Skip to content

fix(bigquery): support LOAD DATA FROM FILES syntax#7482

Open
RedZapdos123 wants to merge 4 commits intotobymao:mainfrom
RedZapdos123:fix/bigquery-load-data-from-files
Open

fix(bigquery): support LOAD DATA FROM FILES syntax#7482
RedZapdos123 wants to merge 4 commits intotobymao:mainfrom
RedZapdos123:fix/bigquery-load-data-from-files

Conversation

@RedZapdos123
Copy link
Copy Markdown
Contributor

@RedZapdos123 RedZapdos123 commented Apr 9, 2026

Summary:

  • add BigQuery parser support for LOAD DATA ... FROM FILES(...) syntax used in BigQuery docs
  • keep existing LOAD DATA ... INPATH ... parsing behavior by falling back to the base parser when FROM FILES is not matched
  • make LoadData.inpath optional and add optional files payload for BigQuery form
  • add BigQuery generator override to emit LOAD DATA [OVERWRITE|INTO TABLE ...] FROM FILES(...)
  • add BigQuery regression tests for both OVERWRITE ... FROM FILES(...) and INTO TABLE ... FROM FILES(...)

Closes #7480.

Validation:

  • Reproduced issue before fix: ParseError: Required keyword: 'inpath' missing
  • python -m pytest -q tests/dialects/test_bigquery.py -vv
  • python -m pytest -q
  • Direct parse reproduction now succeeds, and returns LoadData

The validations screenshots of the tests run, locally on WSL:

image image

Signed-off-by: Mridankan Mandal <xerontitan90@gmail.com>
Copy link
Copy Markdown
Collaborator

@VaggelisD VaggelisD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @RedZapdos123, thank you for your contribution, leaving a few comments:

Signed-off-by: Mridankan Mandal <xerontitan90@gmail.com>
Signed-off-by: Mridankan Mandal <xerontitan90@gmail.com>
Signed-off-by: Mridankan Mandal <xerontitan90@gmail.com>
@RedZapdos123 RedZapdos123 force-pushed the fix/bigquery-load-data-from-files branch from 5194ed5 to 8e7e831 Compare April 14, 2026 15:41
Copy link
Copy Markdown
Collaborator

@VaggelisD VaggelisD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more comment and should be good to go I believe:

local = " LOCAL" if expression.args.get("local") else ""
inpath = f" INPATH {self.sql(expression, 'inpath')}"
overwrite = " OVERWRITE" if expression.args.get("overwrite") else ""
this = f" INTO TABLE {self.sql(expression, 'this')}"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • We can pull these upwards to avoid duplicating them as well
  • We should avoid doing self.func here, looks a bit hacky given that this is an option list and not a function call

I'd try to rewrite this more like the following (may need minor fixes):

Suggested change
this = self.sql(expression, "this")
overwrite = " OVERWRITE" if expression.args.get("overwrite") else ""
files = self.sql(expression, "files")
if files:
files = f"FILES {self.wrap(files)}"
this = f" {this}" overwrite else f" INTO TABLE {this}"
return f"LOAD DATA{overwrite}{this} FROM {files_sql}"
local = " LOCAL" if expression.args.get("local") else ""
inpath = f" INPATH {self.sql(expression, 'inpath')}"
this = f" INTO TABLE {this}"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BigQuery: LOAD DATA ... FROM FILES(...) raises ParseError: Required keyword 'inpath' missing

3 participants