Read data from files in Google Cloud Storage and store it in BigQuery.
Read data from files in a Google Cloud Storage Bucket and store it in a BigQuery database using DataFrames.
Variables
Credentials to use to authenticate with the Google Cloud Platform to access Google Cloud Storage.
Specify name of the Cloud Storage bucket with the files to load.
Specify the path to folder containing the files to load. This parameter is used as a key prefix filter.
If enabled, files within subfolders under the Folder Path will also be loaded.
Specify the number of load records to examine, from most recent and going backwards, when comparing to determine which files have not yet been loaded. If no value is specified, all load records will be examined.
Number of files to load into the DataFrame in memory before the Dataframe is loaded into BigQuery.
Select the format of the data in the files to load.
Number of rows to ignore at the beginning of files with delimited data format.
Specify the character used to separate data values in files with delimited data format.
Specify the character used to enclose string data values in files with delimited data format.
Specify the characters used to represent a null value in files with delimited data format.
Specify the character used to represent an escaped value in files with delimited data format.
Specify the characters used to represent the end of a line.
Credentials to use to authenticate with the Google Cloud Platform to access BigQuery.
Specify the unique identifier of the Google Cloud project that contains the dataset where the data will be loaded.
Specify the name of the dataset that contains the table where the data will be loaded.
Specify the name of the table where the data will be loaded.
Optionally specify the region where the dataset is located.
Column within the Table where the path to the file that was the source of the data will be stored.
Mapping between the source file columns or JSON paths and BigQuery columns.
Read data from files in a Google Cloud Storage Bucket and store it in a BigQuery database using a native command.
Variables
Credentials to use to authenticate with the Google Cloud Platform to access Google Cloud Storage.
Specify name of the Cloud Storage bucket with the files to load.
Specify the path to folder containing the files to load. This parameter is used as a key prefix filter.
If enabled, files within subfolders under the Folder Path will also be loaded.
Select the format of the data in the files to load.
Number of rows to ignore at the beginning of files with delimited data format.
Specify the character used to separate data values in files with delimited data format.
Specify the character used to enclose string data values in files with delimited data format.
Specify the characters used to represent a null value in files with delimited data format.
Allow quoted data sections that contain newline characters in files with delimited data format.
Allow rows that are missing trailing optional columns in files with delimited data format.
Allow extra values that are not represented in the table schema.
Character encoding of the data.
Maximum number of bad records that BigQuery will allow before failing the entire load.
Credentials to use to authenticate with the Google Cloud Platform to access BigQuery.
Specify the unique identifier of the Google Cloud project that contains the dataset where the data will be loaded.
Specify the name of the dataset that contains the table where the data will be loaded.
Specify the name of the table where the data will be loaded.
Optionally specify the region where the dataset is located.