Firestore to BigQuery Loader

Description

Copy data from a Cloud Firestore database to a BigQuery data warehouse.

Purchasing Options

Firestore to BigQuery Loader - $99.99 / month

Available Actions

Copy Data With Date Range

Read a range of data from a Cloud Firestore database and store it in a BigQuery data warehouse.

Variables

Source Parameters

source_firebase_config required

Credentials to use to access the Cloud Firestore database.

app_name

Name of the Firebase app associated with the Cloud Firestore database that you want to copy data from. If this field is left blank, the Cloud Firestore database associated with the [DEFAULT] Firebase app will be used.

tracking_field_value required

Only documents in the sorted collection from the Cloud Firestore database that have a sort field value higher than the value specified will be processed. For a database that has never been processed before, specify a value that is lower than the sort field value on any document. This field will automatically be updated after each increment of documents is successfully processed.

batch_size required

This Task will process documents in increments. Set a value in this field to determine the size of each increment.

Configuration Parameters

table_config required

JSON specifying the name of the collection in the Firestore Database to use to find documents, which document field to use to sort the collection and how the fields in the documents map to columns in the BigQuery table. Refer to the slide out help for details and an example.

subcollections

JSON specifying the names of the subcollections in the Firestore Database to use to find documents and how the fields in the documents map to columns in the BigQuery table. Refer to the slide out help for details and an example.

quit_on_error required

If enabled, the Task will end when an error is encountered while reading field in a document or writing a row to BigQuery. If disabled, the document triggering the error will be skipped and processing will continue with the next document.

Destination Parameters

target_big_query_config required

Credentials to use to access the BigQuery Cloud Data Warehouse.

project_id required

Specify the unique identifier of the Google Cloud project that contains the dataset where the data from the Cloud Firestore database should be loaded.

bigquery_dataset required

Specify the name of the dataset that contains the table where the data from the Cloud Firestore database should be loaded.

target_table required

Specify the name of the table where the data from the Cloud Firestore database should be loaded.

Copy All Data

Read all data from a Cloud Firestore database and store it in a BigQuery data warehouse. Pre-existing data in the table will not be modified.