You can't use this until you register with us!

Description

Copy files from S3 to Google Cloud Storage.

Purchasing Options

S3 to Google Cloud Storage Copier - $49.99 / month

Available Actions

Copy Files

Copy files from an S3 bucket to a Google Cloud Storage bucket.

Variables

Source Parameters
aws_token_connection

Select the AWS Access Keys to use to access the S3 bucket where the files are stored.

iam_role

Optional. ARN of an IAM role to use for S3 bucket access.

bucket_shared_config required

Select the Shared Config that contains information about the S3 Bucket where the files are stored.

s3_folder_path

Path from the starting folder in the S3 Bucket Configuration to the folder where the files are stored.

process_subfolders required

Enable to copy files within subfolders under the Source Folder.

extensions

Only copy files with the extensions specified. Comma separated list. Leave blank to copy all files.

lookback

Specify the number of tracking records to examine, from most recent and going backwards, when comparing to determine which files have not yet been copied. If no value is specified, all tracking records will be examined.


Destination Parameters
gcloud_connection required

Credentials to use to access the Google Cloud Storage bucket.

gcloud_bucket required

Name of the bucket where the files will be copied.

target_folder

Folder where the files will be copied.

re_upload required

Enable to allow previously copied files to be copied again.


Tracking Parameters
staging_db required

Credentials to use to access the BigQuery Cloud Data Warehouse.

project_id required

Name of the project with the dataset where the tracking data will be stored.

dataset_id required

Name of the dataset with the table where the tracking data will be stored.

bigquery_table required

Name of the table where the tracking data will be stored.

bigquery_region required

Name of the region where dataset is located