You can't use this until you register with us!




Description

Load data from files in an S3 bucket to a Redshift database using Redshift native commands.


Purchasing Options



Available Actions

Initialize

Creates a schema called NomNomData in the database specified and table called registry within it for tracking data loaded into the database. Requires access to create schemas and table. Must be run before the Register and Load actions.


Variables

General Parameters
target_db required

Select the Redshift database where the data from the files will be loaded.


Register

Creates a tracking table, a staging table, and, if needed, the Target Table in the database specified. Also adds a line to the registry table created by the Initialize action. Requires access to create tables. Must be run after the Initialize action and before the Load action.


Variables

Task Options
metadata required

Select the Shared Config that contains the source and destination information, the load options and the column definitions for the data that will be loaded.


General Parameters
target_db required

Select the Redshift database where the data from the files will be loaded.

target_table required

Specify the name of the table where data should be loaded in the format table_schema.table_name. If no table_schema is specified, the public schema will be used.

batch_size required

Specify the number of files to attempt to load simultaneously using the Redshift native load commands.

file_lookback

Specify the number of load records to examine, from most recent and going backwards, when comparing to determine which files have not yet been loaded. If no value is specified, all load records will be examined.


Load

Loads data from files on S3 using the options specified. Requires access to update the Target Table. Must be run after the Initialize action. This action will attempt to perform the Register action, if needed. However, if the credentials in the Redshift Database Connection do not have access to create tables in the database, then a database administrator will need to run the Register action prior to running this action.


Variables

Task Options
metadata required

Select the Shared Config that contains the source and destination information, the load options and the column definitions for the data that will be loaded.


AWS Information
aws_connection required

Select the AWS token with access to S3 bucket with the files that will be loaded.


General Parameters
target_db required

Select the Redshift database where the data from the files will be loaded.

target_table required

Specify the name of the table where data should be loaded in the format table_schema.table_name. If no table_schema is specified, the public schema will be used.

batch_size required

Specify the number of files to attempt to load simultaneously using the Redshift native load commands.

file_lookback

Specify the number of load records to examine, from most recent and going backwards, when comparing to determine which files have not yet been loaded. If no value is specified, all load records will be examined.