You can't use this until you register with us!

Description

Load data from files in Blob Storage to an SQL Server.

Purchasing Options

Blob Storage to SQL Server Loader - $99.99 / month

Available Actions

Initialize

Creates a schema called NomNomData in the database specified and table called registry within it for tracking data loaded into the database. Requires access to create schemata and tables. Must be run before the Register and Load actions.

Variables

Initialization Options
target_db required

Credentials to use to access the SQL Server database.


Register

Creates a tracking table, a staging table, and, if needed, the Target Table in the database specified. Also adds a line to the registry table created by the Initialize action. Requires access to create tables. Must be run after the Initialize action and before the Load action.

Variables

Registration Options
target_db required

Credentials to use to access the SQL Server database.

target_table required

Name of the table, in schema.table format, where data should be loaded.

metadata required

Shared Config that contains the source information, load options and column definitions.

file_lookback

Number of load records to examine, from most recent and going backwards, to determine which files have not yet been loaded. If no value is specified, all load records will be examined.

batch_size required

Number of files to load simultaneously.


Native Load

Load data from files in a Blob Storage container to a SQL Server database using native commands. Loads data from files on Blob Storage using the options specified. Requires access to update the Target Table. Must be run after the Initialize and Register actions.

Variables

Source Options
blob_storage_connection

Credentials to use to access the Blob Storage container.

file_lookback

Number of load records to examine, from most recent and going backwards, to determine which files have not yet been loaded. If no value is specified, all load records will be examined.

batch_size required

Number of files to load simultaneously.


Destination Options
target_db required

Credentials to use to access the SQL Server database.

target_table required

Name of the table, in schema.table format, where data should be loaded.

metadata required

Shared Config that contains the source information, load options and column definitions.


Non-native Load

Load data from files in a Blob Storage container to a SQL Server database. Loads data from files on Blob Storage using the options specified. Requires access to update the Target Table. Must be run after the Initialize and Register actions.

Variables

Source Options
blob_storage_connection

Credentials to use to access the Blob Storage container.

file_lookback

Number of load records to examine, from most recent and going backwards, to determine which files have not yet been loaded. If no value is specified, all load records will be examined.

batch_size required

Number of files to load simultaneously.

file_split_threshold

Maximum number of lines to load into memory at one time from a single file.

filename_mask

RegEx pattern to limit files to load. Specify .* for all files.

skip_error

Continue loading subsequent data if an error occurs.

allow_missing_columns

Use with caution. Allows files that do not contain all of the columns defined in the Task Configuration to be loaded. If enabled, ANY file will successfully load, regardless of content.


Destination Options
target_db required

Credentials to use to access the SQL Server database.

target_table required

Name of the table, in schema.table format, where data should be loaded.

metadata required

Shared Config that contains the source information, load options and column definitions.