Azure Blob Storage

Azure Blob Storage is a collector and forwarder function that allows for data to be received and sent to Microsoft Azure Blob Storage cloud instances. Azure Blob storage data can be used to compose data lakes for analytics needs and utilise the storage to create cloud-native and mobile apps. 

Collector 

The collector is used to receive data from Azure Blob Storage accounts. To use it, you need to have a prepared environment in advance. The collector function is organized into two sections. The first one is the Azure Credentials where the mandatory access information is provided. Optionally, the Secrets Wallet functionality of Usage Engine can be enabled.  

Field

Description

Field

Description

Account Name

Specify the Azure Blob Storage account name

Authentication

Choose the appropriate authentication type – SAS Token or Account Key.

SAS Token/ Account Key Input Field

Here the SAS Token or Account Key needs to be placed. 

The File Information section concerns the actual data handling of the respective entries. 

Field

Description

Field

Description

Container Name

This field designates the path to the target container.

Folder

This is an optional parameter designating the target path of the desired folder from where the data can be acquired. 

File Selection

This toggle selects if All files are to be collected from the designated path, or only a specific file. If the second option is chosen, an additional field will be displayed called File Name where it can be typed. Additionally, a checkbox called Use Regex can be toggled to enable regular expressions. 

File Format

This dropdown menu selects the file format of the target data, the following ones are supported: CSV, Excel, JSON, and XML. Additional options are available for some of the formats: 

  • If CSV is selected an optional checkbox is shown called Include Table Header that enables the table header check.

  • If Excel is selected an optional checkbox is shown called Include Table Header that enables the table header check. Additionally, this file format will reveal the Sheet selection options that specify the target sheets from where data will be collected – this can be either All sheets in file or Specific sheet(s). If the second option is selected, an additional input field will be shown called Select sheet(s) where the desired names can be entered.

Note!

You can only preview Excel files of 10MB or less.

Delimiter

This dropdown menu selects the used delimiter. The following options are supported: Comma, Tab, Semicolon, Space, and Other. If Other is selected, then the Custom Delimiter input box will be displayed in which the character can be entered.

After Collection

This option specifies the source action file that will be executed once the collection has been completed. It can be either Do nothing or Remove.  

Forwarder

The forwarder function sends data to the configured Azure Blob Storage accounts. This feature-rich function allows for a comprehensive configuration of the target Azure Blob Storage service. The forwarder Function is organized into three sections. The first one is the Azure Credentials where the mandatory access information is provided. Optionally, the Secrets Wallet functionality of Usage Engine can be enabled.  

Field

Description

Field

Description

Account Name

Specify the Azure Blob Storage account name

Authentication

Choose the appropriate authentication type – SAS Token or Account Key.

SAS Token/ Account Key Input Field

Here the SAS Token or Account Key needs to be placed. 

The File Information section concerns the actual data handling of the respective entries. 

Field

Description

Field

Description

Container Name

This field designates the path to the target container.

Folder

This is an optional parameter designating the target path of the desired folder from where the data can be acquired. 

The Output file information section controls the handling of the output data coming from Usage Engine over to the Azure Blob Storage account. 

Field

Description

Field

Description

Filename Options

This selection chooses the target filename to which the data will be forwarded. This can be either the Collector filename from where the data was initially acquired or a Custom filename.

If Collector filename is selected and the collector input file does not have a filename, the function generates a name based on the collector function, for example, Counter.

The function appends the file format extension to the filename based on the file format selected below, if the selected extension is not in the filename.

If Custom filename is selected, an additional field will be displayed called Filename where it can be typed.

The function appends the file format extension to the filename based on the file format selected below, so you do not need to add any extensions to the filename.

Note!

For existing streams, if you would like to use the new filename saving method as described above, you can check the New filename saving method check box.

Append Timestamp

This checkbox toggles the time and date recording of the action. 

File Format

This dropdown menu selects the file format of the target data, the following ones are supported: CSV, Excel, JSON, and Buffer. Additional options are available for some of the formats: 

  • If CSV is selected an optional checkbox is shown called Include Table Header that enables the table header check. A drop-down menu allows for the Delimeter to be chosen, supported ones are Comma, Tab, Semicolon, Space, and Other. If Other is selected, enter the desired character in the Custom delimeter input box. 

  • If Excel is selected the Sheet name field will be shown, in which the desired sheet name can be entered.

  • If JSON is selected the users can select the desired Action on records execution, the following options are available: All in one array, All in one array with key, and One file per record. An optional Output file in pretty pink can be toggled on. 

Compress file

This enables or disables compression. If enabled, the users can choose the format – Zip or GZip. 

Â