Azure Blob Storage

Azure Blob Storage is a collector and forwarder Function that allows for data to be received and sent to Microsoft Azure Blob Storage cloud instances. Azure Blob storage data can be used to compose data lakes for analytics needs and utilise the storage to create cloud-native and mobile apps. 

Collector 

The collector is used to receive data from Azure Blob Storage accounts. To use it you need to have a prepared environment in advance. The collector Function is organized into two sections. The first one is the Azure Credentials where the mandatory access information is provided. Optionally, the Secrets Wallet functionality of Cloud Edition can be enabled.  

FieldDescription
Account NameSpecify the Azure Blob Storage account name
AuthenticationChoose the appropriate authentication type – SAS Token or Account Key.
SAS Token/ Account Key Input FieldHere the SAS Token or Account Key needs to be placed. 

The File Information section concerns the actual data handling of the respective entries. 

FieldDescription
Container NameThis field designates the path to the target container.
FolderThis is an optional parameter designating the target path of the desired folder from where the data can be acquired. 
File SelectionThis toggle selects if All files are to be collected from the designated path, or only a specific file. If the second option is chosen, an additional field will be displayed called File Name where it can be typed. Additionally, a checkbox called Use Regex can be toggled to enable regular expressions. 
File Format

This dropdown menu selects the file format of the target data, the following ones are supported: CSV, Excel, JSON, and XML. Additional options are available for some of the formats: 

  • If CSV is selected an optional checkbox is shown called Include Table Header that enables the table header check.
  • If Excel is selected an optional checkbox is shown called Include Table Header that enables the table header check. Additionally, this file format will reveal the Sheet selection options that specify the target sheets from where data will be collected – this can be either All sheets in file or Specific sheet(s). If the second option is selected, an additional input field will be shown called Select sheet(s) where the desired names can be entered. 


DelimiterThis dropdown menu selects the used delimiter. The following options are supported: Comma, Tab, Semicolon, Space, and Other. If Other is selected, then the Custom Delimiter input box will be displayed in which the character can be entered.
After CollectionThis option specifies the source action file that will be executed once the collection has been completed. It can be either Do nothing or Remove.  

Forwarder

The forwarder Function sends data to the configured Azure Blob Storage accounts. This feature-rich Function allows for a comprehensive configuration of the target Azure Blob Storage service. The forwarder Function is organized into three sections. The first one is the Azure Credentials where the mandatory access information is provided. Optionally, the Secrets Wallet functionality of Cloud Edition can be enabled.  

FieldDescription
Account NameSpecify the Azure Blob Storage account name
AuthenticationChoose the appropriate authentication type – SAS Token or Account Key.
SAS Token/ Account Key Input FieldHere the SAS Token or Account Key needs to be placed. 

The File Information section concerns the actual data handling of the respective entries. 

FieldDescription
Container NameThis field designates the path to the target container.
FolderThis is an optional parameter designating the target path of the desired folder from where the data can be acquired. 

The Output file information section controls the handling of the output data coming from Cloud Edition over to the Azure Blob Storage account. 

FieldDescription
Filename OptionsThis selection chooses the target filename to which the data will be forwarded. This can be either the Collector filename from where the data was initially acquired or a Custom filename. If the second option is chosen, an additional field will be displayed called Filename where it can be typed.
Append TimestampThis checkbox toggles the time and date recording of the action. 
File Format

This dropdown menu selects the file format of the target data, the following ones are supported: CSV, Excel, JSON, and Buffer. Additional options are available for some of the formats: 

  • If CSV is selected an optional checkbox is shown called Include Table Header that enables the table header check. A drop-down menu allows for the Delimeter to be chosen, supported ones are Comma, Tab, Semicolon, Space, and Other. If Other is selected, enter the desired character in the Custom delimeter input box. 
  • If Excel is selected the Sheet name field will be shown, in which the desired sheet name can be entered. 
  • If JSON is selected the users can select the desired Action on records execution, the following options are available: All in one array, All in one array with key, and One file per record. An optional Output file in pretty pink can be toggled on. 
Compress fileThis enables or disables compression. If enabled, the users can choose the format – Zip or GZip