The File System Profile is used to configure settings for making file system-specific configurations, currently used by:
Amazon S3 collection agent
Amazon S3 forwarding agent
GCP Storage collection agent
GCP Storage forwarding agent
HDFS collection agent
HDFS forwarding agent
System Importer
System Exporter
The configurations will vary depending on the selected file system, and each file system will be described separately below.
| Table of Contents | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Menus
The External Reference button is specific for the File System profile configurations.
...
Setting
...
Description
...
External References
...
different file systems. It works with https://infozone.atlassian.net/wiki/x/XAurBg , https://infozone.atlassian.net/wiki/x/4xOrBg , https://infozone.atlassian.net/wiki/x/CpipBg, as well as the https://infozone.atlassian.net/wiki/x/voiqBg and https://infozone.atlassian.net/wiki/x/gAerBg. These configurations depend on the specific file system being used. You can find more details about each file system in the subsection links at the bottom of this page.
External References Button
The External References button is specific to the File System profile configuration. Click this button to enable External References in the File System profile configuration. This can be used to configure the
...
fields shown in the following table
File system type | Fields available for configuration |
|---|---|
Amazon S3 file systems |
|
HDFS file systems |
|
GCP file systems |
|
For further information
...
Git
When selecting Git as a file system, you will see the General tab.
...
General Tab
The following settings are available in the General tab in the Git File System profile:
...
Setting
...
Description
...
Repository URL
...
The URL to the repository.
...
Token
...
Token to access the repository. This field is optional.
...
Use Secrets Profile
...
Select the checkbox to use a Secrets Profile to get the Token.
...
Get Branches
...
Click this button to fetch the branches from the repository. If the connection is working the Branch combo box will be populated. If the connection fails, an error dialog will be shown.
...
Branch
...
Select the branch to use.
| Note |
|---|
Note! It is not possible to create a new branch using Usage Engine. The branch must already exist in the repository specified in the Repository URL. |
...
Preview Repository
...
Click here to browse the folders in the repository. It is only possible when the configuration is saved.
| Note |
|---|
Note! When you do a Save As operation, the remote repository is cloned to the platform and may take a little long time. This directory is $MZHOME/gitrepos by default. It can be changed by setting the property mz.git.basePath to some other path accessible from the Platform It is not possible to change the Repository URL or branch once the configuration is saved. |
Import of Git File System Profile
An imported new Git File System Profile configuration will always be invalid since the repository has not been cloned. You clone the repository in the profile by clicking the Clone Repository button.
...
When the cloning is done the text on the button will change to Preview Repository, and the configuration should now be valid, which you can verify by clicking the Validate button.
Amazon S3
When selecting Amazon S3 as a file system, you will see two tabs – General and Advanced.
...
File System profile - Amazon S3 - General tab
The following settings are available in the General tab in the File System profile:
Setting | Description |
|---|---|
File System Type | Select which file system type this profile should be applied for. Currently, only Amazon S3 is available. |
Credentials Settings | |
Credentials from Environment | Select this check box in order to pick up the credentials from the environment instead of entering them in this profile. If this check box is selected, the Access Key and Secret Key fields will be disabled. |
Access Key | Enter the access key for the user who owns the Amazon S3 account in this field. |
Secret Key | Enter the secret key for the stated access key in this field. |
Location Settings | |
Region from Environment | Select this check box in order to pick up the region from the environment instead of entering the region in this profile. If this check box is selected, the Region field will be disabled. |
Region | Enter the name of the Amazon S3 region in this field. |
Bucket | Enter the name of the Amazon S3 bucket in this field. |
Use Amazon Profile | Select this check box if you already have an Amazon Profile set up, this will disable the fields above and allow you to utilize the credentials that you have defined in your chosen Amazon Profile. |
The Advanced tab allows for advanced properties to be configured in the profile.
File System profile - Amazon S3 - Advanced tab
GCP Storage
When selecting GCP as a file system, you will see one tab called General.
...
File System profile - GCP - General tab
The following settings are available in the General tab in the File System profile:
Setting | Description |
|---|---|
Authentication Details - Service Account Settings | |
Environment-Provided Service Account | Select this option, if you are configuring an environment-provided service account with this profile. This will disable the Input Option and Credentials File fields. |
Input Option | Using this option you delegate how the GCP connection credentials are acquired, the available options are to select a JSON File or to fill in the information from a Form. |
Credentials File | Enter the path to the delegated credentials file. This option is visible only when the JSON File option is selected as the input option. |
Import Credentials from File | This button allows for credentials to be imported from a locally stored file. This option is visible only when the Form input option is selected. |
Project ID | Enter the project ID. This option is visible only when the Form input option is selected. |
Private Key ID | Enter the Private key ID. This option is visible only when the Form input option is selected. |
Client Email | Enter the client's email. This option is visible only when the Form input option is selected. |
Client ID | Enter the client ID. This option is visible only when the Form input option is selected. |
Other Information | Enter other information that might be used with this profile. This option is visible only when the Form input option is selected. |
Location Settings | |
Bucket | Enter the target bucket name. |
Use GCP Profile | Select this check box if you already have a GCP Profile set up, this will disable the fields above and allow you to utilize the credentials that you have defined in your chosen GCP Profile. |
HDFS
When selecting HDFS as a file system, you will see two tabs – General and Advanced.
...
File System profile - HDFS - General tab
The following settings are available in the General tab in the File System profile:
Setting | Description |
|---|---|
General Settings | |
Hadoop Mode | Select the Hadoop mode that you want to use, both NON-HA and HA are available. |
Name Node Settings | |
Host | Enter the Hadoop name node host. This option is visible only when the NON-HA Hadoop Mode is selected. |
Port | Enter the Hadoop port number. This option is visible only when the NON-HA Hadoop Mode is selected. |
Replication | Enter the desired number of replication. |
The Advanced tab allows for advanced properties to be configured in the profile.
...
on configuring external reference profiles, see https://infozone.atlassian.net/wiki/x/fwmrBg and https://infozone.atlassian.net/wiki/x/wQyqBg.
Subsections
This section includes the following subsections: