File System Profile

Owned by Jenni Wallin
Last updated: May 02, 2023 by Former user (Deleted)
4 min read

The File System Profile is used for making file system-specific configurations, currently used by the Amazon S3 collection and forwarding agents.

Configuration

To create a new File System profile, click the New Configuration button from the Configuration dialog available from Build View, and then select File System Profile from the menu. The configurations will vary depending on the selected file system, and each file system will be described separately below.

Buttons

The contents of the menus in the button bar may change depending on which configuration type has been opened in the currently displayed tab. The File System profile uses the standard menu items and buttons that are visible for all configurations, and these are described in Common Configuration Buttons.

SettingDescription

External References

Select this menu item to enable the use of External References in the File System profile configuration. This can be used to configure the following fields:

Amazon S3 file systems

  • Access Key
  • Secret Key
  • Bucket
  • Region
  • Advanced Properties

HDFS file systems

  • Host
  • Replication
  • Port
  • Advanced Properties

GCP file systems

  • Project Id
  • Private Key Id
  • Private Key
  • Client Email
  • Client Id
  • Other Information
  • Bucket

For further information, see Using External Reference in Agent Profile Fields and External Reference Profile.

Amazon S3

When selecting Amazon S3 as a file system, you will see two tabs – General and Advanced.

File System profile - Amazon S3 - General tab

The following settings are available in the General tab in the File System profile:

SettingDescription

File System Type

Select which file system type this profile should be applied for. Currently, only Amazon S3 is available.
Credentials Settings
Credentials from EnvironmentSelect this check box in order to pick up the credentials from the environment instead of entering them in this profile. If this check box is selected, the Access Key and Secret Key fields will be disabled.

Access Key

Enter the access key for the user who owns the Amazon S3 account in this field.

Secret Key

Enter the secret key for the stated access key in this field.

Location Settings
Region from EnvironmentSelect this check box in order to pick up the region from the environment instead of entering the region in this profile. If this check box is selected, the Region field will be disabled.
RegionEnter the name of the Amazon S3 region in this field.

Bucket

Enter the name of the Amazon S3 bucket in this field.

Use Amazon ProfileSelect this check box if you already have an Amazon Profile set up, this will disable the fields above and allow you to utilize the credentials that you have defined in your chosen Amazon Profile.

The Advanced tab allows for advanced properties to be configured in the profile. 

 

File System profile - Amazon S3 - Advanced tab

GCP Storage 

When selecting GCP as a file system, you will see one tab called General.

File System profile - GCP - General tab

The following settings are available in the General tab in the File System profile:

SettingDescription

Environment-Provided Service AccountSelect this option, if you are configuring an environment-provided service account with this profile. This will disable the Input Option and Credentials File fields.
Input OptionUsing this option you delegate how the GCP connection credentials are acquired, the available options are to select a JSON File or to fill in the information from a Form.
Credentials FileEnter the path to the delegated credentials file. This option is visible only when the JSON File option is selected as the input option.
Import Credentials from FileThis button allows for credentials to be imported from a locally stored file. This option is visible only when the Form input option is selected. 
Project IDEnter the project ID. This option is visible only when the Form input option is selected. 
Private Key IDEnter the Private key ID. This option is visible only when the Form input option is selected. 
Client EmailEnter the client's email. This option is visible only when the Form input option is selected. 
Client IDEnter the client ID. This option is visible only when the Form input option is selected. 
Other InformationEnter other information that might be used with this profile. This option is visible only when the Form input option is selected. 
Location Settings
BucketEnter the target bucket name.
Use GCP ProfileSelect this check box if you already have a GCP Profile set up, this will disable the fields above and allow you to utilize the credentials that you have defined in your chosen GCP Profile.

HDFS

When selecting HDFS as a file system, you will see two tabs – General and Advanced.

File System profile - HDFS - General tab

The following settings are available in the General tab in the File System profile:

SettingDescription
General Settings
Hadoop ModeSelect the Hadoop mode that you want to use, both NON-HA and HA are available.
Name Node Settings
HostEnter the Hadoop name node host. This option is visible only when the NON-HA Hadoop Mode is selected. 
PortEnter the Hadoop port number. This option is visible only when the NON-HA Hadoop Mode is selected. 
ReplicationEnter the desired number of replication. 

The Advanced tab allows for advanced properties to be configured in the profile.Â