HDFS Collection Agent Input/Output Data and MIM(3.1)

Owned by Jenni Wallin
May 10, 2023
2 min read

Input/Output Data

The Input/Output data is the type of data an agent expects and delivers.

The agent produces bytearray types.

MIM

For information about the MIM and a list of the general MIM parameters, see Administration and Management(3.0).

Publishes

MIM Value

Description

File Modified Timestamp

This MIM parameter contains a timestamp, indicating when the file is stored in the collection directory.

File Modified Timestamp is of the date type and is defined as a header MIM context type.

File Retrieval Timestamp

This MIM parameter contains a timestamp, indicating when the file processing starts.

File Retrieval Timestamp is of the date type and is defined as a header MIM context type.

Source File Count

This MIM parameter contains the number of files, available to this instance for collection at startup. The value is constant throughout the execution of the workflow, even if more files arrive during the execution. The new files will not be collected until the next execution.

Source File Count is of the long type and is defined as a global MIM context type.

Source File Size

This MIM parameter contains the file size, in bytes, of the source file.

Source File Size is of the long type and is defined as a header MIM context type.

Source Filename

This MIM parameter contains the name of the currently processed file, as defined at the source.

Source Filename is of the string type and is defined as a header MIM context type.

Source Filenames

This MIM parameter contains a list of file names of the files that are about to be collected from the current collection directory.

Note

When the agent collects from multiple directories, the MIM value is cleared after collection of each directory. Then, the MIM value is updated with the listing of the next directory.

Source Filenames is of the list<any> type and is defined as a Header MIM context type.

Source Files Left

This parameter contains the number of source files that are yet to be collected. This is the number that appears in the Execution Manager backlog.

Source Files Left is of the long type and is defined as a header MIM context type.

Source Pathname

This MIM parameter contains the path to the directory where the file currently under processing is located.

Source Pathname is of the string type and is defined as a global MIM context type. The path is defined in the HDFS tab.

Accesses

The agent does not itself access any MIM resources.