HDFS Forwarding Agent(3.0)

The HDFS forwarding agent creates files on the remote file system containing the received data. Files are created when a Begin Batch message is received, and closed when an End Batch message is received. In addition, the Filename Template service offers the possibility to compress (gzip) the files, or to further process them, using commands.

To ensure that downstream systems will not use the files until they are closed, they are stored in a temporary directory until the End Batch message is received. This behavior also applies to Cancel Batch messages. If a Cancel Batch is received, file creation is cancelled.

The chapter contains the following sections:

• HDFS Forwarding Agent Configuration(3.0)
• HDFS Forwarding Agent MultiForwardingUDR Example(3.0)
• HDFS Forwarding Agent Transaction Behavior(3.0)
• HDFS Forwarding Agent Input/Output Data and MIM(3.0)
• HDFS Forwarding Agent Events(3.0)