HDFS Forwarding Agent MultiForwardingUDR Input
When the agent is set to use MultiForwardingUDR input, it accepts input of the UDR type MultiForwardingUDR
declared in the package FNT. The declaration follows:
internal MultiForwardingUDR { // Entire file content byte[] content; // Target filename and directory FNTUDR fntSpecification; };
The content of the MultiForwardingUDR
will be stored at the path that you have set in the fntSpecification
field. Use the APL functions fntAddString
and fntAddDirDelimiter
to set the value of this field. For further information about these functions, see FNTUDR Functions in APL Reference Guide.
When the files are received they are written to temp files in the DR_TMP_DIR
directory situated in the root output folder. The files are moved to their final destination when an end batch message is received. A runtime error will occur if any of the fields have a null value or if the path is invalid on the target file system.
A UDR of the type MultiForwardingUDR which has a target filename that is not identical to its precedent is saved in a new output file.
Note
After a target filename that is not identical to its precedent is saved, you cannot use the first filename again. For example: Saving filename B after saving filename A, prevents you from using A again. Instead, you should first save all the A filenames, then all the B filenames, and so forth.
Non-existing directories will be created if the MultiForwardingUDR
. Every configuration option referring to bytearray input is ignored when MultiForwardingUDRs
are expected.
Example - APL code to send MultiForwardingUDRs
This example shows the APL code used in an Analysis agent connected to a forwarding agent expecting input of type MultiForwardingUDRs
.
import ultra.FNT; MultiForwardingUDR createMultiForwardingUDR (string dir, string file, bytearray fileContent){ //Create the FNTUDR FNTUDR fntudr = udrCreate(FNTUDR); fntAddString(fntudr, dir); fntAddDirDelimiter(fntudr);//Add a directory fntAddString(fntudr, file);//Add a file MultiForwardingUDR multiForwardingUDR = udrCreate(MultiForwardingUDR); multiForwardingUDR.fntSpecification = fntudr; multiForwardingUDR.content = fileContent; return multiForwardingUDR; } consume { bytearray file1Content; strToBA (file1Content, "file nr 1 content"); bytearray file2Content; strToBA (file2Content, "file nr 2 content"); //Send MultiForwardingUDRs to the forwarding agent udrRoute(createMultiForwardingUDR ("dir1", "file1", file1Content)); udrRoute(createMultiForwardingUDR ("dir2", "file2", file2Content)); }
The Analysis agent mentioned in this example sends two MultiForwardingUDRs
to the forwarding agent. Two files with different contents are placed in two separate sub folders in the root directory. The Create Non-Existing Directories check box in the Filename Template tab in the configuration of the forwarding agent must be selected if the directories do not previously exist.