Workflow Template
- Kevin Wu
- Yeok Hong Ng (Deactivated)
The workflow template is the area where the workflow is designed. The workflow template has one workflow per default. You can add more workflows via the Add Workflow button.
Note!
A workflow configuration cannot be activated, the workflows in it however can.
Note!
Each configuration only allows 1 user to edit it at any one time. While editing the configuration, it is locked to the user's session. To release the lock, you must navigate away from the configuration to another page. It is advised not to close the browser tab without navigating away from the configuration first as this will cause the lock to stay in place until timeout.
You create a workflow configuration by using one of the following methods:
Click Build → New Configuration. Select Workflow from the Configurations dialog or click the New button in the upper left part of the Workflow Template window.
Select a Workflow Type:Â Batch, Realtime, or Task, and then Ok.
Click Add agent to open the Agents selection dialog.
Click on the agent from the Agents selection dialog to deploy the agent to the template.
A workflow template can have multiple labels located next to the workflow template name. These labels serve to identify the workflow template. The following are the labels that may appear:
Label | Description |
|---|---|
 | This label identifies the workflow as a Batch workflow. This label is mutually exclusive to Realtime and Task. |
This label identifies the workflow as a Realtime workflow. This label is mutually exclusive to Batch and Task. | |
This label identifies the workflow as a Task workflow. This label is mutually exclusive to Batch and Realtime. | |
This label identifies the workflow as a Read Only workflow. Read Only workflows will be locked from any editing and you must click on the Edit button to modify a Read Only workflow. | |
This label identifies the workflow as a valid workflow. For more information about validation, refer to https://infozone.atlassian.net/wiki/spaces/MD93/pages/204638608. | |
This label identifies the workflow as an invalid workflow. For more information about validation, refer to https://infozone.atlassian.net/wiki/spaces/MD93/pages/204638608. |
Here are more things that you can do on the workflow template:
Configuration
Note!
Due to the agents' relationships within a workflow configuration, it is preferable that all agents and routes are added before the configuration is started.
Each agent in the workflow configuration has a specific configuration dialog named after the agent type. You can access these configurations by double-clicking the agent or the route. Subsequently you can also click on the Edit button when selecting an agent or a route. This causes a dialog to appear with options. See Agent Options or Route Options for more information.
When an agent is deployed into the workflow template it receives a default name located underneath it. The same applies to routes when they are added. These names may be modified to ease identification in monitoring facilities and logs.
All editing and triggering from the workflow template generate changes to the workflow configuration. Examples of this are adding and removing agents, altering agent positions, and editing agent settings and preferences. If you have the Workflow Monitor open, the changes made after saving the workflow template will be reflected.Â
The workflow table is affected if it includes columns that correspond to an agent removed from the workflow template.
Routing Agents
To create a data flow, agents need to be connected to each other. To connect the agents, click the left mouse button on the center of the source agent and without releasing the left mouse button, move the pointer to the target agent and release there. This creates a route between the two agents indicating the data flow.
You can also create a route between agents by using the button under agent options and selecting the target agent in the dialog. You can modify a route name by clicking on its name and typing a new name. Route names must be unique within a workflow configuration and may only contain the a-z, A-Z, 0-9, "-" and "_" characters.
Route Options
Clicking on a route in the workflow template provides you with a couple of options.
Options | Description |
|---|---|
Opens the Edit Route dialog where you can then edit the route name and route style. You can also double-click the route itself to open the Edit Route dialog. A route name is modified by typing a new name. Route names must be unique within a workflow configuration and may only contain the a-z, A-Z, 0-9, "-" and "_" characters. You can also change the appearance of the routes by selecting from the following Route Style options; Orthogonal, Bezier, or Straight. The default route style is Bezier. | |
Removes the selected route. |
Agent Options
Clicking an agent in the workflow template provides you with a few options.
Options | Description |
|---|---|
Opens the Agent Configuration dialog where you can add or edit the settings and configuration values for the selected agent. You can also double-click the agent itself to open the Agent Configuration dialog. In a read only configuration mode, you can proceed to edit directly on the agent configuration dialog. To save, click Edit on the dialog bar as shown below. A read only agent configuration dialog | |
Removes the agent and all its stored configuration. This action will not prompt you with any dialog and it is final so be absolutely sure that you want to remove the agent. | |
Opens a dialog with an option to route the selected agent to another agent. Â | |
 | Brings up an information dialog that displays parts of the agent's configuration.  |
Agent Selection Dialog
For more information about the different types of agents, refer to the https://infozone.atlassian.net/wiki/spaces/MD93/pages/204604724 section.
Workflow Tables
For more information about the workflow monitor, refer to the https://infozone.atlassian.net/wiki/spaces/MD93/pages/204997546 section.
Workflow Properties
For more information about the workflow monitor, refer to the https://infozone.atlassian.net/wiki/spaces/MD93/pages/204604801 section.
Workflow Monitor
For more information about the workflow monitor, refer to the https://infozone.atlassian.net/wiki/spaces/MD93/pages/204604893 section.
Visualization
You can manipulate visuals of the workflow template by zooming in or out, or moving the workflow template around. A toolbar is present on the right side of the workflow template to allow you to manipulate the visuals.
Workflow template visualization toolbar
Options | Description |
|---|---|
Zooms in on the workflow template. | |
Zooms out of the workflow template. | |
Centers the workflow template so that all agents and routes are visible on the canvas. | |
Displays all the keyboard shortcuts that you can use within the workflow template. |
There is also a zoom percentage along with an x and y coordinate, which are displayed as soon as you zoom in or out. The default range on the zoom percentage is 100% while the x and y coordinates, which are determined from the point on the top left corner of the workflow template canvas, start at 0, 0, respectively. These values appear at the bottom left corner of the canvas whenever you manipulate the visuals.
Common Agent Services
Some agent configuration dialogues are optionally equipped with additional tabs holding configurations for different common services. This section describes the general services that are available. The user guide for each agent, in turn describe what services it uses.
Thread Buffer Tab
By default, a batch workflow uses one active thread at a time. By configuring a buffer storage for an agent, it is possible for yet another thread to be created. This is also called multithreading. One thread populates the buffer, and another pulls it for data. Adding yet another buffer for another agent adds yet another thread, and so on.
This is especially useful in complex workflows with many agents. All batch agents that receive UDRs can use this functionality.
Â
Open the Configuration dialog of the agent and select the Thread Buffer tab. The tab is present in batch processing and batch forwarding agents.
Item | Description |
|---|---|
Use Buffer | Enables multithreading. For further information, see https://infozone.atlassian.net/wiki/spaces/MD93/pages/204997440 . |
Print Statistics | Statistics to be used when trying out where to use the Thread Buffer in the workflow. After each batch execution, the full and empty percentages of the threads using the buffer are logged in the event area at the bottom of the Workflow Monitor. For information on how to interpret the results, see the section below, Analyzing Thread Buffer Statistics. |
A UDR may be queued up while another thread is busy processing a reference to it. Workflows routing the same UDR on several routes and involving further processing of its data, must consequently be reconfigured to avoid this. A simple workaround is to route the UDR to an Analysis agent for cloning before routing it to the other agents (one unique clone per route).
Analyzing Thread Buffer Statistics
By using the Print Statistics alternative in the Thread Buffer tab, buffer statistics are logged for the whole batch execution and show the full and empty percentages for the threads using the thread buffer. For information about multithreading in a batch workflow, refer to the section Threads in a Batch Workflow, in https://infozone.atlassian.net/wiki/spaces/MD93/pages/204997440.
Â
The percentage values for Empty and Full must be as low as possible, and as equal as possible. The latter may be hard to achieve, since the agents may differ too much in processing complexity. If possible, add and configure another agent to take over some of the processing steps from the most complex agent.
See the section below, Thread Buffer Tab, for how to configure the thread buffer.
Filename Sequence Tab
For batch collection agents such as Disk, FTP, and SFTP, there is a service available, found in the agents' configuration dialog in the Filename Sequence tab. The filename sequence is used when you want to collect files that contain a sequence number in the file name. The sequence number is expected to be found at a specific position in the file name and has a fixed or dynamic size.
Â
Item | Description |
|---|---|
Enable Filename Sequence | Determines if the service will be used or not. |
Start Position | The offset in the file name where the sequence number starts. The first character has offset 0 (zero). In the example file name |
Length | The length of the sequence number, if the sequence number has a static length (padded with leading zeros). If the sequence number length is dynamic this value is set to 0 (zero). Â |
Wrap On | If the Filename Sequence service is enabled, a value must be specified on which the sequence will wrap. This number should be larger than or equal to Next Sequence Number and it must be larger than Wrap To. |
Wrap To | The value that the sequence will wrap to. This value must be less than the Wrap On value and less than or equal to Next Sequence Number. |
Warn On Out Of Sequence | If enabled, the agent logs an informative message to the System Log when detecting an out of sequence file, before deactivating. The collection agent will not continue to collect any files upon the next activation, until either the missing file is found, or the Next Sequence Number is set to a valid value. |
Sort Order Tab
The Sort Order service is available for some batch collection agents and is used to sort matched files before collection.
The sort pattern is expected to occur at a specific position in the file name or to be located using a regular expression. However, if the sort pattern cannot be applied, the workflow will abort, except for files with a name which is shorter than Position plus Length. For information on this exception, see the description below for Position.
Item | Description |
|---|---|
Enable Sort Order | Determines if the service will be used or not. If enabled, the sort order that you select also applies to subfolders if you have selected the Include Subfolders checkbox for the FTP, SFTP, and Disk collection agents. For further information, see the section for the relevant agent in https://infozone.atlassian.net/wiki/spaces/MD93/pages/204606046. |
Modification Time | Select to enable file collection order according to the modification time stamp. If Sort Direction is set to Ascending, the oldest files are collected first. The time resolution depends on the server and protocol. Â |
Value Pattern | The method used to locate the item (part of the file name) to be the target for the sorting. This could be either Position that indicates that the item is located at a fixed position in the file name or Regular Expression indicating that the item should be fetched using a regular expression. |
Position | If Position is enabled, the files are sorted based on the part of the filename indicated by Position plus Length. The Start Position value states the offset in the file name where the sorting item starts. The first character has offset 0 (zero). The Length value states the length of the sorting item (part of the file name) if it has a static length. To sort items with a dynamic length, use the value zero (0). Â |
Regular Expression | If enabled, the sorting item is extracted from the file name using the regular expression. If the file name does not end with a digit, this is the correct option. Â |
Sort Type | Type of sorting. Could be either Alphanumeric or Numeric. |
Ignore Case | If enabled, the sorting is not case sensitive. |
Sort Direction | Indicates if the sort order should be Ascending or Descending. |
Filename Template Tab
The Filename Template service is available to batch forwarding agents that are responsible for creating a file, and the Disk forwarding agent for real-time workflows. The configuration contains MIM resources for all available agents in the workflow, the values of which may be used when constructing a filename for the outgoing file.
Since this service includes a selection of MIM resources from available agents in the workflow, it is advised to add all agents to the workflow, and to assign route and agent names, before the filename template configuration is completed.
Â
The list contains MIM resources or user defined values that create the file name. The order of the items in the list controls the order in the file name.
There are two ways in which you can determine the order of the items. You can use the Up and Down arrow buttons, or you can use the column headers. If you begin to sort the items using the column headers, the Up and Down arrows are disabled.
The table - with MIM resources, user defined values, separators and/or directory delimiters - is used to create the filepath or filename. The order of the items in the table defines the filepath or filename.
Since the service uses a selection of MIM resources from available agents in the workflow, it is advised to add all agents to the workflow before the filename template configuration is completed.
Â
Item | Description |
|---|---|
Create Non-Existing Directories | When this checkbox is selected, non-existing directories stated in the path are created. If not selected, the agent will abort if a required directory is missing. |
Item | Description |
|---|---|
MIM Defined | Determines if the Value will be selected from a MIM resource. The MIM resource of type FNTUDR is represented in the template table in the same way as other MIMs, however it has another appearance when the filename or filepath is presented. A MIM FNTUDR value can represent a sub-path with delimiters, a part of a filename, or a directory. For further information about how to use the FNTUDR in filename templates, see the section below, Defining a MIM Resource of FNTUDR Type. |
User Defined | Determines if the Value will be a user defined constant entered in the text field. |
Directory Delimiter | Determines if the Value will be a directory delimiter indicating that the file sub-path will have a directory delimiter at that specified position. You cannot have two directory delimiters directly after each other, or a delimiter in the beginning or end of a filename template. The MIM resource of the special UDR type FNTUDR can include, begin or/and end with directory delimiters this must be noted when adding delimiters in the template. For further information about using the FNTUDR in filename templates, see the section below, Defining a MIM Resource of FNTUDR Type. |
Size | Number of allocated characters in the file name for the selected MIM resource (or user defined constant). If the actual value is smaller than this number, the remaining space will be padded with the chosen padding character. If left empty, the number of characters allocated in the file name will be equal to the MIM value or the constant. |
Padding | Character to pad remaining space with if Size is set. If Size is not set, this value is ignored. |
Alignment | Left or right alignment within the allocated size. If Size is not set, this value is ignored. |
Separator | Separating character to add to the file name after the MIM value or constant. |
Date Format | Adds a timestamp to the file name in the selected way. For further information about the format, see Date and Time Format Codes in https://infozone.atlassian.net/wiki/spaces/MD93/pages/205655211 . |
Â
Defining a MIM Resource of FNTUDR Type
A MIM resource with a value of the FNTUDR type included in the filename template is treated somewhat differently than other MIM resources. A FNTUDR value is a text string that can contain delimiters. The delimiters in the FNTUDR value will be replaced by directory delimiters when determining the target file path. The FNTUDR is defined in the FNT folder.
Â
For further information about how to manipulate FNTUDRs with APL functions and how to publish MIM resources, see thehttps://infozone.atlassian.net/wiki/spaces/MD93/pages/204646109 and the section Meta Information Model in https://infozone.atlassian.net/wiki/spaces/MD93/pages/205655211 .
Â