Process Modelling and Configuration

The process modeling mechanism Usage Engine leverages to create applications and data processing logic is through a flexible configuration layer. Its high-level graphical environment delivers a complete set of configurable tools to meet the challenging requirements of our customers.

Usage Engine Agents

Workflows are the cornerstones of process modeling and configuration in Usage Engine. A workflow is a set of nodes that are referred to as agents. Processing models are visualized by connecting the agents to each other using drag and drop.

Workflows contain three types of agents:

Collection agents are responsible for gathering data into the workflow from data sources or the client of a bi-directional real-time flow. An example of a simple collection agent could be one that reads a file from disk and sends its contents into the workflow.
Processing agents delivers data on one or many outgoing routes. A processing agent could be as simple as a counter that counts the throughput. It could also be more complex in that it evaluates the data and depending on the result, delivers it on different routes. Amongst the processing agents, so-called transformer agents can be identified. A transformer agent is responsible for translating an incoming byte stream into a UDR object or the opposite. For file distribution the Encoder Agent can be used to create header/trailer records containing meta-data of the file, e.g. record counter, check sum etc. This is commonly referred to as Decoding and Encoding, which the Usage Engine Ultra format system will handle.
Forwarding agents are responsible for distributing the data from the workflow to other systems or devices. An example of a forwarding agent could be one that creates a file from a data stream and transfers it to another system using FTP.

Process Modelling

Workflows

Workflows may be interlinked into data processing and orchestrattion tasks of virtually any complexity, where the output of one Workflow is the input of another.

There are three different types of workflows, that each have a slightly different execution behavior, but their modeling and configuration structure is the same.

Batch workflows – Are used to collect, process and distribute file-based data, also referred to as an Offline model. Batch Workflows can be configured for multi-threaded execution and enforces a first-in-first-out processing order and a strict transaction boundary based on each batch processed.
Real-time workflows – Enables online processing of requests/answers with other systems. These Workflows use multi-threading to enable execution of large numbers of independent execution paths simultaneously.
Task workflows – Used to execute common activities such as cleanup or maintenance tasks. Several System Tasks are pre-configured in Usage Engine and can be complemented with any user-defined activity.

The workflow configuration is used to design and configure workflows, by adding agents and connecting them to each other.

The following illustration shows an example of a simple Batch workflow:

Batch Workflow Example

Workflows that are conceptually related can be grouped together. This enables the setting of common execution or monitoring characteristics on all elements in a group, such as execution scheduling for Workflow Groups or selection of execution environment for Workflows and Workflow Groups.

Workflow and Workflow Group functions include, among other things:  

Function	Description
Scheduling	Workflows Groups can be scheduled for execution periodically, on specific occasions or as a result of an event within the Usage Enginee system.
Distributed execution	A workflow or a group of workflows can be directed to execute on a particular execution node, or it can automatically be distributed to a node with the lowest load or lowest number of active Workflows.
Suspend execution	For a period of time a workflow or a group can be suspended from executing as scheduled.
Version handling	All versions of workflows and groups are saved during configuration, and it is possible to view and rollback to any previous version.
Table configuration	Workflows with the same structure can easily be multiplied in a table-based view, where the user selects which parameters that should be possible to define per workflow.
External References	Workflow configuration can be managed via property files called external references, providing the facility for external control of Workflow execution from outside the Desktop.
Meta Information Model (MIM)	Upon configuration, the workflow and its agents provide runtime attributes – MIMs, which can be fetched and distributed to other parts of the system. MIMs can be both static and dynamic. Examples of MIM values are, for example, agent names and names of collected files.
Encryption	As with all configurations, also Workflows and groups can be saved encrypted. In such a case, read or write access to encrypted configurations is only possible if a correct password is provided.