05. System Extensions MZ 9

General Availability

Name

Type

Included in Subscription base license

Rel.

Description
APL Collection StrategiesSystemY9.0.0.0

An addition (a tab in the UI) to the Disk, FTP, SFTP, and SCP collection agents. Enables APL code to be used to define complex collection rules. Includes function blocks for detailed filtering and processing for:

  • List of directories to scan.
  • List of files to collect.
  • Actions to take before collection.
  • Actions to take after collection.
APL ExtensionsBothY9.0.0.0

Add-ons to the APL library of functions. Contains:

  • File functions: list directories, list files, move and delete files.
  • Dynamic functions: dynamic calls to an agent, a plug-in, or a generic APL function.
  • Base64 functions: decoding and encoding of Base64 data.
  • Map functions: create, delete and handle maps.
  • Log and notifications functions; send debug, system log entries, and alrams.
  • Data signing functions: generate digital signatures for data:
    • Gets keys from Java keyword file:
      • JKS, PKCS8, PKCS12 or CA_CERTIFICATES_PATH.
    • Calculates hash values and signs result using various algorithms:
      • SHA1withDSA, SHA1withRSA or SHA256withRSA.
      • Hmac-SHA256 or Hmac-SHA512.
Amazon RedshiftOfflineY9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for Amazon Redshift. Usage:

  • Database Table Related Functions (APL).
  • SQL Collection/Forwarding Agents.
Amazon S3 External ReferenceSystemY9.0.0.0Enables storage of External Reference files in Amazon S3 buckets.
Authorization ServerSystemY9.2.0.0The Authorization Server is used for generating access tokens that are required for accessing REST APIs hosted by the HTTP/2 Server agent when the OAuth 2.0 Authentication feature is enabled in the HTTP/2 Server agent.

Azure Application Insight Event Notifier

SystemY9.0.0.0

Event Notification support for Azure Application Insight.

Azure KeyVaultSystemY9.0.0.0Enables configuration of access credentials and properties to be used to connect to an Azure KeyVault.
Azure OAuth2 APL pluginSystemY9.0.0.0

APL plugin for acquiring an OAuth2-token from Azure servers.

CSVSystemY

DB connectivity package (JDBC) for CSV files that makes it possible to read CSV files instead of databases when:

  • Using database related table functions in APL.
  • Using shared table configurations.
Cassandra DBSystemY9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for Cassandra. Usage:

  • SQL Collection/Forwarding Agents.

Categorized Grouping

Offline

Y

9.0.0.0

A processing agent that divides incoming data into categories - each resulting in separate forwarding files. A category can contain data from one or several files. Closure criteria for the categories is configurable. 

Collection Strategy - Control Files

Offline

Y

9.0.0.0

An addition (a tab in the UI) to the Disk, FTP, SFTP, and SCP collection agents. Enables collection of files for which a corresponding control file exist.

Collection Strategy – Duplicate Filter

Offline

Y

9.0.0.0

An addition (a tab in the UI) to the Disk, FTP, SFTP, and SCP collection agents. Enables duplication check for all collect files from a directory against a configurable number of files to keep as reference. Verification criteria can be based on filename and optionally time stamp. 

Collection Strategy – Multi Directory

Offline

Y

9.0.0.0

An addition (a tab in the UI) to the Disk, FTP, SFTP, and SCP collection agents. Enables collection of files from a series of directories listed in a control file. 

Compression agent

Offline

Y

9.0.0.0

Processing agents that compresses data (GZIP format).


Conditional TraceRealtime Y9.0.0.0

Conditional Trace has been rewritten to track all changes to selected UDRs in a workflow, and be plug-and-playable to Jaeger, a state-of-the-art trace logging system. This provides for complete trace logs and a rich set of filtering and troubleshooting tools.

Couchbase for Aggregation

Online

Y

9.0.0.0

Aggregation storage based on Couchbase. Couchbase enterprise edition license (commercial support) is required.
Data HubSystemY9.0.0.0

Consist of Profile, that defines connection to an Impala database via Cloudera JDBC and HDFS, and two agents. The Data HubForwarding Agent loads data in bulks into Impala databases. The Data HubTask Agent removes old partitions. 

Reference:

Data MaskingBothY9.0.0.0

A Profile and Processing agent, that either masks or unmasks specified fields in selected UDRs. Available masking methods:

  • Crypto: AES-128 or AES-256. Can be used for both obscuring and unmasking data.
  • Database: stores masked and unmasked data in a database.
  • Hash: obscures data. 

Data Masking Hash/Database Storage

SystemN9.0.0.0

Support for Data Masking Hash/Database Storage.

Data VeracitySystemY9.0.0.0

An alternative to the Error Correction System available out-of-the-box in the platform. Stores data failing validation in an external database, and is better suited for high load operations. 

  • Data Veracity Forwarding Agent: sends data to the storage.
  • Data Veracity Collection Agent: collects data from the storage for reprocessing.
  • Data Veracity Task Agent: configures and schedules repair tasks for data in storage.
  • Data Veracity UI: used to examine the storage, and define configuration like error codes and repair tasks.

Database APL Functions

Both

Y

9.0.0.0

Add-ons to the APL library of functions. Contains:

  • Database bulk SQL; bundles multiple queries into one SLQ statement. Available for batch workflows only.
  • Callable statements; enables use of Stored Procedures with output parameters.
  • Prepared statements; enables efficient execution of an SQL statement multiple times.

Decompression Agent

Offline

Y

9.0.0.0

Processing agents that decompresses data (GZIP format).

Distributed Storage Couchbase

Online

Y

9.0.0.0

Enables access to a Couchbase distributed storage solution from APL without having to provide details about its type or implementation. This makes APL code generic and independent of storage solutions changes, which are handled in the Profile.

APL functions available for data management:

  • Read, store and remove data.
  • Transaction management and bulk processing.
Distributed Storage RedisOnlineY9.0.0.0

Enables access to a Couchbase distributed storage solution from APL without having to provide details about its type or implementation. This makes APL code generic and independent of storage solutions changes, which are handled in the Profile.

APL functions available for data management:

  • Read, store and remove data.
  • Transaction management and bulk processing.
Elasticsearch Aggregation StorageOfflineY9.0.0.0A Profile enabling Elasticsearch to be used as Aggregation storage.

Free format SQL loader agent


OfflineY9.0.0.0Processing agent that populates a database with data from files, using a configurable SQL statement.
HTTP Client APL Functions
Y9.0.0.0

APL plug-in for HTTP requests handling.

IBM DB2OfflineY9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for IBM DB2. Usage:

  • Database Table Related Functions (APL)
  • Database Collection/Forwarding Agents
  • Prepared Statements (APL)
  • SQL Collection/Forwarding Agents

IBM Netezza

System

Y

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for IBM Netezza. Usage:

  • sqlexec (APL)
  •  SQL Loader Agent

JSON Interface

System

Y

9.0.0.0

Enables JSON decoding and encoding. 

Reference:

  • RFC7159
JWT APL FunctionsOnlineY9.0.0.0

Support for JSON Web Tokens (JWT) signature verification.

LDAP APL Plug-inBothY9.0.0.0APL functions for connecting (using anonymous or simple authentication) to an external LDAP directory and performing search, add, delete, and modify operations.

LDAP Authentication

System

Y9.0.0.0Enables LDAP as authentication method in the Access Controller UI. The authentication and administration is delegated to an external LDAP directory. 

LZO Support

SystemY9.0.0.0

Adds LZO decompression support to a collection agent's configuration UI. Any flags to the standard unix command "lzop" can be added.

Reference:

MS SQL Server

System

Y

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for SQL Server. Usage:

  • Audit Profile
  •  Database Bulk Lookup Functions (APL)
  •  Database Table Related Functions (APL)
  •  Database Collection/Forwarding Agents
  •  Event Notifications
  •  Prepared Statements (APL)
  •  SQL Collection/Forwarding Agents
  •  Task Workflows Agents (SQL)
MariaDBSystem Y9.2.0.0

Database connection support for MariaDB database that can be used in:

  • Reference Data Management

Merge File Collection

Offline

Y

9.0.0.0

Collects and merges files based on number of files or bytes. A collected batch will consist of all files matching the search criteria for a particular workflow activation.


MySQL

System

Y

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for MySQL. Usage:

  •  Database Bulk Lookup Functions (APL)
  •  Database Table Related Functions (APL)
  •  Event Notifications
  •  Prepared Statements (APL)
  •  SQL Collection/Forwarding Agents
  •  SQL Loader Agent
  •  Task Workflows Agents (SQL)

PKCS7

System

Y

9.0.0.0

APL functions supporting signing of data using CMS signature.

  • Loading of PKCS7 certificates.
  • Signing of bytearrarys with PKCS7 certificates. 

PostgreSQL

System

Y

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for PostgreSQL. Usage:

  • Audit Profile
  • Aggregation Profile
  • Callable Statements (APL)
  • Database Bulk Lookup Functions (APL)
  • Database Table Related Functions (APL)
  • Database Collection/Forwarding Agents
  • Event Notifications
  • Prepared Statements (APL)
  • Reference Data Management
  • Shared Table Profile
  • SQL Collection/Forwarding Agents
  • SQL Loader Agent
  • Task Workflows Agents (SQL)
Python Task



Realtime Ordered RoutingOnlineN9.0.0.0Queue Strategy that maintains the order of UDRs as provided by the source.
Redis for AggregationOnlineY9.0.0.0Aggregation storage based on Redis. For commercial support RedisLabs license is required.
Reference Data Management GUISystemY9.1.0.0Delivers the capability to query and edit specific table sets in relational databases while schema permissions remain unaltered.

SAP Hana

System

Y

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for Snowflake. Usage for SAP Hana v1 and v2:

  • Aggregation Profile
  • Callable Statements (APL) 
  • Database Bulk Lookup Functions (APL)
  • Database Table Related Functions (APL)
  • Database Agents
  • Event Notifications
  • Prepared Statements (APL)
  • Shared Table Profile
  • SQL Collection/Forwarding Agents
  • SQL Loader Agent
  • Task Workflows Agents (SQL)

In addition to the functions listed above, the SAP HANA database 2.0 can be used with the following functionality:

  • Audit Profile

SCIM

SystemY9.0.0.0

A REST HTTP interface enabling SCIM management of users and groups. Supports POST, GET, DELETE, PUT and PATCH methods for user and group configurations.  

References:

SNMP RequestOnlineY9.0.0.0Polls management and operational status data from SNMP-enabled network elements. The list of network elements to poll from is provided as a CSV file in the agent's configuration. The SNMP Collection profile is used to import the set of MIB files that will be used to build the target UDRs, as well as configure default parameters for connection. The SNMP OID Profile defines which UDR types and fields to poll.
SNMP TrapOnlineY9.0.0.0Receives SNMP notifications, i e traps and informs. The SNMP Collection profile is used to import the set of MIB files that will be used to build the target UDRs, as well as configure default parameters for connection.
SQL CollectionOfflineY9.0.0.0

Free format SQL collection agent:

Collects rows from databases and inserts them as UDRs into a workflow. A configurable SQL query is executed upon workflow activation, and all matching rows will comprise a batch of UDRs. The format of the UDRs is configurable.

Several databases are supported,  please refer to the System Extensions section.

SQL ForwardingOfflineY9.0.0.0

Free format SQL forwarding agent:

Inserts UDRs as rows in database tables. Table columns can be populated with UDR fields or MIM values. Optionally, commit window size can be set. Configurable exception handling allows for alternate routes for failed inserts.

Several databases are supported,  please refer to the System Extensions section.

SQL for AggregationOfflineY9.0.0.0

Aggregation storage based on SQL relational database:

Allows batch aggregation to be done using databases instead of using persistent geo-redundant disk storage.

SSOSystemY9.0.0.0

Enables Single Sign On with external OpenID Provider, along with multi factor authentication.

Shared Memory Table

BothY9.0.0.0Enables workflows to share in-memory tables for lookups. This table lookup service is defined in a Shared Table Profile.
Snowflake DBSystemY9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for Snowflake. Usage:

  • sqlExec Function (APL)
  • SQL Collection/Forwarding Agents

Sybase IQ

System

Y

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for Sybase IQ. Usage:

  • Database Table Related Functions (APL)
  • APL function sqlexec 
  • Database Collection/Forwarding Agents
  • Event Notifications
  • SQL Collection/Forwarding Agents
  • SQL Loader Agent
  • Task Workflows Agents (SQL)
TeradataOfflineY9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for Teradata. Usage:

  • Database Table Related Functions (APL)
  • Database Collection Agent
  • Prepared Statements
  • SQL Collection Agents

TimesTen

System

Y

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for TimesTen. Usage:

  • Audit Profile
  •   Database Bulk Lookup Functions (APL)
  •   Database Table Related Functions (APL)
  •   Event Notifications
  •   Prepared Statements (APL)
  •   Shared Table Profile
  •   SQL Collection/Forwarding Agents
  •   Task Workflows Agents (SQL)

Ultra GPB support (Google Protocol Buffer)

System

Y

9.0.0.0

Enables GPB definition support in UFDL, allowing GPB data to be Decoded and Encoded. Both the proto2 and proto3 versions of the GPB language are supported.

Ultra XML

System

Y

9.0.0.0

Enables XML schema support in UFDL, allowing XML data to be Decoded and Encoded.
Unit TestSystemY9.0.0.0

Enables test of workflows and APL Profiles. Tests are configured using Python programming, and can be exported and imported the same way as any other configuration

VerticaSystemY9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for Vertica. Usage:

  •  Audit Profile
  • Database Table Related Functions (APL)

  • Database Collection/Forwarding Agents

  • Event Notifications

  • Prepared Statements (APL)

  • Shared Table Profile

  • SQL Collection/Forwarding Agents

  • SQL Loader Agent

  • Task Workflows Agents (SQL)

Web API

System

Y

9.0.0.0

Feature (HTTP-based API enabling external systems to collect status information from selected parts of the system, including stop and start workflows and groups) is sunset, and is replaced with Operations API. Note that full coverage and synch PE vs MZ is done incrementally.

Workflow bridge Batch -> Realtime

Both

Y

9.0.0.0

Forwarding and Collection Agents for sending data from a batch workflow to a realtime workflow. The data is stored in memory cache if the workflows are executing on the same JVM, or is transferred using TCP or Aeron in case the workflows reside on different JVMs. 

Workflow bridge Realtime-> Realtime

Online

Y

9.0.0.0

Forwarding and Collection Agents for sending data from a realtime workflow to a realtime workflow. The data is stored in memory cache if the workflows are executing on the same JVM, or is transferred using TCP or Aeron in case the workflows reside on different JVMs. 
Workflow PackagesSystemY9.0.0.0

Configuration can be bundled into workflow packages and exported to other systems. Advantages compared to regular export format:

  • Self-contained; can execute independent of configuration outside the package.
  • Pre-compiled; enables fast and easy upgrades and rollbacks in production.
  • Read only.

Limited Availability

Name

Type

Included in Subscription base license

Rel.

Description

AvroSystem Y9.2.0.0

Consists of two parts (two different ways of using Avro data):

  • Legacy Avro support using Ultra Format Definition Language to create and store the schema definition within the product. Data Decoder and Encoder support.
  • Regular Avro support using URLs to external schema registries. Data Decoder and Encoder support.

System Landscape Directory

SystemN9.0.0.0

Command to extract system topology in SAP SLD format.

TMO CollectionOnlineN9.1.0.0Customer specific extension.