An addition (a tab in the UI) to the Disk, FTP, SFTP, and SCP collection agents. Enables APL code to be used to define complex collection rules. Includes function blocks for detailed filtering and processing for:

• List of directories to scan.
• List of files to collect.
• Actions to take before collection.
• Actions to take after collection.

Add-ons to the APL library of functions. Contains:

• File functions: list directories, list files, move and delete files.
• Dynamic functions: dynamic calls to an agent, a plug-in, or a generic APL function.
• Base64 functions: decoding and encoding of Base64 data.
• Map functions: create, delete and handle maps.
• Log and notifications functions; send debug, system log entries, and alrams.
• Data signing functions: generate digital signatures for data:

• • Gets keys from Java keyword file:
    • JKS, PKCS8, PKCS12 or CA_CERTIFICATES_PATH.
  • Calculates hash values and signs result using various algorithms:
    • SHA1withDSA, SHA1withRSA or SHA256withRSA.
    • Hmac-SHA256 or Hmac-SHA512.

Addition to the Database Profile; DB connectivity package (JDBC) for Amazon Redshift. Usage:

• Database Table Related Functions (APL).
• SQL Collection/Forwarding Agents.

Azure Application Insight Event Notifier

Event Notification support for Azure Application Insight.

APL plugin for acquiring an OAuth2-token from Azure servers.

DB connectivity package (JDBC) for CSV files that makes it possible to read CSV files instead of databases when:

• Using database related table functions in APL.
• Using shared table configurations.

Addition to the Database Profile; DB connectivity package (JDBC) for Cassandra. Usage:

• SQL Collection/Forwarding Agents.

Categorized Grouping

Offline

9.0.0.0

Collection Strategy - Control Files

Offline

9.0.0.0

Collection Strategy – Duplicate Filter

Offline

9.0.0.0

Collection Strategy – Multi Directory

Offline

9.0.0.0

Compression agent

Offline

9.0.0.0

Processing agents that compresses data (GZIP format).

Conditional Trace has been rewritten to track all changes to selected UDRs in a workflow, and be plug-and-playable to Jaeger, a state-of-the-art trace logging system. This provides for complete trace logs and a rich set of filtering and troubleshooting tools.

Couchbase for Aggregation

Online

9.0.0.0

Consist of Profile, that defines connection to an Impala database via Cloudera JDBC and HDFS, and two agents. The Data HubForwarding Agent loads data in bulks into Impala databases. The Data HubTask Agent removes old partitions. 

Reference:

• Cloudera Impala (https://www.cloudera.com)

A Profile and Processing agent, that either masks or unmasks specified fields in selected UDRs. Available masking methods:

• Crypto: AES-128 or AES-256. Can be used for both obscuring and unmasking data.
• Database: stores masked and unmasked data in a database.
• Hash: obscures data. 

Data Masking Hash/Database Storage

Support for Data Masking Hash/Database Storage.

An alternative to the Error Correction System available out-of-the-box in the platform. Stores data failing validation in an external database, and is better suited for high load operations. 

• Data Veracity Forwarding Agent: sends data to the storage.
• Data Veracity Collection Agent: collects data from the storage for reprocessing.
• Data Veracity Task Agent: configures and schedules repair tasks for data in storage.
• Data Veracity UI: used to examine the storage, and define configuration like error codes and repair tasks.

Database APL Functions

Both

9.0.0.0

Add-ons to the APL library of functions. Contains:

• Database bulk SQL; bundles multiple queries into one SLQ statement. Available for batch workflows only.
• Callable statements; enables use of Stored Procedures with output parameters.
• Prepared statements; enables efficient execution of an SQL statement multiple times.

Decompression Agent

Offline

9.0.0.0

Distributed Storage Couchbase

Online

9.0.0.0

Enables access to a Couchbase distributed storage solution from APL without having to provide details about its type or implementation. This makes APL code generic and independent of storage solutions changes, which are handled in the Profile.

APL functions available for data management:

• Read, store and remove data.
• Transaction management and bulk processing.

Enables access to a Couchbase distributed storage solution from APL without having to provide details about its type or implementation. This makes APL code generic and independent of storage solutions changes, which are handled in the Profile.

APL functions available for data management:

• Read, store and remove data.
• Transaction management and bulk processing.

Free format SQL loader agent

APL plug-in for HTTP requests handling.

Addition to the Database Profile; DB connectivity package (JDBC) for IBM DB2. Usage:

• Database Table Related Functions (APL)
• Database Collection/Forwarding Agents
• Prepared Statements (APL)
• SQL Collection/Forwarding Agents

IBM Netezza

System

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for IBM Netezza. Usage:

• sqlexec (APL)
•  SQL Loader Agent

JSON Interface

System

9.0.0.0

Enables JSON decoding and encoding. 

Reference:

• RFC7159

Support for JSON Web Tokens (JWT) signature verification.

LDAP Authentication

System

LZO Support

Adds LZO decompression support to a collection agent's configuration UI. Any flags to the standard unix command "lzop" can be added.

Reference:

• https://www.lzop.org/lzop_man.php

MS SQL Server

System

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for SQL Server. Usage:

• Audit Profile
•  Database Bulk Lookup Functions (APL)
•  Database Table Related Functions (APL)
•  Database Collection/Forwarding Agents
•  Event Notifications
•  Prepared Statements (APL)
•  SQL Collection/Forwarding Agents
•  Task Workflows Agents (SQL)

Database connection support for MariaDB database that can be used in:

• Reference Data Management

Merge File Collection

Offline

9.0.0.0

Collects and merges files based on number of files or bytes. A collected batch will consist of all files matching the search criteria for a particular workflow activation.

MySQL

System

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for MySQL. Usage:

•  Database Bulk Lookup Functions (APL)
•  Database Table Related Functions (APL)
•  Event Notifications
•  Prepared Statements (APL)
•  SQL Collection/Forwarding Agents
•  SQL Loader Agent
•  Task Workflows Agents (SQL)

PKCS7

System

9.0.0.0

APL functions supporting signing of data using CMS signature.

• Loading of PKCS7 certificates.
• Signing of bytearrarys with PKCS7 certificates. 

PostgreSQL

System

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for PostgreSQL. Usage:

• Audit Profile
• Aggregation Profile
• Callable Statements (APL)
• Database Bulk Lookup Functions (APL)
• Database Table Related Functions (APL)
• Database Collection/Forwarding Agents
• Event Notifications
• Prepared Statements (APL)
• Reference Data Management
• Shared Table Profile
• SQL Collection/Forwarding Agents
• SQL Loader Agent
• Task Workflows Agents (SQL)

SAP Hana

System

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for Snowflake. Usage for SAP Hana v1 and v2:

• Aggregation Profile
• Callable Statements (APL) 
• Database Bulk Lookup Functions (APL)
• Database Table Related Functions (APL)
• Database Agents
• Event Notifications
• Prepared Statements (APL)
• Shared Table Profile
• SQL Collection/Forwarding Agents
• SQL Loader Agent
• Task Workflows Agents (SQL)

In addition to the functions listed above, the SAP HANA database 2.0 can be used with the following functionality:

• Audit Profile

SCIM

A REST HTTP interface enabling SCIM management of users and groups. Supports POST, GET, DELETE, PUT and PATCH methods for user and group configurations.  

References:

• https://tools.ietf.org/html/rfc7643
• https://tools.ietf.org/html/rfc7644#section-3.2

Free format SQL collection agent:

Collects rows from databases and inserts them as UDRs into a workflow. A configurable SQL query is executed upon workflow activation, and all matching rows will comprise a batch of UDRs. The format of the UDRs is configurable.

Several databases are supported,  please refer to the System Extensions section.

Free format SQL forwarding agent:

Inserts UDRs as rows in database tables. Table columns can be populated with UDR fields or MIM values. Optionally, commit window size can be set. Configurable exception handling allows for alternate routes for failed inserts.

Several databases are supported,  please refer to the System Extensions section.

Aggregation storage based on SQL relational database:

Allows batch aggregation to be done using databases instead of using persistent geo-redundant disk storage.

Enables Single Sign On with external OpenID Provider, along with multi factor authentication.

Shared Memory Table

Addition to the Database Profile; DB connectivity package (JDBC) for Snowflake. Usage:

• sqlExec Function (APL)
• SQL Collection/Forwarding Agents

Sybase IQ

System

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for Sybase IQ. Usage:

• Database Table Related Functions (APL)
  
• APL function sqlexec 
• Database Collection/Forwarding Agents
• Event Notifications
• SQL Collection/Forwarding Agents
• SQL Loader Agent
• Task Workflows Agents (SQL)

Addition to the Database Profile; DB connectivity package (JDBC) for Teradata. Usage:

• Database Table Related Functions (APL)
• Database Collection Agent
• Prepared Statements
• SQL Collection Agents

TimesTen

System

9.0.0.0

Addition to the Database Profile; DB connectivity package (JDBC) for TimesTen. Usage:

• Audit Profile
•   Database Bulk Lookup Functions (APL)
•   Database Table Related Functions (APL)
•   Event Notifications
•   Prepared Statements (APL)
•   Shared Table Profile
•   SQL Collection/Forwarding Agents
•   Task Workflows Agents (SQL)

Ultra GPB support (Google Protocol Buffer)

System

9.0.0.0

Enables GPB definition support in UFDL, allowing GPB data to be Decoded and Encoded. Both the proto2 and proto3 versions of the GPB language are supported.

Ultra XML

System

9.0.0.0

Enables test of workflows and APL Profiles. Tests are configured using Python programming, and can be exported and imported the same way as any other configuration

Addition to the Database Profile; DB connectivity package (JDBC) for Vertica. Usage:

•  Audit Profile

• Database Table Related Functions (APL)

• Database Collection/Forwarding Agents

• Event Notifications

• Prepared Statements (APL)

• Shared Table Profile

• SQL Collection/Forwarding Agents

• SQL Loader Agent

• Task Workflows Agents (SQL)

Web API

System

9.0.0.0

Feature (HTTP-based API enabling external systems to collect status information from selected parts of the system, including stop and start workflows and groups) is sunset, and is replaced with Operations API. Note that full coverage and synch PE vs MZ is done incrementally.

Workflow bridge Batch -> Realtime

Both

9.0.0.0

Workflow bridge Realtime-> Realtime

Online

9.0.0.0

Configuration can be bundled into workflow packages and exported to other systems. Advantages compared to regular export format:

• Self-contained; can execute independent of configuration outside the package.
• Pre-compiled; enables fast and easy upgrades and rollbacks in production.
• Read only.

Consists of two parts (two different ways of using Avro data):

• Legacy Avro support using Ultra Format Definition Language to create and store the schema definition within the product. Data Decoder and Encoder support.
• Regular Avro support using URLs to external schema registries. Data Decoder and Encoder support.

System Landscape Directory

Command to extract system topology in SAP SLD format.