Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Term or AcronymDefinition

Batch stream

Anchor
batch stream
batch stream

A stream processing high volumes of data in batch(es) within a specific time span. It processes large amount of records in one transaction. For example, data file read via S3 bucket.

For batch streams, the 7799838 happens only when a stream is run.

Bounded Stream

Anchor
collector
collector

A unidirectional 7799838 that processes finite units of input, for example, complete files or a fixed amount of rows in a database.

Ui text box

Example:

Collecting .csv files via SFTP and only setting a "committed" checkpoint for a particular input file once it is guaranteed an entire file has been successfully processed.


Collector

Anchor
collection
collection

A function that collects data from a 7799838.

Collection

Anchor
client_mode
client_mode

The process of Dazzler Cloud Edition acquiring data from a 7799838.

Ui text box

Examples:

  • Collecting .csv files from an S3 bucket by acting as an HTTPS client
  • Collecting RESTful requests by acting as an HTTPS server


Client Mode

Anchor
decode
decode

A communication mode where collectors initiate outgoing requests.

Ui text box
Examples:
  • Dazzler Cloud Edition acting as an HTTPS client.Dazzler acting
  • Cloud Edition acting as an SFTP client.
  • Dazzler acting Cloud Edition acting as a JDBC client.


Custom References

Custom references refer to an optional feature that is used with SAP-related functions. When enabled it allows users to specify one or more typeCode and id values in the relevant usage records. 

Anchor
decode
decode
Decode


The process of parsing input from a 7799838 and converting it into records in a format that can be sent across a 7799838.

Destination

Anchor
destination
destination


An endpoint to which data is sent from a 7799838.

Encode

Anchor
encode
encode

The process of formatting a record into a format that can be understood by a 7799838.

Ui text box

Example:

Creating a JSON string from a 7799838.


Endpoint

Anchor
endpoint
endpoint

An interface used by the service to connect to an 7799838. An endpoint may function in either server mode or client mode.

Ui text box

Examples:

  • S3 bucket
  • SFTP server
  • HTTP server
  • HTTP client
  • Database instance


Flush

Anchor
flush
flush


The flush operation forces to pass the stored/buffered (aggregated) data to the next function in the stream.


Forwarder

Anchor
forwarding
forwarding

A Function that sends data to a given 7799838.

Forwarding

Anchor
function
function

The process of sending data to a 7799838.

Ui text box

Examples:

  • Sending JSON records to a billing system via HTTPS
  • Storing .csv files in an S3 bucket


Function

Anchor
record
record

A Function accepts input, applies logic to it, and then returns a result. Functions may be stateful.

Ui text box

Examples:

  • Filtering fields from records
  • Renaming fields in records
  • Aggregating multiple records.


HTTP Response Headers

This is an option found within the HTTP Client collector and processor function, intended to support the sending out of response headers in the network stream. The users can enter a custom value in the result header key and associated result key to configure the target web service. 

Notification Group

Anchor
Notification Group
Notification Group

 A notification group is an email list to which automated messages regarding erroneous events related to streams are being sent. The content of the error messages include stream execution errors, missing files, and related events. 

Payload

Anchor
payload
payload

Refers to the current record.

Record

Anchor
route
route

A unit of data that can be processed by a Function or multiple functions and sent across routes.

Ui text box

Examples:

  • A decoded line in a .csv file.
  • A decoded JSON payload in an HTTP request.
  • A decoded row in a relational database.
  • A decoded Kinesis message.


RegEx (Regular Expression)

The use of regular expressions implies the creation of custom schemes to which various operations can be executed. 

Real-time stream

Anchor
realtime stream
realtime stream

Stream processing data in real-time. The data size is unknown, infinite and continuous. During real-time streams, the data output rate is as fast as data input rate. When the incoming data is continuous and requires immediate response then in that case real-time streams are used. For example, data continuously send via HTTP collector.

For real-time streams, the 7799838 happens after every 60 seconds. 

Route

Anchor
solution
solution

A route is a link between two functions, across which data flows.

Ui text box

Example:

A 7799838 exposing two outgoing routes, e.g. records that are either successfully processed or erroneous records.


Session

Anchor
agg_session
agg_session

A representation of aggregated records that is stored before being flushed out. 

Solution

Anchor
source
source

A solution is a set of streams that together solve a particular customer problem.

Ui text box

Examples:

  • Streams required to perform billing mediation for a cloud biller on incoming usage records
  • Streams required to acquire the product catalog data for a cloud biller


Source


An 7799838 from which data is collected into a 7799838 using a given protocol and data format.

Stream

Anchor
stream
stream

A graph representation of data flowing from a 7799838, through a series of functions, to a 7799838

Ui text box

Example:

A stream collecting CSV files from an Amazon S3 source and forwarding them to a destination database using JDBC.


Time To Leave (TTL) 

This is a mechanism that limits the lifespan of data in an environment. It is used in functions like aggregation to express the date and time where a record expires. 

Timeout

Anchor
timeout
timeout

A method that allows to set a timer to the execution of a certain function. Timeouts allow for more efficient usage of limited resources. The basic idea is that in situations where a system must wait for something to happen, rather than waiting indefinitely, the waiting will be aborted after the timeout period has elapsed.
Transaction

A transaction is a unit of data that is being processed inside a stream. The definition of a completed transaction is when the stream processes it without any errors. The transactions by design are limited to their specified data sources.

A transaction can refer to a file that is being processed in a stream or a result set from a query. 

The data correction feature also creates transactions of reference and they are regarded as separate processes. 

For more information refer to Transaction Safety.

Transactions Per Second (TPS)

This is a metric showing the number of transactions per second (tps) that can be processed by a given web service. This value can be calculated using this formula:

Tps value for a function = (Total number of output events handled by the function) / (duration of the stream)

...