Streams

Owned by Former user (Deleted)
Last updated: Aug 16, 2024 by Felicity Borrelli
3 min read

In Usage Engine, streams are created inside solutions. A stream is made up of Collector, Forwarder, and Processor Functions that provide specific functionality and are linked to automated data flows of varying complexity. 

Adding Your First Stream to a Solution

Inside a solution, you can create a new stream or import an existing one. The following section provides the different options available to create a stream. When you first create a solution you will be directed to a landing page where you can choose to create or import your own stream, or import an example stream.

 

overview screen - only shown on a newly created Solution

Once you have created or imported a stream you will see a list of your streams in your Solution Overview.

 

Tools in the Solution Overview toolbar

You can use the tools in the Solution Overview toolbar to help you create a stream or import an existing or example stream.

 

Stream options icon

Description

Stream options icon

Description

Import example stream

Import stream

Create new stream

 

Creating a Stream

In your Solutions Overview click on the solution where you want to add a stream. Click on the 'plus' + icon in the toolbar, enter the name of the new stream and click Create Stream

Clicking Create Stream takes you to the Stream Editor where you can create the stream of your choice using the Functions that are part of your subscription package. 

 

Note!

When creating streams, depending on your subscription package, you will be presented with an option to configure your stream according to the number of records it needs to process. This is to support the concept of Performance and Scalability when handling streams in Usage Engine.

After clicking Create Stream, you must select the number of records to be processed per stream execution or stream run. Each stream is configured to process a maximum of 10 million records per execution. If you want to process more than 10 million records you can configure one or more stream replicas. This will enable you to process a larger number of records without compromising on the performance of Usage Engine. You can choose the number of replicas you wish to create.

There are some prerequisites that you need to ensure before configuring replicas. For more information about this see Performance and Scalability.

Importing an Existing Stream

You can import an existing stream into your solution in a JSON file format. Click the top part of the box to select from your saved files, or drag and drop the file into this top section. You can also paste the file directly into the box below. The imported stream is displayed in the Stream Editor.

 

Note!

When importing a stream its name is retained. Please ensure that you import streams into your solution with unique names.

When you click Export, a dialog appears with the code for the stream you have created. Click Copy to Clipboard to copy the contents or click Download to download the file locally.

 

Importing an Example Stream

The example streams have been created for you to import so that you can explore Usage Engine. Once you have created your first stream, you will see the following options in the toolbar. The first icon, Import example stream, will return you to the landing page with the example streams.

 

Choose an example you want to import and click Import stream

 

 

Subsections

This section has the following subsections:

Â