Streams
In Usage Engine, streams are created inside solutions. A stream can comprise various Collector, Forwarder, and Processor Functions each providing specific functionality, that is linked into automated data flows of virtually any complexity.
Adding First Stream to Solution
Once inside a new solution, you can create a new stream or import an existing one. The following section provides the different options available to create a stream:
Creating a Stream
Click Create Stream and enter the name of the new stream as shown below:
Create Stream
Clicking Create Stream takes you to the Stream Editor where you can create the stream of your choice using the Functions that are part of your subscription package.
New stream
Note!
When creating streams, depending on your subscription package, you will be presented with an option to configure your stream according to the number of records it needs to process. This is to support the concept of Performance and Scalability when handling streams in Usage Engine.
After clicking Create Stream, you must select the number of records to be processed per stream execution or stream run. Each stream is configured to process a maximum of 10,000,000 records per execution. If you want to process more than 10,000,000 records, you can configure stream replica(s) to process a large number of records without compromising on the performance of Usage Engine. You can choose the number of replicas you wish to create to effectively process your records with the least impact on performance.
However, there are some prerequisites that you need to ensure before configuring replicas. For more information about this, refer to Performance and Scalability.
Importing an Existing Stream
You can also import an existing stream into your current solution. You can import a stream either by dragging an exported file JSON file onto the upper part of the pop-up or the file contents can alternatively be pasted in the input box. Click on the Import button to confirm and import the stream. The imported stream is displayed in the Stream Editor.
Import stream
Note!
When you export and import a stream, the stream name is also retained. Ensure that you do not already have a stream with the same name in the solution that you import into.
When you click Export, a dialog appears with the code for the stream you have created. Click Copy to Clipboard to copy the contents or click Download the download the file locally.
Importing an Example Stream
There are example streams curated by the team for you to import and start exploring Usage Engine. Go to any example stream of your choice and click Import Stream to import that example stream.
Once you create your first stream, you will see the following options in the toolbar to alternatively enable you to create or import more streams inside the same solution.
Streams view
| Stream options icon | Description |
|---|---|
| Import example stream |
| Import stream |
| Create new stream |
In this section, the following chapters can provide more insight into the functioning of streams and provide examples for further understanding: