Shared store examples

This page provides examples of how to utilize the shared store in both scheduled streaming and when HTTP server functions are used in a stream, along with a sample script that demonstrates how to write to and read from the shared store.

Example scheduled stream

Use case: Preloading reference data for other streams to use

Stream A reads a file from S3 containing 1,000 reference entries — for example, customer tiers or product mappings.
As it processes the file, Stream A writes this reference data (key-value pairs) to the shared store, such as:
user123 → goldTier, user456 → silverTier, etc.
Once Stream A finishes and commits its transaction, the data is persisted in the shared store.
Stream B, which runs later, can read this reference data from the shared store during its processing.

This is a common pattern for sharing lookup or configuration data between streams that run sequentially.

Scheduled stream shared store example

Example stream with HTTP server functions

For more details, see Streaming mode with HTTP server functions.

Example script snippet using shared store

This example shows how two separate streams can share data using the shared store.

We have two CSV files:

Stream A processes reference data:

id,tier
123,goldTier
456,silverTier
789,bronzeTier

Stream B collects usage data:

usage,userId,timestamp
3,123,20250530T13:12:10.001Z
5,456,20250530T13:15:43.221Z
2,789,20250530T13:17:29.100Z
4,123,20250530T13:19:55.010Z
6,456,20250530T13:21:03.331Z
1,789,20250530T13:22:47.401Z
2,123,20250530T13:24:12.789Z
3,456,20250530T13:26:08.050Z
5,789,20250530T13:28:30.900Z

Stream A – Writing reference data to shared store

// Script in Stream A
// Payload contains reference data
await sharedStore.set(`user${payload.id}`, { tier: payload.tier });

Stream A writes each user’s tier to the shared store using their ID as the key.

Stream B – Reading reference data from shared store

// Script in Stream B
// Payload contains usage data
const userReferenceData = await sharedStore.get(`user${payload.userId}`);

Stream B retrieves the corresponding reference data (like tier) based on the user ID in the usage record.