Botbot AI assist

Owned by Jonas Brandvik
Last updated: Aug 28, 2024 by Felicity Borrelli
2 min read

Usage Metering Extension

Licensing Information
The feature described in this documentation is optional and might not be part of the edition covered by your company's license.

This example stream demonstrates the use case of the company Botbot which offers AI-enabled chatbots that can be embedded in websites and products to offer virtual assistance.

Botbot wants to launch their new product offering, AI Assist, which comprises two models of chatbots. They want to charge their customers based on the number of API calls made to each model type within a session.

  • Level 1 (L1): L1 is trained to answer basic questions from customers. It attempts to resolve the user’s request as soon as a new session begins. If unable to solve the request, it will delegate the session to level 2. L1 is the cheaper of the two models.

  • Level 2 (L2): L2 is specially trained on Botbot’s customer data. It can resolve business-specific support requests autonomously and is more expensive than L1.

Botbot’s customers can choose from various Large Learning Models (LLMs) to power the L1 and L2 chatbots. Customers can select one model for each level.

These are the models available:

Level of Service

Available LLMs

Level of Service

Available LLMs

L1

  • Llama3

  • Gemini_Pro

  • Claude3_Sonnet

L2

  • GPT4o

  • Gemini_Advanced

  • Claude3_Opus

Metering Solution

In the Botbot AI assist example stream you will learn:

For more information on Usage Metering, see.Usage Metering

Botbot’s meter types

In this example stream you will create six meter types under two categories. Grouping meter types under categories allows new learning models to be easily added without the need to update the stream configuration.

Categories

Meter Types

Categories

Meter Types

L1_API_Request

  • L1_Llama3_Requests

  • L1_Gemini_Pro_Requests

  • L1_Claude3_Sonnet_Requests

L2_API_Request

  • L2_GPT4o_Requests, category

  • L2_Gemini_Advanced_Requests

  • L2_Claude3_Opus_Requests

Note!

This example stream needs to be executed alongside helper streams. Running the helper streams will configure the meter types and meters. In a real-world scenario, this is done using APIs integrated with your system. See Botbot AI assist - Metering Setup Guide for more details.

Botbot’s AI Assist stream

Botbot AI assist example stream

Functions in this stream and their purpose

Function

Purpose

Function

Purpose

Count

Counter that triggers the stream to run for the pre-defined number of times.

Script

Simulates the AI assist usage data via API calls towards each service.

Deduplicate 

Filters out duplicated records based on session ID.

Validate

Validates the region of the cases and only allows APAC, EMEA, or AMER.

Field organizer

Maps the data to the fields required by the Meter Processor.

Meter

Matches the usage data with its associated meter and aggregates the usage in meters based on user Id and value of API calls.

Log

Stores data in a log. This step substitutes data being sent to a billing system.

Validate Function

All records filtered by the https://infozone.atlassian.net/wiki/x/edZ3 and https://infozone.atlassian.net/wiki/x/WYHDB functions can be further processed and corrected using https://infozone.atlassian.net/wiki/x/AQAkAw.

Botbot’s meters

Meters are created when a B2B customer signs up for one of Botbot’s AI assist services. In this example stream the B2B customer is Cloud AI and they sign up for an AI assist plan with the following LLMs:

  • L1 model - Claude3_Sonnet

  • L2 model- Claude3_Opus

They require two meters to be created with the following meter types:

  • L1_Claude3_Sonnet_Requests

  • L2_Claude3_Opus_Requests

 

Subsections

This section has the following subsections:

Â