This section provides an example of how you can use the Python processing and Python Connector agents for machine learning.

In this example the classic iris dataset is used. With the help of scikit-learn a model of this dataset is built to predict observations. The iris dataset consists of 3 different types of iris' (Setosa, Versicolor, and Virginica) petal and sepal length, with 50 observations each, totaling 150 observations. The problem is presented as a supervised learning problem, with the aim of predicting the species of an iris using measurements of petal and sepal lengths.

There is a production flow, where queries are served via web services requests, and an exploration flow for model validation.

Workflow example with Python processing and Python Connector agents

Workflow Configuration

The sections below provide descriptions of agent configurations for this example workflow.

WS_Handler

The Analysis agent is configured to handle requests from the web services agent. The request is translated into a format required by the machine learned model. Responses from the model are sent back to the web services agent as the result of the prediction.

Example - Code for WS_Handler

import ultra.Iris.UFL_Types;
import ultra.ws.Iris.PRF_WebService.Xtraining;

consume {
    debug(input);
    if (instanceOf(input, WSCycle_predict)) {
        WSCycle_predict cycle = (WSCycle_predict) input;
        PredictObservation obs = udrCreate(PredictObservation);
        obs.observation = listCreate(double, cycle.param.sepal_length, cycle.param.sepal_width, 
            cycle.param.petal_length, cycle.param.petal_width);
        obs.context = cycle;
        udrRoute(obs, "observation");
    }
    if (instanceOf(input, PredictObservation)) {
        PredictObservation obs = (PredictObservation) input;
        WSCycle_predict cycle = (WSCycle_predict) obs.context;
        Response response = udrCreate(Response);
        response.iris = obs.prediction;
        cycle.response = response;
        udrRoute(cycle, "response");
    }
}

Predict

The Python processing agent processes UDRs by defining a consume block, and uses the selected Interpreter profile that is used to configure the Python executables.

The input is either a UDR of type InstallModel in which case the current model is replaced by a recently trained model, or a UDR of type PredictObservation in which case the information in the input is used as input to the model to predict the species of iris.

Example - Code for Predict

import pickle

model = None
targets = None

def consume(input):
    if isinstance(input, InstallModel):
        global model, targets
        model = pickle.loads(input.model)
        targets = input.targets
    elif isinstance(input, PredictObservation):
        if model:
            idx = model.predict([input.observation])[0]
            input.prediction = targets[idx]
        else:
            input.prediction = 'please install model'
        if input.testing:
            debug(input)
            udrRoute(input, 'test_prediction')
        else:
            udrRoute(input, 'prediction')

Python_Connector

The Python Connector agent is configured to bind on port 3810 from which the data will be received. The types accepted for routing are PredictObservation (Iris.UFL_Types) and InstallModel (Iris.UFL_Types) on route r_1.

The Python Connector agent implements a custom API that does all the workflow interaction and provides a high level API to the exploration tool.

Example - Code for Python Connector API

import pickle
from .UFL_Types import InstallModel
from .UFL_Types import PredictObservation

def install_model(model, targets):
    udr = InstallModel(
        model=pickle.dumps(model),
        targets=targets)
    udrRoute(udr)

def predict(observation):
    udr = PredictObservation(
        observation=observation,
        testing=True)
    udrRoute(udr)
    return udrConsume().prediction

__all__ = ['install_model', 'predict']

Exploration Tool

You start the workflow, open the exploration tool of your choice, and run your script.

Usage Engine Private Edition 4 Documentation

Python Agents Example - Real-Time (4.3)