examples/AIServingIrisXGBoostPMMLModel.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inferencing Iris XGBoost PMML Model using AI-Serving"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PMML stands for Predictive Model Markup Language. It is the de facto standard to present the classic machine learning models. With PMML, it is easy to develop a model on one system using one application and deploy the model on another system using another application.\n",
    "\n",
    "In this tutorial, we will use the PMML to show how to deploy the famous Iris classifier using AI-Serving"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites to run the notebook\n",
    "#### 1. Train and export your model to PMML.\n",
    "\n",
    "In this example, we're going to use XGBoost to train our classifier. Once trained, we will convert our model to PMML that will be deployed in later steps. You can train, convert, and validate your model by simply running this notebook [`Training an Iris classifier using XGBoost`](https://github.com/autodeployai/ai-serving/blob/master/examples/IrisXGBoost.ipynb). Once your model is validated, you can deploy it.\n",
    "\n",
    "#### 2. Download the AI-Serving image.\n",
    "\n",
    "Pull the latest docker image of AI-Serving. Please, refer to [Docker Containers for AI-Serving](https://github.com/autodeployai/ai-serving/tree/master/dockerfiles) about more docker images.\n",
    "\n",
    "```bash\n",
    "docker pull autodeployai/ai-serving\n",
    "```\n",
    "\n",
    "#### 3. Start the docker image.\n",
    "\n",
    "Run a docker container of AI-Serving. The port `9090` is the port of HTTP endpoint while `9091` is for gRPC, you could see an error likes `Bind for 0.0.0.0:9090 failed: port is already allocated`, please use another new port instead of the first part as follows `-p $(NEW_PORT):9090` to run the command again, and remember the port is always needed in the URL of HTTP endpoint. \n",
    "\n",
    "```bash\n",
    "docker run --rm -it -v $(PWD):/opt/ai-serving -p 9090:9090 -p 9091:9091 autodeployai/ai-serving\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Additional information about two python files\n",
    "In the current directory, there are two python files `onnx_ml_pb2.py` and `ai_serving_pb2.py`, which are generated from compiling the [two proto files](https://github.com/autodeployai/ai-serving/tree/master/src/main/protobuf) using [protoc](https://developers.google.com/protocol-buffers/docs/pythontutorial), for example, the command as follows:\n",
    "\n",
    "```bash\n",
    "protoc -I=$SRC_DIR --python_out=. ai-serving.proto onnx-ml.proto\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import dependent libraries\n",
    "Import some dependent libraries that we are going to need to run the Iris XGBoost model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import requests\n",
    "from pprint import pprint\n",
    "\n",
    "import onnx_ml_pb2\n",
    "import ai_serving_pb2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the base HTTP URL\n",
    "Change the port number `9090` to the appropriate port number if you had changed it during AI-Serving docker instantiation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "port = 9090\n",
    "base_url = 'http://localhost:' + str(port)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test the server availability\n",
    "Use the specific endpoint `http://host:port/up` to test whether the server has been initialized and is ready to accept requests. The `OK` message indicates it's already available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_url = base_url + '/up'\n",
    "response = requests.get(test_url)\n",
    "print('The status of the server: ', response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy the PMML model into AI-Serving\n",
    "First, we need to deploy the PMML model `xgb-iris.pmml` into AI-Serving, which can serve multiple models or multiple versions for a named model at once.\n",
    "\n",
    "You must specify a correct content type for PMML models when constructing an HTTP request to deploy a PMML model, the candidates are:\n",
    " * application/xml\n",
    " * text/xml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The specified servable name\n",
    "model_name = 'iris'\n",
    "deployment_url = base_url + '/v1/models/' + model_name\n",
    "\n",
    "# The specified content type for the model:\n",
    "headers = {'Content-Type': 'application/xml'}\n",
    "\n",
    "model_path = 'xgb-iris.pmml'\n",
    "with open(model_path, 'rb') as file:\n",
    "    deployment_response = requests.put(deployment_url, headers=headers, data=file)\n",
    "\n",
    "# The response is a JSON object contains the sepcified servable name and the model version deployed\n",
    "deployment_response_info = deployment_response.json()\n",
    "print('The depoyment response: ', deployment_response_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retrieve metadata of the deployed model\n",
    "The metadata will contain model inputs and outputs, which are needed when constructing an input request and consume an output response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_version = deployment_response_info['version']\n",
    "metadata_url = base_url + '/v1/models/' + model_name + '/versions/' + str(model_version)\n",
    "metadata_response = requests.get(metadata_url)\n",
    "\n",
    "print('The model metadata response:\\n')\n",
    "pprint(metadata_response.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## HTTP request formats for the AI-Serving\n",
    "The request for AI-Serving could have two formats: JSON and binary, the HTTP header Content-Type tells the server which format to handle and thus it is required for all requests. The binary payload has better latency, especially for the big tensor value for ONNX models, while the JSON format is easy for human readability.\n",
    "\n",
    "- Content-Type: application/json. The request body must be a JSON object formatted as described [here](https://github.com/autodeployai/ai-serving#4-predict-api).\n",
    "\n",
    "\n",
    "- Content-Type: application/octet-stream, application/vnd.google.protobuf or application/x-protobuf. The request body must be the protobuf message PredictRequest, besides of those common scalar values, it can use the standard TensorProto value directly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Construct JSON requests for the AI-Serving\n",
    "We will create both JSON objects, one is using the `Records` format that contains a single record, the other is using `Split` format that contains two records."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a JSON object using `records` that contains a single record.\n",
    "request_json_recoreds = {\n",
    "    'X': [{\n",
    "        'sepal length (cm)': 5.7,\n",
    "        'sepal width (cm)': 4.4,\n",
    "        'petal length (cm)': 1.5,\n",
    "        'petal width (cm)': 0.4\n",
    "    }]\n",
    "}\n",
    "\n",
    "# Create a JSON object using `split` that contains two records with a filter that\n",
    "# only the output `predicted_Species` is expected.\n",
    "request_json_split = {\n",
    "    'X': {\n",
    "        'columns': ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)'],\n",
    "        'data': [[5.7, 4.4, 1.5, 0.4], [6.4, 2.8, 5.6, 2.1]]\n",
    "    },\n",
    "    'filter': ['predicted_species']\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make the HTTP requests with JSON data to the AI-Serving\n",
    "Make predictions using the AI-Serving, the content type of requests with JSON data must be `application/json`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# When version is omitted, the latest version is used.\n",
    "prediction_url = base_url + '/v1/models/' + model_name\n",
    "\n",
    "# The Content-Type: application/json is specified implicitly when using json instead of data\n",
    "prediction_json_response_records = requests.post(prediction_url, json=request_json_recoreds)\n",
    "prediction_json_response_split = requests.post(prediction_url, json=request_json_split)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Consume the HTTP response with JSON data from the AI-serving\n",
    "Having received the results from the server, we are going to parse the JSON text that we just received for us to make sense of the results. \n",
    "\n",
    "**NOTE: The data format of the output response is always the same as the input request.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('JSON prediction response of the request using `records`:')\n",
    "pprint(prediction_json_response_records.json())\n",
    "\n",
    "print('\\n----------------------------------------------------------------------------\\n')\n",
    "\n",
    "# Only the predicton column `predicted_Species` is expected.\n",
    "print('JSON prediction response of the request using `records` with a filter:')\n",
    "pprint(prediction_json_response_split.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Construct binary requests for the AI-Serving\n",
    "We will create both instances of PredictRequest, one is using the `Records` format that contains a single record, the other is using the `Split` format that contains two records."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ai_serving_pb2 import RecordSpec, Record, PredictRequest, ListValue, Value\n",
    "\n",
    "# Create an instance of RecordSpec using `records` that contains a single record.\n",
    "request_message_records = PredictRequest(X=RecordSpec(\n",
    "    records=[Record(fields={\n",
    "        'sepal length (cm)': Value(number_value=5.7),\n",
    "        'sepal width (cm)': Value(number_value=4.4),\n",
    "        'petal length (cm)': Value(number_value=1.5),\n",
    "        'petal width (cm)': Value(number_value=0.4),\n",
    "    })]))\n",
    "\n",
    "# Create an instance of RecordSpec using `split` that contains two records with a filter that\n",
    "# only the output `predicted_Species` is expected.\n",
    "request_message_split = PredictRequest(\n",
    "    X=RecordSpec(\n",
    "        columns = ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)'],\n",
    "        data = [\n",
    "            ListValue(values=[Value(number_value=5.7), Value(number_value=4.4), Value(number_value=1.5), Value(number_value=0.4)]),\n",
    "            ListValue(values=[Value(number_value=6.4), Value(number_value=2.8), Value(number_value=5.6), Value(number_value=2.1)])]),\n",
    "    filter=['predicted_species'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make the HTTP requests with binary data to the AI-Serving\n",
    "Make predictions using the AI-Serving, the content type of requests with binary data must be one of those three candidates above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "headers = {'Content-Type': 'application/x-protobuf'}\n",
    "\n",
    "# When version is omitted, the latest version is used.\n",
    "prediction_url = base_url + '/v1/models/' + model_name\n",
    "\n",
    "# Make prediction for the `records` request message.\n",
    "prediction_response_records = requests.post(prediction_url, \n",
    "                                           headers=headers, \n",
    "                                           data=request_message_records.SerializeToString())\n",
    "\n",
    "# Make prediciton for the `split` request message.\n",
    "prediction_response_split = requests.post(prediction_url, \n",
    "                                           headers=headers, \n",
    "                                           data=request_message_split.SerializeToString())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Consume the HTTP response with binary data from the AI-serving\n",
    "Having received the results from the server, we are going to parse the \"serialized\" message that we just received for us to make sense of the results.\n",
    "\n",
    "**NOTE: The data format of the output response is always the same as the input request.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parse the response message from the `recrods` request.\n",
    "response_message = ai_serving_pb2.PredictResponse()\n",
    "response_message.ParseFromString(prediction_response_records.content)\n",
    "print('Binary prediction response of the request using `records`:')\n",
    "print(response_message)\n",
    "\n",
    "print('\\n----------------------------------------------------------------------------\\n')\n",
    "\n",
    "# Parse the response message from the `split` request.\n",
    "response_message = ai_serving_pb2.PredictResponse()\n",
    "response_message.ParseFromString(prediction_response_split.content)\n",
    "print('Binary prediction response of the request using `split` with a filter:')\n",
    "print(response_message)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}