Grafana & Prometheus
This tutorial uses docker-compose locally to demonstrate training a model, recording its training data histogram, then deploying it with a simple Flask server and then observing the model's divergence in Grafana & Prometheus.
git clone https://github.com/boxkite-ml/boxkite
python3 virtual environment and activate it. Then install the training dependencies.
pip install -r requirements.txt pip install -r app/requirements.txt
The training script generates a linear regression model using sklearn's diabetes dataset.
Running the command above creates two files in the current directory:
The serving script creates a flask server that uses the trained model.
You can test the server by sending a HTTP request using
curl localhost:5000 -H "Content-Type: application/json" \ -d "[0.03, 0.05, -0.002, -0.01, 0.04, 0.01, 0.08, -0.04, 0.005, -0.1]"
Make sure you have generated
model.pkl in your current directory before continuing.
Start the serving container with Prometheus using
docker-compose. It automatically scrapes the flask server every 15 seconds for feature distribution metrics.
requests library and call
metrics/load.py to generate some load.
pip install requests python3 metrics/load.py
Navigate to http://localhost:3000 for the Grafana GUI, then login with
admin, skip changing the admin password, then navigate to Dashboards -> Manage -> Model Metrics: