Creating and Deploying a Python Machine Learning Service

Creating and Deploying a Python Machine Learning ServiceBuild a hate speech detection system with scikit-learn and deploy it via Docker on Heroku.

Dominik HaitzBlockedUnblockFollowFollowingApr 8Photo by Jon Tyson on UnsplashIntroductionImagine you’re the moderator of a message board or comment section.

You don’t want to read everything your users write online, yet you want to be alerted in case a discussion turns sour or people start spewing racial slurs all over the place.

So, you decide to build yourself an automated system for hate speech detection.

Text classification via machine learning is an obvious choice of technology.

However, turning model prototypes into working services has proven to be a widespread challenge.

To help bridge this gap, this four-step tutorial illustrates an exemplary deployment workflow for a hate speech detection app:Train and persist a prediction model with scikit-learnCreate an API endpoint with fireflyCreate a Docker container for this serviceDeploy the container on HerokuThe code for this project is available here.


Create prediction modelDatasetThe approach is based on the paper Automated Hate Speech Detection and the Problem of Offensive Language by Davidson, Warmsley, Macy and Weber.

Their results are based on more than 20 000 labelled tweets, which are available on the corresponding Github page.

The .

csv file is loaded as a dataframe:import pandas as pdimport redf = pd.


csv', usecols=['class', 'tweet'])df['tweet'] = df['tweet'].

apply(lambda tweet: re.

sub('[^A-Za-z]+', ' ', tweet.

lower()))The last line cleans the tweet column by converting all text to lowercase and removing non-alphabetic characters.

Result:The class attribute can assume three category values: 0 for hate speech, 1 for offensive language and 2 for neither.

Model trainingWe have to convert our predictors, i.


the tweet text, into a numeric representation before we can train a machine learning classifier.

We can use scikit-learn’s TfidfVectorizer for this task, which transforms texts into a matrix of term-frequency times inverse document-frequency (tf-idf) values, suitable for machine learning.

Additionally, we can remove stop words (common words such as the, is, …) from the processing.

For text classification, support vector machines (SVMs) are a reliable choice.

As they are binary classifiers, we will use a One-Vs-Rest strategy, where for each category an SVM is trained to separate this category from all others.

Both text vectorization and SVM training can be performed in one command by using scikit-learn’s Pipeline feature and defining the respective steps:from sklearn.

pipeline import make_pipelinefrom sklearn.


text import TfidfVectorizerfrom sklearn.

multiclass import OneVsRestClassifierfrom sklearn.

svm import SVCfrom stop_words import get_stop_wordsclf = make_pipeline( TfidfVectorizer(stop_words=get_stop_words('en')), OneVsRestClassifier(SVC(kernel='linear', probability=True)))clf = clf.

fit(X=df['tweet'], y=df['class'])Now, the performance of the model should be evaluated, e.


using a cross-validation approach to calculate classification metrics.

However, as this tutorial focusses on model deployment, we will skip this step (never do this in an actual project).

The same goes for parameter tuning or additional techniques of natural language processing which are described in the original paper.

Test the modelWe can now try a test text and have the model predict the probabilities:text = "I hate you, please die!"clf.


lower()])# Output:array([0.

64, 0.

14, 0.

22])The numbers in the array correspond to the probabilities for the three categories (hate speech, offensive language, neither).

Model persistenceUsing the joblib module, we can save the model as a binary object to disk.

This will allow us to load and use the model in an application.

from sklearn import externalsmodel_filename = 'hatespeech.




dump(clf, model_filename)2.

Create REST APICreate API endpointThe python file app.

py loads the model and defines a simple module-level function which wraps the call to the model’s predict_proba function:from sklearn import externalsmodel_filename = 'hatespeech.


z'clf = externals.


load(model_filename)def predict(text): probas = clf.


lower()])[0] return {'hate speech': probas[0], 'offensive language': probas[1], 'neither': probas[2]}Now, we use firefly, a lightweight python module for function as a service.

For advanced configuration or use in a production environment, Flask or Falcon might be a better choice as they’re well established with a large community.

For rapid prototyping, we’re fine with firefly.

We’ll use firefly on the command line to bind the predict function to port 5000 on localhost:$ firefly app.

predict –bind 127.



1:5000Test API locallyVia curl, we can make a POST request to the created endpoint and obtain a prediction:$ curl -d '{"text": "Please respect each other.

"}' http://127.



1:5000/predict# Output:{"hate speech": 0.

04, "offensive language": 0.

31, "neither": 0.

65}Of course, in a full-fledged real application there would be much more additional features (logging, input and output validation, exception handling, …) and work steps (documentation, versioning, testing, monitoring, …), but here we’re merely deploying a simple prototype.


Create a Docker containerWhy Docker?.A Docker container runs an application in an isolated environment, with all dependencies included, and can be shipped as an image, thus simplifying service setup and scaling.

Build imageWe have to configure the contents and start-actions of our container in a file named Dockerfile:FROM python:3.

6RUN pip install scikit-learn==0.


2 firefly-python==0.


15COPY app.

py hatespeech.


z .

/CMD firefly app.

predict –bind 0.



0:5000EXPOSE 5000The first three lines are about taking python:3.

6 as base image, additionally installing scikit-learn and firefly (the same versions as in the development environment) and copying the app and model files inside.

The latter two lines tell Docker the command which is executed when a container is started and that port 5000 should be exposed.

The build process that creates the image hatespeechdetect is started via:$ docker build .

-t hatespeechdetectRun ContainerThe run command starts a container, derived from an image.

Additionally, we’re binding the containers’s port 5000 to the host’s port 3000 via the -p option:$ docker run -p 3000:5000 -d hatespeechdetectUse prediction serviceNow, we can send a request and obtain a prediction:$ curl -d '{"text": "You are fake news media!.Crooked!"}' http://127.



1:3000/predict# Output:{"hate speech": 0.

08, "offensive language": 0.

76, "neither": 0.

16}In this example, the container runs locally.

Of course the actual purpose is to keep it running at a permanent location, and possibly scale the service by starting multiple containers in an enterprise cluster.


Deploy as an Heroku appA way to make the app publicly available to others is using a platform as a service such as Heroku, which supports Docker and offers a free basic membership.

To use it, we have to register an account and install the Heroku CLI.

Heroku’s application containers expose a dynamic port, which requires an edit in our Dockerfile: We have to change port 5000 to the environment variable PORT:CMD firefly app.

predict –bind 0.



0:$PORTAfter this change, we are ready for deployment.

On the command line, we log in to heroku (which will prompt us for credentials in the browser) and create an app named hate-speech-detector:$ heroku login$ heroku create hate-speech-detectorThen we log in to the container registry.

heroku container:push will build an image based on the Dockerfile in the current directory and send it to the Heroku Container registry.

After that, we can release the image to the app:$ heroku container:login$ heroku container:push web –app hate-speech-detector$ heroku container:release web –app hate-speech-detectorAs before, the API can be addressed via curl.

However, this time, the service is not running locally, but is available to the world!$ curl -d ‘{“text”: “You dumb idiot!”}’ https://hate-speech-detector.


com/predict# Output:{"hate speech": 0.

26, "offensive language": 0.

68, "neither": 0.

06}Now, scaling the app would be just a few clicks or commands away.

Also, the service needs to be connected to the message board, the trigger threshold needs to be set and an alerting implemented.

Hopefully this tutorial will help you deploy your own machine learning models and apps.

Got any additional ideas?.Share your opinion in the comments!.

. More details

Leave a Reply