Full Stack Visualizations For Complex Solutions — For Data Scientists

What if the boss asks “Can you put together something so I can play around with it later?” or “Can you make a demo that I use in some meeting that you’re not attending?”.

Basically, they’re asking for a URL to click on and see the magic, plus that ensures it works each time as expected without someone accidentally mess up your code, etc.

Developing something like this demo from scratch and without any tutorial can easily take up to a week for someone who has little knowledge of HTML.

However, I hope after going through the steps below, you can put together something like this in matter of a few hours.

As you may have gathered, such an interface requires two sides.

The back-end, where you run your python magic and the client-side, where you render the results into some fascinating, jaw-dropping graphs to get the message across while giving users some degree of freedom for exploring your solution further.

I usually host my back-end on Google App Engine that can host your python code and run it whenever you trigger it on your front-end but you could very well use AWS or other services as well.

No particular preference here.

Front-end is where we write in HTML, CSS & JavaScript(JS).

Back-end is you can possibly store data, take advantage of cloud computing, or even run code without using your user’s (client) resources (Imagine their phone, PC).

The Back-endFor the purpose of this example, we need to setup our back-end such that it would respond to some URL requests.

In other words, each function would run when you enter a URL in your browser or make that request via some JS code such as fetch(), XMLHttpRequest(), etc.

(you’ll see an example later).

Since we are writing our back-end to execute some python code (so we can enjoy doing some data science cool stuff later — hey, no bias here :p) we take advantage of Flask — one of the simple web development frameworks for python.

Flask is a microframework for Python based on Werkzeug, Jinja 2 and good intentions.

And before you ask: It’s BSD licensed!You need to write your python script and host it somewhere on a server such that each time a URL is requested one or more of your python functions are triggered.

You can pass some data to the function & finally get some data back which in this demo will be the optimized routes & vehicle’s information.

A side note, this simple demo is taking advantage of Google OR Tools for solving some textbook travel salesman problem with constraints being defined here as time and truck load.

If you haven’t used app engine before, the simplest way to get started is to follow the tutorial in the App Engine after signing up.

Select Python as your language and follow to the end of the Hello World tutorial.

Then come back here.

After going through the hello world tutorial on App Engine you should be able to create an app and deploy it to Google App Engine using either your command line or the embedded shell & editor in the browser (I like the latter personally), all you really need to create is a simple Flask app like below.

Keep in mind to install the packages you will use in your back-end code like Flask, OR-tools, etc first.

That means you need to add them to your requirements.

txt file and install then (see below).

Simple Flask app for the back-end to trigger our functions via some URL requestsNow if we make a request to the “baseURL-of-project/vrp” above, we are simply triggering the function associated with it.

Simple right?!.This is great, because you can do whatever you want on the back-end, heck, even add complex modeling and use all the machine learning libraries you need, etc all on your back-end and then pass on the result to your client side to render a pretty graph or chart.

The Front-endHearing “front-end development” might intimidate you but I assure you you don’t need to know everything to get this started.

You could try to do it all via python but I personally like to go all in with HTML, CSS & JS.

Below are a few concepts with a good source for learning each that should be all you need to begin with;Basic JavascriptPromisesFetch() or XMLHttpRequest() for making our URL request on the client-side (front-end)I’m not going to tell you what’s best to use as there are bazillions of front-end libraries, packages & frameworks.

Whatever works for you should be good.

If you have no clue, then I’d recommend starting with going through ( PWA Starter Kit) and you should be able to start writing pretty awesome cutting edge web apps shortly.

We need a simple HTML page to include our demo UI elements such as buttons, input fields, and perhaps a google map since it’s darn simple.

Keep in mind, for using google’s services like map, you would need to create a free API key on Google’s related web page.

For rendering a map you can simply start here and here.

Once you have an action button setup & you want to connect it to the back end, you simply need to make the request to our triggering URL upon some user interaction, e.

g, when user clicks on “Optimize Route” in our case.

We capture that via the click listener (More on this below) we have attached to the button and run the function below.

JavaScript function where we make a POST request to our back-end URLAnd just so you know what I meant by a click listener, below is an example with a functional demo here.

<!DOCTYPE html><html></html><h2>JavaScript addEventListener()</h2><p>This example uses the addEventListener() method to attach a click event to a button.

</p><button id="myBtn">Try it</button><p id="demo"></p><script>document.

getElementById("myBtn").

addEventListener("click", displayDate);function displayDate() { document.

getElementById("demo").

innerHTML = Date();}</script></body></html>Notice how I am passing my data as a string in body of my request inside _optimize() method.

I am also taking advantage of fetch function which returns a promise which allows me to retrieve the result when it comes back.

Done.

Once you have the result you can simply print it to the console via console.

log(response.

json()), head to the developer tools in Google Chrome and confirm you are receiving the results.

Also, you need to replace ****** with your project name in App Engine.

Beyond that it is all JS processing and rendering your data the way you like it.

There are so many JS graphing libraries.

D3.

js is one of my favorites.

Hosting Your Front-endSo, you have the project written.

Now you need to access it via a URL that you can share with the lazy guys (lol).

Well, all you need is a hosting service.

There are many hosting services around.

I have been using the relatively new Google’s Firebase services because of it’s very short learning curve.

You can choose any service you like.

To simply host your project on Firebase you need to create a project via Firebase console (takes a couple minutes).

Then head to your local project folder in your machine and initialize Firebase and connect to your newly created Firebase project.

Don’t worry all these are outlines step-by-step here.

Then enter the command below from your local project directory to deploy your project.

firebase deployThe URL is then shown in Firebase console, under Hosting.

There you have it.

ConclusionIf you start a basic UI, you are easily able to wrap your data science projects into a more advance interactive shell that would give anyone the freedom to play around and who doesn’t like to play!!.I have personally seen the advantages of being able to show case your solution in such a way.

While it is true that you can create some interactive menus via Jupyter notebooks or other tools for simple projects, building more complex dashboards is often left as a task for front-end developers.

I argue that as a data scientist who strive to be better, you don’t have to depend on someone else’s schedule or skills.

So do yourself a favor, spend a weekend on this, master it & enjoy the freedom you will gain as a result.

If you follow the tutorials and the links I provided, you should be able to start building within a day or two.

The back-end should be quite straightforward since it’s mostly python other than the Flask shell.

The front-end might require a bit of more time for someone who has not done any front-end development before.

I am hoping that I have provided a step by step guide to get you going but I might add more details depending on the comment I get.

Please let me know how I can improve this post and I will do that.

ReferencesBuilding Interactive Dashboards with JupyterWelcome to Part II of “Advanced Jupyter Notebook Tricks.

” In Part I, I described magics, and how to calculate notebooks…blog.

dominodatalab.

comA very simple demo of interactive controls on Jupyter notebookNotebooks come alive when interactive widgets are used.

Users can visualize and control changes in the data.

Learning…towardsdatascience.

com.. More details

Leave a Reply