In this post, I’d like to show how to quickly build a Tableau-like BI application on top of your data in a Jupyter Notebook using the Atoti python module. It can give your users an easy way to:
- explore the data produced by your model with an interactive UI,
- play around and see it in various projections without the need to write any code or depending on someone to ingest it into a standalone BI solution.
In the study “2020 State of Data Science – moving from hype towards maturity”, Anaconda discovered that “Fewer than half (48%) of respondents feel they can demonstrate the impact of data science on business outcomes”. As a remedy the Anaconda team has suggested to focus on removing “the barriers to deployment” and for data scientists on learning “to communicate the value of their work”.
My own small research is showing that pro data scientists tend to present the results of the work in a form of a Jupyter Notebook or a pdf file:
Jupyter notebooks are great for working on models, but they are not so great in their raw form for presenting results to users. By building a BI app on top of the collected and enriched data sets and letting business users “experience” the data, slice and dice it, we encourage early feedback, help unlocking imagination and discovering the value of the ML algorithm at earlier stages of development.
In a BI app business users can explore the data – filter, drill-in and even run what-if analysis without the need to write any code – like in excel but from a web gui. Charts, pivot tables, formatting, conditional formatting and interactive recalculations, and scenarios will be available to them.
Python implementation
Now I’ll show you how to quickly build a BI app locally or on a server from a jupyter notebook using Atoti python module and give the end user a link to slice and dice the data set.
These are my 10 lines code:
import Atoti
session = Atoti.create_session()
events_ds = session.read_pandas(
events,
store_name="Events",
keys=["externalsessionid", "eventtype"],
)
cube = session.create_cube(events_ds, "Sales Analytics")
They perform these 4 steps:
- import Atoti. You can find the installation steps for conda and pip in the doc.
- create a session – at this step an in-memory data cube is span on your machine or on a server and the UI is launched. You can open the URL of the app by running
session.url
- Next step is to load the data from a pandas dataframe into the app engine (creating an in-memory datastore). You could source data from other sources, for instance, csv files, parquet, kafka feed and others.
- Finally, the create_cube command is populating measures and dimensions.
Later if you want, you can load more data sources and link them together in a snowflake-type of data schema. Or you can define simple or sophisticated calculations for your end users to look at. You can find some reference examples in the Atoti gallery.
Under the hood
Starting using Atoti python module is not hard – just a few lines of code and one module to install, but do not let this easiness to confuse you – under the hood the bi app you are launching is powered by a fairly sophisticated technology which utilizes an in-memory database engine, datastores capable of handling real-time data updates and reactive UI.
Conclusion
Productionalizing analytical models and helping users adopt them is not easy. I hope this example can help you engage business users as early as possible.
As a power user of Atoti, I build models for risk analytics in financial institutions, please check my other posts for more sophisticated examples of non-linear aggregations and complex calculations.
