Going beyond multidimensional analysis with Python
While many data scientists or analysts are excited about doing their analysis, the analysis itself constitutes only a portion of a full solution. Securing the system, for instance, is critical for most, if not all projects, especially those that involve sensitive data.
Atoti – Creating BI application with Python
Atoti is a free Python library that allows users to create their own BI analytics platform. How so?
With a few lines of code, Atoti loads the data into a multidimensional data cube and spins up a web application:
There’s more to Atoti than just that. Check their features map for a quick overview of what the library can do.
Upsizing Atoti with a plus
As a single library that allows us to perform aggregations, visualizations and dashboarding, that’s a great deal Atoti is offering. Going beyond the analytics needs, there are “plus” features that can be added with Atoti-plus plugin, e.g.:
- Implementing security with granular access control using multiple-level role management
- Replacing Atoti branding with your company’s own branding
- Extension to the web application
- Java API to describe more advanced data models
The Atoti-plus plugin allows us to access features of the Atoti+, the enterprise edition of Atoti. Reach out to support@atoti.io to find out more about the Atoti+ license.
While there are multiple interesting features to zoom into, we are going to focus on the security implementation in this article.
Security implementation for BI web applications
Let’s dive a little deeper into the type of security we can have for an application. Below are some possible setups that a company may have:
- A global user access control policy or each application has its own policy.
- Different network zones restrict access differently via the firewall configuration.
- Different applications are hosted in different zones depending on their security clearance.
Integrating Atoti into the existing infrastructure
We can run Atoti as a Python script on a server within any network zone, depending on the data sensitivity. Atoti can also be hosted on the cloud.
We only require the internet connection to install Atoti and its dependencies from the public PyPI repository or conda-forge. Once installed, Atoti does not require the internet to run.
Access to Atoti web application
The key thing to note is that we create the Atoti web application when we create the Atoti session. This means that it is hosted on the server that the Python script is running on.
Being a web application, its access is restricted by the firewall rules on the network zone. To facilitate the setting up of firewall rules, we can always designate a fixed port for the web application to run on:
session = tt.create_session(config={"port": 9090})
Data connectors
In terms of the data sources connected to Atoti, it depends on the mode of communication accepted by the company across the different network zones.
Atoti is able to consume a Kafka topic and stream its record. Similarly, Atoti supports JDBC connection. Otherwise, Atoti can read the below data type:
Since it’s a Python library, we can always have custom implementations to transform other data sources into one of the accepted data structures.
User authentication and roles management
Up to the previous section, we haven’t utilized any of the “plus” features. However, we require Atoti+ to implement user authentication and access controls.
Most companies have their own systems to manage their users and roles. We can configure authentication for Atoti+ to integrate with any of the below:
What happens if the company doesn’t have any existing systems to manage the users? It’s fine because Atoti supports basic authentication – we can define the users, their password and roles with the BasicSecurity feature of Atoti+.
Below, we have defined an admin user with the native roles – ROLE_ADMIN and ROLE_SHARE:
import Atoti as tt
session = tt.create_session(
config={
"authentication": {"basic": {"realm": "Test security"}},
},
)
session.security.basic.create_user(
"admin_user", password="admin_password", roles=["ROLE_ADMIN", "ROLE_SHARE"]
)
Securing Atoti+ web applications with multiple-level access controls
We have seen earlier that the application can be placed in a designated network zone and is protected behind the firewall. We can also enable Secure Sockets Layer (SSL) for the Atoti web application to strengthen security.
Once we have secured the application from the infrastructure side, let’s look into the application itself.
Atoti+ provides 3 levels of security:
- Application access control
- Folders and dashboards level access control
- Data level control
Access control at application level
Earlier, we mentioned that we can integrate with existing authentication modules such as LDAP server or OpenID Connect. These systems maintain the users and their roles.
We cannot redefine the following predefined roles in Atoti+:
- ROLE_USER – required for application access
- ROLE_ADMIN – gives full access (read, write, delete etc) to the application
- ROLE_SHARE – required for setting access rights on dashboards and folders
Only users with valid roles are able to log in to the application.
Access control at folders and dashboards level using roles
Atoti and Atoti+ are meant to support collaborations company-wide. Therefore, users from different departments and teams will be able to access the system. However, there may be dashboards that are team specific and should not be accessible by others.
In this case, we can have department and team-specific roles. Thereafter, business analysts or key users with the “ROLE_SHARE” access can now design dashboards and share them with their respective teams and departments.
ROLE_SHARE can grant folder and dashboard access to others.
Let’s create some users with restricted role access:
session.security.basic.create_user(
"Market_manager_eu", password="eu_password", roles=["ROLE_MARKET_EU", "ROLE_SHARE"],
)
session.security.basic.create_user(
"market_manager_us", password="us_password", roles=["ROLE_MARKET_US", "ROLE_SHARE"]
)
Now, users with the above roles can only see the folders and dashboards that were shared with them:
-
market_manager_eu can only see folders and dashboards from Europe -
market_manager_us can only see folders and dashboards from USA -
admin_user is able to see all the folders and dashboards
It is interesting to note that we can share dashboards and folders with both users and roles. Also, users can interact with the dashboard such as editing the filters or adding new widgets whether they have “Reader” or “Editor” access. However, they will not be able to overwrite the existing dashboard. Instead, they will have to save the dashboard as a new dashboard that is only available to themselves.
This also means that users on Atoti+ are able to have a personal work folder that is only accessible by themselves and the administrator. This capability empowers them to create their own dashboards to suit their needs.
Access control at data level
A dashboard can sometimes be used across teams with a change of filters. For instance, for a retail company that has business in the USA and Europe, we can have a dashboard on its sales for both its US and European team.
In the event that we do not want the teams to see the sales of other countries, we can define roles with stricter permissions instead of duplicating dashboards with different filters:
session.security.create_role(
"ROLE_MARKET_EU", restrictions={"COUNTRY": ["FRANCE", "GERMANY"]}
)
session.security.create_role("ROLE_MARKET_US", restrictions={"COUNTRY": ["USA"]})
So when both the US and European teams access the same dashboard, they will be looking at their own data only.
-
market_manager_eu is able to see data from France and Germany -
market_manager_us is only able to see data from USA -
admin_user is able to see all data
Seeing data restriction in action, we need to understand the difference between applying a filter and having stricter permissions at the data level. Below explains when to use filter and when to apply data restrictions on roles:
- Apply filters when the users have the access rights to the data but do not require it for the specific insight. After all, users have the ability to update or remove the filters.
- Stricter permissions at the data level ensure that users have no access to the underlying data, even if they change the filter from the web application.
Transitional data in-memory
It is important to understand that Atoti only reads the data from the source and there is no feedback or writes to the source. Hence, any manipulation of data on the Atoti side has no impact on the source systems.
In addition, data is held in memory for the Atoti cube within the server where it is hosted. Therefore, we do not have to worry about data being stored outside of the server.
In conclusion
Below is a high-level summary of how security is achieved in Atoti+:
The examples for Basic Authentication and LDAP Authentication mechanisms are available on GitHub.
Atoti is a powerful business intelligence tool. Its “plus” version – Atoti+ – provides a holistic enterprise solution needed for productionization. Reach out to the community at https://gitter.im/atoti/atoti# for more information.
