Why Atoti is a good fit for their needs

I came across the presentation from Meghana Kshirsagar and Gauri Vaidya on “Business Intelligence with Online Analytical Processing with Atoti”.

I had a quick catch up with Gauri and got to know that she has been sharing Atoti with students along with her mentor, Meghana. Meghana is a postdoctoral researcher at the University of Limerick. I got curious about what these researchers do and how they got to know about Atoti. But more importantly, why are they sharing it with the community?

Join me in my conversation with the mentor and mentee to find out more.

Huifang: Could you introduce yourselves to our community?

Meghana: I’m Meghana Kshirsagar affiliated as a post-doctoral researcher with Lero, The SFI Research Centre for Software. I’m working with Biocomputing and Developmental Systems (BDS) Research Group, attached to the Department of Computer Science and Information Systems (CSIS) at University of Limerick (UL), in Ireland.

I used to work as an Associate Professor with the computer science department back in India at Government College of Engineering, Aurangabad and Gauri was pursuing her Bachelor’s degree in the same department. After I moved to Ireland in 2019, she joined as a remote intern with the BDS group. As her expertise includes machine learning and data science, all the core programming languages like C, Python along with a host of visualization tools, she was a perfect fit for us.

Huifang: (To Meghana) Why did you choose to join the University of Limerick which is so far away from India?

Meghana: I have always wanted to work with a research organization that partners with lots of industries supporting multiple nationalities. So, I applied and got an invite from Professor Conor Ryan, Director, BDS Research Group, UL to work as a Postdoctoral Researcher on the SFI funded project. Here, the task was to design a machine learning tool, which was my area of expertise.

The deliverable of the five-year project was to automate the generation of digital circuits. It has funding of around €1, 822, 053.00. It involves incorporating evolutionary computation, more specifically, Grammatical Evolution, which was a completely new domain for me and hence sounded exciting, especially, with Intel as the main industrial partner!

Huifang: (To Meghana) Tell us about your journey as a researcher?

Meghana: Since September 2019, I have been working primarily on our Automatic Design of Digital Circuit (ADDC) project. I’m also leading teams in various independent projects such as developing an integrated smart transportation system and a smart healthcare system.

I have presented papers and chaired sessions at international conferences, delivered talks and workshop sessions in Science Foundation Week and events such as hackathons, organised workshops, events, challenges. I’m currently co-supervising a postgrad on the ADDC project.

Recently I presented my paper “Hierarchical Clustering Driven Test Case Selection in Digital Circuits” and chaired a session at the “16th International Conference on Software Technologies, ICSOFT 21”, 6–8 July.

Huifang: Who are the key users whom researchers are answerable? Where do the ideas for research topics come from?

Meghana: We report to the Line Manager, Professor Conor Ryan, the principal investigator of the project, and have regular interaction with the industry personnel from Intel. Our research group is affiliated with Lero and we work according to the guidelines prescribed by Lero for its researchers.

Together with three principal investigators, our team of four post-doctoral researchers are also involved in managing and supervising the eight postgrads between us.

The research topics evolve from the interactions with the Irish industries, universities and councils to collaboratively come up with innovative ideas and approaches driving digital transformation, and to provide services aligned with the UN’s sustainable goals. This impacts Ireland’s regional and economic growth leading to competitive domestic markets and opportunities across all sectors.

Huifang: What happens at the end of a research project?

Meghana: The impact of any research project is evaluated in terms of Key Performance Indicators (KPIs), target deliverables and potential for spin-offs. KPIs can be disseminating research in the form of publications in leading international conferences and journals, exploring collaborations among Irish and EU Universities, and industries. Through the research project, we promote science, technology, engineering and maths (STEM) awareness and engage with the Irish public across all levels of the educational system.

Potential funding opportunities and collaborations are explored to develop and start new research projects.

Huifang: Was it difficult to coordinate between Gauri and yourself, having one in Ireland and the other in India?

Gauri: Initially when Meghana went to Ireland, it was challenging to adjust to the difference in time zones.

Over a period of time, overcoming the challenges of virtual meetings at different zones, we have developed an easy-going and friendly bond. We collaborate and work together on multiple projects through virtual meetings. Such collaborations have resulted in many productive outcomes.

Meghana: In-person, Gauri and I could have easily walked through our understanding of the data and concepts. These kinds of things are very challenging through virtual meetings. It was also very difficult for Gauri to communicate with me the way she was interpreting the data, the way she was building the visualizations and the kinds of patterns she wanted to reveal. Later on, we started using whiteboards in our virtual meetings to discuss our ideas.

Huifang: I came across the YouTube presentation by you and Gauri on “Business Intelligence with Online Analytical Processing and Atoti”. Was this also part of your research project?

Meghana: No, this was not a research project, but, started as a series of talks at international venues to familiarize users with online analytical processing (OLAP) operations.

Many of the BI tools that we were associated with in the past were not open source and each of them had features spread across tools.

I was looking at a tool where I can blend the concept into the tool and Atoti was a perfect fit for this purpose.

Huifang: How did you come across Atoti?

Meghana: We were invited for a talk in May, 2021, at Symbiosis Institute of Technology, Pune, India that caters to Information Technology (IT) undergrads. The objective of the talk was to get users acquainted with BI tools for a hands-on demonstration of OLAP operations.

While I was exploring our weekend article on Medium — Insights on Modern Computation with Gauri, I stumbled upon Atoti.

I came across Atoti on LinkedIn when there was some excitement and buzz around its new release and how it helps in developing business use cases. Back then, it came bundled with the sales use case.

Since the case of a retail sale would make sense for our talk, I decided to go along with Atoti. Using Atoti, we made the business intelligence concept come across by building on top of whatever is existing in the template sales example for our talk.

In this way, our Atoti journey commenced.

Huifang: Did you face any difficulties in picking up Atoti and how long was your learning curve for the tool?

Gauri: I would say that the standard tutorial for Atoti is perfect. Each concept, along with the images, was so well explained while also illustrating a similar real-world example. There was no difficulty for us in learning Atoti.

We were planning this with Atoti for a week before our talk and it was sufficient to learn the concepts. We stumbled a lot upon what should be the representation of data. But in terms of the tutorial, it’s perfect to start with the basics.

Huifang: What made you decide on using Atoti for the talk to explain the concept of OLAP?

Meghana: OLAP was done in the past with tools as simple as Microsoft Excel. People are still using Excel for business analytics. They are still doing the drill-down, slicing and dicing and a lot of them would still be comfortable with Excel.

Students are well-versed in Excel. However, in terms of the way data is being organized and relating the concept of a multidimensional cube with keywords, it makes lots of sense to support it directly with the practical using Atoti.

We were targeting a generation of young upcoming researchers like Gauri. Hence, the ease of operations is also one of our main concerns. So we needed tools with a short learning curve . As Gauri mentioned, the learning goal for her with Atoti was hardly four or five days.

Huifang: What are the tools that you have been using previously before you got to know Atoti?

Meghana: We were using only the licensed SPSS software for most of our analytics. Later on, we started writing complex queries in the backend. It was tricky and difficult to write the queries to perform the various kinds of joins. It was also a bit tricky for us to integrate those with the other visualization tools.

With Atoti, users are done away with writing complex queries and don’t have to bother about database management stuff. It also comes with a perfect set of visualization tools, all packed in one. The drag and drop operations with the flexibility of customising the attributes help in simulating and visualizing different what-if scenarios easily with live dashboards.

Huifang: Are there any problems you faced with Atoti? What missing feature would you like to have with the library?

Gauri: I would say the latest release of Atoti has addressed the two concerns that we had with the previous 0.5.5 version.

The first thing we had observed was that the Atoti is unable to tell date columns in different formats. But the date column was one of the significant features of our main table. Now with the latest release, I saw that date is now considered as an object which is great! We can now pass it to the object without any manipulation.

The second thing we faced while delivering the talk was that we say we have a “dimension table” but the Atoti keyword is “store”. So we had to first explain what a store is. Then we relate the store to the table in the real world. But again, with the latest release, the keyword is replaced by table. So now it would be easier to explain to people who are new to the terms.

Huifang: This is the first time we wrap our API in Python and have a free community edition. In your viewpoint, how important is free software and open source to research?

Meghana: Free software and open source, I would say, are the most important things for researchers. As you mentioned, you are not targeting just the enterprises with Atoti but also a whole group of other sectors like education. Education is a key sector here, comprising all these research bodies.

As a researcher, I’m tackling different use cases on a daily basis. Having these tools facilitates rapid prototyping and testing.

Huifang: Lastly, what is your favourite research topic so far?

Meghana: Right now, Gauri and I are working with some clients in the Intel Cryptography division to design a random number generator that is cryptographically secured.

It is a very challenging domain to build such tools. We have to take into consideration key features like speed, unpredictability while being secured from attacks. That was the most interesting work that we are currently associated with.

Many thanks to Meghana and Gauri for letting us take a peek into the busy life of researchers and for sharing Atoti with the community. Follow them on their Medium to gather insights on modern computation.