The Evolution of Business Intelligence and Self-Service Analytics – Whiteboard Walkthrough

Contributed by

7 min read

In this week's Whiteboard Walkthrough, Sameer Nori, Business Intelligence Expert at MapR, explains how BI has evolved over the last 3 decades from being IT driven to analyst driven with Self-Service tools. He explains how Self-Service Analytics can be taken to a new level with schema- free Data Exploration and the role Apache Drill plays in this evolution with some horizontal use case examples.

Here's the unedited transcription:

Hi, my name is Sameer Nori and I'd like to take a few minutes to talk about the evolution of business intelligence and analytics and the context of big data and what we see happening with our customers. I thought it would be useful to talk about what we see as the three different phases or three different eras of business intelligence in some regards. In the 80's and 90's, business intelligence was largely IT-driven. What I mean by that is whatever business users or stakeholders needed, whether it was a particular report or particular type of insight, they went to IT.

IT created this for them, which is why we label this as IT-driven: really, it was where IT was creating the necessary reports and spreadsheets if needed. I think this has served the business world and continues to be a mainstay use case for business intelligence. However, what's occurred is the fact that as business users have gotten more access to information, their appetite for more insights has tremendously grown. IT has become encumbered with a little bit of a problem, which is well-known in the business intelligence industry, called the report backlog.

In the 2000's, there were a number of self-service BI tools that came to the market. Tools like QlikView, Tableau, and Spotfire really helped cure these report backlog problems to some degree, because what self-service business intelligence tools did is that they made analysts a lot more self-dependent and self-reliant, with some support from IT for things like ETL. Analyst were now able to gather the data by themselves and actually develop these analytics and insights that were acquired with support from IT for the necessary transformation and the backend processes. This really helped unleash a new wave of productivity within the whole business analyst community.

Fast forward to where we are now; we're in the context of the big data space and big data movement, where there's a lot more variety of data sources coming from a variety of a lot of different data formats. These include data formats like JSON, and a lot more complex, flat schemas and things that are sitting in the SQL systems that are creating a whole new wave of analytics that can be unleashed. We believe that this is the era of schema-free data exploration, where it is completely analyst-driven without needing really any IT support to get going.

What that means is that analysts can get data sources much more quicker by themselves and not have to rely on the traditional process that has been followed before of schema development and developing appropriate structure. This is where we really see Apache Drill as a technology really helping to solve this problem, because of its ability to discover schemas on-the-fly and be able to let analysts explore data almost immediately. I thought it would be useful to talk about this in the context of a few different use cases. Let's talk a little bit about marketing analytics and marketing campaigns, and how those can get optimized with data exploration.

If you think about the modern day marketing world, where whether you're a B2C company or B2B company, you're dealing with a lot more data sources from a variety of different channels, and all of these are in a variety of different formats. The ability to improve things like customer lifetime value, your conversion rates, and so on, are definitely dependent on having immediate access to data and being able to uncover and figure out what things you can improve from a campaign perspective. We see data exploration and the whole notion of schema-free data and analysts being able to drive this and provide the insights to their campaign teams as a huge use case for us.

In the world of customer service, if you are in the customer service function or responsible for customer experience, there's a lot more information that is available that can be harnessed to improve your customer service that you're providing to your end customers. Whether they are calling about a particular product or service that they use and they have issues with that, or whether it's something about warranties or a variety of different other angles to customer service—all of that can benefit from this notion of schema-free data exploration, where analysts can really help optimize the process and what follow up with you for that.

The last thing to discuss is operations and the supply chain. Operations people are tasked with improving operation efficiency across a number of different business processes, and increasingly these are becoming a lot more data-driven. You have the ability to tap into a lot more data than was available before, but more importantly, you have the ability to apply the analytics acquired to optimize various things. Whether that's a use case in preventive maintenance, if you're in the oil and gas space, or whether you're in manufacturing and looking to improve supply chain and logistics, schema-free data exploration really can help you take that to the next level.

For more information, you can download Apache Drill on the Apache Drill website. If you have any other ideas or questions, feel free to leave a comment. Thanks for your time.

This blog post was published November 25, 2015.

50,000+ of the smartest have already joined!

Stay ahead of the bleeding edge...get the best of Big Data in your inbox.

Get our latest posts in your inbox

Subscribe Now