Innovations in Big Data: All Signs Point to Convergence

Contributed by

5 min read

What’s clear to me is that we are in the midst of the biggest change in enterprise computing in decades: a shift in how data is stored, analyzed and processed is changing the way businesses operate and compete in the marketplace. We recently held our annual Customer Advisory Board meeting, and I had the pleasure of spending time with many companies who've partnered with MapR on their big data journeys. What these big data pioneers have accomplished is amazing and where they want to take their big data use cases is aggressive. By putting data services (enterprise storage, database, and event streaming) and processing tools (Hadoop, Spark, Drill, NoSQL, and others) on one data platform, we’re enabling organizations to gain immediate access to data across operational analytical workloads. This new type of “converged” platform not only supports the broad range of open source projects that provide a rich diversity of processing options, but also integrates more generally across enterprise apps that require file, table, and/or stream access. That means that insurance carriers can base premiums on actual behavior and specific risk profiles, healthcare providers can improve personalized treatment, and retailers can dynamically adjust their offers.

Representing the bellwether for the market, companies are using MapR to incorporate analytics at scale and make real-time adjustments to improve business performance. These companies are doing more than deploying data hubs and performing queries; they are integrating analytics directly into their business operations. This aligns with the concept of HTAP (Hybrid Transaction/Analytical Processing), coined in early 2014 by Gartner, which describes a new generation of data platforms that can perform both online transaction processing (OLTP) and online analytical processing (OLAP) without requiring data duplication.

A common theme that emerges from discussions with customers is that by unifying production workloads with analytics, companies are able to quickly adjust to changing customer preferences, competitive pressures, and business conditions. This means that they’re really able to speed up the “data-to- action” cycle, since they don’t have to deal with the time lag between analytics and changes to operations; by using a converged data platform, they’re able to tear down those silos.

These big data experts are also eager to talk about the many open source projects within the Hadoop and Spark ecosystem that came into prominence last year. We have seen the rapid evolution from MapReduce to YARN to Spark for processing big data. A converged platform actually accelerates value from these kinds of emerging technologies, since it provides a single point for governance, security, and data quality as well as for enterprise readiness. A platform that enables a single investment to support workloads across all of these can also future-proof organizations, since they will be able to easily exploit future innovations. What’s exciting is that companies are running applications that leverage many open source projects with converged data across a wide variety of industries and business functions, including retailers (customized ads and dynamic pricing), chip manufacturers (product issue identification), telcos (enhanced mobile services), and ad tech companies (real-time bidding).

Many industries use real-time data in several areas of their organizations. They are well aware that if the application that collects data on their customers’ purchases and experiences takes longer than a web page to load, it will not be much help in customizing the online experience. These types of real-time applications also require mission-critical capabilities. Data needs to be available, reliable and protected – globally and across cloud and on-premise; anything less and you run the risk of compromising the business. Security is also important, as organizations need data access control, user authentication, and audit capabilities in order to inspect user access and actions. This is where convergence of big data technologies comes into play.

The deployments and expansion plans for big data use cases within organizations have exceeded my expectations. Leveraging big data for operational benefits has proven to be a viable strategy, and as a result, we’re in the midst of the biggest change in enterprise computing in decades. Organizations that can take advantage of data services and processing tools on a single data platform will be able to harness real-time insight from their streaming data, and this translates into real-time views into their customers, products, and operations. The convergence of big data presents a real-time capability that can be the key to a long-term advantage–or, if it is delayed too long, it could be the topic of future turnaround plan.

This blog post was published March 29, 2016.

50,000+ of the smartest have already joined!

Stay ahead of the bleeding edge...get the best of Big Data in your inbox.

Get our latest posts in your inbox

Subscribe Now