Contributor: Jim Scott

MapR Converge Blog author, Jim Scott

Jim Scott is an experienced leader having worked in financial services, regulatory, digital advertising, IoT, manufacturing, healthcare, chemicals and geographical management systems. He is a cofounder of the Chicago Hadoop Users Group (CHUG) where he helped grow a now flourishing community around next generation technologies. Scott has built systems scaling to 50+ billion transactions per day, and his work with high-throughput computing at Dow Chemical was a precursor to more standardized big data concepts. His passion is in building combined big data and blockchain solutions.

Blog Posts by Jim Scott

April 19, 2018 | By Jim Scott

How Orchestration, Edge Computing, and Serverless Computing Impact Your Cloud Strategy

Editor's Note: This is an excerpt from the book, "A Practical Guide to Microservices and Containers: Mastering the Cloud, Data, and Digital Transformation". It is one thing to make a grand statement like "go run it in the cloud,"...

Read more
April 18, 2018 | By Jim Scott

Cloud vs. On-Premises – What Are the Best Options for Deploying Microservices with Containers?

Editor's Note: This is an excerpt from the book, "A Practical Guide to Microservices and Containers: Mastering the Cloud, Data, and Digital Transformation". Microservices are great. Don't just take my word for it: there are many sources...

Read more
March 20, 2018 | By Jim Scott

Application Agility through Kubernetes (k8s) and Containers

Editor's Note: This is an excerpt from the book, "A Practical Guide to Microservices and Containers: Mastering the Cloud, Data, and Digital Transformation" – you can download the ebook here. Containers offer a front-row seat to application...

Read more
February 06, 2018 | By Jim Scott

In Search of Application Agility: What It Actually Means and How to Attain Application Agility

Editor’s Note: This is an excerpt from the book, “A Practical Guide to Microservices and Containers: Mastering the Cloud, Data, and Digital Transformation” – you can download the ebook here. Containers offer a front-row seat to application development...

Read more
February 05, 2018 | By Jim Scott

In Search of Data Agility: What It Actually Means and How to Attain Data Agility

Editor’s Note: This is an excerpt from the book, “A Practical Guide to Microservices and Containers: Mastering the Cloud, Data, and Digital Transformation” – you can download the ebook here. Agility as it relates to digital transformation is what businesses...

Read more
February 02, 2018 | By Jim Scott

3 Typical Problem Areas Facing Enterprise IT on the Road to Digital Transformation

There are many challenges facing organizations today as they initiate or continue their journey to digital transformation as well as the vital replatforming of the IT infrastructure. Whatever the specific problems that you and your organization may be...

Read more
February 01, 2018 | By Jim Scott

How to Modernize Your Enterprise Data Architecture by Tearing Down Data Silos

When big data became a phrase, a way to describe large volumes of data coupled with workloads that required more than one server, a promise was made: “no more data silos.” This promise was riding on the shoulders of Apache Hadoop. As the years went by...

Read more
January 30, 2018 | By Jim Scott

Quantium’s Story: An Expert’s Perspective on Modernizing Infrastructure with Microservices and Containers

Editor’s Note: This is an excerpt from the book, "A Practical Guide to Microservices and Containers: Mastering the Cloud, Data, and Digital Transformation" – you can download the book for free here. Quantium is an early technology adopter that...

Read more
June 21, 2017 | By Jim Scott

Cloudy with a Chance of On-Prem

Editor’s Note: Jim Scott, MapR Director of Enterprise Strategy and Architecture, gave a presentation entitled “Cloudy with a Chance of On-Prem” at Strata + Hadoop World 2017 in San Jose. You can watch Jim’s presentation here, or read his blog post below...

Read more
March 28, 2017 | By Jim Scott

Using microservices to evolve beyond the data lake

Big data tools and technologies started out by meeting the needs of the analytics community, but they have been evolving ever since. These tools and technologies were born out of the necessity to support large-scale analytics that wouldn’t break the bank...

Read more
March 06, 2017 | By Jim Scott

Handling the Extremes: Scaling and Streaming in Finance

Editor’s Note: At Strata+Hadoop World 2016 in New York, MapR Director of Enterprise Strategy & Architecture Jim Scott gave a presentation on “Handling the Extremes: Scaling and Streaming in Finance.” As Jim explains, agility is king in the world of...

Read more
July 25, 2016 | By Jim Scott

Real-time Message-driven Service Oriented Architecture: Bringing the Boom!

Editor's Note: At Strata+Hadoop World in London, MapR Director of Enterprise Strategy & Architecture Jim Scott talked about Real-time Hadoop: The Ideal Messaging System. You can watch his presentation here or read his blog post below to learn...

Read more
May 31, 2016 | By Jim Scott

Big Data: Facial Recognition and the Biometrics Movement

Just a few years ago, using a fingerprint to sign on to your phone seemed futuristic. Today, it’s everywhere and just the beginning of how biometrics will be woven into our lives. Biometrics is a method of digital identity verification that scans a person...

Read more
May 18, 2016 | By Jim Scott

The Changing Economics of Big Data

Perhaps you’re old enough to remember when the library was the place we went to learn. We foraged through card catalogs, encyclopedias and the Reader's Guide to Periodical Literature in hopes that we’d be able to understand what was going on in other...

Read more
May 09, 2016 | By Jim Scott

Selling Hadoop to the C-Suite: It’s all about business value

With all the talk about Big Data, most organizations are barely out of the starting blocks when it comes to exploiting it for business benefit. Gartner estimates that 85% of Fortune 500 companies are yet unable to exploit Big Data for competitive advantage...

Read more
May 02, 2016 | By Jim Scott

Solving Problems with the Right Technology: Hadoop and RDBMS

In some circles today there is a sort of ‘Hadoop vs. RDBMS’ debate ongoing. Often the discussion casts Hadoop as the obvious heir apparent in the data processing world, with RDBMS cast as your father’s Oldsmobile. This debate is somewhat misdirected and...

Read more
March 08, 2016 | By Jim Scott

Apache Flink GA - Planning for the Future

The distributed computation world has seen a massive shift in the last decade. Apache Hadoop showed up on the scene and brought with it new ways to handle distributed computation at scale. It wasn’t the easiest to work with, and the APIs were far from...

Read more
February 11, 2016 | By Jim Scott

How to Evolve from RDBMS to NoSQL + SQL

For the past 25 years, applications have been built using an RDBMS with a predefined schema that forces data to conform with a schema on-write. Many people still think that they must use an RDBMS for applications, even though records in their datasets...

Read more
February 10, 2016 | By Jim Scott

Stream Processing Everywhere – What to Use?

Processing data from social media streams and sensors devices in real time is becoming increasingly prevalent, and there are plenty of open source solutions to choose from. Here is the presentation that I gave at Strata+Hadoop World, where I compared...

Read more
January 26, 2016 | By Jim Scott

Streaming in the Extreme

Are you ready to start streaming all the events in your business? What happens to your streaming solution when you outgrow your single data center? What happens when you are at a company that is already running multiple data centers and you need to implement...

Read more
January 21, 2016 | By Jim Scott

Big Data on the Road

Getting from point A to point B has been one of humanity’s greatest preoccupations throughout history. While we’ve developed new methods of transportation such as railroads, cars, trucks, and airplanes, they never seem to be fast enough. Big data could...

Read more
January 19, 2016 | By Jim Scott

Big Data: The Key to Unlocking Facial Recognition in Business

Companies everywhere are looking for ways to improve customer service. For example, companies with call-in support centers might track how long agents take to answer calls, or how long customers stay on hold. While many companies are looking at every...

Read more
December 30, 2015 | By Jim Scott

The Big Data Guide to Star Wars Vehicles

One of the many high points in Disney’s Star Wars Episode VII: The Force Awakens movie was the return of several classic ships and other vehicles from the original trilogy, as well as the introduction of innovative, new types of vehicles. With all the...

Read more
December 15, 2015 | By Jim Scott

MapR Streams vs. Apache Kafka – Whiteboard Walkthrough

In this week's Whiteboard Walkthrough, Jim Scott, Director of Enterprise Strategy and Architecture at MapR, discusses a business use case that leverages the power of MapR Streams. Here's the unedited transcription: Hi – welcome to this Whiteboard...

Read more
December 07, 2015 | By Jim Scott

Apache Spark vs. Apache Drill

Editor’s note: If you would like to learn more about using Spark there is a free book titled Getting Started with Apache Spark: From Inception to Production. If you’re thinking about working with big data, you might be wondering which tools you should...

Read more
December 04, 2015 | By Jim Scott

Running at Google Scale With the Zeta Architecture

Google has set the standard for most of the world when it comes to running systems at scale. It has created a number of different technologies to benefit its business. It built those technologies in a way that makes sense for its business, but it has...

Read more
November 30, 2015 | By Jim Scott

Turning Data Into Value with Hadoop and Spark – Infographic

The faster questions can be asked the faster you can get answers. Waiting for data to be shipped off of servers to a central processing platform can take time and most businesses these days want to get as close to real time as possible. Technologies like...

Read more
November 24, 2015 | By Jim Scott

Apache Spark in a Hadoop-based Big Data Architecture – Infographic

Editor's Note: If you're interested in learning more about Apache Spark, download this free interactive ebook – Getting Started with Apache Spark: From Inception to Production. Streaming data enables businesses to respond to customers as close...

Read more
November 23, 2015 | By Jim Scott

Using Python with Apache Spark

Apache Spark is awesome. Python is awesome. This post will show you how to use your favorite programming language to process large datasets quickly. Why Python? Python has become one of the major programming languages, joining the pantheon of essential...

Read more
November 18, 2015 | By Jim Scott

The Importance of Apache Drill to the Big Data Ecosystem

There are many lessons that our high school teachers tried to teach us. Some stuck and others went in one ear and out the other. The one that really stuck for me is that “history repeats itself.” The lesson here wasn’t meant to be strictly literal, but...

Read more
November 17, 2015 | By Jim Scott

Zeta Architecture: Hexagon is the new circle

Data processing in the enterprise goes very swiftly from “good enough” to “we need to be faster!” as expectations grow. The Zeta Architecture is an enterprise architecture that enables simplified business processes and defines a scalable way for increasing...

Read more
November 12, 2015 | By Jim Scott

Mesos and YARN: A tale of two clusters

This is a tale of two siloed clusters. The first cluster is an Apache Hadoop cluster. This is an island whose resources are completely isolated to Hadoop and its processes. The second cluster is the description I give to all resources that are not a part...

Read more
November 06, 2015 | By Jim Scott

Why Data Agility is a Key Driver of Big Data Technology Development

As technology advances at breakneck speed, our lives are becoming increasingly digitized. From Twitter feeds to sensor data to medical devices, companies are drowning in big data yet starving for actionable information. Most likely, you've heard a...

Read more
October 08, 2015 | By Jim Scott

Why Apache Spark Is Like a Fighter Jet

At Strata London in 2015, someone said to me, “Spark is like a fighter jet that you have to build yourself. Once you have it built, though, you have a fighter jet. Pretty awesome. Now you have to learn to fly it.” Let’s break down this quote to see the...

Read more
October 06, 2015 | By Jim Scott

A Quick Guide to Spark Streaming

Stream processing is a power that has been added alongside Spark Core and its original design goal of rapid in-memory data processing. (Learn more about Spark’s purposes and uses in the ebook Getting Started with Apache Spark: From Inception to Production...

Read more
September 30, 2015 | By Jim Scott

Using Apache Spark for the First Time

If you read our recent blog post, “Spark 101: What Is It, What It Does, and Why It Matters,” you know that Spark is a general-purpose data processing engine that can be used for stream processing, machine learning, data integration, and interacting with...

Read more
September 28, 2015 | By Jim Scott

Thinking of Apache Spark in Terms of Legos

Apache Spark is a top-level project of the Apache Software Foundation, designed to be used with a range of programming languages on a variety of architectures. Spark’s speed, simplicity, and broad support for existing development environments and storage...

Read more
September 23, 2015 | By Jim Scott

Apache Spark: A Great Companion to a Modern Hadoop Cluster Deployment

Editor's note: If you're interested in learning more about Apache Spark, download the free ebook: Getting Started with Apache Spark: From Inception to Production Apache Spark is a general-purpose data processing engine, suitable for use in a wide...

Read more
September 21, 2015 | By Jim Scott

Spark 101: What Is It, What It Does, and Why It Matters

Recently, a new name has entered many of the conversations about big data. Some people see the popular newcomer Apache Spark™ as a more accessible and more powerful replacement for Hadoop, the original technology of choice for big data. Others recognize...

Read more
August 21, 2015 | By Jim Scott

Real Time Analysis with Apache Hadoop

Apache Hadoop is revolutionizing big data in more than one way. While the Hadoop platform introduced reliable distributed storage and processing, various packages such as Spark on top of Hadoop make it possible to build applications and analyze data much...

Read more
August 14, 2015 | By Jim Scott

Top 3 Ways Ant-Man and MapR are Similar

As you probably know (unless you’ve been living under an ant hill), Ant-Man is a fictional superhero who first appeared in Marvel comic books, and he’s also a proud founding member of The Avengers. He made his debut on the big screen recently with the...

Read more
July 06, 2015 | By Jim Scott

The Evolution of Database Schemas using SQL + NoSQL

pre, code { background-color: #f7f7f7; border: 1px solid #ddd; font-size: 13px; line-height: 19px; overflow-x: auto; overflow-y: auto; padding: 2px 4px; border-radius: 3px; word-wrap: normal; white-space: pre !important;" font-family: Consolas, "Liberation...

Read more
July 02, 2015 | By Jim Scott

Drilling into Healthy Choices

pre, code { background-color: #f7f7f7; border: 1px solid #ddd; font-size: 13px; line-height: 19px; overflow-x: auto; overflow-y: auto; padding: 2px 4px; border-radius: 3px; word-wrap: normal; white-space: pre !important;" font-family: Consolas, "Liberation...

Read more
July 01, 2015 | By Jim Scott

Introduction to the Zeta Architecture – Whiteboard Walkthrough

Editor's Note: In this week's Whiteboard Walkthrough, Jim Scott, Director of Enterprise Strategy and Architecture at MapR, gives you an introduction to the Zeta Architecture, a high-level enterprise architectural construct which enables simplified...

Read more
June 16, 2015 | By Jim Scott

Zeta Architecture – What’s in the Name?

A software design pattern is a description or template which defines how to solve a problem which can then be used in many different situations. The important part of a software design pattern is that it has a common name and way of being described that...

Read more
April 10, 2015 | By Jim Scott

How I Chose My Cloud Storage

Not too long ago it was commonplace for people to have their data sitting on multiple different computers in their home and keeping backups of data between the machines in case one failed. As anyone who lived through that can recall, it was a pain to...

Read more
March 05, 2015 | By Jim Scott

Impacting Business As-It-Happens

To get value out of today’s big and fast data, organizations must evolve beyond traditional analytic cycles that are heavy with data transformation and schema management. The Hadoop revolution is about merging business analytics and production operations...

Read more
February 11, 2015 | By Jim Scott

Apache Mesos vs. Hadoop YARN – Whiteboard Walkthrough

Editor's Note: In this week's Whiteboard Walkthrough, Jim Scott, Director of Enterprise Strategy and Architecture at MapR, explains the differences between Apache Mesos and YARN, and why one may or may not be better in global resource management...

Read more
February 04, 2015 | By Jim Scott

Append-only File System vs. Read-Write File System #WhiteboardWalkthrough

Editor's Note: In this week's Whiteboard Walkthrough, Jim Scott, Director of Enterprise Strategy and Architecture at MapR, talks about the implications of append-only file systems and the impact they have on downstream projects in the Hadoop...

Read more
January 21, 2015 | By Jim Scott

HBase Key Design with OpenTSDB

Editor's Note: In this week's Whiteboard Walkthrough, Jim Scott, Director of Enterprise Strategy and Architecture at MapR, walks you through HBase key design with OpenTSDB. Here is the transcript: Hi. I'm Jim Scott, Director of Enterprise...

Read more
November 07, 2014 | By Jim Scott

Real Experiences from a Hadoop Veteran

Some people say I am biased toward certain technologies. That is a completely true statement! Granted, it does depend on the specific technology. But just because I may be biased with certain technologies doesn’t mean I’m not objective or fair. When it...

Read more
October 07, 2014 | By Jim Scott

Get Real with Hadoop: Read-Write File System

In this blog series, we’re showcasing the top 10 reasons customers are turning to MapR in order to create new insights and optimize their data-driven strategies. Here’s reason #9: MapR provides a read-write file system for real-time Hadoop. When the Apache...

Read more
September 12, 2014 | By Jim Scott

Loading a Time Series Database at 100 Million Points Per Second

Why do this? There are many use cases for time series data, and they usually require handling a decent data ingest rate. Rates of more than 10,000 points per second are common and rates of 1 million points per second are not quite as common, but not outrageously...

Read more
September 10, 2014 | By Jim Scott

The Ultimate 3-Minute Guide to Time Series Data and OpenTSDB

What is a time series? A time series is a sequence of data points which are ordered in time. Time series data can come in multiple shapes, and can be used in many facets of everyday life, such as measuring rainfall, earthquake activity, or even stock...

Read more
September 02, 2014 | By Jim Scott

5 Google Projects That Changed Big Data Forever

“Google is living a few years in the future and sends the rest of us messages,” Doug Cutting, Hadoop founder Because of the nature of its business, Google has long been a pioneer in embracing both the challenges and opportunities of big data. Google has...

Read more
Categories

50,000+ of the smartest have already joined!

Stay ahead of the bleeding edge...get the best of Big Data in your inbox.


Get our latest posts in your inbox

Subscribe Now