The Spyglass Initiative

On The MapR Converged Data Platform

The Spyglass Initiative is a multi-release MapR effort with the vision of increasing user and administrator productivity. It takes a comprehensive, open, and extensible approach to simplifying big data deployments.

Spyglass Phase 1 – Summer 2016: In the first phase of the Spyglass Initiative MapR focuses on operational visibility, to help customers with their ongoing big data successes. Successful big data deployments continue to get bigger and more complex. With new data sources, new use cases, new workloads, and new user groups, managing that growth requires a complete understanding of what is currently happening in the system. MapR Monitoring helps you to manage successful big data deployments by giving you a converged, customizable, and extensible platform for cluster-wide visibility.

In Fall 2016: MapR Installer Stanzas is a new capability that allows programmatic software installation to automate new deployments. This is especially useful for quickly deploying elastic clusters across the cloud and on-premises. “Stanzas” are configuration files that describe a cluster and enable the API-driven installation of MapR clusters.

MapR Monitoring Key Benefits

Convergence

The MapR Converged Data Platform integrates key big data capabilities (Hadoop/Spark, SQL, web-scale storage, NoSQL, event streams) into a common platform with architectural optimizations for high efficiency. MapR Monitoring gives you a converged view of all operations to help you with decisions around use case expansion and resource allocation.

Customizability

MapR Monitoring is highly customizable, giving you the flexibility to collect and view data to meet your specific requirements. This is especially important in multi-tenant environments where you have a specific set of operations that you want to track in a way that gives you the best insights.

Extensibility

By leverage open source technologies with open APIs, you can track new compute engines added to the platform, and also plug in to existing tools to help monitor your deployment.

Customizable Dashboards for Visualizing Metrics Log Analytics
Dashboards and log analytics visualizations allow complete visibility into your cluster.

MapR Monitoring collects, aggregates, stores, and visualizes all critical cluster operations data. Data is collected on node utilization, space utilization, YARN/MapReduce applications, and ecosystem services.

MapR Monitoring is included as a standard feature in all editions of the MapR Converged Data Platform.

The MapR Monitoring architecture consists of 4 primary layers:

  • Data sources create the monitoring data to be collected and tracked.

  • The collection layer is responsible for delivering data from the sources to the storage layer.

  • The aggregation and storage layer consists of two popular open source storage technologies, OpenTSDB and Elasticsearch, for storing metrics and logs and providing APIs for access.

  • The visualization layer ties into the aggregation and storage layer to provide customizable dashboards on cluster operations, using the popular open source technologies Grafana and Kibana.

Visualization dashboards can be easily customized and then exported for sharing. The MapR Community Exchange, part of the MapR Community, lets you share your custom dashboards and associated best practices. The MapR Exchange is ideal for sharing other assets such as code, tutorials, and demos.

Additional resources

Administrator

When applications go from idea to reality, MapR provides the only production-ready platform for Hadoop, Spark, and related technologies.

Learn More

Enterprise Architect

The design of the patented MapR Converged Data Platform speaks directly to Enterprise Architects who know best that architecture matters.

Learn More

Developer

MapR provides developers the widest variety of popular open source projects for developing data applications.

Learn More