Creating and Deploying a Custom Resource for Spark Applications

After setting up your Compute Space and creating a MapR user secret, you can run Spark jobs. To help you create your first Spark job, sample Spark custom resource files are provided. These Spark CR files are the definition used by your Spark jobs. You do not have to use the MapR samples. You can create your own CR file, specifying which JAR files, libraries, and image resources to use in your Spark application.

About the Sample Spark Custom Resource

The examples/spark/ directory includes the following sample Spark CRs:
Sample Spark CR Description
mapr-spark-hive.yaml Shows how to interact with Hive Metastore.
mapr-spark-naive-bayes-r.yaml Shows how to run a Naive Bayes Job in Spark on R.
mapr-spark-pi-affinity.yaml Shows how to run a simple Spark Pi job with affinity to specific nodes.
mapr-spark-pi-scheduled.yaml Shows how to schedule a simple Spark Pi job to run on a timer.
mapr-spark-pi.yaml Simple Spark Pi job. This CR is useful for testing Spark on Kubernetes.
mapr-spark-py-with-dependencies.yaml Shows how to run PySpark with separate dependency libraries.
mapr-spark-streaming-queue.yaml Shows how to run a Spark job with a streaming queue.
mapr-spark-wc-py.yaml Reads a document and counts the number of words. This version uses PySpark to do the word count.
mapr-spark-wc-s3.yaml Shows how to run a Spark job that counts words on a document stored in an S3 bucket.
mapr-spark-wc.yaml Shows how to run a Spark job that does a word count on a document stored on the MapR Data Platform.