About Running Compute Applications in Compute Spaces

The MapR Kubernetes Ecosystem includes operators for setting up Spark application(s) and Drill cluster(s) in Compute Spaces (CSpaces). These operators are installed on the Kubernetes cluster during the bootstrap phase and follow the operator pattern for managing the lifecycle of the applications on the Kubernetes cluster. The following sections describe how to use the Spark operator and Drill operator to build and run the applications in the CSpace.

Requirements

To run applications in a Compute Space, the following requirements must be met:
  • A bare-metal MapR storage cluster (instance) is visible from CSpace pods running in the external Kubernetes cluster. To verify that there is connectivity, an administrator should exec into a shell to a running pod on the Kubernetes cluster, and ping nodes on the storage cluster.
  • A Kubernetes cluster has been successfully bootstrapped and operators installed.
  • User, Client, and Server secrets and hosts for the external cluster have been added to the mapr-external-info namespace.
  • An external CSpace has been created by installing a CSpace CR.
  • Ticket and user secrets have been created as required by the compute application. This can be done by running the /opt/mapr/kubernetes/ticketcreator.sh tool in the CSpace Terminal pod, as described in Using the Ticketcreator Utility to Generate Secrets.

Workflow

To create a Spark or Drill application in your CSpace, do the following:
  1. Create a CR of the proper type for your application. You can run Spark and Drill applications in the CSpace. For more information, see Running Spark Applications in Compute Spaces and Running Drillbits in Compute Spaces.
  2. Apply your CR using the custom resource file:
    kubectl apply -f <path to custom resource file of application>
    
  3. Run the following command to verify whether the CR was successfully submitted:
    kubectl get pods -n <application cspace>