Creating MapR Credentials for Spark Applications in Compute Spaces

Kubernetes and MapR users are not the same by default. Which Kubernetes user you are determines what pods you can run and what CSpaces you can explore. Your MapR user determines what data you can see. If both Kubernetes and MapR users are created in the same corporate directory, the users will be the same. If they are not, the users might be different and things can become confusing.

Spark jobs must run as a specific user to view a user's MapR Filesystem files. In a secure MapR storage cluster, you must also have a user ticket to provide access to data. Without a user ticket, a Spark job cannot access a secure MapR storage cluster. You can log in to a MapR node in a secure cluster and generate a ticket for running your Spark job. Manually copying various tickets to every pod running a Spark job in Kubernetes would be unwieldy. Instead, you can use Kubernetes secrets containing user info and tickets. These secrets are mounted by applications pods such as Spark. They facilitate MapR storage access in applications. A single secret can support multiple pods. The MapR Kubernetes Ecosystem refers to these secrets as user secrets.

In addition to tickets, Spark on Kubernetes also requires user information that includes UIDs, GIDs, and passwords. This information is stored in the user secret along with the ticket. Each Compute Space must have a user secret for every user wishing to run Spark jobs or other containers that also require user info.

In in the CSpace Terminal pod of each Compute Space, the MapR Kubernetes Ecosystem provides a utility called ticketcreator.sh. Users other than root can ssh into a CSpace Terminal and run the ticketcreator.sh utility. The utility generates a user secret containing user information and the user's ticket (if the cluster is secure) and adds it to the CSpace. For more information, see Using the Ticketcreator Utility to Generate Secrets.

You can also manually set up a secret for a CSpace user. For more information, see Manually Setting Up the User Secret.