Settings for Spark Configuration

The Spark resource file should contain values for the following MapR-specific properties in the sparkConf object:
Property Description
spark.mapr.user.secret (Required for communication with a secure MapR cluster) Specifies the name of the object that contains the Spark user secret. This Spark user secret can be generated in the CSpace Terminal pod using the ticketcreator.sh script in the /opt/mapr/kubernetes directory to run the application on a secure MapR cluster with a ticket from the MapR cluster. For more information, see Creating MapR Credentials for Spark Applications in Compute Spaces.
spark.eventLog.enabled Specifies whether or not event logging is enabled. Use one of the following values:
  • true – to enable the event log
  • false – to disable the event log
If true, you must also specify the directory for the event log.
spark.eventLog.dir Specifies the directory where event logs are stored.
Note: If the Spark event log is enabled using the spark.eventlog.enabled property, the spark.evenLog.dir property configures the Spark application to write logs by default to maprfs:///apps/spark/<cspaceName>. However, for the sample minimum CRs, the maprfs:///apps/spark/<cspaceName> must be created manually.