Settings for the Spark Driver and Executors

The object can contain the following properties:
Property Type Description
cores integer Specifies the amount of CPU for the pod.
coreLimit string Specifies the maximum CPU for the pod.
memory string Specifies the amount of memory.
affinity object Specifies the node on which to launch the pod. For more information, see Setting Affinity for Spark Pods Using Node Labels.
labels object Specifies the supported Spark version. The default value is 2.4.4.
serviceAccount string Specifies the name of the service account in the CSpace.
volumeMounts object Specifies the name of the Kubernetes volume (for the driver and executor) and the mount path to that volume from inside the container.
Note: In the Spark application on external CSpaces, if spark.authenticate is set to true, the Spark executor cannot connect to the Spark driver, and the job fails. To prevent the job from failing, set the spark.authenticate property to false.