Configure Myriad

This section provides the steps for configuring Myriad.

  1. Modify the myriad-config-default.yml file.
    As a minimum, the following Myriad configuration parameters must be set:
    • mesosMaster
    • zkServers
    • YARN_HOME
    Note: The value of mesosMaster parameters may be similar to <mesosMasterHost>:5050 or zk://<zkHost>:5181/<path_to_mesos_master>
  2. Configure YARN to use Myriad.
    Edit the $YARN_HOME/etc.hadoop/yarn-site.xml file and add the following properties:
    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>RESOURCEMANAGER_HOSTNAME</value>
    </property>
    
    <property>
      <name>yarn.resourcemanager.recovery.enabled</name>
      <value>true</value>
    </property>
    
    <property>
      <name>yarn.resourcemanager.scheduler.class</name> 	  <value>org.apache.myriad.scheduler.yarn.MyriadFairScheduler</value>
    </property>
    
    <property>
      <name>yarn.nodemanager.resource.cpu-vcores</name>
      <value>${nodemanager.resource.cpu-vcores}</value>
    </property>
    
    <property>
      <name>yarn.nodemanager.resource.memory-mb</name>
      <value>${nodemanager.resource.memory-mb}</value>
    </property>
    
    <property>
      <name>yarn.nodemanager.address</name>
      <value>${myriad.yarn.nodemanager.address}</value>
    </property>
    
    <property>
      <name>yarn.nodemanager.webapp.address</name>
      <value>${myriad.yarn.nodemanager.webapp.address}</value>
    </property>
    
    <property>
      <name>yarn.nodemanager.webapp.https.address</name>
      <value>${myriad.yarn.nodemanager.webapp.address}</value>
    </property>
    
    <property>
      <name>yarn.nodemanager.localizer.address</name>
      <value>${myriad.yarn.nodemanager.localizer.address}</value>
    </property>
    
    <property>
      <name>mapreduce.shuffle.port</name>
      <value>${myriad.mapreduce.shuffle.port}</value>
    </property>
    
    <property>
      <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle,mapr_direct_shuffle,myriad_executor</value>
    </property>
    
    <property>
      <name>yarn.nodemanager.aux-services.myriad_executor.class</name>
      <value>org.apache.myriad.executor.MyriadExecutorAuxService</value>
    </property>
    
    <property>
      <name>yarn.resourcemanager.store.class</name>
    <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore</value>
    </property>
    
    <property>
      <name>yarn.scheduler.minimum-allocation-mb</name>
      <value>0</value>
     </property>
    
    <property>
      <name>yarn.scheduler.minimum-allocation-vcores</name>
      <value>0</value>
    </property>
    
    <property>
      <name>yarn.scheduler.minimum-allocation-disks</name>
      <value>0</value>
    </property>
          

See Configuring Services for information about configuring JobHistoryServer and other services. See Use Myriad for information about using Myriad.