Establish Communication between Hue and ResourceManagers

If you run YARN applications, integrate Hue with the ResourceManager to view the status of applications in the Job Browser.

Note: If you installed Hue with the MapR Installer, steps 2 is not required as it is performed automatically.
  1. Pick the version of MapReduce (MR) you want to use. Set hadoop.yarn_clusters.default.submit_to=True and hadoop.mapred_clusters.default.submit_to=False to use MRv2. Or vice versa to use MRv1. This option is not used in Hue 4.2+, as support of MRv1 was dropped in latest versions of Hue.
  2. Set hadoop.yarn_clusters.default.resourcemanager_host and hadoop.yarn_clusters.default.resourcemanager_port with RM host and RM IPC ports, respectively. This value is only used to set the <jobTracker> property in Oozie workflows that Hue generates.
  3. There are 4 other properties in the hadoop.yarn_clusters.default section of the hue.ini file that are in fact used by Hue to access different RM-related REST APIs:
    • resourcemanager_api_url
    • proxy_api_url (URL of RM ProxyServer API, which typically equals RM REST API)
    • history_server_api_url
    • spark_history_server_url
    The above properties are determined by Hue at runtime by capturing the output of maprcli -urls <service>. And in the case where Hue is not able to get any data from some of these APIs, it reruns the maprcli -urls <service> command to get the updated URL for some of these services.
  4. In the hadoop section, set webhdfs_url to be the node that runs HttpFS.
  5. In the yarn_clusters section within the hadoop section, set the submit_to value to True.
  6. In the mapred_clusters section, set the submit_to value to False.

YARN Example

[desktop]
        
        [hadoop]
        
        # Use WebHdfs/HttpFs as the communication mechanism.
        # This should be the web service root URL.
        # The ip_address corresponds to the node running httpfs.
        webhdfs_url=http://<ip_address>:14000/webhdfs/v1
        
        [[yarn_clusters]]
        [[[default]]]
        # Enter the host on which you are running the ResourceManager
        resourcemanager_host=localhost
        
        # The port where the ResourceManager IPC listens on
        resourcemanager_port=8032
        
        # Whether to submit jobs to this cluster
        submit_to=True
        
        ...
        
        [[mapred_clusters]]
        [[[default]]]
        
        # Whether to submit jobs to this cluster
        submit_to=False