Download Storm from https://github.com/yahoo/storm-yarn/archive/master.zip with the following command:
Unzip the downloaded archive, then change to the storm-yarn-master directory:
# unzip master.zip
# cd storm-yarn-master
Add the following section to the
<hadoop.version>property in the
pom.xmlfile with the following line:
Create a work folder to hold working files for Storm. In this example, the work folder is named
# mkdir storm-work-folder
lib/storm.zipfile to the work folder.
# cp lib/storm.zip /storm-work-folder
storm.zipfile to the MapR-FS layer with a
hadoop fs -putcommand or over an NFS mount point. The following example uses
hadoop fs -put:
# hadoop fs -put storm.zip /lib/storm/0.9.0-wip21
storm-yarn-masterfolder, start Maven:
# cd storm-yarn-master
# mvn package -DskipTests
storm-0.9.0-wip21/conf/storm.yamlfile to include the ZooKeeper servers for your cluster. The file uses the following syntax:
- "<ip address of ZooKeeper1>"
- "<ip address of ZooKeeper2>"
- "<ip address of ZooKeeper3>"
storm.zookeeper.port: <port number>
Launch Storm on Yarn with the following command:
# storm-yarn launch <path to your storm.yaml file>
Use the following command to display the YARN application ID for Storm:
# yarn application -list
storm.yamlfile in the .storm directory to enable Storm to find the configuration file when Storm submits jobs:
# storm-yarn getStormConfig -appId <application ID> -output ~/.storm/storm.yaml
Use the following command to display the Nimbus host:
# cat ~/.storm/storm.yaml | grep nimbus.host
Run two sample topologies to verify Storm is working correctly. To run the word count example, use the following command:
storm jar lib/storm-starter-0.0.1-SNAPSHOT.jar storm.starter.WordCountTopology WordCountTopology -c nimbus.host=<your nimbus host>
To run the Exclamation example, use the following command:
storm jar lib/storm-starter-0.0.1-SNAPSHOT.jar storm.starter.ExclamationTopology ExclamationTopology -c nimbus.host=<your nimbus host>
Monitor the results of the sample topologies by connecting to the Storm Web UI with a browser. The Storm Web UI is typically at the Nimbus host, on port 7070.
MapR 5.0 Documentation : Storm on YARN
This procedure guides you through the process of installing and launching Storm on YARN.
Screen Shot 2014-03-24 at 3.08.14 PM.png (image/png)