hadoop pipes

The hadoop pipes command runs a pipes job.

Warning: This command is deprecated.

Hadoop Pipes is the C++ interface to Hadoop Reduce. Hadoop Pipes uses sockets to enable tasktrackers to communicate processes running the C++ map or reduce functions.


hadoop [GENERIC OPTIONS ] pipes
    [-output <path>]
    [-jar <jar file>]
    [-inputformat <class>] 
    [-map <class>]
    [-partitioner <class>]
    [-reduce <class>]
    [-writer <class>]
    [-program <executable>]
    [-reduces <num>]


Command Options

The following command parameters are supported for hadoop pipes:



-output <path>

Specify the output directory.

-jar <jar file>

Specify the jar filename.

-inputformat <class>

InputFormat class.

-map <class>

Specify the Java Map class.

-partitioner <class>

Specify the Java Partitioner.

-reduce <class>

Specify the Java Reduce class.

-writer <class>

Specify the Java RecordWriter.

-program <executable>

Specify the URI of the executable.

-reduces <num>

Specify the number of reduces.

Generic Options

The following generic options are supported for the hadoop pipes command: -conf <configuration file>, -D <property=value>, -fs <local|file system URI>, -jt <local|jobtracker:port>, -files <file1,file2,file3,...>, -libjars <libjar1,libjar2,libjar3,...>, and -archives <archive1,archive2,archive3,...>. For more information on generic options, see Generic Options.