Recompiling MapReduce V1 Applications

Existing compiled MapReduce V1 applications may need to be recompiled before they can be run as MapReduce V1 applications in MapR Version 5.x.

The small number of API changes that have been made, including removal of classes and methods and conversion of classes to interfaces, are documented here. If your application does not use any of the changes listed in this document, you do not need to recompile the application.

When an application has been compiled against MapReduce V1 or MapReduce V2 (YARN), the application can be run in either mode.

The following list of changes is grouped by package name.

org.apache.hadoop.mapred.jobcontrol

Job
  • extends ControlledJob
  • getMapredJobID: return type changed from String to JobID
JobControl
  • extends org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl
  • String addJob(ControlledJob) instead of String addJob(Job), where Job extends ControlledJob

org.apache.hadoop.mapred

JobContext
  • Changed from class to interface
JobInProgress
  • All methods removed. Counter class remains for backward compatibility.
JobEndNotifier
  • Removed methods:
    • void registerNotification(JobConf, JobStatus)
    • void startNotifier()
    • void stopNotifier()
Operation
  • QueueACL qACLNeeded: Change in type from org.apache.hadoop.mapred.QueueManager.QueueACL to org.apache.hadoop.mapred.QueueACL
TaskAttemptContext
  • Changed from class to interface
  • Progress method removed
Counters
  • Extends AbstractCounters, and some methods have been type parameterized, using generic. This change breaks binary compatibility.
TaskLog
  • Removed methods: captureDebugOut, captureOutAndError, getJobDir, getTaskLogFile, getUserLogDir
TaskStatus
  • Removed methods: boolean getIncludeCounters(), TaskLogFS getTaskLogFs(), void setIncludeCounters(boolean), void setTaskLogFs(TaskLogFS)
TaskUmbilicalProtocol
  • Signature change for several methods where the JvmContext argument has been removed.
Utils
  • getHttpScheme removed
ClusterStatus
  • getMaxPrefetchMapTasks() removed
JobClient
  • New exception thrown
Task
  • Change in exceptions: java.lang.ClassNotFoundException was removed. java.lang.InterruptedException was added.
  • Change of visibility from protected to public.

org.apache.hadoop.mapreduce

Counter
  • Changed from class to interface
  • Removed:
    • boolean equals(Object)
    • int hashCode()
    • void readFields(DataInput): read the binary representation of the counter
    • void write(DataOutput)
CounterGroup:
  • Changed from class to interface
  • Removed:
    • boolean equals(Object)
    • Counter findCounter(String, String): internal to find a counter in a group
    • Counter findCounter(String)
    • String getDisplayName(): get the display name of the group
    • String getName(): get the internal name of the group
    • int hashCode()
    • void incrAllCounters(CounterGroup)
    • Iterator<Counter> iterator()
    • void readFields(DataInput)
    • int size(): return the number of counters in this group
    • void write(DataOutput)
Counters
  • Extends AbstractCounters, and some methods have been type parameterized, using generic. This breaks binary compatibility.
Job
  • Removed: TaskCompletionEventList getTaskCompletionEventList(int), void setUserClassesTakesPrecedence(boolean)
JobContext:
  • Changed from class to interface
  • Removed: boolean userClassesTakesPrecedence()
JobSubmissionFiles
  • getStagingDir: Change in signature from (JobClient, Configuration) to (Cluster, Configuration)
MapContext
  • Changed from class to interface
Mapper.Context
  • Changed from non-abstract to abstract

ReduceContext
  • Changed from class to interface
ReduceContext.ValueIterator
  • Changed from class to interface
Reducer.Context
  • Changed from non-abstract to abstract

TaskAttemptContext
  • Changed from class to interface
  • Removed: progress
TaskInputOutputContext
  • Changed from class to interface

org.apache.hadoop.mapreduce.lib.output

FileOutputCommitter
  • abortTask: Change in exceptions thrown from no exceptions to java.io.IOException.

org.apache.hadoop.mapreduce.security.token

DelegationTokenRenewal
  • Removed class

org.apache.hadoop.mapreduce.util

ProcessTree
  • Change in signature for some methods (emoved arg - signal; now just takes pid): killProcess, killProcessGroup, etc.