job linklogs

Creates symbolic links to all the logs relating to the activity of a specific job.

The maprcli job linklogs command works with the Centralized Logging to provide a job-centric view or an application-centric view of all log files generated during job or application execution.

The output of job linklogs is a directory populated with symbolic links to all log files related to the specified job(s) or to the application. The command can be performed during or after a job or application is processed.

Syntax

CLI
maprcli job linklogs
    -jobid <jobPattern>
    -todir <desinationDirectory>
    [-jobconf <pathToJobXml>]
REST
http[s]://<host>:<port>/rest/job/linklogs?jobid=<jobPattern>&todir=<destinationDirectory> 

Parameters

Parameter

Description

jobid

  • For MapReduce v1, specify a regular expression specifying the target jobs. To specify a pattern, you can use asterisks. You can prefix a pattern with job; otherwise, job will be automatically prepended to the pattern. There is just one match if the full job ID is used.
  • For MapReduce v2, specify the application ID.

todir

The target directory for the symbolic links to the log files.

jobconf
  • For MapReduce v1 jobs, specify this parameter if the centralized job history location has changed since the job of interest was run. This command uses the centralized job history location specified by mapred.job.tracker.history.completed.location. The centralized job history location is by default /var/mapr/cluster/mapred/jobTracker/history/done.
  • For MapReduce v2, this parameter is not applicable.

Output

For MapReduce v1, the following directory structure will be created in the location specified by todir for all jobids matching the jobid parameter.

  • <jobid>/hosts/<host>/ contains symbolic links to log directories of tasks executed for <jobid> on <host>
  • <jobid>/mappers/ contains symbolic links to log directories of all map task attempts for <jobid> across the whole cluster
  • <jobid>/reducers/ contains symbolic links to log directories of all reduce task attempts for <jobid> across the whole cluster

For MapReduce v2, the following directory will be created in the location specified by todir for the application ID that you specify for the jobid parameter:

  • <applicationId>/hosts/<host>/ contains symbolic links to log directories of tasks executed for <applicationId> on <host>

Examples

Link logs for all jobs named "wordcount1" and dump output to /myvolume/joblogviewdir:
CLI
maprcli job linklogs -jobid job_*_wordcount1 -todir /myvolume/joblogviewdir 
REST
https://r1n1.sj.us:8443/rest/job/linklogs?jobid=job_*_wordcount1&todir=/myvolume/joblogviewdir