task table

Retrieves histograms and line charts for task metrics.

A Hadoop job sets the rules that the JobTracker service uses to break an input data set into discrete tasks and assign those tasks to individual nodes. Use the task table to retrieve task analytics about the jobs running on your cluster. The task metric data includes information about the tasks that make up a specific job, as well as the specific task attempts. The job metric data includes a task attempt's data throughput, measured in number of records per second as well as in bytes per second. The metrics data can be formatted for histogram display or line chart display. In order to issue the task table command, the mapr-metrics package must be installed on all the nodes where webserver and jobtracker are configured to run.

Syntax

CLI
N/A
REST
http[s]://<host>:<port>/rest/task/table?output=terse&filter=string&chart=chart_type&columns=list_of_columns&scale=scale_type<parameters> 

Parameters

Parameter

Description

filter

Filters results to match the value of a specified string.

chart

Chart type to use: line for a line chart, bar for a histogram.

columns

Comma-separated list of column names to return.

bincount

Number of histogram bins.

scale

Scale to use for the histogram. Specify linear for a linear scale and log for a logarithmic scale.

Column Names

The following table lists the terse short names for particular metrics regarding task attempts.

Parameter

Description

tacir

Combine Task Attempt Input Records

tacor

Combine Task Attempt Output Records

tamib

Map Task Attempt Input Bytes

tamir

Map Task Attempt Input Records

tamob

Map Task Attempt Output Bytes

tamor

Map Task Attempt Output Records

tamsr

Map Task Attempt Skipped Records

tarig

Reduce Task Attempt Input Groups

tarir

Reduce Task Attempt Input Records

taror

Reduce Task Attempt Output Records

tarsb

Reduce Task Attempt Shuffle Bytes

tarsr

Reduce Task Attempt Skipped Records

tacput

Task Attempt CPU Time

talbr

Task Attempt Local Bytes Read

talbw

Task Attempt Local Bytes Written

tambr

Task Attempt MapR-FS Bytes Read

tambw

Task Attempt MapR-FS Bytes Written

tapmem

Task Attempt Physical Memory Bytes

taspr

Task Attempt Spilled Records

tavmem

Task Attempt Virtual Memory Bytes

tad

Task Attempt Duration (histogram only)

tagct

Task Attempt Garbage Collection Time (histogram only)

td

Task Duration (histogram only)

taid

Task Attempt ID (filter only)

tat

Task Attempt Type (filter only)

tas

Task Attempt Status (filter only)

tapro

Task Attempt Progress (filter only)

tast

Task Attempt Start Time (filter only)

taft

Task Attempt Finish Time (filter only)

tashe

Task Attempt Shuffle End

tase

Task Attempt Sort End

tah

Task Attempt Host Location

talog

Location of logs, stderr, and stdout for this task attempt.

tadi

Freeform information about this task attempt used for diagnosing behaviors.

tarsg

Reduce Task Attempt Skipped Groups (filter only)

tasrb

Reduce Task Attempt Shuffle Bytes

tamirps

Map Task Attempt Input Records per Second

tarirps

Reduce Task Attempt Input Records per Second

tamorps

Map Task Attempt Output Records per Second

tarorps

Reduce Task Attempt Output Records per Second

tamibps

Map Task Attempt Input Bytes per Second

tamobps

Map Output Bytes per Second

tarsbps

Reduce Task Attempt Shuffle Bytes per Second

ts

Task Status (filter only)

tid

Task Duration

tt

Task Type (filter only)

tsta

Primary Task Attempt ID (filter only)

tst

Task Start Time (filter only)

tft

Task End Time (filter only)

th

Task Host Location (filter only)

thl

Task Host Locality (filter only)

Example

Retrieve a Task Histogram:

REST
https://r1n1.sj.us:8443/rest/task/table?chart=bar&filter=%5Btt!=JOB_SETUP%5Dand%5Btt!=JOB_CLEANUP%5Dand%5Bjid==job_201129649560_3390%5D&columns=td&bincount=28&scale=log
CURL
curl -d @json https://r1n1.sj.us:8443/api/task/table 

In the curl example above, the json file contains a URL-encoded version of the information in the Request section below.

Request

GENERAL_PARAMS:
{
        [chart: "bar"|"line"],
        columns: <comma-separated list of column terse names>,
        [filter: "[<terse_field>{operator}<value>]and[...]",]
        [output: terse,]
        [start: int,]
        [limit: int]
}

REQUEST_PARAMS_HISTOGRAM:
{
        chart:bar
        columns:td
        filter: <anything>
}

REQUEST_PARAMS_LINE:
{
        chart:line,
        columns:tapmem,
        filter:  NOT PARSED, UNUSED IN BACKEND
}

REQUEST_PARAMS_GRID:
{
        columns:tid,tt,tsta,tst,tft
        filter:<any real filter expression>
        output:terse,
        start:0,
        limit:50
}

Response

RESPONSE_SUCCESS_HISTOGRAM:
{
        "status" : "OK",
        "total" : 15,
        "columns" : ["td"],
        "binlabels" : ["0-5s","5-10s","10-30s","30-60s","60-90s","90s-2m","2m-5m","5m-10m","10m-30m","30m-1h","1h-2h","2h-6h","6h-12h","12h-24h",">24h"],
        "binranges" : [
                [0,5000],
                [5000,10000],
                [10000,30000],
                [30000,60000],
                [60000,90000],
                [90000,120000],
                [120000,300000],
                [300000,600000],
                [600000,1800000],
                [1800000,3600000],
                [3600000,7200000],
                [7200000,21600000],
                [21600000,43200000],
                [43200000,86400000],
                [86400000]
        ],
        "data" : [33,919,1,133,9820,972,39,2,44,80,11,93,31,0,0]
}

RESPONSE_SUCCESS_GRID:
{
        "status": "OK",
        "total"       : 67,
        "columns" : ["ts","tid","tt","tsta","tst","tft","td","th","thl"],
        "data"        : [
                   ["FAILED","task_201204837529_1284_9497_4858","REDUCE","attempt_201204837529_1284_9497_4858_3680",
                    1301066803229,1322663797292,21596994063,"newyork-rack00-8","remote"],
                   ["PENDING","task_201204837529_1284_9497_4858","MAP","attempt_201204837529_1284_9497_4858_8178",
                    1334918721349,1341383566992,6464845643,"newyork-rack00-7","unknown"],
                   ["RUNNING","task_201204837529_1284_9497_4858","JOB_CLEANUP","attempt_201204837529_1284_9497_4858_1954",
                    1335088225728,1335489232319,401006591,"newyork-rack00-8","local"],
                  ]}

RESPONSE_SUCCESS_LINE:
{
        "status" : "OK",
        "total" : 22,
        "columns" : ["tapmem"],
        "data" : [
                [1329891055016,0],
                [1329891060016,8],
                [1329891065016,16],
                [1329891070016,1024],
                [1329891075016,2310],
                [1329891080016,3243],
                [1329891085016,4345],
                [1329891090016,7345],
                [1329891095016,7657],
                [1329891100016,8758],
                [1329891105016,9466],
                [1329891110016,10345],
                [1329891115016,235030],
                [1329891120016,235897],
                [1329891125016,287290],
                [1329891130016,298390],
                [1329891135016,301355],
                [1329891140016,302984],
                [1329891145016,303985],
                [1329891150016,304403],
                [1329891155016,503030],
                [1329891160016,983038]
        ]
}