Levels of Auditing

Describes how to enable auditing for file, table, and stream operations.

There are two levels of auditing:

  • Auditing for cluster level operations
  • Auditing of file system, table, and stream operations
In contrast to auditing cluster-level operations, auditing of filesystem, table, and stream operations needs to be enabled at multiple levels. For auditing file, table, and stream operations, you must first enable auditing at the cluster level and then enable auditing at the volume level. If you want:
  • Granular or selective auditing of content in the volume, you must also enable auditing on each individual directory, file, table, and/or stream in the volume, recursively from the root directory, using the hadoop command. If auditing is enabled at the root directory, all new files inherit the property.
  • To audit all content (files, tables, and/or streams) in the volume, you can set the forceaudit parameter at the volume level, irrespective of what is set (or whether or not auditing is enabled) at the individual file, table, and/or stream level.

The following table summarizes the requirements:

For this type of auditing... You must enable... Using...
Cluster-level operations Auditing at the cluster level audit cluster command
Granular or selective auditing of content (files, tables, and streams) in the volume
  1. Auditing at the cluster level
  2. Auditing at the volume level
  3. Auditing on each individual file, table, and/or stream in the volume, recursively from the root directory
  1. audit cluster command
  2. audit data, volume create, or volume modify command
  3. hadoop mfs command
Auditing all content (files, tables, and streams) in the volume (whether or not auditing is selectively enabled or disabled on the individual file, table, or stream)
  1. Auditing at the cluster level
  2. Auditing at the volume level
  1. audit cluster command
  2. audit data, volume create, or volume modify command

In the following diagram, the illustration on the left shows data auditing enabled at three levels: the cluster level, through the maprcli audit data command; the volume level, through any of the three volume commands shown in the diagram; and the level of the individual directory, file, table, or stream, recursively from the root directory, using the hadoop command. This allows you to include and/or exclude specific directories, files, tables, and streams for auditing. If auditing is not enabled at any one of these levels, operations on an object are not logged.

Alternatively, after enabling auditing at the cluster level, you can enforce auditing for all directories, files, tables, and streams at the volume level itself, irrespective of audit setting at the individual file, table, and/or stream level, using:

The illustration on the right shows auditing enabled at two levels: the cluster level, through the audit data command and the volume level through volume audit command (enabled and forceenable parameters).

Note: You can enable auditing at the volume level using the volume create and volume modify commands also.

Because all levels are enabled, operations that, for example, a client application makes on a directory, file, table, or stream are recorded in an audit log.



To give another example, in the following diagram, auditing is enabled at the cluster level using the audit data command and at the volume level through the the auditenabled and forceauditenable parameters set using any one of the volume commands. Also note that although auditing is explicitly disabled at the directory, file, table, and/or stream level, operations on all directories, files, tables, and streams in the volume are audited because forceauditenable is set to true at the volume level.



For granular or selective auditing, the next diagram shows auditing enabled at the cluster level and the volume level (with just the auditenabled parameter), but not on the directory, file, table, or stream that an operation is performed on. Although the two higher levels are enabled for auditing, the operation is not logged in an audit log because the objects on which auditing has to be performed on is not enabled for auditing.



For granular or selective auditing, to give one final example, in the next diagram auditing is enabled on the individual directory, file, table, or stream and at the cluster level. However, auditing is not enabled at the volume level. Therefore, the operation that the client application performs on the object is not recorded in an audit log.