Example 4

On this example 10-node cluster, each node has the following characteristics:

  • 8 cores
  • 12 GB memory
  • 3 drives

This hardware is resource-constrained across the board. Reduce the chunk size to improve parallelism, use less memory for the operating system, and set a small number of mappers and reducers:

  • Set chunk size to 128 MB
  • Set io.sort.mb to 190 MB
  • 4 map slots
  • 2 reduce slots
  • Give any leftover memory to MapR-FS