The Two Modes of Publishing

The producer client library batches messages into publish requests that it sends to the MapR Streams server. By default, the client sends multiple publish requests without waiting for acknowledgements from the MapR Streams server. This behavior is determined by the per producer configuration parameter streams.parallel.flushers.per.partition, which defaults to true.

With this behavior turned on, it is possible for messages to arrive to partitions out of order due to the presence of multiple network interface controllers, network errors, or retries.

For example, suppose a producer is sending messages that are specifically for partition 1. The producer client library buffers the messages and sends a batch to partition 1. Meanwhile, the producer keeps sending messages for partition 1 and the client continues to buffer them. The next time the producer client library has enough messages for partition 1, the client sends another batch, whether or not the MapR Streams server has acknowledged the previous batch.

If you always want messages to arrive to partitions in the order in which they were sent, set the configuration parameter streams.parallel.flushers.per.partition to false. The producer client library will wait for acknowledgements from the MapR Streams server before sending subsequent publish requests.