Supported API for MapR Streams C Client

This topic lists the librdkafka APIs supported by the MapR Streams C Client. It also describes behavior differences between librdkafka and the MapR Streams C Client.

Producer APIs

API Behavior
rd_kafka_produce When this API is called with NULL payload, an invalid argument error is sent to the callback. librdkafka creates a message with NULL payload and key value instead.
rd_kafka_produce_batch When this API is called with NULL payload, an invalid argument error is sent to the callback. librdkafka creates a message with NULL payload and key value instead.
rd_kafka_outq_len This API returns a positive number to indicate that messages are waiting to be produced to a MapR streams topic but the value does not indicate the actual number of messages. librdkafka returns the actual number of messages that are waiting to be sent to or acknowledged by the broker.
rd_kafka_poll Same as librdkafka.

Consumer APIs

API Behavior
rd_kafka_assign If this API is called for a consumer that is already subscribed to topics, no operation is performed.
rd_kafka_assignment This API returns the number of topic partitions the consumer is assigned to. However, it returns 0 when topic partitions have yet to be created by the producer. librdkafka returns the number of partitions assigned to a consumer even when the partitions have not been created.
rd_kafka_commit Same as librdkafka.
rd_kafka_commit_message Same as librdkafka.
rd_kafka_committed Same as librdkafka.
Note: The MapR Streams offset starts at 1.
rd_kafka_consumer_close Same as librdkafka.
rd_kafka_consumer_poll Same as librdkafka.
rd_kafka_group_list_destroy Same as librdkafka.
rd_kafka_list_groups This API can only be used by consumers that are subscribed to at least one stream on the cluster and have a default stream configured with the streams.consumer.default.stream parameter. It returns the group list of subscribed consumers associated with the default stream. librdkafka returns all consumer groups from the cluster instead.
Note: This API returns RD_KAFKA_RESP_ERR__TIMED_OUT when the querying consumer is not subscribed to any topic.
rd_kafka_message_destroy Same as librdkafka.
rd_kafka_poll_set_consumer Same as librdkafka.
rd_kafka_position This API returns 0 when the messages have not yet been consumed from partitions. librdkafka returns -1001 instead.
rd_kafka_seek Same as librdkafka.
rd_kafka_subscribe Same as librdkafka.
rd_kafka_subscription This API allows either a list of topics from one or more streams or a regex expression for topics from a single streams. For example, regex expression /streamA:^t*a,/streamA:^t*b is supported but /streamA:^t*a,/streamB:^t*a is not supported. librdkafka accepts both options in the same call.
Note: You cannot use the rd_kafka_subscribe API to subscribe a consumer to topics when that consumer is already assigned to topics. If you call this API for an assigned consumer, error RD_KAFKA_RESP_ERR__CONFLICT is returned.
rd_kafka_unsubscribe Same as librdkafka.

Producer/Consumer Common APIs

API Behavior
rd_kafka_conf_destroy Same as librdkafka.
rd_kafka_conf_dup Same as librdkafka.
rd_kafka_conf_get Same as librdkafka.
rd_kafka_conf_new Same as librdkafka.
rd_kafka_conf_set Same as librdkafka.
rd_kafka_conf_set_consume_cb Same as librdkafka.
rd_kafka_conf_set_dr_msg_cb Same as librdkafka.
rd_kafka_conf_set_error_cb Same as librdkafka.
rd_kafka_conf_set_offset_commit_cb Same as librdkafka.
rd_kafka_conf_set_opaque Same as librdkafka.
rd_kafka_conf_set_rebalance_cb Same as librdkafka.
rd_kafka_conf_set_throttle_cb Same as librdkafka.
rd_kafka_destroy Same as librdkafka.
rd_kafka_new Same as librdkafka.
rd_kafka_opaque Same as librdkafka.
rd_kafka_wait_destroyed Same as librdkafka.
rd_kafka_yield Same as librdkafka.

Topic APIs

API Behavior
rd_kafka_conf_set_default_topic_conf Same as librdkafka.
rd_kafka_topic_conf_destroy Same as librdkafka.
rd_kafka_topic_conf_dup Same as librdkafka.
rd_kafka_topic_conf_new Same as librdkafka.
rd_kafka_topic_conf_set Same as librdkafka.
rd_kafka_topic_conf_set_opaque Same as librdkafka.
rd_kafka_topic_conf_set_partitioner_cb Same as librdkafka.
rd_kafka_topic_destroy Same as librdkafka.
rd_kafka_topic_name Same as librdkafka.
rd_kafka_topic_new Same as librdkafka..
rd_kafka_topic_partition_list_add Same as librdkafka.
rd_kafka_topic_partition_list_add_range Same as librdkafka.
rd_kafka_topic_partition_list_copy Same as librdkafka.
rd_kafka_topic_partition_list_del Same as librdkafka.
rd_kafka_topic_partition_list_del_by_idx Same as librdkafka.
rd_kafka_topic_partition_list_destroy Same as librdkafka.
rd_kafka_topic_partition_list_find Same as librdkafka.
rd_kafka_topic_partition_list_new Same as librdkafka.
rd_kafka_topic_partition_list_set_offset Same as librdkafka.

Cluster Configuration APIs

API Behavior
rd_kafka_brokers_add

This API has no impact on MapR Streams since MapR Streams does not utilize Kafka brokers. When this API is called, the MapR Streams client may print a brokers are down error message to the console.

Miscellaneous API

API Behavior
rd_kafka_err2name Same as librdkafka.
rd_kafka_err2str Same as librdkafka.
rd_kafka_errNo Same as librdkafka.
rd_kafka_errNo2err Same as librdkafka.
rd_kafka_get_err_descs Same as librdkafka.
rd_kafka_last_error Same as librdkafka.
rd_kafka_log_print Same as librdkafka.
rd_kafka_log_syslog Same as librdkafka.
rd_kafka_message_errstr Same as librdkafka.
rd_kafka_set_logger Same as librdkafka.

Additional Information

For more information and API signatures, see rdkafka.h.