kafka单条日志超过1M解决方案

        一、说明

kafka默认会丢弃超过单调日志超过1M的记录。如果单条日志超出1M（压缩后），客户端会报出如下错误。

错误日志

Icon

java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.RecordTooLargeException: The request included a message larger than the max message size the server will accept.

at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:70)

at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:57)

at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:25)

at sina.kafka.group.kafka102ProducerTestForMaxMessage.main(kafka102ProducerTestForMaxMessage.java:63)

Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The request included a message larger than the max message size the server will accept.