I’m using kafka-python library for my fastapi consumer app and I’m consuming messages in batch with maximum of 100 records. Since the topic has huge traffic and have only one partition, consuming, processing and committing should be as quick as possible hence I want to use commit_async(), instead of synchronous commit(). But I’m not able to find a good example
Tag: kafka-consumer-api
Python-Kafka: Keep polling topic infinitely
I am using python-kafka to listen to a kafka topic and use that the records. I want to keep it polling infinitely without any exit. This is my code below: This code just reads the data and exits directly. Is there a way to keep listening to topics even if message is not pushed to it? Any relevant example where