Tag: kafka-consumer-api

How to perform async commit when using kafka-python

apache-kafka kafka-consumer-api kafka-python python

I’m using kafka-python library for my fastapi consumer app and I’m consuming messages in batch with maximum of 100 records. Since the topic has huge traffic and have only one partition, consuming, processing and committing should be as quick as possible hence I want to use commit_async(), instead …

Python-Kafka: Keep polling topic infinitely

apache-kafka kafka-consumer-api kafka-python kafka-topic python

I am using python-kafka to listen to a kafka topic and use that the records. I want to keep it polling infinitely without any exit. This is my code below: This code just reads the data and exits directly. Is there a way to keep listening to topics even if message is not pushed to it? Any relevant example wher…