confluentinc/confluent-kafka-python

Specifying a Python list for config `bootstrap.servers` fails silently with no errors or indications

Open

#711 opened on Oct 28, 2019

View on GitHub
 (5 comments) (2 reactions) (0 assignees)Python (3,526 stars) (909 forks)batch import
code:pythonenhancementgood first issueinvestigate furthersize:smallstatus:planned

Description

Description

The configuration key bootstrap.servers implies that multiple servers can be specified, but it is not explicitly defined what this key's value should be. It appears that only a string using comma separation is allowed for this config key. If a Python list is given, no explicit error or feedback is given to indicate this is not valid. Instead, calls to consumer.poll() or other broker-bound functions simply block and never succeed.

Ideally, a Python list (e.g. ["broker1", "broker2"]) should be valid; or at the very least, constructing the Consumer should throw a runtime/configuration exception.

How to reproduce

Simply specify a Python list as the value for bootstrap.servers in the config given to the Consumer.

conf = {
    'bootstrap.servers': ['kafkabroker1:9092', 'kafkabroker2:9092'],
    ...
}
c = Consumer(conf)
while True:
    c.poll(1) # this will never produce a message due to the Python list specified above as the bootstrap.servers value

Checklist

Please provide the following information:

  • confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):
>>> confluent_kafka.version()
('1.2.0', 16908288)
>>> confluent_kafka.libversion()
('1.2.0', 16908543)
  • Apache Kafka broker version:
  • Client configuration: {...}
  • Operating system: Ubuntu 18.04 and CentOS 7
  • Provide client logs (with 'debug': '..' as necessary)
  • Provide broker log excerpts
  • Critical issue

Contributor guide