Parsely/pykafka

IllegalGeneration exception from SimpleConsumer

Open

#612 aperta il 25 ott 2016

Vedi su GitHub
 (9 commenti) (0 reazioni) (0 assegnatari)Python (231 fork)batch import
bughelp wanted

Metriche repository

Star
 (1114 star)
Metriche merge PR
 (Nessuna PR mergiata in 30 g)

Descrizione

INFO:pykafka.balancedconsumer:Rebalancing consumer "pykafka-88ab9f20-80db-493c-9253-998a14cfb0d6" for topic "my-replicated-topic".
INFO:pykafka.managedbalancedconsumer:Sending JoinGroupRequest for consumer id 'pykafka-88ab9f20-80db-493c-9253-998a14cfb0d6'
INFO:pykafka.balancedconsumer:pykafka-88ab9f20-80db-493c-9253-998a14cfb0d6: Balancing 2 participants for 10 partitions. Owning 5 partitions.
DEBUG:pykafka.balancedconsumer:My partitions: ['my-replicated-topic-0-2', 'my-replicated-topic-0-5', 'my-replicated-topic-0-8', 'my-replicated-topic-1-3', 'my-replicated-topic-1-0']
INFO:pykafka.balancedconsumer:pykafka-88ab9f20-80db-493c-9253-998a14cfb0d6: Balancing 2 participants for 10 partitions. Owning 5 partitions.
DEBUG:pykafka.balancedconsumer:My partitions: ['my-replicated-topic-2-7', 'my-replicated-topic-1-6', 'my-replicated-topic-1-9', 'my-replicated-topic-2-4', 'my-replicated-topic-2-1']
INFO:pykafka.managedbalancedconsumer:Sending SyncGroupRequest for consumer id 'pykafka-88ab9f20-80db-493c-9253-998a14cfb0d6'
DEBUG:pykafka.simpleconsumer:Committing offsets for 10 partitions to broker id 2
INFO:pykafka.simpleconsumer:Continuing in response to IllegalGeneration
ERROR:pykafka.simpleconsumer:Error committing offsets for topic 'my-replicated-topic' from consumer id 'pykafka-88ab9f20-80db-493c-9253-998a14cfb0d6'(errors: {<class 'pykafka.exceptions.IllegalGeneration'>: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]})
DEBUG:pykafka.simpleconsumer:Retrying
DEBUG:pykafka.simpleconsumer:Fetcher thread exiting
INFO:pykafka.simpleconsumer:Continuing in response to IllegalGeneration
ERROR:pykafka.simpleconsumer:Error committing offsets for topic 'my-replicated-topic' from consumer id 'pykafka-88ab9f20-80db-493c-9253-998a14cfb0d6'(errors: {<class 'pykafka.exceptions.IllegalGeneration'>: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]})
DEBUG:pykafka.simpleconsumer:Retrying
INFO:pykafka.simpleconsumer:Continuing in response to IllegalGeneration
ERROR:pykafka.simpleconsumer:Error committing offsets for topic 'my-replicated-topic' from consumer id 'pykafka-88ab9f20-80db-493c-9253-998a14cfb0d6'(errors: {<class 'pykafka.exceptions.IllegalGeneration'>: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]})
DEBUG:pykafka.simpleconsumer:Retrying
INFO:pykafka.simpleconsumer:Continuing in response to IllegalGeneration
ERROR:pykafka.simpleconsumer:Error committing offsets for topic 'my-replicated-topic' from consumer id 'pykafka-88ab9f20-80db-493c-9253-998a14cfb0d6'(errors: {<class 'pykafka.exceptions.IllegalGeneration'>: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]})
DEBUG:pykafka.simpleconsumer:Retrying
INFO:pykafka.simpleconsumer:Continuing in response to IllegalGeneration
ERROR:pykafka.simpleconsumer:Error committing offsets for topic 'my-replicated-topic' from consumer id 'pykafka-88ab9f20-80db-493c-9253-998a14cfb0d6'(errors: {<class 'pykafka.exceptions.IllegalGeneration'>: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]})
INFO:pykafka.cluster:Attempting to discover offset manager for consumer group 'testgroup'
INFO:pykafka.cluster:Found coordinator broker with id 2
DEBUG:pykafka.simpleconsumer:Fetching offsets for 5 partitions from broker id 2
DEBUG:pykafka.simpleconsumer:Set offset for partition 1 to 282963
DEBUG:pykafka.simpleconsumer:Set offset for partition 4 to 283507
DEBUG:pykafka.simpleconsumer:Set offset for partition 9 to 284755
DEBUG:pykafka.simpleconsumer:Set offset for partition 6 to 284410
DEBUG:pykafka.simpleconsumer:Set offset for partition 7 to 282724

I use topic.get_balanced_consumer(consumer_group='testgroup',auto_commit_enable=True,managed=True, consumer_timeout_ms=1500) to get managedbalancedconsumer. this is a self-balancing consumer, but get a error

Guida contributor