Silent fail / exit with streamz >= 0.4.0 when pipeline is being run in a daemon process
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 25/100
- Issue 类型
- 缺陷
- 描述清晰度
- 需要澄清
- 活跃度
- 停滞
- 技术栈
- elasticsearch, kafka, python
调研方向
Reproduce the pipeline under python-daemon while comparing streamz 0.3.0 and 0.4.0, using Kafka input and Elasticsearch output. Investigate whether the Tornado processes introduced in later versions cause the daemon to stop after its first message. Done means the daemon continues processing messages and exposes an error if processing fails.
由索引模型根据 Issue 内容生成。
描述
I've been using streamz to consume events from kafka. For that I built a pipeline which I run in a daemon process, managed by the python daemon package (https://pypi.org/project/python-daemon/). The pipeline ultimately writes to some elasticsearch instance.
This worked great with streamz version 0.3.0 with which I started developing.
Now I wanted to update to v0.5.0 and discovered that while I can still perfectly run the streamz pipeline in the current python process, the daemon running the pipeline suddenly stops without any further detectable error in the pipeline itself after consuming the first message. I downgraded to v0.4.0, then v0.3.0 and observed that the fail is introduced by the 0.3.0 -> 0.4.0 step.
Does anybody has an idea what introduced the problem at hand?
Is there an easy way to fix the issue with the later versions?
.. I somehow suspect that it has something to do with the tornado processes started ..?
- 主要语言
- Python
- 星标
- 1.3k
- 派生
- 151
- 平均合并
- 17 小时 39 分钟
- 30 天内合并 PR
- 1
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
python-streamz/streamz 的其他 Issue
-
难度 3/5 1-2 天 新手友好度 35/100
python-streamz/streamz#481 · 4 条评论 ·
-
难度 5/5 一周以上 新手友好度 25/100
python-streamz/streamz#480 · 2 条评论 ·
-
难度 5/5 一周以上 新手友好度 20/100
python-streamz/streamz#479 · 1 条评论 ·
-
难度 5/5 一周以上 新手友好度 20/100
python-streamz/streamz#478 · 6 条评论 ·
-
难度 4/5 3-5 天 新手友好度 10/100
python-streamz/streamz#476 · 17 条评论 · 2 个 reaction ·
查看 python-streamz/streamz 的全部 Issue
相似的 Issue
-
New Internship未关闭new_internship
难度 1/5 1 小时以内 新手友好度 70/100
-
[BUG] Reports tab: "Unban" button tooltip shows raw `{{ip}}` placeholder instead of the IP address未关闭bug javascript ui
难度 2/5 1-3 小时 新手友好度 68/100
bunkerity/bunkerweb#4001 · 1 条评论 ·
维护者通常 1 天内回复
-
bug
难度 1/5 1 小时以内 新手友好度 92/100
PedestrianDynamics/pyFDS-Evac#476 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
google/differential-privacy#516 ·
-
难度 2/5 1-3 小时 新手友好度 82/100
adobe-fonts/source-serif#153 ·