When monitoring /var/log/journal/remote events are missed when two remote hosts send messages at the same time (updates)
還沒有人認領這個 Issue。
評估
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 新手友好度
- 35/100
- Issue 類型
- 缺陷
- 描述清晰度
- 基本清楚
- 活躍度
- 停滯
- 技術堆疊
- linux, python
研究方向
Start with the journal.Reader polling example, especially seek_tail(), get_previous(), get_events(), iteration, and process(). Reproduce simultaneous writes from two clients and compare the event counter with journalctl -D /var/log/journal/remote/; done means explaining or correcting the missed-event behavior and confirming that all records are observed.
由索引模型根據 Issue 內容生成。
描述
Context
- 1 x log server (VM on Apple M2)
- 2 x log client (VM on Apple M2)
- Using systemd-journal-remote on server (http)
- Using systemd-journal-upload on clients (http)
Testing Method
We first cleanup the server with:
rm /var/log/journal/remote/*
On each client we run (side by side / simultaneously:
for x in {1..10000}; do echo "EVENT $x" | logger ; done
So we expect 20,000 records on the server, which we validate:
root@log-server:/home/ansible# journalctl -D /var/log/journal/remote/ --no-pager | grep -v Boot | wc -l
20000
I've tested this many times to confirm that the systemd-journal-remote/upload mechanism is OK.
The failing code
import select
from systemd import journal
j = journal.Reader(path="/var/log/journal/remote")
j.seek_tail()
j.get_previous() # prevent wrap around
p = select.poll()
p.register(j, j.get_events())
print("Starting....")
event_counter = 0
while p.poll():
for x in j:
event_counter += 1
if j.process() != journal.APPEND:
print("...WAITING....")
else:
print(f"Event counter: {event_counter}")
First of all, I'm not sure if this code is right. If I've made a glaring mistake, I apologise in advance for this report.
That said, I get correct results if I run my '10K log generator command' on one client at a time.
Event counter: 9915
Event counter: 9933
Event counter: 9945
Event counter: 9950
Event counter: 9987
Event counter: 10000
If I run the command at the same time on each client at once, I would expect 20K records, but it gets nowhere near that:
Event counter: 10228
Event counter: 10228
Event counter: 10228
Event counter: 10228
Event counter: 10228
Event counter: 10228
Event counter: 10228
In this example I'm missing almost half of all expected events. Yet I can confirm on the log server with the 'journalctl' command that there are actually 20K log records stored on the server.
Even if I test with just 1K events per client and run that simultaneously it's also clearly visible:
Event counter: 1096
Event counter: 1141
Event counter: 1215
Event counter: 1430
Happy to run any additional tests if that helps.
If I made a huge mistake, please let me know.
Update 1
I've been testing this issue on virtual machines on my mac.
I've also a few physical x86 micro pc's and the issue is the same.
Rate-limiting the logging with sleep .01 doesn't impact the result.
Question, is python-systemd built to handle remote logging from multiple clients through polling?
- 主要語言
- C
- 星號
- 524
- 分支
- 79
- PR 合併指標
- 30 天內沒有已合併 PR
環境準備
這個專案沒有提供開發容器、Dockerfile 或貢獻指南，環境需要你自己搭建：先看它的 README，通用步驟見我們的新手貢獻指南。
從這裡開始
- 先讀完整個 Issue，再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫，在一個分支上完成修改。
- 送出 Pull Request，並在描述裡引用這個 Issue 編號。
systemd/python-systemd 的其他 Issue
-
難度 3/5 1-2 天 新手友好度 48/100
systemd/python-systemd#177 · 1 則留言 · 1 個 reaction ·
-
難度 3/5 1-2 天 新手友好度 42/100
systemd/python-systemd#169 · 1 則留言 ·
-
難度 4/5 3-5 天 新手友好度 48/100
systemd/python-systemd#167 · 19 則留言 ·
-
難度 5/5 一週以上 新手友好度 35/100
systemd/python-systemd#152 · 1 則留言 · 14 個 reaction ·
-
難度 4/5 3-5 天 新手友好度 45/100
systemd/python-systemd#146 · 2 則留言 ·
查看 systemd/python-systemd 的全部 Issue
相似的 Issue
-
bug
難度 2/5 1-3 小時 新手友好度 68/100
-
難度 2/5 1-3 小時 新手友好度 68/100
RPCS3/rpcs3#19705 · 1 個 reaction ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 88/100
維護者通常 1 天內回覆
-
bug llvm
難度 2/5 1-3 小時 新手友好度 88/100
維護者通常 1 天內回覆
-
難度 1/5 1 小時以內 新手友好度 88/100
containers/bubblewrap#813 · 1 個 reaction ·
維護者通常 1 天內回覆