Down-device notifications re-sent every loop: timezone offset applied to UTC eveDateTime in clearPendingEmailFlag() cutoff
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Read clearPendingEmailFlag() in server/models/notification_instance.py and compare its cutoff with the down_devices query in server/messaging/notification_sections.py. Remove the timezone adjustment so both use the same UTC cutoff, then check that a pending down event is cleared after its alert delay and is not sent again.
Written by the indexing model from the issue text.
Description
What installation are you running?
Dev (netalertx-dev) 👩💻
Is there an existing issue for this?
- I have searched the existing open and closed issues and I checked the docs https://docs.netalertx.com/
The issue occurs in the following browsers. Select at least 2.
- N/A - This is an issue with the backend
Current Behavior
With TIMEZONE='America/Los_Angeles' (UTC-7 right now) and NTFPRCS_alert_down_time=5, every "Down devices" alert is published 3–5 times, about once per main loop (~1 min apart). The repeats stop only when the next scan's skip_repeated_notifications() happens to clear the flag.
What I saw on one instance (version dev-20260704-150156, Pushover publisher, devSkipRepeated=0 on all devices):
- One
Device Downevent (a single row inEvents) produced 4 separateNotificationsrows with different GUIDs, created at +5:40, +6:02, +7:07 and +8:14 after the event. Each one was delivered by Pushover. - Every one of those cycles logged
[Notification] Notifications changes: 0, which meansclearPendingEmailFlag()didn't clear the pending flag on the down event. - Since March, 103 distinct alerts went out more than once, and every one of them was a
down_devices-only payload.
Expected Behavior
Once a down alert has been sent, evePendingAlertEmail should be cleared for that event, and the alert should go out exactly once.
Root cause
eveDateTime is stored in UTC. The down_devices section selects events with a plain UTC cutoff (server/messaging/notification_sections.py):
AND eveDateTime < datetime('now', '-{alert_down_minutes} minutes')
But clearPendingEmailFlag() also applies the timezone offset when it clears the same events (server/models/notification_instance.py L253–L264 @ fe29f5d4 (current main)):
minutes = int(get_setting_value("NTFPRCS_alert_down_time") or 0)
tz_offset = get_timezone_offset()
self.db.sql.execute(
"""
UPDATE Events
SET evePendingAlertEmail = 0
WHERE evePendingAlertEmail = 1
AND eveEventType = 'Device Down'
AND eveDateTime < datetime('now', ?, ?)
""",
(f"-{minutes} minutes", tz_offset),
)
With tz_offset = '-07:00', the clear cutoff becomes now - 7h05m, while the select cutoff is now - 5m:
datetime('now') -> 2026-10-19 06:56:07
datetime('now','-5 minutes') -> 2026-10-19 06:51:07 (select cutoff)
datetime('now','-5 minutes','-07:00') -> 2026-10-18 23:51:07 (clear cutoff)
So any down event between 5 minutes and 7 hours old is selected and sent again on every loop, but never cleared. In timezones east of UTC the error goes the other way: the clear cutoff lands in the future. Then any pending down event that is still inside its alert_down_time delay gets cleared whenever some other notification goes out, so that down alert is never sent.
This is the same class of UTC/localtime mismatch as #1737, which fixed skip_repeated_notifications() in 2dc2207228. clearPendingEmailFlag() still has it on current main (fe29f5d4).
Proposed fix
--- a/server/models/notification_instance.py
+++ b/server/models/notification_instance.py
@@ clearPendingEmailFlag
SET evePendingAlertEmail = 0
WHERE evePendingAlertEmail = 1
AND eveEventType = 'Device Down'
- AND eveDateTime < datetime('now', ?, ?)
+ AND eveDateTime < datetime('now', ?)
""",
- (f"-{minutes} minutes", tz_offset),
+ (f"-{minutes} minutes",),
)
(After this change tz_offset and the get_timezone_offset import may no longer be needed.)
I'm running this patch, plus the #1737 change backported to my older image, via read-only bind mounts. After the change, the clear cutoff matches the select cutoff (now - 5m).
Steps To Reproduce
- Set
TIMEZONEto any non-UTC zone west of UTC (e.g.America/Los_Angeles), withNTFPRCS_alert_down_time=5,devAlertDown=1,devSkipRepeated=0, and any publisher enabled. - Take a device offline so a
Device Downevent is created. - After about 5 minutes, a down alert is sent, and then sent again on every main loop until the next scan cycle. Each cycle logs
Notifications changes: 0, and each one adds a new row toNotifications.
Relevant app.conf settings
TIMEZONE = 'America/Los_Angeles'
NTFPRCS_INCLUDED_SECTIONS = ['new_devices','down_devices','events']
NTFPRCS_alert_down_time = 5
PUSHOVER_RUN = 'on_notification'
ARPSCAN_RUN_SCHD = '*/5 * * * *'
docker-compose.yml
services:
netalertx:
container_name: netalertx
image: jokobsk/netalertx-dev:latest # dev-20260704-150156
network_mode: host
restart: unless-stopped
read_only: true
volumes:
- /path/to/netalertx:/data
- /etc/localtime:/etc/localtime:ro
Debug or Trace enabled
- I have read and followed the steps in the wiki link above and provided the required debug logs and the log section covers the time when the issue occurs.
Relevant app.log section
21:31:07 [Scan Stats] New Down Alerts........: 1
21:36:47 [Notification] Check if something to report
21:36:47 [Notification] down_devices section done.
21:36:49 [Plugins] SUCCESS for PUSHOVER received 1 entries
21:36:51 [Notification] Notifications changes: 0
21:37:09 [Notification] Check if something to report
21:37:09 [Notification] down_devices section done.
21:37:11 [Plugins] SUCCESS for PUSHOVER received 1 entries
21:37:19 [Notification] Notifications changes: 0
21:38:14 [Notification] down_devices section done.
...
21:39:21 [Notification] down_devices section done.
21:39:25 [Notification] Notifications changes: 0
21:41:14 [Skip Repeated Notifications] Skip Repeated <- flag finally cleared here
21:41:21 [Notification] No changes to report
Docker Logs
N/A, nothing relevant; the container is healthy. The behavior is fully visible in app.log and the Notifications table.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 456
- Avg merge
- 17h 59m
- Merged PRs (30d)
- 26
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from netalertx/NetAlertX
-
bug 🐛
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
netalertx/NetAlertX#1856 · 1 comment · 7 reactions ·
Maintainers usually reply within 1 day
-
[Bug] MCP SSE endpoint event omits the /server prefix, so clients behind the built-in nginx proxy get a 404 on /mcp/messagesPossibly taken @frdmn claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Docker 🐋
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
[Bug] Internet-Check returns 0.0.0.0 when IPv6 is enabled and OpenDNS lookup failsPossibly taken @pujitha24 claimed this 6 days ago. Openbug 🐛 next release/in dev image 🚀 Waiting for reply⏳
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
netalertx/NetAlertX#1845 · 2 comments ·
Maintainers usually reply within 1 day
-
bug 🐛 next release/in dev image 🚀
Difficulty 2/5 1-3 hours Newbie friendliness 63/100
netalertx/NetAlertX#1841 · 3 comments ·
Maintainers usually reply within 1 day
All issues in netalertx/NetAlertX
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
khuisman/mcp-gee-sweet#1166 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
lugassawan/swe-workbench#823 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
gramaziokohler/compas_timber#870 ·
-
security-scan-failure
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
semantica-agi/semantica#2089 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
autowarefoundation/autoware_carla_scenario#500 ·
Maintainers usually reply within 1 day