Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

PortalNextProcess supervisor killed the /next Next.js server on one memex portal pod after its health probe on a hashed static-chunk URL went unanswered

Closed
  • Is something wrong?
#7,080 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
csharp, next.js
Domain
backend, observability-sre

Research direction

Read PortalNextProcess in the MeshWeaver.Plugins portal hosts (Memex.Portal.Distributed) to inspect probe URL selection, failure details, and the kill/restart logic. Check the pod logs around the incident for the child process output, then compare the probed chunk URL with a known-good /next path from inside the pod. Done means identifying which failure mode fits and whether the probe should use a stable endpoint; the source is not readable from this mesh, so locating it may require repository access.

Written by the indexing model from the issue text.

Description

bug sev:M

Observed

The Plugins PortalNextProcess component (log category Memex.Portal.Distributed.PortalNextProcess) supervises the Next.js /next front end inside the portal process: it unpacks /app/portal-next.zip from the memex-portal-ai image and serves it on a local next port (see /Doc/Architecture/ContinuousDeliveryContract, policy portal-next-in-platform-image). On pod memex-portal-deployment-8556887fd9-hwx6z (namespace memex) it reported that the /next server did not answer its health probe several consecutive times and killed the child process, restarting it as a crash would be — i.e. the supervisor's own designed response fired.

The probed URL was http://127.0.0.1:3000/next/_next/static/chunks/1168.14d04b3e2c337450.js — a content-hashed Next.js build artifact, not a stable health endpoint.

Where and when

This is the first sighting of this fingerprint — one pod, one line, firstSeen == lastSeen, with no folded shapes and no prior incident or issue at this log site anywhere in the mesh. It is a single kill-and-restart event, not (yet) a restart loop.

Impact

Bounded and self-healing so far: the /next front end on that one pod was down for the probe-to-restart window and is served by the restarted child afterwards. Whether any user saw the window, and whether the memex portal's primary UI was affected at all, is not established from this incident.

Not established

  • Why the probe went unanswered. The message says only "did not answer"; the incident carries no exception and no stack, and that wording does not distinguish connection-refused (child not listening) from a timeout (child hung) from a non-2xx answer (URL answered but with an error status).
  • Whether the restarted child came up healthy and whether the event recurred after the restart.
  • Whether it coincided with a pod start or a portal image roll (per the delivery contract, every published set restarts the installs that take it).
  • Whether other pods or instances show the same fingerprint — the watcher has seen it on this one pod only.
  • The PortalNextProcess source itself is not readable from this mesh; it lives in the MeshWeaver.Plugins portal hosts (Memex.Portal.Distributed).

Candidate causes and the reading that separates them

  1. Child process not listening — the Next.js server crashed at boot or never bound port 3000. Predicts connection-refused probe failures, Next.js stderr near the same timestamp, and recurrence as a restart loop. Separating reading: the pod's log around the incident for the child's own output, and whether this fingerprint recurs on the same pod after the restart.
  2. Child hung or overloaded — port bound but requests time out. Predicts the same fingerprint recurring mid-life (not only after start), with no crash output from the child. Separating reading: the per-probe failure detail at a more verbose log level in PortalNextProcess, if one exists.
  3. The probe URL is a content-hashed chunk that can go stale. 1168.14d04b3e2c337450.js is a build-specific artifact; if the probe URL is resolved once (at unpack, or from a manifest) and a non-2xx answer counts as "did not answer", then a re-unpack or a new bundle — e.g. after an image roll — would make a healthy server fail the probe. Predicts: the identical chunk name in every occurrence regardless of restart, and a direct fetch of that exact URL returning non-200 while other /next paths answer 200. Separating reading: from inside the pod, fetch the exact probed URL and a known-good /next path and compare; read how PortalNextProcess chooses the probe URL (fixed at unpack vs. re-resolved per probe).
  4. Startup race — the probe loop starts before the Next server is listening, so consecutive misses land during boot. Predicts occurrences clustered shortly after pod start/unpack and never mid-life. Separating reading: whether occurrence timestamps cluster at pod start times.

Where to look

  • PortalNextProcess in the MeshWeaver.Plugins portal hosts (Memex.Portal.Distributed): the probe loop, the probe URL selection, and the consecutive-miss kill/restart logic.
  • /Doc/Architecture/ContinuousDeliveryContract → "🚂 /next is not a repository of the set" — how the bundle is built into memex-portal-ai, unpacked and served on a next port.
  • Pod memex-portal-deployment-8556887fd9-hwx6z logs around the incident time for the Next.js child's own output and the supervisor's per-probe failure mode.
Evidence
Fingerprint f01a717c24b4ff36
Category Memex.Portal.Distributed.PortalNextProcess
Severity Error
Namespace memex
Pods memex-portal-deployment-8556887fd9-hwx6z
Occurrences 1
First seen 2026-10-23 07:59:58Z
Last seen 2026-10-23 07:59:58Z
Routing not determined — no configured route matches the category Memex.Portal.Distributed.PortalNextProcess. This repository is the configured fallback, not a finding about who owns the fault; the category names the LOGGER, which may not be the subject.
Recent log lines 
2026-10-23 07:59:58Z memex-portal-deployment-8556887fd9-hwx6z fail: Memex.Portal.Distributed.PortalNextProcess[0]
      [PortalNext] the /next server did not answer http://127.0.0.1:3000/next/_next/static/chunks/1168.14d04b3e2c337450.js 4 times in a row — killed; it restarts like a crash

Opened automatically from Admin/_LogIncident/f01a717c24b4ff36. Recurrences are folded into this issue rather than opening new ones.
It also stands for the whole log site 9785d23fef3d0c4b: other fingerprints of this site fold in here as comments rather than opening tickets of their own.

Dominant language
C#
Stars
12
Forks
5
Avg merge
6h 37m
Merged PRs (30d)
929

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Systemorph/MeshWeaver

All issues in Systemorph/MeshWeaver

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.