Server process keeps running after a startup failure

Start with Mirth.run(), Mirth.startup(), and the referenced lines in Mirth.java, then inspect how MirthLauncher waits for the run thread and how EventController owns the consumer threads. Reproduce the occupied-port case with ./oieserver and the migration failure using the provided Derby steps; also compare the shutdown-hook behavior affected by #499. Done means either startup failure terminates the process with a non-zero status, no consumer thread keeps the JVM alive, and failed-migration shutdown does not hang.

Summary

When the server fails to start, it logs the error and stops starting up, but the JVM does not exit. The process stays up with no listening ports, so a service manager sees it as running and nothing restarts or alerts.

This happens with ordinary failures, not just unusual ones. Two confirmed triggers:

• A port already in use, for example from a second instance or another service on 8443.
• A failed database migration (MigrationException).

Steps to reproduce

Run these from the server's install directory.

A port already in use

1. Occupy the HTTPS port, for example in another terminal:

   nc -l 8443
   

   (Traditional netcat needs nc -l -p 8443.)
2. Start the server:

   ./oieserver
   

A failed migration

With the default Derby database:

1. Start a fresh install once so the database is created, then stop it (Ctrl+C once startup has finished):

   ./oieserver
   

2. With the server stopped, set a schema version this release doesn't know:

   java -cp 'server-lib/database/*' org.apache.derby.tools.ij <<'EOF'
   connect 'jdbc:derby:appdata/mirthdb';
   UPDATE SCHEMA_INFO SET VERSION = '99.0.0';
   exit;
   EOF
   

3. Start the server again:

   ./oieserver
   

The migration fails with or without the fix for #496 (unrecognized schema version). Only the logged error differs.

Expected

The process exits with a non-zero status after logging the failure.

Actual

Port in use: the log shows https.port port is already in use: 8443, then nothing. The Java process keeps running with no listening ports. SIGTERM stops it within a couple of seconds, because the shutdown hook was never registered.

Failed migration: reproduced with Derby and with PostgreSQL 16. The log shows Failed to migrate database schema (and on Derby, Database shut down normally.), then nothing. The Java process keeps running with no listening ports.

In both cases a thread dump shows the same two non-daemon threads keeping the JVM alive:

• AuditableEventListener Consumer Thread
• DefaultAlertWorker Consumer Thread

Cause

Production starts the server through MirthLauncher, which runs Mirth as a thread, so the process only exits once every non-daemon thread has ended. Each EventListener starts its consumer thread in its constructor, when Mirth creates its EventController, and nothing stops those threads on a failed startup.

Both failures end Mirth.run() without stopping them:

• Port in use: the port check returns from Mirth.run() before startup() is called and before the shutdown hook is registered (Mirth.java:148-154).
• Failed migration: Mirth.startup() sets running = false and returns (Mirth.java:320-326), so Mirth.run() ends.

The System.exit(0) in Mirth.main() never runs on these paths, because main() is only used by devRun. When the server is started that way, main() calls System.exit(0) once run() returns, so the status is 0. After a failed migration, that exit also runs the shutdown hook, which on Derby should hang as described in #499. This path was not tested.

Suggested fix

On a startup failure, end the process with a non-zero status. One option is for Mirth.run() to record the failure and for the launcher, or Mirth itself, to call System.exit(1). The early System.exit(0) on a failed database connection (Mirth.java:303) should use a non-zero status too. The shutdown hook is already registered by then (Mirth.java:162), so that exit also runs it.

For failures inside startup(), such as a failed migration, this has to be fixed together with #499. System.exit runs the JVM shutdown hook, and after a migration failure on Derby that hook hangs, so on its own a System.exit(1) would leave the process stuck in the hook instead of idle. The port check runs before the hook is registered, so that case doesn't depend on the shutdown-hook fix.

---

🤖 Drafted with Claude Code