jac-scale microservice: architecture hardening and post-merge cleanup
#5,750 opened on Apr 29, 2026
Repository metrics
- Stars
- (555 stars)
- PR merge metrics
- (PR metrics pending)
Description
Follow-up cleanup and architecture-hardening items identified during review of the scale-micro-service branch / PR #5483.
The branch is directionally right: sv import becomes the service boundary, jaclang exposes additive hooks (ensure_sv_service, sv_service_call, get_sv_registry), and jac-scale layers registry/deployer/process-manager/gateway concerns instead of hard-coding everything into one path. That is the architecture we should keep.
The remaining concern is that some implementation details are still local-dev/prototype shaped. If they calcify, they will make K8s, async/queue-backed transports, and larger projects harder than they need to be.
Related context:
- #5483 - current PR
- #5557 - microservice follow-up tracker
- #5139 - original
sv importmicroservice proposal, including async/queue-backed future phases
Architecture Hardening
1. Preserve import-path-aware service identity
Where:
jac-scale/jac_scale/microservices/setup.jacjac-scale/jac_scale/microservices/orchestrator.jacjac-scale/jac_scale/plugin.jacensure_sv_service
Problem: setup.jac derives service identity with Path(file_path).stem, so services/cart.jac becomes cart. Later, the orchestrator and ensure_sv_service reconstruct the file as {module_name}.jac at the project root. This breaks nested service files, dotted/module package layouts, and any future import-path-based URL assignment.
Action: Store and route by canonical module/import path plus source file path. Reuse Jac import resolution where possible instead of rebuilding paths from file stems. The registry entry should have enough information to launch the service without assuming it lives at project root.
Why now: #5557 already hints at "auto url assignment using import path". This is the foundation for that and for K8s service naming.
2. Persist local service state beyond the current CLI process
Where:
jac-scale/jac_scale/microservices/impl/process_manager.impl.jacjac-scale/jac_scale/microservices/impl/local_deployer.impl.jac
Problem: Local lifecycle state is split between in-memory pm.processes and pid-only files under .jac/run. A fresh jac scale status invocation has an empty pm.processes, so it cannot accurately report services started by a previous jac start. Pidfiles alone do not preserve module name, file path, port, URL, command, cwd, or last health state.
Action: Introduce a small persisted local service-state registry, for example .jac/run/services.json, containing service name/import path, source file, pid, port, URL, command, cwd, started_at, and last health result. status, stop, restart, logs, and destroy should use that persisted state and validate liveness/health at read time.
Why now: The PR advertises jac scale status/stop/restart/logs. Those commands need to work across separate CLI invocations to be reliable developer tooling.
3. Replace Python hash() port selection with stable allocation
Where:
jac-scale/jac_scale/microservices/_util.jacpick_free_port- core default
ensure_sv_serviceif it mirrors the same strategy
Problem: Python's hash() is randomized per process unless PYTHONHASHSEED is fixed. That means 18000 + hash(name) % 1000 is not stable across CLI invocations. A process may start a service on one port, then the next CLI invocation computes a different port for the same service.
Action: Use a stable hash (hashlib.sha256(name).digest()) or allocate once and persist the chosen port in the service-state registry. Prefer persisted allocation so collisions and user overrides are explicit.
4. Make gateway route ownership explicit instead of probing every service
Where:
jac-scale/jac_scale/microservices/impl/gateway.impl.jachandle_builtin_passthrough
Problem: Built-in passthrough routes (/user, /walker, /function, etc.) are resolved by trying each healthy service until one returns non-404/405. This is workable for a small local demo, but it creates ambiguous ownership when multiple services expose the same function/walker and adds O(N) latency to common paths.
Action: Build an endpoint ownership map from service introspection, OpenAPI aggregation, or compile/runtime metadata. The gateway should know which service owns /function/foo or /walker/Bar instead of discovering by trial request.
Why now: This is important for predictable production behavior and better error messages. It also reduces pressure on downstream services.
5. Hide sv_client module globals behind a real registry/resolver API
Where:
jac/jaclang/runtimelib/sv_client.jacjac-scale/jac_scale/plugin.jacjac-scale/jac_scale/microservices/impl/process_manager.impl.jac
Problem: The hookspec direction is good, but implementation code still reaches into _registry, _test_clients, and _consumer_providers. ensure_all() also parallelizes resolution while these registries are plain dicts. This makes thread-safety and future resolver replacement more fragile than the public hooks imply.
Action: Add explicit functions for lookup, snapshot, register, unregister, test-client lookup, provider declaration, and provider listing. Internally guard mutable registry state with a lock where concurrent resolution can touch it. Keep direct global access test-only or remove it entirely.
6. Fix config/schema drift before adding more knobs
Where:
jac-scale/jac_scale/plugin_config.jacjac-scale/jac_scale/impl/config_loader.impl.jacjac-scale/jac_scale/microservices/orchestrator.jac
Problem: The gateway/orchestrator reads some config that normal config loading does not return. One example is client.static_mounts: the orchestrator handles it, but get_microservices_config() currently returns only entry, dist_dir, and base_route. This silently drops real user config.
Action: Add tests that round-trip every documented [plugins.scale.microservices] option through plugin_config and get_microservices_config(). If an option is documented or consumed, it should be preserved by the loader.
7. Keep async/queue-backed sv import in the design envelope
Where:
jaclang.runtimelib.sv_clientJacAPIServer.sv_service_callhookspec/default implementationjac-scaletransport override
Problem: #5139 explicitly calls for future sv import async with Redis queues/streams for long-running AI work. The current branch is sync HTTP, which is fine as Phase 1, but the APIs should not assume all service calls are request/response HTTP forever.
Action: Document the transport boundary and avoid baking HTTP-only assumptions into compiler metadata, service registry naming, or gateway ownership. Consider a future-facing ServiceResolver / ServiceTransport split: resolver finds the service, transport decides HTTP vs queue/stream.
Cleanup / Deslop
8. Move the circuit breaker out of plugin.jac
Where: jac-scale/jac_scale/plugin.jac lines around the _breaker_* helpers.
Problem: plugin.jac is now a kitchen-sink entrypoint. The circuit breaker is a self-contained subsystem with module-global state and several helpers, while its peers already live in microservices/_*.jac files.
Action: Extract to jac-scale/jac_scale/microservices/_breaker.jac. plugin.jac's sv_service_call hookimpl should import from there. No behavior change.
9. Pick one HTTP client for the feature, or document the split
Where:
microservices/impl/http_forward.jacusesaiohttpplugin.jacsv_service_calluseshttpxmicroservices/_openapi_agg.jacusesurllib
Problem: Three HTTP clients means three timeout, retry, TLS, proxy, streaming, and dependency surfaces to maintain.
Action: Prefer consolidation around httpx if practical since it supports sync, async, and streaming. If not, add top-level rationale comments explaining why each client is used.
10. Stop _openapi_agg.jac from blocking the event loop
Where: jac-scale/jac_scale/microservices/_openapi_agg.jac
Problem: It performs sequential blocking urllib.request.urlopen calls inside an async function. A single hung downstream blocks the gateway event loop for the timeout window.
Action: Use asyncio.to_thread as a low-risk fix, or switch to httpx.AsyncClient plus parallel fetch with asyncio.gather.
11. Make ForwardResult resource ownership structural
Where: jac-scale/jac_scale/microservices/impl/http_forward.jac
Problem: The current contract says callers must call either to_response() or cleanup(). That is easy to violate during future branching/refactors and can leak outbound connections.
Action: Wrap forwarding in an async context manager or otherwise encode cleanup into the API shape, not just a docstring.
12. Audit _drain.jac against uvicorn/ASGI lifecycle support
Where:
jac-scale/jac_scale/microservices/_drain.jacjac-scale/jac_scale/tests/test_drain.jac
Problem: Uvicorn already has signal handling, graceful shutdown, and ASGI lifespan hooks. The custom drain layer may be justified, especially for in-flight proxied streams/WebSockets and explicit 503 behavior, but that rationale is not written down.
Action: Either simplify by leaning on uvicorn/ASGI hooks, or add a top-level comment explaining exactly which gaps require custom drain behavior.
13. Clarify setup.jac's relationship to project/template infrastructure
Where:
jac-scale/jac_scale/microservices/setup.jacjac/jaclang/project/template_loader.jacjac/jaclang/project/template_registry.jacjac-client/jac_client/templates/
Problem: The original concern was that setup.jac may duplicate jac-client template generation. After spot-checking, setup.jac is more of an interactive config mutator than a scaffold generator. The real smell is ad hoc TOML/string editing and project scanning living outside the project/config/template infrastructure.
Action: Decide and document the boundary:
- If
jac setup microserviceremains a config mutator, move toward structured TOML/project config updates and robust path handling. - If it grows scaffolding behavior, reuse
ProjectTemplate/ template loader machinery rather than inventing a second template system.
Out Of Scope / Fine As-Is
plugin.jachaving two top-level classes (JacCmdandJacScalePlugin) is the plugin-entry shape.gateway.impl.jacis large, but it is split into named handlers and is acceptable after the route-ownership issue is addressed.get_sv_registryis the right public hook direction.- WebSocket catch-all routing is expected for a gateway, though ownership and drain behavior should remain tested.
cc @MusabMahmoodh - none of this rejects the branch direction. This is the set of things that should be cleaned up before the microservice subsystem becomes the long-term foundation.