security: replace ML pipeline filesystem paths with owned job and artifact handles

Summary

Replace the ML pipeline filesystem and shared-output contract with server-owned upload, job, and artifact handles plus a protected sidecar boundary. Owner surfaces: dashboard/backend/handlers/mlpipeline.go, dashboard/backend/mlpipeline/, src/training/model_selection/ml_model_selection/server.py, and deploy/kubernetes/observability/dashboard/deployment.yaml.

Current behavior

The sidecar accepts caller-selected paths, the shipped containers do not share one consistent absolute path namespace, multipart input can grow disk without a total cap, jobs share output state, and downloads can trust returned paths. The sidecar also lacks a workload authentication boundary when broadly bound.

Expected behavior

External APIs exchange opaque owned handles only. One private data root is mounted consistently, all file resolution is containment-checked, jobs have isolated resources and bounded lifecycle, and the sidecar is loopback-only or strongly workload-authenticated.

Acceptance

• Remove arbitrary input, output, cache, and benchmark paths from public requests; use server-owned upload/job/artifact IDs.
• Resolve every path symlink-aware beneath one private root mounted at the same absolute path in both containers.
• Enforce aggregate request, per-file, file-count, and content limits before disk growth; use private random job directories and file names.
• Bound job concurrency, queueing, deadlines, cancellation, output/cache isolation, and retention cleanup.
• Validate all returned artifact handles against the owning completed job before download.
• Bind a co-located sidecar to loopback; require authenticated workload transport and network policy for any remote mode.

Validation

• Add a two-container integration test using the shipped mount layout and restart recovery.
• Add adversarial multipart, containment, symlink, malformed index, sidecar-result, concurrent-job, cancellation, and cleanup tests.
• Run cd dashboard/backend && go test -race ./handlers ./mlpipeline.
• Run the Python sidecar unit suite and affected Kubernetes E2E profile with explicit file, job, and network-boundary assertions.

Related

• Parent audit: #2375
• Security index: #2357
• Dashboard state history: #1609
• Dashboard hotspot history: #1548