Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

feat(ops): phase 3 verify service health resource budgets and deployment observability

Đang mở
  • Có gì không ổn?
#663 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
25/100
Loại issue
Tính năng
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
docker-compose, grafana, prometheus
Lĩnh vực
devops, observability

Hướng nghiên cứu

Read docs/platform/README.md, docs/platform/agent-execution.md, and the referenced current-state.md, validation.md, and migration-runbook.md contracts first; check that prerequisites #653, #658, #660, and #661 are merged. Then inspect the listed Prometheus, OpenTelemetry, Grafana, and Compose files and run the required platform smoke and observability commands for the target environment. Done means all acceptance checks have evidence, including readiness failures, telemetry and alert behavior, measured resource limits, and documented operator recovery.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Work ID: P3-03 · Phase 3: Delivery and recovery

Parent: #652 · Documentation PR: #651

Goal and current context

Several services have no container healthcheck, APISIX's current check only looks for a PID file, and cross-stack telemetry targets assume Swarm names. Platform deployment status alone cannot establish service readiness.

Source of truth

Read the implementation plan and agent execution contract before editing. Task-specific contracts:

These are immutable planning references. After the documentation PR merges, maintain the corresponding repository paths as the current canonical contract. Do not silently override an ADR; update it and affected issue criteria together.

Prerequisites

  • P0-01: #653 has merged implementation and applicable passing evidence.
  • P2-02: #658 has merged implementation and applicable passing evidence.
  • P2-04: #660 has merged implementation and applicable passing evidence.
  • P3-01: #661 has merged implementation and applicable passing evidence.

Existing source entry points

Baseline commit: e80a48a8b8d7d4d9732840b618e73a2f4b58a1c3. Confirm newer dev changes before implementation; repository evidence is not proof of live state.

Implementation scope and deliverables

  1. Define real readiness/liveness and black-box smoke contracts for every enabled deployment unit. Verify actual upstream application endpoints from their repository/image; do not invent /health routes or require curl in images that lack it.

  2. Update service DNS, scrape targets and Grafana datasource/provisioning mounts for Dokploy's networks. Prove a request's trace/log/metric correlation where supported and investigate the APISIX global-rule declaration against actual runtime configuration.

  3. Set and validate effective memory/CPU/log/disk/retention limits from measured baseline and target capacity, including Dokploy overhead and build avoidance on the production host. Record headroom and architecture compatibility.

  4. Add alerts for public endpoint failure, deployment failure, data volume space, OOM/restart loops, certificate expiry and backup age/failed restore drills. Put at least one availability check outside the target host's failure domain.

  5. Record deployment IDs, release digests and environment in observability without secrets. Document UI actions for status/logs/redeploy and a break-glass path for controller or VPN outage.

  6. Exercise a failed readiness check, stopped app, controller outage, missing backup and telemetry interruption. Establish an operator runbook and bounded thresholds from test evidence.

Validation and acceptance

  • Healthy smoke requests succeed and intentionally unhealthy services fail the gate; a running process or container count alone is insufficient.
  • Telemetry survives redeploy and uses valid runtime names; dashboards are private.
  • Alert tests demonstrate external detection when the host/controller is unavailable, with a documented notification destination.
  • Measured load/latency/resource use and chosen limits are recorded; the full target fits without relying on unverified memory assumptions.
  • Routine inspect, logs, compatible rollback and redeploy are possible through the documented workflow without an interactive host shell.

Required command interfaces to implement/use, as specified in validation.md; these did not exist at planning baseline:

make platform-smoke ENV=lab TIER=vm
make platform-observability-test ENV=lab TIER=vm

Run the actual assertions appropriate to this scope. Missing infrastructure, credentials or a daemon means blocked/skipped evidence, not a passed gate. Existing repository lint, semantic stack-drift and relevant risk-policy checks must continue to pass for changed paths.

Completion evidence

Attach the implementation PR and tested commit, exact commands/results, environment/tier, versions/digests/architecture, meaningful positive/negative assertions, cleanup/rollback results and remaining blockers. Required acceptance must be complete and merged through repository policy before closing. Keep public evidence sanitized; never upload secret values, raw inspect/env dumps, database contents or private keys.

Rollback and safety boundary

Restore previous tested config/provisioning and limits. Do not make metrics/admin endpoints public to debug ingestion.

Issue creation itself does not authorize cloud spend, production mutation or destruction. Use existing execution authorization and the plan's concrete environment gates; complete preparatory work before requesting any genuinely missing final approval.

Downstream handoff

P4-01: #664

Primary technical references

Ngôn ngữ chính
Python
Star
0
Fork
0
Merge trung bình
21 giờ 11 phút
Pull request đã merge (30 ngày)
19

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của AniTrend/local-stack

Tất cả issue của AniTrend/local-stack

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.