Upgrade the Lambda runtime from Python 3.13 to 3.14

Start with Dockerfile and pyproject.toml: inspect the Lambda base image tag and the [tool.mypy]/[tool.black] Python targets. Rebuild the arm64 image on a pinned 3.14...-arm64 tag and verify native deps (confluent-kafka, psycopg2) compile with 3.14, then run unit and integration tests against that image. Check CI/pipeline files for leftover 3.13 pins, and mark it done only when tests pass and the smoke checks for POST /topics/{topic_name} and /health pass.

Description of Technical Debt

The local toolchain and the deployed runtime are on different Python versions:

Where	Version	Source
Local venv / test runs	3.14.7	./.venv/Scripts/python.exe --version
Container image	3.13	Dockerfile: public.ecr.aws/lambda/python:3.13-arm64
mypy target	3.13	pyproject.toml [tool.mypy] python_version
black target	py313	pyproject.toml [tool.black] target-version

Every local pytest, mypy, pylint, and black run therefore executes under a different interpreter than production, and nothing in CI closes the gap.

Surfaced during review of PR #204, where the difference in annotation semantics between the two versions came up directly.

Impact of Technical Debt

• Local green does not prove production green. Anything that changed between 3.13 and 3.14 passes locally and fails on deploy. Annotation evaluation is the concrete example: PEP 649 makes annotations lazy by default in 3.14 but not in 3.13, so a forward reference that resolves on a developer machine can raise NameError in Lambda.
• The static analysis is checking the wrong target. mypy is told python_version = "3.13" while running on a 3.14 interpreter, so its view of the stdlib and the developer's actual runtime disagree.
• Review time is spent on it. The 3.13-vs-3.14 distinction already consumed a PR #204 thread, and will keep resurfacing while the two differ.
• The divergence widens on its own. Nothing pins the local version, so a fresh venv on a newer interpreter moves further from production without anyone noticing.

Category

DevOps / Infrastructure

Priority

Medium - Should be addressed soon

Proposed Solution

Move the deployed runtime to 3.14 so it matches what the project is already developed and tested on. The code already runs there — 290 unit tests pass locally on 3.14.7 — so this is a deployment-target change rather than a migration.

Verified against ECR Public and PyPI:

• public.ecr.aws/lambda/python publishes GA arm64 builds for 3.14 — 37 non-preview arm64 tags, latest 3.14.2026.08.26.13-arm64.
• There is no floating 3.14-arm64 tag, unlike 3.13-arm64. The plain 3.14 tag is a single-arch manifest, not a multi-arch list, so it cannot stand in for an arm64 build. The Dockerfile must pin a dated arm64 tag.
• Compiled dependencies publish cp314 artifacts: confluent-kafka==2.15.0 (5 cp314 wheels), psycopg2==2.9.12 (cp314 wheel, 3.14 classifier), cryptography==50.0.0 (13 cp314 wheels, 3.14 classifier). aws-lambda-powertools==3.31.1 declares the 3.14 classifier.

Work:

1. Dockerfile: FROM --platform=linux/arm64 public.ecr.aws/lambda/python:3.14.<dated>-arm64.
2. pyproject.toml: [tool.mypy] python_version = "3.14", [tool.black] target-version = ['py314'].
3. Rebuild the image and confirm the source builds still succeed. This is the main risk: the Dockerfile passes --no-binary confluent-kafka to compile against the librdkafka 2.15.0 built in the image, so the published cp314 wheels are not used and the C extension is compiled against the 3.14 C API directly. psycopg2 (not psycopg2-binary) compiles from source too.
4. Confirm the remaining pure-Python pins install cleanly on 3.14: jsonschema, PyJWT, requests, boto3/botocore, aiosql.
5. Run the integration tests against the rebuilt image, not just unit tests, so the Kafka and Postgres paths exercise the recompiled extensions.
6. Check whether CI pins a Python version anywhere and update it, so the mismatch cannot silently reappear.

Alternative: keep the runtime on 3.13 and recreate the venv on 3.13 instead, pinning the version in CI. That closes the gap equally well and avoids a runtime bump, at the cost of developing against an older interpreter. Either direction is acceptable; leaving the two apart is not.

Effort Estimate

1–2 days, dominated by rebuilding the image and validating the compiled extensions

Dependencies / Related

• PR #204, where this surfaced
• #193

Additional Context

Done when:

• The image builds on 3.14 with librdkafka and psycopg2 compiled from source.
• Unit and integration tests pass against the rebuilt image.
• No tooling config still names 3.13.
• A deployed smoke test covers one POST /topics/{topic_name} fan-out and one /health probe.