Migrate the generated TypeScript bindings from ts-proto to protobuf-es v2

Đang mở
  • Có gì không ổn?
#3,632 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
30/100
Loại issue
Tái cấu trúc
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
grpc, typescript
Lĩnh vực
api, frontend

Hướng nghiên cứu

Start in app/controlplane/api/buf.gen.yaml, which holds the ts-proto plugin entry, and then run make api to see the current output in gen/frontend. The work is to swap the generator to buf.build/bufbuild/es with target=ts and include_imports: true, remove the old ts-proto files, and regenerate. Done means the acceptance criteria hold: no @improbable-eng/grpc-web, browser-headers or protobufjs imports, and no diff from make api in CI. The consumer migration and breaking-change plan still need a maintainer decision.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

component/frontend dependencies

Summary

Migrate the generated TypeScript bindings of the control plane API from ts-proto to protobuf-es v2 (@bufbuild/protobuf v2). Use Connect-ES v2 (@connectrpc/connect v2) for the clients.

Current state

app/controlplane/api/buf.gen.yaml generates the TypeScript bindings with ts-proto:

- remote: buf.build/community/stephenh-ts-proto:v1.151.1
  out: ./gen/frontend
  opt:
    - outputClientImpl=grpc-web
    - esModuleInterop=true
    - useOptionals=messages
  • The output goes to app/controlplane/api/gen/frontend.
  • The output contains the control plane, attestation, workflow contract, errors and jsonfilter protos.
  • The output also contains third-party imported protos. Examples: protoc-gen-openapiv2/options/annotations.ts, protoc-gen-openapiv2/options/openapiv2.ts, buf/validate/... and google/api/....
  • The generated code depends on @improbable-eng/grpc-web, browser-headers and protobufjs/minimal.

This is the only buf.gen.yaml in the repository that emits TypeScript. The other files (app/artifact-cas/api, pkg/attestation/crafter/api, pkg/credentials/api, the plugin SDK and the conf packages) emit Go only. So the scope is this one file and its output folder.

Motivation

  • ts-proto with the improbable-eng grpc-web transport is in maintenance mode.
  • protobuf-es v2 and Connect-ES v2 are the stack that Buf maintains. They have a smaller runtime, first-class ESM and TypeScript support, and they pass the Protobuf conformance tests.
  • Some downstream consumers already generate their own protos with protobuf-es v2 into the same source tree. Today they get two different generated files for the same imported proto. For example, protoc-gen-openapiv2/options/annotations.ts from ts-proto sits next to annotations_pb.ts from protobuf-es. The consumers must filter files out when they sync the bindings. One generator for all bindings removes this problem.

Proposed implementation

1. Replace the generator plugin

protobuf-es v2 also generates the service descriptors. A separate connect-es plugin is not necessary.

- remote: buf.build/bufbuild/es:v2.16.0
  out: ./gen/frontend
  opt: target=ts
  include_imports: true

include_imports: true generates the imported dependencies (openapiv2 options, buf/validate, google/api) as *_pb.ts files. Consumers then get a self-contained set of files.

2. Changes in the generated code
  • File names change to <name>_pb.ts.
  • Messages are plain types. Create them with create(FooSchema, {...}).
  • Serialize with toBinary, fromBinary, toJson and fromJson from @bufbuild/protobuf.
  • Enums are TypeScript enums.
  • Oneofs use the { case, value } form.
  • Use @bufbuild/protobuf/wkt for Timestamp (timestampDate, timestampFromDate).
3. Clients
  • Create clients with createClient(Service, transport) from @connectrpc/connect v2.
  • Use createGrpcWebTransport from @connectrpc/connect-web v2. The transport stays gRPC-web, so the server needs no change.
  • Remove the @improbable-eng/grpc-web, browser-headers and protobufjs dependencies.
import { createClient } from "@connectrpc/connect";
import { createGrpcWebTransport } from "@connectrpc/connect-web";
import { WorkflowService } from "./controlplane/v1/workflow_pb";

const transport = createGrpcWebTransport({ baseUrl: "https://cp.example.com" });
const client = createClient(WorkflowService, transport);
4. Transition
  • This is a breaking change for every consumer of gen/frontend.
  • If a transition period is necessary, emit the protobuf-es output to a new folder first (for example gen/frontend-es). Keep the ts-proto output in gen/frontend until the consumers migrate. Then remove the ts-proto plugin.
5. Regeneration and CI
  • Run make api from the repository root (it runs buf generate in app/controlplane/api) and commit the new output.
  • Remove the old ts-proto files from the output folder. buf generate does not delete stale files.
  • The test.yml workflow runs make api and fails if the tree is dirty. The regenerated files must be in the same pull request.

Acceptance criteria

  • app/controlplane/api/buf.gen.yaml uses buf.build/bufbuild/es v2 with target=ts and include_imports: true.
  • The generated output has no imports of @improbable-eng/grpc-web, browser-headers or protobufjs.
  • The generated output contains the imported third-party protos as *_pb.ts files.
  • make api produces no diff in CI.
  • The release notes state the breaking change for consumers of gen/frontend.

🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri

Ngôn ngữ chính
Go
Star
582
Fork
61
Merge trung bình
12 giờ 47 phút
Pull request đã merge (30 ngày)
137

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của chainloop-dev/chainloop

Tất cả issue của chainloop-dev/chainloop

Issue tương tự

Thêm issue về Go

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.