Completing Protobuf / gRPC support (JENA-2167): message-level StreamRDF/RowSet adapters, Protobuf RDFPatch, and RDFLink over gRPC

オープン
  • 何か問題がありますか？
#4,364 コメント 14 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
30/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
活発
技術スタック
java
領域
backend-api-design

調査の方向性

Start by reading the attached reviewed approach and examining Jena's existing Protobuf converters and Thrift schema, which the issue identifies as the relevant starting points. The proposed work spans message-level adapters, RDF Patch support, and a gRPC transport, so confirm the boundaries and expected APIs with maintainers before starting. Done means the agreed work is implemented with the promised test coverage; the issue proposes splitting it into three PRs.

索引モデルが issue の本文から書いたものです。

説明

enhancement
Version

6.x (main)

Feature
TLDR - the full roadmap

I would like to contribute to the Jena project and help improve it's support for Protobuf and gRPC integration. As an apache contributor, I am trying to expand gRPC support in the ecosystem as part of greater efforts for Apache licensed software.

Digging through your tickets, I found some interest in gRPC support. I'd like to propose 3 PRs to get this done.

What I'd like to contribute
  • Direct Protobuf message streaming without byte-stream overhead: Allow applications to convert RDF triples, quads, and SPARQL result rows directly to and from Jena's existing Protobuf objects in memory, rather than having to serialize and re-parse them through Java byte streams when using frameworks like gRPC or Kafka.
  • Protobuf support for RDF Patch: Bring Jena's Protobuf schema up to parity with its Thrift schema by adding support for RDF Patches, so transactional dataset additions, deletions, and prefix changes can be streamed and recorded in Protobuf format.
  • A gRPC server and client for Jena datasets: Provide an optional gRPC transport that plugs into Jena's standard connection APIs and storage engines (such as TDB2) without pulling in an embedded HTTP/servlet server like Jetty.
  • End-to-end streaming for graph reads, writes, and SPARQL queries: Support streaming quad pattern lookups, bulk RDF loading, transactional patch replication, and SPARQL query results natively over gRPC streams.

I'd make 3 PRs to get this done.

  1. Expose the in-memory Protobuf message converters
  2. RDF Patch support
  3. gRPC transport module

As far as approach goes, it's pretty straightforward and I attached an AI-assisted approach to it. I've fully vetted and edited it, but because it has massive references to the codebase that a normal human wouldn't write, I have attached it as an ai-generated plan and won't call it my own (but think it's exactly right and would seek feedback).

ai-generated-approach-reviewed.md

I'd recommend we get an implementation going first then we can figure out the server infrastructure for serving gRPC if you're interested.

Prior experience to get this quickly with quality

I've contributed gRPC improvements to a multitude of projects

  • Quarkus - fixed critical bugs + massive speed improvements
  • Micronaut - created gRPC <--> REST automation
  • Apicurio - better serialization of protobufs and better performance enhancements
  • OpenNLP - I'm a committer to this project. We're going to push an initial gRPC server that'll support all of OpenNLPs rich feature set (probably early 2027)
  • Tika - working with the team to come up with a parser design with their v2 of their grpc service

and many more.

I'll make sure that there's plenty of test coverage, I'll attribute all code that is AI written and review it carefully before opening a PR, and am totally OK if the patch is too large to break it up into smaller tickets that are easy for a maintainer to understand. This is an important feature to me; and I'll start working on it now. If you make a branch for this, push my changes there as a PR from my branch or directly. I'd only recommend this so we can approve the code incrementally and push to a branch before merging a full branch and allowing for contributions and collaboration from maintainers and committers within jena.

Are you interested in contributing a solution yourself?

Yes

主要言語
Java
スター
1.5k
フォーク
729
平均マージ
1日 8時間
マージ済み PR（30日）
56

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

apache/jena のほかの issue

apache/jena の issue をすべて見る

似ている issue

Java の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。