Completing Protobuf / gRPC support (JENA-2167): message-level StreamRDF/RowSet adapters, Protobuf RDFPatch, and RDFLink over gRPC

Open
  • Is something wrong?
#4,364 15 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
java
Domain
backend-api-design

Research direction

Start by reading the attached reviewed approach and examining Jena's existing Protobuf converters and Thrift schema, which the issue identifies as the relevant starting points. The proposed work spans message-level adapters, RDF Patch support, and a gRPC transport, so confirm the boundaries and expected APIs with maintainers before starting. Done means the agreed work is implemented with the promised test coverage; the issue proposes splitting it into three PRs.

Written by the indexing model from the issue text.

Description

enhancement
Version

6.x (main)

Feature
TLDR - the full roadmap

I would like to contribute to the Jena project and help improve it's support for Protobuf and gRPC integration. As an apache contributor, I am trying to expand gRPC support in the ecosystem as part of greater efforts for Apache licensed software.

Digging through your tickets, I found some interest in gRPC support. I'd like to propose 3 PRs to get this done.

What I'd like to contribute
  • Direct Protobuf message streaming without byte-stream overhead: Allow applications to convert RDF triples, quads, and SPARQL result rows directly to and from Jena's existing Protobuf objects in memory, rather than having to serialize and re-parse them through Java byte streams when using frameworks like gRPC or Kafka.
  • Protobuf support for RDF Patch: Bring Jena's Protobuf schema up to parity with its Thrift schema by adding support for RDF Patches, so transactional dataset additions, deletions, and prefix changes can be streamed and recorded in Protobuf format.
  • A gRPC server and client for Jena datasets: Provide an optional gRPC transport that plugs into Jena's standard connection APIs and storage engines (such as TDB2) without pulling in an embedded HTTP/servlet server like Jetty.
  • End-to-end streaming for graph reads, writes, and SPARQL queries: Support streaming quad pattern lookups, bulk RDF loading, transactional patch replication, and SPARQL query results natively over gRPC streams.

I'd make 3 PRs to get this done.

  1. Expose the in-memory Protobuf message converters
  2. RDF Patch support
  3. gRPC transport module

As far as approach goes, it's pretty straightforward and I attached an AI-assisted approach to it. I've fully vetted and edited it, but because it has massive references to the codebase that a normal human wouldn't write, I have attached it as an ai-generated plan and won't call it my own (but think it's exactly right and would seek feedback).

ai-generated-approach-reviewed.md

I'd recommend we get an implementation going first then we can figure out the server infrastructure for serving gRPC if you're interested.

Prior experience to get this quickly with quality

I've contributed gRPC improvements to a multitude of projects

  • Quarkus - fixed critical bugs + massive speed improvements
  • Micronaut - created gRPC <--> REST automation
  • Apicurio - better serialization of protobufs and better performance enhancements
  • OpenNLP - I'm a committer to this project. We're going to push an initial gRPC server that'll support all of OpenNLPs rich feature set (probably early 2027)
  • Tika - working with the team to come up with a parser design with their v2 of their grpc service

and many more.

I'll make sure that there's plenty of test coverage, I'll attribute all code that is AI written and review it carefully before opening a PR, and am totally OK if the patch is too large to break it up into smaller tickets that are easy for a maintainer to understand. This is an important feature to me; and I'll start working on it now. If you make a branch for this, push my changes there as a PR from my branch or directly. I'd only recommend this so we can approve the code incrementally and push to a branch before merging a full branch and allowing for contributions and collaboration from maintainers and committers within jena.

Are you interested in contributing a solution yourself?

Yes

Dominant language
Java
Stars
1.5k
Forks
733
Avg merge
1d 4h
Merged PRs (30d)
48

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/jena

All issues in apache/jena

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.