[DESIGN PROPOSAL] Generate a gRPC request/response based on a GraphQL query at build time

SUMMARY

With GraphQL the structure of a responses is influenced by the query (for example field names can be aliased) and thus a static proto cannot be used on the server at build time.

This design proposes having the client generate a proto definition at client build time from it’s GraphQL queries. The client can use this proto to generate standard Java, Js, etc code for use by the client. The server won’t have access to this generated proto but will infer and dynamically create its definition based on the received query and the GraphQL schema on the server.

GraphQL queries and mutations can declare variables which are usually encoded using JSON. These variables can be used to create a request proto for a query at build time.

In summary, request and response protos can be generated at build time for clients to use. The server will be able to respond to these requests without access to the generated protos by dynamically creating the proto response.

OBJECTIVE

• Binary proto over the wire (gRPC)
• Regular proto responses on the client
• Support full GraphQL spec
• Leverage existing open source libraries and tools
• Work for all gRPC clients and languages
• Only deps are GraphQL-Java and gRPC (Rejoiner isn’t required but recommend)

Non-goals

• Use the same exact types as the underlying protos (future work)

BACKGROUND

Although GraphQL libraries usually return data as JSON, the GraphQL spec allows for any protocol/encoding that can accommodate dynamically structured responses. GraphQL queries can also define variables that are supplied as a JSON object alongside the string query.

DETAILED DESIGN

GraphQL Query & Mutation to proto mapping

Response message

Enum support Options

• Generate a single Enums.proto file for entire server schema
• Generate enum on demand
  • Cons: Duplicates
• Import it from source
  • Pros: simple in theory
  • Cons: proto visibility,

Request message

Static Proto A bytes field can be used in a service definition to represent the data section of a GraphQL response. This would be ideal for clients using the binary wire format, and is how the Any type is implemented. For clients that want a JSON response the Struct type can be used to cleanly model the standard GraphQL JSON response.

message GraphQlResponse {
  repeated bytes data = 1;
  // errors, etc ...
}

Build time proto generation

A RelayJS compiler plugin will generate a response proto at build time. The Relay compiler can be used to generate artifacts for any language, in this case a simple proto definition. A build rule will be added to the Relay-compiler Skylark build rule to generate this proto.

Rather than using the Relay compiler (which is implemented in Js and runs in Node), the Java GraphQL library could be used to generate the proto. This approach would be simpler and should be compared to the Relay compiler. Relay is more powerful and it’s advantages show when constructing a query from composing multiple fragments.

The GraphQL API explorer has been updated to include a proto preview, which can be seen in the above screen shots.

Dynamic message

The server will dynamically construct the proto descriptor and populate the response based on the schema on the server and the run-time query.

The query is parsed into an AST, which is then transformed via a QueryTraversal into a DescriptorProto. Additional type information supplied from the Schema during query traversal. The same heuristics are used as those in the client when creating a proto from the query.

Creating response proto

public static DescriptorProto createProto(String query, GraphQLSchema schema) {
  DescriptorProtos.DescriptorProto.Builder descriptorProto =
      DescriptorProtos.DescriptorProto.newBuilder().setName("...");

  // add fields to descriptorProto …

  return descriptorProto.build();
}

Fill response

public static Message createResponse(
    DescriptorProto descriptor, Map<String,Object> response) {
  
  DynamicMessage.Builder responseBuilder =
      DynamicMessage.newBuilder(  descriptorProtoToDescriptor(descriptorProto));

  // build response ...

  return responseBuilder.build();
}

REJECTED ALTERNATIVES

The previous attempt at offering a proto based GraphQL API didn’t support the full GraphQL spec. It created a static proto based on the GraphQL schema for a GraphQL server. This resulted in a standard looking proto response, but didn’t support the full GraphQL spec.

Another attempt used the Struct, ListValue, etc prototypes to model the response similar to JSON. This resulted in proto that was difficult to consume on the client and a less than optimal over the wire encoding.