envoyproxy/envoy

gRPC Access Log Service sends invalid UTF-8 in the user agent field

Open

#5141 aperta il 28 nov 2018

Vedi su GitHub
 (4 commenti) (1 reazione) (0 assegnatari)C++ (5373 fork)batch import
bughelp wanted

Metriche repository

Star
 (27.997 star)
Metriche merge PR
 (Merge medio 8g) (378 PR mergiate in 30 g)

Descrizione

I have configured a custom gRPC ALS made in Go to log all the requests to Stackdriver. We are continuously receiving bad requests as we have our services exposed to the Internet. From time to time (~40 daily for us) Go reports an error in the gRPC streaming service with this:

grpc: failed to unmarshal the received message proto: field "envoy.data.accesslog.v2.HTTPRequestProperties.user_agent" contains invalid UTF-8

This is probably related to the fact that Go protobuf is validating the encoding of string fields since April:

Strict validation of string fields as valid UTF-8. It is regrettable that the Go implementation has not enforced the encoding thus far, and that was an oversight on our part. However, in order for protobuf messages to be properly parsed by implementations in other languages, it is important to produce an error on the Go side to protect users from generating invalid messages. If your strings do not use UTF-8, then consider using the bytes type.

Two possible solutions to this problem are:

  1. Changing the string field to a bytes one.
  2. Removing any invalid UTF-8 character of the user agent or replacing them with a placeholder character before sending the ALS request to the stream.

This happens both in the latest commit uploaded to Docker Hub and v1.8.0.

Guida contributor