gRPC Access Log Service sends invalid UTF-8 in the user agent field
#5,141 opened on 2018年11月28日
Repository metrics
- Stars
- (27,997 stars)
- PR merge metrics
- (平均マージ 8d) (30d で 378 merged PRs)
説明
I have configured a custom gRPC ALS made in Go to log all the requests to Stackdriver. We are continuously receiving bad requests as we have our services exposed to the Internet. From time to time (~40 daily for us) Go reports an error in the gRPC streaming service with this:
grpc: failed to unmarshal the received message proto: field "envoy.data.accesslog.v2.HTTPRequestProperties.user_agent" contains invalid UTF-8
This is probably related to the fact that Go protobuf is validating the encoding of string fields since April:
Strict validation of string fields as valid UTF-8. It is regrettable that the Go implementation has not enforced the encoding thus far, and that was an oversight on our part. However, in order for protobuf messages to be properly parsed by implementations in other languages, it is important to produce an error on the Go side to protect users from generating invalid messages. If your strings do not use UTF-8, then consider using the bytes type.
Two possible solutions to this problem are:
- Changing the string field to a
bytesone. - Removing any invalid UTF-8 character of the user agent or replacing them with a placeholder character before sending the ALS request to the stream.
This happens both in the latest commit uploaded to Docker Hub and v1.8.0.