Layer v3.30.0 ships multi_json 1.21.1, which seems to break apps on json 3

まず、Ruby Lambda layer の layer build inputs と依存関係のインストールを確認してください。issue では /opt/ruby/gems/4.0.0 に含まれる multi_json のバージョンが挙げられています。提供された Dockerfile と run.sh を使い、json 3.1.0 で問題を再現します。その後、再ビルドによって multi_json 1.21.2 以降が解決され、handler が正常に解析できることを確認してください。layer が JSON.parse の引数エラーを発生させなくなれば完了です。

Expected Behavior

Our app bundles json 3.x and should keep parsing JSON fine with the Datadog Ruby layer attached.

Actual Behavior

After we bumped json to 3.1.0, every Octokit call in our Lambda started failing with ArgumentError: wrong number of arguments (given 2, expected 1) inside JSON.parse.

From what we can tell, layer v3.30.0 (built Aug 14) ships multi_json 1.21.1 through this chain:

datadog-lambda 3.30.0
└── aws-xray-sdk ~> 0.11.3
    └── multi_json ~> 1   (resolved to 1.21.1, the latest on Aug 14)

multi_json 1.21.1 calls JSON.parse(string, options), and json 3 only takes keyword options. Since the layer's gems end up on the function's GEM_PATH, and Sawyer (used by Octokit) picks multi_json when it can find it, our app ends up using the layer's copy even though we don't depend on multi_json ourselves. Without the layer, the same app seems to work fine on json 3.

It looks like multi_json already fixed this in 1.21.2 (Sep 7): https://github.com/sferik/multi_json/pull/67

Possible fix

We think a plain rebuild of the layer might be enough. The build doesn't use a lockfile, so it should resolve multi_json 1.21.2 now. As far as we can see nothing should conflict:

• aws-xray-sdk 0.11.x wants multi_json ~> 1, which 1.21.2 fits.
• 1.21.2 has the same requirements as 1.21.1 (Ruby >= 3.2, concurrent-ruby ~> 1.2), so it should work for all the layer runtimes.
• We didn't spot anything else in the layer depending on multi_json, but we only looked at direct dependencies.

If you want to keep older versions out of future builds, a multi_json >= 1.21.2 constraint in the gemspec (or an explicit install in the Dockerfile, like ffi) could do it. You'll know better than us whether that fits.

Steps to Reproduce the Problem

We put together a small repro on the official Lambda Ruby 4.0 image, run through the runtime emulator. The app only has json and sawyer as gems and never requires multi_json. Drop the three files below in a directory and run ./run.sh with AWS credentials. It downloads Datadog-Ruby4-0:30, unpacks it into /opt like Lambda does, and calls the handler three times: json 3.1.0 with the layer as is, json 2.21.2 with the same layer, and json 3.1.0 with only the layer's multi_json swapped for 1.21.2 (roughly what we'd expect a rebuild to give).

Dockerfile:

FROM public.ecr.aws/lambda/ruby:4.0
ARG LAYER_DIR
ARG JSON_VERSION
ARG MULTI_JSON_VERSION
# The Datadog layer, unzipped, exactly as Lambda mounts it.
COPY ${LAYER_DIR}/ /opt/
# Optionally swap the layer's multi_json, as a rebuild of the layer would.
RUN if [ -n "$MULTI_JSON_VERSION" ]; then \
      gem uninstall multi_json --all --force --install-dir /opt/ruby/gems/4.0.0 && \
      gem install multi_json:${MULTI_JSON_VERSION} --no-document --install-dir /opt/ruby/gems/4.0.0; \
    fi
# json has a C extension.
RUN dnf install -y gcc make >/dev/null
# The app's only dependencies: json and Sawyer (Octokit's HTTP client), vendored like `sam build` does.
WORKDIR /var/task
RUN printf "source 'https://rubygems.org'\ngem 'json', '${JSON_VERSION}'\ngem 'sawyer', '0.9.3'\n" > Gemfile && \
    bundle config set --local path vendor/bundle && bundle install --quiet && \
    rm -rf Gemfile Gemfile.lock .bundle
COPY app.rb /var/task/
CMD ["app.handler"]

app.rb:

require 'json'
require 'sawyer'

def handler(event:, context:)
  # The app never asks for multi_json. Sawyer picks it when it is loadable.
  data = Sawyer::Serializer.any_json.decode('{"ok":true}')
  { json: JSON::VERSION, multi_json: (MultiJson::VERSION if defined?(MultiJson)), result: data }
rescue => e
  { json: JSON::VERSION, multi_json: (MultiJson::VERSION if defined?(MultiJson)), error: "#{e.class}: #{e.message}",
    at: e.backtrace.first(2) }
end

run.sh:

#!/usr/bin/env bash
# Usage: ./run.sh [layer version, default 30]. Needs AWS credentials to download the layer.
set -euo pipefail
cd "$(dirname "$0")"
version=${1:-30}
dir=layer-$version
if [ ! -d "$dir" ]; then
  url=$(aws lambda get-layer-version-by-arn --region us-east-1 \
    --arn "arn:aws:lambda:us-east-1:464622532012:layer:Datadog-Ruby4-0:$version" \
    --query Content.Location --output text)
  curl -sSfo layer.zip "$url" && mkdir "$dir" && unzip -q layer.zip -d "$dir" && rm layer.zip
fi
echo "multi_json in layer: $(ls "$dir"/ruby/gems/4.0.0/gems | grep '^multi_json-' || echo none)"
# json version, multi_json override ("" keeps the layer's own)
for run in "3.1.0 " "2.21.2 " "3.1.0 1.21.2"; do
  read -r json multi_json <<<"$run"
  docker build -q --platform linux/amd64 --build-arg LAYER_DIR="$dir" --build-arg JSON_VERSION="$json" \
    --build-arg MULTI_JSON_VERSION="${multi_json:-}" -t dd-multi-json-repro . >/dev/null
  id=$(docker run -d --rm --platform linux/amd64 -p 9000:8080 dd-multi-json-repro)
  sleep 2
  echo "json $json, multi_json ${multi_json:-from layer}: $(curl -s -d '{}' http://localhost:9000/2015-03-31/functions/function/invocations)"
  docker stop "$id" >/dev/null
done

What we get:

multi_json in layer: multi_json-1.21.1
json 3.1.0, multi_json from layer: {"json":"3.1.0","multi_json":"1.21.1","error":"ArgumentError: wrong number of arguments (given 2, expected 1)","at":["/var/task/vendor/bundle/ruby/4.0.0/gems/json-3.1.0/lib/json/common.rb:291:in 'JSON.parse'","/opt/ruby/gems/4.0.0/gems/multi_json-1.21.1/lib/multi_json/adapters/json_gem.rb:47:in 'MultiJSON::Adapters::JsonGem#load'"]}
json 2.21.2, multi_json from layer: {"json":"2.21.2","multi_json":"1.21.1","result":{"ok":true}}
json 3.1.0, multi_json 1.21.2: {"json":"3.1.0","multi_json":"1.21.2","result":{"ok":true}}

Specifications

• Datadog Lambda Layer version: Datadog-Ruby4-0:30 (v3.30.0)
• Ruby version: 4.0 (ruby4.0 runtime)

Stacktrace

From our function in production (trimmed):

ArgumentError: wrong number of arguments (given 2, expected 1)
/var/task/vendor/bundle/ruby/4.0.0/gems/json-3.1.0/lib/json/common.rb:291:in 'JSON.parse'
/opt/ruby/gems/4.0.0/gems/multi_json-1.21.1/lib/multi_json/adapters/json_gem.rb:47:in 'MultiJSON::Adapters::JsonGem#load'
/opt/ruby/gems/4.0.0/gems/multi_json-1.21.1/lib/multi_json/adapter.rb:100:in 'MultiJSON::Adapter.load'
/opt/ruby/gems/4.0.0/gems/multi_json-1.21.1/lib/multi_json.rb:182:in 'MultiJSON.parse'
/var/task/vendor/bundle/ruby/4.0.0/gems/sawyer-0.9.3/lib/sawyer/serializer.rb:64:in 'Sawyer::Serializer#decode'
/var/task/vendor/bundle/ruby/4.0.0/gems/sawyer-0.9.3/lib/sawyer/agent.rb:130:in 'Sawyer::Agent#decode_body'
/var/task/vendor/bundle/ruby/4.0.0/gems/octokit-10.0.0/lib/octokit/connection.rb:159:in 'Octokit::Connection#request'
/var/task/vendor/bundle/ruby/4.0.0/gems/octokit-10.0.0/lib/octokit/client/repositories.rb:27:in 'Octokit::Client::Repositories#repository'
...