SAM/CloudFormation: 9 differences from AWS when deploying an existing serverless app

Open
#32 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
aws, docker, rust

Research direction

Start by reading the CloudFormation, SAM, Lambda, and API Gateway implementation paths for the nine reproduced differences. The report gives concrete examples and expected AWS behavior, but does not identify source files or tests, and the fixes span several services. Done means the listed deployments and invocations match AWS behavior, including updates and rollback.

Written by the indexing model from the issue text.

Description

What happened

We deployed an existing SAM app with aws cloudformation package and aws cloudformation deploy. The app uses REST API, Lambda, DynamoDB, SNS and an OpenAPI body brought in with AWS::Include. The same templates deploy on AWS. On the emulator, deployment hit the 9 issues below. We worked around each one to get the stacks to CREATE_COMPLETE.

Running the functions is blocked separately by #28 (missing node: built-ins in nodejs24.x) and by issue 1 below.

1. Lambda: provided.al2023 is not supported
Invoke of a provided.al2023 function (bootstrap in a layer) returns InvalidRuntimeException: The runtime provided.al2023 is not supported by this emulator, which runs the Python and node runtimes. Function creation succeeds, so the failure only shows up at invoke time (API Gateway returns 500).

2. CloudFormation: AWS::Include inside a property is not processed

MyApi:
  Type: AWS::Serverless::Api
  Properties:
    StageName: dev
    DefinitionBody:
      Fn::Transform:
        Name: AWS::Include
        Parameters:
          Location: openapi.yml   # rewritten to s3://... by `aws cloudformation package`

The RestApi body that gets created has only the keys Fn::Transform and paths, so creation fails with Invalid OAS input.

3. Lambda::Permission: generated StatementId over 100 characters
SAM generates the permission for an Api event as <FunctionLogicalId><EventKey>Permission. The emulator uses <stack>-<logicalId>-<suffix> as the StatementId without shortening it, which fails with Member must have length less than or equal to 100. Repro: a function with logical ID FnWithAVeryLongLogicalIdForTestingPurposes and event key AnotherVeryLongEventKeyForTestingPurposes. The same template deploys on AWS.

4. CloudFormation: the 51,200-byte limit is applied to templates passed by S3 URL
aws cloudformation deploy --s3-bucket ... uploads the template to S3. CreateChangeSet still fails for a 51,921-byte template with Value at 'templateBody' failed to satisfy constraint: Member must have length less than or equal to 51200. AWS documents 51,200 bytes for an inline template body and 1 MB for a template in an S3 object (CloudFormation quotas).

5. CloudFormation: Template format error: not well-formed for a valid template
A valid 37 KB template, which deploys on AWS, failed with this error as block YAML, as flow YAML and as JSON. Removing either Globals (copied into 9 functions by SAM) or the OpenAPI components section fixed it. This suggests a size limit on the SAM-expanded template that is reported as a format error. We could not find the exact threshold.

6. CloudFormation: Fn::ImportValue is resolved for a resource whose Condition is false

Conditions:
  IsProd: !Equals [!Ref Env, prod]
  NotLocal: !Not [!Equals [!Ref Env, local]]
Resources:
  WafAssoc:
    Type: AWS::WAFv2::WebACLAssociation
    Condition: NotLocal            # false when Env=local
    Properties:
      ResourceArn: ...
      WebACLArn: !If [IsProd, !ImportValue prod-acl, !ImportValue dev-acl]

With Env=local the stack rolls back with No export named prod-acl found. On AWS, the resource is skipped and the stack deploys.

7. SAM: bare AWS managed policy names in Policies are not expanded to ARNs
Policies: [AmazonAPIGatewayInvokeFullAccess] fails role creation with ARN AmazonAPIGatewayInvokeFullAccess is not valid. SAM documents that Policies accepts "the name of an AWS managed policy" (AWS::Serverless::Function Policies).

8. Lambda: number values in Environment.Variables are not converted to strings
Globals: {Function: {Environment: {Variables: {MAX_COUNT: 25}}}} fails function creation with expected string, got number. The same template deploys on AWS.

9. Stack updates and rollback

  • Updating a stack where the RestApi body changed fails with NotImplemented: this emulator does not change the properties of an existing AWS::ApiGateway::RestApi in place.
  • When AWS::ApiGateway::Stage creation fails, rollback cannot delete the deployment (Active stages pointing to this deployment must be moved or deleted). The stack is left in ROLLBACK_FAILED, and only an emulator restart clears it.
What you expected

The same behavior as AWS: the same templates deploy successfully with aws cloudformation package and aws cloudformation deploy. Issues 4 and 7 contradict the AWS documentation (S3 template size limit; SAM Policies accepting managed policy names).

Steps to reproduce
  1. Start a fresh emulator. Point the AWS CLI at it with AWS_ENDPOINT_URL=https://127.0.0.1:9297 and the emulator's credentials, config and CA files, in region us-east-1.
  2. For each issue above, deploy a minimal SAM template containing the snippet shown:
    aws cloudformation package --template-file template.yml --s3-bucket <bucket> --output-template-file packaged.yml, then
    aws cloudformation deploy --template-file packaged.yml --stack-name repro --s3-bucket <bucket> --capabilities CAPABILITY_IAM CAPABILITY_AUTO_EXPAND
  3. Check the result with aws cloudformation describe-stack-events --stack-name repro.

Client: AWS CLI 2.32.34 on macOS (arm64). The emulator runs in Docker, built from the Dockerfile in the user guide with the aarch64 RPM.

Emulator version

Docker image built on 2026-11-12 from the LTS/latest aarch64 RPM (https://downloads.aws.com/aws-emulator/packages/aws-emulator/LTS/latest/). I'll add the exact aws-emulator -v output in a comment.

Impact and workaround

Together these block deploying an existing SAM app without changes. We worked around 2–8 by rewriting the template before packaging:

  • inlined the includes
  • shortened the event keys
  • stubbed the OpenAPI schemas
  • expanded the managed policy ARNs
  • quoted the environment variable values
  • created placeholder exports

We work around 9 by recreating stacks instead of updating them. There's no workaround for 1 or for #28, so the functions can't be run.

Additional context

Happy to provide minimal standalone templates for any item if that helps triage. If you'd prefer one issue per item, we can split this up.

Dominant language
Rust
Stars
2.2k
Forks
44
PR merge metrics
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from aws/aws-emulator

All issues in aws/aws-emulator

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.