Query on a GSI rejects a FilterExpression that references the base table's sort key

Open
  • Is something wrong?
#34 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
52/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
aws, docker, rust
Domain
backend-api-design, databases

Research direction

Start by tracing DynamoDB Query validation for FilterExpression in the Rust service, focusing on how it determines the queried index’s key attributes versus the base table’s key attributes. Reproduce with the supplied GSI setup: filtering on base-table sk should return Count 0, while filtering on the queried index’s key should still raise ValidationException. The issue does not name source files or tests, so locate the relevant validation tests before estimating the change.

Written by the indexing model from the issue text.

Description

Before you file
  • I searched open and recently closed issues and found no report of this.
  • I checked that this still happens with the latest version of the emulator.
What happened

Summary

Querying a global secondary index (GSI) with a FilterExpression that references an attribute of the base table's primary key fails with a ValidationException in AWS Emulator.

Real DynamoDB accepts the same request. Real DynamoDB rejects only filters on the key attributes of the index being queried.

Environment

  • AWS Emulator: 1.0.322 (LTS package, aarch64, run in Docker)
  • Service: Amazon DynamoDB
  • Operation: Query
  • AWS CLI: v2

Control: filtering on the queried index's own key

On real DynamoDB, filtering on the index's sort key (gsi_sk) instead of sk is rejected:

aws dynamodb query --table-name Ite
  --key-condition-expression '#p = :p' \
  --filter-expression 'begins_with(
  --expression-attribute-names '{"#p":"gsi_pk","#g":"gsi_sk"}' \
  --expression-attribute-values '{"' \
  --select COUNT

An error occurred (ValidationException) when calling the Query operation:
Filter Expression can only contain rimary key attribute: gsi_sk

So the restriction exists on real DynamoDB, but it applies only to the key attributes of the index that is queried. From the index's ps key attributes are ordinaryprojected attributes.

Notes

  • The Query API reference says a FilterExpression "cannot contain partition key or sort key attributes" without distinguishing he behavior above was observed onreal DynamoDB.
  • The error message in the emulatoreal DynamoDB, so the validationitself looks correct. Only the set of attributes it applies to differs.
What you expected

Expected result (real DynamoDB)

{
    "Count": 0,
    "ScannedCount": 0,
    "ConsumedCapacity": null
}
Steps to reproduce
1. Create a table with a GSI

The base table has the key pk (HASH) and sk (RANGE). The GSI has the key gsi_pk (HASH) and gsi_sk (RANGE).

aws dynamodb create-table --table-name Items \
  --attribute-definitions \
    AttributeName=pk,AttributeType=S \
    AttributeName=sk,AttributeType=S \
    AttributeName=gsi_pk,AttributeType=S \
    AttributeName=gsi_sk,AttributeType=S \
  --key-schema AttributeName=pk,KeyType=HASH AttributeName=sk,KeyType=RANGE \
  --global-secondary-indexes '[{
    "IndexName": "ByGsiKey",
    "KeySchema": [
      {"AttributeName": "gsi_pk", "KeyType": "HASH"},
      {"AttributeName": "gsi_sk", "KeyType": "RANGE"}],
    "Projection": {"ProjectionType": "INCLUDE", "NonKeyAttributes": ["extra"]}
  }]' \
  --billing-mode PAY_PER_REQUEST

aws dynamodb wait table-exists --ta
2. Query the GSI and filter on the base table's sort key
aws dynamodb query --table-name Ite
  --key-condition-expression '#p = :p' \
  --filter-expression 'begins_with(
  --expression-attribute-names '{"#p":"gsi_pk","#sk":"sk"}' \
  --expression-attribute-values '{"' \
  --select COUNT
Emulator version
## Actual result (AWS Emulator)


An error occurred (ValidationException) when calling the Query operation:
Filter Expression can only contain rimary key attribute: sk
Impact and workaround

No response

Additional context

No response

Dominant language
Rust
Stars
2.2k
Forks
44
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from aws/aws-emulator

All issues in aws/aws-emulator

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.