Transaction log follower logs an error for every transaction deleted from the files log

Open Beginner friendly
  • Is something wrong?
#8,624 2 comments 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@ca61688 is already working on this.

Since Nov 11, 2026.

  • #8646 by @ca61688 — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
aws, java
Domain
backend, cloud

Research direction

Start by reading TransactionLogTransactionStack to find the follower's DynamoEventSource configuration, then check DynamoDBStreamTransactionLogEntryMapper and DynamoDBStreamTransactionLogEntryMapperTest. Filter the follower to INSERT events and cover a REMOVE record in the mapper test; the issue says non-INSERT records should be skipped without ERROR logs.

Written by the indexing model from the issue text.

Description

bug size:small
Description / Background

The transaction log follower lambda reads the DynamoDB Stream on the files transaction log table. The event source has no filter, so it also receives REMOVE records whenever transactions are deleted. This happens when the transaction deletion lambda runs, or when a table is cleared or deleted.

The stream view type is NEW_IMAGE, so a REMOVE record has no new image. DynamoDBStreamTransactionLogEntryMapper assumes every record has one and throws a NullPointerException. The exception is caught, the record is skipped, and an ERROR is logged with a stack trace.

Skipping the record is correct, but every deleted transaction produces an ERROR log that hides real errors. The follower is also invoked for batches with nothing to process.

Steps to reproduce
  1. Deploy an instance and ingest data into a table.
  2. Wait for the transaction deletion lambda to delete old transactions, or delete the table.
  3. The follower's log group has an ERROR Unable to map to transaction log entry with a NullPointerException for each deleted transaction.
Expected behaviour

The follower only receives INSERT records, and any other record is skipped without an ERROR log.

Technical Notes / Implementation Details
  • In TransactionLogTransactionStack, filter the follower's DynamoEventSource to eventName = INSERT. Log items are never updated, so nothing is lost.
  • As a fallback, have DynamoDBStreamTransactionLogEntryMapper skip non-INSERT records at DEBUG level.
  • Add a test to DynamoDBStreamTransactionLogEntryMapperTest for a REMOVE record.
Dominant language
Java
Stars
107
Forks
29
Avg merge
1d 33m
Merged PRs (30d)
115

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from gchq/sleeper

All issues in gchq/sleeper

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.