Buffer should convert to UTF8 not ASCII

Open
#2 0 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
1/5
Estimated time
Under an hour
Newbie friendliness
55/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
javascript
Domain
backend, cloud

Research direction

Start at line 37 of ddb_eventprocessor.js and inspect how the Kinesis payload is decoded before JSON processing. Confirm the change by checking that payloads containing Chinese or other non-ASCII characters are preserved and no longer produce invalid JSON control-character errors.

Written by the indexing model from the issue text.

Description

I think line 37 should be rewritten as payload = new Buffer(record.kinesis.data, 'base64').toString('utf8');

I recently ran into a situation where I could not for the life of me figure out why I kept getting this mysterious unexpected token error in Lambda's Cloudwatch console. After a few hours of testing and debugging, I finally figured out that I was passing invalid JSON Control Characters. BUT, what was really happening was I was converting chinese characters to ASCII.
The standard should be UTF8 to support our global connectivity 😉 .

Dominant language
JavaScript
Stars
344
Forks
127
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from aws-samples/lambda-refarch-streamprocessing

All issues in aws-samples/lambda-refarch-streamprocessing

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.