microsoft/cpprestsdk

utility::conversions::latin1_to_utf16 - Is there a bug for characters above 0x80?

Open

#101 opened on Mar 23, 2016

View on GitHub
 (1 comment) (0 reactions) (0 assignees)C++ (7,596 stars) (1,670 forks)batch import
bughelp wanted

Description

I've tried to use utility::conversions::latin1_to_utf16 to convert latin1 string to UTF-16. The string contained 'í' character (0xed). After converting it should become 0x00ed but it seems that converting method replaced it with 0xffed.

I think this may be result of casting signed char to utf16char and dest[i] = utf16char(s[i]); in asyncrt_utils.cpp should be replaced by dest[i] = utf16char(static_cast<unsigned char>(s[i])); to fix it. Can someone check if I'm right?

Contributor guide

Tech stack
cpp
Domain
apibackend
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
under 1 hour
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
basic C++ knowledgeunderstanding of signed/unsigned char
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70
Research direction
The issue identifies a potential bug in `asyncrt utils.cpp` in the `utility::conversions::latin1 to utf16` function. The problem is that casting a signed `char` to `utf16char` (likely `unsigned short`) causes sign extension for characters above 0x80, resulting in incorrect values like 0xFFED instead of 0x00ED. The suggested fix is to use `static cast<unsigned char>` before casting to `utf16char`. To confirm, examine the source file `asyncrt utils.cpp` in the repository and test the conversion with a Latin 1 character like 'í' (0xED). The maintainer should verify the fix and ensure it doesn't break other use cases.