OpenXRay/xray-16

Switch to UTF-8

Open

#419 opened on Jun 20, 2019

View on GitHub
 (0 comments) (2 reactions) (0 assignees)C++ (519 forks)auto 404
CoreDeveloper ExperienceHelp wantedPortability

Repository metrics

Stars
 (3,524 stars)
PR merge metrics
 (PR metrics pending)

Description

Rationale

Currently, engine uses system native locales and just doesn't support UTF-8. This results in several problems, like player save files having wrong characters when written on disk, inconvenient interoperability with third party libraries that use UTF-8 by default.

Proposed solution

To my knowledge, C++17/C++20 doesn't provide required UTF-8 functionality. There are two big libraries that can be used:

Their functionality should be equal, ICU is preferred, and we likely want to support both (i.e. iconv too), because e.g. iconv is native to some systems and doesn't require additional packages to be installed.

Useful links

FreeBSD documentation:

Methodology

https://utf8everywhere.org/

Contributor guide

Research direction
Research how to integrate ICU and iconv libraries into the CMake build system. Identify all places in the xray 16 codebase that use system locale functions (e.g., setlocale, std::locale, etc.) and replace them with UTF 8 aware calls. Look at existing open source projects that have done similar migrations for reference.
Tech stack
cpp
Domain
game dev
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
C++CMakeGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
45