Rationale

Currently, engine uses system native locales and just doesn't support UTF-8. This results in several problems, like player save files having wrong characters when written on disk, inconvenient interoperability with third party libraries that use UTF-8 by default.

Proposed solution

To my knowledge, C++17/C++20 doesn't provide required UTF-8 functionality. There are two big libraries that can be used:

• https://icu.unicode.org/
• https://www.gnu.org/software/libiconv/

Their functionality should be equal, ICU is preferred, and we likely want to support both (i.e. iconv too), because e.g. iconv is native to some systems and doesn't require additional packages to be installed.

Useful links

FreeBSD documentation:

• https://docs.freebsd.org/en/books/porters-handbook/special/#using-iconv
• https://people.freebsd.org/~olivierd/porters-handbook/using-iconv.html
• https://docs-archive.freebsd.org/doc/9.3-RELEASE/usr/share/doc/freebsd/ru/books/porters-handbook/using-iconv.html

Methodology

https://utf8everywhere.org/