facebook/zstd

zstd fails to process some filenames on Windows [we need a hero]

Open

#3.882 aberto em 24 de jan. de 2024

Ver no GitHub
 (19 comments) (0 reactions) (0 assignees)C (27.100 stars) (2.481 forks)batch import
help wanted

Description

Windows 7 x64, zstd 1.5.5

$ dir /b
birds.txt
pájaros.txt
vögel.txt
птицы.txt

$ zstd *.*
zstd: can't stat pajaros.txt : No such file or directory -- ignored
zstd: can't stat vogel.txt : No such file or directory -- ignored
  2 files compressed : 85.71% (  3.00 KiB =>   2.57 KiB)
 
$ dir /b *.zst
birds.txt.zst
птицы.txt.zst

@Cyan4973 said zstd doesn't do anything special, it uses the C standard fopen() function. This should be naturally compatible with any utf-8 character set. But I suspect Windows does something different with the character set, so there might be a need to employ other non-portable Windows variants of > fopen for these scenarios.

Guia do colaborador

Pilha de tecnologia
c
Domain
cli
Tipo Issue
bug
DifficultyDificuldade de implementação estimada para um novo contribuidor, de 1 para alterações muito pequenas a 5 para trabalho de nível especializado.
4
Tempo estimadoUm intervalo de tempo aproximado para um colaborador experiente investigar, implementar, testar e preparar um pull request.
3-5 days
Status da atividadeQuão disponível o issue aparece agora: novo, ativo, obsoleto, bloqueado ou aguardando entrada do mantenedor.
active
ClarityCom que clareza o issue explica a mudança esperada, os critérios de aceitação e a próxima etapa.
mostly clear
Prerequisites
C programmingWindows API knowledgeunderstanding of Unicode encodingfamiliarity with cross platform compilation
Simpatia para novatosUma pontuação de 1 a 100 que estima o quão acessível este issue é para colaboradores iniciantes.
20
Direção de pesquisa
The issue is that zstd fails to process filenames with non ASCII characters on Windows because it uses the C standard fopen() which does not handle UTF 8 filenames on that platform. The fix likely involves using Windows specific wide character file functions like wfopen() or the newer CreateFileW() API. Look at the source code in programs/fileio.c where file opening is handled, and investigate how to conditionally use the appropriate function based on the WIN32 macro. Refer to the maintainer's comment in the issue for guidance.