GGUF endianness cannot be determined from GGUF itself
#3.957 geöffnet am 5. Nov. 2023
Repository-Metriken
- Stars
- (110.169 Stars)
- PR-Merge-Metriken
- (Durchschn. Merge 6T 8h) (389 gemergte PRs in 30 T)
Beschreibung
As of the time of writing, the big-endian support that was added in https://github.com/ggerganov/llama.cpp/pull/3552 doesn't encode the endianness within the file itself:
This means that there is no way to distinguish a big-endian GGUF file from a little-endian file, which may cause some degree of consternation in the future if these files get shared around 😅
The cleanest solution would be to add the endianness to the header - ideally, it would be in the metadata, but the reading of the metadata is dependent on the endianness - but that would be a breaking change.
Given that, my suggestion would be to use FUGG as the header for big-endian files so that a little-endian executor won't attempt to read it at all unless it knows how to deal with it. The same can go the other way, as well (a big-endian executor won't attempt to read a little-endian executor).