Repository metrics
- Stars
- (3,761 stars)
- PR merge metrics
- (PR metrics pending)
Description
mbtiles and martin-cp standardize how a tile data is hashed in the mbtiles. tilelive-copy has been generating normalized mbtiles schema with MD5 hashes of the tile data as the table foreign key. So it was possible to validate the content of the tile by re-computing MD5 hash.
tippecanoe (cc @bdon) has used a faster fnv1a checksum algorithm, so clearly there is need for more than one algo.
Proposal
- Add
hash_algorithmmetadata value with the possible valuesmd5,fnv1a, ... - If the value is not there, check if agg_tiles_hash metadata value is present, and use its length to decide algorithm
- If not, and mbtiles uses normalized or flat-with-hash schema, use the size of the hashing keys to decide the algorithm
- If the hashing algorithm is detected, add it to the metadata, possibly after validating that all hashes match their data
Multiple Algorithms Consideration
MBTiles use hashes for the tile_data column and for the agg_tiles_hash metadata field. In theory these could use different algorithms for performance and security(?) reasons. If so, we could either have two metadata fields (hash_algorithm and agg_hash_algorithm), or better yet, rename agg_tiles_hash into agg_tiles_hash_md5 (or other algo). This keeps the algorithm and its value together, and is fairly easy to detect at runtime.