[Feature]: Join Field: match a text key with a number key, so Census tables join to TIGER tracts
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 35/100
Research direction
Start in crates/analysis/src/attribute_join.rs where Join Field checks key types and raises the mismatch error. Read the nightly test in #1165 that currently works around the type mismatch. Done means the join matches an all-digit text key to a numeric key (or the error message names the Add/Calculate Field fix) and the workaround in #1165 is removed.
Written by the indexing model from the issue text.
Description
Before you start
- I searched the existing issues and the roadmap and did not find a duplicate request.
What problem does this solve?
The most common US Census join fails. The Census Bureau's tract table (its Gazetteer file, or any table downloaded from data.census.gov) has an 11-digit GEOID such as 47157000100. The CSV reader types that column as a number. The TIGER/Line tract shapefile stores the same GEOID as text. Join Field then refuses:
key field type mismatch: target field 'GEOID' is Text, but source field 'GEOID' is Long
The same happens with any ID that is all digits in one file and text in the other: county and state FIPS codes, ZIP codes, parcel numbers, school or hospital ids.
Found by the nightly real-data workflow test (#1165), which joins Shelby County's 249 tracts to the Gazetteer. To make the join work, the test does what a user has to: it adds a number field to the tracts with Add Field and copies GEOID into it with Calculate Field. That's two extra steps for every join of this kind, and the message doesn't suggest them.
The message also calls the field type Long, while RustGIS's field type is named Integer (FieldType::Integer).
Proposed solution
The choice of approach is the maintainers'. These are the options I can see:
- Compare across types as text. When one key is text and the other a whole number, Join Field compares the number written out in full (
47157000100) with the text. Leading zeros stay meaningful: the CSV reader already keeps01001as text, so a text01001would still not match the number 1001. This fixes the common case with no new parameter. - A parameter for it, such as
"compare": "Exact" | "As text", defaulting to Exact, with the error message pointing to it. This is more explicit, but a user still has to discover it. - Keep the refusal, and improve the message: name the fix (convert one key with Add Field and Calculate Field), and use the same type names the UI uses.
Option 1 plus the clearer message from option 3 looks like the smallest change that makes the everyday case work. Either way, the nightly test in #1165 can then drop its workaround, so it doubles as the regression test.
Alternatives considered
- Have the CSV reader keep long all-digit columns as text (say 10 digits or more, or columns whose names end in
ID,GEOIDorFIPS). This is a heuristic, it changes how existing tables read, and it doesn't help when the number column comes from a GeoPackage or Parquet file. - A way to choose a column's type when adding a CSV. This is useful on its own, but a user has to know to use it before the join fails.
Estimated scope
Small for option 1 or 3 (crates/analysis/src/attribute_join.rs, plus a test). Option 2 adds a tool parameter.
- Dominant language
- Rust
- Stars
- 1.9k
- Forks
- 268
- Avg merge
- 4h 36m
- Merged PRs (30d)
- 851
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from opengeos/RustGIS
-
enhancement
Difficulty 1/5 Under an hour Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
Maintainers usually reply within 1 day
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 55/100
opengeos/RustGIS#1217 · 3 comments ·
Maintainers usually reply within 1 day
All issues in opengeos/RustGIS
Similar issues
-
rust
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
vllm-project/vllm#64939 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
redlib-org/redlib#611 ·
Maintainers usually reply within 1 day
-
area:docs documentation good first issue priority:low
Difficulty 1/5 Under an hour Newbie friendliness 80/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
dividebysandwich/rust-dos#408 ·
Maintainers usually reply within 1 day
-
enhancement performance rust
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day