Filter using string partial matching
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
Research direction
Start at the bcdc_query_geodata() entry point and trace how dplyr filter(), collect(), st_drop_geometry(), select(), and pull() are translated. Check whether attribute-only filtering can avoid downloading geometries while preserving the query result. Done means partial string matching can be applied efficiently before geometry retrieval, with the behavior covered by the package's existing tests.
Written by the indexing model from the issue text.
Description
Problem: I'd like to filter a query but I only know part of the string I am looking for...
bcdata::bcdc_query_geodata("bc-parks-ecological-reserves-and-protected-areas") %>%
filter(stringr::str_detect(PROTECTED_LANDS_NAME, "SUGARBOWL.*PARK*"))
#> Error in stri_detect_regex(string, pattern, negate = negate, opts_regex = opts(pattern)): object 'PROTECTED_LANDS_NAME' not found
Potential solution:
I could download/collect only the column that I want to filter by, detect the sting that I am looking for and then use that string to filter my query.
my_var <- bcdc_query_geodata("bc-parks-ecological-reserves-and-protected-areas") %>%
collect() %>%
st_drop_geometry() %>%
select(PROTECTED_LANDS_NAME) %>%
filter(stringr::str_detect(PROTECTED_LANDS_NAME, "SUGARBOWL.*PARK")) %>%
pull()
bcdc_query_geodata("bc-parks-ecological-reserves-and-protected-areas") %>%
filter(PROTECTED_LANDS_NAME == my_var)
Problem with this solution: I need to download the geometries of the data set to get the column that I am interested in, which is essentially just downloading the entire dataset, so there is no processing speed advantage.
My question:
Is there a way to drop the geometry before collecting the data? I suspect that this would be very fast and efficient for filtering large datasets using string detection.
- Dominant language
- R
- Stars
- 87
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from bcgov/bcdata
-
running air in CI Open
Difficulty 3/5 1-2 days Newbie friendliness 25/100
-
considering
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
considering enhancement wfs
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
Similar issues
-
documentation pkg infrastructure
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
epiverse-trace/epiparameter#511 ·
-
function:write_dwc
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
r-lib/pkgdepends#485 · 3 comments ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
beginners blocker
Difficulty 2/5 1-3 hours Newbie friendliness 78/100