allow tokenizing string_view to string_view(s)
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 42/100
Research direction
Start by locating assign_or_plus_equal and the tokenizer paths that handle string_view, then compare their current dependence on std::string::assign with the proposed specialization. Done means tokenizing into std::basic_string_view works without relying on std::string's assign function and the existing tokenizer behavior remains intact.
Written by the indexing model from the issue text.
Description
This requires getting around the dependency on std::string's assign function. I added the following specialization to assign_or_plus_equal to get it working but it could probably be done better.
template<class Iterator, typename char_t>
static void assign(Iterator b, Iterator e, std::basic_string_view<char_t>& t)
{
t = { std::addressof(*b), static_cast<size_t>(std::distance(b, e)) };
}
- Dominant language
- C++
- Stars
- 30
- Forks
- 45
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from boostorg/tokenizer
All issues in boostorg/tokenizer
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
games-on-whales/wolf#509 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/cuda-samples#453 ·