add tidycjk and tidyEmoji
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- r
- Domain
- documentation
Research direction
Start by comparing how existing CRAN packages are listed in the Natural Language Processing task view, then review the tidycjk and tidyEmoji CRAN pages linked in the issue. Done means both packages appear in the view with accurate, concise descriptions and links.
Written by the indexing model from the issue text.
Description
Two packages that may fit this view, both tidy-style toolkits for text that whitespace-based tooling handles poorly.
tidycjk (0.1.0), for Chinese, Japanese and Korean. CJK writing does not separate words with whitespace, so a standard tokeniser returns either one undifferentiated blob per sentence or isolated characters. It provides cjk_segment() and cjk_tokens() with a pluggable segmenter backend (cjk_segmenters(), register_cjk_segmenter()), script and language identification (cjk_script(), cjk_blocks(), cjk_detect_language(), has_cjk(), cjk_ratio()), counts and summaries (cjk_char_counts(), cjk_summary()), and the width handling that matters whenever CJK text is aligned or truncated (cjk_width(), cjk_pad(), cjk_truncate(), to_fullwidth(), to_halfwidth()).
tidyEmoji (0.4.0), for emoji in text columns. Not every code point is an emoji and a single emoji can span several code points, so emoji statistics are fiddly to get right. It covers extraction (emoji_extract_unnest(), emoji_extract_nest(), emoji_tokens()), counting and density (emoji_frequency(), emoji_density(), emoji_ratio(), emoji_position()), categorisation (emoji_categorize()), sentiment and emotion scoring against published lexicons (emoji_sentiment(), emoji_emotion(), emoji_lexicons(), register_emoji_lexicon()), shortcode conversion both ways for preprocessing and accessibility (as_emoji_shortcode(), as_emoji_name(), emoji_to_text(), text_to_emoji(), emoji_sanitize()), context and collocation (emoji_context(), emoji_collocations(), emoji_ngrams(), emoji_cooccurrence()), and emoji_dfm() for document-by-emoji feature tables. emoji_token_cost() reports what emoji cost under a tokeniser, which matters for LLM preprocessing.
The sentiment lexicon is the Emoji Sentiment Ranking (Kralj Novak et al. 2015, doi:10.1371/journal.pone.0144296).
Disclosure: I am the author of both packages.
- Dominant language
- No language data
- Stars
- 6
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from cran-task-views/NaturalLanguageProcessing
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 55/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 48/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 42/100
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
All issues in cran-task-views/NaturalLanguageProcessing
Similar issues
-
Crush Open
Difficulty 1/5 Under an hour Newbie friendliness 85/100
catppuccin/catppuccin#3125 ·
-
Link Checker Report Openautomated issue report
Difficulty 1/5 Under an hour Newbie friendliness 95/100
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
VoltAgent/awesome-design-md#469 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
KhronosGroup/glTF#2648 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
sccn/sccn.github.io#108 ·