Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

add tidycjk and tidyEmoji

Open Beginner friendly
#15 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Documentation
Clarity
Mostly clear
Activity status
Active
Tech stack
r
Domain
documentation

Research direction

Start by comparing how existing CRAN packages are listed in the Natural Language Processing task view, then review the tidycjk and tidyEmoji CRAN pages linked in the issue. Done means both packages appear in the view with accurate, concise descriptions and links.

Written by the indexing model from the issue text.

Description

Two packages that may fit this view, both tidy-style toolkits for text that whitespace-based tooling handles poorly.

tidycjk (0.1.0), for Chinese, Japanese and Korean. CJK writing does not separate words with whitespace, so a standard tokeniser returns either one undifferentiated blob per sentence or isolated characters. It provides cjk_segment() and cjk_tokens() with a pluggable segmenter backend (cjk_segmenters(), register_cjk_segmenter()), script and language identification (cjk_script(), cjk_blocks(), cjk_detect_language(), has_cjk(), cjk_ratio()), counts and summaries (cjk_char_counts(), cjk_summary()), and the width handling that matters whenever CJK text is aligned or truncated (cjk_width(), cjk_pad(), cjk_truncate(), to_fullwidth(), to_halfwidth()).

tidyEmoji (0.4.0), for emoji in text columns. Not every code point is an emoji and a single emoji can span several code points, so emoji statistics are fiddly to get right. It covers extraction (emoji_extract_unnest(), emoji_extract_nest(), emoji_tokens()), counting and density (emoji_frequency(), emoji_density(), emoji_ratio(), emoji_position()), categorisation (emoji_categorize()), sentiment and emotion scoring against published lexicons (emoji_sentiment(), emoji_emotion(), emoji_lexicons(), register_emoji_lexicon()), shortcode conversion both ways for preprocessing and accessibility (as_emoji_shortcode(), as_emoji_name(), emoji_to_text(), text_to_emoji(), emoji_sanitize()), context and collocation (emoji_context(), emoji_collocations(), emoji_ngrams(), emoji_cooccurrence()), and emoji_dfm() for document-by-emoji feature tables. emoji_token_cost() reports what emoji cost under a tokeniser, which matters for LLM preprocessing.

The sentiment lexicon is the Emoji Sentiment Ranking (Kralj Novak et al. 2015, doi:10.1371/journal.pone.0144296).

Disclosure: I am the author of both packages.

Dominant language
No language data
Stars
6
Forks
6
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from cran-task-views/NaturalLanguageProcessing

All issues in cran-task-views/NaturalLanguageProcessing

Similar issues

More Documentation issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.