Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Problems and future plans on token system

Abierto
  • ¿Algo no está bien?
#75 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
20/100
Tipo de issue
Refactorización
Claridad
Necesita aclaración
Estado de actividad
Estancado
Stack tecnológico
react, typescript
Área
frontend, performance

Línea de trabajo

Start with the linked token system documentation and the mentioned files: src/tokenize/utils.js, normalizeToLines.js, pickRanges.js, and backToTree.js. Review the proposed clone, path, and node-shape changes and their breaking-change implications; this issue is complete only after a concrete migration plan and agreed implementation scope are defined.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

pinned

Token system is a key feature of react-diff-view and is the source of its powerfulness.

I looked into current implementation of token system recently and discovered some vulnerabilities about it, mostly about performance.

Too many clones

To ensure token system works safely by default, we introduce a clone function to copy an entire path, that is clone every node inside this path. This function is used in almost every key functions like normalizeToLines and other manipulation utilities.

Actually, while we guarantee that token node is immutable, we can get rid of all these clones, boosting performance considerably.

In current situation we don't have a statement saying "you cannot mutate token node" and removing clone operations can introduce some potential breaking changes, but I think it should be safe in most cases.

Too many slices

Previously we represent a path as an array of token nodes, the last node in this array is always a text node:

[
  {type: 'element'},
  {type: 'element'},
  {type: 'mark'},
  {type: 'text', value: 'Hello World'}
]

In nearly all cases when we need to manipulate this path, we were not changing the text, we frequently insert a node before text node, the implement could be:

const parents = path.slice(0, -1);
const text = path[path.length - 1];
const newPath = [...parents, customNode, text];

We notice a .slice(0, -1) inside this code which can cause some dropdown on performance.

To optimize this, we can separate node path and its containing text into a tuple like:

[
  [
    {type: 'element'},
    {type: 'element'},
    {type: 'mark'},
  ],
  'Hello World'
]

We can insert a node before text more efficiently:

const [parents, text] = path;
const newPath = [[...parents, customNode], text];

In this solution we also unwrap text node into a single string to reduce the creation and garbage collection of node objects.

This is a breaking change since any custom enhancers written previously will fail to work on the new data structure.

Too dynamic nodes

For the lifetime of react-diff-view, we didn't have a strictly defined type for token node, that is, users can put any properties into a node and use them in custom renderToken function.

This is easy to use, but a dynamic node causes more than expected node clones and a poor performance node merge algorithm.

We decide to later define the node more strictly, leaving only type and a dynamic properties for it, the clone and compare can be much simpler:

const cloneNode = node => {
  return {
    type: node.type,
    properties: node.properties,
  }
};

const areNodesEquals = (x, y) => (x.type === y.type && shallowEquals(x.properties, y.properties));

This is also a breaking change since a more strict structure is forced.

Reuse token system

As a result of performance inspection and reusability consideration, I developed a tiny library source-tokenizer to handle these, on the initial implement we get about 25% performance boost:

highlighting a json file with 11029 lines, 10 times
legacy: 22.384s
current: 16.038s

I'm preparing to develop a react-source-view component based on this library to render source codes, react-diff-view which focused on render diff contents will have a major version to use this library in 2020, an upgrade guide will be published then.

Lenguaje dominante
TypeScript
Estrellas
1k
Forks
88
Sin PR fusionados en 30 d

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de otakustay/react-diff-view

Todos los issues de otakustay/react-diff-view

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.