Performance degradation in concurrent usage with `Promise`, `Promise.all`

Abierto
#766 4 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 3 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
35/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Estancado
Stack tecnológico
javascript
Área
performance

Línea de trabajo

Empieza por benchmark/XmlParserConcurrent.mjs, src/fxp.js y spec/assets/midsize.xml. Ejecuta los benchmarks concurrent y non-concurrent y, después, sigue las rutas del parser utilizadas por XMLParser y el legacy parser. El trabajo estará terminado cuando la ralentización async/concurrent reportada en archivos grandes esté explicada y resuelta, y los resultados de los benchmarks muestren la mejora.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

When implementing XML processing using Promise.all with fast-xml-parser, we encountered significant performance
degradation issues. After conducting benchmark tests, the results suggested that this could be related to how
fast-xml-parser handles concurrent operations, particularly with larger XML files.

Summary

When processing large XML files (e.g., 13MB) concurrently with Promise, Promise.all (10 at the same time), fast-xml-parser is significantly slower than xml2js.
In contrast, for small or lightweight files, fast-xml-parser performs well and even outperforms xml2js.
When running a single parse (no concurrency), fast-xml-parser shows very high throughput, meaning the slowdown only appears with concurrent or async usage.


Reproduction Code

https://github.com/sya-ri/fast-xml-parser/commit/172d53094b2a3324bd8c30e8a53ba72789bb9452

A. Concurrent execution (13MB / 10 at once)

Executed as benchmark/XmlParserConcurrent.mjs.

"use strict";

import Benchmark from "benchmark";
import {XMLParser} from "../src/fxp.js";
import xml2js from "xml2js";
import fxpv3 from "fast-xml-parser";
import { dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
    
// compatibility
const __dirname = dirname(fileURLToPath(import.meta.url));

const suite = new Benchmark.Suite("XML Parser concurrent benchmark");

import fs from "fs";
import path from "path";
const fileNamePath = path.join(__dirname, "../spec/assets/midsize.xml"); // 13MB
const xmlData = fs.readFileSync(fileNamePath).toString();
const promiseSize = 10;

const fxpParser = new XMLParser();
const fxpParserForOrderedJs = new XMLParser({preserveOrder: true});

suite
    .add("fxp v3", {
      defer: true,
      fn: function(deferred) {
        Promise.all(Array.from({ length: promiseSize }).map(async () =>
          fxpv3.parse(xmlData)
        )).then(() => deferred.resolve());
      }
    })
    .add("fxp", {
      defer: true,
      fn: function(deferred) {
        Promise.all(Array.from({ length: promiseSize }).map(async () =>
          fxpParser.parse(xmlData)
        )).then(() => deferred.resolve());
      }
    })
    .add("fxp - preserve order", {
      defer: true,
      fn: function(deferred) {
        Promise.all(Array.from({ length: promiseSize }).map(async () =>
          fxpParserForOrderedJs.parse(xmlData)
        )).then(() => deferred.resolve());
      }
    })
    .add('xml2js ', {
      defer: true,
      fn: function(deferred) {
        Promise.all(Array.from({ length: promiseSize }).map(async () =>
          xml2js.parseStringPromise(xmlData)
        )).then(() => deferred.resolve());
      }
    })

    .on("start", function() {
        console.log("Running Suite: " + this.name);
    })
    .on("error", function(e) {
        console.log("Error in Suite: " + this.name, e);
    })
    .on("abort", function(e) {
        console.log("Aborting Suite: " + this.name, e);
    })
    .on("complete", function() {
        for (let j = 0; j < this.length; j++) {
            console.log(this[j].name + " : " + this[j].hz + " requests/second");
        }
    })
    .run({"async": true});

B. Non-concurrent (wrapped in async)

Executed as benchmark/XmlParserConcurrent.mjs.

"use strict";

import Benchmark from "benchmark";
import {XMLParser} from "../src/fxp.js";
import xml2js from "xml2js";
import fxpv3 from "fast-xml-parser";
import { dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
    
// compatibility
const __dirname = dirname(fileURLToPath(import.meta.url));

const suite = new Benchmark.Suite("XML Parser concurrent benchmark");

import fs from "fs";
import path from "path";
const fileNamePath = path.join(__dirname, "../spec/assets/midsize.xml"); // 13MB
const xmlData = fs.readFileSync(fileNamePath).toString();

const fxpParser = new XMLParser();
const fxpParserForOrderedJs = new XMLParser({preserveOrder: true});

suite
    .add("fxp v3", {
      defer: true,
      fn: function(deferred) {
        (async () =>
          fxpv3.parse(xmlData)
        )().then(() => deferred.resolve());
      }
    })
    .add("fxp", {
      defer: true,
      fn: function(deferred) {
        (async () =>
          fxpParser.parse(xmlData)
        )().then(() => deferred.resolve());
      }
    })
    .add("fxp - preserve order", {
      defer: true,
      fn: function(deferred) {
        (async () =>
          fxpParserForOrderedJs.parse(xmlData)
        )().then(() => deferred.resolve());
      }
    })
    .add('xml2js ', {
      defer: true,
      fn: function(deferred) {
        (async () =>
          xml2js.parseStringPromise(xmlData)
        )().then(() => deferred.resolve());
      }
    })

    .on("start", function() {
        console.log("Running Suite: " + this.name);
    })
    .on("error", function(e) {
        console.log("Error in Suite: " + this.name, e);
    })
    .on("abort", function(e) {
        console.log("Aborting Suite: " + this.name, e);
    })
    .on("complete", function() {
        for (let j = 0; j < this.length; j++) {
            console.log(this[j].name + " : " + this[j].hz + " requests/second");
        }
    })
    .run({"async": true});

Benchmark Results

13MB XML / 10 concurrent
fxp v3 : 0.2865043014339054 requests/second
fxp : 0.16876260944496088 requests/second
fxp - preserve order : 0.1669240166670523 requests/second
xml2js : 1282.977050682912 requests/second

With large XML files under concurrency, fast-xml-parser is drastically slower than xml2js.


1.5KB XML / 10 concurrent
fxp v3 : 3238.1802137107325 requests/second
fxp : 1778.4294041322637 requests/second
fxp - preserve order : 2043.0091729760097 requests/second
xml2js  : 1745.291494844131 requests/second

With small XML, fast-xml-parser performs on par or better than xml2js.


CDATA (lightweight XML) / 10 concurrent
fxp v3 : 9789.883828979968 requests/second
fxp : 6691.052733490235 requests/second
fxp - preserve order : 7227.875838621091 requests/second
xml2js  : 3116.079812611551 requests/second

With lightweight files, fast-xml-parser outperforms xml2js significantly.


13MB XML / single execution (npm run parser)
fxp v3 : 100200.12823920241 requests/second
fxp : 66284.89077189047 requests/second
fxp - preserve order : 71015.28486649034 requests/second
xmlbuilder2 : 29305.972887978354 requests/second
xml2js  : 31336.192966600825 requests/second

In single execution, fast-xml-parser shows excellent performance, faster than xml2js.


13MB XML / non-concurrent (async wrapper)
fxp v3 : 2.807664801903926 requests/second
fxp : 1.8445207259077958 requests/second
fxp - preserve order : 2.0789000186976128 requests/second
xml2js : 13684.611351341111 requests/second

Even with a simple async wrapper, fast-xml-parser slows down drastically, while xml2js maintains good performance.


Notes

  • Compared parsers: fast-xml-parser and xml2js.
    xmlbuilder2 is excluded except in the single execution test.
  • Using xml2js.parseString(xmlData, callback) instead of xml2js.parseStringPromise(xmlData) produced the same results.
Lenguaje dominante
JavaScript
Estrellas
3.1k
Forks
404
Merge medio
2 d 1 h
PR fusionados (30 d)
5

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NaturalIntelligence/fast-xml-parser

Todos los issues de NaturalIntelligence/fast-xml-parser

Issues similares

Más issues de JavaScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.