Performance degradation in concurrent usage with `Promise`, `Promise.all`
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- javascript
- Domain
- performance
Research direction
Start with benchmark/XmlParserConcurrent.mjs, src/fxp.js, and spec/assets/midsize.xml. Run the concurrent and non-concurrent benchmarks, then trace the parser paths used by XMLParser and the legacy parser. Done means the reported large-file async/concurrent slowdown is explained and addressed, with benchmark results showing the improvement.
Written by the indexing model from the issue text.
Description
When implementing XML processing using Promise.all with fast-xml-parser, we encountered significant performance
degradation issues. After conducting benchmark tests, the results suggested that this could be related to how
fast-xml-parser handles concurrent operations, particularly with larger XML files.
Summary
When processing large XML files (e.g., 13MB) concurrently with Promise, Promise.all (10 at the same time), fast-xml-parser is significantly slower than xml2js.
In contrast, for small or lightweight files, fast-xml-parser performs well and even outperforms xml2js.
When running a single parse (no concurrency), fast-xml-parser shows very high throughput, meaning the slowdown only appears with concurrent or async usage.
Reproduction Code
https://github.com/sya-ri/fast-xml-parser/commit/172d53094b2a3324bd8c30e8a53ba72789bb9452
A. Concurrent execution (13MB / 10 at once)
Executed as benchmark/XmlParserConcurrent.mjs.
"use strict";
import Benchmark from "benchmark";
import {XMLParser} from "../src/fxp.js";
import xml2js from "xml2js";
import fxpv3 from "fast-xml-parser";
import { dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
// compatibility
const __dirname = dirname(fileURLToPath(import.meta.url));
const suite = new Benchmark.Suite("XML Parser concurrent benchmark");
import fs from "fs";
import path from "path";
const fileNamePath = path.join(__dirname, "../spec/assets/midsize.xml"); // 13MB
const xmlData = fs.readFileSync(fileNamePath).toString();
const promiseSize = 10;
const fxpParser = new XMLParser();
const fxpParserForOrderedJs = new XMLParser({preserveOrder: true});
suite
.add("fxp v3", {
defer: true,
fn: function(deferred) {
Promise.all(Array.from({ length: promiseSize }).map(async () =>
fxpv3.parse(xmlData)
)).then(() => deferred.resolve());
}
})
.add("fxp", {
defer: true,
fn: function(deferred) {
Promise.all(Array.from({ length: promiseSize }).map(async () =>
fxpParser.parse(xmlData)
)).then(() => deferred.resolve());
}
})
.add("fxp - preserve order", {
defer: true,
fn: function(deferred) {
Promise.all(Array.from({ length: promiseSize }).map(async () =>
fxpParserForOrderedJs.parse(xmlData)
)).then(() => deferred.resolve());
}
})
.add('xml2js ', {
defer: true,
fn: function(deferred) {
Promise.all(Array.from({ length: promiseSize }).map(async () =>
xml2js.parseStringPromise(xmlData)
)).then(() => deferred.resolve());
}
})
.on("start", function() {
console.log("Running Suite: " + this.name);
})
.on("error", function(e) {
console.log("Error in Suite: " + this.name, e);
})
.on("abort", function(e) {
console.log("Aborting Suite: " + this.name, e);
})
.on("complete", function() {
for (let j = 0; j < this.length; j++) {
console.log(this[j].name + " : " + this[j].hz + " requests/second");
}
})
.run({"async": true});
B. Non-concurrent (wrapped in async)
Executed as benchmark/XmlParserConcurrent.mjs.
"use strict";
import Benchmark from "benchmark";
import {XMLParser} from "../src/fxp.js";
import xml2js from "xml2js";
import fxpv3 from "fast-xml-parser";
import { dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
// compatibility
const __dirname = dirname(fileURLToPath(import.meta.url));
const suite = new Benchmark.Suite("XML Parser concurrent benchmark");
import fs from "fs";
import path from "path";
const fileNamePath = path.join(__dirname, "../spec/assets/midsize.xml"); // 13MB
const xmlData = fs.readFileSync(fileNamePath).toString();
const fxpParser = new XMLParser();
const fxpParserForOrderedJs = new XMLParser({preserveOrder: true});
suite
.add("fxp v3", {
defer: true,
fn: function(deferred) {
(async () =>
fxpv3.parse(xmlData)
)().then(() => deferred.resolve());
}
})
.add("fxp", {
defer: true,
fn: function(deferred) {
(async () =>
fxpParser.parse(xmlData)
)().then(() => deferred.resolve());
}
})
.add("fxp - preserve order", {
defer: true,
fn: function(deferred) {
(async () =>
fxpParserForOrderedJs.parse(xmlData)
)().then(() => deferred.resolve());
}
})
.add('xml2js ', {
defer: true,
fn: function(deferred) {
(async () =>
xml2js.parseStringPromise(xmlData)
)().then(() => deferred.resolve());
}
})
.on("start", function() {
console.log("Running Suite: " + this.name);
})
.on("error", function(e) {
console.log("Error in Suite: " + this.name, e);
})
.on("abort", function(e) {
console.log("Aborting Suite: " + this.name, e);
})
.on("complete", function() {
for (let j = 0; j < this.length; j++) {
console.log(this[j].name + " : " + this[j].hz + " requests/second");
}
})
.run({"async": true});
Benchmark Results
13MB XML / 10 concurrent
fxp v3 : 0.2865043014339054 requests/second
fxp : 0.16876260944496088 requests/second
fxp - preserve order : 0.1669240166670523 requests/second
xml2js : 1282.977050682912 requests/second
With large XML files under concurrency, fast-xml-parser is drastically slower than xml2js.
1.5KB XML / 10 concurrent
fxp v3 : 3238.1802137107325 requests/second
fxp : 1778.4294041322637 requests/second
fxp - preserve order : 2043.0091729760097 requests/second
xml2js : 1745.291494844131 requests/second
With small XML, fast-xml-parser performs on par or better than xml2js.
CDATA (lightweight XML) / 10 concurrent
fxp v3 : 9789.883828979968 requests/second
fxp : 6691.052733490235 requests/second
fxp - preserve order : 7227.875838621091 requests/second
xml2js : 3116.079812611551 requests/second
With lightweight files, fast-xml-parser outperforms xml2js significantly.
13MB XML / single execution (npm run parser)
fxp v3 : 100200.12823920241 requests/second
fxp : 66284.89077189047 requests/second
fxp - preserve order : 71015.28486649034 requests/second
xmlbuilder2 : 29305.972887978354 requests/second
xml2js : 31336.192966600825 requests/second
In single execution, fast-xml-parser shows excellent performance, faster than xml2js.
13MB XML / non-concurrent (async wrapper)
fxp v3 : 2.807664801903926 requests/second
fxp : 1.8445207259077958 requests/second
fxp - preserve order : 2.0789000186976128 requests/second
xml2js : 13684.611351341111 requests/second
Even with a simple async wrapper, fast-xml-parser slows down drastically, while xml2js maintains good performance.
Notes
- Compared parsers: fast-xml-parser and xml2js.
xmlbuilder2 is excluded except in the single execution test. - Using
xml2js.parseString(xmlData, callback)instead ofxml2js.parseStringPromise(xmlData)produced the same results.
- Dominant language
- JavaScript
- Stars
- 3.1k
- Forks
- 398
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 1
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NaturalIntelligence/fast-xml-parser
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
NaturalIntelligence/fast-xml-parser#890 · 1 comment ·
Maintainers usually reply within 3 days
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
NaturalIntelligence/fast-xml-parser#888 · 2 comments ·
Maintainers usually reply within 3 days
-
ignoreDeclaration does not ignore lower case doctype declarationMay be free again A pull request for this issue was closed without being merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NaturalIntelligence/fast-xml-parser#870 · 2 comments ·
Maintainers usually reply within 3 days
-
Enhance type definition for XMLParser `isArray` option when using TypeScriptMay be free again A pull request for this issue was closed without being merged. Open
Difficulty 3/5 1-2 days Newbie friendliness 52/100
NaturalIntelligence/fast-xml-parser#803 · 21 comments · 2 reactions ·
Maintainers usually reply within 3 days
-
Feature-Request
Difficulty 4/5 3-5 days Newbie friendliness 52/100
NaturalIntelligence/fast-xml-parser#779 · 9 comments ·
Maintainers usually reply within 3 days
All issues in NaturalIntelligence/fast-xml-parser
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
[quality] Nothing keeps docs/skills/manifest.md in sync with docs/skills/*.mdPossibly taken @hivecommons-hive claimed this today. Openagent/quality hive/covered-by-pr hive/hosted-available-lke648397-260827-5n31 quality testing
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
agent:blocked agent:ready
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
turbot/guardrails-fn#46 · 2 comments ·
Maintainers usually reply within 1 day
-
needs verification
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 6 days