hexojs/hexo

Out of Memory on 33103 html and with image 671111 regular files

Open

#5552 opened on Sep 30, 2024

View on GitHub
 (10 comments) (0 reactions) (0 assignees)TypeScript (41,376 stars) (5,017 forks)batch import
#perfmattershelp wanted

Description

Check List

  • I have already read Docs page & Troubleshooting page.
  • I have already searched existing issues and they are not help to me.
  • I examined error or warning messages and it's difficult to solve.
  • I am using the latest version of Hexo. (run hexo version to check)
  • My Node.js is matched the required version.

Expected behavior

Run:

hexo server

or

hexo generate

And the site is generated in record time :)

Actual behavior

Tried #2165 and no luck.

Get out of memory error as such:

Ok, trying on 33103 files and getting fail at 16 gb

#!/usr/bin/env node --max_old_space_size=16384

in node_modules/hexo-cli/bin/hexo

Error:

<--- Last few GCs --->

[137:0x7964cb02b000]    33039 ms: Scavenge (reduce) (interleaved) 3446.1 (3671.1) -> 3446.1 (3668.1) MB, pooled: 0 MB, 8.24 / 0.00 ms  (average mu = 0.274, current mu = 0.252) external memory pressure; 
[137:0x7964cb02b000]    33791 ms: Mark-Compact (reduce) 3446.3 (3668.1) -> 3444.6 (3648.6) MB, pooled: 0 MB, 96.37 / 0.00 ms  (+ 649.8 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 789 ms) (average mu = 0.262, 

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

Aborted (core dumped)

AND, here is the error at 32gb

<--- Last few GCs --->

[148:0x7aed4cf6f000]    34166 ms: Scavenge (reduce) (interleaved) 3444.6 (3660.1) -> 3444.6 (3657.1) MB, pooled: 0 MB, 8.24 / 0.00 ms  (average mu = 0.262, current mu = 0.248) external memory pressure; 
[148:0x7aed4cf6f000]    34920 ms: Mark-Compact (reduce) 3444.7 (3657.1) -> 3443.1 (3638.9) MB, pooled: 0 MB, 99.86 / 0.00 ms  (+ 648.2 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 791 ms) (average mu = 0.256, 

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

Aborted (core dumped)

How to reproduce?

  1. hexo generate

Is the problem still there under Safe mode?

Yes

Your Node.js & npm version

node -v && npm -v

v22.9.0
10.8.3

Your Hexo and Plugin version

/hexo # npm ls --depth 0
hexo-site@0.0.0 /hexo
+-- hexo-bridge@2.0.3
+-- hexo-generator-archive@2.0.0
+-- hexo-generator-category@2.0.0
+-- hexo-generator-index@4.0.0
+-- hexo-generator-tag@2.0.0
+-- hexo-renderer-ejs@2.0.0
+-- hexo-renderer-marked@6.3.0
+-- hexo-renderer-stylus@3.0.1
+-- hexo-server@3.0.0
+-- hexo-theme-landscape@1.0.0
`-- hexo@7.3.0

Your package.json

{
  "name": "hexo-site",
  "version": "0.0.0",
  "private": true,
  "scripts": {
    "build": "hexo generate",
    "clean": "hexo clean",
    "deploy": "hexo deploy",
    "server": "hexo server"
  },
  "hexo": {
    "version": "7.3.0"
  },
  "dependencies": {
    "hexo": "^7.3.0",
    "hexo-bridge": "^2.0.3",
    "hexo-generator-archive": "^2.0.0",
    "hexo-generator-category": "^2.0.0",
    "hexo-generator-index": "^4.0.0",
    "hexo-generator-tag": "^2.0.0",
    "hexo-renderer-ejs": "^2.0.0",
    "hexo-renderer-marked": "^6.3.0",
    "hexo-renderer-stylus": "^3.0.1",
    "hexo-server": "^3.0.0",
    "hexo-theme-landscape": "^1.0.0"
  }
}

Your site's _config.yml (Optional)

No response

Others

No response

Contributor guide