restify/node-restify

bodyParser ignores maxBodySize and buffers whole file in memory for file uploads

Open

#956 opened on Nov 23, 2015

View on GitHub
 (5 comments) (2 reactions) (0 assignees)JavaScript (10,695 stars) (983 forks)batch import
BugCriticalGood First IssueHelp Wanted

Description

The bodyParser middleware seems to buffer the whole file in memory for file uploads.

Check the following minimum example:

'use strict';

var os = require( 'os' );
var restify = require( 'restify' );

var server = restify.createServer( {} );

server.use( restify.bodyParser( {
  // this should supposedly limit maximum upload size to 10 KiB
  maxBodySize: 10 * 1024,
  mapParms: true,
  mapFiles: true,
  keepExtensions: true,
  uploadDir: os.tmpdir()
} ) );

server.get( '/', function( req, res, next ) {
  res.json( { message: 'hello' } );
  next();
} );

server.post( '/upload', function( req, res, next ) {
  console.log( 'Received files:', req.files );
  res.send( 200 );
  next();
} );

server.listen( 8000 );

Then, upload a really big file to it:

# bigfile.dat has a couple of GiB
curl -F "file=@/path/to/bigfile.dat" localhost:8000/upload

This will send the process into accumulating tons of memory until it eventually runs out of memory and crashes with:

FATAL ERROR: JS Allocation failed - process out of memory

Am I making a mistake in setting up the middleware or is this a bug?

Contributor guide

Tech stack
javascript
Domain
backendapi
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Node.js basicsrestify middleware knowledgeunderstanding of file uploads
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
The issue is that bodyParser ignores maxBodySize for file uploads, causing the entire file to be buffered in memory. The fix likely involves modifying the bodyParser plugin (e.g., lib/plugins/bodyParser.js) to check maxBodySize before or during streaming. Investigate how the plugin currently processes uploads and where the size limit should be enforced. Check the multipart/form data parsing logic and ensure that the size is checked against the upload stream, rejecting early if exceeded. Also verify that the maxBodySize option is properly passed to the underlying parser (like formidable or busboy).