restify/node-restify

bodyParser ignores maxBodySize and buffers whole file in memory for file uploads

Open

#956 aberto em 23 de nov. de 2015

Ver no GitHub
 (5 comments) (2 reactions) (0 assignees)JavaScript (10.695 stars) (983 forks)batch import
BugCriticalGood First IssueHelp Wanted

Description

The bodyParser middleware seems to buffer the whole file in memory for file uploads.

Check the following minimum example:

'use strict';

var os = require( 'os' );
var restify = require( 'restify' );

var server = restify.createServer( {} );

server.use( restify.bodyParser( {
  // this should supposedly limit maximum upload size to 10 KiB
  maxBodySize: 10 * 1024,
  mapParms: true,
  mapFiles: true,
  keepExtensions: true,
  uploadDir: os.tmpdir()
} ) );

server.get( '/', function( req, res, next ) {
  res.json( { message: 'hello' } );
  next();
} );

server.post( '/upload', function( req, res, next ) {
  console.log( 'Received files:', req.files );
  res.send( 200 );
  next();
} );

server.listen( 8000 );

Then, upload a really big file to it:

# bigfile.dat has a couple of GiB
curl -F "file=@/path/to/bigfile.dat" localhost:8000/upload

This will send the process into accumulating tons of memory until it eventually runs out of memory and crashes with:

FATAL ERROR: JS Allocation failed - process out of memory

Am I making a mistake in setting up the middleware or is this a bug?

Guia do colaborador

Pilha de tecnologia
javascript
Domain
backendapi
Tipo Issue
bug
DifficultyDificuldade de implementação estimada para um novo contribuidor, de 1 para alterações muito pequenas a 5 para trabalho de nível especializado.
3
Tempo estimadoUm intervalo de tempo aproximado para um colaborador experiente investigar, implementar, testar e preparar um pull request.
1-3 hours
Status da atividadeQuão disponível o issue aparece agora: novo, ativo, obsoleto, bloqueado ou aguardando entrada do mantenedor.
stale
ClarityCom que clareza o issue explica a mudança esperada, os critérios de aceitação e a próxima etapa.
clear
Prerequisites
Node.js basicsrestify middleware knowledgeunderstanding of file uploads
Simpatia para novatosUma pontuação de 1 a 100 que estima o quão acessível este issue é para colaboradores iniciantes.
40
Direção de pesquisa
The issue is that bodyParser ignores maxBodySize for file uploads, causing the entire file to be buffered in memory. The fix likely involves modifying the bodyParser plugin (e.g., lib/plugins/bodyParser.js) to check maxBodySize before or during streaming. Investigate how the plugin currently processes uploads and where the size limit should be enforced. Check the multipart/form data parsing logic and ensure that the size is checked against the upload stream, rejecting early if exceeded. Also verify that the maxBodySize option is properly passed to the underlying parser (like formidable or busboy).