Multipart: Final boundary missing.

[tomsommer]

I have a customer uploading a 200mb mp4 file through AJAX in Drupal.

With mod_security2 enabled, this results in a stopped 57% upload. The following is visible in the debug log:

[26/Jun/2017:09:55:17 +0200] [XXXX][1] Request body (Content-Length) is larger than the configured limit (134217728).
[26/Jun/2017:09:55:17 +0200] [XXXX][4] Second phase starting (dcfg 6d31318).
[26/Jun/2017:09:55:17 +0200] [XXXX][1] Request body (Content-Length) is larger than the configured limit (134217728).
[26/Jun/2017:09:55:17 +0200] [XXXX][4] Input filter: Reading request body.

....

[26/Jun/2017:09:55:18 +0200] [XXXXlZepdfwwbDzafVgPaKmpJnA][9] Input filter: Bucket type HEAP contains 8000 bytes.
[26/Jun/2017:09:55:18 +0200] [XXXXlZepdfwwbDzafVgPaKmpJnA][9] Input filter: Bucket type HEAP contains 192 bytes.
[26/Jun/2017:09:55:18 +0200] [XXXXlZepdfwwbDzafVgPaKmpJnA][9] Input filter: Bucket type HEAP contains 6408 bytes.
[26/Jun/2017:09:55:18 +0200] [XXXXlZepdfwwbDzafVgPaKmpJnA][9] Input filter: Bucket type HEAP contains 7300 bytes.
[26/Jun/2017:09:55:18 +0200] [XXXXlZepdfwwbDzafVgPaKmpJnA][9] Input filter: Bucket type HEAP contains 7300 bytes.
[26/Jun/2017:09:55:18 +0200] [XXXXlZepdfwwbDzafVgPaKmpJnA][4] Multipart parsing error: Multipart: Final boundary missing.
[26/Jun/2017:09:55:18 +0200] [XXXXlZepdfwwbDzafVgPaKmpJnA][4] Input filter: Completed receiving request body (length 134219089).
[26/Jun/2017:09:55:18 +0200] [XXXXlZepdfwwbDzafVgPaKmpJnA][4] Starting phase REQUEST_BODY.
[26/Jun/2017:09:55:18 +0200] [XXXXlZepdfwwbDzafVgPaKmpJnA][9] This phase consists of 0 rule(s).
[26/Jun/2017:09:55:18 +0200] [XXXXlZepdfwwbDzafVgPaKmpJnA][4] Hook insert_filter: Adding input forwarding filter (r 6d14150).
[26/Jun/2017:09:55:18 +0200] [XXXXlZepdfwwbDzafVgPaKmpJnA][4] Hook insert_filter: Adding output filter (r 6d14150).

The length in the debug is wrong, which is probably why the boundary is missing. Why does the upload stop at 134219089 bytes? The file is 243.376.128 bytes

It looks like it does partial processing, but that's not good when the partial processing results in a boundary error which kills the request? There should be a SecRequestBodyLimitAction Skip

[zimmerle]

Hi @tomsommer,

Please check your ModSecurity configuration file, there is this SecRequestBodyLimit that you may want to double check the value. You can check it here: https://github.com/SpiderLabs/ModSecurity/blob/v2/master/modsecurity.conf-recommended#L38

[tomsommer]

I already did, but the fact that that might not be enough is not good. Basically modsecurity will break any 1GB+ uploads because you cannot disable partial processing?

[victorhora]

Hummm @tomsommer I know there were some issues related to that on older versions of ModSecurity but I haven't heard of such issue with current builds.

Maybe check if the customer is running 2.9.1 or later build from master and also making sure that APR and other dependencies are ok (e.g. loading different version of APR from the build can lead to weird issues). You can check that on the first few lines of the error.log on startup.

I would also suggest checking and experimenting with the following directives which could impact with large file uploads:

SecRequestBodyAccess
SecRequestBodyInMemoryLimit
SecRequestBodyLimit
SecRequestBodyLimitAction

Note there's also a hard limit of 1GiB for the maximum request body size ModSecurity will accept for buffering, excluding the size of any files being transported in the request. This can be tuned with SecRequestBodyNoFilesLimit.
Even though your issue happens when you're actually uploading a file, I'm wondering if for any reason you might be hitting this limit when you're trying your file upload...

[tomsommer]

Running modsecurity-2.9.1, apr-1.5.2, apr-util-1.5.4, httpd-2.4.25

The problem is that the limit cuts off the request and so processing is done on what is naturally a broken request.

If you set SecRequestBodyLimit to something like 1kb and start uploading, it would in theory be reproducible?

[tomsommer]

Also a problem in 2.9.3

There is no way to bypass this check: https://github.com/SpiderLabs/ModSecurity/blob/b392a1ca36181a786a6d68b6eab57a8bb0bd558e/src/request_body_processor/multipart.cc#L1050

[victorhora]

Reopening for further investigation. Thanks for the report @tomsommer.