Modsec can OOM nginx in a container with large response bodies

[jimbobhickville]

I have the latest version of nginx, modsecurity, and modsecurity-nginx built in a containerized environment. It seems that modsecurity is buffering responses in memory even when looking at responses is disabled (i.e. SecResponseBodyAccess Off), causing the nginx container to be killed by the OS (Memory cgroup out of memory: Kill process 2553 (nginx) score 1990 or sacrifice child).

If I reconfigure nginx with modsecurity off I'm able to download the large file without running out of memory, so it's not nginx itself that's the culprit here.

Steps to reproduce:

1. Run nginx+modsec dynamic module with a cgroup memory limit.
2. Download a file through nginx that is larger than that memory limit.
3. When the amount downloaded exceeds the memory limit, the worker process is killed by the OS.

I should note that I was hitting a similar issue with the statically-compiled version of nginx+modsec, but it would OOM before the download even started. I had hoped that this dynamic module would behave better. It does, in some sense, in that it will start to download, but it still OOMs after the amount downloaded exceeds the memory limit.

I'm happy to help debug this issue further as it's breaking websites that we host, but I'm not much of a C programmer, so I could use some pointers on how to go about helping.

[victorhora]

Hi @jimbobhickville,

Have you tried tuning SecResponseBodyLimit and related directives to see if it makes a difference?

[jimbobhickville]

I can try tomorrow, but the docs said the default was 512K, and I'm talking about hundreds of megabytes or gigabytes of memory being used. So either the docs are wrong or that setting is also not being obeyed.

[jimbobhickville]

This is also an application/octet-stream response which is not in the default list of content-types that are supposed to be inspected, according to the docs.

[jimbobhickville]

Well, I'll be darned, adding:

SecResponseBodyLimit 1
SecResponseBodyMimeType text/plain text/html

Fixed it. So there's a bug still in that it's not obeying the defaults on those settings and it's ignoring the whole Off setting, but I can work around it for now. Thanks.

[victorhora]

Hi @jimbobhickville,

We appreciate the feedback :)

This is interesting. Afaik, SecResponseBodyAccess off should be enough to not buffer output body. You've mentioned you're running the latest codebase of ModSecurity (and the connector) right? (Just to be on the safe side, can you confirm you also have #1643 patch on your codebase?)

There is a known issue with Response Body processing discussed at #84 which might be related with the issue you're facing that's worth having a look.

Also, I've just noticed this comment on ModSecurity-nginx which I'm wondering if it might explain the issue still persisting even with SecResponseBodyAccess disabled.

Anyways, as there might be a bug there indeed I'm tagging this one properly and leaving it open for now until we have more time to look into it.

[zimmerle]

The default values are not actually part of the code, but in the recommended configuration:

In the config we can find:

# Buffer response bodies of up to 512 KB in length.
SecResponseBodyLimit 524288

Without it, there is no limit.

[astrcomp]

ModSecurity-nginx v1.0.3 (rules loaded inline/local/remote: 0/0/0)
nginx/1.22.1
OS: Linux 5.4.0-137-generic

I ran my app in container owasp/modsecurity:3.0.8-alpine-202210230310 with 1 GB memory limit and with parameter modsecurity on . If I upload a file larger than 1 GB (simple multipart upload file, nginx worker save files in /var/cache/nginx/client_temp/), nginx worker was killed by OS and in dmesg Memory cgroup out of memory: Killed process 771663 (nginx). Files smaller than 1GB are upload fine. If I set parameter modsecurity off all files upload fine. Is this the normal operation of the program? How I could configure modsecurity or nginx for upload large file?

[martinhsv]

Hello @astrcomp ,

Your question is a little different from what this closed issue was dealing with.

In any case, I'm not sure I have a good suggestion for you. If you are retaining the uploaded files via SecUploadKeepFiles, the code basically assumes that you want to examine the content with FILES_TMP_CONTENT -- which means reading the content into memory. It seems likely that you're hitting your memory limit.

There are some options, but whether they might work for you would depend on what your needs are. For example:

• if you don't need to have rules that examine the file content, then perhaps consider turning off that functionality (performance is likely to be poor for such very large files anyway)
• if you do need to examine the file content, but only sometimes, and only for smaller files, you could consider using different location blocks and not activate the functionality for the block that receives large files

As I said, it depends on what your current configuration is and what your needs are. But those are a couple of possibilities anyway.