Nginx with Modsecurity: POST request gives 500 error

[pijnack]

Nginx with Modsecurity gives 500 errors after a POST request.

In error.log:

2013/10/23 13:21:07 [alert] 29432#0: *4 no upstream configuration, client: 88.159.16.136, server: test.huygenscollege.nl, request: "POST /proefwerk/prog/index.php?day=28&month=10&year=2013&klas=2 HTTP/1.1", host: "test.huygenscollege.nl", referrer: "http://test.huygenscollege.nl/proefwerk/prog/index.php?day=4&month=11&year=2013&klas=2"

In nginx.conf:

upstream smoothrac {
server 88.159.13.153;
}
server {
listen 80;
server_name test.huygenscollege.nl;
location / {
ModSecurityEnabled on;
ModSecurityConfig modsecurity.conf;
proxy_pass http://smoothrac/;
proxy_set_header Host test.huygenscollege.nl;
proxy_read_timeout 150s;
}
}

After changing ModSecurityEnabled on to off, the problem is gone.
I guess something is missing in my upstream configuration?

Piet Pijnacker,
Eindhoven, Netherlands

[zimmerle]

Hi @pijnack, I am reopening this issue, I'm investigating it.

[zimmerle]

Wordpress set multiple cookies in the very same request, which is a issue in our Nginx implementation as report at #154. It is not directly related to this issue, but it is somehow related in the sense that it does affect the behavior of Wordpress.

[yumahil]

I also had two issues regarding "POST request 500 error" and only one set-cookie.
The Fix #154 solved only one set-cookie problem. But "POST request 500 error" still remains.

[jijojv]

Same issue here , no upstream or proxy setting used. just nginx php-fpm.

Even more weird is this only triggers when 'Host: ' header is set - using browser or curl etc
So using this http://wiki.nginx.org/HttpHeadersMoreModule - "more_clear_input_headers 'Host';" is one workaround for some folks where the application doesn't need the host headers.

[shiguti]

I've got the same issue, any ideas?

[infinitnet]

Same issue for me with IPB. Latest NGINX and ModSecurity version and ASL rules. If SecRequestBodyAccess is enabled, it will give a 500 error with POST requests while writing nothing useful on the ModSecurity logs, even with debug level 9 and in the NGINX logs it only produces the lines mention in OP. This issue was reported in #115 10 months ago and nobody took care of it? I can provide server access to one of the devs if no one is able to reproduce it - it's a non-production environment.

[zimmerle]

Hi @infinitnet, there are some problems with the way that ModSecurity is handling the post requests on nginx. If you look at #142 you will be able to see that contents over 8k are causing problems as well. Our version 2.8.0-RC1 contain a fix related to the issue #645. But we still have problems while the request body is bigger than a certain amount of data. The main reason for the problem is that ModSecrurity is not assembling Nginx chunks right, the function is here:

https://github.com/SpiderLabs/ModSecurity/blob/master/nginx/modsecurity/ngx_http_modsecurity.c#L1011-L1020

In certain cases (e.g. when the request body content is not complete) this function should return NGX_AGAIN, so that nginx will call it again with the rest of the body data.

[infinitnet]

@zimmerle I'm actually using 2.8.0-RC1 already. I didn't check the POST data size, but the issue appears even with simple login forms, rendering ModSecurity pretty much unusable with NGINX.

[10/Apr/2014:16:19:03 +0200] "POST /index.php?app=core&module=global&section=login&do=process HTTP/1.1" 500

[alert] 19480#0: *4 no upstream configuration, client: 1.2.3.4, server: domain.com, request: "POST /index.php?app=core&module=global&section=login&do=process HTTP/1.1"

And nothing useful in the audit or debug log.

[infinitnet]

@zimmerle I should have checked dmesg earlier, actually, it's indeed somehow related to APR (my version of libapr1(-dev) is 1.4.6-3+deb7u1), as I'm getting segfaults for each NGINX worker when starting NGINX:

Apr 10 17:01:13 a143 kernel: [787456.369365] nginx[19482]: segfault at 7fe879c61098 ip 00007fe88484792d sp 00007fff04fc4b90 error 4 in libapr-1.so.0.4.6[7fe88482d000+30000]
Apr 10 17:01:13 a143 kernel: [787456.374651] nginx[19480]: segfault at 7fe879c61098 ip 00007fe88484792d sp 00007fff04fc4b90 error 4 in libapr-1.so.0.4.6[7fe88482d000+30000]
Apr 10 17:01:13 a143 kernel: [787456.375342] nginx[19481]: segfault at 7fe879c61098 ip 00007fe88484792d sp 00007fff04fc4b90 error 4 in libapr-1.so.0.4.6[7fe88482d000+30000]
Apr 10 17:01:13 a143 kernel: [787456.377620] nginx[19483]: segfault at 7fe879c61098 ip 00007fe88484792d sp 00007fff04fc4b90 error 4 in libapr-1.so.0.4.6[7fe88482d000+30000]

[zimmerle]

@infinitnet This happens when you start nginx or whenever you receive a new request?

[infinitnet]

@zimmerle Updated my comment, it happens for each NGINX worker process when starting NGINX, not with each request and also not when I get the 500 error, only when starting it.