curl with PUT method can upload a huge file (>2GB) on remote server but hangs and timeout

[jonathansmith9485]

Describe the bug
When we want to upload a huge file (>2GB) to a remote web site (Artifactory in our case) , our http client hangs indefinitely.
However, the file is correctly uploaded to Artifactory.

Logs and dumps
Output of:

1. DebugLogs (level 9), cf attachments
2. AuditLogs : this file is empty
3. Error logs :

To Reproduce
On a GNU/Linux machine do :

dd if=/dev/urandom of=/tmp/2G.txt bs=1M count=2048
curl -u <myuser>:<my_api_key> -T /tmp/2G.txt 'https://artifactory.local/artifactory/generic/perf_debug/2G.txt

Then, the curlhangs.

Expected behavior
curl ends whithout error.

Server :

• ModSecurity version (and connector): ModSecurity v3.0.12 and 3.0.14 with nginx-connector v1.0.3
• WebServer: nginx-1.18.0
• OS: GNU/Linux, Ubuntu 22.04

Rule Set (please complete the following information):

• Running any public or commercial rule set? Core Security Rules, version 4.5.0

Additional context

• The infrastructure is made like : client -> haproxy -> nginx(+ modesec) -> Artifactory
• More than the file size , the time to upload is more than 250 seconds when the upload fails
• The problem persists when we disable Core Security Rules usage
• The only way to avoid the problem is to set modsecurity off; in my vhost configuration
• Artifactory return a HTTP 201
• HTTP method used is PUT
• strace -> see attachments
• TCP segment missing when modsec is enabled (on nginx machine, interface connected to haproxy machine) :

69719  272.819868  172.16.3.13  172.16.3.12  TCP  66  443 → 36912 [FIN, ACK] Seq=6734 Ack=2104577102 Win=2021120 Len=0 TSval=55543961 TSecr=2937796858

Attachments:

• 2000m_debug.log
• final_strace.log

[airween]

Hi @jonathansmith9485,

thanks for reporting this issue. First of all, could you show us your relevant config, especially SecRequestBodyLimit?

These lines are a bit suspicious:

[174222454310.517319] [/artifactory/generic/perf-debug/2000m.txt] [9] Adding request body: 2097152000 bytes. Limit set to: -2147483648.000000
[174222454310.517319] [/artifactory/generic/perf-debug/2000m.txt] [9] Appending request body: 2097152000 bytes. Limit set to: -2147483648.000000

mainly the Limit set to: a negative number...

[jonathansmith9485]

Hello @airween and thanks for the answer ,
Here is the vhost configuration template we use, including modsec parameters :

vhost_template.conf.txt

[airween]

Thanks. And what is in /etc/nginx/modsec/main.conf?

[jonathansmith9485]

Here are the files (crs-setup.conf and modsecurity.conf are imported from main.conf) :

• main.conf.txt
• crs-setup.conf.txt
• modsecurity.conf.txt

[airween]

Hi @jonathansmith9485,

thanks for the config parts above.

I tried to reproduce your issue, and I think I successfully did it - but unfortunately I couldn't find the cause yet.

I suspect a connector/nginx problem rather than libmodsecurity3. See the last line in your debug.log:

Return code `200' is not interesting to audit logs, relevant code(s): `^(?:5|4(?!04|01))'.

This is typically the last line when audit.log is generating.

Unfortunately this is a very unique issue, I'm not sure I have enough time to inspect this more. I can help you and try to help you, but - really sorry - this demands more time. If you can join to Slack (on channel #project-modsecurity), then I can help you online (if I'm ready).

[jonathansmith9485]

Hi @airween and thanks for your answer.
Don't worry, I can understand that this bug isn't obvious to solve, I'll stay available for further tests if needed.