Temporary file is deleted before @inspectFile is triggered if SecUploadKeepFiles set to Off and SecTmpSaveUploadedFiles set to On

[dbaynak-cl]

Bug description

If:

• SecUploadKeepFiles is disabled
• SecTmpSaveUploadedFiles is enabled
• a multipart file is uploaded,

Then the file is deleted immediately and no rules could observe this file - this behaviour looks like a bug for me (because of the 2nd item in additional context and the fact that ModSecurity v2.9 behaved as expected in such situation).

Log, configs, rules

ModSecurity configuration file:

SecRequestBodyAccess On
SecTmpSaveUploadedFiles On
SecResponseBodyLimitAction ProcessPartial
SecTmpDir /var/<dir>/tmp_modsec
SecUploadDir /var/<dir>/tmp_modsec

Related debug log:

[1602836810] [/accept.php] [9] Multipart: Content-Disposition filename: eicar.com.
[1602836810] [/accept.php] [4] Multipart: Created temporary file 1 (mode o0): /var/<dir>/tmp_modsec/20201016-082650-1602836810-file-xrFoHi
[1602836810] [/accept.php] [9] Multipart: Added file part to the list: name "file" file name "eicar.com" (offset 105, length 68)
[1602836810] [/accept.php] [4] Multipart: Cleanup started (keep files set to Not set)
[1602836810] [/accept.php] [4] Multipart: file deleted successfully (part) "/var/<dir>/tmp_modsec/20201016-082650-1602836810-file-xrFoHi"
....
[1602836810] [/accept.php] [4] (Rule: 33331) Executing operator "InspectFile" with param "<python script>" against FILES_TMPNAMES.
[1602836810] [/accept.php] [9] Target value: "/var/<dir>/tmp_modsec/20201016-082650-1602836810-file-xrFoHi" (Variable: FILES_TMPNAMES:/var/<dir>/tmp_modsec/20201016-082650-1602836810-file-xrFoHi)
[1602836810] [/accept.php] [4] Rule returned 0.
[1602836810] [/accept.php] [9] Matched vars cleaned.

Rule:

SecRule FILES_TMPNAMES "@inspectFile <python script>" \
    "id:33331,\
    log,\
    auditlog,\
    deny,\
    severity:2,\
    phase:2,\
    t:none,\
    msg:'...',\
    ctl:RuleEngine=On"

Note 1: <python script> is some python script with a shebang line in it.
Note 2: <dir> is a valid (rwx), existing directory in /var.

To Reproduce
Prerequisites:

• SecRule FILES_TMPNAMES "@inspectFile ",
• SecTmpSaveUploadedFiles set to On,
• SecUploadKeepFiles set to Off.

Test:

• upload a multipart file,
• should receive its path, but the file on this path is deleted - and this is a bug, I believe.

Expected behavior

The uploaded file should be deleted after all rules have checked it, not before.

Server:

• ModSecurity v3.0.4 (master, effectively) with nginx-connector v1.0.1 (master, effectively),
• nginx-1.18.0,
• Linux, CentOS 7.

Additional context

1. If we add SecUploadKeepFiles On - everything works as expected (but the file is kept in SecTmpDir after processing, which we want to avoid).
2. This behavior seems to be triggered by a combination of:

• branch with multipart
• actual rules call
• multipart destructor

so the destructor is called when the branch finishes (which performs a cleanup), but the place where rules are actually evaluated is outside of this branch, so the file gets deleted.

[martinhsv]

Agreed. Clearly a bug.

[zimmerle]

fixed at d72be1c by @martinhsv