SecRuleUpdateTargetById does not work with regex for ARGS_NAMES, ARGS_GET_NAMES, ARGS_POST_NAMES

[lontchianicet]

Describe the bug

After migrating nginx.plus from R24 to R25, false positives were observed again. Especially those that were processed with the "SecRuleUpdateTargetById".

We came to the conclusion that the SecRuleUpdateTargetById directive does not work with a Regex.

Example: SecRuleUpdateTargetById 930120 "!ARGS_NAMES:/^json.*\.profile.*/"

Logs and dumps

Output of:
AuditLogs

---EQGotKRA---A--
[15/Oct/2021:10:32:05 +0200] 1634286725 127.0.0.1 42748 127.0.0.1 6087

---EQGotKRA---H--
ModSecurity: Warning. Matched "Operator PmFromFile' with parameter lfi-os-files.data' against variable ARGS_NAMES' (Value: json.userProfiles.array_0.profileName.de' ) [file "/crs/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf"] [line "78"] [id "930120"] [rev ""] [msg "OS File Access Attempt"] [data "Matched Data: .profile found within ARGS_NAMES: json.userprofiles.array_0.profilename.de"] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-lfi"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "OWASP_CRS/WEB_ATTACK/FILE_INJECTION"] [tag "WASCTC/WASC-33"] [tag "OWASP_TOP_10/A4"] [tag "PCI/6.5.4"] [hostname "127.0.0.1"] [uri "/xxx"] [unique_id "1634286725"] [ref "o25,8v0,47t:utf8toUnicode,t:urlDecodeUni,t:normalizePathWin,t:lowercaseo25,8v0,40t:utf8toUnicode,t:urlDecodeUni,t:normalizePathWin,t:lowercaseo25,8v0,47t:utf8toUnicode,t:urlDecodeUni,t:normalizePathWi (368 characters omitted)"]
ModSecurity: Warning. Matched "Operator Ge' with parameter 5' against variable TX:ANOMALY_SCORE' (Value: 40' ) [file "/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "80"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 40)"] [data ""] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "127.0.0.1"] [uri "/xxx"] [unique_id "1634286725"] [ref ""]

Server (please complete the following information):

• ModSecurity version (and connector): ModSecurity v3
• WebServer: nginx version: nginx/1.21.3 (nginx-plus-r25)
• OWASP CRS : 3.2.0
• OS : CentOS Linux, Debian 11

[martinhsv]

Hello @lontchianicet ,

Although you mention that you are using ModSecurity v3, you don't say which specific version you are using (i.e. what is the 'x' in v3.0.x).

There was a bug in v3.0.4 ( #2251 ), a fix for which was included in v3.0.5.

[lontchianicet]

Thx. It was useful to know that.

[martinhsv]

I'm assuming this means that you were indeed using v3.0.4? If I don't hear otherwise, I'll go ahead and close this within a day or two.

[defanator]

@martinhsv latest modsecurity module package for N+ R25 is based on pure v3.0.5, so this one requires additional investigation. I'm going to schedule some time to work on this.

[martinhsv]

@defanator: Thanks for the clarification.

@lontchianicet: It looks like I can reproduce your finding in v3.0.5 -- but only with ARGS_NAMES and not with ARGS (the latter seems to work correctly).

I'll see if I can take a closer look.

[martinhsv]

I have identified the commit that caused this issue.

Also, I have confirmed that only three collections are affected: ARGS_NAMES, ARGS_GET_NAMES, ARGS_POST_NAMES. I will update the issue title to that effect.

[lontchianicet]

@martinhsv Thx for discovering the source of the problem.

[martinhsv]

To summarize and explain a workaround that may be useful for some users ...

Suppose you have a rule that you want to trigger for any argument name that includes 'abc'. It might look something like this:

SecRule ARGS_NAMES "@contains abc" "id:1000,phase:2,log,deny,status:403"

But perhaps you want to exclude the rule if the 'abc' is merely a prefix to a larger specific string 'abcdef'. In this case you might have constructed an extra directive like:

SecRuleUpdateTargetById 1000 "!ARGS_NAMES:/abcdef/"

This use case is broken in v3.0.5 for the three ARGS_*NAMES collections.

However, an alternative that does appear to work correctly is to specify the exclusion in the rule itself. E.g.

SecRule ARGS_NAMES|!ARGS_NAMES:/abcdef/ "@contains abc" "id:1000,phase:2,log,deny,status:403"

This alternative is not ideal in some cases -- e.g. if your rule set is an acquired rule set (like CRS) that has even moderately frequent updates. After all, one of the advantages of keeping custom exceptions separate from acquired rules is reduced maintenance effort when the acquired rules are updated. However, I thought it worth outlining, as it may be useful to some users until a permanent fix is in place.