SecRuleRemoveById conflict with other Sec*

[rljoia]

Describe the bug
I created a configuration, based on this article, to rate limit a specific url. I'm using apache + modsecurity. This configuration works fine, but I noticed that if I have any SecRuleRemoveById directive in any of my *.conf files, the rate limit doesn't work anymore.

Logs and dumps and To Reproduce
It's works:

SecRuleEngine On
SecStatusEngine On
<Location /influxdb>
    SecAction initcol:ip=%{X-Forwarded-For},pass,nolog,id:11
    SecAction "phase:5,deprecatevar:ip.somepathcounter=1/1,pass,nolog,id:12"
    SecRule IP:SOMEPATHCOUNTER "@gt 60" "phase:2,pause:300,deny,status:429,setenv:RATELIMITED,skip:1,nolog,id:13"
    SecAction "phase:2,pass,setvar:ip.somepathcounter=+1,expirevar:ip.somepathcounter=600,nolog,id:14"
    Header always set Retry-After "10" env=RATELIMITED

    ProxyPass "http://influxdb:8086"
    ProxyPassReverse "http://influxdb:8086"
</Location> 

ErrorDocument 429 "Rate Limit Exceeded"

The output of my test script is:

58 - Status code: 200 - {"version":"1.8.0"}
59 - Status code: 200 - {"version":"1.8.0"}
60 - Status code: 200 - {"version":"1.8.0"}
61 - Status code: 200 - {"version":"1.8.0"}
62 - Status code: 429 - Rate Limit Exceeded
63 - Status code: 200 - {"version":"1.8.0"}
64 - Status code: 429 - Rate Limit Exceeded
65 - Status code: 429 - Rate Limit Exceeded
66 - Status code: 429 - Rate Limit Exceeded

It doesn't work:

SecRuleEngine On
SecStatusEngine On
<Location /influxdb>
    SecAction initcol:ip=%{X-Forwarded-For},pass,nolog,id:11
    SecAction "phase:5,deprecatevar:ip.somepathcounter=1/1,pass,nolog,id:12"
    SecRule IP:SOMEPATHCOUNTER "@gt 60" "phase:2,pause:300,deny,status:429,setenv:RATELIMITED,skip:1,nolog,id:13"
    SecAction "phase:2,pass,setvar:ip.somepathcounter=+1,expirevar:ip.somepathcounter=600,nolog,id:14"
    Header always set Retry-After "10" env=RATELIMITED

    ProxyPass "http://influxdb:8086"
    ProxyPassReverse "http://influxdb:8086"
</Location> 

ErrorDocument 429 "Rate Limit Exceeded"

Include /etc/apache2/conf-available/modsecurity_rules.conf

Content of modsecurity_rules.conf:

SecRuleRemoveById  932130 
SecRuleRemoveById  932100 
SecRuleRemoveById  932105
SecRuleRemoveById  932140
SecRuleRemoveById  942220
SecRuleRemoveById  920180 
SecRuleRemoveById  980130
SecRuleRemoveById  920420

If I comment ALL of these SecRuleRemoveById, the rate limit works.
If I uncomment ANY of these SecRuleRemoveById, the rate limit doesn't work. In this scenario, the output of my test script is:

58 - Status code: 200 - {"version":"1.8.0"}
59 - Status code: 200 - {"version":"1.8.0"}
60 - Status code: 200 - {"version":"1.8.0"}
61 - Status code: 200 - {"version":"1.8.0"}
62 - Status code: 200 - {"version":"1.8.0"}
63 - Status code: 200 - {"version":"1.8.0"}
64 - Status code: 200 - {"version":"1.8.0"}
65 - Status code: 200 - {"version":"1.8.0"}
66 - Status code: 200 - {"version":"1.8.0"}

Expected behavior

• SecRuleRemoveById directive shouldn't affect the rate limit script.

Server (please complete the following information):

• ModSecurity version (and connector): [v2.9.3 with modSecurity-CRS: 3.2.0-1]
• WebServer: [apache v2.4.43]
• OS (and distro): [Linux Ubuntu Focal (20.04), running in Docker]

Rule Set (please complete the following information):

• modSecurity-CRS: 3.2.0-1

Additional context
I develop a script in python to test the rate limit:

import requests 
URL = "https://localhost/influxdb/ping"
PARAMS = {'verbose':'true'} 
for x in range(10000):
    r = requests.get(url = URL, params = PARAMS, verify=False) 
    print(str(x) + ' - Status code: ' + str(r.status_code) + ' - ' + r.text)
print('Finish')

[zimmerle]

@rljoia

You mentioned the rules -

SecRuleRemoveById  932130 
SecRuleRemoveById  932100 
SecRuleRemoveById  932105
SecRuleRemoveById  932140
SecRuleRemoveById  942220
SecRuleRemoveById  920180 
SecRuleRemoveById  980130
SecRuleRemoveById  920420

But forgot the share the actual content of the rule. I would suggest you remove all the extra rules, to narrow down the problem. It is possible that the IP collection is being initialized elsewhere.

[rljoia]

@zimmerle

These numbers refers to OWASP CRS rule set. To clarify the question, I will put all *.conf files from the project here.

Everything starts with locations.conf:

# Disable directory listing in /var/www
<Directory /var/www/>
	Options FollowSymLinks
	AllowOverride None
	Require all granted
</Directory>

# Locations for OBiBa apps
RewriteRule    ^/repo/?$  %{HTTP_REFERER}repo/ui/index.html [R]
ProxyTimeout 1200

<Location /auth>
    ProxyPass "https://agate:8444"
    ProxyPassReverse "https://agate:8444"
</Location>

<Location /repo>
    ProxyPass "https://opal:8443"
    ProxyPassReverse "https://opal:8443"
</Location>

<Location /pub>
    ProxyPass "https://mica:8445"
    ProxyPassReverse "https://mica:8445"
</Location>

<Location /cat>
    ProxyPass "http://mica-drupal:80"
    ProxyPassReverse "http://mica-drupal:80"	
</Location>


# Debug log
SecDebugLog /var/log/mod_debug.log
SecDebugLogLevel 6

# Central configuration
Include /etc/apache2/conf-available/central.conf

# Log requests to the OBiBa apps
Include /etc/apache2/conf-available/obiba_log.conf

# Override SSL/TLS conf
Include /etc/apache2/conf-available/custom_ssl.conf

Now, central.conf:

SecRuleEngine On
SecStatusEngine On

<Location /influxdb>
    SecAction initcol:ip=%{X-Forwarded-For},pass,log,id:20011
    SecAction "phase:1,deprecatevar:ip.somepathcounter=1/1,pass,log,id:20012"
    SecRule IP:SOMEPATHCOUNTER "@gt 60" "phase:2,pause:300,deny,status:429,setenv:RATELIMITED,skip:1,log,id:20013"
    SecAction "phase:2,pass,setvar:ip.somepathcounter=+1,expirevar:ip.somepathcounter=600,log,id:20014"
    Header always set Retry-After "10" env=RATELIMITED

    ProxyPass "http://influxdb:8086"
    ProxyPassReverse "http://influxdb:8086"
</Location> 

ErrorDocument 429 "Rate Limit Exceeded"

<Location /grafana-central>
    ProxyPass "http://grafana-central:3000"
    ProxyPassReverse "http://grafana-central:3000"
</Location>

Now, obiba_log.conf:

# Expose the request ID and cookies/header tokens
#SetEnvIf Request_URI ^.*/repo/ws/project/.*/commands/_export$ opal_log_body=yes
SetEnvIf Request_URI ^/auth.*$ agate=yes
SetEnvIf Request_URI ^/repo.*$ opal=yes
SetEnvIf Request_URI ^/pub.*$ mica=yes

# Main log format
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

# Obiba logs format
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" @tokens@%{X-Agate-Auth}i@%{agatesid}C@%{obibaid}C@tokens@" combined_with_agate_tokens
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" @tokens@%{X-Opal-Auth}i@%{opalsid}C@%{obibaid}C@tokens@" combined_with_opal_tokens
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" @tokens@%{X-Mica-Auth}i@%{micasid}C@%{obibaid}C@tokens@" combined_with_mica_tokens

# Obiba logs format (with request ID)
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"request_id: %{unique_id}e\" @tokens@%{X-Agate-Auth}i@%{agatesid}C@%{obibaid}C@tokens@" combined_with_agate_tokens_and_id
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"request_id: %{unique_id}e\" @tokens@%{X-Opal-Auth}i@%{opalsid}C@%{obibaid}C@tokens@" combined_with_opal_tokens_and_id
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"request_id: %{unique_id}e\" @tokens@%{X-Mica-Auth}i@%{micasid}C@%{obibaid}C@tokens@" combined_with_mica_tokens_and_id

# Change log lines to include the username in all requests to Agate, Opal or Mica
CustomLog "|$bash /etc/apache2/scripts/discover_username.sh >> /var/log/obiba.log" combined_with_agate_tokens_and_id expr=(reqenv('agate')=='yes')
CustomLog "|$bash /etc/apache2/scripts/discover_username.sh >> /var/log/obiba.log" combined_with_opal_tokens_and_id expr=(reqenv('opal')=='yes')
CustomLog "|$bash /etc/apache2/scripts/discover_username.sh >> /var/log/obiba.log" combined_with_mica_tokens_and_id expr=(reqenv('mica')=='yes')

CustomLog /var/log/apache2/access.log combined expr=(!reqenv('agate')=='yes'&&!reqenv('opal')=='yes'&&!reqenv('mica')=='yes')

# Configure ModSecurity to log body of Opal exports and R server requests
SecRuleEngine On
SecAuditEngine Off
SecRequestBodyAccess On
SecAuditLogParts ACZ

SecAuditLog /var/log/obiba.log
SecDefaultAction "nolog,noauditlog,allow,phase:2"
SecRule REQUEST_METHOD "^POST$|^PUT$" "chain,allow,phase:2,id:123"
SecRule REQUEST_URI "^.*/repo/ws/project/.*/commands/_export$|^.*/repo/ws/r/.*/symbol/.*$|^.*/auth/ws/user.*$" "ctl:auditEngine=On"

# Change default ModSecurity rules
Include /etc/apache2/conf-available/modsecurity_rules.conf

Now, modsecurity_rules.conf:

# Default HTTP policy: allowed_methods
SecRule REQUEST_URI "@rx ^(/auth|/repo|/pub|/cat|/api)" "id:900200,\
  phase:1,\
  nolog,\
  pass,\
  t:none,\
  setvar:'tx.allowed_methods=GET HEAD POST PUT DELETE OPTIONS'"

# Default HTTP policy: allowed_request_content_type
SecRule REQUEST_URI "@rx ^(/auth($|/)|/repo($|/)|/pub($|/)|/cat($|/)|/api($|/))" "id:900201,\
  phase:1,\
  nolog,\
  pass,\
  t:none,\
  setvar:'tx.allowed_request_content_type=application/x-protobuf+json|application/x-www-form-urlencoded|multipart/form-data|text/xml|application/xml|application/soap+xml|application/x-amf|application/json|application/octet-stream|text/plain'"

# Increase request body content size (for Opal file uploads)
SecRequestBodyLimit 1073741824

SecRuleRemoveById  932130 # "Remote Command Execution: Unix Shell Expression Found" -> MagmaJS scripts (opal)
SecRuleRemoveById	932100 # "PCRE limits exceeded" -> transient tables when importing data (opal)
SecRuleRemoveById	932105 # "PCRE limits exceeded" -> transient tables when importing data (opal)
SecRuleRemoveById	932140 # "Remote Command Execution: Windows FOR/IF Command Found" -> transient tables when importing data (opal)
SecRuleRemoveById	942220 # "integer overflow attack" -> on search (opal)
SecRuleRemoveById	920180 # "no content-length header" - datashield
SecRuleRemoveById	980130 # "Inbound Anomaly Score Exceeded" -> datashield
SecRuleRemoveById	920420 # "Request content type is not allowed by policy (application/octet-stream)" -> datashield

And, finally, custom_ssl.conf:

SSLProtocol all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1

As I said, If I comment ALL lines with SecRuleRemoveById in modsecurity_rules.conf, the rate limit script in <Location /influxdb> works.

This problem appears some trickery. I'm trying to understand the reason, but even with debug logs from ModSecurity in level 6, I couldn't find the solution.

[zimmerle]

ModSecurity version 2.x has some limitations on the utilization of SecRuleRemove inside virtual hosts. That limitation no longer exists in version 3. I think that is may be the case.