Audtilog not works with nginx default-backend

[Norsu296]

Hello, I'm using ingress-nginx on kubernetes and I have enabled Modsecurity via configmap option. Modsecurity works, auditlog is formatted in JSON.
When I set in nginx configmap custom-http-errors auditlog is not sent as JSON but as normal nginx output as described in this link. kubernetes/ingress-nginx#5679 (comment)
How can I fix it? I want to have custom error pages and I want to keep auditlog. I tried everything from issue above and related another issues I think it maybe related with #255 but as I can see it was fixed but it still doesn't work. Im using ingress-nginx chart 4.7.1

[airween]

Is this an ingress-nginx specific error?

I tried to reproduce that with a "native" Nginx instance. Here is my config:

Nginx:

server {
    error_page 403 /403.html;

    location /403.html {
        root /usr/share/nginx/html;
        internal;
    }
}

ModSecurity:

SecAuditLogType Serial
SecAuditLog /var/log/nginx/modsec_audit.log
SecAuditLogFormat JSON

Sending an attack then I see my custom formatted page. Then I check my audit.log and I get:

{"transaction":{"client_ip":"::1","time_stamp":"Thu Oct 17 13:39:50 2024","server_id":"0d4d434e96996020298a3e094b066f66e6634d3b","client_port":51836,"host_ip":"::1","host_port":8080,"unique_id":"172916519050.865520","request":{"method":"GET","http_version":1.1,"uri":"/?q=/bin/bash","headers":{"Host":"localhost:8080","User-Agent":"curl/8.10.1"...

Versions:
Nginx: 1.26.0
libmodsecurity: 3.0.13 + commits since the release
libnginx-mod-http-modsecurity: 1.0.3

How can I reproduce this issue?

[james-nofrixion]

@airween: I have this issue and it is produced by installing using helm chart ingress-nginx-4.11.3 and the following values.yaml:

controller:
  replicaCount: 2
  enableAnnotationValidations: true
  allowSnippetAnnotations: true
  service:
    type: LoadBalancer
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-name: ingress-nlb
      service.beta.kubernetes.io/aws-load-balancer-type: external
      service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
      service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
      service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: tcp
      service.beta.kubernetes.io/aws-load-balancer-healthcheck-path: /healthz
      service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: preserve_client_ip.enabled=true
      service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
  addHeaders:
    Strict-Transport-Security: max-age=15724800; includeSubDomains
    X-Content-Type-Options: nosniff
  config:
    custom-http-errors: "403"
    proxy-body-size: "200M"
    proxy-buffer-size: "128k"
    proxy-buffers: "4 256k"
    proxy-busy-buffers-size: "256k"
    client-header-buffer-size: "64k"
    large-client-header-buffers: "16 128k"
    use-proxy-protocol: "true"
    proxy-real-ip-cidr: 192.168.0.0/16
    enable-real-ip: "true"
    log-format-escape-json: "true"
    log-format-upstream: >-
      { "timestamp": "$time_iso8601", "remote_addr": "$remote_addr","x-forward-for": "$proxy_add_x_forwarded_for", "request_id": "$req_id", "remote_user": "$remote_user", "bytes_sent": $bytes_sent, "request_time": $request_time, "status": $status, "vhost": "$host", "request_proto": "$server_protocol", "path": "$uri", "request_query": "$args", "request_length": $request_length, "duration": $request_time,"method": "$request_method", "http_referrer": "$http_referer", "http_user_agent": "$http_user_agent"}
    # mod security WAF (https://kubernetes.github.io/ingress-nginx/user-guide/third-party-addons/modsecurity/)
    enable-modsecurity: true
    enable-owasp-modsecurity-crs: true 
    modsecurity-snippet: |
      Include /etc/nginx/owasp-modsecurity-crs/custom/custom-modsecurity.conf
  extraVolumeMounts:
    - name: modsecurity-config
      mountPath: /etc/nginx/owasp-modsecurity-crs/custom/
      readOnly: true
  extraVolumes:
  - name: modsecurity-config
    configMap:
      name: modsecurity-config
defaultBackend:
  enabled: true
  image:
    registry: registry.k8s.io
    image: ingress-nginx/custom-error-pages
    tag: v1.0.2@sha256:b2259cf6bfda813548a64bded551b1854cb600c4f095738b49b4c5cdf8ab9d21
  extraConfigMaps:
  - name: custom-error-pages
    data:
      403: |
        <!DOCTYPE html>
        <html>
          <head><title>Access Denied</title></head>
          <body>
            <h1>Access Denied</h1>
            <p>Access to this page was blocked by the Nofrixion WAF.</p>
          </body>
        </html>
  extraVolumes:
  - name: custom-error-pages
    configMap:
      name: custom-error-pages
      items:
      - key: "403"
        path: "403.html"
  extraVolumeMounts:
  - name: custom-error-pages
    mountPath: /www

The modsecurity configmap is:

apiVersion: v1
kind: ConfigMap
metadata:
  name: "modsecurity-config"
data:
  custom-modsecurity.conf: |    
    SecRuleEngine On
    # Avoid sending status information about ModSecurity in response header
    SecStatusEngine Off
    # Send ModSecurity audit logs to the stdout (only for rejected requests)
    SecAuditLog /dev/stdout
    SecAuditLogFormat JSON
    SecAuditLogParts ABCFHKZ
    SecAuditEngine RelevantOnly # could be On/Off/RelevantOnly
    # Max request sizes in bytes (with/without files) - Note NGINX Ingress has its own parameter/annotation that should be kept in sync
    SecRequestBodyLimit 209715200 # 200Mb (default is 12.5Mb)
    SecRequestBodyNoFilesLimit 2621440 # 2500Kb (default is 128Kb)
    SecRequestBodyLimitAction Reject # Reject if larger (we could also let it pass with ProcessPartial)
    # recommended limits for regular expression recursion. See https://github.com/SpiderLabs/owasp-modsecurity-crs/issues/656#issuecomment-262780221
    SecPcreMatchLimit 500000
    SecPcreMatchLimitRecursion 500000
    SecAction "id:900110, phase:1, nolog, pass, t:none, setvar:tx.inbound_anomaly_score_threshold=5, setvar:tx.outbound_anomaly_score_threshold=4"
    # Include PUT/PATCH/DELETE in the allowed methods, otherwise those verbs will be rejected by rule 911100
    SecAction "id:900200,phase:1,nolog,pass,t:none,\
      setvar:tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE"
    # REQUEST_COOKIE has too much text, keep getting arbitrary function name matches on PHP functions.
    SecRuleUpdateTargetById 933150 !REQUEST_COOKIES

[airween]

@james-nofrixion,

unfortunately I'm afraid I can't help you with this. I've never used Ingress and I don't have enough capability to install that.

Probably you need to check twice the permission (path to audit.log), or some other MACL system (mandatory access control list - Apparmor or SeLinux) settings.