Machine Parseable Audit Log Formats

[rcbarnett-zz]

It would be great to have a new directive such as "SecAuditLog Format" where the user can specify a new output format such as JSON or XML that is more suited for parsing by SIEM systems.

Example directive usage -
SecAuditLogFormat [TEXT JSON XML]

TEXT would be how it is now, with multi-line entries.
JSON would be JSON formatted and XML would be put into an XML schema.

We would need to think about how to tokenize the audit log data into JSON/XML elements. We could simply break it up into the audit log PARTS as defined here -
https://github.com/SpiderLabs/ModSecurity/wiki/ModSecurity-2-Data-Formats#wiki-Parts

[RandallKent]

@p0pr0ck5 has put some effort into JSON logging on a fork as a part of his Master's thesis.

He has also provided a brief write up on his blog.

I've not taken a close look or tested, but thought a comment might be helpful to get it on the radar.

[rcbarnett-zz]

Thanks - coincidentally, we just found his fork the other day and are reviewing it :)

[RandallKent]

Surely a sign of good things to come 😄

[zimmerle]

Trying to have @p0pr0ck5 logic using YAJL instead of json-c as we already depend on YAJL. Also, I have added option to save the output in text or json as suggested on the feature request.

https://github.com/SpiderLabs/ModSecurity/tree/json_logging

[p0pr0ck5]

I ended up taking a different direction in the project I was building out, and so didn't pursue completing a stable and working fork- please forgive that my work is incomplete and rather hackish :) Hopefully it can be of some use!

[p0pr0ck5]

I ended up completely refactoring the idea for presenting audit logs as JSON here: https://github.com/p0pr0ck5/ModSecurity/tree/json_audit_logging

This uses yajl and is a much cleaner approach, in that it doesn't stomp on message generation function signatures.

See https://www.cryptobells.com/mod_security-json-audit-logs-revisited/ for some discussion.

[zimmerle]

Pull request #914 was merged. It will be available as part of ModSecurity version 2.9.1