Rule MULTIPART_UNMATCHED_BOUNDARY, id:'200003' fails on valid multipart/form-data submission

[temyers]

The MULTIPART_UNMATCHED_BOUNDARY rule fires, denying the request if the multipart/form data includes "--" at the beginning of a line. If there is anything else at the start of the line (e.g. " --") then the rule does not trigger.

This issue was identified against a Liferay 6.2 installation fronted by apache+modsecurity, submitting a freemarker code snippet.

A sample curl command:

curl 'https://localhost/group/control_panel/manage?p_auth=MWq0gmZw&p_p_id=166&p_p_lifecycle=1&p_p_state=pop_up&p_p_mode=view&doAsGroupId=10328&refererPlid=10331&controlPanelCategory=current_site.content&_166_refererPortletName=15&_166_refererWebDAVToken=journal&_166_scopeTitle=Templates&_166_struts_action=%2Fdynamic_data_mapping%2Fedit_template' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.5' -H 'Connection: keep-alive' -H 'Cookie: JSESSIONID=E10AB37213FD6D61DE10A72AB8FF1A3A; LFR_SESSION_STATE_10345=1390888801890' -H 'DNT: 1' -H 'Host: localhost' -H 'Referer: https://localhost/group/control_panel/manage?p_p_id=166&p_p_lifecycle=0&p_p_state=pop_up&doAsGroupId=10328&refererPlid=10331&controlPanelCategory=current_site.content&_166_refererPortletName=15&_166_refererWebDAVToken=journal&_166_scopeTitle=Templates&_166_cmd=update&_166_struts_action=%2Fdynamic_data_mapping%2Fedit_template&_166_redirect=https%3A%2F%2Flocalhost%2Fgroup%2Fcontrol_panel%2Fmanage%3Fp_p_id%3D166%26p_p_lifecycle%3D0%26p_p_state%3Dpop_up%26p_p_mode%3Dview%26doAsGroupId%3D10328%26refererPlid%3D10331%26controlPanelCategory%3Dcurrent_site.content%26_166_refererPortletName%3D15%26_166_refererWebDAVToken%3Djournal%26_166_scopeTitle%3DTemplates%26_166_groupId%3D10328%26_166_showHeader%3D0%26_166_classNameId%3D10102%26_166_eventName%3DselectStructure%26_166_struts_action%3D%252Fdynamic_data_mapping%252Fview_template&_166_templateId=10850&_166_groupId=10328&_166_classNameId=10102&_166_classPK=0&_166_type=display&_166_structureAvailableFields=' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0' -H 'Content-Type: multipart/form-data; boundary=---------------------------1835282785777842564651277339' --data-binary $'Content-Type: multipart/form-data; boundary=---------------------------1835282785777842564651277339\r\nContent-Length: 3492\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_formDate"\r\n\r\n1390888796288\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_cmd"\r\n\r\nupdate\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_redirect"\r\n\r\nhttps://localhost/group/control_panel/manage?p_p_id=166&p_p_lifecycle=0&p_p_state=pop_up&p_p_mode=view&doAsGroupId=10328&refererPlid=10331&controlPanelCategory=current_site.content&_166_refererPortletName=15&_166_refererWebDAVToken=journal&_166_scopeTitle=Templates&_166_groupId=10328&_166_showHeader=0&_166_classNameId=10102&_166_eventName=selectStructure&_166_struts_action=%2Fdynamic_data_mapping%2Fview_template\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_closeRedirect"\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_portletResource"\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_templateId"\r\n\r\n10850\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_groupId"\r\n\r\n10328\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_classNameId"\r\n\r\n10102\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_classPK"\r\n\r\n0\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_type"\r\n\r\ndisplay\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_structureAvailableFields"\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_saveAndContinue"\r\n\r\n1\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_name"\r\n\r\nTest\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_name_en_AU"\r\n\r\nTest\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_language"\r\n\r\nftl\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_description"\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_description_en_AU"\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_smallImage"\r\n\r\nfalse\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_type"\r\n\r\nfalse\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_smallImageFile"; filename=""\r\nContent-Type: application/octet-stream\r\n\r\n\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_scriptContent"\r\n\r\n<#--\r\nDisplay templates are used to lay out the fields defined in a data\r\ndefinition.\r\n\r\nPlease use the left panel to quickly add commonly used variables.\r\nAutocomplete is also available and can be invoked by typing "${".\r\n-->\r\n-----------------------------1835282785777842564651277339\r\nContent-Disposition: form-data; name="_166_script"; filename=""\r\nContent-Type: application/octet-stream\r\n\r\n\r\n-----------------------------1835282785777842564651277339--\r\n'

Environment:

$ yum list installed | grep httpd
httpd.x86_64                     2.4.6-2.fc18       @updates                    
httpd-tools.x86_64               2.4.6-2.fc18       @updates                    
libmicrohttpd.x86_64             0.9.22-1.fc18      @koji-override-0/$releasever

$ yum list installed | grep mod_
mod_security.x86_64              2.7.3-2.fc18       @updates                    
mod_ssl.x86_64                   1:2.4.6-2.fc18     @updates          

The form submission is valid, with this an obvious false positive. The workaround is to disable the rule. However, it would be better if this was catered for an didn't trigger the rule in the first place.

[Hadrianus]

Even a parameter with a value starting with "--" causes the request being denied.
A sample curl command:

curl https://localhost/app/ -Fparam=--value

[akrherz]

any hopes of triaging this and getting it fixed? This is causing us great grief at the moment :)

[rcbarnett-zz]

Is this the rule causing the alert?
https://github.com/SpiderLabs/ModSecurity/blob/master/modsecurity.conf-recommended#L83-86
If so, you might want to disable it in the meantime until we can investigate.

[akrherz]

Thanks for the response. Yes that is the rule and we are not permitted to disable the rule :(

[rcbarnett-zz]

Are you not the website (ModSecurity) owner?

[akrherz]

Nope, just the developer

I am affected by this bug as well.
A false positive trigger occurs when uploading a pdf which contains a line that starts with '--abc'.

The rule that triggers this is:
https://github.com/SpiderLabs/ModSecurity/blob/master/modsecurity.conf-recommended#L83-86

[akrherz]

Any updated comments on resolving this bug? :)

[AndreyMZ]

I am not familiar with Modsecurity code, but...
It is an obvious bug in Modsecurity in defining MULTIPART_UNMATCHED_BOUNDARY variable. And it have taken 2 minutes for me to locate it.
Could you fix it please, or correct me if I am wrong?

Problem description:
If all boundaries are OK but POST data contains "--" at the begining of line, then MULTIPART_UNMATCHED_BOUNDARY is wrongly set to 1.

Affected versions:
Modsecurity 2.9.0 (downloaded from https://www.modsecurity.org/tarball/2.9.0/modsecurity-2.9.0.tar.gz).

Root cause:
File modsecurity-2.9.0\apache2\msc_multipart.c, function multipart_process_chunk, line 23.

/* Do we have something that looks like a boundary? */
if (   msr->mpd->buf_contains_line
    && (strlen(msr->mpd->buf) > 3)
    && (*(msr->mpd->buf) == '-')
    && (*(msr->mpd->buf + 1) == '-') )
{
    /* Does it match our boundary? */
    if (   (strlen(msr->mpd->buf) >= strlen(msr->mpd->boundary) + 2)
        && (strncmp(msr->mpd->buf + 2, msr->mpd->boundary, strlen(msr->mpd->boundary)) == 0) )
    {
        // ...
    } else { /* It looks like a boundary but we couldn't match it. */
        // ...
        msr->mpd->flag_unmatched_boundary = 1;
    }
} else { /* We do not think the buffer contains a boundary. */

Possible soulution:
Maybe just this?

         }

-        msr->mpd->flag_unmatched_boundary = 1;
     }
 } else { /* We do not think the buffer contains a boundary. */

[manuel-sousa]

Same thing as rutgerbazen, with a couple of PDF uploads this rule triggers and had to disable it since the PDF encoding is out of our control.

[driehuis]

@AndreyMZ: This rule does what it says on the tin. Your suggested fix would essentially remove the entire check. The MULTIPART_UNMATCHED_BOUNDARY check is intended to detect uploads that would cause problems for a web application that does not properly check the full MIME boundary. If you are 100% confident that your application properly checks MIME boundaries, just disable the rule in your modsecurity.conf with

SecRuleRemoveById 200003

Remember that many rules that try to protect against common programming errors will also trip on legitimate traffic, and as a security administrator, you will have to make the judgement call on whether or not the added security of keeping the rule enabled is worth the occasional false positive (or indeed, that you are sure that this particular programming error was not made anywhere on your system).

I have personally seen badly written MIME parsers that fail when processing a file that contains a newline-dash-dash sequence, so I have seen the use case for having this rule first hand.

[goyocasero]

We are experiencing the same issue (some uploaded files are being denied by this rule).
The log:
Message: Access denied with code 403 (phase 2). Match of "eq 0" against "MULTIPART_UNMATCHED_BOUNDARY" required. [file "E:/Apache2.2.31/conf/httpd.conf"] [line "561"] [id "000003"] [msg "Multipart parser detected a possible unmatched boundary."]

and the rule:
SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" \ "phase:2,id:'000003',t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"

Some files are being denied but some others not...

We're using modsecurity version 2.8.0 in Apache2.2.31 and I'm finding this issue as opened, so I'm wondering if this could be the reason of our problem. If it's already resolved, anyone has any clue?

[AndreyMZ]

The solution is to disable this rule (it's id is 200003).

According to @driehuis, this rule does what it should do.
So the problem is that the documentation is not clear. It should say that this rule in particular blocks uploading of some legitimate files, and it should be disabled unless some badly written MIME parser is used.