POST body inspection

[jvehent]

We discussed inspecting POST body a few times, and so far concluded that it was too difficult and to costly to do, and we should stick to body size validation instead.

Discussing with @psiinon today, he raised a good point: If we add the format of a POST request into the specification, ZAP could use it as a template to do security tests/fuzzing. The cost issue could be solved by enabling/disabling the filtering at the resource level, and them simply turn it off in production.

Example from https://github.com/mozilla/videur/blob/master/spec/mig_example.json#L73-L106

The flag must_validate controls whether validate is enforced or not, and thus can be turned off in production.
The format field indicates that the POST body, and thus validation field, contains valid JSON.
The validation field is then a standardized version of the expected body, with values replaced with placeholders such as [[datetime]] or [[text]] or [[json]].

{
    "service": {
        "resources": {
            "/action/create": {
                "POST": {
                    "format": "json",
                    "must_validate": false,
                    "validation": {
                        "id": "[[float64]]",
                        "name": "[[text]]",
                        "target": "[[text]]",
                        "description": {
                            "author": "[[text]]",
                            "email": "[[email]]",
                            "url": "[[text]]",
                            "revision": "[float64]"
                        },
                        "threat": {
                            "level": "[[text]]",
                            "family": "[[text]]",
                            "type": "[[text]]"
                        },
                        "validfrom": "[[datetime]]",
                        "expireafter": "[[datetime]]",
                        "operations": [
                            {
                                "module": "[[text]]",
                                "parameters": "[[json]]"
                            }
                        ],
                        "pgpsignatures": [
                            "[[text]]"
                        ],
                        "starttime": "[[datetime]]",
                        "finishtime": "[[datetime]]",
                        "lastupdatetime": "[[datetime]]",
                        "counters": "[[json]]",
                        "syntaxversion": "[[float64]]"
                    }
                }
            }
        }
    }
}

[psiinon]

+1 from me - I could definitely use that in ZAP, and I think it would be really useful to have the option to validate JSON POST data.
Gives us basic WAF functionality without all the pain ;)
Theres no reason why this format could not be also used for the std key=value pairs, XML etc etc