Compliance: /spec Validation Checks

The compliance suites an entry subscribes to are listed in the spec field of meta.json, the way profiles are listed in tests. It is a field of its own because nothing in it is measured:

"tests": ["baseline", "baseline-h2", "baseline-h2c", "baseline-h3"],
"spec": ["h1", "h2", "h3"]

Two checks run before anything is built, like the one on tests:

• Unknown names fail. The suites are h1, h2, h3 and ws. A typo would otherwise drop a column without a word.
• A suite needs a profile to run against. h1 needs an HTTP/1.1 profile on 8080; h2 one of baseline-h2, static-h2, baseline-h2c or json-h2c; h3 one of baseline-h3 or static-h3; ws one of the echo-ws profiles.

Then /spec is checked for h1 and h2, on every listener that suite uses: serving it on 8080 and forgetting it on 8443 fails rather than quietly losing the H/2 column. h3 and ws need no route, so there is nothing more to check for them here. A suite the entry does not list is not checked, and shows as not measured in its column.

Bodies are random bytes, so a canned response cannot pass, and every echo is compared byte for byte.

h1: HTTP/1.1 on 8080

h2: HTTP/2 on 8443 and 8082

What h2spec needs, on each HTTP/2 listener the entry runs: