Allow horizontal tabs in header values

[dave-shawley]

Addresses #3450

[bdarnell]

Thanks! You're right that tabs should be allowed here, you're just the first to ask for it.

[bdarnell]

While we're here: I went back to the RFCs to make sure that this change is correct, and found that \x7f (DEL) is also invalid. So let's add that to the list too: \x00-\x08\x0a-\x1f\x7f. Or for a slightly larger code change we could invert it to define valid chars (like the RFCs do) instead of invalid ones: \x09\x20-\x7e\x80-0xff.

[bdarnell]

This is nitpicky, but my first thought on seeing this test case is "why does it look like another header value after the tab? Is that some sort of injection attack I haven't seen?". Header injection is the reason \r\n are banned, but if we're expanding this handler to be the general test of set_header's enforcement of valid header characters we should probably rename it and use examples that look less like injection attacks.

Also, there should probably be a check in test_header_injection to assert that the valid header is readable from the client side.

[dave-shawley]

@bdarnell I force pushed over my branch to clean up the commit history. I took your advice and added a new test case instead of reusing the one from test_header_injection. I also added an exclusion for DEL.

[dave-shawley]

Since inverting the sense of the regular expression required switching from re.search to re.fullmatch, should I implement the requirement that the first and last characters in a header are printable characters?

  field-value    = *field-content
  field-content  = field-vchar [ 1*( SP / HTAB / field-vchar ) field-vchar ]
  field-vchar    = VCHAR / obs-text
  obs-text       = %x80-FF

I think that changing the RE from [\x09\x20-\x7e\x80-\xff]* to [\x21-\x7e\x80-\xff]([\x09\x20-\x7e\x80-\xff]*[\x21-\x7e\x80-\xff])* would do the job. I'm not sure if increasing the RE complexity is worth it or not.