Conformance tests fail when deployed on a cluster with CNI that needs NetworkPolicy for cross namespace traffic

[pmalek]

Recently, we've discovered that when running conformance tests on a cluster with Calico a lot of the tests fail.

This discussion aims to bring this subject up whether maintainers would support the idea of introducing the concept of adding resources like NetworkPolicy to conformance tests that would require them when deployed on a cluster which has a CNI installed that would enforce said resources.

[shaneutt]

A NetworkPolicy is a declaration of intent as to what network communications are allowed (or disallowed). There are network communications we are inherently expecting to be allowed in the conformance tests, but some testing environments may inherently deny traffic. Since NetworkPolicy is available in core I personally think it would be reasonable for us to package along some policies which cover the expected testing traffic. Alternatively, at bare minimum we should probably document somewhere that implementers will be required to turn on a "default allow" type policy or create their own policies for the testing traffic.

[robscott]

Alternatively, at bare minimum we should probably document somewhere that implementers will be required to turn on a "default allow" type policy or create their own policies for the testing traffic.

This is the direction I'd lean. It seems like the simplest approach is just for us to document what we expect a cluster to be capable of to run conformance tests. I think it could get rather challenging to maintain a set of Network Policies that lined up with our tests. In general, I think the expectation is that conformance tests will be run on dev/test clusters that do not have any kind of significant NetworkPolicy restrictions.

[pmalek]

Alrighty. I've created #2296 to address this in the docs. PTAL @robscott @shaneutt