Template: Fix faulty CORS headers handling.

[elizabeth-dev]

What this PR does / why we need it:

CORS headers handling implementation has been producing inconsistent results for a bit due to the unusual way Nginx handles if directives (essentially, they are to include return and rewrite directives only inside them). This change recreates the exact same logic we were trying to apply whe handling CORS, but using NJS scripting.

This also includes the NJS module into our custom build of Nginx, since it was going to be necessary for #12383 anyway.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• CVE Report (Scanner found CVE and adding report)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation only

Which issue/s this PR fixes

fixes #10267

How Has This Been Tested?

Test cases including multiple origin, headers, and methods have been run using curl, in order to ensure all the different CORS headers behave accordingly to the different annotation values we allow. This also included the specific configuration a user reported failing in the linked issue.

Additionally, most of the CORS e2e test suite has been revised and improved, relying more on actually checking the result of HTTP requests rather than just checking the contents of the nginx.conf file.

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I've read the CONTRIBUTION guide
• I have added unit and/or e2e tests to cover my changes.
• All new and existing tests passed.

[k8s-ci-robot]

Hi @elizabeth-dev. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[netlify]

✅ Deploy Preview for kubernetes-ingress-nginx canceled.

Name	Link
🔨 Latest commit	1fe7fe4
🔍 Latest deploy log	https://app.netlify.com/projects/kubernetes-ingress-nginx/deploys/68d91dea6dd52d0008380f31

[Gacko]

I'd not call it a fix if we need to introduce NJS for it. We also already have a different PR for introducing NJS but as this needs a working NGINX base image build, we first need to fix this, introduce NJS and then can rebase this PR on top.

[elizabeth-dev]

@Gacko in that case we can prioritize and merge the NJS PR first, and then add this on top of it as just a fix

[Gacko]

Yes, makes sense! Sadly we can first release it in v1.13. Also I wouldn't word it as a fix then. Does this make sense?

[elizabeth-dev]

@Gacko hmm, not sure if I understand. you wouldn't tag the other PR (adding NJS + some migrations) as a fix, I agree with that, but once that is merged whenever, we can put this change on top, then it'd just be the scripting part, and that would pass as "just a fix" for me, since no new modules would be added and the logic would stay the same (but working as expected)

or do you prefer to pass both changes as new features?

[Gacko]

Hm, yeah, you're right. We can consider this a fix. It's just a bit complex as we are also changing the whole framework the implementation is based on.

[elizabeth-dev]

well, that's settled then. we'll see if we can fix the build issues soon

/hold

[Gacko]

/triage accepted
/kind bug
/priority backlog
/hold

Sadly we cannot back-port this as it requires the NGINX NJS module in the NGINX base image. Therefore I'll first build and promote a new NGINX base image including NJS, include it on main and LGTM this PR afterwards.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: elizabeth-dev, Gacko

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Gacko]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Gacko]

@elizabeth-dev We added NJS to the NGINX base image and included it on main. Sadly E2E tests are failing at the moment. Can you investigate the root cause please? I'd be happy to get this merged as soon as possible. 🙂

[rikatz]

/uncc
/unassign

[strongjz]

/uncc
/unassign

[bo0tzz]

I recently ran into this issue and decided to see if I can help unstuck this PR. The logs from the CI run are old and expired, but running locally I see the same error as mentioned in #13135. A comment there mentions that the move to NJS was dropped, which I think obsoletes the approach taken in this PR? That got me wondering why JS is used here instead of header_filter_by_lua_block - what understanding am I missing there? If 'translating' this PR to Lua is all it would take, I'd be happy to take a stab at that.