Investigate recurring failure of test_apply_context_remote

[PProfizi]

The test_apply_context_remote has been the source of very common random failure of CIs for the past two weeks.
This task track investigations on the reason behind this flakiness.

Let's try and reproduce the random failure locally, design tests to catch a potential underlying bug, or look at redesigning the current test to clarify behavior.
Also waiting on #2167 to get test analytics.