Improved version of ECHCheck test

[hellais]

This issue is to track the work @johnhess is doing on expanding our echcheck test to fully support the ECH handshake, see: johnhess/probe-cli#1.

What we should ensure happens as part of this new iteration is:

• Add support for truly speaking ECH (as opposed to just greased ECH), using the tls client implementation in golang stdlib
• Get rid of the utls dependency
• Add support for setting the queries, tcp_connect and network_events keys

xref: #1453

[johnhess]

The first and second should be done already :-)

Re: the third item, is the intention that the probe would also populate results for the other dataformats e.g. queries? If so, I can add those. Which network events do you want to see captured?

[hellais]

is the intention that the probe would also populate results for the other dataformats e.g. queries

Yes that's correct. These should already be emitted by the measurement library functions you are using.

Which network events do you want to see captured?

Similarly also network_events are automatically collected and you just have to add them to your test_keys.

Here are some examples of how this is done in other tests

network_events: https://github.com/ooni/probe-cli/blob/ef139c525f1b1e751f21cccf842982b4a2720dbd/internal/experiment/simplequicping/simplequicping.go#L190

queries, tcp_connect: https://github.com/ooni/probe-cli/blob/ef139c525f1b1e751f21cccf842982b4a2720dbd/internal/experiment/urlgetter/getter.go#L62

[johnhess]

Working on that. Ran into a semantics question.

When we attempt to do a GREASE connection, we get an error. We expect that error; it's an EchRejectionError! We can & do use the retry_configs that come with it to attempt a real ECH connection.

Question: Do we want to log 2 ArchivalTLSOrQUICHandshakeResults with the GREASE'd one as having a Failure? A SoError? My guess is "no, that's not a failure" since it's doing what we expect (even if golang is returning a conspicuous error to the caller so they know they didn't get the ECH they wanted).

[johnhess]

For net_events: do we want to capture all net events for the DNS queries, then again for each TCP+TLS attempt? Or just a subset?

[hellais]

Do we want to log 2 ArchivalTLSOrQUICHandshakeResults with the GREASE'd one as having a Failure? A SoError? My guess is "no, that's not a failure" since it's doing what we expect (even if golang is returning a conspicuous error to the caller so they know they didn't get the ECH they wanted).

This is a good point.

Is there the possibility that we do not get an error when doing a GREASE'd handshake? If that is a possibility and we would like to differentiate between the "no failure" (which is actually unexpected) and "failure" (which is expected), then we should be setting the failure string to something like "ech_rejected". OTOH if it's never possible to have the GREASE'd handshake succeed, then we should probably not log it as a failure and just set this key to null (as we have been doing up until now).

For net_events: do we want to capture all net events for the DNS queries, then again for each TCP+TLS attempt? Or just a subset?

I would say we should start off with recording all of them. It's going to be quite useful to investigate cases of blocking to understand at which point exactly it's happening and isolate at which point during the network operations they are occuring.

[aanorbel]

At the moment, echcheck does not include a url in status.measurement_start event. It would be great to also have that.

We currently have this workaround to get the URL. ooni/probe-multiplatform#437

[johnhess]

if it's never possible to have the GREASE'd handshake succeed, then we should probably not log it as a failure and just set this key to null (as we have been doing up until now).

It's always rejected; left as-is.

start off with recording all of them

I've added captures of net_events and dns queries using the trace.

At the moment, echcheck does not include a url in status.measurement_start event. It would be great to also have that.

I'm not clear on how to do this. I see the richer input doc & that taskmodel.go defines the type of event to be emitted. But I don't know when in the experiment one would do that or through which facility I would emit that event. Are there experiments that model this?

[johnhess]

@hellais and @aanorbel

I'm happy to leave this PR as-is for y'all to make use of when the rest of OONI makes it to golang 1.23.4 or later. If there are changes you want to see before I put this on the shelf (see comment above) please let me know in the next few days. Thanks!

[DecFox]

Hey @johnhess thanks for contributing to OONI with the new echcheck experiment. We have decided to upgrade to go 1.23.4 for the next probe-cli release. Over the next days, I am going to push some changes and also integrate the PR here: ooni/probe-cli#1693 into master. I hope that's alright. I'll leave some feedback (if required) on the PR and might push some changes directly as well. Thanks!