DVT Compatibility

[AgeManning]

Description

There are two endpoints in the beaconapi specs, specifically designed for DVT middleware:

• https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Validator/submitBeaconCommitteeSelections
• https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Validator/submitSyncCommitteeSelections

We have a --distributed flag in the VC now. We should implement these endpoints on the BN and when this flag is set on the VC use these endpoints.

There is further information and testing links in this comment: #4867 (comment)

[michaelsproul]

My understanding is that we do not need to implement these on the BN: just to use them from the VC.

[michaelsproul]

I will investigate

[OisinKyne]

My understanding is that we do not need to implement these on the BN: just to use them from the VC.

Correct. You just need to call them with the VC, charon returns an aggregated signature in exchange for the partial one sent, and then the aggregation selection process continues as normal from there :)

[OisinKyne]

Hey @michaelsproul, have you had a chance to take a look at this issue yet? :)

[michaelsproul]

@OisinKyne No, sorry! It fell off my todo list, will try to make some time for it now Electra is stabilising

[michaelsproul]

@chong-he is going to take a look at this.

This is the point to get started (for the aggregate attestation API):

lighthouse/validator_client/validator_services/src/duties_service.rs

Lines 128 to 150 in 029b4f2

	async fn make_selection_proof<T: SlotClock + 'static, E: EthSpec>(
	duty: &AttesterData,
	validator_store: &ValidatorStore<T, E>,
	spec: &ChainSpec,
	) -> Result<Option<SelectionProof>, Error> {
	let selection_proof = validator_store
	.produce_selection_proof(duty.pubkey, duty.slot)
	.await
	.map_err(Error::FailedToProduceSelectionProof)?;
	selection_proof
	.is_aggregator(duty.committee_length as usize, spec)
	.map_err(Error::InvalidModulo)
	.map(|is_aggregator| {
	if is_aggregator {
	Some(selection_proof)
	} else {
	// Don't bother storing the selection proof if the validator isn't an
	// aggregator, we won't need it.
	None
	}
	})
	}

We need to:

• Define the type for BeaconCommitteeSelectionRequest (from beacon-api).
• If --distributed mode is enableld: add an HTTP call to the aggregate committee API (using first_success on the list of beacon nodes is fine).
• Use the response from the API as the selection_proof, and keep the same logic for checking whether the validator is an aggregator (is_aggregator).

[OisinKyne]

Hey @chong-he, let me know if you have any questions on what's involved. :)

[chong-he]

Hey @chong-he, let me know if you have any questions on what's involved. :)

Hey there! I have created a PR for this issue and going to test the beacon committee aggregator. I see your reply here that there is a tool ready for testing: #4867 (comment), so going to give this a try

[chong-he]

Hi @OisinKyne , I am setting up a Kurtosis-Charon testnet to get familiar with Obol before the PR #7016 change. I got the testnet up and it is producing blocks normally. But I wanted to know how Charon works and it I have set it up properly, so I have a few questions:

1. How to know which validators are having partial keys, which are full keys? or once make run-charon-lighthouse is run, the validators are already all using partial keys?

2. As I understand Lighthouse VC still doesn't support the endpoint now, so I thought the aggregated attestations will not work, but I am seeing aggregated attestations in the VC logs, and also in Charon:

"log":"03:21:35.326 INFO tracker    Broadcasted attestation included on-chain {\"block_slot\": 4068, \"attestation_slot\": 4067, \"pubkey\": \"b3f_2d9\", \"inclusion_delay\": 1, \"broadcast_delay\": \"4.712200459s\", \"aggregate_len\": 24, \"aggregated\": true}\n","stream":"stderr","time":"2025-02-27T03:21:35.32649264Z"}

3. From the repository, at step 8, there is a Grafana instance that I couldn't access. I asked in the Discord channel about this: https://discord.com/channels/849256203614945310/970759460693901362/1344318464160174090, appreciate if you can provide some insights in this.

I also reached out to you via Discord DM @Okyne so feel free to reply there.

Thanks

[KaloyanTanev]

Hey @chong-he , thank you for taking up this issue!

1. Indeed, once we start the 3 charon nodes, we are working with partial keys from the 3 lighthouses that we've started (images for those can be observed in the docker-compose.yml)

2. Hmmm, how did you start the setup? What I see in Charon logs is:
   node0 | 18:10:02.000 DEBG fetcher Timeout calling fetcher/fetch, duty expired {"duty": "32/aggregator"}
   and
   node0 | 18:10:02.004 DEBG fetcher Timeout calling fetcher/fetch, duty expired {"duty": "32/sync_contribution"}

Then in Lighthouse VC I see:
vc0-lighthouse-1 | Feb 27 18:15:22.018 CRIT Error during attestation routine slot: 61, committee_index: 0, error: "Some endpoints failed, num_failed: 2 http://node0:3600/ => RequestFailed(\"Failed to produce an aggregate attestation: HttpClient(url: http://node0:3600/, kind: timeout, detail: operation timed out)\"), http://node0:3600/ => RequestFailed(\"Failed to produce an aggregate attestation: HttpClient(url: http://node0:3600/, kind: timeout, detail: operation timed out)\")", service: attestation

3. I think there are some credentials required in order to access it, I will check internally and let you know.

Now, couple more things from my side:

1. Feel free to reach out to me on Obol's Discord as well, we have a sigma-prime channel there (you can find me as Kalo).
2. If I can share some examples of how other client teams implemented this flow and it will help you - here is Prysm and how they use the distributed flag we put when we start Prysm VC:

• checking if validator is beacon aggregator
• submitting aggregation
• checking if validator is sync aggregator
• submitting sync aggregation
  Of course, don't take it for granted, as there might be other, better approaches.

[chong-he]

Hey @chong-he , thank you for taking up this issue!

Thank you very much for the detailed reply. I search through the log you gave and also see it in my charon docker container (it's node0, node1 or node2) now:

{"log":"17:54:11.000 DEBG fetcher Timeout calling fetcher/fetch, duty expired {\"duty\": \"1232/aggregator\"}\n","stream":"stderr","time":"2025-02-26T17:54:11.000580253Z"}

I now see the following in the docker container docker logs -f kurtosis-charon-vc1-lighthouse-1 which is like what you shown:

CRIT Error during attestation routine slot: 69, committee_index: 0, error: "Some endpoints failed, num_failed: 1 http://node1:3600/ => RequestFailed(\"Failed to produce an aggregate attestation: HttpClient(url: http://node1:3600/, kind: timeout, detail: operation timed out)\")", service: attestation

Previously I was looking at the Kurtosis instance such as vc-2-geth-lighthouse that shows the aggregated attestations still going on normally:

INFO Successfully published attestation type: aggregated, slot: 94, committee_index: 0, head_block: 0xe9b87e17f6b11bd3525ebdcd9b9b7539ac08f19b4b5560c884b019d6185503f5, signatures: 15, aggregator: 389, service: attestation

So I thought aggregate attestation was working (while it shouldn't)?

I am in the Obol discord but clicking the link shows messages failed to load. Anyway, we can continue in the #support-testnet channel?

Thanks!