Skip to content

universe+tapdb: add new block_height field to universe_leaves use that to implement block height based supply tree queries #1596

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: supply-commit-machine
Choose a base branch
from

Conversation

Roasbeef
Copy link
Member

In this PR, we add a new method FetchSupplyLeavesByHeight, that can be used to query for the leaves of a supply tree by height. This will be useful for writing the new syncing state machine and the sub-system that serves the supply tree syncer. Such a syncer would start from the initially created pre-commitment, then fetch all the leaves for the future updates, assemble those into the supply tree, then verify that everything matches up.

Prompts used during development (aider+Gemini 2.5-pro-06-5)

Initial draft creation

Based on by working memory of this new feature, I scanned the relevant files, then made mental nodes of what needed to be updated. I then drafted this fairly detailed prompt that guided aider w.r.t which files to update, and the set of high level changes to make.

#### Today we assemble the individual updates (burn, ignore, mint) for a given asset  
#### specifier (group key is what we care about) into distinct trees for each type.  
#### We then further aggregrate each of those roots into a unified supply tree for  
#### the entire asset specifier. This tree commits to a root sum of the supply (you  
#### can subtract out any of the roots from the root sum to derive a snapshot of the  
#### supply such as the amt burnt, etc).  
####   
#### We then also have a system to assemble these updates, then commit then in a  
#### batch, this exists in: tapdb/supply_commit.go and tapdb/supply_tree.go.  
####   
#### For each of the supply sub-trees, we actually re-use the existing  
#### universe_leaves table to point to each of the respective leaves.  
####   
#### In this task, we want to add a block height to this leaf, as this'll allow us  
#### to scan the leaves in order, based on the type, so we can send out the set of  
#### leaves added in a state transition to a party that wants to sync the set of  
#### leaves.  
####   
####   
#### For this, we may need to update the set of queries in:  
#### tapdb/sqlc/queries/universe.sql. Of interest is also the  
#### applySupplyUpdatesInternal helper function, as we'll want to make sure that the  
#### helper functions that it calls properly inserts the height into the ultimately  
#### universe leaf. For the mint and burn types, we can obtain this height from the  
#### issuance proof.  
####   
#### For the ignore type, we'll need to add a block height field to  
#### universe.IgnoreTuple (universe/ignore_records.go).  
####   
#### For the leaves, we'll likely need to add a new migration (38) which adds the  
#### new height field (can be NULL) to the universe_leaves table.  
####   
#### We'll likely also want to add a new BlockHeight() method to the  
#### SupplyUpdateEvent interface.  
####   
#### Next, we'll want a new method on the SupplyTreeStore that given a start+end  
#### height and a specifier, the set of updates whose universe_leaves have a block  
#### height between that param. This may need to update  
#### tapdb/sqlc/queries/supply_tree.sql.  
####   
#### Also see tapdb/querier.go.  
####   
#### Ruminate on my sketch above, then let's create an execution plan remembering  
#### the ultimate goal. From there we'll implement each portion.  

Subsequent iterations

From here I made an edit or two to get things compiling. Then I executed a series of prompts to refine the code, refactor slightly, then add unit tests.

inside multiverse.UpsertProof Leaf, let's use SparseDecode instead along with BlockHeightRecord as we don't need to decode the entire proof just to get the block height

let's use the same sparse decode in BaseUniverseStore.UpsertProofLeaf

let's do the same for SupplyTreeStore.RegisterMintSupply (sparse decode)

can we morph that into a function closure invocation that itself returns lfn.Option[uint32]? so we don't leak that intermediate variable to the higher scope

great! but now we have a bit of duplication, can we make a helper function can call that intead of having the same duplication everywhere? we can put the helper function in tapdb/sqlutils.go

nice! now let's extend the unit tests in tapdb/supply_tree_test.go to cover the new FetchSupplyLeavesByHeight method.

I messed when committing and dropped a commit entirely (lol), so I had aider re-add it:

help me ree-add QuerySupplyLeavesByHeight back to tapdb/sqlc/queries/supply_tree.sql, I deleted it accidentally

Comment on lines +599 to +611
// SupplyUpdate is a struct that holds a supply update event and its block
// height.
type SupplyUpdate struct {
supplycommit.SupplyUpdateEvent
BlockHeight uint32
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

supplycommit.SupplyUpdateEvent already includes

// BlockHeight returns the block height of the update.
BlockHeight() uint32

so I don't think we need the separate BlockHeight uint32 field in SupplyUpdate.

@Roasbeef Roasbeef force-pushed the supply-leaf-height-query branch from 3e7ed65 to c57d19d Compare June 18, 2025 01:51
@Roasbeef Roasbeef force-pushed the supply-commit-machine branch from c2c394f to b1173b9 Compare June 18, 2025 01:51
@Roasbeef Roasbeef force-pushed the supply-leaf-height-query branch from c57d19d to bd5d0b4 Compare June 18, 2025 02:06
@Roasbeef Roasbeef force-pushed the supply-commit-machine branch from b1173b9 to c6f9962 Compare June 18, 2025 02:06
@Roasbeef Roasbeef force-pushed the supply-leaf-height-query branch from bd5d0b4 to 37f23fa Compare June 18, 2025 23:02
@Roasbeef Roasbeef force-pushed the supply-commit-machine branch from c6f9962 to da0df38 Compare June 18, 2025 23:02
@Roasbeef Roasbeef force-pushed the supply-leaf-height-query branch from 37f23fa to be0a52c Compare June 18, 2025 23:25
@Roasbeef Roasbeef force-pushed the supply-commit-machine branch 2 times, most recently from ee045a1 to 598067d Compare June 18, 2025 23:51
@Roasbeef Roasbeef force-pushed the supply-leaf-height-query branch from be0a52c to f2a69aa Compare June 18, 2025 23:57
@levmi levmi moved this from 🆕 New to 🏗 In progress in Taproot-Assets Project Board Jun 19, 2025
@levmi levmi moved this from 🏗 In progress to 👀 In review in Taproot-Assets Project Board Jun 19, 2025
@lightninglabs-deploy
Copy link

@Roasbeef, remember to re-request review from reviewers when ready

@Roasbeef Roasbeef force-pushed the supply-leaf-height-query branch from f2a69aa to 93ed469 Compare June 20, 2025 23:49
@Roasbeef Roasbeef force-pushed the supply-commit-machine branch from 598067d to 9f03d81 Compare June 20, 2025 23:49
@Roasbeef Roasbeef force-pushed the supply-leaf-height-query branch from 93ed469 to 633373a Compare June 21, 2025 00:08
@Roasbeef Roasbeef force-pushed the supply-commit-machine branch from 9f03d81 to e77d0cf Compare June 21, 2025 00:08
@Roasbeef Roasbeef force-pushed the supply-leaf-height-query branch from 633373a to 3ca1517 Compare June 21, 2025 00:12
@Roasbeef Roasbeef force-pushed the supply-commit-machine branch from e77d0cf to 3044a87 Compare June 21, 2025 00:12
@Roasbeef Roasbeef force-pushed the supply-leaf-height-query branch from 3ca1517 to 988837b Compare June 21, 2025 00:20
@Roasbeef Roasbeef force-pushed the supply-commit-machine branch from 3044a87 to 9854971 Compare June 21, 2025 00:20
Roasbeef added 2 commits June 20, 2025 18:22
In this commit, we add a migratino to add a block_height to the universe
leaf table. We'll use this in the future to be able to allow clinets to
sync block by block for the new supply universe trees.
Roasbeef added 4 commits June 20, 2025 18:22
In this commit, we add a new method to the SupplyTreeStore that is able
to read out the leaves of a supply tree based on a start and end height.
This will be useful for writing the new syncing state machine and the
sub-system that serves the supply tree syncer.
@Roasbeef Roasbeef force-pushed the supply-leaf-height-query branch from 988837b to a9ef139 Compare June 21, 2025 01:22
@Roasbeef Roasbeef force-pushed the supply-commit-machine branch from 9854971 to 37bdcfa Compare June 21, 2025 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 👀 In review
Development

Successfully merging this pull request may close these issues.

3 participants