[group key addrs 4/5]: implement mailbox server and message store database #1614

guggero · 2025-06-23T14:56:45Z

Depends on #1611.

Extracted some commits from #1587.

Implements the SQL store for the auth mailbox server and fully hooks up the server in tapd.
The implementation is covered by a basic message send and receive integration test.

ffranr · 2025-06-24T18:24:11Z

authmailbox/server.go

 	if err != nil {
 		return nil, fmt.Errorf("error storing message: %w", err)
 	}

 	// Publish the message to all subscribers, meaning it will be
 	// distributed to all subscribed streams. Each stream will filter by
 	// recipient ID by itself.
+	msg.ID = msgID


We assign msg.ID to the db table’s auto-incrementing primary key. Exposing that surrogate key externally couples the database to the API.

Another possible solution: generate a UUIDv4 for each message instead, store it in the db row, and keep the integer primary key internal to the db only. This decouples external identifiers from the DB’s bookkeeping (makes table rebuilds/merge safer because message IDs never collide?).

Not a big deal, but something that might be worth exploring.

Do we actually even use this ID on the proto layer?

IIUC, the client uses a block height range to scan for new messages.

We use the ID to query as well. That makes things a bit easier if you know exactly which ID you received last. Because there could be multiple messages per block, so just using the block height might give you messages you've already seen.

Having a comparable (e.g. numeric) ID is useful here, exactly for the above case. Why do you think exposing the database primary ID is a bad thing on the API level, @ffranr? It's really just a number...

ffranr · 2025-06-24T18:25:17Z

authmailbox/server.go

@@ -234,13 +236,14 @@ func (s *Server) SendMessage(ctx context.Context,
 		// We didn't have the proof before, so we store it now. If at
 		// the same time a different goroutine is trying to store the
 		// same proof, we expect the database to handle the concurrency.
-		err = s.cfg.TxProofStore.StoreProof(txProof.ClaimedOutPoint)
+		err = s.cfg.TxProofStore.StoreProof(ctx, *txProof)


Roasbeef · 2025-06-25T01:15:01Z

Needs to be rebased due to recent merges increasing the highest migration number.

Roasbeef

LGTM 🐩

Roasbeef · 2025-06-25T01:25:45Z

tapdb/sqlc/migrations/000036_authmailbox.up.sql

+    -- The timestamp when the message was created on the server. This is a unix
+    -- timestamp in seconds to allow for easy querying and sorting of messages
+    -- based on their arrival time, without time zone complications.
+    arrival_timestamp BIGINT NOT NULL,


Roasbeef · 2025-06-25T01:27:48Z

tapdb/authmailbox.go

+		// Add context to unique constraint errors.
+		var uniqueConstraintErr *ErrSqlUniqueConstraintViolation
+		if errors.As(dbErr, &uniqueConstraintErr) {
+			return 0, proof.ErrTxMerkleProofExists


Roasbeef · 2025-06-25T01:34:01Z

tapdb/interfaces.go

 	ReadOnly() bool
 }

+// BaseTxOptions defines the set of db txn options the database understands.
+type BaseTxOptions struct {


Nice. TBH, we can prob get rid of all the specific tx options we have like BaseUniverseStoreOptions and just use this instead. Non-blocking can do it in some other PR as it'll increase the diff size and scope here.

Roasbeef · 2025-06-25T01:34:15Z

proof/tx.go

@@ -418,9 +419,9 @@ func UnmarshalTxProof(
 type TxProofStore interface {
 	// HaveProof returns true if the proof for the given outpoint exists in
 	// the store.
-	HaveProof(wire.OutPoint) (bool, error)
+	HaveProof(context.Context, wire.OutPoint) (bool, error)


coveralls · 2025-06-25T08:10:52Z

Pull Request Test Coverage Report for Build 15873763187

Details

251 of 359 (69.92%) changed or added relevant lines in 16 files are covered.
23 unchanged lines in 6 files lost coverage.
Overall coverage increased (+0.2%) to 38.759%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
itest/tapd_harness.go	0	1	0.0%
tapcfg/config.go	0	1	0.0%
authmailbox/mock.go	6	8	75.0%
cmd/commands/conn.go	0	3	0.0%
itest/loadtest/utils.go	0	3	0.0%
log.go	0	3	0.0%
proof/mock.go	0	6	0.0%
tapdb/sqlc/authmailbox.sql.go	73	81	90.12%
tapcfg/server.go	0	12	0.0%
server.go	0	22	0.0%

Files with Coverage Reduction	New Missed Lines	%
cmd/commands/conn.go	1	0.0%
address/mock.go	2	88.24%
commitment/tap.go	3	71.14%
tapchannel/aux_leaf_signer.go	3	43.43%
tapdb/multiverse.go	6	53.28%
tapgarden/caretaker.go	8	67.53%

Totals
Change from base Build 15865241809:	0.2%
Covered Lines:	29824
Relevant Lines:	76948

💛 - Coveralls

This commit implements the MailboxStore which implements a SQL based implementation for the authmailbox.MsgStore and proof.TxProofStore interfaces.

To avoid collisions with the universerpc.Info RPC method, we rename our authmailbox Info RPC method to MailboxInfo.

We now plug everything together and make sure that the auth mailbox server works by adding a new integration test.

The rapid generator for strings seems to occasionally produce invalid UTF-8 strings, causing an error with the Postgres driver. We fix that by using hex encoded bytes instead.

guggero requested review from Roasbeef and ffranr June 23, 2025 14:56

guggero added this to Taproot-Assets Project Board Jun 23, 2025

github-project-automation bot moved this to 🆕 New in Taproot-Assets Project Board Jun 23, 2025

guggero added the group_key label Jun 23, 2025

guggero added this to the v0.7 milestone Jun 23, 2025

guggero self-assigned this Jun 23, 2025

guggero mentioned this pull request Jun 23, 2025

[group key addrs 6/6]: send and receive support for V2 addresses #1587

Draft

levmi moved this from 🆕 New to 🏗 In progress in Taproot-Assets Project Board Jun 23, 2025

guggero moved this from 🏗 In progress to 👀 In review in Taproot-Assets Project Board Jun 23, 2025

guggero force-pushed the misc-refactor branch from 81fbf58 to c7c21af Compare June 24, 2025 07:32

guggero changed the base branch from misc-refactor to main June 24, 2025 08:25

guggero force-pushed the add-mailbox-server branch from 094b31e to 3d1a126 Compare June 24, 2025 08:27

levmi added the P0 label Jun 24, 2025

ffranr approved these changes Jun 24, 2025

View reviewed changes

Roasbeef approved these changes Jun 25, 2025

View reviewed changes

guggero force-pushed the add-mailbox-server branch 2 times, most recently from 636d80d to dc426c6 Compare June 25, 2025 07:48

guggero added 3 commits June 25, 2025 10:49

multi: implement SQL based auth mailbox stores

7793bf9

This commit implements the MailboxStore which implements a SQL based implementation for the authmailbox.MsgStore and proof.TxProofStore interfaces.

authmailbox+taprpc: rename Info RPC to MailboxInfo

a46f6a5

To avoid collisions with the universerpc.Info RPC method, we rename our authmailbox Info RPC method to MailboxInfo.

multi: add, initialize and test mailbox RPC server

74953ff

We now plug everything together and make sure that the auth mailbox server works by adding a new integration test.

guggero force-pushed the add-mailbox-server branch from dc426c6 to 74953ff Compare June 25, 2025 08:49

asset+tapdb: fix flake by generating valid asset names

8b6e3d1

The rapid generator for strings seems to occasionally produce invalid UTF-8 strings, causing an error with the Postgres driver. We fix that by using hex encoded bytes instead.

guggero mentioned this pull request Jun 25, 2025

ecies: add version byte prefix to encoding format #1619

Merged

guggero merged commit 443d61a into main Jun 25, 2025
18 checks passed

github-project-automation bot moved this from 👀 In review to ✅ Done in Taproot-Assets Project Board Jun 25, 2025

guggero deleted the add-mailbox-server branch June 26, 2025 10:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[group key addrs 4/5]: implement mailbox server and message store database #1614

[group key addrs 4/5]: implement mailbox server and message store database #1614

guggero commented Jun 23, 2025

Uh oh!

ffranr Jun 24, 2025

Uh oh!

Roasbeef Jun 25, 2025

Uh oh!

guggero Jun 25, 2025

Uh oh!

ffranr Jun 24, 2025

Uh oh!

Roasbeef commented Jun 25, 2025

Uh oh!

Roasbeef left a comment

Uh oh!

Roasbeef Jun 25, 2025

Uh oh!

Roasbeef Jun 25, 2025

Uh oh!

Roasbeef Jun 25, 2025

Uh oh!

Roasbeef Jun 25, 2025

Uh oh!

coveralls commented Jun 25, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

[group key addrs 4/5]: implement mailbox server and message store database #1614

[group key addrs 4/5]: implement mailbox server and message store database #1614

Conversation

guggero commented Jun 23, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Roasbeef commented Jun 25, 2025

Uh oh!

Roasbeef left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

coveralls commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 15873763187

Details

💛 - Coveralls

Uh oh!

Uh oh!

Uh oh!

coveralls commented Jun 25, 2025 •

edited

Loading