From 8e426287c6b7195f0a95f3b4fb1c92aa6a619ff0 Mon Sep 17 00:00:00 2001 From: Pavel Kalinnikov Date: Mon, 9 May 2022 17:28:04 +0000 Subject: [PATCH] docs: Data model for append-only Merkle trees --- docs/data_model.md | 51 ++- docs/images/data_model.svg | 899 +++++++++++++++++++++++++++++++++++++ 2 files changed, 942 insertions(+), 8 deletions(-) create mode 100644 docs/images/data_model.svg diff --git a/docs/data_model.md b/docs/data_model.md index cc3e131..df48a1b 100644 --- a/docs/data_model.md +++ b/docs/data_model.md @@ -1,13 +1,48 @@ Data Model ========== -This document establishes terminology shared throughout this repositotry around the -structure and components of the Merkle tree. - -TODO. Things to cover: -- Node addressing. Each node is a (level, index) pair. -- Leaves, nodes, roots. -- Perfect and ephemeral nodes. -- Append-only, perfect nodes are immutable. +This document establishes terminology shared throughout this repositotry around +the structure and components of the Merkle tree. + +### Merkle tree + +In this repository, by Merkle trees we mean a slightly modified version of +"history trees" introduced by Crosby and Wallach in the *Efficient Data +Structures for Tamper-Evident Logging* +[paper](https://static.usenix.org/event/sec09/tech/full_papers/crosby.pdf). + +![data_model](images/data_model.svg) + +The basis of the data structure is an append-only **log** consisting of log +**entries**, numbered with consecutive integers starting from 0. + +Built on top of the log entries, there is a highly regular structure of nodes of +a perfect binary tree-like shape. Nodes are addressed as `(level, index)` pairs. +Nodes at level 0 are called **leaf nodes**, and correspond directly to the log +entries. Nodes at higher levels are defined recursively based on nodes of the +lower levels. Specifically, `(level, index)` depends directly on nodes +`(level-1, index*2)` and `(level-1, index*2 + 1)`, and, recursively, nodes of +its entire subtree. + +The data structure evolves dynamically. Initially, the log is empty, and all +the nodes of the tree have no data. When new entries are appended to the log, +nodes that recursively depend on these entries are updated. While the nodes can +be updated, they are in **ephemeral** state. Eventually, when the log grows +past a certain size, a node becomes **perfect**, and is never modified again. + +Effectively, perfect nodes are immutable / write-once registers, and ephemeral +nodes are mutable. + +### Tree state + + + +To represent the state of the entire log, often a single ephemeral node is used +which covers all the corresponding leaves. For example, in a tree with 21 +leaves, as in the picture above, this would be the ephemeral node 5.0. + +### TODO: Things to cover: + +- Append-only. - Merkle tree hasher. - Structured objects, like proofs and compact ranges. diff --git a/docs/images/data_model.svg b/docs/images/data_model.svg new file mode 100644 index 0000000..599c8a0 --- /dev/null +++ b/docs/images/data_model.svg @@ -0,0 +1,899 @@ + + + + + + + + + + + + + + + + Produced by OmniGraffle 7.12.1 + 2022-05-09 21:38:57 +0000 + + + merkle-tree + + + Layer 1 + + + + + + + + + + + 1.0 + + + + + + + 0 + + + + + + + 1 + + + + + + + + + + + + + 1.1 + + + + + + + 2 + + + + + + + 3 + + + + + + + + + + + + + 1.2 + + + + + + + 4 + + + + + + + 5 + + + + + + + + + + + + + 1.3 + + + + + + + 6 + + + + + + + 7 + + + + + + + + + + + + + 1.4 + + + + + + + 8 + + + + + + + 9 + + + + + + + + + + + + + 1.5 + + + + + + + 10 + + + + + + + 11 + + + + + + + + + + + + + 1.6 + + + + + + + 12 + + + + + + + 13 + + + + + + + + + + + + + 1.7 + + + + + + + 14 + + + + + + + 15 + + + + + + + 2.0 + + + + + + + 2.1 + + + + + + + 2.2 + + + + + + + 2.3 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 3.0 + + + + + + + 3.1 + + + + + + + 4.0 + + + + + + + + + + + + + + + + + + + + + + + + 5.0 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 1.8 + + + + + + + 16 + + + + + + + 17 + + + + + + + + + + + + + 1.9 + + + + + + + 18 + + + + + + + 19 + + + + + + + + + + + + + 1.10 + + + + + + + 20 + + + + + + + 21 + + + + + + + + + + + + + 1.11 + + + + + + + 22 + + + + + + + 23 + + + + + + + + + + + + + 1.12 + + + + + + + 24 + + + + + + + 25 + + + + + + + + + + + + 1.13 + + + + + + 26 + + + + + + 27 + + + + + + + + + + + + 1.14 + + + + + + 28 + + + + + + 29 + + + + + + + + + + + + 1.15 + + + + + + 30 + + + + + + 31 + + + + + + + 2.4 + + + + + + + 2.5 + + + + + + 2.6 + + + + + + 2.7 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 3.2 + + + + + + 3.3 + + + + + + 4.1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Entries + + + + + + Level 0 + + + + + + Level 1 + + + + + + Level 2 + + + + + + Level 3 + + + + + + Level 4 + + + + + + Level 5 + + + + +