How to reconstruct zk hyperchain

[fborello-lambda]

Purpose

Reconstruct zk hyperchain using this tool.

Context

Hi, im deploying a zk chain on sepolia using the zksync-era commit core-v24.7.0.

The following reasoning was used to run the program with the custom hyperchain deploy:

1. In order to customize the program, the variables inside state-reconstruct-fetcher/src/constants.rs has to be changed:

pub mod ethereum {
    /// Number of Ethereum blocks to advance in one filter step.
    pub const BLOCK_STEP: u64 = 1_000;

    /// Block number in Ethereum for zkSync genesis block.
    //pub const GENESIS_BLOCK: u64 = 16_627_460;
    pub const GENESIS_BLOCK: u64 = <BlockHeight where the DiamondProxy contract was deployed>;

    /// Block number in Ethereum of the first Boojum-formatted block.
    //pub const BOOJUM_BLOCK: u64 = 18_715_403;
    pub const BOOJUM_BLOCK: u64 = <GenesisBlock>;

    /// Block number in Ethereum of the first block storing pubdata within blobs.
    //pub const BLOB_BLOCK: u64 = 19_412_473;
    // Sepolia Dencun Upgrade
    pub const BLOB_BLOCK: u64 = 5_187_023;

    /// zkSync smart contract address.
    //pub const ZK_SYNC_ADDR: &str = "0x32400084C286CF3E17e7B677ea9583e60a000324";
    pub const ZK_SYNC_ADDR: &str = <DiamondProxyContractAddr>;

    /// Default Ethereum blob storage URL base.
    //pub const BLOBS_URL: &str = "https://api.blobscan.com/blobs/";
    pub const BLOBS_URL: &str = "https://api.sepolia.blobscan.com/blobs/";
}

pub mod storage {
    /// The path to the initial state file.
    pub const INITAL_STATE_PATH: &str = "InitialState.csv";

    /// The default name of the database.
    pub const DEFAULT_DB_NAME: &str = "db";

    /// The name of the index-to-key database folder.
    pub const INNER_DB_NAME: &str = "inner_db";
}

pub mod zksync {
    /// Bytes in raw L2 to L1 log.
    pub const L2_TO_L1_LOG_SERIALIZE_SIZE: usize = 88;
    // The bitmask by applying which to the compressed state diff metadata we retrieve its operation.
    pub const OPERATION_BITMASK: u8 = 7;
    // The number of bits shifting the compressed state diff metadata by which we retrieve its length.
    pub const LENGTH_BITS_OFFSET: u8 = 3;
    // Size of `CommitBatchInfo.pubdataCommitments` item.
    pub const PUBDATA_COMMITMENT_SIZE: usize = 144;
    // The number of trailing bytes to ignore when using calldata post-blobs. Contains unused blob commitments.
    pub const CALLDATA_SOURCE_TAIL_SIZE: usize = 32;
}

2. Reasoning about how to set the variables:

• GENESIS_BLOCK: Is this the block where the DiamondProxy contract was deployed? Based on assumption: Yes.
• BOOJUM_BLOCK: Since version 24.7.0 deployment does not include the BOOJUM update, should BOOJUM_BLOCK match GENESIS_BLOCK? Based on assumption: Yes.
• BLOB_BLOCK: Set to 5_187_023 for the Sepolia Dencun Upgrade.
• ZK_SYNC_ADDR: Is this the DiamondProxyAddr? How does it interact with Hyperchains? Based on assumption: DiamondProxyAddr.

A snapshot was generated using snapshots-creator from the zksync-era repository, downloaded, and placed in ./storage_logs_snapshots. The following command was then executed:

cargo run --release -- reconstruct --snapshot ./storage_logs_snapshots l1 --http-url https://ethereum-sepolia-rpc.publicnode.com

Snapshots were succesfully imported, but there is a mismatch.

Output:

2024-07-08T13:10:16.024141Z  INFO Succesfully imported snapshot containing 574 storage logs!
2024-07-08T13:10:16.069307Z  INFO PROGRESS: [ - ] CUR BLOCK L1: 6135177 L2: 0 TOTAL BLOCKS PROCESSED L1: 0 L2: 0
2024-07-08T13:10:21.084794Z ERROR Root hash mismatch!
Local: 586db64a95fdda64b42c5d4bd80a785b6fba06d796add5258238a985e56895f5
Published: 43ff9d17f184cd0cc84bcc951359d9c2016c93e7f20b123839bdb8c6ffd4942a

3. The InitialState.csv file contains genesis information. Since no relevant information was found directly in the L1 blockchain of the zksync-era mainnet tracing the GENESIS_BLOCK transactions, the information in the InitialState.csv is assumed to be present in the PostgreSQL database."

InitialState.csv format:

E'\\xDFE5063F03287AEC1B9B52B110A060F7AF3EFF32916C9A7539E66619624394C5',E'\\x0000000000000000000000000000000000008002',E'\\x0000000000000000000000000000000000000000000000000000000000008003',E'\\x010000FF484F38E5D074EC2A2F1D012662D65AAF63E83DCCD4E569B515565069',1,E'\\x0000000000000000000000000000000000000000000000000000000000000000',0,2023-02-14 13:52:18.104630,2023-02-14 13:52:18.104630,0

The initial_writes table in the zksync_local database is a good candidate:

                      List of relations
 Schema |               Name               | Type  |  Owner   
--------+----------------------------------+-------+----------
 public | initial_writes                   | table | postgres

initial_writes was copied into a .csv with the following psql command:

\COPY initial_writes TO '/home/admin/InitialState.csv' CSV HEADER

DB output format:

\xf703c2abe67f32afad15f6055bad876ec5f3510c7ed27cc0add665da95b51f21,0,2024-06-18 18:00:29.8804,2024-06-18 18:00:29.8804,1
\x7715a1136f6dc78cdd2011cb8a0f003839c13fe6239cc70539101075c3ad92a7,0,2024-06-18 18:00:29.8804,2024-06-18 18:00:29.8804,2
\x65f891e1f428d2b264c7983f4d7f4615f96a5ea6411df55455ff1ea4319ba72e,0,2024-06-18 18:00:29.8804,2024-06-18 18:00:29.8804,3
\xe499632b8273a7e14b70cb53d633a028b56243b43f326acee6d75cf8aa665e98,0,2024-06-18 18:00:29.8804,2024-06-18 18:00:29.8804,4
\x35cc470860885f9260b520584a90ae1242988b686b48b14d54f8887b95e17aa8,0,2024-06-18 18:00:29.8804,2024-06-18 18:00:29.8804,5
\x394aa1ed2546ac581d65ccdef8b073cbd610098caed9c2c62ee6ec2ee3fdcc70,0,2024-06-18 18:00:29.8804,2024-06-18 18:00:29.8804,6
\x3d0291ae8877887c7e267cc262c9c7975c27909ebb439533b0633c6e368c6031,0,2024-06-18 18:00:29.8804,2024-06-18 18:00:29.8804,7
\xedcf8c1118026d665c4b7fd8db9e74eb93860fa675b6173f359a56ec2f353041,0,2024-06-18 18:00:29.8804,2024-06-18 18:00:29.8804,8

This file was used instead of the original, but since it does not match the expected format, it caused the program to panic.

Questions:

• Can this tool be used for a custom blockchain? Seems that just by changing the constants it should work. If yes:
  • How to determine the GENESIS_BLOCK and the BOOJUM_BLOCK?
  • Is ZK_SYNC_ADDR the DiamondProxyAddr?
  • How was the InitialState.csv file originally created? It seems to involve a complex query.