-
Notifications
You must be signed in to change notification settings - Fork 0
Keysets Master Feature #11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
PASS [ 42.994s] (3/3) lit_node::test toxiproxy::perf_tests::load_with_no_latency |
* Better handling of shadow splicing. * Update rust/lit-node/lit-node/src/tasks/fsm/epoch_change.rs Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: Copilot <[email protected]>
| // Most of this is related to staking / delegation | ||
| s.globalConfig[0] = LibStakingStorage.GlobalConfig({ | ||
| tokenRewardPerTokenPerEpoch: (10 ** 18) / 20, // 18 decimal places in token | ||
| keyTypes: keyTypesTemp, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we're removing keytypes because now which keytypes a network supports is based on which keytypes the keyset supports right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, according to Mike, this variable wasn't used even before key sets 🤷 ! ! !
More just cleanup.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR implements a comprehensive keyset-based architecture that replaces the previous single root key approach. The changes enable the system to support multiple keysets with configurable thresholds, curve types, and realm associations, introducing a more flexible and scalable key management system.
Key changes:
- Refactored from single root key storage to keyset-based configuration with
KeySetConfigmanaging multiple root keys per curve type - Updated contract ABIs and routing data structures to include keyset identifiers throughout the signing and key derivation flows
- Upgraded critical dependencies: scc (2.x→3.3.2), sdd (3.x→4.2.4), ethabi (16.0.0→18.0.0), and blsful (3.0.0-pre8→3.0.0)
Reviewed changes
Copilot reviewed 121 out of 159 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| rust/lit-os/Cargo.lock | Dependency consolidation removing older versions of bitvec, ethabi, ethereum-types, and related crates |
| rust/lit-node/lit-sdk/src/handshake.rs | Renamed handshake request/response types from Json* to SDK* prefix |
| rust/lit-node/lit-sdk/src/cait_sith.rs | Added missing k256 import from lit_frost |
| rust/lit-node/lit-node/tests/integration/signing.rs | Refactored signing tests to use helper function and added key_set_id to PKP info tuple |
| rust/lit-node/lit-node/tests/integration/session_sigs.rs | Updated PKP info destructuring to include key_set_id parameter |
| rust/lit-node/lit-node/tests/integration/lit_actions.rs | Updated PKP info destructuring to include key_set_id parameter |
| rust/lit-node/lit-node/tests/integration/integration_tests.rs | Refactored epoch configuration and added logging for shadow realm processing |
| rust/lit-node/lit-node/tests/integration/backup_datil_long.rs | Added IPFS API key configuration and hardcoded root keys for testing |
| rust/lit-node/lit-node/tests/component/* | Updated signature method calls to pass key_set_id and refactored root_pubkeys from Option to slice |
| rust/lit-node/lit-node/src/utils/web.rs | Refactored get_bls_root_pubkey to accept key_set_id and use CurveState |
| rust/lit-node/lit-node/src/tss/frost/mod.rs | Changed root_pubkeys parameter from Option<Vec> to &[String] and added key_set_id |
| rust/lit-node/lit-node/src/tss/ecdsa_damfast/mod.rs | Changed root_pubkeys parameter from Option<Vec> to &[String] and added key_set_id |
| rust/lit-node/lit-node/src/tss/dkg/* | Major refactoring to support keyset-based DKG with HashMap return types and ShadowOptions |
| rust/lit-node/lit-node/src/tss/common/tss_state.rs | Removed get_dkg_state method and refactored to use CurveState directly |
| rust/lit-node/lit-node/src/tss/common/traits/* | Removed BasicDkg trait, updated Signable and Cipherable trait signatures |
| rust/lit-node/lit-node/src/tss/common/curve_state.rs | Converted from trait implementation to standalone struct with root_keys method |
| rust/lit-node/lit-node/src/tss/blsful/mod.rs | Updated to use CurveState and added key_set_id parameter |
| rust/lit-node/lit-node/src/tasks/fsm/* | Major refactoring for keyset support with HashMap-based root keys and ShadowOptions |
| rust/lit-node/lit-node/src/pkp/utils.rs | Updated to use PubKeyRoutingData and added key_set_id to vote_for_root_key |
| rust/lit-node/lit-node/src/models/mod.rs | Added KeySetConfig and PubKeyRoutingData models |
| rust/lit-node/lit-node/src/functions/action_client.rs | Updated to use CurveState and added key_set_id parameters throughout |
| rust/lit-node/lit-node/src/endpoints/* | Updated handshake endpoints and added v1 handshake with key_sets response |
| rust/lit-node/lit-node/src/config/chain.rs | Major refactoring from root_keys to key_sets with BTreeMap-based storage |
| rust/lit-node/lit-node/src/common/key_helper.rs | Updated to use iter_sync instead of scan for scc 3.x compatibility |
| rust/lit-node/lit-node/Cargo.toml | Updated scc dependency to 3.3.2 and added log dependency |
| rust/lit-node/lit-node-testnet/* | Updated test infrastructure for keyset support |
| rust/lit-node/lit-node-core/src/models/* | Renamed SDK types and added documentation comments |
| rust/lit-core/lit-blockchain/* | Updated contract ABIs and middleware type aliases |
| rust/lit-core/* | Various code quality improvements and cleanup |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
* handle old peers Signed-off-by: Michael Lodder <[email protected]> * Open v2.1.7 --------- Signed-off-by: Michael Lodder <[email protected]> Co-authored-by: Adam Reif <[email protected]> Co-authored-by: Brendon <[email protected]>
* Update test * Update tests * Update rust/lit-node/lit-node/src/tasks/fsm/epoch_change.rs Co-authored-by: Copilot <[email protected]> * resync /w keysets feature. --------- Co-authored-by: Copilot <[email protected]>
* Update test * Update tests * Setup * restore datil keys into a second keyset * remove large binaries * remove large zip * cargo fmt * Updates to DKG / Epoch change processes + correct saving of data to chain. * fix actions in testnet * cargo fmt * patch epoch change retries. * cargo fmt * Fix failed DKGs when doing restores.
* Update test * Update tests * Setup * restore datil keys into a second keyset * remove large binaries * remove large zip * cargo fmt * Updates to DKG / Epoch change processes + correct saving of data to chain. * Add BLS decryption assertions * Use correct keyset for assertion * Update lit-node-testnet to work with ImportedDatilTestnet + start on PKP assertions across chain This is incomplete. * Add notes * Add notes * fix actions in testnet * cargo fmt * Add BLS decryption assertions * Use correct keyset for assertion * Update lit-node-testnet to work with ImportedDatilTestnet + start on PKP assertions across chain This is incomplete. * Add notes * Add notes * Update test to interact with ImportedDatilTestnet * remove unneded code * refactor & remove duplications * Refactor & add datil node secrets * Update rust/lit-node/lit-node/src/tasks/fsm/epoch_change.rs Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/lit-node/src/tss/common/restore/eks_and_ds.rs Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/Cargo.toml Co-authored-by: Copilot <[email protected]> * cargo lock cleanup * cargo.lock update + config.rs in the resolver formatting. * patch epoch change retries. * cargo fmt * Fix failed DKGs when doing restores. * cargo fmt * fix tests * Logging info for CI datil instances. * Open ports for a second instance of anvil in the yml files. * Load hex data from test state into anvil. * remove unused file * comment out #if false * take 2? * fix test reference * Use HEX file to load state, rather than a JSON file. * Refactor to insure that for Datil tests we don't rely on naga cache'd chain state. * switch lit-recovery binaries for testing * Remove keyset_identifier as an enum and clean up tests. * fix datil backup selection of binary. * cargo fmt * yml test. * if false yml * comment out version upgrade * yml update * try a var for the upgrade workflow ? * update lit-os cargo.lock * update gitinfo --------- Co-authored-by: Howard Tam <[email protected]> Co-authored-by: Copilot <[email protected]>
| uint256[] counts; | ||
| /// Set when the recovery DKG completes for the key set | ||
| address[] recoveryPartyMembers; | ||
| bytes recoverySessionId; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think we have to put back address[] recoveryPartyMembers with a comment that it's unused but kept for storage compatibility reasons. and then you can put this bytes recoverySessionId; below it and the diamond storage will be happy and preserved
| struct GlobalConfig { | ||
| uint256 tokenRewardPerTokenPerEpoch; | ||
| // the key type of the node. // 1 = BLS, 2 = ECDSA. Not doing this in an enum so we can add more keytypes in the future without redeploying. | ||
| uint256[] keyTypes; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can't remove this from storage, it will make diamond storage sad. so we have to keep uint256[] keyTypes; where it is, but you can add a comment saying it's unused and kept for storage compatibility reasons.
| event StakingTokenSet(address newStakingTokenAddress); | ||
| event KickPenaltyPercentSet(uint256 reason, uint256 newKickPenaltyPercent); | ||
| event ResolverContractAddressSet(address newResolverContractAddress); | ||
| event ConfigSet( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
did we just remove this event? it seems like it wasn't ever used properly anyway? it would be good to have an event when config is set, tho. maybe it should be called RealmConfigSet or something
| } | ||
|
|
||
| const SUPPORTED_KEY_TYPES = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']; | ||
| async function askForKeyTypes(): Promise<number[]> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for removing this - i don't think there's a scenario where we would want to only run with a subset of key types, so no point in asking the user what key types they want. i like your solution - just use all of them.
…to support pkps in both chains ) (#42) * Update test * Update tests * Setup * restore datil keys into a second keyset * remove large binaries * remove large zip * cargo fmt * Updates to DKG / Epoch change processes + correct saving of data to chain. * Add BLS decryption assertions * Use correct keyset for assertion * Update lit-node-testnet to work with ImportedDatilTestnet + start on PKP assertions across chain This is incomplete. * Add notes * Add notes * fix actions in testnet * cargo fmt * Add BLS decryption assertions * Use correct keyset for assertion * Update lit-node-testnet to work with ImportedDatilTestnet + start on PKP assertions across chain This is incomplete. * Add notes * Add notes * Update test to interact with ImportedDatilTestnet * remove unneded code * refactor & remove duplications * Refactor & add datil node secrets * Update rust/lit-node/lit-node/src/tasks/fsm/epoch_change.rs Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/lit-node/src/tss/common/restore/eks_and_ds.rs Co-authored-by: Copilot <[email protected]> * Update rust/lit-node/Cargo.toml Co-authored-by: Copilot <[email protected]> * whole lotta shaking going on. ( endpoints changed throughout ) * PKP sign works. * cargo clippy * cargo lock cleanup * git merge fixes * cargo toml update ( to trigger cargo.lock update ) * cargo.lock update + config.rs in the resolver formatting. * cargo fmt * fix panic when node was kicked (realmid = 0) * patch epoch change retries. * cargo fmt * Expose underlying error when peer_connect attestation fails * Fix failed DKGs when doing restores. * cargo fmt * fix tests ! * fix tests * Logging info for CI datil instances. * Open ports for a second instance of anvil in the yml files. * Load hex data from test state into anvil. * Feature/price setting script (#43) * first pass at price setting script * price setting script seems like it will work * script works on dev and prod * Update blockchain/contracts/scripts/generatePriceSettingTransactions.ts Co-authored-by: Copilot <[email protected]> * remove unused map * regen contracts * 100x decrease for testnets when setting prices * divide by 10000 when setting naga-test prices * actually, made naga-test 100000x cheaper than naga prod --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Brendon <[email protected]> * Fix: Add `correlation_id` to `lit-actions` spans (#25) * Add spans for actions * Add correlation_id to TracingMiddleware * Checkout server.rs * Extract correlation_id' --------- Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Brendon <[email protected]> * Refactors OsQuery metrics emission to use OpenTelemetry Gauges (#28) * Add spans for actions * Improve metrics * first pass at adding request id to logs * Update rust/lit-node/lit-node/src/main.rs Co-authored-by: Copilot <[email protected]> * change how we set the request id on the span * clippy * Add correlation_id to TracingMiddleware * Checkout server.rs * Extract correlation_id' * Fix clippy wanrning * separate info from gauge metrics (#44) * Revert "Merge branch 'inf-852-fix-tracing-lit-actions' into inf-753-olly-metrics" This reverts commit b0a6d38, reversing changes made to 4a58992. * Revert "Merge remote-tracking branch 'origin/feature/NODE-4860-correlation-id-in-logs' into inf-753-olly-metrics" This reverts commit 7be7bbe, reversing changes made to b0a6d38. * cargo fmt --------- Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Adam Reif <[email protected]> * remove unused file * comment out #if false * take 2? * fix test reference * Use HEX file to load state, rather than a JSON file. * Refactor to insure that for Datil tests we don't rely on naga cache'd chain state. * switch lit-recovery binaries for testing * Remove keyset_identifier as an enum and clean up tests. * Fix up tests * cargo fmt * fix datil backup selection of binary. * logging, shorten handshake challange value. * fix "default" keyset bug :( * cargo fmt * yml test. * if false yml * comment out version upgrade * yml update * try a var for the upgrade workflow ? * upgrade yml with default if statement. * cargo clippy. * Update lit-os cargo.lock * update lit-os cargo.lock * update gitinfo * fix restore test to use Datil directly. * cargo fmt * patch so both long running restores work :( * Update epoch change long to deal with keysets * small shiva refactor * difference between pkp & root key lookups when getting a keyset. * remove datil from lit action testing. * clean up integration test setup to account for keyset settings. * limit caches to only active states. * Fix Shadow splicing test, make some integration force builds * update abis & lit-os lock file * cargo fmt + minor cleanup * Refactor one of the last timing related tests. * ABI staleness, remove more testing feature flags * Make the StakerAccountSetupMapper Send+Sync to allow Shiva to build. * Clean up complaints. * Update to determine key set for recovery (#58) * update to determine key set for recovery Signed-off-by: Michael Lodder <[email protected]> * add test when no key set is on chain Signed-off-by: Michael Lodder <[email protected]> * update test Signed-off-by: Michael Lodder <[email protected]> --------- Signed-off-by: Michael Lodder <[email protected]> Co-authored-by: Brendon <[email protected]> * Better logging. --------- Signed-off-by: Michael Lodder <[email protected]> Co-authored-by: Howard Tam <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Adam Reif <[email protected]> Co-authored-by: Adam Reif <[email protected]> Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Abhi... <[email protected]> Co-authored-by: Chris Cassano <[email protected]> Co-authored-by: Michael Lodder <[email protected]>
* fix invalid signature combination Signed-off-by: Michael Lodder <[email protected]> * fix tests Signed-off-by: Michael Lodder <[email protected]> --------- Signed-off-by: Michael Lodder <[email protected]>
NOTE
This is a long running feature branch that receives regular updates from
master. This is the primary feature that allows for keyset functionality ( multiple concurrent sets of root keys, that are unrelated to each other ) within the nodes.Pull Request Overview
This PR implements threshold enforcement for keysets by introducing a keyset-based architecture that replaces the previous single root key approach. The changes enable the system to support multiple keysets with configurable thresholds, curve types, and realm associations.
Key changes:
See linear project : https://linear.app/litprotocol/project/keysets-56c26d54587f/overview
MISC
INCOMPATIBILITIES
Check LibStakingStorage - this does not match develop at the moment.
Must re-enable upgrade test against develop or some ansible target.