feat(lee,lez)!: add private execution of immutable programs and shadow programs by jonesmarvin8 · Pull Request #824 · logos-blockchain/logos-execution-zone

jonesmarvin8 · 2026-09-02T20:15:20Z

🎯 Purpose

Builds off of program deployment (in terms of linked list). Currently, from PR 815/812, privacy execution must disclose each program that is executed. In this PR, we introduce function privacy for immutable programs and shadow programs.

Once a program becomes immutable (ProgramHeader.immutable = true), a commitment is emitted to the private state. This commitment can be used to (1) prove program's existence for program_owner in private state, (2) image_id stored in committed Data is used with provided program elf.

A shadow program is a program that has never been emitted to the public state. Privacy execution allows for execution of such programs by providing the elf. This is a feature that was possible in previous versions of LEZ.

BREAKING CHANGE!: Modifies privacy circuit; thus, proofs generated before this PR will no longer verify.

⚙️ Approach

create_header/update_header emit a private mirror commitment the moment a header's immutable becomes true, seeded by the header's own account_id (not just its content).
ProgramImageClaim splits into Public (today's public lookup) / Private (anchored against the emitted commitment via membership proof). Sequencer verifies both at deploy-time and privacy-tx time.
AccountId::for_shadow_program + ShadowProgramWitness: a third way to reference a program never deployed anywhere, identity established fresh in-circuit from an ELF witness each dispatch.
ProgramWithDependencies::as_shadow_program/with_shadow_dependency/as_private_program/with_private_dependency so callers can opt a program into either path.

🧪 How to Test

All previous tests should pass; new tests added:

lez::programs::program_loader::core / lee::state_machine::src::state::tests::deploy:

create_header_immutable_from_birth_lands_immutable_mirror_commitment
create_header_mutable_leaves_commitment_tree_unchanged
update_header_flip_to_immutable_lands_immutable_mirror_commitment

lee::state_machine::core::src::program::tests:

for_shadow_program_matches_pinned_value

lee::state_machine::src::state::tests::circuit:

private_claim_matching_a_real_commitment_passes_verification
private_claim_with_no_matching_commitment_is_rejected
shadow_program_claims_a_private_pda_it_legitimately_owns
shadow_program_claims_a_public_pda_it_legitimately_owns

🔗 Dependencies

N/A

🔜 Future Work

Indexer wire-protocol coverage for Private claims (currently lossy/display-only in that direction).
Negative test for a ShadowProgramWitness whose ELF doesn't hash to its declared account_id.

📋 PR Completion Checklist

Complete PR description
Implement the core functionality
Add/update tests
Add/update documentation and inline comments

… programs Reimplements the marvin/privacy-and-shadow-executions feature (PR #824) as a single commit directly on current dev, rather than rebasing the branch's full 84-commit history: the vast majority of that history was foundational program-as-account/program_loader plumbing that has since landed on dev independently through other means, leaving only this feature's own ~850-line delta as genuinely new work. Once a program's header becomes permanently immutable (immutable: true, set at CreateHeader or flipped by UpdateHeader), program_loader_core emits a commitment mirroring its finalized ProgramHeader directly into private state - no proof, nullifier, or ciphertext needed, since every validating node can independently recompute it from the same public transaction. The mirror is seeded by the header's own account_id, so the commitment anchors a specific deployment rather than any header with coincidentally identical data. ProgramImageClaim splits into Public (today's public lookup) and Private (anchored against the emitted commitment via a membership proof instead). The sequencer independently reconstructs either claim before accepting a proof, on both the deploy-time write path and the later privacy-transaction verify path. Shadow programs add a third way to reference a program in a privacy- preserving transaction: one never deployed anywhere, public or private. AccountId::for_shadow_program derives its dispatch address purely from its image_id, domain-separated from every other PDA formula. Its identity is established fresh, in-circuit, on every dispatch from an ELF supplied as a witness. ProgramWithDependencies gains as_shadow_program/with_shadow_dependency and as_private_program/with_private_dependency so callers can opt a program into either path. Every API this touches was independently reshaped on dev since the original branch forked (AccountPostState/Claim -> AccountStateDiff/BalanceDiff, the ProgramWithDependencies.self_account_id rename, program_loader_core's Data/ProgramHeader/ProgramSegment codec), so this is a from-scratch port against dev's current shapes rather than a mechanical cherry-pick. Rebuilds every guest artifact and the prebuilt sequencer test fixture, since lee_core (compiled into every guest) changed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

… programs Reimplements the marvin/privacy-and-shadow-executions feature (PR #824) as a single commit directly on current dev, rather than rebasing the branch's full 84-commit history: the vast majority of that history was foundational program-as-account/program_loader plumbing that has since landed on dev independently through other means, leaving only this feature's own delta as genuinely new work. Once a program's header becomes permanently immutable (immutable: true, set at CreateHeader or flipped by UpdateHeader), program_loader_core emits a commitment mirroring its finalized ProgramHeader directly into private state - no proof, nullifier, or ciphertext needed, since every validating node can independently recompute it from the same public transaction. The mirror is seeded by the header's own account_id, so the commitment anchors a specific deployment rather than any header with coincidentally identical data. ProgramImageClaim splits into Public (today's public lookup) and Private (anchored against the emitted commitment via a membership proof instead). The sequencer independently reconstructs either claim before accepting a proof, on both the deploy-time write path and the later privacy-transaction verify path. Shadow programs add a third way to reference a program in a privacy- preserving transaction: one never deployed anywhere, public or private. AccountId::for_shadow_program derives its dispatch address purely from its image_id, domain-separated from every other PDA formula. Its identity is established fresh, in-circuit, on every dispatch from an ELF supplied as a witness. ProgramWithDependencies gains as_shadow_program/with_shadow_dependency and as_private_program/with_private_dependency so callers can opt a program into either path. Every API this touches was independently reshaped on dev since the original branch forked (AccountPostState/Claim -> AccountStateDiff/BalanceDiff, the ProgramWithDependencies.self_account_id rename, program_loader_core's Data/ProgramHeader/ProgramSegment codec), so this is a from-scratch port against dev's current shapes rather than a mechanical cherry-pick. Rebuilds every guest artifact and the prebuilt sequencer test fixture, since lee_core (compiled into every guest) changed. Trims the test suite to what's actually load-bearing: drops the unused, dead force_insert_commitment test helper and 3 of 4 for_shadow_program tests that were fully implied by the remaining pinned-value test, adds a regular-private-account variant of the shadow-program-ownership test so a future regression can be isolated to PDA binding vs. shadow-identity resolution, and trims added doc/inline comments for concision.

…d in-circuit Addresses PR #824 review feedback: - ProgramImageClaim::Private previously echoed the full account_id and ProgramHeader into the public journal, and had its "membership" checked by the sequencer doing a local existence lookup rather than a real Merkle proof — so it leaked more than Public claims while proving nothing. Splits the circuit input into ProgramImageWitness (carries account_id/program_header/ membership_proof, never published) and a trimmed ProgramImageClaim (Private now only publishes a commitment-tree root), with membership verified in-circuit the same way private account updates already are. The sequencer now only checks that root is one the tree has actually had (V03State::is_known_commitment_root, shared with check_nullifiers_are_valid). - Shadow program witnesses were resolved (ELF decoded and hashed) lazily, once per dispatch, so a shadow program invoked N times in one call graph paid the cost N times. Resolution now happens once per witness up front, merged into image_id_by_account_id alongside the claims. - Adds a negative test asserting a shadow witness whose ELF doesn't hash to its declared account_id is rejected even if the call graph never dispatches it, closing the gap the PR's own Future Work list flagged. Relocates the immutable-mirror commitment derivation from program_loader_core into lee_core (program_loader_core re-exports it), since the circuit needs it too and can't depend on a specific program's core crate. Rebuilds affected RISC0 artifacts and the test fixture.

moudyellaz

lgtm, thanks for addressing my comments!

agureev

Generally very cool! Very smart with the domain separator for shadow programs, makes sense, even if I am unsure how viable it is in terms of computation.

Also left a bunch of abstarction/doc comments/suggestions. Doc comments are merely suggestions, feel free to skip, but some parts I feel would be good like getting the benchmarks in and removing some redundant/unstructured data.

Thank you beforehand!

…ependency Addresses agureev's PR #824 review comment: ProgramWithDependencies tracked each account's Public/Shadow/Private status across three independent collections (dependencies, shadow_account_ids, private_program_headers), so an account could end up in more than one of them at once with no coherent meaning, never enforced anywhere. Introduces ProgramKind (Public/Shadow/Private) and Dependency { program, kind }, collapsing the three collections into one dependencies: HashMap<AccountId, Dependency> plus a self_kind field for the top-level program. One value per account instead of three independent memberships, so the "both shadow and private" state is no longer representable at all. The builder API (new/as_shadow_program/with_shadow_dependency/ as_private_program/with_private_dependency/From<Program>) keeps its exact existing signatures, so every existing caller (wallet, cycle_bench, examples/program_deployment, lee's own tests) compiles unchanged. The witness-building logic in execute_and_prove_with_padded_inputs collapses from a two-pass filter/chain into a single match over each account's one kind. Also extends wallet-ffi's C surface, which never gained shadow/private support when PR #824 introduced it upstream (it was only ever touched enough to keep compiling). Adds FfiProgramKind/FfiProgramHeader/ FfiMembershipProof/FfiDependency following this crate's existing tag-plus-flat-struct convention, and threads them through FfiProgramWithDependencies. account_id is always derived from (program, kind), never caller-supplied, so it can't disagree with what the circuit itself computes.

* refactor(lee,lez): unify ProgramWithDependencies around ProgramKind/Dependency Addresses agureev's PR #824 review comment: ProgramWithDependencies tracked each account's Public/Shadow/Private status across three independent collections (dependencies, shadow_account_ids, private_program_headers), so an account could end up in more than one of them at once with no coherent meaning, never enforced anywhere. Introduces ProgramKind (Public/Shadow/Private) and Dependency { program, kind }, collapsing the three collections into one dependencies: HashMap<AccountId, Dependency> plus a self_kind field for the top-level program. One value per account instead of three independent memberships, so the "both shadow and private" state is no longer representable at all. The builder API (new/as_shadow_program/with_shadow_dependency/ as_private_program/with_private_dependency/From<Program>) keeps its exact existing signatures, so every existing caller (wallet, cycle_bench, examples/program_deployment, lee's own tests) compiles unchanged. The witness-building logic in execute_and_prove_with_padded_inputs collapses from a two-pass filter/chain into a single match over each account's one kind. Also extends wallet-ffi's C surface, which never gained shadow/private support when PR #824 introduced it upstream (it was only ever touched enough to keep compiling). Adds FfiProgramKind/FfiProgramHeader/ FfiMembershipProof/FfiDependency following this crate's existing tag-plus-flat-struct convention, and threads them through FfiProgramWithDependencies. account_id is always derived from (program, kind), never caller-supplied, so it can't disagree with what the circuit itself computes. * fix(wallet-ffi): avoid FfiProgramKind/FfiAccountIdentityKind enum collision FfiProgramKind's Public variant collided with FfiAccountIdentityKind's existing Public at the C level: unscoped C/C++ enums put their enumerators in the enclosing scope, so two enums both defining PUBLIC in the same header is a hard redefinition error, not just a lint. Confirmed by actually compiling the generated header standalone (it now compiles cleanly). Renames the variants to ProgramPublic/ProgramShadow/ProgramPrivate (PROGRAM_PUBLIC/PROGRAM_SHADOW/PROGRAM_PRIVATE in the generated header). --------- Co-authored-by: jonesmarvin8 <marvin.jones.phd@gmail.com>

agureev

lgtm, thank you for addressing the comments!

schouhy

hey lgtm! left some comments

…programs Renamed ProgramImageWitness/ProgramImageClaim/ProgramKind's Public/Private variants to Disclosed/Undisclosed: both are publicly deployed programs, the distinction is only whether the claim discloses which one, and the old names read as if Private meant not publicly deployed. Asserted program_header.immutable in to_claim()'s Undisclosed arm, where prover-supplied witness data first crosses the trust boundary -- fail-fast defense in depth alongside the real invariant every other caller of immutable_mirror_commitment already enforces. Added account_id/self_account_id fields to FfiDependency and FfiProgramWithDependencies: ffi_kind_account_id derived every account id from kind + bytecode, silently assuming a Disclosed/Undisclosed program's header always lives at its own bijection address. It doesn't -- CreateHeader lets a deployer place a header at any address, and the native ProgramWithDependencies API already takes that address as an explicit parameter, but the FFI had no way to express it. Shadow keeps deriving from the bytecode, since that's definitional and there's no header to consult. Required is_authorized unconditionally for CreateHeader targets in finalize_header, matching UpdateHeader's existing rule. CreateHeader previously let a deployer place a header at any empty account with no authorization check at all; since AccountId::for_shadow_program is a plain forward hash, an attacker could compute a target program's shadow address in advance, deploy an unrelated "honest" header there to collect real public trust and funds, then dispatch a weaker program privately under the same address later to drain them -- no hash needs breaking, since the deployer chooses the target directly. The production wallet facade already always signs for the header's own key, so this closes the gap without touching any real deployment path; genesis bypasses CreateHeader entirely via insert_program, so it's unaffected too. Rebuilt the privacy_preserving_circuit RISC0 artifact, last regenerated at #903, one commit before #905 changed lee_core's circuit-input types -- the checked-in guest binary no longer matched the host's current encoding of PrivacyPreservingCircuitInput, so any shadow-program proof failed in-guest with a Borsh length mismatch.

…w programs Adds two new ways a program can be invoked in a privacy-preserving transaction alongside the existing publicly-disclosed path: - Undisclosed: an immutable, publicly-deployed program whose identity isn't revealed on the resulting claim -- membership against the header's mirrored commitment is proven in-circuit instead of disclosing which image_id it is. - Shadow: a program never deployed to public state at all, resolved fresh each proof from its bytecode via a witness, dispatched at AccountId::for_shadow_program(image_id). Renamed the Public/Private naming throughout (ProgramImageWitness, ProgramImageClaim, ProgramKind, and their wallet-ffi mirrors) to Disclosed/Undisclosed: both are publicly deployed programs, the distinction is only whether the claim discloses which one. Security fixes from review: - program_loader's CreateHeader now requires is_authorized for its target unconditionally, closing an attack where a deployer could precompute a target program's shadow address, park an unrelated "honest" header there to collect real public trust and funds, then dispatch the real program privately under the same address later to drain them. - ProgramImageWitness::to_claim returns a Result instead of panicking when an Undisclosed claim's header isn't immutable, keeping the panic-or-not decision at the guest call site rather than baked into a shared type. - wallet-ffi's FfiDependency/FfiProgramWithDependencies gained explicit account_id/self_account_id fields: a program's header can live at any address a deployer was authorized for, never assumed to be its bytecode's bijection address, and the FFI had no way to express that before. Rebuilt the privacy_preserving_circuit RISC0 artifact to match.

…uit artifact Rebasing onto dev's shard-based account model required source changes to the guest-visible circuit_io/program_loader/state_machine code, which invalidates the previously committed guest artifact. Rebuilds privacy_preserving_circuit.bin against the rebased source and fixes clippy/fmt violations surfaced by dev's restriction lints (arbitrary_source_item_ordering, shadow_unrelated) plus a stale double-kernel-wrap bug in force_insert_segment_chain that broke segment-chain test fixtures with "Invalid Magic Bytes" errors.

CI's artifacts check caught these: the shard-based account-model rebase changed lee_core types shared by every lez program's guest code, so all of their compiled RISC0 binaries were stale even though only privacy_preserving_circuit's artifact had been rebuilt. Rebuilds via `just build-artifacts` and refreshes the fixture dump to match.

…ss bijection addresses private_pda's and scoped_reads' program deployments targeted AccountId::from_builtin_program(image_id) via CreateHeader without holding its key, relying on the permissionless header-creation model the shadow-account-collision fix (comment #2) closed. Production deployment (lez/wallet/src/program_facades/program_loader.rs) always signs for the header's own key already; genesis bypasses CreateHeader entirely via insert_program. Fixes both tests by generating a real key for the deployment target instead, matching how their own segment accounts are already deployed.

jonesmarvin8 marked this pull request as draft September 2, 2026 20:41

jonesmarvin8 force-pushed the marvin/remove-program-deployment branch 3 times, most recently from 08385cb to aa10df2 Compare September 3, 2026 21:26

jonesmarvin8 changed the base branch from marvin/remove-program-deployment to dev September 8, 2026 20:35

jonesmarvin8 force-pushed the marvin/privacy-and-shadow-executions branch from dfb3c5f to 5cc58e1 Compare September 8, 2026 21:39

jonesmarvin8 force-pushed the marvin/privacy-and-shadow-executions branch from fdeea79 to 48bc7b9 Compare September 9, 2026 18:00

jonesmarvin8 requested review from Arjentix, Pravdyvy, agureev, erhant, moudyellaz and schouhy September 9, 2026 19:55

jonesmarvin8 marked this pull request as ready for review September 9, 2026 19:55

jonesmarvin8 changed the title ~~feat(lee,lez): add private execution of immutable programs and shadow programs~~ feat(lee,lez)!: add private execution of immutable programs and shadow programs Sep 9, 2026

moudyellaz reviewed Sep 11, 2026

View reviewed changes

Comment thread lee/state_machine/core/src/circuit_io.rs Outdated

Comment thread lee/privacy_preserving_circuit/src/execution_state.rs Outdated

Comment thread lee/state_machine/src/state/tests/circuit.rs

jonesmarvin8 requested a review from moudyellaz September 11, 2026 23:30

jonesmarvin8 mentioned this pull request Sep 14, 2026

feat!: ciphertext padding #630

Merged

moudyellaz approved these changes Sep 17, 2026

View reviewed changes

jonesmarvin8 removed request for Arjentix, Pravdyvy and erhant September 17, 2026 21:40

moudyellaz added the priority:medium Normal priority label Sep 17, 2026

agureev requested changes Sep 18, 2026

View reviewed changes

This was referenced Sep 19, 2026

refactor(lee)!: derive immutable-mirror AccountId without npk/vpk #903

Merged

fix(lee): immutable genesis program emit commitment #904

Merged

refactor(lee,lez): ProgramKind abstraction for ppe #905

Merged

jonesmarvin8 requested a review from agureev September 19, 2026 18:01

agureev approved these changes Sep 22, 2026

View reviewed changes

schouhy approved these changes Sep 22, 2026

View reviewed changes

Comment thread lez/wallet-ffi/src/generic_transaction.rs Outdated

Comment thread lee/state_machine/core/src/program/mod.rs

Comment thread lee/state_machine/core/src/circuit_io.rs

Comment thread lee/state_machine/core/src/circuit_io.rs

jonesmarvin8 added 2 commits September 23, 2026 09:17

jonesmarvin8 force-pushed the marvin/privacy-and-shadow-executions branch from 70beae5 to 4f9c996 Compare September 23, 2026 14:20

jonesmarvin8 added 2 commits September 23, 2026 10:47

jonesmarvin8 merged commit cc4951b into dev Sep 23, 2026
51 checks passed

Navigation Menu

Repository navigation

feat(lee,lez)!: add private execution of immutable programs and shadow programs - #824

feat(lee,lez)!: add private execution of immutable programs and shadow programs#824
jonesmarvin8 merged 4 commits into
devfrom
marvin/privacy-and-shadow-executions

jonesmarvin8 commented Sep 2, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

moudyellaz left a comment •

edited

Loading

Uh oh!

agureev left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

agureev left a comment

Uh oh!

schouhy left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jonesmarvin8 commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🎯 Purpose

⚙️ Approach

🧪 How to Test

🔗 Dependencies

🔜 Future Work

📋 PR Completion Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

moudyellaz left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agureev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

agureev left a comment

Choose a reason for hiding this comment

Uh oh!

schouhy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jonesmarvin8 commented Sep 2, 2026 •

edited

Loading

moudyellaz left a comment •

edited

Loading