Skip to content

tutorial.md

Latest commit

 

History

History
547 lines (353 loc) · 21.5 KB

tutorial.md

File metadata and controls

547 lines (353 loc) · 21.5 KB

Bringing External Data On-Chain: Oracle Patterns for Midnight

TL;DR

Midnight does not ship a built-in oracle mechanism. That is not an omission so much as a design constraint: Compact circuits prove computation over explicit inputs and ledger state, while off-chain data remains off-chain unless your application deliberately introduces it. In practice, developers on Midnight reach for three oracle patterns:

  1. Witness-provided data with on-chain verification
    A witness supplies external data to the circuit, and the contract verifies enough evidence on-chain to decide whether to accept it. This is the most flexible pattern, but it only becomes trustworthy if the circuit validates signatures, rounds, digests, or other authenticity constraints.

  2. Admin-updated ledger fields with access control
    A designated updater account writes the current value into contract state. This is operationally simple and often the easiest thing to ship, but it concentrates trust in the admin key and updater process.

  3. Cross-contract calls for composed state
    One contract consumes state maintained by another contract. This is the cleanest pattern when the “oracle” is really another on-chain component in your application architecture, but it inherits the source contract’s trust model and upgrade policy.

This tutorial walks through all three patterns, shows minimal Compact examples for each, and explains the tradeoffs. Where the current public primer does not include a specific library API or import form, I call that out explicitly and point you to the official docs as the source of truth.

Context: what “oracle” means on Midnight, and why there is no native oracle support

The Compact language reference defines three building blocks that matter here:

  • ledger declarations hold public on-chain state,
  • circuits are the contract entry points that may read or write ledger state, and
  • witnesses are off-chain callbacks provided by the DApp.

That last point is the most important one for oracle design. The docs warn:

“Do not assume in your contract that the code of any witness function is the code that you wrote in your own implementation. Any DApp may provide any implementation that it wants for your witness functions.”

So witness output is untrusted input unless your circuit constrains it.

That is the reason Midnight does not have “native oracle support” in the way some developers expect from other chains. A Compact contract does not directly fetch HTTP data, read a market API, or trust a protocol-owned off-chain feed by default. The proof system verifies that the circuit was executed correctly over the supplied inputs; it does not prove that an external website was honest or that a witness callback consulted the source you intended.

From the application developer’s perspective, this means:

  • external data must arrive through an explicit path,
  • that path must have a trust model,
  • the contract must encode whatever verification rules it relies on, and
  • the user experience depends on how much work the prover or operator must do off-chain.

In other words, Midnight gives you the primitives, not the oracle.

That is why the contributor bounty itself points to the three patterns in this article. They are not arbitrary examples. They are the practical design space when you need off-chain information in a Compact contract.

Before we get into code, two implementation notes:

  1. I use only Compact syntax and concepts that are directly supported by the supplied primer whenever possible.
    That includes ledger, sealed ledger, circuit, witness, tuples, structs, and disclose(...).

  2. Two details depend on the current docs and library version in your project:

    • the exact assertion helper or failure primitive, and
    • the exact syntax for cross-contract imports/calls and signature-verification helpers.
      I mark these places clearly. Use the official Midnight docs and the Compact language reference to bind them to the current release before compiling.

Witness-provided data with on-chain verification

This is the most “oracle-like” pattern in Midnight.

A witness supplies a piece of external data—say, a price, an exchange rate, a weather reading, or a compliance attestation—and the circuit checks enough evidence to decide whether the value is acceptable.

The core idea is simple:

  1. the witness returns the data payload plus proof material,
  2. the circuit verifies the proof material, and
  3. the contract either uses the value or rejects it.

When to use this pattern

Use witness-provided data when:

  • the data changes frequently,
  • you do not want a central admin constantly writing to chain,
  • different transactions may need different data points, or
  • you want the proving client to carry the burden of fetching the data.

Typical examples include:

  • signed price feeds,
  • signed attestations from a KYC or compliance service,
  • Merkle inclusion proofs for off-chain datasets,
  • authenticated API snapshots packaged by a relayer.

What must be verified on-chain

Because the witness is untrusted, a secure circuit usually checks some combination of:

  • authenticity: was this message signed by the trusted publisher?
  • freshness: is this update newer than the previous accepted update?
  • domain separation: was the message intended for this contract or this use case?
  • replay protection: has this exact round/nonce already been used?
  • range constraints: is the value within acceptable bounds?
  • binding to transaction context: if needed, is the message tied to the caller or operation?

A “signed price feed” is just one concrete instance of this general pattern.

Minimal Compact structure

The following example models a feed update with a round number and a signer. The exact signature-verification helper depends on the current library release, so I leave that one helper isolated as the only assumption.

struct PriceUpdate {
  price: Uint<32>,
  round: Uint<32>,
  signer: Bytes<32>,
  signature: Bytes<64>
}

export sealed ledger trustedSigner: Bytes<32>;
export ledger lastAcceptedRound: Uint<32>;
export ledger currentPrice: Uint<32>;

witness nextPriceUpdate(): PriceUpdate;

// Assumption: bind this helper to the current crypto-verification API from the official docs.
// The shape is shown to isolate the trust logic in one place.
pure circuit verifySignedPriceUpdate(update: PriceUpdate): Boolean {
  // Replace with the current standard-library or project-specific signature check.
  // Expected policy:
  // 1. signer must equal trustedSigner
  // 2. signature must authenticate (price, round) under signer
  return true;
}

export circuit initializeSigner(signer: Bytes<32>): [] {
  trustedSigner = disclose(signer);
}

export circuit submitVerifiedPrice(): Uint<32> {
  const update = nextPriceUpdate();

  // Assumption: use the current assertion/failure primitive from the Compact stdlib/docs.
  assert(verifySignedPriceUpdate(update));
  assert(update.signer == trustedSigner);
  assert(update.round > lastAcceptedRound);

  lastAcceptedRound = update.round;
  currentPrice = update.price;

  return update.price;
}

Why this pattern works

The witness itself is not trusted. The verification logic is trusted.

If the witness returns a fake price:

  • the signer check should fail,
  • the signature check should fail,
  • the monotonic round check should fail, or
  • some policy constraint should fail.

That is the right mental model for witnesses on Midnight: they are input channels, not trust anchors.

A simpler fully-constrained variant

If you are not yet using signatures, you can still apply the same pattern using a precommitted digest or versioned value. This is less flexible than real signed updates, but it stays closer to the minimal syntax shown in the public primer.

struct QuotedValue {
  value: Uint<32>,
  round: Uint<32>
}

export ledger approvedValue: Uint<32>;
export ledger approvedRound: Uint<32>;

witness quotedValue(): QuotedValue;

export circuit acceptQuotedValue(): Uint<32> {
  const q = quotedValue();

  assert(q.round > approvedRound);

  approvedValue = q.value;
  approvedRound = q.round;

  return q.value;
}

This second snippet is not a substitute for cryptographic authentication. It shows the structural pattern only: witness input becomes acceptable only after the circuit enforces policy.

What to test

For this pattern, your tests should at minimum cover:

  • valid signed update accepted,
  • update from wrong signer rejected,
  • stale round rejected,
  • same round replay rejected,
  • malformed proof/signature rejected,
  • large-but-valid value accepted only if it meets range policy.

If your feed is economically sensitive, also test edge conditions like zero prices, overflow boundaries on Uint<n>, and out-of-order delivery.

Trust tradeoffs

This pattern gives you the best decentralization story if the verification logic is strong.

Its upside:

  • no privileged updater needs to write every value,
  • users can carry the oracle data with their own transactions,
  • the contract can accept data from any relayer as long as the cryptographic proof checks out.

Its downside:

  • correctness depends on careful validation,
  • witness-driven flows are easy to get wrong if you forget freshness or replay checks,
  • signature verification and proof packaging may increase implementation complexity.

A useful rule of thumb is this: witnesses are great at transport; they are bad as sources of truth. The truth must come from whatever the circuit verifies.

Admin-updated ledger fields with access control

The second pattern is operationally simpler: appoint an updater, store the latest value in ledger state, and gate writes with access control.

This is the closest thing to a conventional centralized oracle.

When to use this pattern

Use admin-updated fields when:

  • your application can tolerate a trusted operator,
  • values update on a fixed schedule,
  • the simplicity of one write path matters more than decentralizing the feed,
  • you want downstream consumers to read a canonical on-chain value without packaging witness data every time.

Typical examples include:

  • a project-maintained configuration value,
  • a daily settlement rate,
  • a governance-controlled parameter,
  • a manually reviewed compliance or risk flag.

Design goals

A good admin-updated oracle contract usually wants:

  • an immutable or tightly controlled admin identity,
  • a monotonic version or round number,
  • a clear initialization path,
  • an optional rotation path if the admin key must change,
  • a policy for what consumers should do if the value is stale.

Here is a minimal version.

export sealed ledger adminKey: Bytes<32>;
export ledger latestValue: Uint<32>;
export ledger latestVersion: Uint<32>;

witness callerKey(): Bytes<32>;

export circuit initializeAdmin(key: Bytes<32>): [] {
  adminKey = disclose(key);
}

export circuit updateValue(nextValue: Uint<32>, nextVersion: Uint<32>): [] {
  const caller = callerKey();

  assert(caller == adminKey);
  assert(nextVersion > latestVersion);

  latestValue = disclose(nextValue);
  latestVersion = disclose(nextVersion);
}

export circuit readValue(): [Uint<32>, Uint<32>] {
  return [latestValue, latestVersion];
}

What this contract does

  • adminKey is sealed, so it cannot be rebound after initial setup.
  • updateValue requires the witness-reported caller key to match adminKey.
  • latestVersion prevents accidental replays or out-of-order updates.
  • consumer circuits can read the current value and version from on-chain state.

Important security note

This example uses a witness to provide a caller identity because that is one of the primitives explicitly described in the supplied reference material. In a production contract, you should bind authorization to the current official Midnight identity/authentication mechanism rather than relying on a naked witness-supplied caller value.

That distinction matters because the same witness warning still applies: a witness alone is not self-authenticating.

So the production pattern is:

  • use the platform’s documented caller/auth model for authorization,
  • store the updater identity on-chain,
  • reject writes from everyone else.

The structural lesson remains the same: this oracle pattern trusts an administrator, not a witness.

Operational model

In a real system, the updater process is usually an off-chain service that:

  1. polls the upstream source,
  2. normalizes the value,
  3. increments the version,
  4. submits the update transaction,
  5. monitors the chain for success.

That operational simplicity is why this pattern is common even in systems that aspire to decentralize later. It is also why it is dangerous to hide the trust model. You are trusting a key, an operator, and a process.

What to test

At minimum, test:

  • initialization succeeds once,
  • non-admin update rejected,
  • stale or repeated version rejected,
  • newer version accepted,
  • read circuit returns expected value and version.

If you add rotation:

  • only current admin can rotate,
  • rotated-out admin can no longer update,
  • rotated-in admin can update.

Trust tradeoffs

This pattern is easy to reason about and easy to integrate. Consumers do not need to package signatures or witness proofs with every call. They simply read the on-chain value.

But the tradeoff is direct:

  • if the admin is compromised, the feed can be corrupted;
  • if the admin is offline, the feed can go stale;
  • if governance rotates the key carelessly, downstream contracts may break or freeze.

So this is not a trustless oracle. It is an explicit trusted-operator oracle. That is acceptable in many applications, but it should be documented plainly.

Cross-contract calls for composed state

The third pattern is different. Instead of importing data from the outside world directly, one contract consumes state that another contract already maintains on-chain.

This is useful when the “oracle” is not really an oracle service at all, but a specialized contract whose state should be reused by other contracts.

Examples:

  • a dedicated price registry contract used by multiple markets,
  • a system-wide configuration contract,
  • a risk engine contract that stores collateral parameters,
  • a contract that records approved attestations or membership state.

When to use this pattern

Use cross-contract composition when:

  • the source value is already maintained on-chain,
  • multiple contracts need the same canonical state,
  • you want to separate responsibilities across contracts,
  • you want one contract to own update policy and others to consume it.

This is not a way to eliminate trust. It is a way to centralize trust in one on-chain source instead of duplicating it.

Provider contract

A minimal provider can look like this:

export ledger sharedPrice: Uint<32>;
export ledger sharedRound: Uint<32>;

export circuit setSharedPrice(price: Uint<32>, round: Uint<32>): [] {
  sharedPrice = disclose(price);
  sharedRound = disclose(round);
}

export circuit getSharedPrice(): [Uint<32>, Uint<32>] {
  return [sharedPrice, sharedRound];
}

This contract does one thing: expose a canonical price and round.

Consumer contract

The exact cross-contract import/call syntax is not included in the supplied Compact primer, so the following example shows the intended structure and should be bound to the current docs before compilation.

// Assumption: replace this import form with the current Compact cross-contract syntax.
import "./PriceProvider" prefix PriceProvider;

export ledger lastObservedPrice: Uint<32>;
export ledger lastObservedRound: Uint<32>;

export circuit syncFromProvider(): [Uint<32>, Uint<32>] {
  // Assumption: replace this call with the current cross-contract call syntax.
  const [price, round] = PriceProvider.getSharedPrice();

  lastObservedPrice = price;
  lastObservedRound = round;

  return [price, round];
}

Why this pattern is useful

Suppose you have three application contracts:

  • LendingMarket
  • LiquidationEngine
  • Treasury

If each one used its own admin-updated price field, you would create three separate trust surfaces and three chances for divergence. A shared provider contract avoids that.

The provider becomes the single state authority, and consumers inherit its answer.

What to verify

Even with cross-contract composition, consumers should still think about:

  • is the provider contract authenticated and governed correctly?
  • what happens if the provider value is stale?
  • is there an expected minimum round or version?
  • can the provider be upgraded, replaced, or paused?
  • should the consumer cache the value or read it fresh every time?

For example, a consumer may want to reject a provider round lower than a locally remembered minimum, or it may want to store the observed round to prevent regressions.

What to test

At minimum, test:

  • provider state update works,
  • consumer reads the provider correctly,
  • consumer handles unchanged provider state predictably,
  • consumer does not regress to an older round if you enforce monotonicity,
  • integration behaves correctly after provider redeployment or upgrade, if your architecture allows that.

Trust tradeoffs

Cross-contract composition often feels more “on-chain” than the other two patterns, but the trust question has only moved, not vanished.

The consumer trusts:

  • the provider contract’s write policy,
  • the provider contract’s upgrade/governance policy,
  • the correctness of the provider’s own upstream oracle model.

If the provider itself uses an admin-updated feed, then every consumer indirectly trusts that admin. If the provider uses witness-submitted signed updates, every consumer indirectly trusts that verification logic.

So cross-contract calls are best thought of as a trust distribution pattern, not a source-of-truth pattern on their own.

Choosing among the three patterns

These patterns solve different problems.

Choose witness-provided data with on-chain verification when:

  • you want low coordination between users and a central updater,
  • the data source can produce verifiable proofs or signatures,
  • freshness and per-transaction customization matter.

This is usually the best answer for highly dynamic data, assuming you are prepared to implement verification carefully.

Choose admin-updated ledger fields when:

  • you need something simple and dependable,
  • the update cadence is modest,
  • your application already has a trusted operator or governance process.

This is often the right first version for internal or low-risk applications.

Choose cross-contract composition when:

  • the data is already on-chain elsewhere,
  • multiple contracts must share one canonical value,
  • you want to isolate oracle maintenance from business logic.

This is usually an architectural improvement rather than a replacement for the other two.

A common production architecture

Many production systems combine them:

  • a provider contract accepts witness-submitted signed updates,
  • that provider stores the latest verified value on-chain,
  • multiple consumer contracts then use cross-contract reads,
  • governance retains an admin emergency path to pause, rotate signer keys, or switch providers.

That hybrid architecture usually gives the cleanest separation of duties.

Pitfalls and common errors

1. Treating witness output as trusted

This is the biggest mistake. The docs explicitly warn against it. If a witness supplies a price, signature, caller identity, nonce, or timestamp-like value, your circuit must constrain it.

2. Verifying authenticity but not freshness

A real signed update can still be stale. Always pair authenticity checks with a round, version, or nonce policy.

3. Forgetting replay protection

If the same signed payload can be submitted twice, you may open the door to duplicate state transitions or stale-value reuse. Track the last accepted round or used nonce.

4. Using an admin-updated feed without documenting the trust model

If a single team-controlled key can set the oracle, say so. Hidden trust assumptions are worse than centralized ones.

5. Duplicating shared values across many contracts

If several contracts need the same value, a provider/consumer architecture is usually cleaner than independent copies. Otherwise you create synchronization problems.

6. Ignoring staleness in consumers

A consumer contract should decide what to do when the source value is old, unchanged, or unavailable. “Latest” is not always “fresh enough.”

7. Relying on undocumented auth shortcuts

Do not infer caller identity from a witness unless the platform docs explicitly define that mechanism as authoritative. Use the official Midnight authorization model for access control.

8. Overfitting to one oracle source

If your contract hardcodes one signer, one provider, or one updater path, plan for rotation and recovery. Key compromise and operator failure are operational realities, not edge cases.

References