KBS: Missing Features for Production

[zvonkok]

Here is a list of things Trustee doesn't do:

• KBS is not close to FIPS 140-3 level 1 (software module) nor is it aligned with the relevant protection profiles for HSMs (including soft HSMs).
• It is not well suited for a system of system attestation.
• It’s not compliant with KMIP either
• Does not plug into OSCP
• Handling of the RIM database
• Customizable attestation policies
• Sealed secrets
• No authorization for resource retrieval (There should be authorization policies so users can only retrieve their own resources)
• No authentication for resource retrieval
• Designed for just one project or single build (multi-project or mult-build or namespace separation)
• There is only one resource policy. If two projects both set up different resource policies, the new policy can overwritten the previous one
• Resource identification only by path
• High availability/Scalabitly
• HSM requirements / PKCS#11 ?
• Rate limitation to other services like AMD KDS, caching service?

Should an OTS solution like Vault be preferred? (or a cloud HSM so long as you’re not single infrastructure vendor) and Trustee brokers it to the backend?

[Xynnn007]

Hi @zvonkok Nice topic! Some naive questions and thanks in advance

It is not well suited for a system of system attestation.

What do you mean by system attestation?

Customizable attestation policies

What do you mean by "Customizable"?

[zvonkok]

What do you mean by system attestation?

Aiming here for a composite, hierarchical attestation flow that produces one verifiable statement for the entire distributed platform, not just the individual parts, also referring to layered attestation, where each part depends on the layer below it, throughout the complete stack, host + guest + container.

[zvonkok]

What do you mean by "Customizable"?

There is no way for a relying party to inject a Verifier policy for attestation at runtime with multiple different policies.

[fitzthum]

There is no way for a relying party to inject a Verifier policy for attestation at runtime with multiple different policies.

Ah right. This should be pretty easy to add once we have plaintext init data, which is coming soon. We can define a special (optional) init-data key which is used to select the policy.

[Xynnn007]

Aiming here for a composite, hierarchical attestation flow that produces one verifiable statement for the entire distributed platform, not just the individual parts, also referring to layered attestation, where each part depends on the layer below it, throughout the complete stack, host + guest + container.

This is the aim we are going to. We are perfecting the guest stack attestation framework and hopefully it is a right way to go

1. underlying hardware stack. We have finished the device-cpu attestation framework (Composite).
2. underlying software stack ovmf/kernel/rootfs. The ovmf and kernel is supported to be verified for TDX platform and we are getting things better with initdata for SNP. for rootfs, hopefully we could share a same simplified and verifiable guest image building process like using bootc, etc as PeerPod is looking forward
3. container attestation is based on the runtime measurement and AAEL. We are getting closer by add AAEL spec and try to integrate it into TCG2 eventlog format. After the work we will add more details about the policy.

In summary, we will provide a full measurement solution for each level within the guest, and the final user-facing thing is an easy-to-use configurable policy, which could define all different layer expectations.

About the host side, as it is not inside TEE, it would be somehow out of scope. We can use supply chain security to ensure the provenance is good, but it's hard for us to provide verification method for TEE users than CoCo platform deployers.

[fitzthum]

I think Zvonko also wants a report that will cover multiple pods that are part of the same complex workload. Today we don't really have any abstractions that capture this. If all the related pods ask for secrets with similar resource URIs, you can grep the log for this. We only keep track of the workloads in terms of secret requests. This topic could intersect the workload id work, the prometheus work, the plaintext init-data, etc.

[Xynnn007]

Yes. AAEL and runtime measurement can cover this, by something similar with confidential-containers/guest-components#636

[zvonkok]

Composite attestation is acceptable, but I am also targeting software attestation, meaning that regardless of who or what provides the hash, we can include it in our pipeline. Let's say we have a model fingerprint (e.g., Merkle-tree). How can we easily integrate this into our chain of trust, and how can we add this attester/verifier combination easily, such as through plugins, configuration, or initialization data?

[Xynnn007]

How can we easily integrate this into our chain of trust, and how can we add this attester/verifier combination easily, such as through plugins, configuration, or initialization data?

For CoCo or more general CVM, there will be a need for dynamic loading or runtime measurement of objects. For example, the model digest/hash in the inference service scenario you mentioned.

AAEL defines a general framework to support. Let me give two specific examples to illustrate

1. The deployer's app container is responsible for loading the model and then adding it to the boot chain for verification by the caller of the inference service: Then you need to add a piece of code to the logic of the app to calculate the merkle tree root of the model and explicitly call the ASR restful API to record the model loading action. In this case we assume that the ASR redirects the model loading action to AA's extend_rtmr API.
2. The kata/ itself is responsible for loading the model. Then the kata side needs to add same logic to calculate the merkle tree root of the model and call the attestation to extend_rtmr.

Both cases the requests look like (the name LoadModel and the following JSON can be defined. Now we only support PullImage in CoCo)

LoadModel {"name":"gpt","digest":"000.."}

and then we could have attestation service policy for data owner side, with AAEL+TCG2 Eventlog support, e.g.

loadExpectedModel  if {
    event := input.tdx.uefi_event_logs[_]
    event.type_name == "EV_EVENT_TAG"
    event.data.operation == "LoadModel"
    event.data.content.name == "gpt"
    event.data.content.digest == "000.."
}

default executables := 97
executables := 3 if {
    loadExpectedModel
}

[zvonkok]

FYI: Here is the current implementation for model signing: https://openssf.org/blog/2025/04/04/launch-of-model-signing-v1-0-openssf-ai-ml-working-group-secures-the-machine-learning-supply-chain/

[zvonkok]

Event logs integrity is either vTPM or runtime register, do we have any plans to have support for a ledger?

[Xynnn007]

sorry, what do you mean by "ledger"?

[zvonkok]

Ledger, as in blockchain, but we're going off-topic here, lets focus on the matter at hand :)

[Xynnn007]

Ledger, as in blockchain, but we're going off-topic here, lets focus on the matter at hand :)

Yes. Let's go back, but using blockchain as underlying immutable data structure to store rtmr seems an intriguing idea.