Skip to content

WIP :Use key-value-storage/policy-engine crates to make Trustee stateless #1129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Xynnn007 wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

WIP :Use key-value-storage/policy-engine crates to make Trustee stateless #1129

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
9333028
rvps: replace storage impl with kv-storage crate
Xynnn007 Nov 13, 2025
7555dab
deps/policy-engine: refactor to generic design and enhance Regorus su…
Xynnn007 Dec 2, 2025
639cc01
AS: migrate to external policy-engine crate
Xynnn007 Dec 2, 2025
ad5b345
kbs: migrate to external policy-engine/kv-storage crate
Xynnn007 Dec 2, 2025
e8989f0
integration: apply for new configuration shapes
Xynnn007 Dec 2, 2025
28af42c
trustee-cli/kbs: default to deny_all with modular policy store
Xynnn007 Dec 9, 2025
bb0fbd7
docs: update AS/KBS config docs due to storages replacement
Xynnn007 Dec 10, 2025
8f5ade0
kbs/deployments: update kbs config for deployments
Xynnn007 Dec 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
344 changes: 160 additions & 184 deletions Cargo.lock
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions attestation-service/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,10 @@ futures = "0.3.17"
hex.workspace = true
jsonwebtoken.workspace = true
kbs-types.workspace = true
key-value-storage.path = "../deps/key-value-storage"
lazy_static.workspace = true
openssl.workspace = true
policy-engine.path = "../deps/policy-engine"
prost = { workspace = true, optional = true }
rand = "0.8.5"
reference-value-provider-service.path = "../rvps"
Expand Down
127 changes: 97 additions & 30 deletions attestation-service/docs/config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,16 @@ section:

| Property | Type | Description | Required | Default |
|----------------------------|-----------------------------|-----------------------------------------------------|----------|---------|
| `work_dir` | String | The location for Attestation Service to store data. | False | Firstly try to read from ENV `AS_WORK_DIR`. If not any, use `/opt/confidential-containers/attestation-service` |
| `rvps_config` | [RVPSConfiguration][2] | RVPS configuration | False | - |
| `attestation_token_broker` | [AttestationTokenBroker][1] | Attestation result token configuration. | False | - |
| `work_dir` | String | The location for Attestation Service to store data. | No | ENV `AS_WORK_DIR`, otherwise `/opt/confidential-containers/attestation-service` |
| `rvps_config` | [RVPSConfiguration][3] | RVPS configuration | No | `BuiltIn` |
| `attestation_token_broker` | [AttestationTokenBroker][1] | Attestation result token configuration. | No | See below |
| `verifier_config` | Object | Optional verifier specific configuration (for example TPM) | No | None |

[1]: #attestationtokenbroker
[2]: #rvps-configuration
[2]: #tokensignerconfig
[3]: #rvps-configuration
[4]: #policyengineconfig
[5]: #keyvaluestorage

#### AttestationTokenBroker

Expand All @@ -31,10 +35,8 @@ section:
| `developer_name` | String | The developer name to be used as part of the Verifier ID in the EAR | No |`https://confidentialcontainers.org`|
| `build_name` | String | The build name to be used as part of the Verifier ID in the EAR | No | Automatically generated from Cargo package and AS version|
| `profile_name` | String | The Profile that describes the EAR token | No |tag:github.com,2024:confidential-containers/Trustee`|
| `policy_dir` | String | The path to the work directory that contains policies to provision the tokens. | No |`/opt/confidential-containers/attestation-service/token/policies`|
| `signer` | [TokenSignerConfig][1] | Signing material of the attestation result token. | No | None |

[1]: #tokensignerconfig
| `signer` | [TokenSignerConfig][2] | Signing material of the attestation result token. | No | None |
| `policy_engine`| [PolicyEngineConfig][4] | Storage backend used to keep EAR policies. | No | In‑memory storage |

#### TokenSignerConfig

Expand All @@ -56,23 +58,11 @@ This section is **optional**. When omitted, a new EC key pair is generated and u

If `type` is set to `BuiltIn`, the following extra properties can be set

| Property | Type | Description | Required | Default |
|----------------|-------------------------|-----------------------------------------------------------------------|----------|----------|
| `storage` | ReferenceValueStorageConfig | Configuration of storage for reference values (`LocalFs` or `LocalJson`) | No | `LocalFs`|

`ReferenceValueStorageConfig` can contain either a `LocalFs` configuration or a `LocalJson` configuration.

For `LocalFs`, the following properties can be set

| Property | Type | Description | Required | Default |
|----------------|-------------------------|----------------------------------------------------------|----------|----------|
| `file_path` | String | The path to the directory storing reference values | No | `/opt/confidential-containers/attestation-service/reference_values`|

For `LocalJson`, the following properties can be set
| Property | Type | Description | Required | Default |
|----------|----------------------------|-----------------------------------------------|----------|----------|
| `storage`| [KeyValueStorage][5] | Storage backend for reference values | No | `Memory` |

| Property | Type | Description | Required | Default |
|----------------|-------------------------|----------------------------------------------------------|----------|----------|
| `file_path` | String | The path to the file that storing reference values | No | `/opt/confidential-containers/attestation-service/reference_values.json`|
See [KeyValueStorage][5] for available storage backends and their configuration options.

##### Remote RVPS

Expand All @@ -90,16 +80,21 @@ Running with a built-in RVPS:
```json
{
"work_dir": "/var/lib/attestation-service/",
"policy_engine": "opa",
"rvps_config": {
"type": "BuiltIn",
"storage": {
"type": "LocalFs"
"file_path": "/var/lib/attestation-service/reference-values"
"type": "LocalFs",
"dir_path": "/var/lib/attestation-service/reference-values"
}
},
"attestation_token_broker": {
"duration_min": 5
"duration_min": 5,
"policy_engine": {
"storage": {
"type": "LocalJson",
"file_path": "/var/lib/attestation-service/ear-policies.json"
}
}
}
}
```
Expand All @@ -109,7 +104,6 @@ Running with a remote RVPS:
```json
{
"work_dir": "/var/lib/attestation-service/",
"policy_engine": "opa",
"rvps_config": {
"type": "GrpcRemote",
"address": "127.0.0.1:50003"
Expand All @@ -125,14 +119,16 @@ Configurations for token signer
```json
{
"work_dir": "/var/lib/attestation-service/",
"policy_engine": "opa",
"rvps_config": {
"type": "GrpcRemote",
"address": "127.0.0.1:50003"
},
"attestation_token_broker": {
"duration_min": 5,
"issuer_name": "some-body",
"policy_engine": {
"storage": { "type": "Memory" }
},
"signer": {
"key_path": "/etc/coco-as/signer.key",
"cert_url": "https://example.io/coco-as-certchain",
Expand All @@ -141,3 +137,74 @@ Configurations for token signer
}
}
```

Running with PostgreSQL storage:

```json
{
"work_dir": "/var/lib/attestation-service/",
"rvps_config": {
"type": "BuiltIn",
"storage": {
"type": "Postgres",
"host": "localhost",
"port": 5432,
"db": "coco_as",
"username": "postgres",
"password": "password",
"table": "reference_values"
}
},
"attestation_token_broker": {
"duration_min": 5,
"policy_engine": {
"storage": {
"type": "Postgres",
"host": "localhost",
"port": 5432,
"db": "coco_as",
"username": "postgres",
"password": "password",
"table": "ear_policies"
}
}
}
}
```

#### PolicyEngineConfig

| Property | Type | Description | Default |
|----------|----------------------|---------------------------------------------------|----------|
| `storage`| [KeyValueStorage][5] | Backend used to persist EAR policies | `Memory` |

See [KeyValueStorage][5] for available storage backends and their configuration options.

#### KeyValueStorage

The `KeyValueStorage` configuration defines the storage backend used for key-value pairs. It is used in multiple places throughout the configuration, including:

- Policy Engine storage (see [PolicyEngineConfig][4])
- RVPS storage (see [RVPS Configuration][3])

The following storage types are supported:

| `type` value | Extra fields | Description | Default path |
|--------------|--------------|-------------|--------------|
| `Memory` | None | Ephemeral in‑memory store (data is lost on restart) | N/A |
| `LocalFs` | `dir_path` | Store each value as a file | `/opt/confidential-containers/storage/local_fs` |
| `LocalJson` | `file_path` | Store all values in one JSON file | `/opt/confidential-containers/storage/local_json/key_value.json` |
| `Postgres` | See below | Store key-value pairs in a PostgreSQL database table | N/A |

When `type` is `Postgres`, the following properties can be set:

| Property | Type | Description | Required | Default |
|------------|---------|------------------------------------------------|----------|--------------|
| `db` | String | The name of the PostgreSQL database | No | `postgres` |
| `username` | String | The username of the PostgreSQL database | No | `postgres` |
| `password` | String | The password of the PostgreSQL database | No | None |
| `port` | Integer | The port of the PostgreSQL database | No | `5432` |
| `host` | String | The host of the PostgreSQL database | No | `localhost` |
| `table` | String | The name of the table to store key-value pairs | No | `key_value` |

> NOTE: If the `POSTGRES_URL` environment variable is set with a PostgreSQL connection URI, it will be used instead of the configuration parameters above.
2 changes: 1 addition & 1 deletion attestation-service/src/bin/restful/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,7 @@ pub async fn get_policies(
.await
.context("get policies")?
.into_iter()
.map(|(id, digest)| json!({"policy-id": id, "policy-hash": digest}))
.map(|id| json!({"policy-id": id}))
.collect::<Vec<_>>();

let policy_list =
Expand Down
13 changes: 8 additions & 5 deletions attestation-service/src/config.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,27 +89,28 @@ impl TryFrom<&Path> for Config {

#[cfg(test)]
mod tests {
use policy_engine::PolicyEngineConfig;
use rstest::rstest;
use std::path::PathBuf;

use super::Config;
use crate::ear_token::TokenSignerConfig;
use crate::rvps::RvpsCrateConfig;
use crate::{ear_token::EarTokenConfiguration, rvps::RvpsConfig};
use reference_value_provider_service::storage::{local_fs, ReferenceValueStorageConfig};
use key_value_storage::{local_fs, KeyValueStorageConfig};

#[rstest]
#[case("./tests/configs/example1.json", Config {
work_dir: PathBuf::from("/var/lib/attestation-service/"),
rvps_config: RvpsConfig::BuiltIn(RvpsCrateConfig {
storage: ReferenceValueStorageConfig::LocalFs(local_fs::Config::default()),
storage: KeyValueStorageConfig::LocalFs(local_fs::Config::default()),
extractors: None,
}),
attestation_token_broker: EarTokenConfiguration {
duration_min: 5,
issuer_name: "test".into(),
signer: None,
policy_dir: "/var/lib/attestation-service/policies".into(),
policy_engine: PolicyEngineConfig::default(),
developer_name: "someone".into(),
build_name: "0.1.0".into(),
profile_name: "tag:github.com,2024:confidential-containers/Trustee".into()
Expand All @@ -119,13 +120,15 @@ mod tests {
#[case("./tests/configs/example2.json", Config {
work_dir: PathBuf::from("/var/lib/attestation-service/"),
rvps_config: RvpsConfig::BuiltIn(RvpsCrateConfig {
storage: ReferenceValueStorageConfig::LocalFs(local_fs::Config::default()),
storage: KeyValueStorageConfig::Memory,
extractors: None,
}),
attestation_token_broker: EarTokenConfiguration {
duration_min: 5,
issuer_name: "test".into(),
policy_dir: "/var/lib/attestation-service/policies".into(),
policy_engine: PolicyEngineConfig {
storage: KeyValueStorageConfig::Memory,
},
developer_name: "someone".into(),
build_name: "0.1.0".into(),
profile_name: "tag:github.com,2024:confidential-containers/Trustee".into(),
Expand Down
Loading
Loading