Skip to content

Commit 19447aa

Browse files
authored
Tee dev cleanup (#759)
* build: added scripts for local tee/sgx development * Improved documentation: move all explanation to one README file
1 parent 8afb7a4 commit 19447aa

File tree

6 files changed

+186
-83
lines changed

6 files changed

+186
-83
lines changed

.github/workflows/ci.yml

+8-14
Original file line numberDiff line numberDiff line change
@@ -280,7 +280,7 @@ jobs:
280280
crates/notary/server/tee/notary-server.manifest
281281
crates/notary/server/tee/notary-server.manifest.sgx
282282
crates/notary/server/tee/config
283-
crates/notary/server/tee/notary-server-sgx.md
283+
crates/notary/server/tee/README.md
284284
if-no-files-found: error
285285

286286
- name: Attest Build Provenance
@@ -304,24 +304,17 @@ jobs:
304304
CONTAINER_REGISTRY: ghcr.io
305305
if: github.ref == 'refs/heads/dev' || (startsWith(github.ref, 'refs/tags/v') && contains(github.ref, '.'))
306306
steps:
307+
- name: Checkout repository
308+
uses: actions/checkout@v4
309+
with:
310+
sparse-checkout: './crates/notary/server/tee/notary-server-sgx.Dockerfile'
311+
307312
- name: Download notary-server-sgx.zip from gramine-sgx job
308313
uses: actions/download-artifact@v4
309314
with:
310315
name: notary-server-sgx.zip
311316
path: ./notary-server-sgx
312317

313-
- name: Create Dockerfile
314-
run: |
315-
cat <<EOF > ./Dockerfile
316-
FROM gramineproject/gramine:latest
317-
WORKDIR /work
318-
COPY ./notary-server-sgx /work
319-
RUN chmod +x /work/notary-server
320-
LABEL org.opencontainers.image.source=https://github.com/tlsnotary/tlsn
321-
LABEL org.opencontainers.image.description="TLSNotary notary server in SGX/Gramine."
322-
ENTRYPOINT ["gramine-sgx", "notary-server"]
323-
EOF
324-
325318
- name: Log in to the Container registry
326319
uses: docker/login-action@v2
327320
with:
@@ -342,7 +335,8 @@ jobs:
342335
push: true
343336
tags: ${{ steps.meta-notary-server-sgx.outputs.tags }}
344337
labels: ${{ steps.meta-notary-server-sgx.outputs.labels }}
345-
file: ./Dockerfile
338+
file: ./crates/notary/server/tee/notary-server-sgx.Dockerfile
339+
346340
build_and_publish_notary_server_image:
347341
name: Build and publish notary server's image
348342
runs-on: ubuntu-latest

crates/notary/server/tee/README.md

+112-1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,114 @@
11
This folder contains the necessary files to build a Docker image for running the Notary Server on Intel SGX-enabled hardware.
22

3-
The container is built as part of the CI pipeline. For details on the build process, refer to the [CI workflow configuration](../../../../.github/workflows/ci.yml).
3+
## Compile the Notary Server for Intel SGX
4+
5+
We use [Gramine](https://github.com/gramineproject/gramine) to run the Notary Server on Intel SGX. Gramine allows the Notary Server to run in an isolated environment with minimal host requirements.
6+
7+
The isolated environment is defined via the manifest template (`notary-server.manifest.template`).
8+
9+
The Notary Server for SGX is compiled with the Rust feature flag `tee_quote`. This enables the server to use an ephemeral private notary key for signing attestations (`private_key_pem_path: "/ephemeral/notary.key"`) and also adds the SGX *quote* to the server's `/info` endpoint.
10+
11+
### CI
12+
13+
The [notary-server-sgx Docker container](https://github.com/tlsnotary/tlsn/pkgs/container/tlsn%2Fnotary-server-sgx) is built as part of the CI pipeline. For details on the build process, refer to the [CI workflow configuration](../../../../.github/workflows/ci.yml).
14+
15+
CI builds a zip file named `notary-server-sgx.zip`, which contains the compiled binary and the signed manifest. This zip file is available for all releases and `dev` builds in the build artifacts. We also publish a Docker image `notary-server-sgx` at <https://github.com/tlsnotary/tlsn/pkgs/container/tlsn%2Fnotary-server-sgx>. Check the section below for details on running this container.
16+
17+
### Development
18+
19+
You can also build everything locally using the `run-gramine-local.sh` script.
20+
21+
This script creates and signs the Gramine manifest for the Notary Server in a local development environment. It requires the Gramine SDK, so the most convenient way to use it is within a Docker container that includes the necessary dependencies and tools.
22+
23+
> ⚠️ This script assumes that the `notary-server` binary is already built (for `linux/amd64`) and available in the current directory. Make sure it is built with the `tee_quote` feature:
24+
> `cargo build --bin notary-server --release --features tee_quote`
25+
26+
#### Build the Docker Image
27+
28+
To build the Docker image for local development, run:
29+
```sh
30+
docker build -f gramine-local.Dockerfile -t gramine-local .
31+
```
32+
#### Run the Gramine Script
33+
34+
Once the image is built, you can run the `run-gramine-local.sh` script inside the container:
35+
```
36+
docker run --rm -it \
37+
--platform=linux/amd64 \
38+
-v "${PWD}:/app" \
39+
-w /app/ \
40+
gramine-local \
41+
"bash -c ./run-gramine-local.sh"
42+
```
43+
44+
If successful, the script will generate the following files:
45+
* `notary-server.sig`
46+
* `notary-server-sigstruct.json`
47+
* `notary-server.manifest`
48+
* `notary-server.manifest.sgx`
49+
50+
51+
You can verify that the provided **enclave signature (`notary-server.sig`)** matches the expected **`MR_ENCLAVE` and `MR_SIGNER`** values in `notary-server-sigstruct.json`, by running the following command inside a **Gramine Docker container** to inspect the enclave's signature:
52+
53+
```sh
54+
docker run --rm -v "$(pwd):/work" -w /work gramineproject/gramine:latest \
55+
"gramine-sgx-sigstruct-view --verbose --output-format=json notary-server.sig"
56+
```
57+
58+
The output should be the same as `notary-server-sigstruct.json`
59+
60+
## How to Run TLSNotary on Intel SGX?
61+
62+
Before running the Notary Server on Intel SGX hardware, ensure your system has the required Intel SGX components installed:
63+
```sh
64+
wget https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key
65+
cat intel-sgx-deb.key | sudo tee /etc/apt/keyrings/intel-sgx-keyring.asc > /dev/null
66+
67+
# Add the repository to your sources:
68+
echo 'deb [signed-by=/etc/apt/keyrings/intel-sgx-keyring.asc arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu noble main' | sudo tee /etc/apt/sources.list.d/intel-sgx.list
69+
70+
sudo apt-get update
71+
sudo apt-get install libsgx-epid libsgx-quote-ex libsgx-dcap-ql -y
72+
```
73+
74+
For more details, refer to the official **[Intel SGX Installation Guide](https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_SGX_SW_Installation_Guide_for_Linux.pdf).**
75+
76+
### Docker Compose
77+
78+
To run the Notary Server using Docker Compose, create a docker-compose.yml file like the following:
79+
```yaml
80+
services:
81+
dev:
82+
container_name: dev
83+
image: ghcr.io/tlsnotary/tlsn/notary-server-sgx:dev
84+
restart: unless-stopped
85+
devices:
86+
- /dev/sgx_enclave
87+
- /dev/sgx_provision
88+
volumes:
89+
- /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket
90+
ports:
91+
- "7047:7047"
92+
entrypoint: [ "gramine-sgx", "notary-server" ]
93+
```
94+
95+
To retrieve the SGX attestation quote, query the `/info` endpoint:
96+
```sh
97+
curl localhost:7047/info | jq
98+
```
99+
100+
### Run local build directly with Gramine
101+
102+
To run a locally built Notary Server inside a Gramine-protected SGX enclave, execute:
103+
```sh
104+
docker run --detach \
105+
--restart=unless-stopped \
106+
--device=/dev/sgx_enclave \
107+
--device=/dev/sgx_provision \
108+
--volume=/var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket \
109+
--publish=7047:7047 \
110+
--volume="$(pwd):/work" \
111+
--workdir=/work \
112+
gramineproject/gramine:latest \
113+
"gramine-sgx notary-server"
114+
```
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
FROM --platform=linux/amd64 gramineproject/gramine:latest
2+
3+
RUN apt update && \
4+
apt install -y jq openssl zip && \
5+
apt clean && \
6+
rm -rf /var/lib/apt/lists/*
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
FROM gramineproject/gramine:latest
2+
WORKDIR /work
3+
4+
# Copies `notary-server-sgx.zip` from the CI build or created locally via `run-gramine-local.sh`.
5+
COPY ./notary-server-sgx /work
6+
RUN chmod +x /work/notary-server
7+
8+
LABEL org.opencontainers.image.source=https://github.com/tlsnotary/tlsn
9+
LABEL org.opencontainers.image.description="TLSNotary notary server in SGX/Gramine."
10+
11+
ENTRYPOINT ["gramine-sgx", "notary-server"]

crates/notary/server/tee/notary-server-sgx.md

-68
This file was deleted.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
#!/bin/bash
2+
3+
set -euo pipefail
4+
5+
echo "[*] Generating SGX signing key..."
6+
gramine-sgx-gen-private-key
7+
8+
if [ ! -f notary-server ]; then
9+
echo "[!] notary-server binary not found. Please copy it from ci, or build it first."
10+
echo "Note that notary-server must be built for linux/amd64 with tee_quote feature enabled"
11+
exit 1
12+
fi
13+
14+
chmod +x notary-server
15+
16+
echo "[*] Creating Gramine manifest..."
17+
gramine-manifest \
18+
-Dlog_level=debug \
19+
-Darch_libdir=/lib/x86_64-linux-gnu \
20+
-Dself_exe=notary-server \
21+
notary-server.manifest.template \
22+
notary-server.manifest
23+
24+
echo "[*] Signing manifest..."
25+
gramine-sgx-sign \
26+
--manifest notary-server.manifest \
27+
--output notary-server.manifest.sgx
28+
29+
echo "[*] Viewing SIGSTRUCT..."
30+
gramine-sgx-sigstruct-view --verbose --output-format=json notary-server.sig >notary-server-sigstruct.json
31+
32+
cat notary-server-sigstruct.json | jq .
33+
34+
mr_enclave=$(jq -r ".mr_enclave" notary-server-sigstruct.json)
35+
mr_signer=$(jq -r ".mr_signer" notary-server-sigstruct.json)
36+
37+
echo "=============================="
38+
echo "MRENCLAVE: $mr_enclave"
39+
echo "MRSIGNER: $mr_signer"
40+
echo "=============================="
41+
42+
zip -r notary-server-sgx.zip \
43+
notary-server \
44+
notary-server-sigstruct.json \
45+
notary-server.sig \
46+
notary-server.manifest \
47+
notary-server.manifest.sgx \
48+
config \
49+
README.md

0 commit comments

Comments
 (0)