-
Notifications
You must be signed in to change notification settings - Fork 99
ci: calculate SGX mr_enclave for notary server in gramine docker #701
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 9 commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
077f17f
ci: calculate SGX mr_enclave for notary server in gramine docker
heeckhau fc6f8e2
test: different caching + remove intermediate artifact
heeckhau 8798d3b
test
heeckhau f10cc97
wip
heeckhau 2952ab1
test
heeckhau 4165870
Added documentation
heeckhau 1fcbe96
improvements
heeckhau ae9f49f
ci: remove old tee github workflow
heeckhau 2edcc28
ci: attest build result for dev branch builds and releases
heeckhau 22210f6
Update .github/workflows/ci.yml
heeckhau a9bb4bd
review feedback
heeckhau File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,6 +8,11 @@ on: | |
- "[v]?[0-9]+.[0-9]+.[0-9]+*" | ||
pull_request: | ||
|
||
permissions: | ||
id-token: write | ||
contents: read | ||
attestations: write | ||
|
||
env: | ||
CARGO_TERM_COLOR: always | ||
CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse | ||
|
@@ -17,7 +22,7 @@ env: | |
# - https://github.com/privacy-scaling-explorations/mpz/issues/178 | ||
# 32 seems to be big enough for the foreseeable future | ||
RAYON_NUM_THREADS: 32 | ||
|
||
GIT_COMMIT_HASH: ${{ github.event.pull_request.head.sha || github.sha }} | ||
jobs: | ||
fmt: | ||
name: Check formatting | ||
|
@@ -38,6 +43,7 @@ jobs: | |
|
||
- name: Check formatting | ||
run: cargo +nightly fmt --check --all | ||
|
||
build-and-test: | ||
name: Build and test | ||
runs-on: ubuntu-latest | ||
|
@@ -62,6 +68,7 @@ jobs: | |
|
||
- name: Test | ||
run: cargo test | ||
|
||
wasm: | ||
name: Build and Test wasm | ||
runs-on: ubuntu-latest | ||
|
@@ -136,6 +143,7 @@ jobs: | |
|
||
- name: Run integration tests | ||
run: cargo test --profile tests-integration --workspace --exclude tlsn-tls-client --exclude tlsn-tls-core -- --include-ignored | ||
|
||
coverage: | ||
runs-on: ubuntu-latest | ||
env: | ||
|
@@ -155,4 +163,118 @@ jobs: | |
with: | ||
token: ${{ secrets.CODECOV_TOKEN }} | ||
files: lcov.info | ||
fail_ci_if_error: true | ||
fail_ci_if_error: true | ||
|
||
build-sgx: | ||
runs-on: ubuntu-latest | ||
needs: build-and-test | ||
container: | ||
image: rust:latest | ||
|
||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
|
||
- name: Install Clang | ||
run: | | ||
apt update | ||
apt install -y clang | ||
|
||
- name: Use caching | ||
uses: Swatinem/[email protected] | ||
|
||
- name: Build Rust Binary | ||
run: | | ||
cargo build --bin notary-server --release --features tee_quote | ||
heeckhau marked this conversation as resolved.
Show resolved
Hide resolved
|
||
cp --verbose target/release/notary-server $GITHUB_WORKSPACE | ||
|
||
- name: Upload Binary for use in the Gramine Job | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: notary-server | ||
path: notary-server | ||
if-no-files-found: error | ||
|
||
gramine-sgx: | ||
runs-on: ubuntu-latest | ||
needs: build-sgx | ||
environment: tee | ||
heeckhau marked this conversation as resolved.
Show resolved
Hide resolved
|
||
container: | ||
image: gramineproject/gramine:latest | ||
|
||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
|
||
- name: Restore SGX signing key from secrets | ||
run: | | ||
mkdir -p "${HOME}/.config/gramine/" | ||
echo "${{ secrets.SGX_SIGNING_KEY }}" > "${HOME}/.config/gramine/enclave-key.pem" | ||
# verify key | ||
openssl rsa -in "${HOME}/.config/gramine/enclave-key.pem" -check -noout | ||
|
||
- name: Download notary-server binary from build job | ||
uses: actions/download-artifact@v4 | ||
with: | ||
name: notary-server | ||
path: crates/notary/server/tee | ||
|
||
- name: Install jq | ||
run: | | ||
apt update | ||
apt install -y jq | ||
|
||
- name: Run notary-server in Gramine | ||
heeckhau marked this conversation as resolved.
Show resolved
Hide resolved
|
||
run: | | ||
cd crates/notary/server/tee | ||
|
||
chmod +x notary-server | ||
|
||
gramine-manifest \ | ||
-Dlog_level=debug \ | ||
-Darch_libdir=/lib/x86_64-linux-gnu \ | ||
-Dself_exe=notary-server \ | ||
notary-server.manifest.template \ | ||
notary-server.manifest | ||
|
||
gramine-sgx-sign \ | ||
--manifest notary-server.manifest \ | ||
--output notary-server.manifest.sgx | ||
|
||
gramine-sgx-sigstruct-view --verbose --output-format=json notary-server.sig | tee >> notary-server-sigstruct.json | ||
|
||
cat notary-server-sigstruct.json | ||
|
||
mr_enclave=$(jq -r '.mr_enclave' notary-server-sigstruct.json) | ||
mr_signer=$(jq -r '.mr_signer' notary-server-sigstruct.json) | ||
|
||
echo "mrenclave=$mr_enclave" >>"$GITHUB_OUTPUT" | ||
echo "#### sgx mrenclave" | tee >>$GITHUB_STEP_SUMMARY | ||
echo "\`\`\`mr_enclave: ${mr_enclave}\`\`\`" | tee >>$GITHUB_STEP_SUMMARY | ||
echo "\`\`\`mr_signer: ${mr_signer}\`\`\`" | tee >>$GITHUB_STEP_SUMMARY | ||
|
||
- name: Upload notary-server and signatures | ||
id: upload-notary-server-sgx | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: notary-server-sgx.zip | ||
path: | | ||
crates/notary/server/tee/notary-server | ||
crates/notary/server/tee/notary-server-sigstruct.json | ||
crates/notary/server/tee/notary-server.sig | ||
crates/notary/server/tee/notary-server.manifest | ||
crates/notary/server/tee/notary-server.manifest.sgx | ||
crates/notary/server/tee/config | ||
crates/notary/server/tee/notary-server-sgx.md | ||
if-no-files-found: error | ||
|
||
- name: Attest Build Provenance | ||
if: startsWith(github.ref, 'refs/tags/') || github.ref == 'refs/heads/dev' | ||
uses: actions/attest-build-provenance@v2 | ||
with: | ||
subject-name: notary-server-sgx.zip | ||
subject-digest: sha256:${{ steps.upload-notary-server-sgx.outputs.artifact-digest }} | ||
|
||
- uses: geekyeggo/delete-artifact@v5 # Delete notary-server from the build job, It is part of the zipfile with the signature | ||
with: | ||
name: notary-server |
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
# TLSNotary Notary Server for Intel SGX | ||
heeckhau marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
This package contains the **SGX-enabled version of the TLSNotary Notary Server**, built with **Gramine** to run securely on Intel SGX hardware. Follow the instructions below to install dependencies, verify the integrity of the enclave, and launch the server. | ||
|
||
## Install Intel SGX Drivers & Runtime Dependencies | ||
|
||
Ensure your system has the required Intel SGX components installed. | ||
|
||
```sh | ||
wget https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | ||
cat intel-sgx-deb.key | sudo tee /etc/apt/keyrings/intel-sgx-keyring.asc > /dev/null | ||
|
||
# Add the repository to your sources: | ||
echo 'deb [signed-by=/etc/apt/keyrings/intel-sgx-keyring.asc arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu noble main' | sudo tee /etc/apt/sources.list.d/intel-sgx.list | ||
|
||
sudo apt-get update | ||
sudo apt-get install libsgx-epid libsgx-quote-ex libsgx-dcap-ql -y | ||
``` | ||
|
||
For more details, refer to the official **[Intel SGX Installation Guide](https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_SGX_SW_Installation_Guide_for_Linux.pdf).** | ||
|
||
## Verify the Enclave Signature (`MR_ENCLAVE`, `MR_SIGNER`) | ||
|
||
Before running the Notary Server, verify that the provided **enclave signature (`.sig`)** matches the expected **`MR_ENCLAVE` and `MR_SIGNER`** values. | ||
heeckhau marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Run the following command inside a **Gramine Docker container** to inspect the enclave's signature: | ||
heeckhau marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
```sh | ||
docker run --rm -v "$(pwd):/work" -w /work gramineproject/gramine:latest \ | ||
"gramine-sgx-sigstruct-view --verbose --output-format=json notary-server.sig" | ||
``` | ||
|
||
The output should be the same as `notary-server-sigstruct.json` | ||
|
||
## Run the Notary Server in SGX | ||
|
||
Once verification is complete, you can start the Notary Server inside a **Gramine-protected SGX enclave**. | ||
|
||
```sh | ||
docker run --detach \ | ||
--restart=unless-stopped \ | ||
--device=/dev/sgx_enclave \ | ||
--device=/dev/sgx_provision \ | ||
--volume=/var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket \ | ||
--publish=7047:7047 \ | ||
--volume="$(pwd):/work" \ | ||
--workdir=/work \ | ||
gramineproject/gramine:latest \ | ||
"gramine-sgx notary-server" | ||
heeckhau marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` | ||
|
||
Notes: | ||
- `--device /dev/sgx_enclave --device /dev/sgx_provision` → Exposes SGX devices. | ||
- `--volume=/var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket` → Enables access to **Intel's AESM daemon** (required for remote attestation). | ||
- `-p 7047:7047` → Exposes the notary server on port **7047**. | ||
|
||
## Attestation & Verification | ||
|
||
The Notary Server runs inside an **Intel SGX enclave**, which supports **remote attestation**. When connecting to it, clients should request an **SGX quote** to verify: | ||
|
||
- **MR_ENCLAVE** (ensures the correct enclave binary is running). | ||
- **MR_SIGNER** (ensures the enclave was signed by the expected key). | ||
- **Quote Freshness** (prevents replay attacks). | ||
|
||
To retrieve the SGX attestation quote, navigate to `<your notary server>:7047/info`: | ||
``` | ||
curl localhost:7047/info | jq | ||
``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -42,4 +42,4 @@ sys.disallow_subprocesses = true | |
#### tlsn rev | ||
sgx.isvprodid = 7 | ||
#### F | ||
sgx.isvsvn = 46 | ||
sgx.isvsvn = 1 |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.