feat: bump bindle to v0.9.0-rc.1 #786

FrankYang0529 · 2022-09-23T15:46:10Z

resolve #689

Setup

# clone this PR and build spin
cargo build

# clone bindle dockerize PR https://github.com/deislabs/bindle/pull/343 and build a docker image
docker build -t bindle .

# create a bindle directory and a key
BINDLE_TEMP=$(mktemp -d)
echo $BINDLE_TEMP
export BINDLE_KEYRING=$BINDLE_TEMP/keyring.toml
./target/debug/bindle keys create "VishnuJin<[email protected]>" -f $BINDLE_TEMP/secret_keys.toml

# start bindle server
docker run --name bindle -d --restart=unless-stopped -e RUST_LOG=debug -v $BINDLE_TEMP:/bindle-data -p 8080:8080 bindle
# show bindle server logs
docker logs bindle -f

Test spin bindle prepare

# export BINDLE_TEMP to the folder which we created in the last terminal
export BINDLE_TEMP=

export BINDLE_KEYRING_FILE=$BINDLE_TEMP/keyring.toml
export BINDLE_SECRET_FILE=$BINDLE_TEMP/secret_keys.toml
export BINDLE_LABEL="VishnuJin<[email protected]>"

# create a spin app
./target/debug/spin new http-rust myapp

# build spin app
cd myapp
# remember to update spin_sdk version to v0.5.0 before building it
../target/debug/spin build

# run spin bindle prepare
export STAGING_DIR=$(mktemp -d)
../target/debug/spin bindle prepare --staging-dir $STAGING_DIR

Test spin bindle push

export BINDLE_URL=http://localhost:8080/v1
../target/debug/spin bindle push

Test integration with hippo

# start consul
consul agent -dev

# start nomad
nomad agent -dev

# clone the latest hippo and start hippo
export Database__Driver=sqlite
export ConnectionStrings__Database="Data Source=hippo.db;Cache=Shared"
export ConnectionStrings__Bindle="Address=http://127.0.0.1:8080/v1"
export Nomad__Driver="raw_exec"
export Jwt__Key="ceci n'est pas une jeton"
export Jwt__Issuer="localhost"
export Jwt__Audience="localhost"
cd src/Web
dotnet build
dotnet run

# Run spin deploy in myapp
export HIPPO_USERNAME="admin"
export HIPPO_PASSWORD="p@ssword"
export HIPPO_URL="http://localhost:5309"
../target/debug/spin deploy --deploy-existing-bindle

vdice

Thanks for this much-needed PR @FrankYang0529! I'm excited to test.

The updates look good/correct at a high level. The one question I had was around the scenario when spin (specifically its bindle client) doesn't share the keyring file with the bindle server, eg when the bindle server is fully remote. Do we have logic here to fetch the public key from the bindle server in this case? (... speaking of, I wonder if it would be worth adding a new test for this scenario as well -- I love seeing how we already have existing tests that have been updated!)

FrankYang0529 · 2022-09-27T15:09:59Z

Hi @vdice, I agree we need to fetch the public key. However, even if I use different folders for client and server, I still can run spin up --bindle myapp/0.1.0. Could you help me check where is wrong? I can't figure out why it works.

Setup

# clone this PR and build spin
cargo build

# clone bindle dockerize PR https://github.com/deislabs/bindle/pull/343 and build a docker image
docker build -t bindle .

# create a bindle client directory and a key
BINDLE_TEMP=$(mktemp -d)
echo $BINDLE_TEMP
export BINDLE_KEYRING=$BINDLE_TEMP/client/keyring.toml
./target/debug/bindle keys create "VishnuJin<[email protected]>" -f $BINDLE_TEMP/client/secret_keys.toml

# create a bindle server directory
mkdir $BINDLE_TEMP/server
cp $BINDLE_TEMP/client/keyring.toml $BINDLE_TEMP/server/keyring.toml

# start bindle server
docker run --name bindle -d --restart=unless-stopped -e RUST_LOG=debug -v $BINDLE_TEMP/server:/bindle-data -p 8080:8080 bindle

# show bindle server logs
docker logs bindle -f

Push spin app

# export BINDLE_TEMP to the folder which we created in the last terminal
export BINDLE_TEMP=

export BINDLE_KEYRING_FILE=$BINDLE_TEMP/client/keyring.toml
export BINDLE_SECRET_FILE=$BINDLE_TEMP/client/secret_keys.toml
export BINDLE_LABEL="VishnuJin<[email protected]>"

# create a spin app
./target/debug/spin new http-rust myapp

# build spin app
cd myapp
# remember to update spin_sdk version to v0.5.0 before building it
../target/debug/spin build

# run spin bindle push
export BINDLE_URL=http://localhost:8080/v1
../target/debug/spin bindle push

Run spin app

export BINDLE_URL=http://localhost:8080/v1
./target/debug/spin up --bindle myapp/0.1.0

FrankYang0529 · 2022-09-27T15:23:18Z

Hi @vdice, I agree we need to fetch the public key. However, even if I use different folders for client and server, I still can run spin up --bindle myapp/0.1.0. Could you help me check where is wrong? I can't figure out why it works.

I know the reason! We didn't update bindle package version to v0.9.0-rc.1 in creates/loader. I will fix it.

FrankYang0529 · 2022-09-28T16:20:19Z

I know the reason! We didn't update bindle package version to v0.9.0-rc.1 in creates/loader. I will fix it.

Hi @vdice, I updated creates/loader and used different keyring files for testing. If you want to run make test-e2e on your computer, please remember to update bindle-server to v0.9.0-rc.1. Thank you.

FrankYang0529 · 2022-09-30T16:52:05Z

I will resolve conflicts tomorrow.

vdice · 2022-09-30T19:02:58Z

I will resolve conflicts tomorrow.

👍 I'll be away next week (10/3-10/7) but hopefully another maintainer can take a look during that time as well. I do know that we'll probably want to wait to merge until after the next v0.6.0 release. Thanks!

FrankYang0529 · 2022-10-15T07:30:33Z

Resolve conflicts with #794. Add spin deploy in test steps of this PR.

Setup

# clone this PR and build spin
cargo build

# clone bindle dockerize PR https://github.com/deislabs/bindle/pull/343 and build a docker image
docker build -t bindle .

# create a bindle client directory and a key
BINDLE_TEMP=$(mktemp -d)
echo $BINDLE_TEMP
export BINDLE_KEYRING=$BINDLE_TEMP/client/keyring.toml
./target/debug/bindle keys create "VishnuJin<[email protected]>" -f $BINDLE_TEMP/client/secret_keys.toml

# create a bindle server directory
mkdir $BINDLE_TEMP/server
cp $BINDLE_TEMP/client/keyring.toml $BINDLE_TEMP/server/keyring.toml

# start bindle server
docker run --name bindle -d --restart=unless-stopped -e RUST_LOG=debug -v $BINDLE_TEMP/server:/bindle-data -p 8080:8080 bindle

# show bindle server logs
docker logs bindle -f

Push spin app

# export BINDLE_TEMP to the folder which we created in the first terminal
export BINDLE_TEMP=

export BINDLE_KEYRING_FILE=$BINDLE_TEMP/client/keyring.toml
export BINDLE_SECRET_FILE=$BINDLE_TEMP/client/secret_keys.toml
export BINDLE_LABEL="VishnuJin<[email protected]>"

# create a spin app
./target/debug/spin new http-rust myapp

# build spin app
cd myapp
# remember to update spin_sdk version to v0.5.0 before building it
../target/debug/spin build

# run spin bindle push
export BINDLE_URL=http://localhost:8080/v1
../target/debug/spin bindle push

Run spin app

export BINDLE_URL=http://localhost:8080/v1
./target/debug/spin up --bindle myapp/0.1.0

Start Hippo

# start consul
consul agent -dev

# start nomad
nomad agent -dev

# clone the latest hippo and start hippo
export Database__Driver=inmemory
export ConnectionStrings__Bindle="Address=http://127.0.0.1:8080/v1"
export Nomad__Driver="raw_exec"
export Jwt__Key="ceci n'est pas une jeton"
export Jwt__Issuer="localhost"
export Jwt__Audience="localhost"
cd src/Web
dotnet build
dotnet run

Login and deploy

# export BINDLE_TEMP to the folder which we created in the first terminal
export BINDLE_TEMP=

export BINDLE_KEYRING_FILE=$BINDLE_TEMP/client/keyring.toml
export BINDLE_SECRET_FILE=$BINDLE_TEMP/client/secret_keys.toml
export BINDLE_LABEL="VishnuJin<[email protected]>"
export BINDLE_URL=http://localhost:8080/v1

export HIPPO_USERNAME="admin"
export HIPPO_PASSWORD="p@ssword"
export HIPPO_URL="http://localhost:5309"

# Login
./target/debug/spin login

# Run spin deploy in myapp
cd myapp
../target/debug/spin deploy --deploy-existing-bindle

vdice · 2022-10-18T03:41:32Z

@FrankYang0529 Thanks so much for updating to work with the latest spin deploy changes! I hope to revisit this PR asap.

One other item I was thinking about... thoughts on how we'd surface spin-bindle incompatibilities? For instance, if a user is attempting to deploy using spin v0.5.0 and the bindle server is 0.9.0+, how best to relay that this combination isn't supported. Conversely, when we release a version of spin updated to talk to 0.9.0+, how best to make it obvious that the newer spin client can't talk to previous bindle server versions?

FrankYang0529 · 2022-10-18T12:52:25Z

One other item I was thinking about... thoughts on how we'd surface spin-bindle incompatibilities? For instance, if a user is attempting to deploy using spin v0.5.0 and the bindle server is 0.9.0+, how best to relay that this combination isn't supported. Conversely, when we release a version of spin updated to talk to 0.9.0+, how best to make it obvious that the newer spin client can't talk to previous bindle server versions?

Yeah, this is a good question. If we can determine bindle-server version, we can give a proper message to users. Do you think it's good to return bindle-server version in /v1/healthz API? If yes, I can help to create a PR in bindle. Thanks.

vdice · 2022-10-18T23:47:28Z

Do you think it's good to return bindle-server version in /v1/healthz API? If yes, I can help to create a PR in bindle. Thanks.

Indeed, if bindle-server doesn't yet return its version in another endpoint, that seems like a good candidate. I'll create an issue there just in case other bindle maintainers have input.

Signed-off-by: Frank Yang <[email protected]>

FrankYang0529 · 2022-11-09T14:07:11Z

The CI will fail since we haven't updated the bindle version in CI pipeline, but I add the bindle health check first. I will change the bindle version in Cargo.toml back to https://github.com/fermyon/bindle once we fetch the latest commit and release a new version v0.9.1-rc.2. Currently, we are still blocked by deislabs/bindle#354.

vdice · 2023-01-26T17:27:02Z

@FrankYang0529 thanks so much for your effort here. It's looking like Spin moving towards using OCI registries as its default form of artifact support. Full details can be seen in SIP-008: Using OCI Registries and a working (albeit in-progress) prototype can be seen here.

It remains to be seen whether Spin deprecates support for using Bindle or not. However, I'm anticipating that we'll be sticking with Bindle 0.8.x for the forseeable future, at least while we gain experience and collect feedback around OCI support. Lastly, if we do indeed update Bindle at some point, we'll most likely wait until a non-rc release of the next version is available and hardened.

With the notes above, I suggest we close this PR. However, I wanted to check in with you first before doing so.

itowlson · 2023-07-05T04:18:40Z

As this was suggested to be closed in January, and there's been no feedback to keep it open, I'm going to close it. If anyone wants it furthering then do feel free to reopen though!

FrankYang0529 force-pushed the bump-bindle branch from 499e7d0 to 5371607 Compare September 26, 2022 12:42

FrankYang0529 marked this pull request as ready for review September 26, 2022 13:03

vdice reviewed Sep 27, 2022

View reviewed changes

FrankYang0529 force-pushed the bump-bindle branch from 5371607 to b2109b8 Compare September 28, 2022 16:18

FrankYang0529 force-pushed the bump-bindle branch 2 times, most recently from 5a37fe9 to 3915f41 Compare October 10, 2022 02:47

FrankYang0529 force-pushed the bump-bindle branch 2 times, most recently from d0f4edb to 978787a Compare October 15, 2022 06:52

FrankYang0529 mentioned this pull request Oct 27, 2022

feat(healthz): add version deislabs/bindle#353

Merged

FrankYang0529 force-pushed the bump-bindle branch from 978787a to fb85df1 Compare November 4, 2022 15:22

FrankYang0529 mentioned this pull request Nov 6, 2022

feat(aws): support multiple nodes fermyon/installer#100

Merged

FrankYang0529 force-pushed the bump-bindle branch from fb85df1 to 1ba8e0e Compare November 6, 2022 14:17

FrankYang0529 added 3 commits November 9, 2022 12:00

feat: bump bindle to v0.9.0-rc.1

bcd57ee

Signed-off-by: Frank Yang <[email protected]>

test: add bindle keyring & secret keys

4f41515

Signed-off-by: Frank Yang <[email protected]>

feat(bindle): check version

48066a4

Signed-off-by: Frank Yang <[email protected]>

FrankYang0529 force-pushed the bump-bindle branch from 1ba8e0e to 48066a4 Compare November 9, 2022 14:03

itowlson closed this Jul 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: bump bindle to v0.9.0-rc.1 #786

feat: bump bindle to v0.9.0-rc.1 #786

Uh oh!

FrankYang0529 commented Sep 23, 2022 •

edited

Loading

Uh oh!

vdice left a comment

Uh oh!

FrankYang0529 commented Sep 27, 2022 •

edited

Loading

Uh oh!

FrankYang0529 commented Sep 27, 2022

Uh oh!

FrankYang0529 commented Sep 28, 2022

Uh oh!

FrankYang0529 commented Sep 30, 2022

Uh oh!

vdice commented Sep 30, 2022

Uh oh!

FrankYang0529 commented Oct 15, 2022

Uh oh!

vdice commented Oct 18, 2022 •

edited

Loading

Uh oh!

FrankYang0529 commented Oct 18, 2022

Uh oh!

vdice commented Oct 18, 2022

Uh oh!

FrankYang0529 commented Nov 9, 2022

Uh oh!

vdice commented Jan 26, 2023

Uh oh!

itowlson commented Jul 5, 2023

Uh oh!

Uh oh!

feat: bump bindle to v0.9.0-rc.1 #786

feat: bump bindle to v0.9.0-rc.1 #786

Uh oh!

Conversation

FrankYang0529 commented Sep 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vdice left a comment

Choose a reason for hiding this comment

Uh oh!

FrankYang0529 commented Sep 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FrankYang0529 commented Sep 27, 2022

Uh oh!

FrankYang0529 commented Sep 28, 2022

Uh oh!

FrankYang0529 commented Sep 30, 2022

Uh oh!

vdice commented Sep 30, 2022

Uh oh!

FrankYang0529 commented Oct 15, 2022

Uh oh!

vdice commented Oct 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FrankYang0529 commented Oct 18, 2022

Uh oh!

vdice commented Oct 18, 2022

Uh oh!

FrankYang0529 commented Nov 9, 2022

Uh oh!

vdice commented Jan 26, 2023

Uh oh!

itowlson commented Jul 5, 2023

Uh oh!

Uh oh!

FrankYang0529 commented Sep 23, 2022 •

edited

Loading

FrankYang0529 commented Sep 27, 2022 •

edited

Loading

vdice commented Oct 18, 2022 •

edited

Loading