CI build mode for WASM runtime artifact

[quasiyoke]

1. In the 1st commit, I disable all CI jobs except those necessary for demonstrating the new artifact type.
2. In the 2nd commit, the new artifact is added.

You can also review the draft release: https://github.com/humanode-network/humanode/releases/tag/untagged-c97d2543b7a7af640755

I've tested releases as well 😁

[quasiyoke]

I believe we can remove cfg...feature here to include substrate_wasm_builder by default in no-std:

humanode/crates/humanode-runtime/build.rs

Line 8 in adab1b4

#[cfg(feature = "std")]

— and then we will be able to build WASM using the following command:

cargo build --locked --workspace --release --package humanode-runtime --no-default-features --features evm-tracing --target wasm32-unknown-unknown

This would reduce the number of dependency crates to build from the current 1270 to just 449. The only thing is, I haven't tested it yet to see if it'll break anything else.

[MOZGIII]

Looks good overall! Small nits, but this is definitely on the right track

[MOZGIII]

I believe we can remove cfg...feature here to include substrate_wasm_builder by default in no-std:

humanode/crates/humanode-runtime/build.rs

Line 8 in adab1b4

#[cfg(feature = "std")]

— and then we will be able to build WASM using the following command:

cargo build --locked --workspace --release --package humanode-runtime --no-default-features --features evm-tracing --target wasm32-unknown-unknown

This would reduce the number of dependency crates to build from the current 1270 to just 449. The only thing is, I haven't tested it yet to see if it'll break anything else.

If we remove it it will trigger a wasm build while doing a wasm build - an infinite loop

[MOZGIII]

What build command should work just fine though
cargo build --locked --workspace --release --package humanode-runtime --no-default-features --features evm-tracing --target wasm32-unknown-unknown

Except it doesn't compress the built artifact

[MOZGIII]

After some research, I deem the way of building the runtime via the WasmBuilder preferable; at present that would mean building via build.rs script. It we could avoid building the dependencies of the humanode-runtime before the build script it executed for the usual std mode - then we will have an advantage of not having to pay the price for building them twice (build script will only build the dependencies once, for wasm, and here are the savings)...

[dmitrylavrenov]

Clear on my side.

Not sure, that I'm in context having a lot of commented code. Is it fully ready for merging or just some kind of checkpoint ?

[quasiyoke]

@MOZGIII

I deem the way of building the runtime via the WasmBuilder preferable; at present that would mean building via build.rs script.

I noticed that Polkadot itself sometime uses wasm-builder from helper scripts. I also believe it's easier to build from build.rs, especially since that was the method envisioned in their template, and we don’t need any specific customization yet.

If we could avoid building the dependencies of the humanode-runtime before the build script it executed for the usual std mode - then we will have an advantage of not having to pay the price for building them twice (build script will only build the dependencies once, for wasm, and here are the savings)...

I don’t yet know how to do this.

[quasiyoke]

@dmitrylavrenov

[what about] having a lot of commented code. Is it fully ready for merging or just some kind of checkpoint?

We agreed with Mozgiii right away that launching CI here would require several attempts, and that running too many jobs would consume a significant portion of our CI budget. Therefore, in the first commit, I disabled all CI jobs except those necessary for demonstrating the new artifact type. I think we should first ensure that there are no comments on the Draft PR, and then I'll remove this commit in the final PR.

[MOZGIII]

Would be nice to have the syntax and formatting unified; I recommend using Bash IDE and shellcheck extensions

[quasiyoke]

shellcheck — I had it installed,
BashIDE — I installed it, but it didn't offer any formatting corrections…

[quasiyoke]

I removed the CLI arguments --package humanode-runtime --lib – the number of crates during compilation remained at 1270. The included std feature likely prevented limiting the scope of compiled packages. It turns out --package only posed a potential risk of breaking features without providing any performance benefits.

I will still see what can be done to avoid building so many crates.

[MOZGIII]

Maybe we could just structure the builds in such a way that we also publish a whole node with tracing support? I.e. so that the node would, like, also have the embedded tracing runtime initially, and the tracing variant of the on-chain runtime (code).

[quasiyoke]

An experiment confirmed that the cargo flags --package and --workspace work exactly as @MOZGIII initially explained:
--package builds the selected package with the minimal possible set of features.
--bin builds the selected binary with a set of features for a complete workspace build.
--workspace builds the entire workspace in any case.
— these options work this way both in 2021 and 2024 edition, with resolver 2 and 3 respectively.

Added Cargo target --bin humanode-runtime-build-selector — the number of crates necessary for building is 673, compared to 1270 when using --workspace.

also publish a whole node with tracing support

I think a separate WASM artifact is more flexible to use. If it’s not needed, I can add node with EVM-tracing here. If another artifact (EVM-tracing node) is required, perhaps it would be best to create it in a separate task?

[MOZGIII]

I think it, like this, will still build all the dependencies for the host triple before the actual runtime is built (for the wasm triple).

If we, instead, make another crate, and have the contents of the build.rs script in there (with slight changes to target a different crate) - we could essentially directly trigger the runtime build for wasm from a separate package - as opposed to doing it from the build script.

[MOZGIII]

If we, instead, make another crate, and have the contents of the build.rs script in there (with slight changes to target a different crate) - we could essentially directly trigger the runtime build for wasm from a separate package - as opposed to doing it from the build script.

Tested this with 842c319 - the build script fails (substrate-wasm-builder panics at https://github.com/paritytech/substrate/blob/033d4e86cc7eff0066cd376b9375f815761d653c/utils/wasm-builder/src/wasm_project.rs#L837); the other variation where the build script it the bin itself also didn't work - but for other reason https://github.com/paritytech/substrate/blob/033d4e86cc7eff0066cd376b9375f815761d653c/utils/wasm-builder/src/builder.rs#L163.