v1.1.5

What's Changed

• change to orchestrator endoint file to allow cross origin comms by @XandraMcC in #2998
• Improve bid strategy error messages by @simonwo in #3006
• Show node ranking failures in the CLI (again) by @simonwo in #3008
• Fix Python linting and ensure packages are built on PRs. by @simonwo in #3007
• Bugfix: invalid wasm memory request by @Nasfame in #2992
• refactor: no longer write a config file by default by @frrist in #3009

Full Changelog: v1.1.4...v1.1.5

v1.1.5-rc1

What's Changed

• change to orchestrator endoint file to allow cross origin comms by @XandraMcC in #2998
• Improve bid strategy error messages by @simonwo in #3006
• Show node ranking failures in the CLI (again) by @simonwo in #3008
• Fix Python linting and ensure packages are built on PRs. by @simonwo in #3007
• Bugfix: invalid wasm memory request by @Nasfame in #2992
• refactor: no longer write a config file by default by @frrist in #3009

Full Changelog: v1.1.4...v1.1.5-rc1

v1.1.4

What's Changed

• allow describing nodes with their short IDs by @wdbaruni in #2986
• support shortIDs for GetHistory and GetExecutions by @wdbaruni in #2990
• fixes: image manifest cache by @Nasfame in #2993
• Choose better log stream addresses internally by @rossjones in #2996
• enable downloading s3 results by @wdbaruni in #2995
• change prod bootstrapping peers by @wdbaruni in #2999

Full Changelog: v1.1.3...v1.1.4

v1.1.4-rc2

What's Changed

• allow describing nodes with their short IDs by @wdbaruni in #2986
• support shortIDs for GetHistory and GetExecutions by @wdbaruni in #2990
• fixes: image manifest cache by @Nasfame in #2993
• Choose better log stream addresses internally by @rossjones in #2996
• enable downloading s3 results by @wdbaruni in #2995
• change prod bootstrapping peers by @wdbaruni in #2999

Full Changelog: v1.1.3...v1.1.4-rc2

v1.1.3

What's Changed

• Fix auto-adding GPU labels to nodes. by @simonwo in #2951
• UserID to updates,create/set userID,rename userID by @js-ts in #2943
• Increase the default job resource limits by @rossjones in #2956
• Provide option to turn on HTTPS for SDK by @rossjones in #2853
• Fix: return a reason when a node rejects a job due to capacity. by @frrist in #2964
• Fix bug in date formatting during test assertion by @rossjones in #2975
• Adds SourcePath in conversion to legacy spec by @rossjones in #2974
• Add job Running state by @wdbaruni in #2977
• Don't wait for long running jobs by @wdbaruni in #2978
• Fix only one job is restarted during startup by @wdbaruni in #2979
• Middleware change to enable CORS by @XandraMcC in #2976

New Contributors

• @XandraMcC made their first contribution in #2976

Full Changelog: v1.1.2...v1.1.3

v1.1.3-rc1

What's Changed

• Fix auto-adding GPU labels to nodes. by @simonwo in #2951
• userID to updates,create/set userID,rename userID by @js-ts in #2943
• Increase the default job resource limits by @rossjones in #2956
• Provide option to turn on HTTPS for SDK by @rossjones in #2853
• fix: return a reason when a node rejects a job due to capacity. by @frrist in #2964
• Fix bug in date formatting during test assertion by @rossjones in #2975
• Adds SourcePath in conversion to legacy spec by @rossjones in #2974
• add job Running state by @wdbaruni in #2977
• don't wait for long running jobs by @wdbaruni in #2978
• fix only one job is restarted during startup by @wdbaruni in #2979
• Middleware change to add CORS support by @XandraMcC in #2976

New Contributors

• @XandraMcC made their first contribution in #2976

Full Changelog: v1.1.2...v1.1.3-rc1

v1.1.2

⚠️ Breaking Change

The CLI flag --max-timeout for the bacalhau serve command has been renamed to --max-job-execution-timeout. This new flag more accurately describes its functionality: setting the maximum job execution timeout that the node can accept. If you have any automation or scripts that use bacalhau serve, please update them to use this new flag name.

Main Changes

• Enforce Normalization in Marshaller Implementations by @wdbaruni in #2890
• Flyte integration by @enricorotundo in #2761
• whitelist: python package manager by @Nasfame in #2875
• Use non-p2p multiaddress for peer by @rossjones in #2897
• Run update check when any client command is executed by @simonwo in #2909
• fix migration of new jobs submitted with local directory by @wdbaruni in #2900
• Fix time/date substitutions for S3 urls by @rossjones in #2922
• Limit update checks to once per day by @simonwo in #2918
• Changes the Publisher interface to take an Execution rather than an ExecutionID by @rossjones in #2924
• fix job selection policy flag by @wdbaruni in #2936
• Add update checking to Bacalhau nodes by @simonwo in #2939
• fix: allow compute timeouts to be configured by @frrist in #2931

New Contributors

• @criadoperez made their first contribution in #2881
• @Nasfame made their first contribution in #2875

Full Changelog: v1.1.1...v1.1.2

v1.1.1

What's Changed

• release v1.1.0 by @wdbaruni in #2857
• Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.11.83 to 1.11.86 by @dependabot in #2859
• Convert job spec from lowercase to PascalCase by @wdbaruni in #2856
• ENG-167 - Flaky test: Allow different target states by @rossjones in #2863
• Fix builds for OSX by @rossjones in #2823
• Bump github.com/ipfs/go-unixfsnode from 1.7.4 to 1.9.0 by @dependabot in #2862
• Bump github.com/jedib0t/go-pretty/v6 from 6.4.6 to 6.4.7 by @dependabot in #2867
• [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in #2869
• Remove unused schema site by @rossjones in #2860
• Bump github.com/ipld/go-car/v2 from 2.11.0 to 2.13.1 by @dependabot in #2871
• Simple Version Check and Update Notification by @js-ts in #2832
• Clarify existence of --wait=false by @rossjones in #2873
• Bump k8s.io/kubectl from 0.28.1 to 0.28.2 by @dependabot in #2874
• Remove tracing mutex by @rossjones in #2872
• Bump github.com/invopop/jsonschema from 0.9.0 to 0.11.0 by @dependabot in #2876
• Ensure GPU tools are present in Bacalhau GPU image by @simonwo in #2880
• use title-case for version spec by @wdbaruni in #2883
• make name optional in job spec and use jobID by default by @wdbaruni in #2884

Full Changelog: v1.1.0...v1.1.1

v1.1.0

v1.1.0 Release Notes

📢 Introducing Bacalhau v1.1.0 - Unleash the Power!

We are thrilled to announce the release of Bacalhau v1.1.0, a significant milestone in our quest for unparalleled computing capabilities. Packed with exciting new features like Full Fleet Targeting, Configurable Compute Timeouts, persistent storage, integration with private data swarms and API TLS support, this release is sure to take your computational experience to new heights! 🚀

But that's not all! We invite you to explore the experimental features of this release, such as Long-Running Jobs, as we continue to push the boundaries of computational possibilities.

So, what are you waiting for? Upgrade to Bacalhau v1.1.0 and unlock a world of infinite possibilities in distributed computing! 🌟

curl https://get.bacalhau.org/install.sh | bash

New features

Full Fleet Targeting

Jobs can now target all nodes in a network simultaneously, allowing for more efficient and parallel operations jobs that need to query or modify an entire fleet.

Full fleet jobs are perfect for fleet management, allowing an operator to quickly understand the state of all of their nodes at once with a single command.

Full fleet jobs will only succeed if all known nodes in a network can be reached and can execute the job successfully. Jobs can still be targeted at a subset of the fleet by using labels or resource requirements.

Pass the --target=all parameter to any Bacalhau job command or set Deal.TargetAll: true in an existing Bacalhau job spec.

New node CLI and APIs

New CLI and APIs have been introduced allowing users to easily list nodes in a network and see what compute resources that are available.

Use the new command bacalhau node list to get a tabular output of all known nodes:

You can then use bacalhau node describe to get in-depth output about a specific node.

Configurable Timeouts

Jobs can now last for days or weeks, enabling the execution of big computations that require longer processing times.

By default, compute nodes now do not enforce an execution timeout and jobs default to the longest allowed timeout. Job submitters can still request a timeout using the --timeout flag or the Timeout field in their job spec.

Node operators can still choose to limit the maximum timeout allowed by passing the --max-timeout flag to the serve command or by specifying the new Node.Compute.Capacity.JobTimeouts.MaxJobExecutionTimeout property in their config file.

Richer Node Configuration

We're excited to unveil enhanced configuration options in Bacalhau v1.1.0! With a heightened focus on flexibility, we've expanded the ways you can configure Bacalhau, whether it be via a configuration file, command-line flags, or environment variables.

The new release introduces a persistent configuration file that provides more flexibility and control over node configurations. Read the documentation for how to get started with configuration files.

Key Changes from v1.0.3 to v1.1.0:

• The enriched config.yaml now has a trove of default configuration values, an improvement from the empty version in v1.0.3.
• Event and Libp2p tracing is no longer activated by default. Enable this by specifying paths for via EventTracerPath and Libp2PTracerPath in config.yaml.
• The node’s private key is no longer called**private_key.1235** and is named **libp2p_private_key**by default. Configure its path with Libp2PKeyPath in config.yaml.
• user_id.pem remains consistent. Direct its location using KeyPath in config.yaml.
• Directory name has changed from execution-state-<NODE_ID> to <NODE_ID>-compute, and now, apart from jobStats.json, it also includes executions.db using BoltDB when using persistent storage mode. Define its path using ExecutionStore.Path in config.yaml.
• New directories include <NODE_ID>-requester (stores the state for the requester node using BoltDB), executor_storages (hosts data for Bacalhau storage types), and plugins (houses executor plugin binaries). Configure their paths respectively via JobStore.Path, ComputeStoragePath, and ExecutorPluginPath in config.yaml.

⚠️Note: there are optional migration steps for existing Bacalhau users who want to keep their previous configuration. See the end of this note for how to migrate.

Support for TLS on public APIs

TLS certificates for serving client-facing APIs are now supported, ensuring secure and encrypted communication between Bacalhau clients and requester nodes.

To use a TLS certificate to encrypt communication, you can:

• Configure automatic certificates from Let’s Encrypt by passing --autocert=<your-hostname> and ensuring the Bacalhau binary can respond to challenges by running sudo setcap CAP_NET_BIND_SERVICE+ep $(which bacalhau).
• Pass a certificate to --tlscert and the corresponding private key to --tlskey.

By default, if none of the above options are used, the server will continue to serve its API endpoints over HTTP.

Persistent Storage of Jobs and Executions

Compute and requester nodes now support persistent storage, ensuring data integrity and allowing for long-term job and execution audit records. This feature is now switched on by default and records are persisted to the Bacalhau repository.

See the documentation for how to configure persistence.

Improved Error Messages

Clearer error messages are now displayed when no node is available to run a job, making troubleshooting easier and more efficient.

Instead of receiving ‘not enough nodes to run the job’, users will now get more specific help messages, such as ‘Docker image does not exist or repo is inaccessible’ or ‘job timeout exceeds the maximum allowed’.

Fine-Grained Control Over Image Entrypoint and Parameters

Users now have finer control over the entrypoint and parameters passed to a Docker image. Previously, Bacalhau would ignore the default entrypoint to the image and replace it with the first argument after bacalhau docker run <image>. Now, the default entrypoint in the image is used and all of the positional arguments are passed as the command to that entrypoint.

The entrypoint can still be explicitly overriden by using the --entrypoint flag or by setting the Entrypoint field in a Docker job spec.

GPU Support Inside Docker Containers

Bacalhau now has the capability to automatically utilize GPUs when the Bacalhau node is running inside a Docker container. Ensure that the Bacalhau node is started with a GPU capability by passing --gpus=all to docker run, and Bacalhau nodes will automatically detect GPUs running on the host machine.

Submit a job to a node running inside Docker using bacalhau docker run --gpu=1 to run the job in a new GPU-enabled container on the host.

Support for Private IPFS Clusters

Integration with private IPFS clusters has been added, providing enhanced security and control over data storage and retrieval.

To connect to a private swarm, pass the path to a swarm key to --ipfs-swarm-key, set the BACALHAU_IPFS_SWARM_KEY environment variable or configure the Node.IPFS.SwarmKeyPath configuration property.

When connecting to a private swarm, Bacalhau will no longer bootstrap using or connect to public peers and will rely on the swarm for all data retrieval.

These steps are also necessary on clients who use bacalhau get to download from a private IPFS swarm.

Note that these steps are not necessary if using the --ipfs-connect flag, which already can connect to IPFS nodes running a private swarm.

New Experimental Features

All of these features are experimental, meaning that their APIs are liable to change in an upcoming release. You are encouraged to try out these features and provide feedback or bug reports on Bacalhau Slack.

Long-Running Jobs

Bacalhau jobs can now run indefinitely and will automatically restart when nodes come back online, allowing for continuous and uninterrupted processing.

Long-running jobs allow compute workloads to process data that arrives continuously, and is perfect for tasks such as pre-filtering logs, processing real-time analytics, or working with edge sensors.

With the introduction of long-running jobs, ML inference tasks can now operate in a "warm-boot" environment. This means that the necessary resources and dependencies are already loaded, significantly reducing the time taken to run an inference job.

With this experimental feature, you can now unleash the power of Bacalhau to handle dynamic and ever-changing data streams, ensuring continuous and uninterrupted processing of your computational workloads.

Deprecated Features

Estuary

The Estuary publisher is no longer supported in this release. Compute nodes will now reject jobs that require the Estuary publisher.

Verification

The Verifiers feature is no longer supported in this release. Compute nodes will silently ignore verification requirements on jobs.

⚠️ Migration steps

Users who wish to continue using their previous Bacalhau private key or their previous Bacalhau Client ID as their identity will need to either:

• Rename private_key.1235 to libp2p_private_key
• Modify the config.yaml to use the previous key by editing the value of Libp2PKeyPath to point to its path.

Up Next

These upcoming features aim to provide users with increased flexibility and convenience in their computational workflows while maintaining a ...

v1.0.3

What's Changed

• release v1.0.0 by @wdbaruni in #2458
• Bump markdown-it-py from 2.1.0 to 2.2.0 in /python by @dependabot in #2454
• Bump github.com/ipfs/go-cid from 0.3.2 to 0.4.1 by @dependabot in #2457
• Bump github.com/docker/distribution from 2.8.1+incompatible to 2.8.2+incompatible by @dependabot in #2462
• Bump github.com/ipfs/kubo from 0.18.2-0.20230214211946-0ff406170dd9 to 0.19.0 by @dependabot in #2460
• Bump github.com/cloudflare/circl from 1.1.0 to 1.3.3 in /ops/aws/canary/lambda by @dependabot in #2464
• Bump github.com/docker/distribution from 2.8.1+incompatible to 2.8.2+incompatible in /ops/aws/canary/lambda by @dependabot in #2465
• Bump github.com/ipfs/kubo from 0.18.2-0.20230214211946-0ff406170dd9 to 0.19.0 in /ops/aws/canary/lambda by @dependabot in #2461
• Bump github.com/cloudflare/circl from 1.1.0 to 1.3.3 by @dependabot in #2463
• Strip debug symbols from wasm builds by @rossjones in #2471
• Skip S3 tests if no credentials are present by @rossjones in #2473
• Fix examples for 1.0.0 Python SDK by @rossjones in #2474
• Bump github.com/ipfs/go-unixfsnode from 1.5.2 to 1.7.0 by @dependabot in #2477
• Insulated jobs: post-job results moderation by @simonwo in #2468
• [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in #2478
• Storage cleanup of inputs by @rossjones in #2449
• Resolve docker digests from supplied image ids by @rossjones in #2410
• Print errors if no nodes complete the job successfully by @simonwo in #2487
• Bump requests from 2.28.2 to 2.31.0 in /python by @dependabot in #2490
• Bump github.com/lib/pq from 1.10.7 to 1.10.9 by @dependabot in #2488
• Bump requests from 2.28.1 to 2.31.0 in /ops by @dependabot in #2489
• Bump github.com/go-git/go-git/v5 from 5.6.1 to 5.7.0 by @dependabot in #2493
• Add docker secrets to terraform by @rossjones in #2417
• Record Canary Jobs in Postgres Database by @OwainJones in #2494
• chore: remove python and wasm_python executors by @frrist in #2373
• [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in #2495
• Cache docker manifests for compute node bidding by @rossjones in #2491
• Extend deadline for serve tests by @rossjones in #2497
• migrate to expanso's grafana by @wdbaruni in #2501
• Bump k8s.io/apimachinery from 0.27.0 to 0.27.2 by @dependabot in #2496
• enable mounting writable local paths by @wdbaruni in #2499
• update to libp2p v0.27.4 by @wdbaruni in #2506
• Bump github.com/multiformats/go-multihash from 0.2.1 to 0.2.2 by @dependabot in #2509
• [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in #2510
• Disable requester node docker caching until we can address issues by @rossjones in #2511
• 2482 output peering information to run by @aronchick in #2485
• Revert removal of python and wasm_python executors by @frrist in #2513
• Bump github.com/samber/lo from 1.36.0 to 1.38.1 by @dependabot in #2515
• [circleci] Build swagger reference - this is an automatic commit [skip ci] by @enricorotundo in #2517

New Contributors

• @OwainJones made their first contribution in #2494

Full Changelog: v1.0.0...v1.0.3