v0.19.0

⚠️ Version 0.19.0 has minor breaking changes for the Worker.Middleware, introduced fairly recently in 0.17.0 that has a worker's Middleware function now taking a non-generic JobRow parameter instead of a generic Job[T]. We tried not to make this change, but found the existing middleware interface insufficient to provide the necessary range of functionality we wanted, and this is a secondary middleware facility that won't be in use for many users, so it seemed worthwhile.

Added

• Added a new "hooks" API for tying into River functionality at various points like job inserts or working. Differs from middleware in that it doesn't go on the stack and can't modify context, but in some cases is able to run at a more granular level (e.g. for each job insert rather than each batch of inserts). PR #789.
• river.Config has a generic Middleware setting that can be used as a convenient way to configure middlewares that implement multiple middleware interfaces (e.g. JobInsertMiddleware and WorkerMiddleware). Use of this setting is preferred over Config.JobInsertMiddleware and Config.WorkerMiddleware, which have been deprecated. PR #804.

Changed

• The river.RecordOutput function now returns an error if the output is too large. The output is limited to 32MB in size. PR #782.
• Breaking change: The Worker interface's Middleware function now takes a JobRow parameter instead of a generic Job[T]. This was necessary to expand the potential of what middleware can do: by letting the executor extract a middleware stack from a worker before a job is fully unmarshaled, the middleware can also participate in the unmarshaling process. PR #783.
• JobList has been reimplemented to use sqlc. PR #795.

v0.18.0

⚠️ Version 0.18.0 has breaking changes for the rivertest.Worker type that was just introduced. While attempting to round out some edge cases with its design, we realized some of them simply couldn't be solved adequately without changing the overall design such that all tested jobs are inserted into the database. Given the short duration since it was released (over a weekend) it's unlikely many users have adopted it and it seemed best to rip off the bandaid to fix it before it gets widely used.

Added

• Jobs can now store a recorded "output" value, a JSON-encoded payload set by the job during execution and stored in the job's metadata. The river.RecordOutput function makes it easy to use the job row to store transient/temporary values that are needed for introspection or for other downstream jobs. The output can be accessed using the JobRow.Output() helper method.

  This output is stored at the same time as the job is completed following execution, so it does not require additional database calls or overhead. Output can be anything that can be stored in a Postgres JSONB field, though for performance reasons it should be limited in size. PR #758.

Changed

• Breaking change: The rivertest.Worker type now requires all jobs to be inserted into the database. The original design allowed workers to be tested without hitting the database at all. Ultimately this design made it hard to correctly simulate features like JobCompleteTx and the other potential solutions seemed undesirable.

  As part of this change, the Work and WorkJob methods now take a transaction argument. The expectation is that a transaction will be opened by the caller and rolled back after test completion. Additionally, the return signature was changed to return a WorkResult struct alongside the error. The struct includes the post-execution job row as well as the event kind that occurred, making it easy to inspect the job's state after execution.

  Finally, the implementation was refactored so that it uses the real river.Client insert path, and also uses the same job execution path as real execution. This minimizes the potential for differences in behavior between testing and real execution.
  PR #766.

• Adjusted panic stack traces to filter out irrelevant frames like the ones generated by the runtime package that constructed the trace, or River's internal rescuing code. This makes the first panic frame reflect the actual panic origin for easier debugging. PR #774.

Fixed

• Fix error message on unsuccessful client subscribe that erroneously referred to "Workers" not configured. PR #771.
• Fix an issue with encoding unique keys in riverdatabasesql driver. PR #777.

v0.17.0

Added

• Exposed TestConfig struct on Config under the Test field for configuration that is specific to test environments. For now, the only field on this type is Time, which can be used to set a synthetic TimeGenerator for tests. A stubbable time generator was added as rivertest.TimeStub to allow time to be easily stubbed in tests. PR #754.
• New rivertest.Worker type to make it significantly easier to test River workers. Either real or synthetic jobs can be worked using this interface, generally without requiring any database interactions. The Worker type provides a realistic execution environment with access to the full range of River features, including river.ClientFromContext, middleware (both global and per-worker), and timeouts. PR #753.

Changed

• Errors returned from retryable jobs are now logged with warning logs instead of error logs. Error logs are still used for jobs that error after reaching max_attempts. PR #743.
• Remove range variable capture in for loops and use simplified range syntax. Each of these requires Go 1.22 or later, which was already our minimum required version since Go 1.23 was released. PR #755.

Fixed

• riverdatabasesql driver: properly handle nil values in bytea[] inputs. This fixes the driver's handling of empty unique keys on insert for non-unique jobs with the newer unique jobs implementation. PR #739.
• JobCompleteTx now returns rivertype.ErrNotFound if the job doesn't exist instead of panicking. PR #753.
• • NeverSchedule.Next now returns the correct maximum time value, ensuring that the periodic job truly never runs. This fixes an issue where an incorrect maximum timestamp was previously used. Thanks Hubert Krauze (@krhubert)! PR #760

v0.16.0

Added

• NeverSchedule returns a PeriodicSchedule that never runs. This can be used to effectively disable the reindexer or any other maintenance service. PR #718.
• Add SkipUnknownJobCheck client config option to skip job arg worker validation. PR #731.

Changed

• The reindexer maintenance process has been enabled. As of now, it will reindex only the river_job_args_index and river_jobs_metadata_index GIN indexes, which are more prone to bloat than b-tree indexes. By default it runs daily at midnight UTC, but can be customized on the river.Config type via ReindexerSchedule. Most installations will benefit from this process, but it can be disabled altogether using NeverSchedule. PR #718.

• Periodic jobs now have a "periodic": true attribute set in their metadata to make them more easily distinguishable from other types of jobs. PR #728.

• Snoozing a job now causes its attempt to be decremented, whereas previously the max_attempts would be incremented. In either case, this avoids allowing a snooze to exhaust a job's retries; however the new behavior also avoids potential issues with wrapping the max_attempts value, and makes it simpler to implement a RetryPolicy based on either attempt or max_attempts. The number of snoozes is also tracked in the job's metadata as snoozes for debugging purposes.

  The implementation of the builtin RetryPolicy implementations is not changed, so this change should not cause any user-facing breakage unless you're relying on attempt - len(errors) for some reason. PR #730.

• ByPeriod uniqueness is now based off a job's ScheduledAt instead of the current time if it has a value. PR #734.

v0.15.0

Added

• The River CLI will now respect the standard set of PG* environment variables like PGHOST, PGPORT, PGDATABASE, PGUSER, PGPASSWORD, and PGSSLMODE to configure a target database when the --database-url parameter is omitted. PR #702.
• Add missing doc for JobRow.UniqueStates + reveal rivertype.UniqueOptsByStateDefault() to provide access to the default set of unique job states. PR #707.

Changed

• Sleep durations are now logged as Go-like duration strings (e.g. "10s") in either text or JSON instead of duration strings in text and nanoseconds in JSON. PR #699.
• Altered the migration comments from river migrate-get to include the "line" of the migration being run (main, or for River Pro workflow and sequence) to make them more distinguishable. PR #703.
• Fewer slice allocations during unique insertions. PR #705.

Fixed

• Exponential backoffs at degenerately high job attempts (>= 310) no longer risk overflowing time.Duration. PR #698.

v0.14.3

Changed

• Dropped internal random generators in favor of math/rand/v2, which will have the effect of making code fully incompatible with Go 1.21 (go.mod has specified a minimum of 1.22 for some time already though). PR #691.

Fixed

• 006 migration now tolerates previous existence of a unique_states column in case it was added separately so that the new index could be raised with CONCURRENTLY. PR #690.

v0.14.2

Fixed

• Cancellation of running jobs relied on a channel that was only being received when in the job fetch routine, meaning that jobs which were cancelled would not be cancelled until the next scheduled fetch. This was fixed by also receiving from the job cancellation channel when in the main producer loop, even if no fetches are happening. PR #678.
• Job insert middleware were not being utilized for periodic jobs. This insertion path has been refactored to rely on the unified insertion path from the client. Fixes #675. PR #679.

v0.14.1

Fixed

• In PR #663 the client was changed to be more aggressive about re-fetching when it had previously fetched a full batch. Unfortunately a clause was missed, which resulted in the client being more aggressive any time even a single job was fetched on the previous attempt. This was corrected with a conditional to ensure it only happens when the last fetch was full. PR #668.

v0.14.0

Added

• Expose JobCancelError and JobSnoozeError types to more easily facilitate testing. PR #665.

Changed

• Tune the client to be more aggressive about fetching when it just fetched a full batch of jobs, or when it skipped its previous triggered fetch because it was already full. This should bring more consistent throughput to poll-only mode and in cases where there is a backlog of existing jobs but new ones aren't being actively inserted. This will result in increased fetch load on many installations, with the benefit of increased throughput. As before, FetchCooldown still limits how frequently these fetches can occur on each client and can be increased to reduce the amount of fetch querying. Thanks Chris Gaffney (@gaffneyc) for the idea, initial implementation, and benchmarks. PR #663.

Fixed

• riverpgxv5 driver: Hijack() the underlying listener connection as soon as it is acquired from the pgxpool.Pool in order to prevent the pool from automatically closing it after it reaches its max age. A max lifetime makes sense in the context of a pool with many conns, but a long-lived listener does not need a max lifetime as long as it can ensure the conn remains healthy. PR #661.

v0.13.0

⚠️ Version 0.13.0 removes the original advisory lock based unique jobs implementation that was deprecated in v0.12.0. See details in the note below or the v0.12.0 release notes.

Added

• A middleware system was added for job insertion and execution, providing the ability to extract shared functionality across workers. Both JobInsertMiddleware and WorkerMiddleware can be configured globally on the Client, and WorkerMiddleware can also be added on a per-worker basis using the new Middleware method on Worker[T]. Middleware can be useful for logging, telemetry, or for building higher level abstractions on top of base River functionality.

  Despite the interface expansion, users should not encounter any breakage if they're embedding the WorkerDefaults type in their workers as recommended. PR #632.

Changed

• Breaking change: The advisory lock unique jobs implementation which was deprecated in v0.12.0 has been removed. Users of that feature should first upgrade to v0.12.1 to ensure they don't see any warning logs about using the deprecated advisory lock uniqueness. The new, faster unique implementation will be used automatically as long as the UniqueOpts.ByState list hasn't been customized to remove required states (pending, scheduled, available, and running). As of this release, customizing ByState without these required states returns an error. PR #614.
• Single job inserts are now unified under the hood to use the InsertMany bulk insert query. This should not be noticeable to users, and the unified code path will make it easier to build new features going forward. PR #614.

Fixed

• Allow river.JobCancel to accept a nil error as input without panicking. PR #634.