Skip to content

Multi-bet queue can strand fresh bets and stall mech requests #907

Description

@OjusWiZard

Summary

In multi-bet mode, agents can keep cycling normally but stop producing new mech requests because sampling_round repeatedly exits with Event.NONE after logging There were no unprocessed bets available to sample from!.

The strongest evidence points to a queue-liveness issue between market_manager_abci and decision_maker_abci, not a mech-tool selection or staking stop-trading problem.

Observed behavior

Healthy agents show the expected progression:

  • Available bets to sample from: 16
  • Selected the mech tool 'superforcaster'
  • Calling method get_request_data ...

Failing agents repeatedly show:

  • Computed stop_trading=False
  • Entered in the 'sampling_round'
  • There were no unprocessed bets available to sample from!
  • sampling_round ... Event.NONE
  • policy can still report a best tool, but the flow never reaches mech request creation

This was reproduced across multiple operator datasets, including another user's snapshot (peanut), which still favored superforcaster historically but hit the same sampling failure.

Why this looks like a trader queue-liveness bug

Upstream: fresh promotion in multi-bet mode is all-or-nothing

In packages/valory/skills/market_manager_abci/behaviours/update_bets.py:

all_bets_fresh = all(
    bet.queue_status.is_fresh()
    for bet in self.bets
    if not bet.queue_status.is_expired()
)

if all_bets_fresh:
    for bet in self.bets:
        bet.queue_status = bet.queue_status.move_to_process()

This means that if the non-expired queue becomes mixed (FRESH + PROCESSED / REPROCESSED), newly fresh bets are not promoted to TO_PROCESS.

Downstream: sampling excludes fresh bets

In packages/valory/skills/decision_maker_abci/behaviours/sampling.py:

processable_statuses = {
    QueueStatus.TO_PROCESS,
    QueueStatus.PROCESSED,
    QueueStatus.REPROCESSED,
}

FRESH bets are not processable. So once the queue is mixed, new bets can remain stranded while mature bets age out or get blacklisted.

Additional depletion pressure

Both upstream and downstream can permanently expire bets:

  • update_bets.py::_blacklist_expired_bets()
  • sampling.py::processable_bet() via blacklist_forever() when within_safe_range is false

That makes the workflow biased toward depletion, while requeue paths are conditional (checkpoint reached, sell-review path).

Likely failure mode

  1. Bets progress normally: FRESH -> TO_PROCESS -> PROCESSED -> REPROCESSED
  2. New bets arrive as FRESH
  3. Existing mature bets remain PROCESSED / REPROCESSED
  4. Because not all non-expired bets are fresh, _bet_freshness_check_and_update() does nothing
  5. Fresh bets stay FRESH
  6. Sampling ignores FRESH
  7. Mature bets age out / get blacklisted
  8. sampling_round returns Event.NONE
  9. Mech request count plateaus even though the service keeps cycling

Existing test evidence

There is already upstream test coverage proving the all-or-nothing promotion behavior:

  • packages/valory/skills/market_manager_abci/tests/test_behaviours_update_bets.py
  • test_multi_bets_not_all_fresh

I also added a downstream regression test that passes and shows the sampling-side manifestation:

  • packages/valory/skills/decision_maker_abci/tests/behaviours/test_sampling.py
  • test_sample_excludes_fresh_bets_left_unpromoted_in_mixed_queue

That test shows a FRESH bet can remain safely eligible by time, yet _sample() still returns None because the only mature candidate is blacklisted and fresh bets are excluded.

Suggested fix direction

Please evaluate whether multi-bet queue promotion is too strict. In particular, consider whether fresh bets should be promoted independently instead of requiring all non-expired bets to be fresh.

At minimum, it would also help to add sampling diagnostics before returning no bets, e.g. counts by:

  • expired
  • fresh
  • to_process
  • processed
  • reprocessed
  • rejected by opening range
  • rejected by safe range
  • rejected by selling restriction

Relevant files

  • packages/valory/skills/market_manager_abci/behaviours/update_bets.py
  • packages/valory/skills/decision_maker_abci/behaviours/sampling.py
  • packages/valory/skills/market_manager_abci/bets.py
  • packages/valory/skills/trader_abci/composition.py

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions