Skip to content

[Bug] valid_solved_count increments on same-account solves, bypassing MIN_VALID_SOLVED_ISSUES eligibility gate #1255

@web-dev0521

Description

@web-dev0521

Description

In _score_miner_mirror_issues (gittensor/validator/issue_discovery/mirror_scan.py:301-337), valid_solved_count is incremented before the same-account check (issue.author_github_id == solving_pr.author_github_id) is applied. As a result, a miner who files an issue and solves it with their own PR has that issue counted toward valid_solved_count, which is the counter used by check_issue_eligibility against MIN_VALID_SOLVED_ISSUES (7).

The same-account gate at line 332 correctly suppresses the discovery score for that issue ("credibility only"), but does not suppress its contribution to the eligibility counter. A miner can therefore use ≥7 self-filed + self-solved issues to bootstrap past the issue-discovery eligibility gate, unlocking the issue-discovery emission pool (10% of total emissions) for subsequent legitimate third-party discoveries — without ever earning discovery score from a real cross-author discovery.

Steps to Reproduce

  1. Run the validator with LOG_LEVEL=debug for at least one complete scoring round on the mirror path.
  2. Grep validator output for the same-account debug line emitted at mirror_scan.py:333-335:
    grep "same-account (discoverer == solver" validator.log
    
  3. For each UID that emitted the line, cross-reference the per-UID summary line at mirror_scan.py:411-415:
    grep "UID <N>: .* solved (.* valid)" validator.log
    
  4. Observe that the (N valid) figure includes the same-account solves that the same round logged as "credibility only".
  5. If is_issue_eligible=True AND the same-account count alone could push valid_solved_count over MIN_VALID_SOLVED_ISSUES = 7, the gate has been bypassed by self-loops.

Expected Behavior

Self-account solves (issue.author_github_id == solving_pr.author_github_id) should not increment valid_solved_count. The eligibility gate MIN_VALID_SOLVED_ISSUES is intended to signal that a miner has demonstrated discovery activity — closing a self-filed issue with one's own PR is not discovery and is already excluded from the discovery score on that basis.

After the fix, a miner with N self-account solves and zero cross-author solves should be is_issue_eligible=False with reason = "N/7 valid solved PRs (need 7)", where N counts only cross-author qualifying solves.

Actual Behavior

Order of operations in mirror_scan.py:305-332:

solved_count += 1                                     # counts self-loop

cached = await _resolve_solving_pr_score(...)
if cached is None:
    continue

if cached.token_score >= MIN_TOKEN_SCORE_FOR_BASE_SCORE:
    valid_solved_count += 1                           # counts self-loop

if issue.author_github_id == solving_pr.author_github_id:
    bt.logging.debug(... "credibility only")
    continue                                          # only blocks score

valid_solved_count is incremented before the same-account continue. The subsequent check_issue_eligibility(solved_count, valid_solved_count, closed_count) call at mirror_scan.py:380 therefore counts same-account solves toward the 7-minimum gate.

Concrete consequence: a miner with 7 self-filed-and-self-solved issues (each with a solving PR whose token_score >= 5) and zero cross-author solves passes is_issue_eligible=True despite having performed no discovery activity, and will earn discovery score on any subsequent cross-author discovery within the same lookback window.

Note: the in-code comment at mirror_scan.py:331 reads "Same-account: discoverer == solver gets credibility only, no score" — the "no score" contract is preserved (the issue is not added to scored_issues), but the implicit contract that same-account also doesn't help the miner qualify for the pool is silently broken by the counter ordering.

Environment

  • OS: Linux 6.17.0-23-generic
  • Python version: 3.12
  • Commit/Version: 3724d0d (v5.0.0)

Additional Context

Scope of the fix. Move two lines so the same-account check runs before the counters:

# classification == 'solved'
solving_pr = issue.solving_pr

cached = await _resolve_solving_pr_score(...)
if cached is None:
    bt.logging.debug(... "fetch failed — credibility only")
    continue

if issue.author_github_id == solving_pr.author_github_id:
    bt.logging.debug(... "credibility only")
    continue

solved_count += 1
if cached.token_score >= MIN_TOKEN_SCORE_FOR_BASE_SCORE:
    valid_solved_count += 1

Affected file count: 1 (gittensor/validator/issue_discovery/mirror_scan.py) plus one test in tests/validator/issue_discovery/. No protocol change, no new abstraction, no change to the legacy path (already removed for issue discovery).

Behavioral contract change. Same-account solves no longer contribute to solved_count either, so issue_credibility = solved / (solved + adjusted_closed) is unaffected by self-loops in both directions. This matches the eligibility gate's intent: self-loops are inert to the discovery pool entirely. The "credibility only" log line should be updated to "no scoring credit" to keep the docstring honest.

Test shape. The test asserts: with a MinerEvaluation whose mirror response contains exactly 7 issues where author_github_id == solving_pr.author_github_id and solving_pr.token_score = 25, _score_miner_mirror_issues produces evaluation.is_issue_eligible == False and evaluation.issue_discovery_score == 0.0. A second test asserts that 7 cross-author solves under the same conditions produce is_issue_eligible == True (i.e. the fix only removes the bypass, doesn't break the legitimate path).

Why this is one concrete vector, not a parity argument. Prior closed-not_planned issues #1049 and #1089 raised parity / data-correctness concerns about the same-account check itself (which field, which value). This issue is different: the check works correctly, but its result is applied after the counter that depends on it. The bug is a control-flow ordering bug, not a field-comparison bug.

Not a duplicate of: PR #1245 / Issue #1244 (stale issue_discovery_score on cache-restored ineligibility — different code path, different field). PR #1247 / #1246 / Issue #1243 (cross-repo Closes refs — different anti-gaming vector). PR #1173 / #1176 / Issue #1172 (fixed_base_score override on cache-miss — unrelated subsystem).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions