Skip to content

Commit c80aea3

Browse files
arpjwclaude
andcommitted
phase 1: positioning and cuts complete
- SWE-bench harness at benchmarks/swe_bench/ with cost estimate gate and concurrency control - Removed Web IDE: deleted ide.py route, IDESession model, fly_machines service, frontend /ide pages; CSP headers cleaned - Mothballed marketplace: all 4 routes return HTTP 410; CLI commands show paused message; footer link removed - Stripped voice mode: deleted voice.py, removed voice_mode from NimbusREPL and CLI - Fixed get_or_create_queue import errors in automation_engine, pipeline_executor, slack/linear handlers -- these functions were removed when ws.py moved to Redis pub/sub - CLI version now reads from package metadata instead of hardcoded string - README rewritten: benchmarks table, comparison table, known limitations (Redis/ARQ, reconciliation) - BYOK key wiring, sandboxed verification, skill injection scanning, cross-provider review (from phase 3 hardening) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 8de6dd2 commit c80aea3

32 files changed

Lines changed: 657 additions & 1952 deletions

.github/workflows/swe-bench.yml

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: SWE-bench Verified
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
tasks:
7+
description: "Number of tasks (integer or 'all')"
8+
required: true
9+
default: "10"
10+
concurrency:
11+
description: "Max parallel tasks"
12+
required: false
13+
default: "3"
14+
run_id:
15+
description: "Optional run identifier"
16+
required: false
17+
18+
jobs:
19+
swe-bench:
20+
runs-on: ubuntu-latest
21+
timeout-minutes: 360
22+
23+
steps:
24+
- uses: actions/checkout@v4
25+
26+
- uses: actions/setup-python@v5
27+
with:
28+
python-version: "3.12"
29+
cache: pip
30+
31+
- name: Install dependencies
32+
working-directory: backend
33+
run: |
34+
pip install -r requirements.txt -e .
35+
pip install datasets swebench
36+
37+
- name: Run harness
38+
working-directory: backend
39+
env:
40+
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
41+
VOYAGE_API_KEY: ${{ secrets.VOYAGE_API_KEY }}
42+
REQUIRE_API_KEY: "false"
43+
run: |
44+
python -m benchmarks.swe_bench.harness \
45+
--tasks ${{ github.event.inputs.tasks }} \
46+
--confirm-cost \
47+
--yes \
48+
--concurrency ${{ github.event.inputs.concurrency || '3' }} \
49+
${{ github.event.inputs.run_id && format('--run-id {0}', github.event.inputs.run_id) || '' }}
50+
51+
- name: Upload results
52+
uses: actions/upload-artifact@v4
53+
if: always()
54+
with:
55+
name: swe-bench-results-${{ github.run_id }}
56+
path: backend/benchmarks/swe_bench/results/
57+
retention-days: 90

0 commit comments

Comments
 (0)