Skip to content

feat: Ask Sourcebot #392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 123 commits into from
Jul 23, 2025
Merged
Show file tree
Hide file tree
Changes from 121 commits
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
5e400bd
wip
brendan-kellam May 7, 2025
e4ee716
wip
brendan-kellam Jun 10, 2025
c73b00d
wip: add inline citations
brendan-kellam Jun 10, 2025
fb0e44a
Add read file tool
brendan-kellam Jun 10, 2025
446112b
code block rendering
brendan-kellam Jun 10, 2025
571b55f
Add find all refs / get defs tool
brendan-kellam Jun 11, 2025
c7a771a
Add range field to citations
brendan-kellam Jun 12, 2025
bbf4bec
wip on context system
brendan-kellam Jun 13, 2025
4c1ebae
wip on adding context window panel
brendan-kellam Jun 13, 2025
fe8479e
wip on adding context system
brendan-kellam Jun 14, 2025
36202ee
Add useBrowsePath
brendan-kellam Jun 17, 2025
8534924
try out claude
brendan-kellam Jun 17, 2025
f7ab5f9
clickable links
brendan-kellam Jun 17, 2025
53fcb5a
WIP: playing around with slate to drive @ mentions
brendan-kellam Jun 19, 2025
187a092
experiment: reduce input token usage by altering file search payload …
brendan-kellam Jun 19, 2025
97bbebd
small improvements
brendan-kellam Jun 19, 2025
cea3962
further improvements to @ mention system
brendan-kellam Jun 19, 2025
2b1952f
chat persistance + working on new chat UI
brendan-kellam Jun 19, 2025
828ba99
add input message support
brendan-kellam Jun 20, 2025
aea6c18
switch to openai
brendan-kellam Jun 20, 2025
824c2bf
fix
brendan-kellam Jun 20, 2025
1ab148b
fix
brendan-kellam Jun 20, 2025
b76ca87
Add markdown rendering using marked
brendan-kellam Jun 21, 2025
63b87e8
formatting nits
brendan-kellam Jun 21, 2025
a9bb163
wip on annotations using markdown
brendan-kellam Jun 22, 2025
fd02802
Add line numbers to source to make it easier for LLMs to reference sp…
brendan-kellam Jun 22, 2025
d9facb2
nits
brendan-kellam Jun 22, 2025
aa21412
centralize vscode icons
brendan-kellam Jun 22, 2025
74b3517
stylelized read files tool component
brendan-kellam Jun 22, 2025
14a985c
much progress on styled tool results
brendan-kellam Jun 23, 2025
b1997da
fix bug with scroll area width
brendan-kellam Jun 23, 2025
d8816fb
switch over to rendering using react-markdown. Render code blocks usi…
brendan-kellam Jun 23, 2025
fea6e47
Add metadata to generated codeblocks. Make codemirror rendering nicer
brendan-kellam Jun 23, 2025
5d1e1e6
fix: only encode path part
brendan-kellam Jun 24, 2025
83cda04
fix: remove forced session update on settings dropdown since that was…
brendan-kellam Jun 24, 2025
1e6e18d
wip on perf improvements: render lightweight code editor when streaming
brendan-kellam Jun 24, 2025
d661bde
resolve conflicts
brendan-kellam Jun 24, 2025
bb02fb1
scroll restoration + some improvements to code block rendering
brendan-kellam Jun 25, 2025
1d65f25
fix
brendan-kellam Jun 25, 2025
907027a
rendering nits + auto scroll
brendan-kellam Jun 26, 2025
90992da
Add hints to inline code blocks
brendan-kellam Jun 26, 2025
4ec001e
fix issues with scroll restoration
brendan-kellam Jun 26, 2025
6f32b76
refactor everything into seperate components
brendan-kellam Jun 26, 2025
956d98e
proper initial message handling
brendan-kellam Jun 26, 2025
f4e3bf8
Make the repo selector actually do something
brendan-kellam Jun 26, 2025
d40acaa
configurable model providers
brendan-kellam Jun 27, 2025
83a6231
save search mode to local storage. Other small nits
brendan-kellam Jun 27, 2025
9059907
Add support for gemini
brendan-kellam Jun 27, 2025
c468a3c
Add bedrock support
brendan-kellam Jun 27, 2025
70a9395
small ux nits: submit button, keyboard shortcuts, auto focus, etc.
brendan-kellam Jun 28, 2025
10cdcba
persist chats to DB
brendan-kellam Jun 28, 2025
8c4ad65
feat: Add chat history side panel
brendan-kellam Jun 29, 2025
d9fcdcf
Add chat title summarization
brendan-kellam Jun 30, 2025
cafbcd8
add ability to rename / delete chats
brendan-kellam Jun 30, 2025
3662977
chore: add chat name to top-bar; make things work when user is unauth…
brendan-kellam Jul 3, 2025
ddc9159
Add stop button
brendan-kellam Jul 3, 2025
752d8b8
wip: Add reasoning output
brendan-kellam Jul 4, 2025
5b8ace7
chore: upgrade to ai-sdk v5 beta
brendan-kellam Jul 4, 2025
466695b
Add remark reference plugin
brendan-kellam Jul 7, 2025
1bfd7ee
small overhall in how we present the chat thread
brendan-kellam Jul 8, 2025
af45097
Add table of contents
brendan-kellam Jul 8, 2025
624b48b
Adds a navigation guard to chat that warns the user that leaving the …
brendan-kellam Jul 8, 2025
9fb0503
fix issue with rendering code blocks
brendan-kellam Jul 8, 2025
f2100c9
fix docker build errors
msukkari Jul 8, 2025
2f253a6
Revert "fix docker build errors"
brendan-kellam Jul 8, 2025
ec96fd2
wip: Reference list viewer
brendan-kellam Jul 8, 2025
25ceee2
nits
brendan-kellam Jul 9, 2025
1c3e93b
chore: remove full codemirror cell output in chat thread
brendan-kellam Jul 9, 2025
7e3a658
Add tool UX for find refs / defs
brendan-kellam Jul 9, 2025
df50e28
Clarify the concept of a source and a reference. Various cleanup work
brendan-kellam Jul 10, 2025
cb0fe49
Add reference scrolling
brendan-kellam Jul 10, 2025
9071159
scroll handling
brendan-kellam Jul 10, 2025
87f3381
small nit
brendan-kellam Jul 11, 2025
c16fcfc
improved highlight range styling
brendan-kellam Jul 11, 2025
e1f616a
color selection tweak
brendan-kellam Jul 11, 2025
90187f6
color tweak
brendan-kellam Jul 11, 2025
b5a3d74
feat: Add code folding to file reference viewer
brendan-kellam Jul 12, 2025
5b50c04
Collapsable code cells
brendan-kellam Jul 12, 2025
86c56d0
Small UX improvements like collapsable chat
brendan-kellam Jul 12, 2025
4193a21
support file references with only a start line
brendan-kellam Jul 12, 2025
27fbb78
cleanup & some devex improvements to how we handle file references
brendan-kellam Jul 13, 2025
e20d810
output openai reasoning tokens
brendan-kellam Jul 13, 2025
fe6fdcc
Add tooltip to mention elements
brendan-kellam Jul 13, 2025
44e28d0
wip on refactoring homepage
brendan-kellam Jul 15, 2025
35b5a46
feat: Add repo suggestions. todo: wire up repo selector to chat box
brendan-kellam Jul 15, 2025
281d8f0
wip on homepage redesign - add agentic search examples
brendan-kellam Jul 15, 2025
110d1d7
Add model provider info to 'new chat' page
brendan-kellam Jul 15, 2025
3e7acf8
wip: source selected repos from @ mentions found in the chat box
brendan-kellam Jul 16, 2025
6b2df65
chore: add a mechanism to truncate large files
brendan-kellam Jul 16, 2025
cc06480
chore: bump to latest ai sdk version
brendan-kellam Jul 16, 2025
e38f469
Various refactors, including: 1) move LLM invocation into a sperate '…
brendan-kellam Jul 21, 2025
6d2081b
resolve conflicts
brendan-kellam Jul 21, 2025
585d3ed
Add back table of contents
brendan-kellam Jul 21, 2025
507d537
Copy button
brendan-kellam Jul 21, 2025
c349add
Add support for like / dislike buttons
brendan-kellam Jul 21, 2025
27ef6d3
Add chat history to LLM input
brendan-kellam Jul 21, 2025
dd99109
switch over to using config.json for configuring models. Add language…
brendan-kellam Jul 21, 2025
cda8a7f
Add tooltips explaining each search mode
brendan-kellam Jul 22, 2025
9270f29
more nits with search mode tooltip
brendan-kellam Jul 22, 2025
8a5c7e7
Load repos in server component. Various other nit fixes and improvements
brendan-kellam Jul 22, 2025
4fbb459
Remove repositories from being a mentionable item, instead opting for…
brendan-kellam Jul 22, 2025
4293cbd
simplification: remove file truncation
brendan-kellam Jul 22, 2025
ac8126c
instruct the LLM to output the answer and stop researching when we ar…
brendan-kellam Jul 22, 2025
334ac73
Add news data
brendan-kellam Jul 22, 2025
da21a5b
Add basic chat history component to homepag
brendan-kellam Jul 22, 2025
253b055
small tweaks here and there
brendan-kellam Jul 22, 2025
6e531fd
rename to ask sourcebot
brendan-kellam Jul 22, 2025
14e9660
update readme
msukkari Jul 22, 2025
e61ae50
Improved anonymous access handling: add concept of chat visibility an…
brendan-kellam Jul 22, 2025
3e0c5b1
Fix build errors
brendan-kellam Jul 22, 2025
c9a7fcc
add support for google vertex
msukkari Jul 23, 2025
0d894cf
small nits with example queries
brendan-kellam Jul 23, 2025
fca9c68
add antrhopic vertex support:
msukkari Jul 23, 2025
95cfd11
remove log
msukkari Jul 23, 2025
0377e2f
changelog
brendan-kellam Jul 23, 2025
096b245
Make repo selection required
brendan-kellam Jul 23, 2025
d54ffd7
docs
msukkari Jul 23, 2025
64b9f99
Add langfuse integration
brendan-kellam Jul 23, 2025
2cc5027
Update og metadata description
brendan-kellam Jul 23, 2025
eea0e06
nits
brendan-kellam Jul 23, 2025
fa74f03
readme nit
brendan-kellam Jul 23, 2025
1707be8
more docs
msukkari Jul 23, 2025
d29a972
feedback
brendan-kellam Jul 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/_gcp-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ jobs:
NEXT_PUBLIC_SENTRY_ENVIRONMENT=${{ vars.NEXT_PUBLIC_SENTRY_ENVIRONMENT }}
NEXT_PUBLIC_SENTRY_WEBAPP_DSN=${{ vars.NEXT_PUBLIC_SENTRY_WEBAPP_DSN }}
NEXT_PUBLIC_SENTRY_BACKEND_DSN=${{ vars.NEXT_PUBLIC_SENTRY_BACKEND_DSN }}
NEXT_PUBLIC_LANGFUSE_PUBLIC_KEY=${{ vars.NEXT_PUBLIC_LANGFUSE_PUBLIC_KEY }}
NEXT_PUBLIC_LANGFUSE_BASE_URL=${{ vars.NEXT_PUBLIC_LANGFUSE_BASE_URL }}
SENTRY_SMUAT=${{ secrets.SENTRY_SMUAT }}
SENTRY_ORG=${{ vars.SENTRY_ORG }}
SENTRY_WEBAPP_PROJECT=${{ vars.SENTRY_WEBAPP_PROJECT }}
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- Introducing Ask Sourcebot - ask natural langauge about your codebase. Get back comprehensive Markdown responses with inline citations back to the code. Bring your own LLM api key. [#392](https://github.com/sourcebot-dev/sourcebot/pull/392)

## [4.5.3] - 2025-07-20

### Changed
Expand Down
14 changes: 9 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,16 +44,20 @@

# About

Sourcebot lets you index all your repos and branches across multiple code hosts (GitHub, GitLab, Bitbucket, Gitea, or Gerrit) and search through them using a blazingly fast interface.
Sourcebot is a self-hosted tool that helps you understand your codebase.

https://github.com/user-attachments/assets/ced355f3-967e-4f37-ae6e-74ab8c06b9ec
- **Ask Sourcebot:** Ask questions about your codebase and have Sourcebot provide detailed answers grounded with inline citations.
- **Code search:** Search and navigate across all your repos and branches, no matter where they’re hosted.

https://github.com/user-attachments/assets/286ad97a-a543-4eef-a2f1-4fa31bea1b32


## Features
- 💻 **One-command deployment**: Get started instantly using Docker on your own machine.
- 🔍 **Multi-repo search**: Index and search through multiple public and private repositories and branches on GitHub, GitLab, Bitbucket, Gitea, or Gerrit.
- ⚡**Lightning fast performance**: Built on top of the powerful [Zoekt](https://github.com/sourcegraph/zoekt) search engine.
- 🎨 **Modern web app**: Enjoy a sleek interface with features like syntax highlighting, light/dark mode, and vim-style navigation
- 🤖 **Bring your own model**: Connect Sourcebot to any of the reasoning models you're already using.
- 🔍 **Multi-repo support**: Index and search through multiple public and private repositories and branches on GitHub, GitLab, Bitbucket, Gitea, or Gerrit.
- ⚡ **Lightning fast performance**: Built on top of the powerful [Zoekt](https://github.com/sourcegraph/zoekt) search engine.
- 🎨 **Modern web app**: Enjoy a sleek interface with features like syntax highlighting, light/dark mode, and vim-style navigation.
- 📂 **Full file visualization**: Instantly view the entire file when selecting any search result.

You can try out our public hosted demo [here](https://demo.sourcebot.dev)!
Expand Down
5 changes: 4 additions & 1 deletion docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,15 @@
"group": "Features",
"pages": [
{
"group": "Search",
"group": "Code Search",
"pages": [
"docs/features/search/overview",
"docs/features/search/syntax-reference",
"docs/features/search/multi-branch-indexing",
"docs/features/search/search-contexts"
]
},
"docs/features/ask",
"docs/features/code-navigation",
"docs/features/analytics",
"docs/features/mcp-server",
Expand Down Expand Up @@ -66,6 +68,7 @@
"docs/connections/request-new"
]
},
"docs/configuration/language-model-providers",
"docs/license-key",
"docs/configuration/environment-variables",
{
Expand Down
4 changes: 4 additions & 0 deletions docs/docs/configuration/language-model-providers.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title: Language Model Providers
sidebarTitle: Language model providers
---
46 changes: 46 additions & 0 deletions docs/docs/features/ask.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
title: Ask Sourcebot
---

Ask Sourcebot gives you the ability to ask complex questions about your codebase in natural language.

It uses Sourcebot’s existing [code search](/docs/features/search/overview) and [navigation](/docs/features/code-navigation) tools to allow reasoning models to search your code,
follow code nav references, and provide an answer that’s rich with inline citations and navigable code snippets.

<CardGroup>
<Card title="Configure language models" icon="robot" href="/docs/configuration/language-model-providers" horizontal="true">
Learn how to connect your language model to Sourcebot
</Card>
<Card title="Index repos" icon="book" href="/docs/connections/overview" horizontal="true">
Learn how to index your repos so you can ask questions about them
</Card>
</CardGroup>

<video
autoPlay
muted
loop
playsInline
className="w-full aspect-video"
src="/images/ask_sourcebot_low_res.mp4"
></video>

# Why do we need another AI dev tool?

Existing AI dev tools (Cursor, Claude Code, Copilot) are great at generating code. However, we believe one of the hardest parts of being
a software engineer is **understanding code**.

In this domain, these tools fall short:
- You can only ask questions about the code you have checked out locally
- You get a wall of text that's difficult to parse, requiring you to go back and forth through different code snippets in the response
- The richness of the explanation is limited by the fact that you're in your IDE

We built Ask Sourcebot to address these problems. With Ask Sourcebot, you can:
- Ask questions about your teams entire codebase (even on repos you don't have locally)
- Easily parse the response with side-by-side citations and code navigation
- Share answers with your team to spread the knowledge

Being a web app is less convenient than being in your IDE, but it allows Sourcebot to provide responses in a richer UI that isn't constrained by the IDE.

We believe this experience of understanding your codebase is superior, and we hope you find it useful. We'd love to know what you think! Feel free to join the discussion on our
[GitHub](https://github.com/sourcebot-dev/sourcebot/discussions).
34 changes: 34 additions & 0 deletions docs/docs/features/search/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
title: Overview
---

Search across all your repos/branches across any code host platform. Blazingly fast, and supports regular expressions, repo/language search filters, boolean logic, and more.

<Accordion title="Key benefits">
- **Regex support:** Use regular expressions to find code with precision.
- **Query language:** Scope searches to specific files, repos, languages, symbol definitions and more using a rich [query language](/docs/features/search/syntax-reference).
- **Branch search:** Specify a list of branches to search across ([docs](/docs/features/search/multi-branch-indexing)).
- **Fast & scalable:** Sourcebot uses [trigram indexing](https://en.wikipedia.org/wiki/Trigram_search), allowing it to scale to massive codebases.
- **Syntax highlighting:** Syntax highlighting support for over [100+ languages](https://github.com/sourcebot-dev/sourcebot/blob/57724689303f351c279d37f45b6406f1d5d5d5ab/packages/web/src/lib/codemirrorLanguage.ts#L125).
- **Multi-repository:** Search across all of your repositories in a single search.
- **Search suggestions:** Get search suggestions as you craft your query.
- **Filter panel:** Filter results by repository or by language.
</Accordion>

<CardGroup>
<Card title="Index repos" icon="book" href="/docs/connections/overview" horizontal="true">
Learn how to index your repos so you can ask questions about them
</Card>
<Card title="Branches" icon="split" href="/docs/features/search/multi-branch-indexing" horizontal="true">
Learn how to index and search through your branches
</Card>
</CardGroup>

<video
autoPlay
muted
loop
playsInline
className="w-full aspect-video"
src="https://framerusercontent.com/assets/cEqHNSLiMbNeG3bk5xheQWXmKqc.mp4"
></video>
28 changes: 25 additions & 3 deletions docs/docs/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
title: "Overview"
---

[Sourcebot]((https://github.com/sourcebot-dev/sourcebot)) is an open-source, self-hosted code search tool. It allows you to search and navigate across millions of lines of code across several code host platforms.
[Sourcebot](https://github.com/sourcebot-dev/sourcebot) is a self-hosted tool that helps you understand your codebase.

- [Code search](/docs/features/search/overview): Search and navigate across all your repos and branches, no matter where they’re hosted
- [Ask Sourcebot](/docs/features/ask): Ask questions about your codebase and have Sourcebot provide detailed answers grounded with inline citations

<CardGroup>
<Card title="Deployment guide" icon="server" href="/docs/deployment-guide" horizontal="true">
Expand Down Expand Up @@ -30,9 +33,28 @@ title: "Overview"
Find an overview of all Sourcebot features below. For details, see the individual documentation pages.
</Info>

### Fast indexed based search
### Ask Sourcebot

[Ask Sourcebot](/docs/features/ask) gives you the ability to ask complex questions about your codebase, and have Sourcebot provide detailed answers with inline citations.

<Accordion title="Key benefits">
- **Bring your own model:** [Configure](/docs/configuration/language-model-providers) to any language model you'd like
- **Inline citations:** Every answer Sourcebot provides is grounded with inline citations directly into your codebase
- **Mutli-repo:** Ask questions about any repository you have indexed on Sourcebot
</Accordion>

<video
autoPlay
muted
loop
playsInline
className="w-full aspect-video"
src="/images/ask_sourcebot_low_res.mp4"
></video>

### Code Search

Search across millions of lines of code instantly using Sourcebot's blazingly fast indexed search. Find exactly what you are looking for with regular expressions, search filters, boolean logic, and more.
Search across all your repos/branches across any code host platform. Blazingly fast, and supports regular expressions, repo/language search filters, boolean logic, and more.

<Accordion title="Key benefits">
- **Regex support:** Use regular expressions to find code with precision.
Expand Down
Binary file added docs/images/ask_sourcebot_low_res.mp4
Binary file not shown.
Loading