Skip to content

Commit b7552da

Browse files
authored
Merge pull request #950 from srtab/chore/improve-local-dev-setup
Improve local development setup for new contributors
2 parents e054313 + a546a0d commit b7552da

File tree

17 files changed

+162
-63
lines changed

17 files changed

+162
-63
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -160,6 +160,7 @@ cython_debug/
160160
#.idea/
161161

162162
config.secrets.env
163+
config.toml
163164
state-snapshots.txt
164165

165166
.ruff_cache/

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1414
- Added `--repo-id` option to `setup_webhooks` command to restrict setup to a specific repository.
1515
- Added `allowed_usernames` option to `.daiv.yml` to restrict which users can interact with DAIV on a per-repository basis. Useful for public repositories where you want to limit who can trigger DAIV.
1616

17+
### Fixed
18+
19+
- Fixed `set -eu pipefail` in shell scripts — `pipefail` is not valid in POSIX `#!/bin/sh`.
20+
1721
### Changed
1822

1923
- Changed `setup_webhooks` command to only create new webhooks by default, skipping existing ones.

CONTRIBUTING.md

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ Thank you for your interest in contributing to DAIV! This document provides guid
55
## Table of Contents
66

77
- [Code of Conduct](#code-of-conduct)
8+
- [Development Environment](#development-environment)
89
- [Development Guidelines](#development-guidelines)
910
- [Code Style](#code-style)
1011
- [Testing](#testing)
@@ -20,6 +21,25 @@ Thank you for your interest in contributing to DAIV! This document provides guid
2021

2122
We expect all contributors to be respectful and constructive. Please ensure that your interactions with the community are positive and inclusive.
2223

24+
## Development Environment
25+
26+
To get started quickly:
27+
28+
```bash
29+
git clone https://github.com/srtab/daiv.git && cd daiv
30+
make setup # creates config files from templates
31+
docker compose up --build # starts core services (db, redis, app, worker, scheduler)
32+
```
33+
34+
Optional services are available via Docker Compose profiles:
35+
36+
- `--profile gitlab` — local GitLab instance and runner
37+
- `--profile sandbox` — sandbox code executor
38+
- `--profile mcp` — MCP proxy
39+
- `--profile full` — all services
40+
41+
See the [README](README.md) for full setup details.
42+
2343
## Development Guidelines
2444

2545
### Code Style
@@ -66,7 +86,7 @@ DAIV uses pytest for testing:
6686

6787
### Type Checking
6888

69-
We use mypy for static type checking but we don't enforce it, we encourage you to use it to improve your code quality:
89+
We use [ty](https://github.com/astral-sh/ty) for static type checking but we don't enforce it, we encourage you to use it to improve your code quality:
7090

7191
```bash
7292
make lint-typing

Makefile

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
11
# Makefile
22

3-
.PHONY: help test test-ci lint lint-check lint-format lint-fix lint-typing evals
3+
.PHONY: help setup test test-ci lint lint-check lint-format lint-fix lint-typing evals
44

55
help:
66
@echo "Available commands:"
7+
@echo " make setup - Set up local development environment"
78
@echo " make test - Run tests with coverage report"
89
@echo " make lint - Run lint check and format check"
910
@echo " make lint-check - Run lint check only (ruff)"
@@ -13,6 +14,30 @@ help:
1314
@echo " make lock - Update uv lock"
1415
@echo " make integration-tests - Run integration tests"
1516

17+
setup:
18+
@if [ ! -f docker/local/app/config.secrets.env ]; then \
19+
cp docker/local/app/config.secrets.env.example docker/local/app/config.secrets.env && \
20+
echo "Created docker/local/app/config.secrets.env from template."; \
21+
else \
22+
echo "docker/local/app/config.secrets.env already exists, skipping."; \
23+
fi
24+
@if [ ! -f docker/local/gitlab-runner/config.toml ]; then \
25+
cp docker/local/gitlab-runner/config.toml.example docker/local/gitlab-runner/config.toml && \
26+
echo "Created docker/local/gitlab-runner/config.toml from template."; \
27+
else \
28+
echo "docker/local/gitlab-runner/config.toml already exists, skipping."; \
29+
fi
30+
@echo ""
31+
@echo "Next steps:"
32+
@echo " 1. Edit docker/local/app/config.secrets.env and add your API keys"
33+
@echo " (at minimum: one LLM provider key + CODEBASE_GITLAB_AUTH_TOKEN if using GitLab)"
34+
@echo " 2. Start core services: docker compose up --build"
35+
@echo " 3. Optional services:"
36+
@echo " docker compose --profile gitlab up # local GitLab instance"
37+
@echo " docker compose --profile sandbox up # sandbox code executor"
38+
@echo " docker compose --profile mcp up # MCP proxy"
39+
@echo " docker compose --profile full up # all services"
40+
1641
test:
1742
LANGCHAIN_TRACING_V2=false uv run pytest -s tests/unit_tests
1843

README.md

Lines changed: 43 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -62,13 +62,14 @@ DAIV's agent has access to a set of capabilities that make this possible:
6262
cd daiv
6363
```
6464

65-
2. **Configure Environment**:
66-
Copy `docker/local/app/config.secrets.env.example` to `docker/local/app/config.secrets.env` and update it with your Git platform credentials (GitLab token or GitHub App credentials), OpenAI API Key, Anthropic API Key, Google API Key, and LangChain API Key.
65+
2. **Run setup**:
6766

6867
```bash
69-
cp docker/local/app/config.secrets.env.example docker/local/app/config.secrets.env
68+
make setup
7069
```
7170

71+
This creates config files from their templates (`config.secrets.env` and `config.toml`). Edit `docker/local/app/config.secrets.env` and add your API keys — at minimum one LLM provider key (Anthropic, OpenAI, Google, or OpenRouter) and `CODEBASE_GITLAB_AUTH_TOKEN` if using GitLab.
72+
7273
3. **Install Dependencies** (optional):
7374
We use [uv](https://docs.astral.sh/uv/) to manage dependencies on DAIV.
7475

@@ -79,30 +80,37 @@ DAIV's agent has access to a set of capabilities that make this possible:
7980
> [!NOTE]
8081
> This will install the project dependencies into a virtual environment. Useful for running linting outside of Docker or enabling autocompletion in VSCode.
8182
82-
4. **Start the Server**:
83+
4. **Start core services**:
8384

8485
```bash
8586
docker compose up --build
8687
```
8788

88-
This will start all needed services locally. You can access them at:
89+
This starts the core services (db, redis, app, worker, scheduler). SSL certificates are auto-generated on first run.
8990

9091
- DAIV API documentation: https://localhost:8000/api/docs/
91-
- GitLab (local test repository platform): http://localhost:8929
92-
- Sandbox (secure code execution): http://localhost:8888/docs
92+
93+
5. **Start optional services** (as needed):
94+
95+
```bash
96+
docker compose --profile gitlab up # local GitLab instance + runner
97+
docker compose --profile sandbox up # sandbox code executor
98+
docker compose --profile mcp up # MCP proxy
99+
docker compose --profile full up # all services
100+
```
93101

94102
> [!NOTE]
95-
> The local development setup includes a GitLab instance for testing. For GitHub integration, you'll need to use GitHub.com or your own GitHub Enterprise instance.
103+
> Profiles can be combined: `docker compose --profile gitlab --profile sandbox up`
96104
97-
5. **Run the tests** (optional):
105+
6. **Run the tests** (optional):
98106
DAIV includes a comprehensive test suite. To run tests with coverage:
99107

100108
```bash
101109
$ docker compose exec -it app bash
102110
$ make test
103111
```
104112

105-
6. **Run linting** (optional):
113+
7. **Run linting** (optional):
106114
To ensure code quality:
107115

108116
```bash
@@ -111,37 +119,42 @@ DAIV's agent has access to a set of capabilities that make this possible:
111119
$ make lint-fix # to automatically fix linting and formatting issues
112120
```
113121

114-
7. **Configure test repository**:
115-
To be able to test DAIV, you need to configure a test repository on the local GitLab instance (or use your own GitHub repository).
122+
### Optional: Local GitLab
123+
124+
To test DAIV with a local GitLab instance:
125+
126+
1. **Start GitLab**:
116127

117-
1. First you need to obtain root password to authenticate with [local GitLab](http://localhost:8929):
128+
```bash
129+
docker compose --profile gitlab up
130+
```
118131

119-
```bash
120-
$ docker compose exec -it gitlab grep 'Password:' /etc/gitlab/initial_root_password
121-
```
132+
2. **Get the root password**:
122133

123-
2. Then you need to configure a personal access token (you can use the root user or create a new user) and add it to the `docker/local/app/config.secrets.env` file.
134+
```bash
135+
docker compose exec -it gitlab grep 'Password:' /etc/gitlab/initial_root_password
136+
```
124137

125-
3. Now you need to create a new project in GitLab and follow the instructions to push your testing code to it.
138+
3. **Configure a personal access token** at [http://localhost:8929](http://localhost:8929) (use the root user or create a new user) and add it to `docker/local/app/config.secrets.env` as `CODEBASE_GITLAB_AUTH_TOKEN`.
126139

127-
> [!TIP]
128-
> You can import using repository URL, go to `Admin Area` -> `Settings` -> `General` -> `Import and export settings` and check the `Repository by URL` option.
140+
4. **Create a test project** in GitLab and push your testing code to it.
129141

130-
4. After you push/import your code to the repository, you need to set up webhooks and index the repository in DAIV:
142+
> [!TIP]
143+
> You can import using repository URL: go to `Admin Area` -> `Settings` -> `General` -> `Import and export settings` and check the `Repository by URL` option.
131144
132-
```bash
133-
# Enter the app container
134-
$ docker compose exec -it app bash
145+
5. **Set up webhooks**:
135146

136-
# Set up webhooks to trigger automatically DAIV actions. You can disable SSL verification for local development by adding `--disable-ssl-verification` to the command.
137-
$ django-admin setup_webhooks
147+
```bash
148+
docker compose exec -it app django-admin setup_webhooks
149+
```
138150

139-
```
151+
> [!NOTE]
152+
> If you get the error `Invalid url given`, go to `Admin Area` -> `Settings` -> `Network` -> `Outbound requests` and check `Allow requests to the local network from webhooks and integrations`.
140153
141-
> [!NOTE]
142-
> If you're getting the error `Invalid url given` when setting up the webhooks on local GitLab, go to `Admin Area` -> `Settings` -> `Network` -> `Outbound requests` and check the `Allow requests to the local network from webhooks and integrations` option.
154+
6. **Test DAIV** by creating an issue in your repository with the `daiv` label. DAIV will automatically present a plan to address the issue.
143155

144-
5. Finally, you can test DAIV by creating an issue in your repository, add `daiv` label to it and see how DAIV will automatically present a plan to address the issue.
156+
> [!NOTE]
157+
> For GitHub integration, you'll need to use GitHub.com or your own GitHub Enterprise instance. Set `CODEBASE_CLIENT=github` in `docker/local/app/config.env` and configure the GitHub App credentials.
145158
146159

147160
## Roadmap

daiv/automation/agent/middlewares/skills.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
from __future__ import annotations
2+
13
import logging
24
import shlex
35
from pathlib import Path

docker-compose.yml

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,6 @@ x-app-defaults: &x_app_default
2020
redis:
2121
condition: service_healthy
2222
restart: true
23-
gitlab:
24-
condition: service_healthy
25-
sandbox:
26-
condition: service_healthy
2723

2824
services:
2925
db:
@@ -90,6 +86,9 @@ services:
9086
gitlab:
9187
image: gitlab/gitlab-ce:latest
9288
container_name: daiv-gitlab
89+
profiles:
90+
- gitlab
91+
- full
9392
restart: on-failure
9493
logging:
9594
driver: "none"
@@ -113,6 +112,9 @@ services:
113112
gitlab-runner:
114113
image: gitlab/gitlab-runner:latest
115114
container_name: daiv-gitlab-runner
115+
profiles:
116+
- gitlab
117+
- full
116118
restart: unless-stopped
117119
volumes:
118120
- ./docker/local/gitlab-runner/config.toml:/etc/gitlab-runner/config.toml
@@ -124,23 +126,27 @@ services:
124126
condition: service_healthy
125127

126128
sandbox:
127-
build:
128-
context: ./../daiv-sandbox
129-
args:
130-
DOCKER_GID: 999
129+
image: ghcr.io/srtab/daiv-sandbox:latest
130+
profiles:
131+
- sandbox
132+
- full
131133
stdin_open: true
132134
restart: unless-stopped
133135
container_name: daiv-sandbox
136+
group_add:
137+
- "${DOCKER_GID:-999}"
134138
env_file:
135139
- ./docker/local/sandbox/config.env
136140
ports:
137141
- "8888:8000"
138142
volumes:
139143
- /var/run/docker.sock:/var/run/docker.sock
140-
- ./../daiv-sandbox/daiv_sandbox:/home/app/daiv_sandbox
141144

142145
mcp-proxy:
143146
image: ghcr.io/tbxark/mcp-proxy:v0.43.2
147+
profiles:
148+
- mcp
149+
- full
144150
restart: unless-stopped
145151
container_name: daiv-mcp-proxy
146152
entrypoint:

docker/local/app/Dockerfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ RUN apt-get update \
2525
gnupg \
2626
ca-certificates \
2727
ripgrep \
28+
openssl \
2829
# Install GitHub CLI
2930
&& curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \
3031
| dd of=/etc/apt/keyrings/githubcli-archive-keyring.gpg \

docker/local/app/config.env

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# default settings to run django manage commands
1+
# Django settings
22
DJANGO_SETTINGS_MODULE=daiv.settings.local
33
DJANGO_SECRET_KEY='v6_=1=)!_$aoeswo@hdo@5kov)v9844!1r&+w!!@@+_7&*!jb+'
44
DJANGO_ALLOWED_HOSTS=*
@@ -7,23 +7,25 @@ DJANGO_REDIS_SESSION_URL=redis://redis:6379/1
77
DJANGO_REDIS_CHECKPOINT_URL=redis://redis:6379/0
88
DJANGO_DEBUG=True
99

10-
# Database secrets
10+
# Database
1111
DB_HOST=db
1212
DB_NAME=db
1313
DB_USER=dbuser
1414
DB_PASSWORD=dbpass
1515
DB_PORT=5432
1616
DB_SSLMODE=prefer
1717

18-
# Codebase
18+
# Codebase client — set to "gitlab" for local GitLab or "github" for GitHub
1919
CODEBASE_CLIENT=gitlab
2020
CODEBASE_GITLAB_URL=http://gitlab:8929
21-
CODEBASE_GITHUB_APP_ID=2763557
22-
CODEBASE_GITHUB_INSTALLATION_ID=107051972
2321

24-
# LANGSMITH
22+
# GitHub App credentials (only needed when CODEBASE_CLIENT=github)
23+
# CODEBASE_GITHUB_APP_ID=
24+
# CODEBASE_GITHUB_INSTALLATION_ID=
25+
26+
# LangSmith observability
2527
LANGCHAIN_TRACING_V2=true
2628
LANGCHAIN_PROJECT=default
2729

28-
# SANDBOX
30+
# Sandbox
2931
DAIV_SANDBOX_NETWORK_ENABLED=True
Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,29 @@
1-
# Codebase
1+
# =============================================================================
2+
# REQUIRED — At least one LLM provider key (+ GitLab token if using GitLab)
3+
# =============================================================================
4+
5+
# GitLab personal access token (required for CODEBASE_CLIENT=gitlab)
26
CODEBASE_GITLAB_AUTH_TOKEN=
37

4-
# Models
5-
OPENAI_API_KEY=
8+
# LLM provider keys — at least one is required
69
ANTHROPIC_API_KEY=
10+
OPENAI_API_KEY=
711
GOOGLE_API_KEY=
812
OPENROUTER_API_KEY=
913

10-
# Monitoring
14+
# =============================================================================
15+
# OPTIONAL — Enable these as needed for additional features
16+
# =============================================================================
17+
18+
# LangSmith observability (traces and monitoring)
1119
LANGCHAIN_API_KEY=
1220

13-
# WebSearch
21+
# Web search capability
1422
WEB_SEARCH_API_KEY=
1523

16-
# MCP
24+
# MCP proxy (requires --profile mcp)
1725
MCP_CONFIG_API_KEY=
1826
MCP_SENTRY_ACCESS_TOKEN=
1927

20-
# Sandbox
28+
# Sandbox secure execution (requires --profile sandbox)
2129
DAIV_SANDBOX_API_KEY=

0 commit comments

Comments
 (0)