Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 0 additions & 31 deletions .github/actions/docker-check-containers/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,34 +31,3 @@ runs:
fi
done
shell: bash
- name: Check Prometheus is running
run: |
for i in {1..${{ inputs.num-retries }}}; do
if curl http://localhost:9090; then
echo "Health check passed"
break
else
echo "Health check failed, attempt $i/${{ inputs.num-retries }}"
if [ $i -eq ${{ inputs.num-retries }} ]; then
exit 1
fi
sleep ${{ inputs.timeout-seconds }}
fi
done
shell: bash

- name: Check Grafana is running
run: |
for i in {1..${{ inputs.num-retries }}}; do
if curl http://localhost:3000; then
echo "Health check passed"
break
else
echo "Health check failed, attempt $i/${{ inputs.num-retries }}"
if [ $i -eq ${{ inputs.num-retries }} ]; then
exit 1
fi
sleep ${{ inputs.timeout-seconds }}
fi
done
shell: bash
11 changes: 3 additions & 8 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Developers extend `TemplateServer` to create application-specific servers (see `
### Application Factory Pattern

- Entry: `main.py:run()` → instantiates `ExampleServer` (subclass of `TemplateServer`) → calls `.run()`
- `TemplateServer.__init__()` sets up middleware, rate limiting, metrics, and calls `setup_routes()`
- `TemplateServer.__init__()` sets up middleware, rate limiting, and calls `setup_routes()`
- **Critical**: Middleware order matters - request logging → security headers → rate limiting
- **Extensibility**: Subclasses implement `setup_routes()` to add custom endpoints and `validate_config()` for config validation

Expand All @@ -29,7 +29,6 @@ Developers extend `TemplateServer` to create application-specific servers (see `
- **Hash Storage**: Only hash stored in `.env` (API_TOKEN_HASH), raw token shown once
- **Token Loading**: `load_hashed_token()` loads hash from .env on server startup, stored in `TemplateServer.hashed_token`
- **Verification Flow**: Request → `_verify_api_key()` dependency → `verify_token()` → hash comparison
- **Metrics**: Success/failure counters with labeled reasons (missing/invalid/error)
- **Health Endpoint**: `/api/health` does NOT require authentication, reports unhealthy if token not configured
- Header: `X-API-Key` (defined in `constants.API_KEY_HEADER_NAME`)

Expand All @@ -42,8 +41,6 @@ Developers extend `TemplateServer` to create application-specific servers (see `

### Observability Stack

- **Prometheus**: `/metrics` endpoint always exposed (no auth), custom auth/rate-limit metrics
- **Grafana**: Pre-configured dashboards in `grafana/dashboards/*.json`
- **Logging**: Dual output (console + rotating file), 10MB per file, 5 backups in `logs/`
- **Request Tracking**: `RequestLoggingMiddleware` logs all requests with client IP

Expand Down Expand Up @@ -107,7 +104,6 @@ docker compose down # Stop and remove containers

- **Prefix**: All routes under `/api` (API_PREFIX constant)
- **Authentication**: Applied via `dependencies=[Security(self._verify_api_key)]` in route registration
- **Unauthenticated Endpoints**: `/health` and `/metrics` do not require authentication
- **Response Models**: All endpoints return `BaseResponse` subclasses with code/message/timestamp
- **Health Status**: `/health` includes `status` field (HEALTHY/DEGRADED/UNHEALTHY), reports unhealthy if no token configured

Expand Down Expand Up @@ -155,15 +151,14 @@ All PRs must pass:

### Key Files

- `template_server.py` - Base TemplateServer class with middleware/metrics/auth setup
- `template_server.py` - Base TemplateServer class with middleware/auth setup
- `main.py` - ExampleServer implementation showing how to extend TemplateServer
- `authentication_handler.py` - Token generation, hashing, verification
- `prometheus_handler.py` - Prometheus metrics setup and custom metric definitions
- `certificate_handler.py` - Self-signed SSL certificate generation and loading
- `logging_setup.py` - Logging configuration (executed on import)
- `models.py` - All Pydantic models (config + responses)
- `constants.py` - Project constants, logging config
- `docker-compose.yml` - FastAPI + Prometheus + Grafana stack
- `docker-compose.yml` - Container stack

### Environment Variables

Expand Down
12 changes: 0 additions & 12 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,6 @@ jobs:
REQUIRED_DIRS=(
"${SITE_PACKAGES}/${PACKAGE_NAME}"
"${SITE_PACKAGES}/configuration"
"${SITE_PACKAGES}/grafana"
"${SITE_PACKAGES}/prometheus"
)

for dir in "${REQUIRED_DIRS[@]}"; do
Expand All @@ -91,13 +89,3 @@ jobs:
exit 1
fi
done

# Show directory structure
echo "Package structure in site-packages:"
tree "${SITE_PACKAGES}/${PACKAGE_NAME}" --dirsfirst -F -L 2
echo "Configuration structure:"
tree "${SITE_PACKAGES}/configuration" --dirsfirst -F
echo "Grafana structure:"
tree "${SITE_PACKAGES}/grafana" --dirsfirst -F -L 2
echo "Prometheus structure:"
tree "${SITE_PACKAGES}/prometheus" --dirsfirst -F
13 changes: 0 additions & 13 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@ COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
# Copy backend source files
COPY python_template_server/ ./python_template_server/
COPY configuration/ ./configuration/
COPY grafana/ ./grafana/
COPY prometheus/ ./prometheus/
COPY pyproject.toml .here LICENSE README.md ./

# Build the wheel
Expand All @@ -38,8 +36,6 @@ RUN mkdir -p /app/logs /app/certs
# Copy included files from installed wheel to app directory
RUN SITE_PACKAGES_DIR=$(find /usr/local/lib -name "site-packages" -type d | head -1) && \
cp -r "${SITE_PACKAGES_DIR}/configuration" /app/ && \
cp -r "${SITE_PACKAGES_DIR}/grafana" /app/ && \
cp -r "${SITE_PACKAGES_DIR}/prometheus" /app/ && \
cp "${SITE_PACKAGES_DIR}/.here" /app/.here && \
cp "${SITE_PACKAGES_DIR}/LICENSE" /app/LICENSE && \
cp "${SITE_PACKAGES_DIR}/README.md" /app/README.md
Expand All @@ -48,15 +44,6 @@ RUN SITE_PACKAGES_DIR=$(find /usr/local/lib -name "site-packages" -type d | head
RUN echo '#!/bin/sh\n\
set -e\n\
\n\
# Copy monitoring configs to shared volume if they do not exist\n\
if [ -d "/monitoring-configs" ]; then\n\
echo "Setting up monitoring configurations..."\n\
mkdir -p /monitoring-configs/prometheus /monitoring-configs/grafana\n\
cp -r /app/prometheus/* /monitoring-configs/prometheus/ 2>/dev/null || true\n\
cp -r /app/grafana/* /monitoring-configs/grafana/ 2>/dev/null || true\n\
echo "Monitoring configurations ready"\n\
fi\n\
\n\
# Generate API token if needed\n\
if [ -z "$API_TOKEN_HASH" ]; then\n\
echo "Generating new token..."\n\
Expand Down
12 changes: 4 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@
<!-- omit from toc -->
# Python Template Server

A production-ready FastAPI server template with built-in authentication, rate limiting, security headers, and Prometheus metrics. This repository provides a solid foundation for building secure, observable FastAPI applications.
A production-ready FastAPI server template with built-in authentication, rate limiting and security headers.
This repository provides a solid foundation for building secure, observable FastAPI applications.

<!-- omit from toc -->
## Table of Contents
Expand All @@ -30,7 +31,6 @@ A production-ready FastAPI server template with built-in authentication, rate li

- **TemplateServer Base Class**: Reusable foundation
- **FastAPI Framework**: Modern, fast, async-ready web framework
- **Observability Stack**: Pre-configured Prometheus + Grafana dashboards
- **Docker Support**: Multi-stage builds with docker-compose orchestration
- **Production Patterns**: Token generation, SSL certificate handling, health checks

Expand All @@ -42,7 +42,6 @@ This project uses a **`TemplateServer` base class** that encapsulates cross-cutt
- **Security Headers**: HSTS/CSP/X-Frame-Options automatically applied
- **API Key Verification**: SHA-256 hashed tokens with secure validation
- **Rate Limiting**: Configurable limits using `slowapi` (in-memory/Redis/Memcached)
- **Prometheus Metrics**: Custom authentication/rate-limit metrics + HTTP instrumentation

**Application-specific servers** (like `ExampleServer` in `main.py`) extend `TemplateServer` to implement domain-specific endpoints and business logic. The base class handles all infrastructure concerns, letting you focus on your API functionality.

Expand Down Expand Up @@ -94,7 +93,6 @@ uv run python-template-server
# Server runs at https://localhost:443/api
# Swagger UI: https://localhost:443/api/docs
# Redoc: https://localhost:443/api/redoc
# Metrics: curl -k https://localhost:443/api/metrics
# Health check: curl -k https://localhost:443/api/health
# Login (requires authentication): curl -k -H "X-API-Key: your-token-here" https://localhost:443/api/login
```
Expand All @@ -115,21 +113,19 @@ See the [Software Maintenance Guide](./docs/SMG.md) for detailed setup instructi
## Docker Deployment

```sh
# Start all services (FastAPI + Prometheus + Grafana)
# Start all services
docker compose up -d

# View logs
docker compose logs -f python-template-server

# Access services:
# - API: https://localhost:443/api
# - Prometheus: http://localhost:9090
# - Grafana: http://localhost:3000 (admin/admin)
```

## Documentation

- **[API Documentation](./docs/API.md)**: Endpoints, authentication, metrics
- **[API Documentation](./docs/API.md)**: API architecture and endpoints
- **[Software Maintenance Guide](./docs/SMG.md)**: Development setup, configuration
- **[Docker Deployment Guide](./docs/DOCKER_DEPLOYMENT.md)**: Container orchestration
- **[Workflows](./docs/WORKFLOWS.md)**: CI/CD pipeline details
Expand Down
56 changes: 0 additions & 56 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
services:
# Python Template Server
python-template-server:
image: ${CYBER_QUERY_AI_IMAGE:-ghcr.io/javidahmed64592/cyber-query-ai:latest}
build:
context: .
dockerfile: Dockerfile
Expand All @@ -14,9 +13,6 @@ services:
volumes:
- certs:/app/certs
- logs:/app/logs
- monitoring-configs:/monitoring-configs
networks:
- monitoring
restart: unless-stopped
healthcheck:
test:
Expand All @@ -31,58 +27,6 @@ services:
retries: 3
start_period: 10s

# Prometheus for metrics collection
prometheus:
image: prom/prometheus:latest
container_name: prometheus
ports:
- "9090:9090"
volumes:
- monitoring-configs:/monitoring-configs:ro
- prometheus-data:/prometheus
command:
- "--config.file=/monitoring-configs/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus"
- "--web.console.libraries=/usr/share/prometheus/console_libraries"
- "--web.console.templates=/usr/share/prometheus/consoles"
- "--web.enable-lifecycle"
networks:
- monitoring
restart: unless-stopped
depends_on:
python-template-server:
condition: service_started

# Grafana for metrics visualization
grafana:
image: grafana/grafana:latest
container_name: grafana
ports:
- "3000:3000"
environment:
- GF_SECURITY_ADMIN_USER=admin
- GF_SECURITY_ADMIN_PASSWORD=admin
- GF_INSTALL_PLUGINS=
- GF_PATHS_PROVISIONING=/monitoring-configs/grafana/provisioning
volumes:
- monitoring-configs:/monitoring-configs:ro
- grafana-data:/var/lib/grafana
networks:
- monitoring
restart: unless-stopped
depends_on:
python-template-server:
condition: service_started
prometheus:
condition: service_started

networks:
monitoring:
driver: bridge

volumes:
certs:
logs:
prometheus-data:
grafana-data:
monitoring-configs:
54 changes: 0 additions & 54 deletions docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,6 @@ All endpoints are mounted under the `/api` prefix.
- [Request Logging](#request-logging)
- [Security Headers](#security-headers)
- [Rate Limiting](#rate-limiting)
- [Prometheus Metrics](#prometheus-metrics)
- [GET /api/metrics](#get-apimetrics)
- [Standard HTTP Metrics (via `prometheus-fastapi-instrumentator`)](#standard-http-metrics-via-prometheus-fastapi-instrumentator)
- [Custom Application Metrics](#custom-application-metrics)
- [Accessing Dashboards](#accessing-dashboards)
- [Prometheus Dashboard](#prometheus-dashboard)
- [Grafana Dashboards](#grafana-dashboards)
- [Endpoints](#endpoints)
- [GET /api/health](#get-apihealth)
- [GET /api/login](#get-apilogin)
Expand Down Expand Up @@ -121,53 +114,6 @@ Default rate limit: **100 requests per minute** per IP address.

Rate limits can be configured in `config.json`.

## Prometheus Metrics

The server exposes Prometheus-compatible metrics for monitoring and observability.

### GET /api/metrics

- **Purpose**: Expose Prometheus metrics for scraping and monitoring.
- **Format**: Prometheus text-based exposition format.

**Metrics Exposed**:

#### Standard HTTP Metrics (via `prometheus-fastapi-instrumentator`)
- `http_requests_total`: Total number of HTTP requests by method, path, and status code
- `http_request_duration_seconds`: HTTP request latency histogram by method and path
- `http_requests_in_progress`: Number of HTTP requests currently being processed

#### Custom Application Metrics

**Authentication Metrics**:
- `auth_success_total`: Counter tracking successful API key validations
- `auth_failure_total{reason}`: Counter tracking failed authentication attempts with labels:
- `reason="missing"`: No API key provided in request
- `reason="invalid"`: Invalid or incorrect API key
- `reason="error"`: Error during token verification

**Rate Limiting Metrics**:
- `rate_limit_exceeded_total{endpoint}`: Counter tracking requests that exceeded rate limits, labeled by endpoint path

### Accessing Dashboards

The application includes pre-configured monitoring dashboards for visualization:

#### Prometheus Dashboard
- **URL**: http://localhost:9090
- **Purpose**: Query and visualize raw metrics data
- **Features**: Built-in query interface, graphing, and alerting

#### Grafana Dashboards
- **URL**: http://localhost:3000
- **Credentials**: admin / admin (change after first login)
- **Pre-configured Dashboards**:
- **Authentication Metrics**: Tracks successful and failed authentication attempts, including reasons for failures
- **Rate Limiting Metrics**: Monitors requests that exceed rate limits by endpoint

To access the dashboards, the containers for Grafana and Prometheus must be running.
See the [Docker documentation](./DOCKER_DEPLOYMENT.md) for information on how to run these.

## Endpoints

### GET /api/health
Expand Down
Loading