diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000..c9d9f20 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,174 @@ +# Python Template Server - AI Agent Instructions + +## Project Overview + +FastAPI-based template server providing reusable infrastructure for building secure HTTPS applications. +Implements authentication, rate limiting, security headers, and observability foundations via a base `TemplateServer` class. +Developers extend `TemplateServer` to create application-specific servers (see `ExampleServer` in `main.py`). + +## Architecture & Key Components + +### Application Factory Pattern + +- Entry: `main.py:run()` → instantiates `ExampleServer` (subclass of `TemplateServer`) → calls `.run()` +- `TemplateServer.__init__()` sets up middleware, rate limiting, metrics, and calls `setup_routes()` +- **Critical**: Middleware order matters - request logging → security headers → rate limiting +- **Extensibility**: Subclasses implement `setup_routes()` to add custom endpoints and `validate_config()` for config validation + +### Configuration System + +- `config.json` loaded via `TemplateServer.load_config()` method +- Validated using Pydantic models in `models.py` (TemplateServerConfig hierarchy) +- Subclasses override `validate_config()` to provide custom config models +- Logging configured automatically on `logging_setup.py` import with rotating file handler +- Environment variables stored in `.env` (API_TOKEN_HASH only, never commit) + +### Authentication Architecture + +- **Token Generation**: `uv run generate-new-token` creates secure token + SHA-256 hash +- **Hash Storage**: Only hash stored in `.env` (API_TOKEN_HASH), raw token shown once +- **Token Loading**: `load_hashed_token()` loads hash from .env on server startup, stored in `TemplateServer.hashed_token` +- **Verification Flow**: Request → `_verify_api_key()` dependency → `verify_token()` → hash comparison +- **Metrics**: Success/failure counters with labeled reasons (missing/invalid/error) +- **Health Endpoint**: `/api/health` does NOT require authentication, reports unhealthy if token not configured +- Header: `X-API-Key` (defined in `constants.API_KEY_HEADER_NAME`) + +### Rate Limiting + +- Uses `slowapi` with configurable storage (in-memory/Redis/Memcached) +- Applied via `_limit_route()` wrapper when `config.rate_limit.enabled=true` +- Custom exception handler increments `rate_limit_exceeded_counter` per endpoint +- Format: `"100/minute"` (supports /second, /minute, /hour) + +### Observability Stack + +- **Prometheus**: `/metrics` endpoint always exposed (no auth), custom auth/rate-limit metrics +- **Grafana**: Pre-configured dashboards in `grafana/dashboards/*.json` +- **Logging**: Dual output (console + rotating file), 10MB per file, 5 backups in `logs/` +- **Request Tracking**: `RequestLoggingMiddleware` logs all requests with client IP + +## Developer Workflows + +### Essential Commands + +```powershell +# Setup (first time) +uv sync # Install dependencies +uv run generate-certificate # Create self-signed SSL certs (certs/ dir) +uv run generate-new-token # Generate API key, save hash to .env + +# Development +uv run python-template-server # Start server (https://localhost:443/api) +uv run -m pytest # Run tests with coverage +uv run -m mypy . # Type checking +uv run -m ruff check . # Linting + +# Docker Development +docker compose up --build -d # Build + start all services +docker compose logs -f python-template-server # View logs +docker compose down # Stop and remove containers +``` + +### Testing Patterns + +- **Fixtures**: All tests use `conftest.py` fixtures, auto-mock `pyhere.here()` to tmp_path +- **Config Mocking**: Use fixtures for consistent test config +- **Integration Tests**: Test via FastAPI TestClient with auth headers +- **Coverage Target**: 99% (currently achieved) +- **Pattern**: Unit tests per module (test\_\*.py) + integration tests (test_template_server.py) + +### Docker Multi-Stage Build + +- **Stage 1 (builder)**: Uses `uv` to build wheel, copies `configuration/` directory and other required files +- **Stage 2 (runtime)**: Installs wheel, copies runtime files (.here, configs, LICENSE, README.md) from wheel to /app +- **Startup Script**: `/app/start.sh` generates token/certs if missing, starts server +- **Config Selection**: Uses `config.json` for all environments +- **Build Args**: `PORT=443` (exposes port) +- **Health Check**: Curls `/api/health` with unverified SSL context (no auth required) +- **User**: Switches to non-root user `templateserver` (UID 1000) + +## Project-Specific Conventions + +### Code Organization + +- **Handlers**: Separate modules for auth (`authentication_handler.py`), certs (`certificate_handler.py`) +- **Middleware**: Dedicated package `middleware/` with base classes extending `BaseHTTPMiddleware` +- **Constants**: All magic strings/numbers in `constants.py` (ports, file names, log config) +- **Models**: Pydantic models for config + API responses, use `@property` for derived values + +### Security Patterns + +- **Never log secrets**: Print tokens via `print()`, not `logger` (see `generate_new_token()`) +- **Path validation**: Use Pydantic validators, Path objects for cert paths +- **Security headers**: HSTS, CSP, X-Frame-Options via `SecurityHeadersMiddleware` +- **Cert generation**: RSA-4096, SHA-256, 365-day validity, SANs for localhost + +### API Design + +- **Prefix**: All routes under `/api` (API_PREFIX constant) +- **Authentication**: Applied via `dependencies=[Security(self._verify_api_key)]` in route registration +- **Unauthenticated Endpoints**: `/health` and `/metrics` do not require authentication +- **Response Models**: All endpoints return `BaseResponse` subclasses with code/message/timestamp +- **Health Status**: `/health` includes `status` field (HEALTHY/DEGRADED/UNHEALTHY), reports unhealthy if no token configured + +### Logging Format + +- Format: `[DD/MM/YYYY | HH:MM:SS] (LEVEL) module: message` +- Client IPs logged in requests: `"Request: GET /api/health from 192.168.1.1"` +- Auth failures: `"Invalid API key attempt!"` + +## Development Constraints + +### What's NOT Implemented Yet + +- Custom domain-specific endpoints (template provides base functionality only) +- Database/metadata storage (users implement as needed in subclasses) +- CORS configuration (can be added by subclasses) +- API key rotation/expiry +- Multi-user auth (JWT/OAuth2) + +### Testing Requirements + +- Mock `pyhere.here()` for all file path tests (see `conftest.py`) +- Use fixtures for TemplateServer/ExampleServer instantiation +- Test async endpoints with `@pytest.mark.asyncio` +- Mock `uvicorn.run` when testing server `.run()` methods + +### CI/CD Validation + +All PRs must pass: + +**CI Workflow:** + +1. `validate-pyproject` - pyproject.toml schema validation +2. `ruff` - linting (120 char line length, strict rules in pyproject.toml) +3. `mypy` - 100% type coverage (strict mode) +4. `pytest` - 99% code coverage, HTML report uploaded +5. `version-check` - pyproject.toml vs uv.lock version consistency + +**Docker Workflow:** + +1. `docker-development` - Build and test dev image with docker compose +2. `docker-production` - Build and test prod image with ENV=prod, PORT=443 + +## Quick Reference + +### Key Files + +- `template_server.py` - Base TemplateServer class with middleware/metrics/auth setup +- `main.py` - ExampleServer implementation showing how to extend TemplateServer +- `authentication_handler.py` - Token generation, hashing, verification +- `logging_setup.py` - Logging configuration (executed on import) +- `models.py` - All Pydantic models (config + responses) +- `constants.py` - Project constants, logging config +- `docker-compose.yml` - FastAPI + Prometheus + Grafana stack + +### Environment Variables + +- `API_TOKEN_HASH` - SHA-256 hash of API token (only var required) + +### Configuration Files + +- `configuration/config.json` - Configuration (used for all environments) +- `.env` - API token hash (auto-created by generate-new-token) +- **Docker**: Startup script uses config.json for all environments diff --git a/README.md b/README.md index 441716a..5d62c80 100644 --- a/README.md +++ b/README.md @@ -1,40 +1,128 @@ [![python](https://img.shields.io/badge/Python-3.12-3776AB.svg?style=flat&logo=python&logoColor=ffd343)](https://docs.python.org/3.12/) [![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv) [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) +[![FastAPI](https://img.shields.io/badge/FastAPI-Latest-009688?style=flat&logo=fastapi&logoColor=white)](https://fastapi.tiangolo.com/) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) -# Template Python Repository -This repository can be used as a template for a Python application. +# Python Template Server + +A production-ready FastAPI server template with built-in authentication, rate limiting, security headers, and Prometheus metrics. This repository provides a solid foundation for building secure, observable FastAPI applications. ## Table of Contents -- [uv](#uv) -- [Installing Dependencies](#installing-dependencies) -- [Testing, Linting, and Type Checking](#testing-linting-and-type-checking) +- [Features](#features) +- [Architecture](#architecture) +- [Quick Start](#quick-start) + - [Prerequisites](#prerequisites) + - [Installation](#installation) + - [Generate Certificates and API Token](#generate-certificates-and-api-token) + - [Run the Server](#run-the-server) +- [Using as a Template](#using-as-a-template) +- [Docker Deployment](#docker-deployment) +- [Documentation](#documentation) + +## Features + +- **TemplateServer Base Class**: Reusable foundation +- **FastAPI Framework**: Modern, fast, async-ready web framework +- **Observability Stack**: Pre-configured Prometheus + Grafana dashboards +- **Docker Support**: Multi-stage builds with docker-compose orchestration +- **Production Patterns**: Token generation, SSL certificate handling, health checks + +## Architecture + +This project uses a **`TemplateServer` base class** that encapsulates cross-cutting concerns: + +- **Request Logging**: All requests/responses logged with client IP tracking +- **Security Headers**: HSTS/CSP/X-Frame-Options automatically applied +- **API Key Verification**: SHA-256 hashed tokens with secure validation +- **Rate Limiting**: Configurable limits using `slowapi` (in-memory/Redis/Memcached) +- **Prometheus Metrics**: Custom authentication/rate-limit metrics + HTTP instrumentation + +**Application-specific servers** (like `ExampleServer` in `main.py`) extend `TemplateServer` to implement domain-specific endpoints and business logic. The base class handles all infrastructure concerns, letting you focus on your API functionality. + +## Quick Start -## uv -This repository is managed using the `uv` Python project manager: https://docs.astral.sh/uv/ +### Prerequisites -To install `uv`: +- Python 3.12+ +- [uv](https://docs.astral.sh/uv/) package manager + +Install `uv`: ```sh -curl -LsSf https://astral.sh/uv/install.sh | sh # Linux/Mac -powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" # Windows +# Linux/Mac +curl -LsSf https://astral.sh/uv/install.sh | sh + +# Windows (PowerShell) +powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" ``` -## Installing Dependencies -Install the required dependencies using `uv`: +### Installation + +```sh +# Clone the repository +git clone https://github.com/javidahmed64592/python-template-server.git +cd python-template-server + +# Install dependencies +uv sync --extra dev +``` - uv sync +### Generate Certificates and API Token -To install with `dev` dependencies: +```sh +# Generate self-signed SSL certificate (saves to certs/ directory) +uv run generate-certificate - uv sync --extra dev +# Generate API authentication token (saves hash to .env) +uv run generate-new-token +# ⚠️ Save the displayed token - you'll need it for API requests! +``` + +### Run the Server + +```sh +# Start the server +uv run python-template-server + +# Server runs at https://localhost:443/api +# Health check: curl -k https://localhost:443/api/health +# Metrics: curl -k https://localhost:443/api/metrics +``` + +## Using as a Template + +To create your own server: + +1. **Create a subclass of `TemplateServer`** (see `python_template_server/main.py:ExampleServer` as reference) +2. **Implement required methods**: + - `validate_config()`: Validate your config model + - `setup_routes()`: Define your API endpoints +3. **Add custom routes** using FastAPI decorators on `self.app` +4. **Configure** via `configuration/config.json` + +See the [Software Maintenance Guide](./docs/SMG.md) for detailed setup instructions. + +## Docker Deployment + +```sh +# Start all services (FastAPI + Prometheus + Grafana) +docker compose up -d + +# View logs +docker compose logs -f python-template-server + +# Access services: +# - API: https://localhost:443/api +# - Prometheus: http://localhost:9090 +# - Grafana: http://localhost:3000 (admin/admin) +``` -## Testing, Linting, and Type Checking +## Documentation -- **Run tests:** `uv run pytest` -- **Lint code:** `uv run ruff check .` -- **Format code:** `uv run ruff format .` -- **Type check:** `uv run mypy .` +- **[API Documentation](./docs/API.md)**: Endpoints, authentication, metrics +- **[Software Maintenance Guide](./docs/SMG.md)**: Development setup, configuration +- **[Docker Deployment Guide](./docs/DOCKER_DEPLOYMENT.md)**: Container orchestration +- **[Workflows](./docs/WORKFLOWS.md)**: CI/CD pipeline details diff --git a/docs/API.md b/docs/API.md new file mode 100644 index 0000000..87c36c1 --- /dev/null +++ b/docs/API.md @@ -0,0 +1,223 @@ + +# API + +This document summarizes the backend API provided by the Python Template Server. +All endpoints are mounted under the `/api` prefix. + +**Note**: The `TemplateServer` base class provides common behaviors (authentication, rate limiting, security headers, instrumentation) that are automatically applied to all endpoints. Application-specific servers (like `ExampleServer`) extend `TemplateServer` to add custom endpoints and business logic. + + +## Table of Contents +- [Authentication](#authentication) +- [Logging Configuration](#logging-configuration) +- [Request Logging](#request-logging) +- [Security Headers](#security-headers) +- [Rate Limiting](#rate-limiting) +- [Prometheus Metrics](#prometheus-metrics) + - [GET /api/metrics](#get-apimetrics) + - [Standard HTTP Metrics (via `prometheus-fastapi-instrumentator`)](#standard-http-metrics-via-prometheus-fastapi-instrumentator) + - [Custom Application Metrics](#custom-application-metrics) + - [Accessing Dashboards](#accessing-dashboards) + - [Prometheus Dashboard](#prometheus-dashboard) + - [Grafana Dashboards](#grafana-dashboards) +- [Endpoints](#endpoints) + - [GET /api/health](#get-apihealth) +- [Request and Response Models (Pydantic)](#request-and-response-models-pydantic) + +## Authentication + +All API endpoints require authentication via an API key passed in the `X-API-Key` header. + +**Request Header**: +``` +X-API-Key: your-api-token-here +``` + +**Error Responses**: +- `401 Unauthorized`: Missing or invalid API key + +## Logging Configuration + +The server uses Python's built-in logging system with both console and rotating file handlers for comprehensive log management. + +**Log Files**: +- Location: `logs/server.log` (relative to project root) +- Rotation: Automatic when file reaches 10 MB +- Backup Count: 5 backup files retained (e.g., `server.log.1`, `server.log.2`, etc.) +- Total Storage: Up to 60 MB (10 MB active + 5 × 10 MB backups) + +**Log Format**: +``` +[DD/MM/YYYY | HH:MM:SS] (LEVEL) module: message +``` + +**Example**: +``` +[22/11/2025 | 14:30:45] (INFO) template_server: Server starting on https://localhost:443/api +[22/11/2025 | 14:30:46] (INFO) request_logging_middleware: Request: GET /api/health from 192.168.1.100 +[22/11/2025 | 14:30:46] (INFO) request_logging_middleware: Response: GET /api/health -> 200 +``` + +## Request Logging + +All incoming requests and responses are automatically logged for monitoring and debugging purposes. + +**Logged Information**: +- **Request**: HTTP method, path, client IP address +- **Response**: HTTP method, path, status code +- **Authentication**: API key validation attempts + +**Log Levels**: +- `INFO`: Successful requests and responses +- `WARNING`: Authentication failures (missing or invalid API keys) +- `DEBUG`: Successful API key validations +- `ERROR`: Server errors and exceptions + +**Example Log Output**: +``` +INFO: Request: GET /api/health from 192.168.1.100 +DEBUG: API key validated successfully +INFO: Response: GET /api/health -> 200 +``` + +## Security Headers + +All API responses include security headers to protect against common web vulnerabilities: + +**Headers Included**: +- `Strict-Transport-Security`: Forces HTTPS connections (HSTS) +- `X-Content-Type-Options`: Prevents MIME-type sniffing +- `X-Frame-Options`: Prevents clickjacking attacks +- `Content-Security-Policy`: Controls which resources can be loaded +- `X-XSS-Protection`: Enables browser XSS filtering +- `Referrer-Policy`: Controls referrer information sent with requests + +**Configuration** (`config.json`): +```json +{ + "security": { + "hsts_max_age": 31536000, + "content_security_policy": "default-src 'self'" + } +} +``` + +- `hsts_max_age`: Duration in seconds that browsers should remember to only access the site via HTTPS (default: 1 year) +- `content_security_policy`: CSP directive controlling resource loading (default: only allow resources from same origin) + +## Rate Limiting + +API endpoints are rate-limited to prevent abuse. When the rate limit is exceeded, the server responds with: + +**Response**: +- Status Code: `429 Too Many Requests` +- Headers: `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset` + +Default rate limit: **100 requests per minute** per IP address. + +Rate limits can be configured in `config.json`. + +## Prometheus Metrics + +The server exposes Prometheus-compatible metrics for monitoring and observability. + +### GET /api/metrics + +- **Purpose**: Expose Prometheus metrics for scraping and monitoring. +- **Format**: Prometheus text-based exposition format. + +**Metrics Exposed**: + +#### Standard HTTP Metrics (via `prometheus-fastapi-instrumentator`) +- `http_requests_total`: Total number of HTTP requests by method, path, and status code +- `http_request_duration_seconds`: HTTP request latency histogram by method and path +- `http_requests_in_progress`: Number of HTTP requests currently being processed + +#### Custom Application Metrics + +**Authentication Metrics**: +- `auth_success_total`: Counter tracking successful API key validations +- `auth_failure_total{reason}`: Counter tracking failed authentication attempts with labels: + - `reason="missing"`: No API key provided in request + - `reason="invalid"`: Invalid or incorrect API key + - `reason="error"`: Error during token verification + +**Rate Limiting Metrics**: +- `rate_limit_exceeded_total{endpoint}`: Counter tracking requests that exceeded rate limits, labeled by endpoint path + +### Accessing Dashboards + +The application includes pre-configured monitoring dashboards for visualization: + +#### Prometheus Dashboard +- **URL**: http://localhost:9090 +- **Purpose**: Query and visualize raw metrics data +- **Features**: Built-in query interface, graphing, and alerting + +#### Grafana Dashboards +- **URL**: http://localhost:3000 +- **Credentials**: admin / admin (change after first login) +- **Pre-configured Dashboards**: + - **Authentication Metrics**: Tracks successful and failed authentication attempts, including reasons for failures + - **Rate Limiting Metrics**: Monitors requests that exceed rate limits by endpoint + +To access the dashboards, the containers for Grafana and Prometheus must be running. +See the [Docker documentation](./DOCKER_DEPLOYMENT.md) for information on how to run these. + +## Endpoints + +### GET /api/health + +**Purpose**: Simple health check of the server. + +**Authentication**: Not required (publicly accessible) + +**Rate Limiting**: Subject to rate limits (default: 100/minute) + +**Request**: None + +**Response Model**: `GetHealthResponse` +- `code` (int): HTTP status code +- `message` (string): Status message +- `timestamp` (string): ISO 8601 timestamp +- `status` (string): Health status indicator (HEALTHY/DEGRADED/UNHEALTHY) + +**Health Status Indicators**: +- `HEALTHY`: Server is fully operational and token is configured +- `UNHEALTHY`: Server token is not configured (returns 500 status code) + +**Example Request**: +```bash +curl -k https://localhost:443/api/health +``` + +**Example Response** (200 OK - Healthy): +```json +{ + "code": 200, + "message": "Server is healthy", + "timestamp": "2025-11-22T12:00:00.000000Z", + "status": "HEALTHY" +} +``` + +**Example Response** (500 Internal Server Error - Unhealthy): +```json +{ + "code": 500, + "message": "Server token is not configured", + "timestamp": "2025-11-22T12:00:00.000000Z", + "status": "UNHEALTHY" +} +``` + +## Request and Response Models (Pydantic) + +The primary Pydantic models are defined in `python_template_server/models.py`: +- `BaseResponse`: Base model with code, message, and timestamp fields +- `GetHealthResponse`: Extends BaseResponse with status field (HEALTHY/UNHEALTHY) +- `TemplateServerConfig`: Configuration model for server settings (security, rate limiting, JSON response) + +**Extending Configurations**: Extend the `TemplateServerConfig` class to get the necessary server setup configuration. + +**Extending Models**: When building your own server, create custom response models by extending `BaseResponse` for consistent API responses. diff --git a/docs/DOCKER_DEPLOYMENT.md b/docs/DOCKER_DEPLOYMENT.md new file mode 100644 index 0000000..69cd015 --- /dev/null +++ b/docs/DOCKER_DEPLOYMENT.md @@ -0,0 +1,254 @@ + +# Docker Deployment Guide + +This guide provides comprehensive instructions for deploying the Python Template Server using Docker and Docker Compose, including metrics visualization with Prometheus and Grafana. + + +## Table of Contents +- [Prerequisites](#prerequisites) + - [Check Prerequisites](#check-prerequisites) +- [Quick Start](#quick-start) + - [1. Generate API Key](#1-generate-api-key) + - [2. Start Services](#2-start-services) +- [Configuration](#configuration) + - [Docker Compose Services](#docker-compose-services) + - [Environment Variables](#environment-variables) + - [Server Configuration](#server-configuration) +- [Building and Running](#building-and-running) + - [Development Mode](#development-mode) + - [Managing Containers](#managing-containers) +- [Accessing Services](#accessing-services) + - [Python Template Server](#python-template-server) + - [Prometheus](#prometheus) + - [Grafana](#grafana) +- [Metrics Visualization](#metrics-visualization) + - [Available Metrics](#available-metrics) + - [Authentication Metrics](#authentication-metrics) + - [Rate Limiting Metrics](#rate-limiting-metrics) + - [HTTP Metrics (provided by prometheus-fastapi-instrumentator)](#http-metrics-provided-by-prometheus-fastapi-instrumentator) + - [Custom Dashboard Setup](#custom-dashboard-setup) + - [View Container Logs](#view-container-logs) + +## Prerequisites + +- **Docker**: Version 20.10 or higher +- **Docker Compose**: Version 2.0 or higher +- **API Key**: Generate using `generate-new-token` command (see below) + +### Check Prerequisites + +```bash +docker --version +``` + +## Quick Start + +### 1. Generate API Key + +Before starting the containers, you need to generate an API key: + +```bash +# Install the package locally (if not already installed) +uv sync + +# Generate a new API token (automatically creates/updates .env file) +uv run generate-new-token +``` + +This will: +- Generate a cryptographically secure **raw token** (displayed once - keep this secret!) +- Hash the token using SHA-256 +- Automatically save the hash to `.env` as `API_TOKEN_HASH` + +**Important**: +- Save the displayed raw token - you'll need it for authenticated API requests +- The `.env` file is automatically created/updated by the command +- **The same `.env` file is shared between local development and Docker containers**, allowing you to use the same token across both environments + +### 2. Start Services + +```bash +# Start all services (FastAPI server, Prometheus, Grafana) +docker compose up -d + +# View logs +docker compose logs -f + +# Stop services +docker compose down +``` + +**Token Management in Docker**: + +The Docker startup script automatically handles token generation with the following logic: + +1. **Existing `.env` file**: If you've already run `uv run generate-new-token` locally, the Docker container will use the existing token hash from your `.env` file. This means: + - You can generate a token once during development and use it consistently across local and Docker environments + - The same API token works for both `uv run python-template-server` and `docker compose up` + - No need to regenerate tokens when switching between environments + +2. **No `.env` file**: If no `.env` file exists (e.g., fresh checkout), the Docker startup script will automatically generate a new token. However: + - The auto-generated token is only displayed in the container logs + - It's recommended to generate tokens locally using `uv run generate-new-token` so you can save the raw token for API requests + +## Configuration + +### Docker Compose Services + +The `docker-compose.yml` defines three services: + +1. **python-template-server** (Port 443) + - FastAPI application with HTTPS + - Auto-generates self-signed certificates on first run (if not present) + - Uses existing `.env` file if available, otherwise generates a new token on startup + - Exposes `/api/metrics` endpoint for Prometheus + +2. **prometheus** (Port 9090) + - Metrics collection and storage + - Scrapes `/api/metrics` endpoint every 15 seconds + - Persistent storage via Docker volume + +3. **grafana** (Port 3000) + - Metrics visualization dashboards + - Pre-configured Prometheus datasource + - Custom dashboards for authentication and rate limiting + - Default credentials: `admin/admin` + +### Environment Variables + +Configure the FastAPI server using environment variables in `docker-compose.yml`: + +```yaml +environment: + - API_TOKEN_HASH=${API_TOKEN_HASH} +``` + +The `API_TOKEN_HASH` is loaded from your local `.env` file. +If the `.env` file exists when you run `docker compose up`, the container will use that token hash. +Otherwise, the container startup script will generate a new token and create the `.env` file. + +### Server Configuration + +Modify `config.json` to customize: + +- **Host and Port**: Change server binding address +- **Security Headers**: Configure HSTS and CSP policies +- **Rate Limiting**: Adjust rate limit rules +- **Certificates**: Set certificate validity period + +## Building and Running + +### Development Mode + +```bash +# Stop containers if required +docker compose down + +# Build and start in background +docker compose up --build -d +``` + +### Managing Containers + +```bash +# View running containers +docker compose ps + +# Stop services +docker compose stop + +# Start stopped services +docker compose start + +# Restart services +docker compose restart python-template-server + +# Remove containers and volumes +docker compose down -v +``` + +## Accessing Services + +### Python Template Server + +**Base URL**: `https://localhost:443` + +**API Endpoints**: +- Health Check: `GET /api/health` (publicly accessible, no authentication required) +- Metrics: `GET /api/metrics` (publicly accessible, no authentication required) +- Custom Endpoints: Defined in your server subclass (authentication may be required) + +**Example Request**: +```bash +# Using curl (with self-signed cert) +curl -k https://localhost:443/api/health + +# Authenticated request to custom endpoint +curl -k -H "X-API-Key: your-token-here" https://localhost:443/api/your-endpoint +``` + +### Prometheus + +**URL**: `http://localhost:9090` + +**Features**: +- Query metrics directly +- View scrape targets and status +- Create custom queries + +### Grafana + +**URL**: `http://localhost:3000` + +**Default Credentials**: +- Username: `admin` +- Password: `admin` (change on first login) + +**Pre-installed Dashboards**: +1. **Authentication Metrics** (`/d/auth-metrics`) + - Success/failure rates + - Total authentication attempts + - Failure reasons breakdown + - Success rate percentage + +2. **Rate Limiting & Performance** (`/d/rate-limit-metrics`) + - Rate limit violations by endpoint + - HTTP request rates + - Request duration percentiles + - Total violations gauge + +## Metrics Visualization + +### Available Metrics + +#### Authentication Metrics +- `auth_success_total`: Successful authentication attempts +- `auth_failure_total{reason}`: Failed attempts by reason (missing, invalid, error) + +#### Rate Limiting Metrics +- `rate_limit_exceeded_total{endpoint}`: Rate limit violations per endpoint + +#### HTTP Metrics (provided by prometheus-fastapi-instrumentator) +- `http_requests_total`: Total HTTP requests +- `http_request_duration_seconds`: Request latency histogram +- `http_requests_in_progress`: Current in-flight requests + +### Custom Dashboard Setup + +1. **Access Grafana**: Navigate to `http://localhost:3000` +2. **Login**: Use `admin/admin` +3. **Navigate**: Go to Dashboards → Browse → Python Template Server folder +4. **View**: Select either dashboard to visualize metrics + +### View Container Logs + +```bash +# All services +docker compose logs -f + +# Specific service +docker compose logs -f python-template-server + +# Last 100 lines +docker compose logs --tail=100 prometheus +``` diff --git a/docs/SMG.md b/docs/SMG.md new file mode 100644 index 0000000..0c59736 --- /dev/null +++ b/docs/SMG.md @@ -0,0 +1,143 @@ + +# Software Maintenance Guide + +This document outlines how to configure and setup a development environment to work on the Python Template Server application. + + +## Table of Contents +- [Backend (Python)](#backend-python) + - [Directory Structure](#directory-structure) + - [Architecture Overview](#architecture-overview) + - [Installing Dependencies](#installing-dependencies) + - [Setting Up Certificates and Authentication](#setting-up-certificates-and-authentication) + - [Generating SSL Certificates](#generating-ssl-certificates) + - [Generating API Authentication Tokens](#generating-api-authentication-tokens) + - [Running the Backend](#running-the-backend) + - [Testing, Linting, and Type Checking](#testing-linting-and-type-checking) + +## Backend (Python) + +[![Python](https://img.shields.io/badge/Python-3.12-3776AB?style=flat-square&logo=python&logoColor=ffd343)](https://docs.python.org/3.12/) +[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json&style=flat-square)](https://github.com/astral-sh/uv) +[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json&style=flat-square)](https://github.com/astral-sh/ruff) +[![FastAPI](https://img.shields.io/badge/FastAPI-Latest-009688?style=flat-square&logo=fastapi&logoColor=white)](https://fastapi.tiangolo.com/) + +### Directory Structure + +``` +python_template_server/ +├── middleware/ +│ ├── request_logging_middleware.py # Request logging +│ └── security_headers_middleware.py # Security headers +├── authentication_handler.py # Authentication (token generation/verification) +├── certificate_handler.py # SSL certificate generator +├── constants.py # Server constants +├── logging_setup.py # Logging configuration +├── main.py # Application entry point with ExampleServer +├── models.py # Pydantic models (config + API responses) +└── template_server.py # TemplateServer base class (reusable foundation) +``` + +### Architecture Overview + +The Python Template Server uses a **`TemplateServer` base class** that provides reusable infrastructure for building FastAPI applications: + +**TemplateServer Responsibilities:** +- **Middleware Setup**: Request logging and security headers +- **Authentication**: API key verification with SHA-256 hashing +- **Rate Limiting**: Configurable request throttling per endpoint +- **Metrics**: Prometheus instrumentation for observability +- **Configuration**: JSON-based config loading and validation + +**Application-Specific Servers** (like `ExampleServer` in `main.py`) extend `TemplateServer` to: +- Define custom API endpoints via `setup_routes()` +- Implement domain-specific business logic +- Validate custom configuration models via `validate_config()` + +This separation ensures that cross-cutting concerns (security, logging, metrics) are handled by the base class, while application developers focus on building their API functionality. + +### Installing Dependencies + +This repository is managed using the `uv` Python project manager: https://docs.astral.sh/uv/ + +To install `uv`: + +```sh +powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" # Windows +``` + +Install the required dependencies: + +```sh +uv sync +``` + +To include development dependencies: + +```sh +uv sync --extra dev +``` + +### Setting Up Certificates and Authentication + +Before running the server, you need to generate SSL certificates and an API authentication token. + +#### Generating SSL Certificates + +The server requires self-signed SSL certificates for HTTPS support: + +```sh +uv run generate-certificate +``` + +This command: +- Creates a self-signed certificate valid for 365 days +- Generates RSA-4096 key pairs +- Saves certificates to the `certs/` directory (`cert.pem` and `key.pem`) + +#### Generating API Authentication Tokens + +Generate a secure API token for authenticating requests: + +```sh +uv run generate-new-token +``` + +This command: +- Creates a cryptographically secure token using Python's `secrets` module +- Hashes the token with SHA-256 for safe storage +- Stores the hash in `.env` file +- Displays the plain token (save it securely - it won't be shown again) + +### Running the Backend + +Start the FastAPI server: + +```sh +uv run python-template-server +``` + +The backend will be available at `https://localhost:443/api` by default. + +**Available Endpoints:** +- Health Check: `https://localhost:443/api/health` +- Prometheus Metrics: `https://localhost:443/api/metrics` + +**Testing the API:** +```sh +# Health check (no auth required) +curl -k https://localhost:443/api/health + +# Metrics endpoint (no auth required) +curl -k https://localhost:443/api/metrics + +# Add custom authenticated endpoints in your server subclass +curl -k -H "X-API-Key: your-token-here" https://localhost:443/api/your-endpoint +``` + +### Testing, Linting, and Type Checking + +- **Run tests:** `uv run pytest` +- **Lint code:** `uv run ruff check .` +- **Format code:** `uv run ruff format .` +- **Type check:** `uv run mypy .` diff --git a/docs/TEMPLATE_SERVER_PACKAGE_NOTES.md b/docs/TEMPLATE_SERVER_PACKAGE_NOTES.md new file mode 100644 index 0000000..a97d703 --- /dev/null +++ b/docs/TEMPLATE_SERVER_PACKAGE_NOTES.md @@ -0,0 +1,228 @@ +# TemplateServer Extraction & Documentation Updates + +This file captures suggested documentation updates and a recommended migration plan for extracting the `TemplateServer` class from this repository into a standalone, reusable Python package. + +Summary +------- +- `TemplateServer` was extracted from `CloudServer` and moved to `python_cloud_server/template_server.py`. +- `CloudServer` now inherits from `TemplateServer`, which centralizes middleware, authentication verification, rate limiting and Prometheus metrics. +- The `TemplateServer` class is intended to be a reusable foundation for building FastAPI servers and will be published as a separate Python package in the future. + +Purpose of this document +------------------------ +- Provide a list of documentation and CI changes required to reflect the new project architecture. +- Offer exact wording snippets, file-level guidance and a migration checklist so the maintainers can implement everything once the package is published. + +Recommended package name (examples) +----------------------------------- +- `python-template-server` +- `fastapi-template-server` +- `template-server` (namespace choice depends on PyPI/ownership) + +If you have a specific package name you prefer, update the steps below accordingly. + +Files to update (anthology) +--------------------------- +Below are each of the doc files that should be updated, why an update is necessary, and an example patch or snippet you can apply. + +1) README.md +- Why: README is the main entry and should document the new architecture, the `TemplateServer` concept and the migration plan. +- Suggested snippets: + +Add under "Features": + +> - TemplateServer (reusable base): Consolidates request logging, security headers, rate limiting, Prometheus instrumentation, and token validation into a single base class. `CloudServer` now inherits from `TemplateServer` to define cloud-storage-specific endpoints and domain logic. + +Add a new "Architecture" section (after Features): + +``` +## Architecture + +The project uses a `TemplateServer` base class that encapsulates cross-cutting concerns: request logging, security headers (HSTS/CSP), API key verification, rate limiting, and Prometheus metrics. Application-specific server classes (e.g., `CloudServer`) extend `TemplateServer` to implement domain-specific endpoints and behavior. + +Optionally: the `TemplateServer` is planned to be extracted into a reusable package for publishing on PyPI or an internal registry; this repo may add it as a dependency and remove the local module in a follow-up. +``` + +2) docs/SMG.md (Software Maintenance Guide) +- Why: Directory listing and setup instructions should reflect the new module (and note the future package) + +Add to "Directory Structure": + +``` +python_cloud_server/ +├── template_server.py # Base server class (reusable). Note: may be published as separate package. +├── cloud_server.py # Cloud server subclass that defines cloud-specific endpoints via TemplateServer +├── authentication_handler.py # Authentication +├── certificate_handler.py # Certificate generator +├── config.py # Configuration management +├── constants.py # Server constants +├── main.py # Application entry point +└── models.py # Pydantic models +``` + +Add a short "Architecture - TemplateServer" section: + +``` +### Architectural Notes — TemplateServer + +`TemplateServer` centralizes middleware, authentication verification, rate limiting, and Prometheus instrumentation. `CloudServer` extends this to add cloud-storage-specific routes. The `TemplateServer` class will be extracted as a reusable package in a future release; when it is, update the repo's `pyproject.toml` to include it as a dependency. +``` + +3) docs/API.md +- Why: API docs should clarify that most cross-cutting functionality is implemented by `TemplateServer` (so users or maintainers know where that behavior comes from) + +Add a short note near the top: + +``` +Note: The `CloudServer` implementation inherits common behaviors (authentication, rate limiting, security headers, instrumentation) from a `TemplateServer` base class. These behaviors are applied by the base class to endpoints. +``` + +No endpoint behavior changes are required unless you change the public API. + +4) docs/DOCKER_DEPLOYMENT.md +- Why: Docker builds may need to install the new external package; mention packaging options and Docker build considerations. + +Add a short note under `Building and Running` or `Configuration`: + +``` +If the `TemplateServer` class is published as a separate package, ensure it is included in `pyproject.toml` and installed during `uv sync` (or `pip install`) so the Docker image contains the dependency. For local development, continue to ship the module in the repo as a development fallback until the package is published and version pinned. +``` + +5) docs/WORKFLOWS.md +- Why: CI should install the package if it becomes external and not part of the repo. + +Add or update steps in `validate-pyproject` and `test` jobs: + +``` +- Ensure `pyproject.toml` lists the extracted `TemplateServer` package (if it's published and required as an external dependency); CI will install it during `uv sync --extra dev`. +``` + +6) Optionally update `.github/cicd` and internal docs +- Why: If publishing the package, maintainers may want to add automation to build and publish the package, or update release notes and checkers to ensure compatibility. + +Add a short reference in relevant CI docs about publishing/pinning the `TemplateServer` package and the versioning policy for new releases. + + +Test updates +------------ +- If the `TemplateServer` package is published under a PyPI name and the same code is removed locally, tests must import `TemplateServer` from the published package (e.g., `from template_server_pkg import TemplateServer`) or install it in CI. +- Keep the tests referencing `python_cloud_server.template_server` while the module remains in the repo. +- After publishing, update tests to import from the package, or add an alias import in the project root to preserve compatibility during migration. + +Example update steps for tests + +- While TemplateServer is still in-repo (recommended during initial migration): no test changes necessary. +- After external package release: update test imports to point to the package or update `sys.path` to include the new package location in tests. + +Note: If you extract the module to a separate repo and publish the package, consider adding an integration test that installs the package in a virtualenv and runs server tests to validate parity. + +CI & Docker changes +------------------- +- CI job changes (if needed): + - Add the new package as a dependency in `pyproject.toml` lock file, or ensure CI runs `uv sync` to install it. + - If you publish a separate package, consider pinning the package in `pyproject.toml` and using a `pip` index or internal package registry for CI installs. + +- Docker build changes: + - Ensure `pyproject.toml` includes the package, so the builder installs it during the runtime stage. + - If you publish to an internal index or private registry, update Docker build steps to authenticate and install that package. + +Example `pyproject.toml` snippet to add: + +``` +# Add to your dependency list (example syntax may differ depending on packaging tool) +[tool.poetry.dependencies] +python-template-server = "^0.1.0" +``` + +Implementation & Migration Plan +------------------------------- +This migration plan aims to keep the repo functional during the extraction and publication process. + +1) Keep `TemplateServer` local until first package release + - Rationale: Minimizes immediate refactor changes for tests and deployment. + - Action: Keep `python_cloud_server/template_server.py` in the repo and continue using the local import in the `CloudServer` module. + +2) Publish the package + - Prepare a simple package skeleton and tests in the `template-server` repo (or the new package's repo) + - Publish the package to PyPI (or your private registry) with a version, e.g., `0.1.0` + +3) Add the package to `pyproject.toml` and updates images/CI + - Update `pyproject.toml` to include the new package as a dependency. + - Update Dockerfile and CI workflows if you're using a private registry or the package has specific installation needs. + +4) Remove local module (optional and final step) + - Remove `python_cloud_server/template_server.py` from the cloud server repo after a new release or a deprecation window. + - Update tests and imports to refer to the package instead of local module. + +5) Update docs & README + - Apply the docs changes outlined in this document (snippets above). + - Add a migration note describing the package ZIP and version. + +6) Update CI & workflows to install the package in the environment + - Ensure `uv sync` installs new package; update reusable steps that run `uv run -m pytest` accordingly. + +7) Verify everything with tests + - Run unit and integration tests locally or in the CI environment. + - For Docker, build the image with the package installed and run the health checks. + +PR checklist for implementation +------------------------------ +- [ ] Add `TemplateServer` package to `pyproject.toml` (once published) +- [ ] Add `TemplateServer` architecture text to `README.md` (snippet added above) +- [ ] Add `template_server.py` note to `docs/SMG.md` +- [ ] Add short note to `docs/API.md` indicating behavior comes from `TemplateServer` +- [ ] Add a Docker note to `docs/DOCKER_DEPLOYMENT.md` if the package will be installed +- [ ] Update `docs/WORKFLOWS.md` to mention unit/CI installation +- [ ] Update unit tests to import from the final package (or add alias compatibility) after package publication +- [ ] Validate the new architecture and docs in CI & Docker builds +- [ ] Add a short example `pyproject` dependency snippet to docs (seen above) + +Sample docs patches (copy-paste) +------------------------------- +1) README.md Feature bullet: + +> - TemplateServer (reusable base): Consolidates request logging, security headers, rate limiting, Prometheus instrumentation, and token verification into a base class. `CloudServer` inherits from `TemplateServer` and focuses on cloud-specific endpoints. + +2) README.md Architecture section: + +``` +## Architecture + +The project uses a `TemplateServer` base class that encapsulates cross-cutting concerns: request logging, security headers (HSTS/CSP), API key verification, rate limiting, and Prometheus metrics. Application-specific servers (e.g., `CloudServer`) extend `TemplateServer` to implement domain-specific endpoints and behavior. + +When `TemplateServer` is released as a separate package, this repo will add it as a dependency and use that package instead of the local module. +``` + +3) docs/SMG.md Directory Structure addition: + +``` +python_cloud_server/ +├── template_server.py # Base server class (reusable) - may be published separately +├── cloud_server.py # Cloud server subclass that extends TemplateServer +``` + +4) docs/API.md mention: + +``` +Note: The `CloudServer` implementation inherits common behaviors (authentication, rate limiting, security headers, instrumentation) from a `TemplateServer` base class. These behaviors are automatically applied to endpoints. +``` + +Misc notes & tips +----------------- +- Keep the repo's tests referencing `TemplateServer` while it stays local to avoid large refactoring work. +- When publishing the package, consider a small compatibility release (e.g., `0.1.0`) and maintain a deprecation note for the local module before removing it. +- If you use an internal package feed, store credentials in CI secret stores and update Docker build steps for authentication. + +Questions for you +------------------ +- Do you already have a package name and PyPI or internal registry planned for `TemplateServer`? If yes, I will include explicit package name references in this file. +- Do you prefer to keep `TemplateServer` as a local module until the package is published? (Strongly recommended during extraction) + +If you'd like, I can now: +- Create these doc updates directly in the repo as a PR (I will base the changes on the `extract-template-server` branch), OR +- Keep this document as is for you to implement once the package is ready. + +Notes +----- +- This document intentionally includes copy-paste ready snippets and a migration checklist to speed up applying updates once the `TemplateServer` package is published. +- If you want me to produce a PR with these doc updates or update tests to reflect the external package after publishing, tell me the package name and the preferred release cadence. diff --git a/docs/WORKFLOWS.md b/docs/WORKFLOWS.md new file mode 100644 index 0000000..6d34f46 --- /dev/null +++ b/docs/WORKFLOWS.md @@ -0,0 +1,59 @@ +# GitHub Workflows + +This document details the CI/CD workflows to build and release the Python Template Server application. +They run automated code quality checks to ensure code remains robust, maintainable, and testable. + +## CI Workflow + +The CI workflow runs on pushes and pull requests to the `main` branch. +It consists of the following jobs: + +### validate-pyproject +- Checkout code +- Install uv with caching +- Set up Python from `.python-version` +- Install dependencies with `uv sync --extra dev` +- Validate `pyproject.toml` using `uv run validate-pyproject pyproject.toml` + +### ruff +- Checkout code +- Run Ruff linter using `chartboost/ruff-action@v1` + +### mypy +- Checkout code +- Install uv with caching +- Set up Python from `.python-version` +- Install dependencies with `uv sync --extra dev` +- Run mypy type checking with `uv run -m mypy .` + +### test +- Checkout code +- Install uv with caching +- Set up Python from `.python-version` +- Install dependencies with `uv sync --extra dev` +- Run pytest with coverage report using `uv run -m pytest --cov-report html` +- Upload coverage report as artifact + +## Docker Workflow + +The Docker workflow runs on pushes and pull requests to the `main` branch. +It consists of the following jobs: + +### docker-development +- Checkout code +- Install uv with caching and set up Python from `.python-version` +- Create directories with proper permissions +- Build and start services with docker compose +- Show server logs +- **Health check** using reusable composite action `.github/actions/docker-check-containers` that checks Python Template Server, Prometheus, and Grafana +- Stop services + +### docker-production +- Checkout code +- Install uv with caching and set up Python from `.python-version` +- Create directories with proper permissions +- Build production image with `ENV=prod` and `PORT=443` build arguments +- Start services with docker compose using production environment variables +- Show server logs +- **Health check** using reusable composite action `.github/actions/docker-check-containers` that checks Python Template Server, Prometheus, and Grafana +- Stop services diff --git a/grafana/README.md b/grafana/README.md new file mode 100644 index 0000000..3b13fdb --- /dev/null +++ b/grafana/README.md @@ -0,0 +1,135 @@ + +# Grafana Configuration + +This directory contains Grafana provisioning configuration and custom dashboards for the Python Template Server. + + +## Table of Contents +- [Directory Structure](#directory-structure) +- [Dashboards](#dashboards) + - [1. Health Metrics Dashboard](#1-health-metrics-dashboard) + - [2. Authentication Metrics Dashboard](#2-authentication-metrics-dashboard) + - [3. Rate Limiting \& Performance Metrics Dashboard](#3-rate-limiting--performance-metrics-dashboard) +- [Accessing Dashboards](#accessing-dashboards) +- [Customizing Dashboards](#customizing-dashboards) + - [Adding New Panels](#adding-new-panels) + - [Creating New Dashboards](#creating-new-dashboards) +- [Available Metrics](#available-metrics) + - [Health Metrics](#health-metrics) + - [Authentication Metrics](#authentication-metrics) + - [Rate Limiting Metrics](#rate-limiting-metrics) + - [HTTP Metrics (from prometheus-fastapi-instrumentator)](#http-metrics-from-prometheus-fastapi-instrumentator) + + +## Directory Structure + +``` +grafana/ +├── provisioning/ +│ ├── datasources/ +│ │ └── prometheus.yml # Prometheus datasource configuration +│ └── dashboards/ +│ └── dashboards.yml # Dashboard provisioning configuration +└── dashboards/ + ├── authentication-metrics.json # Authentication monitoring dashboard + ├── health-metrics.json # Health monitoring dashboard + └── rate-limiting-metrics.json # Rate limiting & performance dashboard +``` + +## Dashboards + +### 1. Health Metrics Dashboard + +**UID**: `health-metrics` +**Path**: `/d/health-metrics` + +**Panels**: +- **API Token Configuration Status**: Gauge showing if API token is configured +- **Health Checks (Last 5 Minutes)**: Gauge of recent health check requests +- **Health Check Average Response Time**: Gauge of average response time for health checks +- **Token Configuration Status Over Time**: Timeseries of token configuration status +- **Health Check Request Rate (per second)**: Timeseries of health check request rates +- **Health Check Response Time Percentiles**: Timeseries of p50, p95, p99 response times + +**Use Cases**: +- Monitor server health and configuration status +- Track health check performance and frequency +- Detect configuration issues (e.g., missing API token) + +### 2. Authentication Metrics Dashboard + +**UID**: `auth-metrics` +**Path**: `/d/auth-metrics` + +**Panels**: +- **Authentication Rate**: Success and failure rates per second +- **Total Successful Authentications**: Gauge showing cumulative successes +- **Total Failed Authentications**: Gauge showing cumulative failures +- **Authentication Failures by Reason**: Breakdown of failures (missing, invalid, error) +- **Authentication Success Rate**: Percentage of successful attempts + +**Use Cases**: +- Monitor authentication health +- Detect brute force attacks (high failure rates) +- Identify common authentication issues + +### 3. Rate Limiting & Performance Metrics Dashboard + +**UID**: `rate-limit-metrics` +**Path**: `/d/rate-limit-metrics` + +**Panels**: +- **Rate Limit Exceeded Events**: Rate of violations per second by endpoint +- **Total Rate Limit Violations**: Cumulative violation count +- **Rate Limit Violations by Endpoint**: Breakdown by endpoint +- **HTTP Request Rate**: Overall request rates by method, handler, and status +- **HTTP Request Duration**: 95th and 99th percentile latency + +**Use Cases**: +- Monitor rate limit effectiveness +- Identify endpoints being abused +- Track API performance and latency +- Capacity planning + +## Accessing Dashboards + +1. Generate API key (if not done): `uv run generate-new-token` +2. Start services: `docker compose up -d` +3. Open Grafana: http://localhost:3000 +4. Login with default credentials: `admin/admin` +5. Navigate to: Dashboards → Browse → Python Template Server folder + +## Customizing Dashboards + +### Adding New Panels + +1. Open a dashboard in Grafana +2. Click "Add panel" → "Add a new panel" +3. Configure visualization and query +4. Save the dashboard +5. Export JSON: Dashboard settings → JSON Model +6. Save to this directory for version control + +### Creating New Dashboards + +1. Create dashboard in Grafana UI +2. Export as JSON: Dashboard settings → JSON Model → Copy to clipboard +3. Save JSON file in `grafana/dashboards/` +4. Restart Grafana: `docker compose restart grafana` + +## Available Metrics + +### Health Metrics +- `token_configured` - Binary metric indicating if API token is configured (0 or 1) + +### Authentication Metrics +- `auth_success_total` - Successful authentication count +- `auth_failure_total{reason="missing|invalid|error"}` - Failed authentication count by reason + +### Rate Limiting Metrics +- `rate_limit_exceeded_total{endpoint="/api/health"}` - Rate limit violations per endpoint + +### HTTP Metrics (from prometheus-fastapi-instrumentator) +- `http_requests_total{method, handler, status}` - Total requests +- `http_request_duration_seconds_bucket` - Request duration histogram +- `http_requests_in_progress` - Current active requests