feat: Set up comprehensive Python testing infrastructure with Poetry

[llbbl]

Set Up Python Testing Infrastructure

Summary

This PR establishes a comprehensive testing infrastructure for the RecurrentGPT project using Poetry as the package manager and pytest as the testing framework. The setup provides a solid foundation for writing and maintaining tests throughout the project lifecycle.

Changes Made

Package Management

• Poetry Configuration: Created pyproject.toml with Poetry as the package manager
• Dependency Migration: Migrated all project dependencies (openai, gradio, sentence-transformers) to Poetry
• Development Dependencies: Added pytest, pytest-cov, and pytest-mock as development dependencies

Testing Framework Setup

• Directory Structure:

  • Created tests/ directory with proper Python package initialization
  • Added tests/unit/ and tests/integration/ subdirectories for organized test structure

• Pytest Configuration:

  • Configured test discovery patterns
  • Set up coverage reporting with 80% threshold requirement
  • Added HTML and XML coverage report generation
  • Created custom test markers: @pytest.mark.unit, @pytest.mark.integration, @pytest.mark.slow

• Coverage Configuration:

  • Configured source directories and exclusion patterns
  • Set up branch coverage tracking
  • Configured coverage report formats and output directories

Shared Test Fixtures

Created tests/conftest.py with comprehensive fixtures including:

• temp_dir: Temporary directory management
• mock_config: Mock configuration data
• mock_init_prompt: Mock prompt JSON data
• mock_paragraphs: Mock paragraph generation results
• mock_openai_response: Mock OpenAI API responses
• mock_embedder: Mock SentenceTransformer
• test_file: Temporary file creation helper
• mock_env_vars: Environment variable mocking
• mock_writer_input & mock_human_input: Mock data for core components
• cleanup_cache: Automatic cache cleanup between tests
• mock_gradio_request: Mock Gradio request objects
• capture_stdout: Stdout capture for testing print statements

Additional Changes

• Updated .gitignore: Added comprehensive ignore patterns for:

  • Python artifacts and cache files
  • Testing and coverage reports
  • Virtual environments
  • IDE files
  • Claude settings (.claude/*)

• Bug Fix: Fixed missing os import in utils.py

• Validation Tests: Created tests/test_setup_validation.py to verify the testing infrastructure works correctly

How to Use

Running Tests

# Install dependencies
poetry install

# Run all tests
poetry run pytest

# Alternative commands (both work)
poetry run test
poetry run tests

# Run specific test file
poetry run pytest tests/test_setup_validation.py

# Run tests with specific markers
poetry run pytest -m unit
poetry run pytest -m integration
poetry run pytest -m "not slow"

# Run tests with coverage report
poetry run pytest --cov-report=html

Writing New Tests

1. Unit Tests: Place in tests/unit/ directory
2. Integration Tests: Place in tests/integration/ directory
3. Use Fixtures: Leverage the shared fixtures in conftest.py
4. Mark Tests: Use appropriate markers (@pytest.mark.unit, etc.)

Example test structure:

import pytest
from tests.conftest import *  # Import fixtures

@pytest.mark.unit
def test_example_function(mock_config, temp_dir):
    # Your test code here
    assert True

Notes

• The coverage threshold is set to 80% - tests will fail if coverage drops below this
• Coverage reports are generated in both HTML (htmlcov/) and XML (coverage.xml) formats
• The testing infrastructure is ready for immediate use - developers can start writing tests right away
• All pytest standard options and plugins are available for use

Next Steps

With this testing infrastructure in place, the team can now:

1. Write unit tests for individual functions and classes
2. Create integration tests for component interactions
3. Add end-to-end tests for complete workflows
4. Monitor and improve code coverage
5. Set up CI/CD pipelines with test execution