Skip to content

testing.md

Latest commit

 

History

History
74 lines (48 loc) · 1.76 KB

testing.md

File metadata and controls

74 lines (48 loc) · 1.76 KB

Testing Guide

This guide covers running and extending the NeoBERT regression suite.

Run Tests

Full suite

conda run --name neobert pytest -q

Or via helper:

conda run --name neobert python tests/run_tests.py

Subsets

# One file
conda run --name neobert pytest tests/kernels/test_attention.py -q

# One directory
conda run --name neobert pytest tests/training -q

# Verbose investigation
conda run --name neobert pytest tests/test_model_forward.py -vv --showlocals

Helper flags

conda run --name neobert python tests/run_tests.py --test-dir training
conda run --name neobert python tests/run_tests.py --pattern "test_*compile*.py"
conda run --name neobert python tests/run_tests.py --no-pytest

Manual Validation Scripts

tests/manual/ contains opt-in validation/benchmark scripts and is excluded from default pytest -q discovery.

Manual script commands are in tests/manual/README.md.

Test Authoring Guidelines

  • Prefer tiny configs in tests/configs/.
  • Keep tests deterministic and local (avoid network where possible).
  • Disable external logging for training tests (wandb.mode: disabled).
  • Guard GPU-only assertions with torch.cuda.is_available().
  • For performance-sensitive paths (packing/compile), include regression tests for both correctness and expected control-flow behavior.

Common Failures

  1. Import errors
  • install editable package (pip install -e .[dev]).
  1. Device mismatches
  • ensure tensors and models are on the same device in assertions.
  1. Slow tests
  • lower steps/batch sizes and use tiny configs.

Related Docs