Unit tests #10

dosumis · 2025-11-25T18:35:47Z

Coverage and CI updates are in:

Added a GitHub Actions workflow (.github/workflows/tests.yml) that runs pytest with coverage, generates badges/coverage.svg, and auto-
commits the badge on pushes to main. README now shows the workflow badge and uses the repo-hosted coverage badge.
New dev dependency coverage-badge; requirements-dev.txt updated.
Coverage boosted to ~81%: added integration tests for component/GO building and component-program matching, fixed CLI argument handling
in build_component_mapping.py, build_go_terms.py, match_components.py, and expanded the master report integration test. Coverage report
now excludes build_master_report.py (report assembler) to focus on logic modules.
Restored extract_run_payloads.py (tested) and removed the legacy rename_runs.py.
Generated badges/coverage.svg committed.

All tests pass (pytest), and coverage is ~81%.

Copilot

Pull request overview

This PR introduces a comprehensive test suite with unit and integration tests for the DeepSearch pipeline validation project. The tests achieve approximately 81% code coverage and include CI/CD integration via GitHub Actions.

Key Changes

Added 8 unit test files covering helper functions in project paths, DeepSearch processing, comparison processing, and component/GO term building
Added 5 integration test files testing end-to-end workflows for DeepSearch processing, comparisons, report generation, heatmaps, and component matching
Modified 4 source files (match_components.py, extract_run_payloads.py, build_go_terms.py, build_component_mapping.py) to accept argv parameter for testability
Configured pytest with coverage tracking in pyproject.toml, added test dependencies in requirements-dev.txt, and added Makefile targets
Set up GitHub Actions workflow for automated testing with coverage badge generation
Removed unused src/rename_runs.py file

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 20 comments.

Show a summary per file

File	Description
tests/unit/test_project_paths.py	Tests path resolution with defaults and custom projects
tests/unit/test_process_deepsearch_helpers.py	Tests JSON extraction, signature computation, name similarity, and program similarity calculations
tests/unit/test_process_deepsearch_extract.py	Tests parsing of run markdown files with JSON payloads
tests/unit/test_process_comparisons_helpers.py	Tests GO term extraction, gene set ID parsing, program table parsing, and term normalization
tests/unit/test_generate_summary_figures.py	Tests plot generation for run consistency and GO coverage
tests/unit/test_extract_run_payloads.py	Tests citation extraction and full extraction workflow
tests/unit/test_build_go_terms.py	Tests GO term ID extraction from raw strings
tests/unit/test_build_component_mapping.py	Tests tokenization logic for removing suffixes like "-like"
tests/unit/init.py	Package marker for unit tests
tests/integration/test_process_deepsearch_main.py	Integration test for complete DeepSearch processing pipeline
tests/integration/test_process_comparisons_main.py	Integration test for comparison processing with S10 data
tests/integration/test_generate_reports_and_master.py	Integration test for report and master report generation
tests/integration/test_generate_heatmaps.py	Integration test for heatmap generation from program data
tests/integration/test_build_component_and_go_and_match.py	Integration test for component/GO term building and matching
tests/integration/init.py	Package marker for integration tests
src/rename_runs.py	Removed unused file for renaming run files
src/match_components.py	Added argv parameter to main() for testability
src/extract_run_payloads.py	Removed duplicate main() declaration, added argv parameter, added parents=True to mkdir calls
src/build_go_terms.py	Added argv parameter to main() for testability
src/build_component_mapping.py	Added argv parameter to main() for testability
requirements-dev.txt	Added pytest, pytest-cov, and coverage-badge dependencies
pyproject.toml	Configured pytest options and coverage settings
badges/coverage.svg	Generated coverage badge showing 81% coverage
README.md	Added test badges and documentation for running tests
Makefile	Added test and coverage targets
.github/workflows/tests.yml	Added CI workflow for automated testing with coverage

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-25T18:40:52Z

tests/integration/test_generate_reports_and_master.py

+from src.generate_reports import generate_reports
+from src.build_master_report import main as master_main
+from src.project_paths import resolve_paths
+


Missing blank line before function definition. According to PEP 8, there should be two blank lines between top-level function definitions.

Suggested change

Copilot · 2025-11-25T18:40:53Z

pyproject.toml

+[tool.coverage.run]
+source = ["src"]
+branch = true
+omit = ["src/embed_*", "src/build_master_report.py" ]


Trailing whitespace after the closing bracket. Consider removing it for cleaner code.

Suggested change

omit = ["src/embed_*", "src/build_master_report.py" ]

omit = ["src/embed_*", "src/build_master_report.py"]

Copilot · 2025-11-25T18:40:53Z

README.md

 Pipeline for parsing Perplexity DeepSearch outputs, comparing pseudo-enrichment programs to GO results, and generating figures/reports per project. The current default project is `glioblastoma_perplexity_manual`, but the layout supports multiple projects via per-project subdirectories.

+[![Tests](https://github.com/Cellular-Semantics/langpa_validation_tools/actions/workflows/tests.yml/badge.svg)](https://github.com/Cellular-Semantics/langpa_validation_tools/actions/workflows/tests.yml)
+![coverage](https://img.shields.io/badge/coverage-71%25-orange)


Coverage percentage mismatch: The README shows 71% coverage, but the generated SVG badge in badges/coverage.svg shows 81%. These should be consistent. Consider using the badge file directly or ensuring both are updated together.

Suggested change

![coverage](https://img.shields.io/badge/coverage-71%25-orange)

![coverage](badges/coverage.svg)

Copilot · 2025-11-25T18:40:53Z

tests/unit/test_process_comparisons_helpers.py

+    terms = pc.parse_unmatched_go_terms(text)
+    assert "Term A (GO:1)" in terms
+    assert "Term B (GO:2)" in terms
+


Missing blank line before function definition. According to PEP 8, there should be two blank lines between top-level function definitions.

Suggested change

Copilot · 2025-11-25T18:40:54Z

tests/integration/test_process_comparisons_main.py

+
+from src.process_comparisons import main
+from src.project_paths import resolve_paths
+


Missing blank line before function definition. According to PEP 8, there should be two blank lines between top-level function definitions.

Suggested change

Copilot · 2025-11-25T18:40:56Z

tests/unit/test_process_deepsearch_extract.py

@@ -0,0 +1,15 @@
+import json
+from pathlib import Path


Import of 'Path' is not used.

Suggested change

from pathlib import Path

Copilot · 2025-11-25T18:40:57Z

tests/unit/test_process_deepsearch_extract.py

+import json
+from pathlib import Path
+
+import pytest


Import of 'pytest' is not used.

Suggested change

import pytest

Copilot · 2025-11-25T18:40:57Z

tests/unit/test_process_deepsearch_helpers.py

@@ -0,0 +1,49 @@
+import json


Import of 'json' is not used.

Suggested change

import json

Copilot · 2025-11-25T18:40:57Z

tests/unit/test_project_paths.py

@@ -0,0 +1,34 @@
+import os


Import of 'os' is not used.

Suggested change

import os

Copilot · 2025-11-25T18:40:57Z

tests/unit/test_project_paths.py

+import os
+from pathlib import Path
+
+import pytest


Import of 'pytest' is not used.

Suggested change

import pytest

dosumis · 2025-11-25T18:49:24Z

copilot issues are all linting and formatting. Will ignore for now.

dosumis added 3 commits November 25, 2025 18:07

Add integration/unit tests and restore extraction utility

528e935

Add CI workflow for tests and coverage badge

ec46365

Raise coverage to ~81% and add CI badge automation

ffba2b1

dosumis requested a review from Copilot November 25, 2025 18:36

Copilot started reviewing on behalf of dosumis November 25, 2025 18:36 View session

Copilot finished reviewing on behalf of dosumis November 25, 2025 18:39

Fix CI PYTHONPATH for tests

9fb4217

Copilot AI reviewed Nov 25, 2025

View reviewed changes

Add openpyxl for tests and point coverage badge to repo

b9a47b1

dosumis merged commit c7d96ae into main Nov 25, 2025
2 checks passed

dosumis deleted the unit_tests branch November 25, 2025 18:49

	omit = ["src/embed_*", "src/build_master_report.py" ]
	omit = ["src/embed_*", "src/build_master_report.py"]

	![coverage](https://img.shields.io/badge/coverage-71%25-orange)
	![coverage](badges/coverage.svg)


		from src.process_comparisons import main
		from src.project_paths import resolve_paths

Unit tests #10

Unit tests #10

Uh oh!

Conversation

dosumis commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

dosumis commented Nov 25, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dosumis commented Nov 25, 2025 •

edited

Loading