feat(report): per-scan summary of skipped paths

Walking system roots as a non-root user routinely hits inaccessible
sub-trees and unreadable files. Each occurrence is correct
best-effort behaviour, but in aggregate the operator should see how
much of the filesystem was actually inspected — a partial scan from
missing privileges is not a clean scan.

New `skip_report.py` module with a `SkipReport` singleton that the
walk / read helpers append to whenever they swallow PermissionError
or OSError. Renderer in report.py prints a yellow block at the end
of the scan listing the first few paths in each category and the
total counts, with a hint to re-run with elevated privileges. Silent
when no paths were skipped.

Instrumented sites:
  * config.pruned_walk — now uses os.walk's onerror callback so
    depth-N permission denials are recorded, not just top-level
    root denials (os.walk silently drops per-subdirectory errors
    without onerror set).
  * config.read_if_contains
  * git_repo_index._find_repo_roots (same onerror treatment) and
    its targeted file readers (description, refs/heads, packed-refs,
    workflow files).
  * persistence_scanner (config-dir glob, shell-rc read, /tmp iter).
  * ioc_scanner (known-paths exists check, walk-files hash-read).

Intentionally not instrumented:
  * subprocess_utils.run_safe — tool-availability is already loud
    where the operator needs to know.
  * network_scanner /proc/$PID/exe reads — PermissionError there is
    the normal case for other users' PIDs and would flood the
    summary on any multi-user host.
  * json.JSONDecodeError / ast.SyntaxError on third-party files —
    different category (parse failure of code we don't own), not
    a "skipped path" in the user-facing sense.

Test isolation: new autouse fixture in conftest.py resets the
singleton before and after every test. Live PermissionError tests
guard with `pytest.skip` when running as root, since POSIX
permission bits don't apply.

Smoke test on a real Linux machine: 65 paths recorded (mostly
/opt/containerd, /var/cache/*, /var/log/journal/*) — all expected
for a non-root scan.

408 tests pass (was 398).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: Robcs01Bot

CHANGELOG.md

@@ -2,6 +2,9 @@
 
 ## Unreleased
 
+### Added
+- **Per-scan skip summary.** New `skip_report.py` module with a `SkipReport` singleton that the walk / read helpers append to whenever they swallow a `PermissionError` or `OSError`. Instrumented sites: `config.pruned_walk` (with `onerror` callback so depth-N denials are caught, not just the top-level root), `config.read_if_contains`, `git_repo_index._find_repo_roots` and its targeted file readers, `persistence_scanner` (config-dir, shell-rc, /tmp iteration), and `ioc_scanner` (known-paths exists check + walk-files hash-read). A new `print_skip_summary` block at the end of the report tells the operator how many paths were not inspected and shows the first few of each category, so a partial scan from missing privileges is no longer indistinguishable from a clean scan. Silent when no paths were skipped. Test isolation handled by an autouse fixture in `tests/conftest.py` that resets the singleton between tests.
+
 ### Changed
 - **Fail loud on malformed threat profiles.** `_load_from_dir` no longer swallows `KeyError` / `tomllib.TOMLDecodeError` / `re.error` from individual profiles. Any broken TOML — missing required field, syntax error, or invalid regex — now raises `InvalidThreatProfileError` with the offending file path and stops the scan. A user who wrote a profile expects it to be active; silently logging at WARNING and continuing was the kind of stealth-failure mode that §8 (Fail Fast, Fail Loud) explicitly warns against.
 - **Regex compilation moved from scan time to load time.** `GitArtifactsIOC.workflow_name_regexes` and `branch_name_regexes` are now `tuple[re.Pattern[str], ...]` (was `tuple[str, ...]`). `_parse_git_artifacts` compiles each pattern; a malformed pattern raises `re.error` with the field name and the offending pattern. The `aggregate_indicators` function no longer needs `_compile_into` or any exception handling — patterns are pre-validated.

scan_supply_chain/config.py

@@ -12,6 +12,8 @@
 from collections.abc import Generator
 from pathlib import Path
 
+from .skip_report import note_permission_error, note_read_error
+
 logger = logging.getLogger(__name__)
 
 # Directories always skipped during any filesystem walk
@@ -59,19 +61,36 @@ def read_if_contains(path: Path, keyword: str) -> str | None:
         return None
     try:
         text = path.read_text(errors="ignore")
-    except (PermissionError, OSError):
-        logger.debug("Cannot read %s", path)
+    except PermissionError:
+        note_permission_error(path)
+        return None
+    except OSError as exc:
+        note_read_error(path, type(exc).__name__)
         return None
     return text if keyword in text else None
 
 
 def pruned_walk(
     root: Path, skip_dirs: frozenset[str]
 ) -> Generator[tuple[str, list[str], list[str]], None, None]:
-    """os.walk with directory pruning and PermissionError handling."""
+    """os.walk with directory pruning and skip-report instrumentation.
+
+    Per-directory ``OSError``s (typically ``PermissionError`` on
+    inaccessible sub-trees) are routed through the ``onerror`` callback
+    so each one is recorded individually — by default ``os.walk``
+    silently drops them.
+    """
+
+    def _on_error(exc: OSError) -> None:
+        path = Path(exc.filename) if exc.filename else root
+        if isinstance(exc, PermissionError):
+            note_permission_error(path)
+        else:
+            note_read_error(path, type(exc).__name__)
+
     try:
-        for dirpath, dirnames, filenames in os.walk(root):
+        for dirpath, dirnames, filenames in os.walk(root, onerror=_on_error):
             dirnames[:] = [d for d in dirnames if d not in skip_dirs]
             yield dirpath, dirnames, filenames
     except PermissionError:
-        logger.debug("Permission denied walking %s", root)
+        note_permission_error(root)

scan_supply_chain/git_repo_index.py

@@ -24,6 +24,7 @@
 from pathlib import Path
 
 from .config import GIT_REPO_WALK_SKIP_DIRS
+from .skip_report import note_permission_error, note_read_error
 from .subprocess_utils import run_safe
 
 logger = logging.getLogger(__name__)
@@ -74,14 +75,24 @@ def _find_repo_roots(roots: Iterable[str]) -> Iterator[Path]:
     """Yield the parent directory of each ``.git`` directory found.
 
     Does not descend into ``.git/`` itself. Prunes heavy/uninteresting
-    sibling trees per ``GIT_REPO_WALK_SKIP_DIRS``.
+    sibling trees per ``GIT_REPO_WALK_SKIP_DIRS``. Per-subdirectory
+    permission errors are routed through ``onerror`` into the
+    skip-report so the post-scan summary reflects them.
     """
+
+    def _on_error(exc: OSError) -> None:
+        path = Path(exc.filename) if exc.filename else Path("<unknown>")
+        if isinstance(exc, PermissionError):
+            note_permission_error(path)
+        else:
+            note_read_error(path, type(exc).__name__)
+
     for raw in roots:
         root_path = Path(raw)
         if not root_path.is_dir():
             continue
         try:
-            for dirpath, dirnames, _ in os.walk(root_path):
+            for dirpath, dirnames, _ in os.walk(root_path, onerror=_on_error):
                 dirnames[:] = [d for d in dirnames if d not in GIT_REPO_WALK_SKIP_DIRS]
                 if ".git" in dirnames:
                     yield Path(dirpath)
@@ -91,7 +102,7 @@ def _find_repo_roots(roots: Iterable[str]) -> Iterator[Path]:
                     # per outer repo is enough for the anti-worm scan.
                     dirnames.remove(".git")
         except PermissionError:
-            logger.debug("Permission denied walking %s", root_path)
+            note_permission_error(root_path)
 
 
 # ── Per-repo snapshot ───────────────────────────────────────────────────
@@ -113,7 +124,14 @@ def _read_description(git_dir: Path) -> str:
     desc_path = git_dir / "description"
     try:
         return desc_path.read_text(errors="ignore").strip()
-    except (FileNotFoundError, PermissionError, OSError):
+    except FileNotFoundError:
+        # Missing description file is normal — don't record.
+        return ""
+    except PermissionError:
+        note_permission_error(desc_path)
+        return ""
+    except OSError as exc:
+        note_read_error(desc_path, type(exc).__name__)
         return ""
 
 
@@ -125,8 +143,10 @@ def _read_local_branches(git_dir: Path) -> Iterator[str]:
             for entry in heads_dir.rglob("*"):
                 if entry.is_file():
                     yield entry.relative_to(heads_dir).as_posix()
-        except (PermissionError, OSError):
-            logger.debug("Cannot read %s", heads_dir)
+        except PermissionError:
+            note_permission_error(heads_dir)
+        except OSError as exc:
+            note_read_error(heads_dir, type(exc).__name__)
 
     packed = git_dir / "packed-refs"
     if packed.is_file():
@@ -141,8 +161,10 @@ def _read_local_branches(git_dir: Path) -> Iterator[str]:
                 prefix = "refs/heads/"
                 if ref.startswith(prefix):
                     yield ref[len(prefix):]
-        except (PermissionError, OSError):
-            logger.debug("Cannot read %s", packed)
+        except PermissionError:
+            note_permission_error(packed)
+        except OSError as exc:
+            note_read_error(packed, type(exc).__name__)
 
 
 def _list_workflow_files(repo_root: Path) -> Iterator[Path]:
@@ -154,8 +176,10 @@ def _list_workflow_files(repo_root: Path) -> Iterator[Path]:
         for entry in workflows.iterdir():
             if entry.is_file() and entry.suffix in _WORKFLOW_SUFFIXES:
                 yield entry
-    except (PermissionError, OSError):
-        logger.debug("Cannot read %s", workflows)
+    except PermissionError:
+        note_permission_error(workflows)
+    except OSError as exc:
+        note_read_error(workflows, type(exc).__name__)
 
 
 def _read_recent_emails(repo_root: Path, git_available: bool) -> Iterator[str]:

scan_supply_chain/ioc_scanner.py

@@ -22,6 +22,7 @@
     print_ioc_found,
 )
 from .models import ScanResults
+from .skip_report import note_permission_error, note_read_error
 
 if TYPE_CHECKING:
     from .ecosystem_base import EcosystemPlugin
@@ -55,7 +56,7 @@ def _check_known_paths(
                 results.iocs.append(str(path))
                 found = True
         except PermissionError:
-            logger.debug("Permission denied checking %s", path)
+            note_permission_error(path)
     if not found:
         print_clean()
 
@@ -90,9 +91,14 @@ def _scan_walk_files(
                             digest = hashlib.sha256(file_path.read_bytes()).hexdigest()
                             if digest not in known_hashes:
                                 continue
-                        except (PermissionError, OSError):
-                            # Can't read — still report as suspicious
-                            pass
+                        except PermissionError:
+                            # Can't read — still report as suspicious,
+                            # but record the path for the post-scan
+                            # summary so the operator knows why no hash
+                            # comparison happened.
+                            note_permission_error(file_path)
+                        except OSError as exc:
+                            note_read_error(file_path, type(exc).__name__)
                     print_ioc_found(str(file_path))
                     results.iocs.append(str(file_path))
                     found = True

scan_supply_chain/persistence_scanner.py

@@ -21,6 +21,7 @@
 
 from .config import read_if_contains
 from .models import FindingCategory, ScanResults, scanner_check
+from .skip_report import note_permission_error, note_read_error
 from .subprocess_utils import run_safe
 
 logger = logging.getLogger(__name__)
@@ -96,8 +97,10 @@ def _check_config_dir(
                     str(config_file),
                     2,
                 )
-    except (PermissionError, OSError):
-        logger.debug("Cannot read %s", directory)
+    except PermissionError:
+        note_permission_error(directory)
+    except OSError as exc:
+        note_read_error(directory, type(exc).__name__)
 
 
 # ── Individual checkers ─────────────────────────────────────────────────
@@ -141,8 +144,10 @@ def _check_shell_rc(results: ScanResults, search_terms: Sequence[str]) -> None:
                         str(rc_path),
                         2,
                     )
-        except (PermissionError, OSError):
-            logger.debug("Cannot read %s", rc_path)
+        except PermissionError:
+            note_permission_error(rc_path)
+        except OSError as exc:
+            note_read_error(rc_path, type(exc).__name__)
 
 
 def _check_tmp_scripts(results: ScanResults, package: str) -> None:
@@ -162,8 +167,10 @@ def _check_tmp_scripts(results: ScanResults, package: str) -> None:
                 _check_tmp_python_file(results, f, package)
             elif f.suffix in (".sh", ".bash"):
                 _check_tmp_shell_file(results, f, package)
-    except (PermissionError, OSError):
-        logger.debug("Cannot read /tmp")
+    except PermissionError:
+        note_permission_error(tmp)
+    except OSError as exc:
+        note_read_error(tmp, type(exc).__name__)
 
 
 def _check_tmp_python_file(results: ScanResults, path: Path, package: str) -> None:

scan_supply_chain/report.py

@@ -7,6 +7,7 @@
 from .formatting import BOLD, GREEN, RED, RESET, YELLOW, print_separator
 from .models import Confidence, ConfigReference, ScanResults, SourceReference
 from .scoring import compute_confidence
+from .skip_report import SkipReport
 
 if TYPE_CHECKING:
     from .threat_profile import ThreatProfile
@@ -278,6 +279,62 @@ def print_anti_worm_report(results: ScanResults) -> None:
         print()
 
 
+# ── Skip-report renderer ────────────────────────────────────────────────
+
+
+_SKIP_SUMMARY_HEAD = 5
+
+
+def print_skip_summary(report: SkipReport) -> None:
+    """Render the per-scan skip report. Silent when empty.
+
+    Filesystem paths the scanner couldn't walk or read are listed here
+    so the operator knows how much of the disk was actually inspected —
+    a partial scan from missing privileges is not a clean scan.
+    """
+    if report.is_empty:
+        return
+
+    print_separator()
+    print(
+        f"\n{YELLOW}{BOLD}Skipped {report.total} path(s) during this scan{RESET}\n"
+    )
+
+    if report.permission_errors:
+        print(
+            f"  {YELLOW}{BOLD}Permission denied "
+            f"({len(report.permission_errors)}):{RESET}"
+        )
+        _print_path_head(sorted(report.permission_errors))
+
+    if report.read_errors:
+        print(
+            f"  {YELLOW}{BOLD}Read errors "
+            f"({len(report.read_errors)}):{RESET}"
+        )
+        for path, reason in sorted(report.read_errors.items())[:_SKIP_SUMMARY_HEAD]:
+            print(f"    {path}  ({reason})")
+        remaining = len(report.read_errors) - _SKIP_SUMMARY_HEAD
+        if remaining > 0:
+            print(f"    ... and {remaining} more")
+        print()
+
+    print(
+        f"  {YELLOW}Hint: re-run with elevated privileges to cover paths "
+        f"the current user cannot access.{RESET}\n"
+    )
+
+
+def _print_path_head(paths: list) -> None:
+    """Print the first few paths plus a count of the rest."""
+    for p in paths[:_SKIP_SUMMARY_HEAD]:
+        print(f"    {p}")
+    remaining = len(paths) - _SKIP_SUMMARY_HEAD
+    if remaining > 0:
+        print(f"    ... and {remaining} more")
+    print()
+
+
 # ── Multi-threat summary ────────────────────────────────────────────────
 
 

scan_supply_chain/scanner.py

@@ -26,9 +26,11 @@
     print_anti_worm_report,
     print_config_refs,
     print_multi_threat_summary,
+    print_skip_summary,
     print_source_refs,
 )
 from .search_roots import build_search_roots, deduplicate_roots
+from .skip_report import get_current_report, reset_current_report
 from .source_scanner import scan_source_and_configs
 from .threat_profile import (
     ThreatProfile,
@@ -183,6 +185,9 @@ def main():
         print("No threat profiles found. Nothing to scan.")
         sys.exit(0)
 
+    # Fresh skip-report for this scan — instrumented helpers append to it.
+    reset_current_report()
+
     policy = detect_platform()
 
     print_banner(__version__)
@@ -222,10 +227,12 @@ def main():
         )
         all_results.append((threat, results))
 
-    # Final combined report — anti-worm section first, then per-threat
+    # Final combined report — anti-worm section first, then per-threat,
+    # then the skip-report telling the operator what coverage was missed.
     print()
     print_anti_worm_report(anti_worm_results)
     print_multi_threat_summary(all_results)
+    print_skip_summary(get_current_report())
 
     any_compromised = any(not r.is_clean for _, r in all_results)
     any_worm_signals = not anti_worm_results.is_clean