Skip to content

Commit 4710591

Browse files
committed
feat: add A.6 strict mode detection and A.7 naming consistency assessors
A.6 - TypeAnnotationsAssessor: - Add _check_strict_mode() to detect mypy/pyright strict mode in pyproject.toml, mypy.ini, setup.cfg, and pyrightconfig.json - Score combines coverage (80 pts) + strict mode bonus (20 pts) - Add comprehensive tests for all config file formats A.7 - StandardLayoutAssessor: - Add _check_naming_consistency() to detect mixed snake_case/camelCase in the same directory, reducing 'glob-ability' for agents - 10-point penalty per directory, capped at 20 points - Add 10 tests covering all edge cases (venv excluded, single file, etc.) docs/attributes.md: - Document strict mode scoring and remediation for A.6 - Document naming consistency criteria and penalty for A.7
1 parent 0fd1975 commit 4710591

5 files changed

Lines changed: 809 additions & 8 deletions

File tree

docs/attributes.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,6 +266,8 @@ Type annotations give agents reliable information about what a function expects
266266
- All public functions have parameter and return type hints
267267
- Generic types from `typing` module used appropriately
268268
- Coverage: >80% of functions typed
269+
- **Strict mode configured** (prevents new violations): mypy `strict = true` or `disallow_untyped_defs`, or pyright `strict = true` in `pyproject.toml`, `mypy.ini`, `setup.cfg`, or `pyrightconfig.json`
270+
- Strict mode is scored as a 20-point bonus on a 100-point scale (80 pts max for coverage + 20 pts for strict mode)
269271
- Tools: mypy, pyright
270272

271273
**TypeScript**:
@@ -362,6 +364,29 @@ def create_user(email, role):
362364

363365
4. **Fix errors iteratively**
364366

367+
5. **Enable strict mode** in your type checker config:
368+
369+
```toml
370+
# pyproject.toml
371+
[tool.mypy]
372+
strict = true
373+
```
374+
375+
Or alternatively:
376+
377+
```ini
378+
# mypy.ini
379+
[mypy]
380+
disallow_untyped_defs = true
381+
```
382+
383+
For pyright:
384+
385+
```json
386+
// pyrightconfig.json
387+
{ "typeCheckingMode": "strict" }
388+
```
389+
365390
**TypeScript**:
366391

367392
1. **Enable strict mode** in `tsconfig.json`:
@@ -406,8 +431,21 @@ Using community-recognized directory structures for each language/framework (e.g
406431

407432
Models trained on open-source code have seen the standard layouts thousands of times. When a repo uses the Python `src/` layout or Go's `cmd/internal/pkg` structure, the agent knows where to look for things and where to put new ones. Non-standard layouts force it to explore, and it may still place files in the wrong location.
408433

434+
**Naming consistency** also matters for agent efficiency. When files in the same directory follow a consistent naming convention (e.g., all snake_case in Python), agents can use glob patterns like `src/**/*.py` reliably. Mixed conventions (snake_case and camelCase coexisting in the same directory) reduce "glob-ability" and make it harder for agents to predict file locations.
435+
409436
#### Measurable Criteria
410437

438+
**Python** (naming consistency applies to all languages but assessed via Python files):
439+
440+
- Consistent naming convention within directories:
441+
- snake_case (`module_name.py`) — Python community standard
442+
- camelCase (`moduleHandler.py`) — also valid but must be consistent
443+
- **Mixed naming in the same directory** (e.g., `module_a.py` and `moduleHandler.py` in the same directory) reduces the score
444+
- 10-point penalty per directory with mixed naming, capped at 20 points
445+
- Directories are checked independently: different dirs can have different conventions
446+
- Single-file directories or directories with only one convention are not penalized
447+
- Directories with only 1-2 files need both to be from different conventions to trigger a penalty
448+
411449
**Python (src/ layout)**:
412450

413451
```

src/agentready/assessors/code_quality.py

Lines changed: 107 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
import ast
44
import logging
55
import re
6+
import tomllib
67

78
from ..models.attribute import Attribute
89
from ..models.finding import Citation, Finding, Remediation
@@ -73,7 +74,13 @@ def assess(self, repository: Repository) -> Finding:
7374
)
7475

7576
def _assess_python_types(self, repository: Repository) -> Finding:
76-
"""Assess Python type annotations using AST parsing."""
77+
"""Assess Python type annotations using AST parsing.
78+
79+
Checks both annotation coverage (current state) and strict mode
80+
configuration (prevents new violations). Both matter — strict mode
81+
stops new untyped code from entering; coverage measures what's already
82+
typed.
83+
"""
7784
# Use AST parsing to accurately detect type annotations
7885
try:
7986
# Security: Use safe_subprocess_run for validation and limits
@@ -130,28 +137,121 @@ def _assess_python_types(self, repository: Repository) -> Finding:
130137
)
131138

132139
coverage_percent = (typed_functions / total_functions) * 100
133-
score = self.calculate_proportional_score(
140+
141+
# Check for strict mode configuration (mypy/pyright)
142+
strict_mode = self._check_strict_mode(repository)
143+
144+
# Score combines coverage (up to 80 pts) + strict mode bonus (up to 20 pts)
145+
coverage_score = self.calculate_proportional_score(
134146
measured_value=coverage_percent,
135147
threshold=80.0,
136148
higher_is_better=True,
137149
)
150+
score = min(100.0, coverage_score + (20.0 if strict_mode else 0.0))
138151

139152
status = "pass" if score >= 75 else "fail"
140153

154+
evidence = [
155+
f"Typed functions: {typed_functions}/{total_functions}",
156+
f"Coverage: {coverage_percent:.1f}%",
157+
]
158+
if strict_mode:
159+
evidence.append("Strict mode configured")
160+
else:
161+
evidence.append("No strict mode configuration found")
162+
141163
return Finding(
142164
attribute=self.attribute,
143165
status=status,
144166
score=score,
145167
measured_value=f"{coverage_percent:.1f}%",
146-
threshold="≥80%",
147-
evidence=[
148-
f"Typed functions: {typed_functions}/{total_functions}",
149-
f"Coverage: {coverage_percent:.1f}%",
150-
],
168+
threshold="≥80% coverage + strict mode",
169+
evidence=evidence,
151170
remediation=self._create_remediation() if status == "fail" else None,
152171
error_message=None,
153172
)
154173

174+
@staticmethod
175+
def _check_strict_mode(repository: Repository) -> bool:
176+
"""Check whether the Python type checker is configured in strict mode.
177+
178+
Looks for mypy/pyright strict mode configuration in pyproject.toml,
179+
mypy.ini, setup.cfg, or pyrightconfig.json.
180+
181+
ADR A.6: Strict mode prevents new violations; coverage measures
182+
current state. Both matter.
183+
"""
184+
# Check pyproject.toml for mypy/pyright strict config
185+
pyproject_path = repository.path / "pyproject.toml"
186+
if pyproject_path.exists():
187+
try:
188+
with open(pyproject_path, "rb") as f:
189+
data = tomllib.load(f)
190+
191+
# mypy: [tool.mypy] strict = true
192+
tool = data.get("tool", {})
193+
mypy_cfg = tool.get("mypy", {})
194+
if mypy_cfg.get("strict", False):
195+
return True
196+
if "disallow_untyped_defs" in mypy_cfg and mypy_cfg[
197+
"disallow_untyped_defs"
198+
]:
199+
return True
200+
201+
# pyright: [tool.pyright] strict = true
202+
pyright_cfg = tool.get("pyright", {})
203+
if pyright_cfg.get("strict", False):
204+
return True
205+
206+
except (OSError, tomllib.TOMLDecodeError):
207+
pass
208+
209+
# Check mypy.ini
210+
mypy_ini_path = repository.path / "mypy.ini"
211+
if mypy_ini_path.exists():
212+
try:
213+
content = mypy_ini_path.read_text(encoding="utf-8")
214+
if "[mypy]" in content:
215+
for line in content.splitlines():
216+
stripped = line.strip().lower()
217+
if stripped == "strict = true" or stripped == "strict=true":
218+
return True
219+
if stripped.startswith("disallow_untyped_defs"):
220+
return True
221+
except OSError:
222+
pass
223+
224+
# Check setup.cfg
225+
setup_cfg_path = repository.path / "setup.cfg"
226+
if setup_cfg_path.exists():
227+
try:
228+
content = setup_cfg_path.read_text(encoding="utf-8")
229+
if "[mypy]" in content or "[mypy]" in content.lower():
230+
for line in content.splitlines():
231+
stripped = line.strip().lower()
232+
if "strict = true" in stripped or "strict=true" in stripped:
233+
return True
234+
if stripped.startswith("disallow_untyped_defs"):
235+
return True
236+
except OSError:
237+
pass
238+
239+
# Check pyrightconfig.json
240+
pyright_path = repository.path / "pyrightconfig.json"
241+
if pyright_path.exists():
242+
try:
243+
import json
244+
245+
content = json.loads(pyright_path.read_text(encoding="utf-8"))
246+
if content.get("typeCheckingMode", "") == "strict":
247+
return True
248+
if content.get("strict", False):
249+
return True
250+
except (json.JSONDecodeError, OSError):
251+
pass
252+
253+
return False
254+
155255
def _assess_typescript_types(self, repository: Repository) -> Finding:
156256
"""Assess TypeScript type configuration across all tsconfig.json files.
157257

src/agentready/assessors/structure.py

Lines changed: 101 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,9 @@ def assess(self, repository: Repository) -> Finding:
126126
- Go: cmd/, internal/, pkg/ with go.mod, tests in *_test.go
127127
128128
Fix for #246, #305: Support multiple valid Python layouts
129+
ADR A.7: Adds naming consistency check for Python repos — mixed
130+
naming conventions (camelCase vs snake_case in same directory) reduce
131+
"glob-ability" for agents.
129132
"""
130133
if self._primary_language(repository, {"Python", "Go"}) == "Go":
131134
return self._assess_go_layout(repository)
@@ -162,6 +165,16 @@ def assess(self, repository: Repository) -> Finding:
162165
higher_is_better=True,
163166
)
164167

168+
# ADR A.7: Naming consistency check for Python repos
169+
naming_penalty = 0.0
170+
naming_evidence = []
171+
if repository.languages.get("Python", 0) > 0:
172+
penalty, evidence = self._check_naming_consistency(repository)
173+
naming_penalty = penalty
174+
naming_evidence = evidence
175+
176+
score = max(0.0, score - naming_penalty)
177+
165178
status = "pass" if score >= 75 else "fail"
166179

167180
# Build evidence with detailed source directory info
@@ -180,7 +193,7 @@ def assess(self, repository: Repository) -> Finding:
180193
f"Found {found_dirs}/{required_dirs} standard directories",
181194
source_evidence,
182195
f"tests/: {'✓' if has_tests else '✗'}",
183-
]
196+
] + naming_evidence
184197

185198
return Finding(
186199
attribute=self.attribute,
@@ -497,6 +510,93 @@ def _create_go_remediation(
497510
],
498511
)
499512

513+
def _check_naming_consistency(self, repository: Repository) -> tuple:
514+
"""Check for mixed naming conventions in the same directory.
515+
516+
ADR A.7: Inconsistent naming (camelCase vs snake_case coexisting in
517+
the same directory) reduces "glob-ability" for agents. A consistent
518+
naming convention makes it easier for agents to use glob patterns like
519+
`src/**/*.py` to find all relevant files.
520+
521+
Returns:
522+
(penalty_score, evidence_list)
523+
- penalty_score: 0-20, where 20 is maximum penalty for mixed conventions
524+
- evidence_list: human-readable evidence strings
525+
"""
526+
# Only check source directories, not test directories
527+
# We look at Python files (.py, .pyi) in source locations
528+
source_path = repository.path / "src"
529+
if not source_path.exists():
530+
# Try project-named directory
531+
package_name = self._get_package_name_from_pyproject(repository)
532+
if package_name:
533+
source_path = repository.path / package_name.replace("-", "_")
534+
if not source_path.exists():
535+
# Fall back to root for small repos
536+
source_path = repository.path
537+
538+
file_stats: dict = {"snake_case": 0, "camelCase": 0, "mixed": set()}
539+
total_files = 0
540+
541+
try:
542+
for py_file in source_path.rglob("*.py"):
543+
# Skip venv, node_modules, etc.
544+
if any(
545+
part in py_file.parts
546+
for part in [".venv", "venv", "node_modules", ".git", ".tox"]
547+
):
548+
continue
549+
total_files += 1
550+
dirname = py_file.parent.name
551+
filename = py_file.name
552+
553+
# Group files by directory and check naming per directory
554+
is_snake = bool(re.fullmatch(r"[a-z][a-z0-9]*(?:_[a-z0-9]+)*\.py(?:i)?$", filename))
555+
is_camel = bool(re.fullmatch(r"[a-z][a-zA-Z0-9]*\.py(?:i)?$", filename))
556+
557+
if is_snake and is_camel:
558+
pass # lowercase is snake_case
559+
elif is_snake:
560+
key = f"{dirname}/"
561+
if key not in file_stats:
562+
file_stats[key] = {"snake_case": 0, "camelCase": 0}
563+
file_stats[key]["snake_case"] += 1
564+
elif is_camel:
565+
key = f"{dirname}/"
566+
if key not in file_stats:
567+
file_stats[key] = {"snake_case": 0, "camelCase": 0}
568+
file_stats[key]["camelCase"] += 1
569+
570+
except OSError:
571+
return 0.0, []
572+
573+
# Check if any directory has mixed naming
574+
penalty = 0.0
575+
evidence = []
576+
577+
for dir_key, counts in file_stats.items():
578+
snake_count = counts.get("snake_case", 0)
579+
camel_count = counts.get("camelCase", 0)
580+
581+
# Only count directories with at least 2 files (need meaningful sample)
582+
if snake_count > 0 and camel_count > 0:
583+
total_in_dir = snake_count + camel_count
584+
if total_in_dir >= 2:
585+
penalty += 10.0
586+
evidence.append(
587+
f"Mixed naming in {dir_key}: {snake_count} snake_case, {camel_count} camelCase"
588+
)
589+
590+
# Cap the penalty at 20 points
591+
penalty = min(20.0, penalty)
592+
593+
if penalty > 0:
594+
evidence.insert(0, "Inconsistent naming conventions detected")
595+
elif total_files > 0:
596+
evidence.append("Naming convention consistent")
597+
598+
return penalty, evidence
599+
500600
def _create_remediation(self, has_source: bool, has_tests: bool) -> Remediation:
501601
"""Create context-aware remediation guidance for standard layout.
502602

0 commit comments

Comments
 (0)