A two-layer NLP tool that automatically detects quality issues in systems engineering requirements. Built and tested on 625 real industrial requirements from the PROMISE NFR dataset.
Poor requirements are one of the leading causes of project failure in safety-critical systems. Issues like ambiguous language ("should", "possibly") and compound requirements make verification impossible and introduce risk.
Two layers working together:
Layer 1 — Rule-based: Regex patterns flag ambiguous words (should, could, possibly, etc.) and compound requirements (and/or). Fast and consistent.
Layer 2 — LLM (Llama 3.3 via Groq API): Analyzes flagged requirements in context. Filters false positives and catches subtle ambiguity that rules miss (e.g. "easy to use", "cater to", "appeal to").
- 625 requirements analyzed from the PROMISE NFR dataset
- Rule-based layer flagged 28 requirements for ambiguous language
- LLM confirmed 27 as genuinely problematic (false positive rate: 3.6%)
- LLM catches subjective terms with no measurable criteria that rule-based patterns miss
Python, Groq API (Llama 3.3-70b), pandas, re
PROMISE NFR Dataset — 625 real software requirements across 12 categories.
Source: Zenodo (https://zenodo.org/records/268542) — openly available for research use.