Skip to content

Commit 5584115

Browse files
committed
fix: parser and semantic correctness gaps, enforce benchmark parity
Parser: - lock: parse and discard the optional trailing `on fail` clause, so `lock { } on fail e { }` no longer leaves `on` to expression parsing - member assignment: retain the compound operator on Expr::MemberAssign via a new `op` field; `self.f += self.f` was being reported as a self-assignment because the `+` was dropped - let expression: accept `let var v = ... in ...`, matching query_let_clause and the let-var-decl grammar - correct the parse_leading_qualifiers doc: readonly/distinct are consumed when a class follows, transactional via contextual keywords Semantic: - union annotations resolve to Unknown unless every member resolves to the same type; picking types[0] made `int|string x = "s"` an error Benchmark script: - fail immediately when `cargo build --release` fails or the binary is missing, so a stale binary cannot be benchmarked - gate the timing on a findings diff between bal scan and blazelint, and reject timeouts or nonzero exits in the timed runs Docs: - relabel the stale "current state" figures in both plans as baselines - record unused-parameters as shipping `off`, an intentional divergence Corpus: diagnostic-free files 149 -> 151 of 163, false positives still 0.
1 parent b2dcacb commit 5584115

8 files changed

Lines changed: 135 additions & 27 deletions

File tree

.blazerc

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,14 @@ self-assignment = "warn"
1111
invalid-range = "warn"
1212
unused-variables = "error"
1313

14+
# Advisory scan rules — opt in by raising these to "warn".
15+
# A rule omitted from this file does not run at all, so listing them as "off"
16+
# is documentation, not a behaviour change.
17+
unused-parameters = "off" # ballerina:2
18+
isolated-public-function = "off" # ballerina:3
19+
isolated-public-method = "off" # ballerina:4
20+
isolated-public-class = "off" # ballerina:5
21+
1422
[settings]
1523
max-line-length = 120
1624
max-function-length = 50

docs/SCAN_RULES_PLAN.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@ change and no new analysis capability.
2121
| | Count |
2222
|---|---|
2323
| Official scan rules | 27 (12 language + 15 library) |
24-
| Implemented | **1**`ballerina:2` unused function parameter |
24+
| Baseline (before this plan) | **1**`ballerina:2` unused function parameter |
25+
| Implemented now | **7** |
2526
| After this plan | **7** (26%) |
2627

2728
Our other six rules (`camel-case`, `constant-case`, `line-length`,
@@ -130,13 +131,12 @@ Add to `Config::default()`:
130131
| `isolated-public-function` | `off` | Advisory; noisy outside concurrent code |
131132
| `isolated-public-method` | `off` ||
132133
| `isolated-public-class` | `off` ||
133-
| `unused-parameters` | `off` **`warn`** | Reconsider: `bal scan` ships this on as `ballerina:2`. Aligning improves parity honestly, at the cost of noise on callback signatures. |
134+
| `unused-parameters` | `off` | Intentional divergence from `bal scan`, which ships this on as `ballerina:2`: it fires on callback signatures that cannot drop a parameter. |
134135

135-
Note the `unused-parameters` question is a genuine trade-off, not an oversight:
136-
we turned it off last session precisely because it fired 27 times on the example
137-
corpus, nearly all on signatures that cannot drop a parameter. Recommend keeping
138-
it `off` by default and documenting the divergence, rather than importing the
139-
noise to match a number.
136+
The `unused-parameters` divergence is a deliberate trade-off, not an oversight:
137+
we turned it off because it fired 27 times on the example corpus, nearly all on
138+
signatures that cannot drop a parameter. It stays `off` by default and the
139+
divergence is documented, rather than importing the noise to match a number.
140140

141141
## 6. Verification
142142

docs/SEMANTIC_PLAN.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,10 @@
44
parser now structures, so Blazelint's *output* is trustworthy on real code, not
55
just its parsing.
66

7-
**Current state** — 99/163 corpus files (60%) are diagnostic-free. The other 64
8-
produce **207 diagnostics, and the sampled ones are all false positives**: the
9-
official Ballerina 2201.10.0 compiler accepts every file Blazelint rejects.
7+
**Baseline before this plan** — 99/163 corpus files (60%) were diagnostic-free.
8+
The other 64 produced **207 diagnostics, and the sampled ones were all false
9+
positives**: the official Ballerina 2201.10.0 compiler accepted every file
10+
Blazelint rejected. (Phases A–E have since shipped — see §4 for the results.)
1011

1112
**Target** — ≥95% of corpus files diagnostic-free, with zero false positives
1213
maintained, and every parser AST node reachable by both analysis passes.

scripts/benchmark_vs_scan.sh

Lines changed: 63 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,11 @@ WORK="$(mktemp -d)"
3636
trap 'rm -rf "$WORK"' EXIT
3737

3838
[ -x "$BAL" ] || { echo "Ballerina not found at $BAL (set BAL_HOME)"; exit 1; }
39-
cargo build --release --quiet
39+
command -v jq >/dev/null || { echo "jq is required to compare findings"; exit 1; }
40+
41+
# A stale or missing binary must never be benchmarked: fail here, not later.
42+
cargo build --release --quiet || { echo "cargo build --release failed"; exit 1; }
43+
[ -x "$BLZ" ] || { echo "blazelint binary not found at $BLZ"; exit 1; }
4044

4145
# ---------------------------------------------------------------- corpus setup
4246
PKG="$WORK/bench"
@@ -95,29 +99,84 @@ echo "Corpus: $FILES files in a single package"
9599
echo "Rule set: ballerina:1 (avoid-checkpanic), ballerina:2 (unused parameter)"
96100
echo
97101

102+
# ------------------------------------------------------- correctness gate
103+
# A timing comparison is only meaningful if both tools actually did the work and
104+
# found the same things. Run each once, unsuppressed, and diff the findings
105+
# before any number is reported.
106+
run_or_die() { # label timeout cmd...
107+
local label="$1" limit="$2"; shift 2
108+
local out status
109+
out=$(timeout "$limit" "$@" 2>&1); status=$?
110+
if [ "$status" -eq 124 ]; then
111+
echo "$label timed out after ${limit}s" >&2; exit 1
112+
elif [ "$status" -ne 0 ]; then
113+
echo "$label failed (exit $status):" >&2; echo "$out" >&2; exit 1
114+
fi
115+
printf '%s\n' "$out"
116+
}
117+
118+
# Canonical finding: file:line:rule-id. Columns differ between the tools'
119+
# anchor points, so equivalence is compared at line granularity.
120+
(cd "$PKG" && run_or_die "bal scan" 900 "$BAL" scan) >/dev/null
121+
jq -r '.[] | "\(.location.filePath):\(.location.startLine + 1):\(.rule.id)"' \
122+
"$PKG/target/report/scan_results.json" | sort > "$WORK/scan.findings"
123+
124+
: > "$WORK/blz.findings"
125+
for f in "$PKG"/*.bal; do
126+
(cd "$PKG" && run_or_die "blazelint $(basename "$f")" 300 "$BLZ" "$(basename "$f")")
127+
done | awk '
128+
/^(Warning|Error):/ { msg = $0 }
129+
/^[[:space:]]*--> / {
130+
rule = "unmatched"
131+
if (msg ~ /never used/) rule = "ballerina:2"
132+
else if (msg ~ /checkpanic/) rule = "ballerina:1"
133+
split($2, p, ":"); print p[1] ":" p[2] ":" rule
134+
}' | sort > "$WORK/blz.findings"
135+
136+
if ! diff -u "$WORK/scan.findings" "$WORK/blz.findings" > "$WORK/findings.diff"; then
137+
echo "Findings differ between bal scan and blazelint; refusing to benchmark." >&2
138+
echo "(-) bal scan only, (+) blazelint only:" >&2
139+
sed -n '3,$p' "$WORK/findings.diff" >&2
140+
exit 1
141+
fi
142+
FINDINGS=$(wc -l < "$WORK/scan.findings")
143+
[ "$FINDINGS" -gt 0 ] || { echo "No findings produced; corpus or rule set is wrong" >&2; exit 1; }
144+
echo "Findings: $FINDINGS, identical between both tools"
145+
98146
# --------------------------------------------------------------- measure tools
99147
median() { sort -n | awk '{a[NR]=$1} END {print (NR%2) ? a[(NR+1)/2] : (a[NR/2]+a[NR/2+1])/2}'; }
100148

149+
# Timed runs stay quiet for clean measurement, but a nonzero exit or a timeout
150+
# invalidates the sample rather than being recorded as a fast run.
151+
check_timed() { # label status
152+
if [ "$2" -eq 124 ]; then echo "$1 timed out during timing run" >&2; exit 1
153+
elif [ "$2" -ne 0 ]; then echo "$1 failed during timing run (exit $2)" >&2; exit 1; fi
154+
}
155+
101156
# JVM floor: what `bal` costs before doing any analysis at all.
102157
jvm_times=()
103158
for _ in $(seq 1 "$RUNS"); do
104-
s=$(date +%s%N); (cd "$PKG" && timeout 300 "$BAL" version >/dev/null 2>&1); e=$(date +%s%N)
159+
s=$(date +%s%N); (cd "$PKG" && timeout 300 "$BAL" version >/dev/null 2>&1); status=$?; e=$(date +%s%N)
160+
check_timed "bal version" "$status"
105161
jvm_times+=( $(( (e - s) / 1000000 )) )
106162
done
107163
JVM=$(printf '%s\n' "${jvm_times[@]}" | median)
108164

109165
scan_times=()
110166
for _ in $(seq 1 "$RUNS"); do
111-
s=$(date +%s%N); (cd "$PKG" && timeout 900 "$BAL" scan >/dev/null 2>&1); e=$(date +%s%N)
167+
s=$(date +%s%N); (cd "$PKG" && timeout 900 "$BAL" scan >/dev/null 2>&1); status=$?; e=$(date +%s%N)
168+
check_timed "bal scan" "$status"
112169
scan_times+=( $(( (e - s) / 1000000 )) )
113170
done
114171
SCAN=$(printf '%s\n' "${scan_times[@]}" | median)
115172

116173
blz_times=()
117174
for _ in $(seq 1 "$RUNS"); do
118175
s=$(date +%s%N)
119-
(cd "$PKG" && for f in *.bal; do "$BLZ" "$f" >/dev/null 2>&1; done)
176+
(cd "$PKG" && for f in *.bal; do timeout 300 "$BLZ" "$f" >/dev/null 2>&1 || exit $?; done)
177+
status=$?
120178
e=$(date +%s%N)
179+
check_timed "blazelint" "$status"
121180
blz_times+=( $(( (e - s) / 1000000 )) )
122181
done
123182
BLZ_MS=$(printf '%s\n' "${blz_times[@]}" | median)

src/ast.rs

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,9 +134,13 @@ pub enum Expr {
134134
},
135135
/// Assignment to a non-variable lvalue (field or index target), e.g.
136136
/// `self.count = 1` or `arr[i] = x`. Simple-variable assignment uses `Assign`.
137+
///
138+
/// `op` is `None` for a plain `=` and carries the operator for a compound
139+
/// assignment (`arr[i] += 1`), which reads `target = target op value`.
137140
MemberAssign {
138141
target: Box<Expr>,
139142
value: Box<Expr>,
143+
op: Option<BinaryOp>,
140144
span: Span,
141145
},
142146
/// Method call expression (e.g., obj.method()).

src/linter/rules/self_assignment.rs

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,14 @@ impl LintRule for SelfAssignmentRule {
3939
Expr::Assign { name, value, .. } => {
4040
matches!(value.as_ref(), Expr::Variable { name: v, .. } if v == name)
4141
}
42-
// `self.f = self.f`, `a[0] = a[0]`
43-
Expr::MemberAssign { target, value, .. } => same_lvalue(target, value),
42+
// `self.f = self.f`, `a[0] = a[0]`. A compound assignment such as
43+
// `self.f += self.f` does change the value, so only plain `=` counts.
44+
Expr::MemberAssign {
45+
target,
46+
value,
47+
op: None,
48+
..
49+
} => same_lvalue(target, value),
4450
_ => false,
4551
};
4652
if redundant {

src/parser.rs

Lines changed: 27 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -413,10 +413,13 @@ impl Parser {
413413
/// Consumes a run of module-level qualifier keywords, returning whether
414414
/// `public` was seen and the list of qualifier lexemes (for AST retention).
415415
///
416-
/// Only `public` and `isolated` are treated as leading qualifiers here.
417-
/// `readonly`, `distinct`, and `transactional` are deliberately excluded
418-
/// because they are primarily type-level constructors (`readonly & T`,
419-
/// `distinct T`) that must reach `parse_type_descriptor` at declaration start.
416+
/// `public` and `isolated` are always consumed here. `readonly` and
417+
/// `distinct` are consumed only when `qualifies_a_class` confirms a class
418+
/// declaration follows, so the type-level constructors (`readonly & T`,
419+
/// `distinct T`) still reach `parse_type_descriptor` at declaration start.
420+
/// `transactional`, `client`, and `service` are consumed as contextual
421+
/// keywords, with `service` excluded when a `/` marks a service declaration
422+
/// (`service /path on ...`) rather than a class qualifier.
420423
fn parse_leading_qualifiers(&mut self) -> ParseResult<(bool, Vec<String>)> {
421424
let mut is_public = false;
422425
let mut qualifiers = Vec::new();
@@ -1054,6 +1057,10 @@ impl Parser {
10541057
self.advance()?; // 'lock'
10551058
let start = self.previous_span().start;
10561059
let body = self.braced_block()?;
1060+
// `lock { } on fail e { }` is legal. `Stmt::Lock` has no on-fail fields,
1061+
// so the clause is parsed and discarded — otherwise `on` would reach
1062+
// expression parsing as a stray statement.
1063+
let _on_fail = self.parse_on_fail()?;
10571064
let end = self.previous_span().end;
10581065
Ok(Stmt::Lock {
10591066
body,
@@ -1737,14 +1744,24 @@ impl Parser {
17371744
}
17381745

17391746
// Field/index lvalues (`self.count = x`, `arr[i] += 1`) are valid
1740-
// assignment targets. The compound-operator distinction is not retained
1741-
// here (member-assignment semantics are deferred).
1747+
// assignment targets. The compound operator is retained on the node
1748+
// rather than desugared, since `Expr` is not `Clone` and the target
1749+
// would have to be duplicated to build the binary form.
17421750
if matches!(expr, Expr::FieldAccess { .. } | Expr::MemberAccess { .. }) {
17431751
let span_start = expr.span().start.min(assign_span.start);
17441752
let span_end = value_span_end.max(assign_span.end);
1753+
1754+
let op = match op_token {
1755+
Token::Eq => None,
1756+
Token::PlusEq => Some(BinaryOp::PlusAssign),
1757+
Token::MinusEq => Some(BinaryOp::MinusAssign),
1758+
_ => unreachable!(),
1759+
};
1760+
17451761
return Ok(Expr::MemberAssign {
17461762
target: Box::new(expr),
17471763
value: Box::new(value),
1764+
op,
17481765
span: span_start..span_end,
17491766
});
17501767
}
@@ -2506,7 +2523,10 @@ impl Parser {
25062523
let mut bindings = Vec::new();
25072524
loop {
25082525
self.match_token(&[Token::Final])?; // optional 'final'
2509-
let _ty = self.parse_type_descriptor()?;
2526+
// A binding is `var name` or `<type> name`, as in `query_let_clause`.
2527+
if !self.match_token(&[Token::Var])? {
2528+
let _ty = self.parse_type_descriptor()?;
2529+
}
25102530
let name = self.expect_ident("Expected variable name in let binding")?;
25112531
self.consume(Token::Eq, "Expected '=' in let binding", Some("'='"))?;
25122532
let value = self.expression()?;

src/semantic.rs

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1465,10 +1465,20 @@ impl Analyzer {
14651465
}
14661466
TypeDescriptor::Optional(inner) => self.type_from_annotation(inner),
14671467
TypeDescriptor::Union(types) => {
1468-
if !types.is_empty() {
1469-
self.type_from_annotation(&types[0])
1470-
} else {
1471-
Type::Unknown("union".to_string())
1468+
// Picking the first member would let `int|string x = "s"` be
1469+
// reported as a type error. Only a union whose members all resolve
1470+
// to the same type carries usable information; anything wider stays
1471+
// Unknown so downstream checks accept every valid member.
1472+
let mut distinct: Vec<Type> = Vec::new();
1473+
for ty in types {
1474+
let resolved = self.type_from_annotation(ty);
1475+
if !distinct.contains(&resolved) {
1476+
distinct.push(resolved);
1477+
}
1478+
}
1479+
match distinct.len() {
1480+
1 => distinct.remove(0),
1481+
_ => Type::Unknown("union".to_string()),
14721482
}
14731483
}
14741484
// Parse-tolerant, deferred-semantics types: represented but not fully

0 commit comments

Comments
 (0)