Skip to content
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
9d342f0
Compiler fixes from the self-hosting effort
cuzzo Aug 12, 2026
32924a2
Specs and transpile tests for the self-hosting fixes
cuzzo Aug 12, 2026
639ba63
Tooling, runtime and docs from the self-hosting effort
cuzzo Aug 12, 2026
cae098e
Sync the fuzz README cell count for curated_gap_corpus
cuzzo Aug 12, 2026
5e09abd
Decide owned-branch sources by construction, not by naming reads
cuzzo Aug 12, 2026
fed2220
Reject frame frees of memory the arena never allocated
cuzzo Aug 12, 2026
1129eb0
Always emit a struct's __clear_drop and __clear_clone, even when noth…
cuzzo Aug 12, 2026
a924175
Fuzz the provenance round trip: read a value back out and let it drop
cuzzo Aug 12, 2026
1fc122a
Benchmark symbol interning: lock elision vs a shared pool, against Rust
cuzzo Aug 12, 2026
1684431
Add an optimized test lane: clear test --safe, fuzz --safe
cuzzo Aug 12, 2026
5990c6e
Give String@symbol its own type instead of spelling it []const u8
cuzzo Aug 12, 2026
2cd6d6d
Ask a MIR node whether it materializes a value, instead of listing th…
cuzzo Aug 12, 2026
3ca3886
Match the Symbol key by name, not by having a `bytes` field
cuzzo Aug 12, 2026
96c5b74
Say when the frame free check turns itself off
cuzzo Aug 12, 2026
134a813
Thread the fuzz --safe flag instead of passing it through a global
cuzzo Aug 12, 2026
91280b9
Prove the provenance matrix detects the bug it was built for
cuzzo Aug 12, 2026
65653dd
Reconcile the rebase with master's newer pipeline and test expectations
cuzzo Aug 12, 2026
7b4b17f
Decide the symbol-widening cast from the Type, not the rendered Zig s…
cuzzo Aug 12, 2026
6005438
Widen a Symbol at every String coercion boundary, not just CAST and p…
cuzzo Aug 12, 2026
e847b5e
Track retired arena storage as an envelope, not a table -- the table …
cuzzo Aug 12, 2026
9115d6c
Delete the interned container variants the Symbol type made dead
cuzzo Aug 12, 2026
2b4034d
Pin symbol widening per boundary in the spec suite; fix lower_cast's …
cuzzo Aug 12, 2026
24754b6
Name the widening target: Type#byte_string?
cuzzo Aug 12, 2026
3260f5d
Unit-test CheatArena.owns: current, retired, and foreign storage
cuzzo Aug 12, 2026
63ffa1b
Widen a Symbol at the registry-call boundary too
cuzzo Aug 13, 2026
bd4fd2a
Coerce an OR_ELSE fallback to the merge type while its stamp is in hand
cuzzo Aug 13, 2026
cd81ada
Print the rejected bytes in the frame free check
cuzzo Aug 13, 2026
1cd9fb3
Drop the fact-mine alias analysis this branch swept in by accident
cuzzo Aug 13, 2026
1cf10f6
Restore the @node handle payload fix the rebase reverted
cuzzo Aug 13, 2026
0729d82
Prune the build cache by age, not by entry count
cuzzo Aug 13, 2026
e839ba1
Widen a Symbol at the equality-assert boundary too
cuzzo Aug 13, 2026
5ea2c91
Make the compiler type-check under Sorbet again
cuzzo Aug 13, 2026
5013495
Run instrumented tests from the invocation directory again
cuzzo Aug 13, 2026
c331e4e
Restore two more `clear test` fixes the rebase dropped
cuzzo Aug 13, 2026
e29ab2b
Take master's matrix specs back: the runtime lanes live in the fuzz h…
cuzzo Aug 13, 2026
ba18651
Release a union inline-variant payload that owns through a capability
cuzzo Aug 13, 2026
ee9a5e0
Sync the fuzz README cell count for the new transpile test
cuzzo Aug 13, 2026
5bc6052
Report a leaking fuzz bundle as a leak, and show the end of a silent …
cuzzo Aug 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,9 @@ zig/transpile-test.zig
zig/zig-out
zig/.zig-cache
zig/.zig-cache-*
# Per-build Zig/CLEAR caches. 854 of these had been committed by accident.
zig/.zig-global-cache
zig/.clear-module-cache
zig/.clear-cache
zig/.clear-transpile-cache
zig/fiber-stack-check/pass/build
Expand Down Expand Up @@ -219,3 +222,4 @@ compiler/.ruby-rbs/
compiler/.ruby-original/
# Local Spinel fork used for the AOT experiment
tmp/spinel/

1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ Reference docs: `mir-bugs.md` (known MIR violations), `alloc-bugs.md` (frame-the
**Sigils:** `$` pipeline/interp, `&` mutation, `|>` SMOOTH (safe pipeline w/ error prop), `_` placeholder.

**Tense Sigils:** `!` = Error / Error handling, `?` = Option / nil handling, `~` = Stream / future handling.
**Sigils:** `$` pipeline/interp, `&` explicit mutable call-site path, `|>` SMOOTH (safe pipeline w/ error prop), `_` placeholder, `TRY` explicit propagation.

**Ownership / capabilities — bindings, not types.** Two sigil groups:
- **Group 1 (sync / ownership wrappers):** `@locked`, `@writeLocked`, `@shared` (Arc), `@multiowned` (Rc), `@local`. Stored on `SymbolEntry#sync` and `#storage`. Composed via `MIR::CapWrap`.
Expand Down
157 changes: 116 additions & 41 deletions clear

Large diffs are not rendered by default.

18 changes: 8 additions & 10 deletions compiler/ruby/annotator/domains/control_flow.rb
Original file line number Diff line number Diff line change
Expand Up @@ -487,8 +487,11 @@ def declare_is_a_binding!(condition)
return unless payload_type

scope = current_scope
scope.declare(binding, nil, payload_type, false, false, nil, :stack)
og_declare(binding, nil, payload_type)
# The IS_A node is the binding's declaration site. Recording it gives
# lowering a stable identity to key a rename on when a nested MATCH
# binds the same name.
scope.declare(binding, condition, payload_type, false, false, nil, :stack)
og_declare(binding, condition, payload_type)
classify_ownership!(scope.local_entry!(binding))
borrow_match_payload_binding!(binding)
return
Expand Down Expand Up @@ -529,13 +532,8 @@ def visit_IfBind(node)
# getPtr/getAtPtrOpt), so mutation through the capture is
# legal and lands in the container. Rc/node-handle payloads
# are value captures and stay immutable borrows.
# A @node handle is itself a pointer into the NodeStore, so
# assigning through the capture lands in the stored node --
# excluding it here made `IF nodes[i] EXISTS AS n THEN n.f = ...`
# fail as an immutable-field assignment. Rc payloads stay
# immutable value captures.
mutable_slot_payload = (unwrapped.struct? || unwrapped.collection?) &&
!unwrapped.any_rc?
!unwrapped.node_reference? && !unwrapped.any_rc?
mutable_list_alias = b.expr.is_a?(AST::GetIndex) && root &&
!current_scope.is_immutable?(root.name) && mutable_slot_payload
current_scope.declare(b.name, nil, unwrapped, mutable_list_alias, false, nil, :stack)
Expand Down Expand Up @@ -941,8 +939,8 @@ def declare_union_payload_binding!(node, match_case, plan, variant_name, binding
end

payload_type = match_payload_binding_type(plan, variant_name, T.unsafe(raw_payload), match_case)
current_scope.declare(binding, nil, payload_type, false, false, nil, :stack)
og_declare(binding, nil, payload_type)
current_scope.declare(binding, match_case, payload_type, false, false, nil, :stack)
og_declare(binding, match_case, payload_type)
classify_ownership!(current_scope.local_entry!(binding))
borrow_match_payload_binding!(binding) unless node.takes
end
Expand Down
1 change: 0 additions & 1 deletion compiler/ruby/annotator/domains/errors.rb
Original file line number Diff line number Diff line change
Expand Up @@ -681,7 +681,6 @@ def visit_OrElse(node)
).returns(TenseOperationPlan)
end
def plan_or_else_with_diagnostic(node, left_type, right_type, operation, recovery)
T.bind(self, Annotator::Phases::TypeAnalysisSession)
TenseOperationPlanner.or_else(
left_type,
right_type,
Expand Down
1 change: 0 additions & 1 deletion compiler/ruby/annotator/domains/expressions.rb
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,6 @@ def visit_UnaryOp(node)

sig { params(node: AST::UnaryOp, plan_input: Type, raw_type: Type).returns(T.nilable(TenseOperationPlan)) }
def try_value_plan_with_diagnostic(node, plan_input, raw_type)
T.bind(self, Annotator::Phases::TypeAnalysisSession)
TenseOperationPlanner.try_value(plan_input)
rescue ArgumentError
error!(node, :UNWRAP_NON_OPTIONAL, got: raw_type)
Expand Down
8 changes: 7 additions & 1 deletion compiler/ruby/annotator/domains/lifetimes.rb
Original file line number Diff line number Diff line change
Expand Up @@ -601,7 +601,13 @@ def handle_assign_borrow(node)
error!(node, :BORROWED_VAR_NOT_FOUND) if borrowed_scope.nil?
return if T.must(borrowed_scope).is_immutable?(root_var)

lhs_name = node.name.is_a?(AST::Identifier) ? node.name.name : "__borrow_#{root_var}"
# VarDecl#name is a String, Assignment#name an Identifier. Both are real
# bindings and must borrow under their own name; only a genuinely
# unbound result falls back to the synthetic name, whose lifetime
# nothing ever ends.
lhs_name = node.name
lhs_name = lhs_name.name if lhs_name.is_a?(AST::Identifier)
lhs_name = "__borrow_#{root_var}" unless lhs_name.is_a?(String)
mutable = node.is_a?(AST::VarDecl) && node.mutable
err = ownership_graph.borrow(lhs_name, root_var, mutable: mutable)
error!(node, :LIFETIME_ALREADY_BORROWED, name: root_var) if err
Expand Down
6 changes: 5 additions & 1 deletion compiler/ruby/annotator/domains/member_access.rb
Original file line number Diff line number Diff line change
Expand Up @@ -380,7 +380,11 @@ def visit_HashLit(node)

values = node.pairs.values
if values.all? { |value| Type.new(value.resolved_type).string? }
value_type = :String
# Symbols are strings, but interned ones: collapsing them to a plain
# String drops @symbol and the map's values become owned slices that
# COPY deep-clones and cleanup frees. List literals already preserve
# the element capability.
value_type = values.all? { |value| value.full_type!(context: "hash literal symbol value").symbol? } ? :"String@symbol" : :String
else
value_type = values.first.resolved_type
symbol_key_map = node.pairs.keys.all? { |key| key.is_a?(AST::Literal) && key.type == :SYMBOL }
Expand Down
6 changes: 6 additions & 0 deletions compiler/ruby/annotator/helpers/capabilities.rb
Original file line number Diff line number Diff line change
Expand Up @@ -946,6 +946,12 @@ def declare_capability_scope!(fact)
declare_unwrapped_capability_alias!(fact) if fact.unwraps_sync_alias?
declare_capability_binding_or_error!(fact)
declare_capability_projection!(fact)
# declare_with_new_capability marks the SOURCE binding, but the body reads
# through the alias and Scope#is_restricted? answers per binding. Without
# this the alias looks unrestricted, so borrowing through it -- e.g. calling
# a `RETURNS self: T` accessor -- is refused.
alias_name = fact.alias_name
current_scope.resolve_entry(alias_name)&.capabilities&.add(fact.capability) if alias_name
nil
end

Expand Down
51 changes: 41 additions & 10 deletions compiler/ruby/annotator/helpers/function_analysis.rb
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ def replace_arg!(index, arg)
def explicit_mutable_argument?(index)
method_node = T.cast(node, T.nilable(AST::MethodCall)) if node.is_a?(AST::MethodCall)
if method_node && args.length == method_node.args.length + 1
concrete_method = method_node
concrete_method = T.must(method_node)
return concrete_method.explicit_mutable_receiver? if index == 0
return concrete_method.explicit_mutable_argument?(index - 1)
end
Expand All @@ -58,7 +58,7 @@ def explicit_mutable_argument?(index)
def explicit_mutable_argument_token(index)
method_node = T.cast(node, T.nilable(AST::MethodCall)) if node.is_a?(AST::MethodCall)
if method_node && args.length == method_node.args.length + 1
concrete_method = method_node
concrete_method = T.must(method_node)
return concrete_method.explicit_mutable_receiver_token_value if index == 0
return concrete_method.explicit_mutable_argument_token(index - 1)
end
Expand Down Expand Up @@ -196,11 +196,30 @@ def analyze_routine(node, body, declared_return, is_implicit)
return_type
end

# The root scope also holds imported names and function entries; a routine
# body must only inherit the module's own variable declarations.
sig { returns(Scope) }
def module_variable_scope
T.bind(self, Annotator::Phases::TypeAnalysisSession) rescue nil
seeded = Scope.new
semantic_root_scope.binding_entries.each do |name, entry|
next unless entry.reg.is_a?(AST::VarDecl)

seeded.binding_entries[name] = entry
end
seeded
end

sig { params(node: RoutineNode, blk: T.proc.void).void }
def with_routine_analysis_scope(node, &blk)
T.bind(self, Annotator::Phases::TypeAnalysisSession) rescue nil

with_new_scope do
# A routine body sees the module's own declarations: a module-level
# `MUTABLE x = ...` is Ruby module state, and `x = value` inside a function
# must reassign it. Seeding from the root scope (rather than a fresh one)
# is what makes the assignment resolve instead of silently declaring a
# shadowing local.
with_new_scope(module_variable_scope) do
og_push_scope
begin
blk.call
Expand Down Expand Up @@ -258,7 +277,7 @@ def signature_from_function_type(fn_type)
name: "arg#{i}",
type: param.type,
required: true,
mutable: false,
mutable: param.mutable,
takes: false
)
i += 1
Expand Down Expand Up @@ -1287,7 +1306,8 @@ def verify_param_lifetime!(arg_node, param, signature)
end
return true unless base_paths.include?(:wildcard) || base_paths.include?(param.name)

error!(arg_node, :MUTABLE_PARAM_NEEDS_RESTRICT, name: param.name)
error!(arg_node, :MUTABLE_PARAM_NEEDS_RESTRICT,
name: param.name, arg: arg_node.name, callee: (signature.respond_to?(:fn_name) ? signature.fn_name : nil) || "the callee")
end

# `node.return_lifetime` shapes:
Expand Down Expand Up @@ -1558,7 +1578,15 @@ def declare_captures(node)
nil,
cap.storage
)
capture_entry.inherit_ownership_identity!(owner_entry) if owner_entry
next unless owner_entry

capture_entry.inherit_ownership_identity!(owner_entry)
# A capture is the same binding seen from inside the lambda, so it keeps
# the source's capabilities. Without this, capturing a WITH alias --
# `USE(MUTABLE view)` -- yields an entry with none, so
# Scope#is_restricted? is false for it and borrowing through the capture
# is refused.
capture_entry.capabilities.merge(owner_entry.capabilities)
end
nil
end
Expand Down Expand Up @@ -1670,12 +1698,15 @@ def return_is_borrow?(node)
end
if node.is_a?(AST::Identifier)
return false unless ownership_graph[node.name]&.kind == :borrowed
# Parameters (reg=nil) and MATCH bindings (reg=nil) are safe to return —
# the caller controls their lifetime. Only flag variables explicitly assigned
# from a collection index borrow (BindExpr with container_borrow=true).
# Parameters and MATCH/IS_A payload bindings are safe to return — the
# caller controls their lifetime. (A payload binding records its MATCH
# arm as `reg` so lowering can rename a nested rebind of the same name;
# that node carries no container_borrow.) Only flag variables explicitly
# assigned from a collection index borrow (BindExpr with
# container_borrow=true).
scope = lookup_scope_for(node.name)
reg = scope&.resolve_entry(node.name)&.reg
return reg&.container_borrow == true
return !!(reg.respond_to?(:container_borrow) && reg.container_borrow == true)
end
return true if node.is_a?(AST::GetIndex)
return true if node.is_a?(AST::GetField)
Expand Down
7 changes: 6 additions & 1 deletion compiler/ruby/annotator/helpers/function_return.rb
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,11 @@ def resolve(receiver, args = [])
element_list(value)
when Kind::KeyList
key = T.must(receiver).key_type
# keys() hands back the map's OWN keys, which it duplicated on insert and
# frees at deinit -- owned Strings, even when lookups are spelled with
# interned `String@symbol` handles. A Symbol is a handle nobody owns, so
# claiming one here would label map-owned bytes as immortal.
key = Type.new(:String) if key.symbol?
element_list(key)
when Kind::Infer
resolve_infer(args)
Expand Down Expand Up @@ -188,7 +193,7 @@ def infer_to_list(args)
receiver = T.must(args.first)
receiver_type = receiver.type_object
raise "toList receiver: unresolved type info" unless receiver_type
receiver_type = receiver_type
receiver_type = T.must(receiver_type)
raise "toList receiver: unresolved type info" if receiver_type.untyped?
element_type = if receiver_type.dynamic_stream? || receiver_type.promise_list?
receiver_type.tense_type.element_type
Expand Down
42 changes: 41 additions & 1 deletion compiler/ruby/annotator/helpers/function_signature.rb
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,11 @@ def initialize(params:, visibility: nil, type_params: [], reentrant: false,
end
end

# Stand-in for a registry `validate:` lambda while a signature is
# serialized. Procs cannot be marshalled, but every validator is a
# registry singleton, so the name is enough to re-link on load.
ValidatorRef = Struct.new(:name)

class AnalysisFacts < T::Struct
extend T::Sig

Expand Down Expand Up @@ -137,6 +142,29 @@ def copy
return_def: return_def
)
end

sig { returns(T::Hash[Symbol, T.untyped]) }
def marshal_dump
state = T.let({}, T::Hash[Symbol, T.untyped])
instance_variables.each { |ivar| state[ivar] = instance_variable_get(ivar) }
validator = state[:@arg_validator]
return state unless validator

name = IntrinsicRegistry.validator_name(validator)
raise TypeError, "arg_validator is not a registry validator and cannot be serialized" unless name

state[:@arg_validator] = ValidatorRef.new(name)
state
end

sig { params(state: T::Hash[Symbol, T.untyped]).void }
def marshal_load(state)
state.each { |ivar, value| instance_variable_set(ivar, value) }
reference = @arg_validator
return unless reference.is_a?(ValidatorRef)

@arg_validator = T.let(IntrinsicRegistry.validator_for_name(reference.name), T.nilable(Proc))
end
end
private_constant :Contract, :AnalysisFacts

Expand Down Expand Up @@ -723,6 +751,15 @@ def dup
copy
end

sig { params(entry: T.nilable(SymbolEntry)).returns(T.nilable(SymbolEntry)) }
def self.import_kept_identity_symbol(entry)
return nil unless entry&.kept_identity

copy = entry.dup
copy.kept_identity = entry.kept_identity
copy
end

sig { params(params: T::Array[AST::Param]).returns(T::Array[AST::Param]) }
def self.copy_params_for_import(params)
params.map do |param|
Expand All @@ -736,7 +773,10 @@ def self.copy_params_for_import(params)
name_token: param.name_token,
required: param.required,
sync: param.sync,
symbol: nil
# The entry itself is mutable per-unit state, but kept_identity is a
# fact about the callee: drop it and an importer cannot tell the callee
# keeps the argument, so an Rc crosses the boundary without a retain.
symbol: import_kept_identity_symbol(param.symbol)
)
end
end
Expand Down
15 changes: 12 additions & 3 deletions compiler/ruby/annotator/helpers/generic_analysis.rb
Original file line number Diff line number Diff line change
Expand Up @@ -139,10 +139,19 @@ def type_annotation_facts(node, type_obj, is_param)

sig { params(type_obj: Type).returns(Type) }
def type_annotation_inner(type_obj)
return T.must(type_obj.payload_type) if type_obj.error_union?
return T.must(type_obj.wrapped_type) if type_obj.optional?
# Tense prefixes stack (`!?T`), so peel every layer -- a single unwrap
# leaves `!?String[]@set` looking like a non-array to the shape checks.
inner = type_obj
loop do
next_inner = if inner.error_union?
inner.payload_type
elsif inner.optional?
inner.wrapped_type
end
return inner unless next_inner

type_obj
inner = next_inner
end
end

sig { params(facts: TypeAnnotationFacts).void }
Expand Down
Loading
Loading