Releases: tokens-studio/tokenscript-interpreter
v0.17.0
Added
- Persistent color alpha channel: Added
alphaproperty toColorSymbol(stored outsidevaluechannels, defaults tonull). Alpha is now preserved across all color format conversions (RGB → HSL → Hex, etc.). All color types (RGB, HSL, Hex) now support alpha via.alphaproperty. CSS color formatting includes alpha as fourth parameter when set and < 1 (e.g.,rgb(255, 0, 0, 0.5)).
Breaking Changes
- Color alpha property: Replace
rgbaColor.value.awithrgbaColor.alpha - Color alpha access: Use
color.alphaproperty on any color type instead of RGBA-specific handling - ColorSymbol constructor: Constructor signature changed from
(value, subType?, config?, alpha?)to(value, subType?, alpha?, config?). Update any directColorSymbolconstructor calls to use new parameter order.
What's Changed
Full Changelog: v0.16.1...v0.17.0
v0.16.1
Added
-
Script mode for REPL: Two execution modes for interactive development
- Inline mode (default): Executes each line immediately, like traditional REPLs
- Script mode (
--mode script): Accumulates lines into a multi-statement script for complex workflows - Smart output handling: Only prints assigned/reassigned values and expression results
- New commands:
exit()to quit,clear()to reset script buffer
-
Interpreter
- Public
getSymbol()method on Interpreter for accessing symbol table values
- Public
What's Changed
Full Changelog: v0.16.0...v0.16.1
v0.16.0
Changed
-
BREAKING: Cleaned up processor output interface: Removed internal properties from public API
- Removed
unresolvedproperty fromProcessorResult- internal tracking no longer exposed - Removed
subFieldPathsproperty fromProcessorResult- internal tracking no longer exposed - Added
getSubFieldPaths()method onPrefixResolverfor internal use - Simplified
ProcessorOutputby removing redundant filtering logic in builder - Users should rely on
tokensMap for resolved values andissuesMap for problems
- Removed
-
BREAKING: Unified error tracking in
issuesMap: Removed separateerrorsMap from processor outputProcessorOutputno longer includeserrorsproperty - all errors now tracked inissuesMap- Added
getTokenError()helper to extract Error objects from issues - Added
tokenHasError()helper to check if token has error vs lint issue - Errors in issues can be identified by presence of
codeproperty (vs lint issues) - Simplifies error handling with single source of truth for all token problems
- Migration: Replace
result.errors.get(token)withgetTokenError(result.issues, token) - Migration: Replace
result.errors.has(token)withtokenHasError(result.issues?.get(token))
What's Changed
Full Changelog: v0.15.1...v0.16.0
v0.15.1
Fixed
- Phantom Token Resolution: Resolved type-safety issue in missing dependency tracking
- Removed unsafe type casts (
error as unknown as InterpreterResult) inreferenceCache - Added dedicated
missingDependenciesset to track references to non-existent tokens - Ensured
referenceCacheonly contains validInterpreterResultvalues (ISymbolType | string | null) - Fixed circular dependency detection to correctly report
CIRCULAR_DEPENDENCYinstead ofTOKEN_NOT_FOUND - Prevents phantom tokens (missing dependencies) from appearing in resolved output
- Removed unsafe type casts (
What's Changed
Full Changelog: v0.15.0...v0.15.1
v0.15.0
Added
-
CSS and Penpot Preset Validators: Pre-built validation rulesets for token types
css.createLintRunner(): Standards-compliant CSS validators (opacity, fontWeight, borderRadius, boxShadow, lineHeight, letterSpacing, textTransform, textDecoration)penpot.createLintRunner(): Penpot-specific validators extending CSS rules (typography, shadow, strokeWidth)- Composable primitives:
number(),string(),boolean(),color(),numberWithUnit() - Combinators:
or(),all(),oneOrList(),list(),struct(),arrayOf()
-
LintRunner Extension API: Composable rule system with
.extend()method- Override specific validators while preserving others
- Combine validators using
all()combinator - Immutable API returns new instances
-
Structured Token Field-Level Linting: Validation with field-level granularity
pathproperty onLintIssueidentifies specific fields (e.g.,["fontSize"]or[0, "blur"])- Supports Map-based (typography) and array-based (box-shadow) tokens
- Cross-field validation (e.g., lineHeight requires fontSize)
-
TokenSymbol Validator Support: Validators can use
.get(),.keys(),.values(),.length()methods -
Helper Functions:
getAffectedTokens(),getBrokenReferences(),getModifiedDependants(),getRenamedReferences()
Changed
-
BREAKING: LintResult Type: Changed from object to
Map<RefPath, LintIssue[]>- Before:
result.lint.errors,result.lint.warnings,result.lint.hasErrors - After:
result.issues(Map), filter byissue.severity
- Before:
-
BREAKING: LintIssue Structure:
- Removed
ruleIdproperty (usecodeinstead) - Changed
tokenNametype fromstringtoRefPath - Added optional
pathproperty:(string | number)[]
- Removed
-
BREAKING: CreateIssueFn Signature: Now accepts full issue object instead of individual parameters
-
BREAKING: Removed
aggregateResults(): Work directly withMap<RefPath, LintIssue[]> -
ProcessorOutput: Renamed
lintIssuestoissues(now includes lint issues and language errors) -
Error Handling:
DependencyErrorclass replaced withcreateDependencyError()helper returningProcessorError -
CRUD Result Types: Unified to
TokenOperationResultwithtokens,resolved,issues,dependantsproperties -
Circular dependency handling: Circular dependencies no longer throw errors but are handled gracefully
What's Changed
Full Changelog: v0.14.0...v0.15.0
v0.14.0
Changed
-
BREAKING: Builder output refactoring: Simplified builder architecture and separated string vs symbol outputs
- Any output is now defined by the
builderproperty - Migration: Replace
{ output: "string" }with{ builder: new StringMapBuilder() }for string Map output - Migration: Replace
{ output: "symbols" }by removing the option (symbols are now the default)
- Any output is now defined by the
-
Enhanced CRUD operations with linter support: All TokenResolver CRUD methods now return lint results
createToken(),updateToken(), anddeleteToken()now returnlintIssuesarray in their results- Linter is preserved across CRUD operations when passed to
buildTokens()orprocessTokens() - Enables real-time validation feedback during token editing workflows
Added
-
buildTokens()export: Core token processing function now available in public API- Skips normalization overhead when you already have
Map<string, TokenData> - Lower-level API for performance-critical applications
- Skips normalization overhead when you already have
-
Error class exports: All error classes now exported from main library entry point
- New
errorClassesexport includes:LanguageError,LexerError,ParserError,InterpreterError,ProcessorError,DependencyError - Enables consumers to handle and throw interpreter errors properly
- Useful for custom validators and error handling in applications
- New
-
isList,isDictionarytype guard helpers: Added type checking utilities for List and Dictionary symbols
What's Changed
Full Changelog: v0.13.3...v0.14.0
v0.13.3
- Exported
linter
Full Changelog: v0.13.2...v0.13.3
v0.13.2
v0.13.1
Added
- Exported
renameReferencesutility function from library
What's Changed
Full Changelog: v0.13.0...v0.13.1
v0.13.0
Added
- Symbol
toJs()method: All symbol types now have atoJs()method for converting to plain JavaScript values- Supports
ToJsOptionswithrecursive(default: true) andstringify(default: false) options - NumberWithUnitSymbol: Returns
{ value: number | null, unit: string }by default, or string whenstringify: true - ColorSymbol: Returns structured object with
typefield and component values by default- Hex colors:
{ type: "hex", value: "#ff0000" } - Dynamic colors:
{ type: "rgb", r: 255, g: 0, b: 0, ... } - String representation when
stringify: true
- Hex colors:
- ListSymbol, DictionarySymbol, TokenSymbol: Recursively convert nested values with
recursiveoption - Value types extracted and reused:
NumberValue,StringValue,BooleanValue,NumberWithUnitValue,ColorValue - Comprehensive test coverage across all symbol types
- Supports
Changed
- Removed
symbolTypeToJsValue()function: Developers should callsymbol.toJs()directly instead - Updated
serializeInterpreterResult(): Now acceptsToJsOptionsparameter for customization - Builder serialization: Uses
stringify: trueoption to maintain backward-compatible string output forNumberWithUnitSymbolandColorSymbol
Added
-
TokenResolver CRUD operations: Complete token lifecycle management with dependency tracking
createToken(): Add new tokens with automatic dependency graph updatesupdateToken(): Modify existing tokens with options for renaming and reference updatesupdateReferencesoption: Automatically updates all dependent tokens when renaming- Returns
renamedReferencesset tracking which tokens had references updated - Returns
brokenReferencesset when renaming withoutupdateReferences
deleteToken(): Remove tokens with broken reference detection- Returns
brokenReferencesset identifying tokens with missing dependencies
- Returns
- All operations return affected tokens and dependency subgraphs for UI updates
- Full integration test coverage for complex CRUD workflows
-
AST utilities for reference management
walkAST(): Traverse AST nodes with visitor patternfilterAST(): Collect matching nodes into flat arraycollectReferenceNodes(): Extract all reference nodes, optionally filtered by namerenameReferences(): Safely rename token references in expressions using AST- Token positions (
pos,endPos) now tracked on all tokens for precise text manipulation
-
Enhanced token position tracking
- Lexer now records start and end positions for all tokens
- Enables precise source code manipulation for renames and updates
- Reference tokens include exact character ranges in original string
Changed
-
BREAKING: Removed
@tokenscript/stencil-componentspackage- Stencil web components moved out of monorepo to reduce dependencies
- Removed from CI/CD pipeline and build scripts
- Examples no longer depend on components package
- Runtime UI example now uses native form implementation
-
Enhanced error codes
- Added
ProcessorErrorCode.TOKEN_ALREADY_EXISTSfor duplicate token detection - Improved error messages with structured data for better debugging
- Added
Fixed
-
Build configuration preserves class names in production bundles
- Added
keepNames: trueto tsup build configs to prevent Symbol class name mangling - Ensures runtime type checking and debugging remain accurate in minified builds
- Added
-
Test suite cleanup
- Removed debug console.log statements from reference method tests
Removed
- Stencil Components package (
packages/stencil-components/)<token-form>web component and related infrastructure- Stencil build configuration and dependencies
- CI/CD integration for component builds and tests
- Example integrations using Stencil components
What's Changed
- Add token crud operations by @floscr in #75
- Fix vercel build by @floscr in #76
- DevExp Fixes by @floscr in #77
- Add .toJs interface method by @floscr in #78
- Add release script by @floscr in #79
- Prevent library rebuild on CI by @floscr in #80
Full Changelog: v0.12.1...v0.13.0