Skip to content

Releases: tokens-studio/tokenscript-interpreter

v0.17.0

29 Dec 12:46

Choose a tag to compare

Added

  • Persistent color alpha channel: Added alpha property to ColorSymbol (stored outside value channels, defaults to null). Alpha is now preserved across all color format conversions (RGB → HSL → Hex, etc.). All color types (RGB, HSL, Hex) now support alpha via .alpha property. CSS color formatting includes alpha as fourth parameter when set and < 1 (e.g., rgb(255, 0, 0, 0.5)).

Breaking Changes

  • Color alpha property: Replace rgbaColor.value.a with rgbaColor.alpha
  • Color alpha access: Use color.alpha property on any color type instead of RGBA-specific handling
  • ColorSymbol constructor: Constructor signature changed from (value, subType?, config?, alpha?) to (value, subType?, alpha?, config?). Update any direct ColorSymbol constructor calls to use new parameter order.

What's Changed

Full Changelog: v0.16.1...v0.17.0

v0.16.1

29 Dec 11:07

Choose a tag to compare

Added

  • Script mode for REPL: Two execution modes for interactive development

    • Inline mode (default): Executes each line immediately, like traditional REPLs
    • Script mode (--mode script): Accumulates lines into a multi-statement script for complex workflows
    • Smart output handling: Only prints assigned/reassigned values and expression results
    • New commands: exit() to quit, clear() to reset script buffer
  • Interpreter

    • Public getSymbol() method on Interpreter for accessing symbol table values

What's Changed

  • feat: adds script mode to repl by @floscr in #89

Full Changelog: v0.16.0...v0.16.1

v0.16.0

17 Dec 18:48

Choose a tag to compare

Changed

  • BREAKING: Cleaned up processor output interface: Removed internal properties from public API

    • Removed unresolved property from ProcessorResult - internal tracking no longer exposed
    • Removed subFieldPaths property from ProcessorResult - internal tracking no longer exposed
    • Added getSubFieldPaths() method on PrefixResolver for internal use
    • Simplified ProcessorOutput by removing redundant filtering logic in builder
    • Users should rely on tokens Map for resolved values and issues Map for problems
  • BREAKING: Unified error tracking in issues Map: Removed separate errors Map from processor output

    • ProcessorOutput no longer includes errors property - all errors now tracked in issues Map
    • Added getTokenError() helper to extract Error objects from issues
    • Added tokenHasError() helper to check if token has error vs lint issue
    • Errors in issues can be identified by presence of code property (vs lint issues)
    • Simplifies error handling with single source of truth for all token problems
    • Migration: Replace result.errors.get(token) with getTokenError(result.issues, token)
    • Migration: Replace result.errors.has(token) with tokenHasError(result.issues?.get(token))

What's Changed

Full Changelog: v0.15.1...v0.16.0

v0.15.1

16 Dec 14:36

Choose a tag to compare

Fixed

  • Phantom Token Resolution: Resolved type-safety issue in missing dependency tracking
    • Removed unsafe type casts (error as unknown as InterpreterResult) in referenceCache
    • Added dedicated missingDependencies set to track references to non-existent tokens
    • Ensured referenceCache only contains valid InterpreterResult values (ISymbolType | string | null)
    • Fixed circular dependency detection to correctly report CIRCULAR_DEPENDENCY instead of TOKEN_NOT_FOUND
    • Prevents phantom tokens (missing dependencies) from appearing in resolved output

What's Changed

  • Fix phantom tokens in token output by @floscr in #86

Full Changelog: v0.15.0...v0.15.1

v0.15.0

16 Dec 11:57

Choose a tag to compare

Added

  • CSS and Penpot Preset Validators: Pre-built validation rulesets for token types

    • css.createLintRunner(): Standards-compliant CSS validators (opacity, fontWeight, borderRadius, boxShadow, lineHeight, letterSpacing, textTransform, textDecoration)
    • penpot.createLintRunner(): Penpot-specific validators extending CSS rules (typography, shadow, strokeWidth)
    • Composable primitives: number(), string(), boolean(), color(), numberWithUnit()
    • Combinators: or(), all(), oneOrList(), list(), struct(), arrayOf()
  • LintRunner Extension API: Composable rule system with .extend() method

    • Override specific validators while preserving others
    • Combine validators using all() combinator
    • Immutable API returns new instances
  • Structured Token Field-Level Linting: Validation with field-level granularity

    • path property on LintIssue identifies specific fields (e.g., ["fontSize"] or [0, "blur"])
    • Supports Map-based (typography) and array-based (box-shadow) tokens
    • Cross-field validation (e.g., lineHeight requires fontSize)
  • TokenSymbol Validator Support: Validators can use .get(), .keys(), .values(), .length() methods

  • Helper Functions: getAffectedTokens(), getBrokenReferences(), getModifiedDependants(), getRenamedReferences()

Changed

  • BREAKING: LintResult Type: Changed from object to Map<RefPath, LintIssue[]>

    • Before: result.lint.errors, result.lint.warnings, result.lint.hasErrors
    • After: result.issues (Map), filter by issue.severity
  • BREAKING: LintIssue Structure:

    • Removed ruleId property (use code instead)
    • Changed tokenName type from string to RefPath
    • Added optional path property: (string | number)[]
  • BREAKING: CreateIssueFn Signature: Now accepts full issue object instead of individual parameters

  • BREAKING: Removed aggregateResults(): Work directly with Map<RefPath, LintIssue[]>

  • ProcessorOutput: Renamed lintIssues to issues (now includes lint issues and language errors)

  • Error Handling: DependencyError class replaced with createDependencyError() helper returning ProcessorError

  • CRUD Result Types: Unified to TokenOperationResult with tokens, resolved, issues, dependants properties

  • Circular dependency handling: Circular dependencies no longer throw errors but are handled gracefully

What's Changed

Full Changelog: v0.14.0...v0.15.0

v0.14.0

12 Dec 15:00

Choose a tag to compare

Changed

  • BREAKING: Builder output refactoring: Simplified builder architecture and separated string vs symbol outputs

    • Any output is now defined by the builder property
    • Migration: Replace { output: "string" } with { builder: new StringMapBuilder() } for string Map output
    • Migration: Replace { output: "symbols" } by removing the option (symbols are now the default)
  • Enhanced CRUD operations with linter support: All TokenResolver CRUD methods now return lint results

    • createToken(), updateToken(), and deleteToken() now return lintIssues array in their results
    • Linter is preserved across CRUD operations when passed to buildTokens() or processTokens()
    • Enables real-time validation feedback during token editing workflows

Added

  • buildTokens() export: Core token processing function now available in public API

    • Skips normalization overhead when you already have Map<string, TokenData>
    • Lower-level API for performance-critical applications
  • Error class exports: All error classes now exported from main library entry point

    • New errorClasses export includes: LanguageError, LexerError, ParserError, InterpreterError, ProcessorError, DependencyError
    • Enables consumers to handle and throw interpreter errors properly
    • Useful for custom validators and error handling in applications
  • isList, isDictionary type guard helpers: Added type checking utilities for List and Dictionary symbols

What's Changed

Full Changelog: v0.13.3...v0.14.0

v0.13.3

11 Dec 10:16

Choose a tag to compare

  • Exported linter

Full Changelog: v0.13.2...v0.13.3

v0.13.2

11 Dec 10:05

Choose a tag to compare

Added

  • Exported linter

What's Changed

Full Changelog: v0.13.1...v0.13.2

v0.13.1

10 Dec 14:13

Choose a tag to compare

Added

  • Exported renameReferences utility function from library

What's Changed

  • Use crud operations in runtime UI by @floscr in #81

Full Changelog: v0.13.0...v0.13.1

v0.13.0

10 Dec 13:08

Choose a tag to compare

Added

  • Symbol toJs() method: All symbol types now have a toJs() method for converting to plain JavaScript values
    • Supports ToJsOptions with recursive (default: true) and stringify (default: false) options
    • NumberWithUnitSymbol: Returns { value: number | null, unit: string } by default, or string when stringify: true
    • ColorSymbol: Returns structured object with type field and component values by default
      • Hex colors: { type: "hex", value: "#ff0000" }
      • Dynamic colors: { type: "rgb", r: 255, g: 0, b: 0, ... }
      • String representation when stringify: true
    • ListSymbol, DictionarySymbol, TokenSymbol: Recursively convert nested values with recursive option
    • Value types extracted and reused: NumberValue, StringValue, BooleanValue, NumberWithUnitValue, ColorValue
    • Comprehensive test coverage across all symbol types

Changed

  • Removed symbolTypeToJsValue() function: Developers should call symbol.toJs() directly instead
  • Updated serializeInterpreterResult(): Now accepts ToJsOptions parameter for customization
  • Builder serialization: Uses stringify: true option to maintain backward-compatible string output for NumberWithUnitSymbol and ColorSymbol

Added

  • TokenResolver CRUD operations: Complete token lifecycle management with dependency tracking

    • createToken(): Add new tokens with automatic dependency graph updates
    • updateToken(): Modify existing tokens with options for renaming and reference updates
      • updateReferences option: Automatically updates all dependent tokens when renaming
      • Returns renamedReferences set tracking which tokens had references updated
      • Returns brokenReferences set when renaming without updateReferences
    • deleteToken(): Remove tokens with broken reference detection
      • Returns brokenReferences set identifying tokens with missing dependencies
    • All operations return affected tokens and dependency subgraphs for UI updates
    • Full integration test coverage for complex CRUD workflows
  • AST utilities for reference management

    • walkAST(): Traverse AST nodes with visitor pattern
    • filterAST(): Collect matching nodes into flat array
    • collectReferenceNodes(): Extract all reference nodes, optionally filtered by name
    • renameReferences(): Safely rename token references in expressions using AST
    • Token positions (pos, endPos) now tracked on all tokens for precise text manipulation
  • Enhanced token position tracking

    • Lexer now records start and end positions for all tokens
    • Enables precise source code manipulation for renames and updates
    • Reference tokens include exact character ranges in original string

Changed

  • BREAKING: Removed @tokenscript/stencil-components package

    • Stencil web components moved out of monorepo to reduce dependencies
    • Removed from CI/CD pipeline and build scripts
    • Examples no longer depend on components package
    • Runtime UI example now uses native form implementation
  • Enhanced error codes

    • Added ProcessorErrorCode.TOKEN_ALREADY_EXISTS for duplicate token detection
    • Improved error messages with structured data for better debugging

Fixed

  • Build configuration preserves class names in production bundles

    • Added keepNames: true to tsup build configs to prevent Symbol class name mangling
    • Ensures runtime type checking and debugging remain accurate in minified builds
  • Test suite cleanup

    • Removed debug console.log statements from reference method tests

Removed

  • Stencil Components package (packages/stencil-components/)
    • <token-form> web component and related infrastructure
    • Stencil build configuration and dependencies
    • CI/CD integration for component builds and tests
    • Example integrations using Stencil components

What's Changed

Full Changelog: v0.12.1...v0.13.0