Skip to content

Escape string literals used as stateful transform keys - #279

Open
lllleolin-max wants to merge 1 commit into
matthewwardrop:mainfrom
lllleolin-max:codex/escape-stateful-expression-keys
Open

lllleolin-max wants to merge 1 commit into
matthewwardrop:mainfrom
lllleolin-max:codex/escape-stateful-expression-keys

Conversation

@lllleolin-max

Copy link
Copy Markdown

Stateful transforms fail when their arguments contain quotation marks, backslashes or escaped newlines. The generated state/metadata lookups interpolate an expression into quoted Python source, changing its value or creating invalid syntax. Serialize each lookup key with repr and keep the same unmodified key in the state mapping. Tests exercise both metadata access and reuse of training state for strings containing either quote, backslashes, newlines and non-ASCII text.

Validation: the added regression tests fail on main (81e675b) and pass with this patch. Related tests: python -m pytest tests/utils/test_stateful_transforms.py tests/transforms/test_patsy_compat.py tests/utils/test_variables.py -q. Changed files pass ruff check and ruff format.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant