Skip to content

Compute scaling state from non-missing observations - #277

Open
lllleolin-max wants to merge 1 commit into
matthewwardrop:mainfrom
lllleolin-max:codex/scale-observed-values-with-missing-data
Open

lllleolin-max wants to merge 1 commit into
matthewwardrop:mainfrom
lllleolin-max:codex/scale-observed-values-with-missing-data

Conversation

@lllleolin-max

Copy link
Copy Markdown

A single NaN currently makes center(x) and scale(x) return NaN for every observation, so the default missing-value policy drops the whole sample. Compute each column's mean, sum of squares and observation count from non-missing values while leaving the original missing rows as NaN. This follows the documented R scale semantics, including uncentered root-mean-square scaling. Tests verify column-specific counts, ddof, reused training state and end-to-end materialization.

Validation: the added regression tests fail on main (81e675b) and pass with this patch. Related tests: python -m pytest tests/transforms/test_scale.py tests/transforms/test_patsy_compat.py -q. Changed files pass ruff check and ruff format.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant