-
-
Notifications
You must be signed in to change notification settings - Fork 386
Feature: expose exclude_columns on pandas profile accessor #1374
Copy link
Copy link
Open
Labels
area:integrationsIntegrations with pandas, NumPy, scikit-learn, DuckDB, Arrow, and notebooksIntegrations with pandas, NumPy, scikit-learn, DuckDB, Arrow, and notebooksarea:pandas-interopto_pandas, from_pandas, dtype compatibility, pandas workflowsto_pandas, from_pandas, dtype compatibility, pandas workflowsarea:qualityprofile, suggest_cleaning, auto_clean, quality reportingprofile, suggest_cleaning, auto_clean, quality reportingdifficulty:beginnerGood for new contributors with limited project contextGood for new contributors with limited project contextgssocPart of the GSSoC 2026 contributor programPart of the GSSoC 2026 contributor programgssoc:good-first-issueBeginner-friendly GSSoC task with clear scopeBeginner-friendly GSSoC task with clear scopegssoc:level-1GSSoC beginner-level taskGSSoC beginner-level tasklevel:beginnerGSSoC scoring label for beginner-level merged PRsGSSoC scoring label for beginner-level merged PRspriority:lowNice to have or low-risk improvementNice to have or low-risk improvementsize:xsTiny change; often docs, tests, validation, or one small fixTiny change; often docs, tests, validation, or one small fixstatus:claimedA contributor is currently working on thisA contributor is currently working on thistype:featureGSSoC-compatible feature labelGSSoC-compatible feature label
Description
Metadata
Metadata
Assignees
Labels
area:integrationsIntegrations with pandas, NumPy, scikit-learn, DuckDB, Arrow, and notebooksIntegrations with pandas, NumPy, scikit-learn, DuckDB, Arrow, and notebooksarea:pandas-interopto_pandas, from_pandas, dtype compatibility, pandas workflowsto_pandas, from_pandas, dtype compatibility, pandas workflowsarea:qualityprofile, suggest_cleaning, auto_clean, quality reportingprofile, suggest_cleaning, auto_clean, quality reportingdifficulty:beginnerGood for new contributors with limited project contextGood for new contributors with limited project contextgssocPart of the GSSoC 2026 contributor programPart of the GSSoC 2026 contributor programgssoc:good-first-issueBeginner-friendly GSSoC task with clear scopeBeginner-friendly GSSoC task with clear scopegssoc:level-1GSSoC beginner-level taskGSSoC beginner-level tasklevel:beginnerGSSoC scoring label for beginner-level merged PRsGSSoC scoring label for beginner-level merged PRspriority:lowNice to have or low-risk improvementNice to have or low-risk improvementsize:xsTiny change; often docs, tests, validation, or one small fixTiny change; often docs, tests, validation, or one small fixstatus:claimedA contributor is currently working on thisA contributor is currently working on thistype:featureGSSoC-compatible feature labelGSSoC-compatible feature label
Summary
ar.profile()supportsexclude_columns, but the pandas accessor wrapperdf.arnio.profile()does not expose that option.This creates an integration parity gap: pandas users can profile through Arnio, but they cannot use the privacy/noisy-column exclusion path without leaving the accessor API and manually converting to
ArFrame.Why it matters
The pandas accessor is meant to make Arnio easy to use inside existing pandas workflows.
exclude_columnsis important for production profiling because datasets often contain sensitive, high-cardinality, or irrelevant columns such as IDs, tokens, emails, or free-text fields.Users should be able to write:
instead of switching APIs for the same profiling feature.
Evidence
arnio/quality.py:profile(frame, *, exclude_columns=None, ...)supports excluding columns and validates the requested names.arnio/integrations/pandas.py:ArnioPandasAccessor.profile()forwards profiling options but has noexclude_columnsparameter.tests/test_pandas_accessor.py: accessor tests cover basic profiling but do not cover excluding columns throughdf.arnio.profile().Reproduction or trigger
df.arnio.profile(exclude_columns=["secret"]).TypeErrorbecause the accessor method does not accept the parameter, even though the underlyingar.profile()API does.Expected behavior
df.arnio.profile()should acceptexclude_columnsand forward it toar.profile().Actual behavior
df.arnio.profile()exposes only sample/top-value parameters, soexclude_columnscannot be used through the pandas accessor.Suggested direction
exclude_columns: Sequence[str] | None = NonetoArnioPandasAccessor.profile().profile(self.to_arframe(), exclude_columns=exclude_columns, ...).DataQualityReport.References
arnio/integrations/pandas.pyarnio/quality.pytests/test_pandas_accessor.pyPriority: low