Skip to content

Exposing transformations applied by Cleaner and TableVectorizer - #2122

Open
emassoulie wants to merge 26 commits into
skrub-data:mainfrom
emassoulie:issue-1265-expose-cleaner-and-tablevectorizer-transformations
Open

Exposing transformations applied by Cleaner and TableVectorizer#2122
emassoulie wants to merge 26 commits into
skrub-data:mainfrom
emassoulie:issue-1265-expose-cleaner-and-tablevectorizer-transformations

Conversation

@emassoulie

Copy link
Copy Markdown
Contributor

Closes #1265
Adds a method to the Cleaner and TableVectorizer transformers enabling them to give a human-readable list of the transformations (DropUninformative, ToFloat, ToDatetime) they are set to apply once fitted.

@emassoulie

Copy link
Copy Markdown
Contributor Author

I've got something going with the transformation lists for both transformers! I feel the TableVectorizer transformations could be equally relevant to expose by raw transformer type (ToFloat, ToDatetime etc, like the Cleaner), or by transformer category ("numeric", "low cardinality" etc, since they are customisable). I'm thinking of going towards the latter, but I wanted to ask for a second opinion before I go ahead.

@emassoulie
emassoulie marked this pull request as ready for review July 16, 2026 12:53
@emassoulie

Copy link
Copy Markdown
Contributor Author

It's starting to come together! The direction chosen is to output a string viewable in any terminal, and to list columns grouped by transformation. On this second point, I've been having thoughts on doing it the other way, adding a way of describing the pipeline each individual column goes through: does that seem relevant or does the current way work fine?

I've attached the output to TableVectorizer.list_transformations() after running fit on two different examples, one on the employee dataset and one on the custom-made dataset created for the test. Does it look like the kind of output we want?

Capture d’écran du 2026-07-16 14-59-52 Capture d’écran du 2026-07-16 15-09-32

Eloi Massoulié added 2 commits July 16, 2026 15:27

@rcap107 rcap107 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for the PR @emassoulie ! I did a review of the current material.

In general, I think we're on the right path. I think the main thing to decide is the actual format of the returned string. I think there is too much whitespace, so we may want to investigate how other symbols may make it easier to parse the output.

Other than that, good job!

Comment thread skrub/_table_vectorizer.py Outdated
Comment thread skrub/_table_vectorizer.py Outdated
Comment thread skrub/_table_vectorizer.py Outdated
Comment thread skrub/_table_vectorizer.py Outdated
Comment thread skrub/_table_vectorizer.py Outdated
Comment thread skrub/_table_vectorizer.py Outdated
Comment thread skrub/_table_vectorizer.py Outdated
Comment thread skrub/tests/test_table_vectorizer.py Outdated
Comment thread skrub/tests/test_table_vectorizer.py Outdated
Comment thread skrub/tests/test_table_vectorizer.py Outdated
Eloi Massoulié added 2 commits July 28, 2026 12:33

@rcap107 rcap107 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for improving the PR!

I think we're getting close to the end, though I think the code can still be improved and simplified in places.

I opened a PR targeting this branch in emassoulie#2 to propose some changes in that direction

Could you also add a mention of list_transformations in the docstring of TableVectorizer and Cleaner? Without printing the entire thing, only the first few lines followed by ... to elide the rest

Comment thread skrub/_table_vectorizer.py Outdated
Comment thread skrub/_table_vectorizer.py Outdated
Comment thread skrub/_table_vectorizer.py Outdated
Comment thread skrub/_table_vectorizer.py
Comment thread skrub/_table_vectorizer.py Outdated
Comment thread skrub/tests/test_table_vectorizer.py Outdated
Comment thread skrub/tests/test_table_vectorizer.py Outdated
Comment thread skrub/tests/test_table_vectorizer.py
Comment thread skrub/tests/test_table_vectorizer.py Outdated
Comment thread skrub/tests/test_table_vectorizer.py Outdated
@rcap107 rcap107 added this to the Release 0.10.1 milestone Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose the transformations done by the TableVectorizer

2 participants