DOC - adding dev documentation about the dataframe API and df_module - #2253
Draft
rcap107 wants to merge 6 commits into
Draft
DOC - adding dev documentation about the dataframe API and df_module#2253rcap107 wants to merge 6 commits into
rcap107 wants to merge 6 commits into
Conversation
Member
Author
|
Somewhat unrelated, we currently don't have a dedicated place that we use to keep developer-specific documentation, so we might want to rework a bit the material so that the new contributor guide and the release guide are in the same place as the pages added in this PR, though that can be in a separate PR. |
lisaleemcb
reviewed
Aug 26, 2026
|
|
||
| ``df_module`` solves a different problem: when a test needs to construct | ||
| inputs, call a function, and check the output, it must do so in a way that | ||
| works for all three configurations. ``df_module`` provides a uniform |
Contributor
There was a problem hiding this comment.
configurations is sort of vague here. maybe something about the 3 supported dataframe types?
It specifies it further below but I think it could be moved up to make it clearer.
| itself is backend-specific. | ||
|
|
||
| The design principle is the same in both cases: write the general case once | ||
| and isolate backend differences to dedicated, clearly labelled places. |
Contributor
There was a problem hiding this comment.
Suggested change
| and isolate backend differences to dedicated, clearly labelled places. | |
| and isolate backend differences to dedicated, clearly labeled places. |
|
|
||
| skrub targets both pandas and polars as first-class backends. Rather than | ||
| scattering ``if pandas … else polars …`` branches throughout the codebase, all | ||
| dataframe and column operations are funnelled through a thin dispatch layer that |
Contributor
There was a problem hiding this comment.
Suggested change
| dataframe and column operations are funnelled through a thin dispatch layer that | |
| dataframe and column operations are funneled through a thin dispatch layer that |
| ``df_module`` is defined in ``skrub/conftest.py`` and returns a | ||
| :class:`types.SimpleNamespace` with a consistent set of attributes. The | ||
| attributes are designed to normalise the differences between libraries so test | ||
| bodies need no ``if pandas / if polars`` branches (with few exceptions). |
Contributor
There was a problem hiding this comment.
Suggested change
| bodies need no ``if pandas / if polars`` branches (with few exceptions). | |
| bodies need no ``if pandas / if polars`` branches (with a few exceptions). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is adding documentation about the dispatched dataframe API and the df_module and testing framework.
For the moment, they have not been added to the rendered documentation, so they're only available from inside the repository. We can decide to put them on the website if needed.
I asked an agent to generate the bulk of the material, then I reviewed it to remove/rephrase some of the content. I'll leave this in draft mode to get a few more eyes on and trim it more if necessary.