Skip to content

[FEA] Improve logging and reporting of expected pandas incompatibilites in cudf.pandas #20640

@vyasr

Description

@vyasr

Is your feature request related to a problem? Please describe.
As we work towards #18659, we are also taking note of places where incompatibilities are unavoidable. Even allowing for fallback, we are unlikely to pass every single test for a variety of reasons (tests of internal functionality, tests of flaky behavior, tests of behaviors that inherently differ on CPU and GPU like memory usage, etc). While such incompatibilities may be unavoidable, we need a way to communicate these incompatibilities to users.

Describe the solution you'd like
Currently we maintain the full list of failing tests in conftest-patch.py. We should eventually switch to instead storing the lists of failing/skipped/etc tests in a more structured file format (e.g. YAML or JSON) that can be parsed out in conftest-patch.py but also by our documentation. Then, we can easily include clear indications of why certain behaviors are not supported. We will need to come up with a sufficiently general schema for the file so that we can capture the information we need. Some of my initial unstructured thoughts here are:

  • We need a way to link to GH issues in cases where there are specific cudf or pandas (or third-party library) bugs leading to the incompatibility.
  • We need a way to include comments so that those comments are what is shown in the documentation
  • We may need a way to represent sublists since there may be many related issues with subgroupings (e.g. underneath the "uses internal APIs" heading we may have separate groups for different APIs in case we eventually need to reconsider some and not others).

Metadata

Metadata

Assignees

No one assigned

    Labels

    PythonAffects Python cuDF API.cudf.pandasIssues specific to cudf.pandasfeature requestNew feature or request

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions