Add pybind11 docs

[kwokcb]

Purpose

• Add in Python docs into the MaterialX package for all modules
• This allows for standard Python help and the ability to integrate this with things like type stubs (pyi)
  An example of this integrated into VSCode and PyCharm is shown

• Tooling used to create html docs via tools like Sphinx but does not add any reliance on such tools. Here is an example custom help page.

Changes

• This adds in C++ docs into the pybind11 C++ wrappers
• This is done by building XML Doxygen and then parsing and inserting into the C++ wrappers via
  a Python script.
• This script is provided as a utility which can be run on any MaterialX given Doxygen XML (as input) and matching target C++ PyMaterialX pair (as output). It can incrementally update docs from C++ as desired and is not an automated build step.

Workflow

The basic logic for build workflow is below. The key is whether doc building is enabled. When it is (and doxygen is available) it will parse and insert comments into the C++. This can be done a local change and checked in. The regular CI does not build docs so there is no effect. Wheels builds will build docs + install doxygen.

graph TB
    Start([Workflow Start])
    
    Start --> RepoCheck{Repository?}
    RepoCheck -->|AcademySoftwareFoundation/MaterialX| OfficialRepo[Official Repository]
    RepoCheck -->|Other Fork| ForkRepo[Fork Repository]
    
    %% Regular Build Path
    OfficialRepo --> RegularBuild[Regular Build Job]
    ForkRepo --> RegularBuild
    
    RegularBuild --> CMakeBuild[CMake Build]
    
    CMakeBuild --> DoxygenRegular{MATERIALX_BUILD_DOCS?}
    DoxygenRegular -->|ON| GenDocsRegular[MaterialXDocs target<br/>Generate Doxygen XML]
    DoxygenRegular -->|OFF| SkipDocs[No Documentation]
    
    GenDocsRegular --> PyBindDocsRegular[PyBindDocs target<br/>pybind_docs.py extracts XML]
    
    PyBindDocsRegular --> ForceCheckRegular{MATERIALX_PYTHON_FORCE_REPLACE_DOCS?}
    ForceCheckRegular -->|ON| ForceReplace[--force flag<br/>Replace existing docs]
    ForceCheckRegular -->|OFF| NoForce[Update only if newer]
    
    ForceReplace --> PyModulesRegular[Build PyMaterialX modules<br/>with embedded docs]
    NoForce --> PyModulesRegular
    SkipDocs --> PyModulesNoDoc[Build PyMaterialX modules<br/>without docs]
    
    PyModulesRegular --> UploadRegular[Upload Artifact]
    PyModulesNoDoc --> UploadRegular
    
    %% Wheels Build Path
    OfficialRepo --> SDist[SDist Job]
    
    SDist --> WheelsJob[Wheels Job]
    
    WheelsJob --> OSSetup{OS-Specific Setup}
    
    OSSetup -->|Linux| LinuxDoxy[yum install doxygen]
    OSSetup -->|MacOS| MacDoxy[brew install doxygen]
    OSSetup -->|Windows| WinDoxy[choco install doxygen]
    
    LinuxDoxy --> BuildWheel[Build Wheel<br/>CMake runs inside cibuildwheel]
    MacDoxy --> BuildWheel
    WinDoxy --> BuildWheel
    
    BuildWheel --> DoxygenWheel{CMake finds Doxygen?}
    DoxygenWheel -->|Yes| GenDocsWheel[MaterialXDocs target<br/>Generate Doxygen XML]
    DoxygenWheel -->|No| WheelNoDocs[Skip docs]
    
    GenDocsWheel --> PyBindDocsWheel[PyBindDocs target<br/>pybind_docs.py extracts XML]
    
    PyBindDocsWheel --> ForceCheckWheel{FORCE flag?}
    ForceCheckWheel -->|Set| ForceReplaceWheel[--force flag<br/>Replace existing docs]
    ForceCheckWheel -->|Not set| NoForceWheel[Update only if newer]
    
    ForceReplaceWheel --> PyModulesWheel[Build PyMaterialX modules<br/>with embedded docs]
    NoForceWheel --> PyModulesWheel
    WheelNoDocs --> PyModulesWheelNoDoc[Build PyMaterialX modules<br/>without docs]
    
    PyModulesWheel --> WheelPackage[Package into wheel]
    PyModulesWheelNoDoc --> WheelPackage
    WheelPackage --> UploadWheel[Upload Wheel]
    
    %% Styling - only color decision nodes
    style DoxygenRegular fill:#404040,stroke:#666,stroke-width:2px,color:#fff
    style DoxygenWheel fill:#404040,stroke:#666,stroke-width:2px,color:#fff
    style ForceCheckRegular fill:#404040,stroke:#666,stroke-width:2px,color:#fff
    style ForceCheckWheel fill:#404040,stroke:#666,stroke-width:2px,color:#fff
    style RepoCheck fill:#404040,stroke:#666,stroke-width:2px,color:#fff
    style OSSetup fill:#404040,stroke:#666,stroke-width:2px,color:#fff

Loading

[kwokcb]

This runs very fast and will allow anyone to parse the XML docs as desired.

[kwokcb]

This can be updated / modified as desired. I would like to check it in, even if it's an internal script.

[jstone-lucasfilm]

This looks promising, @kwokcb, and I'm linking this to our long-standing GitHub Issue for Python API documentation:

#342

For our first Dev Days event, @StefanHabel started a project to implement this feature, and he ended up making some great improvements to MaterialX Python in general, but we were never able to come up with a solution that avoided duplication between the documentation strings for C++ and Python.

In your latest proposal, there's still duplication of doc strings, but it seems slightly clearer how a developer would navigate this when they add a new function to C++ and Python , as the doc strings are now part of the Python bindings themselves.

I'd be very interested in thoughts from the community on this proposal, to judge whether the benefits of Python API documentation would be worth the additional burden we'd be placing on developers when they add or modify API methods.

[kwokcb]

There is a variant that could be done but it's really not as developer friendly. Basically we put doc strings in separate resource files and then include that file in C++ and pybind C++ and write macros to insert. This however is a lot of work on the C++ side.

The best I could come up with to build Doxygen + pybind insertion via script. It could be run as needed to update, or add in missing pybind docs before a release.

[kwokcb]

I've added in an build step so doc building is done if Python is being built and Docs are building built.
This changes not for the regular workflow since docs are not built in main.yml.

[kwokcb]

We don't want the html docs files for now in the package. If / when html docs are build for Python they can be included.

[kwokcb]

Auto build docs. Replaces all existing docs.

[kwokcb]

Run doc extraction.

[kwokcb]

Option to allow replacement of docs at build level. Used for wheels and can be used in local or CI builds.

[kwokcb]

@jstone-lucasfilm . All checked in Py* files could be reverted as I've made it so that it can build as part of wheel building. This leaves a "single source of truth" for docs. Either way works (build and check-in) or build only on when building packages. I did not add in a non-wheels route but it would not be hard to do.