Skip to content

Run API compliance tests automatically for API-touching PRs #2440

@enyst

Description

@enyst

Problem

We have API compliance tests, but they are currently optional/manual:

  • label-triggered or manually dispatched
  • not a default enforcement path for API-touching PRs
  • configured with continue-on-error

So today they are a useful tool, but not a reliable part of public API enforcement.

What we may want

Automatically run at least a reduced API compliance suite for PRs that likely affect public APIs, for example:

  • PRs touching openhands-sdk/
  • PRs touching openhands-agent-server/
  • PRs labeled api-change
  • release PRs

Possible rollout options:

  1. Run a smoke subset automatically for likely API changes.
  2. Keep the full suite manual/label-triggered.
  3. Fail loudly or at least post a very visible result summary.

Why

Structural diff checks are necessary, but runtime/API compliance coverage gives extra confidence that behavior remains usable for real clients.

Metadata

Metadata

Assignees

No one assigned

    Labels

    invariantsthe design invariants of the codebase

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions