Skip to content

docs: add cross-repo testing skill for SDK ↔ OH Cloud e2e workflow#2477

Open
xingyaoww wants to merge 6 commits intomainfrom
add-cross-repo-testing-skill
Open

docs: add cross-repo testing skill for SDK ↔ OH Cloud e2e workflow#2477
xingyaoww wants to merge 6 commits intomainfrom
add-cross-repo-testing-skill

Conversation

@xingyaoww
Copy link
Collaborator

@xingyaoww xingyaoww commented Mar 17, 2026

Summary

Adds .agents/skills/cross-repo-testing/SKILL.md — a skill that guides AI agents through the end-to-end testing workflow when an SDK feature requires coordinated changes in the OpenHands Cloud backend.

What it covers

  1. Repository map — explains the relationship between software-agent-sdk (agent core), OpenHands (Cloud backend), and deploy (infrastructure)
  2. Step-by-step workflow — from writing server-side changes, to deploying a staging feature environment, to running SDK e2e tests against it
  3. Key gotchas — feature-env auth isolation, dual SHA updates in deploy.yaml, DNS propagation, enterprise Docker image prerequisites

Why both repos?

This same skill is being added to the OpenHands repo in a companion PR. Agents working from either side of a cross-repo feature need this guide:

  • SDK agents need it when building features like OpenHandsCloudWorkspace.get_llm() that call Cloud APIs
  • Server agents need it when their API changes must be validated end-to-end with the SDK client

Context

This skill was distilled from the real cross-repo testing workflow performed for SDK PR #2409 + Server PR OpenHands/OpenHands#13383 (SaaS credentials inheritance for SDK-created conversations).


Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.13-nodejs22 Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:07278b2-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-07278b2-python \
  ghcr.io/openhands/agent-server:07278b2-python

All tags pushed for this build

ghcr.io/openhands/agent-server:07278b2-golang-amd64
ghcr.io/openhands/agent-server:07278b2-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:07278b2-golang-arm64
ghcr.io/openhands/agent-server:07278b2-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:07278b2-java-amd64
ghcr.io/openhands/agent-server:07278b2-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:07278b2-java-arm64
ghcr.io/openhands/agent-server:07278b2-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:07278b2-python-amd64
ghcr.io/openhands/agent-server:07278b2-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-amd64
ghcr.io/openhands/agent-server:07278b2-python-arm64
ghcr.io/openhands/agent-server:07278b2-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-arm64
ghcr.io/openhands/agent-server:07278b2-golang
ghcr.io/openhands/agent-server:07278b2-java
ghcr.io/openhands/agent-server:07278b2-python

About Multi-Architecture Support

  • Each variant tag (e.g., 07278b2-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., 07278b2-python-amd64) are also available if needed

Adds a skill that guides agents through the full end-to-end testing
workflow when an SDK feature requires changes in the OpenHands Cloud
backend. Covers: repo relationships, staging deployment via deploy
repo, feature-env auth gotchas, and result recording.

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions
Copy link
Contributor

github-actions bot commented Mar 17, 2026

Python API breakage checks — ✅ PASSED

Result:PASSED

Action log

@github-actions
Copy link
Contributor

github-actions bot commented Mar 17, 2026

REST API breakage checks (OpenAPI) — ✅ PASSED

Result:PASSED

Action log

…ng skill

Covers the reverse dependency: when the OpenHands server needs new SDK
packages/agent-server image. Steps: pin SDK to git commit, use SDK PR's
merge-commit SHA for agent-server image, regenerate lock files, wait for
enterprise image build, then deploy.

Co-authored-by: openhands <openhands@all-hands.dev>
@xingyaoww xingyaoww marked this pull request as ready for review March 17, 2026 12:52
Copy link
Collaborator

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taste Rating: 🟡 Acceptable - Comprehensive documentation solving a real cross-repo testing pain point.

[IMPROVEMENT OPPORTUNITIES]

One minor clarity improvement suggested inline, but the document is ready to merge as-is.

VERDICT:Worth merging - Accurate, comprehensive documentation distilled from real workflow experience (PR #2409).

KEY INSIGHT: Complex workflows can't always be simplified away—the best you can do is document them clearly with concrete examples, which this does well.

@xingyaoww xingyaoww requested review from enyst and neubig March 17, 2026 14:35
Co-authored-by: Engel Nyst <engel.nyst@gmail.com>
@@ -0,0 +1,214 @@
---
name: cross-repo-testing
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
name: cross-repo-testing
name: saas-cloud-workspace-testing

This skill is fully tied to deploy/, I’m not sure… I think it’s not useful for anyone not in AH.

It’s not really about e2e testing with the cloud, either; I can test e2e with the cloud. 🤔

I wonder if there’s some alternative… I understand you need it now. And I’m not sure a non-default marketplace works on the cloud… You’re using the SDK repo on the cloud, right?

Copy link
Collaborator Author

@xingyaoww xingyaoww Mar 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i agree this is not a great idea... Maybe we can hold off merging this, and we can first figure out how we can get these as custom secrets so it doesn't ends up in the open-source repo

Copy link
Collaborator

@enyst enyst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for my hesitations, Xingyao, honestly I doubt we have a lot of choice - you need this in context, but any other contributor to the SDK should, maybe to the contrary, not have it in context. 🫠

We really need your work on automations ❤️ so IMHO we could take this PR, though I kinda hope we can remove it soon / move it somewhere or somehow not interfere with everyone else’s attempt to work with the repo…

I think non-default marketplaces work in the SDK, and skills can individually be specified in them. So maybe that will be an option, though I don’t know if it works yet on the cloud, WDYT?

Co-authored-by: openhands <openhands@all-hands.dev>
@neubig neubig removed their request for review March 18, 2026 14:34
Co-authored-by: Engel Nyst <engel.nyst@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants