Goal

• Migrate the internal E2E pipeline to GitHub Actions under goharbor.

This goal was discussed in our meeting, recording is available in the Slack:
https://cloud-native.slack.com/files/U051J3DBMJS/F08MSD1P739/e2e_pipeline.mp4

The updates will be tracked on this issue.

Migration Plan

After consideration, it’s best to run these pipelines on PRs instead of nightly runs.

The rationale: if 3 - 4 PRs get merged in a day and the nightly run fails, it becomes harder to identify the issue, and the main branch ends up broken. My goal is to keep main working as much as possible.

Strategy

The idea is to:

• Break down the large test suite into smaller, isolated components.
• Run these subtests concurrently.
• Each test currently takes around 10–30 mins.
• The existing pipeline takes 45 mins or more (worst case scenario), So, I believe this approach is feasible.

If we split and run separately, total runtime may reduce to ~20-30 mins.

These tests include components such as:

• Playwright Setup (feat: Add Playwright setup #22462)
• DB.robot (Feat: Add Playwright tests for DB scenarios #22472)
• Routing.robot (Feat: Playwright tests for Routing Scenarios #22533)
• Teardown.robot ( )
• Common_GC.robot ( )
• Common.robot
• Webhook.robot #22150
• P2P_Preheat.robot
• Trivy.robot

!!Webhook.robot

This e2e test involves infra requirement such as

1. Webhook endpoint
2. Webhook endpoint UI. where it can find the triggered webhook.

So, we need to setup webhook server and with an accessible ui as defined in the robot tests.

Note

if the time to setup the environment in the github actions is too high (i.e time to setup the environemnt take more than 20 - 30 mins). Because at this stage people can't wait an hour to get the tests complete on PR. This is the overall idea. hope this make sense. we will create a deployed service under Harbor AWS acc.

The migration to playwright testcases would include all keywords inside - tests/resources/Harbor-Pages, and
All testcases inside

• Only these two testcases in https://github.com/goharbor/harbor/blob/b0e54cad7645ab9fa64180dd6589087391d04b01/tests/robot-cases/Group0-BAT/BAT.robot#L25C1-L46C18
• Group1-Nightly
• Group2-Longevity (*if needed)

Tests that will NOT be migrated.

• all of tests/apitests
• tests/robot-cases/Group0-*
• tests/robot-cases/Group3-Upgrade
• tests/robot-cases/Group2-Longevity

tests that doesn't include UI elements will be skipped.

Approximate no of tests to be migrated is just ~110 robot testcases have been found with UI elements involved. 90% of it are from Group1-Nightly. query: https://github.com/search?q=repo%3Agoharbor%2Fharbor+Init+Chrome+Driver&type=code&p=1

Feel free to help me out on this one. We are actively looking for contributors to help.

Thanks