feat: add distributed browser load-testing framework#2112
Open
PeterBaker0 wants to merge 24 commits into
Open
Conversation
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
…twork smashing and timeouts Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disclaimer
This load testing framework is experimental and should not be pointed at prod. For a couple of reasons
With this being said, the purpose of this is a throwaway infra stack to scale up a distributed agent using the app - it is not on the critical app path so doesn't need to be as well engineered from a code quality perspective.
I have tested this up to 30 concurrent agents on AWS for 30 minutes and works - we see significant DB stress on our AWS dashboards + other key metrics in the grafana dashboard. The DASS plan will be to deploy a temporary prod clone with no real data, users or risk, which emulates the performance characteristics of our real prod - we can then not worry about security posture of this ephemeral infra.
With this in mind, I don't think this PR needs to be heavily scrutinised from a code quality perspective, however the integration points with the 'main' code (e.g. instrumentation), the overall architecture/approach and other aspects are worth considering. If this can't make it into the main codebase - I may just maintain a fork or branch with this code in it, as it is useful enough that I will use it for the DASS regardless of the outcome of this review, but would rather see it contribute to the open source repo.
AI generated PR below:
Summary
Adds a distributed, browser-based load-testing framework for the FAIMS3 collection app. A coordinator executes JSON sequence plans and assigns steps to Playwright agents; metrics flow to Prometheus/Grafana via Pushgateway. Includes local docker-compose workflow, AWS CDK infra for on-demand ECS Fargate runs, and supporting app instrumentation + test hooks.
Not for production — designed for ephemeral staging/dev environments with disposable test accounts.
What's included
Load-testing framework (
load-testing/)loop/split/ phase steps), progress reporting, metrics pushGET /step; scenarios for onboarding, online/offline collection, patchy network, and export stress@faims3/load-testing-shared) — Zod schemas, coordinator client, bundled sequence plans and collection profiles (declarative record-fill workflows)run-load-test.sh, account seeding, coordinator status polling, cost estimation, debug helpersApp & library support
@faims3/instrumentation— User Timing API wrapper with standardiseddass.*measure namesdata-testidhooks on activation, add-record, sync status, form navigation, and finish-anyway dialog — for stable Playwright selectorsAPI tooling
seedLoadTestAccounts.ts— create/update load-test users (no DB migrations)env-from-cdk-stack.sh— generate.envfrom a deployed Conductor ECS task definitionOther
CMDtypoArchitecture note
Plans are loaded only by the coordinator (file, inline, or S3 URI). Agents receive individual steps over HTTP and never hold plan JSON. Collection profile filenames are inlined before S3 upload on AWS runs.
Known limitations