Skip to content

Reject Test Instance Submissions with "bad" YAML #24

@wmwolf

Description

@wmwolf

Test instances that are missing mandatory information (broadly defined), like a test case name, or a failure_type (if passed is false) should never make it into the database and should be spat back to the submitter while accepting the rest. Might also spit back unrecognized keys since they are likely wrong, or at least as a reminder to the developer to make them do something on the backend.

Background

mesa handles creating the YAML files, and mesa_test handles submitting them. In principle, mesa_test could check the YAML files for consistency so it never makes it to the server, but that would require mesa_test to be "smart", and more importantly, updated. Kicking this up to the testhub means that mesa_test can remain a blameless messenger between the testing scripts of mesa and the database in the testhub.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions