-
Notifications
You must be signed in to change notification settings - Fork 1
Description
Test instances that are missing mandatory information (broadly defined), like a test case name, or a failure_type
(if passed
is false
) should never make it into the database and should be spat back to the submitter while accepting the rest. Might also spit back unrecognized keys since they are likely wrong, or at least as a reminder to the developer to make them do something on the backend.
Background
mesa
handles creating the YAML files, and mesa_test
handles submitting them. In principle, mesa_test
could check the YAML files for consistency so it never makes it to the server, but that would require mesa_test
to be "smart", and more importantly, updated. Kicking this up to the testhub means that mesa_test
can remain a blameless messenger between the testing scripts of mesa
and the database in the testhub.