Description
A requested feature is a new command line tool, say cmsReRunSubmission
, that takes the ID of a submission (plus, possibly, a set of testcase IDs, or a regex), compiles and evaluates the submission on a worker (in the same conditions as would normally happen in a contest) but forcefully keeps the sandbox and doesn't store the result (so, in particular, it doesn't overwrite the score): instead it prints the results on the console where the command is run (say, score, time and memory consumption, and the paths to the sandboxes where the compilation/evaluation happened).
Some changes are needed for this:
- Jobs needs to support a
keep_sandbox
flag that overrides the one in the configuration; - EvalutionService needs to provide an RPC endpoint where arbitrary JobGroups can be provided for insertion in the queue and worker pool.
Additionally a "local" mode could also be provided, that compiles and evaluates the submission locally on the machine where the script is run. This would change the conditions in which the operations run but would make it easier and faster to access the data.