It is frustrating that the debug: True flag is only tested once the cluster has started up. It would make more sense to check for debug: True - and other cluster-relevant flags - before starting the cluster.
I think that the same could be said for any error that results from a problem with the experiment yaml. Another example is passing a wrong argument name.
A possible solution would be a "dry run" of the config locally first, maybe.