Skip to content

job-output: configurable fallback globs for MPI rank output that bypasses the -o spool #35

Description

@ultimatile

job-output shows the scheduler spool files crewster pointed -o / -e at. Some MPI runtimes route rank stdout elsewhere: on one Fujitsu TCS deployment, mpirun under PJM was observed writing per-rank output to <workdir>/output.<jobid>/<rank>/1/stdout.1.0 while leaving the -o spool empty (the exact layout likely varies by site configuration), so a COMPLETED job that produced output looks silent in crewster job-output.

This routing is site/runtime policy rather than part of crewster's contract, so hardcoding the Fujitsu layout would be wrong. Two mechanism-only options, composable:

  • a config such as [pjm].extra_output_globs (with {job_id} substitution): when the spool is empty, job-output falls back to files matching the globs, keeping the site-specific layout in user config and only the fallback mechanism in crewster
  • minimal variant: when the job is terminal and the spool is empty, print a one-line hint that MPI runtimes may route rank output to job-local files

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions