Feature/singularity support#64
Open
lucas-ebi wants to merge 14 commits into
Open
Conversation
- Add use_singularity, singularity_image, and singularity_bind_paths parameters - Implement _wrap_with_singularity() method to execute commands inside container - Automatically bind necessary paths (onedep_root, log_dir, run_dir, /tmp) - Support custom bind paths via singularity_bind_paths parameter - Set required environment variables (WWPDB_SITE_ID, WWPDB_SITE_LOC, ONEDEP_PATH) - Add CLI arguments for Singularity options - Default image: /nfs/public/services/onedep_gpfs/docker-tools/onedep-builder_rocky8.sif This enables running tools built for Rocky Linux 8 on RedHat 9 cluster nodes.
- Add use_singularity, singularity_image, and singularity_bind_paths instance variables - Implement setUseSingularity(), setSingularityImage(), and setSingularityBindPaths() methods - Update __run() method to pass Singularity parameters to RunRemote - Enables all RcsbDpUtility operations to run inside Singularity containers when enabled Works together with RunRemote Singularity support for RedHat 9 migration.
- Add __getSingularity() method to read Singularity settings from ConfigInfo - Call __getSingularity() in __init__ to auto-configure from site-config - Supports use_singularity, singularity_image, and singularity_bind_paths config keys - No code changes needed in calling code - Singularity is enabled automatically via configuration This allows enabling Singularity site-wide by adding to site.cfg: use_singularity = true singularity_image = /path/to/image.sif (optional) singularity_bind_paths = /path1,/path2 (optional)
- Detect if already running inside a Singularity container - Skip container wrapping when SINGULARITY_CONTAINER or SINGULARITY_NAME env vars are set - Prevents container-in-container issues when jobs spawn sub-jobs This handles the case where: 1. Job A runs in Singularity container on SLURM 2. Job A spawns Job B and submits it to SLURM 3. Job B would be wrapped in Singularity again (nested) 4. Now Job B detects it's already in a container and skips wrapping
- Remove hardcoded default path from RunRemote.__init__ - Singularity image path must be provided via singularity_image parameter - Or set via site.cfg configuration: singularity_image = /path/to/image.sif - This ensures paths are always configuration-driven, never hardcoded
- Only use TOP_WWPDB_SITE_CONFIG_DIR from config for bind mounts. - Do not fall back to ONEDEP_PATH or any hardcoded path. - Ensures cross-site compatibility and restores original wwPDB environment standards.
At runtime, inspect the shebang of every executable invoked in the shell command. If all of them are Python scripts, bypass the container (the host Python works fine without it). ELF binaries and mixed commands still run inside the container as before.
Extend __commandOnlyCallsPythonScripts to recognise bare python/python3
invocations (e.g. python -m wwpdb.apps.validation) in addition to
absolute paths with Python shebangs. Shell builtins and variable
assignments are skipped.
Pass PYTHON=/usr/bin/python3 into the Singularity container so that
env.sh (which now uses ${PYTHON:-python3}) calls the container system
python for ConfigInfoShellExec.py instead of tools-rhel-9/bin/python3
which requires glibc 2.34.
Contributor
|
You should fix the auto test errors... |
Route all remote workflow jobs through Singularity when enabled now that the Rocky 8 toolchain is active. Remove the forced container PYTHON override so env.sh owns interpreter selection.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds support for running remote jobs inside a Singularity container in the
RcsbDpUtilityandRunRemotemodules. It introduces configuration options and logic to enable or disable containerization, specify images and bind paths, and avoid unnecessary container usage for pure Python jobs. The main changes can be grouped as follows:Singularity container integration:
__use_singularity,__singularity_image,__singularity_bind_paths,__partition) toRcsbDpUtilityand initialized them from config files. Provided setters for these options. ([[1]](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-2e2869b065e067698eb47b67f09dfb23c80c48f00226941f5dea3c8587cb6e7dR433-R444),[[2]](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-2e2869b065e067698eb47b67f09dfb23c80c48f00226941f5dea3c8587cb6e7dR496-R528),[[3]](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-2e2869b065e067698eb47b67f09dfb23c80c48f00226941f5dea3c8587cb6e7dR537-R554))__runinRcsbDpUtilityto pass Singularity options toRunRemote, and added logic to only use Singularity if the command is not purely Python-based. ([[1]](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-2e2869b065e067698eb47b67f09dfb23c80c48f00226941f5dea3c8587cb6e7dR5050-R5089),[[2]](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-2e2869b065e067698eb47b67f09dfb23c80c48f00226941f5dea3c8587cb6e7dR5103-R5106))RunRemote, added parameters and logic to wrap commands using Singularity, including detection of nested containers and construction of bind mounts and environment variables. ([[1]](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-0bec41aa508ed1c2af5339f02d9075f7c749d732fee45898b42a3b13e832edd0R47-R50),[[2]](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-0bec41aa508ed1c2af5339f02d9075f7c749d732fee45898b42a3b13e832edd0R61-R76),[[3]](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-0bec41aa508ed1c2af5339f02d9075f7c749d732fee45898b42a3b13e832edd0R166-R216),[[4]](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-0bec41aa508ed1c2af5339f02d9075f7c749d732fee45898b42a3b13e832edd0R235-R238))Partition and SLURM integration:
F75435fbL45R44,[wwpdb/utils/dp/RunRemote.pyR61-R76](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-0bec41aa508ed1c2af5339f02d9075f7c749d732fee45898b42a3b13e832edd0R61-R76))Python job detection:
__commandOnlyCallsPythonScriptsinRcsbDpUtilityto detect if a command only runs Python scripts, allowing the system to skip containerization for such jobs. ([wwpdb/utils/dp/RcsbDpUtility.pyR5050-R5089](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-2e2869b065e067698eb47b67f09dfb23c80c48f00226941f5dea3c8587cb6e7dR5050-R5089))Command-line interface enhancements:
RunRemoteCLI with options for Singularity usage, image selection, and bind paths. ([[1]](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-0bec41aa508ed1c2af5339f02d9075f7c749d732fee45898b42a3b13e832edd0R276-R287),[[2]](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-0bec41aa508ed1c2af5339f02d9075f7c749d732fee45898b42a3b13e832edd0R297-R299))Other improvements:
reimport for regex matching in Python job detection. ([wwpdb/utils/dp/RcsbDpUtility.pyR153](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-2e2869b065e067698eb47b67f09dfb23c80c48f00226941f5dea3c8587cb6e7dR153))[wwpdb/utils/dp/RunRemote.pyR144-R155](https://github.com/wwPDB/py-wwpdb_utils_dp/pull/64/files#diff-0bec41aa508ed1c2af5339f02d9075f7c749d732fee45898b42a3b13e832edd0R144-R155))These changes enable flexible and configurable use of Singularity containers for remote job execution, improving compatibility and reproducibility for heterogeneous compute environments.