|
1 | | -# Model-to-Data Workflow Template |
| 1 | +<div align="center"> |
| 2 | + <h1> |
| 3 | + Model-to-Data Challenge Infrastructure Template |
| 4 | + </h1> |
2 | 5 |
|
3 | | -This repository provides a ready-to-use template for setting up a |
4 | | -model2data workflow to be used with a challenge hosted on Synapse. |
| 6 | + <h3> |
| 7 | + Ready-to-use workflow template for evaluating Docker images submitted on |
| 8 | + <a href="https://www.synapse.org" title="Synapse.org">Synapse.org</a> |
| 9 | + </h3> |
5 | 10 |
|
6 | | -→ See also: [SynapseWorkflowOrchestrator](https://github.com/Sage-Bionetworks/SynapseWorkflowOrchestrator) |
| 11 | + <br/> |
| 12 | + |
| 13 | + <img |
| 14 | + alt="GitHub release (latest by date)" |
| 15 | + src="https://img.shields.io/github/release/sage-bionetworks-challenges/model-to-data-challenge-workflow?label=latest%20release&display_name=release&style=flat-square&color=blue"> |
| 16 | + <img |
| 17 | + alt="GitHub Release Date" |
| 18 | + src="https://img.shields.io/github/release-date/sage-bionetworks-challenges/model-to-data-challenge-workflow?style=flat-square&color=green"> |
| 19 | + <img |
| 20 | + alt="GitHub" |
| 21 | + src="https://img.shields.io/github/license/sage-bionetworks-challenges/model-to-data-challenge-workflow?style=flat-square&color=orange"> |
| 22 | +</div> |
| 23 | + |
| 24 | + |
| 25 | +### 💡 Should You Use This Template? |
| 26 | + |
| 27 | +The model-to-data (m2d) workflow is ideal for scenarios where participants |
| 28 | +train their algorithm locally using provided training data. Participants will |
| 29 | +then submit their algorithm as a containerized Docker image to be run against |
| 30 | +the hold-out validation/test data (which they will have no access to) in |
| 31 | +order to generate a predictions file, which is then evaluated against the |
| 32 | +hidden groundtruth data. |
| 33 | + |
| 34 | +### 🚀 Quick Start |
| 35 | + |
| 36 | +* **Customize evaluation logic:** modify the scoring and validation scripts |
| 37 | + within the `evaluation` folder |
| 38 | +* **Configure workflow:** adapt `workflow.cwl` (and `writeup-workflow.cwl`, |
| 39 | + if applicable) to define the inputs and steps specific to your challenge |
| 40 | +* **Test your changes:** use [`cwltool`](https://github.com/common-workflow-language/cwltool) |
| 41 | + to test your CWL scripts within the `steps` folder |
| 42 | + |
| 43 | +--- |
| 44 | + |
| 45 | +### Technical Details & Resources |
| 46 | + |
| 47 | +#### Repository structure |
| 48 | + |
| 49 | +This template provides all necessary components for a full challenge pipeline: |
| 50 | + |
| 51 | +``` |
| 52 | +. |
| 53 | +├── evaluation // core scoring and validation scripts |
| 54 | +├── README.md |
| 55 | +├── scripts // scripts called by the individual CWL scripts in `steps` |
| 56 | +├── steps // individual CWL scripts (called by the main workflow CWLs) |
| 57 | +├── workflow.cwl // CWL workflow for evaluating submissions |
| 58 | +└── writeup-workflow.cwl // CWL workflow to validate and archive writeup submissions |
| 59 | +``` |
| 60 | + |
| 61 | +#### Resource docs |
| 62 | + |
| 63 | +This workflow uses CWL, Docker, and the SynapseWorkflowOrchestrator to run |
| 64 | +the challenge pipeline. For more information on how to utilize these tools, |
| 65 | +see their docs below: |
| 66 | + |
| 67 | +* CWL: https://www.commonwl.org/user_guide/ |
| 68 | +* Docker: https://docs.docker.com/get-started/ |
| 69 | +* SynapseWorkflowOrchestrator: https://github.com/Sage-Bionetworks/SynapseWorkflowOrchestrator |
0 commit comments