Skip to content
This repository was archived by the owner on Aug 9, 2023. It is now read-only.

Commit b453d17

Browse files
authored
Merge pull request #4 from aws-samples/develop
merged copy edits
2 parents 559fbd8 + 825576e commit b453d17

11 files changed

+28
-32
lines changed

docs/core-env/create-custom-compute-resources.md

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
# Creating Custom Compute Resources
22

33
Genomics is a data-heavy workload and requires some modification to the defaults
4-
used for batch job processing. In particular, instances running the Tasks/Jobs need scalable storage to meet unpredictable runtime demands.
4+
used for batch job processing. In particular, instances running the Tasks/Jobs
5+
need scalable storage to meet unpredictable runtime demands.
56

67
By default, AWS Batch relies upon the [Amazon ECS-Optimized AMI](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html)
78
to launch container instances for running jobs. This is sufficient in most cases, but specialized needs, such as the large
@@ -20,11 +21,13 @@ The simplest method for customizing an instance is to use an EC2 Launch Template
2021
This works best if your customizations are relatively light - such as installing
2122
a few small utilities or making specific configuration changes.
2223

23-
This is because Launch Templates run `UserData` when an instance first launches.
24+
This is because Launch Templates run a `UserData` script when an instance first launches.
2425
The longer these scripts / customizations take to complete, the longer it will
2526
be before your instance is ready for work.
2627

27-
Since this will be working with AWS Batch, you only need to supply the `UserData`
28+
Launch Templates are capable of pre-configuring a lot of EC2 instance options.
29+
Since this will be working with AWS Batch, which already does a lot of automatic
30+
instance configuration on its own, you only need to supply the `UserData`
2831
script below:
2932

3033
```text
@@ -49,9 +52,10 @@ runcmd:
4952
--==BOUNDARY==--
5053
```
5154

52-
By default the `ebs-autoscale` monitor will add a 20GB EBS volume to the logical volume
53-
mounted at `/scratch`. If you want this volume to be larger initially, you can
54-
specify a `/dev/sdc` volume in the Launch Template.
55+
The above will add an `ebs-autoscale` daemon to an instance. By default it will
56+
add a 20GB EBS volume to the logical volume mounted at `/scratch`.
57+
If you want this volume to be larger initially, you can specify a bigger one
58+
mapped to `/dev/sdc` the Launch Template.
5559

5660
!!! note
5761
The mount point is specific to what orchestration method / engine you intend
14.2 KB
Loading

docs/core-env/introduction.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
A high level view of the architecture you will need is below.
44

5+
![high level architecture](images/aws-genomics-workflows-high-level-arch.png)
6+
57
To start you will need the following components:
68

79
* A place to store your input data and generated results
45.8 KB
Loading

docs/images/custom-ami-storage.png

-215 KB
Binary file not shown.
-418 KB
Binary file not shown.

docs/index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Genomics Workflows on AWS
22

3+
![banner](images/aws-genomics-workflows-banner.png)
4+
35
## Introduction
46

57
Welcome!
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Step Functions Workflow Examples
22

3+
THIS IS A STUB
4+
35
![Example Workflow](./images/example-state-machine.png)
46

57
this was created from [this file](./files/example-state-machine.json).

docs/orchestration/step-functions/step-functions-overview.md

Lines changed: 9 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -6,22 +6,15 @@
66

77
In the context of genomics workflows, the combination of AWS Step Functions with Batch and Lambda constitutes a robust, scalable, and serverless task orchestration solution.
88

9-
<!--// not ready for release yet
10-
// TODO: create the example sfn state-machine cfn template
11-
## TL;DR
12-
13-
If you need something up and running in a hurry, and you've followed all of the
14-
steps in the [Getting Started](../../../core-env/introduction/) section, you
15-
already have the majority of what you need setup.
16-
17-
The last component you need is an AWS Step Functions state machine for your workflow.
18-
Below is a CloudFormation template that creates an example state-machine that
19-
you can modify to suit your needs.
20-
21-
| Name | Description | Source | Launch Stack |
22-
| -- | -- | :--: | :--: |
23-
{{ cfn_stack_row("AWS Step Functions", "GenomicsWorkflow-Sfn", "aws-genomics-sfn.template.yaml", "Create IAM roles, Batch job definitions, and an example genomics workflow using an AWS Step Functions state machine") }}
24-
-->
9+
## Full Stack Deployment (TL;DR)
10+
11+
If you need something up and running in a hurry, a fully automated setup process
12+
is provided at this GitHub repository:
13+
14+
[AWS Batch Genomics](https://github.com/aws-samples/aws-batch-genomics)
15+
16+
If you are interested in creating your own solution with AWS Step Functions and AWS Batch,
17+
read through the rest of this page.
2518

2619
## Prerequisites
2720

docs/tldr.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,9 +45,9 @@ Once completed, click on the `Outputs` tab and copy down the AWS Batch Job Queue
4545

4646
### Option B: Individual components
4747

48-
The CloudFormation templates above are [nested stacks](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-nested-stacks.html), a hierarchy of templates that pass values from a parent template to dependent templates.
48+
The `Full Stack` CloudFormation template above is a [nested stack](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-nested-stacks.html), a hierarchy of templates that pass values from a parent template to dependent templates.
4949

50-
Below are the stand-alone CloudFormation templates for S3, IAM, and AWS Batch. These are handy in case you need to modify the individual components, or need to have another individual with elevated privileges to execute one of them (e.g. the IAM template). They are in order of dependency, and you will need to provide output values from one template to the dependent templates.
50+
Below are the stand-alone CloudFormation templates for each of the sub-stacks. These are handy in case you need to modify the individual components, or need to have another individual with elevated privileges to execute one of them (e.g. the IAM template). They are in order of dependency, and you will need to provide output values from one template to the dependent templates.
5151

5252
| Name | Description | Source | Launch Stack |
5353
| -- | -- | :--: | :--: |

0 commit comments

Comments
 (0)