Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
63adb4f
add codebundles/aws-c7n-ebs-health/sli.robot
saurabh3460 Nov 20, 2024
7a205bd
add c7n ebs policies
saurabh3460 Nov 20, 2024
9ee4894
add script to create test infra
saurabh3460 Nov 20, 2024
b86124b
added runbook.robot with List Unattached EBS Volumes task
saurabh3460 Nov 20, 2024
e1bd6e0
Merge branch 'runwhen-contrib:main' into main
saurabh3460 Nov 20, 2024
774460f
added parse_ebs_results func in Core.py
saurabh3460 Nov 22, 2024
9d5dd28
change name of unused-ebs-snapshots policy
saurabh3460 Nov 22, 2024
3dd7314
change secret__aws_account_id -> secret__aws_access_key_id
saurabh3460 Nov 22, 2024
b9505d0
updated create/delete snapshot script in .test
saurabh3460 Nov 22, 2024
aa77f67
added List Unused EBS Snapshots and List Unencrypted EBS Volumes task…
saurabh3460 Nov 22, 2024
780854e
add runwhen generation rule and template yaml
saurabh3460 Nov 22, 2024
3455556
clean cc lib
saurabh3460 Nov 25, 2024
ecc92ff
replace ebs test script with terraform
saurabh3460 Nov 27, 2024
cfb684b
remove volume check and add encrypted false in ebs.tf
saurabh3460 Nov 27, 2024
e9f4513
added taskfile in ebs health codebundle
saurabh3460 Nov 27, 2024
6102c59
add account_id in ebs gen rule qualifiers
saurabh3460 Nov 27, 2024
4c59821
add check-rwp-config task in ebs cb test's taskfile
saurabh3460 Nov 27, 2024
90b306b
update ebs cb test README
saurabh3460 Nov 27, 2024
7f81bd3
add encrypted filed in ebs tf file
saurabh3460 Nov 28, 2024
9602af8
add suite variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in eb…
saurabh3460 Nov 28, 2024
becadf9
add rw-cli-keywords dependency in requirements.txt
saurabh3460 Nov 29, 2024
37fd8b7
fix sli locations filed in both ebs and s3 cb
saurabh3460 Nov 29, 2024
5672dc9
update Author in sli
saurabh3460 Nov 29, 2024
53c55bc
fix Add Issue and change AWS_ACCOUNT_NAME -> AWS_ACCOUNT_ID
saurabh3460 Nov 29, 2024
406e2fd
ebs Taskfile: add custom field and terraform/cb.secret
saurabh3460 Nov 29, 2024
b0d6153
ebs sli: fix score logic
saurabh3460 Nov 29, 2024
4580a3b
ebs runbook: update next steps string and task title
saurabh3460 Nov 29, 2024
d7fce89
EBS CB: fix typo and update image url in templates
saurabh3460 Dec 4, 2024
59c5463
update intervalSeconds 300 -> 600
saurabh3460 Dec 6, 2024
d1caa2e
EBS CB: update Metadata and thresholds defaults 1->0
saurabh3460 Dec 6, 2024
e4df799
ebs cb: rename gereration rule file
saurabh3460 Dec 6, 2024
b41e23d
ebs cb: update Metadata, statements in add issues in runbook.robot
saurabh3460 Dec 6, 2024
5d5f2c8
ebs cb: fix score logic
saurabh3460 Dec 11, 2024
7f557d0
ebs cb: update REDME.md with how to test steps
saurabh3460 Dec 12, 2024
1b2699e
ebs cb: update REDME.md with how to test steps
saurabh3460 Dec 12, 2024
c78fbe9
Merge remote-tracking branch 'upstream/main'
saurabh3460 Dec 13, 2024
4c9681b
Merge remote-tracking branch 'upstream/main'
saurabh3460 Jan 6, 2025
638f32b
Merge remote-tracking branch 'upstream/main'
saurabh3460 Jan 8, 2025
191873c
Merge remote-tracking branch 'upstream/main'
saurabh3460 Jan 17, 2025
0548d37
Merge remote-tracking branch 'upstream/main'
saurabh3460 Jan 31, 2025
5e63e7c
Update c7n version to 0.9.43 in requirements.txt
saurabh3460 Feb 7, 2025
9658c05
Add README for AWS Cloud Custodian Service Usage CodeBundle
saurabh3460 Feb 7, 2025
718a5f3
Add service usage policy template for AWS service quotas
saurabh3460 Feb 7, 2025
20ee761
Add utility function to generate usage tables from AWS resource logs
saurabh3460 Feb 7, 2025
1c689f3
Add generation rules and templates for AWS service usage monitoring
saurabh3460 Feb 7, 2025
57c73ba
Add AWS Service Usage monitoring suite with threshold checks
saurabh3460 Feb 7, 2025
cf9c6cb
Add runbook for listing AWS service usage exceeding defined thresholds
saurabh3460 Feb 7, 2025
318c9b7
add placeholder for test infra
saurabh3460 Feb 7, 2025
0b2b15f
Add Util.py library to SLI robot suite
saurabh3460 Feb 7, 2025
50c39e9
Fix metric value conversion in AWS service usage check
saurabh3460 Feb 7, 2025
a943aac
Update usage percentage pattern to enforce positive integer values
saurabh3460 Feb 7, 2025
ca9be89
Update default AWS resource providers and set suite variables in runb…
saurabh3460 Feb 7, 2025
bdec9b8
Add timeout parameter to AWS service usage commands
saurabh3460 Feb 10, 2025
093db1c
Improve AWS service usage reporting with detailed account context and…
saurabh3460 Feb 11, 2025
18f8bad
Improve descriptions for AWS resource providers and usage percentage …
saurabh3460 Feb 11, 2025
fc4448c
Clarify SLI metrics description in README for better understanding
saurabh3460 Feb 11, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
apiVersion: runwhen.com/v1
kind: GenerationRules
spec:
platform: aws
generationRules:
- resourceTypes:
- aws_ec2_security_groups
matchRules:
- type: pattern
pattern: ".+"
properties: [name]
mode: substring
slxs:
- baseName: aws-c7n-service-usage
qualifiers: ["account_id"]
baseTemplateName: aws-c7n-service-usage
levelOfDetail: basic
outputItems:
- type: slx
- type: sli
- type: runbook
templateName: aws-c7n-service-usage-taskset.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
apiVersion: runwhen.com/v1
kind: ServiceLevelIndicator
metadata:
name: {{slx_name}}
labels:
{% include "common-labels.yaml" %}
annotations:
{% include "common-annotations.yaml" %}
spec:
displayUnitsLong: OK
displayUnitsShort: ok
locations:
- {{default_location}}
description: Count AWS Service Usage Exceeding defined threshold in AWS account {{match_resource.resource.account_id}}
codeBundle:
{% if repo_url %}
repoUrl: {{repo_url}}
{% else %}
repoUrl: https://github.com/runwhen-contrib/rw-c7n-codecollection.git
{% endif %}
{% if ref %}
ref: {{ref}}
{% else %}
ref: main
{% endif %}
pathToRobot: codebundles/aws-c7n-service-usage/sli.robot
intervalStrategy: intermezzo
intervalSeconds: 600
configProvided:
- name: AWS_REGION
value: "{{match_resource.resource.region}}"
- name: AWS_ACCOUNT_ID
value: "{{match_resource.resource.account_id}}"
secretsProvided:
- name: AWS_ACCESS_KEY_ID
workspaceKey: {{custom.aws_access_key_id}}
- name: AWS_SECRET_ACCESS_KEY
workspaceKey: {{custom.aws_secret_access_key}}
alerts:
warning:
operator: '>'
threshold: '1'
for: '20m'
ticket:
operator: '>'
threshold: '1'
for: '40m'
page:
operator: '=='
threshold: '0'
for: ''
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
apiVersion: runwhen.com/v1
kind: ServiceLevelX
metadata:
name: {{slx_name}}
labels:
{% include "common-labels.yaml" %}
annotations:
{% include "common-annotations.yaml" %}
spec:
imageURL: https://storage.googleapis.com/runwhen-nonprod-shared-images/icons/aws/Resource-Icons_06072024/Res_Networking-Content-Delivery/Res_Amazon-VPC_Virtual-private-cloud-VPC_48.svg
alias: AWS Service Usage Exceeding defined threshold in AWS Account {{match_resource.resource.account_id}}
asMeasuredBy: The number of AWS Service Usage Exceeding defined threshold in AWS account {{match_resource.resource.account_id}}
configProvided:
- name: SLX_PLACEHOLDER
value: SLX_PLACEHOLDER
owners:
- {{workspace.owner_email}}
statement: List AWS Service Usage Exceeding defined threshold in the AWS account {{match_resource.resource.account_id}}
additionalContext:
region: "{{match_resource.resource.region}}"
account_id: "{{match_resource.resource.account_id}}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
apiVersion: runwhen.com/v1
kind: Runbook
metadata:
name: {{slx_name}}
labels:
{% include "common-labels.yaml" %}
annotations:
{% include "common-annotations.yaml" %}
spec:
location: {{default_location}}
description: List AWS Service Usage Exceeding defined threshold in the AWS account {{match_resource.resource.account_id}}
codeBundle:
{% if repo_url %}
repoUrl: {{repo_url}}
{% else %}
repoUrl: https://github.com/runwhen-contrib/rw-c7n-codecollection.git
{% endif %}
{% if ref %}
ref: {{ref}}
{% else %}
ref: main
{% endif %}
pathToRobot: codebundles/aws-c7n-service-usage/runbook.robot
configProvided:
- name: AWS_REGION
value: "{{match_resource.resource.region}}"
- name: AWS_ACCOUNT_ID
value: "{{match_resource.resource.account_id}}"
secretsProvided:
- name: AWS_ACCESS_KEY_ID
workspaceKey: {{custom.aws_access_key_id}}
- name: AWS_SECRET_ACCESS_KEY
workspaceKey: {{custom.aws_secret_access_key}}
91 changes: 91 additions & 0 deletions codebundles/aws-c7n-service-usage/.test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
### How to test this codebundle?

#### IAM User Configuration

We create two distinct AWS IAM users with carefully scoped access:

**CloudCustodian IAM User**

Purpose: Service Level Indicator (SLI) monitoring and runbook automation and configured with least privilege access principles

with the following policy:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"tag:GetResources",
"ec2:DescribeRegions",
"ec2:DescribeSecurityGroups",
"servicequotas:ListServices",
"servicequotas:ListAWSDefaultServiceQuotas",
"servicequotas:ListServiceQuotas",
"cloudwatch:GetMetricStatistics"
],
"Resource": "*"
}
]
}
```

**Infrastructure Deployment User**

Purpose: Cloud infrastructure provisioning and management using Terraform

#### Credential Setup

Navigate to the `.test/terraform` directory and configure two secret files for authentication:

`cb.secret` - CloudCustodian and RunWhen Credentials

Create this file with the following environment variables:

```sh
export RW_PAT=""
export RW_WORKSPACE=""
export RW_API_URL="papi.beta.runwhen.com"

export AWS_DEFAULT_REGION="us-west-2"
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)
```


`tf.secret` - Terraform Deployment Credentials

Create this file with the following environment variables:

```sh
export AWS_DEFAULT_REGION=""
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
export AWS_SESSION_TOKEN="" # Optional: Include if using temporary credentials
```

#### Testing Workflow

1. Build test infra:
**Note** WIP

2. Generate RunWhen Configurations
```sh
tasks
```

3. Upload generated SLx to RunWhen Platform

```sh
task upload-slxs
```

4. At last, after testing, clean up the test infrastructure.

```sh
task clean
```

Loading