[ALERT] cmp-prod-alb-target-5xx

[alexeygrigorev]

This issue was opened automatically from a CloudWatch alarm notification.

Alarm

Field	Value
Name	cmp-prod-alb-target-5xx
State	ALARM
Changed at	2026-08-13T14:16:01.837+0000
Description	ALB target 5xx responses for CMP prod.
Reason	Threshold Crossed: 1 datapoint [2.0 (13/08/26 14:09:00)] was greater than or equal to the threshold (1.0).
AWS account	387546586013
Region	eu-west-1

Open this alarm in CloudWatch

Metric

Field	Value
Namespace	AWS/ApplicationELB
Metric	HTTPCode_Target_5XX_Count
Dimensions	TargetGroup=targetgroup/course-management-prod/164455bdbcc646d8, LoadBalancer=app/course-management-shared/bd279e4cc4f97601
Statistic	SUM
Period	300 seconds
Evaluation periods	1
Threshold	GreaterThanOrEqualToThreshold 1.0

Investigation

• Check application and load-balancer logs around the state-change time.
• Check the current service version at /api/health/.
• Record the cause and resolution here before closing the issue.

Application logs

Field	Value
Log group	/ecs/course-management-prod
Suggested window	2026-08-13T14:06:01.837000+00:00 to 2026-08-13T14:21:01.837000+00:00
Investigator role	arn:aws:iam::387546586013:role/cmp-alert-investigator

Open CloudWatch Logs Insights, select /ecs/course-management-prod, or use the
cmp-alert-investigator AWS CLI profile described in the infrastructure README:

aws --profile cmp-alert-investigator logs filter-log-events \
  --region eu-west-1 \
  --log-group-name /ecs/course-management-prod \
  --start-time 1786629961837 \
  --end-time 1786630861837 \
  --limit 100

Review log output for sensitive information before copying any excerpt into
this public issue.

Raw CloudWatch notification

{
  "AWSAccountId": "387546586013",
  "ActionExecutedAfterMuteWindow": false,
  "AlarmActions": [
    "arn:aws:sns:eu-west-1:387546586013:course-management-operator-alerts"
  ],
  "AlarmArn": "arn:aws:cloudwatch:eu-west-1:387546586013:alarm:cmp-prod-alb-target-5xx",
  "AlarmConfigurationUpdatedTimestamp": "2026-07-08T22:06:44.752+0000",
  "AlarmDescription": "ALB target 5xx responses for CMP prod.",
  "AlarmName": "cmp-prod-alb-target-5xx",
  "InsufficientDataActions": [],
  "NewStateReason": "Threshold Crossed: 1 datapoint [2.0 (13/08/26 14:09:00)] was greater than or equal to the threshold (1.0).",
  "NewStateValue": "ALARM",
  "OKActions": [],
  "OldStateValue": "OK",
  "Region": "EU (Ireland)",
  "StateChangeTime": "2026-08-13T14:16:01.837+0000",
  "Trigger": {
    "ComparisonOperator": "GreaterThanOrEqualToThreshold",
    "Dimensions": [
      {
        "name": "TargetGroup",
        "value": "targetgroup/course-management-prod/164455bdbcc646d8"
      },
      {
        "name": "LoadBalancer",
        "value": "app/course-management-shared/bd279e4cc4f97601"
      }
    ],
    "EvaluateLowSampleCountPercentile": "",
    "EvaluationPeriods": 1,
    "MetricName": "HTTPCode_Target_5XX_Count",
    "Namespace": "AWS/ApplicationELB",
    "Period": 300,
    "Statistic": "SUM",
    "StatisticType": "Statistic",
    "Threshold": 1.0,
    "TreatMissingData": "notBreaching",
    "Unit": null
  }
}