SMS-Checker - Operation Registry

This repository is the main entry into this project. It introduces your high-level architecture and links to the other corresponding repositories, so visitors can easily understand your project and find all relevant information. This repository contains a single Docker Compose setup that orchestrates and runs the full application, including:

model-service: Backend Model Service
app: Frontend Web Application

Repository Structure

The following are the repositories present in this organization:

app : Frontend Java Spring Boot service providing the UI
model-service : Backend Python microservice that loads and serves the spam classifier
lib-version : Library dependencies ?? //TODO

A1: Version Release and Containerization

Requirements

The following needs to be installed before running this project:

Docker
Docker Compose Other dependencies such as Python or Java will be installed in respective docker containers.

Environment Variables Setup

The docker compose uses .env file for setting up the environment variables:

APP_PORT = 8080
MODEL_PORT = 8081
APP_IMAGE = ghcr.io/doda2025-team21/frontend
MODEL_IMAGE = ghcr.io/doda2025-team21/backend

You can change the ports or image versions here.

How to start the application?

Make sure to clone all the repositories using

git clone ....

Navigate to the appropriate directory

cd operation

Run the following command:

docker compose up --pull always

The following command does the following:

it will run the latest version of the image from MODEL_IMAGE
Start the backend on http://localhost:8081 (or replace the port with one mentioned in MODEL_PORT).
Start the frontend on http://localhost:8080/sms (or replace port with one mentioned in APP_PORT). An alternative to running the latest version of the image is specifying the version of the you'd like to run using the TAG_VERSION=*** variable:

TAG_VERSION=0.0.1 docker compose up --pull always

To stop running everything, run the following command:

docker compose down

Other handy docker commands

Action	Command	Description
Start everything	`docker compose up`	Starts all services (shows logs)
Start in background	`docker compose up -d`	Runs services in detached mode
Stop all running services	`docker compose down`	Stops and removes containers, networks
Rebuild images	`docker compose up --build`	Rebuilds images before starting
View logs	`docker compose logs`	Shows combined logs from all services
View logs for a specific service	`docker compose logs app`	Shows logs only for the app
Restart one service	`docker compose restart app`	Restarts only the app service

A2: Provisioning a Kubernetes Cluster

Goal: reproducibly provision the Kubernetes base platform described in the A2 brief (Vagrant + VirtualBox + Ansible, Flannel CNI, MetalLB, ingress, dashboard, Istio). The controller lives at 192.168.56.100 and workers start at 192.168.56.101; counts and resources are configurable.

What the automation sets up

Vagrant builds one ctrl and NUM_WORKERS node-* hosts on a host-only network (controller at 192.168.56.100, workers starting at 192.168.56.101/102/...); CPU/memory/worker count come from .env (NUM_WORKERS, CTRL_CPUS, CTRL_MEMORY, NODE_CPUS, NODE_MEMORY).
ansible/general.yml (Steps 4-12): imports team SSH keys from ansible/files/ssh-keys/*.pub, disables swap and fstab entries, loads overlay/br_netfilter, enables IP forwarding sysctls, templates /etc/hosts with all nodes, adds the Kubernetes apt repo, installs containerd 1.7.24 + runc 1.1.12 + kubeadm/kubelet/kubectl 1.32.4, writes containerd config (pause 3.10, AppArmor off, SystemdCgroup=true), restarts containerd, enables kubelet.
ansible/ctrl.yml (Steps 13-17): kubeadm init with advertise address 192.168.56.100 and pod CIDR 10.244.0.0/16 (idempotent via /etc/kubernetes/admin.conf check), copies kubeconfig to ~/.kube for vagrant and fetches ./kubeconfig for the host, installs Flannel with --iface=eth1, installs Helm and the helm-diff plugin.
ansible/node.yml (Steps 18-19): delegates kubeadm token create --print-join-command to the controller and joins each worker if kubelet.conf is missing.
ansible/finalization.yml (Steps 20-23): MetalLB with pool 192.168.56.90-192.168.56.99 and readiness waits; ingress-nginx with class nginx and load balancer IP 192.168.56.90; Kubernetes Dashboard via Helm with an ingress on dashboard.local (HTTPS backend) and admin ServiceAccount; Istio 1.25.2 with ingress gateway IP 192.168.56.91.

Preparation

Install VirtualBox, Vagrant, and Ansible on the host.
Add your public key(s) to ansible/files/ssh-keys/ so SSH works without passwords.
Optional: adjust worker count/resources in .env (e.g., NUM_WORKERS=2 NODE_MEMORY=6144).

Bring up the cluster

git pull

# Create VMs and run general/ctrl/node provisioning
vagrant up --no-provision            # rerun with `vagrant provision` if needed

Inventory generation

Our Vagrantfile generates a valid inventory.cfg for Ansible that contains all (and only) active nodes.
To avoid potential race conditions, first create VMs as stated in the previous step, and then run provisioning.
Inspect the generated inventory (inventory.cfg) by running:

vagrant provision

Testing inventory generation

To test inventory.cfg contains only active nodes:
- halt one of the nodes, for instance, in the case that number of workers are adjusted to 3 in the Preparation step (e.g.,NUM_WORKERS=3), run:

vagrant halt node-3
vagrant provision
cat inventory.cfg

node-3 should not be visible under [workers].
Afterwards:

vagrant up node-3
vagrant provision
cat inventory.cfg

node-3 should now be visible under [workers].

The folowing commands can be used to rerun the playbooks manually.

# Rerun playbooks manually (useful after edits)
ansible-playbook -u vagrant -i inventory.cfg ansible/general.yml
ansible-playbook -u vagrant -i inventory.cfg ansible/ctrl.yml
ansible-playbook -u vagrant -i inventory.cfg ansible/node.yml

# Finish cluster features (MetalLB, ingress, dashboard, Istio)
ansible-playbook -u vagrant -i inventory.cfg ansible/finalization.yml

Smoke tests (matches A2 expectations)

# Reach the VMs (if keys are missing, fallback user/password is vagrant/vagrant)
ssh vagrant@192.168.56.100
ssh vagrant@192.168.56.101
ssh vagrant@192.168.56.102

# To get out of node, press ctrl+D

# Check cluster health from host
KUBECONFIG=./kubeconfig kubectl get nodes -o wide
KUBECONFIG=./kubeconfig kubectl get pods -A

# Verify load balancers / ingresses
KUBECONFIG=./kubeconfig kubectl get svc -n metallb-system
KUBECONFIG=./kubeconfig kubectl get svc -n ingress-nginx ingress-nginx-controller -o wide
KUBECONFIG=./kubeconfig kubectl get svc -n kubernetes-dashboard
KUBECONFIG=./kubeconfig kubectl get svc -n istio-system istio-ingressgateway -o wide

A3: Operate and Monitor Kubernetes

Deploy the stack with Helm

This will run very long time, might be few minutes or even more ........ 🥲

vagrant up --no-provision && \
    ansible-playbook -u vagrant -i inventory.cfg ansible/general.yml && \
    ansible-playbook -u vagrant -i inventory.cfg ansible/ctrl.yml && \
    ansible-playbook -u vagrant -i inventory.cfg ansible/node.yml && \
    ansible-playbook -u vagrant -i inventory.cfg ansible/finalization.yml && \
    helm upgrade --install sms-checker ./sms-checker-helm-chart \
    -f sms-checker-helm-chart/values.yaml \
    --set app.ingress.hosts.stable=sms.local \
    --set app.ingress.hosts.preview=sms-preview.local \
    --set app.ingress.className=nginx \
    --kubeconfig kubeconfig

Check resources and ingress:

KUBECONFIG=./kubeconfig kubectl get pods,svc,ingress
KUBECONFIG=./kubeconfig kubectl get svc -n ingress-nginx ingress-nginx-controller -o wide

Expose ingress to access the app

MetalLB assigns the nginx ingress controller an IP (default 192.168.56.90 from finalization.yml). Point your host name to it:

# sms.local sms-preview.local are the --set host, change that if you used a different host name
echo "192.168.56.90 sms.local sms-preview.local" | sudo tee -a /etc/hosts

Then open http://sms.local/ to reach the frontend; it talks to model-service via the cluster service. Open sms.local/sms/ to open the SMS Checker web. (Note, the sms.local service might takes up 1 or 2 minutes to be ready and I don't know why 😅).

Ingress hosts (stable + preview)

Stable host: app.ingress.hosts.stable (defaults to sms.local) drives the main ingress rule.
Preview/experiment host: app.ingress.hosts.preview (defaults empty). If you set a value (e.g., sms-preview.local), the chart renders a second ingress rule to the same service—useful for A4 experiments.
Ingress class: app.ingress.className defaults to nginx; override if your cluster uses a different controller.
Point both hostnames to your ingress external IP (MetalLB or Minikube tunnel) via /etc/hosts so browsers resolve correctly.

Minikube note

If using Minikube, enable ingress and get an IP via minikube addons enable ingress and minikube tunnel, add that IP to /etc/hosts for sms.local, and browse the same URL.

Using ConfigMap and Secrets

The deployed application uses ConfigMap and Secrets. In app-configmap.yaml we have "WELCOME_MESSAGE", and "DUMMY_VALUE". In app-secrets.yaml we have "API_KEY", and "PASSWORD".

These values are not hardcoded. They are used in stable and canary versions of our app.

To test it out:

KUBECONFIG=./kubeconfig kubectl get configmap app-config
KUBECONFIG=./kubeconfig kubectl get secret app-secret

KUBECONFIG=./kubeconfig kubectl get pods -n default | grep app-

KUBECONFIG=./kubeconfig kubectl exec <app-pod> -- env | grep WELCOME_MESSAGE
KUBECONFIG=./kubeconfig kubectl exec <app-pod> -- env | grep DUMMY_VALUE

KUBECONFIG=./kubeconfig kubectl exec <app-pod> -- env | grep API_KEY
KUBECONFIG=./kubeconfig kubectl exec <app-pod> -- env | grep PASSWORD

Additionally, our app does not hardcode the model-service location; it reads the model address from "MODEL_HOST" environment variable. Therefore, model-service can be relocated by changing the Kubernetes configuration. You can test it out by:

helm upgrade --install sms-checker ./sms-checker-helm-chart \
  -f sms-checker-helm-chart/values.yaml \
  --set model.name=model-service-new-test-name \
  --kubeconfig kubeconfig

Then run:

KUBECONFIG=./kubeconfig kubectl get deploy app-stable -o yaml | grep -A2 MODEL_HOST

KUBECONFIG=./kubeconfig kubectl get svc | grep model

you should see:

model-service-new-test-name

Using a Shared VirtualBox Folder to Create Shared Storage Across All Pods

All VMs mount the same shared VirtualBox folder as /mnt/shared into the VM. The deployed application mounts this path as a hostPath Volume into at least one Deployment (In our case this deployment is app-deployment.yaml; specifically, the stable version of the app).

In order to prove this functionality, we have a3-kubernetes-proof.txt file in the shared directory.

According to the official Kubernetes documentation, "if you allow a read-write mount of any host path by an untrusted Pod, the containers in that Pod may be able to subvert the read-write host mount." Therefore, in order to avoid any possible issues that may arrive, any mounts of hostPath volume are "read only."

To verify shared storage, read the file from /mnt/shared on two different VMs (the controller and a worker):

ssh vagrant@192.168.56.100 "cat /mnt/shared/a3-kubernetes-proof.txt"
ssh vagrant@192.168.56.101 "cat /mnt/shared/a3-kubernetes-proof.txt"

You should see:

If you are seeing this message inside a pod, shared storage is working correctly!

This shows that /mnt/shared exists on both VMs, and its contents are identical.

Afterwards, verify inside the Kubernetes pod, since we implemented this functionality for the stable version of app, do:

KUBECONFIG=./kubeconfig kubectl get pods -n default | grep app-stable

pick one of the running pods, replace <copy-your-pod-name-here>with your actual pod name, and run : cat /mnt/shared/a3-kubernetes-proof.txt in that specific container.

KUBECONFIG=./kubeconfig kubectl exec -n default -it <copy-your-pod-name-here> -- cat /mnt/shared/a3-kubernetes-proof.txt

You should see:

If you are seeing this message inside a pod, shared storage is working correctly!

This means that this pod can successfully see /mnt/shared. Pod mount is working.

Prometheus Monitoring

The Helm chart includes kube-prometheus-stack which installs:

Prometheus → collects metrics
Grafana → view dashboards
ServiceMonitors → auto-discover app metrics

Access Grafana

# Add to /etc/hosts
echo "192.168.56.90 grafana.local" | sudo tee -a /etc/hosts

Open http://grafana.local in your browser.

Default credentials: admin / admin
You'll be asked to change the password on first login.

Access Prometheus

# Add to /etc/hosts
echo "192.168.56.90 prometheus.local" | sudo tee -a /etc/hosts

Open http://prometheus.local in your browser.

Application Metrics

Both services expose Prometheus-compatible metrics:

App Service (port 9090, path /actuator/prometheus):

Metric	Type	Description
`sms_requests_total`	Counter	Total SMS classification requests (labels: endpoint)
`sms_queue_size`	Gauge	Current messages in processing queue (labels: priority)
`sms_classification_duration_seconds`	Histogram	Time to classify SMS messages (labels: model_version)

Model Service (port 9091, path /metrics):

Metric	Type	Description
`model_predictions_total`	Counter	Total predictions made (labels: model_name, prediction, confidence_bucket)
`model_loaded`	Gauge	Whether model is ready (1=yes, 0=no)
`model_inference_duration_seconds`	Histogram	Model inference time (labels: model_name)

See METRICS.md for detailed documentation.

Verify Monitoring is Working

# Check ServiceMonitors
ssh vagrant@192.168.56.100 "kubectl get servicemonitors"

# Check Prometheus targets (should show UP)
# Open http://prometheus.local → Status → Targets

Disable Monitoring

To deploy without Prometheus/Grafana, set in values.yaml:

prometheus:
  enabled: false

Displaying Grafana Dashboards

Since a ConfigMap is provided (sms-checker-helm-chart/templates/grafana-a3.yaml), there is no need for the manual installation.

There are two separate dashboards, one for A3 and one for the decision process for A4. Their names in Grafana Dashboard list are: A3 Dashboard 1, and A4 Supporting the Decision Process. Once you navigate to the Dashboards in Grafana they should be at the top of the list.

Open http://grafana.local in your browser.

Go to the Dashboards from the side menu. And you chould be able to see our two dashboards at the top. (A3 Dashboard 1, and A4 Supporting the Decision Process)

How to clean up

vagrant destroy

Email Alerting using Prometheus

Installation Pre-requisites

Minikube (running with application deployed)
Helm

istioctl (install istio to your cluster if not installed and enable sidecar injection)

istioctl install --set profile=demo -y
kubectl label namespace default istio-injection=enabled

Docker (Minikube driver)

Alert Details

HighRequestRate

Condition: More than 15 requests per minute, aggregated across all pods

Expression: rate(<request_counter_metric>[1m]) * 60 > 15

Duration: Must hold for 2 consecutive minutes

Severity: warning

Notification: Email (with resolve notification enabled)

Install/Upgrade the Helm Chart

helm install sms-checker . \
  --set alertmanager.smtp.user="YOUR_GMAIL@gmail.com" \
  --set alertmanager.smtp.password="YOUR_APP_PASSWORD" \
  --set alertmanager.recipient="YOUR_EMAIL@example.com"

run this command from the sms-checker-helm-chart folder. Replace YOUR_EMAIL@example.com with your actual email address.

Note: Gmail requires an App Password, not your normal account password.

Verify Setup

Check pods: Ensure the following pods are running:
- Prometheus
- AlertManager
- Grafana
- Application pods (app & model-service)
```
kubectl get pods
```
Verify existence of alert rule in kubernetes; you're expected to see high-traffic-alert rule in the list of rules.
```
kubectl get prometheusrules
```
Verify Alertmanager Config Mount: Look for the following file in the directory: alertmanager.yaml.gz
```
kubectl exec -it alertmanager-<pod-name> -- ls /etc/alertmanager/config
```
Verify Alert Rule in Prometheus:
```
kubectl port-forward svc/prometheus-operated 9090:9090
```
Open: http://localhost:9090
Navigate to Status -> Alerts, you should see:
```
traffic-alerts
└── HighRequestRate
```
The initial state should be INACTIVE (green).

Testing the alert end-to-end

View the Rule in Prometheus by port-forwarding:
```
kubectl port-forward svc/prometheus-operated 9090:9090
```
Now you can open prometheus on http://localhost:9090. Verify under Status -> Rule Health to see high traffic alert rule with the status as OK. Additionally under Alerts, you can find the rule with status INACTIVE.

Keep this terminal running.
In a separate terminal window, to view the metrics and dashboard in Grafana, use a different port to port-forward to:
```
kubectl port-forward svc/sms-checker-grafana 3000:80
```
Go to http://localhost:3000 and use username: admin and password: admin to login. Open dashboard -> Application Metrics.

Generate traffic to observe metrics & trigger alert. Port-forward istio ingress:

kubectl port-forward -n istio-system svc/istio-ingressgateway 8080:80

Keep this terminal running.

Open a new terminal and generate high traffic for about 3-4 minutes:

while true; do
  curl -X POST http://localhost:8080/sms/ \
    -H "Host: sms-istio.local" \
    -H "Content-Type: application/json" \
    -d '{"sms":"alert test"}' >/dev/null
  sleep 0.05
done

Observe Alert Lifecycle: In Prometheus (Alerts page);

Time State

< 2 min Pending

≥ 2 min Firing
Verify Alertmanager:
```
kubectl port-forward svc/alertmanager-operated 9093:9093
```
Open: http://localhost:9093
You should see HighRequestRate in Firing state.
Verify Email Delivery
- Alert email is sent when the alert fires
- A resolve email is sent after traffic stops
Check spam folder if not visible.

A4: Traffic Management with Istio

This section documents the Istio based traffic management configuration for canary releases with sticky sessions. After the explanation, there will be a section where it could be verified.

Istio Ingress Gateway Configuration

If you already ran A3 and deployed with helm chart, you must label the namespace to enable automatic Istio sidecar injection. Without this label, pods will not have the istio-proxy container and Istio traffic management will not work.

1. Label namespace for sidecar injection

KUBECONFIG=./kubeconfig kubectl label namespace default istio-injection=enabled --overwrite

If pods existed before you labelled the namespace, you need to restart pods so that they get sidecars too (optional) kubectl rollout restart deployment app-stable app-canary model-service-stable model-service-canary

2. for full deployment of A4 release with all the requirements, run this command ( Deploying with Istio enabled, canary release, and monitoring)

KUBECONFIG=./kubeconfig helm upgrade --install sms-checker ./sms-checker-helm-chart
-f sms-checker-helm-chart/values.yaml
--set istio.enabled=true
--set istio.host=sms-istio.local
--set istio.canary.enabled=true
--set istio.canary.weight=10
--set istio.stickySession.enabled=true
--set prometheus.enabled=true
--set kube-prometheus-stack.prometheusOperator.admissionWebhooks.enabled=false #webhook admission just validates the servicemonitor yaml and prometheus grafana metrics all work without this

sometimes upgrade fails saying another operation is in progress. In this case, just unistall the release and install again using the above command. You can uninstall first with the following:

KUBECONFIG=./kubeconfig helm uninstall sms-checker --no-hooks

then run the install again.

Also, prometheus gets easily stuck. You can delete the job by using

KUBECONFIG=./kubeconfig kubectl delete job --all

and try using the above command to retry the upgrade.

3. Restart pods to inject sidecars (helm upgrade may not trigger this)

KUBECONFIG=./kubeconfig kubectl rollout restart deployment app-stable app-canary model-service-stable model-service-canary

#4 Add Host Entry

# Point the Istio host to the ingress gateway IP
echo "192.168.56.91 sms-istio.local" | sudo tee -a /etc/hosts

Verification

1. App accessible via Istio Gateway

curl http://sms-istio.local/

2. Check 90/10 traffic split (run 10+ times, ~90% stable, ~10% canary)

for i in {1..10}; do 
  curl -s http://sms-istio.local/sms/ -X POST -H "Content-Type: application/json" -d '{"sms": "test '$i'"}' > /dev/null
  sleep 1
done

echo "=== STABLE ===" && KUBECONFIG=./kubeconfig kubectl logs deploy/app-stable -c app --tail=20 | grep "test"
echo "=== CANARY ===" && KUBECONFIG=./kubeconfig kubectl logs deploy/app-canary -c app --tail=20 | grep "test"

3. Sticky sessions (same cookie = same version)

Test sticky sessions - same cookie should hit same version

curl -c cookies.txt -b cookies.txt http://sms-istio.local/sms/ -X POST -H "Content-Type: application/json" -d '{"sms": "sticky test 1"}' curl -c cookies.txt -b cookies.txt http://sms-istio.local/sms/ -X POST -H "Content-Type: application/json" -d '{"sms": "sticky test 2"}' curl -c cookies.txt -b cookies.txt http://sms-istio.local/sms/ -X POST -H "Content-Type: application/json" -d '{"sms": "sticky test 3"}'

echo "=== STABLE ===" && KUBECONFIG=./kubeconfig kubectl logs deploy/app-stable -c app --tail=10 | grep "sticky" echo "=== CANARY ===" && KUBECONFIG=./kubeconfig kubectl logs deploy/app-canary -c app --tail=10 | grep "sticky"

(all 3 should go to the same version)

4. Consistent routing: old→old, new→new

Clear old logs by checking fresh requests

First, make requests that go to STABLE app

for i in {1..5}; do curl -s http://sms-istio.local/sms/ -X POST -H "Content-Type: application/json" -d '{"sms": "routing test '$i'"}' > /dev/null sleep 1 done

Check which modelservice received the requests

echo "=== MODEL-STABLE ===" && KUBECONFIG=./kubeconfig kubectl logs deploy/model-service-stable -c model-service --tail=10 | grep POST echo "=== MODEL-CANARY ===" && KUBECONFIG=./kubeconfig kubectl logs deploy/model-service-canary -c model-service --tail=10 | grep POST

5. Grafana dashboard

KUBECONFIG=./kubeconfig kubectl port-forward svc/sms-checker-grafana 3000:80

Open http://localhost:3000 (admin/prom-operator)

Check SMS Checker dashboard exists with A/B comparison

6. Rate limiting (additional use case)

for i in {1..15}; do curl -s http://sms-istio.local/sms/ -d "message=test$i" -w "%{http_code}\n" -o /dev/null; done

After ~10 requests, should get 429 Too Many Requests

The Istio ingress gateway is provisioned during cluster setup (ansible/finalization.yml) and is not a part of the Helm chart, as specified in the assignment description. The gateway runs in the istio-system namespace with the following configuration:

Gateway Labels and Selector

Component	Label	Value	Description
Ingress Gateway Pod	`istio`	`ingressgateway`	Label used by Gateway resources to select the ingress gateway
Ingress Gateway Pod	`app`	`istio-ingressgateway`	Application identifier
Service	External IP	`192.168.56.91`	MetalLB-assigned load balancer IP

Configurable Gateway Selector

The ingress gateway selector is configurable in values.yaml to support different cluster configurations:

istio:
  ingressGateway:
    # Default istio installations use 'ingressgateway'
    selector: ingressgateway
    # Below is the namespace where ingress gateway is deployed
    namespace: istio-system

If deploying to a cluster with a different ingress gateway configuration as mentioned in the assignment, you can override the selector using the following:

helm upgrade --install sms-checker ./sms-checker-helm-chart \
  --set istio.ingressGateway.selector=my-custom-gateway \
  --kubeconfig kubeconfig

Deploy with Istio Traffic Management

Enable Istio Sidecar Injection (Required)

Before deploying, you must label the namespace to enable automatic Istio sidecar injection. Without this label, pods will not have the istio-proxy container and Istio traffic management will not work.

# Label the namespace for Istio sidecar injection
kubectl label namespace default istio-injection=enabled

# Verify the label
kubectl get namespace default --show-labels

If you've already deployed the Helm chart without the namespace label, restart the deployments to inject the sidecar:

kubectl rollout restart deployment app-stable app-canary model-service-stable model-service-canary

After restart, verify pods have 2 containers (app + istio-proxy):

kubectl get pods -l app=app -o jsonpath='{range .items[*]}{.metadata.name}: {.spec.containers[*].name}{"\n"}{end}'

Full Deployment Command

# Deploy with Istio enabled, canary release, and monitoring
helm upgrade --install sms-checker ./sms-checker-helm-chart \
  -f sms-checker-helm-chart/values.yaml \
  --set istio.enabled=true \
  --set istio.host=sms-istio.local \
  --set istio.canary.enabled=true \
  --set istio.canary.weight=10 \
  --set istio.stickySession.enabled=true \
  --set prometheus.enabled=true \
  --kubeconfig kubeconfig

Add Host Entry

# Point the Istio host to the ingress gateway IP
echo "192.168.56.91 sms-istio.local" | sudo tee -a /etc/hosts

Istio Resources Created

The Helm chart creates the following Istio resources when istio.enabled=true. The chart deploys three istio objects to be precise, to expose throught the provisioned istio ingressgateway.

1. Gateway (`app-gateway`)

It works as a front door for incoming http traffic. It tells the Istio ingress gateway to accept requests for sms-istio.local on port 80 and pass them into the mesh.

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: app-gateway
spec:
  selector:
    istio: ingressgateway  # Configurable via values.yaml
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - "sms-istio.local"

2. VirtualService (`app-vs`)

The virtualservice contains routing rules that support:

force canary (for testing): Requests with x-canary: true header go to canary 100%
sticky sessions: Requests with sms-session=stable or sms-session=canary cookie route to the matching subset
normal traffic split: New requests follow the default traffic split (90% stable, 10% canary) and receive a Set-Cookie header

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: app-vs
spec:
  hosts: ["sms-istio.local"]
  gateways: ["app-gateway"]
  http:
    # Explicit canary testing via header
    - match:
        - headers:
            x-canary:
              exact: "true"
      route:
        - destination:
            host: app
            subset: canary
          weight: 100
    # Sticky session: route to stable if cookie says stable
    - match:
        - headers:
            cookie:
              regex: ".*sms-session=stable.*"
      route:
        - destination:
            host: app
            subset: stable
    # Sticky session: route to canary if cookie says canary
    - match:
        - headers:
            cookie:
              regex: ".*sms-session=canary.*"
      route:
        - destination:
            host: app
            subset: canary
    # Default weighted routing - sets sticky session cookie
    - route:
        - destination:
            host: app
            subset: stable
          weight: 90
          headers:
            response:
              add:
                Set-Cookie: "sms-session=stable; Path=/; Max-Age=3600"
        - destination:
            host: app
            subset: canary
          weight: 10
          headers:
            response:
              add:
                Set-Cookie: "sms-session=canary; Path=/; Max-Age=3600"

3. DestinationRule (`app-destinationrule`)

It defines two subsets that Istio can route to:

stable → pods labelled with version: stable
canary → pods labelled with version: canary

Note: Sticky sessions are handled by VirtualService cookie matching (see above), not by DestinationRule consistentHash, because consistentHash does not work correctly with weighted routing across subsets.

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: app-destinationrule
spec:
  host: app
  subsets:
    - name: stable
      labels:
        version: stable
    - name: canary
      labels:
        version: canary

Sticky Sessions

Sticky sessions ensure that once a user is routed to a specific version (stable or canary), they continue to see that version on subsequent requests. This is implemented using VirtualService cookie matching with response headers.

How It Works

On first request, Istio routes based on the configured weights (90/10)
The VirtualService adds a Set-Cookie response header (sms-session=stable or sms-session=canary) based on which version handled the request
On subsequent requests, the VirtualService checks for the cookie using regex matching
If sms-session=stable cookie is found, traffic routes directly to stable subset
If sms-session=canary cookie is found, traffic routes directly to canary subset
Cookie TTL is 3600 seconds (1 hour) by default

Configuration

istio:
  stickySession:
    enabled: true
    cookieName: "sms-session"
    ttl: 3600s

Testing the Canary release

Verify Istio Resources

# Check Gateway, VirtualService, and DestinationRule
KUBECONFIG=./kubeconfig kubectl get gateway,virtualservice,destinationrule

# View detailed configuration
KUBECONFIG=./kubeconfig kubectl get virtualservice app-vs -o yaml
KUBECONFIG=./kubeconfig kubectl get destinationrule app-destinationrule -o yaml

Verify Deployments

# Check both stable and canary deployments are running
KUBECONFIG=./kubeconfig kubectl get pods -l app=app --show-labels

# Expected output shows pods with version=stable and version=canary labels

Test Traffic Routing

Before testing, make sure you can access the cluster.

Verify nodes are ready KUBECONFIG=./kubeconfig kubectl get nodes

Verify pods are running

KUBECONFIG=./kubeconfig kubectl get pods

Verify Istio resources exist

KUBECONFIG=./kubeconfig kubectl get gateway,virtualservice,destinationrule

1. Normal Request, this demonstrates that the app is accessible through the Istio gateway and virtual services

# Multiple requests will be distributed ~90% stable, ~10% canary
# First request sets the sticky session cookie
curl -v -H "Host: sms-istio.local" http://192.168.56.91/sms/

# It should return the http health and show that you are connected to the local app through Istio.

2. Force Canary via Header

# Always routes to canary version
curl -v -H "Host: sms-istio.local" -H "x-canary: true" http://192.168.56.91/sms/

3. Test Sticky Sessions

Sticky sessions use a cookie to ensure users stay on the same version (stable or canary) across requests.

Test 1: Verify cookie is set on first request

rm -f cookies.txt
curl -v -c cookies.txt -H "Host: sms-istio.local" http://192.168.56.91/sms/ 2>&1 | grep -i "set-cookie"

Expected output: set-cookie: sms-session=stable; Path=/; Max-Age=3600 (or canary)

Test 2: Verify sticky routing to stable

# Set cookie to stable
rm -f cookies.txt
echo "sms-istio.local	FALSE	/	FALSE	1769999999	sms-session	stable" > cookies.txt

# Send multiple requests with the stable cookie
for i in {1..3}; do
  curl -s -b cookies.txt -H "Host: sms-istio.local" -H "Content-Type: application/json" \
    -X POST -d "{\"sms\":\"STABLE_TEST_$i\"}" http://192.168.56.91/sms/
  echo ""
  sleep 2
done

# Check logs - all requests should appear in stable, none in canary
KUBECONFIG=./kubeconfig kubectl logs -l app=app,version=stable --tail=20 | grep "STABLE_TEST"
KUBECONFIG=./kubeconfig kubectl logs -l app=app,version=canary --tail=20 | grep "STABLE_TEST"

Test 3: Verify sticky routing to canary

# Set cookie to canary
rm -f cookies.txt
echo "sms-istio.local	FALSE	/	FALSE	1769999999	sms-session	canary" > cookies.txt

# Send multiple requests with the canary cookie
for i in {1..3}; do
  curl -s -b cookies.txt -H "Host: sms-istio.local" -H "Content-Type: application/json" \
    -X POST -d "{\"sms\":\"CANARY_TEST_$i\"}" http://192.168.56.91/sms/
  echo ""
  sleep 2
done

# Check logs - all requests should appear in canary, none in stable
KUBECONFIG=./kubeconfig kubectl logs -l app=app,version=canary --tail=20 | grep "CANARY_TEST"
KUBECONFIG=./kubeconfig kubectl logs -l app=app,version=stable --tail=20 | grep "CANARY_TEST"

Expected results:

With sms-session=stable cookie: All requests go to stable pods only
With sms-session=canary cookie: All requests go to canary pods only

4. Verify Version in Response

The app includes an APP_VERSION environment variable that can help identify which deployment served the request. Check application logs:

# View logs from stable deployment
KUBECONFIG=./kubeconfig kubectl logs -l app=app,version=stable

# View logs from canary deployment
KUBECONFIG=./kubeconfig kubectl logs -l app=app,version=canary

Testing Model-Service Canary Routing

The model-service has its own canary deployment with Istio routing based on source labels. This ensures consistent routing: requests from stable app pods go to stable model, requests from canary app pods go to canary model.

1. Verify Deployments

KUBECONFIG=./kubeconfig kubectl get pods -l app=model-service --show-labels

You should see pods with version=stable and version=canary labels.

2. Verify Istio Configuration

KUBECONFIG=./kubeconfig kubectl get virtualservice,destinationrule | grep model

3. Verify Routing Configuration in Envoy

The routing is configured per-workload. Check that stable app routes to stable model:

KUBECONFIG=./kubeconfig kubectl exec deploy/app-stable -c istio-proxy -- \
  pilot-agent request GET config_dump | grep -A5 '"cluster": "outbound|8081|' | head -10

You should see outbound|8081|stable|model-service for app-stable pods.

4. Check Logs After Using the App

Open the web UI at http://sms-istio.local/sms/ and submit a few SMS messages. Then check which model-service received the requests:

KUBECONFIG=./kubeconfig kubectl logs deploy/model-service-stable -c model-service --tail=5 | grep POST
KUBECONFIG=./kubeconfig kubectl logs deploy/model-service-canary -c model-service --tail=5 | grep POST

How we control the canary rollout

Image Tags (two versions running)

The helm chart can deploy stable deployment using 0.0.1, and canary deployment using 0.0.2.

app:
  tag: "0.0.1"       # Stable version
  canaryTag: "0.0.2" # Canary version

Traffic Weight

Adjust the percentage of traffic going to canary:

istio:
  canary:
    enabled: true
    weight: 10  # 10% to canary, 90% to stable

Progressive Rollout Example (how it can be gradually increased)

# Start with 5% canary
helm upgrade sms-checker ./sms-checker-helm-chart --set istio.canary.weight=5 --kubeconfig kubeconfig

# Monitor and increase to 25%
helm upgrade sms-checker ./sms-checker-helm-chart --set istio.canary.weight=25 --kubeconfig kubeconfig

# Full rollout  
helm upgrade sms-checker ./sms-checker-helm-chart --set app.tag=0.0.2 --set istio.canary.enabled=false --kubeconfig kubeconfig

Additional Use Case

Rate Limiting

name: envoy.filters.http.local_ratelimit
 typed_config:
  "@type": type.googleapis.com/udpa.type.v1.TypedStruct
  type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
  value:
    stat_prefix: http_local_rate_limiter
    # token_bucket: This sections has 3 attributes
    token_bucket:
      # max_tokens: Specifies the maximum number of tokens in the bucket, 
      # representing the maximum allowed requests within a certain time frame.
      max_tokens: {{ .Values.istio.rateLimiting.burstSize }}
      # tokens_per_fill: Indicates the number of tokens added to the bucket with each fill, 
      # essentially the rate at which tokens are replenished.
      tokens_per_fill: {{ .Values.istio.rateLimiting.tokensPerFill }}
      # fill_interval: Defines the time interval at which the bucket is replenished with tokens.
      fill_interval: {{ .Values.istio.rateLimiting.fillIntervalSeconds }}s

This configuration allows a burst that is up to 5 requests, and it refills 1 token every 60 seconds.

When there are more requests, the receive HTTP 429 Too Many Requests.

We also implemented a custom response header for visibility purposes.

response_headers_to_add: 
  - append: false
    header:
      key: {{ quote .Values.istio.rateLimiting.headerName }}
      value: {{ quote .Values.istio.rateLimiting.headerValue }}

Also, our rate limiting is applied to both app and model-service by attaching the filter to the Istio ingress gateway.

workloadSelector:
  labels: 
    istio: {{ .Values.istio.ingressGateway.selector }}

Check if everything is running

KUBECONFIG=./kubeconfig kubectl -n istio-system get envoyfilter

You should see "local-rate-limit" in the list.

Make Istio host resolve to the Istio gateway IP

echo "192.168.56.91 sms-istio.local" | sudo tee -a /etc/hosts

Testing Rate Limiting

User-based rate-limiting is implemented, the users that we know have the user IDs: 001, 002, and 003.

In the case that the user ID does not match, we fall back to global rate-limiting.

For demonstration purposes, we have kept the "tokens_per_fill" relatively low.

echo "waiting just in case"
sleep 65

echo User 1 with user ID: 001
for i in {1..5}; do
  code=$(curl -s -o /dev/null -w "001 $i %{http_code}\n" \
  -H "Host: sms-istio.local" \
  -H "x-instance-id: 001" \
  http://sms-istio.local/sms/)
  echo "Request $i: $code"
done

echo "waiting for refill"
sleep 65

echo User 2 with user ID: 002
for i in {1..7}; do
  code=$(curl -s -o /dev/null -w "002 $i %{http_code}\n" \
  -H "Host: sms-istio.local" \
  -H "x-instance-id: 002" \
  http://sms-istio.local/sms/)
  echo "Request $i: $code"
done

echo "waiting for refill"
sleep 65

echo User 3 with user ID: 003
for i in {1..9}; do
  code=$(curl -s -o /dev/null -w "003 $i %{http_code}\n" \
  -H "Host: sms-istio.local" \
  -H "x-instance-id: 003" \
  http://sms-istio.local/sms/)
  echo "Request $i: $code"
done

echo "waiting for refill"
sleep 65

echo Unknown user with user ID: 777
for i in {1..15}; do
  code=$(curl -s -o /dev/null -w "777 $i %{http_code}\n" \
  -H "Host: sms-istio.local" \
  -H "x-instance-id: 777" \
  http://sms-istio.local/sms/)
  echo "Request $i: $code"
done

echo "waiting for refill"
sleep 65

echo User with no header
for i in {1..15}; do
  code=$(curl -s -o /dev/null -w "NOHEADER $i %{http_code}\n" \
  -H "Host: sms-istio.local" \
  http://sms-istio.local/sms/)
  echo "Request $i: $code"
done

For user 1:

You should be seeing HTTP 200 for the first ~3 requests
Afterwards, you should be seeing HTTP 429

For user 2:

You should be seeing HTTP 200 for the first ~5 requests
Afterwards, you should be seeing HTTP 429

For user 3:

You should be seeing HTTP 200 for the first ~7 requests
Afterwards, you should be seeing HTTP 429

For unknown user and no header:

You should be seeing HTTP 200 until the global limit is reached
Afterwards, you should start seeing HTTP 429

It is expected that you will start seeing HTTP 200 after some HTTP 429 responses. The fill interval is 60s, meaning that every minute, the specified amount of tokens (tokens per fill) is added to the token bucket. Since we are sending low numbers of requests with the first three users, the 60s limit is not reached. However, as you can see in the last two users, since we send more requests and more time passes, we start seeing HTTP 200 responses as the tokens are getting refilled.

In order to inspect,

curl -i http://sms-istio.local/sms/

That should give you:

HTTP/1.1 429 Too Many Requests
x-local-rate-limit: TOO_MANY_REQUESTS

Name		Name	Last commit message	Last commit date
ansible		ansible
docs		docs
shared		shared
sms-checker-helm-chart		sms-checker-helm-chart
.env		.env
.gitignore		.gitignore
ACTIVITY.md		ACTIVITY.md
README.md		README.md
Vagrantfile		Vagrantfile
docker-compose.yml		docker-compose.yml

Time	State
< 2 min	Pending
≥ 2 min	Firing

Uh oh!

Latest commit

History

Folders and files

Repository files navigation