samples/aks-store-demo - ai-service with local LLM model

[mathieu-benoit]

samples/aks-store-demo - ai-service with local LLM model

Just for score-compose for now, not yet supported with score-k8s.

score-compose init --no-sample \
    --provisioners https://raw.githubusercontent.com/score-spec/community-provisioners/refs/heads/main/service/score-compose/10-service.provisioners.yaml \
    --provisioners https://raw.githubusercontent.com/score-spec/community-provisioners/refs/heads/main/llm-model/score-compose/10-dmr-llm-model.provisioners.yaml

score-compose generate \
    order/score.yaml \
    ai/score.yaml \
    product-ai/score.yaml \
    store-front/score.yaml \
    store-admin/score.yaml \
    makeline/score.yaml

docker compose up --build -d --remove-orphans

docker images:

ghcr.io/azure-samples/aks-store-demo/ai-service:2.1.0         55be3a444826        202MB         47.5MB    U   
ghcr.io/azure-samples/aks-store-demo/makeline-service:2.1.0   d4898dd5cd9d       44.7MB         14.8MB    U   
ghcr.io/azure-samples/aks-store-demo/order-service:2.1.0      5904458f4bb7        272MB         59.8MB    U   
ghcr.io/azure-samples/aks-store-demo/product-service:2.1.0    17686395e713        177MB         43.7MB    U   
ghcr.io/azure-samples/aks-store-demo/store-admin:2.1.0        510d7b397d25       24.1MB         7.68MB    U   
ghcr.io/azure-samples/aks-store-demo/store-front:2.1.0        ffad6d063d89       27.2MB         9.24MB    U      
mirror.gcr.io/mongo:8                                         7f5bbdafebde       1.26GB          330MB    U   
mirror.gcr.io/nginx:1-alpine                                  8491795299c8         82MB         23.9MB    U   
mirror.gcr.io/rabbitmq:3-management-alpine                    606d8c0d6b3c        277MB         87.7MB    U

docker models list:

MODEL NAME              PARAMETERS  QUANTIZATION  ARCHITECTURE  MODEL ID      CREATED       CONTEXT  SIZE       
smollm2:135M-Q2_K       134.52 M    Q2_K          llama         eba11bf8f361  8 months ago           82.41 MiB

docker ps:

CONTAINER ID   IMAGE                                                         COMMAND                  CREATED          STATUS                    PORTS                                                                  NAMES
b10ea15b3409   ghcr.io/azure-samples/aks-store-demo/product-service:2.1.0    "./product-service"      41 minutes ago   Up 41 minutes                                                                                    aks-store-demo-product-service-product-service-1
6ddc9fd1c0c4   ghcr.io/azure-samples/aks-store-demo/store-front:2.1.0        "/docker-entrypoint.…"   54 minutes ago   Up 50 minutes             80/tcp, 8080/tcp                                                       aks-store-demo-store-front-store-front-1
00680675e107   ghcr.io/azure-samples/aks-store-demo/store-admin:2.1.0        "/docker-entrypoint.…"   54 minutes ago   Up 50 minutes             80/tcp, 8081/tcp                                                       aks-store-demo-store-admin-store-admin-1
1a9212f61931   ghcr.io/azure-samples/aks-store-demo/ai-service:2.1.0         "uvicorn main:app --…"   54 minutes ago   Up 54 minutes             5001/tcp                                                               aks-store-demo-ai-service-ai-service-1
7c36d5b28fd2   ghcr.io/azure-samples/aks-store-demo/order-service:2.1.0      "docker-entrypoint.s…"   54 minutes ago   Up 54 minutes             3000/tcp                                                               aks-store-demo-order-service-order-service-1
279aa4c6e3ca   ghcr.io/azure-samples/aks-store-demo/makeline-service:2.1.0   "./main"                 54 minutes ago   Up 54 minutes             3001/tcp                                                               aks-store-demo-makeline-service-makeline-service-1
30320b0e6159   mirror.gcr.io/rabbitmq:3-management-alpine                    "docker-entrypoint.s…"   54 minutes ago   Up 54 minutes (healthy)   4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 15691-15692/tcp, 25672/tcp   aks-store-demo-rabbitmq-nX9AKg-1
9ec4a3c84e0e   mirror.gcr.io/nginx:1-alpine                                  "/docker-entrypoint.…"   54 minutes ago   Up 54 minutes             0.0.0.0:8080->80/tcp, [::]:8080->80/tcp                                aks-store-demo-routing-FFV3Yh-1
764b215b105f   mirror.gcr.io/mongo:8                                         "docker-entrypoint.s…"   54 minutes ago   Up 54 minutes (healthy)   27017/tcp                                                              aks-store-demo-mongo-Kt1dmI-1
b36f22e8ad0a   docker/model-runner:latest                                    "/app/model-runner"      2 days ago       Up 3 hours                127.0.0.1:12434->12434/tcp, 172.17.0.1:12434->12434/tcp                docker-model-runner

Website deployed and available at:

curl $(score-compose resources get-outputs dns.default#store-front.dns --format '{{ .host }}:8080')
curl $(score-compose resources get-outputs dns.default#store-admin.dns --format '{{ .host }}:8080')

We can now see the Ask AI Assistant button next to the Description field on the Product page now: