Fix integration test failures

[tae0y]

Describe the bug
Since adding the Ollama server to the GitHub Actions CI/CD pipeline, the integration tests have been intermittently failing.

To Reproduce
Run GitHub Actions Azure Dev or Azure Dev - Build Only Workflow.

Expected behavior
Integration tests succeed.

Additional context
I’m attaching the link to the recently failed GitHub Actions run.

• https://github.com/aliencube/open-chat-playground/actions/runs/18973322004
• https://github.com/aliencube/open-chat-playground/actions/runs/18970998475
• https://github.com/aliencube/open-chat-playground/actions/runs/18970812999
• https://github.com/aliencube/open-chat-playground/actions/runs/18945315875
• https://github.com/aliencube/open-chat-playground/actions/runs/18943359984

[tae0y]

The issue cannot be reproduced locally.
The tests that fail are inconsistent each time.
The failure messages themselves are all timeouts.

It’s suspected that the timeouts occur due to Ollama’s parallel processing limitations.
Since increasing Ollama’s processing capacity is difficult,
we should try adjusting the number of concurrent requests to see if that mitigates the issue.

[tae0y]

When I ran only the LLMRequired integration tests separately as shown below, the CI did not fail.
However, it seems that the issue was not due to the number of requests sent to Ollama,
but rather because the total number of test executions affected Ollama.

I configured the forked repository as shown below and attempted Azure Dev builds 3 times.
They all succeeded.

• https://github.com/tae0y/open-chat-playground/actions/runs/19037960679
• https://github.com/tae0y/open-chat-playground/actions/runs/19038135314
• https://github.com/tae0y/open-chat-playground/actions/runs/19038285831

# .github/workflows/azure-dev.yml
- name: Run integration tests
  shell: bash
  run: |
    dotnet test ./test/OpenChat.PlaygroundApp.Tests -c Release \
        --no-build --logger "trx" --collect:"XPlat Code Coverage" \
        --filter "Category=IntegrationTest & Category!=IgnoreGitHubActions & Category!=LLMRequired"

- name: Run LLMRequired integration tests
  shell: bash
  run: |
    dotnet test ./test/OpenChat.PlaygroundApp.Tests -c Release \
        --no-build --logger "trx" --collect:"XPlat Code Coverage" \
        --filter "Category=IntegrationTest & Category!=IgnoreGitHubActions & Category=LLMRequired"

[justinyoo]

It’s suspected that the timeouts occur due to Ollama’s parallel processing limitations. Since increasing Ollama’s processing capacity is difficult, we should try adjusting the number of concurrent requests to see if that mitigates the issue.

이것 같긴 하네요. 동시성 이슈: https://github.com/ollama/ollama/blob/main/docs/faq.mdx#how-does-ollama-handle-concurrent-requests

그러면 위에 LLMRequired 랑 아닌거랑 따로 한 번 해서 PR 날려볼까요? 그리고 한 번 보시죠. UI 테스트할 때도 테스트별로 LLM이 정말로 필요한 건지 아닌지를 확인해야 할 듯.