[Bug] API call via LAN triggers HuggingFace download error when offline

[l14666]

Describe the bug

When I call the rkllama API via LAN, the model initialization fails with an error that tries to connect to HuggingFace, even though I have already prepared the model files locally.

To Reproduce

Steps to reproduce the behavior:

1. Start rkllama server on device A

   rkllama serve --model Qwen2.5-3B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-1.0
   

curl -X POST http://172.16.20.254:8080/api/chat
-H "Content-Type: application/json"
-d '{
"model": "Qwen2.5-3B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-1.0",
"messages": [{"role": "user", "content": "Hello"}]
}'

{"error":"We couldn't connect to 'https://huggingface.co' to load the files, and couldn't find them in the cached files.\nCheck your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'."}

[bigworldsmall]

I’m having the same issue. How can I fix it?

Using the local model, I can run Q&A normally in the command line, but when calling http://127.0.0.1:8080/api/chat it throws an error.

{"error":"We couldn't connect to 'https://huggingface.co/' to load the files, and couldn't find them in the cached files.\nCheck your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'."}