Nix flake to run llama.cpp with CUDA acceleration on the Jetson Orin Nano.
To run the default server config:
nix runTo run llama-server in router mode:
nix run .#llama-server-routeregentway/llama_cpp_orin
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|