How to run in termux

[belog2867]

I solved this problem through ai.

/data/data/com.termux/files/home/llamaqnn/ggml/src/ggml-qnn/utils.cpp:253:17: error: reference to unresolved using declaration
253 | return std::aligned_alloc(alignment, size);
| ^
/data/data/com.termux/files/usr/include/c++/v1/cstdlib:150:1: note: using declaration annotated with 'using_if_exists' here
150 | using ::aligned_alloc _LIBCPP_USING_IF_EXISTS;
| ^
/data/data/com.termux/files/home/llamaqnn/ggml/src/ggml-qnn/utils.cpp:253:12: error: excess elements in scalar initializer
253 | return std::aligned_alloc(alignment, size);
| ^ ~~~~~~
2 errors generated.
make[2]: *** [ggml/src/ggml-qnn/CMakeFiles/ggml-qnn.dir/build.make:177: ggml/src/ggml-qnn/CMakeFiles/ggml-qnn.dir/utils.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:2370: ggml/src/ggml-qnn/CMakeFiles/ggml-qnn.dir/all] Error 2

Then how to let backend find correct lib. I am using Xiaomi 14 Pro

[belog2867]

he full capacity of the model will not be utilized extend_lib_search_path is nullptr, will use . as defaultunable to find a valid qnn system interface failed to load QNN system lib
failed to init qnn backend qnn-gpu解

[chraac]

oh, do you mind build it with offical ndk-toolchain on your host machine? thats the official way google suggested, and its clang is relatively new and support the newer c/c++ standard

Can have a look here, hope it helps: https://github.com/chraac/llama.cpp/wiki/How-to-Build

[belog2867]

I set the location of the library, but it doesn't seem to be used. Can I locate these libraries in the approximate location of /vendor/lib64?

extend_lib_search_path is nullptr, will use /data/local/tmp as defaultfailed to load /data/local/tmp/libQnnSystem.so, fallback to libQnnSystem.so
can not load QNN library libQnnSystem.so, error: dlopen failed: library "libQnnSystem.so" not found
failed to load QNN system lib
failed to init qnn backend qnn-gpu
llama_init_from_model: failed to initialize qnn-gpu backend

This is my build command.
cd llama.cpp &&
mkdir build-android && cd build-android

cmake .. -G Ninja
-DCMAKE_TOOLCHAIN_FILE=$HOME/android-sdk/ndk/26.3.11579264/build/cmake/android.toolchain.cmake
-DANDROID_ABI=arm64-v8a
-DANDROID_PLATFORM=android-28
-DBUILD_SHARED_LIBS=OFF
-DGGML_RPC=ON
-DGGML_QNN=ON
-DGGML_QNN_SDK_PATH=~/qairt/2.31.0.250130
-DGGML_OPENMP:BOOL=OFF
-DGGML_QNN_DEFAULT_LIB_SEARCH_PATH=/vendor/lib64

ninja

[belog2867]

At the best of times, it told me that libQnnGpu.so was missing.

[chraac]

actually, you have to push the qnn related dynamic library to you cellphone before run, a complete list of those libs can be found here:
https://github.com/chraac/llama-cpp-qnn-builder/blob/dd7ba303a8e3213c8cafe330c0938b25c6bd788f/docker/build_in_container.sh#L83

[belog2867]

How to make this backend use lib in the./ directory?

[chraac]

maybe you can try to overwrite the LD_LIBRARY_PATH env var

[belog2867]

Now it gives me a new problem.extend_lib_search_path is nullptr, will use /data/local/tmp as defaultfailed to load /data/local/tmp/libQnnSystem.so, fallback to libQnnSystem.so
can not load QNN library libQnnSystem.so, error: dlopen failed: library "libstdc++.so.6" not found: needed by /data/data/com.termux/files/home/a/libQnnSystem.so in namespace (default)

Missing cpp library? ?
Thank you for your help.

[chraac]

hmmm, may I ask which Android version and device model you're using? looks like libQnnSystem.so relies on libstdc++.so.6 and has failed to load

[belog2867]

android 15 xiaomi 14 pro hyperos2.0.104.0

[chraac]

android 15 xiaomi 14 pro hyperos2.0.104.0

interesting....
my phone is also android 15, but there's no such error come out....

and this is symbol table dump of my libQnnSystem.so

libQnnSystem.so:     file format elf64-little

DYNAMIC SYMBOL TABLE:
0000000000000000      DF *UND*  0000000000000000 (LIBC)       __cxa_finalize
0000000000000000      DF *UND*  0000000000000000 (LIBC)       __cxa_atexit
0000000000000000      DF *UND*  0000000000000000 (LIBC)       strncmp
0000000000000000      DF *UND*  0000000000000000 (LIBC)       posix_memalign
0000000000000000      DF *UND*  0000000000000000 (LIBC)       syslog
0000000000000000      DF *UND*  0000000000000000 (LIBC)       snprintf
0000000000000000      DF *UND*  0000000000000000 (LIBC)       openlog
0000000000000000      DF *UND*  0000000000000000 (LIBC)       free
0000000000000000      DF *UND*  0000000000000000 (LIBC)       memchr
0000000000000000      DF *UND*  0000000000000000 (LIBC)       fwrite
0000000000000000      DF *UND*  0000000000000000 (LIBC)       realloc
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_getspecific
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_rwlock_unlock
0000000000000000      DF *UND*  0000000000000000 (LIBC)       memcpy
0000000000000000      DF *UND*  0000000000000000 (LIBC)       strndup
0000000000000000      DF *UND*  0000000000000000 (LIBC)       __errno
0000000000000000      DF *UND*  0000000000000000 (LIBC)       memmove
0000000000000000      DF *UND*  0000000000000000 (LIBC)       __system_property_get
0000000000000000      DF *UND*  0000000000000000 (LIBC)       memset
0000000000000000      DF *UND*  0000000000000000 (LIBC)       vfprintf
0000000000000000      DF *UND*  0000000000000000 (LIBC)       malloc
0000000000000000      DF *UND*  0000000000000000 (LIBC)       fprintf
0000000000000000      DF *UND*  0000000000000000 (LIBC)       abort
0000000000000000      DF *UND*  0000000000000000 (LIBC)       closelog
0000000000000000      DF *UND*  0000000000000000 (LIBC)       strlen
0000000000000000      DF *UND*  0000000000000000 (LIBC)       getauxval
0000000000000000      DO *UND*  0000000000000000 (LIBC)       __sF
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_cond_broadcast
0000000000000000      DF *UND*  0000000000000000 (LIBC)       strcmp
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_rwlock_rdlock
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_key_delete
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_setspecific
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_cond_wait
0000000000000000      DF *UND*  0000000000000000 (LIBC)       strerror_r
0000000000000000      DF *UND*  0000000000000000 (LIBC)       syscall
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_once
0000000000000000      DF *UND*  0000000000000000 (LIBC)       vasprintf
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_mutex_lock
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_rwlock_wrlock
0000000000000000      DF *UND*  0000000000000000 (LIBC)       android_set_abort_message
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_mutex_destroy
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_mutex_unlock
0000000000000000      DF *UND*  0000000000000000 (LIBC)       getpid
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_key_create
0000000000000000      DF *UND*  0000000000000000 (LIBC)       __assert2
0000000000000000      DF *UND*  0000000000000000 (LIBC)       fflush
0000000000000000      DF *UND*  0000000000000000 (LIBC)       fputc
0000000000000000      DF *UND*  0000000000000000 (LIBC)       dl_iterate_phdr
0000000000019f10 g    DF .text  000000000000002c  Base        QnnSystemInterface_getProviders

as you can see, there's no relied deps with libstdc++
wondering where did you get your libQnnSystem.so? it should be from <qnn_sdk_path>/qairt/2.31.0.250130/lib/aarch64-android for android

[belog2867]

Oh I got them in QNN_SDK_PATH/lib/aarch64-oe-linux-gcc11.2 and QNN_SDK_PATH/lib/hexagon-v*

[belog2867]

I got another problem,maybe I'm not far from success.

he full capacity of the model will not be utilized
extend_lib_search_path is nullptr, will use /data/local/tmp as defaultfailed to load /data/local/tmp/libQnnSystem.so, fallback to libQnnSystem.so
initialize qnn system successfully
failed to load /data/local/tmp/libQnnGpu.so, fallback to libQnnGpu.so
can not open QNN library libQnnGpu.so, with error: dlopen failed: cannot locate symbol "Xzs_Construct" referenced by "/system/lib64/libunwindstack.so"...
failed to load QNN backend
failed to init qnn backend qnn-gpu
llama_init_from_model: failed to initialize qnn-gpu backend
common_init_from_params: failed to create context with model '../.ollama/models/blobs/sha256-d547a29843982f182448eeacc5d3a9ddfb0f29f88d986b7339f5b533477023ac'
main: error: unable to load model

I have this lib in /system/lib64.