Qwen3-VL-4B-Instruct on QAIRT 2.45: "Invalid Node IO value passed: GENIE_NODE_WILDCARD" - is deepstack actually wired? + question on Genie deprecation

[korkland]

Hi,

Two related questions about the Qwen3-VL-4B-Instruct Genie bundle and the Genie
runtime direction in general. Would appreciate any clarification.

Setup

• Model: Qwen3-VL-4B-Instruct (v0.58.0 release)
• Bundle: the official prebuilt for SA8775P from HF
  (qwen3_vl_4b_instruct-genie-w4a16-qualcomm_sa8775p.zip)
• Runtime: QAIRT 2.45, genie-app -s genie-app-script.txt

1. GENIE_NODE_WILDCARD warning - is deepstack actually connected?

The genie-app-script.txt shipped inside the SA8775P bundle contains:

pipeline connect GeniePipeline imageEncoder GENIE_NODE_IMAGE_ENCODER_EMBEDDING_OUTPUT textGenerator GENIE_NODE_TEXT_GENERATOR_EMBEDDING_INPUT
pipeline connect GeniePipeline lutEncoder GENIE_NODE_TEXT_ENCODER_EMBEDDING_OUTPUT textGenerator GENIE_NODE_TEXT_GENERATOR_EMBEDDING_INPUT
pipeline connect GeniePipeline imageEncoder GENIE_NODE_WILDCARD textGenerator GENIE_NODE_WILDCARD

The third line prints in red at runtime:

Invalid Node IO value passed: GENIE_NODE_WILDCARD

Root cause on our side is clear: GENIE_NODE_WILDCARD is not in the public
GenieNode_IOName_t enum in include/Genie/GenieNode.h for QAIRT 2.45, and
genie-app's stringToNodeIO map has no such key, so it throws
std::invalid_argument. The 2.45 genie-app catches this per-line and keeps
going (2.44 did not - it would abort the script), which is why the pipeline
still runs and we still get a coherent image description.

Meanwhile the decoder graphs really do expect deepstack inputs
(we can see them in our own compile logs, and it matches your architecture):

visual_pos_masks               1,128 (or 1,1 in token stage)
deepstack_visual_embeds_0      256,2560
deepstack_visual_embeds_1      256,2560
deepstack_visual_embeds_2      256,2560

So my question is: on QAIRT 2.45, are these 4 tensors auto-wired by name
inside libGenie (WILDCARD line is just informational and safe to ignore),
or are they left unfed and effectively zeroed, meaning the deepstack
re-injection at decoder layers 0-2 is silently disabled?

If it is the latter, will proper deepstack wiring (or a real
GENIE_NODE_WILDCARD / equivalent) be supported in an upcoming Genie release?
is there a canonical way to wire them on 2.45?
We would prefer to stay on the Genie path rather than move to GenieX (see
question 2 below), so knowing this is on the Genie roadmap would help a lot.

2. Why deprecate Genie for embedded targets?

The README notes that "Genie support will be deprecated soon" in favor of
GenieX. From our perspective this feels backwards for embedded / IoT use
cases, and I'd love to understand the reasoning.

Genie today is essentially a thin C++ wrapper over QNN (the qualla layer
plus dialog/pipeline orchestration). For an embedded stack that already
links libQnnHtp and manages its own I/O, this is close to ideal:

• No extra runtime dependencies beyond QAIRT
• Small, embeddable C/C++ API surface
• Source is available, so we can trace behavior and fork behavior when needed
• Fits directly into our own scheduler / power / memory policies

GenieX on the other hand is packaged as a user-facing runtime (CLI, Python,
Kotlin/Android, Docker, OpenAI-compatible server). Its QAIRT plugin is
BSD-3-Clause which is great, but the docs show it also removes control that
raw QNN gives us today (on Android for example nGpuLayers and nCtx are
rejected, precision / context / KV size are baked in the bundle, etc.).

For a system integrator wanting to build a controllable inference layer on
Snapdragon / Dragonwing platforms, Genie's shape (a lean C++ wrapper over
QNN) is a much better starting point than GenieX. Deprecating it seems to
push everyone either onto a higher-level runtime that gives us less control,
or back to hand-rolling everything directly against QNN.

Could you share:

• The rationale behind the Genie -> GenieX transition (support burden,
  architectural direction, feature parity plan?)
• Whether the Genie C++ API (or something at that level of abstraction) will
  continue to be maintained for embedded / OEM use cases
• Whether the deepstack / VLM pieces that are missing on Genie 2.45 will land
  in a future Genie release

Thanks!
Chen

[gustavla]

Chen (@korkland)
Really sorry this was left unanswered for a long time. I don't have all the answer yet, but this is what I tell you right now:

Issue 1:
I tried this myself on QDC. I'm actually struggling to run the VLM demo through genie-app. The process consistently gets killed, so I never reach a repro of the wildcard message you are seeing. The genie-t2t-run as pure LLM works though.

This device isn't yet being verified through automatic infrastructure, which is why the guarantee that it works isn't as high. Note, we are no longer listing this device as part of our model cards, but you are right we are still providing assets.

As far as the Genie error (warning). I agree it does suggest the deepstack features aren't being passed in correctly. I'll need to reach out to the Genie team to understand this error better.

Issues 2:
The goal of GenieX is that it should work well in this setting as well and be a full replacement of Genie. You can use it from C or C++ through the C API without engaging any of the higher-level paths (Python, CLI). It does come packaged in two separate packages (QAIRT and GenieX). If there are high-level constructs (Python API, CLI) that you don't want, I think it should be possible to delete those files during installation. However, let's see what GenieX folks say (AlexCHEN (@alexchen4ai)).

Source is available for GenieX as well (https://github.com/qualcomm/GenieX), so it's it's possible to ship using a custom version (similar to current typical OEM use of Genie). In fact, developing it in the open is one of the big shifts from Genie to GenieX for us, and if customers feel the need to fork, we encourage upstream contributions.

Fits directly into our own scheduler / power / memory policies

What are the levers that you were able to change before and can't change in GenieX?

it also removes control that raw QNN gives us today (on Android for example nGpuLayers and nCtx are rejected, precision / context / KV size are baked in the bundle, etc.).

AlexCHEN (@alexchen4ai) Can you take a look at this?
For my own understanding, what does nGpuLayers do in a context where you have only context binaries? And precision / context / KV size all have to be decided during assets build time, so this doesn't change between Genie and GenieX. Or, maybe I'm misunderstanding?

[alexchen4ai]

Chen (@korkland) Really sorry this was left unanswered for a long time. I don't have all the answer yet, but this is what I tell you right now:

Issue 1: I tried this myself on QDC. I'm actually struggling to run the VLM demo through genie-app. The process consistently gets killed, so I never reach a repro of the wildcard message you are seeing. The genie-t2t-run as pure LLM works though.

This device isn't yet being verified through automatic infrastructure, which is why the guarantee that it works isn't as high. Note, we are no longer listing this device as part of our model cards, but you are right we are still providing assets.

As far as the Genie error (warning). I agree it does suggest the deepstack features aren't being passed in correctly. I'll need to reach out to the Genie team to understand this error better.

Issues 2: The goal of GenieX is that it should work well in this setting as well and be a full replacement of Genie. You can use it from C or C++ through the C API without engaging any of the higher-level paths (Python, CLI). It does come packaged in two separate packages (QAIRT and GenieX). If there are high-level constructs (Python API, CLI) that you don't want, I think it should be possible to delete those files during installation. However, let's see what GenieX folks say (AlexCHEN (@alexchen4ai)).

Source is available for GenieX as well (https://github.com/qualcomm/GenieX), so it's it's possible to ship using a custom version (similar to current typical OEM use of Genie). In fact, developing it in the open is one of the big shifts from Genie to GenieX for us, and if customers feel the need to fork, we encourage upstream contributions.

Fits directly into our own scheduler / power / memory policies

What are the levers that you were able to change before and can't change in GenieX?

it also removes control that raw QNN gives us today (on Android for example nGpuLayers and nCtx are rejected, precision / context / KV size are baked in the bundle, etc.).

AlexCHEN (@alexchen4ai) Can you take a look at this? For my own understanding, what does nGpuLayers do in a context where you have only context binaries? And precision / context / KV size all have to be decided during assets build time, so this doesn't change between Genie and GenieX. Or, maybe I'm misunderstanding?

Thanks Gustav Larsson (@gustavla) for the ping! Chen (@korkland) I suggest you can take a look at https://github.com/qualcomm/geniex-qairt-plugin where you can have granular control like Genie. We have have good documentation of the runtime: https://geniex.aihub.qualcomm.com/en/get-started/what-is-geniex.
Deepstack / VLM (Q1 + Q3): Qwen3-VL deepstack is already supported in geniex-qairt-plugin. No need to wait on a Genie release.

The rational from Genie to GenieX:
Genie → GenieX rationale (Q2): Two drivers. First, usability and multimodality — Genie's dialog/pipeline model was built around text generation, with multimodal bolted on per architecture; the deepstack friction you hit is a good example. GenieX treats multimodal as first-class. Second, GenieX also has a llama.cpp path, so one runtime and one API cover both.

On low-level control: geniex-qairt-plugin has the same granularity as Genie — same layer over QNN, not a higher-level wrapper — so the config surface you use for your own scheduler / power / memory policy is still there. Please read through the repo directly, and if you find a lever Genie exposes that the plugin doesn't, file it and we'll treat it as a gap to close.

nGpuLayers / nCtx are llama.cpp-path parameters. With precompiled QNN context binaries, backend partitioning and max context/KV sizing are fixed at asset build time, so there's nothing for nGpuLayers to offload at runtime — hence rejected rather than ignored. On the QAIRT plugin path, context and KV sizing behave as they do under Genie today.

So, directly: the Genie-level abstraction for embedded / OEM isn't going away — it's geniex-qairt-plugin.

Also, we have carefully prepared the skills for GenieX so that you can build additional new features using agents easily.

[ashish-frozo]

Chen (@korkland) not arguing Genie vs GenieX. The open question you asked is the one that ships: on QAIRT 2.45, are visual_pos_masks + deepstack_visual_embeds_0/1/2 actually fed on SA8775P, or zeroed while genie-app still prints a caption.

gustavla already said the WILDCARD warning suggests they are not passed correctly, and SA8775P is not on automatic infrastructure.

We run the official Qwen3-VL-4B w4a16 bundle on the target and fail the merge if those four tensors never move (or NPU% drops) even when the caption looks coherent. 10 hosted runs if you want that as a PR check: https://edgegate.frozo.ai

Ashish / EdgeGate