Feat Request: Constrained Decoding

[alan-insam]

Hey,

I’m trying to confirm whether constrained decoding is currently supported by the downstream inference engines used in this project. Specifically, I’m interested in the ability to enforce structured output formats, such as generating valid JSON or following a defined schema, directly during decoding rather than relying on post-processing.

This capability would greatly improve reliability for applications that require well-formed structured responses.

Thanks for your work on the project!
Alan

[DenisovAV]

Hi @alan-insam,

Constrained decoding (structured output / JSON schema enforcement during decoding) is not currently supported by either inference engine we use:

• MediaPipe GenAI — no constrained decoding API
• LiteRT-LM — added llguidance support in v0.9.0-alpha02, but only for function calling schemas, not general JSON schema enforcement

What's available today:

1. Function calling — supported for Gemma 3, Hammer, DeepSeek, Qwen models. Constrains output to function call format with defined parameter schemas
2. Prompt engineering — request JSON output in the prompt (less reliable)
3. Post-processing — validate and parse the response after generation

If constrained decoding is important for your use case, I'd suggest filing feature requests on the upstream SDKs:

• https://github.com/google-ai-edge/mediapipe
• https://github.com/google-ai-edge/LiteRT-LM