[OpenVINO backend] Adding support for OpenVINO backend & support inference for Mistral & Gemma & GPT2

[Mohamed-Ashraf273]

Description of the change

As a part of my GSoC25 project to support inference with the openvino backend for Gemma , Mistral and GPT-2,
This is my PR for supporting Gemma , Mistral and GPT-2 pipelines.

import os
os.environ["KERAS_BACKEND"] = "openvino"
import keras_hub

model = keras_hub.models.GPT2CausalLM.from_preset(
    "gpt2_large_en", dtype="float32"
)
model.summary()
output = model.generate("Keras is ", max_length=20)
print("Generated text:", output)

Reference

https://docs.openvino.ai/2025/index.html
https://keras.io/api/
https://keras.io/keras_hub/

Colab Notebook

Checklist

• I have added all the necessary unit tests for my change.
• I have verified that my change does not break existing code and works with all backends (TensorFlow, JAX, and PyTorch).
• My PR is based on the latest changes of the main branch (if unsure, rebase the code).
• I have followed the Keras Hub Model contribution guidelines in making these changes.
• I have followed the Keras Hub API design guidelines in making these changes.
• I have signed the Contributor License Agreement.

[Mohamed-Ashraf273]

@fchollet
@mattdangerw
@divyashreepathihalli
I'd appreciate any help reviewing this PR so we can get it merged. Thanks in advance!

[gemini-code-assist]

Summary of Changes

Hello @Mohamed-Ashraf273, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

I've implemented support for running inference with Gemma, Mistral, and GPT-2 models on the OpenVINO backend. This work is part of my GSoC25 project and aims to broaden the compatibility of Keras Hub models with various hardware platforms. The changes involve integrating OpenVINO's inference capabilities directly into the CausalLM models, introducing new utility functions for OpenVINO-specific operations, and updating the testing infrastructure to properly handle the OpenVINO backend's characteristics, including skipping tests that involve training or have known incompatibilities.

Highlights

• OpenVINO Backend Integration: I've added comprehensive support for running inference with Gemma, Mistral, and GPT-2 models using the OpenVINO backend. This significantly expands the deployment options for these models within Keras Hub.
• New OpenVINO Utility Module: To facilitate OpenVINO integration, I've introduced a new utility module, openvino_utils.py, which handles critical aspects like input parameterization, structured output processing, and intelligent model reusability checks for optimized performance.
• CausalLM Model Adaptations for OpenVINO: I've updated the core CausalLM model's make_generate_function to dynamically switch to OpenVINO-optimized inference when the OpenVINO backend is active. This includes handling specific ops.slice and ops.reshape behaviors required by OpenVINO for certain models like Gemma and GPT-2.
• Enhanced Test Skipping and Compatibility for OpenVINO: To ensure test suite compatibility and efficiency with the OpenVINO backend, I've implemented a new requires_trainable_backend pytest marker. This allows tests that rely on training functionalities (not yet supported by OpenVINO) to be skipped appropriately. Additionally, I've introduced exclusion lists (openvino_excluded_concrete_tests.txt and openvino_excluded_tests.txt) to manage specific test skips for OpenVINO.
• Sampler Logic Adjustments for OpenVINO: The sampler.py module has been modified to ensure its ops.while_loop constructs are compatible with OpenVINO's requirements, specifically by explicitly passing the mask parameter in the loop's condition and body functions.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds support for OpenVINO backend inference for several models, which is a significant enhancement. The changes include modifications to the testing infrastructure to handle backend-specific tests, updates to the core model generation logic, and the addition of new OpenVINO utility functions and tests. Overall, the implementation is solid, but I've identified a critical bug in the CausalLM class that could lead to an AttributeError, along with several opportunities for code simplification, efficiency improvements, and increased robustness. Please see my detailed comments below.

[Mohamed-Ashraf273]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for the OpenVINO backend for Gemma, Mistral, and GPT-2 models. The changes are extensive and well-structured, including:

• Adding OpenVINO-specific logic in CausalLM for model compilation and inference.
• Introducing a new openvino_utils.py module with helper functions and corresponding tests.
• Updating conftest.py to selectively skip tests that are not compatible with the OpenVINO backend, using a new marker and an exclusion list.
• Adding openvino to the project dependencies.

The implementation is solid, but I have a few suggestions to improve robustness and maintainability, such as using more specific exception handling, avoiding hardcoded device names for OpenVINO, and cleaning up test exclusion files. Overall, this is a great contribution to extend backend support.

[Mohamed-Ashraf273]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for the OpenVINO backend for Gemma, Mistral, and GPT-2 models. The changes are comprehensive, touching the test configuration to skip non-compatible tests, adding core OpenVINO inference logic to CausalLM, and including necessary workarounds in model implementations. My review focuses on improving the correctness and maintainability of the new OpenVINO integration. I've identified a potential issue with input handling for the OpenVINO model, a recommendation to improve exception handling, and pointed out some opportunities for code cleanup.

[Mohamed-Ashraf273]

@mattdangerw
@divyashreepathihalli
@fchollet
Could you please take a look?

[Mohamed-Ashraf273]

@mattdangerw
I've addressed all your feedback, and the PR is ready for review.
Thanks for your help!

[Mohamed-Ashraf273]

@mattdangerw
@divyashreepathihalli

[Mohamed-Ashraf273]

@mattdangerw
@divyashreepathihalli
@hertschuh
@abheesht17
@fchollet

[Mohamed-Ashraf273]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for the OpenVINO backend for inference with Gemma, Mistral, and GPT-2 models. The changes are comprehensive, including modifications to the test configuration to selectively skip unsupported tests, adjustments in the CausalLM and Sampler implementations for OpenVINO compatibility, and the addition of a new openvino_utils.py module with helper functions for inference and test management. I've found a couple of areas for improvement. There's a bug in how the test whitelist file is parsed, as it doesn't handle comments correctly. I've also suggested a refactoring in the TrainingMethodDetector class for better performance and code style. Overall, this is a great contribution to extend Keras Hub's backend support.

[Mohamed-Ashraf273]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for the OpenVINO backend for inference with several models like Gemma, Mistral, and GPT-2. The changes are extensive, including a new OpenVINO utilities module with helpers for model compilation, inference, and test management. The test configuration is updated to intelligently skip tests that are not compatible with the OpenVINO backend. The core model generation logic is also adapted to use the OpenVINO inference path when the backend is active.

My review focuses on improving performance in the test suite and enhancing code quality in the new test files. I've identified an inefficiency in how OpenVINO-specific tests are configured and a code style issue regarding local imports in the new test file. Overall, this is a solid contribution that significantly expands the backend support of Keras Hub.

[Mohamed-Ashraf273]

@mattdangerw
@divyashreepathihalli
@fchollet
@hertschuh
@abheesht17

[Mohamed-Ashraf273]

@mattdangerw
@divyashreepathihalli
@fchollet
@hertschuh
@abheesht17