[Bug]: GPU inference with static batch size produces corrupted outputs

[MaKlaib]

OpenVINO Version

tested with 2023.3 & 2024.6 & 2025.3

Operating System

Windows System

Device used for inference

GPU

Framework

Keras (TensorFlow 2)

Model used

Adapted U-Net architecture

Issue description

The issue arises when running GPU inference with a static batch size on a U-Net segmentation model (trained with TF 2.20) in OpenVINO. Models exported via ov.convert_model/ov.save_model (tested across versions 2023.3, 2024.6, and 2025.3) or with the Model Optimizer (2023.3) using legacy_frontend=false consistently produce corrupted segmentation results in this setup. In contrast, CPU inference works correctly with both static and dynamic batch sizes, and GPU inference also succeeds with static batch size when the Model Optimizer (2023.3) is used with legacy_frontend=true.

Using dynamic batch size on the GPU is not a viable option, as throughput drops significantly compared to a fixed static batch size.

Correct result (e.g., CPU or GPU dynamic batch size or GPU legacy model optimizer):

Corrupted result (GPU static batch size with model conversion tool):

Step-by-step reproduction

Model Conversion

• Model Conversion which results in corrupt GPU Model:

import openvino as ov

model = ov.convert_model("tf_saved_model_path", input=[1,256,256,1])
ov.save_model(model, "model_path.xml") # saves model with input shape [1,256,256,1]

• Model Conversion which results in working GPU Model:

import openvino as ov

model = ov.convert_model("tf_saved_model_path")
ov.save_model(model, "model_path.xml") # saves model with input shape [?,256,256,1]

• For OpenVINO 2023.3 Model Optimizer variants (command line):

# Variant 1: corrupt GPU Model
mo --saved_model_dir path/to/tensorflow_saved_model --output_dir path/to/mo_output --shape [1,256,256,1] --legacy_frontend false

# Variant 2: working GPU Model
mo --saved_model_dir path/to/tensorflow_saved_model --output_dir path/to/mo_output_legacy --shape [1,256,256,1] --legacy_frontend true`

Model Inference

import openvino as ov
import cv2

core = ov.Core()
compiled_model = core.compile_model("path_to_xml", device="GPU") # or 'CPU'
infer_request = compiled_model.create_infer_request()

img = cv2.imread("img.png", 0)
img = img / 255.0
img = img.reshape([1,256,256,1])

infer_request.set_input_tensor(ov.Tensor(img))
infer_request.infer()
output_tensor = infer_request.get_output_tensor()
arr = output_tensor.data[0]
arr = arr[:,:,1] # get channel information of channel with index 1 (shape 256x256)

arr = arr * 255
cv2.imwrite("result.png", arr)

Relevant log output

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[YuChern-Intel]

Could you share your tf_saved_model files and also the input image for the replication purpose?

[MaKlaib]

The tf_saved_model folder and a sample input image are attached in the zip folder:
SavedModel_and_image.zip

[YuChern-Intel]

Thanks for sharing the model.

However, I unable to reproduce the issue by running your codes. Is there anything missing from the code snippet as you able to run the inference?