Are these stats normal for openvino on an iGPU?

[zolakt]

Hi,

my dual tpu coral has finally started dying after trying to plug it with the pcie adapter (I would definitely advise anyone against that thing btw). It freezes my whole proxmox host when runs detection. So, after months on nagging you guys if I should switch to openvino (sorry about that btw), I'm left with no options and had to switch.

I have a Intel Core i5 14400 with the UHD 730 iGPU. And I have a Frigate+ subscription. So I've been trying the yolov9s model, both 320x320 and 640x640. I only have a single camera.

First I tried 640x640 model and I find it barely usable. Inference time is around 50ms (which is ok I guess), but the "Detector CPU Usage" is almost 100%. And while I live view when it's doing detection it's very laggy, basically slow motion.

Then I switched to the 320x320 model. Here I have a 15-20ms interence time (which is ok I guess), and the "Detector CPU Usage" goes to around 50-75% when it's doing detection. There is still some lag, more than on the coral, but much less than with the 640x640 model. Here is a screenshot of the stats:

So I have a few questions:

1. are these stats expected for my iGPU
2. is the 640x640 too large for my iGPU, or is something wrong with my setup
3. is the "Detector CPU Usage" normal? I was a little surprised it would be so high, Is this really GPU usage or is it falling back to the CPU for some reason? Although, when I monitor the CPU in proxmox it doesn't go above 10%, and I also don't see any logs in Frigate that would indicate it's falling back to the CPU
4. since the "Detector CPU Usage" is quite high, will this even be usable for multiple cameras? Or is that normal behavior, that it pegs the GPU usage regardless of the number of cameras
5. I also have a Nvidia GPU on this machine, and I'm passing it to Frigate (I'm using the tensorrt image). I don't want to use it for inference. I want to use it for semantic search, face and licence plate detection and running ollama (for genAI). Are those enhancements now still running on the Nvidia card, or are they going to the iGPU? I'm not sure how can it know which one to use, both are GPUs

Here is my full config:

mqtt:
  enabled: true
  host: 192.168.2.10
  user: '{FRIGATE_MQTT_USER}'
  password: '{FRIGATE_MQTT_PASSWORD}'

auth:
  session_length: 604800
  refresh_time: 345600

model:
  path: plus://REDACTED_PLUS_MODEL_ID

detectors:
  ov:
    type: openvino
    device: GPU

objects:
  track:
    - person
    - face
    - baby
    - car
    - license_plate
    - bicycle
    - motorcycle
    - dog
    - cat
    - bird
    - fox
    - mustelid
  filters:
    person:
      min_score: 0.56
      threshold: 0.68
    car:
      threshold: 0.90
    cat:
      min_score: 0.4
      threshold: 0.6
    license_plate:
      min_score: 0.4

review:
  alerts:
    labels:
      - person
      - car
      - bicycle
      - motorcycle
      - cat
      - dog
      - fox
      - mustelid

record:
  enabled: true
  sync_recordings: false
  retain:
    days: 14
    mode: motion
  alerts:
    pre_capture: 20
    post_capture: 20
    retain:
      days: 30
      mode: motion
  detections:
    pre_capture: 10
    post_capture: 10
    retain:
      days: 30
      mode: motion

snapshots:
  enabled: true
  clean_copy: true
  timestamp: false
  retain:
    default: 30

go2rtc:
  streams:
    front_yard_camera:
      - ffmpeg:http://192.168.20.101/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_RTSP_USER}&password={FRIGATE_RTSP_PASSWORD}#video=copy#audio=copy#hardware=vaapi
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.20.101:554/Preview_01_main
  webrtc:
    candidates:
      - 192.168.20.2:8555
      - stun:8555

ffmpeg:
  hwaccel_args: preset-intel-qsv-h264
  input_args: preset-rtsp-restream

semantic_search:
  enabled: true
  reindex: false
  model_size: large

genai:
  enabled: true
  provider: ollama
  base_url: http://192.168.2.50:11434
  model: lukaspetrik/gemma3-tools:12b

cameras:
  front_yard_camera:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/front_yard_camera
          roles:
            - record
            - detect
    onvif:
      host: 192.168.20.101
      port: 8000
      user: '{FRIGATE_ONVIF_USER}'
      password: '{FRIGATE_ONVIF_PASSWORD}'
    live:
      streams:
        front_yard_camera: front_yard_camera
    detect:
      enabled: true
      width: 1440
      height: 1080
    motion:
      mask: 
        0,0.212,0.222,0.076,0.259,0.096,0.442,0.086,0.451,0.077,0.563,0.076,0.57,0.087,1,0.085,1,0,0.003,0.001
      threshold: 37
      contour_area: 12
      improve_contrast: true
      lightning_threshold: 0.9
    zones: {}
    objects:
      filters:
        person:
          mask: 0.582,0.058,0.357,0.06,0.352,0,0.584,0
        car:
          mask: 0.738,0.075,0.742,0,0,0,0,0.083
        motorcycle:
          mask: 0.567,0.061,0.568,0,0.222,0.001,0.223,0.07
    genai:
      enabled: true
tls:
  enabled: false

logger:
  default: warning

version: 0.16-0

detect:
  enabled: true
  fps: 5
  max_disappeared: 50

face_recognition:
  enabled: true
  min_area: 600
  model_size: large
  save_attempts: 200

lpr:
  enabled: true
  model_size: small
  recognition_threshold: 0.6
  enhancement: 0
  debug_save_plates: false
  device: GPU
  known_plates:
    Kia:
      - KT ?17[0O]-[DO0]F.?
    Hyundai:
      - KT ?6[8B]1-[B8]J.?

classification:
  bird:
    enabled: false

And here is my docker compose file:

services:
  frigate:
    container_name: frigate
    privileged: true
    restart: unless-stopped
    stop_grace_period: 30s
    image: ghcr.io/blakeblackshear/frigate:stable-tensorrt
    shm_size: "128mb"
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
      - /dev/dri/card0:/dev/dri/card0
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    cap_add:
      - CAP_PERFMON
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config:/config
      - /media/frigate:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "8971:8971"
      - "5000:5000"
      - "8554:8554"
      - "8555:8555/tcp"
      - "8555:8555/udp"
    environment:
      FRIGATE_RTSP_USER: RECACTED_USERNAME
      FRIGATE_RTSP_PASSWORD: REDACTED_PASSWORD
      FRIGATE_MQTT_USER: RECACTED_USERNAME
      FRIGATE_MQTT_PASSWORD: REDACTED_PASSWORD
      FRIGATE_ONVIF_USER: RECACTED_USERNAME
      FRIGATE_ONVIF_PASSWORD: REDACTED_PASSWORD
    secrets:
      - PLUS_API_KEY
secrets:
  PLUS_API_KEY:
    file: ./frigate_plus_key.txt

And sorry for nagging again... I've been forced to this unplaned, so I don't really know what to expect.

[inkeep-ai-assistant[bot]]

Based on your UHD 730 iGPU and the documentation, your stats are within expected ranges. Let me address your questions:

1. Are these stats expected for my iGPU?

Yes, your performance is consistent with documented benchmarks. For the Intel UHD 730, the YOLOv9 s-320 model shows inference times of approximately 20ms, and the s-640 model shows approximately 40ms(1). Your 15-20ms for 320x320 and ~50ms for 640x640 align with these expectations(1).

2. Is the 640x640 too large for my iGPU?

The 640x640 model is usable but pushes the limits of the UHD 730. The documentation shows s-640 models take approximately 40ms on this GPU(1). Your 50ms inference time with high detector CPU usage suggests the GPU is working hard but not falling back to CPU.

3. Is the "Detector CPU Usage" normal?

The "Detector CPU Usage" metric in Frigate actually represents the detector process's overall resource usage, not strictly CPU usage. When using OpenVINO with device: GPU, the detector process orchestrates GPU operations, which can show high usage even though the actual inference runs on the iGPU(2). Since your Proxmox host CPU stays around 10%, your GPU is doing the work correctly.

4. Will this be usable for multiple cameras?

For multiple cameras with the UHD 730, you should use the 320x320 model. The documentation notes that the UHD 730 can handle one detector instance effectively(1). With one camera at 320x320 showing 50-75% detector usage, you have headroom for additional cameras, though you may need to tune detect fps and motion settings to balance load.

5. Which GPU handles enrichments?

This is where your setup needs attention. You're using the -tensorrt image and passing through both the Intel iGPU and Nvidia GPU. According to the documentation:

• OpenVINO will automatically detect and use Intel GPUs for enrichments in the default Frigate image(3)
• Nvidia GPUs will automatically detect and use with enrichments in the -tensorrt Frigate image(3)
• TensorRT for object detection and OpenVINO for enrichments is not a supported combination(3)

Since you're using the -tensorrt image with an Nvidia GPU passed through, your enrichments (semantic search, face recognition, LPR) are likely attempting to use the Nvidia GPU via TensorRT, not the Intel iGPU(3).

Recommendations:

1. For your use case (OpenVINO detection + Nvidia enrichments), continue using the -tensorrt image as you are
2. Stick with the 320x320 model for better performance with the UHD 730(1)
3. Your Nvidia GPU should handle semantic search, face recognition, and LPR enrichments(3)(4)
4. Consider the default Frigate image instead if you want OpenVINO to handle both detection and enrichments on the Intel iGPU, though this would not utilize your Nvidia GPU(3)

Your current setup should work well: Intel iGPU for object detection via OpenVINO, and Nvidia GPU for enrichments via the tensorrt image.

📚 Sources:

1. Recommended hardware
2. Supported Hardware
3. Enrichments
4. License Plate Recognition (LPR)

---

Have another question? Just tag @inkeep.

[hawkeye217]

The AI is correct here.

I run a 12th gen i7 iGPU and I see similar numbers as you.

A 640x640 model won't run very well on an iGPU either, in my experience. And we've found that users are vastly overestimating the benefit of a 640x640 model. Bigger is not necessarily better. Frigate was specifically designed to compensate for using a smaller 320x320 model. Using a 640x640 model can actually make performance worse because it limits how far Frigate will zoom in on the image when running detection.

Also, Detector CPU Usage is the measurement of the CPU used to prepare inputs for the model for inference on GPU. Those values are normal, as well.

[zolakt]

Thanks for the quick reply. I'm glad it's in the expected range. Although I was expecting lower resource usage with all the "igpu is better than coral" talk. I know coral is running a much smaller model, but in terms of how many cameras it can run (based on these single camera stats) it doesn't look like the igpu can compete with the coral.

I'll also give it a try and run everything on the Nvidia GPU. It's much more powerful that the iGPU. Although there I'm worried that genAI might interfere too much. Ollama pegs the GPU to 100% when it does genAI, and that goes on in parellel with object detection, face recognition etc. Someone here suggested previously (Nick or even you) that might not be the best idea.

[NickM-27]

but in terms of how many cameras it can run (based on these single camera stats) it doesn't look like the igpu can compete with the coral.

You can run multiple detectors on an iGPU, it outclasses the coral in many ways

[zolakt]

You can run multiple detectors on an iGPU, it outclasses the coral in many way

Thanks for the reply. You guys truly have the fastest support there is :)

I was just assuming that if one camera utilizes 50-75% "Detector CPU Usage" there isn't much room for more. Maybe I'm just misinterpreting these stats. Luckily, I'll just need one more cam, so hopefully it won't be too much for the iGPU.

[hawkeye217]

In addition to what @NickM-27 said, the model that the Coral runs is a mobiledet model, which is a much less accurate model than what you can run on your iGPU or Nvidia card. It really isn't a 1-to-1 comparison.

Unless you are seeing skipped detections in your camera metrics, I would not worry about the numbers you're seeing at all.