[Detector Support]: Detection hardware utilization split between iGPU and dGPU

[drmrfancypants]

Describe the problem you are having

Running frigate in TRUENAS SCALE 25.10 on an old Dell Precision machine with a 1245v6 CPU with iGPU and an ARC PRO A40 GPU. I have "GPU" set under detector in the config file and I had good inference times. Then I switched to YOLOv9s 640x640 and my inference time skyrocketed from less than 10ms to about 70ms. I decided to check the following in 2 separate shell side by side so I could compare:

intel_gpu_top -d drm:/dev/dri/renderD128 (the iGPU)
intel_gpu_top -d drm:/dev/dri/renderD129 (the dGPU)

frigate.detecto is processing under both devices with minimal load on D129 but heavy intermittent load on D128
ffmpeg seems to be running on D129 only (which is what I want)

I guess I have more than 1 question related to this.

1. Is this a bug
2. Is there a config file command/code to put in the detection settings section to specify which GPU rather than just "GPU" or
3. Is the only way to fix this to convert frigate to a custom app and comment out the D128 device in the compose file
4. How much VRAM is needed to run all of the enhancements & YOLOv9s 640x640 (I'm not opposed to getting different hardware, currently have 5 cameras but will soon have 20)
5. Is the only way to see the more detailed GPU metrics in the metrics page to convert to custom app and add the code specified on the frigate config guide for GPU stats? (if so, perhaps this can be made a default change when selecting the GPU pass through option when installing the truenas app?)

Version

0.16.2

Frigate config file

mqtt:
  enabled: false

ffmpeg:
  hwaccel_args: preset-intel-qsv-h265


motion:
  contour_area: 30
  delta_alpha: 0.2
  frame_alpha: 0.01
  frame_height: 50
  lightning_threshold: 0.8

detectors:
  ov:
    type: openvino
    device: GPU

model:
  path: plus://75eafa2f3843347dd8e431c1763a0fb8
#  width: 300
#  height: 300
#  input_tensor: nhwc
#  input_pixel_format: bgr
#  path: /openvino-model/ssdlite_mobilenet_v2.xml
#  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

detect:
  width: 704
  height: 480
  fps: 5
  enabled: true
  max_disappeared: 5

record:
  enabled: true
  retain:
    days: 10
    mode: motion #Can be all, motion, or active_objects
  alerts:
    retain:
      days: 90
  detections:
    retain:
      days: 60

snapshots:
  enabled: true
  retain:
    default: 90 # Keep snapshots longer since they're tiny

objects:
  track:
    - person
    - face
    - car
    - dog
    - cat
    - amazon
    - fedex
    - ups
    - usps
    - package

cameras:
  barn_west:
    enabled: true
    ffmpeg:
      inputs:
        # Dahua - substream for detection only (704x480 H.265)
        - path: 
            rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
        # Dahua - main stream for recording only (3840x2160 H.265)
        - path: 
            rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
    detect:
      width: 704
      height: 480
      fps: 5

    motion:
      mask: 0.583,0.19,0.857,0.182,0.985,0.402,0.888,0.773,0.576,0.514
  barn_parking:
    enabled: true
    ffmpeg:
      inputs:
        # Uniview - substream for detection only (704x480 H.265)
        - path: 
            rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=2
          roles:
            - detect
        # Uniview - main stream for recording only (3840x2160 H.265)
        - path: 
            rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
    detect:
      width: 1280
      height: 720
      fps: 5

    motion:
      mask:
        - 0.372,0.626,0.46,0.995,0,1,0,0.597
        - 0.794,0.409,0.601,0.79,0.787,0.995,0.963,0.632,0.891,0.434
      threshold: 40
      contour_area: 30
      improve_contrast: true

  multipurpose_fisheye:
    enabled: true
    ffmpeg:
      inputs:
        # Hikvision - using main stream for detection and recording (no substream available)
        - path: 
            rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=2
          roles:
            - detect
        - path: 
            rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
    detect:
      width: 1280
      height: 720
      fps: 5

  barn_banquet:
    enabled: true
    ffmpeg:
      inputs:
        # Dahua - substream for detection only (704x480 H.265)
        - path: 
            rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
        # Dahua - main stream for recording only (3840x2160 H.265)
        - path: 
            rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
    detect:
      width: 704
      height: 480
      fps: 5

  barn_south:
    enabled: true
    ffmpeg:
      inputs:
        # Dahua - substream for detection only (704x480 H.265)
        - path: 
            rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
        # Dahua - main stream for recording only (3840x2160 H.265)
        - path: 
            rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
    detect:
      width: 704
      height: 480
      fps: 5

    motion:
      mask: 0.607,0.02,0.449,0.567,0.26,1,0.318,0.998,0.575,0.679,0.636,0.026
go2rtc:
  streams:
    barn_west: 
      rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
    barn_parking: 
      rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
    multipurpose_fisheye: 
      rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
    barn_banquet: 
      rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
    barn_south: 
      rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0

version: 0.16-0
semantic_search:
  enabled: true
  model_size: small
face_recognition:
  enabled: true
  model_size: small
lpr:
  enabled: true
classification:
  bird:
    enabled: true

docker-compose file or Docker CLI command

Unable to acquire (truenas app, unmodified)

Relevant Frigate log output

No errors

Install method

Docker Compose

Object Detector

OpenVino

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

I was using large size for the enrichments but changed to small to see if it made a difference, if it did it was maybe 3ms at most.

[hawkeye217]

The ability to specify a GPU for enrichments has been implemented for the next version of Frigate. You can offload ffmpeg decoding to another device by using manual ffmpeg arguments.

@inkeep can demonstrate

[drmrfancypants]

Ok so waiting for v0.17 I guess, not a big deal.

When you say for enrichments, does this mean that the enrichments are using the CPU because there is currently no way to set the GPU as default or that there will be a way to specify a specific GPU when a config has more than 1 available? I was under the impression that enrichments automatically used openvino in the default image when available.

If you are referring to specifying the particular GPU in a multi GPU config, will this functionality also extend to "detectors"? Asking because they are 2 different sections in the config file.

[NickM-27]

Actually, in 0.17 both multi-GPU enrichments and multi-GPU ffmpeg are supported.

Enrichments is on top of object detection, using multiple GPUs for object detection is already supported

[drmrfancypants]

Actually, in 0.17 both multi-GPU enrichments and multi-GPU ffmpeg are supported.

Enrichments is on top of object detection, using multiple GPUs for object detection is already supported

As it is already supported in 0.16 can you help me out with how to specify which GPU for detections, please. I asked the AI in the documentation guide, and I was unable to figure it out.

[NickM-27]

device: GPU:0 or device: GPU:1 for openvino detector

[drmrfancypants]

device: GPU:0 or device: GPU:1 for openvino detector

This worked. GPU:1 selected the dGPU and dropped my inference time to about 30ms. frigate.detecto and frigate.embeddi still show as processes under the iGPU but there is no activity. Looking forward to 0.17. Unless I missed it, this is not clearly explained in the documentation, might be something to clarify. Again, thank you!

[hawkeye217]

Great, glad to hear it's working. This is covered in the OpenVINO docs: https://docs.frigate.video/configuration/object_detectors/#openvino-detector

Specifically:

The OpenVINO device to be used is specified using the "device" attribute according to the naming conventions in the Device Documentation.

[inkeep-ai-assistant[bot]]

You can offload ffmpeg video decoding to a specific device in Frigate by setting manual ffmpeg arguments for each camera input. In your Frigate configuration, inside the ffmpeg section for a camera, you can use the input_args and hwaccel_args fields to set hardware acceleration and specify device paths.

For Intel hardware, for example, you can use the preset-intel-qsv-h264 or preset-vaapi presets for hardware acceleration(1)(2). To control which device is used, you can run Frigate in Docker and map only the desired device (such as /dev/dri/renderD129) into the container(1). You can also set custom ffmpeg arguments to specify the device node directly if needed.

Example for mapping a specific device in docker-compose:

services:
  frigate:
    ...
    devices:
      - /dev/dri/renderD129:/dev/dri/renderD129

(1)

And for manual ffmpeg arguments, you can use:

ffmpeg:
  inputs:
    - path: rtsp://...
      input_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format yuv420p
      roles:
        - detect

(3)

For more advanced setups (such as distributing load manually across multiple GPUs), this approach is supported, but you must be familiar with ffmpeg and hardware device management(4)(3).

For enrichments (semantic search, face recognition, etc.), the ability to specify which GPU is used will be supported in the next version of Frigate(5). Currently, the GPU used for enrichments is detected automatically based on the Frigate Docker image and hardware present(5).

📚 Sources:

1. Video Decoding
2. FFmpeg presets
3. Help understanding ffmpeg errors
4. Recommended GPU for Video Decoding with ~80 Cameras
5. Enrichments

---

Have another question? Just tag @inkeep.