[HW Accel Support]: "Memcpy nodes are added to the graph main_graph for CUDAExecutionProvider." when using large model for LPR and setting the device to GPU

[zolakt]

Describe the problem you are having

I have a Nvidia GPU, and I want to run LPR on it. I'm using a Frigate+ model. I've had it working with this config:

lpr:
  enabled: true
  model_size: large

However, I wasn't really sure if this was running on the GPU (it should according to docs), or if it was falling back to the CPU. So I've tried pointing it to the GPU explicitly, with this config:

lpr:
  enabled: true
  model_size: large
  device: GPU

As soon as I do that, I see this message in the logs:
[W:onnxruntime:, transformer_memcpy.cc:74 ApplyImpl] 2 Memcpy nodes are added to the graph main_graph for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.

Version

0.16.1-e664cb2

Frigate config file

mqtt:
  enabled: true
  host: 192.168.2.10
  user: '{FRIGATE_MQTT_USER}'
  password: '{FRIGATE_MQTT_PASSWORD}'

model:
  path: plus://REDACTED_PLUS_ID

detectors:
  coral:
    type: edgetpu
    device: pci

objects:
  track:
    - person
    - face
    - baby
    - car
    - license_plate
    - bicycle
    - motorcycle
    - dog
    - cat
    - bird
  filters:
    person:
      threshold: 0.70
    car:
      threshold: 0.90
    cat:
      min_score: 0.45
      threshold: 0.65
    license_plate:
      min_score: 0.4

review:
  alerts:
    labels:
      - person
      - car
      - bicycle
      - motorcycle
      - cat
      - dog

record:
  enabled: true
  sync_recordings: false
  retain:
    days: 14
    mode: motion
  alerts:
    pre_capture: 20
    post_capture: 20
    retain:
      days: 30
      mode: motion
  detections:
    pre_capture: 10
    post_capture: 10
    retain:
      days: 30
      mode: motion

snapshots:
  enabled: true
  clean_copy: true
  timestamp: false
  retain:
    default: 30

go2rtc:
  streams:
    front_yard_camera:
      - ffmpeg:http://192.168.20.101/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_RTSP_USER}&password={FRIGATE_RTSP_PASSWORD}#video=copy#audio=copy#audio=opus
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.20.101:554/Preview_01_main
  webrtc:
    candidates:
      - 192.168.20.2:8555
      - stun:8555

ffmpeg:
  hwaccel_args: preset-intel-qsv-h264
  input_args: preset-rtsp-restream

semantic_search:
  enabled: false
  reindex: false
  model_size: large

genai:
  enabled: true
  provider: ollama
  base_url: http://192.168.2.50:11434
  model: lukaspetrik/gemma3-tools:12b

cameras:
  front_yard_camera:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/front_yard_camera
          roles:
            - record
            - detect
    onvif:
      host: 192.168.20.101
      port: 8000
      user: '{FRIGATE_ONVIF_USER}'
      password: '{FRIGATE_ONVIF_PASSWORD}'
    live:
      streams:
        front_yard_camera: front_yard_camera
    detect:
      enabled: true
      width: 1440
      height: 1080
    motion:
      mask: 
        0,0.212,0.185,0.078,0.231,0.068,0.259,0.096,0.436,0.077,0.443,0.067,0.556,0.069,0.563,0.081,1,0.085,1,0,0.003,0.001
      threshold: 37
      contour_area: 12
      improve_contrast: true
    zones: {}
    objects:
      filters:
        person:
          mask: 0.568,0.041,0.356,0.039,0.352,0,0.57,0
        car:
          mask: 0.738,0.075,0.742,0,0,0,0,0.083
        motorcycle:
          mask: 0.567,0.061,0.568,0,0.222,0.001,0.223,0.07
    genai:
      enabled: true
tls:
  enabled: false

logger:
  default: warning

version: 0.16-0

detect:
  enabled: true
  fps: 5
  max_disappeared: 50

face_recognition:
  enabled: true
  min_area: 600
  model_size: large
  save_attempts: 200

lpr:
  enabled: true
  model_size: large
  recognition_threshold: 0.6
  enhancement: 0
  debug_save_plates: false
  device: GPU

classification:
  bird:
    enabled: false

docker-compose file or Docker CLI command

services:
  frigate:
    container_name: frigate
    privileged: true
    restart: unless-stopped
    stop_grace_period: 30s # allow enough time to shut down the various services
    image: ghcr.io/blakeblackshear/frigate:stable-tensorrt
    shm_size: "128mb"
    devices:
      - /dev/apex_0:/dev/apex_0
      - /dev/dri/renderD128:/dev/dri/renderD128
      - /dev/dri/card0:/dev/dri/card0
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    cap_add:
      - CAP_PERFMON
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config:/config
      - /media/frigate:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "8971:8971"
      - "5000:5000".
      - "8554:8554"
      - "8555:8555/tcp"
      - "8555:8555/udp"

    environment:
      FRIGATE_RTSP_USER: REDACTED_USERNAME
      FRIGATE_RTSP_PASSWORD: REDACTED_PASSWORD
      FRIGATE_MQTT_USER: REDACTED_USERNAME
      FRIGATE_MQTT_PASSWORD: REDACTED_PASSWORD
      FRIGATE_ONVIF_USER: REDACTED_USERNAME
      FRIGATE_ONVIF_PASSWORD: REDACTED_PASSWORD
    secrets:
      - PLUS_API_KEY
secrets:
  PLUS_API_KEY:
    file: ./frigate_plus_key.txt

Relevant Frigate log output

2025-10-14 10:19:35.231600242  [INFO] Preparing Frigate...
2025-10-14 10:19:35.520510489  [INFO] Starting Frigate...
2025-10-14 10:19:36.817231819  [2025-10-14 10:19:36] frigate.util.config            INFO    : Checking if frigate config needs migration...
2025-10-14 10:19:36.830189068  [2025-10-14 10:19:36] frigate.util.config            INFO    : frigate config does not need migration...
2025-10-14 10:19:36.855646253  [2025-10-14 10:19:36] frigate.config.config          WARNING : front_yard_camera is configured to track ['baby'] objects, which are not supported by the current model.
2025-10-14 10:19:36.889140591  INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
2025-10-14 10:19:37.247939113  �[0;93m2025-10-14 10:19:37.247839105 [W:onnxruntime:, transformer_memcpy.cc:74 ApplyImpl] 2 Memcpy nodes are added to the graph main_graph for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.�[m
2025-10-14 10:19:37.263934864  loading data from : /config/model_cache/facedet/landmarkdet.yaml

Relevant go2rtc log output

2025-10-14 10:19:35.473093079  [INFO] Preparing new go2rtc config...
2025-10-14 10:19:35.682898364  [INFO] Starting go2rtc...
2025-10-14 10:19:35.774520087  10:19:35.773 INF go2rtc platform=linux/amd64 revision=fa580c5 version=1.9.9
2025-10-14 10:19:35.774524580  10:19:35.773 INF config path=/dev/shm/go2rtc.yaml
2025-10-14 10:19:35.774526523  10:19:35.773 INF [rtsp] listen addr=:8554
2025-10-14 10:19:35.774528478  10:19:35.773 INF [api] listen addr=:1984
2025-10-14 10:19:35.774530075  10:19:35.774 INF [webrtc] listen addr=:8555
2025-10-14 10:19:45.181725922  [INFO] Starting go2rtc healthcheck service...

FFprobe output from your camera

[{"return_code":0,"stderr":"","stdout":{"programs":[],"stream_groups":[],"streams":[{"codec_long_name":"H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10","width":2560,"height":1920,"avg_frame_rate":"0/0"},{"codec_long_name":"AAC (Advanced Audio Coding)","avg_frame_rate":"0/0"}]}}]

Install method

Proxmox via Docker

Object Detector

Coral

Network connection

Wired

Camera make and model

Reolink E2 Outdoor

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[NickM-27]

This is not a problematic message

[zolakt]

Ok, thanks.

Btw. is it required to set device: GPU if I want it to run on the GPU? From this page I assumed it would force it to the GPU automatically. But there is obviously some differerence, since I'm not getting this message if I don't do device: GPU explicitly

[NickM-27]

For most of the enrichments all you need is the size to be large. But for LPR the large is only recommended for countries where the plates have multiple lines. Most users should run the small model, you can still specify the GPU device for the small model (but in many cases LPR runs just as fast on CPU)

[inkeep-ai-assistant[bot]]

The warning message you're seeing is expected behavior when explicitly setting device: GPU for LPR(1). This warning comes from ONNX Runtime's CUDA execution provider and indicates that some operations in the model cannot be optimized for GPU execution, requiring memory copy operations between CPU and GPU(1).

When using the large model size without explicitly setting the device, Frigate automatically detects and uses GPU acceleration if available(1). The documentation states that "using a GPU may increase performance of the models"(1) and that the device configuration "can be CPU or GPU"(1).

Your original configuration was likely already running on the GPU:

lpr:
  enabled: true
  model_size: large

(1)

The warning doesn't indicate a problem - it's just informing you that the model has some operations that can't be fully optimized for GPU execution. This is normal for many ONNX models when using CUDA. The LPR feature should still function correctly and benefit from GPU acceleration for the operations that can be accelerated.

If you want to avoid the warning message, you can remove the explicit device: GPU setting and let Frigate handle GPU detection automatically, or you can set the logging level as suggested in the warning message to suppress it.

📚 Sources:

1. License Plate Recognition (LPR)

---

Have another question? Just tag @inkeep.