[Detector Support]: Frigate Plus ONNX model on Jetson Orin Nano and Jetpack 6 is using cpu instead of gpu

[jshumaker]

Describe the problem you are having

Trying to use a new Jetson Orin Nano with Jetpack 6 to run 0.16 jp6 image and use the hardware for detection, but it seems to only use the cpu. Is there something wrong with my config, device, or otherwise? The logs mention some warnings about smallest subnormal for various numpy float values, but not sure if that's related.

Version

0.16.0-0b7a33d

Frigate config file

mqtt:
  enabled: false

auth:
  session_length: 864000 # 10 days
  refresh_time: 776000 # 9 days
  trusted_proxies:
    - 192.168.0.187/32
tls:
  enabled: false

go2rtc:
  streams:
    garage:
      - rtsp://.../cam/realmonitor?channel=1&subtype=0
  webrtc:
    candidates:
      - 192.168.0.187:8555
      - stun:8555

detectors:
  onnx_0:
    type: onnx
  onnx_1:
    type: onnx

model:
  path: plus://717c77b1c548a9f5371b44e1ec8466fd
  # path: plus://45e79bc69e9a5188669fcce0ff4e2c00 # 2025.2 640x640 onnx

ffmpeg:
  hwaccel_args: preset-jetson-h264

objects:
  track:
    - person
    - face
    - car
    - motorcycle
    - license_plate
    - bicycle
    - dog
    - cat
    - fox
    - deer
    - bird
    - raccoon
    - fox
    - squirrel
    - rabbit
    - ups
    - fedex
    - amazon
    - usps
    - dhl
    - package
    - umbrella

record:
  enabled: true
  retain:
    days: 1
    mode: all
live:
  height: 1080

detect:
  fps: 5
  max_disappeared: 300
  enabled: true

cameras:
  garage:
    snapshots:
      enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://.../cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
            - rtmp
    detect:
      width: 2592
      height: 1944
    mqtt:
      height: 512
      quality: 85
    zones:
      Property:
        coordinates: 
          0.05,0.204,0.176,0.159,0.338,0.145,0.517,0.136,0.686,0.142,0.765,0.161,0.82,0.181,0.925,0.26,0.992,0.295,1,1,0,1,0,0.224
        loitering_time: 0
        inertia: 3

camera_groups:
  Home:
    order: 1
    icon: LuHome
    cameras:
      - garage
version: 0.16-0

docker-compose file or Docker CLI command

services:
  frigate:
    container_name: frigate
    image: ghcr.io/blakeblackshear/frigate:0.16.0-rc1-tensorrt-jp6
    restart: unless-stopped
    privileged: true
    runtime: nvidia 
    environment:
      - PLUS_API_KEY=redacted
    shm_size: 512mb
    volumes:
      - ./config:/config
      - ./storage:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554"
      - "8555:8555"

Relevant Frigate log output

2025-08-01 13:43:56.679888479  [INFO] Preparing Frigate...
2025-08-01 13:43:57.996029256  [INFO] Starting Frigate...
2025-08-01 13:44:05.283677977  /usr/local/lib/python3.11/dist-packages/numpy/core/getlimits.py:500: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
2025-08-01 13:44:05.283694746    setattr(self, word, getattr(machar, word).flat[0])
2025-08-01 13:44:05.283699770  /usr/local/lib/python3.11/dist-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
2025-08-01 13:44:05.283702778    return self._float_to_str(self.smallest_subnormal)
2025-08-01 13:44:05.283891838  /usr/local/lib/python3.11/dist-packages/numpy/core/getlimits.py:500: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
2025-08-01 13:44:05.283899135    setattr(self, word, getattr(machar, word).flat[0])
2025-08-01 13:44:05.283938240  /usr/local/lib/python3.11/dist-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
2025-08-01 13:44:05.283940064    return self._float_to_str(self.smallest_subnormal)
2025-08-01 13:44:10.967817830  [2025-08-01 13:44:10] frigate.util.config            INFO    : Checking if frigate config needs migration...
2025-08-01 13:44:11.011870709  [2025-08-01 13:44:11] frigate.util.config            INFO    : frigate config does not need migration...
2025-08-01 13:44:12.304681400  [2025-08-01 13:44:12] frigate.app                    INFO    : Starting Frigate (0.16.0-0b7a33d)
2025-08-01 13:44:12.323936766  [2025-08-01 13:44:12] peewee_migrate.logs            INFO    : Starting migrations
2025-08-01 13:44:12.325316638  [2025-08-01 13:44:12] peewee_migrate.logs            INFO    : There is nothing to migrate
2025-08-01 13:44:12.386765704  [2025-08-01 13:44:12] frigate.app                    INFO    : Recording process started: 552
2025-08-01 13:44:12.409644644  [2025-08-01 13:44:12] frigate.app                    INFO    : Review process started: 565
2025-08-01 13:44:12.416487333  [2025-08-01 13:44:12] frigate.app                    INFO    : go2rtc process pid: 211
2025-08-01 13:44:12.464833338  [2025-08-01 13:44:12] detector.onnx_0                INFO    : Starting detection process: 577
2025-08-01 13:44:12.467449175  [2025-08-01 13:44:12] frigate.detectors.plugins.onnx INFO    : ONNX: loaded onnxruntime module
2025-08-01 13:44:12.467892802  [2025-08-01 13:44:12] frigate.detectors.plugins.onnx INFO    : ONNX: loading /config/model_cache/717c77b1c548a9f5371b44e1ec8466fd
2025-08-01 13:44:12.476889814  [2025-08-01 13:44:12] detector.onnx_1                INFO    : Starting detection process: 580
2025-08-01 13:44:12.489449278  [2025-08-01 13:44:12] frigate.detectors.plugins.onnx INFO    : ONNX: loaded onnxruntime module
2025-08-01 13:44:12.490446326  [2025-08-01 13:44:12] frigate.detectors.plugins.onnx INFO    : ONNX: loading /config/model_cache/717c77b1c548a9f5371b44e1ec8466fd
2025-08-01 13:44:12.538953422  [2025-08-01 13:44:12] frigate.app                    INFO    : Output process started: 606
2025-08-01 13:44:12.957411070  [2025-08-01 13:44:12] frigate.app                    INFO    : Camera processor started for garage: 629
2025-08-01 13:44:12.994091871  [2025-08-01 13:44:12] frigate.app                    INFO    : Capture process started for garage: 631
2025-08-01 13:44:13.485977875  [2025-08-01 13:44:13] frigate.detectors.plugins.onnx INFO    : ONNX: /config/model_cache/717c77b1c548a9f5371b44e1ec8466fd loaded
2025-08-01 13:44:13.485977875  [LOGGING] Last message repeated 1 times
2025-08-01 13:44:14.289094773  [2025-08-01 13:44:14] frigate.api.fastapi_app        INFO    : Starting FastAPI app
2025-08-01 13:44:14.571630622  [2025-08-01 13:44:14] frigate.api.fastapi_app        INFO    : FastAPI started

Install method

Docker Compose

Object Detector

TensorRT

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

the ffmpeg preset preset-jetson-h264 doesn't seem to be working which might be related
ffprobe output confiorming it is an h264 stream

[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"codec_long_name":"H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10","width":0,"height":0,"avg_frame_rate":"0/0"}]}}]

nvidia-smi output showing no active processes and driver details:

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 540.4.0                Driver Version: 540.4.0      CUDA Version: 12.6     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  Orin (nvgpu)                  N/A  | N/A              N/A |                  N/A |
| N/A   N/A  N/A               N/A /  N/A | Not Supported        |     N/A          N/A |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+

[NickM-27]

Are you running nvidia-smi inside of the docker container? This will not show any processes, based on the inference speed it does suggest that the CPU is not being used as it would likely be much slower

[NickM-27]

yeah, that may be too slow, 320x320 is generally the recommended resolution as it performs better in most scenarios

[jshumaker]

With 2 detectors, I'm getting 70ms inference times for 640x640. Is my math right that means, 350ms per second, with the 2 detectors, that this then could potentially handle ~5 cameras? with 3-4 being more comfortable?

[NickM-27]

70ms is ~ 15 detections per second, so you have 30 detections per second total (although it is clear that the inference speed fluctuates, it may get higher during peak activity). It is important to remember that Frigate runs detections on regions not the entire frame, so you may have multiple detections per frame, though with 640x640 this is less common, 5 cameras may be fine. You will just want to watch the skipped fps

[jshumaker]

When running 2 cameras the detection process seems to regularly freeze up, I've dropped it down to 320x320 as that seems far more stable on this device. Thanks again.

To summarize:
The jp6 image is properly using the GPU on the Jetson Orin Nano. nvidia-smi however is not appropriate on the device to check for usage. tegrastats is built in and a better option. jtop can be installed and is better, but currently does not support JetPack 6.2.1 and requires manual modification to run.

Jetson Orin Nano seems to maybe be able to do 640x640 detection for one camera, but unstable for 2. 320x320 works well for 2 cameras with 1 detector, 30-40ms inference time, with 2, 50-60ms, and 3 40-65ms. I am seeing some dropped detections varying on activity with a max of around 20-22fps before it starts dropping frames. Seems decent for 25W power running yolonas, but not quite as much as I had been hoping for.