[Detector Support]: onnx on tensorrt image v0.16 beta4

[levski]

Describe the problem you are having

saw beta4 come out.
was still running cpai for a detector but wanted to move to onnx on my nvidia gpu so i wanted to test the new release.

used the notebook from the documentation to create the yolo-nas model. tried the default setting and a custom with the 640x640.

Version

frigate:0.16.0-beta4-tensorrt

Frigate config file

/opt/frigate$ cat config/config.yaml 
mqtt:
  enabled: false

ffmpeg:
  hwaccel_args: preset-nvidia

detectors:
  onnx:
    type: onnx
    model:
      path: /config/model_cache/yolo_nas/yolo_nas_s.onnx
      model_type: yolonas
      width: 320
      height: 320

docker-compose file or Docker CLI command

/opt/frigate$ cat docker-compose.yml 
version: "3.9"
services:
  frigate:
    container_name: frigate
    image: ghcr.io/blakeblackshear/frigate:0.16.0-beta4-tensorrt
    restart: unless-stopped
    privileged: true
    shm_size: "2gb"
    runtime: nvidia
    deploy:    # <------------- Add this section
      resources:
        reservations:
          devices:
            - driver: nvidia
              device_ids: ['0'] # this is only needed when using multiple GPUs
              capabilities: [gpu]
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility,video
      - TZ=Asia/Manila
      - FRIGATE_RTSP_PORT=8554
      - YOLO_MODELS=yolov7-320,yolov7-640,yolov7x-320,yolov7x-640
      - USE_FP16=false

    volumes:
      - /opt/frigate/config:/config
      - /tank/frigate:/media/frigate
      - /opt/frigate/logs:/var/log/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 10g

    ports:
      - "8971:8971"         # Frigate Web UI
      - "8554:8554"         # RTSP output

Relevant Frigate log output

2025-07-14 12:42:41.147207559  [2025-07-14 20:42:41] frigate.app                    INFO    : go2rtc process pid: 120
2025-07-14 12:42:41.168567273  [2025-07-14 20:42:41] frigate.app                    INFO    : Embedding process started: 511
2025-07-14 12:42:41.315737382  Process detector:onnx:
2025-07-14 12:42:41.315742491  Traceback (most recent call last):
2025-07-14 12:42:41.315744595    File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
2025-07-14 12:42:41.315750631      self.run()
2025-07-14 12:42:41.315753257    File "/opt/frigate/frigate/util/process.py", line 41, in run_wrapper
2025-07-14 12:42:41.315754918      return run(*args, **kwargs)
2025-07-14 12:42:41.315773544             ^^^^^^^^^^^^^^^^^^^^
2025-07-14 12:42:41.315775576    File "/usr/lib/python3.11/multiprocessing/process.py", line 108, in run
2025-07-14 12:42:41.315777676      self._target(*self._args, **self._kwargs)
2025-07-14 12:42:41.315779572    File "/opt/frigate/frigate/object_detection/base.py", line 112, in run_detector
2025-07-14 12:42:41.315781429      object_detector = LocalObjectDetector(detector_config=detector_config)
2025-07-14 12:42:41.315783189                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-14 12:42:41.315804374    File "/opt/frigate/frigate/object_detection/base.py", line 57, in __init__
2025-07-14 12:42:41.315806360      self.detect_api = create_detector(detector_config)
2025-07-14 12:42:41.315823647                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-14 12:42:41.315825755    File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
2025-07-14 12:42:41.315827240      return api(detector_config)
2025-07-14 12:42:41.315828619             ^^^^^^^^^^^^^^^^^^^^
2025-07-14 12:42:41.315830455    File "/opt/frigate/frigate/detectors/plugins/onnx.py", line 53, in __init__
2025-07-14 12:42:41.315831960      self.model = ort.InferenceSession(
2025-07-14 12:42:41.315833351                   ^^^^^^^^^^^^^^^^^^^^^
2025-07-14 12:42:41.315835634    File "/usr/local/lib/python3.11/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 458, in __init__
2025-07-14 12:42:41.315852769      raise TypeError(f"Unable to load from type '{type(path_or_bytes)}'")
2025-07-14 12:42:41.315854678  TypeError: Unable to load from type '<class 'NoneType'>'
2025-07-14 12:42:41.321883086  [2025-07-14 20:42:41] detector.onnx                  INFO    : Starting detection process: 510
2025-07-14 12:42:41.323298983  [2025-07-14 20:42:41] frigate.detectors.plugins.onnx INFO    : ONNX: loaded onnxruntime module
2025-07-14 12:42:41.325586444  [2025-07-14 20:42:41] frigate.detectors.plugins.onnx INFO    : ONNX: loading None
2025-07-14 12:42:41.326739363  [2025-07-14 20:42:41] frigate.app                    INFO    : Output process started: 538
2025-07-14 12:42:41.409310672  [2025-07-14 20:42:41] frigate.app                    INFO    : Camera processor started for perimetercam01: 630
2025-07-14 12:42:41.417527142  [2025-07-14 20:42:41] frigate.app                    INFO    : Camera processor started for perimetercam02: 632

Install method

Docker Compose

Object Detector

TensorRT

Screenshots of the Frigate UI's System metrics pages

UI is dead

Any other information that may be helpful

other than that, there was no other error.

[NickM-27]

your config is invalid, model should not be indented

[levski]

yeah, was kinda confused with the two docs.
ok, made some changes.
that's using the default notebook setting S and 320x320

[NickM-27]

your config does not include all of the necessary fields. Please just copy directly from the docs https://deploy-preview-16390--frigate-docs.netlify.app/configuration/object_detectors#yolo-nas-1

[levski]

am sure i missed this because i don't have it.
where do i get this file? labelmap_path: /labelmap/coco-80.txt
i don't think that was part of the download from the notebook.

[NickM-27]

it is built in to the docker image

[levski]

and of course i made custom directory for the model so it didn't load until i figured that out too. smh.
appreciate the time to straighten this fool.