[Config Support]: Generative AI not automatically running but manual regenerate works #18351

illuzn · 2025-05-22T09:11:34Z

illuzn
May 22, 2025

Describe the problem you are having

I must be doing something wrong but I can't for the life of me get genai to automatically generate descriptions.

Manually generating descriptions through the semantic search dialog works fine (either from snapshot or thumbnails).

The only thing I can think is my slightly unusual setup where I use a lower quality substream for the detect role and the full quality main stream for recordings.

detect, record and snapshots are enabled for the relevant camera and the camera is enabled.

The record stream is a 1080P H265 video.
The detect stream is a 432x320 H265 video.

Version

0.15.1

Frigate config file

logger:
  default: info

detectors:
  ov:
    type: openvino
    device: CPU

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  path: /openvino-model/ssdlite_mobilenet_v2.xml
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

record:
  enabled: true
  sync_recordings: true
  retain:
    days: 7
    mode: all
  alerts:
    retain:
      days: 21
    pre_capture: 30
    post_capture: 60
  detections:
    retain:
      days: 21
    pre_capture: 30
    post_capture: 60
detect:
  annotation_offset: -500

snapshots:
  enabled: true
  clean_copy: true
  quality: 100
  retain:
    default: 30

go2rtc:
  streams:
    Front_Door:
      - rtsp://xxxx:[email protected]:554/ch02/0
    Front_Door_sub:
      - rtsp://xxxx:[email protected]:554/ch02/1

objects:
  track:
    - person
    - car
    - bicycle
    - motorcycle
    - bus
  filters:
    person:
      min_score: 0.6
      threshold: 0.85
    car:
      min_score: 0.6
      threshold: 0.8
    bicycle:
      min_score: 0.6
      threshold: 0.8
    motorcycle:
      min_score: 0.6
      threshold: 0.8
    bus:
      min_score: 0.6
      threshold: 0.8

ffmpeg:
  hwaccel_args: preset-vaapi
  input_args: preset-rtsp-restream
  output_args:
    record: preset-record-generic

cameras:
  Front_Door:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Front_Door
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Front_Door_sub
          roles:
            - detect
    detect:
      enabled: true
    record:
      enabled: true
    snapshots:
      enabled: true
    mqtt:
      enabled: true
      timestamp: false
      bounding_box: false
      crop: true
      quality: 100
      height: 320
    motion:
      mask: 0.681,0.948,0.682,0.999,0.956,0.998,0.954,0.947
      threshold: 20
      contour_area: 25
      improve_contrast: true
    genai:
      enabled: true

version: 0.15-1

semantic_search:
  enabled: true
  reindex: false
  model_size: small

genai:
  enabled: true
  provider: ollama
  base_url: http://frigate.lan:11434
  model: moondream

Relevant Frigate log output

With logger level at info no relevant outputs.

Relevant go2rtc log output

With logger level at info no relevant outputs.

Frigate stats

{"cameras":{"Garage":{"camera_fps":5.0,"process_fps":5.0,"skipped_fps":0.0,"detection_fps":0.0,"detection_enabled":true,"pid":627,"capture_pid":692,"ffmpeg_pid":697,"audio_rms":0.0,"audio_dBFS":0.0},"Front_Door":{"camera_fps":5.1,"process_fps":5.1,"skipped_fps":0.0,"detection_fps":0.0,"detection_enabled":true,"pid":635,"capture_pid":696,"ffmpeg_pid":707,"audio_rms":0.0,"audio_dBFS":0.0},"Front_Yard_Left":{"camera_fps":5.0,"process_fps":5.1,"skipped_fps":0.0,"detection_fps":1.7,"detection_enabled":true,"pid":643,"capture_pid":706,"ffmpeg_pid":715,"audio_rms":0.0,"audio_dBFS":0.0},"Front_Yard_Right":{"camera_fps":5.1,"process_fps":5.1,"skipped_fps":0.0,"detection_fps":1.7,"detection_enabled":true,"pid":651,"capture_pid":714,"ffmpeg_pid":738,"audio_rms":0.0,"audio_dBFS":0.0},"Courtyard":{"camera_fps":5.0,"process_fps":5.0,"skipped_fps":0.0,"detection_fps":0.0,"detection_enabled":false,"pid":659,"capture_pid":739,"ffmpeg_pid":750,"audio_rms":0.0,"audio_dBFS":0.0},"Backyard_Left":{"camera_fps":5.1,"process_fps":5.1,"skipped_fps":0.0,"detection_fps":0.0,"detection_enabled":false,"pid":667,"capture_pid":749,"ffmpeg_pid":764,"audio_rms":0.0,"audio_dBFS":0.0},"Side":{"camera_fps":5.0,"process_fps":5.0,"skipped_fps":0.0,"detection_fps":0.0,"detection_enabled":false,"pid":676,"capture_pid":756,"ffmpeg_pid":769,"audio_rms":0.0,"audio_dBFS":0.0},"Backyard_Right":{"camera_fps":5.0,"process_fps":5.0,"skipped_fps":0.0,"detection_fps":0.0,"detection_enabled":false,"pid":684,"capture_pid":768,"ffmpeg_pid":787,"audio_rms":0.0,"audio_dBFS":0.0}},"detectors":{"ov":{"inference_speed":4.55,"detection_start":0.0,"pid":579}},"detection_fps":3.4,"gpu_usages":{"amd-vaapi":{"gpu":"0.00%","mem":"3.57%"}},"cpu_usages":{"frigate.full_system":{"cpu":"4.2","mem":"25.9"},"1":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/package/admin/s6/command/s6-svscan -d4 -- /run/service"},"15":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise s6-linux-init-shutdownd"},"16":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B"},"24":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise frigate-log"},"25":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise s6rc-oneshot-runner"},"26":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise go2rtc-log"},"27":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise s6rc-fdholder"},"28":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise go2rtc"},"29":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise go2rtc-healthcheck"},"30":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise nginx-log"},"31":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise nginx"},"32":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise certsync"},"33":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise frigate"},"34":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise certsync-log"},"45":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --"},"46":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules"},"87":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx"},"88":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc"},"89":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate"},"90":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/certsync"},"96":{"cpu":"2.2","cpu_average":"2","mem":"0.1","cmdline":"/usr/local/go2rtc/bin/go2rtc -config=/dev/shm/go2rtc.yaml"},"105":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"bash ./run.user go2rtc-healthcheck"},"107":{"cpu":"2.4","cpu_average":"9","mem":"0.7","cmdline":"python3 -u -m frigate"},"126":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"nginx: master process nginx"},"173":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"nginx: worker process"},"174":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"nginx: worker process"},"175":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"nginx: worker process"},"177":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"nginx: worker process"},"193":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"nginx: cache manager process"},"339":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"bash ./run.user certsync"},"563":{"cpu":"0.6","cpu_average":"0","mem":"0.3","cmdline":"frigate.recording_manager"},"572":{"cpu":"0.6","cpu_average":"0","mem":"0.3","cmdline":"frigate.review_segment_manager"},"579":{"cpu":"27.2","cpu_average":"22","mem":"0.5","cmdline":"frigate.detector.ov  "},"581":{"cpu":"0.1","cpu_average":"0","mem":"2.3","cmdline":"frigate.embeddings_manager"},"602":{"cpu":"1.3","cpu_average":"1","mem":"0.3","cmdline":"frigate.output       "},"627":{"cpu":"0.6","cpu_average":"0","mem":"0.3","cmdline":"frigate.process:Garage"},"635":{"cpu":"0.6","cpu_average":"3","mem":"0.3","cmdline":"frigate.process:Front_Door"},"643":{"cpu":"7.2","cpu_average":"2","mem":"0.3","cmdline":"frigate.process:Front_Yard_Left"},"651":{"cpu":"6.0","cpu_average":"5","mem":"0.3","cmdline":"frigate.process:Front_Yard_Right"},"659":{"cpu":"0.5","cpu_average":"0","mem":"0.3","cmdline":"frigate.process:Courtyard"},"667":{"cpu":"0.5","cpu_average":"0","mem":"0.3","cmdline":"frigate.process:Backyard_Left"},"676":{"cpu":"0.6","cpu_average":"0","mem":"0.3","cmdline":"frigate.process:Side "},"684":{"cpu":"0.6","cpu_average":"0","mem":"0.3","cmdline":"frigate.process:Backyard_Right"},"692":{"cpu":"0.2","cpu_average":"0","mem":"0.3","cmdline":"frigate.capture:Garage"},"695":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 432x320 -i pipe: -threads 1 -f mpegts -s 972x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"696":{"cpu":"0.2","cpu_average":"0","mem":"0.3","cmdline":"frigate.capture:Front_Door"},"697":{"cpu":"0.4","cpu_average":"0","mem":"0.2","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Garage_sub -r 5 -vf fps=5,scale_vaapi=w=432:h=320,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"706":{"cpu":"0.1","cpu_average":"0","mem":"0.3","cmdline":"frigate.capture:Front_Yard_Left"},"707":{"cpu":"0.4","cpu_average":"0","mem":"0.2","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Front_Door_sub -r 5 -vf fps=5,scale_vaapi=w=432:h=320,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"710":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 432x320 -i pipe: -threads 1 -f mpegts -s 972x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"713":{"cpu":"0.2","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Garage -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/Garage@%Y%m%d%H%M%S%z.mp4"},"714":{"cpu":"0.2","cpu_average":"0","mem":"0.3","cmdline":"frigate.capture:Front_Yard_Right"},"715":{"cpu":"0.4","cpu_average":"0","mem":"0.2","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Front_Yard_Left_sub -r 5 -vf fps=5,scale_vaapi=w=432:h=320,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"726":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 432x320 -i pipe: -threads 1 -f mpegts -s 972x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"727":{"cpu":"0.3","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Front_Door -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/Front_Door@%Y%m%d%H%M%S%z.mp4"},"730":{"cpu":"0.2","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Front_Yard_Left -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/Front_Yard_Left@%Y%m%d%H%M%S%z.mp4"},"737":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 432x320 -i pipe: -threads 1 -f mpegts -s 972x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"738":{"cpu":"0.5","cpu_average":"0","mem":"0.2","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Front_Yard_Right_sub -r 5 -vf fps=5,scale_vaapi=w=432:h=320,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"739":{"cpu":"0.1","cpu_average":"0","mem":"0.3","cmdline":"frigate.capture:Courtyard"},"748":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 432x320 -i pipe: -threads 1 -f mpegts -s 972x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"749":{"cpu":"0.2","cpu_average":"0","mem":"0.3","cmdline":"frigate.capture:Backyard_Left"},"750":{"cpu":"0.4","cpu_average":"0","mem":"0.2","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Courtyard_sub -r 5 -vf fps=5,scale_vaapi=w=432:h=320,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"755":{"cpu":"0.2","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Front_Yard_Right -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/Front_Yard_Right@%Y%m%d%H%M%S%z.mp4"},"756":{"cpu":"0.2","cpu_average":"0","mem":"0.3","cmdline":"frigate.capture:Side "},"764":{"cpu":"0.4","cpu_average":"0","mem":"0.2","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Backyard_Left_sub -r 5 -vf fps=5,scale_vaapi=w=432:h=320,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"767":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 432x320 -i pipe: -threads 1 -f mpegts -s 972x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"768":{"cpu":"0.2","cpu_average":"0","mem":"0.3","cmdline":"frigate.capture:Backyard_Right"},"769":{"cpu":"0.4","cpu_average":"0","mem":"0.2","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Side_sub -r 5 -vf fps=5,scale_vaapi=w=432:h=320,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"787":{"cpu":"0.5","cpu_average":"0","mem":"0.2","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-6cb5cfb -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Backyard_Right_sub -r 5 -vf fps=5,scale_vaapi=w=432:h=320,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"788":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 432x320 -i pipe: -threads 1 -f mpegts -s 972x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"796":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 432x320 -i pipe: -threads 1 -f mpegts -s 972x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"805":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"16388":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"sleep 60"},"16748":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"sleep 30s"}},"service":{"uptime":1629,"version":"0.15.0-6cb5cfb","latest_version":"0.15.1","storage":{"/media/frigate/recordings":{"total":937796.7,"used":291035.7,"free":599051.6,"mount_type":"ext4"},"/media/frigate/clips":{"total":937796.7,"used":291035.7,"free":599051.6,"mount_type":"ext4"},"/tmp/cache":{"total":953.7,"used":5.5,"free":948.2,"mount_type":"tmpfs"},"/dev/shm":{"total":200.0,"used":63.1,"free":136.9,"mount_type":"tmpfs"}},"temperatures":{},"last_updated":1747904943},"processes":{"recording":{"pid":563},"review_segment":{"pid":572},"go2rtc":{"pid":96},"embeddings":{"pid":581}}}

Operating system

Debian

Install method

Docker Compose

docker-compose file or Docker CLI command

services:
  frigate:
    container_name: frigate
    restart: unless-stopped
    volumes:
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 1000000000
      - /drives/nvme/frigate/recordings:/media/frigate
      - /drives/nvme/frigate/config:/config
    devices:
      - /dev/dri
      - /dev/kfd
    shm_size: 200m
    environment:
      - FRIGATE_RTSP_PASSWORD=XXXX
      - HSA_OVERRIDE_GFX_VERSION=10.3.0
    image: ghcr.io/blakeblackshear/frigate:stable
    network_mode: host

Object Detector

OpenVino

Screenshots of the Frigate UI's System metrics pages

Note: the GPU usage spike is from me manually generating a gen ai description.

Any other information that may be helpful

No response

Answered by hawkeye217

Jun 1, 2025

When I was testing and writing code for GenAI, I saw this occasionally with the llava model (and others) when I was low on GPU memory and/or inference time was very high.

Since the requests are getting to Ollama, this isn't anything we can improve on the Frigate side.

View full answer

NickM-27 · 2025-05-22T11:06:59Z

NickM-27
May 22, 2025
Collaborator Sponsor

What is the request time for the regeneration request? Is it possible the original is timing out because it's taking longer than a minute?

1 reply

illuzn May 22, 2025
Author

Between 1-2 seconds - certainly nowhere longer than a minute.

hawkeye217 · 2025-05-22T11:08:37Z

hawkeye217
May 22, 2025
Collaborator

All Frigate does is pass requests to Ollama via its API. If this is working manually but isn't happening as tracked object lifecycles are ending, you may have Ollama misconfigured. You should watch your Ollama logs.

https://docs.frigate.video/configuration/genai#ollama

15 replies

illuzn May 27, 2025
Author

Yes.
Not sure if related but I sometimes get this in my ollama logs:

time=2025-05-27T16:07:29.224+09:30 level=WARN source=runner.go:128 msg="truncating input prompt" limit=4096 prompt=4440 keep=4 new=4096

My prompt is nowhere near 4KiB long.

hawkeye217 May 27, 2025
Collaborator

This is your issue. In your case, the prompt to your model may include the images sent to it - I'm not sure how moondream handles multimodal inputs. If that's what's happening, this is a model limitation, not an Ollama one. So the PR you opened is not actually correct.

Could you try another model? Perhaps llava?

illuzn Jun 1, 2025
Author

The reason I was using moondream was because llava was not working at all. Thanks to your help thus far, I realised it was due to parallel requests causing a timeout.

So now with llava:7b manual generation works perfectly. However, with automatic generation I'm getting a very strange bug.
Here is the automatically generated response to an event:

2023. A truck parked near a fence with a car in a driveway or garage, not driving on the road, behind a fence, but pulling into a parking lot, under a bridge over a fence and crossing it. In reality,

and here is the response when I manually generate it:

1. The first image shows a blue car parked in front of a pink car. It's possible that the driver of the blue car has just arrived at this location and parked their vehicle before moving on to explore or attend an event nearby. They may return shortly, so they left their car stationary for now.
2. The second image shows the blue car moving away from the pink car. This could indicate that the driver of the blue car has decided to leave this location, perhaps due to a change in plans or because they have finished whatever activity brought them here in the first place. As the car leaves, it might be preparing for its next destination or task.
3. The third image shows the blue car parked again at a different spot, possibly indicating that the driver has arrived at another location after leaving the previous one. They may have stopped to pick up passengers, attend an appointment, or run errands before continuing their journey.

I've tried sending multiple manual generation requests and they still work fine so it's not a parallel issue.

Have enabled verbose debugging in Ollama to try and further diagnose this issue.

hawkeye217 Jun 1, 2025
Collaborator

When I was testing and writing code for GenAI, I saw this occasionally with the llava model (and others) when I was low on GPU memory and/or inference time was very high.

Since the requests are getting to Ollama, this isn't anything we can improve on the Frigate side.

Answer selected by illuzn

illuzn Jun 26, 2025
Author

Turns out on my underpowered hardware the extra overhead of ollama just passes over the tipping point where this occurs for me.

I've built llama.cpp using HIP as the backend and hosting the exact same model/ parameters and this issue hasn't arisen again for the last week.

Using llama-server and the openai provider (with custom base url) works perfectly.

zolakt Sep 3, 2025

I have this exact same problem, also using moondream through ollama. And I can't figure out what to try, or how to debug.
I'm not underpowered. Moondream is the only model that I'm currently running on a 16GB 4060Ti.
Manual regenerating descriptions works almost instantly.
But automatic is a hit and miss. Sometimes it generates, but most of the time it doesn't.
Any suggestions?

hawkeye217 Sep 3, 2025
Collaborator

This is likely related to the context window and image support of your model. Regeneration only sends one single image, while the requests during the object lifecycle include multiple images. You should watch your Ollama logs.

zolakt Sep 3, 2025

But, when you regenerate from thumbnails, doesn't that send multiple images as well? And that works perfectly fine

Uh oh!

[Config Support]: Generative AI not automatically running but manual regenerate works #18351

Uh oh!

illuzn May 22, 2025

Describe the problem you are having

Version

Frigate config file

Relevant Frigate log output

Relevant go2rtc log output

Frigate stats

Operating system

Install method

docker-compose file or Docker CLI command

Object Detector

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

Replies: 2 comments · 16 replies

Uh oh!

NickM-27 May 22, 2025 Collaborator Sponsor

Uh oh!

illuzn May 22, 2025 Author

Uh oh!

hawkeye217 May 22, 2025 Collaborator

Uh oh!

illuzn May 27, 2025 Author

Uh oh!

hawkeye217 May 27, 2025 Collaborator

Uh oh!

illuzn Jun 1, 2025 Author

Uh oh!

hawkeye217 Jun 1, 2025 Collaborator

Uh oh!

illuzn Jun 26, 2025 Author

Uh oh!

zolakt Sep 3, 2025

Uh oh!

hawkeye217 Sep 3, 2025 Collaborator

Uh oh!

zolakt Sep 3, 2025

illuzn
May 22, 2025

Replies: 2 comments 16 replies

NickM-27
May 22, 2025
Collaborator Sponsor

illuzn May 22, 2025
Author

hawkeye217
May 22, 2025
Collaborator

illuzn May 27, 2025
Author

hawkeye217 May 27, 2025
Collaborator

illuzn Jun 1, 2025
Author

hawkeye217 Jun 1, 2025
Collaborator

illuzn Jun 26, 2025
Author

hawkeye217 Sep 3, 2025
Collaborator