Unconditional scaling 1920x1080 in ffmpeg cmdline

[warez4me]

Describe the problem you are having

Following the docs recommendations I prepared the input streams to have res 1920x1080 and f_rate 5fps for some sreams and 7fps for another ones. Scaling and fps changing were implemented by separate go2rtc/ffmpeg instance on standalone RPi. Values for fps were derived by integer division by 3 from 15 and by 2 from 14 source cameras fps. All this was done to reduce CPU load on Frigate RPi.
I didn't set these values anywhere in config.
But when Frigate was working under that config I've checked CPU state with htop command and saw that every ffmpeg cmdline contains "-r 5 -vf fps=5,scale=1920:1080" .
It can't be autodetected because of 7fps on some streams.
I decided that it's done because I haven't set values in config.
Then I set all these values for all streams in config and after that I got ffmpeg cmdlines exactly with these values for fps and scale. And what is strange that the CPU load became even higher for ~7%.
I don't understand the purpose to hardcode such a relatively CPU-heavy ops into cmdline.
I didn't saw in the docs any requirements for resolution and fps values - only the recommendations.
Or perhaps it's possible to remove that options from ffmpeg cmdline?

Version

0.16.1-e664cb2

Frigate config file

mqtt:
  enabled: true
  host: raspberrypi
  user: ha_mqtt
  password: MqTt_pass

#go2rtc:
#  streams: # 
#    Vyezd: rtsp://192.168.10.11:8554/8m_m

ffmpeg:
  hwaccel_args: -hwaccel drm

cameras:
  8m_m:
    name: 8m_m
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://rpig2r:8554/8m_m_264 # <----- The stream you want to use for detection
          roles:
            - detect
    detect:
      enabled: true # <---- disable detection until you have a working camera feed
      width: 1920
      height: 1080
      fps: 5
    zones:
      car_num:
        coordinates: 0.362,0.003,0.685,0.005,0.876,0.761,0.445,0.932
        loitering_time: 0
    motion:
      threshold: 30
      contour_area: 15
      improve_contrast: true

  v380_top:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://raspberrypi:8554/v380_m_264_top # <----- The stream you want to use for detection
          roles:
            - detect
    detect:
      enabled: false # <---- disable detection until you have a working camera feed
      width: 1920
      height: 1080
      fps: 7

  v380_btm:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://raspberrypi:8554/v380_m_264_btm # <----- The stream you want to use for detection
          roles:
            - detect
    detect:
      enabled: false # <---- disable detection until you have a working camera feed
      width: 1920
      height: 1080
      fps: 7

detect:
  enabled: true

detectors: # <---- add detectors
  coral:
    type: edgetpu
    device: pci

version: 0.16-0

Relevant Frigate log output

no any log output

Relevant go2rtc log output

no any output

Frigate stats

No response

Operating system

Debian

Install method

Docker Compose

docker-compose file or Docker CLI command

name: nvr

services:

#  go2rtc:
#    container_name: go2rtc
#    image: "alexxit/go2rtc"
#    restart: unless-stopped  # autorestart on fail or config change from WebUI
#    network_mode: host       # important for WebRTC, HomeKit, UDP cameras
#    privileged: true         # only for FFmpeg hardware transcoding
#    environment:
#      - TZ=Europe            # timezone in logs
#    volumes:
#      - "./go2rtc:/config"   # folder for go2rtc.yaml file (edit from WebUI)

  frigate:
    container_name: frigate
    image: "ghcr.io/blakeblackshear/frigate:stable"
    restart: unless-stopped
    stop_grace_period: 30s
    privileged: true         # ?? for FFmpeg hardware transcoding
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/nvr/frigate/config:/config
      - /opt/nvr/frigate/storage:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 384000000
    shm_size: 384000000
    ports:
      - "8971:8971"
      #- "8554:8554" # RTSP feeds
    devices:
    #  - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
      - /dev/apex_0:/dev/apex_0

Object Detector

Coral

Screenshots of the Frigate UI's System metrics pages

No response

Any other information that may be helpful

RPi 5 8Gb

[inkeep-ai-assistant[bot]]

Based on the configuration documentation, Frigate applies automatic scaling and frame rate adjustments when detect stream settings are specified (1).

When you configure detect settings like this:

detect:
  width: 1920
  height: 1080
  fps: 5

(1)

Frigate automatically adds scaling and frame rate conversion to ensure the detect stream matches these specifications (1). This is by design to optimize object detection performance.

The documentation shows that detect configuration includes optional width, height, and fps settings that default to using native stream resolution when not specified (1). When you do specify these values, Frigate ensures the detect stream conforms to them through ffmpeg processing.

The scaling operations you're seeing (-vf fps=5,scale=1920:1080) are necessary because:

1. Frigate needs consistent frame dimensions for object detection models (1)
2. The AI inference works more efficiently with standardized input resolutions (1)
3. Frame rate control helps manage CPU load during detection processing (1)

If you want to avoid this scaling overhead, you would need to ensure your input streams already match exactly the resolution and frame rate you want for detection, and potentially omit the detect width/height/fps settings to let Frigate use the native stream properties (1).

However, the documentation recommends a detect resolution and frame rate of 5 fps for optimal performance (2). The scaling operations, while CPU-intensive, are typically necessary for proper object detection functionality.

📚 Sources:

1. Full Reference Config
2. Getting Started

---

Have another question? Just tag @inkeep.

[NickM-27]

I don't understand the purpose to hardcode such a relatively CPU-heavy ops into cmdline.

They are not CPU-heavy in this case as they match what is already provided by the camera.

[warez4me]

@NickM-27
I'll do some additional experiments to be sure, but 1st impression is that CPU load increased for ~7%

[NickM-27]

compared to what?

[warez4me]

compared to situation when any values wasn't set in config.
And "-r 5 -vf fps=5.." didn't according to 2 of 3 stream's native fps

[NickM-27]

that part will always be included in the detect ffmpeg command, no matter what

[warez4me]

it is strange. Why to hardcode options that are don't needed.
"To be sure" isn't explanation
I think must be the possibility to exclude them from the cmdline

[NickM-27]

When you set a resolution in detect width/height Frigate does not check the cameras actual stream resolution. This is in order to be more efficient - checking every cameras resolution before starting up takes additional time and resources, when we already know the users desired resolution.

Sure, we could always check it and if the resolution matches exactly what the camera provides don't set the scale arg, but again this still adds extra startup time and work and there is no point - because when there is nothing to change ffmpeg efficiently does not do anything here.

And especially, in the cases that most users are in where they are using a GPU, it is needed anyway to use the GPU. So this would be a lot of additional config to maintain for a case that is not recommended anyway.

[warez4me]

@NickM-27
Sorry, the last statement is too complicated...

And what to say about this (I did shots with values and then after removing them)? It's not the 7% but 3-4% for sure + this warning about "high usage..."

---

• values are set

---

• all the values are removed

---

P.S. I've double checked these results..

[warez4me]

As You said the ffmpeg checks itself the current resolution and does nothing if it corresponds to scale values.
Then why to force ffmpeg to do check by setting unnecessary cmd option and respectively to do extra unneeded work and increase CPU load?

[NickM-27]

The only reason for additional cpu usage is because you have set the fps to 7 in your configuration. If you want to match the cpu usage then just change it back to 5 as I said previously.

[warez4me]

I set 7 in config because it is native resolution of the stream.
I explicitly talk to Frigate - here's a 7 fps. Take it and don't do any fps manipulations an rescaling.
That's all.
Why to hardcode that options into cmdline?

[NickM-27]

Yes, and processing 7 frames is more work than processing 5. But you are mistaken that the -vf is the part using cpu.

The cpu is required to convert the frames to yuv420 format. Doing this on 7 frames instead of 5 is more work. I've already told you if you set it to 5 it will reduce cpu usage.

If you don't want to do this then that's your choice, but that is what is leading to higher cpu usage. I've already explained why it works the way it does, those options don't add to the cpu usage.

[warez4me]

Ok. Agree with You.
Sorry, I forget about more frequent processing.
Uh-m-m... 2fps delta... IPcam has 14...
Ok. I'll remember about such possibility to decrease CPU load.
Thanks for patience.

[warez4me]

Just a few words for completeness.
Frigate's AI told me that default value for fps is shown in reference config. Yes it is. 5 fps.
But for default resolution there's some misunderstanding.
For the "default detection" resolution are using values from my camera's config.
But default values are 1280x720.

[NickM-27]

That is incorrect, that is just shown as an example

# Optional: width of the frame for the input with the detect role (default: use native stream resolution)
  width: 1280
  # Optional: height of the frame for the input with the detect role (default: use native stream resolution)
  height: 720

[warez4me]

Now trying to tune LPR for my car.
Is it critical for plate to fit 320x320 size or it will be recognized in any case?
I saw recordings - it seems that whole plate is wider - about 420 and above.
But I don't know the distance for that size.

It's all about the default resolution. Seems for LPR it's need to be more than 1920x1080..

[NickM-27]

the region will be resized to the models 320x320 size