Draft: 0.13 Motion Tuning Guide

[NickM-27]

I have a PR up for this guide that will eventually be in the docs, but we need to make sure that it is easy to follow. Feel free to make suggestions or ask questions if something is not clear.

---

id: motion_detection
title: Tuning Motion Detection

Tuning Motion Detection

Frigate uses motion detection as a first line check to see if there is anything happening in the frame worth checking with object detection.

Once motion is detected, it tries to group up nearby areas of motion together in hopes of identifying a rectangle in the image that will capture the area worth inspecting. These are the red "motion boxes" you see in the debug viewer.

The Goal

The default motion settings should work well for the majority of cameras, however there are cases where tuning motion detection can lead to better and more optimal results. Each camera has its own environment with different variables that affect motion, this means that the same motion settings will not fit all of your cameras.

Before tuning motion it is important to understand the goal. The optimal setup for motion detection would be that desired motion like people walking and cars driving but not detecting undesirable motion like grass moving, lighting changes, timestamps, etc.

Masking Out Known Motion Causes

The first step would be to mask out causes of motion that are known to not be objects. This includes things like timestamps, tree limbs, larger bushes, etc. More details can be found in the masks docs..

Prepare For Testing

The easiest way to tune motion detection is to do it live, have one window / screen open with the frigate debug view and motion boxes enabled with another window / screen open allowing for configuring the motion settings. It is recommended to use Home Assistant or MQTT as they offer live configuration of some motion settings meaning that Frigate does not need to be restarted when values are changed.

In Home Assistant the Improve Contrast, Contour Area, and Threshold configuration entities are disabled by default but can easily be enabled and used to tune live, otherwise MQTT can be used.

Tuning Motion Detection During The Day

Now that things are set up, find a time to tune that represents normal circumstances. For example: If you tune your motion on a day that is sunny and windy you may find later that the motion settings are not sensitive enough on a cloudy and still day.

:::note

It is very unlikely to find motion detection settings that only detect desired motion and never detect undesired motion. Realistically, the goal is to find a balance that detects the desired motion without too much undesired motion being detected.

:::

Threshold

The threshold value dictates how much of a change in a pixels luminance is required to be considered motion.

# default threshold value
motion:
  # Optional: The threshold passed to cv2.threshold to determine if a pixel is different enough to be counted as motion. (default: shown below)
  # Increasing this value will make motion detection less sensitive and decreasing it will make motion detection more sensitive.
  # The value should be between 1 and 255.
  threshold: 30

Lower values mean motion detection is more sensitive to changes in color, making it more likely for example to detect motion when a brown dogs blends in with a brown fence or a person wearing a red shirt blends in with a red car. If the threshold is too low however, it may detect things like grass blowing in the wind, shadows, etc. to be detected as motion.

Watching the motion boxes in the debug view, adjust the threshold until the undesired motion is not detected. Once this is done, it is important to test and ensure that desired motion is still detected.

Contour Area

# default contour_area value
motion:
  # Optional: Minimum size in pixels in the resized motion image that counts as motion (default: shown below)
  # Increasing this value will prevent smaller areas of motion from being detected. Decreasing will
  # make motion detection more sensitive to smaller moving objects.
  # As a rule of thumb:
  #  - 10 - high sensitivity
  #  - 30 - medium sensitivity
  #  - 50 - low sensitivity
  contour_area: 10

Once the threshold calculation is run, the pixels that have changed are grouped together. The contour area value is used to decide which groups of changed pixels qualify as motion. Smaller values are more sensitive meaning people that are far away, small animals, etc. are more likely to be detected as motion, but it also means that small changes in shadows, leaves, etc. are detected as motion. Higher values are less sensitive meaning these things won't be detected as motion but with the risk that desired motion won't be detected until closer to the camera.

Watching the motion boxes in the debug view, adjust the contour area until there are no motion boxes smaller than the smallest you'd expect frigate to detect something moving.

Improve Contrast

At this point if motion is working as desired there is no reason to continue with tuning for the day. If you were unable to find a balance between desired and undesired motion being detected, you can try disabling improve contrast and going back to the threshold and contour area steps.

Tuning Motion Detection During The Night

Once daytime motion detection is tuned, there is a chance that the settings will work well for motion detection during the night as well. If this is the case then the preferred settings can be written to the config file and left alone.

However, if the preferred day settings do not work well at night it is recommended to use HomeAssistant or some other solution to automate changing the settings. That way completely separate sets of motion settings can be used for optimal day and night motion detection.

Tuning For Large Changes In Motion

# default lightning_threshold:
motion:
  # Optional: The percentage of the image used to detect lightning or other substantial changes where motion detection
  #           needs to recalibrate. (default: shown below)
  # Increasing this value will make motion detection more likely to consider lightning or ir mode changes as valid motion.
  # Decreasing this value will make motion detection more likely to ignore large amounts of motion such as a person approaching
  # a doorbell camera.
  lightning_threshold: 0.8

Larges changes in motion like PTZ moves and camera switches between Color and IR mode should result in no motion detection. This is done via the lightning_threshold configuration. It is defined as the percentage of the image used to detect lightning or other substantial changes where motion detection needs to recalibrate. Increasing this value will make motion detection more likely to consider lightning or IR mode changes as valid motion. Decreasing this value will make motion detection more likely to ignore large amounts of motion such as a person approaching a doorbell camera.

[dz4news]

Overall I think it reads well and is understandable. I for one, just learned alot about tuning, since I was using "improve contrast" for better detection.

My suggestion...

Watching the motion boxes in the debug view, adjust the threshold until the undesired motion is not detected but be sure to also check with desired motion to ensure it is still detected.

I understand what the point you are making with this sentence, but I would reword the "but be sure to also..." that part seems to run on and you have to read it several times to get the point. It may be as simple as adding a comma.

The only other suggestion I have is that you make reference to using HA several times for tuning... maybe point to some reference as a guide on how to do this or an example.. I know this is not meant to be an HA guide, so if that is out of the scope of the document... then disregard that suggestion.

[NickM-27]

Thanks, I have updated the wording.

The only other suggestion I have is that you make reference to using HA several times for tuning... maybe point to some reference as a guide on how to do this or an example.. I know this is not meant to be an HA guide, so if that is out of the scope of the document... then disregard that suggestion.

yeah that is out of scope 👍

[kirsch33]

i like this guide, although when reading I had it open on one screen with my config on the other, editing as I go. i kept finding myself having to also open up the config file reference page on the documentation to compare the default values to what I'm editing, so maybe adding the default values to this guide would be helpful

[NickM-27]

thanks, added that to the guide

[benklop]

Just a nit, the phrase "Now that things are setup" should be "now that things are set up"

[vajonam]

somewhat related to tuning contour in HA the contour area slider only goes to a min of 15, is this by design, or changed in the beta? I haven't updated to the beta yet.

[NickM-27]

Oh, yeah that needs to be fixed in the integration

[vajonam]

should I open an issue for that? what is the minimum? 10 as per the default?

[NickM-27]

I just put up a PR, made the minimum 1

[OnFreund]

Really helpful guide. Shouldn't frame_height also be tweaked along with contour_area?

[NickM-27]

Right, like the guide says it's never going to be perfect, those are very difficult conditions.

[OnFreund]

I guess the main question is what's the impact of these undesired motions. The options I see are:

1. False positive detections - I can try to handle those in other ways
2. Increased TPU usage - I don't mind that too much since I'm far from capacity
3. Increased CPU usage - that could be concerning but I'm not sure that's true (?)

[NickM-27]

1. Potentially, that can really depend though on each camera / model. I haven't had false positive issues with the tree shadows for a while now.
2. Marginally, should result in 1 or 2 detections per second depending how spread out the motion is
3. Yes, it will use more during this acidity, but that's expected. And the new motion detection approach uses less cpu overall due to fewer operations on the image

[OnFreund]

Thanks. RE 2, that could be 1 or 2 per object type, right? Over several cameras, some of which are configured to detect several objects, that could be a meaningful increase in detection FPS.

[NickM-27]

No that is incorrect, one inference is done for all object types and returns any that are detected.

[NathanVanCleave]

Might be more relevant for tuning masks, but it could be worth adding a note about how to test on a specific clip using input_args: -re -stream_loop -1 -fflags +genpts.

I've used this when trying to understand false positives and missed events since motion detection can play a large role in that.

[kirsch33]

i didnt know this was an option! good advice. much better than making some changes and hoping my neighbors are walking by shortly after or waiting a full day between tweaks to see how it does.

[OnFreund]

Can you elaborate on that?

[NickM-27]

You can use an event clip as a camera to make changes https://deploy-preview-6262--frigate-docs.netlify.app/development/contributing#2-modify-your-local-config-file-for-testing

The exact same thing won't happen each time, and it is crucial when doing it that you have the exact same config for the test camera, otherwise results will not be even close to the same.

[kirsch33]

i think it would be helpful to add some info on frame_height

for example, my 2 main cameras are soffit mounted (so ~28ft elevated from street level) and look down the entire road, and have a fairly wide FOV with a 2.8mm lens, so while the general advice is that a high detect resolution isnt needed, in some cases i think its warranted.

with the default 1280x720 res, a person on the other side of my street is very very small in a 320x320 frame, so im currently trying out a detect resolution of 2304x1296 and frame_height of 150 to compensate.

[NickM-27]

I think it's difficult, there are only some specific instances where an increased frame height is needed like you mentioned, and in my testing I've found that it can lead to increased motion detection even with larger contour area, I fear that users would change this when in most cases it isn't needed.

Maybe worth having a link to the full config reference to explain there are other options that are available as well.

[kirsch33]

not sure if you're looking to add some example data to the guide as a reference, but after over a week of testing ive settled in on these settings for my very high detect resolution of 2304x1296. also attached a pic of each cam which tells the story of why this high resolution works better for my case (you can see how far away things are with it being soffit mounted)

motion:
  threshold: 50
  contour_area: 55
  frame_height: 150

[NickM-27]

Thanks for sharing. With the goal of brevity I don't know that it will make sense to include in the guide itself, but it might be cool to provide links to different users configurations at the end so people can get some references

[ljmerza]

@kirsch33 how small of objects are you able to detect with those settings?

[kirsch33]

@ljmerza i cant really say the full capabilities of this setup since I have Frigate configured to only create events for certain zones which are fairly close to the house, but heres a pretty good one of my dog. area of ~6000, and for reference camera is ~28ft above ground and ~45ft away in this instance, so straight line distance of about 55ft from camera directly to object

[ljmerza]

i believe the issue with changing values in home assistant is if you go and change the yaml in frigate then save+restart then all your changes from home assistant are gone. i would expect when i change settings in home assistant they are persisted

[ljmerza]

oklay this can all be setup although i worry about the increased dependency of home assistant since not everyone running frigate may want home assistant as well

[NickM-27]

I mean it can be done with anything that can send MQTT / websockets including HA, NodeRED, etc. It could even be done locally with a docker dependency in docker compose.

I've personally not had to have any of my cameras switch motion settings after 0.13, just tuning the camera's night mode settings was good enough with the new motion detector

[NathanVanCleave]

I just have an automation that runs when frigate restarts

Are you using sensor.frigate_status to trigger this?

[NickM-27]

No I am using the state of the buttons going from unavailable to any other state

[NathanVanCleave]

Makes perfect sense. Not sure why I hadn't thought of that before.

[mdisabato]

I have gone through this, the documentation, and searched the forum. The only question not answered is "Should motion detection be done in the camera, or is that best shut off and let Frigate take care of it all?". This may be one of those "intuitively obvious" things, but clarity is always good. :)

[NickM-27]

the cameras motion detection is not used by frigate, it does not matter how the camera is configured for motion detection

[mdisabato]

Thank you! That will simplify things and likely take a few microseconds of load of the camera.