Presence Detection not working relibly

[jb123567]

Describe the problem you are having

I am trying to utilise the new presence detection to determine if my little one is in his cot. Frigate consistently identifies him with a score of between 75 and 80%. The bounding box appears to consistently report him, however in the home assistant device the detected object seems to vary between 0 and 1 inconsistently, with no apparent reason.

Im new to Frigate, so may be missing something!

Version

10.0.0

Frigate config file

{
  "birdseye": {
    "enabled": true,
    "height": 720,
    "mode": "objects",
    "quality": 8,
    "width": 1280
  },
  "cameras": {
    "nursery": {
      "best_image_timeout": 60,
      "detect": {
        "enabled": true,
        "fps": 1,
        "height": 270,
        "max_disappeared": 5,
        "stationary": {
          "interval": 0,
          "max_frames": {
            "default": null,
            "objects": {}
          },
          "threshold": 10
        },
        "width": 480
      },
      "ffmpeg": {
        "global_args": [
          "-hide_banner",
          "-loglevel",
          "warning"
        ],
        "hwaccel_args": [],
        "input_args": [
          "-avoid_negative_ts",
          "make_zero",
          "-fflags",
          "+genpts+discardcorrupt",
          "-rtsp_transport",
          "tcp",
          "-stimeout",
          "5000000",
          "-use_wallclock_as_timestamps",
          "1"
        ],
        "inputs": [
          {
            "global_args": [],
            "hwaccel_args": [],
            "input_args": [],
            "path": "rtsp:https://192.168.1.1:7447/xK9LDPNt3sUe7Jnu",
            "roles": [
              "record",
              "rtmp",
              "detect"
            ]
          }
        ],
        "output_args": {
          "detect": [
            "-f",
            "rawvideo",
            "-pix_fmt",
            "yuv420p"
          ],
          "record": [
            "-f",
            "segment",
            "-segment_time",
            "10",
            "-segment_format",
            "mp4",
            "-reset_timestamps",
            "1",
            "-strftime",
            "1",
            "-c",
            "copy",
            "-an"
          ],
          "rtmp": [
            "-c",
            "copy",
            "-f",
            "flv"
          ]
        }
      },
      "ffmpeg_cmds": [
        {
          "cmd": "ffmpeg -hide_banner -loglevel warning -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp:https://192.168.1.1:7447/xK9LDPNt3sUe7Jnu -c copy -f flv rtmp:https://127.0.0.1/live/nursery -r 1 -s 480x270 -f rawvideo -pix_fmt yuv420p pipe:",
          "roles": [
            "record",
            "rtmp",
            "detect"
          ]
        }
      ],
      "live": {
        "height": 720,
        "quality": 8
      },
      "motion": {
        "contour_area": 30,
        "delta_alpha": 0.2,
        "frame_alpha": 0.2,
        "frame_height": 50,
        "mask": "",
        "threshold": 25
      },
      "mqtt": {
        "bounding_box": true,
        "crop": true,
        "enabled": true,
        "height": 270,
        "quality": 70,
        "required_zones": [],
        "timestamp": true
      },
      "name": "nursery",
      "objects": {
        "filters": {
          "person": {
            "mask": null,
            "max_area": 24000000,
            "min_area": 0,
            "min_score": 0.5,
            "threshold": 0.7
          }
        },
        "mask": "",
        "track": [
          "person"
        ]
      },
      "record": {
        "enabled": false,
        "events": {
          "max_seconds": 300,
          "objects": null,
          "post_capture": 5,
          "pre_capture": 5,
          "required_zones": [],
          "retain": {
            "default": 10,
            "mode": "motion",
            "objects": {}
          }
        },
        "expire_interval": 60,
        "retain": {
          "days": 0,
          "mode": "all"
        },
        "retain_days": null
      },
      "rtmp": {
        "enabled": true
      },
      "snapshots": {
        "bounding_box": true,
        "clean_copy": true,
        "crop": false,
        "enabled": false,
        "height": null,
        "quality": 70,
        "required_zones": [],
        "retain": {
          "default": 10,
          "mode": "motion",
          "objects": {}
        },
        "timestamp": false
      },
      "timestamp_style": {
        "color": {
          "blue": 255,
          "green": 255,
          "red": 255
        },
        "effect": null,
        "format": "%m/%d/%Y %H:%M:%S",
        "position": "tl",
        "thickness": 2
      },
      "zones": {
        "cot": {
          "coordinates": [
            "180,1",
            "310,1",
            "360,270",
            "135,270"
          ],
          "filters": {
            "person": {
              "mask": null,
              "max_area": 24000000,
              "min_area": 0,
              "min_score": 0.4,
              "threshold": 0.5
            }
          },
          "objects": [
            "person"
          ]
        }
      }
    }
  },
  "database": {
    "path": "/media/frigate/frigate.db"
  },
  "detect": {
    "enabled": true,
    "fps": 5,
    "height": 720,
    "max_disappeared": null,
    "stationary": {
      "interval": 0,
      "max_frames": {
        "default": null,
        "objects": {}
      },
      "threshold": null
    },
    "width": 1280
  },
  "detectors": {
    "cpu": {
      "device": "usb",
      "num_threads": 3,
      "type": "cpu"
    }
  },
  "environment_vars": {},
  "ffmpeg": {
    "global_args": [
      "-hide_banner",
      "-loglevel",
      "warning"
    ],
    "hwaccel_args": [],
    "input_args": [
      "-avoid_negative_ts",
      "make_zero",
      "-fflags",
      "+genpts+discardcorrupt",
      "-rtsp_transport",
      "tcp",
      "-stimeout",
      "5000000",
      "-use_wallclock_as_timestamps",
      "1"
    ],
    "output_args": {
      "detect": [
        "-f",
        "rawvideo",
        "-pix_fmt",
        "yuv420p"
      ],
      "record": [
        "-f",
        "segment",
        "-segment_time",
        "10",
        "-segment_format",
        "mp4",
        "-reset_timestamps",
        "1",
        "-strftime",
        "1",
        "-c",
        "copy",
        "-an"
      ],
      "rtmp": [
        "-c",
        "copy",
        "-f",
        "flv"
      ]
    }
  },
  "live": {
    "height": 720,
    "quality": 8
  },
  "logger": {
    "default": "info",
    "logs": {}
  },
  "model": {
    "height": 320,
    "labelmap": {},
    "labelmap_path": null,
    "path": null,
    "width": 320
  },
  "motion": null,
  "mqtt": {
    "client_id": "frigate",
    "host": "192.168.1.14",
    "password": "mqtt567",
    "port": 1883,
    "stats_interval": 60,
    "tls_ca_certs": null,
    "tls_client_cert": null,
    "tls_client_key": null,
    "tls_insecure": null,
    "topic_prefix": "frigate",
    "user": "mqtt"
  },
  "objects": {
    "filters": null,
    "mask": "",
    "track": [
      "person"
    ]
  },
  "record": {
    "enabled": false,
    "events": {
      "max_seconds": 300,
      "objects": null,
      "post_capture": 5,
      "pre_capture": 5,
      "required_zones": [],
      "retain": {
        "default": 10,
        "mode": "motion",
        "objects": {}
      }
    },
    "expire_interval": 60,
    "retain": {
      "days": 0,
      "mode": "all"
    },
    "retain_days": null
  },
  "rtmp": {
    "enabled": true
  },
  "snapshots": {
    "bounding_box": true,
    "clean_copy": true,
    "crop": false,
    "enabled": false,
    "height": null,
    "quality": 70,
    "required_zones": [],
    "retain": {
      "default": 10,
      "mode": "motion",
      "objects": {}
    },
    "timestamp": false
  },
  "timestamp_style": {
    "color": {
      "blue": 255,
      "green": 255,
      "red": 255
    },
    "effect": null,
    "format": "%m/%d/%Y %H:%M:%S",
    "position": "tl",
    "thickness": 2
  }
}

Relevant log output

None

FFprobe output from your camera

N/A

Frigate stats

No response

Operating system

HassOS

Install method

HassOS Addon

Coral version

CPU (no coral)

Network connection

Wireless

Camera make and model

Unifi G3 Instant

Any other information that may be helpful

No response

[NickM-27]

@blakeblackshear This seems like it may be related to #2853 ? 🤔

@jb123567 Do you mind including some images with the zone defined? Also, are you looking at the "zone" sensor in HA or the camera sensor?

[jb123567]

@blakeblackshear This seems like it may be related to #2853 ? 🤔

@jb123567 Do you mind including some images with the zone defined? Also, are you looking at the "zone" sensor in HA or the camera sensor?

I am looking at the zone, defined at Cot, opposed to the whole camera. Here is the area.

[NickM-27]

@jb123567 Thanks, just for some insight can you please also post a screenshot of the Home Assistant graph showing the object going between 0 and 1 and back?

[jb123567]

@jb123567 Thanks, just for some insight can you please also post a screenshot of the Home Assistant graph showing the object going between 0 and 1 and back?

This is the past 4 hours.

[NickM-27]

Okay thanks, last question: Do you have frigate events enabled for this zone? If so do you see one long zone or does it get split into events like this graph from HA?

[jb123567]

Okay thanks, last question: Do you have frigate events enabled for this zone? If so do you see one long zone or does it get split into events like this graph from HA?

I don't, no, I was purely using for presence detection.

[blakeblackshear]

I need to try and reproduce this again.

[jb123567]

I need to try and reproduce this again.

Thank you. Is there anything you need from me?

[Clooos]

Hi, I'm "glad" to see that I'm not the only one, after a few days with this issue I can say in my case that this is very strange, I even bought a 1080p IP cam. When I sit in my kitchen the zone and the binary sensor are always triggered correctly and it's a solid presence detection, it's why I'm still using Frigate for my kitchen lights. It's only when I sit in my sofa (on that same camera) that I can see that a person is detected in the sofa zone on the debug view, but I got some random states on the binary sensor. I wish I could help you more and thank you again for all your work.

[angrycatmeowmeow]

I'm having a similar issue where once a person is detected, binary_sensor_person_motion_detection stays on and debug shows no boundary box for a detected person once they've left the FOV. I'm not using zones or anything besides dog, cat and person motion detection. This is only happening on one of my Reolink E1 Zoom cameras.

[NickM-27]

I'm having a similar issue where once a person is detected, binary_sensor_person_motion_detection stays on and debug shows no boundary box for a detected person once they've left the FOV. I'm not using zones or anything besides dog, cat and person motion detection. This is only happening on one of my Reolink E1 Zoom cameras.

Just curious, if you check the logs do you have any connection issues with the camera or is everything working as expected? I'm assuming the event ended at the appropriate time?

[NickM-27]

Also, I recently put a camera up inside my garage and have seen this same thing with it tracking my car. It seems to be stuck at '0' even though the car is still there (but then some time later jumps back to '1') I haven't been able to ascertain a pattern to reproduce to try and fix, this is a tricky one it seems 🤔

[angrycatmeowmeow]

I'm having a similar issue where once a person is detected, binary_sensor_person_motion_detection stays on and debug shows no boundary box for a detected person once they've left the FOV. I'm not using zones or anything besides dog, cat and person motion detection. This is only happening on one of my Reolink E1 Zoom cameras.

Just curious, if you check the logs do you have any connection issues with the camera or is everything working as expected? I'm assuming the event ended at the appropriate time?

I am actually having connection problems with one of them, but not the one in question. The one with connection issues works fine. The events start and end at the appropriate times but it seems like if I spend an extended period of time in the garage the sensor hangs even though Frigate reports no people.

[DrSpaldo]

I am getting the same thing. It isn't consistent though, I am getting a person detection (usually incorrect object) and then it says in HA as being still detected, even though the event in Frigate has finished. Any ideas?

#3415

[NickM-27]

I wonder if it has something to do with lower score objects

[jb123567]

As an update I'm still noticing some really odd behaviour. The events section of Frigate is identifying that a person is inside the area, with the detection showing "In Progress" but the Home Assistant integration is showing 0, am I missing something??

[NickM-27]

What does the HA sensor for the camera (not zone) show?

This could certainly be a valid state, as the person was in the cot zone but now the bottom center of their bounding box is outside the zone so the sensor shows 0

The event will continue "in progress" as long as the object is in view and it was at one point in the zone

[jb123567]

What does the HA sensor for the camera (not zone) show?

This could certainly be a valid state, as the person was in the cot zone but now the bottom center of their bounding box is outside the zone so the sensor shows 0

The event will continue "in progress" as long as the object is in view and it was at one point in the zone

It was also showing 0.

It definitely isn't a issue with the bounding box position, as you can imagine the cot is a very clearly defined area, and no matter how hard he tries, he is always in the zone. I'll monitor over the next few days.

[jb123567]

Issue was due to a change of resolution of the stream, but I am still noticing some odd behaviour with the Home Assistant not reflecting what is being reported by Frigate. As you can see, it seems to jump from 0 to 1 and back, despite Frigate reporting a constant detection over the same period.

[NickM-27]

That's expected even under normal circumstances. The sensor is a raw output meanwhile the events (by default) have a max_disappeared of 5 seconds.

This means that the object can be lost for 2 seconds and then detected again. The event will remain as a single event however the sensor will reflect those lost detections (and loss of presence in a zone).

[jb123567]

That's expected even under normal circumstances. The sensor is a raw output meanwhile the events (by default) have a max_disappeared of 5 seconds.

This means that the object can be lost for 2 seconds and then detected again. The event will remain as a single event however the sensor will reflect those lost detections (and loss of presence in a zone).

That makes sense, I'll just extend the required time to automations.

[DrSpaldo]

That's expected even under normal circumstances. The sensor is a raw output meanwhile the events (by default) have a max_disappeared of 5 seconds.

This means that the object can be lost for 2 seconds and then detected again. The event will remain as a single event however the sensor will reflect those lost detections (and loss of presence in a zone).

Is there a way to make the sensor reflect the same as the event?

[NickM-27]

Is there a way to make the sensor reflect the same as the event?

I'm sure there's an HA helper that can filter that out, but there's no frigate option. Would probably require a rewrite of the current logic to do it that way.

[jb123567]

Is there a way to make the sensor reflect the same as the event?

I'm sure there's an HA helper that can filter that out, but there's no frigate option. Would probably require a rewrite of the current logic to do it that way.

I've increased the max disappear to 20 seconds (100 frames) to avoid any unnecessary jumping between detected and not. I'm really just experimenting to see if I can reliably identify if my son is in the cot to change certain house features so not a huge deal.

Last night again HA didn't represent what Frigate reported. Frigate reported a detection between 0408 - 0538, yet HA showed 0 constantly between 0400 - 0455, am I missing something?

[DrSpaldo]

Last night again HA didn't represent what Frigate reported. Frigate reported a detection between 0408 - 0538, yet HA showed 0 constantly between 0400 - 0455, am I missing something?

What's the history graph for that time frame within HA show?

[jb123567]

Last night again HA didn't represent what Frigate reported. Frigate reported a detection between 0408 - 0538, yet HA showed 0 constantly between 0400 - 0455, am I missing something?

What's the history graph for that time frame within HA show?

Sorry, edited the original post with the screenshots!!

[DrSpaldo]

Try reducing the time in the history area to see how quickly those drops are going off for. I'd be guessing it is something along the lines of what @NickM-27 has previously mentioned where it is somewhat of an instant off then back on because Frigate has momentarily stopped detecting the motion and sent it through to HA, but not enough for the event to stop within Frigate itself.

[NickM-27]

To be clear what I was saying is max_disappeared affects how frigate works with events ending. There are no config options in frigate to adjust the HA sensor behavior. That needs to be done within HA

[bagobones]

Personally I think both the Occupancy and the count sensors should be filtered in frigate like the events #3990 . I don't find ether stable enough to do any automation with them on indoor objects..

If you watch debug view long enough on any occupied room you will find the detection jumps all over the place.

In HomeAssitant you can somewhat work around this with FOR based event triggers or a Template Sensor with a filter that will smooth out the output over a period of time.. This works better for the NUMBER output like the person count than it does for a binary on and off.