-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Objects detection at night #1511
Comments
I second this, I am having trouble triggering my flood light. interestingly, I am getting clips for the required events but its not triggering motion detected. |
maybe the AI-model has to be trained (additionally) on black/white images ....? |
I got it (kinda) working by changing the detect stream to low res. Still won't trigger for my fiance leaving in the wee hours for work. but now catches me coming home from the bar 😆 |
There is a trained DeepStack model for night and dark environments, but I don't know if this can be integrated in frigate. |
this has been a well known issue since day 1. Using Coral TPU you are pretty much limited on models you can run, one next step would be to use a different hardware which allows better models. Lower resolution images give worse result. So, better model, HD resultion, good quality EXIR Hikvision cams, new hardware for example nvidia jetson xavier nx would solve this of course |
I don't understand how you're limited. You can create your own models can't you? |
can you run YOLO v3 on TPU? No. As i know you can not, just the tiny YOLO v3. All i heard is very limited compared to a GPU or device like nvidia jetson xavier nx. I would prefer low power consumption device like Jetson but only if the detection is not terrible bad with it. |
We have the same here, The cat was not detected at night. Even lowering the score and threshold has not helped. The same Hikvision cam I use has detected the motion an added a bounding box and noticed the Line crossing and Zone entering cinfigured in the cam |
@hwinkel Which Hikvision model do you have? |
DS-2CD2443G0-IW, it detected the motion at line crossing and entering zone. it detected not "a cat" |
same here, cat not always gets detected even at daylight, not too high prio for me, but the night time person detection should be improved somehow |
Indeed. That's a critical issue for me. |
usually the embedded Deep Learning in for example Hikvision NVRs are not too good, they use some entry level chips like you have in Jetson Nano and limited dataset. So, opensource projects like Frigate still gives better result. Coral is a must for frigate. Reolink cams have a downside, they are not waterproof, the water goes in via the backside hole where the cables are. Reolink is junk compared to Hikvision EXIR bullets, worth the double price |
Thanks for the suggestions @belmont ! |
I have same trouble - at night not detect movement of any objects: person, cat, dog. |
until there is no improvement on the Deep Learning side, we would need the good old ways, like line crossing detection. Yep, this will also give back the good old false alarms but still better 10 false alarm than a unnoticed break in event. |
In my case, it would be enough to receive a mqtt signal about a simple movement. |
Did anyone try retraining the ssdlite_mobiledet with the COCO dataset converted to grayscale? I downloaded the COCO 2017 dataset and converted the images to grayscale now have to figure out how to retrain the existing model. Only info I could get is from here: https://colab.research.google.com/github/google-coral/tutorials/blob/master/retrain_ssdlite_mobiledet_qat_tf1.ipynb Now how to do it on the COCO dataset and not the Oxford Pets one. Edit: So I think I got it to run. It is doing something anyway. |
I have three cameras and one seems to detect ok at night while the other two appear to be less. So i'm trying to compare the differences to see what it might be. Only big thing that stands out is the fov is much narrower on my better night cam, so it could be it just has more resolution on people, but i feel like that is offset by the fact that it is lower resolution overall than the other cams: good night cam: xiaomi mi cam, 1080 resolution, not wide angle (don't know exact fov), night color is black, detect uses sub stream, 640x386 other cams |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
Resurrecting this as I seem to have the same problem. While in debug view for my camera, I noticed the main issue seems that no motion is detected at all (hence no object recognition is performed either). If I walk with a laptop in my hand, which generates a bright light source, motion is more easily detected, and then my person is reliably identified. Resolution doesn't seem to matter much: I set up two cameras in frigate: one using the main physical camera stream (4K) for detection; the other using the sub camera stream (640x320). Then I walked through the yard with both camera views in debug mode. There was hardly any difference in the motion and object recognition results. Maybe one should be able to define two motion sensing thresholds, for daylight and IR. Ideally, frigate should be able to detect a daylight image vs IR, and apply the different settings accordingly, and different defaults could be provided. Thoughts? |
@alfredopironti I have been actively working on this. If you can provide an example mp4 clip with low contrast IR for testing, that would be incredibly helpful. |
Hi @blakeblackshear That's great news! I should be able to provide some useful footage in the second half of next week. What would be the best way to transfer that privately? |
@alfredopironti you can upload to Google drive and share the folder with [email protected] |
Thanks @blakeblackshear I've now shared a few minutes of footage where a person is moving, both walking slowly and running (but not sprinting) -- I appreciate the motion algorithm can be tuned for both slow and fast movements. I suspect the low-contrast you mentioned is key: when the bright laptop screen gets in the scene, it gets quickly picked up by the motion algorithm; otherwise, everything else isn't picked as motion at all. Btw, the camera has some 3D noise reduction enabled. This seems especially apparent with the footage of the person running, which appears quite blurred in the recording. The recordings are in high-quality, but as anticipated I was getting the same results with the low-quality (640x320) stream for detection. Please let me know if I can support with anything further at this stage. And thanks for all the hard work and great development! |
@alfredopironti Already tested it, and it very clearly gets detected with the updates coming in 0.10.0. I attached a small video clip of the internal motion detection output. It combines 4 different stages of processing into a single video, but the bottom right is clearly being identified as motion with the red bounding box. I expect to have a beta available in the next few days. out.mp4 |
Thanks @blakeblackshear ! It indeed looks impressive! Looking forward to the update then! |
Hi, I'm still having this issue in 0.10.1. Have shared an example video with you @blakeblackshear. |
Have you enabled |
I hadn't, but have now. Will report back. Thanks! |
Still no luck last night with a couple of cats running around and improve_contrast enabled. Wonder if there's a way to boost contrast/sensitivity when greyscale is detected? LMK if it would help to share some example clips. |
That's what improve_contrast is doing. It works with the brightness channel of the image to increase the contrast. Motion detection is only done on the brightness channel. An example clip & your config may be helpful to see what is going on in your particular case |
Thanks! I've shared a couple of clips. I have HA set up to turn on the garden lights when detection fps > 0. You'll see that in both clips the cat doesn't trigger any detection fps, but in one clip a flying insect flies in front of the camera and does trigger detection (and the garden lights turn on after quite a bit of (likely Nest-stream-caused) delay). This could perhaps be something to do with the small roof section in the bottom left being so much brighter than the rest of the picture (because it's much closer and so reflecting more IR)? Here's my frigate.yml:
Note in the above that I've masked off areas of the garden with plants/flowers to avoid motion triggering when they move in the wind. The lawn is the main unmasked area. Appreciate the help! |
@no1knows Thanks, I played around with it for a bit and by setting Unfortunately, I think that the grayscale images of animals just don't work super well with this default google model. I put the same config up with a couple of my own cameras clips. The grayscale ones worked a bit better since my backyard camera is closer to the ground, but when I used a day time video it got it very well. I even set the scores required to 0 and it wasn't finding the cat in your case so I believe it is just an issue with the default model. I am not sure there is much that can be done in this case besides uploading examples to frigate+ and testing that once it is out as it should be much better than the google model (which only has like 1000 sample images for cat / dog) |
@NickM-27 thank you! I'll give the threshold setting a try and start uploading some examples to frigate+. |
grayscale or night vision is look hard to detect (and my mom want that haha) |
Given that we now have a support for beefier hw (tensorrt), is there anything that could be done to improve the current situation? Maybe switching from yolo tiny to the full one? |
My detect config is set for main stream on 2560x1900 10fps but the night detection seems it can be better. Here's an example with no detection whatsoever. (cameras Reolink 520A) motion_clip_camera5.mp4my config is this:
|
@alez007 probabably because you don't have improve_contrast enabled |
thanks, will enable it and try again |
I've been testing Frigate against Blue Iris for a couple months now and I also am having trouble detecting motion at night. Here's my setup: I'm wondering if there's anything else I can do to try and get the night time detection working. My full config
|
@ShortPutt you are going to want to enable
also, frigate only looks for objects after motion is detected so if objects are detected then motion has been detected as well. |
you can see if frigate is detecting motion live by using the debug live view |
interesting, not sure how it would not be detecting motion at night in that case. In the upcoming frigate 0.13 motion detection has been rewritten so this likely won't be a problem there. You can run motion detection on saved clips https://docs.frigate.video/development/contributing#2-modify-your-local-config-file-for-testing |
Thanks Nick, |
Will improve_contrast work for stationary object detection (like cars parked on a driveway / in a garage) ? |
improve contrast is only for motion detection. If the object is stationary then there's likely no motion around it. |
Is there any way to make stationary object detection work at night then? |
When an object is stationary it will continue to be detected at night. Frigate will scan some parts of the image on startup but generally requires motion for the object to be initially detected. |
That's not the case here :-). Can I provide something to help diagnose or
run some diagnostics?
…On Fri, Nov 3, 2023, 19:47 'Nicolas Mowen' via github < ***@***.***> wrote:
When an object is stationary it will continue to be detected at night.
Frigate will scan some parts of the image on startup but generally requires
motion for the object to be initially detected.
—
Reply to this email directly, view it on GitHub
<#1511 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABRJVJISP7U4Z5RF5FWVPELYCV67LAVCNFSM5CDOSWAKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZZGMZDINRVHAYQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
If the car is not remaining detected then that's a detection issue meaning improper config, the object is being lost due to ID swap, or the model is struggling to detect the object. |
Hi. Sorry for resurrecting the old thread, but are there any better models, or settings helping for detection at night? |
I've used my phone to watch the debug view while walking outside on the camera and it worked well. It is also possible that the default model is not working well, I don't believe it has any IR examples in the COCO dataset |
I have tried Improve Contrast option, unfortunately the difference - if any - is marginal and only noticeable on one of the cameras. |
I'm testing Frigate for a week and I'm very impressed with it. One thing however bothers me which is the object detection at night.
I have Unifi G3 camera with IR range extender which helps a lot to illuminate a scene. However Frigate almost never detects objects at night.
Below is an example of a recording captured by Unifi Video NVR software. The dog was not detected by Frigate. Same with other types of objects.
Is there something I can tweak in the configuration to improve that (apart from cleaning the foggy lens 😉) ?
The text was updated successfully, but these errors were encountered: