Add support for NGINX VOD Module #1116

hunterjm · 2021-05-18T06:01:48Z

This adds support for viewing recordings as a "video on demand". It can be tested by loading the stream URL into this demo site or VLC.

The VOD stream format currently follows the recording directory format - i.e.: http:https://localhost:5000/vod/2021-05/17/20/{camera}/master.m3u8

This first version is obviously limited to an hour's worth of recordings based on the current recording file structure. We can expand it later by modifying the vod function in http.py. Regardless, The Home Assistant integration can be updated to use this new URL to allow Media Browser recordings of entire hour long segments without having to use FFMPEG to combine the videos. The Web UI can also use this along with something like videojs to build a recording playback page. I left the /recordings directive in tact for backwards compatibility until things switch over.

hunterjm

Hope these comments/notes are helpful as you look through this.

hunterjm · 2021-05-18T06:02:40Z

frigate/app.py

+
+ try:
+ server.serve_forever()
+ except KeyboardInterrupt:


This doesn't necessarily belong as a part of this PR, but in testing I noticed some exceptions being raised on ctrl+C so this handles it.

hunterjm · 2021-05-18T06:03:05Z

frigate/log.py

@@ -35,7 +35,7 @@ def log_process(log_queue):
 while True:
 try:
 record = log_queue.get(timeout=5)
- except queue.Empty:
+ except (queue.Empty, KeyboardInterrupt):


Similar to above, an exception was being raised here on shutdown, so handling it here.

hunterjm · 2021-05-18T06:04:07Z

frigate/http.py

@@ -446,10 +449,37 @@ def latest_frame(camera_name):
 return "Camera named {} not found".format(camera_name), 404


+@bp.route("/vod/<path:path>")
+def vod(path):


This builds a JSON object to be consumed by nginx-vod-module. Currently we are just using the existing recording path, but can expand this to add additional filters or functionality later.

hunterjm · 2021-05-18T06:06:00Z

docker/Dockerfile.nginx

@@ -0,0 +1,46 @@
+FROM ubuntu:20.04 AS base


We need to build NGINX from source to include support for aio files and multiple architectures. nginx-vod-module has a repo we could add, but it is only built for amd64.

hunterjm · 2021-05-18T06:06:43Z

docker/Dockerfile.base

@@ -18,16 +20,13 @@ ENV DEBIAN_FRONTEND=noninteractive
 # Install packages for apt repo
 RUN apt-get -qq update \
 && apt-get upgrade -y \
- && apt-get -qq install --no-install-recommends -y \
- gnupg wget unzip tzdata nginx libnginx-mod-rtmp \


removing nginx and libnginx-mod-rtmp as they are built-in to our compiled version of NGINX along with nginx-vod-module.

hunterjm · 2021-05-18T06:25:46Z

Will update docs before you merge if this looks like a good path forward to you.

blakeblackshear · 2021-05-18T11:13:21Z

frigate/http.py

+ clips.append({"type": "source", "path": filename})
+ video = cv2.VideoCapture(filename)
+ duration = int(
+ video.get(cv2.CAP_PROP_FRAME_COUNT) / video.get(cv2.CAP_PROP_FPS) * 1000


How long does it take to do this dynamically? It could take a while if using a network drive.

It’s two seconds on my dev machine, but it’s also pretty beefy and the files were local. I almost just hardcoded the 60 seconds here, but know that the command (and thus the time) can be changed in configuration.

I saw the use of ffprobe elsewhere for clips, but since OpenCV was already imported and this stackoverflow answer said it was faster, I just used it. Granted I did not do any verification myself. It’s probably worth a shot testing this on a Pi with video on a network drive to see.

Verified timing in comment below.

mitchross · 2021-05-18T14:14:16Z

Just out of curiosity, what benefits would this have for the 'clips' feature?

hunterjm · 2021-05-18T16:24:03Z

Just out of curiosity, what benefits would this have for the 'clips' feature?

This doesn't really have any benefit for clips as they are non-continuous and already are the length you expect them to be. This is just a way to combine the 60 second recording files together in an hour long video to make viewing the 24/7 recording easier.

What we could do, eventually, is in the front-end automatically add markers to the timeline for the continuous recording at the timestamps that events occurred to be able to seek to them with relative ease.

stephengolub · 2021-05-18T16:49:00Z

I literally just built a small script to generate a downloadable mp4 file to facilitate large chunks of time. I'm wondering if instead of path, you take in camera and then have query parameters that are date/time objects...

Then you just glob the files and do a greater than/less than check. You could default to the last 60 minutes.

This is my relevant bash code:

file_list=$(ls **/**/**/$CAMERA/*.mp4)                                                                                    
                                                                                                                          
real_list=()                                                                                                              
                                                                                                                          
for file in $file_list; do                                                                                                
    if [ "$file" \> "$START_FILE" ] && [ "$file" \< "$END_FILE" ]; then                                                   
        real_list+=("file '$PWD/$file'")                                                                                  
    fi                                                                                                                    
done

hunterjm · 2021-05-18T18:33:14Z

I literally just built a small script to generate a downloadable mp4 file to facilitate large chunks of time. I'm wondering if instead of path, you take in camera and then have query parameters that are date/time objects...

Then you just glob the files and do a greater than/less than check. You could default to the last 60 minutes.

This is my relevant bash code:

file_list=$(ls **/**/**/$CAMERA/*.mp4)                                                                                    
                                                                                                                          
real_list=()                                                                                                              
                                                                                                                          
for file in $file_list; do                                                                                                
    if [ "$file" \> "$START_FILE" ] && [ "$file" \< "$END_FILE" ]; then                                                   
        real_list+=("file '$PWD/$file'")                                                                                  
    fi                                                                                                                    
done

I honestly think that's the next step, but wanted to keep the initial implementation simple (easier to review just the relevant VOD module code) and expand the functionality of filters/etc in a follow-up PR.

stephengolub · 2021-05-18T19:37:59Z

If you take a look at the PR I opened against your branch, I illustrated what I meant (I haven't tested it fully yet), but, tbh, I don't think it's too much more complicated.

hunterjm#1

hunterjm · 2021-05-18T23:25:48Z

If you take a look at the PR I opened against your branch, I illustrated what I meant (I haven't tested it fully yet), but, tbh, I don't think it's too much more complicated.

hunterjm#1

After some initial testing, query string parameters do not get passed on from the VOD module. We can do some path based filtering like camera/start/end But another potential issue is the amount of time it takes to recursively glob the entire recording directory and iterate over the 43,200 files that would exist with the default retention period. I am having trouble finding a fast solution for just the latest 24 hours. I’m happy to research further and put it in a follow-up PR, but don’t want to block this one because of it.

stephengolub · 2021-05-18T23:37:39Z

I was afraid of that. Everyone knows you shouldn’t use the directory tree as a database.

Might be best to table it like you said and work on optimizing how we look up available recordings and clips

…are not saved locally

hunterjm · 2021-05-19T03:42:18Z

Did some testing on the cv.VideoCapture time for a full hour's worth of recordings (60) on my dev box:

Glob: 0.000562
Sort: 0.000004
CV Read Total: 2.192811
CV Read Avg: 0.036547

versus FFProbe that is used to verify clips and recordings:

Glob: 0.000504
Sort: 0.000005
FFProbe Read Total: 2.946326
FFProbe Read Avg: 0.049105

Granted it is a Ryzen 3900x so it may take longer on a Raspberry Pi, but it's encouraging that the OpenCV method is not only faster than a ffprobe sub process, but is less lines of code as well.

Trying to load something like 24 hours worth of video would take 53 seconds however, which is too inefficient for the initial release.

I also have NGINX aggressively caching these manifests for 100 days, so this lookup is only seen the first time you load the VOD.

The only other option would be to default duration to 60, as that is what it will be the vast majority of time. I figured if it didn't take too long to be certain though, it's better for cases where recordings ended early or a user modified the recording segment time in their configuration.

I'm happy with this PR now - the next commit updating docs should be the last.

hunterjm · 2021-05-22T15:38:18Z

@blakeblackshear - could you run the nginx make command to push the images to docker hub? That would make it so others can successfully build frigate.

blakeblackshear · 2021-05-22T19:32:54Z

My buildx command is failing to build the armv7 image. Did you do some set up to get it to work?

hunterjm · 2021-05-22T19:48:05Z

No, it built just fine for me. What step does it fail at?

hunterjm · 2021-05-22T20:42:53Z

Built and pushed for all 3 architectures here: https://hub.docker.com/r/hunterjm/frigate-nginx/tags?page=1&ordering=last_updated

blakeblackshear · 2021-05-22T22:19:39Z

Got it working with these steps. Mine are pushed.

Add support for NGINX VOD Module

e6109ac

hunterjm commented May 18, 2021

View reviewed changes

Make command has to be different than directory

f2c704c

blakeblackshear approved these changes May 18, 2021

View reviewed changes

remove comments from nginx.conf

d6ea3b1

hunterjm added 2 commits May 18, 2021 22:31

Add nginx build arg for other architectures

53eaadb

add --push so the images actually get published for nginx since they …

2ab1f88

…are not saved locally

update HTTP API docs

f099731

blakeblackshear merged commit f4bc68d into blakeblackshear:release-0.9.0 May 22, 2021

hunterjm deleted the nginx-vod-module branch May 22, 2021 15:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for NGINX VOD Module #1116

Add support for NGINX VOD Module #1116

hunterjm commented May 18, 2021 •

edited

Loading

hunterjm left a comment

hunterjm May 18, 2021

hunterjm May 18, 2021

hunterjm May 18, 2021

hunterjm May 18, 2021

hunterjm May 18, 2021

hunterjm commented May 18, 2021 •

edited

Loading

blakeblackshear May 18, 2021

hunterjm May 18, 2021

hunterjm May 19, 2021

mitchross commented May 18, 2021

hunterjm commented May 18, 2021

stephengolub commented May 18, 2021 •

edited

Loading

hunterjm commented May 18, 2021

stephengolub commented May 18, 2021

hunterjm commented May 18, 2021

stephengolub commented May 18, 2021

hunterjm commented May 19, 2021 •

edited

Loading

hunterjm commented May 22, 2021

blakeblackshear commented May 22, 2021

hunterjm commented May 22, 2021 •

edited

Loading

hunterjm commented May 22, 2021

blakeblackshear commented May 22, 2021

Add support for NGINX VOD Module #1116

Add support for NGINX VOD Module #1116

Conversation

hunterjm commented May 18, 2021 • edited Loading

hunterjm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hunterjm commented May 18, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mitchross commented May 18, 2021

hunterjm commented May 18, 2021

stephengolub commented May 18, 2021 • edited Loading

hunterjm commented May 18, 2021

stephengolub commented May 18, 2021

hunterjm commented May 18, 2021

stephengolub commented May 18, 2021

hunterjm commented May 19, 2021 • edited Loading

hunterjm commented May 22, 2021

blakeblackshear commented May 22, 2021

hunterjm commented May 22, 2021 • edited Loading

hunterjm commented May 22, 2021

blakeblackshear commented May 22, 2021

hunterjm commented May 18, 2021 •

edited

Loading

hunterjm commented May 18, 2021 •

edited

Loading

stephengolub commented May 18, 2021 •

edited

Loading

hunterjm commented May 19, 2021 •

edited

Loading

hunterjm commented May 22, 2021 •

edited

Loading