Frigate + Wyze Cam V3 wyze_mini_hacks RTSP- unable to listen to audio live and unable to record audio with video.

[bizwog]

Does anyone have a working config they'd be willing to share with RTSP Wyze V2/3 cameras?

I am also curious if there is a RTSP tool I can use from an android phone or some tool on the home assistant I can probe the camera for the correct video/audio info.

Here's my config if anyone can eyeball it and help. Everything in here works except the audio.

ui:
  live_mode: mse
  
record:
  enabled: true
detect:
  enabled: true

mqtt:
 host: mqtt
 stats_interval: 10
 user: frigate
 password: &&&**
 client_id: frigate

detectors:
  coral_pci:
    type: edgetpu
    device: pci

ffmpeg:
  hwaccel_args: preset-vaapi
  output_args: 
   record: preset-record-generic-audio-aac

go2rtc:
  streams:
    WYZE-C9:
      - rtsp:https://admin:[email protected]:554/video1_unicast # High
      - "ffmpeg:WYZE-C9#audio=aac"  
  
  
cameras:
  C9:
   ffmpeg:
      inputs: # High and Low Streams from C2
        - path: rtsp:https://localhost:8554/WYZE-C9 # 
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record

   live: 
    stream_name: WYZE-C9
    
   detect:
      enabled: True 
      width: 1920 
      height: 1080 
      fps: 5
   objects:
    track:
      - person
      - dog
      - cat
      - bicycle

[mcondren]

I am having the same issue with no audio in recordings or live. I’m using gotortc like you, but I saw it the docs that webrtc requires the opus codec so in the go2rtc section I convert to that like in the docs. - "ffmpeg:rtsp_cam#audio=opus" Still no audio though.

[NickM-27]

but I saw it the docs that webrtc requires the opus codec so in the go2rtc section I convert to that like in the docs. - "ffmpeg:rtsp_cam#audio=opus" Still no audio though.

rtsp_cam needs to change to be the name of the stream otherwise it does nothing

[mcondren]

Yes, I did rename that in my config. I just pasted that from the docs for the purposes of my post.

[bizwog]

I haven't solved it yet but know the issue is related to my setup, maybe frigate/go2rtc or the docker container as I've confirmed the cameras send audio with outside applications. I had no luck with webrtc or MSE in multiple browsers and the recordings do not include audio.

Next step is playing with the CLI ffmpeg fprobe tool to prove it works or not under the docker container shell with the included version of ffmpeg. I haven't had the time with my day job taking up too much time.

I did find 1-2 examples of wyze with home-assistant or wyze with frigate on other websites but the configurations settings didn't work for me.

[TheDuke427]

What codec is your camera set to in the camera settings? I have mine set to G711 and my ffmpeg is set to use opus and my audio works in both the live and recordings.

[mcondren]

My Wyze cameras are set to the aac preset, and before I updated to version 11 I was using the manual ffmpeg copy to aac argument and I got audio. So I am assuming that part is unchanged. The only thing that has changed is the move to 11 and using the preset and using go2rtc

[bizwog]

Could you share your frigate configuration, please?

[mcondren]

Sure.

---
detectors:
  coral:
    type: edgetpu
    device: usb

mqtt:
  host: mosquitto.xxx.xxx
  topic_prefix: frigate

ffmpeg:
  #  global_args:
  #    - -hide_banner
  #    - -loglevel
  #    - panic
  hwaccel_args: preset-vaapi
  #  input_args:
  #    - -avoid_negative_ts
  #    - make_zero
  #    - -fflags
  #    - nobuffer
  #    - -flags
  #    - low_delay
  #    - -strict
  #    - experimental
  #    - -fflags
  #    - +genpts+discardcorrupt
  #    - -rtsp_transport
  #    - tcp
  #    - -stimeout
  #    - '10000000'
  #    - -use_wallclock_as_timestamps
  #    - '1'
  #  # Optional: global output args (default: shown below)
  output_args:
    record: preset-record-generic-audio-aac

objects:
  track:
    - person
  filters:
    person:
      threshold: 0.75
      min_area: 0

record:
  enabled: true
  retain:
    days: 10
    mode: all
  events:
    retain:
      default: 30
      mode: all

snapshots:
  enabled: true
  bounding_box: true
  timestamp: true

birdseye:
  enabled: true
  width: 1920
  height: 1080
  quality: 1
  mode: objects

go2rtc:
  streams:
    frontporch:
      - rtsp:https://admin:[email protected]:8554/video1_unicast
      - "ffmpeg:frontporch#audio=opus"
    frontyard:
      - rtsp:https://admin:[email protected]:8554/video1_unicast
      - "ffmpeg:frontporch#audio=opus"
  webrtc:
    candidates:
      - 192.168.1.43:8555
      - stun:8555

cameras:
  frontporch:
    ffmpeg:
      inputs:
        - path: rtsp:https://admin:[email protected]:8554/video2_unicast
          roles:
            - detect
        - path: rtsp:https://localhost:8554/frontporch
          input_args: preset-rtsp-restream
          roles:
            - record
    detect:
      width: 640
      height: 360
      fps: 10
    mqtt:
      timestamp: true
      bounding_box: true
      quality: 100
    motion:
      mask:
        - 0,0,242,0,243,265,0,279
    objects:
      mask:
        - 0,0,242,0,243,265,0,279

  frontyard:
    ffmpeg:
      inputs:
        - path: rtsp:https://admin:[email protected]:8554/video2_unicast
          roles:
            - detect
        - path: rtsp:https://localhost:8554/frontyard
          input_args: preset-rtsp-restream
          roles:
            - record
    detect:
      width: 640
      height: 360
      fps: 10
    mqtt:
      timestamp: true
      bounding_box: true
      quality: 100
    motion:
      mask:
        - 529,0,0,0,0,275,537,278
        - 526,0,640,0,640,225,542,260
    objects:
      mask:
        - 529,0,0,0,0,275,537,278
        - 526,0,640,0,640,225,542,260

[bizwog]

Gracias!

[mitchross]

Why not just use Wyze Bridge? https://github.com/mrlt8/docker-wyze-bridge

[johnvprcam24573455623]

Does this looks correct to use the wyze_mini server with the two streams

    # mais stream (video1_unicast)
    cam7:
    - exec:ffmpeg -hide_banner -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport udp -analyzeduration 1000M -probesize 1000M -timeout 30000000 -use_wallclock_as_timestamps 1 -i rtsp:https://[username]:[password]@[ip]:[port]/video1_unicast -codec:v copy -codec:a aac -fflags +igndts -f rtsp {{output}}

    # SUB stream (video2_unicast)
   cam7_sub
    - exec:ffmpeg -hide_banner -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport udp -analyzeduration 1000M -probesize 1000M -timeout 30000000 -use_wallclock_as_timestamps 1 -i rtsp:https://[username]:[password]@[ip]:[port]/video2_unicast -codec:v copy -codec:a aac -fflags +igndts -f rtsp {{output}}

and

   cam7:
    ffmpeg:
      hwaccel_args: preset-vaapi
      inputs:
        - path: rtsp:https://127.0.0.1:8554/cam7?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp:https://127.0.0.1:8554/cam7_sub?video=copy
          input_args: preset-rtsp-restream
          roles:
            - detect        

[danlo315]

@johnvprcam24573455623

Did this work for you? A few things:

1. I don't think these Wyze cams do udp, so even though i defined rtsp_transport as udp it is still forcing tcp. I still have it in my command line but I don't think it does anything. I can look at ps -ef and see what ffmpeg command line options Frigate is triggering.
2. I find the input_args messes with the defined command line options in go2rtc section so I took it out.
3. I made a mistake in -fflags; you could just add +igndts to the first -fflags option.
4. The latest go2rtc seems to allow for the audio codec, so I took out the audio transcode to aac option. You have to make sure you declare the webrtc candidate though.
5. I had to transcode to aac for my recording stream though.

Here are my new relevant config.yml settings:

go2rtc:
  webrtc:
    candidates:
      - [hostip:rtsp port] #local candidate
      - stun.l.google.com:19302
.
.
  streams:
    cam1:
      - exec:ffmpeg -hide_banner -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt+igndts -rtsp_transport udp -analyzeduration 1000M -probesize 1000M -timeout 30000000 -use_wallclock_as_timestamps 1 -i rtsp:https://[user:password]@[ip:port]/video2_unicast -codec:v copy -codec:a copy -f rtsp {{output}}
    cam1_rec:
      - exec:ffmpeg -hide_banner -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt+igndts -rtsp_transport udp -analyzeduration 1000M -probesize 1000M -timeout 30000000 -use_wallclock_as_timestamps 1 -i rtsp:https://[user:password]@[ip:port]/video1_unicast -codec:v copy -codec:a copy -f rtsp {{output}}
.
.
cameras:
  cam1:
    ffmpeg:
      inputs:
        - path: rtsp:https://127.0.0.1:8554/cam1?video=copy&audio=copy
          #          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp:https://127.0.0.1:8554/cam1_rec?video=copy&audio=copy
          roles:
            - record
      output_args:
        record: -hide_banner -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt+igndts -rtsp_transport udp -analyzeduration 1000M -probesize 1000M -timeout 30000000 -use_wallclock_as_timestamps 1 -dn -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -codec:v copy -codec:a aac -b:a 24k -ac 1

[NickM-27]

so even though i defined rtsp_transport as udp it is still forcing tcp

if you specifically define udp and ffmpeg can't connect then it will just fail, not connect via tcp

I find the input_args messes with the defined command line options in go2rtc section so I took it out.

input_args in the frigate config does not interact with the go2rtc config in any way

[danlo315]

Hi @NickM-27 thanks for providing feedback. This is what I see in my ffmpeg command line when I look at my processes:

go2rtc process:
ffmpeg -hide_banner -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt+igndts -rtsp_transport udp -analyzeduration 1000M -probesize 1000M -timeout 30000000 -use_wallclock_as_timestamps 1 -i rtsp:https://[wyze cam ip/port]/video2_unicast -codec:v copy -codec:a copy -f rtsp [rtsp_url]

stream process:
ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.12.1-367d724 -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp:https://127.0.0.1:8554/cam1?video=copy&audio=copy -r 8 -vf fps=8,scale_vaapi=w=1280:h=720,hwdownload,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:

Does this look right to you, or do you have any suggestions on what I should be doing?

Thanks.

[NickM-27]

Seems fine if it is working, I just wanted to clarify those points

[johnvprcam24573455623]

@danlo315
so on wz_mini are you are using go2rtc or v4l2rtspserver server?

Could you please also share:

• frigate.cfg cameras: / cameraname: / ffmpeg: config,
• wz_mini.conf

Thanks!

[danlo315]

Sorry everyone I don't know what else I'm doing that is different. The only hwaccel_args is I'm using preset-vaapi. My output_args is only for record and because I want the recording to be every minute. I am using the latest 1.6 version of go2rtc, but it was working for me even when I was on the version that came with Frigate.

ffmpeg:
  hwaccel_args: preset-vaapi
  output_args:
    record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac

Here is my ffprobe on the docker host.

Input #0, rtsp, from 'rtsp:https://[host ip]:[port]/camera1':
Metadata:
title : go2rtc/1.6.0
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 640x360, 20 fps, 20 tbr, 90k tbn, 40 tbc
Stream #0:1: Audio: aac (LC), 8000 Hz, mono, fltp

[danlo315]

Did you guys enable audio in your wz_mini_hacks?

[devinslick]

Yes, audio is enabled on both streams in wz_mini_hacks and ffprobe shows the stream, but when looking at RTSP stream provided by Frigate it doesn't have audio.

The camera's RTSP stream:
ffprobe -v error -show_streams -print_format json rtsp:https://username:[email protected]:8554/video2_unicast -rtsp_transport tcp
{
"streams": [
{
"index": 0,
"codec_name": "h264",
"codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
"profile": "Main",
"codec_type": "video",
"codec_tag_string": "[0][0][0][0]",
"codec_tag": "0x0000",
"width": 640,
"height": 360,
"coded_width": 640,
"coded_height": 360,
"closed_captions": 0,
"has_b_frames": 0,
"pix_fmt": "yuv420p",
"level": 50,
"color_range": "tv",
"color_space": "bt709",
"color_transfer": "bt709",
"color_primaries": "bt709",
"chroma_location": "left",
"field_order": "progressive",
"refs": 1,
"is_avc": "false",
"nal_length_size": "0",
"r_frame_rate": "241/12",
"avg_frame_rate": "20/1",
"time_base": "1/90000",
"start_pts": 170589,
"start_time": "1.895433",
"bits_per_raw_sample": "8",
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0
}
},
{
"index": 1,
"codec_name": "pcm_s16be",
"codec_long_name": "PCM signed 16-bit big-endian",
"codec_type": "audio",
"codec_tag_string": "[0][0][0][0]",
"codec_tag": "0x0000",
"sample_fmt": "s16",
"sample_rate": "8000",
"channels": 1,
"bits_per_sample": 16,
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/8000",
"start_pts": 0,
"start_time": "0.000000",
"bit_rate": "128000",
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0
}
}
]
}

Via frigate:
ffprobe -v error -show_streams -print_format json rtsp:https://frigate.internal.lan:8554/camera1 -rtsp_transport tcp
{
"streams": [
{
"index": 0,
"codec_name": "h264",
"codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
"profile": "Main",
"codec_type": "video",
"codec_tag_string": "[0][0][0][0]",
"codec_tag": "0x0000",
"width": 640,
"height": 360,
"coded_width": 640,
"coded_height": 360,
"closed_captions": 0,
"has_b_frames": 0,
"pix_fmt": "yuv420p",
"level": 50,
"color_range": "tv",
"color_space": "bt709",
"color_transfer": "bt709",
"color_primaries": "bt709",
"chroma_location": "left",
"field_order": "progressive",
"refs": 1,
"is_avc": "false",
"nal_length_size": "0",
"r_frame_rate": "20/1",
"avg_frame_rate": "20/1",
"time_base": "1/90000",
"start_pts": 67465,
"start_time": "0.749611",
"bits_per_raw_sample": "8",
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0
}
}
]
}

I've mirrored what you provided as your config and still, no audio. The main difference is that I'm using preset-intel-qsv-h264 but I also tested with preset-vaapi and got the same results.

Any ideas on debugging or steps to try? It looks like go2rtc 1.6 solves some PCM and webrtc audio issues and seems like a reasonable solution if there aren't other ideas.

[danlo315]

Here is my ffprobe on audio via Frigate:

    {
        "index": 1,
        "codec_name": "aac",
        "codec_long_name": "AAC (Advanced Audio Coding)",
        "profile": "LC",
        "codec_type": "audio",
        "codec_tag_string": "[0][0][0][0]",
        "codec_tag": "0x0000",
        "sample_fmt": "fltp",
        "sample_rate": "8000",
        "channels": 1,
        "channel_layout": "mono",
        "bits_per_sample": 0,
        "r_frame_rate": "0/0",
        "avg_frame_rate": "0/0",
        "time_base": "1/8000",
        "start_pts": 0,
        "start_time": "0.000000",
        "disposition": {
            "default": 0,
            "dub": 0,
            "original": 0,
            "comment": 0,
            "lyrics": 0,
            "karaoke": 0,
            "forced": 0,
            "hearing_impaired": 0,
            "visual_impaired": 0,
            "clean_effects": 0,
            "attached_pic": 0,
            "timed_thumbnails": 0
        }
    }

Are you guys all using exec:ffmpeg under go2rtc in frigate config.yml? I couldn't get it to work using ffmpeg:rtsp.

[danlo315]

FYI, go2rtc 1.6.2 supports wyze audio codecs. You can actually set c:a copy (rather than c:a aac to transcode). It will not show up in the go2rtc ffprobe for some reason, but the streams will stream with audio in Home Assistant. You'll still need to transcode for the recordings.

[johnvprcam24573455623]

Yeah, and the cam will also have a go2rtc, so my goal will be to call WyzeCam go2rtc server direclty from the:
ffmpeg:
inputs:
- path:

I successfully activated the go2rtc server on the CAM:
https://github.com/gtxaspec/wz_mini_hacks/wiki/Configuration-File#go2rtc-streaming-server

however was not able to figure out how to call the stream from Frigate:

[itkfilelor]

I'm a little late to the party but here is the config I have to make it work with webrtc:
first you need the link.
open the rtc page :1984
select the stream you want and click links
Under the first Header copy the link for
- rtsp for MP4 recording (Hass or Frigate) / codecs: H264, H265, AAC

In the frigate config.yaml I have:

go2rtc:
    streams:
        wyze1:
            - <wyze1-ip>:8554/1080p?mp4  # replace 1080p with your chosen stream name
            - ffmpeg:wyze1#video=copy#audio=opus
cameras:
    wyze1:
        enabled: True
        ui:
             dashboard: True
        live:
            stream_name: wyze1
        ffmpeg:
            inputs:
                 - path: rtsp:https://127.0.0.1:8554/wyze1
            roles:
                - detect
...