PaddleGAN的视频唇形同步模型Wav2lip实现了人物口型与输入语音同步，也就是俗称的「对口型」。已经有许多有趣的模型应用，比如下面的两位“女神”、“男神”的图片配音：
Wav2lip实现唇形与语音精准同步突破的关键在于，它采用了唇形同步判别器，以强制生成器持续产生准确而逼真的唇部运动。我们也可以将Wav2lip理解为一种使用脸部图片和语音音频作为监督信息的 Conditional GAN。
此外，该研究通过在鉴别器中，使用多个连续帧而不是单个帧，并使用视觉质量损失（而不仅仅是对比损失）来考虑时间相关性，从而改善了视觉质量。因此，Wav2lip不但能够给输入的图片，而且能够给输入视频对口型。
该wav2lip模型适用于任何人脸、任何语音、任何语言，对任意视频都能达到很高的准确率，可以无缝地与原始视频融合，还可以用于转换动画人脸，并且导入合成语音也是可行的。因为无论使用何种语言的语料进行训练，口型与发音的对应还是一样的。
当然，wav2lip模型对转化的视频还是有一点点要求的，就是视频帧（图片同样）中必须包含至少一个嘴部较完整面部。毕竟没有嘴也是没法对嘴型的~
下面我们就通过一个 demo 来展示如何使用 PaddleGAN 来实现“5行代码配嘴型”。
如果 github 较慢可是使用 gitee 上的镜像下载PaddleGAN。下载后进入目录进行安装。安装过程主要是安装依赖和设置环境变量以正常运行 PaddleGAN 集成的各种模型训练、预测脚本。
!git clone https://gitee.com/paddlepaddle/PaddleGAN.git
%cd PaddleGAN/
!pip install -v -e .-
在项目目录下建立 audio 文件夹用来存储生成嘴型的音频文件。这里使用“第8套广播体操”的伴奏音频，文件名为 gymnastics.m4a 。m4a是win10录音用的默认格式，当然也可以使用 wav 和 mp3 格式的音频文件。
-
在项目目录下建立 face 文件夹用来存储进行对嘴型的视频或图片。这里使用一个《疯狂的外星人》中的片段，文件名为 alien.mp4。
# 这段代码用于展示视频不算在那“5行代码配嘴型”之内~
import cv2
import imageio
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from IPython.display import HTML
import warnings
warnings.filterwarnings("ignore")
def display(driving, fps, size=(8, 6)):
fig = plt.figure(figsize=size)
ims = []
for i in range(len(driving)):
cols = []
cols.append(driving[i])
im = plt.imshow(np.concatenate(cols, axis=1), animated=True)
plt.axis('off')
ims.append([im])
video = animation.ArtistAnimation(fig, ims, interval=1000.0/fps, repeat_delay=1000)
plt.close()
return video
input_video_path = '/home/aistudio/face/alien.mp4'
video_frames = imageio.mimread(input_video_path, memtest=False)
# 获得视频的原分辨率
cap = cv2.VideoCapture(input_video_path)
fps = cap.get(cv2.CAP_PROP_FPS)
HTML(display(video_frames, fps).to_html5_video())首先进入 PaddleGAN 的 applications 目录，然后调用 tools 文件夹下的 wav2lip.py 脚本进行唇形合成。
--face 参数指定进唇形行合成的图片或视频。
--audio 参数指定口型依据的音频。
--outfile 参数指定输出的合成好的视频文件路径。
%cd /home/aistudio/PaddleGAN/applications/
!python tools/wav2lip.py --face /home/aistudio/face/alien.mp4 --audio /home/aistudio/audio/gymnastics.m4a --outfile /home/aistudio/output/output.mp4/home/aistudio/PaddleGAN/applications
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:26: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
def convert_to_list(value, n, name, dtype=np.int):
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/scipy/linalg/__init__.py:217: DeprecationWarning: The module numpy.dual is deprecated. Instead of using dual, use the functions directly from numpy or scipy.
from numpy.dual import register_func
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/scipy/special/orthogonal.py:81: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
from numpy import (exp, inf, pi, sqrt, floor, sin, cos, around, int,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/scipy/io/matlab/mio5.py:98: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
from .mio5_utils import VarReader5
Reading video frames...
�[0;36m[h264 @ 0x5621ce16a400] �[0m�[1;31mno frame!
�[0m�[0;36m[h264 @ 0x5621ce3ce0c0] �[0m�[1;31mno frame!
�[0mNumber of frames available for inference: 140
Extracting raw audio...
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/aistudio/audio/gymnastics.m4a':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: mp41isom
creation_time : 2021-04-21 07:12:51
date : 2021
title : 录音
album_artist : 录音机
Duration: 00:00:07.53, start: 0.000000, bitrate: 196 kb/s
Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 194 kb/s (default)
Metadata:
creation_time : 2021-04-21 07:12:51
handler_name : SoundHandler
Output #0, wav, to 'temp/temp.wav':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: mp41isom
album_artist : 录音机
ICRD : 2021
INAM : 录音
ISFT : Lavf56.40.101
Stream #0:0(und): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s (default)
Metadata:
creation_time : 2021-04-21 07:12:51
handler_name : SoundHandler
encoder : Lavc56.60.100 pcm_s16le
Stream mapping:
Stream #0:0 -> #0:0 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
size= 1412kB time=00:00:07.53 bitrate=1536.1kbits/s
video:0kB audio:1412kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.007469%
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/numba/types/__init__.py:110: DeprecationWarning: `np.long` is a deprecated alias for `np.compat.long`. To silence this warning, use `np.compat.long` by itself. In the likely event your code does not need to work on Python 2 you can use the builtin `int` for which `np.compat.long` is itself an alias. Doing this will not modify any behaviour and is safe. When replacing `np.long`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
long_ = _make_signed(np.long)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/numba/types/__init__.py:111: DeprecationWarning: `np.long` is a deprecated alias for `np.compat.long`. To silence this warning, use `np.compat.long` by itself. In the likely event your code does not need to work on Python 2 you can use the builtin `int` for which `np.compat.long` is itself an alias. Doing this will not modify any behaviour and is safe. When replacing `np.long`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
ulong = _make_unsigned(np.long)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/linear_model/_least_angle.py:30: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
method='lar', copy_X=True, eps=np.finfo(np.float).eps,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/linear_model/_least_angle.py:169: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
method='lar', copy_X=True, eps=np.finfo(np.float).eps,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/linear_model/_least_angle.py:286: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
eps=np.finfo(np.float).eps, copy_Gram=True, verbose=0,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/linear_model/_least_angle.py:858: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
eps=np.finfo(np.float).eps, copy_X=True, fit_path=True):
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/linear_model/_least_angle.py:1094: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
eps=np.finfo(np.float).eps, copy_X=True, fit_path=True,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/linear_model/_least_angle.py:1120: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
eps=np.finfo(np.float).eps, positive=False):
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/linear_model/_least_angle.py:1349: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
max_n_alphas=1000, n_jobs=None, eps=np.finfo(np.float).eps,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/linear_model/_least_angle.py:1590: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
max_n_alphas=1000, n_jobs=None, eps=np.finfo(np.float).eps,
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/linear_model/_least_angle.py:1723: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
eps=np.finfo(np.float).eps, copy_X=True, positive=False):
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/decomposition/_lda.py:29: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
EPS = np.finfo(np.float).eps
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/feature_extraction/image.py:167: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
dtype=np.int):
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/numba/ir_utils.py:1512: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if (hasattr(numpy, value)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/numba/ir_utils.py:1513: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
and def_val == getattr(numpy, value)):
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/numba/ir_utils.py:1512: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if (hasattr(numpy, value)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/numba/ir_utils.py:1513: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
and def_val == getattr(numpy, value)):
(80, 603)
Length of mel chunks: 112
W0421 23:12:35.299253 2458 device_context.cc:362] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
W0421 23:12:35.305552 2458 device_context.cc:372] device: 0, cuDNN Version: 7.6.
2021-04-21 23:12:40,733 - INFO - unique_endpoints {''}
2021-04-21 23:12:40,734 - INFO - Found /home/aistudio/.cache/paddle/hapi/weights/wav2lip_hq.pdparams
Model loaded
0%| | 0/1 [00:00<?, ?it/s]2021-04-21 23:12:41,101 - INFO - unique_endpoints {''}
2021-04-21 23:12:41,101 - INFO - Found /home/aistudio/.cache/paddle/hapi/weights/s3fd_paddle.pdparams
0%| | 0/7 [00:00<?, ?it/s]�[A/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:143: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if data.dtype == np.object:
14%|██████▍ | 1/7 [00:03<00:20, 3.34s/it]�[A
29%|████████████▊ | 2/7 [00:06<00:16, 3.21s/it]�[A
43%|███████████████████▎ | 3/7 [00:09<00:12, 3.20s/it]�[A
57%|█████████████████████████▋ | 4/7 [00:12<00:09, 3.13s/it]�[A
71%|████████████████████████████████▏ | 5/7 [00:15<00:06, 3.14s/it]�[A
86%|██████████████████████████████████████▌ | 6/7 [00:18<00:03, 3.09s/it]�[A
100%|█████████████████████████████████████████████| 7/7 [00:21<00:00, 3.04s/it]�[A
100%|█████████████████████████████████████████████| 1/1 [00:22<00:00, 22.53s/it]
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100
�[0;33mGuessed Channel Layout for Input Stream #0.0 : stereo
�[0mInput #0, wav, from 'temp/temp.wav':
Metadata:
date : 2021
title : 录音
encoder : Lavf56.40.101
Duration: 00:00:07.53, bitrate: 1536 kb/s
Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, 2 channels, s16, 1536 kb/s
Input #1, avi, from 'temp/result.avi':
Metadata:
encoder : Lavf58.31.101
Duration: 00:00:07.47, start: 0.000000, bitrate: 2450 kb/s
Stream #1:0: Video: mpeg4 (Simple Profile) (DIVX / 0x58564944), yuv420p, 874x328 [SAR 1:1 DAR 437:164], 2463 kb/s, 15 fps, 15 tbr, 15 tbn, 15 tbc
�[1;36m[libx264 @ 0xc43ba0] �[0m�[0;33m-qscale is ignored, -crf is recommended.
�[0m�[1;36m[libx264 @ 0xc43ba0] �[0musing SAR=1/1
�[1;36m[libx264 @ 0xc43ba0] �[0musing cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
�[1;36m[libx264 @ 0xc43ba0] �[0mprofile High, level 2.2
�[1;36m[libx264 @ 0xc43ba0] �[0m264 - core 148 r2643 5c65704 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http:https://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=10 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=15 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/home/aistudio/output/output.mp4':
Metadata:
date : 2021
title : 录音
encoder : Lavf56.40.101
Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 874x328 [SAR 1:1 DAR 437:164], q=-1--1, 15 fps, 15360 tbn, 15 tbc
Metadata:
encoder : Lavc56.60.100 libx264
Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:
encoder : Lavc56.60.100 aac
Stream mapping:
Stream #1:0 -> #0:0 (mpeg4 (native) -> h264 (libx264))
Stream #0:0 -> #0:1 (pcm_s16le (native) -> aac (native))
Press [q] to stop, [?] for help
frame= 112 fps= 91 q=-1.0 Lsize= 1169kB time=00:00:07.53 bitrate=1271.9kbits/s
video:1047kB audio:118kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.384919%
�[1;36m[libx264 @ 0xc43ba0] �[0mframe I:1 Avg QP:21.83 size: 32718
�[1;36m[libx264 @ 0xc43ba0] �[0mframe P:107 Avg QP:22.80 size: 9596
�[1;36m[libx264 @ 0xc43ba0] �[0mframe B:4 Avg QP:26.60 size: 2884
�[1;36m[libx264 @ 0xc43ba0] �[0mconsecutive B-frames: 92.9% 7.1% 0.0% 0.0%
�[1;36m[libx264 @ 0xc43ba0] �[0mmb I I16..4: 9.4% 80.2% 10.4%
�[1;36m[libx264 @ 0xc43ba0] �[0mmb P I16..4: 0.7% 5.2% 0.5% P16..4: 53.0% 16.7% 10.7% 0.0% 0.0% skip:13.2%
�[1;36m[libx264 @ 0xc43ba0] �[0mmb B I16..4: 0.0% 0.7% 0.0% B16..8: 55.4% 7.0% 1.2% direct: 2.2% skip:33.4% L0:49.2% L1:22.5% BI:28.4%
�[1;36m[libx264 @ 0xc43ba0] �[0m8x8 transform intra:81.2% inter:60.3%
�[1;36m[libx264 @ 0xc43ba0] �[0mcoded y,uvDC,uvAC intra: 69.3% 76.0% 24.4% inter: 34.3% 34.4% 1.7%
�[1;36m[libx264 @ 0xc43ba0] �[0mi16 v,h,dc,p: 37% 30% 27% 6%
�[1;36m[libx264 @ 0xc43ba0] �[0mi8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 27% 29% 2% 2% 1% 3% 2% 4%
�[1;36m[libx264 @ 0xc43ba0] �[0mi4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 39% 20% 12% 4% 5% 4% 7% 4% 6%
�[1;36m[libx264 @ 0xc43ba0] �[0mi8c dc,h,v,p: 43% 28% 24% 5%
�[1;36m[libx264 @ 0xc43ba0] �[0mWeighted P-Frames: Y:0.0% UV:0.0%
�[1;36m[libx264 @ 0xc43ba0] �[0mref P L0: 79.4% 11.9% 7.0% 1.8%
�[1;36m[libx264 @ 0xc43ba0] �[0mref B L0: 90.2% 9.8%
�[1;36m[libx264 @ 0xc43ba0] �[0mkb/s:1147.58
�[0m
合成的视频存放在 --outfile 参数指定的 /home/aistudio/output 文件夹下。合成的视频已经配好口型，并替换了音轨。如果打开时间过长或想听音频效果，请将合成的文件下载。
这里展示下合成视频配嘴型的效果，看看“地外总代理”是怎么忽悠外星人的吧~
output_video_path = '/home/aistudio/output/output.mp4'
video_frames = imageio.mimread(output_video_path, memtest=False)
# 获得视频的原分辨率
cap = cv2.VideoCapture(output_video_path)
fps = cap.get(cv2.CAP_PROP_FPS)
HTML(display(video_frames, fps).to_html5_video())