CN101877791A - MPEG4-based audio/video synchronization control method in video network transmission - Google Patents
MPEG4-based audio/video synchronization control method in video network transmission Download PDFInfo
- Publication number
- CN101877791A CN101877791A CN 201010191804 CN201010191804A CN101877791A CN 101877791 A CN101877791 A CN 101877791A CN 201010191804 CN201010191804 CN 201010191804 CN 201010191804 A CN201010191804 A CN 201010191804A CN 101877791 A CN101877791 A CN 101877791A
- Authority
- CN
- China
- Prior art keywords
- video
- frame number
- audio
- buffer
- playing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000005540 biological transmission Effects 0.000 title claims abstract description 14
- 239000000872 buffer Substances 0.000 claims abstract description 11
- 238000006243 chemical reaction Methods 0.000 claims abstract description 8
- 238000005070 sampling Methods 0.000 abstract 1
- 230000003139 buffering effect Effects 0.000 description 21
- 230000001360 synchronised effect Effects 0.000 description 9
- 230000006870 function Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses an MPEG4-based audio/video synchronization control method in video network transmission, which comprises the following steps of: calculating the time that the current audio has been played according to an audio sampling rate and the sampled number contained in each frame, and calculating the frame number of the audio to be played according to the frame number of the audio transmitted per second; if the currently actually played frame number is greater than that to be played, not executing the audio playing during dispatching until enough frame number of the audio is played and a new requirement is generated for the audio playing, and continuing the playing program; and when the frame number to be played is greater than the sum of the actually played frame number and the buffer number, before the video data decoding is completed and the YUV conversion is carried out, discarding decoded results directly. The method avoids multiple buffer modules, improves the CPU utilization rate and reduces the cost.
Description
Technical field
The present invention relates to the video network technical field, especially a kind of audio-video synchronization control method in the video network transmission based on MPEG4.
Background technology
Along with network direct broadcasting, the introducing (for example mpeg 4 standard) of the communication technology and audio-video frequency content transmission aspect advanced person's audio frequency and video compression standard, people for the sequential relationship degree of audio ﹠ video also in continuous raising.The decoding of Voice ﹠ Video and broadcast must be finished according to certain sequencing, so just can guarantee " to the last shape of the mouth as one speaks ".Therefore, guarantee the decoded synchronization of data streams broadcast of Voice ﹠ Video, this problem that insider paid close attention to just.
Nowadays, the technology that audio-visual synchronization is play is a lot, IP network stream medium audio and video synchronous playing is arranged, the method for internet stream media format audio-visual synchronization etc.For a kind of stream medium audio and video synchro system, its feature has comprised cache module: audio frequency cache module and video cache module when comprising pre decoding and decoding; Input/output control module: be used to control the output of decoded audio frequency and video data; Decoder module: comprising audio decoder module and video decode module etc.
To this, we can find some problems, and cache module is too much, because resource-constrained will influence the utilance of central processing unit (CPU), and may make waits for too long.
Summary of the invention
For the cache module that overcomes existing audio video synchronization control method too much, restriction cpu busy percentage, deficiency that cost is high, the invention provides in a kind of video network transmission of avoiding adopting more cache module, improve cpu busy percentage, reducing cost audio-video synchronization control method based on MPEG4.
The technical solution adopted for the present invention to solve the technical problems is:
Based on the audio-video synchronization control method of MPEG4, described audio-video synchronization control method may further comprise the steps in a kind of video network transmission:
The number of samples that sample rate and every frame by audio frequency comprises calculates the time that current audio frequency has been play, and the per second transmission frame number by video calculates the frame number that video should be play then;
If the frame number of current actual play greater than the frame number that should play, is not carried out the broadcast of video when then dispatching, up to voice playing enough frame numbers, when video playback had been produced new demand, playing program just can continue;
When the video frame number that the frame number of playing when needs has been play greater than reality adds number of buffers, finish at video data decoding, handle before the YUV conversion, directly will decode afterwards, the result abandons.
Further, after decoded result abandoned, the mark of current decoding buffer is set to buffer_skip, and move behind the pointer of will decoding, when playing program is seen buffer_skip, directly the buffer flag position is set as buffer_free, will moves behind the playing pointer again, then the video frame number of current actual play is added 1.
Technical conceive of the present invention is: in this simultaneous techniques, we need control four processes (audio decoder, video decode, voice playing and video playback), realize between them synchronously.Therefore, it is synchronous that we mainly are divided into two aspects: decoding and play synchronously, Voice ﹠ Video synchronously.
Mainly finish in the audio-visual synchronization process: mpeg4_initial (), mpeg4_continue () and mpeg4_stop () by three functions control.
Mpeg4_initial (): decoding initialization function.Mainly comprise and opening file, relevant hardware parameter is set (as codec (CODEC), direct memory (DMA) etc.), decoding parametric initialization (as the frame number statistics, baud rate setting, decoding initial position etc.), Control Parameter is set (as audio stream format according to file type, video stream format, quiet setting and time-out are provided with etc.), the applied address space (as output buffering (BUFFER) etc.).
Mpeg4_continue (): the entity function of decode procedure.Finish and read input, the work of decoding and writing output is unit with the frame.If it is normal to return 1 representative decoding, can call again itself, realize the decoding of next frame.Return 0 representation file end or occur unusual.Its continuous broadcast that mainly realizes audio frequency (uses DMA to finish the work of writing of CODEC, CPU finishes the decoding of audio frequency simultaneously, and realize that DMA, CODEC play and the decoding of CPU) with a group mark position, (synchronously) synchronously of audio frequency and video with the realization of audio frequency control of video, other functions such as time-out and volume adjusting, audio adjusting.
Mpeg4_stop (): coding is finished function.Finish setting (as closing CODEC), and discharge memory headroom, set and finish parameter etc. hardware.
(1) decoding and broadcast is synchronous
For voice playing and decoding, and the method for synchronization of video playback and decoding is similar.
Play with the synchronous flow process of decoding and see accompanying drawing 1;
We make the decoding can be prior to playing, and when can guarantee when irresistible factor causes decoding the short time in pause like this, the broadcast process still has enough data to play.Simultaneously, write the output BUFFER of decoded result during decoding and the input BUFFER when playing is actually same BUFFER.This BUFFER is equivalent to a circulation array, and require playing pointer be no more than the decoding pointer and the decoding pointer can not move too fast.For realizing control, all establish a sign (FLAG) for each BUFFER.
A) during program initialization, each FLAG is set to buffer_free.If the BUFFER flag bit that decoding program detects decoding pointer correspondence carries out decoding program during for buffer_free, decoded data are write BUFFER after, the sign of this BUFFER is made as buffer_wait, move behind the pointer of will decoding simultaneously; If the BUFFER of decoding pointed is other states, then decoding is not carried out in this scheduling.
B) if it is buffer_wait that playing program detects the flag bit of BUFFER, then begins playing program, and the flag bit of BUFFER is made as buffer_busy.After play finishing, write the flag bit of BUFFER as buffer_free and will be moved behind the playing pointer; If the BUFFER flag bit of BUFFER pointed is not buffer_wait, then broadcast is not carried out in this scheduling.
Because the continuity that voice playing requires, we realize the broadcast of audio frequency with DMA.Being enabled in the common code of playing program, the work of treatment after playing (promptly write the process of moving behind buffer_free and the pointer then need to be placed on to handle in the data interruption that is produced carry out) through DMA.
The synchronous realization of Voice ﹠ Video: in file was play, the speed of audio frequency was certain, can not speed also to slow down, and therefore, we realize synchronously with the audio frequency control of video, and in this control procedure, solve two problems:
If a) video is faster than audio frequency, the broadcasting speed of control of video how
Method is to come control of video to allow the frame number of playing by the frame number that audio frequency has been play.The calculation of parameter such as number of samples that sample rate and every frame by audio frequency comprises go out the time that current audio frequency has been play, and the per second transmission frame number (FPS) by video calculates the frame number that video should be play then.If the frame number of current actual play greater than the frame number that should play, is not carried out the broadcast of video when then dispatching, up to voice playing enough frame numbers, when video playback had been produced new demand, playing program just can continue.Like this, we have just effectively controlled the broadcasting speed of video.
B) if video is slower than audio frequency, how frame losing makes video catch up with audio frequency
Situation about must consider is: because system guarantees the decoding of audio frequency always, might be when some picture of broadcast to switch very fast video file, and system has little time the situation of decoded video.Therefore, in order to make video can catch up with audio frequency, must abandon the frame of some videos.Certainly, because the characteristic of MPEG4 decoder, except key frame, the decoded result of other frame numbers must depend on the decoded result of previous frame.Therefore, just each frame color space (YUV) conversion that abandons and the part of LCDs (LCD) output.
Beneficial effect of the present invention mainly shows: reduce spatial cache, resource is effectively utilized, audio-visual synchronization is respond well.
Description of drawings
Fig. 1 plays and the synchronous flow chart of decoding.
Fig. 2 is the synchronous flow chart of video and audio frequency.
Embodiment
Below in conjunction with accompanying drawing the present invention is further described.
With reference to Fig. 2, based on the audio-video synchronization control method of MPEG4, described audio-video synchronization control method may further comprise the steps in a kind of video network transmission:
The number of samples that sample rate and every frame by audio frequency comprises calculates the time that current audio frequency has been play, and the per second transmission frame number by video calculates the frame number that video should be play then;
If the frame number of current actual play greater than the frame number that should play, is not carried out the broadcast of video when then dispatching, up to voice playing enough frame numbers, when video playback had been produced new demand, playing program just can continue;
When the video frame number that the frame number of playing when needs has been play greater than reality adds number of buffers, finish at video data decoding, handle before the YUV conversion, directly will decode afterwards, the result abandons.
After decoded result abandoned, the mark of current decoding buffer is set to buffer_skip, and move behind the pointer of will decoding, when playing program is seen buffer_skip, directly the buffer flag position is set as buffer_free, to move behind the playing pointer again, then the video frame number of current actual play is added 1.
In the present embodiment, in file was play, the speed of audio frequency was certain, can not speed also to slow down, and therefore, we realize synchronously with the audio frequency control of video, and in this control procedure, solve two problems:
If a) video is faster than audio frequency, the broadcasting speed of control of video how
Method is to come control of video to allow the frame number of playing by the frame number that audio frequency has been play.The calculation of parameter such as number of samples that sample rate and every frame by audio frequency comprises go out the time that current audio frequency has been play, and the per second transmission frame number (FPS) by video calculates the frame number that video should be play then.If the frame number of current actual play greater than the frame number that should play, is not carried out the broadcast of video when then dispatching, up to voice playing enough frame numbers, when video playback had been produced new demand, playing program just can continue.Like this, we have just effectively controlled the broadcasting speed of video.
B) if video is slower than audio frequency, how frame losing makes video catch up with audio frequency
At this moment, the synchronous flow process of Voice ﹠ Video is seen accompanying drawing 2.
Situation about must consider is: because system guarantees the decoding of audio frequency always, might be when some picture of broadcast to switch very fast video file, and system has little time the situation of decoded video.Therefore, in order to make video can catch up with audio frequency, must abandon the frame of some videos.Certainly, because the characteristic of MPEG4 decoder, except key frame, the decoded result of other frame numbers must depend on the decoded result of previous frame.Therefore, just each frame color space (YUV) conversion that abandons and the part of LCDs (LCD) output.
Implementation is as follows: when the video frame number that the frame number of playing when needs has been play greater than reality adds the BUFFER number, can think the video frame number that system can't decode enough.If this occurs, finish in decoding, handle before the YUV, the back result that directly will decode abandons, and current mark of decoding BUFFER is set to buffer_skip, and moves behind the pointer of will decoding, and has so just reduced the YUV conversion operations of this frame.When playing program is seen buffer_skip, (this step must carry out directly the BUFFER marker bit to be set as buffer_free, because among the BUFFER be invalid data), will move behind the playing pointer again, then the frame number of current actual play is added 1 (the same with normal play operation).So just having reduced by a LCD plays.
Abandoning a frame means and reduces a YUV conversion and a LCD output (output at present be based on CPU but not based on DMA).Therefore, a frame-skipping means the million instructions per second (MIPS) of saving about 3.Here so-called frame-skipping just a frame be set to invalid, but this frame still equally is present in the whole video BUFFER sequence with the normal frame of playing, and therefore can not have influence on the synchronous of video and audio frequency.
Claims (2)
1. based on the audio-video synchronization control method of MPEG4, it is characterized in that: described audio-video synchronization control method may further comprise the steps during a video network transmitted:
The number of samples that sample rate and every frame by audio frequency comprises calculates the time that current audio frequency has been play, and the per second transmission frame number by video calculates the frame number that video should be play then;
If the frame number of current actual play greater than the frame number that should play, is not carried out the broadcast of video when then dispatching, up to voice playing enough frame numbers, when video playback had been produced new demand, playing program just can continue;
When the video frame number that the frame number of playing when needs has been play greater than reality adds number of buffers, finish at video data decoding, handle before the YUV conversion, directly will decode afterwards, the result abandons.
2. in a kind of video network transmission as claimed in claim 1 based on the audio-video synchronization control method of MPEG4, it is characterized in that: after decoded result is abandoned, the mark of current decoding buffer is set to buffer_skip, and move behind the pointer of will decoding, when playing program is seen buffer_skip, directly the buffer flag position is set as buffer_free, will moves behind the playing pointer again, then the video frame number of current actual play is added 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010191804 CN101877791A (en) | 2010-06-04 | 2010-06-04 | MPEG4-based audio/video synchronization control method in video network transmission |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010191804 CN101877791A (en) | 2010-06-04 | 2010-06-04 | MPEG4-based audio/video synchronization control method in video network transmission |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101877791A true CN101877791A (en) | 2010-11-03 |
Family
ID=43020240
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010191804 Pending CN101877791A (en) | 2010-06-04 | 2010-06-04 | MPEG4-based audio/video synchronization control method in video network transmission |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101877791A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103391467A (en) * | 2013-07-16 | 2013-11-13 | 四川省迪特尔电子有限公司 | Method for achieving synchronization of decoding and displaying of audio and video of network set-top box |
| CN115412757A (en) * | 2022-08-31 | 2022-11-29 | 海宁奕斯伟集成电路设计有限公司 | Video playing method and device and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1175854A (en) * | 1996-08-19 | 1998-03-11 | 松下电器产业株式会社 | Synchronous reproduction apparatus |
| US20050094966A1 (en) * | 2003-10-29 | 2005-05-05 | David Elberbaum | Method and apparatus for digitally recording and synchronously retrieving a plurality of video signals |
| US20060268993A1 (en) * | 2003-09-29 | 2006-11-30 | Eric Hamilton | Method and Apparatus for Coding Information |
| CN101119461A (en) * | 2006-08-02 | 2008-02-06 | 广达电脑股份有限公司 | System and method for maintaining video frame and audio frame synchronous broadcasting |
| CN101193311A (en) * | 2006-12-21 | 2008-06-04 | 腾讯科技（深圳）有限公司 | Audio and video data synchronization method in P2P system |
-
2010
- 2010-06-04 CN CN 201010191804 patent/CN101877791A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1175854A (en) * | 1996-08-19 | 1998-03-11 | 松下电器产业株式会社 | Synchronous reproduction apparatus |
| US20060268993A1 (en) * | 2003-09-29 | 2006-11-30 | Eric Hamilton | Method and Apparatus for Coding Information |
| US20050094966A1 (en) * | 2003-10-29 | 2005-05-05 | David Elberbaum | Method and apparatus for digitally recording and synchronously retrieving a plurality of video signals |
| CN101119461A (en) * | 2006-08-02 | 2008-02-06 | 广达电脑股份有限公司 | System and method for maintaining video frame and audio frame synchronous broadcasting |
| CN101193311A (en) * | 2006-12-21 | 2008-06-04 | 腾讯科技（深圳）有限公司 | Audio and video data synchronization method in P2P system |
Non-Patent Citations (3)
| Title |
|---|
| 《博研论坛-计算机与网络学院,https://bbs.myboyan.com/read-htm-tid-270649.html》 20081031 wwwxxx12345 Mpeg4解码流程 1-4 1-2 , 2 * |
| 《电子科技大学学报》 20060630 高科 互联网中基于MPEG-4流媒体播放的研究 381-384 1-2 第35卷, 第3期 2 * |
| 《计算机应用》 20050131 曹理宇 MPEG-4流媒体系统中的视音频同步 128-131 1-2 第25卷, 第1期 2 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103391467A (en) * | 2013-07-16 | 2013-11-13 | 四川省迪特尔电子有限公司 | Method for achieving synchronization of decoding and displaying of audio and video of network set-top box |
| CN115412757A (en) * | 2022-08-31 | 2022-11-29 | 海宁奕斯伟集成电路设计有限公司 | Video playing method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101634500B1 (en) | Media workload scheduler | |
| US7020892B2 (en) | Time-shifted video signal processing | |
| US20110002376A1 (en) | Latency Minimization Via Pipelining of Processing Blocks | |
| CN103310820B (en) | A kind of method that multimedia player is optimized | |
| CN101710992B (en) | Pre-decoding high definition player and playing method | |
| CN103475927B (en) | Online-video real-time speed change play method and system thereof | |
| KR20080014843A (en) | Method and system for improving interactive media response systems using visual cues | |
| CN102447906A (en) | Low-latency video decoding | |
| WO2011038565A1 (en) | Streaming media audio-video synchronization method and system | |
| CN109862357A (en) | Cloud game image encoding method, device, equipment and the storage medium of low latency | |
| CN106303379A (en) | A kind of video file backward player method and system | |
| CN105100871A (en) | Mute control method of intelligent television under one-screen multi-window mode | |
| CN103391467A (en) | Method for achieving synchronization of decoding and displaying of audio and video of network set-top box | |
| CN104683823A (en) | Multi-screen linked audio and video synchronizing system | |
| US7151800B1 (en) | Implementation of a DV video decoder with a VLIW processor and a variable length decoding unit | |
| US20060080109A1 (en) | Audio decoding apparatus | |
| CN101877791A (en) | MPEG4-based audio/video synchronization control method in video network transmission | |
| CN103957445A (en) | Video redirecting system and method based on application virtualization technology | |
| CN103974062B (en) | Image display device, image display system and method for displaying image | |
| CN102769737A (en) | Video image switching method and system | |
| CN202210851U (en) | Control device for standard definition and high-definition audio and video coding and decoding | |
| CN103716651A (en) | Image processing device, image processing method, and image processing system | |
| CN1310498C (en) | Digital video-audio decoder | |
| CN103716650B (en) | Server unit and information processing method | |
| CN104185028A (en) | Video playing method and terminal thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20101103 |