CN104683823A - Multi-screen linked audio and video synchronizing system - Google Patents
Multi-screen linked audio and video synchronizing system Download PDFInfo
- Publication number
- CN104683823A CN104683823A CN201310631832.9A CN201310631832A CN104683823A CN 104683823 A CN104683823 A CN 104683823A CN 201310631832 A CN201310631832 A CN 201310631832A CN 104683823 A CN104683823 A CN 104683823A
- Authority
- CN
- China
- Prior art keywords
- coding
- audio
- video data
- module
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 claims abstract description 9
- 230000009123 feedback regulation Effects 0.000 claims description 8
- 230000001105 regulatory effect Effects 0.000 claims description 4
- 238000000034 method Methods 0.000 abstract description 4
- 238000005516 engineering process Methods 0.000 description 5
- 230000001360 synchronised effect Effects 0.000 description 4
- 238000012856 packing Methods 0.000 description 3
- 238000005538 encapsulation Methods 0.000 description 2
- 230000033228 biological regulation Effects 0.000 description 1
- 239000007795 chemical reaction product Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
- H04N21/6437—Real-time Transport Protocol [RTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a multi-screen linked audio and video synchronizing system. The multi-screen linked audio and video synchronizing system comprises a signal source coding end and a receiving end, wherein the signal source coding end comprises a video coding module for coding original video data; an audio coding module for coding original audio data; a video data TCP (Transmission Control Protocol) transmitting module for transmitting the coded video data to the receiving end through the TCP protocol; an audio data UDP (User Datagram Protocol) transmitting module for transmitting the coded audio data to the receiving end through the UDP protocol, wherein the coding of the original video data and the coding of the original audio data are simultaneously performed, the interval between the transmitting time of the coded video data and the transmitting time of the coded audio data is smaller than 40ms, and the receiving end is used for receiving and playing the coded video data and the coded audio data. Through the system, the process of signal source end coding is simplified to a very great extent and the synchronism of the video data and the audio data is also guaranteed.
Description
Technical field
The present invention relates to Streaming Media field, particularly the multi-screen ganged audio-visual synchronization system of one.
Background technology
Along with Web TV, developing rapidly of the communication technology and media product, stream media technology becomes the online play-back technology of one of rising in recent years, and namely download limit in limit is play, player also need not wait until that multimedia file is play after all downloading to local storage again, but plays while downloading.Audio and Video be synchronously the important step that media server provides video capability, under normal circumstances, Voice & Video deposit data is in a media file, and undertaken synchronously by timestamp, audio-visual synchronization adopts PTS to realize audio-visual synchronization, audio sync video PTS, audio video synchronization audio frequency PTS or audio-visual synchronization STC.At present, multi-screen ganged technology relative maturity, be widely used between each end product mutual, an intelligent terminal is as main frame, host terminal can be mobile phone or panel computer, another terminal is as from machine, can be TV or mobile phone from machine terminal, can by main frame control from machine display content, so multi-screen ganged audio-visual synchronization is a very important problem, also the synchronous of audio frequency and video is controlled mainly through timestamp in the prior art, so general stream media play system middle pitch video data synchronization needs the steps such as packing encapsulation and write time stamp.
Summary of the invention
In order to overcome the deficiencies in the prior art, the invention provides a kind of multi-screen ganged audio-visual synchronization system, by this system, without the need to packing to original audio, video data, insert the steps such as timestamp, as long as sent by each the frame audio, video data after coding, simultaneously for receiving terminal, directly audio, video data is decoded immediately and play, so this system used leaves much room simplifies the flow process of information source end coding, also ensure that the synchronism of video data and voice data simultaneously.
Technical scheme provided by the invention is:
A kind of multi-screen ganged audio-visual synchronization system, comprises message sink coding end and receiving terminal;
Described message sink coding end comprises:
Video encoding module, it is in order to encode to original video data;
Audio coding module, it is in order to encode to original audio data;
Video data TCP sending module, the video data after coding is sent to described receiving terminal by Transmission Control Protocol by it;
Voice data UDP sending module, the voice data after coding is sent to described receiving terminal by udp protocol by it;
Wherein, coding to original video data carries out with the coding to original audio data simultaneously, the interval of the transmitting time of the transmitting time of the video data after coding and the voice data after encoding is less than 40ms, and described receiving terminal is for receiving and the video data after broadcast encoder and the voice data after coding.
Preferably, described multi-screen ganged audio-visual synchronization system, it is characterized in that, described receiving terminal comprises:
Video decoding module, in order to decode to the video data after coding;
Audio decoder module, in order to decode to the voice data after coding.
Preferably, described multi-screen ganged audio-visual synchronization system, it is characterized in that, described receiving terminal comprises:
Video data encoding frame number feedback regulation module, the difference of the frame number of its frame number for the video data after calculation code and decoded video data, when described difference exceeds predetermined interval, then regulate the code check parameter of described video encoding module, reduce code check, described predetermined interval is 6 ~ 8 frames.
Preferably, described multi-screen ganged audio-visual synchronization system, is characterized in that, described message sink coding end comprises:
Coding frame per second feedback regulation module, it is used for calculating the frame per second of described video encoding module coding, by regulating the code check parameter of described video encoding module, reducing or improving code check, making frame rate stable at 24fps.
Preferably, described multi-screen ganged audio-visual synchronization system, is characterized in that, described video encoding module adopts when H.264RP encoding and only exports I frame and P frame.
Preferably, described multi-screen ganged audio-visual synchronization system, is characterized in that, described audio coding module is AAC encoder.
Preferably, the multi-screen ganged audio-visual synchronization system stated, it is characterized in that, described receiving terminal comprises a display screen, and described display screen is used for the video data after broadcast decoder and decoded voice data.
Multi-screen ganged audio-visual synchronization system of the present invention, compared with prior art, adopt different network protocol to send audio, video data, video data adopts Transmission Control Protocol to send, the transmitting of TCP ensure that the complete and accurate of video data, avoids the mosaic because frame losing causes.Voice data adopts UDP to send, and to ensure that when the network bandwidth is taken in a large number by video data data still can fast transport, ensures the synchronous of audio, video data.Audio, video data adopts different network protocol subchannel to send, and farthest make use of the network bandwidth, ensure that the real-time of voice data and video data, synchronism.Adopt this system to simplify the coding flow process of information source end and the decoding process of receiving terminal, eliminate the step such as audio, video data packing encapsulation and write time stamp in general stream media play system, effectively reduce overhead.
Accompanying drawing explanation
Fig. 1 is the composition schematic diagram of multi-screen ganged audio-visual synchronization system of the present invention.
Embodiment
Below in conjunction with accompanying drawing, the present invention is described in further detail, can implement according to this with reference to specification word to make those skilled in the art.
As shown in Figure 1, the invention provides a kind of multi-screen ganged audio-visual synchronization system, comprising:
A kind of multi-screen ganged audio-visual synchronization system, comprises message sink coding end and receiving terminal;
Described message sink coding end comprises:
Video encoding module, it is in order to encode to original video data, and described video encoding module adopts and H.264RP encodes, and only exports I frame and P frame during coding;
Audio coding module, it is in order to encode to original audio data, and described audio coding module is AAC encoder;
Video data TCP sending module, the video data after coding is sent to described receiving terminal by Transmission Control Protocol by it;
Voice data UDP sending module, the voice data after coding is sent to described receiving terminal by udp protocol by it;
Coding frame per second feedback regulation module, it is used for calculating the frame per second of described video encoding module coding, by regulating the code check parameter of described video encoding module, reduce or improve code check, make frame rate stable at 24fps, such encoder is designed to coding frame per second in systems in which and follows video requency frame data transmission beat dynamic adjustments, therefore Internet Transmission situation is reflected in real time by frame per second, when frame per second is not equal to 24fps, video encoder code check parameter is regulated, to reduce or to improve code check, ensure that frame rate stable is at 24fps.
Described receiving terminal, can be multiple, each receiving terminal includes:
Video decoding module, in order to decode to the video data after coding;
Audio decoder module, in order to decode to the voice data after coding.
Video data encoding frame number feedback regulation module, the difference of the frame number of its frame number for the video data after calculation code and decoded video data, when described difference exceeds predetermined interval, then regulate the code check parameter of described video encoding module, to reduce code check, described predetermined interval is 6 ~ 8 frames, namely because network data time delay is 180 ~ 240ms.
Display screen, described display screen is used for the video data after broadcast decoder and decoded voice data.
Wherein, coding to original video data carries out with the coding to original audio data simultaneously, can be controlled by a control switch, because the encoding rate to original video data is different with the encoding rate to original audio data, but the interval of the transmitting time of the transmitting time of the video data after control coding and the voice data after encoding is less than 40ms, described receiving terminal for receive and broadcast decoder after video data and decoded voice data, by the regulation and control of encode frame per second feedback regulation module and video data encoding frame number feedback regulation module, the time interval of display screen playing video data and voice data is no more than 160ms, people see and listen and do not feel this time interval, so ensure that the synchronous of video data and voice data.
Because the video data volume is far longer than amount of audio data, in order to avoid video takies the problem that a large amount of network bandwidth causes audio transmission time delay, different network protocol is adopted to send audio, video data, video data adopts Transmission Control Protocol to send, the transmitting of TCP ensure that the complete and accurate of video data, avoids the Ma Saike because frame losing causes; Voice data adopts UDP to send, and to ensure that when the network bandwidth is taken in a large number by video data data still can fast transport, ensures the synchronous of audio, video data.
Although embodiment of the present invention are open as above, but it is not restricted to listed in specification and execution mode utilization, it can be applied to various applicable the field of the invention completely, for those skilled in the art, can easily realize other amendment, therefore do not deviating under the universal that claim and equivalency range limit, the present invention is not limited to specific details and illustrates here and the legend described.
Claims (7)
1. a multi-screen ganged audio-visual synchronization system, is characterized in that, comprises message sink coding end and receiving terminal;
Described message sink coding end comprises:
Video encoding module, it is in order to encode to original video data;
Audio coding module, it is in order to encode to original audio data;
Video data TCP sending module, the video data after coding is sent to described receiving terminal by Transmission Control Protocol by it;
Voice data UDP sending module, the voice data after coding is sent to described receiving terminal by udp protocol by it;
Wherein, coding to original video data carries out with the coding to original audio data simultaneously, the interval of the transmitting time of the transmitting time of the video data after coding and the voice data after encoding is less than 40ms, and described receiving terminal is for receiving and the video data after broadcast encoder and the voice data after coding.
2. multi-screen ganged audio-visual synchronization system as claimed in claim 1, it is characterized in that, described receiving terminal comprises:
Video decoding module, in order to decode to the video data after coding;
Audio decoder module, in order to decode to the voice data after coding.
3. multi-screen ganged audio-visual synchronization system as claimed in claim 2, it is characterized in that, described receiving terminal comprises:
Video data encoding frame number feedback regulation module, the difference of the frame number of its frame number for the video data after calculation code and decoded video data, when described difference exceeds predetermined interval, then regulate the code check parameter of described video encoding module, reduce code check, described predetermined interval is 6 ~ 8 frames.
4. multi-screen ganged audio-visual synchronization system as claimed in claim 2 or claim 3, is characterized in that, described message sink coding end comprises:
Coding frame per second feedback regulation module, it is used for calculating the frame per second of described video encoding module coding, by regulating the code check parameter of described video encoding module, reducing or improving code check, making frame rate stable at 24fps.
5. multi-screen ganged audio-visual synchronization system as claimed in claim 2, is characterized in that, described video encoding module adopts when H.264RP encoding and only exports I frame and P frame.
6. multi-screen ganged audio-visual synchronization system as claimed in claim 3, is characterized in that, described audio coding module is AAC encoder.
7. multi-screen ganged audio-visual synchronization system as claimed in claim 2, it is characterized in that, described receiving terminal comprises a display screen, and described display screen is used for the video data after broadcast decoder and decoded voice data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310631832.9A CN104683823A (en) | 2013-11-29 | 2013-11-29 | Multi-screen linked audio and video synchronizing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310631832.9A CN104683823A (en) | 2013-11-29 | 2013-11-29 | Multi-screen linked audio and video synchronizing system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104683823A true CN104683823A (en) | 2015-06-03 |
Family
ID=53318302
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310631832.9A Pending CN104683823A (en) | 2013-11-29 | 2013-11-29 | Multi-screen linked audio and video synchronizing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104683823A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106982346A (en) * | 2017-04-07 | 2017-07-25 | 上海良讯科技股份有限公司 | Multi-screen telepresence system and method |
| CN108304753A (en) * | 2017-01-24 | 2018-07-20 | 腾讯科技（深圳）有限公司 | Video communication method and video communication device |
| CN109120974A (en) * | 2018-07-25 | 2019-01-01 | 深圳市异度信息产业有限公司 | A kind of method and device that audio-visual synchronization plays |
| CN109361945A (en) * | 2018-10-18 | 2019-02-19 | 广州市保伦电子有限公司 | The meeting audiovisual system and its control method of a kind of quick transmission and synchronization |
| CN113347468A (en) * | 2021-04-21 | 2021-09-03 | 深圳市乐美客视云科技有限公司 | Audio and video transmission method and device based on Ethernet frame and storage medium |
| CN114143584A (en) * | 2021-09-29 | 2022-03-04 | 杭州当虹科技股份有限公司 | Broadcasting system and method for realizing synchronous broadcasting among multiple terminals |
| CN115396406A (en) * | 2022-07-07 | 2022-11-25 | 百果园技术(新加坡)有限公司 | Audio code rate adjusting method and system for audio and video transmission |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070263725A1 (en) * | 2006-03-23 | 2007-11-15 | Kei Matsubayashi | Video signal coding system and method of coding video signal for network transmission, video output apparatus, and signal conversion apparatus |
| CN101198047A (en) * | 2007-12-26 | 2008-06-11 | 腾讯科技（深圳）有限公司 | Television broadcasting method and system |
| CN102014286A (en) * | 2010-12-21 | 2011-04-13 | 广东威创视讯科技股份有限公司 | Video coding and decoding method and device |
| CN103269448A (en) * | 2013-05-24 | 2013-08-28 | 浙江工商大学 | Method for achieving synchronization of audio and video on the basis of RTP/RTCP feedback early-warning algorithm |
-
2013
- 2013-11-29 CN CN201310631832.9A patent/CN104683823A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070263725A1 (en) * | 2006-03-23 | 2007-11-15 | Kei Matsubayashi | Video signal coding system and method of coding video signal for network transmission, video output apparatus, and signal conversion apparatus |
| CN101198047A (en) * | 2007-12-26 | 2008-06-11 | 腾讯科技（深圳）有限公司 | Television broadcasting method and system |
| CN102014286A (en) * | 2010-12-21 | 2011-04-13 | 广东威创视讯科技股份有限公司 | Video coding and decoding method and device |
| CN103269448A (en) * | 2013-05-24 | 2013-08-28 | 浙江工商大学 | Method for achieving synchronization of audio and video on the basis of RTP/RTCP feedback early-warning algorithm |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108304753A (en) * | 2017-01-24 | 2018-07-20 | 腾讯科技（深圳）有限公司 | Video communication method and video communication device |
| CN106982346A (en) * | 2017-04-07 | 2017-07-25 | 上海良讯科技股份有限公司 | Multi-screen telepresence system and method |
| CN109120974A (en) * | 2018-07-25 | 2019-01-01 | 深圳市异度信息产业有限公司 | A kind of method and device that audio-visual synchronization plays |
| CN109361945A (en) * | 2018-10-18 | 2019-02-19 | 广州市保伦电子有限公司 | The meeting audiovisual system and its control method of a kind of quick transmission and synchronization |
| CN113347468A (en) * | 2021-04-21 | 2021-09-03 | 深圳市乐美客视云科技有限公司 | Audio and video transmission method and device based on Ethernet frame and storage medium |
| CN113347468B (en) * | 2021-04-21 | 2023-01-13 | 深圳市乐美客视云科技有限公司 | Audio and video transmission method and device based on Ethernet frame and storage medium |
| CN114143584A (en) * | 2021-09-29 | 2022-03-04 | 杭州当虹科技股份有限公司 | Broadcasting system and method for realizing synchronous broadcasting among multiple terminals |
| CN114143584B (en) * | 2021-09-29 | 2024-03-26 | 杭州当虹科技股份有限公司 | Broadcasting system and method for realizing synchronous broadcasting among multiple terminals |
| CN115396406A (en) * | 2022-07-07 | 2022-11-25 | 百果园技术(新加坡)有限公司 | Audio code rate adjusting method and system for audio and video transmission |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104683823A (en) | Multi-screen linked audio and video synchronizing system | |
| CN114503599B (en) | Supporting video and audio data using extensions in GLTF < 2 > scene descriptions | |
| CN100568966C (en) | A kind of network transfer method of audio/video data | |
| CN102123303B (en) | Audio/video file playing method and system as well as transmission control device | |
| JP6049728B2 (en) | Network streaming of encoded video data | |
| CN105900445B (en) | The method and apparatus of steady live operation for dynamic self-adapting stream transmission | |
| JP5728736B2 (en) | Audio splitting at codec applicable frame size | |
| CN109327728A (en) | One kind is one-to-many to shield together methods, devices and systems, with screen equipment and storage medium | |
| CN104618786A (en) | Audio/video synchronization method and device | |
| CN107005729A (en) | The coffret transmitted for multimedia and file | |
| CN102325181B (en) | Instant audio/video interactive communication method based on sharing service and instant audio/video interactive communication system based on sharing service | |
| CN107370714A (en) | The high efficiency communication method that facing cloud renders | |
| CN105407361A (en) | Audio and video live broadcast data processing method and device | |
| CN101895750A (en) | Set-top box and PC-oriented real-time streaming media server and working method | |
| CN202759552U (en) | Multi-terminal video synchronous playing system based on IP network | |
| CN114554277B (en) | Multimedia processing method, device, server and computer readable storage medium | |
| CN202907119U (en) | Digital television terminal with WIFI display function | |
| CN105430451A (en) | Multi-cam HLS description method and multi-cam video direct broadcasting system based on HLS | |
| CN108494792A (en) | A kind of flash player plays the converting system and its working method of hls video flowings | |
| CN111263217A (en) | Wireless screen projection method, wireless screen projection system and computer readable storage medium thereof | |
| CN109040818A (en) | Audio and video synchronization method, storage medium, electronic equipment and system when live streaming | |
| CN105376595A (en) | Video mixing encoding system and method | |
| CN103974062B (en) | Image display device, image display system and method for displaying image | |
| CN108665749A (en) | The display device and multimedia education system of multimedia education system under cloud desktop | |
| CN103269448A (en) | Method for achieving synchronization of audio and video on the basis of RTP/RTCP feedback early-warning algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150603 |