JP2006054921A - Method of transmitting video signal, method of receiving video signal, and video-signal transmission/reception system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display a synthetic video image required by a user at each reception terminal to reduce the cost, space, and power consumption of a camera side in a network image transmission system that provides a wide dynamic-range video image. <P>SOLUTION: At the camera side, a long-time-exposure video image and a short-time-exposure video image are separately compressed and transmitted, while in the reception-terminal side, the transmitted data are separately expanded and synthesized at an arbitrary ratio to display an video image of a wide dynamic range. Therefore, it is not necessary for the camera side to have a function for synthesizing the long-time-exposure video image and the short-time-exposure video image and a function for transmitting a plurality of video image data that meet a plurality of reception terminals to respective reception terminals, thereby reducing the cost, space, and power consumption of the camera side. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a network image transmission system that provides images from a plurality of surveillance cameras to a plurality of receiving terminals, and more particularly to a transmission method, a reception method, and a transmission / reception system that provide images with a wide dynamic range.

  An imaging apparatus has been developed that can clearly capture even a subject in which extremely bright and dark images are mixed. In such an imaging apparatus, a video signal captured with a long exposure time (hereinafter referred to as a long exposure video signal) and a video signal captured with a short exposure time (hereinafter referred to as a short exposure video signal) for one subject. Both of them can be output. From the long exposure video signal, a clear image of a dark area of the subject can be obtained, and from the short exposure video signal, a clear image of a bright area can be obtained.

  In a conventional imaging device such as a television camera, when generating a wide dynamic range image, the imaging device itself synthesizes the long exposure video signal and the short exposure video signal at a predetermined ratio, and images the synthesized video signal. Output from the device. From the synthesized video signal, an image in which the dark area and the bright area of the subject are clearly reflected can be obtained. Hereinafter, an image sensor that outputs such a long-exposure video signal and a short-exposure video signal will be referred to herein as a wide dynamic image sensor.

  Such a wide dynamic range imaging device (wide dynamic television camera) is described in, for example, prior patent documents 1 to 5. In the techniques disclosed in Patent Documents 1 to 5 described above, a wide dynamic imaging device is used to realize a wide dynamic television camera.

  However, a long exposure video signal and a short exposure video signal can be output for each frame even if a general CCD or other image sensor is used without using a wide dynamic image sensor. For example, acquisition of long exposure video signals transfers the charge accumulated for each pixel of the image sensor during the normal exposure time of a general image sensor (for example, the time excluding the blanking signal of the vertical synchronization signal). It is realized by doing. The acquisition of the short-time exposure video signal is performed by discharging the charge accumulated for each pixel of the image sensor using the electronic shutter function of the image sensor while the normal exposure time elapses. This can be realized by transferring the charge accumulated up to the exposure time, and the exposure time can be arbitrarily changed.

  Since the conventional wide dynamic television camera has a complicated video composition circuit in the camera itself, the size of the camera increases, the power consumption increases, and the cost increases. This is a disadvantageous condition when used for a camera of a video surveillance system.

Further, the conventional wide dynamic television camera outputs a signal obtained by combining the long long exposure video signal and the short long exposure video signal at a predetermined fixed composition ratio. On the monitor device (receiving terminal) side that receives such a composite signal, the long long exposure video signal and the short long exposure video signal cannot be separated. When a composite signal from a conventional wide dynamic television camera is transmitted to a plurality of receiving terminals via a communication network, the same composite signal with the same composition ratio (composition ratio) is received by the monitor device on any of the reception terminals. Therefore, the composite video displayed on a display device such as a monitor or a display is the same in any receiving terminal. Therefore, the conventional wide dynamic television camera cannot meet the user's request on the individual video monitor (reception terminal) side, such as wanting to clearly see the dark part of the subject or observing the bright part more clearly.
Japanese Patent Laid-Open No. 2001-094870 (2nd page, FIG. 5) JP 2001-094999 A (2nd page, FIG. 10) JP 2000-350220 A (page 2-4, FIG. 1-5) Japanese Unexamined Patent Publication No. 2001-094871 (2nd page, FIG. 8-10) Japanese Patent Laid-Open No. 2001-094772 (second page, FIG. 8-10)

  SUMMARY OF THE INVENTION An object of the present invention is to eliminate the above-mentioned conventional drawbacks and to use a video camera having a low dynamic cost, size, and power consumption, and a video signal transmission / reception method and system having a wide dynamic range. Is to provide. Another object of the present invention is to provide a wide dynamic range video signal transmission / reception method and video transmission / reception system capable of realizing advanced synthesis processing on the receiving terminal side, which cannot be processed inside the camera. It is in. Still another object of the present invention is to eliminate the above-mentioned conventional drawbacks and to display a video signal transmission / reception method and video transmission / reception system with a wide dynamic range capable of displaying video with different composition ratios for each receiving terminal. Is to provide.

  In order to achieve the above object, in the video signal transmission / reception system of the present invention, a transmission device and a reception device are coupled by a signal path. The transmission device includes an imaging unit that outputs first and second video signals obtained by photographing a subject with different exposure times, a compression unit that compresses data of the first and second video signals, and a first unit A transmission interface unit that adds a signal for identifying a transmission device and a frame number to the compressed signals of the first and second video signals, and transmits the signal to a signal path; The receiving apparatus includes a reception interface unit that receives the compressed video signal from the signal path, a decompression unit that decompresses the received compressed video signal and reproduces the first and second video signals, and the reproduced first and second video signals. And a synthesis unit that synthesizes the second video signal with an arbitrary synthesis ratio (synthesis ratio).

  According to the present invention having the above characteristics, the image signal combining unit or combining step is arranged on the receiving device side that receives the image signal, displays the image, and observes it. Therefore, the television camera is compact and has low power consumption and low cost. Furthermore, the long-exposure video signal and the short-exposure video signal can be synthesized and output at an arbitrary composition ratio on the reception device (reception terminal) side. It is possible to respond to individual requests on the individual video monitor device (reception terminal) side such as wanting to clearly see a dark part or observing a bright part more clearly, thereby improving usability.

  That is, the video signal transmission method of the present invention is a network system in which a plurality of cameras and a plurality of receiving terminals are connected via a network, and the plurality of cameras are respectively connected to a long exposure video and a short exposure. The video is picked up separately, the long exposure video and the short exposure video taken are separately compressed, and the compressed long exposure video and the compressed short exposure video are compressed in the network. It is transmitted via the network.

  Also, the video signal receiving method of the present invention separately captures a long exposure video and a short exposure video, compresses the captured long exposure video and the short exposure video separately, and A network system comprising a plurality of cameras for transmitting the long-time exposure video and the compressed short-time exposure video via the network and a plurality of receiving terminals connected via the network The plurality of receiving terminals respectively receive the compressed long-exposure video and the compressed short-exposure video transmitted over the network, and the received long-exposure video and the short-exposure video received. Are expanded separately, the expanded long exposure video and the extended short exposure video are combined, and the combined image is displayed.

  The video signal transmission / reception system of the present invention is a network system in which a plurality of cameras and a plurality of receiving terminals are connected via a network, wherein the plurality of cameras are respectively a long exposure video and a short exposure. A camera unit that separately captures the video, a compression unit that separately compresses the captured long-exposure video and the short-exposure video, and the compressed long-exposure video and the compressed short-time A network interface unit that transmits the exposure video via the network.

  The video signal transmission / reception system of the present invention separately captures a long exposure video and a short exposure video, compresses the captured long exposure video and the short exposure video separately, and In a video monitoring system comprising a plurality of cameras that transmit the long-time exposure video and the compressed short-time exposure video via the network network and a plurality of receiving terminals connected via the network network, Each of the plurality of receiving terminals receives the compressed long-exposure video and the compressed short-exposure video transmitted through the network, the received long-exposure video and the short-exposure video, respectively. An image decompression unit that separately decompresses the time exposure video, an image composition unit that synthesizes the decompressed long exposure video and the decompressed short exposure video, and the composite And a display unit for displaying an image, to display an image of wide dynamic range.

  According to the present invention, the function of synthesizing the long-exposure video and the short-exposure video on the camera side becomes unnecessary, so that the cost, space, and power consumption on the camera side can be reduced. Furthermore, since the function of transmitting a plurality of pieces of video information corresponding to a plurality of receiving terminals to each receiving terminal becomes unnecessary on the camera side, the cost, space, and power consumption on the camera side can be reduced. In addition, it is possible to synthesize the images of one camera at different ratios depending on the personal computer, and it is possible to set a fine composition ratio so that the monitor can easily see the part he wants to see.

An embodiment of the video surveillance system of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of an embodiment when a transmission / reception system using a wide dynamic imaging apparatus according to the present invention is applied to a video surveillance system using a communication network. 10 to 15 are surveillance cameras installed at each surveillance location, and each has an internal configuration of a block indicated by 10. Reference numerals 20 to 25 are receiving terminals, and each has an internal configuration of a block indicated by 20. The plurality of monitoring cameras 10 to 15 and the plurality of receiving terminals 20 to 25 are coupled to each other via a communication network 30 such as the Internet. The long-exposure video signal and the short-exposure video signal obtained by imaging with any of the monitoring cameras 10 to 15 are transmitted to at least one of the receiving terminals 20 to 25 via the communication network 30. The receiving terminals 20 to 25 are, for example, personal computers.

  Hereinafter, the embodiment will be described by taking the case of video signal transmission / reception between the monitoring camera 10 and the receiving terminal 20 as an example. In FIG. 1, the number of surveillance cameras and receiving terminals is the same, and there are six sets of them. However, the number is not necessarily the same, and any number is possible. That is, for example, a system in which the cameras 10 to 1m (m is a natural number of 1 or more) and the receiving terminals 20 to 2n (n is a natural number of 1 or more) can be constructed by the communication network 30, and m and n Since it can be arbitrarily connected to or disconnected from the network 30, it is not constant in time. The communication network 30 may be either a wireless network or a wired network. For example, third-generation mobile phone networks such as LAN (Local Area Network), xDSL (x Digital SubscriberLine), CATV Internet (Cable Television), or W-CDMA (Wide-Code Division Multiple Access) can be used. Or a combination of these different networks.

  A surveillance camera 10 shown in FIG. 1 includes a dynamic range imaging device 101 including an optical system that guides light from a subject, an A / D conversion circuit 102 that converts an analog video signal from the dynamic range imaging device 101 into a digital signal, A memory device 103 capable of storing digital image data for one frame from the A / D conversion circuit 102; a compression unit 104 such as a JPEG encoder for compressing the digital image data read from the memory device 103; A memory device 105 that stores the output compressed image data (compressed image signal), and the image data read out from the memory device 105 are processed so as to conform to the communication protocol of the communication network 30, for example, TCP / IP. For example, the network interface 106 that outputs to the communication network 30 and the control inside the surveillance camera 10 And a control unit 107, such as a processor. That is, the control unit 107 controls the imaging device 101 to output a short exposure video signal and a long exposure video signal, compresses the digital video signal to a predetermined compression ratio according to the transmission rate of the communication network 30, for example, compresses it. The compressor 104 is controlled to add an identification signal to the digital signal data. The network interface 106 is capable of bidirectional communication, and can receive a signal from the communication network 30 and the control unit 107 can control the operation of the monitoring camera 10 according to the received signal.

  The receiving terminal 20 receives a signal from the communication network 30, converts it into a signal format that can be processed inside the receiving terminal 20, and outputs it, and a memory device 202 that stores the signal from the network interface 201, The decompression unit 203 such as a JPEG decoder for performing the decompression process on the signal data read from the memory device 202, and the short-exposure video signal and the long-exposure video signal are sorted according to the selection signal 208. A selector unit 204 that outputs to the combining unit 205, and a combining unit 205 that combines both signals at a combination ratio (composition ratio) set for each of the short exposure video signal and the long exposure video signal; A display unit 206 that receives the synthesized video signal from the synthesis unit 205 and displays it as a video, and a decompression unit 203 for decompressing the video signal data. Settings for detecting an identification signal to specify whether it is a short exposure video signal or a long exposure video signal, generating a selection signal 208, and setting a composition ratio K, K ′ (described later) A control unit 207 such as a microprocessor for generating the signal 209 and controlling the inside of the receiving terminal 20 is provided.

  The synthesizing unit 205 stores the short-exposure video signal and the long-exposure video signal from the selector 204, respectively, and the signals read from the memory devices 210 and 211, respectively, and the synthesis ratios K and K ′. Multipliers 212 and 213, and an adder 214 that adds and outputs the signals multiplied by the synthesis ratio. In order to increase the speed, for example, the memory devices 210 and 211 each include two sets. For example, when one of the memory devices 210 and 211 is reading, the other memory device 210 and 211 performs writing. In some cases, a method is adopted.

  Next, the operation of the monitoring camera 10 will be described in more detail with reference to the time chart of FIG. In addition, the signal delay which generate | occur | produces between each apparatus between Fig.2 (a) (b) (c) is not shown in figure. As shown in FIG. 2 (a), the image sensor, for example, a CCD repeats a short exposure of 1/1000 seconds and a long exposure of 1/60 seconds, for example, and the image pickup device 101 receives a frame image unit. The short exposure video signal S and the long exposure video signal L are alternately output. Here, one frame is, for example, 1/30 second.

  The short exposure video signal and the long exposure video signal are converted into digital signals by the A / D converter 102. Then, the digital image data for each frame is stored in the memory device 103 as shown in FIG. The short-exposure and long-exposure digital image signals (image data) read from the memory device 103 are respectively compressed by the compression unit 104, for example, about 1/10 of the original data amount as shown in FIG. The image data is compressed into image data having a compression rate of. However, the value of this compression rate is determined as appropriate, but generally, it often depends on the transmission speed of the network 30 to be connected.

  At this time, identification signals S1, S2,..., L1, L2,... Are added to the compressed short exposure video signal and long exposure video signal. The identification signals S 1, S 2,..., L 1, L 2,... Further include information for specifying the monitoring camera 10 (for example, IP address, etc.) and a short exposure video signal. Information (for example, information indicating a frame number) or the like for distinguishing whether it is a short-time exposure video signal. The identification signals S 1, S 2,..., L 1, L 2,... Can be generated by the control unit 107 in synchronization with the operation switching of the imaging device 101. The compressed image data from the compression unit 104 is temporarily stored in the memory device 105. The memory device 105 can store, for example, a long exposure video signal and a short exposure video signal for 30 frames each, and the video signal output from the compression unit 104 is stored in the data of the stored oldest video signal video signal. It is updated by overwriting. When the memory device 105 receives a transmission request from one of the receiving terminals 20 to 25, the identification signals S1, S2,..., L1 read from the memory device 105 at a predetermined read timing. , L2,... Are added to the communication network 30 as shown in FIG. 3 according to the network communication protocol via the network interface 106 and the compressed short exposure video signal and long exposure video signal. Output alternately.

  Next, the operation of the receiving terminal 20 will be described in more detail with reference to the signal time chart of FIG. Note that signal delays occurring between the devices in FIGS. 4 (a), (b), (c), (b '), and (d) are not shown. The network interface 201 receives from the communication network 30 a compressed signal in which the short-exposure video signal and the long-exposure video signal that are transmitted from any of the monitoring cameras 10 to 15 are alternately continued. The short exposure video signals S1, S2, S3,... And the long exposure video signals L1, L2, L3,. 3 (a)). The decompression unit 203 performs decompression processing on the signal (data) read from the memory device 202 to obtain the original short-time exposure video signals S1, S2, S3,. , L2, L3,... Are reproduced (FIG. 3 (b)). The control device 207 generates a selection signal 208 for distinguishing between the short exposure video signal and the long exposure video signal based on the identification signal detected by the expansion unit 203. The selection unit 204 sorts the short-exposure video signal and the long-exposure video signal in accordance with the selection signal 208 and stores them in separate memory devices 210 and 211, respectively.

  The synthesizing unit 205 multiplies the short-time exposure video signal and long-time exposure video signal read from the memory devices 210 and 211 by the set synthesis ratios K and K ′, respectively, and multiplies the short-time exposure video signal and the long-time exposure video signal. The time exposure video signal and the long exposure video signal are added. The composition ratio of the short exposure video signal and the long exposure video signal can be arbitrarily set by a setting signal 209 from the control unit 207. For example, if no change is specified by the control unit 207, the initial value (default value) is a short exposure video signal ratio K = 0.5 and a long exposure video signal ratio K ′ = 0.5. Works. If the user instructs the control unit 207 to change the setting while looking at the display unit 206, the control unit 207 instructs the synthesis unit 205 to set the values of the synthesis ratios K and K ′ (K + K ′ = 1). A setting signal 209 instructing the change may be given. The display unit 206 receives the video signal of the synthesized image output from the synthesis unit 205 and displays it as a video.

  In this way, the receiving terminal 20 can synthesize and output the received long-exposure video signal and short-exposure video signal at an arbitrary composition ratio, so that the dark part of the subject can be observed more clearly. In the case where the long exposure video signal composition ratio K is set to a larger value and the bright part is to be seen more clearly, the long exposure video signal composition ratio K ′ can be set to a larger value. Therefore, according to the area to be monitored and the purpose of monitoring, it is possible to adapt to individual requests on the receiving terminals 20 to 25 side, and the usability as a monitoring system is improved. That is, since the synthesizing process is performed by the receiving terminals 20 to 25, it is possible to synthesize one camera image at a different ratio for each receiving terminal. Therefore, it is possible to display a wide dynamic range image required by the supervisor for each receiving terminal.

  Further, the receiving terminal (for example, the receiving terminal 20) can select any of the monitoring cameras 10 to 15 and view the video of the selected monitoring camera. In that case, identification information (namely, IP address, etc.) of the monitoring camera designated by the user and a video signal transmission request are given to the network interface 201. The network interface 201 transmits the transmission request to the designated surveillance camera designated via the communication network 30. The monitoring camera that has received the transmission request can also transmit the long exposure video signal and the compressed data of the short exposure video signal to the receiving terminal 20 that has requested the transmission data.

  Here, the readout method of the long exposure video signal and the short exposure video signal from the memory devices 210 and 211 in the synthesizing unit 205 will be described with reference to FIGS. 4 (b) (c) and (b) ′ (d). I will explain. 4 (b) and 4 (b) 'are the same figure and are drawn so that the relationship between (c) and (d) can be understood. FIG. 4 (b), (b)' and FIG. 4 (c) In (d) and (d), the frame image signal of the long-exposure video signal is represented by Ln (where n is a frame number and n = 1, 2, 3,..., N, where N is a natural number). The frame image signal of the short-exposure video signal is represented by Sn (n indicates a frame number, n = 1, 2, 3,..., N, where N is a natural number).

  In the case of combining by the reading method of FIG. 4C, the combining unit 205 combines and outputs Ln and Sn of the same frame number read from the memory devices 210 and 211. 4D, the combining unit 205 combines the frame numbers Ln-1 and Sn read from the memory devices 210 and 211, and then combines the frame numbers Sn and Ln. Next, the frame numbers Ln and Sn + 1 are combined and output sequentially after combining. That is, at the time of reading, the long exposure video signal Ln and the short exposure video signal Sn having the same frame number are always read twice in succession. In the method of FIG. 4 (d), the frame rate is doubled compared to the method of FIG. 4 (c) and the same frame rate as the video signal on the original surveillance camera side. The movement becomes smooth. In the method of FIG. 4 (d), the output frame is output at half the reading speed of the method of FIG. 4 (c).

  In the embodiment described above, the signal compression and decompression has been described using the JPEG system as an example. However, the present invention can also be applied to the MPEG system. FIG. 5 shows the configuration of the compression unit 140 of the surveillance camera that transmits an MPEG image. Since the configuration other than the compression unit 140 and the memory device 130 is basically the same as that of the monitoring camera 10 in FIG. The long exposure video signal and the short exposure video signal output from the A / D converter 102 are stored separately in separate memory areas 131 and 132 of the memory device 130, respectively.

  For example, an MPEG picture signal consists of an I picture (Inter Picture), a P picture (Predictive Picture), and a B picture (Biderectionary Predictive Picture). The I picture is encoded based on the frame data itself, and the P picture is the previous picture. The B picture is information that is encoded with the difference from the data and encoded with the difference from the previous and subsequent image data. Since it contains these, it cannot be processed in units of frames. And the short-time exposure video signal must be individually compressed by the compression unit. The compression unit 140 includes MPEG encoders 141 and 142 for individually compressing the long exposure video signal and the short exposure video signal. The compressed long exposure video signal and the short exposure video signal are temporarily stored in the memory devices 143 and 144, respectively. Then, the switch unit 145 alternately outputs the long exposure video signal and the short exposure video signal compressed from the memory devices 143 and 144 and supplies them to the network interface 106.

  On the receiving terminal side, the compressed long exposure video signal and the short exposure video signal are received, separated, individually decompressed, and combined as in the case of FIG. Since the synthesizing process is performed by the receiving terminals 20 to 25, it is possible to synthesize at different ratios depending on the receiving terminal based on the video from one monitoring camera. Therefore, it is possible to display a composite video required by the supervisor (user) for each receiving terminal. Further, by using a device such as a personal computer that can be individually set by the user for each receiving terminal as the receiving terminal, it is possible to perform high-level synthesis that could not be processed inside the surveillance camera.

The block diagram which shows the structure of one Example of the video surveillance system of this invention. The time chart for demonstrating an example of the operation | movement by the surveillance camera side in the video surveillance system of FIG. The time chart for demonstrating an example of the operation | movement by the surveillance camera side in the video surveillance system of FIG. The time chart for demonstrating an example of the operation | movement by the side of the receiving terminal in the video surveillance system of FIG. The block diagram which shows the structure of one Example of the surveillance camera side at the time of applying the image signal of an MPEG system to the system of this invention.

Explanation of symbols

  10, 11-15: Surveillance camera, 20-25: Receiving terminal, 30: Communication network, 101: Dynamic range imaging device, 102: A / D conversion circuit, 103, 105, 130: Memory device, 104, 140: Compression Section, 106, 201: network interface, 107, 207: control section, 203: expansion section, 204: selector section, 205: composition section, 206: display section, 208: selection signal, 209: setting signal, 210, 211: Memory device 212, 213: Multiplier 214: Adder 131, 132, 143, 144: Memory area 141, 142: MPEG encoder 145: Switch unit

Claims (2)

Two images with different exposure times are alternately imaged, an identification signal for distinguishing the two images with different exposure times is added to each of the two images with different exposure times, and compression is performed for each frame. In a network system comprising a plurality of cameras that alternately transmit two images having different exposure times through the network and a plurality of receiving terminals connected through the network.
Each of the plurality of receiving terminals is
The network-transmitted two images with different compressed exposure times are received, the received two images with different compressed exposure times are decompressed, and the two images with different expanded exposure times are received. Based on the identification signal, the stored two images having different exposure times are alternately read out twice, the two images having different exposure times are combined for each frame, and the combined image A video signal receiving method characterized by displaying. Two images with different exposure times are alternately imaged, an identification signal for distinguishing the two images with different exposure times is added to each of the two images with different exposure times, and compression is performed for each frame. In a video signal transmission system comprising a plurality of cameras that alternately transmit two videos having different exposure times via the network network and a plurality of receiving terminals connected via the network network,
Each of the plurality of receiving terminals is
The network interface unit that receives the two images having different compressed exposure times transmitted through the network, the image decompressing unit that decompresses the two images that have received different compressed exposure times, and the decompressed A storage unit that stores two images having different exposure times based on the identification signal, and a control unit that alternately reads two images having different exposure times stored in the storage unit, and An image signal transmitting / receiving system comprising: an image combining unit that combines two images with different exposure times read from the storage unit for each frame; and a display unit that displays the combined image.

Images