JP2022098663A - Monitoring system, abnormality sensing detection method of monitoring system, and abnormality sensing detection program of monitoring system - Google Patents

Abstract

To provide a monitoring system that can accurately perform abnormality sensing such as left-behind sensing or intrusion sensing.SOLUTION: A monitoring system 1 includes an image processing server 20 having: video receiving units 2(1) to 2(n) that receive image information from a camera 2; and an image processing unit 22 that calculates an amount of angle of view deviation of the camera 2 based on the image information from the video receiving units 2(1) to 2(n) in a mask region, which is a set range of image processing, obtains a corrected mask region from the calculated amount of angle of view deviation, analyzes the image information from the video receiving units 2(1) to 2(n) in the obtained corrected mask region, and detects abnormality sensing such as left-behind sensing or intrusion sensing.SELECTED DRAWING: Figure 2

Description

In the present disclosure, a mask area is set for the image information from a security camera or a surveillance camera for the purpose of improving the accuracy of the image processing result and reducing the calculation cost, and the image information in the mask area is image-processed to detect the leftover. Or related to a monitoring system that detects abnormalities such as intrusion detection.

Patent Document 1 shows a surveillance camera system that detects and tracks a person from an image taken by a camera.
This Patent Document 1 shows a camera setting device suitable for use in a surveillance camera system, and when a builder installs a camera, the number of times a face is detected for a tracked person in a temporarily installed state. , The optimum camera setting device that uses any one of the face orientation, the angle of view, and the movement line as the visitor information, estimates whether the camera installation condition is suitable, and prompts the camera setting change is shown.

Japanese Patent No. 5958716

However, in the conventional surveillance camera system, after the camera is installed, the angle of view of the camera body shifts, which is difficult for the human eye to see, due to disturbances such as vibration or deterioration over time, and the mask area deviates from the expected area. Therefore, unless maintenance is performed on the camera, there arises a problem that abnormality detection such as abandonment detection or intrusion detection cannot be performed with high accuracy.

The present disclosure has been made in view of the above points, and it is desired to obtain a monitoring system capable of correcting the mask area for the angle of view deviation of the camera body and detecting an abnormality such as abandonment detection or intrusion detection with high accuracy. The purpose.

The monitoring system according to the present disclosure determines the amount of angle of view deviation of the camera based on the image information from the image receiving unit that receives the image information from the camera and the image receiving unit in the mask area that is the range of the set image processing. Image processing that performs calculation, obtains a correction mask area from the calculated angle of view deviation amount, analyzes image information from the video receiving unit in the obtained correction mask area, and detects abnormality detection such as abandonment detection or intrusion detection. It is provided with an image processing server having a unit.

According to the present disclosure, it is possible to obtain a monitoring system capable of accurately detecting anomalies such as abandonment detection or intrusion detection.

It is a system block diagram of the monitoring system which concerns on Embodiment 1. FIG. It is a block diagram for demonstrating the function of the monitoring system which concerns on Embodiment 1. FIG. It is a figure which shows an example of the camera list table in the monitoring system which concerns on Embodiment 1. FIG. It is a figure which shows an example of the event list table managed by the image processing server in the monitoring system which concerns on Embodiment 1. FIG. It is a figure which shows an example of the angle of view deviation camera list table managed by the image processing server in the monitoring system which concerns on Embodiment 1. FIG. It is a figure which shows an example of the display example displayed on the display screen of the monitoring terminal apparatus in the monitoring system which concerns on Embodiment 1. FIG. It is a flowchart which shows the operation of the image processing part in the monitoring system which concerns on Embodiment 1. FIG. It is a system block diagram of the monitoring system which concerns on Embodiment 2. FIG.

Embodiment 1.
The monitoring system 1 according to the first embodiment will be described with reference to FIGS. 1 to 7.
As shown in FIG. 1, a security camera or a surveillance camera (hereinafter collectively referred to as a camera) 2 (1), which is a plurality of network cameras, to a camera 2 (n) in a station yard or a place to be monitored such as a store. Will be installed.
In the following description, in particular, when it is not necessary to specify and explain the individual cameras 2 (1) to 2 (n), the reference numerals in () will be omitted in order to avoid complexity. ..

The camera 2 displays the captured image in JPEG or H.I. It is encoded and compressed into a digital video such as 264, converted into an RTP packet by a video transfer protocol such as RTP (Real-time Transport Protocol, RFC3551), and output as image information.
The image information output from the camera 2 is transmitted on the network by multicast distribution via the network switch 3, and is transmitted to the monitoring system 1 via the network.

The monitoring system 1 receives image information from a plurality of cameras 2 distributed via the network switch 3 and performs image processing on the image information for each camera 2, that is, analyzes, analyzes, and leaves the image information. Anomaly detection such as detection or intrusion detection, kerosene can detection with a kerosene can, or long object detection with a long object is detected, and when the abnormality detection is detected, the abnormality detection information is notified.
The monitoring system 1 includes a video recording device 10, an image processing server 20, and a monitoring terminal device 30.
The image recording device 10, the image processing server 20, and the monitoring terminal device 30 simultaneously distribute the image information output from the camera 2 by multicast distribution.

As shown in FIG. 2, the image processing server 20 and the monitoring terminal device 30 store the camera list table 40, the video recording device list table 50, and the threshold value list 60 as shared information.
As shown in FIG. 3, the camera list table 40 is information associated with a "camera number", a "camera name", an "installation location name", a "map", a "map position", and a "code in the figure". It is a table of.
In the monitoring system 1, the cameras 2 (1) to 2 (n) are specified by the "camera number" of the camera list table 40, and the camera information is referred to from the camera list table 40 by using the "camera number".

The "camera number" represents an index number uniquely determined in this monitoring system. For example, in order to distinguish it from the sensor, it is represented by a total of 5 digits, which is a combination of the installation floor and the 3-digit index number, with "C" at the head for convenience. In the system, the camera information is referred from the camera list table 40 using the camera number.
The "camera name" and "installation location name" represent the camera names displayed on the GUI (Graphical User Interface) screen of the monitoring terminal device 30.

The video recording device 10 records the received image information from the camera 2 in association with the camera number of the camera 2 and the time based on the camera number of the camera list table 40 shown in FIG. The video recording device 10 is equipped with a large-capacity recording device such as a hard disk or an SSD (Solid State Drive) device.

As shown in FIG. 2, the image processing server 20 is provided corresponding to each of the cameras 2 (1) to 2 (n), and the image information from the corresponding cameras 2 (1) to 2 (n) is provided. The image information from the plurality of image receiving units 21 (1) to 21 (n) for receiving the image and the image information from the image receiving units 21 (1) to 21 (n) in the mask area which is the set image processing range is analyzed. The image angle deviation amount of the target cameras 2 (1) to 2 (n) is calculated, a correction mask area is obtained from the calculated angle of view deviation amount, and the image receiving unit 21 (in the obtained correction mask area) ( 1) It has an image processing unit 22 that reanalyzes the image information from 21 (n) and re-detects an abnormality detection such as abandonment detection or intrusion detection, kerosene can detection, or long object detection.
In the following description, in particular, when it is not necessary to specify and explain the individual video receiving units 21 (1) to 21 (n), the reference numerals shown in () are used to avoid complexity. The explanation will be omitted.

The image processing server 20 analyzes the image information from the video receiving unit 21 in the mask area within the set image processing range based on the received image information from the camera 2, and detects the abandonment or intrusion, or kerosene. The first function of detecting an abnormality such as can detection or long object detection and the image information from the image receiving unit 21 in the mask area are analyzed, the amount of angle of view deviation of the camera 2 is calculated, and the calculation is performed. It has a second function of obtaining a correction mask area from the amount of deviation in the angle of view.

The second function of the image processing server 20 is to calculate the amount of angle of view deviation by using the received image information from the camera 2 and the past video acquired from the video recording device 10 when the abnormality detection is detected. This is a function to obtain an updated correction mask area by performing coordinate conversion of the mask area set by the amount of angle of view deviation.
The first function of the image processing server 20 performs image processing based on the set mask area when the amount of angle of view deviation is less than the threshold value for the camera number shown in the threshold value list 60 by the second function. It is a function to detect an abnormality detection, and if it is equal to or more than a threshold value, perform image processing based on the correction mask area obtained by the second function to detect the abnormality detection.
The updated correction mask area will be the set mask area from the next time onward.

The first function of the image processing server 20 further states that if the angle of view deviation amount is equal to or greater than the second threshold value larger than the above threshold value due to the second function, automatic adjustment for the angle of view deviation amount is not possible. It has a function to output the angle of view deviation alarm information.
The second threshold value is a value at which it is determined that the camera 2 itself needs to be adjusted when a large vibration, for example, some kind of impact or an angle change due to mischief is given to the camera 2.
The second threshold value is also stored in the threshold value list 60 as a second threshold value for the camera number, similarly to the threshold value.

The image processing server 20 is a server-type computer, and has a high-speed and highly parallel CPU (Central Processing Unit), a large-capacity semiconductor memory (RAM: Random Access Memory), and non-volatile recording of a hard disk device or SSD device. It is composed of a storage device (ROM: Read only memory) such as a device, loads a program stored in the ROM into the RAM, and the CPU executes various processes based on the program loaded in the RAM. The image processing server 20 is driven by a general-purpose OS (Operating System).
It is also possible to distribute the processing load by configuring the image processing server 20 with a plurality of computers according to the magnitude of the processing load. Further, by finely dividing the division of processing contents, it is possible to perform a plurality of complicated image processings in parallel in the monitoring system 1.

As shown in the functional block diagram of FIG. 2, the image processing server 20 has each functional block which is a unit of processing as a processing function. Each functional block is a task or program or thread unit managed by the OS. In addition, data is transmitted / received between the processing unit and the processing unit that are linked by the shared memory interface of the OS or the socket interface via the network.

The functional blocks of the image processing server 20 include video receiving units 21 (1) to 21 (n), image conversion units 23 (1) to 23 (n), and recorded video receiving units 24 (1) to 24 (n). ), An image processing unit 22, an event storage unit 25, and an event notification unit 26.
The image processing unit 22 includes a current image processing unit 221, an angle of view deviation detecting unit 222, a mask area correction unit 223, and a re-image processing unit 224.

Each of the video receiving units 21 (1) to 21 (n) is sequentially assigned image information from the cameras 2 (1) to 2 (n) based on the camera numbers of the camera list table 40 shown in FIG. , It operates corresponding to each of the cameras 2 (1) and 2 (n).
The video receiving unit 21 receives the RTP packet delivered from the camera 2, extracts video data from the payload of the RTP packet, and temporarily buffers it in a memory (not shown). The buffered video data is, for example, in the case of JPEG, in frame units, H.I. If it is 264, a continuous data group (GOP: Group Of Picture) of related frames starting from the I frame is extracted as image information, and the corresponding image conversion units 21 (1) to 21 (n) correspond to each video receiving unit 21 (1) to 21 (n). It is sent as image information to 23 (1) to 23 (n).

Each of the image conversion units 23 (1) to 23 (n) is paired with each of the corresponding video reception units 21 (1) to 21 (n), and the paired video reception units 21 (1) to 21 (n). ) Works in cooperation with each.
In the following description, in particular, when it is not necessary to specify and explain the individual image conversion units 23 (1) to 23 (n), the reference numerals in () are omitted in order to avoid complexity. explain.

The image conversion unit 23 decodes and decompresses digital video data, which is image information received from the video reception unit 21, and generates image data for each frame taken by the camera 2. The generated image data is sent to the current image processing unit 221 of the image processing unit 22 as image information in frame units and processed.

The current image processing unit 221 extracts a feature of a human face or an object by using VCA (Video Content Analysis) technology (video analysis technology) based on the image information from the image conversion unit 23 to obtain a specific object. Recognition, identification and extraction, leaving or taking away or analyzing the presence or absence of abnormalities such as belongings, intrusion, sick or drunk, leaving detection or intrusion detection, kerosene can detection or long object detection, sick or drunk detection, etc. Abnormality detection is detected.
The image recognition function in the current image processing unit 221 has already been put into practical use in a surveillance camera system, for example, a function of constructing a learning model and detecting a person's behavior, or a function of detecting a person's behavior using background subtraction, or a leftover object or a person's retention. Use the detection function.

The abandonment detection in the current image processing unit 221 is performed by background subtraction in the image information from the image conversion unit 23.
In addition, the current image processing unit 221 may perform intrusion detection, shutter detection for confirming the opening / closing of the shutter, kerosene can detection using a learning model by AI, long object detection, prowl detection, wheelchair detection, etc. Detects a person with specific conditions.

The current image processing unit 221 performs a plurality of image processing operations of the image processing server 20 in parallel from the image information in frame units received from the image conversion unit 23, and outputs an image processing result.
In the image processing calculation at this time, a mask area, which is the range of image processing, is set for the image information from the image conversion unit 23 for the purpose of improving the accuracy of the image processing result and reducing the calculation cost, and the set mask area is set. Image processing of the image information is performed in.

The image processing result indicates whether or not anomaly detection has occurred, such as whether or not a person has been left behind, whether or not a person has invaded, whether or not a person has a kerosene can, and whether or not a person has a long object. Or, the absence of abnormality detection is set to "0".
The image information and the image processing result from the image conversion unit 23 acquired by the current image processing unit 221 are sent to the angle of view deviation detecting unit 222.

Each of the recorded image receiving units 24 (1) to 24 (n) is provided corresponding to each of the cameras 2 (1) to 2 (n), and is based on the camera number of the camera list table 40 shown in FIG. The past image information recorded in the video recording apparatus 10 is acquired.
In the following description, in particular, when it is not necessary to specify and explain the individual recorded video receiving units 24 (1) to 24 (n), the reference numerals in () are omitted in order to avoid complexity. I will explain.

The recorded video receiving unit 24 operates in conjunction with the image processing result of the current image processing unit 221, and when the image processing result is "1", the past image information recorded in the video recording device 10 is acquired.
The past image information acquired by the recorded image receiving unit 24 is sent to the angle of view deviation detecting unit 222.
The recorded image receiving unit 24 can send a plurality of image information to the angle of view deviation detecting unit 222.

The angle of view deviation detection unit 222 calculates the amount of angle of view deviation of the image information from the camera 2 based on the image information from the image conversion unit 23 acquired by the current image processing unit 221.
When the angle of view deviation detection unit 222 receives the image processing result with abnormality detection from the current image processing unit 221, the image information from the image conversion unit 23 acquired by the current image processing unit 221 and the recorded image receiving unit 24 acquire it. The amount of angle of view deviation is calculated by comparing with the past image information.
That is, the image information A from the image conversion unit 23 acquired by the current image processing unit 221, the past image information B acquired by the recorded video receiving unit 24, and the image information C for image processing registered in advance are used. Then, the amount of deviation in the angle of view is calculated.
The image information used when calculating the amount of angle of view deviation is the image information in the mask area.

The calculation timing of the angle of view deviation amount in the angle of view deviation detection unit 222 may be always synchronized with the timing of the image processing of the current image processing unit 221, but the angle of view deviation occurs in the camera 2 in the camera 2. Since it is often caused by disturbance such as vibration or deterioration over time, it is preferable to perform the image processing result in synchronization with the timing when the image processing result of the current image processing unit 221 is "1" with abnormality detection.
Further, the timing of calculating the angle of view deviation amount in the angle of view deviation detection unit 222 may be performed at regular intervals.

The calculated angle of view deviation amount is sent to the mask area correction unit 223 and the event storage unit 25.
At the same time, the presence / absence of calculation of the angle of view deviation amount in the angle of view deviation detection unit 222, that is, the presence / absence of the angle of view deviation amount is sent to the event storage unit 25.
The presence or absence of the angle of view deviation in the angle of view detection unit 222 is determined by comparing the amount of angle of view deviation calculated by the angle of view detection unit 222 with the threshold value stored in the threshold list 60, and the amount of angle of view deviation is the threshold value. In the above case, it is determined that the angle of view deviation is "Yes", and if it is less than the threshold value, it is determined that the angle of view deviation is "None".

Further, the angle of view deviation detecting unit 222 compares the calculated angle of view deviation amount with the second threshold value stored in the threshold value list 60, and when the angle of view deviation amount is equal to or greater than the second threshold value, the angle of view is adjusted. The deviation alarm information is sent to the monitoring terminal device 30.

In the event storage unit 25, the presence / absence of calculation of the angle of view deviation amount in the angle of view deviation detection unit 222 is displayed on the event list table shown in FIG. 4, and the image in the angle of view deviation detection unit 222 is displayed on the angle of view deviation camera list table shown in FIG. Whether or not the angle deviation amount is calculated and the angle of view deviation amount are stored.

The event list table shown in FIG. 4 is a table of information associated with "event ID", "occurrence time", "release time", "detection type", "angle of view deviation", and "camera number".
The "event ID" is a number assigned to each event accumulated in the event storage unit 25, and whether or not the angle of view deviation detection unit 222 calculates the angle of view deviation amount is sent to the event storage unit 25. Sometimes granted automatically.

The “occurrence time” is the time when the event occurred, and is the time when the event ID is given when the presence / absence of calculation of the angle of view deviation amount by the angle of view deviation detection unit 222 is sent to the event storage unit 25.
The "cancellation time" represents the time when the user manually cancels the event, and the time when the user cancels the event is automatically stored.

The "detection type" is a number assigned according to the image processing content when the image processing server 20 performs a plurality of image processing. For example, "1" is assigned to "left-behind detection" and "2" is assigned to "intrusion detection", which is a number unique to the type of abnormality detection.
The “angle of view deviation” indicates the presence or absence of the amount of the angle of view deviation by the angle of view deviation detection unit 222, and “1” is added to the presence of the angle of view deviation and “0” is added to the case of no angle of view deviation.

The "camera number" indicates a name that identifies the camera 2 that has obtained the image processing result by the current image processing unit 221 sent to the event storage unit 25, that is, the number of the camera 2, and is a camera list table shown in FIG. It is based on 40 camera numbers.

The angle of view deviation camera list table shown in FIG. 5 shows the "camera number", "camera name", the amount of angle of view deviation in the "X direction (horizontal direction)", and the angle of view in the "Y direction (vertical direction)". This is a table of information in which the amount of deviation and the "angle of view deviation" are linked.
The “camera number” indicates the number of the camera 2 obtained from the image processing result by the current image processing unit 221 sent to the event storage unit 25, and is based on the camera number of the camera list table 40 shown in FIG.

The "camera name" represents the camera name displayed on the GUI screen of the monitoring terminal 150, and is based on the camera name of the camera list table 40 shown in FIG.
“X direction” represents the amount of angle of view deviation in the X direction calculated by the angle of view deviation detection unit 222.
The “Y direction” represents the amount of angle of view deviation in the Y direction calculated by the angle of view deviation detection unit 222.
For example, FIG. 5 shows that the camera number C2001 is +10 mm in the X direction, +9 mm in the Y direction, and the angle of view is deviated.
The “angle of view deviation” indicates the presence or absence of the amount of the angle of view deviation by the angle of view deviation detection unit 222, and “1” is added to the presence of the angle of view deviation and “0” is added to the case of no angle of view deviation.

When the angle of view deviation detection unit 222 determines that the angle of view deviation is present, the mask area correction unit 223 is "set" used in the current image processing unit 221 from the angle of view deviation amount calculated by the angle of view deviation detection unit 222. A correction mask area for correcting the "image area" is obtained.
The correction mask area at this time is an area obtained by updating the "set mask area" by performing coordinate conversion according to the amount of angle of view deviation.
The correction mask area is sent to the reimage processing unit 224.

When the angle of view deviation detection unit 222 determines that there is no angle of view deviation, the correction mask area by the mask area correction unit 223 may remain the "set mask area", and the mask area correction unit 223 does not require re-image processing. Is sent to the re-image processing unit 224.
When the angle of view deviation detecting unit 222 determines that there is no angle of view deviation, the image processing result by the current image processing unit 221 may be sent directly to the monitoring terminal device 30.

The re-image processing unit 224 analyzes the image information from the video receiving unit 21 again by the correction mask area from the mask area correction unit 223, and performs an abnormality such as abandonment detection or intrusion detection, kerosene can detection, or long object detection. Rediscover detection.
The reimage processing using the correction mask area in the reimage processing unit 224 is the same image processing as the image processing by the current image processing unit 221.
The image processing result by the re-image processing unit 224 is sent to the monitoring terminal device 30.
When the mask area correction unit 223 does not require the re-image processing, the re-image processing unit 224 does not perform the re-image processing and sends the image processing result by the current image processing unit 221 to the monitoring terminal device 30.

The image processing unit 22 shown above has, in order, a current image processing unit 221, an angle of view deviation detection unit 222, a mask area correction unit 223, and a reimage processing unit 224 as functional blocks possessed by the image processing unit 22. However, the function of the angle of view deviation detection unit 222 may be performed first, and then the mask area correction unit 223 and the current image processing unit 221 may be used.

That is, when the functional blocks of the image processing unit 22 are, in order, the image angle deviation detection unit 222, the mask area correction unit 223, and the current image processing unit 221, first, the image angle deviation detection unit 222 determines the amount of image angle deviation. When the current image processing unit 221 performs the calculation and the image angle deviation detection unit 222 determines that there is no image angle deviation, the image of the image information in the mask area set for the image information from the image conversion unit 23 When processing is performed and the image angle deviation is determined by the image angle deviation detection unit 222, the image of the image information in the mask area corrected by the mask area correction unit 223 with respect to the image information from the image conversion unit 23 is imaged. Perform processing.

The event notification unit 26 reads out a list of either the event list in the event list table shown in FIG. 4 stored in the event storage unit 25 or the angle-of-view shift camera list in the angle-of-view shift camera list table shown in FIG. , The read list is sent to the monitoring terminal device 30.

As shown in FIG. 2, the monitoring terminal device 30 includes video receiving units 31 (1) to 31 (n), image conversion units 32 (1) to 32 (n), a reproduction control unit 33, and processing result reception. It has a unit 34 and a display means 35 having a GUI screen.
The video receiving units 31 (1) to 31 (n) and the image conversion units 32 (1) to 32 (n) are the video receiving units 21 (1) to 21 (n) and the image conversion unit 23 of the image processing server 20, respectively. (1) to 23 (n). That is, the video receiving units 31 (1) to 31 (n) of the monitoring terminal device 30 and the image conversion units 32 (1) to 32 (n) are the video receiving units 21 (1) to 21 (n) of the image processing server 20. ) And the image conversion units 23 (1) to 23 (n).

Therefore, the image information from the camera 2 is sent to the image processing unit 22 and the display means 35 as image information in frame units via the image receiving unit 21 (31) and the image conversion unit 23 (32).
The video receiving unit 31 and the image conversion unit 32 send the image information of the selected camera 2 to the display means 35 by the "camera switching request" from the display means 35.

The monitoring terminal device 30 has a camera selection function, a live image display function, a recorded image display function, and an event information / angle of view deviation amount display function controlled by a monitoring program.
The camera selection function is a function of receiving image information from the cameras 2 (1) to 2 (n) and selecting an arbitrary camera from the cameras 2 (1) to 2 (n).
The live image display function is a function of displaying image information from the selected camera 2 as a live image on the display screen 352 (see FIG. 6) of the display means 35 by a single screen or a split screen.

The recorded video display function records image information from the selected camera 2 in the camera 2 recorded in the video recording device 10 or an event selected from the event list table shown in FIG. 4 recorded in the video recording device 10. This is a function for reproducing and displaying an image on the display screen 352 of the display means 35.
The event information / angle-of-view deviation amount display function is a display means 35 based on an image processing result indicating the presence / absence of abnormality detection received from the image processing server 20 and a color change or blinking of the camera display targeting the presence / absence of the angle-of-view deviation amount. The function to be displayed on the display screens 351 and 353, 355 (see FIG. 6) and the event list based on the event list table shown in FIG. 4 stored in the event storage unit 25 received from the event notification unit 26 or shown in FIG. This is a function of displaying the angle-of-view camera list based on the angle-of-view camera list table on the display screen 355 (see FIG. 6) of the display means 35.

The monitoring terminal device 30 includes a video receiving unit 31, an image conversion unit 32, a reproduction control unit 33, and a processing result receiving unit 34 represented by functional blocks in FIG. 2, and is composed of a general PC (Personal Computer), a CPU, and a semiconductor. It is composed of a memory (RAM) and a non-volatile recording device (ROM), a monitoring program stored in the ROM is loaded into the RAM, and a CPU executes various processes based on the monitoring program loaded in the RAM. The monitoring terminal device 30 is driven by a general-purpose OS.
The display means 35 has a GUI unit having a graphical user interface using a general OS window system.

The video receiving unit 31 and the image conversion unit 32 send the image information of the selected camera 2 to the display means 35 by the "camera switching request" from the display means 35.
The video receiving unit 31, the image conversion unit 32, and the display means 35 control the live image display function for the selected camera 2.

The playback control unit 33 sends a "search / playback request" from the display means 35 to the video recording device 10, and when it receives a "search / playback request response" from the video recording device 10, prepares for playback and prepares the video recording device 10. The image information recorded in the camera 2 is received as a reproduced image and sent to the display means 35, and the display means 35 reproduces the image information from the camera 2 recorded in the image recording device 10.

Specifically, the video recording device 10 that transmits the "search / playback request" to which the "camera number" and the "playback start time" are added from the display means 35 and receives the "search / playback request" has the "camera number" and the "camera number". The presence / absence of image information from the camera 2 is searched by "playback start time", and if it exists, the video recording device 10 uses the image information from the camera 2 searched by the playback control unit 33 together with the "search / playback request response" as the playback video. Send.
The reproduction control unit 33 sends the image information from the camera 2 transmitted from the video recording device 10 to the display means 35.

Further, when the reproduction control unit 33 receives the "search reproduction request" from the display means 35 that identifies the event, the reproduction control unit 33 reproduces the image information from the camera 2 recorded in the image recording device 10 corresponding to the requested event. The display means 35 reproduces the image information from the camera 2 corresponding to the event recorded in the video recording device 10.
The reproduction control unit 33 and the display means 35 control the recorded image display function.

The processing result receiving unit 34 sends the image processing result from the reimage processing unit 224 in the image processing server 20 to the display means 35, and the event list shown in FIG. 4 stored in the event storage unit 25 from the event notification unit 26. The event list read from the table or the angle-of-view shift camera list read from the image angle shift camera list shown in FIG. 5 is always received, and one of the list is sent to the display means 35.

Further, the processing result receiving unit 34 requests an event to the event notification unit 26 using the event ID as a search key, the event notification unit 26 instructs the event storage unit 25 to search the event list table, and the event storage unit 25 uses the event ID as the event ID. If the corresponding event information exists, the event information is returned to the event notification unit 26 together with the response of "search success". The processing result receiving unit 34 receives the event information for the event request via the event notification unit 26 and sends it to the display means 35.
If the event information corresponding to the event ID does not exist, the event storage unit 25 responds to the event notification unit 26 with a response of "search failure".
The processing result receiving unit 34 and the display means 35 control the event information / angle of view deviation amount display function.

The screen design to be displayed on the display screen of the GUI unit in the display means 35 has the following four functions.
First, in order to control the live video display function, a "camera switching request" in which the camera 2 is selected based on the camera number from the camera list table shown in FIG. 3 is sent to the video receiving unit 31 and the image conversion unit 32. This is a function of receiving image information from the selected camera 2 via the image receiving unit 31 and the image conversion unit 32 and displaying it as a live image on the display screen 352.

Second, in order to control the recorded video display function, a "search / playback request" is sent to the playback control unit 33, and the image information of the camera 2 recorded in the video recording device 10 and searched is displayed on the display screen 352 as a recorded video. It is a function to display.
Thirdly, in order to control the event information / angle of view deviation amount display function, the event content notified from the event notification unit 26 in the event list table shown in FIG. 4 stored in the event storage unit 25 in the image processing server 20. This is a function for displaying a certain "event list" on the display screen 355 and updating the displayed "event list".

Fourth, in order to control the event information / angle of view deviation amount display function, the event notification unit 26 in the angle of view deviation camera list table shown in FIG. 5 stored in the event storage unit 25 in the image processing server 20 notified. This is a function for displaying the "angle-of-view deviation camera list" on the display screen 355.
Further, the "angle-of-view camera list" displayed on the display screen 355 is notified by the event notification unit 26, and is based on the information given to the "event list" displayed on the display screen 355. You can decide the display contents and update them.

FIG. 6 shows an example of a display example displayed on the display screen of the GUI unit of the display means 35.
The display screen of the GUI unit is composed of a camera list display unit 351, a video display unit 352, a map display unit 353, a playback control unit 354, and an angle-of-view deviation camera list display unit 355.

The camera list display unit 351 is an area for displaying a list of cameras 2 managed by the image processing server 20 and the monitoring terminal device 30, and the cameras 2 are used for each map and each camera based on the camera list table 40 shown in FIG. It is displayed hierarchically.
The displayed camera name, for example, "2nd floor west staircase", is based on the camera name of the camera list table 40 shown in FIG.

When the user selects a camera icon or a camera name from the list displayed on the camera list display unit 351, a "camera switching request" indicating the selected camera 2 is sent to the image receiving unit 31 and the image conversion unit 32, and the image is displayed. A live image based on image information from the camera 2 selected via the receiving unit 31 and the image conversion unit 32 is displayed on the video display unit 352.
Further, when the processing result receiving unit 34 receives the image processing result with the angle of view deviation by the re-image processing unit 224, the color display of the camera icon and the camera name corresponding to the target camera 2 is changed or blinks. ..

The video display unit 352 specifies that the live video based on the image information from the selected camera 2 input via the video receiving unit 31 and the image conversion unit 32 is recorded in the video recording device 10 by the playback control unit 33. This is an area for displaying a reproduced image based on image information from the camera 2.
FIG. 6 shows a display area divided into four, and the live image from the four cameras 2 or the live image from the three cameras 2 and the reproduced image from one camera 2 recorded in the image recording device 10 are shown. Can be displayed at the same time.

The map display unit 353 is an area for displaying a map of the station premises when the camera 2 is arranged in the station premises. The camera 2 installed in the station yard is displayed as an icon, and is arranged based on the map and the map position of the camera list table 40 shown in FIG.
When the user selects the camera icon displayed on the map display unit 353, the camera 2 is selected in conjunction with the camera icon displayed on the camera list display unit 351 and the live image based on the image information from the selected camera 2 is selected. Is displayed on the image display unit 352. Further, when the processing result receiving unit 34 receives the image processing result with the image angle deviation by the re-image processing unit 224, the color display of the camera icon corresponding to the target camera 2 changes. Or blinks.
In FIG. 6, the icon of the sensor installed in the station yard is also displayed.

The reproduction control unit 354 is necessary for the time input portion for designating the reproduction start position and the reproduction control of the reproduced image by the image information from the camera 2 recorded in the image recording device 10 displayed on the image display unit 352. This area displays buttons such as "fast rewind", "play", "pause", and "fast forward", and a slider that controls the playback speed.

The angle-of-view deviation camera list display unit 355 is an event list read from the event list table shown in FIG. 4 stored in the event storage unit 25, or an image read from the angle-of-view deviation camera list table shown in FIG. This is the area where the angle-of-view camera list is displayed.
FIG. 6 shows an example of displaying the angle-of-view deviation camera list, showing the “camera number”, the amount of angle-of-view deviation in the “X direction”, the amount of angle-of-view deviation in the “Y direction”, and the “camera name”. , The status of "angle of view deviation" is displayed in real time.

When the user sees the camera list display unit 355, for example, the camera number C2001, the camera name 2nd floor / west staircase camera 2 has an angle of view deviation of +10 mm in the X direction and +9 mm in the Y direction. You can see that there is.
Further, since the camera number and the camera icon of the map display unit 353 are linked by selecting the camera number of the camera list display unit 355 with an angle of view deviation, the camera icon of the map display unit 353 changes color or blinks, etc. The display changes depending on. As a result, the installation location of the camera 2 selected by the camera list display unit 355 with an angle of view deviation can be known from the camera icon of the map display unit 353.

The angle-of-view shift camera list display unit 355 appropriately switches from the display of the angle-of-view shift camera list to the display of the event list, and vice versa.
When the user selects an event ID while the event list is displayed on the camera list display unit 355, for example, when the displayed event ID is double-clicked, the camera number and the occurrence time stored in the event list table are displayed. With the key, the reproduction control unit 33 selects the image information of the camera 2 recorded in the image recording device 10, and displays the image information of the selected camera 2 on the image display unit 352 as a reproduction image.

Next, the operation of the image processing server 20 in the monitoring system, mainly the operation of the image processing unit 22, will be described with reference to FIG. 7.
In step ST1, the image receiving unit 21 receives the image information from the camera 2, and the received image information from the camera 2 is transmitted to the current image processing unit 221 via the image conversion unit 23.
The current image processing unit 221 detects an abnormality such as abandonment detection or intrusion detection, kerosene can detection, or long object detection for the image information of the camera 2 received based on the camera number of the camera list table 40 shown in FIG. Performs arithmetic processing corresponding to the image processing related to, detects abnormality detection, and proceeds to step ST2.

In step ST2, the image information and the image processing result from the image conversion unit 23 acquired by the current image processing unit 221 and the detection information (detection type) such as left-behind detection or intrusion detection or kerosene can detection or long object detection are the angles of view. It is sent to the deviation detection unit 222 and proceeds to step ST3.
In step ST3, the angle of view deviation detection unit 222 detects the angle of view deviation.
That is, the angle of view deviation detecting unit 222 calculates the amount of the angle of view deviation of the camera based on the image information from the image receiving unit 21 in the mask area which is the range of the set image processing.

Specifically, the angle-of-view deviation detection unit 222 uses the camera number of the camera list table 40 as a key, and the image information from the image conversion unit 23 acquired by the current image processing unit 221 and the recorded image reception unit 24 acquire it. The amount of angle of view deviation is calculated by comparing with the past image information.

If the angle of view deviation detection unit 222 does not calculate the angle of view deviation amount, no angle of view deviation is sent to the event storage unit 25, and the process ends. Although not shown in FIG. 7, the image processing result of the current image processing unit 221 is sent to the monitoring terminal device 30.
When the angle of view deviation detection unit 222 calculates the angle of view deviation amount, the calculated angle of view deviation amount is sent to the event storage unit 25 and proceeds to step ST4.
The event storage unit 25 stores the angle-of-view deviation amount in the X-direction and the Y-direction together with the camera number and the camera name in the angle-of-view deviation camera list table shown in FIG.

In step ST4, using the camera number in the camera list table 40 as a key, the threshold value stored in the threshold list 60 corresponding to the detection type is acquired, and the process proceeds to step ST5.
In step ST5, the angle of view deviation detection unit 222 compares the calculated angle of view deviation amount with the threshold value stored in the threshold list 60, and when the angle of view deviation amount is less than the threshold value, the angle of view deviation is "none". Is sent to the event storage unit 25, and the process returns to step ST2.
If the angle of view deviation is equal to or greater than the threshold value, the angle of view deviation is set to "Yes" and sent to the event accumulating unit 25, the angle of view deviation amount is sent to the mask area correction unit 223, and the process proceeds to step ST6.

The event storage unit 25 stores the event ID, the occurrence time, the detection type, and the camera number as well as the presence or absence of the angle of view deviation in the event list table shown in FIG.
Further, in the event storage unit 25, the angle-of-view deviation camera list table shown in FIG. 5 stores the presence or absence of the angle-of-view deviation with respect to the camera number of the camera 2 in which the angle-of-view deviation is compared.

In step ST6, the mask area correction unit 223 obtains a correction mask area for correcting the "set mask area" used in the current image processing unit 221 from the angle of view deviation amount calculated by the angle of view deviation detection unit 222. , The obtained correction mask area is sent to the reimage processing unit 224, and the process proceeds to step ST7.

Specifically, the mask information indicating the mask area registered for each camera number in the camera list table 40 according to the angle deviation amount calculated by the mask area correction unit 223 by the image angle deviation detection unit 222, that is, , Performs coordinate conversion processing on the mask information indicating the set mask area, and updates the mask information indicating the mask area.
The updated mask information is the mask information indicating the correction mask area, and is the mask information indicating the set mask area for the next image processing.

In step ST7, the re-image processing unit 224 analyzes the image information from the video receiving unit 21 using the correction mask area, and detects or intrudes the image in the same manner as the arithmetic processing corresponding to the image processing by the current image processing unit 221. Detects anomaly detection such as detection, kerosene can detection, or long object detection.
Although not shown in FIG. 7, the image processing result of the re-image processing unit 224 is sent to the monitoring terminal device 30, and the processing is completed.

The processing step shown in FIG. 6 may be stored as a program in the ROM of the computer constituting the image processing server 20.
That is, the program stored in the ROM shifts the angle of view of the camera based on the procedure for receiving the image information from the camera and the image information from the received camera in the mask area which is the range of the set image processing. The procedure for calculating the amount, the procedure for obtaining the correction mask area from the calculated angle of view deviation amount, and the image information from the received camera in the obtained correction mask area are analyzed, and left-behind detection or intrusion detection, or It is an abnormality detection detection program of a monitoring system that causes a computer to execute a procedure for detecting an abnormality such as kerosene can detection or long object detection.

As described above, in the monitoring system according to the first embodiment, the image processing unit 22 calculates the amount of the angle of view deviation of the camera 2 based on the image information from the image receiving unit 21, and the calculated angle of view deviation is calculated. A correction mask area is obtained from the amount, and the image information from the image receiving unit 21 in the obtained correction mask area is analyzed to detect an abnormality such as abandonment detection or intrusion detection, kerosene can detection, or long object detection. Therefore, the mask area, which is the range of image processing, can be automatically corrected, the frequency of maintenance for image recognition by the camera 2 and the frequency of parameter adjustment for the camera 2 can be suppressed, and highly accurate abnormality detection can be detected.

As a result, the monitoring system according to the first embodiment can be improved against false alarms and false alarms due to abnormality detection.
For example, in the detection of anomaly detection due to an angle of view shift in the camera 2 and abandonment detection, anomaly detection by detecting background subtraction based on image information from the camera 2 is detected even though there are no suspicious objects or forgotten objects. This may result in false alarms due to the detection of anomalies.
In the monitoring system according to the first embodiment, since the mask area is automatically corrected, it is possible to reduce the occurrence of such false alarms and reduce the user's load of confirmation operation due to false alarms.

Further, in the monitoring system according to the first embodiment, the image processing server 20 has a camera number which is a uniquely determined index number assigned to the camera 2, a camera name indicating an installation position assigned to the camera 2, and a camera. The X-direction angle-of-view deviation amount indicating the X-direction angle-of-view deviation amount of 2 and the Y-direction angle-of-view deviation amount indicating the Y-direction angle-of-view deviation amount of the camera 2 are linked to the presence or absence of the angle-of-view deviation of the camera 2. Since it has an event storage unit 25 in which the angle-of-view deviation camera list table is stored, the user can read and display the angle-of-view deviation camera list from the angle-of-view deviation camera list table stored in the event storage unit 25. It is possible to easily recognize the deviation of the angle of view.

Further, in the monitoring system according to the first embodiment, the event ID and the occurrence time assigned to the image processing server 20 when the image processing unit 22 detects the abnormality detection, and the video receiving unit 21 that detects the abnormality detection. Since it has an event storage unit that stores an event list table that includes a name that identifies the camera 2 and the presence or absence of an angle-of-view deviation amount with respect to the image information from, the event list is read from the event list table stored in the event storage unit 25. By displaying the image, the user can easily recognize the time when the angle of view shift occurs.
Further, it is possible to display an image based on the image information of the camera 2 when the abnormality detection is detected by using the event ID.

Further, in the monitoring system according to the first embodiment, when the image processing unit 22 exceeds the second threshold value having a value larger than the threshold value, the angle-of-view deviation warning information is output, so that the detection of abnormality is merely detected. It is notified that the angle of the camera 2 has changed due to a large vibration to the camera 2, for example, an angle change due to some kind of impact or mischief, not due to the deviation of the angle of view. As a result, the user's inspection of the camera 2 can be reduced.

Embodiment 2.
The monitoring system 1 according to the second embodiment will be described with reference to FIG.
In each figure, the same reference numerals indicate the same or corresponding parts.
The monitoring system 1 according to the second embodiment has a mask area correction unit 225 including an image processing parameter correction unit 225a with respect to the mask area correction unit 223 of the image processing unit 22 in the monitoring system 1 according to the first embodiment. It is the same as the monitoring system 1 according to the first embodiment except that the points are different and the functions of the re-image processing unit 226 that receives the correction parameters from the image processing parameter correction unit 225a are slightly different.
That is, the monitoring system 1 according to the second embodiment is the same as the components in the monitoring system 1 according to the first embodiment except for the mask area correction unit 225 and the reimage processing unit 226.

The mask area correction unit 225 has the same function as the mask area correction unit 223, that is, the "set mask area" used in the current image processing unit 221 from the angle of view deviation amount calculated by the angle of view deviation detection unit 222. In addition to having a function to obtain a correction mask area, it corrects parameters for detecting a specific person such as person detection parameters for detecting a person used in the current image processing unit 221, kerosene can detection, and long object detection. It includes an image processing parameter correction unit 225a having a parameter correction function for obtaining correction parameters to be performed.

The parameter means an image processing parameter and is simply expressed as a parameter to avoid complexity.
The image processing parameter correction unit 225a in the mask area correction unit 225 targets the camera 2 having an angle of view deviation from the angle of view detection unit 222, and is a specific person with respect to the image information from the target camera 2. Get the correction parameters to detect.

The parameter correction function receives the image information in which kerosene can detection or long object detection is detected from the image recording device 10 in which the past image information of the target camera 2 is recorded, and also receives the image information from the recording image receiving unit 24. The image information of the camera 2 related to the target camera 2 is received via the video receiving unit 21 and the image conversion unit 23, and the image information of the target camera 2 and the past of the target camera 2 are received. This is a function of comparing the image information with the related image information of the camera 2 and obtaining a correction parameter for the image information of the target camera 2.
The correction parameter is sent to the reimage processing unit 226.

The reimage processing unit 226 analyzes the image information from the video receiving unit 21 again by the correction mask area and the correction parameter from the mask area correction unit 225, and detects the leftover or intrusion, or the kerosene can detection or the long object detection. Rediscover anomaly detection such as.
The image processing result by the re-image processing unit 226 is sent to the monitoring terminal device 30.

As described above, the monitoring system 1 according to the second embodiment obtains correction parameters for the parameters for correcting the angle of view deviation and performing image processing for the image information of the camera 2 having the angle of view deviation. Since the image processing is performed above, highly accurate abnormality detection can be detected.

As a parameter correction function, the target camera 2 detects kerosene cans or long objects using the detection type as a key from the event list table shown in FIG. 4 stored in the event storage unit 25. , Acquires the event ID of the camera 2 related to the target camera 2, receives the image information of the target camera 2 and the related camera 2 from the video recording device 10 based on the event ID, and is targeted. It may be a function of comparing the image information of the target camera 2 with the image information of the camera 2 related to the target camera 2 and obtaining a correction parameter for the image information of the target camera 2.

It is possible to freely combine the embodiments, modify any component of each embodiment, or omit any component of each embodiment.

The surveillance system according to the present disclosure is based on image information from a security camera or a surveillance camera installed in a station yard or a place to be monitored such as a store, and is used for abandonment detection or intrusion detection, kerosene can detection, long object detection, etc. It is suitable for a monitoring system that detects anomaly detection.

1 Surveillance system, 2 (1) to 2 (n) camera, 3 network switch, 10 video recording device, 20 image processing server, 21 (1) to 21 (n) video receiver, 22 image processing unit, 221 current image Processing unit, 222 Image angle deviation detection unit, 223 Mask area correction unit, 224 Re-image processing unit, 23 (1) to 23 (n) image conversion unit, 24 Recorded video reception unit, 25 Event storage unit, 26 Event notification unit , 30 Monitoring terminal device, 31 (1) to 31 (n) video receiving unit, 32 (1) to 32 (n) image conversion unit, 33 playback control unit, 34 processing result receiving unit, 35 display means, 225 mask area Correction unit, 225a Image processing parameter correction unit, 226 Re-image processing unit.

Claims (10)

The angle of view deviation of the camera is calculated based on the image information from the image receiving unit that receives the image information from the camera and the image receiving unit in the mask area that is the range of the set image processing, and the calculation is performed. Image processing having an image processing unit that obtains a correction mask area from the amount of angle of view deviation, analyzes image information from the video receiving unit in the obtained correction mask area, and detects abnormality detection such as abandonment detection or intrusion detection. A monitoring system with a server. A video receiver that receives image information from the camera,
The current image processing unit that analyzes the image information from the video receiving unit in the mask area within the set image processing range and detects abnormality detection such as abandonment detection or intrusion detection, and the image information from the video receiving unit. An image angle deviation detection unit that calculates the amount of angle of view deviation of the camera, and a mask area correction unit that obtains a correction mask area that corrects the mask area from the amount of angle of view deviation calculated by the image angle deviation detection unit. An image processing unit including a re-image processing unit that re-analyzes image information from the video receiving unit by the correction mask area from the mask area correction unit and rediscovers abnormality detection such as abandonment detection or intrusion detection.
Surveillance system with an image processing server. The monitoring system according to claim 1 or 2, wherein the calculation of the angle of view deviation amount of the camera is performed by comparing the current image information from the video receiving unit with the past image information. The image processing server has a camera number, which is a uniquely determined index number assigned to the camera, a camera name indicating an installation position assigned to the camera, and an X indicating the amount of angle of view deviation of the camera in the X direction. A camera list table is stored in which the amount of directional angle of view deviation, the amount of Y-direction angle of view deviation indicating the amount of deviation in the Y direction of the camera, and the presence or absence of deviation in the angle of view of the camera are linked. The monitoring system according to any one of claims 1 to 3, which has an event storage unit. The image processing server stores an event storage in which an angle-of-view camera list table including the camera name of the camera and the amount of angle-of-view deviation is stored for the image information from the video receiving unit when the image processing unit detects an abnormality detection. Has a part,
The claim is provided with a monitoring terminal device having a display means for displaying the camera name and the angle-of-view deviation camera list including the angle-of-view deviation amount read from the angle-of-view deviation camera list table stored in the event storage unit. The monitoring system according to any one of claims 1 to 4. The image processing server is an event storage unit that stores an event list table including a name that identifies the camera and the presence or absence of an angle of view deviation for image information from the video receiving unit for which the image processing unit has detected an abnormality detection. Have,
1. Claim 1 comprising a monitoring terminal device having a display means for displaying an event list including a name specifying the camera read from the event list table stored in the event storage unit and the presence / absence of the angle of view deviation amount. The monitoring system according to any one of claims 4. When the calculated angle of view deviation amount is equal to or greater than the threshold value, the image processing unit determines that the angle of view deviation is present, obtains a correction mask area from the calculated angle of view deviation amount, and obtains a value from the threshold value. The monitoring system according to any one of claims 1 to 6, which outputs angle-of-view deviation warning information when a large second threshold value is exceeded. The image processing unit detects an abnormality detection of a kerosene can holding a kerosene can or a long object holding a long object by using an image processing parameter based on the image information from the image receiving unit. When the abnormality detection is detected, if the camera targeted for the abnormality detection has an image angle deviation, the image information of the camera targeted for the abnormality detection and the camera related to the target camera are used. The monitoring system according to any one of claims 1 to 7, wherein a corrected image processing parameter for the image information of the target camera is obtained based on the image information of the above. The step in which the video receiver receives the image information from the camera,
From the step of calculating the angle of view deviation amount of the camera based on the image information from the video receiving unit in the mask area which is the range of the set image processing, and the calculated angle of view deviation amount. An abnormality detection detection method of a monitoring system having a step of obtaining a correction mask area and a step of analyzing image information from the video receiving unit in the obtained correction mask area and detecting an abnormality detection such as abandonment detection or intrusion detection. The procedure for receiving image information from the camera and
A procedure for calculating the amount of angle of view deviation of the camera based on the image information from the received camera in the mask area which is the range of the set image processing, and
The procedure for obtaining the correction mask area from the calculated angle of view deviation amount, and
A procedure for analyzing the image information from the received camera in the obtained correction mask area and detecting an abnormality detection such as abandonment detection or intrusion detection.
Anomaly detection detection program for monitoring systems run by computers.

Images