JP4642136B1 - Interlocking display measurement system using surveillance camera images - Google Patents

Abstract

The present invention uses a monocular monitoring camera image that can provide a specified location on a monitoring camera image, a three-dimensional image, and a plan view image at a glance without taking stepwise means. Get a linked display measurement system.
A camera selection unit 31, a mode determination unit 32, an angle-of-view setting unit 33, a display processing unit 34, a camera image designation location linked display portion 11, a GIS image designation location linkage display portion 12, A three-dimensional GIS image and a plan view photographed by the surveillance camera with a three-dimensional terrain model associated with a camera image photographed over a wide range of several kilometers, including a plan view designated location interlocking display unit 13 and the like. In addition to displaying images in conjunction with each other, the designated position (cursor) on any image (camera image, 3D GIS image, or plan view image) is displayed in conjunction with each image, and the 3D coordinates of the designated position are displayed. Inform.
[Selection] Figure 8

Description

  The present invention displays a surveillance camera image obtained by photographing a subject to be photographed with a surveillance camera, a pseudo image of the surveillance camera image, and a planar image including the subject to be photographed on a display unit, and displays designated portions on these images in conjunction with each other. The present invention relates to an interlocked display measurement system using surveillance camera images.

  For the purpose of monitoring natural disasters such as volcanic eruptions and debris flows, camera monitoring equipment (hereinafter referred to as surveillance cameras) may be installed.

  However, while surveillance camera images from surveillance cameras (hereinafter referred to as surveillance camera images) are effective for a general understanding of phenomena such as pyroclastic flows and volcanic blocks, it is not possible to capture detailed location information such as the location of occurrence and arrival from there. It is difficult except for some who are familiar with the local area.

  Especially for surveillance organizations that need to quickly communicate the occurrence status, this problem of location determination is important, and it is possible to immediately grasp the relationship between events in surveillance camera images and map information such as disaster prevention maps. Means are desired.

  Usually, as means for acquiring position information from a monitoring camera image, there are a stereo measurement method using two monitoring cameras or a single photo measurement method using one monitoring camera.

  Stereo measurement needs to overlap so that the images of the two surveillance cameras capture the same range to some extent, and single photo measurement needs to know the distance to the measurement object or the altitude of the object.

  In addition, for these measurements, there are internal orientation elements such as the size of the imaging surface of the surveillance camera, the deviation between the center of the imaging surface and the optical center of the lens, the number of pixels of the imaging device and lens distortion, and the surveillance camera in the geographic coordinate system. The external orientation elements such as the position of and the rotation angle with respect to the three-dimensional coordinate axis are necessary.

  The aforementioned internal orientation element is calculated from nine or more reference points whose three-dimensional coordinates in the video are known.

  On the other hand, the external orientation element is similarly calculated by using three or more reference points with known three-dimensional geographic coordinates in the video, and directly obtained by attaching a measuring device such as GPS or gyro to the photographing apparatus. There is.

  The stereo measurement method is the most ideal for improving the measurement accuracy. However, the existing surveillance cameras have fewer pairs of cameras in order to minimize the number of installed cameras. In many cases, it cannot be secured.

  In addition, it is disadvantageous in terms of cost to add surveillance cameras for measurement, and because of the nature of disaster areas, it is impossible to place reference points on site for safety, and if the shooting range is wide, Because there is a problem that a reference point of a size that can be recognized by the camera cannot be located far from the surveillance camera, there is a problem with the measurement means that does not require a reference point in the field with a single surveillance camera. Realization is required.

  Therefore, a method has already been proposed in which an image that looks the same as a monitoring camera image is generated from numerical map data or three-dimensional shape data, and is superimposed on a photographic image.

JP 2007-271869 A JP 2007-18347 A

Japan Photogrammetry Society Analytical Photogrammetry Rev. P. 46-P56

  However, although both Patent Document 1 and Patent Document 2 have been proposed sequentially for 3D information and photographic image superimposition processing and its usage, the actual monitoring site (volcano explodes at night) where immediacy is required. Etc.), for example, there may be a case where the time lost by taking stepwise measures such as selecting the coordinates on the photograph, outputting the corresponding three-dimensional coordinates, and displaying them on the map software. is there.

  In addition, since a pseudo image is synthesized and displayed on a video (image) actually taken, the actual image cannot be viewed accurately. In order to view the image, it is necessary to take a measure such as removing a pseudo image.

  Therefore, in the present invention, the monitoring camera image is not subjected to any processing, and the designated portions on the monitoring camera image, the three-dimensional image, and the plan view image are displayed at a glance without taking stepwise means. It is an object of the present invention to obtain a linked display measurement system using a monocular surveillance camera image that can be provided accurately and directly on an image.

The interlocking display measurement system using the surveillance camera image of the present invention,
The screen of the display unit is divided, and the surveillance camera image obtained by photographing the subject to be photographed by the surveillance camera having the CCD, the three-dimensional image corresponding to the surveillance camera image, and the plan view image including the subject to be photographed are linked to the corresponding display screen. A linked display measurement system that uses a monitoring camera image to display and display a mark at a specified location on one of the display screens and display the mark in a linked position on another display screen,
A storage means for storing a three-dimensional terrain model of the imaging target, the monitoring camera image, an external orientation element and an internal orientation element of the monitoring camera, and plan view data including the imaging target;
(A) means for defining an angle of view at the photographing point of the surveillance camera in the three-dimensional terrain model based on the external orientation element and the internal orientation element;
(B) The coordinates of the designated location on the display screen of the monitoring camera image are converted into image coordinates on the monitoring camera image, and the image coordinates on the monitoring camera image are converted into photographic coordinates which are actual coordinates on the CCD surface. Converting the photographic coordinates into camera coordinates within the angle of view, and converting the camera coordinates into three-dimensional coordinates of the three-dimensional terrain model;
(C) The coordinates of the designated portion on the display screen of the three-dimensional image are converted into image coordinates on the three-dimensional image, the image coordinates on the three-dimensional image are converted into the photographic coordinates, and the photographic coordinates are converted into the image coordinates. Means for converting into camera coordinates in a corner, and converting the camera coordinates into three-dimensional coordinates of the three-dimensional terrain model;
(D) means for converting the coordinates of a designated portion on the display screen of the plan view image into two-dimensional coordinates of the plan view data, and converting the two-dimensional coordinates into the three-dimensional coordinates of the three-dimensional terrain model;
With
(E) Reading the three-dimensional coordinates of the three-dimensional terrain model, converting the coordinate conversion process of the means (c) into the coordinates of the display screen of the three-dimensional image, and converting the mark to the coordinates. Means for displaying;
(F) The three-dimensional coordinates of the three-dimensional terrain model are read, and the coordinate transformation process of the means (d) is inversely transformed to be transformed into the coordinates of the display screen of the plan view image. Means for displaying,
(G) The three-dimensional coordinates of the three-dimensional terrain model are read, the coordinate conversion process of the means (b) is inversely converted into coordinates on the display screen of the monitoring camera image, and the mark is added to the coordinates. Means for displaying;
With
(H1) When the designated location is the display screen of the monitoring camera image, the (b) means, (e) means are activated, and the (f) means are activated,
(H2) When the designated portion is the display screen of the three-dimensional image, the (c) means, (g) means are activated, and the (f) means are activated,
(H3) In the case where the designated portion is a display screen of the plan view image, the (d) and (e) means are activated and the (g) means is activated. To do.

  As described above, according to the present invention, the monitoring camera image, the three-dimensional GIS image, and the plan view are displayed in conjunction with each other, and the designated portion of any screen is instantaneously displayed in the same position. The relative positional relationship on these images can be grasped at a glance without using.

  In addition, for example, information that cannot be known only by the monitoring camera, such as the eruption start point and the location / measurement of the disaster area, is obtained.

It is a schematic block diagram of the interlocking display measurement system using the monitoring camera image of this Embodiment. It is explanatory drawing explaining the characteristic of the interlocking display measurement system using the monitoring camera image of this Embodiment. It is explanatory drawing explaining schematic operation | movement of the camera image designation | designated location interlocking display part 11, the GIS image designation | designated location interlocking display part 12, and the plane image designation | designated location interlocking display part 13. FIG. It is explanatory drawing explaining the coordinate transformation used for this Embodiment. It is explanatory drawing explaining the coordinate transformation used for this Embodiment. It is explanatory drawing explaining the coordinate transformation used for this Embodiment. It is explanatory drawing explaining the coordinate transformation used for this Embodiment. It is a schematic block diagram of the three-dimensional interlocking measurement display system using the image of the surveillance camera of Embodiment 1. It is explanatory drawing of an orientation element. 4 is a flowchart for explaining the overall operation of the three-dimensional interlocking measurement display system using an image of the surveillance camera according to the first embodiment. 6 is a flowchart for explaining the operation of the camera image designation location interlocking display unit of the three-dimensional interlocking measurement display system using the image of the surveillance camera of the first embodiment. 6 is a flowchart for explaining the operation of the camera image designation location interlocking display unit of the three-dimensional interlocking measurement display system using the image of the surveillance camera of the first embodiment. 6 is a flowchart for explaining the operation of the camera image designation location interlocking display unit of the three-dimensional interlocking measurement display system using the image of the surveillance camera of the first embodiment. FIG. 6 is a configuration diagram of a GIS image designation location interlocking display unit 12 according to a second embodiment. 6 is a flowchart for explaining a GIS image designation location interlocking display unit 12 according to a second embodiment. 6 is a flowchart for explaining a GIS image designation location interlocking display unit 12 according to a second embodiment. 6 is a flowchart for explaining a GIS image designation location interlocking display unit 12 according to a second embodiment. FIG. 11 is a configuration diagram of a plan view image designation location interlocking display unit 14 according to a third embodiment. 10 is a flowchart for explaining a plan view image designation location interlocking display unit 14 according to the third embodiment. 10 is a flowchart for explaining a plan view image designation location interlocking display unit 14 according to the third embodiment. FIG. 10 is a configuration diagram of distance / area calculation processing according to a fourth embodiment. FIG. 10 is an explanatory diagram illustrating distance / area calculation processing according to the fourth embodiment. FIG. 10 is an explanatory diagram for explaining a three-dimensional image of viewpoint change according to a fifth embodiment. FIG. 10 is a schematic configuration diagram of a three-dimensional interlocking measurement display system using an image of a surveillance camera according to a sixth embodiment. 16 is a flowchart for explaining the operation of a three-dimensional interlocking measurement display system using an image of a surveillance camera according to a sixth embodiment. 16 is a flowchart for explaining the operation of a three-dimensional interlocking measurement display system using an image of a surveillance camera according to a sixth embodiment. FIG. 20 is an explanatory diagram illustrating cursor display according to a sixth embodiment. It is explanatory drawing explaining the method of the setting (calculation) of an angle of view.

  In this embodiment, a wide range of images (hereinafter simply referred to as camera images: still images or moving images) covering several kilometers of volcanoes, sea areas, plains, etc., captured by a surveillance camera (fixed) are obtained by aerial laser measurement or the like. By associating with the obtained digital elevation model (three-dimensional geographic coordinate model), a three-dimensional image (referred to as a three-dimensional GIS image in this embodiment) of a camera image or a photographing range (angle of view) taken by the monitoring camera. Alternatively, a two-dimensional plan view image (hereinafter simply referred to as a plan view image) of the photographing range is displayed in an interlocked manner, and a designated portion (including position, area, and line) is displayed as a cross mark (this embodiment) in any of these images. In this form, it is a system that displays in conjunction with the cursor or target point. Also, the coordinates, area, and length of the specified location are displayed.

  In other words, this embodiment uses the fact that a 3D GIS image is a 3D image in which an aerial photograph or satellite image is pasted as a texture on a 3D geographical geographic coordinate model, and the viewpoint and texture can be changed freely. This makes it possible to check the position with the drawings required at the surveillance site and to change the viewpoint to observe the topographic shape.

  That is, since a three-dimensional GIS image is used, a two-dimensional plan view image can be displayed in conjunction with a screen (also referred to as a two-dimensional plan view screen) in the same manner, so that a surveillance camera image, a three-dimensional GIS image, and a plan view are displayed. When images (for example, maps, disaster prevention maps, etc.) are linked and displayed, if a location is specified on any of the images, the mutual relationship between the specified locations (also referred to as specified locations) can be grasped at a glance. Easy to understand.

Therefore, the present embodiment can display the surveillance camera image, the three-dimensional GIS image, and the plan view image in conjunction with each other without taking stepwise means so as to quickly grasp the mutual positional relationship between the designated locations. Yes.

  Embodiments will be described below with reference to the drawings.

  FIG. 1 is a schematic configuration diagram of an interlocked display measurement system using surveillance camera images of the present embodiment.

  In the present embodiment, a wide-range imaging target A such as a volcano, a mountain, a river, a plain, an island, or a sea area is photographed with a monocular surveillance camera 1 (fixed camera: may be a plurality of points), and this camera image data is obtained via a LAN. It transmits to the image storage device 2 (server) at a remote location and stores it.

  Alternatively, the camera image data is stored in an external storage device such as a CD-ROM 3 and is taken back and stored in the image storage device 2. Then, these camera images, internal orientation elements, external orientation elements, and the like described later are stored in the database 5 (also referred to as a three-dimensional measurement data set) (1).

  For the setting of the angle of view by the above-described external orientation elements and internal orientation elements, and mutual conversion between screen coordinates and three-dimensional geographic coordinates, calculation is performed using a coordinate conversion method disclosed in Non-Patent Document 1.

  The database 5 stores a three-dimensional geographic coordinate model and its texture data (hereinafter collectively referred to as a three-dimensional terrain model), and stores two-dimensional plan view data such as a numerical map and a disaster prevention map. . Two-dimensional map data may be used as texture.

  The three-dimensional geographic coordinate model of the above-mentioned three-dimensional terrain model stores the geographical coordinates (latitude and longitude) of the four corners and the number of vertical and horizontal meshes as header information, and only the altitude is recorded as the actual data in the order of mesh intersections. Data.

  Further, the database 5 stores an internal orientation element of the camera, an external orientation element of the camera, and the like.

  The internal orientation element of the camera and the external orientation element of the camera are calculated by the preprocessing unit 6.

  The pre-processing unit 6 prepares a target plate whose coordinate values are precisely measured by a total station or the like in advance, installs the test target plate in front of the lens of the monitoring camera 1, and photographs the tester target plate (2). This is to accurately calculate the internal orientation element of the camera (deviation between the center of the imaging surface and the optical center of the lens = principal point position deviation, lens distortion, focal length,...). The imaging surface size and the number of pixels need to be known in advance.

  Next, the internal orientation elements (principal point position deviation, lens distortion, focal length) of the surveillance camera are calculated from the relationship between the coordinate values of each target plate (coordinates obtained precisely by the total station, etc.) and the image coordinates of the camera. 5 is stored (3).

  Next, multiple corresponding points of characteristic terrain such as ridges reflected in the camera image are obtained on the 3D terrain model (obtaining the ground reference point of the camera image), and the external orientation element of the surveillance camera (the surveillance camera's (Position, posture) is calculated (4). The reference point is acquired by displaying a camera image and a three-dimensional GIS image of the camera image on a screen (camera image display screen, GIS image display screen).

  Next, the external orientation elements (camera position and orientation) of the surveillance camera are obtained from the relationship between the 3D geographic coordinates obtained from the 3D terrain model and the image coordinates of the corresponding surveillance camera, and stored in the database 5 (5). .

  Then, the angle-of-view setting unit 10 reproduces the angle of view of the surveillance camera on the three-dimensional terrain model using the internal orientation element and the external orientation element of the surveillance camera. A pseudo camera image (three-dimensional GIS image) of A can be displayed on the display unit 20. That is, the three-dimensional GIS image corresponding to the imaging target A is displayed on the display unit 20 by reproducing the angle of view of the surveillance camera on the three-dimensional terrain model.

  On the other hand, a camera image designation location interlocking display section 11, a GIS image designation location interlocking display section 12, and a plane image designation location interlocking display section 13 are provided, which operate in conjunction with each other.

  When a cursor indicating the designated position is designated on the camera image, the three-dimensional GIS image, or the plan view image, the corresponding coordinate positions of the three-dimensional GIS image and the plan view image are displayed in conjunction (see FIG. 2).

  FIG. 2 shows that the cursor is displayed in conjunction with the three-dimensional GIS image and the plan view image when the volcano is photographed with a surveillance camera at night.

  FIG. 3 is an explanatory diagram for explaining the schematic operation of the camera image designation location interlocking display unit 11, the GIS image designation location interlocking display unit 12, and the planar image designation location interlocking display unit 13.

  The camera image designation location interlocking display unit 11 performs position measurement on the camera image and interlocking measurement display on the three-dimensional GIS image and the plan view image.

  That is, when the camera image designation location interlocking display unit 11 designates an arbitrary position on the camera image (7-1), the cursor is displayed at that position (7-1-1), and on the three-dimensional GIS image. Is calculated (7-2), a cursor is displayed at that position (7-2-1; see the dotted line K1-1 in FIG. 2), and the imaging range on the GIS image The three-dimensional geographical coordinates are calculated (7-3), the three-dimensional topographic coordinates are displayed (7-3-1), and the two-dimensional geographical coordinates excluding the height direction are obtained (7-4), and correspondingly. A cursor is displayed on the two-dimensional geographic coordinates on the plan view image (7-4-1: see dotted line K1-2 in FIG. 2).

  That is, when the position is input from the camera image, the movement angle of the cursor between the various screens can be linked by reproducing the angle of view of the surveillance camera in the three-dimensional geographic coordinates.

  Furthermore, when measuring an arbitrary distance or area on a camera image, apexes (hereinafter simply referred to as points) constituting the distance or area are continuously specified on the camera image, and the three-dimensional geographic coordinates obtained each time are used. These distances and areas are obtained (7-5).

  The GIS image designated location interlocking display unit 12 performs position measurement on a three-dimensional GIS image, and interlocking measurement display on a plan view image and a camera image.

  When an arbitrary position on the three-dimensional GIS image is designated (8-1), a cursor is displayed at that position (on the three-dimensional GIS image) (8-1-1), and the three-dimensional geographical coordinates in the three-dimensional terrain model are displayed. Calculate (8-2), display the three-dimensional coordinates on the three-dimensional GIS image (8-2-1), and obtain two-dimensional geographic coordinates excluding the height direction (8-3) The cursor is displayed on the plan view image (8-3-1); see the dotted line K2-1 in FIG. 2).

  Then, to a relational expression using a collinear conditional expression (an object in space, an image point on the photograph of the object, and a camera geometric condition for three points of the camera lens center on the same straight line). The image coordinates on the camera image are obtained by substituting the three-dimensional geographic coordinates (8-4), and the cursor is displayed at the corresponding position on the camera image (8-4-1: see dotted line K2-2 in FIG. 2). ).

  That is, it is possible to display the movement of the cursor between the various screens when the position is input from the three-dimensional GIS image.

  Furthermore, when measuring an arbitrary distance or area on a three-dimensional GIS image, the points constituting the distance or area are continuously designated on the three-dimensional GIS image, and the three-dimensional geographic coordinates obtained each time are used. The distance and area are determined (8-5).

  The plane image designation location interlocking display unit 13 performs position measurement on the plan view image, and interlocking measurement display on the three-dimensional GIS image and the camera image.

  When an arbitrary position on the plan view image is designated (9-1), a cursor is displayed on the plan view image (9-1-1), and the cursor position is displayed on the two-dimensional geographic coordinate system of the plan view image. To determine the three-dimensional geographic coordinates of the geographic coordinate model included in the three-dimensional terrain model (9-2), and display the cursor Ki on the three-dimensional GIS image corresponding to the plan view image (9-2-1). ; See dotted line K3-1 in FIG. 2) On the other hand, three-dimensional geographic coordinates are obtained and displayed (9-2-2). At this time, similarly, the three-dimensional geographic coordinates are substituted into the relational expression using the collinear conditional expression to obtain the image coordinates on the camera image (9-3), and the cursor is displayed at the corresponding position on the camera image ( 9-3-1; refer to the dotted line K3-2 in FIG. 2) In addition, the cursor is displayed at the corresponding position on the plan view (see the dotted line K1-2).

  In addition, when measuring an arbitrary distance or area on a plan view image, apexes constituting the distance or area are continuously specified on the plan view, and the three-dimensional geographic coordinates obtained each time are used. The area is obtained (9-5).

  The obtained three-dimensional coordinates and two-dimensional coordinates may be displayed in a coordinate value display box.

<Description of each coordinate system>
This embodiment uses a coordinate system described below with reference to FIGS.

(Coordinate conversion 1: screen-image coordinate conversion)
As shown in FIGS. 4 (a) and 4 (b), coordinate conversion 1 is performed by calculating the pixel coordinates on the display screen from the ratio of the number of vertical and horizontal pixels of the display screen to the number of vertical and horizontal pixels of the image (also referred to as image data). (Mx, My) is converted into pixel coordinates (Px, Py) on the camera image (image data). Further, the inverse transformation of the coordinate transformation 1 is referred to as inverse coordinate transformation 1.

(Coordinate transformation 2: Image-photo coordinate transformation)
As shown in FIGS. 4 (c) and 4 (d), the coordinate transformation 2 associates the actual size of the CCD surface (or film surface) of the camera with the number of vertical and horizontal pixels of the camera image (image data). Thus, pixel coordinates (Px, Py) on the image data are converted into actual coordinates on the CCD plane of the camera = photograph coordinates (X ′, Y ′). Further, the inverse transformation of the coordinate transformation 2 is referred to as inverse coordinate transformation 2.

  Further, the coordinate transformation 2 is combined with a general expression for correcting systematic distortion of a photographic image such as a radial distortion of the lens or a displacement of the principal point (deviation between the center of the optical axis of the lens and the center of the CCD). To obtain the photographic coordinates without distortion.

(Coordinate transformation 3: Photo-camera coordinate transformation)
Coordinate transformation 3 assumes a three-dimensional space in which the photographic coordinate plane is the XY axis and the orthogonal axis is the Z axis, and the focal distance (f) is separated from the CCD center in the Z axis direction = the optical center as the origin. Then, as shown in FIGS. 5A and 5B, the camera coordinates (X ′, Y) in which the photographic coordinates (X ′, Y ′), which are two-dimensional coordinates, are replaced with the three-dimensional relative positions from the optical center. ', Z').

This transformation that reversely transforms the coordinate transformation 3 is referred to as inverse coordinate transformation 3.

(Coordinate transformation 4: camera-geographic coordinate transformation)
As shown in FIGS. 5C and 5D, the camera coordinate system is converted into a geographic coordinate system using the geographic coordinates of the optical center of the camera and the inclination of the camera coordinate system with respect to the geographic coordinate system. This transformation that reversely transforms the coordinate transformation 4 is referred to as inverse coordinate transformation 4.

(Coordinate transformation 5: screen-plan view coordinate transformation)
As shown in FIG. 6A and FIG. 6B, the display is based on the ratio between the number of pixels in the display screen (plan view image display screen) and the number of pixels in the screen display range on the plan view image. Pixel coordinates (Mx, My) on the screen are converted to pixel coordinates on the screen display range, and further, the pixel coordinates on the screen display range are converted into a two-dimensional plane from the relative positional relationship of the screen display range on the two-dimensional plan view data. Conversion into pixel coordinates (Px, Py) on the diagram data. Further, the inverse transformation of the coordinate transformation 5 is referred to as an inverse coordinate transformation 5.

(Coordinate transformation 6: Plan view-Plane geographic coordinate transformation)
As shown in FIGS. 6C and 6D, the coordinate transformation 6 associates the geographical coordinates of the four corners of the two-dimensional plan view data with the number of vertical and horizontal pixels of the displayed plan view image data. Thus, the pixel coordinates (Px, Py) on the plan view image data (plan view image) are converted into plane geographic coordinates (X, Y). Further, the inverse transformation of the coordinate transformation 6 is referred to as the inverse coordinate transformation 6.

(Coordinate transformation 7: Plane-three-dimensional geographic coordinate transformation)
As shown in FIGS. 7A and 7B, the coordinate transformation 7 retrieves an elevation value (Z) corresponding to the planar geographic coordinates (X, Y) from a digital elevation model (three-dimensional terrain model). To do. The inverse transformation of coordinate transformation 7 is referred to as inverse coordinate transformation 7.

<Embodiment 1>
FIG. 8 is a schematic configuration diagram of a three-dimensional interlocking measurement display system using an image of the surveillance camera according to the first embodiment. In the first embodiment, the interlocking display of the cursor will be described.

  As shown in FIG. 8, the three-dimensional interlocking measurement display system using the image of the surveillance camera according to the first embodiment includes a display unit 20, a computer main unit 30 (hereinafter simply referred to as main unit 30), a mouse (not shown), It consists of a keyboard.

  As shown in FIG. 8, the main body unit 30 includes a camera selection unit 31, a mode determination unit 32, an angle-of-view setting unit 33, a display processing unit 34, a camera image designation location continuous image display unit 11, A GIS image designated location interlocking display unit 12 and a plan view designated location interlocking display unit 13 are provided. In association with the coordinate model, the 3D GIS image and the plan view image captured by the surveillance camera are displayed in conjunction with each other, and at a specified position (cursor) on any image (camera image, 3D GIS image or plan view image) ) Are displayed in conjunction with each image, and the three-dimensional coordinates of the designated position are notified.

  As a result, even if the surveillance camera 1 is photographing at night, for example, the same three-dimensional GIS image is displayed next to the night surveillance camera image. You can see the video directly (see Figure 2).

  Furthermore, a cursor is displayed at a designated position on the camera image, a curl is displayed at the same position on the three-dimensional GIS image displayed in conjunction with the camera image, and a tertiary on the plan view image. Since the cursor is linked and displayed at the position on the plan view image of the cursor position on the original GIS image, the positional relationship of the cursor on these images can be grasped instantly at a glance without using stepwise means.

  The display of the cursor on the plan view image or the three-dimensional GIS image and the interlocking display of the cursor on another image (such as a disaster prevention bop) will be described in another embodiment.

  Further, the camera image designated location linked display unit 11, the GIS image designated location linked display unit 12, and the plan view designated location linked display unit 13 are collectively referred to as a designated location linked display unit. Will be described as a second embodiment to be described later, and the plan view designated location interlocking display unit 13 will be described as a third embodiment to be described later.

  In addition, when a cursor is specified on the night surveillance camera image, the cursor is linked and displayed on the 3D GIS image and the plan view image, and 3D coordinates are displayed. The three-dimensional position of the point where the landed is immediately known.

(Configuration of each part)
The above-described surveillance cameras 1 (fixed) are arranged at different points, and the surveillance camera images continuously taken by these surveillance cameras 1 are stored in a CD-ROM 2 or the like, and the CD-ROM 2 is regularly analyzed by a supervisor. Deliver to the center.

  The database 5 stores surveillance camera images, 3D terrain data, 2D floor plan data (numerical maps, disaster prevention maps, etc.), internal orientation elements, external orientation elements, etc. (see FIG. 9). Yes. These pieces of information are for each shooting date and are classified and stored for each monitoring camera.

  The monitoring camera image may be stored in a video storage server (not shown), connected to the video storage server and the main unit 30 via a LAN, and displayed on the display unit.

  Further, the monitoring camera 1 and the video storage server may be connected via a LAN to acquire camera image data from the monitoring camera.

  The internal orientation elements are principal point position deviation, lens distortion, focal length, imaging surface size, resolution, and the like, and the external orientation elements are the three-dimensional position and orientation of the surveillance camera at the time of photographing. These orientation elements will be described later.

  The camera selection unit 31 displays the type of monitoring camera (installation location, name, date, etc.) stored in the database 5, and displays the number of the selected monitoring camera as a display processing unit 34 and an angle of view setting unit 33. To inform.

  The angle-of-view setting unit 33 defines the angle of view of the surveillance camera on the three-dimensional terrain model using the orientation elements (internal orientation element, external orientation element) of the surveillance camera of the number notified from the camera selection unit 31. To do. The setting of this angle of view will be described later.

  The display processing unit 34 includes a camera image reproduction processing unit 34a, a GIS image display processing unit 34b, a plan view image display processing unit 34c, and the like, and divides the screen of the display unit 20 and displays camera images on these divided screens. 3D GIS image, plan view image, 3D position, etc. are displayed.

  The screen on which the camera image is displayed is referred to as a camera image display screen, the screen on which the three-dimensional GIS image is displayed is referred to as a GIS image display screen, and the screen on which the plan view image is displayed is referred to as a plan view image display screen. .

  In addition, the plan view image may be a map, a disaster prevention map, a house map, a satellite photograph, an aerial photograph, and other drawings, or a combination of these, but in this embodiment, The map will be described as a plan view image.

  The camera image reproduction processing unit 34a searches the database 5 for a monitoring camera image of the numbered monitoring camera notified from the camera selection unit 31 and displays it (camera display screen).

  In addition, when the camera image is first designated and a point is designated on the image, the camera image reproduction processing unit 34a displays a cursor at the point position and also displays the GIS image display processing unit 34b and the plan view. A cursor is displayed on the image display processing unit 34c. This cursor display is realized by coordinate conversion.

  Furthermore, when each image of the GIS image display processing unit 34b or the plan view image display processing unit 34c is first specified, the camera image reproduction processing unit 34a receives the cursor pixel coordinates (camera When the image display screen coordinates) are notified, the cursor is displayed at the pixel coordinates.

  The GIS image display processing unit 34b uses the 3D terrain model (3D terrain model and texture data) in the database 5 to generate a 3D graphic image (3D GIS image) corresponding to the shooting range (angle of view) of the surveillance camera. Display (GIS image display screen).

  In addition, when a three-dimensional GIS image is first designated and a point is designated on the image, the GIS image display processing unit 34b displays a cursor at the point position, and the camera image reproduction processing unit 34a and The plan view image display processing unit 34c is caused to display a cursor.

  Furthermore, when each image of the camera image reproduction processing unit 34a or the plan view image display processing unit 34c is first designated, the GIS image display processing unit 34b receives the cursor pixel coordinates (GIS) from the designated location interlocking display unit described later. When the image display screen coordinates (Mx, My) are notified, the cursor is displayed at the pixel coordinates.

  The plan view image display processing unit 34c displays the plan view image including the position of the monitoring camera and the photographing object using the plane coordinates of the 3D terrain model of the 3D terrain model in the database 5 and the texture data of the plane (plan view). Graphic image display screen).

  In addition, when a plan view image is first designated and a point is designated on the image, the plan view image display processing unit 34c displays a cursor at the point position, and also displays the camera image reproduction processing unit 34a and A cursor is displayed on the GIS image display processing unit 34b.

  Furthermore, when each image of the camera image reproduction processing unit 34a or the GIS image display processing unit 34b is first designated, the plan view image display processing unit 34c receives the cursor pixel coordinates (plane When the graphic image display screen coordinates are notified, the cursor is displayed at the pixel coordinates.

  When the cursor button (position designation point) or measurement button of each box (not shown) of the display unit 20 is selected, the mode determination unit 32 determines the position designation mode (also simply referred to as a cursor mode) or the measurement mode, and The selected screen is determined, and the determination result is notified to the display processing unit 34 and the corresponding designated location interlocking display unit.

  As shown in FIG. 8, the camera image designation location interlocking display unit 11 includes a camera image designation location three-dimensional geographic coordinate transformation unit 40, a GIS image inverse transformation unit 41, a plan view image inverse transformation unit 42, and a position. A calculation unit 43 and the like are provided.

  In the camera image designation portion, the three-dimensional geographic coordinate conversion unit 40 causes the camera image reproduction processing unit 34a to convert the cursor position (camera screen pixel coordinates Mx, My) on the camera image display screen into camera image data pixel coordinates (Px, Py). (Coordinate conversion 1: refer to FIGS. 4A and 4B), the camera image data pixel coordinates (Px, Py) are converted into photographic coordinates (X ′, Y ′) (coordinate conversion 2: FIG. 4 (c) and FIG. 4 (d)). At this time, the lens distortion is corrected.

  Then, the photographic coordinates (X ′, Y ′) are converted into camera coordinates (X ′, Y ′, Z ′) (coordinate conversion 4: see FIGS. 5A and 5B), and the optical center of the camera. The camera coordinate system is converted into a three-dimensional geographic coordinate system using the geographic coordinates and the inclination with respect to the camera coordinate system (see coordinate conversion 4: FIGS. 5C and 5D).

  The position calculation unit 43 coordinates the coordinates (X, Y) of the line where the straight line connecting the optical center of the camera coordinate system converted into the three-dimensional geographic coordinates and the cursor of the shooting range (on the photograph) intersects the solid surface of the three-dimensional terrain model. , Z) is also obtained as a three-dimensional position and displayed.

  Further, the position calculation unit 43 calculates the coordinates (X, Y) of the above-mentioned points in the two-dimensional geographic coordinates and displays them.

  The GIS image inverse transform unit 41 transforms a point on the solid surface of the three-dimensional terrain model onto a photograph in the camera coordinate system (X ′, Y ′, Z ′) (inverse coordinate transform 4), and this is transformed into the photograph coordinate system. Conversion to (X ′, Y ′) (inverse coordinate conversion 3). However, the inverse coordinate transformation in the GIS image inverse transformation unit does not perform distortion inverse correction.

  However, in some cases, when the image is a distorted three-dimensional GIS image, reverse distortion correction may be performed.

  This is converted into pixel coordinates (Px, Py) in the GIS image coordinate system (inverse coordinate conversion 2), and further converted into pixel coordinates (Mx, My) in the GIS image display screen (inverse coordinate conversion 1). Then, the data is output to the GIS image display processing unit 34b, and the cursor is displayed at the same position as the cursor on the camera image on the three-dimensional GIS image.

  The plan view image inverse conversion unit 42 converts the cursor position (X, Y, Z) obtained on the three-dimensional terrain model into the coordinates of the planar geographic coordinate system (X, Y) (inverse coordinate conversion 7: 7A and 7B), an imaging range is defined in this planar geographic coordinate system, and this imaging range is defined (converted) into an image coordinate system of two-dimensional plan view data (inverse coordinate transformation 6). : Refer to FIG. 6 (c) and FIG. 6 (d)).

  Then, the pixel coordinates (Px, Py) of this image coordinate system are converted to the pixel coordinates (Mx, My) of the screen for displaying a plan view image (inverse coordinate conversion 5: see FIGS. 6A and 6B). ).

  Then, the pixel coordinates (Mx, My) of the screen for displaying the plan view image are output to the plan view image display processing unit 34c, and the cursor is positioned on the plan view image at the same position as the cursor of the camera image and the three-dimensional GIS image. Is displayed.

(Operation)
FIG. 10 is a flowchart for explaining the overall operation of the three-dimensional interlocking measurement display system using the image of the surveillance camera according to the first embodiment.

  FIGS. 11 to 13 are flowcharts for explaining the operation of the camera image designation location interlocking display unit of the three-dimensional interlocking measurement display system using the image of the surveillance camera according to the first embodiment.

  First, initial setting processing is performed (S1). This initial setting processing includes camera selection, preprocessing, type of plan view (either map or disaster prevention map, or both), the size of the display frame of each screen, and the like.

  Next, the angle-of-view setting unit 33 sets the angle of view (shooting range) using the three-dimensional terrain model (three-dimensional terrain model) (S3). This setting of the angle of view sets the angle of view on the three-dimensional terrain model using the internal orientation elements and external orientation elements of the selected camera stored in the database 5.

  In other words, a three-dimensional space is assumed in which the focal point in the Z direction from the CCD center in the camera coordinate system (camera optical center) is the origin, and this camera coordinate system is represented on the three-dimensional terrain model of the camera optical center. The angle of view is set on the three-dimensional terrain model using the geographical coordinates and the tilt of the camera coordinate system (see step S3 in FIG. 10). A specific method is defined by the same conversion formula as in Non-Patent Document 1.

  Next, the camera image reproduction unit 34 reproduces the camera image of the camera selected from the database 5 and displays it on the camera image display screen of the display unit 20, and the 3D image display processing unit 34 b A three-dimensional GIS image is displayed on the GIS image display screen using a three-dimensional terrain model (a three-dimensional geographic coordinate model of the place taken by the surveillance camera), and the plan view image display processing unit 34c is the second of the database 5. The plan view image of the dimensional plan view is displayed on the plan view display screen (S5).

  At this time, the plan view image display processing unit 34c reads and displays the map or the disaster prevention map when the plan view display is only a map or the disaster prevention map, for example. To display.

  Next, the mode determination unit 32 determines the mode (S7), and performs one of cursor-linked display, distance / area calculation processing, or viewpoint change processing based on the determination result (S10, S20, S30). Then, it is determined whether or not it is finished, and the process is finished (S40).

  Next, the cursor-linked display process will be described. The cursor-linked display processing is mainly performed by the camera image designated location linked display section 11, the GIS image designated location linked display portion 12, and the plan view designated location linked display portion 13.

  The camera image designated part interlocking display shown in FIG. 8 will be described with reference to FIGS. 11, 12, and 13.

  The mode determination unit 32 monitors whether there is a cursor instruction (S101). Next, if there is a cursor instruction, it is determined whether the screen is a camera image display screen, a GIS image display screen, or a plan view image display screen (S102). In the case of the camera image display screen, the camera image designation location interlocking display unit 11 is activated. In the case of the GIS image display screen, the GIS image designation location interlocking display unit 12 is activated (S130), and in the case of the plan view image display screen, the plan view designation location interlocking display unit 13 is activated (S150). In this embodiment, the camera image display screen is used.

  When it is determined that the screen is a camera image display screen, the camera image designation location interlocking display unit 11 is activated and performs the following processing.

  The camera image designation location three-dimensional geographic coordinate conversion section 40 of the camera image designation location interlocking display section 11 displays the camera image display screen on the camera image display screen based on the ratio of the number of vertical and horizontal pixels of the camera image display screen and the number of vertical and horizontal pixels of the camera image. Coordinate conversion 1 is performed to convert pixel coordinates (Mx, My) into pixel coordinates (Px, Py) on the camera image (including the cursor) (S103).

  Next, the pixel coordinates (Px, Py) on the camera image are obtained by associating the actual size (shooting range) of the CCD surface (or film surface) of the camera with the pixel coordinates (Px, Py) on the camera image. Coordinate conversion 2 is performed to convert the actual coordinates on the CCD surface of the camera = photograph coordinates (X ′, Y ′) (S104). At this time, systematic distortion of the photographic image such as the radial distortion of the lens and the displacement of the principal point (deviation of the lens optical axis center and the CCD center) is corrected.

Next, a three-dimensional space is assumed in which the photographic coordinate plane is the XY axis, the orthogonal axis is the Z axis, and the focal point (f) is separated from the CCD center in the Z axis direction = the optical center is the origin. Coordinate conversion 3 is performed to convert the photograph coordinates (X ′, Y ′), which are two-dimensional coordinates, into camera coordinates (X ′, Y ′, Z ′) in which the three-dimensional relative position from the optical center is replaced (S105).

  Next, as shown in FIG. 12, coordinate conversion 4 is performed to convert the camera coordinate system to the geographic coordinate system using the geographic coordinates of the optical center of the camera and the tilt of the camera coordinate system with respect to the geographic coordinate system (S106). That is, it is set using the angle of view on the three-dimensional terrain model.

  Note that the coordinates of the point where the straight line connecting the optical center of the camera converted to the geographic coordinate system and the object on the photograph intersects the solid surface of the elevation model = three-dimensional geographic coordinates (X, Y, Z) are calculated. It is determined in part 43.

  Then, the GIS image inverse transform unit 41 and the plan view image inverse transform unit 42 are activated (S107).

  Processing of the GIS image inverse transform unit 41 and the plan view image inverse transform unit 42 will be described with reference to FIG. These conversion units are operating simultaneously.

  The GIS image inverse transform unit 41 performs inverse coordinate transform 4 that transforms the points of the three-dimensional terrain model onto the photograph (corresponding to the photographing range) of the camera coordinate system (X ′, Y ′, Z ′) (S110). .

Next, inverse coordinate transformation 3 is performed to transform this into a photographic coordinate system (X ′, Y ′) (S111). However, since the inverse coordinate transformation in the GIS image inverse transformation unit uses a three-dimensional terrain model, the distortion inverse correction is not performed. However, in the case of a three-dimensional GIS image with distortion, reverse distortion correction is performed.

  Then, inverse coordinate conversion 2 is performed to convert this into pixel coordinates (Px, Py) of the GIS image coordinate system (S112), and further, this is converted to pixel coordinates (Mx, My) of the GIS image display screen. 1 is performed (S113).

  Then, by outputting the display instruction information to the GIS image display processing unit 34b, the cursor is displayed at the same position as the cursor on the camera image on the GIS image (S114).

On the other hand, the plan view image inverse transform unit 42 performs inverse coordinate transform 7 for transforming the cursor position (X, Y, Z) obtained on the three-dimensional terrain model into the coordinates of the planar geographic coordinate system (X, Y). Do (S
120).

  Next, inverse coordinate conversion 6 is performed to convert the planar geographic coordinates into coordinates on the plan view image data (S121).

  Next, inverse coordinate conversion 5 is performed to convert the pixel coordinates (Px, Py) into pixel coordinates (Mx, My) of the plan view image display screen (S122).

  Then, by outputting display instruction information to the plan view image display processing unit 34c for the pixel coordinates (Mx, My) of the plan view image display screen, the cursors of the camera image and the three-dimensional GIS image are displayed on the plan view image. A cursor is displayed at the same position (S123).

  Therefore, for example, even if a volcano explodes at night and lava flies, it is possible to accurately confirm where the position is on the 3D GIS image, and to accurately know the position of the landing point in 3D coordinates, In addition, the position can be known also in the plan view image.

<Embodiment 2>
FIG. 14 is a schematic configuration diagram of a three-dimensional interlocking measurement display system using an image of the monitoring camera according to the second embodiment, and is a configuration diagram of the GIS image designation location interlocking display unit 12. In FIG. 14, the camera selection unit 31, the angle of view setting unit 33, and the mode determination unit 32 are omitted. Further, the description of the position calculation unit 43 is omitted.

  14 includes a GIS image designation location three-dimensional geographic coordinate conversion section 50, a camera image reverse conversion section 51, a plan view image reverse conversion section 52, and the like.

  In the GIS image designation location three-dimensional geographic coordinate conversion unit 50, the GIS image display processing unit 34b converts the cursor position (pixel coordinates Mx, My) of the GIS image display screen into GIS image data pixel coordinates (Px, Py) ( Coordinate conversion 1: See FIGS. 4A and 4B), and converts the GIS image data pixel coordinates (Px, Py) into photographic coordinates (X ′, Y ′) (coordinate conversion 2: FIG. 4 ( c) and FIG. 4 (d)). At this time, the lens distortion is not corrected.

  However, in the case of a three-dimensional GIS image with distortion, distortion correction is performed.

  Then, the photographic coordinates (X ′, Y ′) are converted into camera coordinates (X ′, Y ′, Z ′) (coordinate conversion 4: see FIGS. 5A and 5B), and the optical center of the camera. The camera coordinate system is converted into a three-dimensional geographic coordinate system using the geographic coordinates and the inclination with respect to the camera coordinate system (see coordinate conversion 4: FIGS. 5C and 5D).

  The camera image inverse transform unit 51 transforms the point of the three-dimensional terrain model onto a photograph in the camera coordinate system (X ′, Y ′, Z ′) (inverse coordinate transform 4), and converts this into the photograph coordinate system (X ′). , Y ′) (inverse coordinate transformation 3). At this time, lens distortion correction is performed.

  This is converted into pixel coordinates (Px, Py) in the camera image coordinate system (inverse coordinate conversion 2), and further converted into pixel coordinates (Mx, My) in the camera image display screen (inverse coordinate conversion 1). Then, display instruction information is output to the camera image display processing unit 34a, and the cursor is displayed at the same position as the cursor on the three-dimensional GIS image on the camera image.

  The plan view image inverse transform unit 52 transforms the cursor position (X, Y, Z) obtained on the three-dimensional terrain model into the coordinates of the planar geographic coordinate system (X, Y) (inverse coordinate transform 7: 7 (a) and 7 (b)), this planar geographic coordinate system is defined (transformed) (inverse coordinate transformation 6: see FIGS. 6 (c) and 6 (d)).

  Then, the pixel coordinates (Px, Py) in the imaging range of the image coordinate system are converted into the pixel coordinates (Mx, My) on the plan view image display screen (inverse coordinate conversion 5: FIGS. 6A and 6B). )reference).

  Then, display instruction information of the pixel coordinates (Mx, My) of the plan view image display screen is output to the plan view image display processing unit 34c, and the cursor of the camera image and the three-dimensional GIS image is displayed on the plan view image. Display the cursor at the same position.

(Description of operation)
15 to 17 are flowcharts for explaining the GIS image designation location interlocking display unit 12 according to the second embodiment.

  The GIS image designation location three-dimensional geographic coordinate converter 50 converts the pixel coordinates (Mx, My) on the GIS image display screen into pixel coordinates (Px, Py) on the three-dimensional GIS image (including a cursor). (S131).

  Next, the pixel coordinates (Px, Py) on the three-dimensional GIS image are converted into the photographic coordinates (X ′, Py) by associating the angle of view (shooting range) with the pixel coordinates (Px, Py) on the three-dimensional GIS image. Y ′) is performed for coordinate conversion 2 (S132).

  Next, a three-dimensional space is assumed in which the photographic coordinate plane is the XY axis, the orthogonal axis is the Z axis, and the focal point (f) is separated from the CCD center in the Z axis direction = the optical center is the origin. Coordinate conversion 3 is performed to convert the photograph coordinates (X ′, Y ′), which are two-dimensional coordinates, into camera coordinates (X ′, Y ′, Z ′) in which the three-dimensional relative position from the optical center is replaced (S133).

  Next, as shown in FIG. 16, by using the inclination of the camera coordinate system with respect to the three-dimensional geographic coordinate system, coordinate conversion 4 for converting the camera coordinate system to the geographic coordinate system is performed (S134). That is, conversion is performed using the angle of view set on the three-dimensional geographic coordinate model.

  Then, the camera image inverse transform unit 51 and the plan view image inverse transform unit 52 are activated (S135).

  Processing of the camera image inverse transform unit 51 and the plan view image inverse transform unit 52 will be described with reference to FIG. These conversion units are operating simultaneously.

  The inverse transformation unit for camera image 51 performs inverse coordinate transformation 4 for transforming the points of the three-dimensional terrain model onto the photograph (corresponding to the photographing range) of the camera coordinate system (X ′, Y ′, Z ′) (S136). .

  Next, inverse coordinate transformation 3 is performed to transform this into a photographic coordinate system (X ′, Y ′) (S137). At this time, distortion is corrected for conversion.

  Next, inverse coordinate transformation 2 is performed to convert this into pixel coordinates (Px, Py) in the camera image coordinate system (S138), and further, inverse coordinates are converted into pixel coordinates (Mx, My) on the camera image display screen. Conversion 1 is performed (S139).

  Then, by outputting the display instruction information to the camera image reproduction processing unit 34a, the cursor is displayed at the same position as the cursor on the three-dimensional GIS image on the camera image (S140).

  On the other hand, the plan view image inverse transform unit 52 performs an inverse coordinate transform 7 for transforming the cursor position (X, Y, Z) obtained on the three-dimensional terrain model into the coordinates of the planar geographic coordinate system (X, Y). This is performed (S141).

  Next, inverse coordinate transformation 6 is performed (S142) that defines (transforms) this planar geographic coordinate system.

  Next, inverse coordinate conversion 5 is performed to convert the pixel coordinates (Px, Py) of the image coordinate system into pixel coordinates (Mx, My) of the plan view image display screen (S143).

  Then, by outputting display instruction information to the plan view image display processing unit 34c for the pixel coordinates (Mx, My) of the plan view image display screen, the cursor of the camera image and the three-dimensional GIS image can be displayed on the plan view image. A cursor is displayed at the same position (S144). At this time, the two-dimensional position (X, Y) of the cursor in the planar geographic coordinate system is displayed.

  Therefore, for example, even if a volcano explodes at night and lava flies, it is possible to accurately confirm where the position is from the plan view image, and it is possible to accurately know the position of the landing point with two-dimensional coordinates, and The position can also be known on the camera image and the three-dimensional GIS image.

<Embodiment 3>
FIG. 18 is a schematic configuration diagram of a three-dimensional interlocking measurement display system using an image of a monitoring camera according to the third embodiment, and is a configuration diagram of a plan view image designation location interlocking display unit 14. In FIG. 18, the camera selection unit 31 and the mode determination unit 32 are omitted. Further, the description of the position calculation unit 43 is omitted.

  The plan view image designation location link display unit 14 shown in FIG. 18 includes a plan view image designation location three-dimensional geographic coordinate conversion unit 60, a camera image reverse conversion unit 61, a GIS image reverse conversion unit 62, and the like. .

  The plan view image designation location 3D geographic coordinate conversion unit 60 converts the cursor position (X, Y, Z) obtained on the 3D terrain model into the coordinates of the plane geographic coordinate system (X, Y) (inversely). Coordinate transformation 7: See FIG. 7 (a) and FIG. 7 (b)) and define (transform) this planar geographic coordinate system (inverse coordinate transformation 6: see FIG. 6 (c) and FIG. 6 (d)).

  Then, the pixel coordinates (Px, Py) of the imaging range of the image coordinate system are converted into the pixel coordinates (Mx, My) of the screen for displaying a plan view image (inverse coordinate conversion 5: FIG. 6A and FIG. 6). b)).

  Then, the pixel coordinates (Mx, My) of the screen for displaying the plan view image are output to the plan view image display processing unit 34c, and the cursor is positioned on the plan view image at the same position as the cursor of the camera image and the three-dimensional GIS image. Is displayed.

The camera image inverse transformation process 61 transforms the points of the three-dimensional terrain model onto a photograph in the camera coordinate system (X ′, Y ′, Z ′) (inverse coordinate transformation 4), which is converted into the photograph coordinate system (X ′). , Y ′) (inverse coordinate transformation 3). At this time, lens distortion correction is performed.

  This is converted into pixel coordinates (Px, Py) in the camera image coordinate system (inverse coordinate conversion 2), and further converted into pixel coordinates (Mx, My) in the camera image display screen (inverse coordinate conversion 1). Then, display instruction information is output to the camera image display processing unit 34a, and the cursor is displayed at the same position as the cursor on the three-dimensional GIS image on the camera image.

  The GIS image inverse transform unit 62 transforms the points of the three-dimensional surface of the three-dimensional terrain model onto the camera coordinate system (X ′, Y ′, Z ′) photograph (inverse coordinate transform 4), and this is transformed into the photographic coordinate system. Conversion to (X ′, Y ′) (inverse coordinate conversion 3). However, the inverse coordinate transformation in the GIS image inverse transformation unit does not perform distortion inverse correction. However, in the case of a three-dimensional GIS image with distortion, reverse distortion correction is performed.

  This is converted into pixel coordinates (Px, Py) in the GIS image coordinate system (inverse coordinate conversion 2), and further converted into pixel coordinates (Mx, My) in the GIS image display screen (inverse coordinate conversion 1). Then, display instruction information is output to the GIS image display processing unit 34b, and the cursor is displayed at the same position as the cursor on the camera image on the three-dimensional GIS image.

(Description of operation)
19 and 20 are flowcharts for explaining the plan view image designation part interlocking display unit 14 of the third embodiment.

  The three-dimensional geographic coordinate conversion unit 60 for designating the plan view image pixel coordinates from the ratio of the number of vertical and horizontal pixels of the plan view image display screen and the number of vertical and horizontal pixels of the screen display range on the plan view image display screen. (Mx, My) is converted into pixel coordinates on the display range of the plan view display screen (corresponding to the shooting range), and further, the pixels on the screen display range are calculated from the relative positional relationship of the screen display range on the two-dimensional plan view data. Coordinate conversion 5 is performed to convert the coordinates into pixel coordinates (Px, Py) on the two-dimensional plan view data (S151: see FIGS. 6A and 6B).

  Next, the pixel coordinates (Px) on the plan view image are obtained by associating the geographical coordinates of the four corners of the two-dimensional plan view data with the number of vertical and horizontal pixels of the displayed plan view data (plan view image of the imaging range). , Py) is converted to plane geographic coordinates (X, Y) (S152): see FIGS. 6C and 6D).

  Next, coordinate conversion 7 is performed to search the elevation value (Z) corresponding to the planar geographic coordinates (X, Y) from the digital elevation model (three-dimensional geographic coordinate model) (S153: FIG. 7A and FIG. 7). (See (b)).

Then, the GIS image reverse conversion and camera image reverse conversion unit is activated and the position calculation unit is activated (S154).

  The GIS image inverse transform unit 62 performs an inverse coordinate transform 4 for transforming the points of the three-dimensional terrain model onto the photograph (corresponding to the photographing range) of the camera coordinate system (X ′, Y ′, Z ′) (S155, S156).

Next, inverse coordinate transformation 3 is performed to convert this into a photographic coordinate system (X ′, Y ′) (S157). However, since the inverse coordinate transformation in the GIS image inverse transformation unit uses a three-dimensional terrain model, the distortion inverse correction is not performed. However, in the case of a three-dimensional GIS image with distortion, reverse distortion correction is performed.

  Then, inverse coordinate conversion 2 is performed to convert this into pixel coordinates (Px, Py) in the GIS image coordinate system (S158), and further, this is converted into pixel coordinates (Mx, My) on the GIS image display screen. 1 is performed (S159).

  Then, by outputting to the GIS image display processing unit 34b, the cursor is displayed at the same position as the cursor on the camera image on the three-dimensional GIS image (S160).

  On the other hand, the camera image inverse transform unit 61 performs inverse coordinate transform 4 that transforms the points of the three-dimensional terrain model onto a photograph (corresponding to the photographing range) of the camera coordinate system (X ′, Y ′, Z ′) ( S161, S162).

  Next, inverse coordinate transformation 3 is performed to transform this into a photographic coordinate system (X ′, Y ′) (S163). At this time, distortion is corrected for conversion.

  Next, inverse coordinate conversion 2 is performed to convert this into pixel coordinates (Px, Py) in the camera image coordinate system (S164), and further, this is converted into pixel coordinates (Mx, My) on the camera image display screen. Conversion 1 is performed (S165).

  Then, by outputting to the camera image reproduction processing unit 34a, the cursor is displayed at the same position as the cursor on the three-dimensional GIS image on the camera image (S166).

  Therefore, when the cursor is designated on the plan view image, the cursor is displayed at the corresponding position on the three-dimensional GIS image and the camera image, and the three-dimensional position and the two-dimensional position are displayed. It will be possible to grasp the relative relationship of.

<Embodiment 4>
FIG. 21 is a schematic configuration diagram of a three-dimensional interlocking measurement display system using an image of the surveillance camera according to the fourth embodiment, and is a configuration diagram of a distance / area calculation process. The distance / area calculation can be performed from any screen, but in this embodiment, an example of calculating the distance / area from the camera image will be described.

  As shown in FIG. 21, a surface area / distance calculator 70 is provided. The surface area / distance calculation unit includes an outline data generation unit 71, an outline point sequence data generation unit 72, a surface area / length calculation unit 73, and the like.

  Each of the camera image reproduction processing unit 34a and the plan view display processing unit 34c of the display processing unit 34 has a function of forming a polygon in a specified range.

  When a polygon (a surface closed by sequentially specifying a start point and an end point) is displayed on the camera image, the contour data calculation unit 71 obtains the pixel coordinates of the vertex of the contour of the polygon, and calculates this as the camera image designation location. It is sent to the interlocking display unit 11.

When a polygon is displayed on the camera image, the contour point sequence data generation unit 72 obtains the pixel coordinates of each line forming the contour of the polygon, and sends this to the camera image designation location interlocking display unit 11.

  That is, the camera image designation location three-dimensional geographic coordinate conversion unit 40 of the camera image designation location interlocking display section 11 converts the vertex pixel coordinates (camera screen pixel coordinates Mx, My) into camera image data pixel coordinates (Px, Py), respectively. (Coordinate conversion 1), and the camera image data pixel coordinates (Px, Py) are converted into photographic coordinates (X ′, Y ′) (coordinate conversion 2). At this time, the lens distortion is corrected.

  Then, the photographic coordinates (X ′, Y ′) are converted into camera coordinates (X ′, Y ′, Z ′) (coordinate conversion 4), and the camera coordinates are converted into a three-dimensional geographic coordinate system (coordinate conversion 4).

  Then, the plan view image inverse conversion unit 42 converts the four-corner three-dimensional coordinates (X, Y, Z) obtained on the three-dimensional terrain model into the coordinates of the planar geographic coordinate system (X, Y) ( Inverse coordinate transformation 7), defined (transformed) into this planar geographic coordinate system (inverse coordinate transformation 6).

  Next, the pixel coordinates (Px, Py) at the apex of the imaging range of the image coordinate system are respectively converted into pixel coordinates (Mx, My) on the plan view image display screen (inverse coordinate conversion 5).

  Then, the pixel coordinates (Mx, My) of the vertex of the plan view image display screen are output to the plan view image display processing unit 34c, and the polygons of the camera image are displayed at the same position on the plan view image.

  On the other hand, when the point sequence data (pixel coordinates) from the contour point sequence data generation unit 72 is input, the GIS image inverse conversion unit 41 converts each point into a camera coordinate system (X ′, Y ′, Z ′). (Inverse coordinate transformation 4), this is transformed into a photographic coordinate system (X ', Y') (inverse coordinate transformation 3). However, the inverse coordinate transformation in the GIS image inverse transformation unit does not perform distortion inverse correction. However, in the case of a three-dimensional GIS image with distortion, reverse distortion correction is performed.

  These points are converted into pixel coordinates (Px, Py) in the GIS image coordinate system (inverse coordinate conversion 2), and further, these points are converted into pixel coordinates (Mx, My) on the GIS image display screen (inverse). Coordinate conversion 1) and output to the GIS image display processing unit 34b.

  Therefore, on the three-dimensional GIS image, a range corresponding to the polygon three-dimensionally is displayed as shown in FIG.

  Furthermore, the surface area / length calculation unit 73 extracts the three-dimensional values of the respective points corresponding to the polygons of the three-dimensional terrain model calculated by the position calculation unit 43 in accordance with the surface area calculation instruction to obtain the surface area (see FIG. See 22 (b).

  In addition, when the start point and the end point are determined on the polygon and the distance is specified, the three-dimensional coordinates of each point between the start point and the end point are obtained from the position calculation unit 43, and the three-dimensional length (on the ground surface) is determined. (Refer to FIG. 22B).

  It is possible to select a straight line distance or a distance on the ground surface.

<Embodiment 5>
In the fifth embodiment, as shown in FIG. 23, a three-dimensional image in which the viewpoint is changed by selecting the viewpoint change button is displayed. At this time, a camera position is designated on the three-dimensional terrain model, and a three-dimensional GIS image of the viewpoint corresponding to the control button (up, left, right, rotation) is displayed in the photographing range (angle of view) around this. That is, the angle of view of the designated camera position and orientation is set on the three-dimensional terrain model in the same manner as in the above embodiment, and a three-dimensional GIS image of the shooting range at this angle of view is displayed.

  Therefore, usually, many volcano monitoring cameras can quantitatively suppress the location (three-dimensional geographic coordinates) of the pyroclastic flow, cinder, etc., the reach distance / area, and the like three-dimensionally.

  Furthermore, the disaster area digitized on the camera video can be projected onto a plan view such as a numerical map image so as to assume and confirm the disaster area.

  In each of the above embodiments, the monitoring camera images continuously taken by the monitoring camera 1 are stored in the CD-ROM 2 or the like, and the CD-ROM 2 is periodically delivered to the analysis center by the monitoring person. It may be transmitted from the camera directly to the video storage server via the network and stored.

<Embodiment 6>
In the embodiment described above, the camera image reproduction processing unit 34a is provided with the camera image designated location continuous image display unit 11 correspondingly, and the GIS image display processing unit 34b is provided with the GIS image designated location linked display unit 12 corresponding thereto. In the above description, the plan view designated location interlocking display unit 13 is provided in the plan view image display processing unit 34c. However, interlocking display may be realized by configuring as shown in FIG.

  The display processing unit 34 is the same as that in each of the above embodiments, but will be described more specifically in the sixth embodiment. Note that there are various methods for specific processing, and it goes without saying that this embodiment is an example.

  Further, the camera image reproduction processing unit 34a will be described separately for a camera image reproduction unit 34ab (video capture) and a camera image display processing unit 34aa.

  FIG. 24 is a schematic configuration diagram of a three-dimensional interlocking measurement display system using an image of the surveillance camera according to the sixth embodiment. However, FIG. 24 omits the camera selection unit 31, the mode determination unit, and the position calculation unit 43.

  As shown in FIG. 24, a display control unit 80 for controlling the screen of the display unit 20, a display processing unit 34 for sending image data to be displayed by the display control unit 80, and a number corresponding to the type of the screen. An image display processing unit creating unit 81 for generating an image processing unit is provided.

  Furthermore, the coordinate conversion part 84, the selection part 85, etc. are provided.

  The display control unit 80 includes a display buffer 80a, and generates display areas (first area, second area,...) Based on the number of divisions from the screen display processing unit creation unit 81 in the display buffer 80a ( For example, it is divided into four).

  That is, the display control unit 80 grasps the size of each display area. Then, the display buffer 80a is periodically read and the image data is displayed on the screen of the display unit 20.

  The image display processing unit creation unit 81 determines the type of the input screen, causes the display buffer 80a (image memory) of the display control unit 80 to generate this number of areas, and the screen type (camera image, tertiary). The display processing unit 34 generates an image display processing unit corresponding to the original GIS image, plan view,.

  At this time, if the image display processing unit creating unit 81 has four types of display, the upper left area of the screen is an area for a camera image (also referred to as a first area), and the upper right area of the screen is an area of a three-dimensional GIS image ( The display processing unit 34 indicates that the lower left area is also a plan view (map) area (also referred to as a third area), and the lower right area is a disaster prevention map area (also referred to as a fourth area). Set to. The conditions for these areas can be changed. For example, a third area below the camera image area (first area) may be used as an area for a three-dimensional GIS image.

  The display processing unit 34 generates a number of screen display processing units corresponding to the type of screen from the screen display processing unit creation unit 81. In the present embodiment, a camera image display processing unit 34aa, a GIS image display processing unit 34b, a planar image display processing unit 34c, and the like are generated and associated with the area of the display buffer 80a of the display control unit 80, each of which is a display buffer. It has a function to monitor events on the area.

  These image display processing units generate image display buffers 34ad1, 34bd1, 34cd1,. These image display buffers are preferably regions corresponding to the areas generated in the display buffer of the display control unit 80.

  In addition, these image display processing units generate new buffers 34ad2, 34bd2,... (For example, a cursor buffer, a 3D coordinate display buffer each time a data write instruction is issued after an image is written to the image buffer.・ ・) Is generated. The new buffer has a size corresponding to the image buffer.

  Further, each of these image display processing units synthesizes the data written in the image display buffer and the new buffer at regular time intervals, and outputs them to the display control unit 80.

  The display control unit 80 writes the image data output from each image display processing unit in a corresponding area of the display buffer.

  For example, the camera image display processing unit 34aa generates a camera image display buffer 34ad1 upon activation, activates the camera image reproduction unit 34ab, and outputs a reproduction image from the camera image reproduction unit 34ab. (Movie) is written in the camera image display buffer 34ad1.

  In addition, when a cursor is designated on the monitored screen (camera screen: first area), the camera image display processing unit 34aa immediately generates the cursor display buffer 34ad2 and designates the camera screen. A cursor (mark) is written at the position of the cursor display buffer 34ad2 corresponding to the position. Then, the camera image in the camera image display buffer 34ad1 and the cursor image in the cursor display buffer 34ad2 are combined and output to the display control unit 80.

  Furthermore, when the cursor mode (including the cursor display instruction) is notified from another image display processing unit, the cursor display buffer 34ad2 is immediately generated and obtained by the coordinate conversion selected by the selection unit 85. The cursor position on the camera image is written in the cursor display buffer 34ad2, and is combined with the camera image in the camera image display buffer 34ad1 and output to the display control unit 80.

  The GIS image display processing unit 34b generates a GIS image display buffer 34bd1 upon activation, and converts the three-dimensional GIS image of the imaging range obtained by the coordinate conversion selected by the selection unit 85 into the three-dimensional landform of the database 5. Data is read out and written into the GIS image display buffer 34bd1.

  In addition, when a cursor is designated on the monitored screen (GIS screen: second area), the GIS image display processing unit 34b immediately generates the cursor display buffer 34bd2 and also displays the cursor display buffer 34bd2. Then, a cursor (mark) is written at a position corresponding to the designated position on the GIS screen. Then, the GIS image in the GIS image display buffer 34bd1 and the cursor image in the cursor display buffer 34bd2 are combined and output to the display control unit 80.

  Further, when the cursor mode (including the cursor display instruction) is notified from another image display processing unit, the cursor display buffer 34bd2 is immediately generated and obtained by the coordinate conversion selected by the selection unit 85. The cursor position on the three-dimensional GIS image is written into the cursor display buffer 34ad2. Then, the GIS image in the GIS image display buffer 34bd1 and the cursor image in the cursor display buffer 34bd2 are combined and output to the display control unit 80. As a result, a cursor corresponding to the cursor position designated on another screen is displayed on the three-dimensional GIS image.

  The plan view image display processing unit 34c generates a plan view display buffer 34cd1 upon activation, and a predetermined range obtained by the coordinate conversion selected by the selection unit 85 (a range including the monitoring camera position and the photographing object). ) Is read from the database 5, written in the plan view display buffer 34 cd 1, and output to the display control unit 80.

  Further, when a cursor is designated on the monitored screen (plan view screen: third area), the plan view image display processing unit 34c immediately generates the cursor display buffer 34cd2 and displays the cursor. A cursor (mark) is written in the buffer 34cd2 at a position corresponding to the designated position on the screen. Then, the plan view of the plan view display buffer 34cd1 and the cursor image of the cursor display buffer 34cd2 are combined and output to the display control unit 80.

  Further, when the cursor mode (including the cursor display instruction) is notified from another image display processing unit, the cursor display buffer 34cd2 is immediately generated and obtained by the coordinate conversion selected by the selection unit 85. The cursor is written in the pixel coordinates (cursor display buffer 34cd2: plan view screen). Then, the plan view image of the plan view display buffer 34cd1 and the cursor image of the cursor display buffer 34cd2 are combined and output to the display control unit 80.

  The coordinate conversion unit 84 includes a coordinate conversion 1 processing unit 101a, an inverse coordinate conversion 1 processing unit 101b, a coordinate conversion 2 processing unit 102a, an inverse coordinate conversion 2 processing unit 102b, a coordinate conversion 3 processing unit 103a, and an inverse coordinate conversion 3 processing unit 103b. , Coordinate transformation 4 processing unit 104a, inverse coordinate transformation 4 processing unit 104b, inverse coordinate transformation 5 processing unit 105a, coordinate transformation 5 processing unit 105b, inverse coordinate transformation 6 processing unit 106a, inverse coordinate transformation 6 processing unit 106b, coordinate transformation 7 A processing unit 107 a and a coordinate conversion 7 processing unit 107 b are provided, and coordinate conversion is performed using the memory 92.

  The coordinate conversion 1 processing unit 101a displays the pixel coordinates (Mx, My) of the cursor on the display screen (camera image display screen, GIS image display screen or plan view image display screen) from the cursor position reading unit 82 on the display screen. Conversion into pixel coordinates (Px, Py) of image data (coordinate conversion 1). That is, the coordinate conversion 1 is performed by converting the pixel coordinates (Mx, My) of the cursor on the display screen to the pixel coordinates (Px, Py) on the image data from the ratio of the number of vertical and horizontal pixels on the display screen to the number of vertical and horizontal pixels on the image data. The coordinate conversion 1 is converted to (1).

  This will be specifically described below.

  For example, when a cursor is first designated on the camera image display screen and the cursor is displayed on the camera image display screen, the camera image display processing unit 34aa is a cursor on the camera image display screen (the display buffer 34ad2 corresponds). When the pixel coordinates Mx, My (cursor coordinates of the camera screen) are output, the cursor coordinates of the camera screen are converted into pixel coordinates (Px, Py) of the camera image (coordinate conversion 1: FIG. 4A). And FIG. 4B).

  In addition, when a cursor is first designated on the GIS image display screen and the cursor is displayed on the GIS image display screen, the GIS image display processing unit 34b performs a cursor on the GIS image display screen (corresponding to the display buffer 34bd2). When the pixel coordinates Mx, My (cursor coordinates of the GIS image display screen) are output, the cursor coordinates of the GIS image display screen are converted into GIS image data pixel coordinates (Px, Py) (coordinate conversion 1: FIG. 4 (a) and FIG. 4 (b)).

  In addition, when a cursor is first designated on the plan view image display screen and the cursor is displayed on the plan view image display screen, the pixel coordinates Mx of the cursor in the display buffer 34cd2 of the plan view image display processing unit 34c, When My (cursor coordinates of the plan view image display screen) is output, the cursor coordinates on the plan view image display screen are converted into pixel coordinates (Px, Py) of the plan view image data (coordinate conversion 1: FIG. 4). (See (a) and FIG. 4 (b)). ).

  The inverse coordinate conversion 1 processing unit 101b is input from the ratio between the pixel coordinates of the aspect ratio of the pixel coordinates (Px, Py) on the image (camera image, three-dimensional GIS image or plan view image) and the number of pixels on the display screen. Inverse conversion coordinate conversion 1 is performed to convert the pixel coordinates (Px, Py) of the cursor on the displayed image into the cursor pixel coordinates (Mx, My) on the display screen.

  The coordinate conversion 2 processing unit 102a performs coordinate conversion 2 for converting the pixel coordinates (Px, Py) of the cursor on the image data into photo coordinates (X ′, Y ′).

This will be specifically described.
First, a cursor is specified on the camera image display screen, the cursor is displayed on the camera image display screen, and the pixel coordinates (Mx, My) of the cursor are changed to the pixel coordinates (Px, When converted to Py), the actual coordinate on the CCD surface of the camera = photograph coordinate (X ′, Y ′). At this time, systematic distortion of the photographic image such as the radial distortion of the lens and the displacement of the principal point (deviation between the center of the optical axis of the lens and the center of the CCD) is corrected.

  First, a cursor is designated on the GIS image display screen, the cursor is displayed on the GIS image display screen, and the pixel coordinates (Mx, My) of the cursor on the GIS image display screen are displayed on the GIS image by the coordinate conversion 1 described above. Is converted into the actual coordinates on the CCD surface of the camera = photograph coordinates (X ′, Y ′).

  First, a cursor is designated on the plan view image display screen, and the cursor is displayed on the plan view image display screen. By the above-described coordinate conversion 1, the pixel coordinates (Mx, When My) is converted into pixel coordinates (Px, Py) on the plan view image, this is converted into real coordinates on the CCD plane of the camera = photograph coordinates (X ′, Y ′).

  The inverse coordinate conversion 2 processing unit 102b performs inverse coordinate conversion 2 for converting the cursor coordinates converted into the photographic coordinate system (X ', Y') into the pixel coordinates (Px, Py) of the cursor on the image data.

  The coordinate conversion 3 processing unit 103a converts the pixel coordinates of the cursor on the image data converted into the photographic coordinate system (X ′, Y ′) by the coordinate conversion 2 in the camera coordinate system (X ′, Y ′, Z ′). It converts into a coordinate (refer Fig.5 (a) and FIG.5 (b)).

  This will be specifically described.

  First, a cursor is designated on the camera image display screen, the cursor is displayed on the camera image display screen, and the above-described coordinate transformation 2 (via coordinate transformation 1) is used to obtain a photographic coordinate system (X ′, Y ′). The pixel coordinates of the cursor on the camera image converted into (2) are converted into the coordinates of the camera coordinate system (X ′, Y ′, Z ′).

  First, a cursor is designated on the GIS image display screen, the cursor is displayed on the GIS image display screen, and the above-described coordinate transformation 2 (via coordinate transformation 1) is used to obtain a photographic coordinate system (X ′, Y When converted to '), the pixel coordinates of the cursor in the photographic coordinate system are converted to the coordinates of the camera coordinate system (X', Y ', Z').

  The inverse coordinate conversion 3 processing unit 103b converts the cursor coordinates of the camera coordinate system (X ′, Y ′, Z ′) to the photographic coordinate system (X ′, Y ′).

  The coordinate conversion 4 processing unit 104a calculates each point of the shooting range of the angle of view (position, tilt, shooting range; photograph set in the camera coordinate system (photo coordinate system)) set by the view angle setting processing unit 33. The coordinates of each point of the 3D terrain model in the database 5 are converted.

  When the cursor is designated on the camera image display screen or the GIS image display screen, the cursor position on the photograph and the optical center in the camera coordinate system obtained through coordinate conversion 1, coordinate conversion 2, and coordinate conversion 3 are displayed. Are converted into 3D geographic coordinates at the point where the straight line passing through and intersects the 3D terrain model of the 3D terrain data.

  The inverse coordinate conversion 4 processing unit 104b is a conversion that performs the reverse of the coordinate conversion 4. 3D coordinates of each point of the 3D terrain model of each point in the shooting range of the angle of view (position, tilt, shooting range; photograph set in the camera coordinate system (photo coordinate system)) converted by coordinate conversion 4 Is converted into the coordinates on the photo set in the camera coordinate system.

  When the cursor is specified on the camera image display screen or the GIS image display screen, the three-dimensional geographic coordinates of the cursor of the three-dimensional terrain model converted by the coordinate conversion 4 are set in the above-mentioned camera coordinate system. Perform inverse transformation to convert to coordinates on the photograph.

  The coordinate conversion 5 processing unit 105b determines the pixel coordinates (Mx, My: from the display buffer 34cd2) of the cursor of the plan view image display screen from the cursor position reading unit 82, the number of pixels in the vertical and horizontal directions of the plan view display screen, and the plane. The pixel coordinates on the plan view image display screen are converted into the pixel coordinates on the screen display range from the ratio of the vertical and horizontal numbers of pixels in the screen display range on the plan image data (plan view memory 35). Then, coordinate conversion is performed to convert pixel coordinates on the screen display range into pixel coordinates (Px, Py) on the plan view image data from the relative positional relationship between the plan view image data and the screen display range on the plan view image data. 5 is performed (see FIG. 6A and FIG. 6B).

  The inverse coordinate transformation 5 processing unit 105 b is an inverse transformation of the coordinate transformation 5. This performs an inverse transformation that converts the pixel coordinates (Px, Py) of the plan view image data in the converted screen display range to the pixel coordinates (Mx, My) of the plan view image display screen.

  The coordinate transformation 6 processing unit 106a associates the coordinates of the four corners of the plan view image data with the number of vertical and horizontal pixels of the two-dimensional plane geographic coordinate system, thereby causing pixel coordinates (Px, Py) on the plan view image data. Is converted into geographic coordinates in a two-dimensional planar geographic coordinate system (see FIGS. 6C and 6D).

  The inverse coordinate transformation 6 processing unit 106 b is an inverse transformation of the coordinate transformation 6. This converts the cursor position of the plane image display screen converted into the two-dimensional plane geographic coordinate system into pixel coordinates (Px, Py) on the plane view image data.

  The coordinate conversion 7 processing unit 107a performs coordinate conversion 7 for converting the cursor position converted into the geographic coordinates of the two-dimensional planar geographic coordinate system into the three-dimensional coordinates on the three-dimensional terrain model.

  The inverse coordinate transformation 7 processing unit 107 b is an inverse transformation of the coordinate transformation 7. This converts the 3D coordinates of the cursor defined in the 3D terrain model into a 2D planar geographic coordinate system.

  The selection unit 85 determines a screen (camera display screen, GIS display screen, or plan view display screen) in which an initial cursor is designated from the display processing unit 34 in accordance with an image display instruction (system start). The above-described various coordinate conversion processes are selected according to the determined type of screen. A program for selection is stored in the memory 93.

  By selecting these various coordinate conversion processes, the process of the camera image designation location interlocking display unit 11, the GIS image designation location interlocking display unit 12 or the plan view designation location interlocking display unit 13 of the above embodiment is performed.

  The operation of this selection unit will be described below with reference to the flowcharts of FIGS.

As an initial setting (S200),
The angle of view is already set (stored in the memory 91) by the angle of view setting processing unit 33 (S201), a camera image is displayed (S202), a plan view image is displayed (S203), and a three-dimensional GIS image is displayed. Is displayed (S204).

  When displaying the camera image, the three-dimensional GIS image, and the plan view image, the display control unit 80 generates a number of areas based on the number of divisions from the screen display processing unit creation unit 81 in the display buffer 80a (4 Screens (first area: camera image display screen, second area: three-dimensional GIS image display screen) monitored by each image display processing unit (camera, three-dimensional GIS, plan view) of the display processing unit 34, A display buffer (34ad1, 34bd1,...) Having the number of pixels corresponding to the size (also referred to as a screen display range) is generated.

  The camera image display processing unit 34aa activates the camera image playback unit 34ab, and each time a playback image is output from the camera image playback unit 34ab, the camera image display buffer 34ad1 Is output to the display control unit 80.

  The plan view image display processing unit 34c reads the input photographing object information (the position of the photographing object, the position of the monitoring camera, etc.), and uses the two-dimensional plan view of the database 5 as pixel coordinates based on the size of the camera image display screen. (Px, Py), and a predetermined range including the position of the monitoring camera and the object to be photographed in the two-dimensional plan view data is written as a display range in the plan view display buffer 34cd1, and this is written to the display control unit 80. Output.

  The GIS image display processing unit 34b reads the image data (texture) of the region of the 3D terrain data at the angle of view set by the angle of view setting unit 33 to the 3D image display buffer 34bd1, and this is read out. Output to 80.

  In displaying the three-dimensional GIS image, when the selection unit 85 is notified from the camera image display processing unit 34aa of the display processing unit 34 that the display of the camera image is completed, the coordinate conversion 4 processing unit 104a is activated. The coordinate conversion 4 processing unit 104a is configured to have a photographic area (also referred to as a photographic range) of the angle of view (position, tilt, shooting range; photo set in the camera coordinate system (photo coordinate system)) set by the view angle setting processing unit 33. Are converted into the coordinates of each point of the three-dimensional terrain data in the database 5. That is, an area corresponding to the imaging range at the angle of view of the surveillance camera in the three-dimensional terrain data is determined.

  Then, the selection unit 8 activates the inverse transformation coordinate 4 processing unit 104b, and the inverse coordinate transformation 4 processing unit 104b converts the three-dimensional coordinates of the image data of the region corresponding to the three-dimensional terrain data into the camera coordinate system, The inverse coordinate conversion 3 processing unit is activated to convert to the coordinates of the photographic coordinate system, the inverse coordinate conversion 2 processing unit is activated to convert to the image coordinate system (Px-Py), and the inverse coordinate conversion 1 processing is further performed. It is activated and converted to a screen coordinate system (Mx-My) for displaying a three-dimensional GIS image.

  Then, the image data of the imaging range of the converted three-dimensional terrain data is output to the GIS image display processing unit 34b. The 3D GIS image display processing unit 34b writes this in the display buffer and writes it in the second area of the display control unit, thereby displaying the 3D GIS image of FIG.

  In such a state, the selection unit 85 monitors whether or not the cursor is designated (S205).

This is determined by each of the image processing units (camera, three-dimensional GIS, plan view) of the display processing unit 34 by outputting the cursor designation and the screen type to the selection unit 85 when the cursor is designated.

  Next, the selection unit 85 determines the screen (camera, three-dimensional GIS, plan view) on which the cursor is designated (S206).

  When it is determined that the cursor is designated on the camera image display screen, the selection unit 85 selects and activates the coordinate conversion 1, coordinate conversion 2, coordinate conversion 3, and coordinate conversion 4 in this order to correspond to the imaging range of the surveillance camera. The cursor position of the 3D terrain data (cursor position on the 3D GIS image) is obtained (S207 to S210).

  Furthermore, the selection is started in the order of inverse coordinate transformation 4, inverse coordinate transformation 3, inverse coordinate transformation 2, and inverse coordinate transformation 1 (S211 to S214), and the pixel coordinates (Mx, My of the GIS image display screen obtained as a result are obtained. ) Is written in the display buffer 34bd2 to display the cursor on the three-dimensional GIS image at the same position as the cursor designated on the camera screen (S215).

  Further, as shown in FIG. 26, the selection unit 85 selectively activates in the order of inverse coordinate transformation 7, inverse coordinate transformation 6, and inverse coordinate transformation 5 to define the cursor position on the two-dimensional plane of the three-dimensional terrain data, Pixel coordinates (Mx, My) of the plan view image display screen are obtained (S216 to S218).

  Then, by writing the pixel coordinates (Mx, My) of the plan view image display screen obtained as a result to the display buffer 34cd2 for plan view image display, two cursors designated on the plan view image on the camera screen are displayed. A cursor is displayed at the dimension position (S219).

  Next, it is determined whether or not the process is finished (S221). If not, the process returns to step S205.

  If it is determined that the cursor is designated on the 3D GIS image display screen by the processing in steps S205 and S206, the selection unit 85 selects coordinate conversion 1, coordinate conversion 2, coordinate conversion 3, and coordinate conversion 4 in this order. The cursor position is defined (X, Y, Z) in an area corresponding to the imaging range of the surveillance camera of the three-dimensional terrain data when activated (S231 to S234).

  Further, the coordinates on the camera image display screen on the same camera image as the cursor position on the three-dimensional GIS image screen are selected and activated in the order of inverse coordinate transformation 4, inverse coordinate transformation 3, inverse coordinate transformation 2, and inverse coordinate transformation 1. (S235 to S238), and by writing the pixel coordinates (Mx, My) to the display buffer 34ad1 for displaying the camera image, the cursor is placed on the camera image at the same position as the cursor specified on the three-dimensional GIS image screen. It is displayed (S239).

  Furthermore, as shown in FIG. 26, the selection unit 85 performs selection and activation in the order of inverse coordinate transformation 7, inverse coordinate transformation 6, and inverse coordinate transformation 5, thereby enabling two selections corresponding to the cursor position designated on the three-dimensional GIS image. The position on the three-dimensional plan view image is obtained (S240 to S242), and the pixel coordinates (Mx, My) of the screen for displaying the plan view image are written in the display buffer 34cd1 for displaying the plan view image, so that the top view image is displayed. The cursor is displayed at the two-dimensional position of the cursor designated in the three-dimensional GIS image (S243).

  Furthermore, when it is determined that the cursor has been designated on the plan view image display screen by the processing of steps S205 and S206, the selection unit selectively activates the coordinate conversion 5, coordinate conversion 6, and coordinate conversion 7 in this order to start the plan view image. The cursor position specified above is defined (X, Y, Z) in the three-dimensional terrain data (S251 to S253).

  Then, on the three-dimensional GIS image corresponding to the cursor position of the three-dimensional terrain data in the photographing range of the surveillance camera by selectively starting in the order of inverse coordinate transformation 4, inverse coordinate transformation 3, inverse coordinate transformation 2, and inverse coordinate transformation 1. The cursor position is converted into pixel coordinates (Mx, My) of the 3D GIS image display screen, and this is written into the display buffer 34bd1 of the 3D GIS image display processing unit 34b (S254 to S258).

  Then, as shown in FIG. 26, the selection unit 85 selectively activates in the order of inverse coordinate transformation 4, inverse coordinate transformation 3, inverse coordinate transformation 2, and inverse coordinate transformation 1 to correspond to the cursor position on the plan view image. The cursor is displayed at a position on the image (S259 to S263). Specifically, the coordinates are converted into pixel coordinates (Mx, My) of the camera image display screen, and this is written in the display buffer of the camera image display processing unit to display the cursor on the camera screen.

  That is, the camera image display processing unit 34aa combines the camera image in the camera image display buffer 34ad1 and the cursor image in the cursor display buffer 34ad2 and outputs the synthesized image to the display control unit 80, and the GIS image display processing unit 34b outputs the GIS image. The three-dimensional GIS image at the angle of view (shooting range) of the monitoring camera in the display buffer 34bd1 and the cursor image in the cursor display buffer 34bd2 are combined and output to the display control unit 80.

  Further, the plan view image display processing unit 34c combines the plan view image of the plan view display buffer 34cd1 and the cursor image of the cursor display buffer 34cd2 and outputs them to the display control unit 80.

  Then, the display control unit 80 reads out these images in the corresponding area of the display buffer 80a, and displays these images in the display buffer 80a on the display unit 20 (see FIG. 27B).

  That is, as in the above embodiment, when the cursor is designated on the camera image, the selection unit 85 performs the camera image designation location three-dimensional coordinate transformation unit, the GIS image inverse transformation unit, and the plan view image inverse transformation. The function of the department is executed.

  When the cursor is designated on the GIS image, the functions of the three-dimensional coordinate conversion unit, the camera image reverse conversion unit, and the plan view image reverse conversion unit are executed.

  Further, when a cursor is designated on the plan view image, the functions of the plan view image designation location three-dimensional coordinate conversion unit, GIS image reverse conversion unit, and camera image reverse conversion unit are executed.

  Therefore, even if the camera image from the surveillance camera, the three-dimensional GIS image, and the plan view image are displayed in conjunction on one screen, the camera image is a real image that has not been subjected to any processing. The camera image is reliable.

  Furthermore, when a cursor is specified on any screen, the same cursor is displayed at the corresponding position on the other screen, so that the position can be specified while looking at a reliable real camera image, a plan view, a GIS image, The geographical spatial position can be grasped at a glance on the camera image.

  Here, the description of setting the angle of view will be supplemented with reference to FIG.

Coordinates in the geographic coordinate system of the ground object P _(X, Y, Z) coordinates _P in the camera coordinate system _{(x p, y p, z} p) is obtained by the following equation.

At this time, the inclination of the camera coordinate system with respect to the geographic coordinate system is (ω, ψ, κ), and the geographic coordinates of the camera projection center are 0 (X _0, Y ₀ , Z ₀ ).

Next, the rotation matrix R of the above equation can be expressed as shown in Equation 2).

Here, let c be the focal length of the camera, and p (x, y) be the screen coordinates of the intersection of the straight line connecting the object P and the camera projection center 0 (X _0, Y ₀ , Z ₀ ) with the CCD plane. The collinear conditional expression that the projection center, the screen coordinates, and the ground object are in a straight line is obtained as follows.

Therefore, using this relational expression, the position and orientation ((X _0, Y ₀ , Z ₀ ) & (ω, ψ, κ)) of the camera with respect to the geographic coordinate system, that is, the external orientation element and the focal length c of the camera are known. If so, coordinates (x, y) on the CCD plane corresponding to arbitrary geographic coordinates (X, Y, Z) can be derived.

Further, by transforming the above formula into the following formula, the geographical coordinates (X, Y) can be obtained from the screen coordinates (x, y) and altitude (Z) on the CCD.

When the geographical coordinates (X, Y, Z) are obtained from the screen coordinates (x, y), the elevation (Z ₀ ) of the camera projection center is set as an initial value, and the value of the elevation Z is continuously changed to obtain the above formula. There is a method of determining the geographic coordinates (X, Y), comparing the actual elevations in the three-dimensional terrain model corresponding to the obtained geographic coordinates (X, Y), and setting the matching points as true values. At this time, by selecting a matching point closest to the camera, it is possible to avoid a mistake such as selecting a point that is hidden by the shadow of the terrain and is not originally visible.

  In each of the above embodiments, the plan view image has been described as a map with contour lines. However, there are three types of plan views: a map with contour lines, a disaster prevention map, and a red map with special emphasis on unevenness. In this case, the screen is divided into five, and each image display processing unit is generated as described above. When the cursor is designated on each screen, the cursor is displayed in conjunction with another corresponding position. Like that.

DESCRIPTION OF SYMBOLS 11 Camera image designated part continuous image display part 12 GIS image designated part interlocking display part 13 Plan view designated part interlocking display part 31 Camera selection part 32 Mode determination part 33 Angle setting part 34 Display processing part

Claims (9)

The screen of the display unit is divided, and the surveillance camera image obtained by photographing the subject to be photographed by the surveillance camera having the CCD, the three-dimensional image corresponding to the surveillance camera image, and the plan view image including the subject to be photographed are linked to the corresponding display screen. A linked display measurement system that uses a monitoring camera image to display and display a mark at a specified location on one of the display screens and display the mark in a linked position on another display screen,
A storage means for storing a three-dimensional terrain model of the imaging target, the monitoring camera image, an external orientation element and an internal orientation element of the monitoring camera, and plan view data including the imaging target;
(A) means for defining an angle of view at the photographing point of the surveillance camera in the three-dimensional terrain model based on the external orientation element and the internal orientation element;
(B) The coordinates of the designated location on the display screen of the monitoring camera image are converted into image coordinates on the monitoring camera image, and the image coordinates on the monitoring camera image are converted into photographic coordinates which are actual coordinates on the CCD surface. Converting the photographic coordinates into camera coordinates within the angle of view, and converting the camera coordinates into three-dimensional coordinates of the three-dimensional terrain model;
(C) The coordinates of the designated portion on the display screen of the three-dimensional image are converted into image coordinates on the three-dimensional image, the image coordinates on the three-dimensional image are converted into the photographic coordinates, and the photographic coordinates are converted into the image coordinates. Means for converting into camera coordinates in a corner, and converting the camera coordinates into three-dimensional coordinates of the three-dimensional terrain model;
(D) means for converting the coordinates of a designated portion on the display screen of the plan view image into two-dimensional coordinates of the plan view data, and converting the two-dimensional coordinates into the three-dimensional coordinates of the three-dimensional terrain model;
With
(E) Reading the three-dimensional coordinates of the three-dimensional terrain model, converting the coordinate conversion process of the means (c) into the coordinates of the display screen of the three-dimensional image, and converting the mark to the coordinates. Means for displaying;
(F) The three-dimensional coordinates of the three-dimensional terrain model are read, and the coordinate transformation process of the means (d) is inversely transformed to be transformed into the coordinates of the display screen of the plan view image. Means for displaying,
(G) The three-dimensional coordinates of the three-dimensional terrain model are read, the coordinate conversion process of the means (b) is inversely converted into coordinates on the display screen of the monitoring camera image, and the mark is added to the coordinates. Means for displaying;
With
(H1) When the designated location is the display screen of the monitoring camera image, the (b) means, (e) means are activated, and the (f) means are activated,
(H2) When the designated portion is the display screen of the three-dimensional image, the (c) means, (g) means are activated, and the (f) means are activated,
(H3) When the designated portion is the display screen of the plan view image, the (d) and (e) means are activated, and the (g) means is activated. Linked display measurement system using surveillance camera images. (I) means for polygon-displaying a range designated on the monitor camera image display screen or the plan view image display screen;
The interlocking display measurement system using the surveillance camera image according to claim 1. Said (i) means comprises:
(I1) When the polygon display is made on the monitor camera image display screen, the coordinates of each vertex of the polygon are read and output to the means (b) as the designated location to obtain the coordinates of each vertex. A range in which the coordinates are converted into three-dimensional coordinates on the three-dimensional terrain model, the coordinates of the vertices are converted into the coordinates on the display screen of the plan view image by the inverse conversion of the means (f), and the coordinates are connected. Means for displaying the polygon;
(I2) When the polygon is displayed on the display screen of the plan view image, the coordinates of each vertex of the polygon are read and output to the means (d) as the designated portion to output the coordinates of these vertices. The three-dimensional coordinates on the three-dimensional terrain model are converted into three-dimensional coordinates and converted into coordinates on the monitoring camera image display screen by the inverse conversion of the means (g), and a range connecting these coordinates is displayed in the polygon display. The interlocking display measurement system using the monitoring camera image according to claim 2, further comprising: (M) When polygons are displayed on the monitoring camera image display screen, the coordinates of the lines forming the polygons are read and converted into the three-dimensional coordinates on the three-dimensional terrain model by the means (b). Means for converting each of these three-dimensional coordinates into the coordinates of the display screen of the three-dimensional image by the means (e), and displaying them on the display screen of the three-dimensional image as points.
(N) When the polygon is displayed on the display screen of the plan view image, each coordinate of a line forming the polygon is converted into a three-dimensional coordinate on the three-dimensional terrain model by the means (d), 3. A means for converting each of the three-dimensional coordinates into coordinates of a display screen of the three-dimensional image by the means (e) and displaying them on the three-dimensional image as points. A linked display measurement system using the described surveillance camera image. On the display screen of the monitoring camera image, every time the designated portion is designated continuously, the (b) means converts the three-dimensional coordinates on the three-dimensional terrain model, and obtains the distance for each conversion. The interlocking display measurement system using the surveillance camera image according to claim 1, further comprising: means for displaying the image on a screen. 4. The surveillance camera image according to claim 3, further comprising means for reading the three-dimensional coordinates of the vertices on the three-dimensional terrain model, obtaining an area of a range in which the vertices are connected, and displaying the area. The linked display measurement system. The linked display measurement system using a monitoring camera image according to claim 1, further comprising means for reading the three-dimensional coordinates on the three-dimensional terrain model and displaying the three-dimensional coordinates . The means (b) is:
The pixel coordinates (Mx, My) of the designated location on the monitoring camera image display screen are determined from the ratio of the vertical and horizontal pixel counts of the monitoring camera image display screen to the vertical and horizontal pixel counts of the monitoring camera image. First coordinate conversion means for converting to (Px, Py);
By associating the actual size of the photographic surface of the surveillance camera with the number of vertical and horizontal pixels of the surveillance camera image, the pixel coordinates (Px, Py) of the surveillance camera image become the photographic coordinates (X ′, Y ′) of the surveillance camera. Second coordinate conversion means for converting;
The plane corresponding to the photographic coordinates is the X and Y axes, the axis orthogonal to the origin is the Z axis, the point separated from the CCD surface by the focal length in the Z direction is the optical center, and this optical center is the origin. Assuming a three-dimensional space, camera coordinates (X ′, Y ′, Z ′) obtained by replacing the photographic coordinates (X ′, Y ′), which are the two-dimensional coordinates of the three-dimensional space, with the three-dimensional relative position from the optical center. Third coordinate conversion means for converting to
Using the three-dimensional coordinates of the optical center of the monitoring camera and the tilt of the camera coordinate system with respect to the three-dimensional coordinates, the camera coordinate system is converted into the three-dimensional coordinate system, and the three-dimensional coordinate system is converted into the three-dimensional coordinate system. Fourth coordinate conversion means for obtaining a three-dimensional coordinate (X, Y, Z) of a point where a straight line connecting the optical center of the surveillance camera and the object on the CCD plane intersects the solid surface of the three-dimensional terrain model ;
The interlocking display measurement system using the surveillance camera image according to claim 1. The monitoring camera image displayed on the display screen of the monitoring camera image is
Means for reading the surveillance camera image of the storage means and reproducing the surveillance camera image ;
Means for outputting the reproduced surveillance camera image to a display screen of the surveillance camera image;
The interlocking display measurement system using the surveillance camera image according to claim 1 , wherein the display is performed by the following .