JP4811249B2 - Remote indication system - Google Patents

Description

  The present invention relates to a remote instruction system, a photographing apparatus, an instruction apparatus, and a program thereof.

Conventionally, a remote instruction is given in which a work instructor gives an instruction by voice such as a telephone to a worker in a remote place while watching a photographed image of the remote place.
As an invention that supports such a remote instruction, a technique is disclosed in which a drawing content written on a captured image by a work instructor is projected onto an actual subject by a projector (see Patent Documents 1 and 2). .
In addition, a technique for controlling a camera based on a designation made on a wide-angle image is disclosed (see Patent Document 3).

US Patent Publication No. 2004/0070674 JP 2003-209832 A Japanese Patent No. 3265893 JP 2001-94857 A JP 2004-64784 A JP 2000-92361 A Japanese Patent No. 0301352 JP 07-226940 A

  For example, when the subject is a precision part or the like, there is a request for the work instructor to enlarge the subject. Thus, it is conceivable to realize enlarged display with a configuration in which the camera and the projector are operated in conjunction with each other, but in this configuration, only the central portion of the photographing range can be enlarged and displayed. In addition, it is conceivable to realize enlarged display by increasing the resolution of the captured image itself, but considering the current communication performance and image processing performance, the smoothness of motion is lost from the captured image due to the increased resolution. It is not preferable.

  As a countermeasure, there is disclosed a technique (see Patent Document 4) in which a designated place is magnified by a PTZ (Pan Tilt Zoom) camera (see Patent Document 4), and an entire captured image (wide-angle image) captured by an ordinary camera is disclosed. A magnified display is realized by combining a photographed image (PTZ image) obtained by enlarging a part with a PTZ camera. However, it does not support drawing and writing on PTZ images.

  Therefore, FlySpec (refer to Patent Document 5), which is a technique for selecting a region from a captured video to control the PTZ camera, and iLight, which is a technique for drawing and writing on the captured video and projecting the drawn image onto the subject. In combination, it is conceivable to realize drawing writing on the PTZ video. However, since there is a parallax between the cameras, there is a difference between the wide-angle video and the PTZ video, so that the drawing content on the PTZ video cannot be accurately projected onto the three-dimensional object. Moreover, since it is necessary to prepare a PTZ camera that requires precise driving due to the configuration of the apparatus, the apparatus is increased in size and cost.

  The present invention has been made in view of the above-described conventional circumstances, and enables to accurately project a drawing content written on an enlarged displayed image onto a subject while making it possible to enlarge and display a subject at a remote place. It is an object.

  The first aspect of the present invention includes a photographing side device and an instruction side device, and the photographing side device photographs a subject and takes a first image by means of photographing side communication means that communicates with the instruction side device. First imaging means for generating image, second imaging means for imaging the subject and generating a second image having a higher resolution than the first image, and drawing information received from the instruction side device by the imaging side communication means Projecting means for projecting an image based on the subject onto the subject, optical axis making means for matching the optical axis of the first photographing means, the optical axis of the second photographing means, and the optical axis of the projecting means, and the instruction Extraction means for extracting an image of an area in the second image corresponding to the area information received from the side apparatus by the imaging side communication means; and the imaging side communication of the first image and the area image to the instruction side apparatus Image transmission control means for transmitting by means; The instruction side device includes: an instruction side communication unit that communicates with the photographing side device; a normal display unit that displays the first image received from the photographing side device by the instruction side communication unit; and the display For receiving the designation of the area in the first image from the user and transmitting the area information indicating the designated area to the photographing side device by the instruction side communication means and the transmission of the area information. Correspondingly, enlarged display means for displaying the area image received from the photographing side apparatus by the instruction side communication means on the screen, and drawing for the displayed first image or the area image is received from the user, and the drawing contents And a projection instruction means for transmitting drawing information indicating the image information to the photographing side apparatus by the instruction side communication means.

  According to a second aspect of the present invention, in the remote instruction system according to the first aspect, the optical axis forming means uses a prism to connect the optical axis of the first imaging means and the optical axis of the second imaging means. The remote indication system is characterized in that the optical axis and the optical axis of the projection means are matched by a half mirror.

  According to a third aspect of the present invention, in the remote instruction system according to the first aspect, the same optical axis unit is configured such that the optical axis of the first imaging unit and the optical axis of the second imaging unit are the first. The remote pointing system is characterized in that the optical axis and the optical axis of the projection means are made to coincide with each other by a second half mirror.

  According to a fourth aspect of the present invention, in the remote instruction system according to any one of the first to third aspects, an angle of view of the first photographing unit is equal to an angle of view of the second photographing unit. This is a remote indication system characterized by that.

  According to a fifth aspect of the present invention, in the remote instruction system according to the fourth aspect, the angle of view of each of the photographing means is equal to the angle of view of the projection means.

  According to a sixth aspect of the present invention, the photographing side communication means for communicating with the instruction side device, the first photographing means for photographing the subject to generate the first image, and photographing the subject to obtain the first image from the first image. A second photographing unit that generates a second image with a high resolution; and the designated region received by the photographing-side communication unit from the instruction-side device that has received designation of the region in the first image from a user. Extraction means for extracting an image of an area in the second image corresponding to the area information, and the first image and the area image are transmitted to the instruction side apparatus by the photographing side communication means and displayed on the instruction side apparatus. An image transmission control means to be executed, and an image based on the drawing information indicating the drawing content received by the photographing side communication means from the instruction side device that has received drawing on the first image or the region image from a user. Projection means for projecting to the optical axis, and optical axis making means for matching the optical axis of the first imaging means, the optical axis of the second imaging means, and the optical axis of the projection means. It is a photographing side device.

  According to a seventh aspect of the present invention, an optical axis of a first photographing unit for photographing a subject to generate a first image, and a second image having a higher resolution than the first image by photographing the subject. An instruction-side communication unit that communicates with an imaging-side device that matches an optical axis of a second imaging unit and an optical axis of a projection unit that projects an image based on drawing information onto the subject; Normal display means for displaying the first image received by the side communication means on the screen, and designation of an area in the displayed first image from the user, and area information indicating the designated area is received on the photographing side In the second image corresponding to the area information received by the instruction side communication means from the photographing side apparatus in response to the transmission of the area information, the enlarged image request means for transmitting to the apparatus by the instruction side communication means The area image is displayed on the screen. An enlargement display means for receiving, a projection instruction means for receiving drawing on the displayed first image or the region image from a user, and transmitting drawing information indicating the drawing content to the photographing side device by the instruction side communication means, It is the instruction | indication side apparatus provided with.

  The present invention according to claim 8, the optical axis of the first photographing means for photographing the subject to generate the first image, and the second image having a higher resolution than the first image by photographing the subject. A photographing side communication means for communicating a computer of the photographing side device, which matches the optical axis of the second photographing means and the optical axis of the projecting means for projecting an image based on the drawing information on the subject, with the pointing side device; Extracting the image of the region in the second image corresponding to the region information indicating the specified region received by the photographing side communication means from the instruction side device that has received the designation of the region in the first image from the user Extraction means, image transmission control means for transmitting the first image and the region image to the instruction side device by the photographing side communication means and displaying the image on the instruction side device, drawing on the first image or the region image Received from users Projection control means for projecting an image from attaching said instruction-side apparatus based on the drawing information indicating the drawing contents received by the imaging-side communication means to the object by the projection means is a program for functioning as a.

  According to the ninth aspect of the present invention, the computer causes the optical axis of the first photographing unit to photograph the subject to generate the first image, and the second image having a higher resolution than the first image by photographing the subject. An instruction-side communication unit that communicates with an imaging-side device in which the optical axis of the second imaging unit that generates the image and the optical axis of the projection unit that projects an image based on drawing information onto the subject are matched, from the imaging-side device Normal display means for displaying the first image received by the instruction side communication means on the screen, designation of an area in the displayed first image is received from a user, and area information indicating the designated area is captured. In the second image corresponding to the area information received by the instruction side communication means from the photographing side apparatus in response to the transmission of the area information, the enlarged image request means for transmitting to the side apparatus by the instruction side communication means Region image An enlarged display means for displaying on the screen, a projection instruction means for accepting drawing for the displayed first image or region image from a user and transmitting drawing information indicating the drawing content to the photographing side device by the instruction side communication means It is a program to make it function as.

  According to the present invention, since each photographing unit and projection unit are arranged on the same optical axis, there is no need to consider parallax in the magnified display of the captured image and the projection of the drawing content on the display image. The drawing content can be accurately projected onto the subject.

In addition, since the enlarged display process can be performed using a common coordinate system by matching the angle of view of each photographing unit, the process can be simplified.
Also, by matching the angle of view of the projection means, the projection process can be performed using a common coordinate system, so the process can be simplified.

The present invention will be specifically described based on an example.
As shown in FIG. 1, the remote instruction system according to the present embodiment includes a photographing device 20 that photographs a subject 10, a projection device 30 that projects an image onto the subject 10, a photographing device 20, and a projection device 30. A control device 40 that performs control and the like, and a remote instruction device 50 that is connected to the control device 40 via a network N are provided.

  The photographing apparatus 20 includes a branching unit 21 for branching light from the subject 10, a low-resolution photographing unit 22 for generating an image (hereinafter referred to as an “entire display image”) from one branched light, a branched other unit, and the like. The high-resolution imaging means 23 for generating an image with a higher resolution than the entire display image (hereinafter referred to as “enlarged display image”) from the light of the above-mentioned light, and the projection device 30 is an image supplied from the control device 40 Is projected on the subject 10.

  In this example, a prism is used as the branching unit 21, and the light from the subject 10 is branched and input to the photographing units 22 and 23, thereby matching the optical axes of the photographing units 22 and 23. Moreover, the optical axis of each imaging means 22 and 23 and the optical axis of the projection means 31 are made to correspond using the half mirror 35 which permeate | transmits the light from one surface and reflects the light from the other surface. . That is, in this example, the prism 21 and the half mirror 35 constitute the same optical axis unit according to the present invention. Note that the optical axis forming means can be configured in various ways, for example, using a half mirror that transmits 50% of the light and reflects the rest as the branching means 21.

  The control device 40 includes a communication unit 41 that communicates with the remote instruction device 50 via the network N, a region information memory 42 that stores region information received from the remote instruction device 50, and an enlarged display generated by the photographing device 20. Extraction means 43 for extracting an image of an area corresponding to the area information (hereinafter referred to as “partially enlarged image”) from the image, an image based on the drawing information received from the remote instruction device 50 is supplied to the projection device 30 and the subject Projection control means 44 for projecting to 10, and image transmission control means 45 for transmitting the entire display image and the partially enlarged image to the remote instruction device 50.

  The remote instruction device 50 includes a communication means 51 that communicates with the control device 40 via the network N, a screen 52 that is configured by a liquid crystal display, an input device 53 that is configured by a keyboard / mouse, and the remote instruction device 50. The normal display means 54 for displaying the received whole display image on the screen 52, the enlarged display means 55 for displaying the partial enlarged image received from the remote instruction device 50 on the screen 52, and the whole display image displayed on the screen 52. The designation of the area is received from the user, and the enlarged image requesting means 56 for transmitting the area information indicating the designated area to the photographing side device 40, the drawing for the whole display image or the partial enlarged image displayed on the screen 52 is used. A projection instructing means 57 for receiving drawing information indicating the drawing contents and transmitting the drawing information to the photographing side apparatus 40.

  Here, in the photographing apparatus 20 of this example, as shown in FIG. 3A, a prism 21 that branches light incident from the subject 10 via the imaging lens 25 and one branched light are imaged. The image sensor 22a for low resolution arranged in this way, and the image sensor 23a for high resolution arranged so that the other light is imaged. The arrangement positions of the image sensors 22a and 23a are optically equivalent, and CCDs (Charge Coupled Devices) having the same photosensitive surface size are used as the image sensors 22a and 23a. Instead of the CCD, for example, another imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) may be used.

The low-resolution imaging unit 22 generates an entire display image based on the output signal LoCh output when the low-resolution imaging element 22a receives light. The high-resolution imaging unit 23 generates an enlarged display image based on the output signal HiCh output when the high-resolution imaging device 23a receives light.
In the remote instruction system of this example, moving image distribution (or still image continuous distribution) is performed at a resolution of NTSC level (640 × 480 pixels: VGA). Therefore, the low-resolution image sensor 22a has a resolution of VGA. Further, the high-resolution image pickup device 23a has a resolution of 3200 × 2400 pixels in order to realize an enlarged display of up to 5 times.

Here, an enlarged display by the remote instruction system of this example will be described with reference to FIG.
For example, when displaying a part of the VGA-size whole display image magnified 5 times, an enlarged display image of 3200 × 2400 pixels is 640 × 480 pixels as shown in FIG. Separation and a corresponding section are extracted as a partially enlarged image and used for enlarged display.
Further, for example, when a part of the VGA-size whole display image is enlarged by 2.5 times, as shown in FIG. 4B, the enlarged display image of 3200 × 2400 pixels is 1280 × 960 pixels. The corresponding section is extracted as a partially enlarged image and used for enlarged display.

  In the remote instruction system of this example, it is assumed that a partially enlarged image is displayed in VGA size. When the entire area of 3200 × 2400 pixels is converted to VGA and displayed, the zoom is 1 × (no zoom), 1280 When a partial area of × 960 pixels is converted to VGA and displayed, the zoom is 2.5 times. Note that such a digital zoom technique based on resolution conversion is disclosed in, for example, Patent Document 6.

  Here, when considering at the pixel level without using the digital zoom, the resolution of the image sensor determines the magnification. In other words, since the enlargement magnification is determined by the resolution, the pixel enhancement technology becomes important. For example, if a 10 × zoom is required for the VGA output, an image sensor with 6400 × 4800 pixels is required, and a magnification that can be enlarged is determined by a technique for increasing the definition of the image sensor. Therefore, in order to further improve the enlargement magnification, the following method of pixel shift may be used.

  For example, FIG. 3B shows a configuration example in the case of applying 3CCD pixel shift. According to the figure, a prism 21 that splits light on the high-resolution imaging means 23 side into an R (red) component, a G (green) component, and a B (blue) component, and an R imaging device arranged corresponding to each color component. 23b, a G image sensor 23c, and a B image sensor 23d. The G image sensor 23c is arranged with a half pixel shift with respect to the R image sensor 23b and the B image sensor 23d, and outputs the output signals RCh, GCh, BCh output from the image sensors 23b, 23c, 23d. In combination, an enlarged display image is generated. According to this 3CCD system pixel shift, a resolution of about 4 times is realized (see Patent Document 7).

 Also, for example, FIG. 3C shows a configuration example in the case of applying a 4CCD pixel shift. According to the figure, a prism 21 that splits light on the high-resolution imaging means 23 side into R component, G1 component, G2 component, and B component, and an R imaging device 23b and a G1 imaging device arranged corresponding to each color component. 23c, the G2 image sensor 23e, and the B image sensor 23d. The G1 image sensor 23c and the G2 image sensor 23e are arranged with a half-pixel shift with respect to the R image sensor 23b and the B image sensor 23d, and are output from the image sensors 23b, 23c, 23e, and 23d. The enlarged display image is generated by combining the output signals RCCh, G1Ch, G2Ch, and BCh. Luminance signal (Y) that determines resolution is 0.7152G + 0.0722B + 0.2126R, and since about 70% of G signal is included, the performance improvement of GCh greatly contributes to the improvement of the luminance signal. (See Patent Document 8).

  In FIG. 3A, for example, a 2/3 type 6 million pixel CCD is used as the high resolution image sensor 23a, and a low resolution CCD is a VGA class. However, in response to the trend of higher density, higher definition, and smaller size, there is little demand for large-format and low-resolution image sensors, leading to increased costs. Therefore, an image generated using a high-resolution image sensor may be reduced to obtain an entire display image. For example, when VGA output is performed with an image sensor of SXGA (1280 × 960 pixels), vertical 2 pixels × horizontal 2 pixels = 4 pixels are set as one pixel unit, or one pixel is thinned out every two pixels. , VGA resolution images can be acquired.

The flow of processing by the remote instruction system of this example will be described with reference to FIG.
The entire display image generated by the low-resolution imaging unit 22 is distributed as a moving image to the remote instruction device 50 by the image transmission control unit 45 of the control device 40 and is taken as a captured image on the screen 52 by the normal display unit 54 of the remote instruction device 50. Is displayed.

  The remote instruction device 50 monitors whether the user has drawn graphics, characters, etc. on the captured video (entire display image) (step S11). The instructing means 57 transmits drawing information indicating the drawn content to the control device 40, and the control device 40 supplies a drawing image based on the drawing information to the projection device 30 to project it onto the subject 10 (step S12).

  Further, the remote instruction device 50 monitors whether or not the user has selected the zoom designation area to be enlarged and displayed in the captured video (entire display image) (step S13), and the zoom designation area has been selected. In this case, the enlarged image request unit 56 transmits coordinate values (region information) indicating the zoom designated region to the control device 40, and the control device 40 stores the coordinate values in the region information memory 42. The extracting means 43 converts the coordinate value stored in the area information memory 42 into the coordinate value in the high resolution video (enlarged display image), and zooms in the corresponding area in the high resolution video (enlarged display image). Extracted as (partial enlarged image), the zoomed image is distributed as a moving image to the remote instruction device 50 by the image transmission control means 45 and displayed on the screen 52 by the enlarged display means 55 (step S14).

  Further, the remote instruction device 50 monitors whether a user has drawn a figure or a character on the zoomed video (partially enlarged image) (step S15). The projection instruction unit 57 transmits drawing information indicating the drawn content to the control device 40, and the control device 40 captures a drawing image based on the drawing information with reference to the coordinate value (region information) of the zoom designation region. The image is converted into a drawing image in the video (normal display image) (step S16), supplied to the projection device 30, and projected onto the subject 10 (step S17).

FIG. 5 shows a screen display example and a projection example in the above processing.
FIG. 5A illustrates a normal display window 60 that displays a normal display image on the screen 52 as a captured video. According to the figure, the normal display window 60 displays the entire display image of the subject 10 on which the drawing image 61 is projected, and the zoom designation area 65 (dashed line portion) is designated by the user. ing.

  Thus, when the zoom designation area 65 is designated, area information (x0, y0, x1, y1) indicating the upper left coordinates (x0, y0) and the lower right coordinates (x1, y1) of the zoom designation area 65 is designated. Is transmitted to the control device 40 and stored in the area information memory 42 of the control device 40. Thereby, a region in the enlarged display image corresponding to the region information is extracted as a partially enlarged image and distributed as a moving image. As a result, as shown in FIG. 5B, the broken line portion (zoom designation area 65) of FIG. 5A is displayed on the enlarged display window 70 that displays the partial display image as a zoomed image on the screen 52. Is done.

  Then, as illustrated in FIG. 5C, when the drawing image 71 is written by the user with respect to the partial display image displayed in the enlarged display window 70, the drawing information of the drawing image 71 is transferred to the control device 40. Sent. The control device 40 uses a relational expression (xa, xb) = g (xb, yb) between the coordinate system (xa, xb) of the entire display image and the coordinate system (xb, yb) of the partial display seat image. Then, the drawing image 71 drawn on the partial enlarged image extracted from the partial display image is converted into the coordinate system of the whole display image.

  Here, as shown in FIG. 6, if the partially enlarged image has an image size (w, h), the function g is expressed by the upper left coordinates (x0, y0) and the lower right coordinates (x1) of the zoom designation area 65. , Y1) and the image size (w, h) of the partially enlarged image, it is obtained by the following mathematical formula (affine transformation).

  As a result, the drawing image 71 formed as the partially enlarged image is converted into the coordinate system of the entire display image. Thereby, as shown in FIGS. 5D and 5E, a drawing image 61 for the whole display image and a drawing image 62 obtained by converting the drawing image 71 for the partially enlarged image into the coordinate system of the whole display image are as follows. Projected images 61 a and projected images 71 a are projected onto the subject 10.

  As described above, according to the present invention, since the low-resolution imaging means 22, the high-resolution imaging means 23, and the projection means 31 are optically equivalently provided, drawing on the partially enlarged image can be accurately projected onto the subject. Image processing can be simplified as compared with. Further, since the configuration does not require a PTZ camera, it can be configured mechanically and the apparatus can be reduced in size and cost.

In this example, the control device 40 performs the affine transformation process. However, the remote instruction device 50 may perform the transformation process and transmit the converted information to the control device 40.
Further, in this example, the angle of view (shooting range) of the low-resolution imaging means 22 and the high-resolution imaging means 23 are made to coincide with each other, and the angle of view (projection range) of the projection means 31 is also made to match. In this case, for example, a coordinate system may be set based on the narrowest range, and the coordinates may be adjusted by the control device 40 or the remote indication device 50.

  As a method for specifying the zoom designation area 65, for example, a range selected by a mouse drag operation is accepted as the zoom designation area 65, or when one point in the entire display image is designated, the point is used as a reference. The predetermined range is accepted as a zoom designation area 65, a section selected from a plurality of images for whole display divided in advance is accepted as the zoom designation area 65, the zoom designation area 65 is accepted by coordinate input with a keyboard, etc. The area can be designated by various methods.

  The control device 40 of this example has hardware resources such as a medium access unit that performs data access to an external storage medium, an HDD (Hard Disk Drive), a RAM (Random Access Memory), and a CPU (Central Processing Unit). The computer reads the program from an external storage medium or HDD that stores and holds the program according to the present invention, develops the program on the RAM, and performs arithmetic processing by the CPU, whereby the communication unit 41, the extraction unit 43, the projection control unit 44, Although the image transmission control means 45 is configured, these may be configured by dedicated hardware.

  In addition, the remote instruction device 50 of this example is an external storage in which a computer having hardware resources such as a medium access unit, HDD, RAM, and CPU that performs data access to an external storage medium stores and holds the program according to the present invention. The program is read from the medium or HDD, loaded on the RAM, and processed by the CPU to constitute the communication means 51, normal display means 54, enlarged display means 55, enlarged image request means 56, and projection instruction means 57. However, these may be configured by dedicated hardware.

It is a figure which shows the structure of the remote instruction | indication system which concerns on one Example of this invention. It is a figure explaining the enlarged display which concerns on one Example of this invention. It is a figure which shows the prism structure which concerns on one Example of this invention. It is a figure explaining the flow of processing concerning one example of the present invention. It is a figure which shows the example of a screen display and projection example which concern on one Example of this invention. It is a figure explaining the projection process which concerns on one Example of this invention.

Explanation of symbols

21: Branch means,
22: Low resolution photographing means,
22a: an image sensor for low resolution,
23: High-resolution imaging means,
23b, 23c, 23d, 23e: high-resolution imaging device,
25: Imaging lens,
31: Projection means,
35: Half mirror,
42: area information memory,
43: Extraction means
44: Projection control means,
45: Image transmission control means,
54: Normal display means,
55: Enlarged display means,
56: Enlargement display request means
57: Projection means

Claims (9)

A photographing side device and an instruction side device;
The photographing side device is:
Photographing side communication means for communicating with the instruction side device;
First photographing means for photographing a subject and generating a first image;
Second photographing means for photographing the subject and generating a second image having a higher resolution than the first image;
Projecting means for projecting an image based on the drawing information received from the instruction side device by the photographing side communication means onto the subject;
The same optical axis unit for matching the optical axis of the first imaging unit, the optical axis of the second imaging unit and the optical axis of the projection unit;
Extraction means for extracting an image of a region in the second image corresponding to region information received from the instruction side device by the photographing side communication unit;
Image transmission control means for transmitting the first image and the region image to the instruction side device by the photographing side communication means,
The pointing device is
Instruction side communication means for communicating with the photographing side device;
Normal display means for displaying the first image received from the photographing side device by the instruction side communication means on a screen;
An enlarged image requesting unit that receives designation of a region in the displayed first image from a user, and transmits region information indicating the designated region to the photographing side device by the instruction side communication unit;
In response to the transmission of the area information, an enlarged display means for displaying the area image received from the imaging side device by the instruction side communication means on a screen;
Projection instruction means for receiving drawing on the displayed first image or the region image from a user and transmitting drawing information indicating the drawing content to the photographing side device by the instruction side communication means. Remote instruction system.   The same optical axis unit makes the optical axis of the first imaging unit and the optical axis of the second imaging unit coincide with each other by a prism, and makes the optical axis and the optical axis of the projection unit coincide with each other by a half mirror. The remote instruction system according to claim 1.   The same optical axis unit causes the optical axis of the first imaging unit and the optical axis of the second imaging unit to coincide with each other by a first half mirror, and the optical axis of the projection unit is set to the second optical axis. The remote indication system according to claim 1, wherein matching is performed by using a half mirror.   4. The remote instruction system according to claim 1, wherein an angle of view of the first imaging unit is equal to an angle of view of the second imaging unit. 5.   5. The remote instruction system according to claim 4, wherein an angle of view of each of the photographing units is equal to an angle of view of the projection unit. Photographing side communication means for communicating with the instruction side device;
First photographing means for photographing a subject and generating a first image;
Second photographing means for photographing the subject and generating a second image having a higher resolution than the first image;
The image of the region in the second image corresponding to the region information indicating the specified region received by the photographing side communication means from the instruction side device that has received the specification of the region in the first image from the user. Extracting means for extracting;
Image transmission control means for transmitting the first image and the region image to the instruction side apparatus by the photographing side communication means and causing the instruction side apparatus to display the image transmission control means;
Projection means for projecting an image based on drawing information indicating the drawing content received by the photographing side communication means from the instruction-side device that has received drawing for the first image or the region image from the user onto the subject;
The same optical axis unit for matching the optical axis of the first imaging unit, the optical axis of the second imaging unit and the optical axis of the projection unit;
An apparatus on the photographing side characterized by comprising: An optical axis of a first photographing unit that photographs a subject and generates a first image, an optical axis of a second photographing unit that captures the subject and generates a second image having a higher resolution than the first image, and drawing An instruction-side communication unit that communicates with an imaging-side device that matches an optical axis of a projection unit that projects an image based on information onto the subject;
Normal display means for displaying the first image received from the photographing side device by the instruction side communication means on a screen;
An enlarged image requesting unit that receives designation of a region in the displayed first image from a user, and transmits region information indicating the designated region to the photographing side device by the instruction side communication unit;
An enlarged display means for displaying an image of an area in the second image corresponding to the area information received from the imaging-side apparatus by the instruction side communication means in response to the transmission of the area information;
A projection instruction means for accepting drawing for the displayed first image or the region image from a user and transmitting drawing information indicating the drawing content to the photographing side device by the instruction side communication means;
An instruction-side device comprising: An optical axis of a first photographing unit that photographs a subject and generates a first image, an optical axis of a second photographing unit that captures the subject and generates a second image having a higher resolution than the first image, and drawing A computer of a photographing side device in which an optical axis of a projecting unit that projects an image based on information onto the subject is matched,
Photographing side communication means for communicating with the instruction side device;
The image of the region in the second image corresponding to the region information indicating the specified region received by the photographing side communication means from the instruction side device that has received the specification of the region in the first image from the user. Extraction means for extracting,
Image transmission control means for transmitting the first image and the region image to the instruction-side device by the photographing-side communication means and causing the instruction-side device to display the first image and the region image;
Projection by which the projection unit projects an image based on the drawing information indicating the drawing content received from the instruction-side device that has received the drawing on the first image or the region image from the user by the photographing-side communication unit. Control means,
Program to function as. Computer
An optical axis of a first photographing unit that photographs a subject and generates a first image, an optical axis of a second photographing unit that captures the subject and generates a second image having a higher resolution than the first image, and drawing An instruction side communication means for communicating with the photographing side apparatus in which the optical axis of the projection means for projecting an image based on the information onto the subject is matched;
Normal display means for displaying the first image received from the photographing side device by the instruction side communication means on a screen;
An enlarged image requesting unit that receives designation of a region in the displayed first image from a user, and transmits region information indicating the designated region to the photographing side device by the instruction side communication unit;
An enlarged display means for displaying an image of an area in the second image corresponding to the area information received by the instruction side communication means from the imaging side device in response to the transmission of the area information;
A projection instruction means for accepting drawing for the displayed first image or the region image from a user and transmitting drawing information indicating the drawing content to the photographing side device by the instruction side communication means;
Program to function as.

Images