JP2011210232A - Image conversion device, image generation system, image conversion method, and image generation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image conversion device capable of converting a spherical surface image into an image that is easy to be stored, and an image generation system, an image conversion method, and an image generation method.SOLUTION: The image conversion device 30 has a spherical surface image generation part 32 for generating the spherical surface image 40 in which a photographed image is projected on a discrete spherical surface, and an image conversion part 33 for converting the spherical surface image into a rectangular image 70. A pixel that the discrete spherical surface has corresponds to a vertex 51 of a geodetic dome 50. A pixel value of each quadrangular pixel that the rectangular image has is a pixel value of the spherical surface image pixel corresponding to the vertex of the position of each quadrangular pixel in a two-dimensional array structure in which a plurality of vertices of an intermediate array structure are arranged in a quadrangle shape by deleting vertices corresponding to first and second poles of the geodetic dome from the intermediate array structure 60 in which, with each vertex on one meridian of the geodetic dome as a base point, a plurality of base points are arranged in a first direction, and the vertices of the same latitude as the respective base points are arranged in the second direction orthogonal to the first direction for each base point, and moving predetermined vertices of the intermediate array structure.

Description

  The present invention relates to an image conversion apparatus, an image generation system, an image conversion method, and an image generation method.

  An image of the whole sky can be obtained by acquiring a surrounding image around a certain viewpoint. In this case, the whole sky is divided and acquired by a plurality of images, but a spherical image is known as one that displays these as one image (see Patent Document 1). A spherical image is an image projected on a spherical surface. When a spherical image is represented by a computer or the like, the spherical image is discretized using a geodetic dome formed by dividing a continuous spherical surface by a geodetic dome division method (see Non-Patent Document 1).

JP 2005-56295 A

Kenichi Morooka and 2 others, “Viewpoint Space Representation and Viewpoint Planning Calculation Method for Automatic Modeling of 3D Objects”, IEICE Transactions D-II, January 2001, Vol. J84-D-II No. 1 pp. 64-74

  As a method for expressing the data structure of the geodetic dome 120 corresponding to the discretized spherical surface, as shown in FIG. 23, a method in which the geodetic dome 120 is developed in a plane is known. FIG. 23 is a drawing showing an example of a developed view of a conventional geodetic dome.

  However, when such a data structure is used, for example, the index u for designating the parallelogram shown in the area A surrounded by the broken line in FIG. 23 and each pixel in the parallelogram are designated two-dimensionally. Two indexes v and w are required, and a three-dimensional expression method is adopted. For this reason, data storage and the like may be complicated.

  The present invention has been made to solve the above-described problems, and provides an image conversion apparatus, an image generation system, an image conversion method, and an image generation method capable of converting a spherical image into an image that can be more easily stored. With the goal.

  In order to solve the above problems, an image conversion apparatus according to the present invention includes a spherical image generation unit that generates a spherical image in which a plurality of captured images are projected on a discrete spherical surface discretely represented by a plurality of pixels, and a spherical surface. An image conversion unit for converting an image into a rectangular image, wherein a plurality of pixels representing a discrete spherical surface correspond to each vertex of a geodetic dome obtained by dividing the spherical surface by a geodetic dome method, and a rectangular image includes a plurality of rectangular pixels. The pixel value of each quadrangle pixel is based on a plurality of vertices on one meridian of the geodetic dome, and a plurality of base points are arranged in the first direction, and vertices having the same latitude as each base point are relative to each base point. Vertices corresponding to the first and second poles of the geodetic dome are deleted from the intermediate array structure arranged in the second direction orthogonal to the first direction, and predetermined vertexes of the intermediate array structure are first and Intermediate by moving in the direction of 2 In two-dimensional array structure in which a plurality of vertices are arranged in a square shape column structure has a pixel value of a pixel of the spherical image corresponding to the vertex position of each square pixel, it is characterized.

  The image conversion method according to the present invention generates a spherical image by projecting a plurality of captured images onto a discrete spherical surface represented by pixels corresponding to each vertex of a geodetic dome formed by dividing a spherical surface by a geodetic dome method. And a plurality of vertices on one meridian of the geodetic dome as base points, and a plurality of base points are arranged in a first direction, and vertices having the same latitude as each base point are orthogonal to the first direction with respect to each base point A step of generating an intermediate array structure arranged in a second direction; and a predetermined vertex among the vertices other than the vertices corresponding to the first and second poles of the geodetic dome included in the intermediate array structure A step of generating a two-dimensional array structure in which vertices other than the vertices corresponding to the first and second poles of the geodetic dome are arranged in a square shape by moving in the second direction, and the two-dimensional array structure is configured Each vertex to As square pixels, characterized in that it comprises the steps of: generating a rectangular image by assigning pixel values of the pixels of the corresponding spherical image to the vertices of the position of each rectangle pixel.

  In such an image conversion apparatus and image conversion method, a spherical image having a plurality of captured images as one image is converted into a rectangular image. In this case, it is only necessary to store the rectangular image, so that it is easy to store the image data. Further, by storing the image as a rectangular image, a compression method such as JPEG can be easily applied.

In the above-described image conversion apparatus, the intermediate array structure has the number of columns in the i-th row (i is an integer of 0 to 3 · 2 ^L ) when the first and second directions are the row direction and the column direction, respectively. N _i has an array structure represented by the formula (1), and the two-dimensional array structure includes the (2 ^{L + 1} +1) th row to the (R _L −1) th row of the intermediate array structure. Using the area as a shift area, each vertex in the shift area is column-shifted toward the (5 × 2 ^L ) th column, and then the (2 ^{L + 1} +1) th row matches the 0th row. together to row shift to, and is configured by deleting the vertices corresponding to the 0th row and the R _L rows in the intermediate array structure, it is preferable. Alternatively, in the image conversion method according to the present invention, when the first and second directions are the row direction and the column direction, respectively, the intermediate array structure has the i-th row (i is 0 or more and 3 · 2 ^L or less). In the step of generating a two-dimensional array structure, the number of columns N _i of (integer) has a structure represented by the formula (1). In the step of generating the two-dimensional array structure, the (2 ^{L + 1} +1) th row of the intermediate array structure ( R _L −1) Using the area up to the row as a shift area, each vertex in the shift area is column-shifted toward the (5 × 2 ^L ) th column side, and then the shift area is (2 ^{L + 1} +1) -th with row to row shift to match the 0th row, delete the vertices corresponding to the 0th row and the R _L rows in the intermediate array structure to produce a two-dimensional array structure, it is preferable.

  Thus, a rectangular image can be easily created by moving the vertices of the intermediate array structure.

  The image generation system according to the present invention includes an image acquisition unit that acquires a plurality of captured images, and the plurality of captured images acquired by the image acquisition unit are projected onto a discrete spherical surface discretely represented by a plurality of pixels. A spherical image generation unit that generates a spherical image, and an image conversion unit that converts the spherical image into a rectangular image, and each of the plurality of pixels representing a discrete spherical surface has a spherical surface divided by the geodetic dome method. The rectangular image has a plurality of square pixels corresponding to the vertices, and the pixel values of each of the square pixels are arranged in the first direction with the plurality of vertices on the meridian of the geodetic dome as the base points. The vertexes corresponding to the first and second poles of the geodetic dome are deleted from the intermediate array structure in which vertices having the same latitude as each base point are arranged in the second direction orthogonal to the first direction with respect to each base point As well as the intermediate sequence structure In a two-dimensional array structure in which a plurality of vertices of the intermediate array structure are arranged in a quadrangular shape by moving the vertices in the first and second directions, the pixels of the spherical image corresponding to the vertices at the positions of the respective quadrangular pixels It is the pixel value of this.

  The imaging method according to the present invention includes a step of acquiring a plurality of captured images, and a discrete spherical surface in which the acquired plurality of captured images are pixels corresponding to each vertex of the geodetic dome obtained by dividing the spherical surface by the geodetic dome method. A step of generating a spherical image by projecting to the base, and using a plurality of vertices on one meridian of the geodetic dome as base points, a plurality of base points are arranged in the first direction, and vertices having the same latitude as each base point are set as the base points. Generating an intermediate array structure arranged in a second direction orthogonal to the first direction, and vertices other than the vertices corresponding to the first and second poles of the geodetic dome included in the intermediate array structure Is moved in the first and second directions to generate a two-dimensional array structure in which vertices other than the vertices corresponding to the first and second poles of the geodetic dome are arranged in a square shape. Step Generating a rectangular image by assigning pixel values of spherical image pixels corresponding to the vertices of the positions of the respective quadrangular pixels, with each vertex constituting the two-dimensional array structure being a quadrilateral pixel. .

  According to such an imaging system and imaging method, a spherical image having a plurality of captured images as one image is converted into a rectangular image. In this case, it is only necessary to store the rectangular image, so that it is easy to store the image data. Further, by storing the image as a rectangular image, a compression method such as JPEG can be easily applied.

  In the image conversion apparatus according to the present invention, a reduced image generating unit that generates a reduced image by converting a rectangular image having L divisions into a rectangular image having (L-1) divisions by the geodetic dome method. It is preferable to further provide. Similarly, in the image conversion method according to the present invention, a reduced image generation step of generating a reduced image by converting a rectangular image having L divisions into a rectangular image having (L-1) divisions by the geodetic dome method. It is preferable to further include Thereby, a reduced image obtained by reducing the generated rectangular image can be generated.

In this case, in the intermediate array structure in which the number of divisions by the geodetic dome method is L, the position of the j-th column (j is an integer not less than 0 and N _i ) in the i-th row is represented as (i _L , j _L ). In a rectangular image corresponding to an intermediate arrangement structure with L times, when the square pixel corresponding to (i _L , j _L ) of the intermediate arrangement structure is represented as S _L (i _L , j _L ), the reduced image generation is performed. (Or in the reduced image generation step), the pixel value of S _L-1 (i _L−1 , j _L−1 ), which is a square pixel in the rectangular image as the reduced image, is divided into L times. A reduced image can be generated by making it correspond to the pixel value of S _L (2i _L−1 , 2j _L−1 ), which is a square pixel in the rectangular image.

Alternatively, the position of the j-th column in the i-th row (j is an integer of 0 or more and _Ni or less) is represented as (i _L , j _L ) in the intermediate array structure in which the number of divisions by the geodetic dome method is L times. In the rectangular image corresponding to the intermediate arrangement structure of L times, when the square pixel corresponding to (i _L , j _L ) of the intermediate arrangement structure is represented as S _L (i _L , j _L ), the reduced image generation unit (Or in the reduced image generation step), the pixel value of S _L-1 (i _L−1 , j _L−1 ), which is a square pixel in the reduced image, is V ^L−1 and the number of divisions is L times. when the pixel value of a certain rectangular a square pixels in the image _{S L (2i L-1,} 2j L-1) was ^{V L,} based on the equation (2) ^may calculate the ^{V L-1} .

(In Formula (2), M is the position of an adjacent vertex adjacent to the vertex corresponding to the position of (2i _L−1 , 2j _L−1 ) of the corresponding intermediate array structure in the geodetic dome having L divisions. And V _m ^L is the pixel value of each adjacent vertex.)

  In addition, the image conversion apparatus according to the present invention further includes an enlarged image generating unit that generates an enlarged image by converting a rectangular image having the number of divisions of L by the geodetic dome method into a rectangular image having the number of divisions of (L + 1). It is preferable. Similarly, the image conversion method according to the present invention further includes an enlarged image generation step of generating an enlarged image by converting a rectangular image having L divisions into a rectangular image having (L + 1) divisions by the geodetic dome method. It is preferable to provide. Thereby, the enlarged image which expanded the produced | generated rectangular image can be produced | generated.

In this case, in the intermediate array structure in which the number of divisions by the geodetic dome method is L, the position of the j-th column (j is an integer not less than 0 and N _i ) in the i-th row is represented as (i _L , j _L ). When a pixel corresponding to (i _L , j _L ) of the intermediate array structure is represented as S _L (i _L , j _L ) in a rectangular image corresponding to the intermediate array structure of L times, the enlarged image generation unit (Or in the magnified image generation step), if the square pixel in the rectangular image as the magnified image corresponds to the apex of the geodetic dome whose number of divisions is L, the square pixel in the rectangular image as the magnified image is S _{L + 1} the pixel value of the _{(i L + 1, j L} + 1), S L (2i L, 2j L) in the rectangular image division number is L times the pixel value of square pixels in the rectangular image as an enlarged image, dividing the number of If corresponding to the new vertex created between two adjacent vertices of the vertex of the geodesic dome which is L times, a square pixel in the rectangular image as an enlarged image S _{L + 1} of the _{(i L + 1, j L} + 1) It is preferable that the pixel value is V _new ^{L + 1,} and V _new ^{L + 1} is an average value of pixel values corresponding to two adjacent vertices among the vertices of the geodetic dome whose number of divisions is L.

Alternatively, the position of the j-th column in the i-th row (j is an integer of 0 or more and _Ni or less) is represented as (i _L , j _L ) in the intermediate array structure in which the number of divisions by the geodetic dome method is L times. In the rectangular image corresponding to the intermediate arrangement structure of L times, when the square pixel corresponding to (i _L , j _L ) of the intermediate arrangement structure is represented as S _L (i _L , j _L ), the enlarged image generation unit (Or in the magnified image generation step), if the square pixel in the rectangular image as the magnified image corresponds to the apex of the geodetic dome whose number of divisions is L, the square pixel in the rectangular image as the magnified image is S _{L + 1} the pixel value of the _{(i L + 1, j L} + 1), S L (2i L, 2j L) in the rectangular image division number is L times the pixel value of square pixels in the rectangular image as enlarged image, (L 1) A quadrilateral pixel in a rectangular image as an enlarged image in the case of corresponding to a new vertex generated between two adjacent vertices among the vertices of a geodetic dome whose number of divisions is L times It is also preferable that the pixel value of S _{L + 1} (i _{L + 1} , j _{L + 1} ) is V _new ^{L + 1,} and V _new ^{L + 1} is calculated by equation (3).

(In Expression (3), V _n1 ^L and V _n2 ^L are pixel values of two adjacent vertices among the vertices of the geodetic dome whose number of divisions is L, and V _t1 ^L and V _t2 ^L are divided. (It is a pixel value of two vertices adjacent to any of the two adjacent vertices among the vertices of the geodetic dome having L times.)

Furthermore, the image conversion apparatus according to the present invention further includes a perspective image generation unit that generates a perspective image from a spherical image as a geodetic dome corresponding to a rectangular image, and the perspective image generation unit includes a pixel array of the perspective image to be generated A structure is generated, and a point on the spherical image corresponding to the pixel p in the pixel arrangement structure is m, and the coordinates of the point m are (θ, φ) (θ is connecting the first and second poles of the geodetic dome. x _s y having a linear and z _s axis, the origin midpoint between the first and second pole on the z _s axis, the two axes orthogonal to the z _s axis as x _s-axis and y _s axis _s z _s is the angle with respect to z _s axis in the coordinate system, when φ is set to the angle) with respect to x _s axis, the pixel value of the pixel p, the geodesic dome, the pixel values of the three vertices surrounding the point m It is preferable to generate a fluoroscopic image by calculating based on the above. Similarly, the image conversion method according to the present invention further includes a perspective image generation step of generating a perspective image from a spherical image as a geodetic dome corresponding to a rectangular image, and in the perspective image generation step, pixels of the perspective image to be generated An array structure is generated, and the point on the spherical image corresponding to the pixel p in the pixel array structure is m, and the coordinates of the point m are (θ, φ) (θ is the first and second poles of the geodetic dome. x _s with a line connecting the z _s axis, the middle point between the first and second pole on the z _s axis as the origin, the two axes orthogonal to the z _s axis as x _s-axis and y _s axis y _s z _s is the angle with respect to z _s axis in the coordinate system, when φ is set to the angle) with respect to x _s axis, the pixel value of the pixel p, the geodesic dome, pixels of three vertices surrounding the point m Generating a perspective image by calculating based on the value It is preferred.

In this case, the perspective image generation unit (or the perspective image generation step) sets the pixel value of the point m to V _m and sets the pixel values of three vertices surrounding the point m in the geodetic dome corresponding to the rectangular image to V _tk (k is , 1, 2, 3), V _m can be calculated based on equation (4).

(In the equation (4), the coefficients a, b, c are for the three vertices when θ _k and φ _k are angles with respect to the z _s axis and the x _s axis of each of the three vertices,
V _tk = aθ _k + bφ _k + c
It is a real number specified by. )

  The image conversion apparatus according to the present invention further includes a perspective image generation unit that generates a perspective image from a spherical image as a geodetic dome corresponding to a rectangular image, and the perspective image generation unit includes a pixel array of the perspective image to be generated When the point on the spherical image corresponding to the pixel p in the pixel array is m, the pixel value of the pixel p is the pixel value of the apex of the geodetic dome corresponding to the rectangular image closest to the point m Therefore, it is preferable to generate a fluoroscopic image. Similarly, the image conversion method according to the present invention further includes a perspective image generation step of generating a perspective image from a spherical image as a geodetic dome corresponding to a rectangular image, and in the perspective image generation step, pixels of the perspective image to be generated When an array is generated and the point on the spherical image corresponding to the pixel p in the pixel array is m, the pixel value of the pixel p is the pixel closest to the point m and the vertex of the geodetic dome corresponding to the rectangular image By setting the value, a fluoroscopic image can be generated.

  According to the present invention, a spherical image can be converted into an image that can be more easily stored.

1 is a diagram schematically illustrating a configuration of an embodiment of an imaging system according to the present invention. 2A and 2B are diagrams illustrating examples of a pair of captured images acquired by the imaging apparatus. It is drawing which shows the correspondence of a captured image and a spherical image. It is a schematic diagram of the spherical image using a discrete spherical surface. FIG. 5A to FIG. 5C are diagrams showing a geodetic dome of 0 division level to 2 division level. It is a schematic diagram of a rectangular image. FIGS. 7A to 7C are diagrams showing intermediate arrangement structures in which the geodetic domes shown in FIGS. 5A to 5C are developed, respectively. It is drawing which shows grouping of an intermediate arrangement structure. FIG. 9A to FIG. 9D are diagrams showing steps for generating a two-dimensional array structure from an intermediate array structure. It is a flowchart of an example of the image generation method of an imaging system. It is drawing which shows an example of a rectangular image. It is drawing which shows an example of the imaging device which acquires the picked-up image for producing | generating the rectangular image shown in FIG. It is an example of the captured image acquired with the imaging device shown in FIG. It is drawing which shows schematically the structure of 2nd Embodiment of the image generation system which concerns on this invention. It is a schematic diagram which shows the correspondence of a spherical image and a fluoroscopic image. It is a schematic diagram of an example of the geodetic dome corresponding to the spherical image which has the projection point m shown in FIG. (A) is a drawing showing an image obtained by reducing the rectangular image shown in FIG. 11 using the first pixel value assignment method of the reduced image in the enlarged / reduced image generation unit. (B) is a drawing showing an example of a spherical image of the rectangular image as the reduced image shown in (a). (A) is a drawing showing an image obtained by reducing the rectangular image shown in FIG. 11 using the second pixel value assignment method for the reduced image in the enlarged / reduced image generation unit. (B) is a drawing showing an example of a spherical image of the rectangular image as the reduced image shown in (a). (A)-(e) is drawing which shows the image when an enlarged / reduced image generation part employ | adopts the 2nd pixel value allocation method, and reduces an image sequentially. (A) is a figure which shows the image which expanded the rectangular image shown in FIG. 11 using the 1st pixel value allocation method of a newly produced | generated pixel in an enlarged / reduced image production | generation part. (B) is a drawing showing an example of a spherical image of the rectangular image as the enlarged image shown in (a). (A) is a figure which shows the image which expanded the rectangular image shown in FIG. 11 using the 2nd pixel value allocation method of a newly produced | generated pixel in an enlarged / reduced image production | generation part. (B) is a drawing showing an example of a spherical image of the rectangular image as the enlarged image shown in (a). (A) is drawing which shows a fluoroscopic image at the time of employ | adopting the 1st pixel value allocation method for fluoroscopic images as a pixel value allocation method. (B) is a figure which shows a fluoroscopic image at the time of employ | adopting the 2nd pixel value allocation method for fluoroscopic images as an allocation method of a pixel value. It is drawing which shows an example of the expanded view of the conventional geodetic dome.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same reference numerals are used for the same elements, and redundant descriptions are omitted.

  FIG. 1 is a diagram showing a schematic configuration of an imaging system including an embodiment of an image conversion apparatus according to the present invention. The imaging system 10 includes an imaging device (image acquisition means) 20 and an image conversion device 30.

  The imaging device 20 includes two cameras 21 and 21 to acquire an image of the entire sky. Each camera 21 and 21 includes a fish-eye lens that is a wide-angle lens, and an image formed by the fish-eye lens 21A is detected by the image sensor 21B. An example of the angle of view of the fisheye lens 21A is 85 °, but may be 180 ° or more. An example of the image sensor 21B is a CCD. The two cameras 21 and 21 are provided back to back as schematically shown in FIG. As a result, the two cameras 21 and 21 capture opposite directions from substantially the same viewpoint, and acquire hemispherical images in the entire sky.

  As illustrated in FIG. 1, the imaging device 20 outputs image data of images acquired by the two cameras 21 and 21 to the image conversion device 30.

  FIGS. 2A and 2B are diagrams illustrating examples of the captured images 22 and 22 captured by the two cameras 21 and 21, respectively. Since each camera 21 uses a fisheye lens 21A, as shown in FIG. 2, the imaging region 22a in each captured image 22 is circular. The periphery of the imaging region 22 a has an overlap region 22 b that overlaps with each captured image 22. In the present embodiment, a region excluding the overlap region 22b in the circular imaging region 22a is referred to as an effective image region 22c.

  Returning to FIG. 1, the configuration of the image conversion apparatus 30 will be described. The image conversion apparatus 30 includes an image input unit 31, a spherical image generation unit 32, an image conversion unit 33, and an image output unit 34 as functional components. In FIG. 1, the configuration of the image conversion apparatus 30 is shown as a functional block. The image conversion apparatus 30 includes, as hardware, a CPU that performs arithmetic processing, a storage unit that stores predetermined data, and the like, and includes an image input unit 31, a spherical image generation unit 32, an image conversion unit 33, and an image output unit 34. Realize the function. An example of the image conversion apparatus 30 is a computer apparatus.

  The image input unit 31 receives input of image data of the captured image 22 acquired by the camera 21 from the camera 21. The image input unit 31 may receive data via at least one of wireless and wired communication, or may receive data from a recording medium (DVD, SD card, etc.).

  The spherical image generation unit 32 converts the image data of the captured image 22 received by the image input unit 31 into spherical image data of a spherical image. The spherical image is an image obtained by projecting a pair of captured images 22 onto the virtual unit spherical surface S (see FIG. 3). Since the pair of photographed images 22 and 22 each photograph a half area of the entire sky, a spherical image displaying the entire sky can be generated by using each of the captured images 22. The spherical image generation unit 32 generates a single spherical image by synthesizing the hemispherical images of the captured images 22 using the relative positions of the cameras 21 and 21 and the overlap region 22b.

  FIG. 3 is a diagram for explaining the correspondence between the captured image 22 and the spherical image. In FIG. 3, the hemispherical part of the unit spherical surface S is shown. The optical axis of the fisheye lens 21A included in the camera 21 is taken as the Z axis, and the plane passing through the principal point O of the fisheye lens 21A and perpendicular to the Z axis is taken as the XY plane. On the Z axis, a point c separated from the principal point O by the focal length f of the fisheye lens 21A corresponds to the center point of the captured image 22 orthogonal to the Z axis. A plane that passes through the center point c and is orthogonal to the Z axis is defined as an xy plane. The captured image 22 is an image formed by projecting a point P in the real space onto the xy plane with the fisheye lens 21A.

The zenith angle theta _A, when the azimuth angle is phi _A, the position coordinates of the point p on the sphere the image projected from the point P in the real space, p _a coordinate point on the captured image 22 (x, y) It is expressed as shown in equation (5).

The above formula (5) is for the case where the projection method of the fisheye lens 21A is the equidistant projection method (that is, r = fθ _A ). However, the projection method of the fisheye lens 21A may employ other projection methods such as a stereoscopic projection, a uniform solid angle projection, and an orthographic projection.

And p of the above spherical image, the correspondence between the point p _a photographed image 22, it is possible to generate the spherical image based on the captured image 22.

  The spherical surface S on which the captured image 22 is projected is theoretically continuous as shown in FIG. However, when data is processed by a computer or the like, a discrete expression is used. In the present embodiment, a data structure using regular hexagonal pixels (hereinafter, regular hexagonal pixels) 42 as shown in FIG. FIG. 4 is a schematic diagram showing an example of a spherical image. A discrete spherical image 40 represented by using regular hexagonal pixels 42 is known as an SCVT (Sperical Centroidal Voronoi Tessellation) image (for example, Q. Du, M. Bunzburger, and L. Ju, “Constrained centroidal Voronoi tessellations on general surface, ”SIAM J. Sci. Comput., 24 (5), 2003, pp1499-1506). In the present embodiment, the spherical image 40 corresponds to an SCVT image using a discretely expressed spherical surface (hereinafter referred to as a discrete spherical surface) 41.

  The discrete spherical surface 41 using the regular hexagonal pixel 42 is a dual polyhedron of the geodetic dome 50 obtained by dividing the theoretical continuous spherical surface S L times (L is an integer of 0 or more) by the geodetic dome method. In this embodiment, when explaining the position in the above-mentioned sphere or geodetic dome, an earth coordinate system such as latitude and longitude is used.

FIG. 5A to FIG. 5C are drawings showing the geodetic dome 50 at each division level in the geodetic dome method. FIG. 5A to FIG. 5C are drawings when the spherical surface 41 is viewed from the north pole (first pole) side when expressed in the earth coordinate system. FIG. 5 (a) shows a geodetic dome 50 with zero division level (L = 0), which corresponds to an icosahedron. FIG. 5B shows a geodetic dome 50 at a one-time division level (L = 1). FIG. 5C shows the case of the two-time division level (L = 2). Each number shown in the curly brackets shown in FIGS. 5A to 5C indicates the vertex number of the vertex 51 of the geodetic dome 50 shown for convenience of explanation. Further, in FIG. 5 (a) ~ FIG 5 (c), in accordance with the division level each geodesic dome 50 is shown as a geodesic dome 50 _L.

  When each vertex 51 of the geodetic dome 50 is represented by one pixel, the pixel representing each vertex 51 and the regular hexagonal pixel 42 of the discrete spherical surface 41 are dual. In this case, the direction of the pixel representing each vertex 51 with respect to the center of the geodetic dome 50 coincides with the direction from the center of the discrete spherical surface 41 to the center of the regular hexagonal pixel 42, that is, the main direction of the regular hexagonal pixel 42.

The number of divisions for generating the geodetic dome 50 is determined as L that satisfies Equation (6).

In Expression (6), N _g is the number of pixels in the effective image area 22c. The effective image area 22c may be calculated in advance based on the relationship between the imaging characteristics of the fisheye lens 21A and the imaging surface of the imaging element 21B and input to a memory or the like. For example, when the number of pixels of the effective image area 22c is 254,628 (= 127, 314 × 2), it is preferable to adopt L = 8. In this case, the number of vertices 51 of the geodetic dome 50 is 655,360, and the number of regular hexagonal pixels 42 is also 655,360.

  When the regular hexagonal pixel 42 does not correspond to the pixel of the effective image area 22c on a one-to-one basis, the regular hexagonal pixel 42 that does not correspond to the pixel of the effective image area 22c is included in the effective image area 22c corresponding to the regular hexagonal pixel 42. The interpolated pixel value is assigned using this pixel.

  The image conversion unit 33 converts the spherical image 40 into a rectangular image. With reference to FIGS. 6 to 8, the correspondence between the rectangular image generated by the image conversion unit 33 and the spherical image 40 will be described.

  FIG. 6 is a diagram showing a rectangular image 60 generated by the image conversion unit 33. FIG. 6 corresponds to the rectangular image 60 based on the geodetic dome 50 in the case of L = 2, and the number in each square pixel 61 in the rectangular image 60 is the vertex number of each vertex 51 in FIG. .

A rectangular image 60 based on a geodetic dome 50 of L division levels has a rectangular pixel 61 arranged in n _r (= 2 ^{L + 1} ) × n _c (= _5.2 ^L ) as shown in FIG. The vertices 51 of the dual geodetic dome 50 corresponding to the image 40 are made to correspond and the pixel values of the corresponding vertices 51 are assigned.

  The image conversion unit 33 includes an intermediate array structure generation unit 33A, a two-dimensional array structure generation unit 33B, and a pixel value assignment unit 33C in order to convert the spherical image 40 into the rectangular image 60.

The intermediate array structure generation unit 33A generates an intermediate array structure 70 in which the geodetic dome 50 is expanded two-dimensionally. FIG. 7A to FIG. 7C are views showing intermediate arrangement structures 70 corresponding to the geodetic domes 50 shown in FIG. 5A to FIG. 5C, respectively. In FIG. 7 (a) ~ FIG 7 (c), shows an intermediate array structure 70 corresponding to the geodesic dome 50 of L times the division level as the intermediate array structure 70 _L.

  As shown in FIGS. 7A to 7C, the intermediate array structure 70 is developed by using a meridian with the geodetic dome 50 as a reference line. That is, in the intermediate array structure 70, each vertex 51 on the reference line is arranged in the i direction (first direction), and a plurality of vertices 51 on the same latitude as each base point are defined with each vertex 51 on the reference line as a base point. The structure is arranged in the j direction (second direction) orthogonal to the i direction. 7A to 7C, each vertex 51 is shown as a square-shaped cell (hereinafter referred to as a vertex cell) 71, and the number in each vertex cell 71 is the number of each corresponding vertex 51. Vertex number. Hereinafter, the position of each vertex cell 71 (or vertex 51) of the intermediate array structure 70 is represented by (i, j).

The intermediate array structure 70 corresponding to the geodetic dome 50 at the time of L divisions includes (3 · 2 ^L +1) rows. Assuming that the number of vertex cells 71 in the i-th row is N _i , N _i is expressed as follows.

However, R _L = 3 · 2 ^L.

Since the intermediate array structure 70 is an expansion of the geodetic dome 50, the vertex cell 71 _n in the intermediate array structure 70 based on the geodetic dome 50 of the L division level (n is the vertex number corresponding to the vertex cell 71). The position (i, j) of the following is determined by using the position (i _K , j _K ) of the vertex cell 71 _n in the intermediate array structure 70 based on the geodetic dome 50 of K (where K <L) division levels. It is expressed as follows.
i = 2 ^(LK) i _K , j = 2 ^(LK) j _K

In the intermediate array structure 70, regions other than the vertex cell 71 corresponding to the north pole point and the south pole (second pole) point of the geodetic dome 50 are divided into three groups G _A , G _B , G as shown in FIG. _C can be divided. FIG. 8 shows a case where the intermediate array structure 70 of FIG. 7C is used as an example of grouping of a plurality of vertex cells 71. Group _{G A,} 0 <a region of the i ≦ ^{2 L,} the group _{G B} ^is of ^{2 L <i ≦ 2 L +} 1 regions, the group _{G C} ^is an area of _{^{2 L + 1 <i ≦ R}} L.

About the position in the intermediate arrangement structure 70 of the vertex 51 adjacent to each vertex 51 of the geodetic dome 50,
(A) When each vertex 51 appearing at the 0 division level is set as a reference vertex 51 _B ,
(B) if each vertex 51 belonging to the group _{G A} as a reference vertex 51 _B of
(C) if each vertex 51 belonging to the group _{G C} as a reference vertex 51 _B,
(D) for the case of the group _G reference vertex 51 _B of each vertex 51 belonging to the _B,
Illustrating the position of the vertex 51 adjacent each apex 51 _B as a reference.

In the following description, the operator “%” means calculation of “remainder”. “J _u ” and “J _d ” satisfy the following relationship.
J _u = N _{i +} 1/5 * {j / (N _i / 5)} + j% (N _i / 5)
J _d = N _i−1 / 5 * {j / (N _i / 5)} + j% (N _i / 5)

The case of (a) will be described.
With respect to the north pole of the geodetic dome 50, the positions of the vertex cells 71 corresponding to the vertex 51 adjacent to the reference vertex 51 _B (north pole) are the following five.

(1, 0), (1, 1), (1, 2), (1, 3), (1, 4)

With respect to the south pole of the geodetic dome 50, the positions of the vertex cells 71 corresponding to the vertex 51 adjacent to the reference vertex 51 _B (south pole) are the following five.
(R _L -1,0), (R _L -1,1), (R _L -1,2),
(R _L -1,3), (R _L -1,4)

For each of the vertices 51 other than the north and south poles at the 0 division level, if the position of the vertex cell 71 of the reference vertex 51 _B satisfies i = 2 ^L and j% 2 ^L = 0, the vertex 51 _B The position of the vertex cell 71 of the vertex 51 adjacent to is the following five positions.
(I-1, N _i-1 / 5 * (j / (N _i / 5)), (i, (j-1 + N _i )% N _i ),
(I, j + 1), (i + 1, (j-1 + N _{i + 1} )% N _{i + 1} ), (i + 1, j)

In addition, when the position of the vertex cell 71 of the reference vertex 51 _B satisfies i = 2 ^{L + 1} and j% 2 ^L = 0, the position of the vertex cell 71 of the vertex 51 adjacent to the vertex 51 _B is There are five positions.
(I−1, j), (i−1, j + 1), (i, (j−1 + N _i )% N _i ),
(I, j + 1), (i + 1, N _{i +} 1/5 * (j / (N _i / 5)))

The case of (b) will be described.
For the vertex 51 on the geodesic line connecting the one vertex 51 belonging to the geodetic dome 50 at the 0 division level and the north pole, the position of the vertex cell 71 of the reference vertex 51 _B is j% (N _i / 5). = when satisfying 0, the position of the vertex cell 71 of the vertex 51 adjacent the apex 51 _B is the following six.
(I−1, J _u ), (i, (j−1 + N _i )% N _i ),
(I, j + 1), (i + 1, (J _d -1 + N _{i + 1} )% N _{i + 1} ),
(I + 1, J _d ), (i + 1, J _d +1)

When the position of the vertex cell 71 of the vertex 51 _B is i = 2 ^L and j% 2 ^L is not 0, the positions of the vertex 51 adjacent to the vertex 51 _B are the following six.
(I-1, _Ju ), (i-1, ( _Ju- 1 + _Ni-1 )% _Ni-1 ),
(I, (j−1 + N _i )% N _i ), (i, (j + 1)% N _i ),
(I + 1, j), (i + 1, (j-1 + N _{i + 1} )% N _{i + 1} )

For vertices 51 other than satisfying the vertices 51 described above belongs to the group G _A, the position of the vertex cell 71 of the vertex 51 adjacent each apex 51 _B serving as the reference is the following six.
(I-1, _Ju ), (i-1, ( _Ju- 1 + _Ni-1 )% _Ni-1 ),
(I, (j−1 + N _i )% N _i ), (i, (j + 1)% N _i ),
(I + 1, J _d ), (i + 1, (J _d +1)% N _{i + 1} )

The case of (c) will be described.
For the vertex 51 on the geodesic line connecting one vertex 51 belonging to the 0-level geodetic dome 50 and the South Pole, the position of the vertex cell 71 of the vertex 51 _{B serving as} the reference is j% (N _i / 5). When = 0 is satisfied, the positions of the vertex cells 71 of the vertex 51 adjacent to the vertex 51 _B are the following six.
(I-1, ( _Ju- 1 + _Ni-1 )% _Ni-1 ), (i-1, _Ju ),
(I−1, J _u +1), (i, (j−1 + N _i )% N _i ),
(I, j + 1), (i + 1, J _d )

With respect to the condition is satisfied vertices 51 other than the apex 51 belongs to the group G _C, the position of the vertex cell 71 of the vertex 51 adjacent each apex 51 _B serving as the reference is the following six.
(I-1, (J _u +1)% N _i-1 ), (i-1, J _u ),
(I, (j−1 + N _i )% N _i ), (i, j + 1),
(I + 1, J _d ), (i + 1, (J _d −1 + N _{i + 1} )% N _{i + 1} ),

The case of (d) will be described.
When the respective vertices 51 belonging to the group G _B and the reference vertex 51 _B, the position of the vertex cell 71 of the vertex 51 adjacent each apex 51 _B serving as the reference is as follows.
(I-1, (j + 1)% N _i-1 ), (i-1, j),
(I, (j−1 + N _i )% N _i ), (i, (j + 1)% N _i ),
(I + 1, j), (i + 1, (j-1 + N _{i + 1} )% N _{i + 1} )

  FIG. 9 is a diagram illustrating a process of generating a secondary array structure. The two-dimensional array structure generation unit 33B converts the vertex cells 71 of the intermediate array structure 70 into a rectangular two-dimensional array structure 72 by performing row shift and column shift.

As shown in FIG. 9 (a), along with shifting the first R _L th vertex cell 71 in an intermediate array structure 70 adjacent to the 0th row of the vertex cell 71, the apexes cells 71 in each row of the group G _C , Shift the column to the ( ^5.2L ) column side. Subsequently, as shown in FIG. 9 (b), each vertex cells 71 of the group Gc, by row shift to the group G _A side, obtaining a sequence structure shown in FIG. 9 (c). Then, as shown in FIG. 9D, the vertex cell 71 corresponding to the north pole and the south pole is deleted, and a 2 ^{L + 1} row (5 · 2 ^L ) column rectangular two-dimensional array structure 72 is generated.

  The pixel value assigning unit 33C sets each vertex cell 71 of the two-dimensional array structure 72 as one square pixel 61, and assigns the pixel value of the corresponding vertex 51 to each square pixel 61, whereby the rectangular image shown in FIG. 60 is generated. Here, the pixel value of the vertex 51 is the pixel value of the pixel of the spherical image 40 corresponding to each vertex 51 of the geodetic dome 50.

As described above, the two-dimensional array structure 72 serving as the data structure of the rectangular image 60 is a predetermined vertex cell 71 of the intermediate array structure 70, that is, each row satisfying 2 ^{L + 1} + 1 ≦ i ≦ R _L −1 (shift region). The vertex cells 71 of FIG. 6 are those shifted in rows and columns. Therefore, the position of each rectangle pixel 61 rectangular image 60 has, as shown in FIG. 9 (d) or FIG. 4, taking the _{i c-axis} and _{j c-axis,} expressed in _{(i c, j} c), The correspondence between (i _c , j _c ) and (i, j) is as follows.
If i _c ^<2 L -1 and _j c> is 5i _c,
i = i _c +2 ^{L + 1} , j = j _c +1
In other cases,
i = i _c +1, j = j _c −5 i _c

The pixel value assigning unit 33C determines the vertex 51 corresponding to each square pixel 61 based on the correspondence relationship between (i _c , j _c ) and (i, j) and the correspondence relationship between the vertex cell 71 and the vertex 51. Assign pixel values.

  The image output unit 34 outputs the rectangular image 60 generated by the image conversion unit 33 or the spherical image 40 generated by the spherical image generation unit 32 to a display device, a network, an external storage medium, or the like.

  The operations of the image generation system 10 and the image conversion apparatus 30 will be described with reference to FIG.

  Photographing is performed by the cameras 21 and 21 of the imaging device 20, and a pair of photographed images 22 is acquired (captured image acquisition step S10). Thereby, the image of the whole sky is acquired.

  When the image input unit 31 receives image data of the pair of captured images 22 and 22 captured by the cameras 21 and 21 (image input step S11), the spherical image generation unit 32 converts the input image data into the spherical image 40. The spherical image 40 is generated by converting into the image data (spherical image generation step S12). The number of regular hexagonal pixels 42 of the discrete spherical surface 41 representing the spherical image 40 may be set in advance based on the equation (6), or when the spherical image 40 is generated, the spherical image generating unit 32 uses the equation. You may calculate based on (6).

  Next, the image conversion unit 33 converts the spherical image 40 into the rectangular image 60 (image conversion step S13). Specifically, the intermediate array structure generation unit 33A generates the intermediate array structure 70 based on the geodetic dome 50 dual to the spherical surface 41 for representing the spherical image 40 (intermediate array structure generation step S13A). Next, the two-dimensional array structure generation unit 33B converts the intermediate array structure 70 into the two-dimensional array structure 72 (two-dimensional array structure generation step S13B). Subsequently, the pixel value assigning unit 33C assigns the pixel value of the corresponding vertex 51 to the vertex cell 71 configuring the two-dimensional array structure 72, and generates the rectangular image 60 (pixel value assigning step S13C).

  Then, the image output unit 34 outputs the rectangular image 60 to a display device or a network (image output step S14).

  Each function of the image input unit 31, the spherical image generation unit 32, the image conversion unit 33, and the image output unit 34 included in the image conversion device 30 can be realized by a computer executing an image conversion program. The image conversion program may be stored in advance in the storage unit of the image conversion apparatus 30, or may be recorded on a computer-readable recording medium. The image conversion program includes a main module that summarizes the processing, and a module that causes the computer to execute the image input step S11, the spherical image generation step S12, the image conversion step S13, and the image output step S14. Can be.

  FIG. 11 is a diagram illustrating an example of a square image based on an actual captured image. FIG. 12 is a diagram illustrating the imaging device 20 that acquires the captured images 22 and 22 that are the basis of the rectangular image 60 illustrated in FIG. 11. FIG. 13 is a diagram illustrating a pair of captured images 22 and 22 captured by the imaging device 20 illustrated in FIG. The imaging device 20 shown in FIG. 12 has two cameras 21 and 21 arranged back to back. The cameras 21 and 21 are CAMEDIA (registered trademark) C-900 ZOOM manufactured by Olympus Corporation equipped with a fisheye lens 21A. The total number of pixels of the pair of captured images 22 and 22 is 4,915,200 pixels, and the total number of pixels of the effective image region 22c is about 1,775,437 pixels. Therefore, the number of pixels of the rectangular image 60 is 2,621,440 pixels corresponding to the number of vertices 51 of the nine-level geodetic dome 50 according to the equation (6).

  When only the rectangular image 60 is displayed as shown in FIG. 11, it is difficult to recall the captured images 22 and 22 or the spherical image 40. However, the spherical image 40 can be restored by using the correspondence between the intermediate array structure 70 and the two-dimensional array structure 72 constituting the rectangular image 60, the positional relationship between the adjacent vertices 51 in the intermediate array structure 70, and the like. Is possible.

  Since the number of pixels of the rectangular image 60 is about 1 million pixels, such as 2,621,440 pixels, as described in the method for generating the rectangular image 60, the two vertex cells 41 corresponding to the north and south poles Even if the pixel value is deleted, the quality of the display image is not substantially changed. Further, when the spherical image 40 is restored from the rectangular image 60, the pixel values of the north and south poles may be interpolated from the pixel values assigned to the surrounding pixels.

  Next, the effects of the image conversion apparatus, the imaging system, the image conversion method, and the imaging method described in this embodiment will be described in comparison with a case where the pair of captured images 22 and 22 are stored as they are.

  Since the two cameras 21 and 21 each shoot half of the entire sky, the pair of captured images 22 and 22 acquired by each camera 21 represents an all-sky image centered on a certain viewpoint. However, as shown in FIG. 2, each captured image 22 has an overlap area 22b and a surplus area 22d in addition to the effective image area 22c. However, it is necessary to secure a data area. Further, when the pair of captured images 22 and 22 are stored as they are, for example, in order to convert them into spherical images, it is necessary to store camera parameters such as the focal length f of the fisheye lens 21A used in the camera 21 together. Will occur.

  On the other hand, in the present embodiment, the pair of captured images 22 and 22 are converted into the spherical image 40 by using the image in the effective image region 22c and the image in the overlap region 22b. Therefore, if the image data of the spherical image 40 is used, it is not necessary to store data in an area other than the effective image area 22c in each captured image 22. In addition, once the spherical image 40 is generated, there is no need to store camera parameters such as the focal length f of the fisheye lens 21A used in the camera 21. In this way, the storage of data can be omitted by converting the captured images 22 and 22 into the spherical image 40.

  Furthermore, since the spherical image data of the spherical image 40 is converted into the image data of the rectangular image 60, data storage is easy. In addition, since the image data of the spherical image 40 is converted into the image data of the rectangular image 60, a compression format that can be applied to a two-dimensional image such as JPEG can be suitably used. Since the rectangular image 60 is generated via the spherical image 40, the rectangular image 60 is stored, so that data such as area data and camera parameters that are not substantially necessary in the captured image 22 as described above are stored. The need to save can also be omitted. Therefore, it is possible to store the whole sky image more compactly than in the case of the pair of captured images 22 and 22.

The first embodiment of the present invention has been described above, but the present invention is not limited to the above embodiment. For example, in the generation of two-dimensional array structure 72, it belongs to the group _{G C,}
2 ^{L + 1} + 1 ≦ i ≦ R _L −1
As shift region an area satisfying, but each vertex cell 71 in the shift region is the row shift and the column shift belongs to the group G _A,
1 ≦ i ≦ 2 ^L
A region satisfying the above condition may be shifted as a shift region in the same manner as in the embodiment to generate a quadrangular two-dimensional array structure 72.

  As an example of the photographing means, a pair of cameras 21 and 21 are arranged back to back. However, for example, a different direction may be taken while rotating one camera 21, or an image of the whole sky may be acquired using two or more cameras 21. The camera 21 has the fisheye lens 21A, but may be a wide-angle lens. Furthermore, as described above, when an image of the entire sky is acquired by rotating one camera 21 or using a plurality of cameras 21, a lens having a narrower angle of view than a wide-angle lens may be used.

  The image conversion unit 33 of the image conversion apparatus 30 includes the intermediate array structure generation unit 33A, the two-dimensional array structure generation unit 33B, and the pixel value assignment unit 33C, but is not limited thereto. The correspondence between the vertex 51 and the vertex cell 71 of the geodesic dome 50 that is dual to the discrete spherical surface 41 and the correspondence between the vertex cell 71 and the quadrangular pixel 61 are stored in advance in a memory or the like as an image conversion formula or a look-up table, for example. Thus, the image conversion unit 33 can assign a pixel value to the rectangular pixel 61 by designating the position of the rectangular pixel 61.

When the above-described image conversion formula or lookup table is used, the image conversion unit 33 can convert the spherical image 40 into the rectangular image 60 as follows. That is, based on the number of divisions L determined by Expression (3), a plane area of n _r (= 2 ^{L + 1} ) rows n _c (= _5.2 ^L ) columns is set as the image area of the rectangular image 60. This image area corresponds to the two-dimensional array structure 72 shown in FIG. Then, the image conversion unit 33 calculates the pixel position of the spherical image 40 corresponding to each square pixel 61 of the rectangular image 60 based on the above-described image conversion formula or lookup table, and sets a predetermined pixel value in the square pixel 61. To generate a rectangular image 60.

  In the above embodiment, the image conversion unit 33 generates the two-dimensional array structure 72 and assigns pixel values to the square pixels corresponding to the vertices. However, when the intermediate array structure 70 is generated, A pixel value corresponding to the cell 71 may be assigned.

(Second Embodiment)
Next, FIG. 14 is a drawing schematically showing a configuration of a second embodiment of the image generation system according to the present invention. The image generation system 80 includes an imaging device (image acquisition means) 20 and an image conversion device 90. Since the imaging device 20 is the same as that of the first embodiment, description thereof is omitted. The image conversion device 90 includes an instruction input unit 91, an instruction determination unit 92, an enlarged / reduced image generation unit 93, and a perspective image generation unit 94, and the image conversion device 30 of the first embodiment. Is different. The image conversion apparatus 90 will be described focusing on this difference.

In this embodiment as well, as in the case of the first embodiment, the geodetic dome 50 whose division level is L (that is, the number of divisions is L) is also referred to as a geodetic dome 50 _L, and the corresponding intermediate array structure 70 or The rectangular image 60 may be similarly applied with a notation method with division levels.

  The instruction input unit 91 receives various instructions input by an operator of the image conversion apparatus 90. The example of the instruction input unit 91 may accept an instruction from a touch panel type such as a liquid crystal display panel, or may accept an instruction from a keyboard. Examples of instructions received by the instruction input unit 91 are, for example, instructions regarding image enlargement or reduction, instructions for generating a fluoroscopic image, and the like.

  The instruction determination unit 92 determines the content of the instruction input by the instruction input unit 91 and inputs the instruction to the component corresponding to the instruction content. For example, if the instruction input by the instruction input unit 91 is an instruction related to image enlargement or reduction, the instruction determination unit 92 inputs that fact to the enlarged / reduced image generation unit 93. In addition, when the instruction input by the instruction input unit 91 is an instruction related to generation of a fluoroscopic image, the instruction determination unit 92 inputs the fact to the fluoroscopic image generation unit 94.

  The enlarged / reduced image generation unit 93 includes an intermediate array structure generation unit 93A, a two-dimensional structure generation unit 93B, and a pixel value assignment unit 93C.

  When the instruction input unit 91 receives an image reduction instruction, the enlargement / reduction image generation unit 93 performs image conversion by the intermediate array structure generation unit 93A, the two-dimensional structure generation unit 93B, and the pixel value assignment unit 93C. A reduced image obtained by reducing the rectangular image 60 generated by the unit 33 is generated. In this case, the enlarged / reduced image generation unit 93 functions as a reduced image generation unit.

  Further, when an instruction for enlarging an image is input by the instruction input unit 91, the enlarged / reduced image generating unit 93 includes an intermediate array structure generating unit 93A, a two-dimensional structure generating unit 93B, and a pixel value assigning unit 93C. An enlarged image obtained by enlarging the rectangular image 60 generated by the image conversion unit 33 is generated. In this case, the enlarged / reduced image generation unit 93 functions as an enlarged image generation unit.

  First, a method for generating a reduced image when the enlarged / reduced image generating unit 93 functions as a reduced image generating unit will be described.

Scaling the image generator 93, a rectangle number of divisions in the geodesic dome method the rectangular image 60 corresponding to the geodesic dome 50 _L is L times, corresponding to the number of pixels when the divided number is (L-1) times A reduced image is generated by converting to an image. In the reduced image, the number of divisions in the geodetic dome method is reduced by one, so the number of pixels is smaller than that of the rectangular image 60 _{L in} the case of L divisions. Therefore, the generation of the reduced image corresponds to a downsampled rectangular image 60.

The intermediate array structure generation unit 93A generates the intermediate array structure 70 _L-1 when the number of divisions in the geodetic dome method is (L-1), similarly to the intermediate array structure generation unit 33A of the image conversion unit 33. . Then, the two-dimensional structure generation unit 93B uses the intermediate array structure 70L _-1 generated by the intermediate array structure generation unit 93A as a reduced image as in the case of the two-dimensional structure generation unit 33B of the image conversion unit 33. The pixel arrangement structure of the image _60L-1 is generated. This pixel arrangement structure is an arrangement structure of quadrangular pixels 61 constituting a rectangular image _60L-1 as a reduced image, and corresponds to the two-dimensional arrangement structure 72L _-1 in the first embodiment.

Here, for explanation, the j-th column of the i-th row in the intermediate array structure 70 _L when dividing the number by the geodesic dome method is L times (j is an integer 0 or N _i) vertex cell in The position of 71 (or the corresponding vertex) is represented as (i _L , j _L ). In addition, in the rectangular image 60 _L corresponding to the intermediate array structure 70 _L when the number of divisions is L, the rectangular pixel 61 corresponding to the vertex cell 71 at the position (i _L , j _L ) of the intermediate array structure 70 _{L. L} is represented as S _L (i _L , j _L ). In this case, i _L = 2i _L−1 and j _L = _2L−1 are established.

The pixel value assigning unit 93C uses one of the following two pixel value assigning methods to convert the pixels in the reduced image, in other words, the rectangular pixels 61 in the pixel arrangement structure that becomes the rectangular image 60 _L-1 to be the reduced image. _A pixel value is assigned to S _L-1 (i _L−1 , j _L−1 ) which is _L−1 .

(First Pixel Allocation Method in Reduced Image Generation)
In, the pixel value assigning unit 93C This _method, based on the relationship _{i L = 2i L-1,} j L = 2 L-1, the _{S L-1 (i L-} 1, j L-1), divided A pixel value of S _L (2i _L−1 , 2j _L−1 ), which is a pixel in the rectangular image 60 _L that is L times, is assigned.

(Second pixel allocation method in reduced image generation)
In this method, the pixel value assigning unit 93C sets the pixel value of S _L-1 (i _L−1 , j _L−1 ), which is the square image 61 _L−1 , to V ^L−1 , and the number of divisions is L times. when the pixel value of a certain rectangular an image 60 square pixels 61 in _{L L S L (2i L-} 1, 2j L-1) was ^{V L,} based on the equation ^(8), calculates the ^{V L-1} To do.

In Expression (8), M is the corresponding vertex of the vertex cell 71 at the position of (2i _L−1 , 2j _L−1 ) in the corresponding intermediate array structure 70 _{L in} the geodetic dome 50 _L corresponding to the L-th division. This is the number of adjacent vertices, and V _m ^L is the pixel value of each adjacent vertex. Therefore, in the equation (8), the pixel value of the pixel corresponding to each vertex number in the geodetic dome 50 _L-1 whose number of divisions is (L-1) is the geodetic dome 50 _L whose number of divisions is _L. Is calculated in consideration of the influence of the pixel value of the vertex adjacent to the corresponding vertex in.

Note that the pixel values of the adjacent vertices adjacent to the corresponding vertex of the vertex cell 71 specified by the position (2i _L−1 , 2j _L−1 ) in the intermediate array structure 70 _L are (a) to (a) to (b) in the first embodiment. The pixel value of the rectangular pixel 61 of the rectangular image 60 _L corresponding to the vertex cell 71 calculated based on the method of calculating the position of the vertex 51 adjacent to the vertex 51 _B described in (d) may be used. .

The case where L = 2 will be specifically described. Geodesic dome 50 ₂ and the intermediate array structure 70 ₂ corresponding to the rectangular image 60 ₂ before downsampling are those shown in FIG. 5 (c) and FIG. 7 (c), the by geodesic dome method (L-1) times geodesic dome 50 ₁ and the intermediate array structure 70 ₁ corresponding to the divided are those shown in FIGS. 5 (b) and 7 (b). Explaining with an example case of calculating the pixel value corresponding to the vertex 51 ₁₄ shown in FIG. 5 (b), the pixel value assignment portion 93C has a vertex pixel values corresponding to the vertex 51 ₁₄ in FIG. 5 (c) 51 _{44, 51 49,} 51 _{57, 51 56,} 51 55, ₅₁ is calculated in consideration of the influence of the pixel values corresponding to _48.

  As can be understood from FIGS. 5A to 5C, in the case of 12 vertices 51 when L = 0, the number of adjacent vertices around the vertex 51 is 5, and L = 1. In the above case, the number of adjacent vertices for the newly generated vertex 51 is six.

  Next, a method for generating an enlarged image when the enlarged / reduced image generating unit 93 functions as an enlarged image generating unit will be described.

Scaling the image generator 93 converts the rectangular image 60 _L of split times in the geodesic dome method corresponds to the geodesic dome which is L times, a rectangular image corresponding to the number of pixels when the divided number is (L + 1) times By doing so, an enlarged image is generated. In the enlarged image, since the number of divisions in the geodetic dome method is increased by one, the number of pixels is larger than that of the rectangular image 60 _{L in} the case of L divisions. Therefore, generation of the enlarged image corresponds to that upsampling a rectangular image 60 _L.

Intermediate sequence structure generating section 93A, as in the intermediate sequence structure generation portion 33A of the image conversion unit 33, dividing the number of geodesic dome method to generate intermediate array structure 70 L _{+ 1} in the case of (L + 1) times. The two-dimensional structure generation unit 93B then uses the rectangular image 60 as an enlarged image from the intermediate array structure 70L _{+ 1} generated by the intermediate array structure generation unit 93A in the same manner as the two-dimensional structure generation unit 33B of the image conversion unit 33. An image arrangement structure of _{L + 1} square pixels 61 is generated. The pixel array structure is an array structure of square pixels 61 constituting a rectangular image 60 _L of a reduced image, which corresponds to a two-dimensional array structure 72 _L in the first embodiment.

Here, similarly to the explanation in the case of the reduced image, the j-th column (j of the i-th row in the intermediate array structure 70 _L when dividing the number by the geodesic dome method is L times, 0 or N _i an integer ) Is represented as (i _L , j _L ). In addition, in the rectangular image 60 _L corresponding to the intermediate array structure 70 _L when the number of divisions is L, the rectangular pixel 61 corresponding to the vertex cell 71 at the position (i _L , j _L ) of the intermediate array structure 70 _{L. L} is represented as S _L (i _L , j _L ). In this case, i _{L + 1} = 2i _L and j _{L + 1} = 2 _L are established.

Square pixels 61 in the enlarged image, when the corresponding one-to-one to the apex of the geodesic dome 50 _L of L times division, _S L _{+ 1} is to be the enlarged image is a square pixel 61 of the pixel array structure _{(i L + 1, j L} + 1) Is the pixel value of S _L (2i _L , 2j _L ) in the rectangular image 60 _L in which the number of divisions is L.

In addition, the quadrature pixel 61 in the enlarged image is a new vertex generated between two adjacent vertices 51 among the vertices 51 of the L-th divided geodetic dome 50 _{L in} the (L + 1) -th divided geodetic dome 50 _{L + 1} . 51, the pixel value of the square pixel 61 corresponding to the new vertex 51 can be assigned by one of the following two pixel value assignment methods.

(First pixel allocation method for newly generated pixels)
In this method, the pixel value of the quadrangular pixel 61 corresponding to the newly generated vertex 51 is calculated as the average value of the pixel values of the quadrangular pixels 61 corresponding to the two vertices 51 and 51 adjacent to the vertex 51. That is, if the pixel value of the square pixel 61 which is a newly generated pixel is V _new ^{L + 1,} and the pixel values of the square pixel 61 corresponding to the two adjacent vertices 51 and 51 are V _n1 ^L and V _n2 ^L , the formula ( Based on 9), V _new ^{L + 1} is calculated.

The case of L = 1 will be specifically described. The geodetic dome 50 ₁ and the intermediate array structure 70 ₁ corresponding to the rectangular image 60 ₁ before up-sampling are shown in FIGS. 5B and 7B, and are divided into (L + 1) times by the geodetic dome method. corresponding geodesic dome 50 ₂ and the intermediate array structure 70 ₂ are those shown in FIG. 5 (c) and FIG. 7 (c).

Figure 5 (c) within the geodesic dome 50 ₂ has vertices 51 shown in square pixels 61 of the pixel array structure corresponding to the vertex _{51 14} having the vertex number of the geodesic dome 50 ₁ and the same shown in FIG. 5 (b) assigns the pixel values at the vertices _{51 14.} On the other hand, calculated as an average value of the geodesic dome 50 when calculating the pixel value of the square pixels 61 corresponding to the vertex _{51 56} newly generated in _2, pixel values corresponding to the geodesic dome 50 ₁ vertex ₅₁ 14, 51 ₃ To do. Here it has been described the vertex _{51 14} and vertex _{51 56} geodesic dome 50 ₂ is the same for the other vertices 51.

Even when an enlarged image is generated, the pixel values of adjacent vertices adjacent to the corresponding vertex of the vertex cell 71 specified by the position (2i _L , 2j _L ) in the intermediate array structure 70 are the same as those in the first embodiment. The pixel value of the rectangular pixel 61 of the rectangular image 60 _L corresponding to the vertex cell 71 calculated based on the method for calculating the position of the vertex 51 adjacent to the vertex 51 _B described in (a) to (d). And it is sufficient. The same applies to the second pixel value assignment method described later.

(Second pixel value assignment method for newly generated pixels)
In this method, when the pixel value of the square pixel 61 corresponding to the newly generated vertex 51 is V _new ^{L + 1} , V _new ^{L + 1} is calculated by Expression (10).

In Expression (10), V _n1 ^L and V _n2 ^L are pixel values of two vertices 51 and 51 adjacent to the newly generated vertex 51 among the vertices 51 of the L-th geodesic dome 50 _L , and V _t1 ^L and V _t2 ^L are pixel values of two vertices 51 and 51 adjacent to both of the two vertices 51 and 51 adjacent to the newly generated vertex 51 out of the vertices 51 of the L-th geodetic dome 50 _L. is there.

The case of L = 1 will be specifically described. As described above, geodesic dome 50 ₁ and the intermediate array structure 70 ₁ corresponding to the up-sampling before rectangular image 60 ₁ are those shown in FIG. 5 (b) and 7 (b), by geodesic dome method ( L + 1) times corresponding to the divided geodesic dome 50 ₂ and the intermediate array structure 70 ₂ are those shown in FIG. 5 (c) and FIG. 7 (c).

In geodesic dome 50 vertices _{51 2} has shown in FIG. 5 (c), the square pixel 61 of the rectangular image 60 ₂ corresponding to the vertex _{51 14} having the same vertex number as geodesic dome 50 ₁ shown in FIG. 5 (b) assigns a pixel value of the vertex _{51 14.}

Further, the comparison of FIGS. 5 (b) and 5 FIG. 5 (c), the example, the vertex _{51 56} is created between the apex _{51 14} and vertex 51 _3, vertices _{51 56} appear at L times division and located between the apexes _{51 14} and vertex 51 ₃ is a vertex 51. In this case, the vertex _{51 56,} apexes _{51 14} and vertex 51 ₃ corresponding to the two vertices 51 and 51 adjacent the. The vertex _{51 19} and vertex _{51 18,} a vertex 51 that is adjacent to any of the vertices _{51 14} and vertex 51 _3. In this case, the vertex _{51 56} apex _{51 14} in geodesic dome 50 ₁ at L times division, vertex 51 _3, are located within the rectangle formed by the vertices _{51 19} and vertex _{51 18.} Equation (10) will determines the pixel value of the square pixels 61 corresponding to the four vertices _{51 14,} 51 _3, 51 19, _{51 18} apex 51 of the new generation in consideration of the pixel value of .

Specifically, in equation _{(10), V n1} ^L and _{V n2} ^L is the pixel value of the vertex _{51 14} and vertex 51 _3, pixels of the _{V t1} ^L and _{V t2} ^L vertices _{51 19} and vertex _{51 18} Value. Here has been described the case of calculating the pixel value of the vertex 51 _56, it is the same for the vertex 51 that is newly generated in the geodesic dome 50 _2.

  The perspective image generation unit 94 generates a perspective image from the rectangular image 60. The rectangular image 60 is another expression of the geodetic dome 50. If the correspondence between the rectangular pixel 61 of the rectangular image 60 and the position of the vertex 51 in the geodetic dome 50 is stored as a lookup table, the rectangular image 60 can be used. The geodetic dome 50 or the spherical image 40 dual to the geodetic dome 50 can be easily generated. Therefore, for the sake of explanation, a case where a perspective image is generated from the spherical image 40 will be described.

  The perspective image generation unit 94 includes a frame formation unit 94A and a pixel value assignment unit 94B. A method for generating a fluoroscopic image by the fluoroscopic image generating unit 94 will be described with reference to FIG. FIG. 15 shows the correspondence between the spherical image 40 and the fluoroscopic image.

  As illustrated in FIG. 15, the frame forming unit 94 </ b> A forms an image frame 100 configured by arranging a plurality of pixels 101 to be a perspective image 110 in a rectangular shape.

  The pixel value assigning unit 94B generates the perspective image 100 by projecting each pixel 101 of the image frame 100 onto the spherical image 40 and assigning the pixel value of the corresponding projection point to the pixel of the image frame 101. Here, the projection point of the pixel p of the image frame 100 is referred to as a projection point m.

  When the projection point m corresponds to the vertex 51 of the geodetic dome 50, the pixel value assigning unit 94B assigns the pixel value of the vertex 51 to the pixel p.

When the projection point m does not correspond to the vertex 41 of the geodetic dome 50, for example, as shown in FIG. 16, the pixel value assigning unit 94B forms the projection point m with three vertices 51 ₁₄ , 51 ₁₃ , 51 _18. When located within the triangle, a pixel value is assigned to the projection point m, that is, the pixel p by any of the following methods. FIG. 16 is a schematic diagram of an example of the geodetic dome 50 corresponding to the spherical image 40 having the projection point m shown in FIG.

(First pixel value assignment method for perspective image)
The pixel value assigning unit 94B assigns the pixel value of the vertex 51 (hereinafter referred to as the nearest vertex) closest to the projection point m as the pixel value of the pixel p. The closest vertex 51 is, for example, the position vector of the projection point m and the position vector of each vertex 51 in the three-dimensional coordinate system (x _s y _s z _s coordinate system shown in FIG. 15) for displaying the spherical image 40. It is possible to search by taking the inner product and setting the vertex having the largest inner product value as the closest vertex.

In the x _s y _s z _s coordinate system shown in FIG. 15, the z _s axis indicates that the spherical image 40 is regarded as the earth, that is, the north pole (first pole) and the south pole (second) in the earth coordinate system representation. And a three-dimensional coordinate system having the origin at the midpoint between the north and south poles on the z _s axis.

(Second pixel value assignment method for perspective image)
The pixel value assigning unit 94B calculates the pixel value of the projection point m using the pixel values of the three vertices 51, 51, 51 surrounding the projection point m, and assigns it to the pixel p. This will be specifically described.

In the x _s y _s z _s coordinate system for representing the spherical image 40, the deflection angles from the z _s axis and the x _s axis with respect to an arbitrary point k on the geodetic dome 50 corresponding to the spherical image 40 are θ _k and Let φ _k . That is, the points on the geodetic dome 50 are represented by spherical coordinates. In this case, it denotes the pixel values V _m of the projection point m by the following equation.

In Expression (11), θ _m and φ _m are the declination angles θ and φ with respect to the projection point m, and the coefficients a, b, and c are values calculated based on the following three expressions.

In the expressions (12a), (12b), and (12c), (θ _t1 , φ _t1 ), (θ _t2 , φ _t2 ), and (θ _t3 , φ _t3 ) are the three vertices surrounding the projection point m, respectively. The declination angle θ and declination φ of the positions 51, 51, 51 are spherical coordinates indicating the positions of these three vertices 51, 51, 51. V _t1 , V _t2, and V _t3 are pixel values of the vertex 51 at the positions (θ _t1 , φ _t1 ), (θ _t2 , φ _t2 ), and (θ _t3 , φ _t3 ).

  In the image conversion apparatus 90, in the image conversion method shown in FIG. 10, when a predetermined instruction is received from the instruction input unit 91 after step S13, the determination unit 92 determines the instruction content and responds to the instruction content. Execute the process.

  For example, when generating a reduced image from the rectangular image 60 generated in step S13, the enlarged / reduced image generation unit 93 generates a reduced image by the method described above. At this time, the enlarged / reduced image generation unit 93 generates a reduced image based on the first and second pixel value assignment methods for the reduced image set in advance.

  When generating an enlarged image from the rectangular image 60 generated in step S13, the enlarged / reduced image generating unit 93 generates an enlarged image by the method described above. At this time, the enlarged / reduced image generation unit 93 generates an enlarged image based on the first and second pixel value assignment methods for newly generated pixels set in advance.

  Furthermore, when it is to generate the fluoroscopic image 100 from the rectangular image 60, the fluoroscopic image generation unit 94 generates the fluoroscopic image 100 by the method described above. At this time, the fluoroscopic image generation unit 94 generates the fluoroscopic image 100 based on the first and second pixel value assignment methods for the fluoroscopic image 100 set in advance. The rectangular image 60 for the perspective image generation unit 94 to generate the perspective image 100 may be a reduced image generated by the enlarged / reduced image generation unit 93 or a rectangular image 60 as an enlarged image. It may be a rectangular image 60 generated by the conversion unit 33 and not enlarged or reduced. These can be executed based on the instruction content input by the instruction input unit 91 and determined by the determination unit 92.

  In the image conversion apparatus 90, after executing one or a plurality of processes described above after step 13, step 14 can be executed to output an image.

  The image conversion apparatus 90 described above further includes the enlarged / reduced image generation unit 93, so that the rectangular image 60 generated by the image conversion unit 33 can be enlarged or reduced to be a new rectangular image 60. Since the rectangular image 60 corresponds to the spherical image 40, the image conversion apparatus 90 can enlarge or reduce the spherical image 40. This point will be described with reference to experimental examples.

  FIG. 17A is an image obtained by reducing the rectangular image 60 using the above-described (first pixel value assignment method for reduced image) in the enlarged / reduced image generation unit 93. FIG. 17B is an example of a spherical image of the rectangular image 60 as the reduced image shown in FIG. FIG. 18A is an image obtained by reducing the rectangular image 60 using the above-described (second image value assigning method of reduced image) in the enlarged / reduced image generation unit 93. FIG. 18B is an example of a spherical image of the rectangular image 60 as the reduced image shown in FIG. The rectangular image 60 before the reduction was formed in the same manner as the rectangular image 60 shown in FIG. 11 except that the number of divisions L in the geodetic dome method was seven. The number of pixels of the rectangular image 60 before the reduction is 640 × 256 = 163,840 pixels.

  By using the enlarged / reduced image generation unit 93, the reduced rectangular image 60 shown in FIGS. 17A and 17B and FIGS. 18A and 18B can be obtained. Note that the number of pixels of the rectangular image 60 shown in FIGS. 17A and 18A is 320 × 128 pixels.

  17 (a) and 18 (a) and FIGS. 17 (a) and 18 (b) are compared with each other, the pixel value changes more when the second pixel value assignment method is adopted. It turns out that it is obtained.

  Further, by repeating the generation of the reduced image by the enlarged / reduced image generation unit 93, a plurality of rectangular images 60 having different image sizes hierarchically as shown in FIGS. 19 (a) to 19 (e) are obtained. It is also possible to generate. FIG. 19A to FIG. 19E are diagrams when the enlarged / reduced image generation unit 93 employs the second pixel value assignment method to sequentially reduce the images.

  FIG. 20A is an image obtained by enlarging the rectangular image 60 using the above (first pixel value allocation method for newly generated pixels) in the enlarged / reduced image generation unit 93. FIG. 20B is an example of a spherical image of the rectangular image 60 as the enlarged image shown in FIG. FIG. 21A is an image obtained by enlarging the rectangular image 60 using the above (second pixel value assignment method for newly generated pixels) in the enlarged / reduced image generation unit 93. FIG. 21B is an example of a spherical image of the rectangular image 60 as the enlarged image shown in FIG. The rectangular image 60 before enlargement was formed in the same manner as the rectangular image 60 shown in FIG. 11 except that the number of divisions L in the geodetic dome method was seven. The number of pixels of the rectangular image 60 before enlargement is 640 × 256 = 163,840 pixels.

  By using the enlarged / reduced image generation unit 93, a rectangular image 60 obtained by enlarging the rectangular image 60 can be obtained as shown in FIGS. 20 (a) and 20 (b) and FIGS. 21 (a) and 21 (b). it can. Note that the number of pixels of the rectangular image 60 shown in FIGS. 20A and 21A is 1280 × 512 pixels.

  When comparing FIG. 20A and FIG. 21A and FIG. 20B and FIG. 21B, an image in which the pixel value changes more smoothly when the second pixel allocation method is adopted. It can be seen that

  Furthermore, since the image conversion apparatus 90 includes the perspective image generation unit 94, the generated rectangular image 60 can be output to the image output unit 34 as a perspective image. As a result, the operator of the image conversion device 90 can view the fluoroscopic image on a display device or the like. The same applies to the rectangular image 60 that has been enlarged / reduced by the enlarged / reduced image generation unit 93. This point will be described with reference to experimental examples. 22A and 22B are perspective images of a part of the rectangular image 60 shown in FIG. 11 generated by the perspective image generation unit 94. FIG. FIG. 22A is a diagram in the case where the first pixel value assignment method for perspective images is adopted as the pixel value assignment method, and FIG. 22B is a perspective image assignment method as the pixel value assignment method. It is a figure at the time of employ | adopting the 2nd pixel value allocation method of this. As shown in FIGS. 22A and 22B, it can be seen that the fluoroscopic image generation unit 94 can generate the fluoroscopic image 100. Then, comparing FIG. 22A and FIG. 22B, it can be seen that an image with smoother pixel value changes can be obtained by adopting the second pixel allocation method.

  Next, the effects of the image conversion device 90 and the image conversion method will be described based on the relationship with the environment map in the computer graphics field.

  In the construction of a virtual reality environment, so-called environmental maps are widely used. Such an environment map corresponds to an all-sky image in which the surroundings are observed from a certain position (point) in the three-dimensional environment. A spherical image obtained by projecting the surrounding environment onto a spherical surface corresponds to the environment map. The image conversion device 90 converts the spherical image into a rectangular image 60. At this time, as described in the first embodiment, a discrete spherical surface as an environment map which is an SCVT (Sperical Centroidal Voronoi Tessellation) image. A rectangular image 60 is generated based on the image 40. Therefore, the rectangular image 60 is an environment map as an all-sky image. In addition, since the rectangular image 60 corresponds to the SCVT image, hereinafter, the rectangular image 60 that is an environment map is also referred to as an SCVT map.

  In the field of computer graphics, a spherical environment map and a cubic environment map are conventionally known. The spherical environment map is obtained by capturing the surroundings in a two-dimensional image, and corresponds to, for example, an image in a case where a mirror ball is captured by a normal projection camera from a predetermined direction. The cubic environment map assumes a cube centered at a certain point, and maps projection images from the center point onto each surface of the cube on each surface.

  Here, in such an environment map, the ratio between the maximum solid angle and the minimum solid angle of a pixel when an arbitrary direction is viewed from the viewpoint is defined as a sampling rate.

  In the spherical environment map and the cubic environment map, the three-dimensional environment is transferred to two dimensions, so that the pixels are difficult to be uniform. On the other hand, in the SCVT image corresponding to the rectangular image 60, the spherical image is discretized, so that the pixel sizes are substantially uniform and the sampling rate is smaller. In addition, it is known that the spherical environment map includes a singular point. On the other hand, such a singular point does not exist in the SCVT map.

  Furthermore, in the image conversion apparatus 90 and the image conversion method using the same, the rectangular image 60 (SCVT image) can be enlarged / reduced by using the enlarged / reduced image conversion unit 93. It is possible to cope with enlargement / reduction of an image displayed as.

  Further, in the image conversion apparatus 90 and the image conversion method using the image conversion apparatus, the SCVT image is represented as the rectangular image 60, so that data storage is easy as described in the first embodiment. In addition, a compression format that can be applied to a two-dimensional image such as JPEG can be preferably used. Since the rectangular image 60 is generated via the spherical image 40 as the SCVT image, the rectangular image 60 is saved, so that data of a substantially unnecessary area in the photographed image 22 and camera parameters are saved. The need to do this can also be omitted. Therefore, it is possible to store the whole sky image more compactly than in the case of the pair of captured images 22 and 22.

  Although the second embodiment has been described above, the present invention is not limited to the above-described embodiment, and modifications such as those exemplified in the first embodiment are also possible in the present embodiment. Furthermore, in the embodiment shown in FIG. 14, the image conversion apparatus 90 has an enlarged / reduced image generation unit 93 and can form both an enlarged image and a reduced image. However, it may include only an enlarged image generation unit having a function of generating only the above-described enlarged image, or may include only a reduced image generation unit having a function of generating only the above-described reduced image. Furthermore, instead of the enlarged / reduced image generation unit 93, a configuration including an enlarged image generation unit and a reduced image generation unit may be employed. In addition, only the perspective image generation unit that generates the perspective image without the enlarged or reduced image may be further provided to the configuration of the first embodiment.

  In the embodiment illustrated in FIG. 14, the instruction input unit and the determination unit have been described as having a function of inputting an instruction to a component such as an enlarged / reduced image generation unit and a perspective image generation unit and determining the instruction. . However, the instruction input unit receives, for example, an operator input to the spherical image generation unit, the image conversion unit, and the output unit, and the determination unit determines the instruction input by the instruction input unit and configures according to the instruction content. An instruction may be input to the element. This is the same in the first embodiment.

  The present invention can be applied to a mobile robot, a surveillance camera, a rescue tool and the like that require an all-sky image. Further, the present invention can be applied to an environment map in computer graphics.

  DESCRIPTION OF SYMBOLS 10 ... Image generation system, 20 ... Imaging device (image acquisition means), 22 ... Captured image, 30 ... Image conversion device, 32 ... Spherical image generation part, 33 ... Image conversion part, 40 ... Spherical image, 41 ... Discrete spherical surface, 42: Regular hexagonal pixel (pixel corresponding to the vertex of the geodetic dome), 50 ... Geodetic dome, 51 ... Vertex of the geodetic dome, 60 ... Rectangular image, 61 ... Square pixel, 70 ... Intermediate array structure, 72 ... Two-dimensional array structure , 80... Image generation system, 90... Image conversion device, 93... Enlarged / reduced image generation unit (reduced image generation unit, enlarged image generation unit), 94.

Claims (12)

A spherical image generation unit that generates a spherical image in which a plurality of captured images are projected on a discrete spherical surface discretely represented by a plurality of pixels;
An image conversion unit for converting the spherical image into a rectangular image;
With
The plurality of pixels representing the discrete sphere correspond to the vertices of the geodetic dome obtained by dividing the spherical surface by the geodetic dome method,
The rectangular image has a plurality of rectangular pixels,
The pixel values of each of the square pixels are based on a plurality of vertices on one meridian of the geodetic dome, and a plurality of the base points are arranged in the first direction, and vertices having the same latitude as the base points are the base points. On the other hand, vertices corresponding to the first and second poles of the geodetic dome are deleted from the intermediate array structure arranged in the second direction orthogonal to the first direction, and the predetermined vertex of the intermediate array structure In the two-dimensional array structure in which the plurality of vertices of the intermediate array structure are arranged in a quadrangular shape by moving the first and second directions in the first and second directions, the position corresponding to each vertex of the quadrangular pixel It is the pixel value of the pixel of the spherical image,
An image conversion apparatus characterized by that. In the intermediate arrangement structure, when the first direction and the second direction are the row direction and the column direction, respectively, the i-th row (i is an integer from 0 to 3 · 2 ^L , and L is a geodetic dome method) The number of columns N _{i of the} number of divisions of the spherical surface has an array structure represented by the formula (1),
In the two-dimensional array structure, an area from the (2 ^{L + 1} +1) -th row to the (R _L -1) -th row of the intermediate array structure is used as a shift region, and each vertex in the shift region is set to the (5 × 2 ^L ) After the column shift toward the column side, the shift region is shifted so that the (2 ^{L + 1} +1) th row matches the 0th row, and the 0th row in the intermediate arrangement structure and it is constructed by deleting the vertices corresponding to the first R _L line,
The image conversion apparatus according to claim 1.
Image acquisition means for acquiring a plurality of captured images;
A spherical image generation unit that generates a spherical image in which a plurality of captured images acquired by the image acquisition unit are projected onto a discrete spherical surface discretely represented by a plurality of pixels;
An image conversion unit for converting the spherical image into a rectangular image;
With
The plurality of pixels representing the discrete sphere correspond to the vertices of the geodetic dome obtained by dividing the spherical surface by the geodetic dome method,
The rectangular image has a plurality of rectangular pixels,
The pixel values of each of the square pixels are based on a plurality of vertices on one meridian of the geodetic dome, and a plurality of the base points are arranged in a first direction, and vertices having the same latitude as the base points are used as the base points. On the other hand, vertices corresponding to the first and second poles of the geodetic dome are deleted from the intermediate array structure arranged in the second direction orthogonal to the first direction, and the predetermined vertex of the intermediate array structure In the two-dimensional array structure in which the plurality of vertices of the intermediate array structure are arranged in a quadrangular shape by moving the first and second directions in the first and second directions, the position corresponding to each vertex of the quadrangular pixel It is the pixel value of the pixel of the spherical image,
An image generation system characterized by that. Generating a spherical image by projecting a plurality of captured images onto a discrete spherical surface represented by pixels corresponding to each vertex of a geodetic dome formed by dividing a spherical surface by a geodetic dome method;
With a plurality of vertices on one meridian of the geodetic dome as base points, a plurality of the base points are arranged in a first direction, and vertices having the same latitude as each of the base points are orthogonal to the first direction with respect to the base points Generating an intermediate arrangement structure arranged in a second direction;
By moving a predetermined vertex out of the vertices other than the vertices corresponding to the first and second poles of the geodetic dome included in the intermediate array structure in the first and second directions, Generating a two-dimensional array structure in which vertices other than the vertices corresponding to the first and second poles are arranged in a square shape;
Generating a rectangular image by assigning pixel values of pixels of the spherical image corresponding to the vertices of the positions of the square pixels, with the vertices constituting the two-dimensional array structure being square pixels,
An image conversion method comprising: In the intermediate array structure, when the first and second directions are the row direction and the column direction, respectively, the number of columns N _i in the i-th row (i is an integer of 0 to 3 · 2 ^L or less) is expressed by the formula It has a structure represented by (2),
In the step of generating the two-dimensional array structure, the regions from the (2 ^{L + 1} +1) -th row to the (R _L -1) -th row of the intermediate array structure are defined as shift regions, and the vertices in the shift region are defined. After shifting the column toward the (5 × 2 ^L ) column side, the shift region is shifted so that the (2 ^{L + 1} +1) row matches the 0th row, and Deleting the vertices corresponding to the zeroth row and the _RLth row to generate the two-dimensional array structure;
The image conversion method according to claim 5.
Acquiring a plurality of captured images;
A step of generating a spherical image by projecting the acquired plurality of captured images onto a discrete spherical surface having pixels corresponding to the vertices of the geodetic dome obtained by dividing the spherical surface by the geodetic dome method; With a plurality of vertices on the line as base points, a plurality of the base points are arranged in a first direction, and vertices having the same latitude as each base point are in a second direction perpendicular to the first direction with respect to each base point Generating an intermediate array structure that is arranged; and
By moving a predetermined vertex of the geodesic dome included in the intermediate array structure other than the vertices corresponding to the first and second poles in the first and second directions, Generating a two-dimensional array structure in which vertices other than the vertices corresponding to the first and second poles are arranged in a square shape;
Generating a rectangular image by assigning pixel values of pixels of the spherical image corresponding to the vertices of the positions of the square pixels, with the vertices constituting the two-dimensional array structure being square pixels,
An image generation method comprising:   The apparatus further comprises a reduced image generation unit that converts the rectangular image having the number of divisions by the geodetic dome method into a rectangular image having the number of divisions of (L-1) and generates a reduced image. Item 3. The image conversion apparatus according to Item 2.   3. The image processing apparatus according to claim 2, further comprising an enlarged image generation unit configured to convert the rectangular image having the number of divisions of L times by the geodetic dome method into a rectangular image having the number of divisions of (L + 1) to generate an enlarged image. Or the image conversion apparatus of 7. Further comprising a perspective image generation unit for generating a perspective image from the rectangular image;
The perspective image generation unit generates a pixel array structure of the perspective image to be generated, and calculates a point on the spherical image corresponding to the rectangular image and corresponding to the pixel p in the pixel array structure m And the coordinates of the point m are (θ, φ) (θ is a line connecting the first and second poles of the geodetic dome with the z _s axis, and the coordinates of the first and second poles on the z _s axis are Is an angle with respect to the z _s axis in the x _s y _s z _s coordinate system having the midpoint between them as the origin and two axes orthogonal to the z _s axis as the x _s axis and the y _s axis, and φ is the x _s axis And generating the perspective image by calculating the pixel value of the pixel p based on the pixel values of the three vertices surrounding the point m in the geodetic dome. The image conversion device according to any one of claims 1, 2, 7, and 8. .   6. The method according to claim 5, further comprising the step of converting the rectangular image having L divisions by the geodetic dome method into a rectangular image having (L-1) divisions to generate a reduced image. Image conversion method.   6. The image according to claim 5, further comprising a step of generating an enlarged image by converting the rectangular image having L divisions by the geodetic dome method into a rectangular image having (L + 1) divisions. Conversion method. Generating a perspective image from the rectangular image;
In the step of generating the perspective image, a pixel array structure of the perspective image to be generated is generated, and a point on the spherical image corresponding to the rectangular image and corresponding to the pixel p in the pixel array structure was a m, coordinates of the point _{m (θ m, φ m)} (θ m , the a line connecting the first and second pole of the geodesic dome and z _s axis, the first and on the z _s axis In the x _s y _s z _s coordinate system having the midpoint between the second poles as the origin and the two axes orthogonal to the z _s axis as the x _s axis and the y _s axis, the direction of the point m relative to the origin and the z _s axis, and φ _m is an angle formed with the x _s axis with respect to the direction of the point m with respect to the origin), and the pixel value of the pixel p is the point m at the geodetic dome. By calculating based on pixel values of three vertices surrounding Image conversion method according to any one of claims 4,5,10,11, characterized in that to generate the image.