JP2016048851A - Marker embedding device, marker detection device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To embed a marker in a dynamic image so as to accurately and fast detect a digital watermark from an image with projective transformation, without being sensed by human sight.SOLUTION: A marker has a predetermined polygonal frame-shaped pattern in which a strength of gradation is monotonously increased or decreased from the inside to the outside of the frame-shaped pattern. Regarding an image of each of frames in an inputted dynamic image, in a color component that is unlikely to be sensed, the marker is superposed in a predetermined region by a marker superposing section 30. Regarding the image of each of the frames in the dynamic image, a digital watermark information superposing section 40 superposes digital watermark information in a region of the image that is determined in a positional relation with the marker superposed on the image. An embedded image output section 50 outputs the image on which the marker and the digital watermark information have been superposed, corresponding to each of the frames in the dynamic image.SELECTED DRAWING: Figure 7

Description

  The present invention relates to a marker embedding device, a marker detection device, a method, and a program for facilitating detection of a digital watermark even when projective transformation is performed on a moving image in which the digital watermark is embedded. .

  For the purpose of content identification / management, copyright protection / management, related information provision, etc. when distributing content such as images and videos, digital watermark technology that embeds other information in the content so that it is not perceived by the human eye The method used is known.

  In particular, in the digital watermark technology for image / video content, a method based on minute change of the pixel value is common. As a specific method, for example, when embedding a digital watermark, based on the embedding target component position information, spectrum spreading is performed to change the real component and imaginary component of the complex matrix independently, and the watermark pattern is independent of the input image. Generate an embedded image by adding an actual image pattern, and when detecting a digital watermark, generate a detection target sequence based on detection target component position information, extract offset information, There is a digital watermark method in which spectral despreading is performed after correcting a detection target sequence and a digital watermark embedded in a cut-out pixel block is detected (Patent Document 1).

  In addition, in digital watermarking, it is necessary that a digital signal representing digital content is resistant to various modifications received after embedding the digital watermark, that is, the digital watermark can be detected after being subjected to various modifications. .

  A particularly important modification is geometric transformation. Resistance to geometric transformation is an indispensable requirement when a target image / video content is re-photographed with a camera and a digital watermark embedded in the content is detected. This is because geometric transformation occurs due to differences in shooting distance and angle.

  As a method of providing resistance to geometric transformation, the edge of the image in which the digital watermark is embedded is detected, and the positions of the four corners of the image are specified to perform reverse projection transformation, and the state before the transformation is determined. There is a method of restoring (Patent Document 2).

  Also, there is a method of embedding a digital watermark as a spatial repetitive pattern and correcting it using the autocorrelation of the pattern at the time of detection (Patent Document 3).

  In addition, there is a method of embedding a marker in an invisible manner on the frequency space of an image and correcting it using the marker (Patent Documents 4 and 5).

JP 2003-219148 A Japanese Patent No. 4020093 Japanese translation of PCT publication No. 2003-510931 Japanese Patent No. 3949679 JP 2010-232886 A

  However, in the method described in Patent Document 2, a digital watermark can be detected from an image accompanied by deformation by projective transformation. However, in order to stably detect the edge of the image, the image is surrounded by a black frame or a white frame. It is necessary and the frame is perceived by the human eye, and thus there is a problem that the design of the content itself is greatly impaired.

  In addition, in the methods described in Patent Documents 3 to 5, it is possible to detect a digital watermark from an image accompanied by a geometric transformation without being perceived by human eyes. There is a problem that only the affine transformation represented by the above and the like, and projective transformation cannot be handled.

  Also, since there is a trade-off relationship between the marker pattern embedding strength and the marker detection stability, it is better to increase the marker pattern embedding strength in order to perform marker detection stably. In particular, there is a problem that the marker is easily visually recognized.

  Furthermore, in order to detect a marker pattern at high speed, it is important how efficiently the marker can be detected. However, the conventional method is based on the brute force method for the entire screen. Therefore, there is a problem that there is a limit when real time processing is required more severely, for example, in combination with augmented reality technology (AR).

  The present invention has been made to solve the above-described problems. Markers are used so that digital watermarks can be detected accurately and quickly from images accompanied by projective transformation without being perceived by the human eye. An object of the present invention is to provide a marker embedding apparatus, method, and program that can be embedded in a moving image.

  It is another object of the present invention to provide a marker detection apparatus, method, and program capable of accurately and quickly detecting a marker used when converting a moving image subjected to projective transformation into an original moving image from a moving image. Objective.

  In order to achieve the above object, a marker embedding device according to a first aspect of the present invention provides a predetermined polygonal frame shape in an image of each frame of an input moving image in which color components of the image are difficult to perceive. A marker superimposing unit that superimposes a marker having a frame-shaped pattern that is a pattern and has a frame-shaped pattern in which the intensity of light intensity increases or decreases monotonously from the inside to the outside of the frame-shaped pattern; A digital watermark information superimposing unit that superimposes digital watermark information on a region of the image determined in a positional relationship with the marker superimposed on the image by the marker superimposing unit for each frame image of the moving image; An embedded image output unit that outputs an image in which the marker and the digital watermark information are superimposed on each of the frames, and The maker superimposing unit is a marker embedding device that superimposes the marker by periodically changing the superposition strength for each image at a predetermined first time frequency for each frame image of the moving image. .

  A marker embedding method according to a second invention is a marker embedding method in a marker embedding device including a marker superimposing unit, a digital watermark information superimposing unit, and an embedded image output unit, The image of each frame of the input moving image is a predetermined polygonal frame-like pattern with a color component that is difficult to perceive in the image, and the gradation is changed from the inside to the outside of the frame-like pattern. A step of superimposing a marker having a frame-like pattern in which the intensity of the monotone increases or decreases monotonously on a predetermined region, and the digital watermark information superimposing unit performs the marker superimposition on the image of each frame of the moving image. A step of superimposing digital watermark information on a region of the image defined by a positional relationship with the marker superimposed on the image by a unit; Outputting an image in which the marker and the digital watermark information are superimposed on each frame of the moving image by the embedded image output unit, the marker embedding unit comprising: The step of superimposing the marker includes a marker embedding method for superimposing the marker by periodically changing the superposition strength for each image at a predetermined first time frequency for each frame image of the moving image. It is.

  According to the first and second aspects of the present invention, the image of each frame of the moving image input by the marker superimposing unit is a predetermined polygonal frame-like pattern in a color component that is difficult to perceive. In addition, a marker having a frame-like pattern in which the intensity of light intensity increases or decreases monotonously from the inside to the outside of the frame-like pattern is superimposed on a predetermined area, and the digital watermark information superimposing unit For each frame image, the digital watermark information is superimposed on the image area defined in the positional relationship with the marker superimposed on the image by the marker superimposing unit, and the embedded image output unit displays the marker for each frame of the moving image. An image in which digital watermark information is superimposed is output.

  In this way, for each frame image of the input moving image, in a color component that is difficult to perceive the image, it is a predetermined polygonal frame pattern, and from the inside to the outside of the frame pattern, A marker having a frame-like pattern in which the intensity of light and shade is monotonously increasing or decreasing is superimposed on a predetermined area, and the image of each frame of the moving image is determined in a positional relationship with the marker superimposed on the image. An image with projective transformation without being perceived by the human eye by superimposing digital watermark information on the image area and outputting an image in which the marker and digital watermark information are superimposed for each frame of the moving image Therefore, a marker can be embedded in a moving image so that a digital watermark can be detected accurately and at high speed.

  In the first invention, the digital watermark information superimposing unit may superimpose an image for each image at a predetermined second time frequency different from the first time frequency for each frame image of the moving image. The digital watermark information may be superimposed by periodically changing.

  According to a third aspect of the present invention, there is provided a marker detection device that is a predetermined polygonal frame-like pattern for each frame image of an input moving image, and that is shaded from the inside to the outside of the frame-like pattern. Using a filter for detecting a component that changes at a predetermined first time frequency used when superimposing a marker having a frame-like pattern in which the intensity of the monotonically increases or decreases A time-frequency filter unit that separates a marker component image composed of components representing the image, and, for each frame image of the moving image, for the marker component image of the image separated by the time-frequency filter unit, the marker component image An edge component is searched for in each of the predetermined adjacent pixel directions from a specific pixel at a predetermined position so as to be within the frame pattern. For each of the edge component pixels searched in the adjacent pixel direction for each of the adjacent pixel directions, an edge component that continues from the edge component pixel toward each of the predetermined adjacent pixel directions. And a side candidate extraction unit that extracts the searched edge component pixels as side candidates, and each side candidate extracted by the side candidate extraction unit for each frame image of the moving image Any one of a polygon candidate configuration unit that configures each of the polygon candidates corresponding to the predetermined polygon and a polygon candidate configured by the polygon candidate configuration unit. A marker determining unit that determines the marker superimposed on the image based on the selected polygon candidate, and the marker for each frame image of the moving image Based on the marker determined by the fixing unit, a projection conversion correcting unit that performs projective conversion on the image, and an image obtained by projective conversion by the projective conversion correcting unit for each frame image of the moving image And a digital watermark information detection unit that detects digital watermark information from an area determined by the positional relationship with the marker of the image determined by the marker determination.

  A marker detection method according to a fourth invention includes a time-frequency filter unit, an edge candidate extraction unit, a polygon candidate configuration unit, a marker determination unit, a projective transformation correction unit, and a digital watermark information detection unit. A marker detection method in a marker detection apparatus, wherein the time-frequency filter unit has a predetermined polygonal frame-like pattern for each frame image input by the time-frequency filter unit, and the frame-like shape A filter for detecting a component that changes at a predetermined first time frequency used when a marker having a frame-like pattern in which the intensity of light intensity increases or decreases monotonously from the inside to the outside of the pattern is used. Using the step of separating the marker component image composed of the component representing the marker from the image, and the image of each frame of the moving image by the edge candidate extraction unit For each of the marker component images of the image separated by the time-frequency filter unit, each of the adjacent pixel directions determined in advance from a specific pixel at a predetermined position so as to be within the frame-shaped pattern of the marker component image For each of the adjacent pixel directions, for each of the edge component pixels searched for in the adjacent pixel direction, from each of the edge component pixels to each of the predetermined adjacent pixel directions. For the edge component pixels that are continuous toward the image, extracting the searched continuous edge component pixels as side candidates, and for each frame image of the moving image by the polygon candidate configuration unit, Each of the candidate polygons corresponding to the predetermined polygon is configured by a combination of each candidate edge extracted by the candidate edge extracting unit. And the marker determining unit selects any one of the polygon candidates configured by the polygon candidate configuration unit, and superimposes on the image based on the selected polygon candidate Determining the determined marker, and performing projective transformation on the image based on the marker determined by the marker determining unit with respect to the image of each frame of the moving image by the projective conversion correcting unit. And a positional relationship between the image of each frame of the moving image by the digital watermark information detection unit and the marker of the image determined by the marker determination in the image obtained by the projective transformation by the projective transformation correction unit Detecting the digital watermark information from the area defined by the above.

  According to the third and fourth aspects of the present invention, the image of each frame of the input moving image is a polygonal frame pattern predetermined for each frame image of the moving image by the time-frequency filter unit. In addition, a component that changes at a predetermined first time frequency used when a marker having a frame-like pattern whose intensity of intensity increases or decreases monotonously from the inside to the outside of the frame-like pattern is detected. A step of separating a marker component image composed of a component representing a marker from the image using a filter for performing image processing, and an image of each frame of the moving image by the edge candidate extraction unit for the image separated by the time-frequency filter unit The marker component image is directed to each of the predetermined adjacent pixel directions from a specific pixel at a predetermined position so as to be within the frame-shaped pattern of the marker component image. For each of the adjacent pixel directions, for each of the edge component pixels searched for in the adjacent pixel direction, the edge component pixels are continuously connected to each of the predetermined adjacent pixel directions. The edge candidate extraction unit extracts the image of each frame of the moving image by the step of extracting the edge component pixels to be searched and extracting the searched edge component pixels as edge candidates and the polygon candidate configuration unit. A step of configuring each of the polygon candidates corresponding to a predetermined polygon by a combination of the respective side candidates, and a marker determining unit, among each of the polygon candidates configured by the polygon candidate configuration unit And selecting a marker superimposed on the image on the basis of the selected polygon candidate, and a projective transformation correction unit for each frame of the moving image. A projective transformation of the image of each frame of the moving image by the digital watermark information detecting unit, and a step of performing a projective transformation on the image based on the marker determined by the marker determining unit. In the image obtained by the conversion, the digital watermark information is detected from the area determined by the positional relationship with the marker of the image determined by the marker determination.

  As described above, the image of each frame of the input moving image is preliminarily obtained from the image using a filter for detecting a component that changes at a predetermined first time frequency used when the marker is superimposed. A marker component image composed of a component representing a marker having a frame shape that is a predetermined polygonal frame shape and has a frame shape pattern in which the intensity of shading monotonously increases or decreases from the inside to the outside of the frame shape pattern. For each frame image of the moving image, for the marker component image of the separated image, the edge component is searched from the specific pixel toward each of the predetermined adjacent pixel directions. For each of the pixels, the edge component pixel is searched for a continuous edge component pixel in each of the predetermined adjacent pixel directions. The pixel of the di component is extracted as a side candidate, and each of the polygon candidates is configured by the combination of the extracted side candidates, and any one of the configured polygon candidates is selected and selected. By determining the marker superimposed on the image based on the polygonal candidate, the marker used when converting the moving image subjected to the projective transformation into the original moving image can be accurately and quickly moved. Can be detected from.

  In the third and fourth inventions, the marker determination unit is configured for each frame image of the moving image by parallel processing for each of the polygon candidates configured by the polygon candidate configuration unit. For the polygon candidate, the attribute of the polygon candidate is compared in time series, and any one of the polygon candidates configured by the polygon candidate configuration unit is selected and selected. Based on the polygon candidates, the image may be superimposed.

  In the third and fourth inventions, the marker determination unit is configured for each frame image of the moving image by parallel processing for each of the polygon candidates configured by the polygon candidate configuration unit. For the polygon candidate, the polygon candidate attributes are compared in time series, the number of occurrences of reversal of the edge direction of each of the edge component pixels included in the polygon candidate is obtained, and the polygon candidate Based on the number of occurrences of reversal of the edge direction acquired for each of the polygon candidates, any one of the polygon candidates configured by the polygon candidate configuration unit is selected, and the selected polygon The marker superimposed on the image may be determined based on the candidate.

  Moreover, the program of this invention is a program for making a computer perform each step of said marker embedding method or marker detection method.

  As described above, according to the marker embedding device, method, and program of the present invention, a predetermined polygonal frame is used for the image of each frame of the input moving image in the color component that is difficult to perceive the image. Each frame of a moving image is superimposed on a predetermined area with a marker having a frame-like pattern and a frame-like pattern in which the intensity of light intensity increases or decreases monotonously from the inside to the outside of the frame-like pattern By superimposing digital watermark information on the image area defined in the positional relationship with the marker superimposed on the image, and outputting an image on which the marker and digital watermark information are superimposed for each frame of the moving image Markers are embedded in moving images so that digital watermarks can be detected accurately and quickly from images with projective transformation without being perceived by the human eye Door can be.

  Moreover, according to the marker detection apparatus, method, and program of the present invention, a component that changes at a predetermined first time frequency used when a marker is superimposed on an image of each frame of an input moving image. Using a filter for detection, a frame shape that is a predetermined polygonal frame pattern from the image, and whose intensity of intensity increases or decreases monotonously from the inside to the outside of the frame pattern A marker component image composed of components representing markers having a pattern is separated, and for each frame image of the moving image, the marker component image of the separated image is directed from the specific pixel to each of the predetermined adjacent pixel directions. The edge component is searched for, and each of the searched edge component pixels continues from the edge component pixel toward each of the predetermined adjacent pixel directions. Search for the pixel of the edge component, extract the searched pixels of the continuous edge component as edge candidates, and configure each of the polygon candidates by combining the extracted edge candidates. By selecting any one of the candidates and determining the marker superimposed on the image based on the selected polygon candidate, the moving image subjected to the projective transformation is changed to the original moving image. A marker used for conversion can be detected from a moving image with high accuracy and high speed.

It is a figure which shows the example which superimposes the frame-shaped shade pattern arrange | positioned around an image as a marker. It is a figure which shows the example of the pattern of a marker, and the embedding strength of a marker. It is a figure which shows the example of embedding a digital watermark. It is a figure which shows the example of the scan of each side direction of a marker. It is a figure which shows the example of the scanning of the direction (plus and minus both directions) perpendicular | vertical to the scanning direction of each side direction of a marker. It is a figure which shows the example which evaluates quadrilateral character. It is a block diagram which shows the functional structure of the marker embedding apparatus which concerns on embodiment of this invention. It is a figure which shows the example of the period of the time frequency used when a marker and a digital watermark are superimposed. It is a block diagram which shows the functional structure of the marker detection apparatus which concerns on embodiment of this invention. It is a flowchart figure which shows the marker embedding apparatus process routine in the marker embedding apparatus which concerns on embodiment of this invention. It is a flowchart figure which shows the marker detection process routine in the marker detection apparatus which concerns on embodiment of this invention. It is a flowchart figure which shows the extraction process routine of the edge candidate in the marker detection apparatus which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Principle of the present invention>

  In the embodiment of the present invention, a polygonal frame-shaped shading pattern arranged around the image as shown in FIG. 1 is superimposed as a marker. However, as shown in FIG. 2, the intensity of the density monotonously increases or decreases from the inside to the outside of the frame-like pattern. Thereby, it can be expected that a density change of a certain strength appears at the outermost edge of the marker.

  The marker is embedded in a color component that is difficult to perceive in visual characteristics so as not to be perceived by human eyes. In the present embodiment, the image is converted from RGB to YCbCr and embedded in the Cb component.

  The digital watermark is embedded at a predetermined position in an area defined by the positional relationship with the marker. FIG. 3 shows an example of the embedded area. Here, if the marker and the digital watermark are embedded in the same place as shown in FIG. 3, generally, the marker pattern and the digital watermark pattern interfere with each other, making it difficult to detect the marker pattern.

  Therefore, the marker pattern and the digital watermark pattern are embedded so as to periodically change at different time frequencies. On the other hand, in detecting a digital watermark, an embedded marker is detected from a color component that is difficult to perceive in an image acquired by camera photography or the like. At that time, in order to separate the acquired image into a marker component, a digital watermark component, and other components, a frequency pass filter specific to the marker component and a frequency pass filter specific to the digital watermark are used.

  Then, the position of the marker is specified from the separated marker component image, and the state before the geometric transformation is restored by performing the reverse projection transformation of the separated digital watermark component image from the position information designated by the marker. A digital watermark is detected from the restored image.

  Here, in order to detect the marker pattern at high speed, the frame detection means used in Patent Document 2 is applied. That is, when the marker is a quadrilateral, as shown in FIG. 4, the center position of the captured camera image or a predetermined position is set as a specific pixel at a predetermined position, and a predetermined adjacent pixel direction is set. Are scanned in each of the four directions in the vertical and horizontal directions of each side of the quadrilateral that is the frame pattern of the marker, and an edge component (a place where the value changes suddenly) in each direction is searched. Once the edge component is found, this time, as shown in FIG. 5, each of the predetermined adjacent pixel directions of the pixel where the edge component is found, each direction perpendicular to the scanning direction (both plus and minus directions). Trace the edge components connected by.

  Candidates related to the line segment at the edge of the image are determined by tracing in both directions in the vertical direction, which is the predetermined adjacent pixel direction. By repeating this until the last position in each scanning direction, each side candidate of the quadrilateral formed by the line segment at the end of the image is extracted. Among the quadrilateral candidates selected from a plurality of virtual quadrilaterals that can be combined with the side candidates, as shown in FIG. 6, the one that can faithfully reproduce the shape of the edge around the image is selected. With this method, a quadrilateral can be detected with high accuracy by performing only the minimum pixel reference without performing a full scan of the image.

  Here, it is considered how to select a correct quadrilateral of an image area from many quadrilateral candidates. In the conventional method as used in Patent Document 2, all quadrilaterals mainly composed of edge line segments that do not change in the time direction of the Y component (brightness) are detected as frames around the image, There were many cases where the wrong quadrilateral area continued to be tracked.

  In addition, when the entire image is a dark background such as a night view and the display color is black, the edges that should occur at the edges of the image do not appear sufficiently, and as a result, the true quadrilateral cannot be detected at all. is there.

  Therefore, in the present embodiment, the variation due to the specific frequency in the time direction, which is another feature of the marker pattern to be embedded, is actively used. In other words, since the edge component of the marker pattern varies positively or negatively at a fixed period, the edge direction of the line segment before the half cycle should be just opposite to the edge direction of the current line segment. By always confirming this in the process of tracking the quadrilateral, it is possible to specify a true quadrilateral as a marker from a plurality of quadrilateral candidates. That is, a plurality of quadrilateral candidates are tracked in parallel, sequentially checked whether their edge directions are reversed compared to the previous half cycle, and the number of occurrences of edge direction inversion exceeds a certain threshold. The object is determined to be a true quadrilateral serving as a marker, and a single true frame (marker position information) is selected from a plurality of candidates. Thereby, it is possible to prevent detection of a confusing quadrilateral such as a quadrilateral object included in the image or a shelf outside the display.

  In this embodiment, a frame-like marker arranged around the image is superimposed on a predetermined position of the target image, and a digital watermark is embedded at a predetermined position in an area defined by the positional relationship with those markers. . At that time, the marker and the digital watermark are embedded so as to periodically change at different frequencies in time, and at the time of detection, the respective components are separated by the respective frequency pass filters so that the marker and the digital watermark do not interfere with each other. Can be embedded in the same place and marker detection accuracy can be improved.

  Further, by performing reverse projection transformation from the detected marker position information, the state before the geometric transformation is restored, and the digital watermark is detected. As a result, the digital watermark can be detected with high accuracy and high speed from an image accompanied by projective transformation without being perceived by human eyes. In particular, in terms of high speed, the search can be efficiently performed without performing a full scan on the camera capture image, and therefore, it is also suitable for real-time processing such as AR. In addition, marker detection is relatively stable even if the marker embedding strength is reduced so that it is difficult for humans to visually recognize the marker.

<Configuration of marker embedding device according to the present embodiment>
Next, the configuration of the marker embedding device according to the present embodiment of the present invention will be described. As shown in FIG. 7, the marker embedding device 100 according to the present embodiment of the present invention includes a CPU, a RAM, and a ROM that stores a program and various data for executing a marker embedding processing routine described later. It can be configured with a computer including. The marker embedding device 100 functionally includes an input unit 10, a calculation unit 20, and an output unit 90 as shown in FIG.

  The marker embedding device 100 periodically inserts a marker having the above-described characteristics in a predetermined region on the video content with respect to the input video content (moving image) at a predetermined first time frequency. Embed by varying the strength of superposition. Furthermore, the marker embedding device 100 has a predetermined second time frequency different from the first time frequency in an area specified based on the position of the marker for video content (moving image) in which the marker is embedded. The digital watermark is embedded by periodically changing the strength of superimposition, and the video content in which the marker and the digital watermark are embedded is output.

  The input unit 10 receives video content prepared in advance as digital data. One image is received as each frame image continuous in time series.

  The computing unit 20 includes a marker superimposing unit 30, a digital watermark information superimposing unit 40, and an embedded image output unit 50.

  The marker superimposing unit 30 superimposes the marker on a predetermined position of the image while periodically varying the intensity of superimposing the marker in the time direction on each frame image of the video received by the input unit 10. For example, as shown in FIG. 1, one marker is superimposed around the image. In the present embodiment, one marker is superimposed (embedded) around the image. In the present embodiment, the square pattern shown in FIG. 2 is used as the marker pattern. Here, the marker pattern is a pattern in which shading increases or decreases monotonously from the inside to the outside.

  As a specific superposition method, an image is converted from RGB to YCbCr, a marker is superimposed on a predetermined area of the image, and a marker pattern is added to a color component that is difficult to perceive in the image at the corresponding position. When FIG. 2 is used as a marker, the numerical value shown as a diagram on the right side of FIG. 2 is an added value. Here, addition is performed when the value is positive, and subtraction is performed when the value is negative. In order to adjust the strength of marker superimposition, a fixed value a set in advance as a parameter may be multiplied as a coefficient and then added.

  Further, the strength for embedding the marker is periodically changed in the time direction as shown in FIG. At this time, as shown in FIGS. 8A and 8B, the digital watermark is changed at a frequency different from the temporal change of the digital watermark. Since the marker may be displaced in time, it is preferable that the temporal variation frequency of the marker is higher than the temporal variation frequency of the digital watermark. In the present embodiment, it is assumed that the strength of superimposing both the marker and the digital watermark is accompanied by temporal variation.

  The digital watermark information superimposing unit 40 is arranged in a time direction for each frame image of the video received by the input unit 10 in an area in the image defined by a relative positional relationship with a marker embedded in each frame image. The digital watermark is superimposed on a predetermined position of each frame image while periodically changing the strength of superimposing the digital watermark on the frame image. For example, an area as shown in FIG. 3 is set as an area where a digital watermark is superimposed. In addition, some variations are conceivable, such as a method in which one marker is arranged in the center of the frame image and the other area is set as an embedded area of the digital watermark. The time frequency for determining the strength for superimposing the digital watermark is different from the time frequency used in the marker superimposing unit 30. Specifically, a temporal frequency as shown in FIG.

  The method for superimposing the digital watermark on the area is based on the adopted digital watermark method. Various methods such as the method described in Patent Document 1 have been proposed as the digital watermark method, but any digital watermark method may be adopted. However, in the case of a digital watermark method that embeds at a specific frequency in the time direction, it is necessary to set the time frequency so as not to be batted with the periodic variation applied to the marker embedding strength. In addition, the digital watermarking method itself employed in the present embodiment may not have geometric conversion tolerance. The information indicated by the digital watermark includes information related to images such as content identification and management, content copyright protection and management, and related information provision.

  The embedded image output unit 50 uses the marker superimposing unit 30 and the digital watermark information superimposing unit 40 as each frame image on which the marker and the digital watermark are superimposed as an “embedded image”, and uses the embedded image as a frame. It is connected to a series of video sequences and output to the output unit 90 as one video content.

  The output unit 90 is, for example, an HDD, a DVD-ROM, an electronic recording medium such as a CD-ROM, a display device, or the like. When the output destination is an electronic recording medium, the video content is recorded on the electronic recording medium, and when the output destination is a display device, the video content output on the display device is displayed.

<Configuration of marker detection apparatus according to the present embodiment>
Next, the configuration of the marker detection device according to this embodiment of the present invention will be described. As shown in FIG. 9, the marker detection device 200 according to the present embodiment of the present invention includes a CPU, a RAM, and a ROM that stores a program and various data for executing a marker detection processing routine described later. It can be configured with a computer including. The marker detection apparatus 200 is functionally configured to include an input unit 210, a calculation unit 220, and an output unit 290 as shown in FIG.

  The marker detection device 200 determines whether or not a predetermined marker is superimposed on the input video content (moving image). When the marker is detected, the marker detection device 200 is superimposed on the video content based on the marker. The detected digital watermark is detected, and information indicated by the detected digital watermark is output.

  The input unit 210 includes digital data obtained by photographing video content embedded with a detection target digital watermark with a camera or the like, and digital data obtained by processing (for example, projective conversion) the video content embedded with the detection target digital watermark. Accept. Note that since the embedded digital watermark requires each frame image continuous in time series of a plurality of videos at the time of detection, the necessary number of frame images are received together in time series order. In the present embodiment, all frame images of digital data are received together in time series order by input unit 210.

  The calculation unit 220 includes a time-frequency filter unit 230, a marker detection unit 240, a projective transformation correction unit 250, and a digital watermark information detection unit 260.

  The temporal frequency filter unit 230 separates each frame image received by the input unit 210 into a marker component image, a digital watermark component image, and other component images by passing the temporal frequency filter. Specifically, two types, a first frequency pass filter that passes a component that changes at a first time frequency peculiar to a marker component, and a second frequency pass filter that passes a component that changes at a second time frequency peculiar to a digital watermark. Are used to separate each frame image into a marker component image, a digital watermark component image, and other component images excluding the marker component image and the digital watermark component image. Since the marker component and the digital watermark component are embedded at different time frequencies, both components can be separated by such processing. The first frequency pass filter unique to the marker component used when separating the marker component image is a filter that passes a component that changes at the same time frequency as the first time frequency used in the marker superimposing unit 30 of the marker embedding device 100. And the second frequency pass filter specific to the digital watermark component used when separating the digital watermark component image changes at the same time frequency as the second time frequency used in the digital watermark information superimposing unit 40 of the marker embedding device 100. It is a filter that allows the component to pass through.

  The marker detection unit 240 identifies the position where the marker on the marker component image is superimposed on the marker component image of each frame image separated by the time-frequency filter unit 230. Specifically, a marker is determined by tracking a quadrilateral candidate, which is a polygon candidate configured for the marker component image of each frame image, and comparing the attributes of the polygon candidate in time series. Hereinafter, the case where the marker embedded in each frame image is the marker shown in FIG. 2 will be described. The marker detection unit 240 includes a side candidate extraction unit 242, a polygon candidate configuration unit 244, and a marker determination unit 246.

  The edge candidate extraction unit 242 first converts each of the marker component images separated from each input frame image from a signal in the RGB color space to a signal in the YCbCr color space, and a marker component of a color component that is difficult for humans to perceive. Extract each of the images. In the YCbCr color space, a Cb component marker component image is extracted.

  Next, as shown in FIG. 4, a marker frame shape that is each predetermined pixel direction from a pixel located at the center of the acquired marker component image, which is a specific pixel at a predetermined position. Scanning is performed in each of the four directions (up, down, left, and right) of each side direction of the quadrilateral that is the shape of the pattern, and an edge component (a portion where the value changes rapidly) in each direction is searched. Here, the scan start position may be set at a center of gravity of the quadrilateral detection position in the marker component image one time before or an arbitrary position on a line connecting the center of gravity and the image center. It should be noted that the marker detection is successful only when the scan start position is inside the detected marker frame pattern.

  Here, unidirectional scanning will be described. As shown in FIG. 5, when scanning is performed from a predetermined specific pixel and an edge component is found once, eight pixels adjacent to the pixel in which the edge component is found are detected. Among these, the edge components in the pixels connected in the predetermined adjacent pixel direction are traced. In the present embodiment, the edge component in the pixel connected in the direction perpendicular to the pixel where the edge component is found (in both plus and minus directions) is traced.

  When no edge component is found in an adjacent pixel in a predetermined adjacent pixel direction, a pixel at a detection position immediately before a position where no edge component is detected is set as an end point on one side of the line segment. If the image is traced to the end, the pixel at the last position is set as an end point on one side of the line segment. Candidates related to the line segments at the end of the embedded image are determined by a predetermined trace in the adjacent pixel direction, and line segments connecting the end points are extracted as edge candidates located in the scan direction. By repeating this process up to the last position in the scanning direction that is each side direction, each of the side candidates in the scanning direction in each side direction constituted by the line segments at the end of the embedded image is extracted. Similarly, by performing the same processing for each scan candidate in all the side directions, each of the side candidates in each side direction constituted by the end line segments of the embedded image is extracted.

  The polygon candidate configuration unit 244 configures each of the quadrangle candidates that are polygon candidates based on each of the quadrangle side candidates extracted by the side candidate extraction unit 242. Specifically, for each of the side candidates extracted for each of the scan directions of each side direction, one side candidate is selected for each side direction, and each of the selected side candidates is combined into a quadrilateral. Configure each of the corresponding virtual quadrilaterals. Note that the virtual quadrilateral is configured for all combinations of side candidates for each scan direction, but the total number of quadrilateral candidates to be detected by the marker is determined in advance. Specifically, an evaluation value (Patent Document 2) is calculated for each of the virtual quadrilaterals configured for all combinations of edge candidates for each scan direction, as shown in FIG. Each quadrilateral candidate to be detected is selected.

  The marker determining unit 246 can faithfully reproduce the shape of the edge around the most embedded image as shown in FIG. 6 for each of the quadrangle candidates for each marker component image acquired by the polygon candidate configuration unit 244. The selected quadrilateral candidate is determined as a marker of the marker component image and extracted.

  Here, the variation due to the specific frequency in the time direction, which is another feature of the marker pattern to be embedded, is actively used. In other words, since the edge component of the marker pattern fluctuates between positive and negative at a fixed period, the edge direction that is the attribute of the line segment before the half cycle is just the opposite of the edge direction that is the attribute of the current line segment. Should be. By confirming this for each of the quadrangle candidates at all times during the quadrangle tracking process, it is possible to identify the true quadrangle as a marker from the plurality of quadrangle candidates. In other words, for each of a plurality of quadrilateral candidates, the quadrilateral candidates for each marker component image are tracked in parallel, and the edge directions are sequentially checked to see if they are reversed compared to the previous half cycle. If the number of occurrences exceeds a preset threshold value, it is determined as a true quadrilateral serving as a marker.

  Specifically, for each quadrilateral candidate tracked in each of the marker component images, if each of the quadrilateral candidates is opposite to the edge direction of the line segment before the half cycle, the tracked quadrilateral Count the number of inversions in the candidate edge direction. As a result of performing the processing for each of the tracked quadrilateral candidates, among the tracked quadrilateral candidates, quadrilateral candidates whose number of inversions in the edge direction exceeds the threshold are detected as markers of the marker component image. . In addition, when there is no quadrilateral candidate whose edge direction inversion number exceeds the threshold value, the quadrilateral candidate having the largest evaluation value may be detected as a marker of the marker component image.

Moreover, in this Embodiment, the tracking method of each quadrangle candidate can utilize the patent document 6, for example (patent document 6: patent 4571115 gazette). When the marker is not detected, it is determined that the digital watermark and the marker are not embedded, or the marker detection has failed, and the processing after the projective transformation correction unit 250 is not performed.

  The marker detection unit 240 can detect a marker for each frame image by the above-described processing, and can specify an area in which a digital watermark is embedded based on the position of the marker.

  The projection conversion correction unit 250 performs reverse projection conversion on the digital watermark component image of each frame image based on the position of the area in which the digital watermark is specified, which is specified by the marker detection unit 240. At this time, the projective transformation correction unit 250 may apply the same technique as that described in Patent Document 2, for example. Further, the projective transformation correcting unit 250 may use the projective transformation matrix calculated by the marker detecting unit 240.

  The digital watermark information detection unit 260 detects the digital watermark embedded by the digital watermark information superimposing unit 40 with respect to the digital watermark component image of each frame image obtained by the reverse projection transformation by a method corresponding to the digital watermark. At this time, assuming that the embedded digital watermark requires a plurality of images in time series at the time of detection, the marker detection unit 240 and the projective transformation correction unit 250 are repeated for the number of frame images, A digital watermark is detected after obtaining a required number of images corrected for projective transformation. The digital watermark information detection unit 260 outputs information indicated by the detected digital watermark to the output unit 290.

  The output unit 290 outputs the information output from the digital watermark information detection unit 260, that is, the digital watermark indicated by the digital watermark to the outside.

<Operation of marker embedding device according to the present embodiment>
Next, the operation of the marker embedding device 100 according to the present embodiment will be described. When the input unit 10 accepts each frame image continuous in time series for a moving image of video content prepared in advance, the marker embedding device 100 executes a marker embedding device processing routine. .

  First, in step S100, with respect to the Cb component of the entire frame image to be processed among the images of each frame received by the input unit 10, the strength of superposition changes at the first time frequency between frames. A marker is superimposed on.

  Next, in step S102, with respect to the frame image to be processed, superimposition of superposition is performed on the region in the frame image determined in the relative positional relationship with the marker superimposed in step S100 at the second time frequency between frames. A predetermined digital watermark is superimposed so that the height changes.

  In step S104, it is determined whether or not processing has been completed for all frame images to be processed. If the process has been completed for all the frame images to be processed, the process proceeds to step S106. If the process has not been completed for all the frame images to be processed, the frame image to be processed is changed. Then, the processing from step S100 to step S104 is performed.

  In step S106, the frame images on which the marker and the digital watermark are superimposed in steps S100 and S102 are joined in the order in which the images of the frames received by the input unit 10 are received.

  In step S108, the moving images connected in step S106 are output to the output unit 90, and the marker embedding processing routine is terminated.

<Operation of Marker Detection Device According to this Embodiment>
Next, the operation of the marker detection device 200 according to the present embodiment will be described. When the input unit 210 accepts a predetermined number of frame images that are continuous in time series with respect to a moving image of video content prepared in advance, the marker detection device 200 causes the FIG. The marker detection processing routine shown in FIG.

  In step S200, the first time frequency filter corresponding to the first time frequency used when the marker embedding device 100 superimposes the marker on the frame image to be processed among the frame images received by the input unit 10 is used. And separating the marker component image from the frame image. Further, the digital watermark component image is separated from the frame image by using a second temporal frequency filter corresponding to the second temporal frequency used when the digital watermark is superimposed in the marker embedding device 100.

  Next, in step S202, each edge candidate is extracted from the marker component image of the frame image to be processed acquired in step S200.

  Next, in step S204, each of the virtual quadrilaterals is configured based on each of the edge candidates in the marker component image of the frame image to be processed acquired in step S202.

  Next, in step S206, the likelihood of each quadrilateral of the virtual quadrilateral in the marker component image of the frame image to be processed acquired in step S204 is evaluated, and the target is a marker detection target based on each of the evaluation values. Select each quadrilateral candidate.

  Next, in step S208, it is determined whether or not the processing from step S200 to step S206 has been completed for all the frame images to be processed. When the process from step S200 to step S206 has been completed for all the frame images to be processed, the process proceeds to step S210, and the process from step S200 to step S206 is performed for all the frame images to be processed. If not completed, the frame image to be processed is changed, and the processing from step S200 to step S206 is repeated.

  Next, in step S210, the quadrilateral candidate is tracked based on each quadrilateral candidate for each frame image acquired in step S206, and the number of edge direction inversion occurrences is acquired for each tracked quadrilateral candidate. Based on the obtained number of edge direction inversion occurrences of each tracked quadrilateral candidate, a true quadrilateral candidate serving as a marker is determined.

  Next, in step S212, the marker of the frame image to be processed is detected based on the true quadrangle candidate to be the marker acquired in step S210.

  Next, in step S214, it is determined whether or not a marker has been detected in the frame image to be processed in step S212. If no marker has been detected for the frame image to be processed, the process proceeds to step S218. If a marker has been detected for the frame image to be processed, the process proceeds to step S216.

  Next, in step S216, based on the position of the marker for the frame image to be processed detected in step S212, the position of the area in which the digital watermark of the frame image is embedded is specified, and Based on the position, the digital watermark component image acquired in step S200 is corrected by reverse projection transformation.

  Next, in step S218, it is determined whether or not the processing of step S212 to step S214 or step S216 has been completed for all the frame images received by the input unit 10. When the process of step S212 to step S214 or step S216 is completed for all the received frame images, the process proceeds to step S220, and the process proceeds to step S212 to step S214 or step S216 for all the received frame images. If the process has not ended, the frame image to be processed is changed and the process proceeds to step S212.

  In step S220, a digital watermark is detected from the image obtained by the reverse projection transformation acquired in step S216 of each frame image received by the input unit 10.

  In step S222, it is determined whether the digital watermark is correctly detected in step S220. If the digital watermark is correctly detected, the process proceeds to step S224. If the digital watermark is not correctly detected, the marker detection processing routine is terminated.

  In step S224, the digital watermark detected in step S220 is output to the output unit 290, and the marker detection processing routine is terminated.

  The edge candidate extraction process shown in step S202 will be described in detail in the edge candidate extraction process routine shown in FIG.

  In step S300 of FIG. 12, the scan start position in the marker component image acquired in step S200 is determined based on the specific pixel at a predetermined position or the position of the marker detected in the previous process.

  Next, in step S302, scanning is performed in the target scan direction in each of the four directions of up, down, left, and right, which are each of the predetermined adjacent pixel directions, from the specific pixel at the scan start position acquired in step S300. The pixel of the edge component or the pixel at the image end of the marker component image acquired in step S200 is searched.

  Next, in step S304, from the edge component pixel searched in step S302 or the image end pixel of the marker component image acquired in step S200, the direction perpendicular to the scan direction in step 302 is the direction of 8 adjacent pixels. Scanning is performed in both directions, the edge component is traced, and each of the end points where the continuous edge component is detected is acquired.

  Next, in step S306, the line segment connecting the end points acquired in step S304 is extracted as an edge candidate in the target scan direction.

  Next, in step S308, it is determined whether or not scanning has been completed up to the end of the marker component image in step S200 for the target scan direction. If the scan has been completed to the end of the marker component image, the process proceeds to step S310. If the scan has not been completed to the end of the marker component image, the process proceeds to step S302, and the target scan direction is reached. Start the scan again.

  Next, in step S310, it is determined whether or not processing has been completed for all target scan directions. If the processes in steps S302 to S308 have been completed for all target scan directions, the edge candidate extraction processing routine is terminated, and the processes in steps S302 to S308 are performed for all target scan directions. If not completed, the target scan direction is changed, and the processes in steps S300 to S308 are performed.

  As described above, according to the marker embedding device according to the present embodiment, for each frame image of the input moving image, a predetermined polygonal frame pattern in the color component that is difficult to perceive the image. In addition, a marker having a frame-like pattern in which the intensity of light intensity increases or decreases monotonously from the inside to the outside of the frame-like pattern is superimposed on a predetermined area, and an image of each frame of the moving image For example, human watermark information is superimposed on an image area defined in the positional relationship with the marker superimposed on the image, and an image in which the marker and the watermark information are superimposed on each frame of the moving image is output. Markers can be embedded in moving images so that digital watermarks can be detected accurately and quickly from images with projective transformation without being perceived by the eyesIn addition, a marker can be embedded in the video in a region overlapping with the digital watermark.

  In addition, according to the marker detection device of the embodiment of the present invention, the image of each frame of the input moving image is a polygonal frame-shaped pattern that is predetermined for each frame image of the moving image. In addition, a component that changes at a predetermined first time frequency used when a marker having a frame-like pattern whose intensity of intensity increases or decreases monotonously from the inside to the outside of the frame-like pattern is detected. A marker component image composed of a component representing a marker from an image using a filter for image processing, and for each frame image of a moving image, a marker component image of the separated image is included in a frame pattern of the marker component image. The edge component is searched for in each of the predetermined adjacent pixel directions from the specific pixel at the predetermined position so that For each of the edge component pixels searched in the direction, the edge component pixel is searched from the edge component pixel in a predetermined adjacent pixel direction, and the searched edge component pixels are searched. Polygons configured by extracting pixels as edge candidates and configuring each of the polygon candidates corresponding to a predetermined polygon by combining the extracted edge candidates with respect to each frame image of the moving image. Select one of each of the candidates, determine a marker superimposed on the image based on the selected polygon candidate, and based on the determined marker for each frame image of the moving image Then, projective transformation is performed on the image, and the image of each frame of the moving image is determined by the positional relationship with the marker of the image determined by the marker determination in the image obtained by the projective transformation. By detecting the electronic watermark information from the area, the marker to be used for converting a moving image projective transformation is performed to the original moving image can be detected from the high precision, moving images at high speed.

  Further, it is possible to provide a method for embedding and detecting a marker so that a digital watermark can be detected accurately and at high speed from an image accompanied by projective transformation without being perceived by human eyes.

  In addition, marker detection can be performed without stress even on a mobile terminal with low CPU performance such as a camera-equipped mobile phone.

  In addition, the marker embedding device and the marker detection device according to the present embodiment are not limited to the purpose of specifying a digital watermark embedding region, but are similar applications using image recognition (for example, Web related to an object photographed by a camera). (Services such as automatically accessing information) can also be used as a pre-recognition process. However, the target object is a planar object and is limited to a case where a marker can be embedded in advance. Of course, when such use is assumed, the digital watermark information superimposing unit, the projective transformation correcting unit, and the digital watermark information detecting unit are not necessary, and the output unit should output the marker position information on the embedded image.

  Further, the recognition module can improve the recognition accuracy by cutting out the object and correcting the geometric transformation based on the received marker position information.

  Further, based on the marker position information, the relative spatial positional relationship between the object and the camera can be calculated and used together with the recognition result.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, in the present embodiment, the case where the change in shading changes as shown in the right of FIG. 2 is described. However, the present invention is not limited to this, and the change monotonously increases or decreases from the inside to the outside of the marker. Anything is acceptable.

  In the present embodiment, the case where the marker pattern is the pattern shown in FIG. 2 has been described. However, the present invention is not limited to this. For example, as long as the marker pattern is a pattern that monotonously increases or decreases from the inside to the outside, the shape may be different from that of FIG.

  In the present embodiment, the case where the waveform of the temporally varying frequency used when embedding the marker and the digital watermark is a sine wave has been described. However, the present invention is not limited to this. For example, the waveform is a rectangular wave. Etc. Further, as long as the condition that “the time frequency of the marker and the digital watermark is different” is satisfied, either the marker or the digital watermark may be a direct current component (which does not give temporal variation).

  In the present embodiment, the case where the time-frequency filter unit separates the marker component image, the digital watermark component image, and the other component images excluding the marker component image and the digital watermark component image has been described. It is not limited to this. For example, in order to shorten the processing time, it is not necessary to separate only the marker component image and separate the digital watermark component image from the other component images. In that case, only the first frequency pass filter may be used.

  In the present embodiment, the case where the criterion for determining a true quadrangle is the number of edge direction inversion occurrences has been described, but the present invention is not limited to this. For example, any index can be used as long as it can distinguish the marker pattern from those that are not. Specifically, the evaluation value of Patent Document 2 may be used.

  In the present embodiment, the case where the method of Patent Document 6 is used as a method of tracking each quadrilateral candidate has been described, but the present invention is not limited to this. For example, other tracking methods may be used.

  In the present embodiment, a case has been described in which a quadrilateral candidate in which the number of edge direction inversion occurrences exceeds the threshold is detected as a marker, but the present invention is not limited to this. For example, if the number of occurrences of edge direction reversal does not exceed the threshold at the present time and the true quadrilateral has not yet been identified, the true quadrilateral is uniquely determined using the same evaluation value as in Patent Document 2. It may be determined.

  In the present embodiment, the marker is described as a quadrilateral, but the present invention is not limited to this. For example, the marker shape may be anything as long as it is a polygon. However, the number and direction of edge search scans to be performed first must be changed in accordance with the number of sides or the ratio of lengths between sides. Polygon candidates are also the types of marker-shaped polygons.

  In the present embodiment, the marker and digital watermark embedding positions have been described as being arranged in the entire target image. However, the present invention is not limited to this. For example, the digital watermark may be embedded in a specific partial area instead of being embedded in the entire image. In this case, the embedding position of the marker may be adjusted accordingly. For example, assuming that two persons appear in the target image, it is possible to embed two different types of digital watermarks in each face area. In this way, the user can be guided from one image content to an information site related to each person.

The present invention also provides a storage medium in which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or apparatus, and a program in which the CPU (MPU) of the system or apparatus is stored in the storage medium. This can also be realized by reading and executing the code. In that case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code, for example, CD-ROM, DVD-ROM, CD-R , CD-RW, MO, HDD and the like constitute the present invention.

DESCRIPTION OF SYMBOLS 10 Input part 20 Operation part 30 Marker superimposition part 40 Digital watermark information superimposition part 50 Embedded image output part 90 Output part 100 Marker embedding apparatus 200 Marker detection apparatus 210 Input part 220 Operation part 230 Time frequency filter part 240 Marker detection part 242 Candidate extraction unit 244 Polygon candidate configuration unit 246 Marker determination unit 250 Projection conversion correction unit 260 Digital watermark information detection unit 290 Output unit

Claims (10)

The image of each frame of the input moving image is a predetermined polygonal frame-like pattern in the color component that is difficult to perceive in the image, and is shaded from the inside to the outside of the frame-like pattern. A marker superimposing unit that superimposes a marker having a frame-like pattern whose intensity monotonously increases or decreases on a predetermined area;
A digital watermark information superimposing unit that superimposes digital watermark information on a region of the image determined in a positional relationship with the marker superimposed on the image by the marker superimposing unit for each frame image of the moving image;
An embedded image output unit that outputs an image in which the marker and the digital watermark information are superimposed on each frame of the moving image;
A marker embedding device comprising:
The marker embedding device, wherein the marker superimposing unit superimposes the marker by periodically changing the intensity of superimposition for each image at a predetermined first time frequency for each frame image of the moving image.   The digital watermark information superimposing unit periodically changes the superposition strength for each image at a predetermined second time frequency different from the first time frequency for the image of each frame of the moving image. The marker embedding apparatus according to claim 1, wherein the digital watermark information is superimposed. A frame that is a predetermined polygonal frame pattern for each frame image of the input moving image, and in which the intensity of light and shade increases monotonously from the inside to the outside of the frame pattern A marker component image composed of a component representing the marker is separated from the image by using a filter for detecting a component that changes at a predetermined first time frequency used when a marker having a pattern is superimposed. A time frequency filter section;
With respect to the image of each frame of the moving image, the specific pixel at a predetermined position so as to be within the frame-shaped pattern of the marker component image of the marker component image of the image separated by the time-frequency filter unit Thus, the edge component is searched for in each of the predetermined adjacent pixel directions, and for each of the edge component pixels searched in the adjacent pixel direction for each of the adjacent pixel directions, the edge component pixels An edge candidate extraction unit that searches for pixels of edge components that are continuous toward each of predetermined adjacent pixel directions, and extracts the searched pixels of edge components that are continuous as edge candidates;
A polygon candidate configuration unit that configures each of the polygon candidates corresponding to the predetermined polygon by combining each of the side candidates extracted by the side candidate extraction unit for each frame image of the moving image; ,
A marker determination unit that selects any one of the polygon candidates configured by the polygon candidate configuration unit and determines the marker superimposed on the image based on the selected polygon candidate When,
A projective transformation correcting unit that performs a projective transformation on the image based on the marker determined by the marker determining unit for each frame image of the moving image;
For each frame image of the moving image, digital watermark information is obtained from an area determined by a positional relationship with the marker of the image determined by the marker determining unit in an image obtained by projective conversion by the projective conversion correcting unit. A digital watermark information detection unit to detect;
A marker detection device.   The marker determination unit tracks the polygon candidates configured for each frame image of the moving image by parallel processing for each of the polygon candidates configured by the polygon candidate configuration unit, and Comparing the attributes of the rectangular candidates in chronological order, selecting any one of the polygon candidates configured by the polygon candidate configuration unit, and based on the selected polygon candidates, The marker detection apparatus according to claim 3, wherein the superimposed marker is determined.   The marker determination unit tracks the polygon candidates configured for each frame image of the moving image by parallel processing for each of the polygon candidates configured by the polygon candidate configuration unit, and Comparing the attributes of the rectangular candidates in chronological order, obtaining the number of occurrences of reversal of the edge direction of each of the edge component pixels included in the polygon candidate, and obtaining the number of the edges obtained for each of the polygon candidates Based on the number of occurrences of direction reversal, select any one of the polygon candidates configured by the polygon candidate configuration unit, and superimpose on the image based on the selected polygon candidates The marker detection apparatus according to claim 4, wherein the marker is determined. A marker embedding method in a marker embedding device including a marker superimposing unit, a digital watermark information superimposing unit, and an embedded image output unit,
The image of each frame of the moving image input by the marker superimposing unit is a predetermined polygonal frame-like pattern in the color component that is difficult to perceive of the image, and inside the frame-like pattern Superimposing a marker having a frame-like pattern in which the intensity of light intensity increases or decreases monotonously from the outside to a predetermined area;
A step of superimposing digital watermark information on an area of the image determined by a positional relationship with the marker superimposed on the image by the marker superimposing unit by the digital watermark information superimposing unit for each frame image of the moving image. When,
Outputting an image obtained by superimposing the marker and the digital watermark information on each frame of the moving image by the embedded image output unit;
A marker embedding method including:
The step of superimposing the marker by the marker superimposing unit periodically changes the intensity of superimposition for each image at a predetermined first time frequency for the image of each frame of the moving image. Marker embedding method to be superimposed. A marker detection method in a marker detection device including a time frequency filter unit, an edge candidate extraction unit, a polygon candidate configuration unit, a marker determination unit, a projective transformation correction unit, and a digital watermark information detection unit. ,
The time-frequency filter unit has a predetermined polygonal frame-like pattern for each frame image of the input moving image, and the intensity of light and shade is monotonous from the inside to the outside of the frame-like pattern. From a component representing the marker from the image, using a filter for detecting a component that changes at a predetermined first time frequency used when a marker having a frame pattern that increases or decreases is superimposed on Separating the marker component image,
With respect to the image of each frame of the moving image by the edge candidate extraction unit, the marker component image of the image separated by the time-frequency filter unit is preliminarily set to be within the frame-shaped pattern of the marker component image. An edge component is searched for in each of the predetermined adjacent pixel directions from a specific pixel at a predetermined position, and each of the edge component pixels searched in the adjacent pixel direction for each of the adjacent pixel directions. For the edge component pixels, searching for edge component pixels continuous in each of the predetermined adjacent pixel directions, and extracting the searched edge component pixels as edge candidates; and
Each polygon candidate corresponding to the predetermined polygon is obtained by combining each of the side candidates extracted by the side candidate extraction unit for each frame image of the moving image by the polygon candidate configuration unit. Configuring steps;
The marker determined by the marker determination unit selects any one of the polygon candidates configured by the polygon candidate configuration unit, and is superimposed on the image based on the selected polygon candidate A step of determining
Performing projective transformation on the image based on the marker determined by the marker determining unit for the image of each frame of the moving image by the projective conversion correcting unit;
The positional relationship between the image of each frame of the moving image by the digital watermark information detection unit and the marker of the image determined by the marker determination unit in the image obtained by the projective conversion by the projection conversion correction unit. Detecting digital watermark information from a defined area;
A marker detection method including:   The step of determining a marker by the marker determining unit includes the polygon candidate configured for the image of each frame of the moving image by parallel processing for each of the polygon candidates configured by the polygon candidate configuring unit. In contrast, the polygon candidate attributes are compared in time series, and one of each of the polygon candidates configured by the polygon candidate configuration unit is selected, and the selected polygon candidate is selected. The marker detection method according to claim 7, wherein the marker superimposed on the image is determined based on the marker.   The step of determining a marker by the marker determining unit includes the polygon candidate configured for the image of each frame of the moving image by parallel processing for each of the polygon candidates configured by the polygon candidate configuring unit. In contrast, the polygon candidate attributes are compared in time series, and the number of occurrences of reversal of the edge direction of each of the edge component pixels included in the polygon candidate is obtained, and obtained for each of the polygon candidates. Based on the generated number of occurrences of reversal of the edge direction, select one of each of the polygon candidates configured by the polygon candidate configuration unit, and based on the selected polygon candidate The marker detection method according to claim 8, wherein the marker superimposed on the image is determined.   The program for making a computer perform each step in the marker embedding method of Claim 6, or the marker detection method of any one of Claims 7-9.