CN114842346A

CN114842346A - Method, device and system for detecting and marking change of remote sensing image and storage medium

Info

Publication number: CN114842346A
Application number: CN202210575260.6A
Authority: CN
Inventors: 阳慧宇
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2022-05-24
Filing date: 2022-05-24
Publication date: 2022-08-02

Abstract

The embodiment of the invention discloses a method, a device and a system for detecting and marking changes of remote sensing images and a storage medium. The method comprises the following steps: acquiring two corresponding remote sensing images of a target scene at different time phases; determining a plurality of changed sub-areas between two remote sensing images and the outer contour corresponding to each sub-area; clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustering regions, wherein one clustering region comprises the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other; and according to the outline position information of each subarea included in the first clustering area, carrying out outline labeling on each subarea in the two remote sensing images, and controlling display equipment to display the images of the labeled parts of the two remote sensing images. The change detection labeling method of the remote sensing image solves the problem that the change detection labeling of the remote sensing image is inconvenient in the prior art.

Description

Method, device and system for detecting and marking change of remote sensing image and storage medium

Technical Field

The invention relates to the technical field of remote sensing image processing, in particular to a method, a device and a system for detecting and marking changes of remote sensing images and a storage medium.

Background

Remote sensing (remote sensing) refers to a non-contact and remote detection technology, and generally refers to the detection of electromagnetic radiation and reflection characteristics of a target object by using a sensor/remote sensor.

The remote sensing image has important application in the fields of natural resource detection, geological disaster early warning, landform change detection, urban planning and the like, the change detection marking of the remote sensing image has obvious effect in a remote sensing change task and determines the performance upper limit of a change detection model, however, the remote sensing change detection marking needs manual naked eyes to compare remote sensing data of two time phases, the areas with changes are compared and marked out by pixels, different application scenes need to be matched with different professional engineering personnel to analyze the remote sensing image, a great amount of manpower is consumed in the process, and the efficiency is not high.

Therefore, the problem that the change detection and labeling of the remote sensing image are inconvenient exists in the prior art. In view of the above problems, no effective solution has been proposed.

The above information disclosed in the background section is only for enhancement of understanding of the background of the technology described herein. The background art may therefore contain certain information that does not form the known prior art to those skilled in the art.

Disclosure of Invention

The embodiment of the invention provides a method, a device and a system for detecting and marking changes of remote sensing images and a storage medium, which at least solve the problem that the change detection and marking of the remote sensing images are inconvenient in the prior art.

According to an aspect of the embodiments of the present invention, there is provided a change detection labeling method for a remote sensing image, including: acquiring two corresponding remote sensing images of a target scene at different time phases; determining a plurality of changed sub-areas between two remote sensing images and the outer contour corresponding to each sub-area; clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustering regions, wherein one clustering region comprises the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other; and according to the outline position information of each subarea included in the first clustering area, carrying out outline labeling on each subarea in the two remote sensing images, and controlling display equipment to display the labeled part images of the two remote sensing images, wherein the first clustering area comprises one, more or all of the clustering areas.

Optionally, clustering outer contours of the plurality of sub-regions to obtain a plurality of clustered regions, including: determining a circumscribed horizontal rectangle corresponding to the outer contour of each sub-region; determining a first target external horizontal rectangle from the external horizontal rectangles, wherein the size of the first target external horizontal rectangle is larger than or equal to a first preset size; and determining a region corresponding to a horizontal rectangle circumscribed by the first target as a clustering region.

Optionally, clustering outer contours of a plurality of sub-regions to obtain a plurality of clustered regions, further comprising: determining a second target external horizontal rectangle from the external horizontal rectangles, wherein the size of the second target external horizontal rectangle is smaller than a first preset size; determining a plurality of target horizontal rectangles, wherein one target horizontal rectangle is arranged around a plurality of second target external horizontal rectangles; and determining a region corresponding to a target horizontal rectangle as a clustering region.

Optionally, determining a plurality of target horizontal rectangles, one target horizontal rectangle being arranged around a plurality of second target circumscribed horizontal rectangles, comprising: determining a plurality of reference points, wherein the reference points correspond to a plurality of second target external horizontal rectangles one to one; performing K-means clustering processing on the plurality of reference points, wherein the K value is traversed to the total number of the second target external horizontal rectangles from 1, so that the second target external horizontal rectangles corresponding to all the reference points belonging to each class are all contained in the range of corresponding second preset sizes; and determining each target horizontal rectangle as the minimum horizontal rectangle of the second target circumscribed horizontal rectangle corresponding to all the reference points containing the corresponding class.

Optionally, determining a plurality of sub-regions with changes between two remote sensing images and an outer contour corresponding to each sub-region includes: inputting the two remote sensing images into a change detection segmentation model obtained by pre-training to obtain a change detection binary image representing the difference between the two remote sensing images; and analyzing the change detection binary image by adopting a connected region analysis algorithm to obtain the outer contour of each connected region.

Optionally, after the outer contour labeling of each sub-region is performed in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region, and the display device is controlled to display the images of the labeled parts of the two remote sensing images, the method for detecting and labeling the change of the remote sensing images further includes: receiving contour modification information for the outer contour label, the contour modification information corresponding to at least one of: deleting at least one outer contour, adjusting at least one outer contour and newly adding an outer contour; and modifying the change detection binary image according to the contour modification information.

Optionally, acquiring two corresponding remote sensing images of the target scene at different time phases includes: receiving two remote sensing general graphs under different time phases; acquiring multiple pairs of corresponding characteristic points on the two remote sensing general graphs; aligning the two remote sensing general graphs according to the multiple pairs of feature points; and performing grid division on the two aligned remote sensing general graphs to obtain a plurality of remote sensing image pairs, wherein each remote sensing image pair comprises two remote sensing images in different time phases.

Optionally, according to the outline position information of each sub-region included in the first clustering region, performing outline labeling of each sub-region in the two remote sensing images, and controlling the display device to display an image of a labeled part of the two remote sensing images, including: in the process of controlling the display equipment to display the images of the marked parts of the two remote sensing images, controlling the mask processing of target areas of the two remote sensing images, wherein the target areas are areas in which a first clustering area is overlapped with sub-areas belonging to a second clustering area, and the first clustering area and the second clustering area are different clustering areas.

According to another aspect of the embodiments of the present invention, there is also provided a change detection labeling apparatus for remote sensing images, including: the acquisition unit is used for acquiring two corresponding remote sensing images of a target scene at different time phases; the determining unit is used for determining a plurality of changed sub-areas between the two remote sensing images and the outer contour corresponding to each sub-area; the clustering unit is used for clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustering regions, one clustering region comprises the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other; and the control unit is used for carrying out outline marking on each subarea in the two remote sensing images according to the outline position information of each subarea included in the first clustering area and controlling the display equipment to display the marked part images of the two remote sensing images, wherein the first clustering area comprises one, more or all of the clustering areas.

Optionally, the clustering unit includes: the first determining module is used for determining a circumscribed horizontal rectangle corresponding to the outer contour of each sub-region; the second determining module is used for determining a first target external horizontal rectangle from the external horizontal rectangles, wherein the size of the first target external horizontal rectangle is larger than or equal to a first preset size; the third determining module is used for determining that a region corresponding to a first target circumscribed horizontal rectangle is a clustering region; the clustering unit includes: the fourth determining module is used for determining a second target external horizontal rectangle from the external horizontal rectangles, wherein the size of the second target external horizontal rectangle is smaller than the first preset size; a fifth determining module, configured to determine a plurality of target horizontal rectangles, where one target horizontal rectangle is arranged around a plurality of second target circumscribed horizontal rectangles; a sixth determining module, configured to determine that a region corresponding to a target horizontal rectangle is a clustering region; the fifth determining module includes: the first determining submodule is used for determining a plurality of reference points, and the reference points correspond to a plurality of second target external horizontal rectangles one to one; the clustering submodule is used for carrying out K-means clustering processing on the plurality of reference points, and traversing the K value from 1 to the total number of the second target external horizontal rectangles so as to enable the second target external horizontal rectangles corresponding to all the reference points belonging to each class to be contained in the corresponding range of a second preset size; the second determining submodule is used for determining each target horizontal rectangle as the minimum horizontal rectangle of a second target external horizontal rectangle corresponding to all the reference points containing the corresponding class; the determination unit includes: the input module is used for inputting the two remote sensing images into a change detection segmentation model obtained by pre-training to obtain a change detection binary image representing the difference between the two remote sensing images; the analysis module is used for analyzing the change detection binary image by adopting a connected region analysis algorithm to obtain the outer contour of each connected region; the change detection labeling device of the remote sensing image further comprises: the receiving unit is used for receiving contour modification information aiming at the outer contour label after the outer contour label of each sub-region is carried out in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region and the display equipment is controlled to display the images of the labeled parts of the two remote sensing images, wherein the contour modification information corresponds to at least one of the following operations: deleting at least one outer contour, adjusting at least one outer contour and newly adding an outer contour; a modification unit for modifying the change detection binary image according to the contour modification information; the acquisition unit includes: the receiving module is used for receiving two remote sensing general diagrams under different time phases; the acquisition module is used for acquiring multiple pairs of corresponding characteristic points on the two remote sensing general graphs; the alignment module is used for aligning the two remote sensing general graphs according to the multiple pairs of feature points; the division module is used for carrying out grid division on the two aligned remote sensing general graphs to obtain a plurality of remote sensing image pairs, and each remote sensing image pair comprises two remote sensing images in different time phases; the clustering unit comprises a control module, and is used for controlling mask processing on target areas of the two remote sensing images in the process of controlling the display equipment to display the images of the marked parts of the two remote sensing images, wherein the target areas are areas where a first clustering area and a sub-area belonging to a second clustering area are overlapped, and the first clustering area and the second clustering area are different clustering areas.

The embodiment of the invention also provides a nonvolatile storage medium which comprises a stored program, wherein the equipment where the nonvolatile storage medium is located is controlled to execute the change detection labeling method of the remote sensing image when the program runs.

The embodiment of the invention also provides a processor, wherein the processor is used for running the program, and the change detection and annotation method of the remote sensing image is executed when the program runs.

The embodiment of the invention also provides a change detection and annotation device of the remote sensing image, which comprises a display device, a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the change detection and annotation method of the remote sensing image is realized when the processor executes the computer program.

The change detection labeling method of the remote sensing image in the embodiment of the invention comprises the following steps: acquiring two corresponding remote sensing images of a target scene at different time phases; determining a plurality of changed sub-areas between two remote sensing images and the outer contour corresponding to each sub-area; clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustering regions, wherein one clustering region comprises the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other; and according to the outline position information of each subarea included in the first clustering area, carrying out outline labeling on each subarea in the two remote sensing images, and controlling display equipment to display the labeled part images of the two remote sensing images, wherein the first clustering area comprises one, more or all of the clustering areas. After two corresponding remote sensing images (namely image pairs) of a target scene at different time phases are obtained, through comparative analysis, a changed sub-region between the two remote sensing images can be determined, the outer contours of the changed sub-regions can be determined, a plurality of clustering regions are formed by clustering the outer contours, each clustering region comprises at least one outer contour, the outer contour marking of each sub-region is carried out in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region, and the display equipment is controlled to display the marked part of the two remote sensing images. Therefore, the change detection and marking of the remote sensing image are realized, and the fragmented outer contour is clustered and then displayed in a centralized manner, so that the subsequent examination or modification of the marked content by an operator is facilitated, the examination efficiency of the marked content is improved, the omission risk of part of fragmented outer contour in the examination process is reduced, the marking processing speed is effectively improved, the labor cost is reduced, and the problem of inconvenience in change detection and marking of the remote sensing image in the prior art is solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

FIG. 1 is a schematic flow chart diagram of an alternative embodiment of a method for change detection labeling of remote sensing images in accordance with the present invention;

FIG. 2 is a schematic view of an alternative embodiment of a change detection annotation appliance for remote sensing images in accordance with the present invention;

FIG. 3 is a schematic diagram of a change detection binary image obtained by the method for change detection labeling of remote sensing images according to the present invention based on two remote sensing images in different time phases;

FIG. 4 is a schematic diagram of a change detection binary image obtained by the change detection labeling method of a remote sensing image according to the present invention;

FIG. 5 is a schematic diagram of determining circumscribed horizontal rectangles corresponding to each outline according to the method for detecting and labeling changes of remote sensing images of the present invention;

FIG. 6 is a schematic diagram of clustering a plurality of outer contours to obtain a plurality of clustering regions according to the change detection labeling method for remote sensing images of the present invention;

fig. 7 is a schematic diagram of the method for detecting and labeling changes in remote sensing images according to the present invention, which controls a display device to display a portion of two remote sensing images corresponding to one cluster region and to display a corresponding outer contour.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first", "second", and the like in the description and claims of the present invention and the accompanying drawings are used for distinguishing different objects, and are not used for limiting a specific order.

Fig. 1 is a method for detecting and labeling changes in remote sensing images according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:

step S102, acquiring two corresponding remote sensing images of a target scene at different time phases;

step S104, determining a plurality of changed sub-areas between two remote sensing images and the outer contour corresponding to each sub-area;

step S106, clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustering regions, wherein one clustering region comprises the outer contour of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other;

and S108, according to the outline position information of each sub-region included in the first clustering region, carrying out outline labeling of each sub-region in the two remote sensing images, and controlling display equipment to display the labeled part images of the two remote sensing images, wherein the first clustering region includes one, more or all of the plurality of clustering regions.

After two corresponding remote sensing images (namely image pairs) of a target scene at different time phases are obtained, by adopting the change detection labeling method of the remote sensing images, the changed sub-regions between the two remote sensing images can be determined through comparison and analysis, the outer contours of the changed sub-regions can be determined, a plurality of clustering regions are formed by clustering the outer contours, each clustering region comprises at least one outer contour, the outer contour labeling of each sub-region is carried out in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region, and the display equipment is controlled to display the images of the labeled parts of the two remote sensing images. Therefore, the change detection and marking of the remote sensing image are realized, and the fragmented outer contour is clustered and then displayed in a centralized manner, so that the subsequent examination or modification of the marked content by an operator is facilitated, the examination efficiency of the marked content is improved, the omission risk of part of fragmented outer contour in the examination process is reduced, the marking processing speed is effectively improved, the labor cost is reduced, and the problem of inconvenience in change detection and marking of the remote sensing image in the prior art is solved. Fig. 7 shows a schematic diagram of displaying the labeled parts of the two remote sensing images by the final display device, and fig. 7 shows the first clustering region and the labeled contour of the image pair, which represent the regions with changes in the identified image pair, so that the operator can effectively check or modify the labeled content conveniently, and the labeling processing speed is increased. Of course, the display device may display only two remote sensing images at a time, or may display a plurality of remote sensing images at a time. In the process of clustering a plurality of outer contours, the clustering basis can be in various forms, for example, clustering according to the distance, clustering according to the size of the outer contour, clustering according to the type of the image element corresponding to the outer contour, and the like.

In this embodiment, clustering outer contours of a plurality of sub-regions to obtain a plurality of clustered regions includes: determining a circumscribed horizontal rectangle corresponding to the outer contour of each sub-region, as shown in fig. 5; determining a first target external horizontal rectangle from the external horizontal rectangles, wherein the size of the first target external horizontal rectangle is larger than or equal to a first preset size; the regions corresponding to the horizontal rectangles circumscribing the first target are determined as a clustering region, such as

regions

1 and 2 in fig. 6.

In the process of clustering the external contours, the approximate size of each external contour can be evaluated more conveniently by circumscribing the horizontal rectangle of each external contour, if the size of the circumscribed horizontal rectangle of a single external contour is larger than or equal to a first preset size, the size of the external contour is also larger, at the moment, the circumscribed horizontal rectangle of the external contour is determined as a first target circumscribed horizontal rectangle, the corresponding area of the first target circumscribed horizontal rectangle is independently used as a clustering area, and then the parts of the two remote sensing images corresponding to the clustering area are independently labeled and displayed, so that a subsequent operator can conveniently check the change detection labeling result.

On this basis, carry out the clustering to the outline of a plurality of subregions, obtain a plurality of cluster regions, still include: determining a second target external horizontal rectangle from the external horizontal rectangles, wherein the size of the second target external horizontal rectangle is smaller than a first preset size; determining a plurality of target horizontal rectangles, wherein one target horizontal rectangle is arranged around a plurality of second target external horizontal rectangles; the region corresponding to one target horizontal rectangle is determined as a clustering region, for example, 3 regions and 4 regions in fig. 6.

The method comprises the steps that small rectangles with the size smaller than a first preset size are determined as second target external horizontal rectangles, in the process of clustering the second target external horizontal rectangles, a plurality of target horizontal rectangles are determined, each target horizontal rectangle surrounds a plurality of small rectangles (second target external horizontal rectangles), and the area corresponding to each target horizontal rectangle is independently used as a clustering area. Therefore, the display equipment is controlled to independently display the parts of the two remote sensing images corresponding to the clustering areas in the follow-up process, so that the outer contours of the small areas with changes are clustered and displayed in a concentrated mode, the small fragmented areas are displayed in a concentrated mode, and the follow-up inspection of the labeling results is facilitated.

In practical applications, the first predetermined size may be measured in various ways, such as area size, length size, width size, etc. In one embodiment, the length of the maximum side of the circumscribed horizontal rectangle is used for measurement, if the length of the maximum side of the circumscribed horizontal rectangle, i.e. max _ len, is greater than or equal to a predetermined value (e.g. 256 pixels), it indicates that the size of the circumscribed horizontal rectangle is greater than or equal to a first predetermined size, and then it is determined that the circumscribed horizontal rectangle is the first target circumscribed horizontal rectangle, and if the length of the maximum side of the circumscribed horizontal rectangle, i.e. max _ len, is less than the predetermined value (e.g. 256 pixels), it indicates that the size of the circumscribed horizontal rectangle is less than the first predetermined size, and then it is determined that the circumscribed horizontal rectangle is the second target circumscribed horizontal rectangle.

In this embodiment, determining a plurality of target horizontal rectangles, one target horizontal rectangle being disposed around a plurality of second target circumscribed horizontal rectangles, includes: determining a plurality of reference points, wherein the reference points correspond to a plurality of second target external horizontal rectangles one to one; performing K-means clustering processing on the plurality of reference points, wherein the K value is traversed to the total number of the second target external horizontal rectangles from 1, so that the second target external horizontal rectangles corresponding to all the reference points belonging to each class are all contained in the range of corresponding second preset sizes; and determining each target horizontal rectangle as the minimum horizontal rectangle of the second target circumscribed horizontal rectangle corresponding to all the reference points containing the corresponding class.

In this embodiment, a K-means clustering processing mode is adopted to perform clustering processing on external horizontal rectangles with a size smaller than a first preset size according to the distance between the external contours, specifically, for each external horizontal rectangle, a reference point is selected first, and the reference point can be flexibly selected, for example, the center of each external horizontal rectangle is selected as a respective reference point, in the K-means clustering process, the K value is traversed from 1 to the total number of the external horizontal rectangles of the second target, each reference point is clustered to the nearest clustering center until all small rectangles of each class can be completely included in the range (in the clipping graph) with the second preset size, and then clustering is finished. In the clustering process, the values of the K values are sequentially taken from small to large, clustering operation is continuously performed, and the finally obtained clustering result is that under the condition that the value of the K is minimum, all the second target external horizontal rectangles belonging to each class can be completely contained in the range of the second preset size, that is, under the condition that the clustering area is as small as possible, all the outer contours are contained, so that the remote sensing image is more efficiently and intensively displayed and marked at the changed positions, wherein the second preset size can be flexibly determined according to the actual situation, can be the same as the first preset size and can also be different from the first preset size, for example, the second preset size can be the size of 512 pixels × 512 pixels, and the specific selection of the first preset size and the second preset size can be from the size of the display area of the display device or based on the observation of the image by an operator (saving the amplification of the image by the operator, Reduction, etc.) and the like. For the small rectangles belonging to each class (second target circumscribed horizontal rectangles), a horizontal rectangle is determined which can contain all the small rectangles (second target circumscribed horizontal rectangles) under that class and which is guaranteed to be minimal in size, that is, the four sides of the horizontal rectangle are all overlapped with the sides of at least one small rectangle (second target circumscribed horizontal rectangle), which is the smallest circumscribed horizontal rectangle for the assembly of all the small rectangles (second target circumscribed horizontal rectangles) under that class. Under the condition that the obtained clustering area is ensured to be proper in size, the centralized display of the outer contour corresponding to each small rectangle (the second target external horizontal rectangle) can be more reasonably realized, and therefore the follow-up labeling inspection is facilitated.

There are many ways to determine the multiple sub-regions with changes between two remote sensing images and the outer contour corresponding to each sub-region, and in this embodiment, the step includes: inputting the two remote sensing images into a change detection segmentation model obtained by pre-training to obtain a change detection binary image representing the difference between the two remote sensing images; and analyzing the change detection binary image by adopting a connected region analysis algorithm to obtain the outer contour of each connected region. The two remote sensing images are input into a change detection segmentation model obtained through pre-training to obtain a change detection binary image representing the difference between the two remote sensing images, the binary image can clearly reflect the difference between the two remote sensing images, and then the change detection binary image is analyzed by adopting a connected region analysis algorithm to obtain the outer contour of at least one connected region, wherein the outer contour is the outer contour of a position with change. Fig. 3 is a schematic diagram of a change detection binary image obtained by the change detection labeling method of remote sensing images according to the present invention based on two remote sensing images in different time phases, and the obtained change detection binary image is shown in fig. 4. In the process of obtaining the change detection binary image by pre-dividing the two remote sensing images by adopting the change detection division model, the adjustment of the output result of the model can be realized by setting the confidence coefficient threshold value of the change detection division model, and the confidence coefficient can be set to be relatively low in order to ensure higher recall rate.

In this embodiment, after performing outline labeling of each sub-region in the two remote sensing images according to the outline position information of each sub-region included in the first clustering region, and controlling the display device to display the images of the labeled parts of the two remote sensing images, the method for detecting and labeling changes in the remote sensing images further includes: receiving contour modification information for the outer contour label, the contour modification information corresponding to at least one of: deleting at least one outer contour, adjusting at least one outer contour and newly adding an outer contour; and modifying the change detection binary image according to the contour modification information.

The contour modification information for the outer contour label may be information input by an operator, information generated by the electronic device in response to a modification operation of a user, or even information automatically generated by the electronic device based on an algorithm. Taking the contour modification information as an example input by an operator, after the display device displays the images of the labeled parts of the two remote sensing images, the operator can manually confirm whether the outer contour labeling of the corresponding object is accurate, and input the contour modification information under the condition that the accuracy does not reach the standard, and perform operations such as deletion, addition, adjustment and the like on the outer contour.

Specifically, two corresponding remote sensing images of the target scene in different time phases are obtained, including: receiving two remote sensing general graphs under different time phases; acquiring multiple pairs of corresponding characteristic points on the two remote sensing general graphs; aligning the two remote sensing general graphs according to the multiple pairs of feature points; and performing grid division on the two aligned remote sensing general graphs to obtain a plurality of remote sensing image pairs, wherein each remote sensing image pair comprises two remote sensing images in different time phases.

That is to say, after receiving two remote sensing general diagrams in different time phases, in order to facilitate the change detection labeling of the remote sensing general diagram with a large size, the two remote sensing general diagrams are aligned and gridded, wherein the alignment is to ensure the accuracy of the subsequent change detection judgment, the gridding divides the two remote sensing general diagrams into a plurality of small images (namely, remote sensing images), and after the gridding division, the remote sensing images belonging to the two time phases correspond to each other, so that the subsequent change detection and labeling are conveniently realized. In the process of aligning the two remote sensing general diagrams, a plurality of pairs of corresponding feature points on the two remote sensing general diagrams are obtained firstly, the feature points can be selected at will, the feature points can be used as positioning references, the two remote sensing general diagrams can be accurately aligned by using the pairs of feature points, and then grid division can be carried out according to actual conditions.

According to the outline position information of each subarea included in the first clustering area, carrying out outline labeling of each subarea in the two remote sensing images, and controlling display equipment to display the images of the labeled parts of the two remote sensing images, wherein the method comprises the following steps: in the process of controlling the display equipment to display the images of the marked parts of the two remote sensing images, the mask processing is controlled to be carried out on the target areas of the two remote sensing images, wherein the target areas are areas where the first clustering areas and the sub-areas belonging to the second clustering areas are overlapped, and the first clustering areas and the second clustering areas are different clustering areas. The target areas of the two remote sensing images are masked, so that the sub-areas of the overlapped areas can be hidden, the overlapped areas refer to the overlapped parts of the sub-areas of the second clustering area and the first clustering area, the situation that the sub-areas which are partially overlapped are repeatedly marked when different clustering areas are marked can be avoided, the simplicity of a marking process is ensured, repeated work is avoided, and the image change detection marking efficiency is favorably ensured.

Next, as shown in fig. 2, an embodiment of the present invention further provides a change detection labeling apparatus for a remote sensing image, including: the acquisition unit is used for acquiring two corresponding remote sensing images of a target scene at different time phases; the determining unit is used for determining a plurality of changed sub-areas between the two remote sensing images and the outer contour corresponding to each sub-area; the clustering unit is used for clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustering regions, one clustering region comprises the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other; and the control unit is used for carrying out outline marking on each subarea in the two remote sensing images according to the outline position information of each subarea included in the first clustering area and controlling the display equipment to display the marked part images of the two remote sensing images, wherein the first clustering area comprises one, more or all of the clustering areas. After the obtaining unit obtains two corresponding remote sensing images (namely image pairs) of a target scene at different time phases, the determining unit can determine sub-areas with changes between the two remote sensing images and the outlines of the sub-areas with changes through comparison and analysis, the clustering unit clusters the outlines to form a plurality of clustering areas, each clustering area comprises at least one outline, the control unit marks the outlines of the sub-areas in the two remote sensing images according to the outline position information of the sub-areas included in the first clustering area and controls the display device to display the marked part images of the two remote sensing images. Therefore, the change detection and marking of the remote sensing image are realized, and the fragmented outer contour is clustered and then displayed in a centralized manner, so that the subsequent examination or modification of the marked content by an operator is facilitated, the examination efficiency of the marked content is improved, the omission risk of part of fragmented outer contour in the examination process is reduced, the marking processing speed is effectively improved, the labor cost is reduced, and the problem of inconvenience in change detection and marking of the remote sensing image in the prior art is solved. Fig. 7 shows a schematic diagram of displaying the labeled parts of the two remote sensing images by the final display device, and fig. 7 shows the first clustering region and the labeled contour of the image pair, which represent the regions with changes in the identified image pair, so that the operator can effectively check or modify the labeled content conveniently, and the labeling processing speed is increased. Of course, the display device may display only two remote sensing images at a time, or may display a plurality of remote sensing images at a time. In the process of clustering a plurality of outer contours, the clustering basis can be in various forms, for example, clustering according to the distance, clustering according to the size of the outer contour, clustering according to the type of the image element corresponding to the outer contour, and the like.

In this embodiment, the clustering unit includes: a first determining module, configured to determine a circumscribed horizontal rectangle corresponding to an outer contour of each sub-region, as shown in fig. 5; the second determining module is used for determining a first target external horizontal rectangle from the external horizontal rectangles, wherein the size of the first target external horizontal rectangle is larger than or equal to a first preset size; a third determining module, configured to determine that a region corresponding to a first target circumscribed horizontal rectangle is a clustering region, for example,

regions

1 and 2 in fig. 6. In the process of clustering the outer contours, the first determining module can more conveniently evaluate the approximate size of each outer contour by externally connecting a horizontal rectangle to each outer contour, if the size of the externally connected horizontal rectangle of a single outer contour is larger than or equal to a first preset size, the size of the outer contour is larger, at the moment, the externally connected horizontal rectangle of the outer contour is determined as a first target externally connected horizontal rectangle by the second determining module, the corresponding area of the externally connected horizontal rectangle is independently used as a clustering area by the third determining module, and the parts of the two remote sensing images corresponding to the clustering area are independently marked and displayed, so that a follow-up operator can conveniently check the change detection marking result.

Specifically, the clustering unit further includes: the fourth determining module is used for determining a second target external horizontal rectangle from the external horizontal rectangles, wherein the size of the second target external horizontal rectangle is smaller than the first preset size; a fifth determining module, configured to determine a plurality of target horizontal rectangles, where one target horizontal rectangle is arranged around a plurality of second target circumscribed horizontal rectangles; and a sixth determining module, configured to determine that a region corresponding to a target horizontal rectangle is a clustering region, for example,

regions

3 and 4 in fig. 6. The method comprises the steps that for small rectangles of which the size of the external horizontal rectangle is smaller than a first preset size, a fourth determining module determines the external horizontal rectangles to be second target external horizontal rectangles, in the process of clustering the second target external horizontal rectangles, a fifth determining module determines a plurality of target horizontal rectangles, each target horizontal rectangle surrounds the small rectangles (the second target external horizontal rectangles), and a sixth determining module independently takes the area corresponding to each target horizontal rectangle as a clustering area. Therefore, the display equipment is controlled to independently display the parts of the two remote sensing images corresponding to the clustering areas in the follow-up process, so that the outer contours of the small areas with changes are clustered and displayed in a concentrated mode, the small fragmented areas are displayed in a concentrated mode, and the follow-up inspection of the labeling results is facilitated.

In practical applications, the first predetermined size may be measured in various ways, such as area size, length size, width size, etc. In one embodiment, the length of the maximum side of the circumscribed horizontal rectangle is used for measurement, if the length of the maximum side of the circumscribed horizontal rectangle, that is, max _ len, is greater than or equal to a preset value (for example, 256 pixels), it indicates that the size of the circumscribed horizontal rectangle is greater than or equal to a first preset size, and then it is determined that the circumscribed horizontal rectangle is the first target circumscribed horizontal rectangle, and if the length of the maximum side of the circumscribed horizontal rectangle, that is, max _ len, is less than the preset value (for example, 256 pixels), it indicates that the size of the circumscribed horizontal rectangle is less than the first preset size, and then it is determined that the circumscribed horizontal rectangle is the second target circumscribed horizontal rectangle.

In this embodiment, the fifth determining module includes: the first determining submodule is used for determining a plurality of reference points, and the reference points correspond to a plurality of second target external horizontal rectangles one to one; the clustering submodule is used for carrying out K-means clustering processing on the plurality of reference points, and traversing the K value from 1 to the total number of the second target external horizontal rectangles so as to enable the second target external horizontal rectangles corresponding to all the reference points belonging to each class to be contained in the corresponding range of a second preset size; and the second determining submodule is used for determining each target horizontal rectangle as the minimum horizontal rectangle of the second target external horizontal rectangle corresponding to all the reference points containing the corresponding class. In this embodiment, a K-means clustering processing mode is adopted to perform clustering processing on external horizontal rectangles with a size smaller than a first preset size according to the distance between the outer contours, specifically, for each external horizontal rectangle, the first determining submodule selects a reference point, which can be flexibly selected, for example, the center of each external horizontal rectangle is selected as a respective reference point, the clustering submodule traverses the K value from 1 to the total number of the second target external horizontal rectangles in the K-means clustering process, clusters each reference point to the nearest clustering center until all small rectangles of each class can be completely contained in the range of a second preset size (in the clipping diagram), and then ends clustering. In the clustering process, the values of the K values are sequentially taken from small to large, clustering operation is continuously performed, and the finally obtained clustering result is that under the condition that the value of the K is minimum, all the second target external horizontal rectangles belonging to each class can be completely contained in the range of the second preset size, that is, under the condition that the clustering area is as small as possible, all the outer contours are contained, so that the remote sensing image is more efficiently and intensively displayed and marked at the changed positions, wherein the second preset size can be flexibly determined according to the actual situation, can be the same as the first preset size and can also be different from the first preset size, for example, the second preset size can be the size of 512 pixels × 512 pixels, and the specific selection of the first preset size and the second preset size can be from the size of the display area of the display device or based on the observation of the image by an operator (saving the amplification of the image by the operator, Reduction, etc.) and the like. For the small rectangles belonging to each class (second target circumscribed horizontal rectangles), a horizontal rectangle is determined which can contain all the small rectangles (second target circumscribed horizontal rectangles) under that class and which is guaranteed to be minimal in size, that is, the four sides of the horizontal rectangle are all overlapped with the sides of at least one small rectangle (second target circumscribed horizontal rectangle), which is the smallest circumscribed horizontal rectangle for the assembly of all the small rectangles (second target circumscribed horizontal rectangles) under that class. Under the condition that the obtained clustering area is ensured to be proper in size, the centralized display of the outer contour corresponding to each small rectangle (the second target external horizontal rectangle) can be more reasonably realized, and therefore the follow-up labeling inspection is facilitated.

In specific implementation, the determining unit may have various forms, for example, the determining unit includes: the input module is used for inputting the two remote sensing images into a change detection segmentation model obtained by pre-training to obtain a change detection binary image representing the difference between the two remote sensing images; and the analysis module is used for analyzing the change detection binary image by adopting a connected region analysis algorithm to obtain the outer contour of each connected region. The input module inputs the two remote sensing images into a change detection segmentation model obtained by pre-training to obtain a change detection binary image representing the difference between the two remote sensing images, the binary image can definitely reflect the difference between the two remote sensing images, and the analysis module analyzes the change detection binary image by adopting a connected region analysis algorithm to obtain the outer contour of at least one connected region, wherein the outer contour is the outer contour of a position with change. Fig. 3 is a schematic diagram of a change detection binary image obtained by the change detection labeling method of remote sensing images according to the present invention based on two remote sensing images in different time phases, and the obtained change detection binary image is shown in fig. 4. In the process of obtaining the change detection binary image by pre-dividing the two remote sensing images by adopting the change detection division model, the adjustment of the output result of the model can be realized by setting the confidence coefficient threshold value of the change detection division model, and the confidence coefficient can be set to be relatively low in order to ensure higher recall rate.

The change detection labeling device of the remote sensing image further comprises: the receiving unit is used for receiving contour modification information aiming at the outer contour marking after the outer contour marking of each sub-region is carried out in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region and the display equipment is controlled to display the marked part images of the two remote sensing images, wherein the contour modification information corresponds to at least one of the following operations: deleting at least one outer contour, adjusting at least one outer contour and newly adding an outer contour; and the modification unit is used for modifying the change detection binary image according to the contour modification information. The contour modification information for the outer contour label may be information input by an operator, information generated by the electronic device in response to a modification operation of a user, or even information automatically generated by the electronic device based on an algorithm. Taking the contour modification information as an example input by an operator, after the display device displays the images of the labeled parts of the two remote sensing images, the operator can manually confirm whether the outer contour label of the corresponding object is accurate, and input the contour modification information under the condition that the accuracy does not reach the standard, and perform operations such as deletion, addition, adjustment and the like on the outer contour, after the receiving unit receives the contour modification information, the modification unit can modify the change detection binary image according to the corresponding contour modification information, and certainly, the change detection binary image needs to be stored after modification.

In this embodiment, the obtaining unit includes: the receiving module is used for receiving two remote sensing general diagrams under different time phases; the acquisition module is used for acquiring multiple pairs of corresponding characteristic points on the two remote sensing general graphs; the alignment module is used for aligning the two remote sensing general graphs according to the multiple pairs of feature points; and the division module is used for carrying out grid division on the two aligned remote sensing general graphs to obtain a plurality of remote sensing image pairs, and each remote sensing image pair comprises two remote sensing images in different time phases. That is to say, after receiving two remote sensing general diagrams in different time phases, in order to facilitate the change detection labeling of the remote sensing general diagram with a large size, the two remote sensing general diagrams are aligned and gridded, wherein the alignment is to ensure the accuracy of the subsequent change detection judgment, the gridding divides the two remote sensing general diagrams into a plurality of small images (namely, remote sensing images), and after the gridding division, the remote sensing images belonging to the two time phases correspond to each other, so that the subsequent change detection and labeling are conveniently realized. In the process of aligning the two remote sensing general diagrams, a plurality of pairs of corresponding feature points on the two remote sensing general diagrams are obtained firstly, the feature points can be selected at will, the feature points can be used as positioning references, the two remote sensing general diagrams can be accurately aligned by using the pairs of feature points, and then grid division can be carried out according to actual conditions.

The clustering unit further comprises a control module, wherein the control module is used for controlling mask processing to be carried out on target areas of the two remote sensing images in the process of controlling the display equipment to display the images of the marked parts of the two remote sensing images, the target areas are areas where the first clustering areas and the sub-areas belonging to the second clustering areas are overlapped, and the first clustering areas and the second clustering areas are different clustering areas. The target areas of the two remote sensing images are masked, so that the sub-areas of the overlapped areas can be hidden, the overlapped areas refer to the overlapped parts of the sub-areas of the second clustering area and the first clustering area, the situation that the sub-areas which are partially overlapped are repeatedly marked when different clustering areas are marked can be avoided, the simplicity of a marking process is ensured, repeated work is avoided, and the image change detection marking efficiency is favorably ensured.

The embodiment of the invention further provides a processor, wherein the processor is used for running the program, and the change detection and annotation method for the remote sensing image is executed when the program runs.

Finally, the embodiment of the invention also provides a change detection and annotation device for the remote sensing image, which comprises a display device, a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the change detection and annotation method for the remote sensing image is realized when the processor executes the computer program. The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.

Claims

1. A change detection labeling method for a remote sensing image is characterized by comprising the following steps:

acquiring two corresponding remote sensing images of a target scene at different time phases;

determining a plurality of sub-areas with changes between the two remote sensing images and the outer contour corresponding to each sub-area;

clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustering regions, wherein one clustering region comprises the outer contour of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other;

and according to the outer contour position information of each sub-region included in a first clustering region, carrying out outer contour labeling on each sub-region in the two remote sensing images, and controlling display equipment to display the labeled part images of the two remote sensing images, wherein the first clustering region includes one, more or all of the clustering regions.

2. The method for detecting and labeling changes in remote sensing images according to claim 1, wherein clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustered regions comprises:

determining a circumscribed horizontal rectangle corresponding to the outer contour of each sub-region;

determining a first target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, wherein the size of the first target circumscribed horizontal rectangle is larger than or equal to a first preset size;

and determining a region corresponding to the first target circumscribed horizontal rectangle as the clustering region.

3. The method for detecting and labeling changes in remote sensing images according to claim 2, wherein the outer contours of the plurality of sub-regions are clustered to obtain a plurality of clustered regions, and further comprising:

determining a second target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, wherein the size of the second target circumscribed horizontal rectangle is smaller than the first preset size;

determining a plurality of target horizontal rectangles, one of the target horizontal rectangles being disposed around a plurality of the second target circumscribing horizontal rectangles;

and determining a region corresponding to one target horizontal rectangle as one clustering region.

4. The method for change detection and annotation of remote sensing images according to claim 3, wherein determining a plurality of target horizontal rectangles, one of which is disposed around a plurality of second target circumscribed horizontal rectangles, comprises:

determining a plurality of reference points, wherein the plurality of reference points are in one-to-one correspondence with a plurality of horizontal rectangles circumscribed to the second target;

performing K-means clustering processing on the plurality of reference points, wherein K values traverse from 1 to the total number of the second target external horizontal rectangles, so that the second target external horizontal rectangles corresponding to all the reference points belonging to each class are all contained in the range of corresponding second preset sizes;

and determining each target horizontal rectangle as the minimum horizontal rectangle of the second target circumscribed horizontal rectangles corresponding to all the reference points containing the corresponding class.

5. The method for detecting and labeling changes in remote sensing images according to claim 1, wherein determining a plurality of subregions where changes exist between two remote sensing images and an outer contour corresponding to each subregion comprises:

inputting the two remote sensing images into a change detection segmentation model obtained by pre-training to obtain a change detection binary image representing the difference between the two remote sensing images;

and analyzing the change detection binary image by adopting a connected region analysis algorithm to obtain the outer contour of each connected region.

6. The method for detecting and labeling changes in remote sensing images according to claim 5, wherein after labeling the outer contours of the sub-regions in the two remote sensing images according to the outer contour position information of the sub-regions included in the first clustering region and controlling the display device to display the images of the labeled parts of the two remote sensing images, the method for detecting and labeling changes in remote sensing images further comprises:

receiving contour modification information for an outer contour label, the contour modification information corresponding to at least one of: deleting at least one outer contour, adjusting at least one outer contour and increasing the outer contour;

and modifying the change detection binary image according to the contour modification information.

7. The method for detecting and labeling changes in remote sensing images according to any of claims 1 to 6, wherein obtaining two corresponding remote sensing images of a target scene at different time phases comprises:

receiving two remote sensing general graphs under different time phases;

acquiring multiple pairs of corresponding characteristic points on the two remote sensing general graphs;

aligning the two remote sensing general graphs according to the plurality of pairs of feature points;

and performing grid division on the two aligned remote sensing general graphs to obtain a plurality of remote sensing image pairs, wherein each remote sensing image pair comprises two remote sensing images in different time phases.

8. The method for detecting and labeling changes in remote sensing images according to any one of claims 1 to 6, wherein the steps of labeling the outer contour of each sub-region in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region, and controlling a display device to display the image of the labeled part of the two remote sensing images comprise:

in the process of controlling the display device to display the images of the labeled parts of the two remote sensing images, controlling mask processing on target areas of the two remote sensing images, wherein the target areas are areas where the first clustering areas and the sub-areas belonging to the second clustering areas are overlapped, and the first clustering areas and the second clustering areas are different clustering areas.

9. A change detection labeling device for a remote sensing image is characterized by comprising:

the acquisition unit is used for acquiring two corresponding remote sensing images of a target scene at different time phases;

the determining unit is used for determining a plurality of changed sub-areas between the two remote sensing images and the outer contour corresponding to each sub-area;

the clustering unit is used for clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustering regions, one clustering region comprises the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other;

and the control unit is used for marking the outer contour of each sub-region in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region and controlling display equipment to display the marked part images of the two remote sensing images, wherein the first clustering region includes one, more or all of the clustering regions.

10. The apparatus for detecting and labeling changes in remote sensing images according to claim 9,

the clustering unit includes: the first determining module is used for determining a circumscribed horizontal rectangle corresponding to the outer contour of each sub-region; the second determining module is used for determining a first target external horizontal rectangle from the external horizontal rectangles, wherein the size of the first target external horizontal rectangle is larger than or equal to a first preset size; a third determining module, configured to determine that a region corresponding to a horizontal rectangle circumscribed by the first target is a clustering region;

the clustering unit includes: a fourth determining module, configured to determine a second target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, where a size of the second target circumscribed horizontal rectangle is smaller than the first preset size; a fifth determining module, configured to determine a plurality of target horizontal rectangles, where one target horizontal rectangle is disposed around a plurality of second target circumscribed horizontal rectangles; a sixth determining module, configured to determine a region corresponding to one of the target horizontal rectangles as one of the clustering regions;

the fifth determining module includes: the first determining submodule is used for determining a plurality of reference points, and the plurality of reference points are in one-to-one correspondence with a plurality of second target external horizontal rectangles; the clustering submodule is used for carrying out K-means clustering processing on the plurality of reference points, and traversing K values from 1 to the total number of the second target external horizontal rectangles so as to enable the second target external horizontal rectangles corresponding to all the reference points belonging to each class to be contained in the range of corresponding second preset sizes; a second determining submodule, configured to determine each target horizontal rectangle as a minimum horizontal rectangle of the second target external horizontal rectangles corresponding to all the reference points including the corresponding class;

the determination unit includes: the input module is used for inputting the two remote sensing images into a change detection segmentation model obtained by pre-training to obtain a change detection binary image representing the difference between the two remote sensing images; the analysis module is used for analyzing the change detection binary image by adopting a connected region analysis algorithm to obtain the outer contour of each connected region;

the change detection labeling device for the remote sensing image further comprises: a receiving unit, configured to perform outline labeling of each sub-region in the two remote sensing images according to outline position information of each sub-region included in the first clustering region, and receive outline modification information for the outline labeling after controlling a display device to display an image of a labeled part of the two remote sensing images, where the outline modification information corresponds to at least one of the following operations: deleting at least one outer contour, adjusting at least one outer contour and increasing the outer contour; a modification unit for modifying the change detection binary image according to the contour modification information;

the acquisition unit includes: the receiving module is used for receiving two remote sensing general diagrams under different time phases; the acquisition module is used for acquiring multiple pairs of corresponding characteristic points on the two remote sensing general diagrams; the alignment module is used for aligning the two remote sensing general graphs according to the plurality of pairs of feature points; the dividing module is used for carrying out grid division on the two aligned remote sensing general graphs to obtain a plurality of remote sensing image pairs, and each remote sensing image pair comprises two remote sensing images in different time phases;

the clustering unit comprises a control module, and is used for controlling mask processing on target areas of the two remote sensing images in the process of controlling the display equipment to display the images of the marked parts of the two remote sensing images, wherein the target areas are areas where the first clustering areas and the sub-areas belonging to the second clustering areas are overlapped, and the first clustering areas and the second clustering areas are different clustering areas.

11. A non-volatile storage medium, characterized in that the non-volatile storage medium comprises a stored program, wherein when the program runs, the non-volatile storage medium is controlled to execute the method for detecting and labeling changes in remote sensing images according to any one of claims 1 to 8.

12. A processor, characterized in that the processor is configured to run a program, wherein the program is configured to execute the method for change detection and annotation of remote sensing images according to any one of claims 1 to 8 when running.

13. A change detection and annotation device for remote sensing images, comprising a display device, a memory, a processor and a computer program stored in the memory and operable on the processor, characterized in that the processor implements the change detection and annotation method for remote sensing images according to any one of claims 1 to 8 when executing the computer program.