JP7261026B2 - Deterioration level determination device, deterioration level determination method, and program - Google Patents

Description

TECHNICAL FIELD The present invention relates to a deterioration level determination device and the like for determining the deterioration level of a building.

2. Description of the Related Art Conventionally, there has been a device that assists in intuitively grasping the state of deterioration of members that constitute a building at the site of a patrol inspection or the like (see, for example, Patent Literature 1).

Japanese Patent No. 5469764

However, the above-described conventional technology cannot automatically determine the deterioration level of a building.

The deterioration level determination device of the first invention comprises a learning information storage unit storing learning information acquired from two or more teacher data corresponding to teacher images of deteriorated parts of a building and deterioration levels. , a determination target image acquisition unit that acquires a determination target image that is an image of a building to be determined, a deterioration level acquisition unit that applies the determination target image to learning information and acquires a deterioration level, and an output that outputs the deterioration level and a deterioration level determination device.

With such a configuration, the deterioration level of the building can be automatically determined.

In addition, the deterioration level determination device of the second invention, in contrast to the first invention, further comprises a teacher data storage unit for storing two or more teacher data, and the deterioration level acquisition unit includes two or more each a teacher feature quantity group acquiring means for acquiring, for each teacher image, a teacher feature quantity group that is a set of two or more feature quantities from at least a part of the teacher image; and two or more teacher features acquired by the teacher feature quantity group acquiring means. learning information acquiring means for acquiring learning information relating to correspondence between two or more feature quantities and deterioration levels using the quantity group and storing the learning information in a learning information storage; Determination target feature quantity group acquisition means for acquiring a determination target feature quantity group that is a set of; and deterioration level acquisition means for applying the determination target feature quantity group to learning information and acquiring a deterioration level corresponding to the determination target feature quantity group and a deterioration level determination device.

With such a configuration, the deterioration level of the building can be automatically determined by learning.

In the deterioration level acquisition unit as described above, the learning unit is configured by the teacher feature group acquisition unit and the learning information acquisition unit, and the automatic determination unit is configured by the determination target feature group acquisition unit and the deterioration level acquisition unit. may be Further, the deterioration level determination unit may have only an automatic determination unit without the learning unit. In that case, the deterioration level of the building can be automatically determined by prior learning.

Further, in the deterioration level determination device of the third invention, in contrast to the second invention, the learning information acquisition means uses two or more teacher feature amount groups acquired by the teacher feature amount group acquisition means to perform machine learning 2 or more feature values are input and the learning information is a learning device that outputs a deterioration level, and the deterioration level acquisition means inputs the determination target feature value group to the learning device, and from the learning device It is a deterioration level determination device that acquires an output deterioration level.

With such a configuration, the deterioration level of a building can be automatically determined by machine learning using a learner.

Further, in contrast to the third invention, the deterioration level determination apparatus of the fourth invention includes determination result data that is correspondence between the determination target image and the deterioration level, and user evaluation that is the user's evaluation of the determination target image. and a determination target image constituting one or more determination result data stored in the set information storage unit satisfies a predetermined first condition. a first judging unit for judging whether a By accumulating the received user evaluation in association with the determination result data composed of the determination target image, the determination result data including the determination target image satisfying the first condition and the determination target Reconfigure a learning device by a machine learning algorithm using a second accumulation unit that configures one or more sets of information including user evaluations of images and one or more sets of information stored in the set information storage unit. The deterioration level determination device further includes a reconstruction unit.

With this configuration, by reconfiguring the learning device using the user evaluation of the determination target image that satisfies the first condition, the deterioration level of the building corresponding to the determination target image that satisfies the first condition can be automatically determined with high accuracy. become.

Further, the deterioration level determination device of the fifth invention, in contrast to the fourth invention, includes a second determination unit that determines whether or not the user evaluation received by the reception unit satisfies a predetermined second condition. Further, the reconstruction unit is a deterioration level determination device that reconstructs the learner when the user evaluation satisfies the second condition.

With such a configuration, when the user evaluation of the determination target image satisfying the first condition is the second condition, the user evaluation is used to reconfigure the learning device, so that the determination target image that satisfies the first condition can be handled. It will be possible to automatically determine the deterioration level of buildings with higher accuracy.

Further, in the deterioration level determination device of the sixth invention, in contrast to the fourth or fifth invention, the first condition is that the deterioration level associated with the image to be determined is equal to or less than a threshold. It is a certain deterioration level determination device.

With such a configuration, by reconfiguring the learning device using the user's evaluation of the determination target image with the low deterioration level, it is possible to automatically determine the deterioration level of the building with the low deterioration level with high accuracy.

Further, in the deterioration level determination device of the seventh invention, for any one of the fourth to sixth inventions, the second condition is that the group information in which the deterioration level and the user evaluation are different is greater than or equal to the threshold value It is a degradation level determination device under the condition that most of them are stored in the set information storage unit.

With such a configuration, when there are many user evaluations of a determination target image with a low deterioration level, and there are many user evaluations different from the deterioration level, the learning device is reconfigured using the user evaluations, thereby determining the deterioration level of the building. Automatic judgment can be performed with higher accuracy.

Further, in the deterioration level determination device of the eighth invention, in any one of the first to seventh inventions, the deterioration level acquisition unit acquires two or more blocks that are part of the determination target image, applying each of the two or more blocks to the learning information to obtain two or more local impairment levels, which are the impairment levels for each block; A deterioration level determination device for acquiring a global deterioration level.

With such a configuration, efficient automatic determination can be performed by combining local determination and global determination.

Further, in the deterioration level determination device of the ninth invention, in contrast to the eighth invention, the deterioration level acquisition unit acquires two or more local deterioration levels by a machine learning algorithm, and calculates the global deterioration level by machine. It is a deterioration level determination device obtained by a learning algorithm.

With such a configuration, both local determination and global determination can be automatically performed by machine learning.

Further, in the deterioration level determination device of the tenth invention, in contrast to the eighth invention, the deterioration level acquisition unit uses two or more local deterioration levels to correspond to the deterioration location in the image to be determined. This is a degradation level determination device that acquires the size of a degraded region, which is an area, and uses the size of the degraded region to acquire a global degradation level.

With such a configuration, the size of the degraded region is obtained using the result of local determination, and the size is used for global determination, so that more efficient automatic determination can be performed.

Further, in the deterioration level determination device of the eleventh invention, in contrast to the tenth invention, the deterioration level acquisition unit acquires the size of the object in the teacher image, and the size of the object and the size of the object in the object This is a degradation level determination device that acquires a global degradation level using the size of a degradation region.

With such a configuration, accurate automatic determination can be performed by using the relative size of the degraded area with respect to the object.

Further, in the deterioration level determination device of the twelfth invention, in any one of the first to eleventh inventions, the deterioration cause of the determination target is determined in advance by using the determination target image. The deterioration level determination device further includes a cause determination unit that determines which of the deterioration causes.

With such a configuration, the cause of deterioration can also be automatically determined.

Further, in the deterioration level determination device of the thirteenth invention, for any one of the first to twelfth inventions, the deterioration level acquisition unit selects the predetermined two or more algorithms, It is a deterioration level determination device that acquires a deterioration level by an algorithm according to the degree of deterioration.

With such a configuration, it is possible to accurately determine the level according to the degree of deterioration of the building.

Further, in the deterioration level determination device of the fourteenth invention, in contrast to the thirteenth invention, the two or more algorithms are a first algorithm that acquires continuous values by a regression equation and a discrete value that is acquired by class classification. The deterioration level obtaining unit obtains a first deterioration level that is a discrete value by the second algorithm, and if the first deterioration level is equal to or less than the threshold value or lower than the threshold value, the first algorithm is a continuous value further obtaining a certain second deterioration level, adopting the deterioration level corresponding to the second deterioration level, and if the first deterioration level is higher than or equal to or greater than the threshold, adopting the deterioration level corresponding to the first deterioration level This is the deterioration level determination device to be adopted.

With this configuration, when the deterioration of the building is light, detailed level judgment can be performed by the first algorithm based on the regression formula, and when the deterioration of the building is severe, the second algorithm by class classification enables various classifications. can do

According to the present invention, the deterioration level of a building can be automatically determined.

Block diagram of a deterioration level determination device according to an embodiment Flowchart explaining the operation of the deterioration level determination device Data structure diagram of training data A diagram showing an example of a set of pairs of a teacher feature quantity group and a deterioration level A diagram showing an example of a set of pairs of determination target feature quantity groups and deterioration levels Data structure diagram of group information External view of computer system A diagram showing an example of the internal configuration of a computer system

Embodiments of a deterioration level determination device and the like will be described below with reference to the drawings. It should be noted that, since components denoted by the same reference numerals in the embodiments perform similar operations, repetitive description may be omitted.

The deterioration level determination device is usually a server, for example, a server of an organization such as a company or an organization that conducts business related to railways. Businesses related to railways include, for example, transportation of passengers and freight by railways, maintenance and management of railways and associated equipment, etc., but the type of business does not matter. Alternatively, the deterioration level determination device may be, for example, a cloud server or a mobile terminal carried by an employee of the organization, regardless of its type or location. Note that the mobile terminal is, for example, a tablet terminal, a smart phone, a mobile phone, a notebook PC, etc., but the type of mobile terminal does not matter.

Alternatively, the deterioration level determination device may be a server or the like of an organization that conducts business other than railway-related work, and the organization to which it belongs does not matter. Non-railway-related business includes, for example, ship-related business, bus-related business, and other non-railway vehicle-related business, as well as office buildings and condominiums that are not directly related to railways. It may be the maintenance and management of things, and the type is not limited.

FIG. 1 is a block diagram of a deterioration level determination device 1 according to this embodiment. The deterioration level determination device 1 includes a storage unit 11 , a reception unit 12 , a processing unit 13 and an output unit 14 . The storage unit 11 includes a learning information storage unit 112 , a teacher data storage unit 111 and a group information storage unit 113 . The processing unit 13 includes a determination target image acquisition unit 131, a deterioration level acquisition unit 132, a first storage unit 133, a first determination unit 134, a second storage unit 135, a second determination unit 136, a reconstruction unit 137, and a cause determination. A portion 138 is provided. The deterioration level acquisition unit 132 includes teacher feature quantity group acquisition means 1321 , learning information acquisition means 1322 , determination target feature quantity group acquisition means 1323 , and degradation level acquisition means 1324 .

The storage unit 11 can store various types of information. The various types of information are, for example, teacher data, which will be described later, learning information, which will be described later, and group information, which will be described later.

The storage unit 11 also normally stores a determination target image. A determination target image is a photographed image of a building to be determined by the deterioration level determination device 1 . The determination target image may be part of group information described later. However, the determination target image does not necessarily have to be stored in the storage unit 11 .

A building is something that is built. A building is typically a building. The building in this embodiment is, for example, a building related to railways. Railway-related buildings are, for example, station buildings. The station building is, for example, a station building, a platform, an overpass, etc., but the type of the building does not matter. Alternatively, the railway-related building may be a building built in a place other than a station. Buildings of this type are, for example, depots for storing trains, substation facilities, etc., but any building related to railways may be used.

Alternatively, the building may be a building associated with non-railway vehicles. Vehicles other than railroads are, for example, ships, buses, etc., but the type of vehicle does not matter. Buildings of this type are, for example, harbor facilities, bus terminals, etc., but the type of building does not matter. It should be noted that the building may be considered to include, for example, vehicles such as railroads and ships.

Alternatively, the building may be a building not directly related to railways. Buildings of this kind are, for example, dedicated bridges for one or more of vehicles or pedestrians, office buildings or apartments, factories, and the like. However, buildings that are not directly related to railways may be excluded from the buildings in this embodiment.

Also, the building may be the whole building or the like, or a part thereof. A part of a building or the like is various members that constitute the building or the like. The various members are, for example, pillars, beams, etc., which constitute the station building, and in particular, metal members such as iron, which are easily rusted or corroded.

Two or more pieces of teacher data are stored in the teacher data storage unit 111 . Teacher data is, for example, data used as a teacher when performing machine learning. Machine learning is a technique that enables computers to perform the same functions as human learning. Machine learning is, for example, supervised learning, but may be unsupervised learning. Note that hereinafter, machine learning may be simply referred to as learning.

The training data in this embodiment is information indicating the correspondence between the training image and the deterioration level. A teacher image is an image used as a teacher when learning the correspondence between the degree of deterioration of a building and the deterioration level. A teaching image is an image of a building captured by a camera. The image is usually a still image, but may be two or more still images that constitute a moving image.

The teacher image has a size of, for example, 800 horizontal pixels by 600 vertical pixels (hereinafter sometimes written as “800×600”). A pixel is an element that constitutes an image, and has information such as RGB and YUV, for example. However, the size of the teacher image may be any size such as 1200×900 or 1600×900. In addition, the representation format of the information possessed by the pixels does not matter. This matter also applies to a determination target image, which will be described later.

Degradation is the deterioration of quality or deterioration of performance over time. Deterioration is, for example, the generation of rust in a building (rusting). Alternatively, the deterioration may be cracks, deformation, or the like in the building, regardless of the type. It should be noted that the deterioration does not necessarily have to be due to secular change, and may include, for example, damage due to natural disasters such as earthquakes and accidents. The degree and speed of deterioration differ, for example, depending on the material of the building, and even buildings made of the same material change depending on the environment, usage conditions, and the like.

The deterioration level is information that numerically expresses the degree of deterioration of the building (hereinafter sometimes referred to as the degree of deterioration) by computer processing. That is, the degree of deterioration in the present embodiment is a numerical value based on human judgment, whereas the deterioration level is a numerical value based on computer processing. The deterioration level may be, for example, discrete numerical values such as "1", "2", and "3" (hereinafter referred to as discrete values), or fractional numbers such as "2.2" and "2.45". A continuous numerical value (hereinafter referred to as a continuous value) having also may be used. The deterioration level in the present embodiment is, for example, a numerical value ranging from 0 to 5, and the larger the numerical value, the greater the degree of deterioration of the building. However, the deterioration level may be, for example, a numerical value in the range of 1 to 10, and the smaller the numerical value, the greater the degree of deterioration of the building. For example, the deterioration level is not limited to numerical values, and may be represented by letters such as "A", "B", and "C".

The training data is composed of pairs of training images and deterioration levels, for example. A teacher image that constitutes the teacher data is an image obtained by photographing a deteriorated portion of the building as described above. The deterioration level that constitutes the teacher data is, for example, a numerical value based on the results of human judgment, but may be a numerical value acquired by machine learning. However, the data structure of the training data does not matter. Any method of acquiring the deterioration level that constitutes the teacher data is also available.

It should be noted that, for example, a deterioration distribution may accompany the deterioration level that constitutes the teacher data. The deterioration distribution is information about distribution of deterioration. The degradation distribution is, for example, global degradation, local degradation, or the like. Global deterioration is deterioration over the entire building, and local deterioration is deterioration of only a part of the building. The deterioration distribution is represented by letters and symbols other than numbers, such as "P" corresponding to local deterioration and "G" corresponding to global deterioration, but the expression format is not limited.

The deterioration level with the deterioration distribution is represented by a numerical value with a subscript, such as "2P" (level 2 local deterioration), "4G" (level 4 local deterioration), and the like. Alternatively, the suffix “P” may be added only to the local deterioration, and no suffix may be added to the overall deterioration, and the expression format is not limited.

However, in the present embodiment, such teacher data as described above is not essential. For example, if learning information is stored in advance in the learning information storage unit 112 and learning is not performed in the deterioration level determination device 1, teacher data is unnecessary. Further, even when unsupervised learning is performed in the deterioration level determination device 1, no teacher data is required. Therefore, the deterioration level determination device 1 does not necessarily have to include the teacher data storage unit 111 .

Learning information is stored in the learning information storage unit 112 . Learning information is information acquired by learning. The stored learning information is information indicating correspondence between two or more feature amounts and deterioration levels. The learning information is usually information acquired by the learning information acquiring means 1322 described later by performing machine learning using two or more teacher data stored in the teacher data storage unit 111 . However, part or all of the learning information may be information obtained by using the results of learning by a person performing calculations or analyses, for example. Alternatively, the learned information may be information acquired by machine learning or the like in an external device, and its origin is not limited.

It can be said that the feature amount is information that quantitatively indicates various features related to the deteriorated region. A degraded region is a region corresponding to a degraded portion of a building in the teacher image or determination target image.

Various features related to the degraded area are, for example, color features, brightness features, and the like. The color feature is, for example, the type of color, but may also be the rate of color change. The brightness feature is, for example, the rate of change of brightness. This type of feature may be, for example, a set of pixel values corresponding to pixels in the degraded region, or a statistic of such pixel values. The statistic is, for example, an average value, but may be a median value, a mode value, or a variance.

Note that the type of color is expressed using hue, for example. Hue is information indicating the positions of various colors on the color wheel. Hue is a numerical value in the range of 0 to 360, for example. Alternatively, the color type may be represented by a set of three numerical values such as RGB, and the representation format is not limited. Also, the brightness is expressed using, for example, lightness (0 to 100), but it may also be expressed by luminance, and the expression format is not limited. Furthermore, the rate of change in color is, for example, the primary differential of hue, and the rate of change in brightness is, for example, the primary differential of lightness, but their expression formats do not matter.

The feature quantity is, for example, a color (characteristic color) specific to the degraded area. The characteristic color is, for example, a rust color such as reddish brown. Alternatively, the characteristic color may be a corroded color such as dark brown, and any kind of color may be used.

Alternatively, the feature quantity may be color variance. Color dispersion is the degree of color dispersion. It should be noted that, in areas where rust or corrosion occurs, for example, as a result of the appearance of color unevenness or mottled patterns, color dispersion increases. For this reason, the color dispersion can be used to determine whether or not there is deterioration and to determine the degree of deterioration.

Alternatively, the feature amount may be the color change rate or the brightness change rate. Note that the rate of color change or the rate of change in brightness may be referred to, for example, as the intensity of edges or the amount of high-frequency components. The rate of change in color or brightness increases (or edges become stronger, or high-frequency components increase) at locations where cracks or surface roughness occur. It can be used to determine whether the

Alternatively, the feature quantity may be the size of the degraded region. The size is, for example, area. The area is expressed, for example, by the number of blocks. The number of blocks is the number of blocks included in the degraded area.

A block is an area corresponding to two or more pixels. A block is obtained by, for example, dividing an image of M pixels×N pixels into m in the horizontal direction and n in the vertical direction into “m×n” pieces of “(M/m) pixels×(N/ n) each of a region of "pixels". However, M is an integer of 2 or more, and m is a divisor of M. Also, N is an integer of 2 or more, and n is a divisor of N. Furthermore, N and M can be different values or the same value. Specifically, a block is each of 19200 "5 pixels x 5 pixels" regions obtained by dividing an 800 x 600 image into 160 in the horizontal direction and 120 in the vertical direction. be. However, the number of pixels in one block and the aspect ratio do not matter.

Alternatively, the size may be length. The length is, for example, the maximum diameter of the degraded region. The maximum diameter is the maximum span of one region, for example, the diagonal line in the case of a rectangular region, and the diameter in the case of a circular region, but the representation format is not limited. The length may be vertical or horizontal. For example, the size of a degraded region in a vertical column may be expressed only by the length in the vertical direction, and the size of a degraded region in a horizontal beam may be expressed by the length in the horizontal direction only. may be

The length is expressed, for example, by the number of pixels in one direction. One direction is, for example, a diagonal direction, a vertical direction, a horizontal direction, or the like. Alternatively, the length may be expressed by the number of blocks in one direction, or may be an actual size, which will be described later, and the form of expression does not matter.

Note that the above matters regarding the size of the degraded area also apply to the sizes of the following objects. Then, the size of the degraded area is preferably a size relative to the object.

An object is an area corresponding to a building (for example, a station building or various members such as pillars and beams that constitute the station building) in the teacher image or the determination target image. The size relative to the object is the ratio of the size of the degraded region to the size of the object.

That is, the feature amount may be the ratio of the size of the degraded area to the size of the object. The ratio is, for example, an area ratio, but may also be a length ratio. The ratio is specifically, for example, "1/15", "5%", "10%", etc., but the expression format does not matter.

However, the size of the degraded area may be an absolute size. That is, the feature amount may be the actual size of the deteriorated portion in the actual building. The actual size of the deteriorated portion is, for example, "60 mm", "15 cm", etc., but the representation format is not limited. The actual size of the deteriorated portion is obtained, for example, by multiplying the known actual size of the building by the above ratio. Alternatively, the actual size of the deteriorated portion may be obtained by accepting manual input.

The learning information is information indicating the correspondence between the above two or more feature amounts and the deterioration level. The information indicating the correspondence may be a set of pair information, which is pairs of two or more feature quantities and deterioration levels, or may be a function.

The former learning information is, for example, class classification. It can be said that class classification is a set of paired information that is a pair of two or more feature values and a degradation level that is a discrete value (hereinafter sometimes referred to as "first degradation level"). . Therefore, class classification may be rephrased as a correspondence table, for example. A class classification is a set of pairs of two or more feature quantities and first deterioration levels. Each of the two or more pairs constituting the class classification is, for example, two or more feature amounts (hereafter referred to as teacher feature amount (sometimes referred to as a group) and the deterioration level that constitutes the teacher data.

Also, the latter learning information is, for example, a regression equation. A regression equation can be said to be an equation representing the relationship between two or more feature quantities and a continuous value of deterioration level (hereinafter sometimes referred to as “second deterioration level”). A regression equation is a function having two or more feature amounts as parameters. The regression equation is, for example, a linear function, but may be a non-linear function.

A regression equation is acquired using two or more teacher data stored in the teacher data storage unit 111 . The regression equation is obtained, for example, by an algorithm such as the least-squares method using a set of pairs of two or more feature quantities and deterioration levels obtained from two or more pieces of teacher data. I don't mind.

The learning information as described above may be a learning device. A learning device is a function that outputs a corresponding deterioration level for input of two or more feature values by machine learning correspondences between two or more feature values and deterioration levels using two or more teacher data. Or it can be said that it is a program. The learning device may be, for example, a learning device based on a regression equation or a learning device based on class classification.

Learning information may be stored in the learning information storage unit 112 in advance. The pre-stored learning information is, for example, information obtained by machine learning in an external device. Alternatively, the pre-stored learning information may be stored in the deterioration level determination device 1 using two or more teacher data stored in the teacher data storage unit 111, for example, by machine learning, or by human judgment results (for example, , user evaluation, which will be described later), may be information acquired, and its origin is not limited.

The group information storage unit 113 stores one or more group information. Group information is information configured by a group of determination result data and user evaluation. The determination result data is information about the deterioration level determined by the deterioration level determination device 1 for the image to be determined. The determination result data is information indicating the correspondence between the determination target image and the deterioration level determined by the deterioration level determination device 1 .

The determination result data is composed of, for example, a pair of a determination target image and a deterioration level. Alternatively, the determination result data may be composed of a pair of an image identifier and a deterioration level, and its data structure is irrelevant. An image identifier is information that identifies an image. The image is a teacher image or a determination target image. The image identifier is, for example, an ID such as "1", "2", "3", etc., but any information that can identify the image may be used.

In the following description, the number of teacher images stored in the teacher data storage unit 111 is assumed to be "K" (where K is a natural number equal to or greater than 2), and the ID is "k" (where k is a natural number equal to or smaller than K). ) may be referred to as “teaching image k”.

Specifically, the determination result data includes, for example, a determination target image acquired by a determination target image acquisition unit 131, which will be described later, and a deterioration level of the determination target image acquired by a deterioration level acquisition unit 132, which will be described later.

A user evaluation is an evaluation by a user. The user evaluation is, for example, the deterioration level determined by the user for the determination target image included in the determination result data. The deterioration level determined by the user may be the same as or different from the deterioration level determined by the deterioration level determination device 1 . Alternatively, the user evaluation may be the result of judgment by the user regarding the appropriateness of the deterioration level of the judgment result data. Alternatively, the user evaluation may be a deterioration level re-determined by the user for the determination target image of the determination result data having the deterioration level determined by the user to be inappropriate.

The reception unit 12 receives various instructions and various information. The various instructions are, for example, instructions to determine the deterioration level. Various types of information are, for example, user evaluations. Acceptance is, for example, acceptance of information input from an input device such as a keyboard or touch panel. It is also possible to receive the information received.

The receiving unit 12 receives, for example, user evaluations of one or more determination target images stored in the group information storage unit 113 via an input device such as a keyboard. More specifically, for example, the output unit 14, which will be described later, acquires one piece of determination result data from the group information storage unit 113 and displays it on the display, and the reception unit 12 outputs a determination target image that constitutes the displayed determination result data. The process of receiving the user evaluation for the image may be repeatedly performed until receiving the user evaluation for all of the stored one or more determination target images is completed.

Further, the reception unit 12 may receive learning information transmitted from an external device, or may receive learning information read from a recording medium, for example. The receiving unit 12 accumulates the received learning information in the learning information storage unit 112 .

Further, the receiving unit 12 receives, for example, one or more determination target images transmitted from an external device, receives one or two or more determination target images read from a recording medium, and determines the deterioration level. An image captured by a built-in camera of a mobile terminal that implements the determination device 1 may be accepted. The reception unit 12 accumulates the one or more determination target images thus received in the group information storage unit 113, for example.

The processing unit 13 performs various types of processing. The various processes are, for example, the determination target image acquisition unit 131, the deterioration level acquisition unit 132, the first storage unit 133, the first determination unit 134, the second storage unit 135, the second determination unit 136, the reconstruction unit 137, These are processes of the cause determination unit 138, the teacher feature amount group acquisition unit 1321, the learning information acquisition unit 1322, the determination target feature amount group acquisition unit 1323, the deterioration level acquisition unit 1324, and the like. In addition, the processing unit 13 also performs determination and discrimination, which will be described in the flow charts.

The determination target image acquisition unit 131 acquires a determination target image. For example, one or more determination target images are stored in the group information storage unit 113 , and the determination target image acquisition unit 131 acquires one determination target image from the group information storage unit 113 . Alternatively, if the deterioration level determination device 1 is a mobile terminal and the building to be determined is captured by the built-in camera of the mobile terminal, the determination target image acquisition unit 131 may acquire the captured determination target image. good.

The deterioration level acquisition unit 132 applies the determination target image acquired by the determination target image acquisition unit 131 to the learning information stored in the learning information storage unit 112, and acquires the degradation level.

The stored learning information is, for example, one or more types of information among regression equations and class classifications. Application is, for example, the execution of machine learning. Machine learning implementations are, for example, imputation into regression equations, or classification according to classification, or both imputation and classification. Acquisition is, for example, acquisition of a calculation result by a regression equation, acquisition of a classification result by class classification, or acquisition of both a calculation result and a classification result.

The deterioration level acquisition unit 132 uses, for example, two or more teacher data stored in the teacher data storage unit 111 to perform learning in which two or more feature quantities are input and a degradation level is output by a machine learning algorithm. Construct learning information that is a container. A machine learning algorithm is, for example, a random forest algorithm.

Alternatively, the machine learning algorithm may be a supervised learning algorithm other than random forest. Algorithms for supervised learning other than random forest include, for example, SVR, decision tree, deep learning, etc., but may also be neural network, SVM, etc., and the type thereof does not matter. Also in machine learning, the number and combination of two or more types of feature quantities used to determine the deterioration level usually need to be determined artificially, but in the case of deep learning, Such numbers and combinations are also determined automatically.

Alternatively, the machine learning algorithm may be an unsupervised learning algorithm such as cluster analysis, principal component analysis, vector quantization, for example. However, the unsupervised learning algorithm may be excluded from the machine learning algorithm in this embodiment.

A learner configured by a machine learning algorithm may be, for example, a regression-based learner, a class classification-based learner, or both of these two types of learners.

The deterioration level acquisition unit 132 uses two or more pieces of teacher data stored in the teacher data storage unit 111, and outputs discrete deterioration levels based on a machine learning algorithm, for example, with two or more feature values as input. or a learner based on class classification, in which two or more feature quantities are input and discrete deterioration levels are output.

Then, the deterioration level acquisition unit 132 acquires two or more feature amounts from the determination target image acquired by the determination target image acquisition unit 131, and the acquired two or more feature amounts (hereinafter sometimes referred to as a determination target feature amount group). is input to each of the one or more types of learners configured as described above, and one or more types of deterioration levels output from the one or more types of learners are obtained. The one or more deterioration levels obtained are one or more of the first deterioration level and the second deterioration level. The first deterioration level is a discrete deterioration level output from a class classification learner (hereinafter sometimes referred to as a first learner). A second deterioration level is a continuous deterioration level output from a learning device (second learning device) based on a regression equation.

In addition, one determination target feature amount group is input to each of the first and second learners, and the first deterioration level output from the first learner and the second deterioration level output from the second learner When both levels are acquired, the deterioration level acquisition unit 132 may adopt either the first or second deterioration level. At that time, the deterioration level acquisition unit 132, for example, adopts the second deterioration level for the deterioration level lower than the threshold or the threshold or less, and adopts the first deterioration level for the deterioration level equal to or higher than the threshold. is preferred.

Note that the application may also include, for example, obtaining a score indicating the likelihood of the deterioration level obtained by the deterioration level obtaining unit 132 through machine learning. For example, the score is represented by a numerical value from 0 to 1, but may be represented by an integer value from 0 to 100, and the representation format is not limited. For example, the learning information may include a likelihood function, and the deterioration level obtaining unit 132 may obtain a score for the obtained deterioration level using the likelihood function.

Alternatively, applying may be performing a search that does not rely on machine learning. For example, the deterioration level acquisition unit 132 searches stored learning information (for example, class classification) using the determination target feature amount group acquired from the determination target image as a key, and determines the determination target feature amount group in advance. A deterioration level paired with a group of teacher features that are more similar to each other as they satisfy the specified conditions may be obtained. The predetermined condition is, for example, the condition that the degree of similarity with the feature amount group to be determined is the highest and equal to or greater than a threshold.

In this case, the deterioration level acquisition unit 132 selects the teacher feature quantity group having the highest similarity with the acquired determination target feature quantity group from among the two or more teacher feature quantity groups that constitute the stored learning information. Then, when the similarity between the determination target feature amount group and the teacher feature amount group is equal to or greater than the threshold value, the deterioration level paired with the teacher feature amount group is acquired.

Alternatively, the predetermined condition may be a condition that the degree of similarity with the feature amount group to be determined is the highest. A teacher feature quantity group having the highest degree of similarity with the acquired determination target feature quantity group is selected from among the feature quantity groups, and a degradation level paired with the teacher feature quantity group is acquired.

A teacher feature quantity group acquisition unit 1321 constituting the deterioration level acquisition unit 132 acquires a teacher feature quantity group from at least a part of each teacher image for each of two or more teacher images stored in the teacher data storage unit 111. get.

A teacher feature quantity group is a set of two or more feature quantities obtained from one teacher image. A set of two or more feature amounts is, for example, a set of three feature amounts "hue", "hue change rate", and "brightness change rate". However, a set of two or more feature quantities, for example, in addition to the above three feature quantities or in place of one of them, may include an LBP (Local Binary Pattern), and the teacher feature quantity group The number and types of feature quantities to be configured do not matter. The LBP is a bit value such as "0" or "1" that indicates whether the difference between each of two or more pixels that make up an image is equal to or greater than a threshold value. It can be said that it is the information shown in The LBP is well known and will not be described in detail.

At least a portion of the teacher image may be the entirety of one teacher image, or may be a portion of one teacher image corresponding to two or more blocks.

More specifically, the teacher feature quantity group acquisition unit 1321 divides the teacher image k of “M pixels×N pixels” into m in the horizontal direction and n in the vertical direction to obtain “m×n” get a block of Each of “m×n” blocks is composed of “(M/m) pixels×(N/n) pixels”.

Specifically, for example, when the teacher image k is composed of 800 pixels×600 pixels (hereinafter, such a teacher image may be referred to as “800×600 teacher image k”), the teacher feature amount The group obtaining means 1321 obtains 19200 “5×5” blocks by dividing this 800×600 teacher image k into 160 in the horizontal direction and 120 in the vertical direction.

Next, the teacher feature quantity group acquisition means 1321 acquires a teacher feature quantity group from each of the acquired “m×n” blocks. As a result, “m×n” teacher feature quantity groups corresponding to “m×n” blocks are acquired from the teacher image k. For example, from an 800×600 teacher image k, 19200 teacher feature quantity groups corresponding to 19200 blocks are acquired.

The teacher feature quantity group acquisition means 1321 associates each of the “m×n” teacher feature quantity groups acquired from the teacher image k in this way with the image identifier “k” and the block identifier (i, j), and stores them in the storage unit 11.

A block identifier is information that identifies a block. A block identifier is, for example, an ID. A block identifier for the “m×n” blocks acquired from the teacher image k is, for example, a set (i, j) of a horizontal block number i and a vertical block number j. The horizontal block number i is information indicating the position (for example, the i-th block from the left) in the array of m blocks in the horizontal direction. The vertical block number j is information indicating the position (for example, the j-th block from the top) in the array of n blocks in the vertical direction.

Specifically, for example, block identifiers for identifying each of 19200 blocks acquired from an 800×600 teacher image k are (1, 1), (1, 2), . . . (120, 160) etc. Such a block identifier (i, j) is also positional information indicating the position of the block in the teacher image k. Note that the block identifier when the block array is one-dimensional may be one block number such as "1", "2", "3", for example. However, the representation format of the block identifier does not matter.

For example, for each of “m×n” blocks obtained from a teacher image k of M pixels×N pixels, a set of three feature amounts “hue”, “hue change rate”, and “brightness change rate” , the teacher feature group acquisition unit 1321 acquires the hue using “(M/m)×(N/n)” pixels that make up the block. do. The teacher feature quantity group acquisition unit 1321 associates the acquired “m×n” hues with the image identifier “k” and the block identifier (i, j) and stores them in the storage unit 11 .

As a result, for example, 19200 hues obtained from an 800×600 teacher image k are represented by image identifier “k” and block identifiers (1, 1), (1, 2), . . . (120, 160 ) etc., and stored in the storage unit 11 .

Next, the teacher feature quantity group acquisition unit 1321 uses “m×n” hues stored in pairs with block identifiers (i, j) in the storage unit 11, for example, to obtain “m×n” For each block, the “hue rate of change”, which is the result of primary differentiation of the hue with respect to the positional information, is obtained. The teacher feature quantity group acquiring means 1321 associates the acquired “m×n” pieces of “hue change rate” with the block identifier (i, j) and further accumulates them in the storage unit 11 .

As a result, for example, 19200 “hue change rates” obtained from an 800×600 teacher image k are represented by image identifier “k” and block identifiers (1, 1), (1, 2), . (120, 160) and the like are further stored in the storage unit 11 .

Further, the teacher feature quantity group acquisition unit 1321 uses, for example, “(M/m)×(N/n)” pixels constituting each “m×n” blocks, Get brightness. Then, the teacher feature value group acquisition unit 1321 acquires the “brightness change rate”, which is the result of first-order differentiation of the brightness with the position information, using the acquired “m×n” brightnesses. The teacher feature quantity group acquiring means 1321 associates the two or more "brightness change rates" thus acquired with the image identifier "k" and the block identifier (i, j), and further accumulates them in the storage unit 11. FIG.

As a result, for example, 19200 "brightness change rates" obtained from an 800×600 teacher image k are represented by the image identifier "k" and the block identifiers (1, 1), (1, 2), . (120, 160), etc., are further stored in the storage unit 11, whereby the storage unit 11 stores three feature values " As a result, a teacher feature amount group composed of a set of "hue", "hue change rate", and "brightness change rate" is stored.

Next, the teacher feature quantity group acquisition unit 1321 acquires the deterioration level corresponding to the one teacher image from the teacher data storage unit 111 . Then, the teacher feature quantity group acquisition unit 1321 acquires " A deterioration level obtained from the teacher data storage unit 111 is associated with a set of m×n″ teacher feature quantity groups.

As a result, in the storage unit 11, for example, 19200 teacher feature quantity groups obtained from the 800×600 teacher image k are stored with the image identifier “k” and the block identifiers (1, 1), (1, 2). , .

The teacher feature quantity group acquisition means 1321 performs the above-described processing for each of K teacher images. In addition, the result of storing "K×m×n" teacher feature amount groups composed of sets of three feature amounts "hue", "hue change rate", and "brightness change rate" Become.

Note that the block size may be variable. That is, the teacher feature quantity group acquisition means 1321 may employ blocks of different sizes according to the size of the teacher image, for example, and acquire the teacher feature quantity group from each block. Specifically, the teacher feature quantity group acquisition means 1321 adopts, for example, 5×5 blocks for an 800×600 teacher image, and 4×4 blocks for a 640×480 teacher image. By adopting, for example, even if the size of the teacher image is small, the teacher feature value group can be acquired with high accuracy, and even if the size of the teacher image is large, the amount of processing for acquiring the teacher feature value group can be reduced. can be suppressed. In other words, it is possible to efficiently acquire two or more teacher feature quantity groups from two or more teacher images of different sizes.

Alternatively, the teacher feature quantity group acquisition means 1321 may adopt blocks of different sizes in one teacher image, for example, according to the change rate of color or brightness, and acquire the teacher feature quantity group from each block. good. Specifically, the teacher feature quantity group acquisition unit 1321 employs, for example, 5×5 blocks for a portion in one teacher image in which the rate of change in color or brightness is greater than or equal to a threshold. , by adopting 10 × 10 blocks for portions where the rate of change in color or luminance is equal to or less than the threshold value, a highly accurate teacher feature amount group is obtained from the portion corresponding to the degraded region, and the degraded region is obtained. A low-accuracy teacher feature quantity group is obtained from a portion corresponding to a region other than the region (hereinafter sometimes referred to as a healthy region). In this way, a group of teaching information is acquired with high accuracy only from the degraded region in one teacher image, and a group of teaching information is acquired with low accuracy from the healthy region. can be obtained efficiently.

Alternatively, the teacher feature quantity group acquisition means 1321 acquires a teacher feature quantity group only from a part in one teacher image in which the rate of change in color or brightness is greater than or equal to the threshold value. It is not necessary to acquire a teacher feature quantity group from a portion whose rate is equal to or less than the threshold or less than the threshold.

Specifically, the teacher feature quantity group acquisition unit 1321, for example, divides the teacher image into two or more blocks, and among the two or more blocks, the color or luminance change rate is greater than or equal to the threshold The teacher feature quantity group is obtained only from the block, and the teacher feature quantity group is not obtained from blocks whose rate of change in color or brightness is equal to or less than the threshold or less than the threshold.

As a result, two or more teacher feature quantity groups can be efficiently obtained with high accuracy only from the degraded region in one teacher image.

It should be noted that the above-mentioned items regarding the block of the teacher image also apply to the block of the determination target image.

Alternatively, the teacher feature quantity group acquisition unit 1321 may acquire the degraded region by performing contour line detection and region extraction for each of two or more teacher images. Contour detection is processing for detecting the contour of an area within an image. The teacher feature quantity group acquisition unit 1321 detects the contour line of the degraded region, for example, by binarizing the teacher image with a predetermined threshold value. Region extraction is processing for extracting a region from an image. The teacher feature quantity group acquisition unit 1321 extracts an area surrounded by the detected outlines from the teacher image. The teacher feature quantity group acquisition unit 1321 may acquire a teacher feature quantity group for each of the two or more degraded regions thus acquired from the two or more teacher images.

The learning information acquisition unit 1322 acquires learning information using two or more teacher feature quantity groups acquired by the teacher feature quantity group acquisition unit 1321 for each of the two or more teacher images stored in the teacher data storage unit 111. Acquired and accumulated in the learning information storage unit 112 .

For example, the learning information acquisition unit 1322 acquires “K×m×n” teacher feature amount groups acquired by the teacher feature amount group acquisition unit 1321 for K teacher images stored in the teacher data storage unit 111. is used to acquire learning information and store it in the learning information storage unit 112 .

The learning information acquisition unit 1322 uses two or more teacher feature quantity groups acquired for each of the two or more stored teacher images to obtain, for example, pairs of teacher images and two or more teacher feature quantity groups. Acquire learning information that is a set.

The learning information acquisition unit 1322 uses, for example, “m×n” teacher feature quantity groups acquired for each of K or more stored teacher images, and “m×n” teacher images and “m×n” Acquire learning information composed of a set of k pairs with the teacher feature quantity group.

Alternatively, the learning information acquisition unit 1322 uses two or more teacher feature quantity groups acquired by the teacher feature group acquisition unit 1321 for each of the two or more teacher images stored in the teacher data storage unit 111, for example , a machine learning algorithm may configure one or more types of learners out of a first learner based on class classification or a second learner based on regression equations.

In addition, the learning information acquisition unit 1322 obtains the teacher feature amount group acquired by the teacher feature amount group acquisition unit 1321 and the two or more acquired teacher feature amount groups for each of the two or more stored teacher images. , using the discrete first deterioration level output from the first learner and the continuous second deterioration level output from the second learner, for example, by a machine learning algorithm, A third learner may also be constructed. The third learner is a learner that receives the first and second deterioration levels as inputs and outputs either the first or second deterioration level.

Further, the learning information acquisition unit 1322 acquires the “m×n” blocks that the teacher feature group acquisition unit 1321 acquires from the “m×n” blocks forming the teacher image for each of the two or more stored teacher images. × n” number of teacher feature quantity groups and the first or second deterioration level output from the first or second learner for the “m × n” number of teacher feature quantity groups. A fourth learner may also be constructed using m×n″ local degradation levels, for example by a machine learning algorithm. The fourth learner is a learner that receives "m×n" local deterioration levels as inputs and outputs global deterioration levels.

The determination target feature amount group acquisition unit 1323 acquires a determination target feature amount group, which is a set of two or more feature amounts, from at least a part of the determination target image acquired by the determination target image acquisition unit 131 . The procedure for the determination target feature quantity group acquisition unit 1323 to acquire the determination target feature quantity group from the determination target image may be the same as the procedure for the teacher feature quantity group acquisition unit 1321 to acquire the teacher feature quantity group from the teacher image.

That is, the determination target feature amount group acquisition unit 1323, for example, divides the acquired determination target image into two or more blocks, and acquires the determination target feature amount group from each of the two or more blocks.

For example, the determination target feature amount group acquisition unit 1323 divides the determination target image of “M pixels×N pixels” into m segments in the horizontal direction and n segments in the vertical direction to obtain “(M/m) pixels x Obtain “m×n” blocks each composed of “(N/n) pixels”. As a result, for example, the 800×600 image to be determined is divided into 19200 blocks of 5×5, and each of the 19200 blocks is divided into A set of features is obtained.

The deterioration level acquisition unit 1324 applies the determination target feature amount group acquired by the determination target feature amount group acquisition unit 1323 to the learning information of the learning information storage unit 112, and acquires the deterioration level corresponding to the determination target feature amount group. do.

For example, the deterioration level acquisition unit 1324 applies the determination target feature amount group acquired from each of two or more blocks by the determination target feature amount group acquisition unit 1323 to the learning information, and acquires the local deterioration level of each block. A local deterioration level is a deterioration level for each of two or more blocks forming a determination target image.

For example, the deterioration level acquisition unit 1324 applies the determination target feature amount group acquired from each of the m×n blocks by the determination target feature amount group acquisition unit 1323 to the learning information, and locally obtains the m×n blocks. Get deterioration level. Each of the m×n local degradation levels thus obtained has, for example, a block identifier (i,j) and a degradation level.

The deterioration level acquisition unit 1324 acquires two or more (eg, m×n: the same shall apply hereinafter) local deterioration levels using, for example, a machine learning algorithm.

Further, the deterioration level obtaining means 1324 obtains the global deterioration level using the two or more local deterioration levels thus obtained. A global deterioration level is a deterioration level for two or more blocks that form a determination target image. The global deterioration level is, for example, the deterioration level for all of the two or more blocks that make up the determination target image (that is, the deterioration level in units of the determination target image). Among them, the deterioration level for two or more blocks corresponding to the object (that is, the deterioration level for each object) may be used.

A global degradation level is obtained using, for example, the size of the degraded region. It is particularly preferable to obtain the global deterioration level using, for example, the ratio of the size of the deteriorated region to the size of the determination target image. The ratio of the size of the degraded region to the size of the image to be determined is, for example, the ratio of the number of blocks whose local deterioration level is smaller than or equal to or less than the threshold to the number of blocks forming the image to be determined.

The deterioration level obtaining means 1324 obtains the size of the deteriorated area using the obtained two or more local deterioration levels, and obtains the global deterioration level using the obtained size of the deteriorated area.

Alternatively, the deterioration level obtaining means 1324 may divide the obtained size of the deteriorated region by the size of the determination target image, and use the division result to obtain the global deterioration level.

For example, when dividing an 800×600 determination target image into 5×5 blocks, the deterioration level acquisition unit 132 divides “800×600” by “5×5” to obtain the size of the determination target image “ 19200" is obtained. If these 19200 blocks include, for example, 600 blocks whose local deterioration level is smaller than the threshold or equal to or less than the threshold, the deterioration level acquisition unit 132 determines the size of the deterioration region to be "600". It is divided by the size of the target image "19200" to obtain the division result "1/32".

The deterioration level obtaining means 1324 uses the obtained size of the deteriorated region or the result obtained by dividing the obtained size of the deteriorated region by the size of the determination target image, for example, using a machine learning algorithm to obtain the overall deterioration level. to get

Alternatively, the correspondence information indicating the correspondence between the size of the degraded area and the global degradation level is stored in the storage unit 11, and the degradation level acquisition means 1324 acquires the global degradation level corresponding to the acquired size of the degraded area. , may be obtained using this correspondence information.

Alternatively, the storage unit 11 stores correspondence information indicating the correspondence between the division result and the global deterioration level, and the deterioration level acquisition unit 1324 divides the size of the acquired deterioration region by the size of the determination target image, A global degradation level corresponding to the division result may be obtained using this correspondence information.

The deterioration level acquisition unit 1324 may acquire the deterioration level by an algorithm corresponding to the degree of deterioration to be determined among two or more predetermined algorithms. Two or more predetermined algorithms include, for example, a first algorithm and a second algorithm. The first algorithm is an algorithm that obtains continuous values using a regression equation. A second algorithm is an algorithm that obtains discrete values by class classification.

The deterioration level acquisition means 1324, for example, first acquires the first deterioration level, which is a discrete value, by the second algorithm. When the obtained first deterioration level is equal to or less than the threshold or lower than the threshold, the deterioration level obtaining means 1324 further obtains a second deterioration level which is a continuous value by the first algorithm, and determines the deterioration corresponding to the second deterioration level. adopt a level. When the first degree of deterioration is higher than the threshold or equal to or greater than the threshold, the deterioration level acquiring means 1324 adopts the deterioration level corresponding to the first degree of deterioration.

Alternatively, the deterioration level acquisition unit 1324 acquires the deterioration degree for one determination target by the first and second algorithms, and among the two or more acquired deterioration degrees, the deterioration level corresponding to the deterioration degree that satisfies the condition. may be adopted. The conditions are, for example, when the degree of deterioration obtained by the second algorithm is greater than or equal to the threshold, the deterioration level corresponding to the degree of deterioration is adopted, and the degree of deterioration obtained by each second algorithm exceeds the threshold If it is less than or equal to the threshold value, the deterioration level corresponding to the deterioration degree obtained by the second algorithm is adopted.

Alternatively, if the average value of the two deterioration degrees obtained by the first and second algorithms is greater than or equal to the threshold, the deterioration level corresponding to the deterioration degree obtained by the second algorithm is adopted. However, if the average value is smaller than the threshold value or equal to or less than the threshold value, the condition may be such that the deterioration level corresponding to the degree of deterioration acquired by the first algorithm is adopted.

In the deterioration level acquisition unit 132 configured as described above, the teacher feature group acquisition unit 1321 and the learning information acquisition unit 1322 constitute a learning unit (not shown). An automatic determination unit (not shown) may be configured together with the level acquisition unit 1324 .

Further, the deterioration level determination unit 132 may have only the automatic determination unit without the learning unit. can be determined automatically. Automatic determination based on prior learning means, for example, determining the deterioration level using learning information stored in advance in the learning information storage unit 112 .

The first storage unit 133 stores the determination result data in the group information storage unit 113 . The first storage unit 133 stores, for example, determination result data composed of a pair of the determination target image acquired by the determination target image acquisition unit 131 and the deterioration level acquired by the degradation level acquisition unit 1324 for the determination target image. are stored in the group information storage unit 113 .

The first determination unit 134 determines whether or not the determination target image forming one or more pieces of determination result data stored in the group information storage unit 113 satisfies a predetermined first condition. The first condition is, for example, that the deterioration level associated with the determination target image is equal to or less than a threshold or smaller than the threshold. The threshold value is, for example, "3", but may be "2", "4", etc., and the numerical value is not limited.

Note that the first condition may be, for example, the condition that the deterioration level for the determination target image is acquired, or the condition that the determination target image is acquired, or any condition related to the determination target image. For example, the first condition may be a condition that the score indicating the likelihood of the deterioration level acquired for the determination target image is equal to or less than a threshold or smaller than a threshold. Alternatively, the first condition is that the deterioration level associated with the determination target image is equal to or less than a threshold or less than the threshold, and the score indicating the likelihood of the deterioration level is equal to or less than the threshold or less than the threshold. It may be a condition that

The second accumulation unit 135 accumulates user evaluations received by the reception unit 12 for each of the one or more determination target images that satisfy the first condition, in association with determination result data formed by the determination target images. As a result, in the grouped information storage unit 113, one or more grouped information having the judgment result data including the judgment target image that satisfies the first condition and the user's evaluation of the judgment target image is configured.

The second accumulation unit 135 stores, for example, the user evaluation received by the receiving unit 12 for one or more determination target images that satisfy the first condition, and the user evaluation indicating the deterioration level determined by the user as the determination target image. They are stored in association with the judgment result data that the image constitutes. As a result, in the group information storage unit 113, one or more groups having the determination result data including the determination target image satisfying the first condition and the user evaluation indicating the deterioration level determined by the user for the determination target image. Information is composed.

The second determination unit 136 determines whether or not the user evaluation received by the reception unit 12 satisfies a predetermined second condition. The second condition is, for example, that group information having different deterioration levels and user evaluations is stored in the group information storage unit 113 in a number equal to or greater than a threshold. The threshold value is, for example, "2", but may be "1", "5", "10", or the like, and any numerical value is acceptable.

Note that the second condition may be, for example, a condition that user evaluation has been received, or any condition related to user evaluation. For example, the second condition may be a condition that the calculated deterioration level is different from the received user evaluation.

The reconstruction unit 137 reconstructs learning information stored in the learning information storage unit 112 using one or more grouped information stored in the grouped information storage unit 113 .

The reconstruction unit 137 reconstructs the learner using one or more sets of information, for example, by a machine learning algorithm.

Alternatively, the reconstruction unit 137 uses one or more group information stored in the group information storage unit 113 in addition to the two or more teacher data stored in the teacher data storage unit 111 to perform machine learning. The algorithm may reconfigure the learner.

The reconfiguration unit 137 normally reconfigures the learning device when the second determination unit 136 determines that the user evaluation received by the reception unit 12 satisfies the second condition. Alternatively, the reconfiguring unit 137 may reconfigure the learning information when the receiving unit 12 receives an instruction to perform reconfiguration, regardless of the trigger or timing for reconfiguring the learning information.

The cause determination unit 138 uses the determination target image acquired by the determination target image acquisition unit 131 to determine which of two or more predetermined deterioration causes the deterioration cause to be determined is. A deterioration cause is a cause of deterioration of a building. The cause of deterioration in the present embodiment may be said to be, for example, a mode of deterioration. The cause of deterioration is, for example, rust, but may be cracks, deformation, or the like, regardless of the type. It should be noted that the factors that cause various types of deterioration are not limited to changes over time. For example, rust is caused by rain leaks, flying salt, etc., and deformation is caused by load, vibration, and the like. In other words, the factor that causes the deterioration does not matter.

The predetermined two or more types are, for example, types related to deterioration modes such as rust, cracks, and deformation, and the determination is to determine which of these two or more modes the deterioration mode is. is. Alternatively, the determination may be, for example, classifying the cause of deterioration into two or more aspects such as rust, cracking, and deformation.

Specifically, for example, the storage unit 11 stores cause determination information. The cause determination information is information for determining the cause of deterioration of the building. The cause determination information is composed of, for example, a set of pairs of one or more feature quantities and deterioration causes. One or more pairs of feature quantity and cause of deterioration include, for example, a pair of feature quantity “reddish brown” indicating a hue characteristic of rust and the cause of deterioration “rust”, and a high frequency component characteristic of cracks (for example , a “brightness change rate” greater than or equal to a threshold) and a deterioration cause “crack”.

For example, the cause determination unit 138 uses the determination target feature amount group (eg, hue, high frequency component, etc.) acquired by the determination target feature amount group acquisition unit 1323 for the determination target image acquired by the determination target image acquisition unit 131. Then, it is determined which one of two or more predetermined causes of deterioration is the cause of deterioration to be determined.

Alternatively, when the deterioration level acquired by the deterioration level acquisition unit 132 is equal to or higher than the threshold value for the determination target image acquired by the determination target image acquisition unit 131, the cause determination unit 138, for example, determines the determination target feature amount group Using the determination target feature amount group acquired by the acquisition unit 1323, it may be determined which of two or more predetermined deterioration causes the deterioration cause to be determined is.

Alternatively, the cause determination unit 138 may determine the cause of deterioration using a machine learning algorithm such as random forest, or may configure a learning device that receives one or more feature values as an input and outputs the cause of deterioration. do not have. The one or more feature amounts used for determining the cause of deterioration are, for example, a part of the two or more feature amounts used for determining the cause of deterioration, but may be a feature amount different from that for determining the cause of deterioration. Note that the cause determination unit 138 may also acquire a score indicating the likelihood of the acquired deterioration cause.

The cause of deterioration determined by the cause determination unit 138 is output by the output unit 14, for example. Alternatively, the determined cause of deterioration may be transferred to the deterioration level acquisition unit 132, and the deterioration level acquisition unit 132 may perform level determination using different algorithms depending on the cause of deterioration.

The output unit 14 outputs the deterioration level acquired by the deterioration level acquisition unit 132 . Output is a concept that includes, for example, display on a display, sound output from a speaker, printing on a printer, storage on a recording medium, delivery to another program, transmission to an external device, and the like. In addition, the output unit 14 may output the score obtained for the deterioration level together with the obtained deterioration level.

Also, the output unit may output the cause of deterioration acquired by the cause determination unit 138, for example.

The output unit 14 normally displays information such as the deterioration level on a display, but it may be stored in a recording medium, transferred to another program, or transmitted to an external device. do not have.

The storage unit 11, the learning information storage unit 112, the teacher data storage unit 111, and the group information storage unit 113 are preferably non-volatile recording media such as hard disks and flash memories, but may also be storage media such as RAMs. It is feasible.

It does not matter how the information is stored in the storage unit 11 or the like. For example, information may be stored in the storage unit 11 or the like via a recording medium, or information transmitted via a network, a communication line, or the like may be stored in the storage unit 11 or the like. Alternatively, information input via an input device may be stored in the storage unit 11 or the like. Any input device such as a keyboard, mouse, touch panel, or the like may be used.

The reception unit 12 may or may not include an input device. The reception unit 12 can be realized by the driver software of the input device, or by the input device and its driver software.

Note that the reception function of the reception unit 12 is usually realized by wired or wireless communication means (for example, a communication module such as a NIC (Network interface controller) or a modem), but means for receiving broadcasts (for example, broadcast reception module).

Processing unit 13, determination target image acquisition unit 131, deterioration level acquisition unit 132, first determination unit 134, first storage unit 133, second storage unit 135, reconstruction unit 137, second determination unit 136, cause determination unit 138 , the teacher feature quantity group acquisition means 1321, the learning information acquisition means 1322, the determination target feature quantity group acquisition means 1323, and the deterioration level acquisition means 1324 are usually implemented by an MPU, a memory, or the like. The processing procedure of the processing unit 13 and the like is normally realized by software, and the software is recorded in a recording medium such as a ROM. However, the processing procedure may be realized by hardware (dedicated circuit).

The output unit 14 may or may not include an output device such as a display or speaker. The output unit 14 can be realized by the driver software of the output device, or by the output device and its driver software.

The transmission function of the output unit 16 is usually realized by wired or wireless communication means, but may be realized by broadcasting means (for example, a broadcasting module).

Next, the operation of the deterioration level determination device 1 will be described using the flowchart of FIG.

(Step S201) The processing unit 13 determines whether or not to determine the deterioration level. For example, when the receiving unit 12 receives an instruction to determine the deterioration level, the processing unit 13 determines to perform the deterioration level determination. Alternatively, the processing unit 13 may determine to determine the deterioration level when the receiving unit 12 receives an image captured by the built-in camera of the mobile terminal that implements the deterioration level determination device 1 . If the deterioration level is determined, the process proceeds to step S202, and if the deterioration level is not determined, the process proceeds to step S206.

(Step S202) The determination target image acquisition unit 131 acquires a determination target image. For example, one or more determination target images are stored in the group information storage unit 113 , and the determination target image acquisition unit 131 acquires one determination target image from the group information storage unit 113 . Alternatively, if the deterioration level determination device 1 is a mobile terminal and the building to be determined is captured by the built-in camera of the mobile terminal, the determination target image acquisition unit 131 may acquire the captured determination target image. good.

(Step S203) The deterioration level obtaining unit 132 applies the determination target image obtained in step S202 to the learning information stored in the learning information storage unit 112, and obtains the deterioration level. Note that the detailed operation of the deterioration level acquisition unit 132 has been described above, so the same description will not be repeated.

(Step S204) The output unit 14 outputs the deterioration level acquired in step S203. The output unit 14 normally displays the obtained deterioration level on a display, but may store it in a recording medium, pass it to another program, or transmit it to an external device.

(Step S205) The first storage unit 133 stores in the group information storage unit 113 determination result data corresponding to the determination target image accepted in step S202 and the deterioration level acquired in step S203. After that, the process returns to step S201.

(Step S<b>206 ) The first determination unit 134 determines whether or not the determination target image forming the one or more determination result data stored in the group information storage unit 113 satisfies the first condition. The first condition is, for example, that the deterioration level associated with the determination target image is equal to or less than a threshold or smaller than the threshold. If any one determination target image satisfies the first condition, the process proceeds to step S206, and if none of the determination target images satisfies the first condition, the process returns to step S201.

(Step S207) The processing unit 13 determines whether or not the receiving unit 12 has received user evaluation of the determination target image that satisfies the first condition. If the user's evaluation of the determination target image that satisfies the first condition has been accepted, the process proceeds to step S208; otherwise, the process returns to step S201.

(Step S208) The second accumulation unit 135 accumulates the user evaluation received in step S207 in association with the judgment result data including the judgment target image that is the object of the user evaluation. As a result, in the grouped information storage unit 113, grouped information including the determination result data including the determination target image that satisfies the first condition and the user's evaluation of the determination target image is configured.

(Step S209) The second judgment unit 136 judges whether or not the user evaluation accepted in step S207 satisfies the second condition. The second condition is, for example, that group information having different deterioration levels and user evaluations is stored in the group information storage unit 113 in a number equal to or greater than a threshold.

(Step S<b>210 ) The reconstruction unit 137 reconstructs learning information stored in the learning information storage unit 112 using one or more group information stored in the group information storage unit 113 . After that, the process returns to step S201.

In the flowchart of FIG. 2, the process starts when the power of the deterioration level determination device 1 is turned on or the program is started, and the process ends when the power is turned off or an interruption to end the process is performed. However, the trigger for starting or ending processing does not matter.

A specific operation example of the deterioration level determination device 1 will be described below. The deterioration level determination device 1 in this example is a server of a company that transports passengers by railroad and maintains and manages facilities such as station buildings. A building is a building such as a station building, or a member such as a pillar or a beam that constitutes the building, and a deteriorated portion is a rusted portion of a member such as a pillar.

The teacher data storage unit 111 stores in advance two or more teacher data, for example, as shown in FIG. FIG. 3 is a data structure diagram of teacher data. Teacher data is a pair of a teacher image and a deterioration level. The teaching image is an image of a rusted portion of a member such as a pillar. The degradation level is a discrete value ranging from 0-5.

Two or more pieces of stored teacher data are associated with IDs (for example, "1", "2", etc.). For example, teacher data corresponding to ID "1" (hereinafter sometimes referred to as teacher data 1) has teacher image 1 and deterioration level "2". Similarly, teacher data (teacher data 2) corresponding to ID "2" has teacher image 2 and deterioration level "4".

Two or more learners can be stored in the learning information storage unit 112 . Two of the two or more learners receive as input a feature amount group that is a set of three feature amounts "hue", "hue change rate", and "brightness change rate", and output the deterioration level. It is a learner. Of these two learners, one is a first learner based on a regression formula and the other is a second learner based on class classification. For the input of the feature amount group, the first learning device outputs the first deterioration level, which is a discrete value in the range of 0 to 5, and the second learning device outputs the continuous value in the same range. A second degradation level is output.

Further, the learning information storage unit 112 receives the first deterioration level from the first learning device and the second deterioration level from the second learning device, and either the first deterioration level or the second deterioration level is input. A third learner with output .

Furthermore, in the learning information storage unit 112, there is a fourth learner that receives "160×120" local deterioration levels from the first or second learner and outputs one global deterioration level. can also be stored. Note that the global deterioration level output from the fourth learner is the first deterioration level regardless of whether the local deterioration level input from the third learner is the first deterioration level or the second deterioration level. be. However, if the local deterioration level input from the third learner is the first deterioration level, the fourth learner outputs a global deterioration level that is the same first deterioration level. If the local deterioration level input from the learning device is the second deterioration level, the global deterioration level, which is the same second deterioration level, may be output.

The first and second learners use two or more teacher data (teacher data 1, teacher data 2, etc.: see FIG. 3) stored in the teacher data storage unit 111 by the learning information acquisition means 1322, It is configured by a machine learning algorithm and stored in the learning information storage unit 112 .

In constructing the first and second learners, the storage unit 11 can store, for example, sets of pairs of teacher feature groups and deterioration levels as shown in FIG.

A pair of a teacher feature quantity group and a deterioration level is, for example, a teacher feature quantity group of “160×120” (=19200) pieces obtained from “160×120” (=19200) blocks constituting the teacher image 1, A pair with deterioration level “2” corresponding to teacher image 1 (hereinafter referred to as a first pair), and “160×120” pieces obtained from “160×120” blocks constituting teacher image 2 A pair of a teacher feature quantity group and a deterioration level "4" corresponding to the teacher image 2 (hereinafter referred to as a second pair).

Each of the “160×120” teacher feature quantity groups that make up the first pair is assigned to the image identifier “1” and block identifiers (1, 1), (1, 2), . . . (160, 120). They are stored in the storage unit 11 in association with each other.

For example, the teacher feature quantity group associated with the image identifier “1” and block identifier (1, 1) is a set of hue “H1”, hue change rate “RH1”, and lightness change rate “RB1”. . Note that, hereinafter, a set of feature amounts corresponding to block (1, 1) may be written as "(H1, HR1, RB1)". Also, (H1, HR1, RB1), etc. are sets of three numerical values.

Similarly, the teacher feature quantity group associated with the image identifier "1" and the block identifier (1, 2) is (H2, HR2, RB2). Also, the teacher feature amount group associated with the image identifier "1" and the block identifier (160, 120) is (H19200, HR19200, RB19200).

Each of the “160×120” teacher feature quantity groups that make up the second pair is assigned to image identifier “2” and block identifiers (1, 1), (1, 2), . . . (160, 120) They are stored in the storage unit 11 in association with each other.

For example, the teacher feature quantity group associated with image identifier “2” and block identifier (1, 1) is (H1, HR1, RB1), and is associated with image identifier “2” and block identifier (1, 2). The teacher feature quantity group obtained is (H2, HR2, RB2), and the teacher feature quantity group associated with the image identifier "2" and the block identifier (160, 120) is (H19200, HR19200, RB19200).

Note that in the set of pairs of teacher feature quantity groups and deterioration levels shown in FIG. 4, the deterioration levels include a local deterioration level (not shown) and a global deterioration level (not shown). A local deterioration level is a discrete first deterioration level or a continuous second deterioration level. A global degradation level is a discrete first degradation level.

To each of the configured first and second learning devices, "160 x 120" determination target feature quantity groups obtained from the determination target image are input, and "160 x 120" feature values are input from the first learning device. , and the second learning device outputs 160×120 second deterioration levels.

The third learning device performs the first Constructed by a machine learning algorithm using "160 × 120" first deterioration levels output from the learner and "160 × 120" second deterioration levels from the second learner .

For the configured third learner, "160 × 120" first deterioration levels output from the first learner and "160 × 120" first deterioration levels output from the second learner Two deterioration levels are input, and "160×120" first or second deterioration levels are output from the third learner.

The fourth learning device outputs "160 x 120" number of third one deterioration level, and the "160×120" second deterioration levels from the second learner, the "160×120" first deterioration levels and the "160×120" second deterioration levels It is configured by a machine learning algorithm using “160×120” first or second deterioration levels output from the third learning device for .

"160 × 120" first or second deterioration levels output from the third learner are input to the configured fourth learner, and one first A global degradation level, which is the degradation level, is output.

When the deterioration level acquiring means 1324 acquires the deterioration level for each of the one or more determination target images stored in the group information storage unit 113 using the first to fourth learning devices, the storage unit 11 stores can store, for example, a set of pairs of determination target feature quantity groups and deterioration levels as shown in FIG. Degradation levels include local degradation levels and global degradation levels.

A pair of a determination target feature amount group and a deterioration level is, for example, “160×120” determination target feature values obtained from “160×120” (=19200) flocks forming the determination target image α. group, "160x120" local degradation levels corresponding to the "160x120" determination target feature quantity groups, and a global degradation level "3" corresponding to the determination target image α. (hereinafter referred to as the first pair).

Alternatively, the pair of the determination target feature amount group and the deterioration level is the "160×120" determination target feature amount group obtained from the "160×120" (=19200) flocks forming the determination target image β. , a deterioration level including “160×120” local deterioration levels corresponding to the “160×120” determination target feature quantity groups, and a global deterioration level “4” corresponding to the determination target image β (hereinafter referred to as the second pair).

The data structure of the determination target feature quantity group is the same as the data structure of the teacher feature quantity group shown in FIG. For example, in the first pair, the determination target feature quantity group associated with the image identifier “α” and the block identifier (1, 1) is (H1, HR1, RB1), and the image identifier “α” and the block identifier ( 1, 2) is (H2, HR2, RB3), and the feature group to be determined associated with the image identifier "α" and the block identifier (160, 120) is (H19200, HR19200, RB19200).

The determination target feature value group (H1, HR1, RB1) in the first pair is associated with the local deterioration level "2.2", and the determination target feature value group (H2, HR2, RB2) is associated with the local A deterioration level of "3.8" is associated, and a local deterioration level of "3.5" is associated with the determination target feature quantity group (H19200, HR19200, RB19200). These “160×120” local deterioration levels, which are continuous second deterioration levels, are associated with global deterioration level “3”, which is discrete first deterioration levels.

Similarly, in the second pair, the determination target feature quantity group associated with the image identifier "β" and the block identifier (1, 1) is (H1, HR1, RB1), and the image identifier "β" and the block identifier The determination target feature amount group associated with (1, 2) is (H2, HR2, RB3), and the determination target feature amount group associated with the image identifier "β" and the block identifier (160, 120) is (H19200 , HR19200, RB19200).

A local deterioration level "4" is associated with the determination target feature value group (H1, HR1, RB1) in the second pair, and a local deterioration level is associated with the determination target feature value group (H2, HR2, RB2). "5" is associated, and the determination target feature amount group (H19200, HR19200, RB19200) is associated with the local deterioration level "4". These “160×120” local deterioration levels, which are discrete first deterioration levels, are associated with global deterioration level “4”, which is discrete first deterioration levels. .

Further, the storage unit 11 stores a first condition "the deterioration level of the determination target image is threshold 3 or less" regarding the determination target image. Further, the storage unit 11 also stores a second condition related to the evaluation level, that is, "grouped information having a deterioration level different from the user's evaluation is stored in the grouped information storage unit 113 with a threshold value of 2 or more".

The set information storage unit 113 can store, for example, one or more pieces of set information as shown in FIG. FIG. 6 is a data structure diagram of group information. The set information is composed of a set of determination result data and user evaluation. The determination result data includes the determination target image and the deterioration level.

One or more sets of stored information are associated with IDs (for example, "1", "2", etc.). For example, group information 1 corresponding to ID "1" has determination target image α, deterioration level "3", and user evaluation "2". Note that the deterioration level “3” is the deterioration level determined by the deterioration level determination device 1 with respect to the determination target image α. ” (see FIG. 5). The user evaluation “2” is the deterioration level determined by the user with respect to the determination target image α, and is a value received by the receiving unit 12 .

Similarly, set information 2 has determination target image β, deterioration level “4”, and user evaluation “Null”. Note that the deterioration level “4” is the deterioration level determined by the deterioration level determination device 1 with respect to the determination target image β, and is the global deterioration level “4” (FIG. 5) output from the fourth learning device. ). The user evaluation “Null” is information indicating that there is no degradation level determined by the user for the determination target image β.

At present, no learning device is stored in the learning information storage unit 112, and only two determination target images α and β are stored in the storage unit 11. FIG.

First, using two or more teacher images 1 and 2 stored in the learning information storage unit 112, the teacher feature quantity group acquisition unit 1321 and the learning information acquisition unit 1322 configure a learning device in the following procedure.

The teacher feature quantity group acquisition means 1321 divides the teacher image 1 into “160×120” blocks, and from the “160×120” blocks, “hue”, “hue change rate”, and “brightness (H1, HR1, RB1), (H2, HR2, RB2), . Then, the teacher feature quantity group acquisition means 1321 assigns each of the acquired “160×120” teacher feature quantity groups to the image identifier “1” and the block identifiers (1, 1), (1, 2), . (160, 120) and stored in the storage unit 11 . Further, the teacher feature quantity group acquisition means 1321 acquires the deterioration level “2” corresponding to the teacher image 1 using the teacher data 1, and converts the acquired deterioration level “2” to the acquired “160×120” are stored in the storage unit 11 in association with the teacher feature amount groups.

Next, the teacher feature quantity group acquisition means 1321 divides the teacher image 2 into “160×120” blocks, and from the “160×120” blocks, similar teacher feature quantity groups (H1, HR1, RB1), (H2, HR2, RB2), . 160, 120) and stored in the storage unit 11. Further, the teacher feature quantity group acquisition means 1321 acquires the deterioration level “4” corresponding to the teacher image 2 using the teacher data 2, and converts the acquired deterioration level “4” to the acquired “160×120” are stored in the storage unit 11 in association with the teacher feature amount groups.

As a result, in the storage unit 11, a set of pairs of teacher feature quantity groups and deterioration levels is formed as shown in FIG. The learning information acquisition means 1322 configures a first learner based on class classification and a second learner based on a regression equation by a machine learning algorithm using a set of pairs of teacher feature groups and deterioration levels. and stored in the learning information storage unit 112 . Since the first and second learners have been described above, detailed description thereof will not be repeated. The same is true for the third and fourth learners.

In addition, the learning information acquisition means 1322 provides a set of pairs of the teacher feature quantity group and the deterioration level, the first deterioration level output from the first learning device for each pair, and the Using the output second deterioration level, a third learning device is also configured by a machine learning algorithm, and stored in the learning information storage unit 112 .

Further, the learning information acquisition means 1322 selects the “160×120” first deterioration levels output from the first learning device and the “160×120” number from the second learning device for each pair. "160 × 120" output from the third learner for the "160 × 120" second deterioration levels and the "160 × 120" second deterioration levels A fourth learning device is configured by a machine learning algorithm using the first and second deterioration levels, and stored in the learning information storage unit 112 .

As a result, the learning information storage unit 112 stores the first to fourth learning devices. Instead of the first to fourth learning devices, the learning information storage unit 112 stores learning information composed of sets of pairs of teacher feature quantity groups and deterioration levels, as shown in FIG. may be stored.

Assume that the user then inputs an instruction to determine the deterioration level via an input device such as a keyboard. In the deterioration level determination device 1, the reception unit 12 receives the instruction. The determination target image acquisition unit 131 acquires the determination target image α from the group information storage unit 113 . The deterioration level obtaining unit 132 applies the obtained determination target image α to the learning information stored in the learning information storage unit 112 to obtain the deterioration level.

Specifically, the determination target feature amount group acquisition unit 1323 constituting the deterioration level acquisition unit 132 divides the determination target image α into “160×120” blocks, and from the “160×120” blocks, the teacher A feature quantity group (H1, HR1, RB1), (H2, HR2, RB2), . . . (H19200, HR19200, RB19200) is acquired. Then, the determination target feature quantity group acquisition unit 1323 obtains the acquired “160×120” determination target feature quantity groups as an image identifier “α” and block identifiers (1, 1), (1, 2), .・Store in the storage unit 11 in association with (160, 120).

The deterioration level acquisition means 1324 first inputs the “160×120” determination target feature amount group to each of the first and second learning devices, and obtains the first deterioration level output from the first learning device. Obtain the level and the second degradation level output from the second learner. Next, the deterioration level obtaining means 1324 inputs the obtained first and second deterioration levels to the third learner, and obtains the first or second deterioration level output from the third learner. .

In this example, from the third learner, "160×120" local deterioration levels "2.2", " Assume that 3.8", . . . "3.5" are output. The degradation level acquisition means 1324 stores the acquired “160×120” local degradation levels “2.2”, “3.8”, . (1, 1), (1, 2), .

The deterioration level acquisition means 1324 obtains the above "160×120" local deterioration levels "2.2", "3.8", . Input "3.5" into the fourth learner to obtain the global degradation level output from the fourth learner. In this example, it is assumed that the global deterioration level "3" is output from the fourth learning device. The first accumulation unit 133 associates the deterioration level “3”, which is the global deterioration level obtained by the deterioration level obtaining unit 1324 with the image identifier “α”, and accumulates them in the storage unit 11 .

As a result, in the storage unit 11, "160×120" determination target feature amount groups (H1, HR1, RB1), (H2, HR2, RB2), . The result is that a pair with a global deterioration level of "3" is stored.

The first accumulation unit 133 associates the determination target image α of the group information storage unit 113 with the global deterioration level “3” associated with the image identifier “α” and accumulates it, thereby storing group information. In section 113, determination result data having the determination target image α and the deterioration level “3”, which is the global deterioration level, is formed.

The output unit 14 acquires the determination result data from the group information storage unit 113 and outputs it via an output device such as a display. The display shows the determination target image α and the deterioration level "3", which is the determination result of the deterioration level determination device 1 for this image.

Since the deterioration level “3” acquired for the determination target image α satisfies the first condition, the reception unit 12 receives the user evaluation for the determination target image α via an input device such as a keyboard. In this example, it is assumed that user evaluation "2" is accepted.

The first storage unit 133 stores the received evaluation level “2” in the group information storage unit 113 in association with the determination result data. As a result, the grouped information 1 shown in FIG. 6 is configured in the grouped information storage unit 113 .

At this point, only group information 1 is stored in the group information storage unit 113 as group information having different deterioration levels and user evaluations, and the second determination unit 136 does not satisfy the second condition. judge not. Therefore, the reconstruction of the learning information by the reconstruction unit 137 is not performed at this point.

Next, the determination target image acquisition unit 131 acquires the determination target image β from the group information storage unit 113, and the deterioration level acquisition unit 132 applies the determination target image β to the learning information in the learning information storage unit 112, Get deterioration level.

Specifically, the determination target feature amount group acquisition unit 1323 divides the determination target image β into “160×120” blocks, and extracts the teacher feature amount group (H1, HR1, RB1), (H2, HR2, RB2), . . . (H19200, HR19200, RB19200) are acquired. Then, the determination target feature quantity group acquisition unit 1323 obtains the acquired “160×120” determination target feature quantity groups as an image identifier “β” and block identifiers (1, 1), (1, 2), .・Store in the storage unit 11 in association with (160, 120).

The deterioration level acquisition means 1324 acquires the first and second deterioration levels by inputting the “160×120” number of determination target feature quantity groups to the first and second learners, respectively, Next, the first or second deterioration level is obtained by inputting the obtained first and second deterioration levels to the third learning device.

In this example, from the third learner, "160 × 120" local deterioration levels "4" and "5" corresponding to "160 × 120" blocks, which are discrete first deterioration levels. , . . . "4" is output. The deterioration level obtaining means 1324 converts the obtained “160×120” local deterioration levels “4”, “5”, . (1, 2), .

The deterioration level obtaining means 1324 obtains the above-described “160×120” local deterioration levels “4”, “5”, . , into the fourth learner, and obtain the global degradation level output from the fourth learner. In this example, it is assumed that the global deterioration level "4" is output from the fourth learning device. The first storage unit 133 associates the acquired deterioration level “4”, which is the global deterioration level, with the image identifier “β” and stores them in the storage unit 11 .

As a result, in the storage unit 11, "160×120" determination target feature quantity groups (H1, HR1, RB1), (H2, HR2, RB2), . ” is stored.

The first storage unit 133 associates and stores the determination target image β in the group information storage unit 113 with the global deterioration level “4” associated with the image identifier “β”, thereby storing group information. In section 113, determination result data having the determination target image β and the deterioration level "4", which is the global deterioration level, is constructed.

The output unit 14 acquires the determination result data from the group information storage unit 113 and outputs it via an output device such as a display. The display shows the determination target image β and the deterioration level "4", which is the determination result for this image.

Note that since the deterioration level "4" acquired for the determination target image β does not satisfy the first condition, user evaluation for the determination target image β is not accepted.

The first storage unit 133 stores “Null” indicating that there is no user evaluation in the group information storage unit 113 in association with the determination result data. As a result, the grouped information 2 shown in FIG. 6 is configured in the grouped information storage unit 113 .

Even at this point, only the set information 1 is stored in the set information storage unit 113 as set information having different deterioration levels and user evaluations, and the second determination unit 136 does not satisfy the second condition. judge not. Therefore, the reconstruction of the learning information by the reconstruction unit 137 is not performed at this time either.

After that, the evaluation target image γ (not shown) is subjected to the same processing as described above, and group information 3 having the determination target image γ, the deterioration level “2”, and the user evaluation “3” is stored in the group information storage unit 113. If it is stored, the second determination unit 136 determines that the second condition is satisfied, and the reconstruction unit 137 stores the two sets of information with different deterioration levels and user evaluations stored in the set information storage unit 113. 1 and 3 are used to reconstruct the learning information.

Specifically, the reconstruction unit 137 inputs the determination target feature amount group acquired from the determination target image α to the first and second learning devices, and outputs the same user evaluation as “2” from the fourth learning device. With the determination target feature quantity group obtained from the determination target image γ outputting the global deterioration level “2” as input to the first and second learning devices, the fourth learning device outputs the user evaluation “ The first to fourth learners are reconfigured so that the same global deterioration level "3" as 3" is output.

As a result, the deterioration level determination device 1 can accurately determine even a deterioration level of threshold value 3 or less.

As described above, according to the present embodiment, the learning information storage unit 112 stores the learning information acquired from two or more teacher data corresponding to the teacher image of the deteriorated part of the building and the deterioration level, The deterioration level determination device 1 acquires a determination target image that is an image of a building to be determined, applies the determination target image to learning information, acquires a deterioration level, and outputs the deterioration level, thereby Deterioration level can be automatically determined.

Two or more pieces of teacher data are stored in the teacher data storage unit 111, and the degradation level determination device 1 sets two or more feature amounts from at least a part of the teacher image for each of the two or more teacher images. A teacher feature quantity group is acquired, and using the two or more acquired teacher feature quantity groups, learning information relating to the correspondence between the two or more feature quantities and the deterioration level is acquired and stored in the learning information storage unit 112 . Then, the deterioration level determination device 1 acquires a determination target feature amount group, which is a set of two or more feature amounts, from at least a part of the determination target image, applies the determination target feature amount group to learning information, and obtains a determination target feature amount. By acquiring the deterioration level corresponding to the quantity group, the deterioration level of the building can be automatically determined by learning.

In addition, the deterioration level determination device 1 uses the acquired two or more teacher feature amount groups to configure a learning device that receives two or more feature amounts as an input and outputs a deterioration level by a machine learning algorithm, and determines By inputting the target feature amount group into the learning device and acquiring the deterioration level output from the learning device, the deterioration level of the building can be automatically determined by machine learning using the learning device.

Further, in the set information storage unit 113, one or more sets of set information having determination result data corresponding to a determination target image and a deterioration level and user evaluation as a user's evaluation of the determination target image are stored. The level judging device 1 judges whether or not the judgment target image constituting one or more pieces of judgment result data stored in the group information storage unit 113 satisfies a predetermined first condition, and determines whether the first condition is met. Receiving user evaluations for each of the one or more determination target images that satisfy the first condition, and accumulating the received user evaluations for each of the one or more determination target images that satisfy the first condition in association with determination result data formed by the determination target images. By doing so, in the group information storage unit 113, one or more group information having determination result data including the determination target image that satisfies the first condition and the user's evaluation of the determination target image are configured, and the group information storage unit 113 Using one or more sets of information stored in , a machine learning algorithm reconfigures the learner. In this way, by reconstructing the learning device using the user evaluation of the determination target image that satisfies the first condition, it becomes possible to automatically determine with high accuracy the deterioration level of the building corresponding to the determination target image that satisfies the first condition. .

Further, the deterioration level determination device 1 determines whether or not the received user evaluation satisfies a predetermined second condition, and reconfigures the learning device when the user evaluation satisfies the second condition. In this way, when the user evaluation of the image to be determined that satisfies the first condition satisfies the second condition, the deterioration level determination device 1 can determine the first condition by reconstructing the learning device using the user evaluation. It becomes possible to automatically determine the deterioration level of the building corresponding to the target image with higher accuracy.

Further, since the first condition is that the deterioration level associated with the determination target image is equal to or less than the threshold, the deterioration level determination device 1 uses the user evaluation for the determination target image with the low deterioration level. By reconfiguring the learner using the method, it becomes possible to automatically determine the deterioration level of a building with a low degree of deterioration with high accuracy.

In addition, the second condition is that the number of pieces of grouped information in which the deterioration level and the user evaluation are different is greater than or equal to the threshold value and is stored in the grouped information storage unit 113. Therefore, the deterioration level determination device 1 If there are many user evaluations of the image to be determined that has a low deterioration level, and there are many user evaluations that are different from the deterioration level, the user evaluation can be used to reconfigure the learning device, so that the deterioration level of the building can be determined with higher accuracy. can be determined automatically.

Further, the deterioration level determination device 1 acquires two or more blocks that are part of the image to be determined, applies each of the two or more blocks to the learning information, and obtains a local deterioration level of two or more, which is the deterioration level of each block. level, and using two or more local impairment levels to obtain a global impairment level, which is the impairment level for two or more blocks. By combining local determination and global determination in this manner, efficient automatic determination can be performed.

Further, the deterioration level determination device 1 obtains two or more local deterioration levels by a machine learning algorithm, and obtains a global deterioration level by a machine learning algorithm, so that both local judgment and global judgment can be performed. It can be done automatically with machine learning.

Further, the deterioration level determination device 1 acquires the size of the deterioration area, which is the area corresponding to the deterioration location in the determination target image, using the local deterioration level of 2 or more, and uses the size of the deterioration area to To obtain the global deterioration level. In this way, by obtaining the size of the degraded region using the result of the local determination and using the size for global determination, more efficient automatic determination can be performed.

Further, the deterioration level determination device 1 obtains the size of the object in the teacher image, and obtains the global deterioration level using the size of the object and the size of the deteriorated region in the object. Thus, by using the size of the degraded area relative to the object, accurate automatic determination can be performed.

Further, the deterioration level determination device 1 automatically determines the cause of deterioration by determining which of two or more predetermined causes of deterioration is the cause of deterioration to be determined using the image to be determined. can.

Further, the deterioration level determination device 1 acquires the deterioration level by an algorithm corresponding to the degree of deterioration to be determined among two or more predetermined algorithms. judgment can be made.

In addition, the two or more algorithms include a first algorithm that acquires continuous values by a regression equation and a second algorithm that acquires discrete values by class classification, and the deterioration level determination device 1 is a discrete value by the second algorithm. Obtaining the first deterioration level, if the first deterioration level is equal to or less than the threshold or lower than the threshold, further obtaining the second deterioration level, which is a continuous value, by the first algorithm, and adopting the deterioration level corresponding to the second deterioration level. However, if the first deterioration level is higher than or equal to or greater than the threshold, the deterioration level corresponding to the first deterioration level is adopted. Therefore, when the deterioration of the building is light, detailed level determination can be performed by the first algorithm based on the regression equation, and when the deterioration of the building is severe, the second algorithm based on class classification can perform various classifications.

Furthermore, the processing in this embodiment may be realized by software. Then, this software may be distributed by software download or the like. Also, this software may be recorded on a recording medium such as a CD-ROM and distributed. Software that implements the information processing apparatus according to the present embodiment is, for example, the following program.

In other words, the computer-accessible recording medium includes a learning information storage unit 112 that stores learning information acquired from two or more pieces of teacher data corresponding to teacher images of deteriorated parts of buildings and deterioration levels. The program includes a determination target image acquisition unit 131 that acquires a determination target image that is an image of a building to be determined, and a deterioration level that is obtained by applying the determination target image to the learning information. This is a program for functioning as a deterioration level acquisition unit 132 to obtain the deterioration level and an output unit 14 for outputting the deterioration level.

FIG. 7 is an external view of a computer system 900 that implements the deterioration level determination device 1 by executing the program in this embodiment. The embodiments can be implemented by computer hardware and computer programs executed thereon. In FIG. 7, computer system 900 comprises computer 901 including disk drive 905 , keyboard 902 , mouse 903 and display 904 . The entire system including the keyboard 902, mouse 903, and display 904 may be called a computer.

FIG. 8 is a diagram showing an example of the internal configuration of the computer system 900. As shown in FIG. In FIG. 8, in addition to the disk drive 905, a computer 901 is connected to an MPU 911, a ROM 912 for storing programs such as a boot-up program, and the MPU 911 to temporarily store instructions of application programs and temporarily A RAM 913 that provides storage space, a storage 914 that stores application programs, system programs, and data, a bus 915 that interconnects the MPU 911, ROM 912, etc., and provides connections to networks such as external networks and internal networks. a network card 916; The storage 914 is, for example, a hard disk, SSD, flash memory, or the like.

A program that causes the computer system 900 to execute the functions of the deterioration level determination device 1 may be stored in a disk 921 such as a DVD or CD-ROM, inserted into the disk drive 905, and transferred to the storage 914. Alternatively, the program may be transmitted to computer 901 over a network and stored in storage 914 . Programs are loaded into RAM 913 during execution. Note that the program may be loaded directly from disk 921 or from the network. Also, the program may be read into the computer system 900 via another removable recording medium (eg, DVD, memory card, etc.) instead of the disk 921 .

The program does not necessarily have to include an operating system (OS) or a third-party program that executes the functions of the deterioration level determination device 1 in the computer details 901 . A program may contain only those portions of instructions that call the appropriate functions or modules in a controlled manner to produce the desired result. How the computer system 900 operates is well known and will not be described in detail.

Although the computer system 900 described above is a server or a stationary PC, the deterioration level determination device 1 may be realized by a portable terminal such as a tablet terminal, a smartphone, or a notebook PC. In this case, for example, the keyboard 902 and mouse 903 are replaced with a touch panel, the disk drive 905 with a memory card slot, and the disk 921 with a memory card. However, the above is an example, and the hardware configuration of the computer that implements the deterioration level determination device 1 is not a concern.

In the above program, the transmission step for transmitting information and the reception step for receiving information are performed by hardware. not included).

Also, the number of computers that execute the above programs may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

Further, in the above embodiment, two or more communication means (for example, the reception function of the reception unit 12 and the transmission function of the output unit 14, etc.) existing in one device are physically realized by one medium. It goes without saying that this is also a good thing.

Further, in each of the above embodiments, each process (each function) may be implemented by centralized processing by a single device (system), or may be implemented by distributed processing by a plurality of devices. may be

It goes without saying that the present invention is not limited to the above-described embodiments, and that various modifications are possible and are also included within the scope of the present invention.

INDUSTRIAL APPLICABILITY As described above, the deterioration level determination device according to the present invention has the effect of being able to automatically determine the deterioration level of a building, and is useful as a deterioration level determination device or the like.

1 deterioration level determination device 11 storage unit 12 reception unit 13 processing unit 14 output unit 111 teacher data storage unit 112 learning information storage unit 113 group information storage unit 131 determination target image acquisition unit 132 deterioration level acquisition unit 133 first accumulation unit 134 th First determination unit 135 Second accumulation unit 136 Second determination unit 137 Reconstruction unit 138 Cause determination unit 1321 Teacher feature quantity group acquisition means 1322 Learning information acquisition means 1323 Determination target feature quantity group acquisition means 1324 Deterioration level acquisition means

Claims (15)

a learning information storage unit storing a learner, which is learning information acquired by machine learning from two or more teacher data corresponding to teacher images of deteriorated parts of a building and deterioration levels;
a determination target image acquisition unit that acquires a determination target image that is an image of a building to be determined;
a deterioration level acquisition unit that applies the determination target image to the learning device and acquires a deterioration level;
an output unit that outputs the deterioration level;
a grouped information storage unit storing one or more grouped information having judgment result data corresponding to a judgment target image and a deterioration level, and user evaluation, which is a user evaluation of the judgment target image;
a first accumulation unit for accumulating judgment result data corresponding to the judgment target image and the deterioration level acquired by the deterioration level acquisition unit for the judgment target image in the group information storage unit;
a first determination unit that determines whether or not a determination target image that constitutes one or more determination result data stored in the group information storage unit satisfies a predetermined first condition;
a reception unit that receives user evaluations of the one or more determination target images that satisfy the first condition;
By accumulating the user evaluation received by the receiving unit for each of the one or more determination target images satisfying the first condition in association with the determination result data formed by the determination target images, the group information storage unit a second accumulation unit that configures one or more sets of information having determination result data including a determination target image that satisfies the first condition and a user evaluation of the determination target image;
and a reconstruction unit that reconstructs the learning device by a machine learning algorithm using one or more group information stored in the group information storage unit. further comprising a teacher data storage unit that stores the two or more teacher data,
The deterioration level acquisition unit,
a teacher feature quantity group acquiring means for acquiring a teacher feature quantity group, which is a set of two or more feature quantities, from at least a part of each of the two or more teacher images;
Acquisition of learning information relating to correspondence between the two or more feature amounts and deterioration levels using the two or more teacher feature amount groups acquired by the teacher feature amount group acquisition means, and accumulation of learning information in the learning information storage unit means and
Determination target feature quantity group acquisition means for acquiring a determination target feature quantity group, which is a set of two or more feature quantities, from at least a portion of the determination target image;
2. The deterioration level determination apparatus according to claim 1, further comprising deterioration level acquisition means for applying the determination target feature amount group to the learning information and acquiring a deterioration level corresponding to the determination target feature amount group. The learning information acquisition means is
Using two or more teacher feature quantity groups acquired by the teacher feature quantity group acquisition means, learning information is configured as a learning device that receives two or more feature quantities as input and outputs a deterioration level by a machine learning algorithm. death,
The deterioration level acquisition means is
3. The deterioration level determination device according to claim 2, wherein the determination target feature quantity group is input to the learning device, and the deterioration level output from the learning device is acquired. further comprising a second determination unit that determines whether the user evaluation received by the reception unit satisfies a predetermined second condition;
The reconstruction unit
The deterioration level determination device according to any one of claims 1 to 3, wherein the learning device is reconfigured when the user evaluation satisfies the second condition. 5. The deterioration level determination device according to any one of claims 1 to 4, wherein the first condition is that the deterioration level associated with the image to be determined is equal to or less than a threshold. 5. The deterioration level determination device according to claim 4, wherein the second condition is that the number of pieces of grouped information having different deterioration levels and user evaluations is greater than or equal to a threshold and stored in the grouped information storage unit. The deterioration level acquisition unit,
Acquiring two or more blocks that are part of the determination target image;
Applying each of the two or more blocks to the learning information to obtain two or more local degradation levels that are degradation levels for each block;
7. The degradation level determination device according to claim 1, wherein the two or more local degradation levels are used to obtain a global degradation level, which is a degradation level for two or more blocks. The deterioration level acquisition unit,
Obtaining the two or more local deterioration levels by a machine learning algorithm,
8. The deterioration level determination device according to claim 7, wherein the global deterioration level is acquired by a machine learning algorithm. The deterioration level acquisition unit,
obtaining the size of a degraded region corresponding to the degraded portion in the determination target image using the two or more local degradation levels;
8. The deterioration level determination device according to claim 7, wherein the size of the deterioration area is used to obtain the global deterioration level. The deterioration level acquisition unit,
obtaining the size of an object in the training image;
10. The deterioration level determination device according to claim 9, wherein the global deterioration level is acquired using the size of the object and the size of the deteriorated area in the object. 11. The method further comprising a cause determination unit that determines, using the image to be determined, which of two or more predetermined causes of deterioration the cause of deterioration to be determined is. The deterioration level determination device according to the item. The deterioration level acquisition unit,
12. The deterioration level determination device according to any one of claims 1 to 11, wherein the deterioration level is acquired by an algorithm corresponding to the degree of deterioration to be determined, from two or more predetermined algorithms. The two or more algorithms include a first algorithm that obtains continuous values by a regression equation and a second algorithm that obtains discrete values by classification,
The deterioration level acquisition unit,
Obtaining a first deterioration level, which is a discrete value, by the second algorithm;
If the first deterioration level is equal to or less than the threshold value or lower than the threshold value, further obtain a second deterioration level that is a continuous value by the first algorithm, adopt the deterioration level corresponding to the second deterioration level,
13. The deterioration level determination device according to claim 12, wherein when the first deterioration level is higher than or equal to or greater than a threshold, a deterioration level corresponding to the first deterioration level is adopted. A learning information storage unit that stores a learning device that is learning information acquired by machine learning from two or more teacher data that correspond to teacher images of deteriorated parts of a building and deterioration levels, and an image to be judged. A set information storage unit storing at least one piece of set information having determination result data corresponding to a deterioration level and user evaluation of the determination target image by a user, a determination target image acquisition unit, and deterioration level acquisition A deterioration level determination method realized by a unit, an output unit, a first storage unit, a first determination unit, a reception unit, a second storage unit, and a reconstruction unit,
a determination target image acquisition step in which the determination target image acquisition unit acquires a determination target image that is an image of a building to be determined;
a degradation level acquisition step in which the degradation level acquisition unit acquires a degradation level by applying the determination target image to the learning device;
an output step in which the output unit outputs the deterioration level;
The first accumulation unit stores determination result data corresponding to the determination target image and the deterioration level acquired by the deterioration level acquisition unit for the determination target image in the group information storage unit. a step;
A first judgment step in which the first judgment unit judges whether or not the judgment target image constituting one or more pieces of judgment result data stored in the group information storage unit satisfies a predetermined first condition. and,
a receiving step in which the receiving unit receives user evaluations of the one or more determination target images that satisfy the first condition;
The second storage unit stores the user evaluations received by the reception unit for each of the one or more determination target images that satisfy the first condition, in association with determination result data formed by the determination target images. a second accumulation step of configuring, in the group information storage unit, one or more group information having judgment result data including a judgment target image satisfying the first condition and a user evaluation of the judgment target image;
and a reconstruction step in which the reconstruction unit reconstructs the learner by a machine learning algorithm using one or more group information stored in the group information storage unit. A learning information storage unit that stores a learning device that is learning information acquired by machine learning from two or more teacher data that correspond to teacher images of deteriorated parts of a building and deterioration levels, and an image to be determined. a computer capable of accessing a paired information storage unit storing one or more pieces of paired information having determination result data corresponding to a deterioration level and user evaluation of the image to be determined;
a determination target image acquisition unit that acquires a determination target image that is an image of a building to be determined;
a deterioration level acquisition unit that applies the determination target image to the learning device and acquires a deterioration level;
an output unit that outputs the deterioration level;
a grouped information storage unit storing one or more grouped information having judgment result data corresponding to a judgment target image and a deterioration level, and user evaluation, which is a user evaluation of the judgment target image;
a first accumulation unit for accumulating judgment result data corresponding to the judgment target image and the deterioration level acquired by the deterioration level acquisition unit for the judgment target image in the group information storage unit;
a first determination unit that determines whether or not a determination target image that constitutes one or more determination result data stored in the group information storage unit satisfies a predetermined first condition;
a reception unit that receives user evaluations of the one or more determination target images that satisfy the first condition;
By accumulating the user evaluation received by the receiving unit for each of the one or more determination target images satisfying the first condition in association with the determination result data formed by the determination target images, the group information storage unit a second accumulation unit that configures one or more sets of information having determination result data including a determination target image that satisfies the first condition and a user evaluation of the determination target image;
A program for functioning as a reconfiguring unit that reconfigures the learning device by a machine learning algorithm using one or more pieces of tuple information stored in the tuple information storage unit.

Images