JP2021144657A - Information collection support program, information collection support method and information processing device - Google Patents

Abstract

To easily collect information for account information on foreign residents.SOLUTION: An information processing device 101 reads residence card information including a face image 501 stored on a residence card 500 (residence information collection). The information processing device 101 actuates a photographing function of the device to photograph a photographic image 502 including a user on completion of the read of the residence card information stored on the residence card 500. At this time, the information processing device 101 outputs a motion indication and determines whether an action corresponding to the motion indication is taken (natural person detection). The information processing device 101 performs face collation processing based upon the face image 501 included in read residence card information and the photographed photographic image 502 once a natural person is detected. Consequently, individual specific items are associated with the natural person, and it can be confirmed that the person who registers the residence information entered into the residence card 500 is the person who possesses the residence card 500.SELECTED DRAWING: Figure 5

Description

The present invention relates to an information collection support program, an information collection support method, and an information processing device.

In recent years, as a measure against money laundering and terrorist financing, identity verification has been tightened under the Act on Prevention of Transfer of Criminal Proceeds. For example, due to concerns about the unauthorized use of funds transferred by banks, etc., stricter deposit account management for foreign residents is being promoted.

As a prior art, the first photographic data is received from the user, information indicating one or more arbitrary poses is selected from the pose DB, and the second photographic data corresponding to the information indicating the selected pose is provided to the user. There is a personal authentication support device that requests submission, accepts second photo data from the user, and outputs the first photo data and the second photo data.

Japanese Unexamined Patent Publication No. 2012-014233

However, the conventional technique has a problem that it takes time and effort to collect information for account management of foreign residents.

In one aspect, it is an object of the present invention to make it possible to easily collect information for account management of foreign residents.

In one embodiment, when the input of the account information is accepted, the input account information is transmitted to the authentication system corresponding to the financial institution, and the authentication result indicating that the authentication is successful based on the account information from the authentication system. Is received, a screen for accepting the input of the residence card number is output, and when the input of the residence card number is accepted, the image of the user stored in the residence card, which is permitted to be read based on the residence card number, is included. An information collection support program that reads information, shoots a captured image including a user, and performs face matching processing based on the captured image of the user included in the read information and the captured image including the captured user. Is provided.

According to one aspect of the present invention, it is possible to easily collect information for managing the accounts of foreign residents.

FIG. 1 is an explanatory diagram showing a system configuration example of the information collection system 100 according to the embodiment. FIG. 2 is a block diagram showing a hardware configuration example of the information processing device 101. FIG. 3 is an explanatory diagram showing an example of the stored contents of the residence information management DB 130. FIG. 4 is a block diagram showing a functional configuration example of the information processing device 101. FIG. 5 is an explanatory diagram showing an example of information collection processing of the information processing apparatus 101. FIG. 6 is an explanatory diagram (No. 1) showing various screen examples displayed on the information processing apparatus 101. FIG. 7 is an explanatory diagram (No. 2) showing various screen examples displayed on the information processing apparatus 101. FIG. 8 is an explanatory diagram (No. 3) showing various screen examples displayed on the information processing apparatus 101. FIG. 9 is an explanatory diagram (No. 4) showing various screen examples displayed on the information processing apparatus 101. FIG. 10 is an explanatory diagram (No. 5) showing various screen examples displayed on the information processing apparatus 101. FIG. 11 is an explanatory diagram (No. 6) showing various screen examples displayed on the information processing apparatus 101. FIG. 12 is an explanatory diagram (No. 7) showing various screen examples displayed on the information processing apparatus 101. FIG. 13 is an explanatory diagram (No. 8) showing various screen examples displayed on the information processing apparatus 101. FIG. 14 is an explanatory diagram (No. 9) showing various screen examples displayed on the information processing apparatus 101. FIG. 15 is an explanatory diagram (No. 10) showing various screen examples displayed on the information processing apparatus 101. FIG. 16 is an explanatory diagram (No. 11) showing various screen examples displayed on the information processing apparatus 101. FIG. 17 is an explanatory diagram (No. 12) showing various screen examples displayed on the information processing apparatus 101. FIG. 18 is an explanatory diagram (No. 13) showing various screen examples displayed on the information processing apparatus 101. FIG. 19 is an explanatory diagram (No. 1) showing a screen example of the registration information confirmation screen. FIG. 20 is an explanatory diagram (No. 2) showing a screen example of the registration information confirmation screen. FIG. 21 is an explanatory diagram (No. 3) showing a screen example of the registration information confirmation screen. FIG. 22 is an explanatory diagram showing a screen example of the residence information detail screen. FIG. 23 is a flowchart (No. 1) showing an example of the information collection support processing procedure of the information processing apparatus 101. FIG. 24 is a flowchart (No. 2) showing an example of the information collection support processing procedure of the information processing apparatus 101. FIG. 25 is a flowchart (No. 3) showing an example of the information collection support processing procedure of the information processing apparatus 101. FIG. 26 is a flowchart showing an example of a specific processing procedure of the face matching process.

Hereinafter, embodiments of the information collection support program, the information collection support method, and the information processing apparatus according to the present invention will be described in detail with reference to the drawings.

(Embodiment)
First, the information collection system 100 including the information processing device 101 according to the embodiment will be described. The information collection system 100 is a computer system that collects residence information of foreign residents. The resident foreigner is, for example, a foreigner residing in Japan.

As a measure against money laundering and terrorist financing, stricter account management of foreign residents is being promoted. For example, from the viewpoint of continuous customer management, the residence information of foreign residents who have an account is collected and managed. Is required to do. The residence information is information on foreign residents, such as the status of residence and the period of stay. The account is, for example, a bank deposit account.

On the other hand, in the past, foreign residents often did not collect residence information when they opened an account. Therefore, in order to confirm the residence information of foreign residents who have already opened an account, it is necessary to collect the residence information in some way. As an existing collection method, information for confirming residence information is sent by mail to foreign residents who have already opened an account.

For example, a bank employee narrows down the accounts of foreign residents from the deposit accounts and then asks the foreign residents, "Please submit a residence card to confirm your residence information, or Please come to the store "by mail. Foreign residents will submit a residence card or visit the store in response to this guidance.

The residence card is an identification card issued to foreigners who stay in Japan for a medium to long term. In addition to a photo of your face, the residence card will contain information such as your name, date of birth, gender, nationality / region, status of residence, period of stay, and whether or not you can work. The residence card has a built-in IC (Integrated Circuit) chip to prevent counterfeiting. All or part of the items written on the card surface are recorded on the IC chip.

However, the existing collection method has a problem that the reply rate and the visit rate are low because it takes time and effort for foreign residents to reply to the residence card by mail or to visit the store. In addition, there is a problem that the cost of reply envelopes and the like and the cost of employees for dealing with customers when visiting the store increase on the bank side as well.

Therefore, if information for managing the accounts of foreign residents can be easily collected using a smartphone or the like, it will be convenient for foreign residents and it will lead to cost reduction on the bank side. For example, it is conceivable to use a smartphone to collect residence information such as the status of residence and the period of stay written on the residence card.

Here, in collecting the residence information written on the residence card, it is necessary to confirm that the residence card belongs to the person himself / herself, not the found item. If this is a face-to-face meeting, for example, a bank employee receives a residence card and confirms that the person who visited the store is the same as the photograph of the face of the residence card. You can check. However, in the case of non-face-to-face, such a series of operations disappears, so a mechanism for appropriately confirming whether the user is the person who holds the residence card (legitimate holder) is required.

Therefore, in the present embodiment, when a resident foreigner registers residence information using a smartphone or the like, he / she can easily confirm whether he / she is the person who holds the residence card (legitimate holder). , Explains the information collection support method that makes it possible to easily collect information for account management of foreign residents.

Here, a system configuration example of the information collection system 100 will be described.

FIG. 1 is an explanatory diagram showing a system configuration example of the information collection system 100 according to the embodiment. In FIG. 1, the information collection system 100 includes an information processing device 101, an account authentication server 102, an information collection server 103, a collation server 104, and a business terminal 105. In the information collection system 100, the information processing device 101, the account authentication server 102, the information collection server 103, the collation server 104, and the business terminal 105 are connected via a wired or wireless network 110. The network 110 is, for example, the Internet, LAN, WAN (Wide Area Network), or the like.

Here, the information processing device 101 is a computer used by the user. A user is a person who conducts a transaction with a business operator. For example, a business operator is a financial institution such as a bank. In this case, the user is a customer who uses a financial institution. The information processing device 101 is, for example, a smartphone, a tablet PC (Personal Computer), or the like.

The account authentication server 102 is a computer provided corresponding to a financial institution and executing an account authentication process. The account authentication server 102 has an account information management DB (Database) 120. The account information management DB 120 stores the account information of the information collection target person. The information collection target is a foreign resident who has an account (deposit account) and whose residence information has not been registered.

In the account information management DB 120, the account information is managed by using the identification key. The identification key is an identifier that uniquely identifies the account information. The account information includes, for example, an account number, a katakana surname, a katakana name, a date of birth, a store number, a subject, a personal identification number, and the like. The account authentication process is a process for authenticating that the account information to be processed is the account of the information collection target person (foreign resident).

The account authentication process is realized by, for example, an API (Application Programming Interface) for authentication. The account authentication server 102 may be, for example, a server of a financial institution, or may be a server of a service provider that provides an authentication service to a financial institution.

The information collection server 103 is a computer provided corresponding to a financial institution and collects residence information. The information collection server 103 has a residence information management DB 130. The stored contents of the residence information management DB 130 will be described later with reference to FIG. The information collection server 103 may be, for example, a server of a financial institution, or may be a server of a service provider that provides an information collection service to a financial institution.

The collation server 104 is a computer that executes a collation process. In the collation process, for example, images are collated to calculate a collation rate indicating the degree of similarity between the images. The collation process is realized by, for example, a collation API. The collation server 104 may be, for example, a server of a financial institution, or may be a server of a service provider that provides a collation service to a financial institution.

The business terminal 105 is a computer used by the staff of the business. The employee of the business operator is, for example, a bank employee. The business terminal 105 is, for example, a PC, a tablet PC, or the like. At the business terminal 105, the bank employee can access the information collection server 103 and confirm, for example, the residence information of the collected information collection target person (foreign resident).

In the example of FIG. 1, only one information processing device 101 is shown, but the present invention is not limited to this. For example, the information processing device 101 is provided for each user. Further, only one each of the account authentication server 102, the information collection server 103, the verification server 104, and the business terminal 105 is described, but the present invention is not limited to this. For example, the account authentication server 102, the information collection server 103, the verification server 104, and the business terminal 105 are provided for each financial institution such as a bank.

(Example of hardware configuration of information processing device 101)
Next, a hardware configuration example of the information processing device 101 will be described.

FIG. 2 is a block diagram showing a hardware configuration example of the information processing device 101. In FIG. 2, the information processing device 101 includes a CPU (Central Processing Unit) 201, a memory 202, a camera 203, a communication I / F (Interface) 204, a display 205, an input device 206, and a reading device 207. Has. Further, each component is connected by a bus 200.

Here, the CPU 201 controls the entire information processing device 101. The memory 202 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash ROM, and the like. Specifically, for example, a flash ROM or ROM stores various programs, and RAM is used as a work area of CPU 201. The program stored in the memory 202 is loaded into the CPU 201 to cause the CPU 201 to execute the coded process.

The camera 203 is a photographing device that captures an image (still image or moving image) and outputs image data. The camera 203 is, for example, an in-camera mounted on the display 205 side or an out-camera mounted on the back side of the display 205.

The communication I / F 204 is connected to the network 110 through a communication line, and is connected to another device (for example, the account authentication server 102, the information collection server 103, the verification server 104, etc. shown in FIG. 1) via the network 110. .. Then, the communication I / F 204 controls the interface between the network 110 and the inside of the own device, and controls the input / output of data from another device.

The display 205 displays data such as documents, images, and functional information, as well as cursors, icons, and toolboxes. The display 205 can function as, for example, a screen for displaying a subject. As the display 205, for example, a liquid crystal display, an organic EL (Electroluminescence) display, or the like can be adopted.

The input device 206 has keys for inputting characters, numbers, various instructions, and the like, and inputs data. The input device 206 may be a keyboard, a mouse, or the like, or may be a touch panel type input pad, a numeric keypad, or the like. The reading device 207 is a device that reads data by short-range wireless communication. For example, the reading device 207 is an NFC (Near Field Communication) reader / writer device.

In addition to the above-described components, the information processing device 101 includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), a portable recording medium I / F, a GPS (Global Positioning System) unit, a microphone, and the like. It may have a speaker or the like.

Further, the business terminal 105 shown in FIG. 1 can also be realized by the same hardware configuration as the information processing device 101. Further, the account authentication server 102, the information collection server 103, and the collation server 104 are realized by, for example, a hardware configuration such as a CPU, a memory, a communication I / F, an HDD, and a portable recording medium I / F.

(Memory contents of residence information management DB 130)
Next, the stored contents of the residence information management DB 130 included in the information collection server 103 will be described with reference to FIG. The residence information management DB 130 is realized by, for example, a storage device such as a memory or an HDD of the information collection server 103.

FIG. 3 is an explanatory diagram showing an example of the stored contents of the residence information management DB 130. In FIG. 3, the residence information management DB 130 has fields for registration time, katakana name, status of residence, period of stay, identification key, and detailed information. By setting information in each field, residence information management data (for example, residence information management data 300-1, 300-2) is stored as a record.

Here, the registration time indicates the date and time when the residence information of the information collection target person (foreign resident) was registered. The katakana name indicates the katakana name of a foreign resident. The status of residence is one of the status of residence and indicates the status of residence of foreign residents. The period of stay is one of the information on residence and indicates the period of stay of a foreign resident.

The identification key is an identifier that uniquely identifies the account held by a foreign resident. The account information of foreign residents can be identified from the identification key. The detailed information is detailed information of the foreign resident, for example, information on all or part of the matters written on the residence card, information on the purpose of transaction, and the like.

(Example of functional configuration of information processing device 101)
FIG. 4 is a block diagram showing a functional configuration example of the information processing device 101. In FIG. 4, the information processing device 101 includes a reception unit 401, an account authentication unit 402, a display control unit 403, a reading unit 404, an imaging control unit 405, a collation unit 406, and a communication unit 407. .. The reception units 401 to the communication units 407 are functions that serve as control units. Specifically, for example, by causing the CPU 201 to execute a program stored in the memory 202 shown in FIG. 2, or by having the CPU 201 execute the communication I / F 204. To realize the function. The processing result of each functional unit is stored in, for example, the memory 202.

The reception unit 401 accepts the input of account information. The input account information is, for example, the account information of the information collection target person (foreign resident). The account information includes, for example, an account number, a katakana surname, a katakana name, a date of birth, a store number, a subject, a personal identification number, and the like. Specifically, for example, the reception unit 401 accepts the input of account information by the user's operation input using the input device 206 shown in FIG.

The account information is input, for example, on the account information input screen displayed on the display 205 shown in FIG. The account information input screen is an operation screen that accepts input of account information. A screen example of the account information input screen will be described later with reference to FIG. 7.

The account authentication unit 402 transmits the input account information to the authentication system corresponding to the financial institution. The authentication system corresponding to a financial institution is a system that authenticates that the account information to be processed is the account of the information collection target person (foreign resident), and is, for example, the account authentication server 102 shown in FIG. .. Specifically, for example, the account authentication unit 402 transmits an authentication request including the input account information to the account authentication server 102.

When the account authentication server 102 receives the authentication request from the information processing device 101, the account information management DB 120 is referred to, and whether or not the account information of the resident foreigner corresponding to the account information included in the received authentication request exists. To judge. The account information of the resident foreigner corresponding to the account information included in the authentication request is, for example, the account information having the same account number, katakana surname, katakana name, date of birth, store number, subject, and PIN.

Here, if the corresponding foreign resident account information exists, the account authentication server 102 transmits an authentication result indicating that the authentication was successful based on the account information to the information processing device 101. When the account authentication unit 402 receives the authentication result indicating that the authentication is successful based on the account information from the account authentication server 102, the account authentication unit 402 authenticates the input account information.

On the other hand, when the corresponding foreign resident account information does not exist, the account authentication server 102 transmits an authentication result to the effect that the authentication has failed based on the account information to the information processing device 101. When the account authentication unit 402 receives the authentication result indicating that the authentication has failed based on the account information from the account authentication server 102, the account authentication unit 402 does not authenticate the input account information.

As a result, it is possible to authenticate whether the entered account information is the information of the legitimate account of the information collection target person (foreign resident). The authentication result indicating that the authentication was successful based on the account information includes, for example, an identification key for the account information.

When the display control unit 403 receives the authentication result indicating that the authentication is successful based on the account information from the authentication system, the display control unit 403 outputs a screen for accepting the input of the residence card number. Specifically, for example, the display control unit 403 displays the residence card number input screen on the display 205 when the authentication result indicating that the authentication is successful is received from the account authentication server 102 based on the account information.

The residence card number input screen is an operation screen that accepts the input of the residence card number required for reading data from the IC chip built in the residence card. A screen example of the residence card number input screen will be described later with reference to FIG.

Further, when the display control unit 403 receives an authentication result indicating that the authentication has failed based on the account information from the account authentication server 102, for example, the display control unit 403 includes a message indicating that the authentication of the account information has failed and the account information. A message prompting re-entry may be displayed on the display 205.

Upon receiving the input of the residence card number, the reading unit 404 reads the residence card information including the image of the user stored in the residence card, which is permitted to be read based on the residence card number. The residence card information is, for example, information recorded on an IC chip built in the residence card. Residence card information includes all or part of the information on the residence card.

For example, the residence card information includes specific personal information, residence information, electronic signature, public key, etc., in addition to the user's image (face image). The specific personal information and residence information are stored as, for example, an image of a residence card (excluding a face image). The electronic signature and public key are for preventing forgery and falsification of the IC chip. In addition, the residence card information may include text information indicating activities outside the status of qualification, the changed address, and the like.

Reading of the residence card information is permitted, for example, when the entered residence card number matches the number stored in advance in the IC chip. On the other hand, if the entered residence card number does not match the number stored in advance in the IC chip, reading of the residence card information is not permitted. Specifically, for example, when the user touches the residence card on the back side (NFC portion) of the device main body, the entered residence card number is checked. When reading is permitted, the reading unit 404 reads the residence card information including the face image from the residence card by the reading device 207 (see FIG. 2).

Further, the reading unit 404 may determine whether or not the read residence card information is valid. Specifically, for example, the reading unit 404 gives a hash function a combination of the image of the user included in the read residence card information and the image of the residence card to acquire the first hash value. The reading unit 404 decrypts the electronic signature included in the read residence card information with the public key to acquire the second hash value.

Then, the reading unit 404 compares the first hash value with the second hash value, and when the first hash value and the second hash value match, the read residence card information is forged (forged) ( It is judged that it has not been forged or altered (it is legitimate). On the other hand, when the first hash value and the second hash value do not match, the reading unit 404 determines that the read residence card information is forged (not valid).

The shooting control unit 405 captures a captured image including the user. Specifically, for example, when the residence card information stored in the residence card is read, the photographing control unit 405 activates the photographing function of the own device and uses the camera 203 shown in FIG. 2 to display the user. Take a picture including. At this time, the shooting control unit 405 may shoot a still image including the user, or may shoot a moving image including the user.

However, the shooting timing of the shot image including the user is not limited to this. For example, when it is determined that the read residence card information is not forged (legitimate), the shooting control unit 405 activates the shooting function of its own device, and the camera 203 takes a picture including the user. You may take an image. Further, when the photographing control unit 405 receives the authentication result indicating that the authentication is successful based on the account information from the authentication system, the photographing control unit 405 activates the photographing function of the own device before outputting the screen for accepting the input of the residence card number. Then, the camera 203 may capture a captured image including the user.

Further, the display control unit 403 may display a guide for positioning the face on the screen for displaying the subject when shooting a captured image including the user. The screen for displaying the subject is displayed on the display 205, for example. The guide may be a figure indicating the position of the face on the screen, or may be a combination of characters and symbols.

As a result, the user can easily take a face image of himself / herself by, for example, aligning the position of the face on the screen displaying the subject according to the guide displayed on the display 205. In the following description, the screen displaying the subject may be referred to as a "finder screen".

The collation unit 406 performs face collation processing based on the image of the user included in the read residence card information and the photographed image including the photographed user. Here, the face matching process is a process for confirming that the user (operator) is the person (legitimate holder) who holds the residence card.

Specifically, for example, the collation unit 406 shows a collation request including a user image (face image) included in the read residence card information and a photographed image including the photographed user in FIG. It is transmitted to the collation server 104. When a moving image including a user is shot as a captured image, the collating unit 406 may select one of the frame images (for example, a front-facing image) from the moving image as the captured image including the user. good.

When the collation server 104 receives the collation request from the information processing device 101, the collation server 104 detects a face from the captured image included in the received collation request and extracts the face image from the captured image. Next, the collation server 104 collates the face image included in the received collation request with the extracted face image to calculate the collation rate.

At this time, the collation server 104 may also calculate the false acceptance rate based on the calculated collation rate. The false acceptance rate is the rate at which a person is recognized as the person even though he / she is not the person. As the technique for calculating the collation rate and the false acceptance rate, for example, any existing technique based on machine learning such as deep learning may be used.

Then, the collation server 104 transmits the collation result including the calculated collation rate (or the collation rate and the false acceptance rate) to the information processing apparatus 101. Upon receiving the collation result from the collation server 104, the collation unit 406 determines whether or not the collation rate included in the received collation result is equal to or greater than the threshold value α. The threshold value α can be arbitrarily set, and is set to a value of, for example, about 80 [%].

Here, if the collation rate is equal to or higher than the threshold value α, the collation unit 406 determines that the face collation was successful. On the other hand, if the collation rate is less than the threshold value α, the collation unit 406 determines that the face collation has failed. Here, the case where the face matching is requested to the matching server 104 has been described as an example, but the present invention is not limited to this. For example, the collation unit 406 may execute a collation process of collating face images with each other to calculate a collation rate.

As a result, when the face matching is successful in the face matching process, it is possible to confirm that the user is the person who holds the residence card. On the other hand, if the face matching fails in the face matching process, it can be determined that the user is not the person who holds the residence card.

When a moving image including a user is shot as a captured image, the collating unit 406 performs face matching processing based on the image of the user included in the residence card information and the moving image including the captured user. You may decide. Specifically, for example, the collation unit 406 selects a plurality of frame images from the captured moving image.

The algorithm for selecting a plurality of frame images can be set arbitrarily. For example, the collation unit 406 may randomly select a plurality of frame images from the moving image, select every other frame image, or select an image in which a face is detected.

Next, the collation unit 406 performs face collation processing based on the result of collating each of the selected plurality of frame images with the face image included in the residence card information. More specifically, for example, the collation unit 406 may make the collation successful when the collation rate of at least one of the plurality of frame images is equal to or greater than the threshold value α.

As a result, the accuracy of confirmation that the residence card belongs to the user can be improved as compared with the case where one image (still image) is taken and face matching is performed. For example, it is possible to prevent a collation failure due to a low collation rate due to an image in which the face is poorly reflected even though it belongs to the person himself / herself.

Alternatively, the collation unit 406 may make the collation successful when all the collation rates obtained by collating each of the plurality of frame images with the face image are equal to or greater than the threshold value α. As a result, it is possible to prevent the collation rate from happening to be high and the collation to be successful even though the person is another person.

Further, the display control unit 403 may output a motion instruction when shooting a moving image (captured image) including the user. Here, the motion instruction is an instruction to instruct the subject to perform a specific motion (motion). Specifically, for example, the display control unit 403 may output a motion instruction randomly selected from a plurality of motion instructions.

The motion instruction is given by, for example, an image displayed on the finder screen or a voice from a speaker (not shown). Specific motions include, for example, blinking, opening and closing of the mouth, nodding, and waving. This makes it possible to instruct the user to perform motions such as blinking, opening and closing the mouth, nodding, and shaking the face.

An example of motion instruction output will be described later with reference to FIG.

The collation unit 406 may determine whether or not an operation corresponding to the output motion instruction is performed based on the moving image including the captured user. For example, if the specific motion is "blinking", the collating unit 406 determines whether or not "blinking" is performed based on the captured moving image.

More specifically, for example, the collation unit 406 states that "blinking" is performed when images of "eyes open ⇒ eyes closed ⇒ eyes open" are detected in order. to decide. Further, for example, if the specific motion is "face swing", the collation unit 406 will perform "face swing" if the face is moving in the order of "leftward ⇒ front ⇒ right ⇒ front" based on the captured video. Is determined to be performed. As a technique for determining whether or not the subject is performing a specific motion from the moving image, any existing technique may be used.

Then, the collating unit 406 may perform the face collating process based on the determination result of whether or not the operation corresponding to the motion instruction is performed. Specifically, for example, when the collation unit 406 determines that the operation corresponding to the motion instruction is performed, the image of the user included in the read residence card information and the image including the photographed user are photographed. Face matching processing is performed based on the image. On the other hand, when it is determined that the operation corresponding to the motion instruction is not performed, the collating unit 406 does not perform the face collating process.

As a result, if the operation corresponding to the motion instruction is not performed, the phase does not shift to the face matching phase. Therefore, for example, fraudulent acts such as taking a face photo of the residence card and using it as a photographed image of the user are prevented. be able to. Further, by randomly selecting and outputting the motion instruction, for example, it is possible to make it difficult to predict the output motion instruction and prevent a moving image corresponding to the motion instruction from being prepared in advance.

The information processing device 101 may output the motion instruction and determine whether or not the operation corresponding to the motion instruction is performed repeatedly a plurality of times while changing the motion instruction.

When the face matching process is successful, the display control unit 403 outputs a screen for accepting the input of residence information. The residence information is, for example, information that specifies the status of residence and the period of stay of a foreign resident, and is written on the surface of the residence card. Specifically, for example, the display control unit 403 displays the residence information input screen on the display 205 when the face matching process is successful.

The residence information input screen is an operation screen that accepts the input of residence information. A screen example of the residence information input screen will be described later with reference to FIG. The residence information may be obtained by performing OCR processing on the image of the residence card included in the read residence card information, for example. In this case, the residence information obtained by OCR (Optical Character Reader) processing is automatically input. The OCR process may be executed in the information processing apparatus 101, or may be executed in an OCR server (not shown) accessible to the information processing apparatus 101.

When the communication unit 407 receives the input of the residence information, the communication unit 407 transmits the input residence information to the information collection system corresponding to the financial institution in association with the input account information. The information management system is a system that collects residence information of an information collection target person (foreign resident), and is, for example, the information collection server 103 shown in FIG.

Specifically, for example, the communication unit 407 transmits a registration request including the input identification key of the account information and the input residence information to the information collection server 103. The identification key of the account information is an identifier that uniquely identifies the account information, and is included in, for example, the authentication result indicating that the authentication was successful based on the account information. However, the registration request may include the entered account information itself.

When the information collection server 103 receives the registration request from the information processing device 101, the information collection server 103 associates it with the identification key of the account information included in the received registration request and associates it with the residence information (for example, status of residence, period of stay) included in the registration request. Is registered in the residence information management DB 130 as shown in FIG.

As a result, new residence information management data is stored as a record in the residence information management DB 130. As the registration time of the new residence information management data, for example, the date and time when the information collection server 103 receives the registration request from the information processing device 101 is set.

Further, when the communication unit 407 receives the input of the residence information, the communication unit 407 may transmit the input residence information and the read residence card information to the information management system in association with the input account information. good.

Specifically, for example, the communication unit 407 requests registration including the input identification key of the account information, the input residence information, and the face image and the residence card image included in the read residence card information. Is transmitted to the information collection server 103. The face image and the image of the residence card may be transmitted with the face image embedded in the image of the residence card.

In this case, when the information collection server 103 receives the registration request from the information processing device 101, the information collection server 103 associates with the identification key of the account information included in the received registration request, and associates the residence information, the face image, and the residence card included in the registration request. The image of is registered in the residence information management DB 130. The face image and the image of the residence card are stored as detailed information of the residence information management data, for example.

Further, when the face matching fails in the face matching process, the communication unit 407 may transmit information indicating that the face matching has failed to the information management system. Specifically, for example, the communication unit 407 transmits alert information including the input identification key of the account information and information indicating that the face matching has failed to the information collection server 103.

As a result, when registering the residence information for the account information identified by the identification key, it is possible to notify an alert indicating that the face verification has failed.

In addition, when it is determined that the residence card information read from the residence card is not valid, the communication unit 407 provides information indicating that the residence card information read from the residence card is forged (invalid). , May be sent to the information management system. Specifically, for example, the communication unit 407 collects alert information including the input identification key of the account information and the information indicating that the residence card information read from the residence card is forged. Send to server 103.

As a result, with respect to the account information identified by the identification key, it is possible to notify an alert indicating that a fake residence card has been used when registering the residence information.

Further, when the communication unit 407 receives an authentication result indicating that the authentication has failed based on the account information from the authentication system, the communication unit 407 transmits information indicating that the authentication has failed to the input account information to the information management system. You may decide.

Further, the display control unit 403 may output a guide for photographing the back surface of the residence card when the face matching is successful in the face matching process. On the back of the residence card, for example, activities outside the status of qualification or the changed address may be written. An output example of a guide for photographing the back surface of the residence card will be described later with reference to FIG.

When an image including the back side of the residence card is taken, the communication unit 407 associates the input account information with the input residence information and the image including the back side of the taken residence card in the information management system. It may be sent to. Specifically, for example, the communication unit 407 transmits a registration request including an input identification key for account information, the input residence information, and an image including the back side of the residence card to the information collection server 103. ..

In this case, when the information collection server 103 receives the registration request from the information processing device 101, the information collection server 103 associates the identification key of the account information included in the received registration request with the residence information included in the registration request and the back side of the residence card. The included image is registered in the residence information management DB 130. The image including the back surface of the residence card is stored as detailed information of the residence information management data, for example.

Further, the display control unit 403 may output information prompting confirmation between the read residence card information and the image including the back surface of the photographed residence card after photographing the image including the back surface of the residence card. .. An example of outputting information for prompting confirmation of the residence card information and the image including the back surface of the residence card will be described later with reference to FIG.

Further, when the face matching is successful in the face matching process, the display control unit 403 may further output a screen for accepting input of information regarding the user's transaction purpose. Here, the information regarding the purpose of transaction is information that specifies the main purpose of use of the account. Specifically, for example, the display control unit 403 displays the transaction purpose input screen on the display 205 when the face matching is successful in the face matching process. The transaction purpose input screen is an operation screen that accepts input of information regarding the transaction purpose of the user. A screen example of the transaction purpose input screen will be described later with reference to FIG.

When the communication unit 407 receives the input of the information regarding the transaction purpose, the communication unit 407 transmits the input residence information and the input information regarding the transaction purpose to the information management system in association with the input account information. Specifically, for example, the communication unit 407 transmits a registration request including the input identification key of the account information, the input residence information, and the input information regarding the transaction purpose to the information collection server 103. ..

In this case, when the information collection server 103 receives the registration request from the information processing device 101, the information collection server 103 associates the identification key of the account information included in the received registration request with the residence information included in the registration request and information on the transaction purpose. , Register in the residence information management DB 130. Information regarding the transaction purpose is stored as detailed information of residence information management data, for example.

In the above description, when the account authentication unit 402 authenticates the input account information, the account information including the account number, the kana surname, the kana name, the date of birth, the store number, the subject, the password, etc. is used. I decided to send it to the account authentication server 102, but it is not limited to this.

For example, the reception unit 401 may accept the input of the reception number and the access key notified from the financial institution to the information collection target person (foreign resident) as account information. The receipt number is a number associated with the account number of the account existing in the financial institution. The reception number associated with the account number is registered in, for example, the account information management DB 120.

The receipt number and access key can be obtained, for example, by reading a QR (Quick Response) code attached to a document mailed from a financial institution to an information collection target person (foreign resident). The QR code is a registered trademark. Then, the account authentication unit 402 may transmit the input reception number and access key as account information to the account authentication server 102.

In this case, if the received reception number is registered in the account information management DB 120 and the received access key is determined to be valid, the account authentication server 102 authenticates that the authentication is successful based on the account information. The result is transmitted to the information processing device 101. On the other hand, if the reception number is not registered or the access key is determined to be invalid, the account authentication server 102 transmits an authentication result indicating that the authentication has failed based on the account information to the information processing device 101. ..

The validity of the access key is, for example, the information obtained by encrypting the received reception number with the received access key and the information obtained by encrypting the received reception number with the valid key (formal access key). It is judged by whether or not to do it. The valid key is stored in the account information management DB 120 in association with the reception number, for example.

Specifically, for example, when the received reception number is registered in the account information management DB 120, the account authentication server 102 acquires a valid key corresponding to the reception number from the account information management DB 120. Then, the account authentication server 102 determines whether or not the information obtained by encrypting the received reception number with the received access key and the information obtained by encrypting the received reception number with the acquired valid key match. .. Here, if the encrypted information matches, the account authentication server 102 determines that the access key is valid. On the other hand, if the encrypted information does not match, the account authentication server 102 determines that the access key is not valid.

Further, the information processing apparatus 101 may determine whether the account information and the residence information are the same person's information when transmitting the residence information to the information collection server 103 in association with the account information. For example, the information processing apparatus 101 determines that the information is the same person when the name (Kana name) and the date of birth match in the account information and the residence information.

Here, the name (Kana name) and the date of birth corresponding to the account information are input as, for example, one piece of information in the account information. In addition, the name (Kana name) and date of birth corresponding to the residence information can be obtained, for example, by performing OCR processing on the image of the residence card read from the IC chip.

Then, when the information processing device 101 determines that the information is the same person, the information processing device 101 may transmit the residence information to the information collection server 103 in association with the account information. On the other hand, if it is determined that the information is not the same person, the information processing apparatus 101 may not transmit the residence information to the information collection server 103 in association with the account information.

If it is determined that the information is not the same person, the information processing device 101 may transmit the residence information and the information indicating that the information is not the same person to the information collection server 103 in association with the account information. .. As a result, it is possible to prevent the residence information of another person different from the person who holds the account from being registered in association with the account information.

Further, the information processing device 101 may check the residence card and the expiration date of the period of stay. The expiration date of the residence card is written at a predetermined position on the residence card, and can be obtained by performing OCR processing on the image of the residence card read from the IC chip. Specifically, for example, the information processing apparatus 101 may check that the expiration date or the expiration date of the residence card is not a past date. Further, the expiration date check result may be transmitted to the information collection server 103 in association with the account information.

Further, the information processing apparatus 101 transmits the input residence information and the residence information included in the read residence card information to the information collection server 103, and includes the input residence information and the read residence card information. The result of comparison with the residence information may be transmitted to the information collection server 103. The residence information included in the residence card information can be obtained by performing OCR processing on the image of the residence card read from the IC chip. This makes it possible to prevent unauthorized registration of residence information.

(Example of information collection processing of information processing device 101)
Next, an example of information collection processing of the information processing apparatus 101 will be described with reference to FIG.

FIG. 5 is an explanatory diagram showing an example of information collection processing of the information processing apparatus 101. In FIG. 5, the information processing device 101 reads the residence card information including the face image 501 stored in the residence card 500. The residence card information includes, in addition to the face image 501, an image of the residence card, an electronic signature, a public key, and the like. As a result, residence information can be collected.

When the information processing device 101 completes reading the residence card information stored in the residence card 500, the information processing device 101 activates the shooting function of the own device and shoots a shot image 502 including the user with the camera 203 (see FIG. 2). (Shooting with a camera). At this time, the information processing device 101 outputs a motion instruction and determines whether or not an operation corresponding to the motion instruction is performed. As a result, it is possible to detect a natural person by detecting the motion as instructed.

When a natural person is detected, the information processing device 101 performs face matching processing based on the face image 501 included in the read residence card information and the captured image 502. As a result, it is possible to confirm that the person who registers the residence information described in the residence card 500 by associating the personal identification matter with the natural person is the person who holds the residence card 500.

(Examples of various screens displayed on the information processing device 101)
Next, various screen examples displayed on the display 205 of the information processing apparatus 101 will be described with reference to FIGS. 6 to 18.

6 to 18 are explanatory views showing various screen examples displayed on the information processing apparatus 101. In FIG. 6, the language selection screen 600 is an example of an operation screen for selecting a language. The language selection screen 600 is displayed, for example, when the residence information collection application (corresponding to this information collection support program) is started in the information processing device 101.

On the language selection screen 600, for example, the language can be selected by selecting any of the buttons 601 to 604 by the user's operation input using the input device 206 shown in FIG. The wording on the subsequent screens will be displayed in the language selected here. For example, when the button 601 is selected, the wording on the subsequent screens is displayed in Japanese.

In the following explanation, it is assumed that "Japanese" is selected as the language. When the language is selected, for example, after the screens (not shown) such as the application terms of use and the application usage explanation are displayed, the account information input screen 700 shown in FIG. 7 is displayed.

In FIG. 7, the account information input screen 700 is an example of an operation screen for inputting account information. Here, it is assumed that the katakana, katakana name, date of birth, store number, subject, and account number are input as the account information.

When the input field 701 is selected on the account information input screen 700, the kana character can be input. If you select the input field 702 on the account information input screen 700, you can enter the katakana name. If the input field 703 is selected on the account information input screen 700, the date of birth can be input.

When the input field 704 is selected on the account information input screen 700, the store number can be input. When the menu button 705 is selected on the account information input screen 700, any subject can be selected from a plurality of subjects displayed in the pull-down menu. If the input field 706 is selected on the account information input screen 700, the account number can be input.

Further, when the Next button 707 is selected with the account information input on the account information input screen 700, the input account information is transmitted from the information processing device 101 to the account authentication server 102. When the authentication result indicating that the authentication is successful based on the account information is received from the account authentication server 102, the residence card number input screen 800 shown in FIG. 8 is displayed.

As a result, it is possible to authenticate the user who is the application operator by authenticating whether the input account information is the account of the information collection target person (foreign resident).

In FIG. 8, the residence card number input screen 800 is an example of an operation screen that accepts the input of the residence card number. By operating the software keyboard 801 on the residence card number input screen 800, the residence card number can be input.

Further, when the camera icon 802 is selected on the residence card number input screen 800, the screen transitions to the operation screen (not shown) for photographing the residence card, and the residence card can be photographed. In this case, the information processing device 101 accepts the input of the residence card number by reading the residence card number from the image of the residence card (front surface) taken by the OCR process.

As a result, the user can input the residence card number simply by taking a picture of the residence card without manually inputting the residence card number. The residence card number is written at a predetermined position (for example, upper right) of the residence card.

The entered residence card number is displayed in the number input box 803. When the residence card number is entered, the IC information reading screen 900 shown in FIG. 9 is displayed.

In FIG. 9, the IC information reading screen 900 is an example of an operation screen for reading the residence card information from the IC chip built in the residence card. When the user touches the residence card on the back side (NFC part) of the main body of the device, the entered residence card number is checked, and when reading is permitted, the reading device 207 (see FIG. 2) includes a face image. Residence card information is read.

When the residence card information including the face image is read, the validity of the residence card (resident card information) is determined based on the residence card information. For the determination of the validity of the residence card (resident card information), for example, a forgery / alteration check based on the electronic signature included in the residence card information is used. When the forgery / alteration check is completed, the reading result confirmation screen 1000 shown in FIG. 10 is displayed.

In FIG. 10, the reading result confirmation screen 1000 is an example of an operation screen showing the reading result 1001 of the residence card information and the forgery / alteration check result 1002. The reading result 1001 of the residence card information is a combination of the face image included in the residence card information read from the IC chip built in the residence card and the image of the residence card (excluding the face image).

The counterfeit / alteration check result 1002 indicates that the residence card (resident card information) is determined to be valid in the counterfeit / alteration check. When the next button 1003 is selected on the reading result confirmation screen 1000, the face image capturing screen 1100 shown in FIG. 11 is displayed.

In FIG. 11, the face image capturing screen 1100 is an example of an operation screen for capturing an image including a user. The user takes a face image of himself / herself according to the instruction displayed on the face image taking screen 1100.

First, when the next button 1101 is selected on the face image capturing screen 1100, the motion instruction 1211 is displayed together with the finder screen 1210 shown in FIG. In addition, a guide 1212 for positioning the face is displayed on the finder screen 1210. The motion instruction 1211 instructs the subject to look at the front.

While looking at the finder screen 1210, the user adjusts the position of the face according to the guide 1212 and performs an operation corresponding to the motion instruction 1211. Here, when the operation corresponding to the motion instruction 1211, that is, the operation of looking at the front is performed, the motion instruction 1211 is switched to the motion instruction 1221 shown in FIG.

While looking at the finder screen 1210, the user adjusts the position of the face according to the guide 1212 and performs an operation corresponding to the motion instruction 1221. Here, when the operation corresponding to the motion instruction 1221, that is, blinking is performed, the motion instruction 1221 is switched to the motion instruction 1231.

While looking at the finder screen 1210, the user adjusts the position of the face according to the guide 1212 and performs an operation corresponding to the motion instruction 1231. Here, when the action corresponding to the motion instruction 1231, that is, the action of opening the mouth is performed, it is determined that the action corresponding to the motion instruction has been performed, and a natural person is detected.

When a natural person is detected, face matching processing is performed based on the face image included in the read residence card information and the photographed image including the photographed user. The captured image including the user is, for example, an image captured when the operation corresponding to the motion instruction 1211 is performed. Here, if the face matching is successful, the face matching completion screen 1300 shown in FIG. 13 is displayed.

If the operation corresponding to each motion instruction 1211, 1221, 1231 is not performed on the face image capturing screen 1100, a screen (not shown) indicating a collation failure is displayed.

In FIG. 13, the face matching completion screen 1300 is an example of an operation screen showing the result of face matching (matching success). Further, on the face matching completion screen 1300, a message 1301 urging the photograph of the back surface of the residence card is displayed. Message 1301 is an example of guidance for photographing the back surface of the residence card.

From this, it can be seen that the user only needs to take a picture of the back side of the residence card. When the next button 1302 is selected on the face matching completion screen 1300, the confirmation screen 1400 shown in FIG. 14 is displayed in a pop-up. Before displaying the confirmation screen 1400, an operation screen for inquiring about permission for shooting or explaining a shooting method may be displayed.

In FIG. 14, the confirmation screen 1400 is an example of an operation screen for selecting whether or not to photograph the back surface of the residence card. When the button 1401 is selected on the confirmation screen 1400, the finder screen 1410 is displayed. If the button 1402 is selected on the confirmation screen 1400, the shooting of the back surface of the residence card can be skipped.

Here, it is assumed that the button 1401 is selected and the finder screen 1410 is displayed. The user can take a picture of the back side of the residence card by aligning the position of the residence card (back side) according to the guide 1411 and selecting the button 1412 while looking at the finder screen 1410.

When the back surface of the residence card is photographed on the finder screen 1410, the residence card confirmation screen 1500 shown in FIG. 15 is displayed.

In FIG. 15, the residence card confirmation screen 1500 is an example of an operation screen that prompts confirmation of the residence card information read from the residence card and the image including the back surface of the photographed residence card. The card image (front surface) 1501 is an image showing the surface of the residence card, and is generated by synthesizing a face image included in the residence card information read from the residence card and an image (image 1510) of the residence card.

The card image (back side) 1502 is an image of the back side of the residence card. On the residence card confirmation screen 1500, the user confirms the card image (front side) 1501 and the card image (back side) 1502, and if there is no problem, selects the check box 1503. Further, if the re-shooting button 1504 is selected on the residence card confirmation screen 1500, the back side of the residence card can be photographed again.

If the next button 1505 is selected with the check box 1503 selected on the residence card confirmation screen 1500, the residence information input screen 1600 shown in FIG. 16 is displayed.

In FIG. 16, the residence information input screen 1600 is an example of an operation screen that accepts the input of residence information. Here, it is assumed that the status of residence, the date of expiration of the period of stay (expiration date of stay), and the e-mail address are entered as the residence information.

When the menu button 1601 is selected on the residence information input screen 1600, any status of residence can be selected from a plurality of statuses of residence displayed in the pull-down menu. If the input field 1602 is selected on the residence information input screen 1600, the expiration date of the period of stay (period of stay) can be input.

If you select the input field 1603 on the residence information input screen 1600, you can enter your e-mail address. The e-mail address entered here is the e-mail address of the contact information of foreign residents. Further, on the residence information input screen 1600, when the next button 1604 is selected with the residence information input, the transaction purpose input screen 1700 shown in FIG. 17 is displayed.

In FIG. 17, the transaction purpose input screen 1700 is an example of an operation screen that accepts input of information regarding the transaction purpose of the user. Here, it is assumed that as the information regarding the transaction purpose of the user, items other than the residence information that may need to be confirmed at the time of transaction are input.

When the input field 1701 is selected on the transaction purpose input screen 1700, the address can be input. When the menu button 1702 is selected on the transaction purpose input screen 1700, any nationality can be selected from the plurality of nationalities displayed in the pull-down menu.

When the check boxes 1703 to 1709 are selected on the transaction purpose input screen 1700, the transaction purpose can be input. For example, if the check box 1703 is selected, "living expense settlement" can be entered as the transaction purpose of the account.

When the next button 1710 is selected on the transaction purpose input screen 1700, the information processing device 101 transmits a registration request to the information collection server 103, and the result transmission screen 1810 shown in FIG. 18 is displayed. The registration request includes, for example, the input identification key of the account information, the input residence information, and the information regarding the input transaction purpose. In addition, the registration request includes information read from the residence card (for example, card image (front side) 1501) and an image including the back side of the residence card (for example, card image (back side) 1502).

In FIG. 18, the result transmission screen 1810 is a screen showing that the input registration request for residence information is being transmitted to the information collection server 103. When new residence information management data is registered in the residence information management DB 130 in response to the residence information registration request on the information collection server 103, the registration completion screen 1820 is displayed.

According to the registration completion screen 1820, the user can confirm that the registration of the residence information has been completed normally. When the end button 1821 is selected on the registration completion screen 1820, a series of registration procedures can be completed.

(Screen example of registration information confirmation screen)
Next, a screen example of the registration information confirmation screen displayed on the business operator terminal 105 when using the residence information collection service will be described. The registration information confirmation screen is displayed on the business terminal 105 under the control of the information collection server 103, for example, when the information collection server 103 is accessed from the business terminal 105.

19 to 21 are explanatory views showing a screen example of the registration information confirmation screen. In FIG. 19, the registration information confirmation screen 1900 is an example of a login screen for performing a login process for the residence information collection service. On the registration information confirmation screen 1900, when the box 1901 is selected by the user's operation input, the user ID can be input.

If you tap the box 1902 on the registration information confirmation screen 1900, you can enter the password. Further, when the button 1903 is tapped on the registration information confirmation screen 1900, the login process can be performed using the entered user ID and password.

In the login process, a valid password registered in the user DB (not shown) associated with the input user ID is collated with the input password. Here, if the passwords match, the authentication is OK. On the other hand, if the passwords do not match or the entered user ID does not exist, the authentication is NG.

When the authentication is OK, the registration information confirmation screen 2000 shown in FIG. 20 is displayed.

In FIG. 20, the registration information confirmation screen 2000 is an example of an operation screen for inputting search conditions when searching for residence information. When the box 2001 is selected by the user's operation input on the registration information confirmation screen 2000, the period can be input as a search condition.

Further, when the box 2002 is selected on the registration information confirmation screen 2000, an identification key can be input as a search condition. When the menu button 2003 is selected on the registration information confirmation screen 2000, any collation result can be selected as a search condition from a plurality of collation results (face collation OK, face collation NG, all) displayed in the pull-down menu.

When the search button is selected on the registration information confirmation screen 2000, residence information matching the entered search conditions is searched. Specifically, for example, in the information collection server 103, the residence information management data matching the input search conditions is searched from the residence information management DB 130, and the registration information confirmation screen 2100 shown in FIG. 21 is displayed.

Here, it is assumed that the period "2020-02-01 to 2020-02-25" is input as the search condition and the collation result "all" is selected.

In FIG. 21, the registration information confirmation screen 2100 is an example of an operation screen for displaying the search results of residence information (for example, search results 2101 to 2104). Here, the registration time is included in the period "2020-02-01 to 2020-02-25", and the search results (all) of the matching result "face matching OK or face matching NG" are displayed.

Search results include registration time, katakana name, period of stay, status of residence, identification key and collation results. The registration time indicates the date and time when the residence information was registered. The katakana name indicates the katakana name of a foreign resident. The period of stay and status of residence indicate residence information. The identification key is an identifier that uniquely identifies the account held by a foreign resident.

For example, the search result 2101 has a registration time of "2020-01-29 15:53:37", a name of Kana "Oushigeo", a period of stay "March 05, 2020", a status of residence "Technology", and an identification key "0123456". "And the collation result" match (face collation OK) "is shown.

As a result, the user of the business terminal 105 (bank employee, etc.) can confirm the registered residence information for the account identified by the identification key.

Further, on the registration information confirmation screen 2100, for example, when the confirmation button 2110 of the search result 2101 is selected, the residence information detail screen 2200 as shown in FIG. 22 is displayed. Further, when the download button 2111 is selected on the registration information confirmation screen 2100, a list of search results can be output in CSV (Comma-Separated Value) format.

FIG. 22 is an explanatory diagram showing a screen example of the residence information detail screen. In FIG. 22, the residence information detail screen 2200 includes a card image (front surface) 2201, a card image (back surface) 2202, and residence information data 2203. The card image (front surface) 2201 is an image showing the surface of the residence card, and is a composite of a face image included in the residence card information read from the residence card and an image of the residence card.

The card image (back side) 2202 is an image of the back side of the residence card. The residence information data 2203 indicates the name of the resident foreigner, the residence information, and the like. However, in FIG. 22, a part of the residence information data 2203 is extracted and displayed. For example, the residence information data 2203 may include a residence card number, date of birth, nationality, address, purpose of transaction, occupation, and the like.

According to the residence information detail screen 2200, the user of the business terminal 105 (bank employee, etc.) can confirm the details of the registered residence information for the account identified by the identification key. Further, when the PDF output button 2204 is selected on the residence information detail screen 2200, the display content of the residence information detail screen 2200 can be output as PDF (Portable Document Format).

(Information collection support processing procedure of information processing device 101)
Next, the information collection support processing procedure of the information processing apparatus 101 will be described with reference to FIGS. 23 to 25.

23 to 25 are flowcharts showing an example of the information collection support processing procedure of the information processing apparatus 101. In the flowchart of FIG. 23, first, the information processing device 101 determines whether or not the input of the account information has been accepted (step S2301). Here, the information processing device 101 waits for the input of the account information to be accepted (step S2301: No).

When the information processing device 101 accepts the input of the account information (step S2301: Yes), the information processing device 101 transmits an authentication request including the input account information to the account authentication server 102 (step S2302). Then, the information processing device 101 determines whether or not the authentication result indicating that the authentication was successful is received from the account authentication server 102 based on the account information (step S2303).

Here, when the authentication result indicating that the authentication has failed based on the account information is received (step S2303: No), the information processing device 101 outputs the authentication result (NG) indicating that the authentication based on the account information has failed. Then (step S2304), a series of processes according to this flowchart is completed.

On the other hand, when the authentication result indicating that the authentication is successful based on the account information is received (step S2303: Yes), the information processing apparatus 101 displays the residence card number input screen on the display 205 (step S2305). Then, the information processing device 101 determines whether or not the input of the residence card number has been accepted (step S2306).

Here, the information processing device 101 waits for the input of the residence card number to be accepted (step S2306: No). When the information processing device 101 accepts the input of the residence card number (step S2306: Yes), the information processing device 101 receives the residence card information including the user's image from the IC chip built in the residence card, which is permitted to be read based on the residence card number. Read (step S2307).

Next, the information processing device 101 executes a forgery / alteration verification process by electronic signature using the public key included in the read residence card information (step S2308). Then, the information processing device 101 determines whether or not the verification result of the forgery / alteration verification process is the verification success (step S2309).

Here, in the case of verification failure (step S2309: No), the information processing device 101 outputs a verification result (NG) indicating that the residence card (resident card information) is not valid (step S2310), and is based on this flowchart. Ends a series of processes. On the other hand, if the verification is successful (step S2309: Yes), the information processing apparatus 101 shifts to step S2401 shown in FIG. 24.

In the flowchart of FIG. 24, first, the information processing apparatus 101 executes the face matching process (step S2401). The face matching process is a process for confirming that the user is the person (legitimate holder) who holds the residence card. However, the face matching process here includes the natural person detection process by motion detection.

The specific processing procedure of the face matching process will be described later with reference to FIG.

Next, the information processing device 101 determines whether or not the face matching is successful in the face matching process (step S2402). Here, when the face matching fails (step S2402: No), the information processing apparatus 101 outputs a matching result (NG) indicating that the face matching has failed (step S2403), and a series of processes according to this flowchart. To finish.

On the other hand, when the face matching is successful (step S2402: Yes), the information processing apparatus 101 displays a guide for photographing the back surface of the residence card on the display 205 (step S2404). Then, the information processing device 101 determines whether or not an image including the back surface of the residence card has been taken (step S2405).

Here, the information processing device 101 waits for an image including the back surface of the residence card to be taken (step S2405: No). When the image including the back surface of the residence card is taken (step S2405: Yes), the information processing device 101 displays the residence card confirmation screen on the display 205 (step S2406).

The residence card confirmation screen is a screen (for example, the residence card confirmation screen 1500) that prompts confirmation of the residence card information read from the residence card and the image including the back surface of the residence card. In the following description, an image including the back side of the residence card may be referred to as a "resident card image (back side)".

Next, the information processing device 101 determines whether or not the residence card information and the residence card image (back surface) have been confirmed (step S2407). Here, when the residence card information and the residence card image (back surface) are not confirmed (step S2407: No), the information processing apparatus 101 returns to step S2404. For example, the information processing device 101 determines that the residence card information and the residence card image (back surface) have not been confirmed when the back surface of the residence card is photographed again (re-photographed).

On the other hand, when the residence card information and the residence card image (back side) are confirmed (step S2407: Yes), the information processing apparatus 101 displays the residence information input screen on the display 205 (step S2408). Then, the information processing device 101 determines whether or not the input of the residence information has been accepted (step S2409).

Here, the information processing device 101 waits for the input of the residence information to be accepted (step S2409: No). When the information processing apparatus 101 receives the input of the residence information (step S2409: Yes), the information processing apparatus 101 proceeds to step S2501 shown in FIG.

In the flowchart of FIG. 25, first, the information processing apparatus 101 displays the transaction purpose input screen on the display 205 (step S2501). Then, the information processing device 101 determines whether or not the input of the information regarding the transaction purpose of the user has been accepted (step S2502).

Here, the information processing device 101 waits for receiving the input of information regarding the transaction purpose (step S2502: No). When the information processing device 101 receives the input of information regarding the transaction purpose (step S2502: Yes), the information processing device 101 transmits a registration request including the input residence information to the information collection server 103 (step S2503).

In addition to the residence information, the registration request includes, for example, an input identification key for account information, information on the input transaction purpose, read residence card information, residence card image (back side), and the like. Further, the registration request may include the collation rate in the face collation process of step S2401 shown in FIG. 24.

Then, when the registration of the residence information on the information collection server 103 is completed, the information processing device 101 displays the registration completion screen on the display 205 (step S2504), and ends a series of processes according to this flowchart. This makes it possible to collect information for managing the accounts of foreign residents.

Next, with reference to FIG. 26, a specific processing procedure of the face matching process in step S2401 shown in FIG. 24 will be described.

FIG. 26 is a flowchart showing an example of a specific processing procedure of the face matching process. In the flowchart of FIG. 26, first, the information processing device 101 activates the shooting function of the own device and shoots a moving image including the user by the camera 203 (step S2601).

Next, the information processing device 101 outputs a motion instruction while the camera 203 is shooting a moving image (step S2602). Then, the information processing device 101 determines whether or not an operation corresponding to the output motion instruction is performed based on the moving image captured by the camera 203 (step S2603).

Here, when the operation corresponding to the motion instruction is not performed (step S2603: No), the information processing apparatus 101 returns to the step in which the face matching process is called, assuming that the face matching has failed (step S2604).

On the other hand, when the operation corresponding to the motion instruction is performed (step S2603: Yes), the information processing device 101 selects from the face image (user's image) included in the read residence card information and the captured moving image. A collation request including a frame image including the user who has performed the information is transmitted to the collation server 104 (step S2605).

Next, the information processing apparatus 101 determines whether or not the collation result has been received from the collation server 104 (step S2606). The collation result includes, for example, a collation rate indicating the degree of similarity between the face image included in the residence card information and the frame image including the user. Here, the information processing apparatus 101 waits for receiving the collation result from the collation server 104 (step S2606: No).

When the collation result is received from the collation server 104 (step S2606: Yes), the information processing apparatus 101 determines whether or not the collation rate included in the received collation result is equal to or greater than the threshold value α (step S2607). Here, when the collation rate is less than the threshold value α (step S2607: No), the information processing apparatus 101 shifts to step S2604.

On the other hand, when the collation rate is equal to or higher than the threshold value α (step S2607: Yes), the information processing apparatus 101 returns to the step in which the face collation process is called, assuming that the face collation is successful (step S2608). As a result, it is possible to perform face collation while detecting a natural person by motion detection and preventing cheating.

As described above, according to the information processing apparatus 101 according to the embodiment, when the input of the account information is received, the input account information is transmitted to the account authentication server 102, and the input account information is transmitted from the account authentication server 102 based on the account information. When the authentication result indicating that the authentication is successful is received, the screen for accepting the input of the residence card number can be output.

As a result, if it can be verified that the entered account information is the account of the person whose information is to be collected (foreign resident), enter the residence card number to read the residence card information recorded on the IC chip of the residence card. Can be accepted.

Further, according to the information processing apparatus 101, when the input of the residence card number is received, the residence card information including the image of the user stored in the residence card, which is permitted to be read based on the residence card number, is read and the user is included. The captured image can be captured.

As a result, when the residence card number is input and the residence card is touched on the back side (NFC part) of the main body of the device, the residence card information is read from the residence card, the shooting function of the own device is activated, and the camera 203 is used. It is possible to shoot a captured image including the user.

Further, according to the information processing device 101, the face matching process can be performed based on the image of the user included in the read residence card information and the photographed image including the photographed user.

As a result, the image of the user taken by the camera 203 is collated with the face image read from the residence card, and whether the user (application operator) is the person who holds the residence card (legitimate holder). It becomes possible to judge whether or not.

Further, according to the information processing apparatus 101, when the face matching is successful in the face matching process, a screen for accepting the input of the residence information is output, and when the input of the residence information is accepted, the input residence information is input. It can be transmitted to the information collection server 103 in association with the account information.

As a result, if it is confirmed that the user is the person who holds the residence card, the input of the residence information can be accepted and the input residence information can be registered in association with the account information.

Further, according to the information processing apparatus 101, the input residence information and the read residence card information can be transmitted to the information collection server 103 in association with the input account information.

As a result, the residence card information recorded on the IC chip of the residence card can be registered in association with the account information together with the input residence information.

Further, according to the information processing device 101, when the face matching is successful in the face matching process, it is possible to output a guide for photographing the back surface of the residence card.

As a result, it is possible to urge the photograph of the back surface of the residence card, which may contain information that is not reflected in the residence card information in the IC chip of the residence card.

Further, according to the information processing device 101, when an image including the back surface of the residence card is taken, the residence information and the image including the back surface can be transmitted to the information collection server 103 in association with the account information.

As a result, the image on the back side of the residence card can be registered in association with the account information together with the entered residence information.

Further, according to the information processing device 101, after taking an image including the back surface of the residence card, it is possible to output information prompting confirmation between the read residence card information and the image including the back surface of the photographed residence card. ..

As a result, the contents of the residence card information recorded on the residence card and the appearance of the image including the back side of the photographed residence card can be confirmed, and the back surface of the residence card can be re-photographed as necessary. be able to.

Further, according to the information processing apparatus 101, when the face matching is successful in the face matching process, a screen for accepting the input of the information regarding the transaction purpose of the user is further output, and when the input of the information regarding the transaction purpose is received, the account information is obtained. In association with, the input residence information and information regarding the transaction purpose can be transmitted to the information collection server 103.

As a result, along with the entered residence information, information related to the user's transaction purpose, which is not described in the residence card, can be registered in association with the account information.

Further, according to the information processing device 101, when shooting a moving image including a user, a motion instruction is output, and based on the moving image including the photographed user, is an operation corresponding to the output motion instruction performed? It is possible to judge whether or not. Then, according to the information processing device 101, when it is determined that the operation corresponding to the motion instruction is performed, the face is based on the image of the user included in the residence card information and the moving image including the photographed user. Collation processing can be performed.

As a result, face matching can be performed after detecting that the subject is a natural person, that is, a real person. Therefore, for example, it is possible to prevent fraudulent acts such as performing face collation by using a photographed image of a face photograph attached to the residence card by a third party who is not the person himself / herself.

Further, according to the information processing apparatus 101, it is possible to determine whether or not the residence card information is valid based on the public key and the electronic signature included in the read residence card information. Then, according to the information processing device 101, when it is determined that the read residence card information is valid, the photographing function of the own device can be activated and the photographed image including the user can be photographed by the camera 203.

As a result, when performing face matching with the face image recorded on the residence card, it is possible to verify the forgery / alteration (authenticity verification) of the residence card using an electronic signature and confirm the validity of the residence card. , It is possible to prevent the information of the residence card that has been forged or altered by fraudulent activity from being registered.

Further, according to the information processing apparatus 101, the input residence information and the residence information included in the read residence card information are transmitted to the information management system (information collection server 103), and the input residence information and the input residence information are combined with each other. The result of comparison with the residence information included in the read residence card information may be transmitted to the information management system. This makes it possible to prevent unauthorized registration of residence information.

From these facts, according to the information processing apparatus 101 according to the embodiment, it can be easily determined that the account specified by the user is the account of the information collection target person and the user is the person who holds the residence card. You can check. Therefore, it is possible to easily collect information for account management of foreign residents by using a smartphone or the like.

For example, the user can register the residence information simply by downloading the residence information collection application to a smartphone or the like and performing an input operation according to the instruction of the application. Therefore, it is possible to reduce the time and effort required for registering the residence information and improve the convenience of the user. Also, for the bank side, it is possible to reduce the cost of reply envelopes and the cost of employees to deal with customers when visiting the store.

The information collection support method described in the present embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This information collection support program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD (Compact Disc) -ROM, a DVD (Digital Versaille Disk), or a USB (Universal Serial Bus) memory, and is recorded by the computer. It is executed by being read from the medium. In addition, this information collection support program may be distributed via a network such as the Internet.

The following additional notes are further disclosed with respect to the above-described embodiment.

(Appendix 1) When the input of account information is accepted, the input account information is transmitted to the authentication system corresponding to the financial institution.
When an authentication result indicating that the authentication was successful based on the account information is received from the authentication system, a screen for accepting the input of the residence card number is output.
When the input of the residence card number is accepted, the information including the image of the user stored in the residence card, which is permitted to be read based on the residence card number, is read, and the photographed image including the user is photographed.
A face matching process is performed based on the image of the user included in the read information and the photographed image including the photographed user.
An information collection support program characterized by having a computer execute processing.

(Appendix 2) If the face matching is successful in the face matching process, a screen for accepting the input of residence information is output.
When the input of the residence information is accepted, the input residence information is associated with the account information and transmitted to the information management system corresponding to the financial institution.
The information collection support program according to Appendix 1, wherein the computer executes the process.

(Appendix 3) The processing to be transmitted is
The information collection support program according to Appendix 2, wherein the residence information and the information are transmitted to the information management system in association with the account information.

(Appendix 4) When the face matching is successful in the face matching process, the computer is made to execute the process of outputting a guide for photographing the back surface of the residence card.
The process of transmitting is
The description in Appendix 2 or 3, wherein when an image including the back surface of the residence card is taken, the residence information and the image including the back surface are transmitted to the information management system in association with the account information. Information gathering support program.

(Appendix 5) After taking an image including the back surface of the residence card, the computer is made to execute a process of outputting information prompting confirmation between the read information and the photographed image including the back surface. The information collection support program described in Appendix 4.

(Appendix 6) When the face matching is successful in the face matching process, the computer is made to execute the process of outputting a screen for accepting the input of information regarding the transaction purpose of the user.
The process of transmitting is
When the input of the information regarding the transaction purpose is received, the residence information and the information regarding the transaction purpose are transmitted to the information management system in association with the account information. Information gathering support program described in one.

(Appendix 7) Output motion instructions when shooting a video that includes the user.
Based on the video including the captured user, it is determined whether or not the operation corresponding to the output motion instruction is performed.
When it is determined that the operation corresponding to the motion instruction is performed, the face matching process is performed based on the image of the user included in the read information and the video including the user captured.
The information collection support program according to any one of Supplementary notes 1 to 6, wherein the computer executes the process.

(Appendix 8) Based on the public key and electronic signature included in the read information, it is determined whether or not the information is valid.
When it is determined that the read information is valid, the shooting function of the own device is activated to shoot a shot image including the user.
The information collection support program according to any one of Supplementary notes 1 to 7, wherein the computer executes the process.

(Appendix 9) The process of performing the face matching process is
When the collation rate indicating the degree of similarity between the user's image and the captured image is equal to or greater than the threshold value, it is determined that the face collation was successful.
If the collation rate is less than the threshold value, it is determined that the face collation has failed.
The information gathering support program described in any one of Appendix 1 to 8, characterized by the above.

(Appendix 10) The input residence information and the residence information included in the read information are transmitted to the information management system, and the input residence information and the residence information included in the read information are included. The result of the comparison is transmitted to the information management system.
The information collection support program according to Appendix 2, wherein the computer executes the process.

(Appendix 11) When the input of account information is accepted, the input account information is transmitted to the authentication system corresponding to the financial institution.
When an authentication result indicating that the authentication was successful based on the account information is received from the authentication system, a screen for accepting the input of the residence card number is output.
When the input of the residence card number is accepted, the information including the image of the user stored in the residence card, which is permitted to be read based on the residence card number, is read, and the photographed image including the user is photographed.
A face matching process is performed based on the image of the user included in the read information and the photographed image including the photographed user.
An information gathering support method characterized in that a computer executes processing.

(Appendix 12) When the input of account information is accepted, the input account information is transmitted to the authentication system corresponding to the financial institution.
When an authentication result indicating that the authentication was successful based on the account information is received from the authentication system, a screen for accepting the input of the residence card number is output.
When the input of the residence card number is accepted, the information including the image of the user stored in the residence card, which is permitted to be read based on the residence card number, is read, and the photographed image including the user is photographed.
A face matching process is performed based on the image of the user included in the read information and the photographed image including the photographed user.
An information processing device characterized by having a control unit.

100 Information collection system 101 Information processing device 102 Account authentication server 103 Information collection server 104 Verification server 105 Business terminal 110 Network 120 Account information management DB
130 Residence Information Management DB
200 bus 201,301 CPU
202 Memory 203 Camera 204 Communication I / F
205 Display 206 Input device 207 Reading device 401 Reception unit 402 Account authentication unit 403 Display control unit 404 Reading unit 405 Shooting control unit 406 Matching unit 407 Communication unit 500 Residence card 501 Face image 502 Photographed image 600 Language selection screen 700 Account information input screen 800 Residence card number input screen 900 IC information reading screen 1000 Reading result confirmation screen 1100 Face image shooting screen 1210, 1410 Finder screen 1211,1221,1231 Motion instruction 1300 Face verification completion screen 1400 Confirmation screen 1500 Residence card confirmation screen 1600 Residence information input Screen 1700 Transaction purpose input screen 1810 Result transmission screen 1820 Registration completion screen 1900, 2000, 2100 Registration information confirmation screen 2200 Residence information details screen

Claims (10)

When the input of the account information is accepted, the input account information is transmitted to the authentication system corresponding to the financial institution, and the input is transmitted.
When an authentication result indicating that the authentication was successful based on the account information is received from the authentication system, a screen for accepting the input of the residence card number is output.
When the input of the residence card number is accepted, the information including the image of the user stored in the residence card, which is permitted to be read based on the residence card number, is read, and the photographed image including the user is photographed.
A face matching process is performed based on the image of the user included in the read information and the photographed image including the photographed user.
An information collection support program characterized by having a computer execute processing. If the face matching is successful in the face matching process, a screen for accepting the input of residence information is output.
When the input of the residence information is accepted, the input residence information is associated with the account information and transmitted to the information management system corresponding to the financial institution.
The information collection support program according to claim 1, wherein the computer executes the process. The process of transmitting is
The information collection support program according to claim 2, wherein the residence information and the information are transmitted to the information management system in association with the account information. When the face matching is successful in the face matching process, the computer is made to execute the process of outputting a guide for photographing the back surface of the residence card.
The process of transmitting is
According to claim 2 or 3, when an image including the back surface of the residence card is taken, the residence information and the image including the back surface are transmitted to the information management system in association with the account information. The listed information gathering support program. The claim is characterized in that, after taking an image including the back surface of the residence card, the computer is made to execute a process of outputting information prompting confirmation between the read information and the photographed image including the back surface. Information gathering support program described in 4. When the face matching is successful in the face matching process, the computer is made to execute the process of outputting a screen for accepting input of information regarding the user's transaction purpose.
The process of transmitting is
Any of claims 2 to 5, wherein when the input of the information regarding the transaction purpose is received, the residence information and the information regarding the transaction purpose are transmitted to the information management system in association with the account information. The information gathering support program described in one. When shooting a video that includes the user, output motion instructions and
Based on the video including the captured user, it is determined whether or not the operation corresponding to the output motion instruction is performed.
When it is determined that the operation corresponding to the motion instruction is performed, the face matching process is performed based on the image of the user included in the read information and the video including the user captured.
The information collection support program according to any one of claims 1 to 6, wherein the computer executes the process. The result of transmitting the input residence information and the residence information included in the read information to the information management system, and comparing the input residence information with the residence information included in the read information. To the information management system,
The information collection support program according to claim 2, wherein the computer executes the process. When the input of the account information is accepted, the input account information is transmitted to the authentication system corresponding to the financial institution, and the input is transmitted.
When an authentication result indicating that the authentication was successful based on the account information is received from the authentication system, a screen for accepting the input of the residence card number is output.
When the input of the residence card number is accepted, the information including the image of the user stored in the residence card, which is permitted to be read based on the residence card number, is read, and the photographed image including the user is photographed.
A face matching process is performed based on the image of the user included in the read information and the photographed image including the photographed user.
An information collection support method characterized in that a computer executes processing. When the input of the account information is accepted, the input account information is transmitted to the authentication system corresponding to the financial institution, and the input is transmitted.
When an authentication result indicating that the authentication was successful based on the account information is received from the authentication system, a screen for accepting the input of the residence card number is output.
When the input of the residence card number is accepted, the information including the image of the user stored in the residence card, which is permitted to be read based on the residence card number, is read, and the photographed image including the user is photographed.
A face matching process is performed based on the image of the user included in the read information and the photographed image including the photographed user.
An information processing device characterized by having a control unit.

Images