KR102593619B1 - Implementation of biometric authentication - Google Patents

Abstract

The present invention generally provides user interfaces for biometric registration process instructions, aligning biometric features for registration, registering biometric features, providing hints during the biometric registration process, and application applications. Based biometric authentication, automatically populating biometric security fields, unlocking devices using biometric authentication, retrying biometric authentication, managing transfers using biometric authentication, biometric authentication Embedded user interface during biometric authentication, preventing retries of biometric authentication, cached biometric authentication, automatically populating fillable fields based on visibility criteria, automatic login using biometric authentication, credential entry Retry biometric authentication in the user interface, provide indications of error conditions during biometric authentication, provide indications regarding the biometric sensor during biometric authentication, and enable the device to register biometric characteristics. It relates to implementing biometric authentication, including orientation.

Description

IMPLEMENTATION OF BIOMETRIC AUTHENTICATION}

Cross-reference to related applications

This application is related to U.S. Provisional Patent Application No. 62/556,413, “FACE ENROLLMENT AND AUTHENTICATION,” filed on September 9, 2017; and No. 62/557,130, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION”, filed September 11, 2017; Danish Patent Application PA 2017 70712, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION”, filed September 22, 2017; PA 2017 70713, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION”, filed September 22, 2017; PA 2017 70714, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION”, filed September 22, 2017; and PA 2017 70715, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION”, filed September 22, 2017; U.S. Provisional Patent Application No. 62/581,025, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION,” filed November 2, 2017; International application PCT/US2018/015603, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION”, filed January 26, 2018; U.S. Patent Application No. 15/894,221, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION,” filed February 12, 2018; U.S. Patent Application No. 15/903,456, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION,” filed February 23, 2018; U.S. Provisional Patent Application No. 62/679,955, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION,” filed June 3, 2018; Danish Patent Application No. PA 2018 70370, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION”, filed June 12, 2018; and Danish Patent Application PA 2018 70371, “IMPLEMENTATION OF BIOMETRIC AUTHENTICATION,” filed June 12, 2018. All of these applications are incorporated herein by reference in their entirety.

Technology field

The present invention relates generally to biometric authentication, and more specifically to interfaces and techniques for registration and authentication of biometric characteristics.

Biometric authentication using electronic devices, for example, face, iris, or fingerprint, is a convenient and efficient way to authenticate users of electronic devices. Biometric authentication allows devices to quickly and easily verify the identity of any number of users.

However, some techniques for implementing biometric authentication using electronic devices are generally cumbersome. For example, some existing techniques, such as those related to facial recognition, require the user to almost perfectly align biometric features in the same way during both registration and each iteration of authentication. Deviation from alignment of biometric features often causes false negative results. As a result, the user is optionally required to unnecessarily perform multiple iterations of biometric authentication, or alternatively, is discouraged from using biometric authentication altogether. As another example, some existing techniques rely only on two-dimensional representations of biometric features. As a result, the user's authentication is optionally limited by failing to analyze one or more three-dimensional characteristics of the biometric feature, and also optionally requires the user to unnecessarily perform additional repetitions of biometric authentication. Considering the aforementioned drawbacks, existing techniques require more time than necessary, wasting both user time and device energy. This latter consideration is particularly important in the operation of battery-operated devices.

Accordingly, the present technique provides electronic devices with faster and more efficient methods and interfaces for implementing biometric authentication. These methods and interfaces optionally complement or replace other methods for implementing biometric authentication. Such methods and interfaces reduce the cognitive burden on the user and create a more efficient human-machine interface. For battery-operated computing devices, such methods and interfaces save power and increase time between battery charges. Such methods and interfaces also reduce the number of unnecessary, irrelevant, or repetitive inputs required in computing devices, such as smartphones and smartwatches.

According to some examples, a method is described, the method comprising: at an electronic device having one or more input devices, one or more biometric sensors, and a display: displaying a first user interface on the display; During the step of displaying the first user interface, detecting the occurrence of a condition corresponding to introduction of a biometric registration process for registering biometric characteristics; In response to detecting the occurrence of a condition corresponding to introduction of a biometric registration process, displaying a biometric registration introduction interface, displaying the biometric registration introduction interface comprising: a representation of a simulation of the biometric features; and simultaneously displaying simulated progress indicators; During the step of displaying the biometric registration introduction interface, displaying a training animation comprising displaying a movement of a representation of the simulation of biometric features and an incremental advancement of a simulated progress indicator; After displaying at least a portion of the training animation, detecting the occurrence of a condition corresponding to initiation of a biometric registration process; and in response to detecting the occurrence of a condition corresponding to initiation of the biometric registration process: displaying a progress indicator corresponding to the simulated progress indicator; and displaying a representation of the user's biometric features, as determined by one or more biometric sensors of the device, in a location previously occupied by a representation of the simulation of the biometric features in the biometric registration introduction interface. do.

According to some examples, a non-transitory computer-readable medium is described, wherein the non-transitory computer-readable storage medium is operated by one or more processors of an electronic device having one or more input devices, one or more biometric sensors, and a display. comprising one or more programs configured to be executed, the one or more programs configured to: display a first user interface on a display; while displaying the first user interface, detecting the occurrence of a condition corresponding to introduction of a biometric registration process for registering biometric characteristics; In response to detecting the occurrence of a condition corresponding to introduction of a biometric registration process, displaying a biometric registration introduction interface comprising: a representation of a simulation of the biometric features; and Includes simultaneous display of progress indicators -; while displaying the biometric registration introduction interface, to display a training animation including displaying a movement of a representation of the simulation of biometric features and an incremental advancement of a simulated progress indicator; After displaying at least a portion of the training animation, detect the occurrence of a condition corresponding to initiation of a biometric registration process; and in response to detecting the occurrence of a condition corresponding to initiation of the biometric registration process: to display a progress indicator corresponding to the simulated progress indicator; and instructions for displaying a representation of the user's biometric characteristic as determined by one or more biometric sensors of the device in the location previously occupied by a representation of the simulation of the biometric feature in the biometric registration introduction interface. Includes.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium configured to be executed by one or more processors of an electronic device having one or more input devices, one or more biometric sensors, and a display. comprising one or more programs configured to display a first user interface on a display; while displaying the first user interface, detecting the occurrence of a condition corresponding to introduction of a biometric registration process for registering biometric characteristics; In response to detecting the occurrence of a condition corresponding to introduction of a biometric registration process, displaying a biometric registration introduction interface comprising: a representation of a simulation of the biometric features; and Includes simultaneous display of progress indicators -; while displaying the biometric registration introduction interface, to display a training animation including displaying a movement of a representation of the simulation of biometric features and an incremental advancement of a simulated progress indicator; After displaying at least a portion of the training animation, detect the occurrence of a condition corresponding to initiation of a biometric registration process; and in response to detecting the occurrence of a condition corresponding to initiation of the biometric registration process: to display a progress indicator corresponding to the simulated progress indicator; and instructions for displaying a representation of the user's biometric characteristic as determined by one or more biometric sensors of the device in the location previously occupied by a representation of the simulation of the biometric feature in the biometric registration introduction interface. Includes.

According to some examples, an electronic device is described, the electronic device comprising: one or more input devices; one or more biometric sensors; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, the one or more programs configured to: display a first user interface on the display; while displaying the first user interface, detecting the occurrence of a condition corresponding to introduction of a biometric registration process for registering biometric characteristics; In response to detecting the occurrence of a condition corresponding to introduction of a biometric registration process, displaying a biometric registration introduction interface comprising: a representation of a simulation of the biometric features; and Includes simultaneous display of progress indicators -; while displaying the biometric registration introduction interface, to display a training animation including displaying a movement of a representation of the simulation of biometric features and an incremental advancement of a simulated progress indicator; After displaying at least a portion of the training animation, detect the occurrence of a condition corresponding to initiation of a biometric registration process; and in response to detecting the occurrence of a condition corresponding to initiation of the biometric registration process: to display a progress indicator corresponding to the simulated progress indicator; and instructions for displaying a representation of the user's biometric characteristic as determined by one or more biometric sensors of the device in the location previously occupied by a representation of the simulation of the biometric feature in the biometric registration introduction interface. Includes.

According to some examples, an electronic device is described, the electronic device comprising: one or more input devices; one or more biometric sensors; display; means for displaying a first user interface on a display; means for detecting, while displaying the first user interface, the occurrence of a condition corresponding to introduction of a biometric registration process for registering a biometric characteristic; In response to detecting the occurrence of a condition corresponding to introduction of a biometric registration process, means for displaying a biometric registration introduction interface, wherein displaying the biometric registration introduction interface comprises: a representation of a simulation of the biometric features; and Includes simultaneous display of progress indicators -; means for displaying a training animation, while displaying a biometric registration introduction interface, including displaying a movement of a representation of the simulation of biometric features and an incremental advancement of a simulated progress indicator; means for detecting, after displaying at least a portion of the training animation, the occurrence of a condition corresponding to initiation of a biometric registration process; and in response to detecting the occurrence of a condition corresponding to initiation of the biometric registration process: means for displaying a progress indicator corresponding to the simulated progress indicator; and means for displaying a representation of the user's biometric features as determined by one or more biometric sensors of the device in a location previously occupied by a representation of the simulation of the biometric features in the biometric registration introduction interface. Includes.

According to some examples, a method is described, the method comprising: in an electronic device having one or more cameras and a display: displaying a first user interface on the display; During the step of displaying the first user interface, detecting the occurrence of a condition corresponding to initiating a biometric registration process for registering each type of biometric characteristic; In response to detecting the occurrence of a condition corresponding to initiating a biometric registration process, displaying, on a display, a digital viewfinder including a preview of image data captured by one or more cameras; and after initiating the biometric registration process: upon determining that each type of biometric feature that meets the alignment criteria has been detected in the field of view of one or more cameras: relative to a second portion of the field of view of the one or more cameras. highlighting a first portion of the field of view; and deemphasizing the first portion of the field of view of the one or more cameras relative to the second portion of the field of view of the one or more cameras in accordance with a determination that each type of biometric feature that meets the alignment criteria has not been detected in the field of view of the one or more cameras. and maintaining the display of the digital viewfinder without displaying the digital viewfinder.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more cameras and a display; , one or more programs for displaying a first user interface on a display; while displaying the first user interface, detecting the occurrence of a condition corresponding to initiating a biometric registration process for registering each type of biometric feature; in response to detecting the occurrence of a condition corresponding to initiating a biometric registration process, to display, on a display, a digital viewfinder including a preview of image data captured by the one or more cameras; and after initiating the biometric registration process: a determination that each type of biometric feature that meets the alignment criteria has been detected in the field of view of the one or more cameras relative to a second portion of the field of view of the one or more cameras. to highlight the first part of the field of view; and de-emphasize the first portion of the field of view of the one or more cameras relative to the second portion of the field of view of the one or more cameras in accordance with a determination that each type of biometric feature that meets the alignment criteria has not been detected in the field of view of the one or more cameras. Contains commands for maintaining the display of the digital viewfinder without

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more cameras and a display, The above programs are for displaying a first user interface on a display; while displaying the first user interface, detecting the occurrence of a condition corresponding to initiating a biometric registration process for registering each type of biometric characteristic; in response to detecting the occurrence of a condition corresponding to initiating a biometric registration process, to display, on a display, a digital viewfinder including a preview of image data captured by the one or more cameras; and after initiating the biometric registration process: a determination that each type of biometric feature that meets the alignment criteria has been detected in the field of view of the one or more cameras relative to a second portion of the field of view of the one or more cameras. to highlight the first part of the field of view; and de-emphasize the first portion of the field of view of the one or more cameras relative to the second portion of the field of view of the one or more cameras in accordance with a determination that each type of biometric feature that meets the alignment criteria has not been detected in the field of view of the one or more cameras. Contains commands for maintaining the display of the digital viewfinder without

According to some examples, an electronic device is described, the electronic device comprising: one or more cameras; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, the one or more programs configured to: display a first user interface on the display; while displaying the first user interface, detecting the occurrence of a condition corresponding to initiating a biometric registration process for registering each type of biometric feature; in response to detecting the occurrence of a condition corresponding to initiating a biometric registration process, to display, on a display, a digital viewfinder including a preview of image data captured by the one or more cameras; and after initiating the biometric registration process: a determination that each type of biometric feature that meets the alignment criteria has been detected in the field of view of the one or more cameras relative to a second portion of the field of view of the one or more cameras. to highlight the first part of the field of view; and de-emphasize the first portion of the field of view of the one or more cameras relative to the second portion of the field of view of the one or more cameras in accordance with a determination that each type of biometric feature that meets the alignment criteria has not been detected in the field of view of the one or more cameras. Contains commands for maintaining the display of the digital viewfinder without

According to some examples, an electronic device is described, the electronic device comprising: one or more cameras; display; one or more processors; means for displaying a first user interface on a display; means for detecting, while displaying the first user interface, the occurrence of a condition corresponding to initiating a biometric registration process for registering each type of biometric characteristic; means for displaying, on a display, a digital viewfinder including a preview of image data captured by one or more cameras, in response to detecting the occurrence of a condition corresponding to initiating a biometric registration process; and after initiating the biometric registration process: upon determining that each type of biometric feature that meets the alignment criteria has been detected in the field of view of the one or more cameras relative to a second portion of the field of view of the one or more cameras. means for emphasizing a first part of the field of view; and deemphasizing the first portion of the field of view of the one or more cameras relative to the second portion of the field of view of the one or more cameras in accordance with a determination that each type of biometric feature that meets the alignment criteria has not been detected in the field of view of the one or more cameras. Includes means for maintaining the display of the digital viewfinder without displaying the display.

According to some examples, a method is described, comprising: in an electronic device having one or more biometric sensors and a display: simultaneously displaying a biometric registration interface on the display, wherein displaying the biometric registration interface comprises: , a representation of the biometric feature having an orientation determined based on the alignment of the biometric feature with respect to one or more biometric sensors of the device, and a first progress indicator at a first location on the display relative to the representation of the biometric feature. simultaneously displaying a progress indicator portion comprising a portion and a second progress indicator portion at a second location on the display relative to the representation of the biometric feature, wherein the representation of the biometric feature is located at a first location and a second location on the display. displayed between positions -; detecting a change in the orientation of the biometric feature relative to one or more biometric sensors while simultaneously displaying a representation of the biometric feature and a progress indicator; and in response to detecting a change in the orientation of the biometric feature relative to the one or more biometric sensors: the change in orientation of the biometric feature relative to the first portion of the biometric feature corresponding to the first progress indicator portion. updating one or more visual characteristics of the first progress indicator portion in accordance with a determination that registration criteria are met; and updating one or more visual characteristics of the second progress indicator portion in accordance with a determination that the change in orientation of the biometric feature meets registration criteria for the second portion of the biometric feature corresponding to the second progress indicator portion. Includes steps.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display. and one or more programs for simultaneously displaying a biometric registration interface on a display, wherein displaying the biometric registration interface is determined based on an alignment of the biometric feature with respect to one or more biometric sensors of the device. A representation of a biometric feature having an orientation, and a first progress indicator portion at a first location on the display for the representation of the biometric feature and a second progress indicator portion at a second location on the display for the representation of the biometric feature. simultaneously displaying a progress indicator comprising a portion, wherein the representation of the biometric characteristic is displayed between a first location and a second location on the display; detecting a change in the orientation of the biometric feature relative to one or more biometric sensors while simultaneously displaying a representation of the biometric feature and a progress indicator; and in response to detecting a change in the orientation of the biometric feature relative to the one or more biometric sensors: registration of the change in orientation of the biometric feature to the first portion of the biometric feature corresponding to the first progress indicator portion. updating one or more visual characteristics of the first progress indicator portion in accordance with a determination that the criteria are met; and updating one or more visual characteristics of the second progress indicator portion in accordance with a determination that the change in orientation of the biometric feature meets registration criteria for the second portion of the biometric feature corresponding to the second progress indicator portion. Includes commands for

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display; , one or more programs for simultaneously displaying a biometric registration interface on a display, wherein displaying the biometric registration interface has an orientation determined based on the alignment of the biometric feature with respect to one or more biometric sensors of the device. a representation of a biometric feature having a first progress indicator portion at a first location on the display for the representation of the biometric feature and a second progress indicator portion at a second location on the display for the representation of the biometric feature. simultaneously displaying a progress indicator comprising: wherein the representation of the biometric characteristic is displayed between a first location and a second location on the display; detecting a change in the orientation of the biometric feature relative to one or more biometric sensors while simultaneously displaying a representation of the biometric feature and a progress indicator; and in response to detecting a change in the orientation of the biometric feature relative to the one or more biometric sensors: registration of the change in orientation of the biometric feature to the first portion of the biometric feature corresponding to the first progress indicator portion. updating one or more visual characteristics of the first progress indicator portion in accordance with a determination that the criteria are met; and updating one or more visual characteristics of the second progress indicator portion in accordance with a determination that the change in orientation of the biometric feature meets registration criteria for the second portion of the biometric feature corresponding to the second progress indicator portion. Includes commands for

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are configured to: simultaneously display a biometric registration interface on the display - displaying the biometric registration interface comprises: a representation of the biometric feature having an orientation determined based on the alignment of the biometric feature with respect to one or more biometric sensors, and a first progress indicator portion and a biometric feature at a first location on the display for the representation of the biometric feature. and simultaneously displaying a progress indicator comprising a second progress indicator portion at a second location on the display relative to the representation of the biometric feature, wherein the representation of the biometric feature is displayed between the first location and the second location on the display. became -; detecting a change in the orientation of the biometric feature relative to one or more biometric sensors while simultaneously displaying a representation of the biometric feature and a progress indicator; and in response to detecting a change in the orientation of the biometric feature relative to the one or more biometric sensors: registration of the change in orientation of the biometric feature to the first portion of the biometric feature corresponding to the first progress indicator portion. updating one or more visual characteristics of the first progress indicator portion in accordance with a determination that the criteria are met; and updating one or more visual characteristics of the second progress indicator portion in accordance with a determination that the change in orientation of the biometric feature meets registration criteria for the second portion of the biometric feature that corresponds to the second progress indicator portion. Includes commands for

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; Means for simultaneously displaying a biometric registration interface on a display, wherein displaying the biometric registration interface comprises: a biometric feature having an orientation determined based on an alignment of the biometric feature with respect to one or more biometric sensors of the device; a presentation, and a progress indicator comprising a first progress indicator portion at a first location on the display for the presentation of the biometric characteristic and a second progress indicator portion at a second location on the display for the presentation of the biometric characteristic. simultaneously displaying, wherein the representation of the biometric characteristic is displayed between the first location and the second location on the display; means for detecting a change in the orientation of a biometric feature relative to one or more biometric sensors while simultaneously displaying a representation of the biometric feature and a progress indicator; and in response to detecting a change in the orientation of the biometric feature relative to the one or more biometric sensors: registration of the change in orientation of the biometric feature to the first portion of the biometric feature corresponding to the first progress indicator portion. means for updating one or more visual characteristics of the first progress indicator portion in accordance with a determination that the criteria are met; and updating one or more visual characteristics of the second progress indicator portion in accordance with a determination that the change in orientation of the biometric feature meets registration criteria for the second portion of the biometric feature that corresponds to the second progress indicator portion. Includes means for

According to some examples, a method is described, comprising, in an electronic device having one or more biometric sensors and a display: displaying, on the display, a biometric registration user interface for registering a biometric feature - biometric Displaying the measurement registration user interface includes displaying a representation of the biometric feature, wherein the appearance of the representation of the biometric feature changes as the orientation of the biometric feature relative to the one or more biometric sensors changes; During the step of displaying a biometric registration user interface, detecting that registration prompt criteria have been met with respect to one or more portions of the biometric characteristics; and in response to detecting that registration prompt criteria have been met with respect to one or more portions of the biometric feature, outputting a respective prompt to move the biometric feature in a respective manner, The prompt is selected based on the registration status of one or more portions of the biometric feature, wherein outputting each prompt is associated with a first portion of the biometric feature that can be enrolled by moving the biometric feature in a first manner. in response to a determination that the enrollment prompt criteria have been met, outputting a prompt to move the biometric feature in a first manner; and outputting a prompt to move the biometric feature in a second manner in response to a determination that enrollment prompt criteria have been met with respect to the second portion of the measurement feature.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display. comprising: one or more programs for displaying, on the display, a biometric registration user interface for registering the biometric characteristic, wherein displaying the biometric registration user interface comprises displaying a representation of the biometric characteristic; The appearance of the representation of a biometric feature changes as the orientation of the biometric feature relative to one or more biometric sensors changes; While displaying a biometric registration user interface, detect that registration prompt criteria have been met with respect to one or more portions of the biometric characteristics; and instructions for outputting a respective prompt to move the biometric feature in a respective manner in response to detecting that registration prompt criteria have been met with respect to one or more portions of the biometric feature, each of Prompts are selected based on the registration status of one or more portions of the biometric feature, outputting each prompt associated with a first portion of the biometric feature that can be enrolled by moving the biometric feature in a first manner. in response to a determination that the enrollment prompt criteria have been met, outputting a prompt to move the biometric feature in a first manner, and moving the biometric feature in a second manner that is different from the first manner. and outputting a prompt to move the biometric feature in a second manner in response to a determination that enrollment prompt criteria have been met with respect to the second portion of the feature.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display; , one or more programs for displaying, on the display, a biometric registration user interface for registering the biometric characteristic, wherein displaying the biometric registration user interface comprises displaying a representation of the biometric feature, The appearance of the representation of a feature changes as the orientation of the biometric feature relative to one or more biometric sensors changes; While displaying a biometric registration user interface, detect that registration prompt criteria have been met with respect to one or more portions of the biometric characteristics; and instructions for outputting a respective prompt to move the biometric feature in a respective manner in response to detecting that registration prompt criteria have been met with respect to one or more portions of the biometric feature, each of Prompts are selected based on the registration status of one or more portions of the biometric feature, outputting each prompt associated with a first portion of the biometric feature that can be enrolled by moving the biometric feature in a first manner. in response to a determination that the enrollment prompt criteria have been met, outputting a prompt to move the biometric feature in a first manner, and moving the biometric feature in a second manner that is different from the first manner. and outputting a prompt to move the biometric feature in a second manner in response to a determination that enrollment prompt criteria have been met with respect to the second portion of the feature.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, the one or more programs configured to: display, on the display, a biometric registration user interface for registering a biometric characteristic - biometric registration; Displaying the user interface includes displaying a representation of the biometric feature, wherein the appearance of the representation of the biometric feature changes as the orientation of the biometric feature relative to the one or more biometric sensors changes; While displaying a biometric registration user interface, detect that registration prompt criteria have been met with respect to one or more portions of the biometric characteristics; and instructions for outputting a respective prompt to move the biometric feature in a respective manner in response to detecting that registration prompt criteria have been met with respect to one or more portions of the biometric feature, each of Prompts are selected based on the registration status of one or more portions of the biometric feature, outputting each prompt associated with a first portion of the biometric feature that can be enrolled by moving the biometric feature in a first manner. in response to a determination that the enrollment prompt criteria have been met, outputting a prompt to move the biometric feature in a first manner, and moving the biometric feature in a second manner that is different from the first manner. and outputting a prompt to move the biometric feature in a second manner in response to a determination that enrollment prompt criteria have been met with respect to the second portion of the feature.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; Means for displaying, on a display, a biometric registration user interface for registering a biometric characteristic, wherein displaying the biometric registration user interface comprises displaying a representation of the biometric feature, and an appearance of the representation of the biometric feature. varies as the orientation of the biometric feature relative to one or more biometric sensors changes; means for detecting, while displaying a biometric registration user interface, that registration prompt criteria have been met with respect to one or more portions of the biometric characteristics; and means for outputting a respective prompt to move the biometric feature in a respective manner in response to detecting that the enrollment prompt criteria have been met with respect to one or more portions of the biometric feature, each comprising: The prompt is selected based on the registration status of one or more portions of the biometric feature, and the means for outputting each prompt comprises: a first portion of the biometric feature that can be enrolled by moving the biometric feature in a first manner; associated means for outputting a prompt to move the biometric feature in a first manner in response to a determination that the enrollment prompt criteria have been met, and whereby the biometric feature can be enrolled by moving it in a second manner that is different from the first manner. and means for outputting a prompt to move the biometric feature in a second manner in response to a determination that enrollment prompt criteria have been met with respect to the second portion of the biometric feature present.

According to some examples, a method is described, comprising: in an electronic device having one or more biometric sensors and a display: on the display: an application interface corresponding to an application, and a biometric device controlled by an operating system of the electronic device. simultaneously displaying a measurement authentication interface, the biometric authentication interface being displayed over a portion of the application interface; While displaying a biometric authentication interface, obtaining biometric data corresponding to at least a portion of the biometric feature from one or more biometric sensors; and upon a determination, based on the biometric data, that at least a portion of the biometric feature satisfies the biometric authentication criteria: sending authentication information to the application indicating that the biometric authentication criteria have been satisfied with respect to one or more portions of the biometric feature. Steps provided to; and maintaining display of the biometric authentication interface for a predetermined period of time after providing authentication information to the application.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display. and one or more programs, on the display, for simultaneously displaying: an application interface corresponding to the application, and a biometric authentication interface controlled by the operating system of the electronic device, the biometric authentication interface being displayed over a portion of the application interface. became -; While displaying a biometric authentication interface, obtain biometric data corresponding to at least a portion of the biometric feature from one or more biometric sensors; and upon a determination, based on the biometric data, that at least a portion of the biometric feature satisfies the biometric authentication criteria: send authentication information to the application indicating that the biometric authentication criteria have been satisfied with respect to the one or more portions of the biometric feature; To provide to; and instructions for maintaining the display of the biometric authentication interface for a predetermined period of time after providing authentication information to the application.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display; , one or more programs for simultaneously displaying on the display: an application interface corresponding to an application, and a biometric authentication interface controlled by the operating system of the electronic device, the biometric authentication interface being displayed over a portion of the application interface. ; While displaying a biometric authentication interface, obtain biometric data corresponding to at least a portion of the biometric feature from one or more biometric sensors; and upon a determination, based on the biometric data, that at least a portion of the biometric feature satisfies the biometric authentication criteria: send authentication information to the application indicating that the biometric authentication criteria have been satisfied with respect to the one or more portions of the biometric feature; To provide to; and instructions for maintaining the display of the biometric authentication interface for a predetermined period of time after providing authentication information to the application.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are: on a display: an application interface corresponding to an application, and biometric authentication controlled by the operating system of the electronic device. for displaying interfaces simultaneously - the biometric authentication interface being displayed over a portion of the application interface; While displaying a biometric authentication interface, obtain biometric data corresponding to at least a portion of the biometric feature from one or more biometric sensors; and upon a determination, based on the biometric data, that at least a portion of the biometric feature satisfies the biometric authentication criteria: send authentication information to the application indicating that the biometric authentication criteria have been satisfied with respect to the one or more portions of the biometric feature; To provide to; and instructions for maintaining the display of the biometric authentication interface for a predetermined period of time after providing authentication information to the application.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; On a display: means for simultaneously displaying an application interface corresponding to an application and a biometric authentication interface controlled by the operating system of the electronic device, the biometric authentication interface being displayed over a portion of the application interface; means for obtaining, while displaying a biometric authentication interface, biometric data corresponding to at least a portion of the biometric feature from one or more biometric sensors; and upon a determination, based on the biometric data, that at least a portion of the biometric feature satisfies the biometric authentication criteria: sending authentication information to the application indicating that the biometric authentication criteria have been satisfied with respect to the one or more portions of the biometric feature. means to provide; and means for maintaining display of the biometric authentication interface for a predetermined period of time after providing authentication information to the application.

According to some examples, a method is described, the method comprising: in an electronic device having one or more biometric sensors and a display: displaying, on the display, an application interface including a fillable field; During displaying the application interface, receiving a request to automatically populate fillable fields of the application interface; and in response to receiving a request to automatically populate the fillable field of the application interface: in accordance with a determination that the fillable field of the application interface is associated with data of the first type, the fillable field with data of the first type. Automatically filling steps; and wherein the fillable field of the application is associated with a second type of data, and wherein at least a portion of the biometric characteristic is determined based on data obtained from one or more biometric sensors corresponding to the biometric characteristic. and automatically populating the fillable field with data of the second type in response to a determination that the conditions are satisfied.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium comprising one or more programs for execution by one or more processors of an electronic device having one or more biometric sensors and a display. and one or more programs for displaying, on a display, an application interface including a fillable field; while displaying the application interface, to receive a request to automatically populate fillable fields of the application interface; and in response to receiving a request to automatically populate a fillable field of the application interface: automatically populating the fillable field with data of a first type, in accordance with a determination that the fillable field of the application interface is associated with data of a first type. to fill with; and wherein the fillable field of the application is associated with a second type of data, and wherein at least a portion of the biometric characteristic is determined based on data obtained from one or more biometric sensors corresponding to the biometric characteristic. and instructions for automatically filling the fillable field with data of the second type, depending on the determination that the conditions are satisfied.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs for execution by one or more processors of an electronic device having one or more biometric sensors and a display; , one or more programs for displaying, on a display, an application interface including fillable fields; while displaying the application interface, to receive a request to automatically populate fillable fields in the application interface; and in response to receiving a request to automatically populate a fillable field of the application interface: automatically populating the fillable field with data of a first type, in accordance with a determination that the fillable field of the application interface is associated with data of a first type. to fill with; and wherein the fillable field of the application is associated with a second type of data, and wherein at least a portion of the biometric characteristic is determined based on data obtained from one or more biometric sensors corresponding to the biometric characteristic. and instructions for automatically filling the fillable field with data of the second type, depending on the determination that the conditions are satisfied.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are configured to: display, on a display, an application interface including fillable fields; while displaying the application interface, to receive a request to automatically populate fillable fields in the application interface; and in response to receiving a request to automatically populate a fillable field of the application interface: automatically populating the fillable field with data of a first type, in accordance with a determination that the fillable field of the application interface is associated with data of a first type. to fill with; and wherein the fillable field of the application is associated with a second type of data, and wherein at least a portion of the biometric characteristic is determined based on data obtained from one or more biometric sensors corresponding to the biometric characteristic. and instructions for automatically filling the fillable field with data of the second type, depending on the determination that the conditions are satisfied.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; means for displaying, on a display, an application interface including fillable fields; means for receiving a request to automatically populate fillable fields of the application interface while displaying the application interface; and in response to receiving a request to automatically populate the fillable field of the application interface: automatically populating the fillable field with data of the first type, in accordance with a determination that the fillable field of the application interface is associated with data of the first type. means for filling; and wherein the fillable field of the application is associated with a second type of data, and wherein at least a portion of the biometric characteristic is determined based on data obtained from one or more biometric sensors corresponding to the biometric characteristic. and means for automatically populating the fillable field with data of the second type in response to a determination that the conditions are satisfied.

According to some examples, a method is described, comprising: in an electronic device having one or more biometric sensors and a display: detecting that device wake criteria have been met; In response to detecting that device wake criteria have been met, transitioning the electronic device from a first visual state to a second visual state; and after transitioning the device to the second visual state: transitioning the electronic device from the second visual state in accordance with a determination that biometric authentication criteria have been met based on the biometric data provided by the one or more biometric sensors. 3 transitioning to a visual state - the transition from the second visual state to the third visual state is a continuation of the transition from the first visual state to the second visual state; and maintaining the electronic device in a second visual state in accordance with a determination that the biometric authentication criteria have not been met based on the biometric data provided by the one or more biometric sensors.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display. and one or more programs for detecting that device wake criteria have been met; transitioning the electronic device from a first visual state to a second visual state in response to detecting that device wake criteria have been met; and after transitioning the device to the second visual state: transitioning the electronic device from the second visual state to the third visual state upon a determination that biometric authentication criteria have been met based on biometric data provided by one or more biometric sensors. to transition to a state - the transition from the second visual state to the third visual state is a continuation of the transition from the first visual state to the second visual state; and instructions for maintaining the electronic device in a second visual state in accordance with a determination that biometric authentication criteria have not been met based on biometric data provided by the one or more biometric sensors.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display; , one or more programs to detect that device wake criteria have been met; transitioning the electronic device from a first visual state to a second visual state in response to detecting that device wake criteria have been met; and after transitioning the device to the second visual state: transitioning the electronic device from the second visual state to the third visual state upon a determination that biometric authentication criteria have been met based on biometric data provided by one or more biometric sensors. to transition to a state - the transition from the second visual state to the third visual state is a continuation of the transition from the first visual state to the second visual state; and instructions for maintaining the electronic device in a second visual state in accordance with a determination that biometric authentication criteria have not been met based on biometric data provided by the one or more biometric sensors.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are configured to: detect that device wake criteria have been met; transitioning the electronic device from a first visual state to a second visual state in response to detecting that device wake criteria have been met; and after transitioning the device to the second visual state: transitioning the electronic device from the second visual state to the third visual state upon a determination that biometric authentication criteria have been met based on biometric data provided by one or more biometric sensors. to transition to a state - the transition from the second visual state to the third visual state is a continuation of the transition from the first visual state to the second visual state; and instructions for maintaining the electronic device in a second visual state in accordance with a determination that biometric authentication criteria have not been met based on biometric data provided by the one or more biometric sensors.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; means for detecting that device wake criteria have been met; means for transitioning the electronic device from a first visual state to a second visual state in response to detecting that device wake criteria have been met; and after transitioning the device to the second visual state: transferring the electronic device from the second visual state to the third visual state, upon a determination that biometric authentication criteria have been met based on biometric data provided by one or more biometric sensors. means for transitioning to a state - the transition from the second visual state to the third visual state is a continuation of the transition from the first visual state to the second visual state; and means for maintaining the electronic device in a second visual state in accordance with a determination that biometric authentication criteria have not been met based on biometric data provided by the one or more biometric sensors.

According to some examples, a method is described, comprising: in an electronic device having one or more biometric sensors and a display: while the electronic device is in a locked state, without explicit input from the user requesting biometric authentication; detecting a condition associated with performing a biometric authentication verification using a biometric sensor; and in response to detecting the condition, performing a first biometric authentication verification, wherein performing the first biometric authentication verification comprises capturing first biometric data using one or more biometric sensors. ; After capturing the first biometric data: transitioning the device from a locked state to an unlocked state upon a determination that the first biometric data satisfies biometric authentication criteria; and maintaining the device in a locked state upon determining that the first biometric data does not meet biometric authentication criteria; After performing the first biometric authentication verification, detecting, via the device, a request to perform the respective action without receiving additional authentication information from the user; and in response to detecting a request to perform the respective operation: in accordance with a determination that the respective operation does not require authentication, performing the respective operation; performing each operation in accordance with a determination that each operation requires authentication and that the device is in an unlocked state; and upon determining that the respective operation requires authentication and the device is in a locked state: capturing second biometric data using one or more biometric sensors without explicit input from the user requesting verification of second biometric authentication. steps; and after capturing the second biometric data, performing a second biometric authentication verification, wherein performing the second biometric authentication verification determines whether the second biometric data meets biometric authentication criteria. Depending on the determination of satisfaction, performing each action; and suspending performance of the respective operation in response to a determination that the second biometric data does not meet the biometric authentication criteria.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display. and one or more programs to detect a condition associated with performing a biometric authentication verification using a biometric sensor without explicit input from the user requesting biometric authentication while the electronic device is in a locked state. for; and in response to detecting the condition, to perform a first biometric authentication verification - performing the first biometric authentication verification includes: capturing first biometric data using one or more biometric sensors; After capturing the first biometric data: transitioning the device from a locked state to an unlocked state following a determination that the first biometric data satisfies biometric authentication criteria; and maintaining the device in a locked state following a determination that the first biometric data does not meet biometric authentication criteria; After performing the first biometric authentication verification, detect, via the device, a request to perform the respective action without receiving additional authentication information from the user; and in response to detecting a request to perform the respective operation: to perform the respective operation, in accordance with a determination that the respective operation does not require authentication; to perform each operation in accordance with a determination that each operation requires authentication and that the device is in an unlocked state; and upon determining that the respective operation requires authentication and the device is in a locked state: capture second biometric data using one or more biometric sensors without explicit input from the user requesting verification of second biometric authentication; to do; and after capturing the second biometric data, instructions for performing a second biometric authentication verification, wherein performing the second biometric authentication verification determines that the second biometric data satisfies the biometric authentication criteria. performing each action according to the decision; and withholding performance of the respective operation in response to a determination that the second biometric data does not meet the biometric authentication criteria.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display; , one or more programs for detecting conditions associated with performing a biometric authentication verification using a biometric sensor without explicit input from the user requesting biometric authentication while the electronic device is in a locked state; and in response to detecting the condition, to perform a first biometric authentication verification - performing the first biometric authentication verification includes: capturing first biometric data using one or more biometric sensors; After capturing the first biometric data: transitioning the device from a locked state to an unlocked state following a determination that the first biometric data satisfies biometric authentication criteria; and maintaining the device in a locked state following a determination that the first biometric data does not meet biometric authentication criteria; After performing the first biometric authentication verification, detect, via the device, a request to perform the respective action without receiving additional authentication information from the user; and in response to detecting a request to perform the respective operation: to perform the respective operation, in accordance with a determination that the respective operation does not require authentication; to perform each operation in accordance with a determination that each operation requires authentication and that the device is in an unlocked state; and upon determining that the respective operation requires authentication and the device is in a locked state: capture second biometric data using one or more biometric sensors without explicit input from the user requesting verification of second biometric authentication; to do; and after capturing the second biometric data, instructions for performing a second biometric authentication verification, wherein performing the second biometric authentication verification determines that the second biometric data satisfies the biometric authentication criteria. performing each action according to the decision; and withholding performance of the respective operation in response to a determination that the second biometric data does not meet the biometric authentication criteria.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are configured to: measure biometric measurements without explicit input from the user requesting biometric authentication while the electronic device is in a locked state; to detect conditions associated with performing biometric authentication verification using a sensor; and in response to detecting the condition, to perform a first biometric authentication verification - performing the first biometric authentication verification includes: capturing first biometric data using one or more biometric sensors; After capturing the first biometric data: transitioning the device from a locked state to an unlocked state following a determination that the first biometric data satisfies biometric authentication criteria; and maintaining the device in a locked state following a determination that the first biometric data does not meet biometric authentication criteria; After performing the first biometric authentication verification, detect, via the device, a request to perform the respective action without receiving additional authentication information from the user; and in response to detecting a request to perform the respective operation: to perform the respective operation, in accordance with a determination that the respective operation does not require authentication; to perform each operation in accordance with a determination that each operation requires authentication and that the device is in an unlocked state; and upon determining that the respective operation requires authentication and the device is in a locked state: capture second biometric data using one or more biometric sensors without explicit input from the user requesting verification of second biometric authentication; to do; and after capturing the second biometric data, instructions for performing a second biometric authentication verification, wherein performing the second biometric authentication verification determines that the second biometric data satisfies the biometric authentication criteria. performing each action according to the decision; and withholding performance of the respective operation in response to a determination that the second biometric data does not meet the biometric authentication criteria.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; means for detecting a condition associated with performing a biometric authentication verification using a biometric sensor without explicit input from the user requesting biometric authentication while the electronic device is in a locked state; and in response to detecting the condition, means for performing a first biometric authentication verification, wherein the means for performing the first biometric authentication verification comprises capturing first biometric data using one or more biometric sensors. means for; After capturing the first biometric data: means for transitioning the device from a locked state to an unlocked state upon a determination that the first biometric data satisfies biometric authentication criteria; and means for maintaining the device in a locked state upon determining that the first biometric data does not meet biometric authentication criteria; means for detecting, after performing the first biometric authentication verification, a request to perform a respective action, via the device, without receiving additional authentication information from the user; and in response to detecting a request to perform the respective operation: means for performing the respective operation, in accordance with a determination that the respective operation does not require authentication; means for performing each operation in response to a determination that each operation requires authentication and that the device is in an unlocked state; and upon determining that the respective operation requires authentication and the device is in a locked state: capturing second biometric data using one or more biometric sensors without explicit input from the user requesting verification of second biometric authentication. means for doing so; and after capturing the second biometric data, means for performing a second biometric authentication verification, wherein the means for performing the second biometric authentication verification comprises: means for performing the second biometric authentication verification, wherein the second biometric data meets biometric authentication criteria; means for performing each operation upon a satisfactory determination; and means for withholding performance of the respective operation in response to a determination that the second biometric data does not meet biometric authentication criteria.

According to some examples, a method is described, the method comprising: in an electronic device having a display, a button, and one or more biometric sensors separate from the button: While in the state, detecting one or more activations of the button; and in response to detecting one or more activations of the button: capturing biometric data with one or more biometric sensors separate from the button; Upon determining that the biometric data satisfies biometric authentication criteria, transitioning the electronic device to a second state in which respective functions of the device are enabled; and, in accordance with the determination that the biometric data does not meet the biometric authentication criteria, maintaining the electronic device in a first state and displaying an indication on the display that biometric authentication has failed.

According to some examples, a non-transitory computer-readable medium is described, wherein the non-transitory computer-readable storage medium is operated by one or more processors of an electronic device having a display, a button, and one or more biometric sensors separate from the button. comprising one or more programs configured to execute, the one or more programs configured to: detect one or more activations of a button while the electronic device is in a first state in which a respective function of the device is disabled; and in response to detecting one or more activations of the button: to capture biometric data with one or more biometric sensors separate from the button; upon a determination that the biometric data satisfies the biometric authentication criteria, transitioning the electronic device to a second state in which respective functions of the device are enabled; and instructions for maintaining the electronic device in a first state and displaying an indication on the display that biometric authentication has failed, upon determining that the biometric data does not meet the biometric authentication criteria.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium configured to be executed by one or more processors of an electronic device having a display, a button, and one or more biometric sensors separate from the button. comprising one or more programs configured to: detect one or more activations of a button while the electronic device is in a first state in which the respective function of the device is disabled; and in response to detecting one or more activations of the button: to capture biometric data with one or more biometric sensors separate from the button; upon a determination that the biometric data satisfies the biometric authentication criteria, transitioning the electronic device to a second state in which respective functions of the device are enabled; and instructions for maintaining the electronic device in a first state and displaying an indication on the display that biometric authentication has failed, upon determining that the biometric data does not meet the biometric authentication criteria.

According to some examples, an electronic device is described, comprising: a display; button; one or more biometric sensors separate from the button; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, wherein the one or more programs are configured to: press one or more of the buttons while the electronic device is in a first state in which respective functions of the device are disabled; to detect activations; and in response to detecting one or more activations of the button: to capture biometric data with one or more biometric sensors separate from the button; upon a determination that the biometric data satisfies the biometric authentication criteria, transitioning the electronic device to a second state in which respective functions of the device are enabled; and instructions for maintaining the electronic device in a first state and displaying an indication on the display that biometric authentication has failed, upon determining that the biometric data does not meet the biometric authentication criteria.

According to some examples, an electronic device is described, comprising: a display; button; one or more biometric sensors separate from the button; means for detecting one or more activations of a button while the electronic device is in a first state in which the respective function of the device is disabled; and in response to detecting one or more activations of the button: means for capturing biometric data with one or more biometric sensors separate from the button; means for transitioning the electronic device to a second state in which respective functions of the device are enabled, upon a determination that the biometric data satisfies the biometric authentication criteria; and means for maintaining the electronic device in a first state and displaying an indication on the display that biometric authentication has failed, in accordance with the determination that the biometric data does not meet the biometric authentication criteria.

According to some examples, a method is described, comprising: at an electronic device having one or more biometric sensors and a display: detecting a request to perform a respective action requiring authentication; and in response to detecting a request to perform each action requiring authentication: in accordance with a determination that the device is unlocked, performing each action; and upon determining that the device is locked and that a first form of authentication is available for use, wherein the first form of authentication is a form of biometric authentication based on data obtained by one or more biometric sensors. and displaying an authentication indicator for a first form of authentication without displaying one or more affordances for using the second form of authentication.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display. and one or more programs for detecting a request to perform each operation requiring authentication; and in response to detecting a request to perform the respective action requiring authentication: to perform the respective action in accordance with a determination that the device is unlocked; and upon determining that the device is locked and that a first form of authentication is available for use, wherein the first form of authentication is a form of biometric authentication based on data obtained by one or more biometric sensors, on the display. and instructions for displaying an authentication indicator for a first form of authentication without displaying one or more affordances for using the second form of authentication.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors and a display; , one or more programs for detecting a request to perform each operation requiring authentication; and in response to detecting a request to perform the respective action requiring authentication: to perform the respective action in accordance with a determination that the device is unlocked; and upon determining that the device is locked and that a first form of authentication is available for use, wherein the first form of authentication is a form of biometric authentication based on data obtained by one or more biometric sensors, on the display. and instructions for displaying an authentication indicator for a first form of authentication without displaying one or more affordances for using the second form of authentication.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are configured to: detect a request to perform a respective operation requiring authentication; and in response to detecting a request to perform the respective action requiring authentication: to perform the respective action in accordance with a determination that the device is unlocked; and upon determining that the device is locked and that a first form of authentication is available for use, wherein the first form of authentication is a form of biometric authentication based on data obtained by one or more biometric sensors, on the display. and instructions for displaying an authentication indicator for a first form of authentication without displaying one or more affordances for using the second form of authentication.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; means for detecting a request to perform each operation requiring authentication; and in response to detecting a request to perform the respective action requiring authentication: means for performing the respective action in accordance with a determination that the device is unlocked; and upon determining that the device is locked and that a first form of authentication is available for use, wherein the first form of authentication is a form of biometric authentication based on data obtained by one or more biometric sensors. and means for displaying an authentication indicator for the first form of authentication without displaying one or more affordances for using the second form of authentication.

According to some examples, a method is described, comprising: at an electronic device having one or more biometric sensors: receiving a first request to perform a respective operation requiring authentication; In response to receiving a first request to perform the respective action: determining, using one or more biometric sensors, whether biometric authentication criteria are met, wherein the biometric authentication criteria are configured to perform the respective action. Includes a requirement that each type of biometric characteristic being authorized be detected by biometric sensors; performing each action in accordance with a determination that the biometric authentication criteria are met; and suspending performing the respective action upon a determination that the biometric authentication criteria are not met; Subsequent to determining that biometric authentication criteria have not been met in response to receiving the first request, receiving a second request to perform the respective action; and in response to receiving a second request to perform the respective operation: determining the biometric authentication criteria in response to the first request due to the failure of the one or more biometric sensors to detect the presence of the respective type of biometric characteristic. Depending on the determination that they are not met, determining whether the biometric authentication criteria are met using one or more biometric sensors in response to the second request; and a second request in response to a determination that biometric authentication criteria have not been met in response to the first request due to one or more biometric sensors detecting a respective type of biometric feature that does not correspond to an authorized biometric feature. and in response to, withholding the step of determining whether the biometric authentication criteria are met using the one or more biometric sensors.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors, One or more programs configured to receive a first request to perform a respective operation requiring authentication; In response to receiving a first request to perform the respective action: to determine whether the biometric authentication criteria are met using one or more biometric sensors - the biometric authentication criteria are authorized to perform the respective action Includes the requirement that each type of biometric characteristic being detected is detected by biometric sensors; to perform each action upon determining that the biometric authentication criteria are met; and to suspend performing the respective operation upon a determination that biometric authentication criteria are not met; Subsequent to determining that the biometric authentication criteria have not been met in response to receiving the first request, receive a second request to perform the respective action; and in response to receiving a second request to perform the respective action: the biometric authentication criteria are met in response to the first request due to one or more biometric sensors not detecting the presence of the respective type of biometric feature. upon a determination that it is not, to determine whether biometric authentication criteria are met using one or more biometric sensors in response to the second request; and upon a determination that the biometric authentication criteria have not been met in response to the first request due to one or more biometric sensors detecting a respective type of biometric characteristic that does not correspond to an authorized biometric characteristic, the second request. and instructions for withholding a determination of whether biometric authentication criteria are met using one or more biometric sensors in response to.

According to some examples, a temporary computer-readable medium is described, the temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors, and storing one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors. The programs are configured to: receive a first request to perform each operation requiring authentication; In response to receiving a first request to perform the respective action: to determine whether the biometric authentication criteria are met using one or more biometric sensors - the biometric authentication criteria are authorized to perform the respective action Includes the requirement that each type of biometric characteristic being detected is detected by biometric sensors; to perform each action upon determining that the biometric authentication criteria are met; and to suspend performing the respective operation upon a determination that biometric authentication criteria are not met; Subsequent to determining that the biometric authentication criteria have not been met in response to receiving the first request, receive a second request to perform the respective action; and in response to receiving a second request to perform the respective action: the biometric authentication criteria are met in response to the first request due to one or more biometric sensors not detecting the presence of the respective type of biometric feature. upon a determination that it is not, to determine whether biometric authentication criteria are met using one or more biometric sensors in response to the second request; and upon a determination that the biometric authentication criteria have not been met in response to the first request due to one or more biometric sensors detecting a respective type of biometric characteristic that does not correspond to an authorized biometric characteristic, the second request. and instructions for withholding a determination of whether biometric authentication criteria are met using one or more biometric sensors in response to.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are configured to: receive a first request to perform a respective operation requiring authentication; In response to receiving a first request to perform the respective action: to determine whether the biometric authentication criteria are met using one or more biometric sensors - the biometric authentication criteria are authorized to perform the respective action Includes the requirement that each type of biometric characteristic being detected is detected by biometric sensors; to perform each action upon determining that the biometric authentication criteria are met; and to suspend performing the respective operation upon a determination that biometric authentication criteria are not met; Subsequent to determining that the biometric authentication criteria have not been met in response to receiving the first request, receive a second request to perform the respective action; and in response to receiving a second request to perform the respective action: the biometric authentication criteria are met in response to the first request due to one or more biometric sensors not detecting the presence of the respective type of biometric feature. upon a determination that it is not, to determine whether biometric authentication criteria are met using one or more biometric sensors in response to the second request; and upon a determination that the biometric authentication criteria have not been met in response to the first request due to one or more biometric sensors detecting a respective type of biometric characteristic that does not correspond to an authorized biometric characteristic, the second request. and instructions for withholding a determination of whether biometric authentication criteria are met using one or more biometric sensors in response to.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; means for receiving a first request to perform each operation requiring authentication; In response to receiving a first request to perform the respective action: to determine whether the biometric authentication criteria are met using one or more biometric sensors - the biometric authentication criteria are authorized to perform the respective action Includes the requirement that each type of biometric characteristic being detected is detected by biometric sensors; to perform each action upon determining that the biometric authentication criteria are met; and means for withholding performance of the respective operation upon a determination that biometric authentication criteria are not met; means for receiving a second request to perform the respective action subsequent to a determination that biometric authentication criteria have not been met in response to receiving the first request; and in response to receiving a second request to perform the respective action: the biometric authentication criteria are met in response to the first request due to one or more biometric sensors not detecting the presence of the respective type of biometric characteristic. upon a determination that it has not been done, to determine whether biometric authentication criteria are met using one or more biometric sensors in response to the second request; and upon a determination that the biometric authentication criteria have not been met in response to the first request due to one or more biometric sensors detecting a respective type of biometric feature that does not correspond to an authorized biometric characteristic, the second request. and instructions for withholding a determination of whether biometric authentication criteria are met using one or more biometric sensors in response to.

According to some examples, a method is described, comprising: at an electronic device having one or more biometric sensors: receiving a first request to perform a first operation requiring authentication; In response to receiving a first request to perform a first operation: determining, using one or more biometric sensors, whether first biometric authentication criteria are met, wherein the first biometric authentication criteria are: Includes a requirement that each type of biometric characteristic authorized to perform an operation is detected by biometric sensors; In accordance with a determination that first biometric authentication criteria are met, performing a first operation; and in response to a determination that the biometric authentication criteria are not met, withholding performing the first action; After performing the first operation, receiving a second request to perform a second operation requiring authentication; and in response to receiving the second request: in accordance with the determination that the re-authentication criteria have been met, determining whether the second biometric authentication criteria are met using one or more biometric sensors - second biometric authentication. The criteria include the requirement that each type of biometric characteristic authorized to perform the second operation is detected by biometric sensors; and in accordance with the determination that the re-authentication criteria have not been met, performing a second operation without performing biometric authentication and withholding the step of determining whether the second biometric authentication criteria are met using the one or more biometric sensors. It includes steps to:

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors, One or more programs configured to receive a first request to perform a first operation requiring authentication; In response to receiving a first request to perform a first action: to determine whether first biometric authentication criteria are met using one or more biometric sensors, wherein the first biometric authentication criteria comprises: Includes a requirement that each type of biometric characteristic that is authorized to perform is detected by biometric sensors; in accordance with a determination that the first biometric authentication criteria are met, to perform a first action; and upon a determination that the biometric authentication criteria are not met, withhold performing the first action; After performing the first action, receive a second request to perform a second action requiring authentication; and in response to receiving the second request: in accordance with a determination that the re-authentication criteria have been met, to determine whether the second biometric authentication criteria have been met using one or more biometric sensors - a second biometric authentication criterion. These include the requirement that each type of biometric characteristic authorized to perform the second operation is detected by the biometric sensors; and in accordance with the determination that the re-authentication criteria have not been met, performing a second operation without performing biometric authentication and withholding from determining whether the second biometric authentication criteria are met using the one or more biometric sensors. Includes commands for

According to some examples, a temporary computer-readable medium is described, the temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors, and storing one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors. The programs are configured to: receive a first request to perform a first operation requiring authentication; In response to receiving a first request to perform a first action: to determine whether first biometric authentication criteria are met using one or more biometric sensors, wherein the first biometric authentication criteria comprises: Includes a requirement that each type of biometric characteristic that is authorized to perform is detected by biometric sensors; in accordance with a determination that the first biometric authentication criteria are met, to perform a first action; and upon a determination that the biometric authentication criteria are not met, withhold performing the first operation; After performing the first action, receive a second request to perform a second action requiring authentication; and in response to receiving the second request: in accordance with a determination that the re-authentication criteria have been met, to determine whether the second biometric authentication criteria are met using one or more biometric sensors - a second biometric authentication criterion. These include the requirement that each type of biometric characteristic authorized to perform the second operation is detected by the biometric sensors; and in accordance with the determination that the re-authentication criteria have not been met, performing a second operation without performing biometric authentication and withholding from determining whether the second biometric authentication criteria are met using the one or more biometric sensors. Includes commands for

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are configured to: receive a first request to perform a first operation requiring authentication; In response to receiving a first request to perform a first action: to determine whether first biometric authentication criteria are met using one or more biometric sensors, wherein the first biometric authentication criteria comprises: Includes a requirement that each type of biometric characteristic that is authorized to perform is detected by biometric sensors; in accordance with a determination that the first biometric authentication criteria are met, to perform a first action; and upon a determination that the biometric authentication criteria are not met, withhold performing the first operation; After performing the first action, receive a second request to perform a second action requiring authentication; and in response to receiving the second request: in accordance with a determination that the re-authentication criteria have been met, to determine whether the second biometric authentication criteria are met using one or more biometric sensors - a second biometric authentication criterion. These include the requirement that each type of biometric characteristic authorized to perform the second operation is detected by the biometric sensors; and in accordance with the determination that the re-authentication criteria have not been met, performing a second operation without performing biometric authentication and withholding from determining whether the second biometric authentication criteria are met using the one or more biometric sensors. Includes commands for

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; means for receiving a first request to perform a first operation requiring authentication; In response to receiving a first request to perform a first action: to determine whether first biometric authentication criteria are met using one or more biometric sensors, wherein the first biometric authentication criteria comprises: Includes a requirement that each type of biometric characteristic that is authorized to perform is detected by biometric sensors; in accordance with a determination that the first biometric authentication criteria are met, to perform a first action; and means for withholding from performing the first operation upon a determination that the biometric authentication criteria are not met; means for receiving, after performing the first operation, a second request to perform a second operation requiring authentication; and in response to receiving the second request: in accordance with a determination that the re-authentication criteria have been met, to determine whether the second biometric authentication criteria are met using one or more biometric sensors - a second biometric authentication criterion. These include the requirement that each type of biometric characteristic authorized to perform the second operation is detected by the biometric sensors; and in accordance with the determination that the re-authentication criteria have not been met, performing a second operation without performing biometric authentication and withholding from determining whether the second biometric authentication criteria are met using the one or more biometric sensors. Includes means for

According to some examples, a method is described, comprising: at an electronic device having a display: receiving a request to display a first portion of respective content; and in response to the request to display the first portion of the respective content: displaying at least a first portion of the respective content on the display, each content comprising an element associated with an authentication operation; initiating biometric authentication upon determining that an element associated with the authentication operation meets visibility criteria; and suspending initiating biometric authentication upon determining that the element associated with the authentication operation does not meet visibility criteria.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having a display, the one or more programs configured to be executed by one or more processors of an electronic device having a display. , for receiving a request to display a first portion of the respective content; and in response to a request to display the first portion of the respective content: to display at least a first portion of the respective content on the display, each content comprising an element associated with an authentication operation; to initiate biometric authentication upon determining that an element associated with the authentication operation meets visibility criteria; and instructions for withholding from initiating biometric authentication upon determining that an element associated with the authentication operation does not meet visibility criteria.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having a display, the one or more programs each comprising: to receive a request to display a first portion of the content of; and in response to a request to display the first portion of the respective content: to display at least a first portion of the respective content on the display, each content comprising an element associated with an authentication operation; to initiate biometric authentication upon determining that an element associated with the authentication operation meets visibility criteria; and instructions for withholding from initiating biometric authentication upon determining that an element associated with the authentication operation does not meet visibility criteria.

According to some examples, an electronic device is described, comprising: a display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are configured to: receive a request to display a first portion of the respective content; and in response to a request to display the first portion of the respective content: to display at least a first portion of the respective content on the display, each content comprising an element associated with an authentication operation; to initiate biometric authentication upon determining that an element associated with the authentication operation meets visibility criteria; and instructions for withholding from initiating biometric authentication upon determining that an element associated with the authentication operation does not meet visibility criteria.

According to some examples, an electronic device is described, comprising: a display; means for receiving a request to display a first portion of each content; and in response to a request to display the first portion of the respective content: to display at least a first portion of the respective content on the display, each content comprising an element associated with an authentication operation; to initiate biometric authentication upon determining that an element associated with the authentication operation meets visibility criteria; and means for withholding from initiating biometric authentication upon determining that an element associated with the authentication operation does not meet visibility criteria.

According to some examples, a method is described, comprising: at an electronic device having a display and one or more biometric sensors: detecting a predefined action corresponding to a credential submission user interface having a credential submission user interface element; step; and in response to detecting the predefined action: upon determining that biometric authentication via one or more biometric sensors is available, display, on the display, one or more biometric features that meet the biometric authentication criteria. and displaying a credential submission user interface with a visual indication that presenting to the sensors will cause credentials to be submitted via the credential submission user interface element.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having a display and one or more biometric sensors. and one or more programs for detecting a predefined action corresponding to a credential submission user interface having a credential submission user interface element; and in response to detecting the predefined action: upon determining that biometric authentication via one or more biometric sensors is available, display, on the display, a biometric feature that meets the biometric authentication criteria. Includes instructions for displaying a credential submission user interface with a visual indication that presenting credentials will cause credentials to be submitted through the credential submission user interface element.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having a display and one or more biometric sensors, One or more programs for detecting a predefined action corresponding to a credential submission user interface with a credential submission user interface element; and in response to detecting the predefined action: upon determining that biometric authentication via one or more biometric sensors is available, display, on the display, a biometric feature that meets the biometric authentication criteria. Includes instructions for displaying a credential submission user interface with a visual indication that presenting credentials will cause credentials to be submitted through the credential submission user interface element.

According to some examples, an electronic device is described, comprising: a display; one or more biometric sensors; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are configured to: detect a predefined action corresponding to a credential submission user interface having a credential submission user interface element; ; and in response to detecting the predefined action: upon determining that biometric authentication via one or more biometric sensors is available, display, on the display, a biometric feature that meets the biometric authentication criteria. Includes instructions for displaying a credential submission user interface with a visual indication that presenting credentials will cause credentials to be submitted through the credential submission user interface element.

According to some examples, an electronic device is described, comprising: a display; one or more biometric sensors; means for detecting a predefined action corresponding to a credential submission user interface with a credential submission user interface element; and in response to detecting the predefined action: upon determining that biometric authentication via one or more biometric sensors is available, display, on the display, a biometric feature that meets the biometric authentication criteria. and means for displaying a credential submission user interface with a visual indication that presenting credentials will cause credentials to be submitted through the credential submission user interface element.

According to some examples, a method is described, comprising: in an electronic device having a touch-sensitive display and one or more biometric sensors: a credential entry user interface having a plurality of character input keys on the touch-sensitive display; displaying; During the step of displaying a credential entry user interface, receiving, via the touch-sensitive display, a touch gesture input comprising movement of a contact on the touch-sensitive display; and in response to receiving a touch gesture input comprising movement of a contact on the touch-sensitive display: upon determining that one or more criteria of the first set are met—one or more criteria of the first set are: biometric authentication is performed; Attempting to biometrically authenticate a user of the electronic device based on biometric information captured using one or more biometric sensors, including a requirement that the electronic device is currently enabled on the electronic device.

According to some examples, a non-transitory computer-readable medium is described, wherein the non-transitory computer-readable storage medium is configured to be executed by one or more processors of an electronic device having a touch-sensitive display and one or more biometric sensors. Comprising programs, one or more programs for displaying, on a touch-sensitive display, a credential entry user interface having a plurality of character entry keys; While displaying a credential entry user interface, receive, through the touch-sensitive display, a touch gesture input, including movement of a contact on the touch-sensitive display; and in response to receiving a touch gesture input comprising movement of a contact on the touch-sensitive display: upon determining that one or more criteria of the first set are met, wherein the one or more criteria of the first set are: Includes instructions for attempting to biometrically authenticate a user of an electronic device based on biometric information captured using one or more biometric sensors - including a requirement that the device is currently enabled on the device.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having a touch-sensitive display and one or more biometric sensors. and one or more programs for displaying, on a touch-sensitive display, a credential entry user interface having a plurality of character entry keys; While displaying a credential entry user interface, receive, through the touch-sensitive display, a touch gesture input, including movement of a contact on the touch-sensitive display; and in response to receiving a touch gesture input comprising movement of a contact on the touch-sensitive display: upon determining that one or more criteria of the first set are met, wherein the one or more criteria of the first set are: Includes instructions for attempting to biometrically authenticate a user of an electronic device based on biometric information captured using one or more biometric sensors - including a requirement that the device is currently enabled on the device.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; touch-sensitive display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are configured to: display, on a touch-sensitive display, a credential entry user interface having a plurality of character entry keys; While displaying a credential entry user interface, receive, through the touch-sensitive display, a touch gesture input, including movement of a contact on the touch-sensitive display; and in response to receiving a touch gesture input comprising movement of a contact on the touch-sensitive display: upon determining that one or more criteria of the first set are met, wherein the one or more criteria of the first set are: Includes instructions for attempting to biometrically authenticate a user of an electronic device based on biometric information captured using one or more biometric sensors - including a requirement that the device is currently enabled on the device.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; touch-sensitive display; means for displaying, on a touch-sensitive display, a credential entry user interface having a plurality of character entry keys; means for receiving, via the touch-sensitive display, a touch gesture input, including movement of a contact on the touch-sensitive display, while displaying a credential entry user interface; and in response to receiving a touch gesture input comprising movement of a contact on the touch-sensitive display: upon determining that one or more criteria of the first set are met—one or more criteria of the first set are: and means for attempting to biometrically authenticate a user of the electronic device based on biometric information captured using one or more biometric sensors.

According to some embodiments, a method is described, comprising, at an electronic device having a display and one or more input devices: receiving, via the one or more input devices, a request to perform an operation requiring authentication; and in response to the request to perform an action requiring authentication: in accordance with a determination that authentication is successful, performing the action; and upon a determination that authentication is not successful and a set of error condition criteria is met: displaying an indication of an error condition on a display, wherein the indication includes information regarding the cause of the error condition; and suspending the step of performing the operation.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having a display and one or more input devices; , one or more programs for receiving, via one or more input devices, a request to perform an operation requiring authentication; and in response to a request to perform an action requiring authentication: to perform the action upon a determination that authentication is successful; and upon a determination that authentication is not successful and a set of error condition criteria are met: to display an indication of the error condition on a display, wherein the indication includes information regarding the cause of the error condition; It also includes instructions for suspending the execution of an action.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having a display and one or more input devices, The above programs are for receiving, via one or more input devices, a request to perform an operation requiring authentication; and in response to a request to perform an action requiring authentication: to perform the action upon a determination that authentication is successful; and upon a determination that authentication is not successful and a set of error condition criteria is met: to display an indication of the error condition on a display, wherein the indication includes information regarding the cause of the error condition; It also includes instructions for suspending the execution of an action.

According to some examples, an electronic device is described, the electronic device comprising: one or more input devices; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs are configured to: receive, via one or more input devices, a request to perform an operation requiring authentication; and in response to a request to perform an action requiring authentication: to perform the action upon a determination that authentication is successful; and upon a determination that authentication is not successful and a set of error condition criteria is met: to display an indication of the error condition on a display, wherein the indication includes information regarding the cause of the error condition; It also includes instructions for suspending the execution of an action.

According to some examples, an electronic device is described, the electronic device comprising: one or more input devices; display; means for receiving, via one or more input devices, a request to perform an operation requiring authentication; and in response to a request to perform an action requiring authentication: to perform the action upon a determination that authentication is successful; and upon a determination that authentication is not successful and a set of error condition criteria is met: to display an indication of the error condition on a display, wherein the indication includes information regarding the cause of the error condition; and means for suspending performance of the operation.

According to some examples, a method is described, the method comprising: in an electronic device having a biometric sensor and a display in a first portion of the electronic device: preventing the biometric sensor from obtaining biometric information about a user of the device; detecting the presence of an error condition; and in response to detecting the presence of an error condition, displaying an error indication on the display, wherein the error indication is displayed at a location proximate to the first portion of the electronic device, displaying the error indication comprising: in response to determining that the user interface of the electronic device is in a first orientation relative to the biometric sensor, displaying an error indication at a first location in the user interface proximate the first portion of the electronic device; and, in response to determining that the user interface of the electronic device is in a second orientation relative to the biometric sensor, displaying an error indication at a second location in the user interface proximate the first portion of the electronic device, wherein the first orientation is different from the second orientation.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium configured to be executed by one or more processors of an electronic device having a biometric sensor and a display in a first portion of the electronic device. comprising one or more programs configured to: detect the presence of an error condition that prevents the biometric sensor from obtaining biometric information about a user of the device; and in response to detecting the presence of an error condition, instructions for displaying an error indication on the display, wherein the error indication is displayed at a location proximate to the first portion of the electronic device, wherein displaying the error indication comprises: in response to determining that the user interface of the device is in a first orientation relative to the biometric sensor, displaying an error indication at a first location in the user interface proximate the first portion of the electronic device; and, in response to determining that the user interface of the electronic device is in a second orientation relative to the biometric sensor, displaying an error indication at a second location in the user interface proximate the first portion of the electronic device, wherein the first orientation is: It is different from the second orientation.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium configured to be executed by one or more processors of an electronic device having a biometric sensor and a display in a first portion of the electronic device. One or more programs, including the above programs, are configured to: detect the presence of an error condition that prevents the biometric sensor from obtaining biometric information about the user of the device; and in response to detecting the presence of an error condition, instructions for displaying an error indication on the display, wherein the error indication is displayed at a location proximate to the first portion of the electronic device, wherein displaying the error indication comprises: in response to determining that the user interface of the device is in a first orientation relative to the biometric sensor, displaying an error indication at a first location in the user interface proximate the first portion of the electronic device; and, in response to determining that the user interface of the electronic device is in a second orientation relative to the biometric sensor, displaying an error indication at a second location in the user interface proximate the first portion of the electronic device, wherein the first orientation is: It is different from the second orientation.

According to some examples, an electronic device is described, the electronic device comprising: a biometric sensor in a first portion of the electronic device; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, wherein the one or more programs: detect the presence of an error condition that prevents the biometric sensor from obtaining biometric information about the user of the device; to do; and in response to detecting the presence of an error condition, instructions for displaying an error indication on the display, wherein the error indication is displayed at a location proximate to the first portion of the electronic device, wherein displaying the error indication comprises: in response to determining that the user interface of the device is in a first orientation relative to the biometric sensor, displaying an error indication at a first location in the user interface proximate the first portion of the electronic device; and, in response to determining that the user interface of the electronic device is in a second orientation relative to the biometric sensor, displaying an error indication at a second location in the user interface proximate the first portion of the electronic device, wherein the first orientation is: It is different from the second orientation.

According to some examples, an electronic device is described, the electronic device comprising: a biometric sensor in a first portion of the electronic device; display; means for detecting the presence of an error condition that prevents the biometric sensor from obtaining biometric information about the user of the device; and means for displaying an error indication on the display in response to detecting the presence of an error condition, wherein the error indication is displayed at a location proximate to the first portion of the electronic device, wherein displaying the error indication comprises: in response to determining that the user interface of the device is in a first orientation with respect to the biometric sensor, displaying an error indication at a first location in the user interface proximate the first portion of the electronic device; and, in response to determining that the user interface of the electronic device is in a second orientation relative to the biometric sensor, displaying an error indication at a second location in the user interface proximate the first portion of the electronic device, wherein the first orientation is: It is different from the second orientation.

According to some examples, a method is described, comprising: in an electronic device having a display and one or more biometric sensors: a biometric registration user interface for initiating biometric registration with the one or more biometric sensors, on the display; displaying; While displaying a biometric registration user interface, receiving input corresponding to a request to initiate biometric registration; and in response to receiving input: initiating a process to register the biometric feature with one or more biometric sensors in accordance with a determination that the orientation of the electronic device satisfies the set of registration criteria; and, upon determining that the orientation of the electronic device does not satisfy the set of registration criteria, outputting one or more prompts to change the orientation of the electronic device to a different orientation that satisfies the set of registration criteria.

According to some examples, a non-transitory computer-readable medium is described, the non-transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having a display and one or more biometric sensors. and one or more programs configured to display, on a display, a biometric registration user interface for initiating biometric registration with one or more biometric sensors; while displaying a biometric registration user interface, to receive input corresponding to a request to initiate biometric registration; and in response to receiving the input: to initiate a process to register the biometric characteristic with one or more biometric sensors in accordance with a determination that the orientation of the electronic device satisfies the set of registration criteria; and instructions for outputting one or more prompts to change the orientation of the electronic device to a different orientation that satisfies the set of registration criteria, in response to a determination that the orientation of the electronic device does not satisfy the set of registration criteria.

According to some examples, a transitory computer-readable medium is described, the transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of an electronic device having a display and one or more biometric sensors, One or more programs configured to display, on a display, a biometric registration user interface for initiating biometric registration with one or more biometric sensors; while displaying a biometric registration user interface, to receive input corresponding to a request to initiate biometric registration; and in response to receiving the input: to initiate a process to register the biometric characteristic with one or more biometric sensors in accordance with a determination that the orientation of the electronic device satisfies the set of registration criteria; and instructions for outputting one or more prompts to change the orientation of the electronic device to a different orientation that satisfies the set of registration criteria, in response to a determination that the orientation of the electronic device does not satisfy the set of registration criteria.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; one or more processors; and a memory storing one or more programs configured to be executed by one or more processors, the one or more programs comprising: on a display, a biometric registration user interface for initiating biometric registration with one or more biometric sensors; to display; while displaying a biometric registration user interface, to receive input corresponding to a request to initiate biometric registration; and in response to receiving the input: to initiate a process to register the biometric characteristic with one or more biometric sensors in accordance with a determination that the orientation of the electronic device satisfies the set of registration criteria; and instructions for outputting one or more prompts to change the orientation of the electronic device to a different orientation that satisfies the set of registration criteria, in response to a determination that the orientation of the electronic device does not satisfy the set of registration criteria.

According to some examples, an electronic device is described, the electronic device comprising: one or more biometric sensors; display; means for displaying, on the display, a biometric registration user interface for initiating biometric registration with one or more biometric sensors; means for receiving input corresponding to a request to initiate biometric registration while displaying a biometric registration user interface; and in response to receiving input: to initiate a process to register the biometric characteristic with one or more biometric sensors in accordance with a determination that the orientation of the electronic device satisfies the set of registration criteria; and means for outputting one or more prompts to change the orientation of the electronic device to a different orientation that satisfies the set of registration criteria, in response to a determination that the orientation of the electronic device does not satisfy the set of registration criteria.

Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors. Executable instructions for performing these functions are optionally included in a transient computer-readable storage medium or other computer program product configured for execution by one or more processors.

Accordingly, devices are provided with faster and more efficient methods and interfaces for implementing biometric authentication, thereby increasing the effectiveness, efficiency and user satisfaction of such devices. These methods and interfaces optionally complement or replace other methods for implementing biometric authentication.

For a better understanding of the various examples described, reference should be made to the drawings below, where like reference numerals indicate corresponding parts throughout the drawings, and the specific details for carrying out the invention below should be referred to.
1A is a block diagram illustrating a portable multifunction device with a touch-sensitive display, according to some embodiments.
1B is a block diagram illustrating example components for event processing, according to some embodiments.
1C is a block diagram illustrating example components for generating tactile output, according to some embodiments.
2 illustrates a portable multifunction device with a touch screen, according to some embodiments.
3 is a block diagram of an example multifunction device having a display and a touch-sensitive surface, according to some embodiments.
4A illustrates an example user interface for a menu of applications on a portable multifunction device, according to some embodiments.
4B illustrates an example user interface for a multifunction device with a touch-sensitive surface separate from the display, according to some embodiments.
4C-4H illustrate example tactile output patterns with specific waveforms, according to some embodiments.
5A illustrates a personal electronic device according to some embodiments.
5B is a block diagram illustrating a personal electronic device according to some embodiments.
5C and 5D illustrate example components of a personal electronic device with a touch-sensitive display and intensity sensors, according to some embodiments.
5E-5H illustrate example components and user interfaces of a personal electronic device, according to some embodiments.
6 illustrates example devices connected via one or more communication channels, according to some embodiments.
7A-7S illustrate example user interfaces for the biometric registration process instructions, according to some examples.
8A-8C are flowcharts illustrating methods of the biometric registration process guidelines.
9A-9AE illustrate example user interfaces for aligning biometric features for registration.
10A-10F are flow diagrams illustrating methods for aligning biometric features for registration.
11A-11O illustrate example user interfaces for registering biometric characteristics.
12A and 12B are flow diagrams illustrating methods for registering biometric characteristics.
13A-13R illustrate example user interfaces for providing hints during the biometric registration process.
14A-14C are flow diagrams illustrating methods for providing hints during the biometric registration process.
15A-15T illustrate example user interfaces for application-based biometric authentication.
16A-16E are flowcharts illustrating methods of application-based biometric authentication.
17A-17AI illustrate example user interfaces for automatically populating biometric security fields.
18A-18D are flow diagrams illustrating methods for automatically populating biometric security fields.
19A-19AB illustrate example user interfaces for unlocking a device using biometric authentication.
20A-20F are flowcharts illustrating methods for unlocking a device using biometric authentication.
Figures 21A-21aq illustrate example user interfaces for retrying biometric authentication.
Figures 22A-22F are flow diagrams illustrating methods for retrying biometric authentication.
Figures 23A-23Q illustrate example user interfaces for managing transmissions using biometric authentication.
24A-24B illustrate example user interfaces for managing transmissions using biometric authentication.
Figures 25A-25C are flow diagrams illustrating methods for managing transmissions using biometric authentication.
26A-26AS illustrate example user interfaces for providing interstitial user interfaces during biometric authentication.
Figures 27A-27E are flow diagrams illustrating methods for providing embeddable user interfaces during biometric authentication.
28A-28AA illustrate example user interfaces for preventing retry of biometric authentication.
29A and 29B are flowcharts illustrating methods for preventing retry of biometric authentication.
30A-30A illustrate example user interfaces for cached biometric authentication.
31A and 31B are flowcharts illustrating methods of cached biometric authentication.
Figures 32A-32W illustrate example user interfaces for automatically populating fillable fields based on visibility criteria.
Figure 33 is a flow diagram illustrating methods for automatically populating fillable fields based on visibility criteria.
Figures 34A-34N illustrate example user interfaces for automatic login using biometric authentication.
35 is a flow chart illustrating methods of automatic login using biometric authentication.
36A-36L illustrate example user interfaces for retrying biometric authentication in a credential entry user interface, according to some examples.
37A and 37B are flow diagrams illustrating a method for reattempting biometric authentication in a credential entry user interface using an electronic device, according to some examples.
38A-38A illustrate example user interfaces for providing indications of error conditions during biometric authentication, according to some examples.
Figures 39A and 39B are flow diagrams illustrating a method for providing indications of error conditions during biometric authentication, according to some examples.
40A-40U illustrate example user interfaces for providing indications regarding a biometric sensor during biometric authentication, according to some examples.
Figures 41A-41C are flow diagrams illustrating a method for providing indications regarding a biometric sensor during biometric authentication, according to some examples.
42A-42P illustrate example user interfaces for orienting a device to register biometric features, according to some examples.
43A-43C are flow diagrams illustrating a method for orienting a device to register biometric features, according to some examples.

The following description sets forth exemplary methods, parameters, etc. However, it should be recognized that this description is not intended as a limitation on the scope of the disclosure but is instead provided as a description of example embodiments.

There is a need for electronic devices that provide efficient methods and interfaces for implementing biometric authentication of biometric features. For example, a need exists for electronic devices that provide a convenient and efficient method for registering one or more portions of a biometric characteristic. As another example, there is a need for electronic devices that provide a quick and intuitive technique for selectively accessing secure data based on biometric authentication. As another example, there is a need for electronic devices that provide a quick and intuitive technique for enabling device functionality following biometric authentication. Such techniques can reduce the cognitive burden on users registering biometric characteristics and/or biometrically authenticating using a device, thereby improving overall productivity. Additionally, such techniques can reduce processor and battery power that would otherwise be wasted on extra user input.

Below, FIGS. 1A-1C, 2, 3, 4A, 4B, and 5A-5H provide descriptions of example devices for performing techniques for implementing biometric authentication. 6 illustrates example devices connected via one or more communication channels, according to some embodiments. 7A-7S illustrate example user interfaces for biometric registration process instructions, according to some embodiments. 8A-8C are flowcharts illustrating methods of the biometric registration process guidelines. The user interfaces of FIGS. 7A-7S are used to illustrate the processes described below, including the processes of FIGS. 8A-8C. 9A-9AE illustrate example user interfaces for aligning biometric features for registration. 10A-10F are flow diagrams illustrating methods for aligning biometric features for registration. The user interfaces of FIGS. 9A-9AE are used to illustrate the processes described below, including the processes of FIGS. 10A-10F. 11A-11O illustrate example user interfaces for registering biometric characteristics. 12A and 12B are flow diagrams illustrating methods for registering biometric characteristics. The user interfaces of FIGS. 11A-11O are used to illustrate the processes described below, including the processes of FIGS. 12A and 12B. 13A-13R illustrate example user interfaces for providing hints during the biometric registration process. 14A-14C are flow diagrams illustrating methods for providing hints during the biometric registration process. The user interfaces of FIGS. 13A-13R are used to illustrate the processes described below, including the processes of FIGS. 14A-14C. 15A-15T illustrate example user interfaces for application-based biometric authentication. 16A-16E are flowcharts illustrating methods of application-based biometric authentication. The user interfaces of FIGS. 15A-15T are used to illustrate the processes described below, including the processes of FIGS. 16A-16E. 17A-17AI illustrate example user interfaces for automatically populating biometric security fields. 18A-18D are flow diagrams illustrating methods for automatically populating biometric security fields. The user interfaces of FIGS. 17A-17AI are used to illustrate the processes described below, including the processes of FIGS. 18A-18D. 19A-19AB illustrate example user interfaces for unlocking a device using biometric authentication. 20A-20F are flowcharts illustrating methods for unlocking a device using biometric authentication. The user interfaces of FIGS. 19A-19AB are used to illustrate the processes described below, including the processes of FIGS. 20A-20F. Figures 21A-21aq illustrate example user interfaces for retrying biometric authentication. Figures 22A-22F are flow diagrams illustrating methods for retrying biometric authentication. The user interfaces of Figures 21A-21aq are used to illustrate the processes described below, including the processes of Figures 22A-22F. Figures 23A-23Q illustrate example user interfaces for managing transmissions using biometric authentication. 24A-24B illustrate example user interfaces for managing transmissions using biometric authentication. Figures 25A-25C are flow diagrams illustrating methods for managing transmissions using biometric authentication. The user interfaces of FIGS. 23A-23Q and 24A-24B are used to illustrate the processes described below, including the processes of FIGS. 25A-25C. 26A-26AS illustrate example user interfaces for providing embeddable user interfaces during biometric authentication. Figures 27A-27E are flow diagrams illustrating methods for providing embeddable user interfaces during biometric authentication. The user interfaces of FIGS. 26A-26AS are used to illustrate the processes described below, including the processes of FIGS. 27A-27E. 28A-28AA illustrate example user interfaces for preventing retry of biometric authentication. 29A and 29B are flowcharts illustrating methods for preventing retry of biometric authentication. The user interfaces of FIGS. 28A-28AA are used to illustrate the processes described below, including the processes of FIGS. 29A and 29B. 30A-30A illustrate example user interfaces for cached biometric authentication. 31A and 31B are flow diagrams illustrating methods of cached biometric authentication. The user interfaces of FIGS. 30A-30A are used to illustrate the processes described below, including the processes of FIGS. 31A and 31B. Figures 32A-32W illustrate example user interfaces for automatically populating fillable fields based on visibility criteria. Figure 33 is a flow diagram illustrating methods for automatically populating fillable fields based on visibility criteria. The user interfaces of Figures 32A-32W are used to illustrate the processes described below, including the processes of Figure 33. Figures 34A-34N illustrate example user interfaces for automatic login using biometric authentication. 35 is a flow diagram illustrating methods of automatic login using biometric authentication. The user interfaces of Figures 34A-34N are used to illustrate the processes described below, including the processes of Figure 35. Figures 36A-36L illustrate example user interfaces for retrying biometric authentication in a credential entry user interface. Figures 37A and 37B are flow diagrams illustrating a method for retrying biometric authentication in a credential entry user interface. The user interfaces of FIGS. 36A-36L are used to illustrate the processes described below, including the processes of FIGS. 37A and 37B. Figures 38A-38A illustrate example user interfaces for providing indications of error conditions during biometric authentication. Figures 39A and 39B are flow diagrams illustrating a method for providing indications of error conditions during biometric authentication. The user interfaces of Figures 38A-38A are used to illustrate the processes described below, including the processes of Figures 39A and 39B. 40A-40U illustrate example user interfaces for providing indications regarding a biometric sensor during biometric authentication. Figures 41A-41C are flow diagrams illustrating a method for providing indications regarding a biometric sensor during biometric authentication. The user interfaces of FIGS. 40A-40U are used to illustrate the processes described below, including the processes of FIGS. 41A-41C. 42A-42P illustrate example user interfaces for orienting a device to register biometric features. Figures 43A-43C are flow diagrams illustrating a method for orienting a device to register biometric features. The user interfaces of Figures 42A-42P are used to illustrate the processes described below, including the processes of Figures 43A-43C.

Although the following description uses terms such as “first,” “second,” etc. to describe various elements, the elements should not be limited by those terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the various embodiments described, a first touch may be referred to as a second touch, and similarly, a second touch may be referred to as a first touch. Although the first touch and the second touch are both touches, they are not the same touch.

The terminology used in the description of various embodiments described herein is for the purpose of describing specific embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless the context clearly dictates otherwise. It is intended. Additionally, it will be understood that the term “and/or” as used herein represents and encompasses any and all possible combinations of one or more of the associated listed items. The terms "include", "including", "comprise", and/or "comprising", when used herein, refer to the features referred to, an integer. Specifies the presence of features, steps, operations, elements, and/or components, but the presence of one or more other features, integers, steps, operations, elements, components, and/or groups thereof It will be further understood that addition is not excluded.

The term “if” means, optionally, “when” or “upon” or “in response to determining” or “ It is interpreted to mean “in response to detecting.” Similarly, the phrases “when it is determined” or “when [the stated state or event] is detected” can, optionally, mean “upon determining” or “in response to determining” or “when [the stated state or event] is detected,” depending on the context. […] upon detection” or “in response to detecting [the stated state or event].”

Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions such as PDA and/or music player functions. Exemplary embodiments of portable multifunction devices include iPhone®, iPod Touch®, and iPad® devices from Apple Inc., Cupertino, CA. Includes without. Other portable electronic devices, such as laptop or tablet computers with touch-sensitive surfaces (eg, touch screen displays and/or touchpads), are optionally used. It should also be understood that in some embodiments, the device is a desktop computer with a touch-sensitive surface (eg, a touch screen display and/or touchpad) rather than a portable communication device.

In the discussion below, an electronic device comprising a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and/or joystick.

Devices typically support a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk authoring applications, spreadsheet applications, gaming applications, phone applications, video conferencing applications, and email. applications, instant messaging applications, exercise support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and/or digital video player applications.

Various applications running on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface as well as corresponding information displayed on the device are optionally adjusted and/or changed from one application to the next and/or within an individual application. In this way, the device's common physical architecture (such as a touch-sensitive surface) optionally supports a variety of applications using user interfaces that are intuitive and transparent to the user.

We now turn our attention to embodiments of portable devices with touch-sensitive displays. 1A is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display system 112, according to some embodiments. Touch-sensitive display 112 is sometimes referred to for convenience as a “touch screen” and is sometimes known or referred to as a “touch-sensitive display system.” Device 100 includes memory 102 (optionally including one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripherals interface 118, and a RF device. Circuitry 108, audio circuitry 110, speaker 111, microphone 113, input/output (I/O) subsystem 106, other input control devices 116, and external port 124. Includes. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact intensity sensors for detecting the intensity of contacts on device 100 (e.g., a touch-sensitive surface such as touch-sensitive display system 112 of device 100). 165). Device 100 may optionally use a touchpad 355 of device 300 or a touch-sensitive display system 112 of device 100 to generate tactile outputs on device 100. and one or more tactile output generators 167 (for generating tactile outputs on the same touch-sensitive surface). These components optionally communicate via one or more communication buses or signal lines 103.

As used in the specification and claims, the term “strength” of contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on a touch-sensitive surface. Refers to a substitute (proxy) for the force or pressure of contact on a surface. The intensity of contact has a range of values, including at least four distinct values and more typically including hundreds (eg, at least 256) distinct values. The intensity of contact is optionally determined (or measured) using various approaches and various sensors or combinations of sensors. For example, one or more force sensors under or adjacent to the touch-sensitive surface are optionally used to measure force at various points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (eg, weighted average) to determine the estimated force of the contact. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and/or changes thereto, the capacitance of the touch-sensitive surface proximate to the contact and/or changes thereto, and/or the touch-sensitive surface proximate to the contact. The resistance of the surface and/or changes thereto are optionally used as a substitute for the force or pressure of contact on the touch-sensitive surface. In some implementations, alternative measurements of contact force or pressure are used directly to determine whether an intensity threshold has been exceeded (eg, the intensity threshold is stated in units corresponding to the alternative measurements). In some implementations, alternative measurements of contact force or pressure are converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is is the pressure threshold measured in units of pressure). By using strength of contact as an attribute of user input, a user may, in some situations, display affordances that would otherwise not be present (e.g., on a touch-sensitive display) and/or Additional device functions that are not accessible by the user may be accessed on a scaled-down device with limited real estate (via the surface or physical/mechanical controls such as knobs or buttons) to receive user input. .

As used in the specification and claims, the term “tactile output” refers to the physical displacement of a device relative to its previous position, the physical displacement of a component of a device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing). ) refers to the physical displacement of the component, or the displacement of the component with respect to the center of mass of the device to be detected by the user using the user's sense of touch. For example, in a situation where a device or a component of a device is in contact with a touch-sensitive surface of a user (e.g., fingers, palm, or other part of the user's hand), the tactile output generated by the physical displacement may be transmitted by the user to the device or It will be interpreted as a tactile sensation that corresponds to a perceived change in the physical properties of the device's components. For example, movement of a touch-sensitive surface (eg, a touch-sensitive display or trackpad) is optionally interpreted by the user as a “down click” or “up click” of a physical actuator button. In some cases, a user will feel a tactile sensation, such as a “down click” or “up click,” even without movement of a physical actuator button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement. . As another example, movement of the touch-sensitive surface is optionally interpreted or perceived by the user as “roughness” of the touch-sensitive surface, even if there is no change in the flatness of the touch-sensitive surface. Although these interpretations of touch by a user will likely be influenced by the user's individualized sensory perception, there are many sensory perceptions of touch that are universal to the majority of users. Therefore, when a tactile output is described as corresponding to a user's specific sensory perception (e.g., “up click”, “down click”, “roughness”), unless otherwise stated, the tactile output produced is typical (or corresponds to a physical displacement of the device or its components that would produce the described sensory perception for an average) user. Providing haptic feedback to the user using tactile outputs improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). It makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some embodiments, the tactile output pattern includes characteristics of the tactile output, such as the amplitude of the tactile output, the shape of the moving waveform of the tactile output, the frequency of the tactile output, and/or the duration of the tactile output. Be specific.

When tactile outputs with different tactile output patterns are generated by a device (e.g., through one or more tactile output generators that move a movable mass to generate tactile outputs), the tactile outputs are generated when the user interacts with the device. Different haptic sensations can be evoked when held or touched. Although the user's senses are based on the user's perception of the tactile output, most users will be able to discern changes in waveform, frequency, and amplitude of the tactile outputs generated by the device. Accordingly, the waveform, frequency and amplitude can be adjusted to indicate to the user that different operations have been performed. As such, a given environment (e.g., a user interface including graphical features and objects, a simulated physical environment with virtual boundaries and virtual objects, a real physical environment with physical boundaries and physical objects, and/or any of the above) properties of objects (e.g., size, material, weight, stiffness, smoothness, etc.); behaviors (eg, oscillation, displacement, acceleration, rotation, expansion, etc.); and/or tactile outputs with tactile output patterns designed, selected, and/or engineered to simulate interactions (e.g., collision, adhesion, repulsion, attraction, friction, etc.), in some situations. , it will provide users with useful feedback that reduces input errors and increases the efficiency of user operation of the device. Additionally, tactile outputs are optionally generated to correspond to feedback that is not related to simulated physical properties, such as an input threshold or selection of an object. Such tactile outputs, in some situations, will provide useful feedback to users that reduces input errors and increases the efficiency of user operation of the device.

In some embodiments, tactile output with a suitable tactile output pattern serves as a cue for the occurrence of an event of interest in a user interface or behind the scenes within the device. Examples of events of interest include activating an affordance provided on the device or in a user interface (e.g., a real or virtual button, or toggle switch), success or failure of a requested action, reaching or crossing a boundary in the user interface. , entering a new state, switching input focus between objects, activating a new mode, reaching or crossing an input threshold, detecting or recognizing a type of input or gesture, etc. In some embodiments, tactile outputs are provided to serve as a warning or alert of an impending event or outcome that will occur if a redirection or interruption input is not detected in a timely manner. Tactile outputs may also, in other contexts, enrich the user experience and/or improve the accessibility of a device for users with visual or motor impairments or other accessibility needs, and/or enhance the user interface and/or Or it is used to improve the efficiency and functionality of the device. Tactile outputs are optionally accompanied by audio outputs and/or visible user interface changes, which further enhance the user's experience when the user interacts with the user interface and/or device, and or facilitate better transfer of information about the state of the device, reduce input errors and increase the efficiency of user operation of the device.

4C-4E illustrate one or more devices for generating haptic feedback suitable for various purposes and in various scenarios, such as those described above and those described in connection with the user interfaces and methods discussed herein. Provides a set of sample tactile output patterns that can be used as is, individually or in combination, or through transformations (e.g., modulation, amplification, truncation, etc.). This example of a palette of tactile outputs shows how a set of three waveforms and eight frequencies can be used to create an array of tactile output patterns. In addition to the tactile output patterns shown in these figures, each of these tactile output patterns is optionally shown in variations, for example, with gains of 1.0, 0.75, 0.5, and 0.25, respectively, in Figure 4f. Tactile output, as shown for FullTap 80 Hz, PullTap 200 Hz, MiniTap 80 Hz, MiniTap 200 Hz, MicroTap 80 Hz, and MicroTap 200 Hz in Figures 4H. The amplitude is adjusted by changing the gain value for the pattern. As shown in Figures 4F-4H, changing the gain of a tactile output pattern changes the amplitude of the pattern without changing the frequency of the pattern or changing the shape of the waveform. In some embodiments, changing the frequency of the tactile output pattern also results in a lower amplitude, as some tactile output generators are limited by how much force can be applied to the movable mass, and thus the waveform This is because the higher frequency movements of the mass are constrained to lower amplitudes (e.g., 230 Hz, 270 Hz, and the peak amplitudes of the pull tab at 300 Hz are lower than the amplitudes of the pull tab at 80 Hz, 100 Hz, 125 Hz, and 200 Hz).

4C to 4H show tactile output patterns with specific waveforms. The waveform of the tactile output pattern represents the pattern of physical displacements relative to a neutral position (e.g., x _zero ) versus the time that the movable mass undergoes to generate the tactile output with that tactile output pattern. For example, the first set of tactile output patterns shown in FIG. 4C (e.g., the tactile output patterns of a “pull tab”) each oscillates with two complete cycles (e.g., starts and ends in a neutral position and moves from neutral to neutral). It has a waveform that includes an oscillation that traverses the position three times. The second set of tactile output patterns shown in FIG. 4D (e.g., the tactile output patterns of the “Minitab”) each oscillates comprising one complete cycle (e.g., starts and ends in a neutral position and moves the neutral position to 1 It has a waveform that includes oscillations that traverse multiple times. The third set of tactile output patterns shown in FIG. 4E (e.g., the tactile output patterns of a “microtap”) each have oscillations comprising one-half of a complete cycle (e.g., starting and ending in a neutral position and has a waveform that contains oscillations that do not traverse the position. The waveform of the tactile output pattern also includes a start buffer and an end buffer that represent gradual acceleration and deceleration of the movable mass at the beginning and end of the tactile output. The example waveforms shown in FIGS. 4C-4H include x _min and x _max values that represent the maximum and minimum limits of movement of the movable mass. For larger electronic devices with larger movable masses, there may be larger or smaller minimum and maximum limits of movement of the masses. Although the examples shown in FIGS. 4C-4H describe the movement of a mass in one dimension, similar principles would also apply to the movement of a movable mass in two or three dimensions.

As shown in FIGS. 4C-4E, each tactile output pattern also has a corresponding characteristic frequency that affects the “pitch” of the haptic sensation, which the user receives from the tactile output with that characteristic frequency. felt by For continuous tactile output, the characteristic frequency represents the number of cycles (eg, cycles per second) completed within a given period of time by the movable mass of the tactile output generator. For discrete tactile output, a discrete output signal (e.g., with 0.5, 1, or 2 cycles) is generated and the characteristic frequency value determines how much the movable mass must produce a tactile output with that characteristic frequency. Determine whether you need to move quickly. As shown in Figures 4C-4H, for each type of tactile output (as defined by the individual waveform, e.g., pull tab, minitap, or microtap), the higher the frequency value is shifted. Faster movement(s) by the enabling mass, and therefore generally shorter time to complete the tactile output (e.g., time to complete the required number of cycle(s) for an individual tactile output + (including start and end buffer times). For example, a pull tab with a characteristic frequency of 80 Hz takes longer to complete than a pull tab with a characteristic frequency of 100 Hz (e.g., 28.3 ms compared to 35.4 ms in Figure 4C). Additionally, for given frequencies, a tactile output with more cycles of its waveform at an individual frequency takes longer to complete than a tactile output with fewer cycles of its waveform at the same individual frequency. For example, a pull tab at 150 Hz takes longer to complete than a minitap at 150 Hz (e.g., 12.8 ms compared to 19.4 ms), and a minitap at 150 Hz takes longer to complete than a microtap at 150 Hz. takes (e.g., 9.4ms compared to 12.8ms). However, for tactile output patterns with different frequencies, this rule may not apply (e.g., tactile outputs with more cycles but a higher frequency than tactile outputs with fewer cycles but a lower frequency). outputs may take less time to complete than the outputs, and vice versa). For example, at 300 Hz, a pull tab takes as long as a minitap (e.g., 9.9 ms).

As shown in FIGS. 4C-4E, the tactile output pattern also has a characteristic amplitude, which affects the amount of energy contained in the tactile signal or the “strength” of the haptic sensation. It can be felt by the user through a tactile output. In some embodiments, the characteristic amplitude of a tactile output pattern refers to an absolute or normalized value that represents the maximum displacement of the movable mass from its neutral position when producing the tactile output. In some embodiments, the characteristic amplitude of the tactile output pattern can be adjusted to various conditions (e.g., customized based on user interface situations and behaviors) and/or preconfigured metrics (e.g., input-based metrics, and /or user interface based metrics), eg by a fixed or dynamically determined gain factor (eg, a value between 0 and 1). In some embodiments, an input-based metric (e.g., an intensity change metric or an input-speed metric) is a characteristic of the input during an input that triggers the generation of a tactile output (e.g., the rate of change in intensity of a touch or the characteristic of a contact in a press input). The speed of movement of a contact across a sensitive surface is measured. In some embodiments, user interface-based metrics (e.g., speed across boundaries metrics) determine characteristics of user interface elements (e.g., hidden or visible boundaries in the user interface) during user interface changes that trigger the generation of tactile output. Measures the speed of movement of the traversed element. In some embodiments, the characteristic amplitude of the tactile output pattern may be modulated by an “envelope”, where peaks of adjacent cycles may have different amplitudes, where one of the waveforms shown above is the tactile output pattern. As the output is being generated, it is further modified by multiplication by an envelope parameter (e.g., 0 to 1) that changes over time to gradually adjust the amplitude of the portions of the tactile output over time.

Specific frequencies, amplitudes, and waveforms are shown in the sample tactile output patterns in FIGS. 4C-4E for illustration purposes, but tactile output patterns with other frequencies, amplitudes, and waveforms may be used for similar purposes. You can. For example, waveforms with 0.5 to 4 cycles can be used. Other frequencies ranging from 60 Hz to 400 Hz may also be used. Table 1 provides examples of specific haptic feedback behaviors, configurations, and examples of their use.

Device 100 is only one example of a portable multi-function device, and device 100 may optionally have more or fewer components than shown, optionally combine two or more components, or optionally use a combination of components. It should be understood that they have different configurations or arrangements. The various components shown in FIG. 1A are implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing circuits and/or application-specific integrated circuits (ASICs).

Memory 102 optionally includes high-speed random access memory, and may also optionally include one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Includes non-volatile memory such as Memory controller 122 optionally controls access to memory 102 by other components of device 100.

Peripheral interface 118 may be used to couple the device's input and output peripherals to CPU 120 and memory 102. One or more processors 120 drive or execute various software programs and/or sets of instructions stored in memory 102 to perform various functions for device 100 and to process data. In some embodiments, peripherals interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, these are optionally implemented on separate chips.

Radio frequency (RF) circuitry 108 receives and transmits RF signals, also referred to as electromagnetic signals. RF circuitry 108 converts electrical signals to and from electromagnetic signals and communicates with communication networks and other communication devices via electromagnetic signals. RF circuitry 108 optionally includes, but is not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, etc. , and includes well-known circuitry for performing these functions. RF circuitry 108 optionally connects to networks, such as the Internet, also referred to as the World Wide Web (WWW), intranets, and/or wireless networks, such as cellular telephone networks, wireless local area networks (LANs), and/or MANs. (metropolitan area network), and communicate with other devices by wireless communication. RF circuitry 108 optionally includes well-known circuitry for detecting near field communication (NFC) fields, such as by a short-range communication radio. Wireless communications may optionally include Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only), HSPA, HSPA+, DC-HSPDA (Dual-Cell HSPA), LTE (long term evolution), NFC (near field communication), W-CDMA (wideband code division multiple access), CDMA (code division multiple access) ), time division multiple access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n and/or IEEE 802.11ac), voice over Internet Protocol (VoIP), Wi-MAX, protocols for email (e.g., Internet message access protocol (IMAP) and/or post office protocol (POP)), instant messaging (e.g., extensible messaging (XMPP)) and presence protocol), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS), and/or Short Message Service (SMS), or communications not yet developed as of the filing date of this document. Utilizes any of a plurality of communication standards, protocols and technologies, including but not limited to protocols, as well as any other suitable communication protocols.

The audio circuit unit 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuit unit 110 receives audio data from the peripheral interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts electrical signals into sound waves that humans can hear. The audio circuitry 110 also receives electrical signals converted from sound waves by the microphone 113. Audio circuitry 110 converts electrical signals into audio data and transmits the audio data to peripherals interface 118 for processing. Audio data, optionally, is retrieved from and/or transmitted to memory 102 and/or RF circuitry 108 by peripherals interface 118. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., 212 in FIG. 2). The headset jack is located between audio circuitry 110 and detachable audio input/output peripherals, such as output-only headphones, or a headset with both an output (e.g., headphones for one or both ears) and an input (e.g., a microphone). Provides an interface.

I/O subsystem 106 couples input/output peripherals on device 100, such as touch screen 112 and other input control devices 116, to peripherals interface 118. The I/O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a haptic feedback controller 161, a depth camera controller 169, and other inputs or controls. Includes one or more input controllers 160 for devices. One or more input controllers 160 receive/transmit electrical signals to/from other input control devices 116. Other input control devices 116 optionally include physical buttons (eg, push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, input controller(s) 160 is optionally coupled to any (or not to any) of a keyboard, infrared port, USB port, and pointer device, such as a mouse. One or more buttons (e.g., 208 in FIG. 2) optionally include up/down buttons for volume control of the speaker 111 and/or microphone 113. One or more buttons optionally include a push button (e.g., 206 in FIG. 2).

A quick press of the push button optionally unlocks the touch screen 112 or initiates processes that optionally use gestures on the touch screen to unlock the device, which As described in U.S. Patent Application Serial No. 11/322,549, “Unlocking a Device by Performing Gestures on an Unlock Image,” filed dated U.S. Pat. No. 7,657,849, which is hereby incorporated by reference in its entirety. do. A longer press of the push button (e.g., 206) selectively powers device 100 on or off. The functionality of one or more buttons may optionally be customizable. Touch screen 112 is used to implement virtual or soft buttons and one or more soft keyboards.

The touch-sensitive display 112 provides an input interface and an output interface between the device and the user. Display controller 156 receives and/or transmits electrical signals to and from touch screen 112. Touch screen 112 displays visual output to the user. Visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively referred to as “graphics”). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

Touch screen 112 has a touch-sensitive surface, sensor, or set of sensors that accepts input from a user based on haptics and/or tactile contact. Touch screen 112 and display controller 156 (along with any associated modules and/or sets of instructions in memory 102) are capable of controlling (and any moving or stopping of contact) contact on touch screen 112. Detects and translates the detected contact into an interaction with user interface objects (eg, one or more soft keys, icons, web pages, or images) displayed on the touch screen 112. In an example embodiment, the point of contact between touch screen 112 and the user corresponds to the user's finger.

The touch screen 112 optionally uses liquid crystal display (LCD) technology, light emitting polymer display (LPD) technology, or light emitting diode (LED) technology, although other display technologies are used in other embodiments. Touch screen 112 and display controller 156 may optionally utilize capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays, or one or more points of contact with touch screen 112. Contact and any movement or interruption thereof may be detected using any of a plurality of touch sensing technologies now known or later developed, including but not limited to other factors for determining. In an exemplary embodiment, projected mutual capacitance sensing technology, such as that found in the iPhone® and iPod Touch® from Apple Inc., Cupertino, California, is utilized.

The touch-sensitive display in some embodiments of the touch screen 112 may optionally include, but is not limited to, U.S. Patents 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and/or 6,677,932 (Westerman et al.) , and/or the multi-touch sensitive touchpads described in US Patent Publication No. 2002/0015024A1, each of which is incorporated herein by reference in its entirety. However, while touch screen 112 displays visual output from device 100, touch-sensitive touchpads do not provide visual output.

A touch-sensitive display in some embodiments of touch screen 112 is described in the following applications: (1) U.S. patent application Ser. No. 11/381,313, “Multipoint Touch Surface,” filed May 2, 2006; Controller"; (2) U.S. Patent Application No. 10/840,862, “Multipoint Touchscreen,” filed May 6, 2004; (3) U.S. Patent Application No. 10/903,964, “Gestures For Touch Sensitive Input Devices,” filed July 30, 2004; (4) U.S. Patent Application No. 11/048,264, “Gestures For Touch Sensitive Input Devices,” filed January 31, 2005; (5) U.S. Patent Application No. 11/038,590, “Mode-Based Graphical User Interfaces For Touch Sensitive Input Devices,” filed January 18, 2005; (6) U.S. Patent Application No. 11/228,758, “Virtual Input Device Placement On A Touch Screen User Interface,” filed September 16, 2005; (7) U.S. Patent Application No. 11/228,700, “Operation Of A Computer With A Touch Screen Interface,” filed September 16, 2005; (8) U.S. Patent Application No. 11/228,737, “Activating Virtual Keys Of A Touch-Screen Virtual Keyboard,” filed September 16, 2005; and (9) U.S. Patent Application No. 11/367,749, “Multi-Functional Hand-Held Device,” filed March 3, 2006. All of these applications are incorporated herein by reference in their entirety.

Touch screen 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touch screen has a video resolution of approximately 160 dpi. The user, optionally, contacts the touch screen 112 using any suitable object or attachment, such as a stylus, finger, etc. In some embodiments, the user interface is designed to operate primarily using finger-based contacts and gestures, which may be less precise than stylus-based input due to the finger's larger contact area on the touch screen. In some embodiments, the device converts coarse finger-based input into precise pointer/cursor positions or commands to perform desired actions by the user.

In some embodiments, in addition to the touch screen, device 100 optionally includes a touchpad (not shown) for activating or deactivating certain functions. In some embodiments, a touchpad is a touch-sensitive area of a device that, unlike a touch screen, does not display visual output. The touch pad is, optionally, a touch-sensitive surface separate from touch screen 112 or an extension of the touch-sensitive surface formed by the touch screen.

Device 100 also includes a power system 162 to power various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator ( such as light emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power within portable devices.

Device 100 also optionally includes one or more optical sensors 164. 1A shows an optical sensor coupled to an optical sensor controller 158 within I/O subsystem 106. The optical sensor 164 optionally includes charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) phototransistors. Light sensor 164 receives light from the surrounding environment, projected through one or more lenses, and converts the light into data representing an image. In conjunction with imaging module 143 (also referred to as a camera module), optical sensor 164 optionally captures still images or video. In some embodiments, the optical sensor is located on the back of device 100, opposite the touch screen display 112 on the front of the device, so that the touch screen display can be used as a viewfinder for still and/or video image acquisition. let it be In some embodiments, an optical sensor is positioned on the front of the device such that an image of the user is optionally obtained for the video conference while the user views other video conference participants on the touch screen display. In some embodiments, the position of optical sensor 164 can be changed by the user (e.g., by rotating the lens and sensor within the device housing), such that a single optical sensor 164 can be used for video conferencing and video conferencing with a touch screen display. Used for both still and/or video image acquisition.

Device 100 also optionally includes one or more contact intensity sensors 165. 1A shows a contact intensity sensor coupled to intensity sensor controller 159 within I/O subsystem 106. Contact intensity sensor 165 may optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of contact on a touch-sensitive surface. Contact intensity sensor 165 receives contact intensity information (eg, pressure information or a proxy for pressure information) from the surrounding environment. In some embodiments, at least one contact intensity sensor collocates with or is proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact intensity sensor is located on the back of device 100, opposite the touch screen display 112 located on the front of device 100.

Device 100 also optionally includes one or more proximity sensors 166. 1A shows proximity sensor 166 coupled to peripherals interface 118. Alternatively, proximity sensor 166 is optionally coupled to input controller 160 within I/O subsystem 106. Proximity sensor 166 may optionally be described in U.S. Patent Application Serial No. 11/241,839, “Proximity Detector In Handheld Device”; No. 11/240,788, “Proximity Detector In Handheld Device”; No. 11/620,702, “Using Ambient Light Sensor To Augment Proximity Sensor Output”; No. 11/586,862, “Automated Response To And Sensing Of User Activity In Portable Devices”; and No. 11/638,251, “Methods And Systems For Automatic Configuration Of Peripherals,” which are incorporated herein by reference in their entirety. In some embodiments, the proximity sensor turns off and disables the touch screen 112 when the multifunction device is positioned near the user's ear (e.g., when the user is talking on the phone).

Device 100 also optionally includes one or more tactile output generators 167. 1A shows a tactile output generator coupled to haptic feedback controller 161 in I/O subsystem 106. Tactile output generator 167 optionally includes one or more electroacoustic devices such as speakers or other audio components and/or motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating devices. Includes electromechanical devices that convert energy into linear motion, such as a component (e.g., a component that converts electrical signals to tactile outputs on a device). Contact intensity sensor 165 receives tactile feedback generation instructions from haptic feedback module 133 to generate tactile outputs on device 100 that can be sensed by a user of device 100. In some embodiments, the at least one tactile output generator is positioned with or proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112) and, optionally, moves the touch-sensitive surface vertically (e.g., , generates a tactile output by moving the device 100 (into/outside the surface of the device 100) or laterally (eg, back and forth in the same plane as the surface of the device 100). In some embodiments, the at least one tactile output generator sensor is located on the back of device 100, opposite the touch screen display 112 located on the front of device 100.

Device 100 also optionally includes one or more accelerometers 168. 1A shows accelerometer 168 coupled to peripherals interface 118. Alternatively, accelerometer 168 is optionally coupled to input controller 160 within I/O subsystem 106. Accelerometer 168 is optionally described in US Patent Publication No. 20050190059, “Acceleration-based Theft Detection System for Portable Electronic Devices” and US Patent Publication No. 20060017692, “Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer. ", both of which are incorporated herein by reference in their entirety. In some embodiments, information is displayed in a portrait view or a landscape view on a touch screen display based on analysis of data received from one or more accelerometers. Device 100 optionally includes, in addition to accelerometer(s) 168, a magnetometer (not shown), and GPS (e.g., vertical or horizontal) to obtain information regarding the location and orientation (e.g., vertical or horizontal) of device 100. or GLONASS or other global navigation system) receiver (not shown).

In some embodiments, device 100 also includes (or communicates with) one or more fingerprint sensors. One or more fingerprint sensors are coupled to peripherals interface 118. Alternatively, one or more fingerprint sensors are optionally coupled to input controller 160 within I/O subsystem 106. However, in one general embodiment, fingerprint identification operations may be performed on secure dedicated computing hardware (e.g., one or more processors, memory and /or communication buses). As used herein, a fingerprint sensor is a device that can distinguish fingerprint features (sometimes referred to as “detailed features”) of the ridges and valleys of the skin, such as those found on human fingers and toes. It's a sensor. The fingerprint sensor may distinguish fingerprint features using any of a variety of techniques including, but not limited to, optical fingerprint imaging, ultrasonic fingerprint imaging, active capacitive fingerprint imaging, and passive capacitive fingerprint imaging. In addition to distinguishing fingerprint features in a fingerprint, in some embodiments, one or more fingerprint sensors may track the movement of fingerprint features over time, thereby The movement of fingerprints can be determined/characterized. The one or more fingerprint sensors may be separate from the touch-sensitive surface (e.g., touch-sensitive display system 112), although in some implementations the touch-sensitive surface (e.g., touch-sensitive display system 112) may It should be understood that 112)) has a sufficiently high spatial resolution to detect fingerprint features formed by individual fingerprint ridges and is used as a fingerprint sensor instead of or in addition to one or more fingerprint sensors. In some embodiments, device 100 may include one or more orientation sensors used to determine the orientation of a finger or hand on or proximate the device (e.g., orientation of a finger over one or more fingerprint sensors). Includes set. Additionally, in some embodiments, a set of one or more orientation sensors are used in addition to or instead of the fingerprint sensor to detect the rotation of a contact interacting with the device (e.g., described below) In one or more methods, instead of using a fingerprint sensor to detect the rotation of a fingerprint/contact, a set of one or more orientation sensors detects the rotation of a contact containing a fingerprint, with or without detecting features of the fingerprint. used to detect).

In some embodiments, features of fingerprints, and comparisons between features of detected fingerprints and features of stored fingerprints, to improve the security of fingerprint data generated, stored, and processed by one or more fingerprint sensors. is performed by secure dedicated computing hardware (e.g., one or more processors, memory and/or communication buses) that is separate from processor(s) 120. In some embodiments, comparisons between the characteristics of the fingerprints and the characteristics of the detected fingerprints and the characteristics of the enrolled fingerprints are performed by the processor(s) 120 using a fingerprint analysis module.

In some embodiments, during the enrollment process, a device (e.g., a fingerprint analysis module or a separate security module in communication with one or more fingerprint sensors) collects biometric information about one or more fingerprints of the user (e.g., the user (identifying the relative positions of a plurality of detailed feature points in the fingerprint). After the enrollment process is complete, the biometric information is stored on the device (e.g., in a secure fingerprint module) for later use in authenticating detected fingerprints. In some embodiments, biometric information stored on the device excludes images of fingerprints and also allows images of fingerprints to be restored so that they are not inadvertently made available if the security of the device is compromised. exclude information. In some embodiments, during the authentication process, the device (e.g., a fingerprint analysis module or a separate security module in communication with one or more fingerprint sensors) collects biometric information about a fingerprint detected on one or more fingerprint sensors ( (e.g., by identifying the relative positions of a plurality of detailed feature points in a fingerprint detected on one or more fingerprint sensors) and comparing biometric information corresponding to the detected fingerprint with biometric information corresponding to the enrolled fingerprint(s). It is determined whether the finger input detected by the above fingerprint sensors includes a fingerprint matching the pre-registered fingerprint. In some embodiments, comparing the biometric information corresponding to the detected fingerprint with the biometric information corresponding to the enrolled fingerprint(s) determines the type and location of detailed feature points in the biometric information corresponding to the detected fingerprint. It includes comparing the type and location of detailed feature points in the biometric information corresponding to the enrolled fingerprints. However, determining whether a finger input includes a fingerprint matching a pre-enrolled fingerprint registered on the device may optionally include a number of well-known fingerprint authentication techniques for determining whether a detected fingerprint matches an enrolled fingerprint. It is performed using any of the following.

Device 100 also optionally includes one or more depth camera sensors 175. 1A shows a depth camera sensor coupled to depth camera controller 169 within I/O subsystem 106. Depth camera sensor 175 receives data from the environment, projected through the sensor. A depth camera sensor 175, optionally in conjunction with imaging module 143 (also referred to as a camera module), is used to determine depth maps of different portions of the image captured by imaging module 143. In some embodiments, a depth camera sensor is located on the front of device 100 so that an image of the user with depth information is available for use by different functions of the device, such as video conferencing, to create a depth map. Capture selfies with data and authenticate the user of the device. In some embodiments, the position of depth camera sensors 175 can be changed by the user (e.g., by rotating the lens and sensor within the device housing) so that depth camera sensors 175 can be used with a touch screen display. It allows for use in both videoconferencing and still and/or video image acquisition.

In some embodiments, software components stored in memory 102 include operating system 126, communications module (or set of instructions) 128, contact/motion module (or set of instructions) 130, graphics module ( or set of instructions) 132, a text input module (or set of instructions) 134, a GPS module (or set of instructions) 135, and applications (or sets of instructions) 136. . Additionally, in some embodiments, memory (102 in FIG. 1A or 370 in FIG. 3) stores device/global internal state 157 as shown in FIGS. 1A and 3. Device/global internal state 157 includes active application state, if present, indicating which applications are currently active; display state indicating which applications, views or other information occupy various areas of the touch screen display 112; sensor state, including information obtained from the device's various sensors and input control devices 116; and location information regarding the location and/or posture of the device.

Operating system 126 (e.g., an embedded operating system such as Darwin, RTXC, LINUX, UNIX, OS Includes various software components and/or drivers for controlling and managing and facilitates communication between various hardware and software components.

The communication module 128 enables communication with other devices through one or more external ports 124, and also provides various functions for processing data received by the RE circuitry 108 and/or external ports 124. Contains software components. External port 124 (e.g., Universal Serial Bus (USB), FIREWIRE, etc.) is configured to couple directly to other devices or indirectly through a network (e.g., Internet, wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same or similar to and/or compatible with the 30-pin connector used in iPod® (a trademark of Apple Inc.) devices.

Contact/motion module 130 optionally detects contact with touch screen 112 (along with display controller 156) and other touch-sensitive devices (e.g., a touch pad or physical click wheel). . Contact/motion module 130 determines whether contact has occurred (e.g., detecting a finger-down event), the strength of the contact (e.g., the force or pressure of the contact, or (a proxy for force or pressure), and track movement across a touch-sensitive surface by determining whether there is movement of the contact (e.g., one or more finger-dragging events). to perform various operations related to detection of contact, such as detecting contact), and determining whether contact has stopped (e.g., detecting a finger-up event or contact interruption). Contains various software components. Contact/motion module 130 receives contact data from a touch-sensitive surface. Determining the movement of the point of contact represented by the set of contact data may optionally determine the speed (magnitude), velocity (magnitude and direction), and/or acceleration (change in size and/or direction) of the point of contact. It includes doing. These operations are optionally applied to single contacts (eg, single finger contacts) or to multiple simultaneous contacts (eg, “multitouch”/multiple finger contacts). In some embodiments, contact/motion module 130 and display controller 156 detect contact on the touch pad.

In some embodiments, contact/motion module 130 may use a set of one or more intensity thresholds to determine whether an action was performed by a user (e.g., to determine whether the user “clicked” on an icon). Use . In some embodiments, at least a subset of the intensity thresholds are determined according to software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators, and changing the physical hardware of device 100 can be adjusted without). For example, the mouse “click” threshold of a trackpad or touch screen display can be set to any of a wide range of predefined threshold values without changing the trackpad or touch screen display hardware. Additionally, in some implementations, a user of the device may control one or more of a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and/or by adjusting multiple intensity thresholds at once with a system-level click “intensity” parameter). Software settings are provided to adjust .

Contact/motion module 130 optionally detects gesture input by the user. Different gestures on a touch-sensitive surface have different contact patterns (eg, different motions, timings, and/or intensities of detected contacts). Accordingly, the gesture is, optionally, detected by detecting a specific contact pattern. For example, detecting a finger tap gesture involves detecting a finger-down event and then finger-down at the same location (or substantially the same location) as the finger-down event (e.g., at the location of the icon). Includes detecting up (liftoff) events. As another example, detecting a finger swipe gesture on a touch-sensitive surface may involve detecting a finger-down event followed by one or more finger-dragging events, followed by a finger-up (liftoff) event. ) includes detecting events.

Graphics module 132 may render and display graphics on a touch screen 112 or other display, including components for changing the visual effect (e.g., brightness, transparency, saturation, contrast, or other visual properties) of displayed graphics. It includes various known software components for display. As used herein, the term “graphics” includes, without limitation, text, web pages, icons (e.g., user interface objects including soft keys), digital images, videos, animations, etc. , contains arbitrary objects that can be displayed to the user.

In some embodiments, graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes specifying the graphics to be displayed, along with coordinate data and other graphics attribute data, if necessary, from applications, etc., and then generates screen image data and outputs it to the display controller 156. do.

Haptic feedback module 133 is used by tactile output generator(s) 167 to generate tactile outputs at one or more locations on device 100 in response to user interactions with device 100. Contains various software components for generating instructions.

Text input module 134, optionally a component of graphics module 132, is used for various applications (e.g., contacts 137, email 140, IM 141, browser 147, and text input). Provides soft keyboards for entering text into any other application).

GPS module 135 determines the location of the device and stores this information for use in various applications (e.g., to phone 138 for use in location-based dialing; to camera 143 as photo/video metadata). and to applications that provide location-based services, such as weather widgets, local yellow pages widgets, and map/navigation widgets.

Applications 136 optionally include the following modules (or sets of instructions), or a subset or superset thereof:

Contacts module 137 (sometimes referred to as address book or contact list);

Telephone module 138;

video conferencing module 139;

email client module 140;

instant messaging (IM) module 141;

exercise support module 142;

Camera module 143 for still and/or video images;

image management module 144;

video player module;

music player module;

browser module 147;

calendar module (148);

Weather widget (149-1), Stock widget (149-2), Calculator widget (149-3), Alarm clock widget (149-4), Dictionary widget (149-5), and other widgets obtained by the user. as well as widget modules 149 optionally including one or more of the user-created widgets 149-6;

a widget generator module 150 for creating user-created widgets 149-6;

Search module 151;

a video and music player module 152 incorporating a video player module and a music player module;

memo module 153;

map module (154); and/or

Online Video Module (155).

Examples of other applications 136 that are optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA-enabled applications, encryption, digital copyright, etc. Includes management, voice recognition and voice replication.

Contacts module 137, along with touch screen 112, display controller 156, contact/motion module 130, graphics module 132, and text input module 134, optionally includes an address book or contact list ( (e.g., stored in memory 102 or application internal state 192 of contacts module 137 in memory 370), including: adding name(s) to the address book; thing; deleting name(s) from the address book; Associating a name with a phone number(s), email address(es), physical address(es) or other information; associating an image with a name; sorting and sorting names; Providing phone numbers or email addresses to initiate and/or facilitate communication by phone 138, video conferencing module 139, email 140 or IM 141, etc.

RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact/motion module 130, graphics module 132, and text. In conjunction with input module 134, phone module 138 may optionally input a sequence of characters corresponding to a phone number, access one or more phone numbers within contact module 137, and modify the entered phone number. It is used to dial individual phone numbers, initiate conversations, and disconnect or hang up when the conversation is complete. As discussed above, wireless communications optionally utilize any of a plurality of communication standards, protocols and technologies.

RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, light sensor 164, light sensor controller 158, touch/motion In conjunction with module 130, graphics module 132, text input module 134, contact module 137, and phone module 138, video conferencing module 139 is capable of interacting with a user and one or more other users according to user instructions. Contains executable commands to initiate, conduct, and terminate video conferences between participants.

Email client module 140, along with RF circuitry 108, touch screen 112, display controller 156, contact/motion module 130, graphics module 132, and text input module 134, provides user instructions. Contains executable commands that allow you to create, send, receive, and manage email in response to Together with the image management module 144 , the email client module 140 makes it very easy to create and send emails with still or video images taken with the camera module 143 .

Instant messaging module 141, along with RF circuitry 108, touch screen 112, display controller 156, contact/motion module 130, graphics module 132, and text input module 134, provides instant messaging Enter a sequence of characters corresponding to a message, modify previously entered characters (e.g., use the Short Message Service (SMS) or Multimedia Message Service for phone-based instant messaging). , MMS) protocol, or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), and contains executable commands for sending individual instant messages, receiving instant messages, and viewing received instant messages. . In some embodiments, instant messages sent and/or received optionally include graphics, photos, audio files, video files, and/or other attachments as supported in MMS and/or Enhanced Messaging Service (EMS). As used herein, “instant messaging” refers to phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS). ) refers to both.

RF circuitry 108, touch screen 112, display controller 156, contact/motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154. , and the music player module, the workout support module 142 is designed to design workouts (e.g., with time, distance, and/or calorie expenditure goals); communicate with exercise sensors (sports devices); receive exercise sensor data; calibrate sensors used to monitor exercise; Select and play music for your workout; Contains executable instructions to display, store, and transmit exercise data.

with touch screen 112, display controller 156, optical sensor(s) 164, optical sensor controller 158, contact/motion module 130, graphics module 132, and image management module 144. , the camera module 143 may capture still images or video (including a video stream) and store them in memory 102, modify the characteristics of the still image or video, or retrieve still images or video from memory 102. Contains executable commands to delete a video.

Image management module 144, along with touch screen 112, display controller 156, contact/motion module 130, graphics module 132, text input module 134, and camera module 143, can /or Contains executable instructions for arranging, modifying (e.g., editing) or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slideshow or album), and storing video images. .

Browser module 147, along with RF circuitry 108, touch screen 112, display controller 156, contact/motion module 130, graphics module 132, and text input module 134, displays a web page. Executable instructions for browsing the Internet according to user instructions, including retrieving, linking to, receiving, and displaying attachments and other files linked to web pages or portions thereof. includes them.

RF circuitry 108, touch screen 112, display controller 156, contact/motion module 130, graphics module 132, text input module 134, email client module 140, and browser module ( In conjunction with 147), the calendar module 148 is an executable module that creates, displays, modifies, and stores calendars and data associated with calendars (e.g., calendar entries, to-do lists, etc.) according to user instructions. Contains commands.

RF circuitry 108, touch screen 112, display controller 156, contact/motion module 130, graphics module 132, text input module 134, and browser module 147, along with widget modules. 149 may be optionally downloaded and used by the user (e.g., weather widget 149-1, stock widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widgets 149-5), or mini-applications that are created by a user (e.g., user-created widgets 149-6). In some embodiments, widgets include Hypertext Markup Language (HTML) files, Cascading Style Sheets (CSS) files, and JavaScript files. In some embodiments, a widget includes an Extensible Markup Language (XML) file and a JavaScript file (eg, Yahoo! widgets).

A widget generator module, along with RF circuitry 108, touch screen 112, display controller 156, contact/motion module 130, graphics module 132, text input module 134, and browser module 147. 150 is optionally used to create widgets by the user (e.g., change a user-specific portion of a web page into a widget).

In conjunction with the touch screen 112, display controller 156, contact/motion module 130, graphics module 132, and text input module 134, search module 151 provides one or more search criteria according to user instructions. and executable instructions for searching text, music, sound, images, video, and/or other files in memory 102 that match (e.g., one or more user-specific search terms).

with touch screen 112, display controller 156, contact/motion module 130, graphics module 132, audio circuitry 110, speakers 111, RF circuitry 108, and browser module 147. , the video and music player module 152 includes executable instructions that allow a user to download and play recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, and videos ( and executable instructions for displaying, playing, or otherwise playing (e.g., on touch screen 112 or on an external connected display through external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an iPod (a trademark of Apple Inc.).

Along with the touch screen 112, display controller 156, contact/motion module 130, graphics module 132, and text input module 134, the memo module 153 creates notes, notes, and actions according to user instructions. Contains executable commands to create and manage task lists, etc.

RF circuitry 108, touch screen 112, display controller 156, contact/motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147. ), the maps module 154 optionally provides maps and data associated with the maps according to user instructions (e.g., driving directions; data about stores and other points of interest at or near a particular location; and other location-based data).

Touch screen 112, display controller 156, contact/motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, In conjunction with the email client module 140 and the browser module 147, the online video module 155 allows users to access online videos in one or more file formats, such as H.264, browse them (e.g., stream and /or by download), play (e.g., on a touch screen or on an external connected display via external port 124), send an email with a link to a particular online video, and otherwise manage Includes commands that In some embodiments, instant messaging module 141 rather than email client module 140 is used to send links to specific online videos. For additional description of online video applications, see U.S. Provisional Patent Application No. 60/936,562, “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” filed June 20, 2007, and December 31, 2007. See U.S. Patent Application Serial No. 11/968,067, “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” the contents of which are hereby incorporated by reference in their entirety. .

Each of the modules and applications identified above may contain executable instructions for performing one or more of the functions described above and methods described herein (e.g., computer implemented methods and other information processing methods described herein). corresponds to a set of These modules (e.g., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules may be optionally combined or otherwise rearranged in various embodiments. . For example, a video player module is optionally combined with a music player module into a single module (e.g., video and music player module 152 in FIG. 1A). In some embodiments, memory 102 optionally stores a subset of the previously identified modules and data structures. Additionally, memory 102 optionally stores additional modules and data structures not described above.

In some embodiments, device 100 is a device in which operation of a predefined set of functions on the device is performed exclusively through a touch screen and/or touchpad. By using a touch screen and/or touchpad as the primary input control device for operation of device 100, the number of physical input control devices (such as push buttons, dials, etc.) on device 100 is optionally reduced.

A predefined set of functions performed entirely via the touch screen and/or touch pad, optionally including navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 to a main, home, or root menu from any user interface displayed on device 100. In these embodiments, the “menu button” is implemented using a touch pad. In some other embodiments, the menu button is a physical push button or other physical input control device instead of a touch pad.

1B is a block diagram illustrating example components for event processing, according to some embodiments. In some embodiments, memory (102 in FIG. 1A or 370 in FIG. 3) stores event sorter 170 (e.g., in operating system 126) and individual applications 136-1 (e.g., the applications described above). Includes any of (137 to 151, 155, 380 to 390).

Event sorter 170 receives event information and determines the application 136-1 to deliver the event information, and the application view 191 of application 136-1. Event sorter 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192 that represents the current application view(s) displayed on touch-sensitive display 112 when the application is active or running. In some embodiments, device/global internal state 157 is used by event sorter 170 to determine which application(s) is currently active, and application internal state 192 is used by event sorter 170 to determine which application(s) are currently active. It is used to determine the application views 191 that will deliver event information.

In some embodiments, application internal state 192 may include resume information to be used when application 136-1 resumes execution, user information that is being displayed or ready to be displayed by application 136-1. interface state information, a state queue to enable the user to return to a previous state or view of the application 136-1, and a redo/undo queue of previous actions taken by the user. Include additional information such as one or more.

The event monitor 171 receives event information from the peripheral interface 118. Event information includes information about sub-events (e.g., a user touch on touch-sensitive display 112 as part of a multi-touch gesture). Peripherals interface 118 may be connected to the I/O subsystem 106 or sensors, such as proximity sensor 166, accelerometer(s) 168, and/or microphone 113 (via audio circuitry 110). Send the information you receive. Information that peripherals interface 118 receives from I/O subsystem 106 includes information from touch-sensitive display 112 or a touch-sensitive surface.

In some embodiments, event monitor 171 sends requests to peripherals interface 118 at predetermined intervals. In response, peripherals interface 118 transmits event information. In other embodiments, peripherals interface 118 transmits event information only when there is a significant event (e.g., receiving input that exceeds a predetermined noise threshold and/or exceeds a predetermined duration) .

In some embodiments, event sorter 170 also includes a hit view determination module 172 and/or an active event recognizer determination module 173.

Hit view determination module 172 provides software procedures for determining where a sub-event occurred within one or more views when touch-sensitive display 112 displays more than one view. Views consist of controls and other elements that the user can see on the display.

Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of an individual application) for which a touch is detected optionally correspond to program levels within the application's program or view hierarchy. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events that are recognized as appropriate inputs is optionally determined based at least in part on the hit view of the initial touch that initiates the touch-based gesture. do.

Hit view determination module 172 receives information related to sub-events of a touch-based gesture. If the application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy for which the subevent should be handled. In most situations, the hit view is the lowest level view in which the initiating subevent (eg, the first subevent in a sequence of subevents forming an event or potential event) occurs. Once a hit view is identified by hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source that led it to be identified as the hit view.

The active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that contain the physical location of a sub-event are actively involved views, such that all actively involved views are It is determined that the sequence should be received. In other embodiments, even if touch sub-events are confined entirely to an area associated with one particular view, higher-level views in the hierarchy will still remain actively participating views.

Event dispatcher module 174 dispatches event information to an event recognizer (e.g., event recognizer 180). In embodiments that include the active event recognizer determination module 173, the event dispatcher module 174 forwards event information to the event recognizer determined by the active event recognizer determination module 173. In some embodiments, event dispatcher module 174 stores event information in an event queue from which event information is retrieved by a respective event receiver 182.

In some embodiments, operating system 126 includes event sorter 170. Alternatively, application 136-1 includes event sorter 170. In still other embodiments, event sorter 170 is a standalone module, or is part of another module stored in memory 102, such as contact/motion module 130.

In some embodiments, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which can handle touch events that occur within a respective view of the application's user interface. Contains commands to process them. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, each application view 191 includes a plurality of event recognizers 180. In other embodiments, one or more of the event recognizers 180 may be a separate user interface kit (not shown) or a higher-level object from which the application 136-1 inherits methods and other properties. It is part of a module. In some embodiments, each event handler 190 may receive one of event data 179 from data updater 176, object updater 177, GUI updater 178, and/or event sorter 170. Includes more. Event handler 190 optionally uses or calls data updater 176, object updater 177, or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views 191 include one or more respective event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included within a respective application view 191.

Each event recognizer 180 receives event information (e.g., event data 179) from event classifier 170 and identifies events from the event information. Event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event recognizer 180 also includes at least metadata 183 and a subset of event delivery instructions 188 (optionally including sub-event delivery instructions).

Event receiver 182 receives event information from event classifier 170. Event information includes information about sub-events, for example, touch or touch movement. Depending on the sub-event, the event information also includes additional information such as the location of the sub-event. If the sub-event involves the motion of a touch, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the events include rotation of the device from one orientation to another (e.g., from portrait to landscape, or vice versa), and the event information includes the device's current orientation (also referred to as device pose). ) includes corresponding information about.

Event comparator 184 compares event information to predefined event or sub-event definitions and, based on the comparison, determines an event or sub-event, or determines or updates the status of the event or sub-event. - Some In embodiments, event comparator 184 includes event definitions 186. Event definitions 186 include definitions of events (e.g., predefined sequences of sub-events), e.g., Event 1 (187-1), Event 2 (187-2), etc. In some embodiments, sub-events within event 187 include, for example, touch start, touch end, touch move, touch cancel, and multiple touches. In one example, the definition for event 1 187-1 is a double tap on a displayed object. A double tap can be, for example, a first touch on a displayed object during a predetermined phase (start of touch), a first liftoff during a predetermined phase (end of touch), and a first touch on the displayed object during a predetermined phase. a second touch on the top (touch start), and a second liftoff (touch end) during a predetermined phase. In another example, the definition for event 2 187-2 is dragging on a displayed object. Dragging includes, for example, a touch (or contact) on a displayed object during a predetermined phase, movement of the touch across the touch-sensitive display 112, and liftoff of the touch (termination of the touch). In some embodiments, an event also includes information about one or more associated event handlers 190.

In some embodiments, event definition 187 includes a definition of an event for the respective user interface object. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view where three user interface objects are displayed on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 determines three user interface objects. Perform a hit test to determine which of them is associated with a touch (sub-event). If each displayed object is associated with a respective event handler 190, the event comparator uses the results of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects an event handler associated with an object and subevent that triggers a hit test.

In some embodiments, the definition for each event 187 also includes deferred actions that delay delivery of event information until after it has been determined whether the sequence of sub-events does or does not correspond to the event type of the event recognizer. .

If the individual event recognizer 180 determines that the series of sub-events does not match any of the events in the event definitions 186, the individual event recognizer 180 determines that the event is not possible, the event is failed, or the event is terminated. state, and then the individual event recognizer ignores subsequent sub-events of the touch-based gesture. In this situation, other event recognizers, if any, that remain active for the hit view continue to track and process subevents of the ongoing touch-based gesture.

In some embodiments, an individual event recognizer 180 may include configurable properties, flags, and/or lists that indicate how the event delivery system should perform sub-event delivery to actively participating event recognizers. It includes metadata 183. In some embodiments, metadata 183 includes configurable properties, flags, and/or lists that indicate how event recognizers interact or can interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and/or lists that indicate whether subevents are delivered to the view or various levels in the program hierarchy.

In some embodiments, a respective event recognizer 180 activates an event handler 190 associated with an event when one or more specific sub-events of the event are recognized. In some embodiments, individual event recognizer 180 passes event information associated with the event to event handler 190. Activating the event handler 190 is separate from sending (and delaying sending) sub-events to individual hit views. In some embodiments, event recognizer 180 sends a flag associated with the recognized event, and event handler 190 associated with that flag catches that flag and performs a predefined process.

In some embodiments, event delivery instructions 188 include sub-event delivery instructions that deliver event information regarding a sub-event without activating an event handler. Instead, sub-event delivery instructions deliver event information to event handlers associated with a series of sub-events or to actively participating views. Event handlers associated with a series of sub-events or actively participating views receive event information and perform predetermined processes.

In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates a new user interface object or updates the location of a user interface object. The GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information for display on a touch-sensitive display and transmits it to graphics module 132.

In some embodiments, event handler(s) 190 includes or accesses data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included within an individual application 136-1 or a single module of application view 191. In other embodiments, they are included within two or more software modules.

1C is a block diagram illustrating a tactile output module according to some embodiments. In some embodiments, I/O subsystem 106 (e.g., haptic feedback controller 161 (FIG. 1A) and/or other input controller(s) 160 (FIG. 1A)) may be configured as shown in FIG. 1C. Includes at least some of the example components. In some embodiments, peripherals interface 118 includes at least some of the example components shown in FIG. 1C.

In some embodiments, the tactile output module includes a haptic feedback module 133. In some embodiments, haptic feedback module 133 provides user interface feedback from software applications on the electronic device (e.g., alerts and other notifications indicating the occurrence of events or performance of actions in the user interfaces of the electronic device, and Collect and combine tactile outputs (feedback) in response to user inputs corresponding to displayed user interfaces. Haptic feedback module 133 includes a waveform module 123 (to provide waveforms used to generate haptic outputs), a mixer 125 (to mix waveforms such as those in different channels), A compressor 127 (to reduce or compress the dynamic range of the waveforms), a low-pass filter 129 (to filter out high-frequency signal components in the waveforms), and a low-pass filter 129 (to adjust the waveforms according to thermal conditions). ) includes one or more of the thermal controllers 131. In some embodiments, haptic feedback module 133 is included within haptic feedback controller 161 (FIG. 1A). In some embodiments, a separate unit of haptic feedback module 133 (or a separate implementation of haptic feedback module 133) is also included within the audio controller (e.g., audio circuitry 110, FIG. 1A) It is used to generate audio signals. In some embodiments, a single haptic feedback module 133 is used to generate waveforms for tactile outputs and to generate audio signals.

In some embodiments, the haptic feedback module 133 may also include a trigger module 121 (e.g., a software application, operating system, or other software modules). In some embodiments, trigger module 121 generates trigger signals to initiate generation of waveforms (e.g., by waveform module 123). For example, the trigger module 121 generates trigger signals based on preset timing standards. In some embodiments, trigger module 121 receives trigger signals from outside of haptic feedback module 133 (e.g., in some embodiments, haptic feedback module 133 receives trigger signals outside of haptic feedback module 133). Receives trigger signals from a hardware input processing module 146 located on a user interface element (e.g., an affordance within an application or an application icon) or a hardware input device (e.g., an intensity-sensitive input device, such as an intensity-sensitive touch screen). Software applications that trigger actions (e.g., using trigger module 121) based on activation of an input surface or home button (e.g., using trigger module 121) or other components within haptic feedback module 133 (e.g., waveform module 123) relayed to In some embodiments, trigger module 121 also receives tactile feedback generating instructions (e.g., from haptic feedback module 133 of FIGS. 1A and 3 ). In some embodiments, trigger module 121 is connected to haptic feedback module 133 (e.g., from haptic feedback module 133 of FIGS. 1A and 3 ) (or trigger module 121 within haptic feedback module 133 ). Trigger signals are generated in response to receiving these tactile feedback commands.

Waveform module 123 receives trigger signals as input (e.g., from trigger module 121) and, in response to receiving the trigger signals, generates waveforms (e.g., FIGS. 4C and 4C ) for generation of one or more tactile outputs. waveforms selected from a predefined set of waveforms designated for use by waveform module 123, such as the waveforms described in more detail below with reference to FIG. 4D.

Mixer 125 receives waveforms as input (e.g., from waveform module 123) and mixes the waveforms together. For example, when mixer 125 receives two or more waveforms (e.g., a first waveform in a first channel and a second waveform that at least partially overlaps the first waveform in a second channel), The mixer 125 outputs a combined waveform corresponding to the sum of two or more waveforms. In some embodiments, mixer 125 may also mix a particular waveform(s) among two or more waveforms (e.g., by increasing the scale of the particular waveform(s) and/or decreasing the scale of the remainder of the waveforms). Modify one or more of two or more waveforms to emphasize them relative to the rest. In some situations, mixer 125 selects one or more waveforms for removal from the combined waveform (e.g., waveforms from more than three sources that were requested to be output simultaneously by tactile output generator 167). When present, the waveform from the oldest source is dropped).

Compressor 127 receives waveforms (e.g., a combined waveform from mixer 125) as input and modifies the waveforms. In some embodiments, compressor 127 reduces the waveforms (e.g., depending on the physical specifications of the tactile output generators (167 in FIG. 1A or 357 in FIG. 3)) to produce tactile output corresponding to the waveforms. causes the outputs to be reduced. In some embodiments, compressor 127 limits the waveforms, such as by enforcing a predefined maximum amplitude for the waveforms. For example, compressor 127 reduces amplitudes of portions of waveforms that exceed a predefined amplitude threshold while maintaining amplitudes of portions of waveforms that do not exceed the predefined amplitude threshold. In some embodiments, compressor 127 reduces the dynamic range of waveforms. In some embodiments, compressor 127 dynamically reduces the dynamic range of the waveforms so that the combined waveforms meet the performance specifications (e.g., force and/or moveable mass displacement limits) of tactile output generator 167. keep it within.

Low-pass filter 129 receives waveforms (e.g., compressed waveforms from compressor 127) as input and filters (e.g., smoothes) the waveforms (e.g., removes high-frequency signal components in the waveforms). or decrease). For example, in some cases, compressor 127 exceeds the performance specifications of tactile output generator 167 and/or generates tactile outputs with compressed waveforms when tactile outputs are generated according to the compressed waveforms. Contains extraneous signals (eg, high frequency signal components) that interfere with the generation of outputs. Low-pass filter 129 reduces or removes such extraneous signals from the waveforms.

Thermal controller 131 receives waveforms (e.g., filtered waveforms from low-pass filter 129) as input and controls internal temperatures (e.g., detected within device 100), e.g., a haptic feedback controller ( Adjust the waveforms according to thermal conditions of device 100 (based on the temperature of 161) and/or external temperatures detected by device 100. For example, in some cases, the output of haptic feedback controller 161 varies with temperature (e.g., haptic feedback controller 161 may, in response to receiving the same waveforms, cause haptic feedback controller 161 to make a first generates a first tactile output when at a temperature, and generates a second tactile output when the haptic feedback controller 161 is at a second temperature that is separate from the first temperature. For example, the magnitude (or amplitude) of tactile outputs may change depending on temperature. To reduce the effects of temperature changes, the waveforms are modified (eg, their amplitude is increased or decreased based on temperature).

In some embodiments, haptic feedback module 133 (e.g., trigger module 121) is coupled to hardware input processing module 146. In some embodiments, other input controller(s) 160 in FIG. 1A include hardware input processing module 146. In some embodiments, hardware input processing module 146 includes hardware input device 145 (e.g., other input or control devices 116 in Figure 1A, such as an intensity-sensitive input surface such as an intensity-sensitive touch screen). or home button). In some embodiments, hardware input device 145 can be any of the input devices described herein, such as touch-sensitive display system 112 (FIG. 1A), keyboard/mouse 350 (FIG. 3), and touch pad. 355 (Figure 3), one of the other input or control devices 116 (Figure 1A), or an intensity-sensitive home button. In some embodiments, hardware input device 145 is other than touch-sensitive display system 112 (FIG. 1A), keyboard/mouse 350 (FIG. 3), or touch pad 355 (FIG. 3). It consists of a intensity-sensitive home button. In some embodiments, in response to inputs from hardware input device 145 (e.g., an intensity-sensitive home button or touch screen), hardware input processing module 146 determines whether a user meets predefined input criteria. One or more trigger signals are provided to the haptic feedback module 133 to indicate that an input, such as an input corresponding to a “click” (eg, “down click” or “up click”) of a home button, has been detected. In some embodiments, haptic feedback module 133 provides waveforms corresponding to a “click” of the home button in response to an input corresponding to a “click” of the home button, simulating the haptic feedback of pressing a physical home button. do.

In some embodiments, the tactile output module includes a haptic feedback controller 161 (e.g., haptic feedback controller 161 in FIG. 1A), which controls the generation of tactile outputs. In some embodiments, the haptic feedback controller 161 is coupled to a plurality of tactile output generators and selects one or more of the plurality of tactile output generators to generate a waveform for generating the tactile outputs. transmits the selected tactile output generators to one or more tactile output generators. In some embodiments, haptic feedback controller 161 controls tactile output requests corresponding to activation of hardware input device 145 and tactile output requests corresponding to software events (e.g., a haptic feedback module ( 133) and (e.g., to prioritize tactile outputs corresponding to activation of hardware input device 145 over tactile outputs corresponding to software events). , one or more of two or more waveforms to emphasize a particular waveform(s) relative to the rest of the two or more waveforms (by increasing the scale of the particular waveform(s) and/or decreasing the scale of the remainder of the waveforms). Modify waveforms.

In some embodiments, as shown in Figure 1C, the output of haptic feedback controller 161 is coupled to audio circuitry of device 100 (e.g., audio circuitry 110 of Figure 1A) and sends audio signals to the device. It is provided to the audio circuit part of (100). In some embodiments, haptic feedback controller 161 provides both waveforms used to generate tactile outputs and audio signals used to provide audio outputs in conjunction with generation of the tactile outputs. In some embodiments, haptic feedback controller 161 modifies the audio signals and/or waveforms (used to generate the haptic outputs) to control the audio (e.g., by delaying the audio signals and/or waveforms). Ensures that the outputs and tactile outputs are synchronized. In some embodiments, haptic feedback controller 161 includes a digital-to-analog converter used to convert digital waveforms into analog signals, which can be output to amplifier 163 and/or tactile output generator 167. ) is received by.

In some embodiments, the tactile output module includes an amplifier 163. In some embodiments, amplifier 163 receives waveforms (e.g., from haptic feedback controller 161), amplifies the waveforms, and then sends the amplified waveforms to tactile output generator 167 (e.g., tactile output generator 167). Transmit to the generators (any of 167 in FIG. 1A or 357 in FIG. 3). For example, amplifier 163 may convert the received waveforms to signal levels that comply with the physical specifications of tactile output generator 167 (e.g., signals transmitted to tactile output generator 167 may be converted to a haptic feedback controller (e.g., amplify the waveforms received from 161) with the required voltage and/or current by a tactile output generator 167 to generate tactile outputs corresponding to the waveforms received from 161), and generate the amplified waveforms tactilely. Sent to output generator 167. In response, tactile output generator 167 generates tactile outputs (e.g., by shifting the movable mass back and forth in one or more dimensions relative to the movable mass's neutral position).

In some embodiments, the tactile output module includes a sensor 169, which is coupled to the tactile output generator 167. Sensor 169 may be a component of tactile output generator 167 or one or more components of tactile output generator 167 (e.g., one or more moving parts such as a membrane used to generate tactile outputs). Detect states or state changes (eg, mechanical position, physical displacement, and/or movement). In some embodiments, sensor 169 is a magnetic field sensor (eg, a Hall effect sensor) or other displacement and/or movement sensor. In some embodiments, sensor 169 provides information (e.g., the position, displacement, and/or movement of one or more components within tactile output generator 167) to haptic feedback controller 161 and provides tactile output. Depending on the information provided by the sensor 169 regarding the state of the generator 167, the haptic feedback controller 161 determines the waveforms output from the haptic feedback controller 161 (e.g., optionally via the amplifier 163), Adjust the waveforms transmitted to the red output generator 167.

The foregoing discussion of event handling of user touches on a touch-sensitive display also applies to other forms of user input for operating multifunction devices 100 with input devices, but not all of which originate on the touch screens. You must understand that this is not the case. For example, mouse movements and mouse button presses, optionally combined with single or multiple keyboard presses or holds; Contact movements such as tapping, dragging, scrolling, etc. on the touchpad; pen stylus inputs; movement of the device; verbal commands; detected eye movements; biometric inputs; and/or any combination thereof are optionally used as inputs corresponding to sub-events defining the event to be recognized.

2 shows a portable multifunction device 100 with a touch screen 112, according to some embodiments. The touch screen optionally displays one or more graphics within user interface (UI) 200. In this embodiment, as well as other embodiments described below, the user may use, for example, one or more fingers 202 (not shown to scale in the figure) or one or more styluses 203 (not to scale in the figure). It is possible to select one or more of the graphics by making a gesture on the graphics using (not shown as shown). In some embodiments, selection of one or more graphics occurs when the user ceases contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (left to right, right to left, upward and/or downward) of a finger in contact with device 100. , and/or rolling (right to left, left to right, upward and/or downward). In some implementations or situations, if a graphic is inadvertently contacted, the graphic is not selected. For example, when the gesture corresponding to selection is a tap, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application.

Device 100 also optionally includes one or more physical buttons, such as a “home” or menu button 204. As described above, menu button 204 is optionally used to navigate to any application 136 in the set of applications optionally running on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in the GUI displayed on touch screen 112.

In some embodiments, device 100 includes a touch screen 112, a menu button 204, a push button 206 for powering on/off and locking the device, and volume control button(s) 208. , a subscriber identity module (SIM) card slot 210, a headset jack 212, and a docking/charging external port 124. Push button 206 optionally powers on/off the device by pressing the button and holding the button pressed for a predefined time interval; lock the device by pressing a button and releasing the button before a predefined time interval has elapsed; Used to unlock the device or initiate the unlocking process. In an alternative embodiment, device 100 also accepts verbal input for activating or deactivating some functions via microphone 113. Device 100 may also optionally include one or more contact intensity sensors 165 to detect the intensity of contacts on touch screen 112 and/or to generate tactile outputs for a user of device 100. Includes one or more tactile output generators 167.

3 is a block diagram of an example multifunction device having a display and a touch-sensitive surface, according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 may be a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., for home or industrial use). controller). Device 300 typically includes one or more processing units (CPU) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses ( 320). Communication buses 320 optionally include circuitry (sometimes referred to as a chipset) that interconnects system components and controls communications between them. Device 300 includes an input/output (I/O) interface 330 that includes a display 340, typically a touch screen display. I/O interface 330 also optionally includes a keyboard and/or mouse (or other pointing device) 350 and touchpad 355, tactile output devices for generating tactile outputs on device 300. Generator 357 (e.g., similar to the tactile output generator(s) 167 described above with reference to FIG. 1A ), and sensors 359 (e.g., the contact intensity sensor(s) described above with reference to FIG. 1A ) includes an optical sensor, an acceleration sensor, a proximity sensor, a touch-sensitive sensor, and/or a contact intensity sensor) similar to 165. Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid state memory devices; Optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU(s) 310. In some embodiments, memory 370 includes programs, modules, and data structures similar to the programs, modules, and data structures stored in memory 102 of portable multifunction device 100 (Figure 1A). , or a subset of them. Additionally, memory 370 optionally stores additional programs, modules and data structures not present within memory 102 of portable multifunction device 100. For example, memory 370 of device 300 may optionally include a drawing module 380, a presentation module 382, a word processing module 384, a website creation module 386, and a disk authoring module 388. ), and/or spreadsheet module 390, while memory 102 of portable multifunction device 100 (FIG. 1A) optionally does not store such modules.

Each of the previously identified elements in Figure 3 is, optionally, stored in one or more of the memory devices described above. Each of the previously identified modules corresponds to a set of instructions for performing the above-described functions. The modules or programs (e.g., sets of instructions) identified above need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules may optionally be combined in various embodiments. Either it is or it is rearranged. In some embodiments, memory 370 optionally stores a subset of the previously identified modules and data structures. Additionally, memory 370 optionally stores additional modules and data structures not described above.

Attention now turns to embodiments of user interfaces that are optionally implemented, for example, on portable multifunction device 100.

FIG. 4A shows an example user interface for a menu of applications on portable multifunction device 100, according to some embodiments. Similar user interfaces are optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:

signal strength indicator(s) 402 for wireless communication(s) such as cellular and Wi-Fi signals;

time(404);

Bluetooth indicator (405);

battery status indicator 406;

Tray 408 with icons for frequently used applications, such as:

° Icon 416 for the phone module 138 labeled “Phone”, optionally including an indicator 414 of the number of missed calls or voicemail messages;

° Icon 418 for the email client module 140 labeled “Mail”, optionally including an indicator 410 of the number of unread emails;

° an icon 420 for the browser module 147 labeled “browser”; and

° an icon 422 for a video and music player module 152 labeled “iPod,” also referred to as iPod (a trademark of Apple Inc.) module 152; and

Icons for different applications, such as:

° Icon 424 for IM module 141 labeled “Messages”;

° Icon 426 for the calendar module 148 labeled “Calendar”;

° Icon 428 for the image management module 144 labeled “Photos”;

° Icon 430 for camera module 143 labeled “Camera”;

° Icon 432 for the Online Video module 155 labeled “Online Video”;

° Icon (434) for stock widget (149-2) labeled “Stocks”;

° Icon 436 for map module 154 labeled “Maps”;

° Icon (438) for weather widget (149-1) labeled “Weather”;

° Icon 440 for an alarm clock widget 149-4 labeled “Clock”;

° Icon 442 for the exercise support module 142 labeled “exercise support”;

° Icon 444 for a notes module 153 labeled “Notes”; and

° Icon 446 for a Settings application or module, labeled “Settings”, which provides access to settings for the device 100 and its various applications 136.

It should be noted that the icon labels shown in Figure 4A are illustrative only. For example, icon 422 for video and music player module 152 is labeled “Music” or “Music Player.” Other labels are optionally used for various application icons. In some embodiments, the label for an individual application icon includes the name of the application that corresponds to the individual application icon. In some embodiments, the label for a particular application icon is separate from the name of the application that corresponds to the particular application icon.

FIG. 4B illustrates a device (e.g., FIG. 3 An example user interface on device 300 is shown. Device 300 may also optionally include one or more contact intensity sensors (e.g., one or more of sensors 359 ) and/or device 300 to detect the intensity of contacts on touch-sensitive surface 451 and one or more tactile output generators 357 for generating tactile outputs for the user.

Although some of the examples that follow will be presented with reference to inputs on touch screen display 112 (a combination of a touch-sensitive surface and display), in some embodiments, the device may be configured separately from the display, as shown in FIG. 4B. Detect inputs on a personal touch-sensitive surface. In some embodiments, the touch-sensitive surface (e.g., 451 in FIG. 4B) has a major axis (e.g., 452 in FIG. 4B) corresponding to a major axis (e.g., 453 in FIG. 4B) on the display (e.g., 450). According to these embodiments, the device interacts with the touch-sensitive surface 451 at positions that correspond to respective positions on the display (e.g., in FIG. 4B, 460 corresponds to 468 and 462 corresponds to 470). Detect contacts (eg, 460 and 462 in FIG. 4B). In this way, user inputs (e.g., contacts 460, 462 and movements thereof) detected by the device on the touch-sensitive surface (e.g., 451 in FIG. 4B) can be performed while the touch-sensitive surface is separate from the display. When used by the device to manipulate the user interface on the display of the multifunction device (e.g., 450 in FIG. 4B). It should be understood that similar methods, optionally, may be used for other user interfaces described herein.

Additionally, while the examples below are given primarily with reference to finger inputs (e.g., finger contacts, finger tap gestures, finger swipe gestures), in some embodiments, one or more of the finger inputs may be used with another input device. It should be understood that input from (eg, mouse-based input or stylus input) is replaced. For example, a swipe gesture is optionally replaced with a mouse click (eg, instead of a contact) followed by movement of the cursor along the path of the swipe (eg, instead of a movement of the contact). As another example, the tap gesture is optionally replaced by a mouse click while the cursor is positioned over the location of the tap gesture (e.g., instead of stopping to detect contact following detection of the contact). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are being used selectively simultaneously, or mouse and finger contacts are being used selectively simultaneously.

FIG. 5A depicts an example personal electronic device 500. Device 500 includes a body 502 . In some embodiments, device 500 may include some or all of the features described with respect to devices 100 and 300 (e.g., FIGS. 1A-4B). In some embodiments, device 500 has a touch-sensitive display screen 504 (hereinafter referred to as touch screen 504). Alternatively or in addition to a touch screen 504, device 500 has a display and a touch-sensitive surface. As with devices 100, 300, in some embodiments, touch screen 504 (or touch-sensitive surface) is optionally configured to detect the intensity of contacts (e.g., touches) being made. Includes more than one intensity sensor. One or more intensity sensors of touch screen 504 (or a touch-sensitive surface) may provide output data representing the intensity of touches. The user interface of device 500 may respond to touches based on the intensity of the touches, meaning that touches of different intensities may invoke different user interface operations on device 500.

Exemplary techniques for detecting and processing touch intensity are described, for example, in related applications: Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to International Patent Application PCT/US2013/040061 (published as WIPO Publication No. WO/2013/169849), filed on November 11, 2013 and entitled "Device, Method, and Graphical User Interface" for Transitioning Between Touch Input to Display Output Relationships", International Patent Application No. PCT/US2013/069483, published as WIPO Publication No. WO/2014/105276, each of which is hereby incorporated by reference in its entirety.

In some embodiments, device 500 has one or more input mechanisms 506, 508. Input mechanisms 506, 508 (if included) may be physical. Examples of physical input mechanisms include push buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. These attachment mechanisms (if included) allow device 500 to attach, for example, a hat, glasses, earrings, necklace, shirt, jacket, bracelet, watch strap, chain, pants, belt, shoe, wallet, etc. , so that it can be attached to a backpack, etc. These attachment mechanisms allow device 500 to be worn by a user.

FIG. 5B depicts an example personal electronic device 500. In some embodiments, device 500 may include some or all of the components described with respect to FIGS. 1A, 1B, and 3. Device 500 has a bus 512 that operably couples I/O section 514 with one or more computer processors 516 and memory 518. I/O section 514 can be connected to display 504, which can have a touch-sensitive component 522 and, optionally, an intensity sensor 524 (eg, a contact intensity sensor). I/O section 514 may also be configured to communicate with communication unit 530 to receive application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular, and/or other wireless communication techniques. can be connected to. Device 500 may include input mechanisms 506 and/or 508 . The input mechanism 506 is optionally a rotatable input device or, for example, a pushable and rotatable input device. In some examples, input mechanism 508 is, optionally, a button.

In some examples, input mechanism 508 is, optionally, a microphone. Personal electronic device 500 optionally includes a GPS sensor 532, an accelerometer 534, a direction sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and/or their It includes a variety of sensors, such as combinations, all of which may be operably connected to the I/O section 514.

Memory 518 of personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions, e.g., to one or more computer processors 516. When executed by computer processors, processes (800, 1000, 1200, 1400, 1600, 1800, 2000, 2200, 2500, 2700, 2900, 3100, 3300, 3500, 3700, 3900, 4100, 4300) (FIG. 8, Figure 10, Figure 12, Figure 14, Figure 16, Figure 18, Figure 20, Figure 22, Figure 25, Figure 27, Figure 29, Figure 31, Figure 33, Figure 35, Figure 37, Figure 39, Figure 41, The techniques described below, including Figure 43), may be performed. A computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a temporary computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic, optical, and/or semiconductor storages. Examples of such storage include persistent solid state memory such as flash, solid state drives, etc., as well as magnetic disks, optical disks based on CD, DVD, or Blu-ray technologies. Personal electronic device 500 is not limited to the components and configuration of FIG. 5B and may include other or additional components in a number of configurations.

As used herein, the term “affordance” refers to a user interaction that is selectively displayed on the display screen of devices 100, 300, and/or 500 (FIGS. 1A, 3, and 5A and 5B). Refers to a user-interactive graphical user interface object. For example, images (eg, icons), buttons, and text (eg, hyperlinks) each optionally constitute an affordance.

As used herein, the term “focus selector” refers to an input element that indicates the current portion of the user interface with which the user is interacting. In some implementations that include a cursor or other location marker, a touch-sensitive surface (e.g., FIG. 3 When an input (e.g., a press input) is detected on the touchpad 355 in or the touch-sensitive surface 451 in FIG. It functions as Some implementations include a touch screen display (e.g., touch-sensitive display system 112 in FIG. 1A or touch screen 112 in FIG. 4A) that enables direct interaction with user interface elements on the touch screen display. In the field, a particular user interface element is detected when an input (e.g., a tactile press input) is detected on the touch screen display at the location of the particular user interface element (e.g., a button, window, slider, or other user interface element). Contacts detected on the touch screen function as “focus selectors” to adjust according to the input received. In some implementations, focus is moved to one of the user interfaces without movement of a corresponding cursor or touch on the touch screen display (e.g., by using the tab key or arrow keys to move focus from one button to another). moves from one area to another area of the user interface; In these implementations, the focus selector moves in response to movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is generally used to communicate the user's intended interaction with the user interface (e.g., by indicating to the device the element of the user interface with which the user wishes to interact). A user interface element controlled by (or by contact on a touch screen display). For example, while a press input is detected on a touch-sensitive surface (e.g., a touchpad or touch screen), the position of the focus selector (e.g., a cursor, contact, or selection box) on each button (on the device's display) (unlike other visible user interface elements) will indicate that the user is attempting to activate the respective button.

As used in the specification and claims, the term “characteristic intensity” of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity may optionally be generated over a predefined number of intensity samples, or (e.g., after detecting the contact, before detecting lift-off of the contact, before or after detecting the onset of movement of the contact, A predetermined period of time for a predefined event (e.g., prior to detecting the end of contact, before or after detecting an increase in the intensity of contact, and/or before or after detecting a decrease in intensity of contact) It is based on a set of intensity samples collected for 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10 seconds). The characteristic intensity of contact may optionally be the maximum value of the intensities of contact, the mean value of the intensities of contact, the average value of the intensities of contact, the top 10 percentile value of the intensities of contact. ), a value of half the maximum value of the contact intensities, a value of 90 percent of the maximum value of the contact intensities, etc. In some embodiments, the duration of contact is used to determine characteristic intensity (eg, when the characteristic intensity is an average of the intensity of contact over time). In some embodiments, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether an action was performed by a user. For example, the set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In this example, as a result of the contact having a characteristic intensity that does not exceed the first intensity threshold, a first action is performed, and as a result of the contact having a characteristic intensity that exceeds the first intensity threshold but does not exceed the second intensity threshold, a second action is performed. An action is performed and, as a result of the contact having a characteristic intensity above the second threshold, a third action is performed. In some embodiments, the comparison between the characteristic strength and one or more thresholds is used to determine whether to perform one or more operations (e.g., perform an individual operation), rather than being used to determine whether to perform a first operation or a second operation. It is used to decide whether to perform an action or to withhold from performing an individual action.

Figure 5C illustrates detecting a plurality of contacts 552A-552E on a touch-sensitive display screen 504 using a plurality of intensity sensors 524A-524D. FIG. 5C further includes intensity diagrams showing current intensity measurements of intensity sensors 524A-524D in intensity units. In this example, the intensity measurements of intensity sensors 524A and 524D are each 9 intensity units, and the intensity measurements of intensity sensors 524B and 524C are each 7 intensity units. In some implementations, the total intensity is the sum of intensity measurements from a plurality of intensity sensors 524A-524D, which is 32 intensity units in this example. In some embodiments, each contact is assigned an individual intensity that is a fraction of the total intensity. FIG. 5D shows assigning total intensity to contacts 552A-552E based on their respective distances from center of force 554. In this example, contacts 552A, 552B, and 552E are each assigned a contact intensity of 8 intensity units out of the total intensity, and contacts 552C, 552D are each assigned a contact intensity of 4 intensity units out of the total intensity. is assigned. More generally, in some implementations, each contact (j) is assigned an individual intensity (Ij) that is a fraction of the total intensity (A) according to a predefined mathematical function Ij = A(Dj/ΣDi); , where Dj is the distance of the individual contact (j) to the center of force, and ΣDi is the sum of the distances of all individual contacts to the center of force (e.g., i=1 to last). The operations described with reference to FIGS. 5C and 5D may be performed using an electronic device similar to or identical to the device 100, 300, or 500. In some embodiments, the characteristic intensity of contact is based on one or more intensities of contact. In some embodiments, intensity sensors are used to determine a single characteristic intensity (eg, a single characteristic intensity of a single contact). It should be noted that the intensity diagrams are not part of the displayed user interface and are included in FIGS. 5C and 5D to assist the reader.

In some embodiments, a portion of the gesture is identified to determine characteristic intensity. For example, the touch-sensitive surface optionally receives successive swipe contacts that transition from a starting position to reach an ending position, at which point the intensity of the contact increases. In this example, the characteristic intensity of the contact at the end position is optionally based on only a portion of the continuous swipe contact (eg, only the portion of the swipe contact at the end position) rather than the entire swipe contact. In some embodiments, a smoothing algorithm is optionally applied to the intensities of the swipe contact before determining the characteristic intensity of the contact. For example, the smoothing algorithm may optionally be one of an unweighted sliding-average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and/or an exponential smoothing algorithm. Contains one or more In some situations, these smoothing algorithms remove narrow spikes or dips in the intensities of a swipe contact to determine the characteristic intensity.

The intensity of the contact on the touch-sensitive surface is optionally characterized against one or more intensity thresholds, such as a contact-detection intensity threshold, a light press intensity threshold, a deep press intensity threshold, and/or one or more other intensity thresholds. In some embodiments, the light press intensity threshold corresponds to the intensity at which the device will perform operations typically associated with clicking a button on a physical mouse or trackpad. In some embodiments, the deep press intensity threshold corresponds to the intensity at which the device will perform operations that are different from operations typically associated with clicking a button on a physical mouse or trackpad. In some embodiments, when contact is detected with a characteristic intensity below the light press intensity threshold (e.g., above the nominal contact detection intensity threshold, below which contact is no longer detected), the device determines the light press intensity. It will move the focus selector according to the movement of the contact on the touch-sensitive surface without performing any actions associated with the threshold or deep press intensity threshold. Generally, unless otherwise stated, these intensity thresholds are consistent between different sets of user interface diagrams.

An increase in the characteristic intensity of a contact from an intensity below the light press intensity threshold to an intensity between the light press intensity threshold and the deep press intensity threshold is sometimes referred to as a “light press” input. An increase in the characteristic intensity of a contact from an intensity below the deep press intensity threshold to an intensity above the deep press intensity threshold is sometimes referred to as a “deep press” input. An increase in the characteristic intensity of a contact from an intensity below the contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light press intensity threshold is sometimes referred to as detecting contact on the touch surface. A decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold to an intensity below the contact detection intensity threshold is sometimes referred to as detecting lift-off of the contact from the touch surface. In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold is greater than zero.

In some embodiments described herein, one or more actions may be performed in response to detecting a gesture comprising an individual press input or to detect an individual press input performed with an individual contact (or multiple contacts). is performed in response to: wherein the individual press input is detected based at least in part on detecting an increase in the intensity of the contact (or plurality of contacts) above a press input intensity threshold. In some embodiments, the respective action is performed in response to detecting an increase in the intensity of the individual contact above a pressing input intensity threshold (e.g., a “down stroke” of the individual pressing input). In some embodiments, the press input includes an increase in the intensity of the individual contacts above the press input intensity threshold and a subsequent decrease in the intensity of the contacts below the press input intensity threshold, and the individual actions are individual contacts below the press input intensity threshold. This is performed in response to detecting a subsequent decrease in the intensity of the contact (e.g., an “up stroke” of an individual press input).

5E-5H illustrate contact 562 from an intensity below the light press intensity threshold (e.g., “IT _L ”) of FIG. 5E to an intensity above the deep press intensity threshold (e.g., “IT _D ”) of FIG. 5H. It shows the detection of a gesture containing a press input corresponding to an increase in intensity. On the displayed user interface 570, which includes application icons 572A to 572D displayed in predefined area 574, while cursor 576 is displayed over application icon 572B corresponding to App 2, A gesture performed using contact 562 is detected on touch-sensitive surface 560. In some embodiments, a gesture is detected on touch-sensitive display 504. Intensity sensors detect the intensity of contacts on the touch-sensitive surface 560. The device determines that the intensity of contact 562 has peaked above a deep press intensity threshold (e.g., “IT _D ”). Contact 562 is maintained on touch-sensitive surface 560. In response to detection of the gesture, and according to contact 562 having an intensity that exceeds the deep press intensity threshold (e.g., "IT _D ") during the gesture, reduced scale representations of recently viewed documents for App 2 ( 578A-578C) (eg, thumbnails) are displayed, as shown in FIGS. 5F-5H. In some embodiments, the intensity compared to one or more intensity thresholds is a characteristic intensity of the contact. It should be noted that the intensity diagram for contact 562 is not part of the displayed user interface and is included in FIGS. 5E-5H to assist the reader.

In some embodiments, the display of representations 578A-578C includes animation. For example, as shown in Figure 5F, representation 578A is initially displayed proximate application icon 572B. As the animation progresses, representation 578A moves up and representation 578B is displayed proximate to application icon 572B, as shown in Figure 5G. Then, as shown in Figure 5H, representation 578A moves up, representation 578B moves up toward representation 578A, and representation 578C is displayed proximate to application icon 572B. Representations 578A-578C form an array over icon 572B. In some embodiments, as shown in FIGS. 5F and 5G, the animation progresses depending on the intensity of contact 562, such that the intensity of contact 562 exceeds a deep press intensity threshold (e.g., “IT _D ”). As we increase toward, representations 578A through 578C appear and move upward. In some embodiments, the intensity on which the progress of the animation is based is the characteristic intensity of the contact. Operations described with reference to FIGS. 5E to 5H may be performed using an electronic device similar to or identical to device 100, 300, or 500.

In some embodiments, the device employs intensity hysteresis to avoid accidental inputs, sometimes referred to as “jitter,” where the device sets a hysteresis intensity threshold ( For example, define or select the hysteresis intensity threshold to be As such, in some embodiments, the pressing input includes an increase in the intensity of the individual contact above the pressing input intensity threshold and a subsequent decrease in the intensity of the contact below the hysteresis intensity threshold corresponding to the pressing input intensity threshold, The operation of is performed in response to detecting a subsequent decrease in the intensity of the individual contact below the hysteresis intensity threshold (eg, an “up stroke” of the individual press input). Similarly, in some embodiments, the press input causes the device to increase the intensity of the contact from an intensity below the hysteresis intensity threshold to an intensity above the press input intensity threshold, and, optionally, to an intensity below the hysteresis intensity threshold. Detected only upon detecting a subsequent decrease in the intensity of the contact, the respective action is performed in response to detecting a pressing input (eg, an increase in the intensity of the contact or a decrease in the intensity of the contact depending on the surroundings).

For ease of explanation, descriptions of actions performed in response to a press input associated with a press input intensity threshold or in response to a gesture that includes a press input include, optionally, an increase in the intensity of the contact above the press input intensity threshold, a hysteresis intensity threshold; increasing the intensity of the contact from an intensity below to an intensity above the pressing input intensity threshold, decreasing the intensity of the contact below the pressing input intensity threshold, and/or increasing the intensity of the contact below the hysteresis intensity threshold corresponding to the pressing input intensity threshold. Triggered in response to detecting either decrease. Additionally, in examples where the operation is described as being performed in response to detecting a decrease in the intensity of the contact below a pressing input intensity threshold, the operation may optionally be performed in response to a pressing input intensity threshold and below a lower hysteresis intensity threshold. It is performed in response to detecting a decrease in the intensity of contact.

As used herein, an “installed application” refers to a software application that has been downloaded onto an electronic device (e.g., devices 100, 300, and/or 500) and will be launched (e.g., opened) on the device. ) Get ready. In some embodiments, a downloaded application becomes an application installed through an installation program that extracts program portions from a downloaded package and integrates the extracted portions with the computer system's operating system.

As used herein, the terms “open application” or “running application” refer to held state (e.g., as part of device/global internal state 157 and/or application internal state 192). Refers to a software application that holds information. The open or running application is optionally any of the following application types:

the active application currently displayed on the display screen of the device on which the application is being used;

a background application (or background process) that is not currently displayed, but one or more processes for the application are being processed by one or more processors; and

A suspended or hibernated application that is not running, but has state information stored in memory (volatile and non-volatile, respectively) and can be used to resume execution of the application.

As used herein, the term “closed application” refers to software applications for which no state information is retained (e.g., state information for closed applications is not stored in the device's memory). Accordingly, closing an application includes stopping and/or removing application processes for the application and removing state information for the application from the device's memory. Typically, opening a second application while in a first application does not close the first application. When the second application is displayed and the first application stops being displayed, the first application becomes a background application.

Attention now is directed to embodiments of user interfaces (“UIs”) and associated processes implemented on an electronic device, such as portable multifunction device 100, device 300, or device 500.

6 illustrates example devices connected via one or more communication channels to engage in a transaction, according to some embodiments. One or more example electronic devices (e.g., devices 100, 300, 500) may optionally detect an input (e.g., a specific user input, an NFC field) and optionally (e.g., use NFC) (using) is configured to transmit payment information. The one or more electronic devices optionally include NFC hardware and are configured to be NFC-enabled.

Electronic devices (eg, devices 100, 300, 500) are configured to optionally store payment account information associated with each of one or more payment accounts. Payment account information may include, for example, the person or company's name, billing address, login, password, account number, expiration date, security code, phone number, bank associated with the payment account (e.g., issuing bank), and card network. Contains one or more of the identifiers. In some examples, payment account information includes an image, such as a photograph, of a payment card (eg, captured by and/or received by the device). In some examples, the electronic devices receive user input including at least some payment account information (e.g., receiving a user-entered credit, debit, account, or gift card number and expiration date). In some examples, electronic devices detect at least some payment account information from an image (e.g., of a payment card captured by a camera sensor of the device). In some examples, electronic devices receive at least some payment account information from another device (eg, another user device or server). In some examples, the electronic device may receive information from a server associated with another service (e.g., an app that rents or sells audio and/or video files) that the user or an account for the user device has previously purchased or identified payment account data for. Receive payment account information.

In some embodiments, a payment account is added to an electronic device (e.g., device 100, 300, 500) such that payment account information is securely stored on the electronic device. In some examples, after the user initiates such a process, the electronic device transmits information about the payment account to a transaction-reconciliation server, which then communicates with the payment network for the account to ensure validity of the information. Communicates with an operating server (e.g., payment server). The electronic device is optionally configured to receive a script from the server that allows the electronic device to program payment information for the account onto the secure element.

In some embodiments, communication between electronic devices 100, 300, 500 facilitate transactions (eg, general or specific transactions). For example, the first electronic device (e.g., 100) may serve as a provisioning or management device and provide notifications of new or updated payment account data (e.g., information about new accounts, existing Updated information about the account, and/or alerts related to an existing account) may be transmitted to a second electronic device (eg, 500). In another example, a first electronic device (eg, 100) may transmit data to a second electronic device, where the data reflects information regarding payment transactions facilitated at the first electronic device. The information optionally includes one or more of the amount paid, account used, time of purchase, and whether the default account has been changed. The second device (eg, 500) optionally uses such information to update the default payment account (eg, based on a learning algorithm or explicit user input).

Electronic devices (eg, 100, 300, 500) are configured to communicate with each other via any of a variety of networks. For example, devices communicate using a Bluetooth connection 608 (e.g., including a traditional Bluetooth connection or a Bluetooth Low Energy connection) or using a WiFi network 606. Communications between user devices are optionally coordinated to reduce the likelihood of inappropriately sharing information across devices. For example, communications related to payment information require that the communicating devices be paired (eg, associated with each other through explicit user interaction) or associated with the same user account.

In some embodiments, an electronic device (e.g., 100, 300, 500) is used to communicate with a point-of-sale (POS) payment terminal 600, optionally NFC-enabled. . Communication optionally occurs using various communication channels and/or technologies. In some examples, the electronic device (eg, 100, 300, 500) communicates with the payment terminal 600 using the NFC channel 610. In some examples, payment terminal 600 communicates with an electronic device (eg, 100, 300, 500) using peer-to-peer NFC mode. The electronic device (e.g., 100, 300, 500) optionally sends a signal containing payment information for a payment account (e.g., a default account, or an account selected for a particular transaction) to payment terminal 600. It is configured to transmit to.

In some embodiments, conducting a transaction includes transmitting a signal containing payment information for an account, such as a payment account. In some embodiments, conducting a transaction reconfigures the electronic device (e.g., 100, 300, 500) to respond as a contactless payment card, such as an NFC-enabled contactless payment card, and then makes the payment via NFC, e.g. It includes transmitting the account credentials to the terminal 600. In some embodiments, following transmitting the account's credentials via NFC, the electronic device is reconfigured not to respond as a contactless payment card (e.g., requires authorization before being reconfigured again to respond as a contactless payment card via NFC). required).

In some embodiments, the generation and/or transmission of the signal is controlled by a secure element within the electronic device (eg, 100, 300, 500). The secure element optionally requires certain user input before releasing payment information. For example, the secure element may optionally detect that the electronic device is being worn, detect a button press, detect the entry of a passcode, detect a touch (e.g., received while interacting with an application), It requires detection of one or more option selections, detection of a fingerprint signature, detection of a voice or voice command, and/or detection of a gesture or movement (e.g., rotation or acceleration). In some examples, if a communication channel (e.g., NFC communication channel) with another device (e.g., payment terminal 600) is established within a defined period of time from detection of the input, the secure element may For example, payment information to be transmitted to the payment terminal 600 is released. In some examples, a secure element is a hardware component that controls the release of secure information. In some examples, a secure element is a software component that controls the release of secure information.

In some embodiments, protocols involved in transaction participation depend on device types, for example. For example, conditions for generating and/or transmitting payment information may be different for a wearable device (e.g., device 500) and a phone (e.g., device 100). For example, a generating and/or transmitting condition for a wearable device includes detecting that a button has been pressed (e.g., after security verification), whereas a corresponding condition for a phone does not require a button press but rather detects that a button has been pressed. Instead, it requires detection of specific interactions with the application. In some examples, conditions for transmitting and/or emitting payment information include receiving specific input on each of multiple devices. For example, release of payment information may optionally include detection of a fingerprint and/or passcode on a device (e.g., device 100) and mechanical input on another device (e.g., device 500). Requests detection of a button press (e.g. a button press).

The payment terminal 600 optionally uses the payment information to generate a signal to be transmitted to the payment server 604 to determine whether the payment is authorized. Payment server 604 optionally includes any device or system configured to receive payment information associated with a payment account and determine whether the proposed purchase is authorized. In some examples, payment server 604 includes a server at an issuing bank. The payment terminal 600 communicates directly with the payment server 604 or indirectly through one or more other devices or systems (eg, an acquiring bank's server and/or a card network's server).

Payment server 604 optionally uses at least a portion of the payment information to identify the user account among a database of user accounts (e.g., 602). For example, each user account includes payment information. An account is located, optionally, by locating an account with specific payment information that matches that from the POS communication. In some instances, the payment information provided does not match (e.g., the expiration date does not correspond to a credit, debit or gift card number) or no account contains payment information that matches that from the POS communication. , the payment is declined.

In some embodiments, data about a user's account may include one or more restrictions (e.g., credit limit); Current or previous balance; The date, location and/or amount of your previous transaction; Further identifies account status (e.g., active or frozen), and/or authorization instructions. In some examples, the payment server (e.g., 604) uses such data to determine whether to authorize the payment. For example, the payment server may determine if the purchase amount added to your current balance would cause you to exceed your account limits, if your account has been frozen, if the amount of previous transactions exceeds the threshold, or if the number or frequency of previous transactions exceeds the threshold. If it exceeds this amount, payment will be refused.

In some embodiments, payment server 604 responds to POS payment terminal 600 with an indication as to whether the proposed purchase is authorized or rejected. In some examples, POS payment terminal 600 transmits a signal to an electronic device (eg, 100, 300, 500) to identify the result. For example, POS payment terminal 600 may be connected to an electronic device (e.g., 100, 300, 500) when a purchase is authorized (e.g., through a transaction-reconciliation server that manages a transaction app on the user device). Send the receipt to In some cases, POS payment terminal 600 presents output (e.g., visual or audio output) indicating the results. Payment may be sent to the merchant as part of the authorization process, or may be sent at a later date.

In some embodiments, the electronic device (e.g., 100, 300, 500) participates in a transaction that is completed without involvement of the POS payment terminal 600. For example, upon detecting that a mechanical input has been received, a secure element within the electronic device (e.g., 100, 300, 500) may emit payment information so that an application on the electronic device can access that information (e.g., and to transmit information to servers associated with the application).

In some embodiments, the electronic device (eg, 100, 300, 500) is in a locked or unlocked state. In the locked state, the electronic device is powered and operating, but is prevented from performing a predefined set of operations in response to user input. The predefined set of operations optionally include navigation between user interfaces, activating or deactivating a predefined set of functions, and activating or deactivating specific applications. The locked state can be used to prevent unintended or unauthorized use of some functions of the electronic device or activation or deactivation of some functions on the electronic device. In the unlocked state, the electronic device 100 is powered and operating and is not prevented from performing at least some of a predefined set of operations that cannot be performed while in the locked state.

When a device is in a locked state, the device is said to be locked. In some embodiments, a device in a locked state may optionally respond to a limited set of user inputs, including an input corresponding to an attempt to transition the device to an unlocked state or an input corresponding to powering down the device. Respond.

In some examples, the secure element (e.g., 115) is a hardware component (e.g., a secure microcontroller chip) configured to securely store data or algorithms such that the securely stored data is accessible by a device without appropriate authentication information from the user of the device. Don't do it. Keeping stored data secure within a secure element that is separate from other storage on the device ensures that stored data remains secure even if other storage locations on the device are compromised (e.g., by malware or other attempts to compromise information stored on the device). prevent access to In some examples, the secure element provides (or emits) payment information (eg, an account number and/or transaction-specific dynamic security code). In some examples, the secure element may be configured to enable the device to authenticate a user (e.g., fingerprint authentication; passcode authentication; when the device is in an unlocked state, and optionally, after the device is unlocked by providing authentication credentials to the device). detecting a double-press of a hardware button while the device is still on the user's wrist, where the continued presence of the device on the user's wrist indicates that the device is in contact with the user's skin. Provide (or release) payment information in response to receiving an authorization (as determined by periodically checking whether the payment information is available). For example, the device detects a fingerprint on the device's fingerprint sensor (eg, a fingerprint sensor integrated into a button). The device determines whether the fingerprint matches the registered fingerprint. Upon determining that the fingerprint matches an enrolled fingerprint, the secure element provides (or emits) payment information. Upon determining that the fingerprint does not match an enrolled fingerprint, the secure element withholds providing (or releasing) payment information.

Attention now is directed to embodiments of user interfaces (“UIs”) and associated processes implemented on an electronic device, such as portable multifunction device 100, device 300, or device 500.

7A-7S illustrate example users for providing training instructions for registering biometric features on an electronic device (e.g., device 100, device 300, or device 500), according to some examples. Interfaces are shown. User interfaces in these figures are used to illustrate the processes described below, including the processes in FIG. 8.

FIG. 7A illustrates an electronic device 700 (e.g., portable multifunction device 100, device 300, or device 500). In the illustrative example illustrated in FIGS. 7A-7S , electronic device 700 is a smartphone. In other examples, electronic device 1500 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 700 has a display 702, one or more input devices (e.g., a touchscreen, buttons, microphone) of display 1502, and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the electronic device includes one or more biometric sensors (e.g., biometric sensor 703), optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, one or more of the biometric sensors may be described in U.S. Application No. 14/341,860, “Overlapping Pattern Projector,” filed July 14, 2014, which is incorporated herein by reference in its entirety for any purpose. Biometric sensors (e.g., facial recognition sensors), such as those described in Application Publication No. 2016/0025993, and U.S. Application No. 13/810,451, “Scanning Projects and Image Capture Modules For 3D Mapping,” and U.S. Patent No. 9,098,931. am. In some examples, the electronic device includes a depth camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR flood light, a structured light projector, or a combination thereof. The light-emitting device is optionally used to illuminate an object during capture of images by a visible light camera and a depth camera (e.g., an IR camera), and information from the visible light camera and depth camera is used to provide information about different parts of the object captured by the visible light camera. Used to determine the depth map. In some examples, the lighting effects described herein use parallax information from two cameras (e.g., two visible light cameras) for back-facing images and for front-facing images (e.g., selfie images). ) is displayed using depth information from the depth camera combined with image data from the visible light camera. In some examples, when two visible light cameras are used to determine depth information and when a depth camera is used to determine depth information, the same user interface is used to determine which information is used when generating lighting effects. It provides users with a consistent experience even when using different technologies. In some examples, while displaying a camera user interface with one of the lighting effects applied, the device detects selection of a camera switching affordance, maintains display of user interface controls for applying the lighting effect, and displays a camera user interface with one of the lighting effects applied. From the front-facing cameras (e.g., a depth camera and a visible light camera) to the back-facing cameras (e.g., two visible light cameras spaced apart from each other), replacing the display of the field of view with that of the back-facing cameras (or vice versa) ) (or vice versa).

As shown in FIG. 7A , device 700 displays device setup user interface 702 on display 701 . In some examples, device setup user interface 702 is displayed when device 700 is first powered on by a user (e.g., when a factory-sealed device is powered on for the first time). . In some examples, device setup user interface 702 is displayed upon resetting device 700 to factory settings. Phone setup user interface 702 includes one or more prompts 704. In the example of Figure 7A, prompt 704 is plain text that prompts the user to proceed with initial device setup (eg, language selection, authentication measures, etc.). Device setup interface 702 includes one or more affordances, such as a continue affordance 706 and a skip (skp) affordance 708. In some examples, in response to detecting a user input corresponding to activation of skip affordance 708, device 700 optionally displays a main user interface, such as the user interface of FIG. 4A, without setup of one or more features. displays.

As illustrated in FIG. 7B , while displaying the setup interface 702 , the electronic device 700 detects activation (e.g., selection) of the continue affordance 706 . In some examples, the activation is a tap gesture 710 on the contact area 710 at the continue affordance 706. In some examples where display 700 is a touch-sensitive display, activation of the continue affordance is a touch, swipe, or other gesture on the display surface at contact area 710. In some examples where the display 700 is not touch-sensitive, the user input is keyboard input or activation of the continue affordance 706 by a focus selector (e.g., a mouse cursor).

In response to detecting activation of the continue affordance 706, the device displays facial authentication instructions interface 712, as shown in FIG. 7C. In some examples, facial authentication setup interface 712 is displayed in response to ending a previous stage of the device setup user interface process, or in response to selecting a facial authentication enrollment option in the settings user interface. Facial authentication setup interface 712 includes one or more prompts 714, a continue affordance 716, and a later affordance 718. In the example of Figure 7C, prompt 714 is plain text indicating that the user has the option to set up facial authentication instead of a numeric passcode. Facial authentication setup interface 712 also includes a graphical representation of the face (e.g., biometric authentication glyph 720) displayed within a framing element 722. In the example of Figure 7C, framing element 722 is rectangular in shape surrounding biometric authentication glyph 720, such that only the corners of the rectangle are displayed. In some examples, the framing element is optionally a solid rectangle or any other shape (eg, a circle or oval) surrounding glyph 720. In some examples, framing element 722, in combination with additional features described below, helps indicate to the user how to properly position their face relative to biometric sensor 703.

Referring to Figure 7D, device 700 detects activation (e.g., selection) of continue affordance 716. In some examples, the activation is a tap gesture 724 at the continue affordance 716. In some examples where display 701 is a touch-sensitive display, activation of the continue affordance is a touch, swipe, or other gesture on the display surface at contact area 724. In some examples where the display 701 is not touch-sensitive, the user input is keyboard input or activation of the continue affordance 716 by a focus selector (e.g., a mouse cursor).

In response to detecting selection of the continue affordance 716, device 700 displays prompt 726, as illustrated in FIG. 7E (e.g., display of prompt 714 as prompt 726) (replaced with). Additionally, the device replaces the display of the continue affordance 716 with the start affordance 728. Upon selection of the continue affordance 716, device 700 maintains displaying (e.g., continues displaying) glyph 720 and framing element 722.

Referring to Figure 7F, device 700 detects activation (e.g., selection) of launch affordance 728. In some examples, the activation is a tap gesture 730 at the start affordance 7728. Activation of initiation affordance 728 optionally represents a user request to initiate a facial authentication enrollment (e.g., setup) process.

As shown in Figures 7H-7Q, device 700 displays facial authentication instructions interface 732 in response to detecting selection of initiation affordance 728. At the same time, the device displays a training animation (e.g., instructions) that shows the user how to properly position and move his or her face relative to the biometric sensor 703, so that the device 700 will enable the collection of sufficient biometric (e.g., facial imaging) data required for secure (e.g., biometric) authentication. Details of the tutorial interface and training animation are described below.

As illustrated in FIGS. 7G and 7H , device 700 modifies the display of framing element 722 so that there is a single continuous framing element 723 surrounding glyph 720 . As shown in FIG. 7G , device 700 optionally rounds each corner of framing element 722 into a portion of a circle and merges and/or constricts the portions into a continuous shape surrounding glyph 720. Forms a circle (e.g., framing element 723 as shown in Figure 7H).

As shown in FIG. 7H , device 700 simultaneously displays an educational progress meter 734 proximate to and/or surrounding glyph 720 . In the example of FIG. 7H , educational progress meter 734 consists of a set of progress elements (e.g., progress ticks 734a, 734b, 734c) evenly distributed around glyph 720. In the example of Figure 7H, progression ticks 734a, 734b, and 734c are equidistant from and extend radially outward from glyph 720, for example, forming a circle around it. In some examples, these progressive elements are optionally dots, circles, line segments, or any other suitable discrete elements. In some examples, these progression elements are optionally arranged around glyph 720 in a square, rectangular, oval, or any other suitable pattern.

While displaying facial authentication tutorial interface 732 (e.g., a glyph surrounded by framing element 723 and training progress meter 734), device 700 collects the user's facial data, as shown in FIG. 7I. It begins displaying an educational animation illustrating the process of registering. As described in more detail below with reference to FIGS. 7I-7P , device 700 moves glyph 720 with corresponding forward and circular motions of training progress meter 734 to emulate successful facial authentication. ) displays the movement.

At the beginning of the training animation, device 700 overlays an orientation guide 736 on top of the display of glyphs 720. In the example of FIG. 7I , orientation guide 736 is a pair of intersecting curves (e.g., crosshairs) that extend from framing element 723 and glyph 720 so that they are out of the plane of the display. Make it appear to bulge outward (e.g., in the simulated z-direction). In some examples, in combination with circular framing element 723, the arcs of orientation guide 736 give an otherwise two-dimensional glyph 720 a three-dimensional appearance, as if it were positioned on the surface of a sphere. Grants. Typically, the training animation maintains the orientation guide 736 at a fixed position relative to the center of the glyph 720, such that the orientation guide appears to be rotated or tilted with (e.g., in the same direction as) the facial expression. . In some examples, glyph 720 itself is a three-dimensional representation of a face, such as a three-dimensional line drawing with lines at simulated z-height. In such examples, orientation guide 736 is optionally omitted. In this case, when the facial representation is tilted in different directions, the lines at different z-heights appear to move relative to each other based on a simulated parallax effect, giving the appearance of three-dimensional movement.

Device 700 configures facial authentication guidelines by displaying movement (e.g., rotation and/or tilt) of glyph 720 and orientation guide 736 in a first direction (e.g., up, down, left, or right). Start the training animation on interface 732. In the example of FIG. 7I , glyph 720 and overlaid orientation guide 736 are tilted to the right with respect to a vertical axis extending from the plane of display 700. Tilting glyph 720 in this manner can optionally reveal a portion of the simulated face (e.g., the left side of the face) and hide another portion of the simulated face (e.g., the right side of the face), thereby tilting or rotating the glyph 720 in a particular direction. It additionally gives the appearance of a three-dimensional head.

As illustrated in FIG. 7I , device 700 changes the appearance of glyphs 720 (and/or orientation guide 736) as they are tilted toward a subset of progressive elements. In particular, progressive elements within meter portion 738 optionally stretch and/or change color from their initial state when the face graphic is tilted toward them. This stretching and/or color change optionally becomes more pronounced as the glyphs 720 are further tilted in their direction. In some examples, progressive elements within meter portion 738 are optionally or otherwise changed in appearance. For example, additionally and/or alternatively, the line thickness, number, or pattern of progressive elements is optionally varied. Changing the appearance of the progressive elements in this way indicates to the user that the biometric sensor 703 is configured to capture image data of the corresponding portion of the face when oriented in that direction. While displaying the training animation, device 700 maintains display progress elements (e.g., elements of meter portion 740) whose facial graphics have not yet been tilted in an initial state. In the example of Figure 7I, device 700 displays progress elements in their initial state as unfilled outlines.

In some examples, device 700 then rotates glyph 720 about a second axis parallel to the plane of the display, causing the simulated face to appear tilted upward or downward. In the example of Figure 7J, glyph 720 is tilted upward from its position in Figure 7I, so that the simulated face appears to be facing upward and to the right. While rotating the glyph 720 in this way, the device 700 changes the appearance of the corresponding meter portion 740, which was previously in its initial state, as shown in FIG. 7I. The device changes the appearance of the meter portion 740 in the same manner as described above with respect to FIG. 7I (e.g., by stretching the progress elements within this portion of the training progress meter and/or by changing its color). At the same time, the device 700 transitions the progress elements in the meter portion 738 to a second state, corresponding to the portion of the facial expression that was registered in FIG. 7I. Progress elements in the success state (e.g., progress elements in the meter portion 738) are different in shape, color, line width, etc. from progress elements in the initial state. In the example of Figure 7I, the progress elements in the success state are displayed at the same size and width as the progress elements in the initial state (e.g., those in meter portion 742), but the facial expression is already oriented in that direction. darkens and/or fills to indicate that

Figure 7K illustrates further tilting and/or rotation of glyph 720 and orientation guide 736 until the simulated face appears to be looking upward. As described above, device 700 changes the appearance of glyphs 720 (e.g., stretches and/or changes color) as they are oriented in the direction of advancing elements within meter portion 742. At the same time, device 700 transitions the progressing elements within meter portion 740 to a success state since the simulated face was previously there, but is no longer oriented in their direction. Progressive elements within meter portion 738 remain successful. Typically, the appearance of progress elements that have transitioned to a successful state is not subsequently modified. In this manner, device 700 changes the appearance of elements within progress meter 734 in response to displaying movement of glyph 720.

In some examples, during the training animation, device 700 optionally displays a fully circular (e.g., clockwise, counterclockwise) motion of the simulated face (e.g., glyph 720 in FIG. 7I ). Continue displaying the rotation and/or tilt of glyph 720 (until it returns to the right-tilted orientation shown). Likewise, device 700 progressively transitions glyph 720 to a success state as it rotates past the elements of training progress meter 734, as described above. After displaying a full rotation of the simulated face, the device displays all progress elements of the training progress meter 734 with a success status, as shown in FIG. 7L. In some examples, device 700 stops displaying orientation guide 736 after a full rotation has been displayed and returns glyph 720 to its initial position.

After all progress elements of training progress meter 724 have transitioned to the success state, device 700 places progress meter 734 (e.g., the progress meter itself) in the authentication success state, e.g., a solid circle surrounding glyph 720. transfers to Displaying the progress meter 724 with an authentication success status optionally indicates successful facial authentication setup. 7L-7O, device 700 provides a training progress meter by shortening each progress tick and merging them together into a continuous solid circle (e.g., success status meter 744) surrounding glyph 720. The display of the individual progress scale at 734 is transitioned to the authentication success state. In the examples of FIGS. 7O and 7P , the circle extends the glyph 720 until the radius of the success status meter 744 is substantially equal to the radius of the framing element 723 (e.g., as shown in FIG. 7P ). shrinks around the circumference.

As shown in FIGS. 7I-7Q , the facial authentication tutorial interface 732 also includes a starting affordance 746 that is optionally displayed throughout the facial authentication tutorial. In some examples, start affordance 746 is enabled for activation after the training animation is completed (e.g., after device 700 displays training progress meter 734 with an authentication success state in FIG. 7Q). In other examples, start affordance 746 is enabled for activation at any time during the display of the facial authentication tutorial animation prior to completion of the training animation.

Referring now to Figure 7Q, device 700 detects activation (e.g., selection) of launch affordance 746. In some examples, the activation is user input in response to a request to begin facial authentication setup. In response to detecting activation of launch affordance 746, device 700 may display a glyph 720, including an image of the user's face captured by biometric sensor 703, as shown in FIG. 7R. 750). In some examples, image 748 is a live preview of the field of view of biometric sensor 703. In other examples, image 750 is a wire-frame representation of the user's face based on the movement of the user's face in the field of view of the optical sensors. Accordingly, image 750 changes (eg, is continuously updated) as the position and orientation of the user's face relative to the biometric sensor changes.

As shown in FIG. 7R , device 700 also displays positioning element 752 around user image 750 . In some examples, positioning element 752 optionally has similar or identical visual characteristics as framing element 722 that was initially positioned surrounding glyph 720 in Figures 7C-7F. In some examples, a positioning element is displayed to highlight a predetermined portion of the display of the electronic device, indicating where the user should position his or her face relative to the biometric sensors for subsequent facial authentication setup. . In some examples, the positioning element has a shape (eg, square) that at least partially demarcates the predetermined display portion from other portions of the display. Device 700 may also include a prompt 754, i.e., prompting the user to move his/or her face relative to the light sensors so that user image 750 is visible inside positioning element 750. Display text.

Referring now to FIG. 7S , in response to detecting that user image 750 is properly positioned within positioning element 750 (e.g., the user's face is properly aligned with biometric sensor 703), the device 700 displays a facial authentication registration interface 756. In the example of Figure 7S, facial authentication registration interface 756 includes a progress meter 758 and a user image 760. In some examples, registration interface 756 includes an orientation guide 762, i.e., a set of curves (e.g., crosshairs) that appear to extend outside the plane of the display. Progress meter 758 optionally has some or all of the features of training progress indicator 734 displayed during the facial authentication tutorial animation. In the example of FIG. 7S , progress meter 758 also includes a set of progress elements distributed around user 750 (e.g., progress ticks 758a, 758b, 758c). Additional explanation of the alignment of the user's face with respect to the light sensors can be found below with reference to FIGS. 9A-9AE and 11A-11O.

8A-8C are flow diagrams illustrating a method for providing training instructions for registering biometric characteristics on an electronic device, according to some examples. Method 800 includes a device having a display, one or more input devices (e.g., a touchscreen, microphone, camera), and a wireless communication radio (e.g., a Bluetooth connection, a WiFi connection, a mobile broadband connection such as a 4G LTE connection). For example, 100, 300, 500, 700). In some examples, the display is a touch-sensitive display. In some examples, the display is not a touch-sensitive display. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the device includes one or more biometric sensors, optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device, such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors. Some operations of method 2000 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 800 provides an intuitive method for providing training instructions for registering biometric characteristics on an electronic device. This method reduces the user's cognitive burden of registering biometric features on the device, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to register biometric features faster and more efficiently saves power and increases time between battery charges.

The device displays 802 a first user interface (e.g., facial authentication setup interface 712) on the display. While displaying the first user interface, the device detects (806) the occurrence of a condition corresponding to the introduction of a biometric registration process for registering biometric features (e.g., face, fingers, eyes, voice, etc.). In some examples, the occurrence of a condition is an input that corresponds to a request to initiate a biometric registration process, such as ending a previous stage of a device setup user interface process or selecting a biometric registration option in the settings user interface. In some examples, biometric features are used for authentication at the device.

In response to detecting the occurrence of a condition corresponding to introduction of a biometric registration process, the device displays 808 a biometric registration introduction interface (e.g., facial authentication guideline interface 732). Displaying the biometric registration introduction interface may include a representation of a simulation of the biometric features (e.g., 720, a video of the biometric features or an animation of the biometric features, such as an animated face/head) and a simulated progress indicator (e.g., 720). and concurrently displaying (810) a training progress meter (734), a displayed element indicating progress of enrollment. In some examples, the simulated progress indicator is located proximate to the simulation's representation of the biometric characteristic. In some examples, the simulated progression indicator may be comprised of a plurality of progression elements (e.g., progression elements 734a, 734b, 734c) selectively distributed around the simulation of the biometric feature, e.g., outward from the simulation of the biometric feature. and a set of graduation marks that extend (eg, extend radially) and form an oval shape, such as a circle.

In some examples, the representation of the simulation of the biometric feature is a representation of the simulation of at least a portion of the face (e.g., 720) (812). In some examples, the representation is a representation of a simulation of a portion of a face. In some examples, the representation is a representation of a simulation of the entire face. In some examples, the simulation of biometric features is a representation of a generic face, such as a line drawing including the eyes, nose, and mouth. In some examples, the simulation's representation of the biometric feature is a three-dimensional representation (814). For example, a representation of a simulation of a biometric feature is a three-dimensional rendered object. Alternatively, the training animation is optionally a 2D animation instead.

In some examples, the representation of the simulation of the biometric feature is a line drawing (e.g., a 3D representation of 720) with lines at different simulated z-heights (816). For example, when a line drawing of a face is tilted in different directions, the lines at different simulated z-heights appear to move relative to each other based on the simulated parallax effect. In some examples, the biometric registration introduction interface is overlaid on a representation of a simulated biometric feature (e.g., 720) and tilted in different directions as the representation of the simulated biometric feature is tilted in different directions. and an orientation guide (e.g., orientation guide 736, a curve that bends backwards in the simulated z direction, as described in more detail below with reference to method 1200) (820).

While displaying the biometric registration introductory interface, the device may move (e.g., tilt and/or rotate) a representation of the simulation of the biometric feature and incrementally advance the progress indicator (e.g., progress elements of the progress indicator may display the biometric feature). a training animation that includes displaying a training animation (e.g., advancing training progress indicator 734 and 720 shown in FIGS. 7H-7L ) (changing color and/or shape in response to display of movement of the representation of the simulation) movement) is displayed (824). Displaying a training animation comprising movement of a representation of a simulation of biometric features and incremental advancement of a simulated progress indicator may first be performed as appropriate for the subsequent biometric registration process (e.g., method 1200 and/or 1400). Requires the device to display biometric registration interfaces (e.g., 756) during this process by illustrating user inputs and thereby helping the user intuitively recognize how to quickly and appropriately register their biometric features. Reduces the duration of presence and reduces the number of user inputs performed on those interfaces. Reducing the number and time of inputs required to perform a registration operation improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). It improves and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the device displays a movement that tilts the simulation of the biometric feature relative to the plane of the device's display (826). For example, movement of a representation of a simulation includes rotation of the representation of the simulation along an axis perpendicular to the plane of the display of the device. In some examples, the biometric sensor and/or the tilt of the sensor's field of view defines a plane of the display. In another example, the device rotates a representation of the simulation of the biometric feature about a first axis (e.g., an axis perpendicular to the display 700) and a second axis different from the first axis (e.g., an axis perpendicular to the display 700). 828). In some examples, the first axis is a vertical axis, allowing movement of the representation from left to right and/or right to left. In some examples, the first axis is perpendicular to the second axis. For example, the second axis is optionally a horizontal axis, allowing movement of the representation downward and/or upward. In some examples, the first axis is any axis other than the axis perpendicular to the display of the device (eg, the representation rotates in any direction), and the second axis is an axis perpendicular to the display of the device. In this example, the simulated head optionally moves in a circular pattern about a second axis. Displaying a movement that tilts the simulation of the biometric feature relative to the plane of the display first illustrates the appropriate user inputs required for the subsequent biometric registration process (e.g., methods 1200 and/or 1400), thereby allowing the user to Helps intuitively recognize how to quickly and appropriately register their biometric features, thereby reducing the duration the device needs to display biometric registration interfaces (e.g., 756) during this process and such interfaces. Reduces the number of user inputs performed in . Reducing the number and time of inputs required to perform a registration operation improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). It improves and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, when displaying an instructional animation, the device is selectively configured to reveal a first portion of the representation (e.g., a first side of 720) and not to reveal a second portion of the representation (e.g., a second different side of 720). Display a representation of the simulation of the biometric feature at the first location (830). Subsequently, the device optionally displays a representation of the simulation of the biometric feature at a second location different from the first location so as to reveal a second portion of the representation and not to reveal a first portion of the representation. In an example where the biometric feature is a face, the simulated face is optionally tilted in a first direction to reveal a first portion of the simulated face and then tilted in a second direction to reveal a second portion of the simulated face. Lose. Displaying simulated biometric features in a first orientation and subsequently displaying simulated biometric features in a second different orientation may first be necessary for a subsequent biometric registration process (e.g., methods 1200 and/or 1400). Instantiating appropriate user inputs and thereby helping users intuitively recognize how to quickly and appropriately register their biometric features, such that the device may display biometric registration interfaces (e.g., 756) during this process. Reduces the duration required and the number of user inputs performed in such interfaces. Reducing the number and time of inputs required to perform a registration operation improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). It improves and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the device displays a simulated progress indicator (e.g., 734) surrounding a representation of the simulation of the biometric feature (e.g., 720). For example, the simulated progress indicator is displayed such that the simulated progress indicator surrounds (or substantially surrounds) a portion or all of the simulation's representation of the biometric feature. In some examples, the simulated progress indicator centers on a representation of the user's biometric characteristics. In some examples, displaying a simulated progress indicator may include a plurality of progress elements (e.g., dots, circles, or line segments, e.g., and displaying (832) progress ticks (734a, 734b, 734c). In some examples, progressive elements are equidistant from the representation and/or extend radially outward from the representation. In some examples, the progression elements are arranged in a circular, square, rectangular, or oval pattern.

In some examples, when displaying incremental advancement of a simulated progress indicator, the device transitions 834 one or more progress elements of the plurality of progress elements from a first state to a second state that is different from the first state. For example, in a first state, the progression elements are optionally of a first color and/or a first length, and in a second state, the progression elements are optionally of a second color and/or a second color different from the first color. It is of a second length different from the first length. In some examples, progression elements are selectively or otherwise changed in appearance. For example, progression elements are selectively changed in line thickness, number, and pattern. Changing the display of portions of the simulated progress indicator allows the user to recognize that changes in the orientation of the simulated biometric features illustrated in the training animation are required to properly register his/or her biometric features. Let's do it. This first helps illustrate the appropriate user inputs needed for the subsequent biometric registration process (e.g., methods 1200 and/or 1400) so that the device may display biometric registration interfaces (e.g., 756) during this process. Reduces the duration required and the number of user inputs performed in such interfaces. Reducing the number and time of inputs required to perform a registration operation improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). It improves and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the representation of the simulation of the biometric feature is a line drawing that includes at least a portion (e.g., some or all) of the glyphs (e.g., success status progress meter 744) used to indicate successful biometric registration. It is (836). In some examples, one or more progress elements of the simulated progress indicator are all updated to a second state (e.g., green and elongated, or the state of meter portion 738 in FIG. 7J) and are not subsequently modified. In some examples, when each of the progress elements has been updated to the second state, the simulated progress indicator transitions to a success state (e.g., success status progress meter 744). In some examples, transitioning the simulated progress indicator to a success state includes transitioning the simulated progress indicator to a solid circle surrounding a representation of the simulation of the biometric feature.

After displaying at least a portion of the training animation, the device detects the occurrence of a condition corresponding to initiation of the biometric registration process (838). In some examples, the condition corresponding to initiating the biometric registration process includes selection of an affordance for initiating the biometric registration process (840). For example, a condition may be an input (e.g., in a contact area 748) that corresponds to a request to “start registration,” such as a “start registration” or a tap on a “next” affordance (e.g., start affordance 746). user input), and optionally following alignment of the user's biometric characteristics with one or more biometric sensors. A more detailed description of the biometric registration process is described in greater detail herein with reference to method 900. In some examples, the electronic device provides tactile output and/or auditory output in response to selection of an affordance.

In response to detecting the occurrence of a condition corresponding to initiation of a biometric enrollment process (842), the device may display a simulation of the biometric features at the biometric enrollment introductory interface (e.g., facial authentication guide interface 732). A representation of the user's biometric characteristics as determined by one or more biometric sensors of the device (e.g., user image 750, the user's face, the user's fingers, the user's eyes, The user's hand) is displayed (844). In some examples, the device optionally displays an enrollment progress user interface (e.g., 756) after a representation of the user's biometric characteristics (e.g., 750, 760) is aligned with one or more biometric sensors (e.g., 703). do.

In some examples, the representation is a representation of a portion of the user's face (e.g., a portion of user image 750) (846). In some examples, the expression is a representation of the user's entire face. In some examples, a representation of a user's biometric characteristics is a representation of a user that is specific to the user. For example, a representation of a user may be a wire frame or images of the user's face that match the contours of the user's face.

In some examples, the biometric registration user interface includes an orientation guide (e.g., orientation guide 736, orientation guide 762) overlaid on a representation of biometric features (e.g., user image 750) (848). ). The orientation guide optionally tilts as the biometric feature tilts in different directions. Displaying an orientation guide that moves with the user's biometric features provides the user with feedback regarding the orientation of his or her biometric features relative to the device's biometric sensors in three-dimensional space, Allows a user to more quickly place his or her biometric features in appropriate orientations during a subsequent registration process (e.g., method 1200 and/or method 1400). Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, a representation of a user's biometric characteristics (e.g., 750) is based (850) on image data captured by one or more cameras (e.g., 703) of the electronic device. For example, the representation of a user's biometric features may optionally be based on sequential images of the user captured by one or more cameras (e.g., 703), or on movement of the user's features in the field of view of one or more cameras. This is a wireframe. In some examples, the representation of a biometric feature changes 852 as the orientation of the biometric feature relative to one or more biometric sensors changes. Updating the orientation of the displayed representation of the biometric features provides the user with feedback regarding the orientation of his or her biometric features relative to the device's biometric sensors, allowing the user to perform the subsequent registration process (e.g., method during 1200 and/or method 1400) to more quickly place his or her biometric features in appropriate orientations. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In response to detecting the occurrence of a condition corresponding to the initiation of the biometric registration process, the device may also display a simulated progress indicator (e.g., a biometric indicator, such as a plurality of progress elements distributed around a representation of the user's biometric characteristics). Display (854) a progress indicator (e.g., 756) corresponding to a progress indicator (e.g., 756) with some or all of the features of the progress indicator displayed surrounding the simulation of the feature. In some examples, displaying a progress indicator includes maintaining display of a simulated progress indicator (856). For example, the simulated progress indicator returns to an initial state (e.g., the state of progress elements 734a, 734b, 734c in Figure 7H) and is used to indicate incremental registration progress of the simulated biometric feature. It is used to indicate the user's incremental registration progress in the same or similar manner as before. Displaying an enrollment progress indicator corresponding to (e.g., similar to) a simulated progress indicator allows the user to: to quickly associate appropriate inputs required during subsequent registration processes (e.g., methods 1200 and/or 1400). This in turn allows the user to complete the enrollment process more quickly, reducing the duration the device needs to display biometric enrollment interfaces (e.g., 756) during this process and user input performed in those interfaces. reduce their number. Reducing the number and time of inputs required to perform a registration operation improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). It improves and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the device displays 858 a progress indicator (e.g., 758) surrounding a representation of the user's biometric characteristics (e.g., 760). For example, a progress indicator optionally has some or all of its features displayed surrounding a simulation of the biometric features. These features optionally include a plurality of progressive elements (eg, 758a, 758b, 758c) distributed around a representation of the user's biometric features. For example, the progress indicator is displayed so that the progress indicator surrounds (or substantially surrounds) a portion or all of the representation of the user's biometric characteristics. In some examples, the progress indicator centers on a representation of the user's biometric characteristics.

In some examples, in response to detecting the occurrence of a condition corresponding to initiation of a biometric registration process, the device displays 860 a location element (e.g., location element 752) on the display of the electronic device. . In some examples, the positioning element is displayed to highlight a predetermined portion of the electronic device's display (eg, 756, 758). In some examples, the positioning element indicates where the user should position a representation of the user's biometric feature (e.g., 750) for subsequent biometric feature registration. In some examples, the positioning element is an object that visually at least partially demarcates the first and second portions of the display (eg, display portion 756 and display portion 758). The positioning element is, in some examples, shaped like a square and is optionally segmented. Displaying a positioning element framing a particular portion of the digital viewfinder allows the user to determine the location and/or orientation of his or her biometric features within the field of view of the biometric sensor during a subsequent biometric registration process (e.g., method (1200 and/or 1400)), allowing the user to more quickly place his or her biometric features in the appropriate orientation. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

Note that the details of the processes described above with respect to method 800 (e.g., FIGS. 8A-8C) are also applicable in a similar manner to the methods described below. For example, method 800 optionally includes one or more of the features of various methods described below with reference to methods 1000, 1200, 1400, 1600, 1800, 2000, 2200, 2500, 2700. do. As another example, the orientation guide described in method 1200 may be applied in conjunction with a training animation displayed on a facial authentication tutorial interface (e.g., 732). For another example, one or more aspects of biometric registration described in method 1200 may be applied in connection with an registration interface (e.g., 756). For another example, one or more aspects of the hints described in method 1400 may be applied to the display of a facial authentication guideline interface (e.g., 732).

Operations in the information processing methods described above optionally utilize one or more functional modules within an information processing device, such as general-purpose processors (e.g., as described with respect to FIGS. 1A, 3, and 5A) or application-specific chips. It is implemented by running it. Additionally, the operations described above with reference to FIGS. 8A-8C are optionally implemented by the components shown in FIGS. 1A and 1B. For example, display operation 802, detection operation 806, display operation 810, display operation 824, detection operation 838, display operation 844, and display operation 854 may optionally be performed. , is implemented by an event classifier 170, an event recognizer 180, and an event handler 190. Event monitor 171 in event sorter 170 detects contact on touch-sensitive surface 604, and event dispatcher module 174 passes event information to application 136-1. The respective event recognizer 180 of application 136-1 compares the event information to the respective event definition 186 and determines that the first contact at the first location on the touch-sensitive surface corresponds to the selection of an object on the user interface. Determines whether it corresponds to the same predefined event or subevent. When an individual predefined event or sub-event is detected, event recognizer 180 activates an event handler 190 associated with detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some examples, event handler 190 accesses the respective GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes may be implemented based on the components shown in FIGS. 1A and 1B.

9A-9E illustrate training instructions for registering biometric features on an electronic device (e.g., device 100, device 300, device 500, or device 700), according to some examples. Example user interfaces are illustrated. User interfaces in these figures are used to illustrate the processes described below, including the processes in FIG. 10.

FIG. 9A illustrates an electronic device 900 (e.g., portable multifunction device 100, device 300, device 500, or device 700). In the illustrative example illustrated in FIGS. 9A-9E , electronic device 900 is a smartphone. In other examples, electronic device 900 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 900 has a display 901, one or more input devices (eg, a touchscreen, buttons, microphone of display 901), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the electronic device includes one or more biometric sensors (e.g., biometric sensor 903), optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, one or more biometric sensors 903 are one or more biometric sensors 703. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

As illustrated in FIG. 9A , device 900 displays facial authentication introduction interface 905 . In some examples, facial authentication introduction interface 905 is similar to facial authentication tutorial interface 732 described above with respect to FIG. 7S. By way of example, facial authentication introduction interface 905 includes a face graphic 902, which is optionally the same as or similar to glyph 720 described above with respect to facial authentication instructions interface 732. Additionally or alternatively, device 900 also optionally displays a success status training progress meter 907, which is optionally the same as or similar to success status training progress meter 744 in FIGS. 7P and 7Q. do. The facial authentication introduction interface 905 also includes a start button 904 (eg, start affordance). As shown in Figure 9A, device 900 detects activation (e.g., selection) of initiation affordance 904. For example, the activation is optionally a user input at the contact area 906 on the starting affordance 904. This user input will, in some situations, correspond to a request to initiate facial authentication setup (eg, initiate face registration).

In some examples, in response to detecting user selection of start button 904, device 900 displays face alignment interface 908 as shown in FIG. 9B. Facial alignment interface 908 includes positioning elements 910, which in some examples are framing circles or brackets that represent alignment boundaries. In some examples, positioning element 910 identifies an inner display portion 912 and an outer display portion 912. In some examples, the electronic device determines that the user's biometric characteristics are properly aligned when substantially positioned within the internal display portion 912 in a predetermined manner. In some examples, positioning element 910 partitions inner display portion 912 from outer display portion 914. Generally, when the user's face is positioned relative to the biometric sensor 903 such that a portion of the user's image is visible in the external display portion 914, the user's face is, in some situations, properly aligned with the cameras. It won't work. As such, the face alignment interface 908 also provides a text prompt 916 instructing the user to position his or her face inside the positioning element 910 (e.g., within the internal display portion 812). ) includes.

Referring to FIG. 9C , in some examples, during the alignment process, the user positions the electronic device 900 substantially in front of the user's face 917. In some examples, the user holds device 900 at approximately the same height as his or her face, such that the face is in the field of view of biometric sensor 903.

As illustrated in Figure 9D, once the user has initiated the alignment process, the device displays a facial alignment interface 908 (recall that the user optionally initiates the registration process by activating affordance 904). . Facial alignment interface 908 includes a digital viewfinder that presents a preview of image data captured by biometric sensor 903. In some examples, the preview of the image data is continuously updated (e.g., as the field of view of these cameras changes (e.g., if device 900 is moved or the user moves closer to/further from the cameras)). , which changes over time) is a live preview. The digital viewfinder includes an image of the user's face 918 as well as a positioning element 910 overlaid on the cameras' field of view. As described above, positioning element 910 partitions inner display portion 912 from surrounding outer display portion 914. To provide additional visual separation between the inner display portion 912 (where the user facial image 918 will be positioned) and the outer display portion 914, device 900 can be configured to: Visually obscure (e.g., shade, darken, or blur) the external display portion 914.

Generally, proper registration of a user's facial features for authentication requires that the user's face be positioned in a predetermined manner and/or be within a predetermined range of distances from the cameras of device 900. In some examples, alignment of the user's face with the cameras of device 900 requires the user to be neither too close nor too far from the device. Accordingly, if the electronic device 900 determines that the user's face is too close or too far, the electronic device instructs the user to position their face at an acceptable distance (e.g., 20 to 40 mm) from the device 900. A text prompt 920 indicating this is displayed within the face alignment interface 908. In the example of FIG. 9D , device 900 determines that the user's face is too far from the cameras on the device (e.g., user face image 918 is within positioning element 910, but internal display portion 912 is not substantially filled) is detected. In some examples, the electronic device prompts the user to move his or her face closer to the device. In some examples, the device may have one or more outputs, such as an audio output 922 (e.g., a series of beeps or other audio output) and a tactile output 924 (e.g., a series of vibrations or other tactile sensations). output) to notify the user of improper alignment. In some examples, audio output 922 and/or tactile output 924 has a magnitude and repetition rate (e.g., frequency) that varies based on the distance between device 900 and the user's face. For example, the output frequency and/or magnitude is optionally increased as the user's face moves closer to an acceptable range of distances from the device (eg, 20 to 40 mm). Conversely, the frequency and/or magnitude of the output is optionally reduced as the user's face moves further away from the acceptable range of distances. In this case, device 900 adjusts the frequency and/or amplitude of audio output 922 and/or tactile output 924 as it detects changes in the distance between the user's face and biometric sensor 903. Continually change (e.g., update) the. In some examples, device 900 provides these outputs as long as the user's face is outside the acceptable range of distances from the device. In some examples, audio output 922 and tactile output 924 are accompanied by corresponding visual output on display 700. These ongoing audio, tactile and/or visual outputs optionally provide intuitive hints to the user about how to accurately align his or her face with the cameras, reducing the time needed to perform successful face alignment. reduce.

9E illustrates the face alignment interface 908 when the user's face is positioned closer to the device 900 (e.g., a significant portion of the user's facial image 918 is within the external display portion 914). In this case, alignment interface 908 also includes text prompt 920 instructing the user to position his or her face at an acceptable distance from device 900. In some examples, the electronic device instructs the user to move his or her face closer to the device. As described above with respect to FIG. 9D , device 900 may optionally provide ongoing audio output 922 and/or tactile output in response to detecting that the user's face is too close to the camera(s). 924). In particular, device 900 changes the frequency and/or magnitude of these outputs as it detects changes in the distance between the user's face and the cameras.

FIG. 9F illustrates the face alignment interface 908 when the user's face is positioned at an acceptable distance from the device 900 but is out of frame (e.g., too far to the right or left). For example, face 918 is optionally positioned such that a significant portion of face 918 lies outside of positioning element 910 within external display portion 914. In this case, device 900 optionally instructs the user to position his or her face within positioning element 910 (e.g., such that user image 918 is displayed within internal display area 912). A text prompt 926 is displayed on the alignment interface 908.

9G-9L , in some examples, electronic device 900 displays face alignment interface 908 in response to determining that the user's face is positioned outside a predetermined range of angles relative to the electronic device. . As shown in FIG. 9G , the electronic device 900 is positioned at a low angle relative to the electronic device such that the electronic device cannot properly acquire (e.g., capture biometric data). is aligned with the user's chin). Referring to FIG. 9H , in response to determining that the electronic device 900 is outside the range of predetermined angles, the electronic device 900 may display a facial alignment interface, such as an internal display portion 912 and an external display portion 914. Blur at least part of 908. In some examples, the electronic device instructs the user to position his or her face within positioning element 910 (e.g., such that user image 918 is displayed within internal display area 912 and at an appropriate angle). An additional prompt (986) is output. 9I and 9K, the user raises device 900 until the electronic device is within a predetermined range of angles. As the user elevates the electronic device, referring to FIGS. 9J and 9K, the electronic device 900 gradually reduces the blur of displayed elements. In this way, the electronic device indicates to the user that the angle of the electronic device relative to the user is approaching an acceptable range of angles. In some examples, the electronic device is so high relative to the user that the electronic device is not within a predetermined range of angles. Similar to the described example, the electronic device optionally reduces or increases blurring of displayed objects as the electronic device is moved relative to the user.

In some examples, if the device detects that an alignment error persists for a predetermined amount of time, device 900 optionally displays accessibility option affordance 928 on facial alignment interface 908, as shown in FIG. 9G. ) is displayed. For example, device 900 may optionally align the user's face at an acceptable distance from the device and/or within a positioning element at a predetermined time after it begins aligning (e.g., after start button 904 is selected). If it does not detect, it displays the accessibility option affordance 928. In some examples, the predetermined time is optionally 10 seconds, 15 seconds, 30 seconds, or any other suitable time. Similarly, device 900 optionally displays an accessibility option affordance after a certain number of failed registration attempts. As discussed in more detail below, device 900 may optionally display additional options or hints and/or initiate alternative facial registration processes in response to detecting selection of accessibility option affordance 928. do. In some examples, activation of accessibility option affordance 928 allows a user to proceed with biometric registration without first correcting alignment errors.

In general, the quality of facial feature registration for the facial authentication methods described herein depends at least in part on the lighting conditions under which the user's facial data is captured. For example, strong backlighting or direct exposure of the user's face will, in some situations, adversely affect the quality of registration. Referring now to FIG. 9H, in response to detecting adverse lighting conditions, device 900 optionally displays a text prompt 930 on alignment interface 908 indicating adverse lighting to the user. Text prompt 930 is optionally accompanied by audio, visual and/or tactile output 932. Output 932 is optionally the same as output 922 and/or 924 described with respect to alignment errors discussed above. In some examples, the outputs are error-specific; Accordingly, output 932 may optionally be a different audio, visual, and/or tactile output than outputs 922 and 924.

In general, the quality of facial feature registration also depends in part on the angle at which the user's face is oriented relative to one or more cameras of device 900 (eg, biometric sensor 903). In particular, the one or more optical sensors of device 900 should be capable of capturing image data of the user's face at a specific angle or within a predetermined range of angles. Even if the user's face is within the acceptable range of distances described above, facial authentication registration may be adversely affected if device 900 is positioned high above or too far below the user's face. Accordingly, in some examples, device 900 requires the user's face to be positioned within a predetermined range of angles relative to one or more of its cameras when detecting successful alignment conditions.

In some examples, device 900 may display an image in the digital viewfinder of alignment interface 808 in response to detecting that the user's face is outside this predetermined range of angles with respect to biometric sensor 903. Blur the data. In some examples, the amount of blur optionally depends on the difference between the detected angle of elevation of the user's face relative to the camera and one or more threshold angles bordering a predetermined angle range. For example, device 900 blurs the preview image to a greater extent the higher or lower device 900 is positioned relative to the user's face. When device 900 detects a change in elevation angle that brings its cameras into closer alignment with the user's face, it optionally reduces the amount of blur as the elevation angle changes (e.g., with a continuous gradient). . In some examples, the preview image is not blurred if the elevation angle between device 900 and the user's face is actively changing (eg, the user is moving device 900 relative to his or her face). Blur may optionally indicate that the angle between the user's face and one or more of its cameras has been outside a predetermined angle range for a set period of time (e.g., 1 second, 2 seconds, 5 seconds, or any suitable period of time) on the device 900. ) is delayed until the decision is made. In some examples, only a portion of the preview image (e.g., external display portion 914) is blurred, while in other examples the entire preview image is selectively blurred. Blurring the preview image in this way optionally prompts the user to more quickly position device 900 at a desired angle relative to his or her face, reducing the time spent during the alignment process. In some examples, device 900 optionally issues a tactile and/or output to inform the user that his or her face is positioned at an appropriate angle relative to biometric sensor 903.

In Figure 9N, the user's face is properly positioned relative to the biometric sensor 903. In this case, face 918 is displayed substantially within alignment element 910 and internal display portion 912. As shown in FIG. 9N , face 918 also occupies a significant portion of internal display portion 912 , indicating that the user's face is within a critical range of distances from device 900 . In response to detecting a face that meets the alignment criteria described above, device 900 issues audio output 934 and tactile output 936 to indicate successful alignment of the user's face with the cameras. In general, outputs 934 and 936 are different from outputs 922, 924 and 932 issued in response to detecting alignment errors. In some examples, device 900 captures and stores one or more images of the user's face upon successful alignment with the cameras.

In some examples, after detecting successful alignment, device 900 visually highlights inner display portion 912 where face 918 is displayed. In the example of FIG. 9P , device 900 displays an outer portion of the digital viewfinder preview while continuing to display a portion of the digital viewfinder preview on inner display portion 914 (e.g., inside positioning element 910). The external display portion 914 is further obscured by further blurring or blackening the image in . In some examples, device 900 further visually emphasizes the content of internal display portion 912 by enlarging or zooming in on an image within internal display portion 912.

In some examples, the device places additional emphasis on the internal display portion 912 by changing the appearance of the positioning element 910. In particular, device 900 may optionally “round” the corners of alignment elements 910 as shown in FIG. 9P and/or round corners of alignment elements 910 to face 918 as shown in FIG. 9Q. Change the appearance of the alignment element by merging it into a circular positioning element 941 that surrounds it.

Referring now to the example of Figure 9R, in response to detecting that the user's face is oriented such that the alignment criteria described above are met, device 900 may display face registration interface 938 (e.g., alignment interface 908 ) by replacing the display of ) with the face registration interface 938) to initiate the facial authentication registration process. In some examples, facial registration interface 938 has similar or identical visual characteristics to facial authentication registration interface 756 described above with respect to FIG. 7S or registration interface 1104 described below with respect to FIG. 11A. In the example of FIG. 9R , face registration interface 938 includes a user facial image 939 displayed within positioning element 941 . In the example of Figure 9R, user facial image 939 is a live preview of image data captured by biometric sensor 903. Facial enrollment interface 938 also optionally includes an enrollment progress meter 940 surrounding user facial image 939 and positioning element 941. As described above with respect to FIGS. 7S and 11A-11H, the enrollment progress meter 940 includes progress elements extending radially outward from the user facial image 939 and, in some examples, surrounding it in a circular pattern. (e.g., 940a, 940b, and 940c). Face registration interface 938 optionally includes an orientation guide 942. In some examples, the orientation guide includes a set of curves (e.g., crosshairs) that appear to extend out of the plane of the display 901 in a virtual z-dimension, intersecting above the center of the user's facial image 939. In some examples, the orientation guide provides a sense of the three-dimensional orientation of the user's face even though facial image 939 is two-dimensional. In this case, orientation guide 942 assists the user in the facial registration process by making the rotation and/or tilt of the user's head relative to device 900 more visually apparent. The face registration interface 938 also includes a text prompt 944 that optionally instructs the user to begin tilting their head, e.g., in a circular motion, to perform registration.

Generally, once the enrollment process is initiated, the quality of enrollment is reduced if device 900 moves too much relative to the user's face (e.g., the device may move while the user slowly moves and rotates/tilts his or her face). must remain stationary). In the example of Figure 9S, device 900 detects excessive movement of one or more of its cameras in relation to the user's face. This excess movement is optionally a significant change in the position and/or orientation of the user's face relative to the device 900 that coincides with movement of the device itself, which prevents reliable alignment and/or registration. In response, device 900 displays a visual prompt 946 at registration interface 938 instructing the user to reduce movement of the device (e.g., prompting the user to keep the device stationary during the registration process). issued on the Device 900 also optionally produces visual and/or auditory output 948 simultaneously. In some examples, movement of the device itself is measured by the accelerometer 168 rather than the biometric sensor 903. Movement of the device is also optionally measured by a magnetometer of device 900, an inertial measurement unit, etc.

Successful registration typically requires that the alignment of the user's face with respect to the cameras on device 900 is maintained throughout the registration process. Accordingly, in some examples, device 900 optionally terminates the face registration process if more than one alignment errors are detected during registration. In some examples, during the enrollment process, if device 900 detects one or more alignment errors, the electronic device terminates the enrollment process (e.g., stops displaying facial enrollment interface 938) and optionally Initiates (e.g., transitions to) an alignment process that displays alignment interface 908-2. In the examples of FIGS. 9T and 9U, alignment interface 908-2 and its components optionally have similar or identical visual characteristics to initial alignment interface 908 described above with respect to FIGS. 9B-9O. In the examples of FIGS. 9T and 9U , device 900 has determined that the user's face is out of frame, and as a result, device 900 has an internal image that is out of position compared to the successful alignment shown in FIG. 9O. Displays the user's face image 918-2 within the display portion 912-2. In some examples, the device outputs an indication of an alignment error, such as text prompt 950 indicating that user facial image 918-2 is not properly aligned within positioning element 910. This example is illustrative only. In some examples, an alignment error is optionally a failure to meet any of the other alignment criteria discussed above (eg, distance from device, orientation angle, adverse lighting, etc.). In such cases, text prompt 950 instructs the user to move the device and/or their face to an acceptable range of distances or to correct the orientation angle. In some other examples, the alignment error is optionally different from the above criteria such that a small change in alignment will not cause the device to terminate the face registration process. In response to detecting one or more alignment errors, the device may display a portion of the image preview displayed on the outer display portion 914-2 and display a positioning element 910-2, as shown in FIG. 9U. Visually de-emphasize the display portion 912-2. For example, device 900 brightens or blurs the preview image in external display portion 914-2 to assist the user in realigning their face with respect to biometric sensor 903. In the example of FIG. 9U , de-emphasizing the inner display portion 912-2 means that a significant portion of the user's facial image 918-2 is positioned within the positioning element 910-2 within the outer display portion 914-2. It reveals that it is positioned outside of .

In some examples, device 900 again detects that the user's face is properly aligned with biometric sensor 903. In response, device 900 outputs audio output 934-2 and/or tactile output 936-2 indicating successful alignment. In some examples, audio output 934-2 and tactile output 934-6 have similar characteristics as audio output 934 and tactile output 936, respectively, as described with reference to FIG. 9O. . In some examples, device 900 then resumes the registration process. For example, device 900 may display internal portion 912-2 and facial image ( 918-2) is emphasized. In some examples, device 900 resumes the enrollment process at the point where the electronic device detects an alignment error (e.g., facial enrollment interface 938 may have the enrollment progress meter advanced to the same state as when the alignment error was detected). 940) is displayed again).

In some examples, if the device does not detect that proper alignment has been established (e.g., re-established) within a predetermined period of time, device 900 may use accessibility option affordance 928-2, as shown in FIG. 9V. ) is displayed. In some examples, accessibility options provide the option to proceed with the registration process without all alignment conditions being met, as described below. In some examples, accessibility options provide the option to set up biometric (eg, facial) authentication with partial registration only (eg, scanning only a portion of the user's face).

In response to detecting activation (e.g., selection) of accessibility option button 928-2 (e.g., by tap gesture 952), the device displays accessibility registration interface 954, as illustrated in FIG. 9W. ) is displayed. One or more features of accessibility registration interface 954 have similar or identical visual characteristics as corresponding features of registration interface 938. For example, in Figure 9W, face registration interface 954 includes a user facial image 939-2 displayed within positioning element 941-2. In some examples, user facial image 939-2 is a live preview of image data captured by biometric sensor 903-2. Accessibility registration interface 954 also optionally includes a registration progress meter 940-2 surrounding a location element 941-2 and a user facial image 939-2. As described above with respect to FIGS. 7S and 11A-11H, registration progress meter 940-2 extends radially outward from user facial image 939-2, and in some examples surrounds it in a circular pattern. consists of a set of progression elements (eg, 940-2a, 940-2b, and 940-2c). Accessibility registration interface 954 optionally includes orientation guide 942-2. In some examples, the orientation guide includes a set of curves (e.g., crosshairs) that appear to extend out of the plane of the display 901 in a virtual z-dimension, intersecting above the center of the user's facial image 939-2. do. Like facial enrollment interface 938, accessibility interface 954 optionally includes a text prompt (e.g., prompt 956) providing written instructions to successfully complete the enrollment process. In some examples, accessibility registration interface 954 also includes a completion affordance 956, the activation of which causes the user to exit the registration process and proceed to set up facial authentication using only a partial scan of their facial features. do. In some examples, partial scan is helpful to a user who, in some situations, has a condition that prevents the user from tilting his or her head in any directions other than those required for registration.

In response to activation (e.g., selection) of completion affordance 956 (e.g., by user input 958 shown in FIG. 9X), the device displays face registration confirmation interface 960, illustrated in FIG. 9Y. . The face registration confirmation interface, in the example of FIG. 9Y, includes a face image 939-3 that has similar visual characteristics as the user face image 939-2. Facial image 939-3 is optionally surrounded by an enrollment progress meter 962 that is displayed in the successful authentication status described above with respect to FIGS. 7P and 7Q. The facial registration confirmation interface also includes a partial scan registration affordance 964 that allows a user to register collected facial data for use in device authentication. Facial registration confirmation interface 960 also includes a back affordance 966 that allows the user to navigate back to accessibility registration interface 954.

As illustrated in Figure 9Z, the device detects user input 968 corresponding to activation (e.g., selection) of back affordance 966. In response to detecting user input, device 900 displays accessibility registration interface 954 (e.g., for a second time). While displaying accessibility registration interface 954 , device 900 detects movement (eg, rotation and/or tilt) of the user's face relative to biometric sensor 903 . In the case of FIG. 9AA , device 900 detects that the user's face is tilted in a particular direction (e.g., downward and/or to the right toward meter portion 970). As described in further detail below with respect to FIGS. 11B-11H, device 900 updates user facial image 939-2 based on detected movement and determines how the user's head is positioned in three-dimensional space. Update the position of orientation guide 942-2 to indicate that it has been tilted and/or rotated. In response to detecting movement of the user's face, device 900 captures image data of a portion of the user's face (e.g., the left side of the face) and a corresponding portion of the enrollment progress meter 940-2 (e.g., the left side of the face). , the appearance of the meter portion (970) is changed at the same time. In some examples, device 900 may display a meter portion (as described in greater detail with respect to FIGS. 7I-7K and FIGS. 11B-11H ) to indicate which portion of the user's face is currently being registered. 970) expand one or more progression elements and/or change their color. In some examples, device 900 maintains the display of meter portion 972 (eg, does not change its appearance) because meter portion 972 corresponds to a facial orientation that has not yet been registered.

As illustrated in FIG. 9AB , in some examples, device 900 detects a change in the orientation of the user's face relative to its one or more cameras (e.g., the user's face is tilted upward) and moves the user accordingly. Update the face image 939-2 and orientation guide 942-2. By way of example, because the image data at the facial orientation corresponding to meter portion 972 has been successfully registered, device 900 may register the meter portion (972) as described in more detail below with respect to FIGS. 972) transition the status of the progression elements to the “registered” state (e.g., by changing or shading the line width and/or color of the progression elements). As shown in Figure 9ab, device 900 again detects activation (e.g., selection) of completion affordance 956 (e.g., by user input 958-2).

In response to detecting activation of completion affordance 956, device 900 returns to displaying face registration confirmation interface 960, as shown in Figure 9ac. Because a portion of the user's face has been successfully registered, device 900 displays registration success indicator 974, for example, proximate to user face image 939-3. In the example of Figure 9ac, registration success indicator 974 indicates orientations of the user's face that were successfully registered. In some examples, registration success indicator 974 is a circular bar. Accordingly, in some examples, registration success indicator 974 indicates (e.g., is located at) locations where a registration progress meter transitioned to a successful state during registration.

In some examples, partial scan registration affordance 964 is optional because accessibility registration interface 960 allows a user to set up facial authentication with only partial registration of their facial features. As shown in Figure 9A, device 900 detects activation (e.g., selection) of partial scan registration affordance 964 (e.g., by user input 976). In response to detecting activation of partial scan registration affordance 964, device 900 displays registration complete interface 978, illustrated in Figure 9ae. Enrollment completion interface 978 includes a text prompt 980 that indicates to the user that the enrollment process is complete and facial authentication is securely set up. Enrollment completion interface 978 optionally includes a generic facial graphic 982 in the location previously occupied by user facial image 939-3. In some examples, enrollment completion interface 978 also includes a completion affordance, the activation of which causes the electronic device to terminate facial authentication setup.

10 is a flow diagram illustrating a method for aligning a biometric feature on a display of an electronic device, according to some examples. Method 1000 includes a device having a display, one or more input devices (e.g., a touchscreen, microphone, camera), and a wireless communication radio (e.g., a Bluetooth connection, a WiFi connection, a mobile broadband connection such as a 4G LTE connection). For example, 100, 300, 500, 900). In some examples, the display is a touch-sensitive display. In some examples, the display is not a touch-sensitive display. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the device includes one or more biometric sensors, optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device, such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors. Some operations of method 2000 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 1000 provides an intuitive method for aligning biometric features on a display of an electronic device. This method reduces the cognitive burden on the user for registering biometric features on the device, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to register biometric features faster and more efficiently saves power and increases time between battery charges.

The device displays 1002 a first user interface (e.g., 905) on the display. For example, the first user interface is optionally a registration introductory user interface as described above with respect to method 700.

While displaying the first user interface, the device detects (1004) the occurrence of a condition corresponding to initiating a biometric registration process to register each type of biometric characteristic (e.g., 917). For example, the occurrence of the condition is optionally an input corresponding to a request to “start registration” (e.g., 906 on start affordance 904).

In response to detecting the occurrence of a condition corresponding to initiating a biometric enrollment process (e.g., a user input selecting to initiate enrollment), the device may display, on the display, capture by one or more cameras (e.g., 903). Display 1006 a digital viewfinder (e.g., display portions 912, 914) containing a preview of the image data (e.g., user facial image 918). In some examples, the preview of the image data includes a first portion of the field of view of one or more cameras (e.g., outer portion of field of view 914) and a second portion of the field of view of one or more cameras (e.g., inner portion of field of view 912). encompasses. In some examples, the second portion of the field of view (e.g., 914) is a portion of the field of view that surrounds (or partially surrounds) the first portion of the field of view (e.g., 912) (1008). In some examples, the inner portion of the field of view is optionally divided from the outer portion by an alignment element (eg, positioning element 910). In some examples, the preview of the image data optionally changes over time as content in the field of view of one or more cameras (e.g., 903) changes. Displaying a preview of the image captured by the biometric sensors provides the user with feedback regarding the location and orientation of his or her biometric features relative to the biometric sensors of the device, allowing the user to Allows proper alignment of biometric features with sensors more quickly and efficiently. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the device, along with a preview of the image data, includes an alignment element (e.g., positioning element) that indicates the portion of the preview (e.g., 912) where the user's face (e.g., 918) should be positioned to proceed with biometric registration. elements 910) are displayed simultaneously (1010). For example, the alignment element may optionally include framing brackets displayed in a central portion of the preview image (e.g., 912) to prompt the user to move the device or their face toward alignment with the central portion of the preview image. Framing One. Displaying alignment elements framing a particular portion of the digital viewfinder provides the user with feedback regarding the position of his or her biometric features relative to a portion of the biometric sensor's field of view that corresponds to proper alignment of the biometric features. to provide. This in turn allows the user to properly position his or her biometric features relative to the sensors more quickly and efficiently. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, after initiating the biometric registration process (1012), the device detects in the field of view of one or more cameras (e.g., 903) each type of biometric feature (e.g., 917) that meets the alignment criteria. Determine whether it has been done (1014). Determining whether a user's biometric features are properly aligned with the biometric sensors can be accomplished by ensuring that image data corresponding to specific portions and/or orientations of the biometric features are captured during registration (e.g., method Improve the quality of subsequent biometric registration (according to 1200 and/or 1400). This in turn improves the device's ability to match the user's biometric characteristics with data captured during biometric authentication on the device. Performing optimized behavior when a set of conditions is met without requiring additional user input (e.g., by helping the user provide appropriate inputs when operating/interacting with a device and reducing user errors) ) improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, in response to detecting (1016) each type of biometric feature (e.g., 917) that meets the alignment criteria, the device outputs a first type of tactile output (e.g., 934, 936, 934- 2, 936-2, for example, the tactile output is an output corresponding to successful alignment) is output (1018). Issuing a tactile output upon detecting that a biometric feature is properly aligned with the biometric sensors provides the user with the biometric information throughout the subsequent biometric registration process (e.g., methods 1200 and/or 1400). Feedback indicating successful alignment is provided to the user, prompting the user to maintain the feature in that alignment. Providing improved tactile feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs during biometric registration and reducing user errors when operating/interacting with the device). and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, in response to detecting each type of biometric feature that meets the alignment criteria (1016), the device stores (1020) image data corresponding to the biometric feature (e.g., 917). In some examples, upon successful alignment, the device captures data associated with the biometric characteristic. Storing biometric (e.g., image) data in response to detecting successful alignment of biometric features allows the device to automatically capture data for reference during subsequent biometric authorization attempts. Performing optimized behavior when a set of conditions is met without requiring additional user input (e.g., by helping the user provide appropriate inputs when operating/interacting with a device and reducing user errors) ) improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the alignment criteria include a requirement that at least a portion of the biometric feature (e.g., 917) is within a first portion of the field of view of one or more cameras (e.g., internal display portion 912, 912-2). (1024). For example, in some examples, the electronic device determines whether the image data includes data corresponding to a biometric characteristic that satisfies alignment criteria. In some examples, the sorting criteria include lighting condition criteria (1050). In some examples, the sorting criteria include a requirement that the lighting conditions of the electronic device be such that at least a first threshold amount of light is detected (e.g., by 903) and/or no more than a second threshold amount of light is detected, such that the biometric characteristic Requires adequate capture of image data during registration.

In some examples, the alignment criteria include a requirement that a portion of the biometric feature (e.g., a portion of 917) be oriented relative to the electronic device in a predetermined manner (1052). In examples where the biometric feature is the user's face, the alignment criteria optionally include a requirement that the user's gaze is directed toward at least one of the electronic device's one or more cameras (e.g., 903) or the electronic device's display (e.g., 901). Includes. In some examples, the requirement that a portion of the biometric feature (e.g., a portion of the user facial image 918) be oriented relative to the electronic device in a predetermined manner may mean that the biometric feature (e.g., 917) is configured with one or more biometric sensors. The requirement is that it is located within a critical angle (e.g., elevation angle) with respect to the angles (e.g., 903). In some examples, the alignment criteria may be such that the biometric feature (e.g., 917) is predefined so that the biometric sensors can capture biometric data corresponding to the biometric feature at a specific angle or within a range of angles. It requires being positioned relative to the biometric sensors (e.g., 903) in a determined manner. In some examples, the device may, for example, based on the degree to which a biometric feature (e.g., 917) deviates from a predefined range of angles with respect to one or more biometric sensors (e.g., 903), Blur the display (e.g., display portions 912 and/or 914) of the electronic device.

In some examples, the alignment criteria include the requirement that the biometric feature (e.g., 917) is within a first threshold distance from one or more biometric sensors (e.g., 903) (e.g., the biometric feature is too far from the biometric sensors). not far) 1042, and a requirement that the biometric feature is not within a second threshold distance from one or more biometric sensors (e.g., the biometric feature is not too close to the biometric sensors) 1026 ).

In some examples, while the biometric feature (e.g., 917) is at a first distance from the electronic device that is not within a predetermined range of distances from the electronic device, the device is monitored by one or more cameras (e.g., 903). , detect 1044 a change in the distance of the biometric feature (e.g., 917) from a first distance to a second distance from the electronic device that is not within a predetermined range of distances from the electronic device. In response to detecting a change in distance, the device generates an output (e.g., an output characteristic (e.g., magnitude or amplitude, or frequency or repetition rate) that changes based on the distance of the biometric feature from a predetermined range of distances. , generate 1046 audio, tactile and/or visual outputs 922, 924). In some examples, the electronic device outputs ongoing audio (e.g., 917) with a frequency that increases as the distance between the biometric feature (e.g., 917) and the electronic device approaches a target distance (or range of distances) from the electronic device. e.g. 924, e.g. a series of beeps). For example, the beeping speed is optionally increased. Conversely, the frequency of the audio output (e.g., 922) is optionally reduced as the distance between the biometric feature and the electronic device moves further away from the target distance (or range of distances) from the electronic device. For example, the beeping rate is selectively reduced. In some examples, similar feedback is generated with tactile outputs (eg, output 924) or visual outputs. Issuing an audio, tactile and/or visual output that varies based on the distance between the biometric feature and the device can be used to control his or her biometric data over a range of distances from the biometric sensors that correspond to appropriate alignment. Provides ongoing feedback to the user regarding the location of measurement features. This in turn reduces the time that alignment interfaces are displayed and reduces the number of user inputs required during the alignment process. Accordingly, providing improved audio, tactile and/or visual feedback to the user may be beneficial to the device's performance (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). It improves operability and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

After initiating the biometric registration process, upon determining (1022) that each type of biometric feature (e.g., 917) that meets the alignment criteria has been detected in the field of view of one or more cameras (e.g., 903), the device represents a first portion of the field of view of one or more cameras (e.g., the inner display portion 912 in FIG. 9J) relative to a second portion of the field of view of the one or more cameras (e.g., the outer display portion 914 in FIG. 9J). Emphasize (e.g., darken, blur, and/or black out a second portion of the field of view of one or more cameras without darkening, blurring, and/or blacking out a first portion of the field of view) (1028) . For example, alignment criteria include a requirement that the user's face (eg, 917) is aligned with the camera (eg, 903) in a predetermined alignment or the user's eyes are aligned with the camera in a predetermined alignment. Providing a visual effect that highlights a portion of the display upon detecting successful alignment of the user's biometric features with the biometric sensors allows the user to determine the current location of his or her biometric features (e.g. , which is optimal for a subsequent biometric registration process (according to methods 1200 and/or 1400). Providing improved visual feedback when a set of conditions are met without requiring additional user input (e.g., helps the user provide appropriate inputs when operating/interacting with a device and reduces user errors) improves the device's operability and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the device darkens (1030) a portion of the digital viewfinder that corresponds to a second portion (e.g., 914 in FIG. 9J) of the field of view of one or more cameras (e.g., 903). Darkening in this manner includes lowering or dimming the brightness of a portion of the digital viewfinder corresponding to the second portion of the field of view.

In some examples, the device stops displaying (1032) a portion of the digital viewfinder that corresponds to a second portion of the field of view of one or more cameras (e.g., second display portion 914). For example, stopping displaying a portion of the viewfinder corresponding to a second portion of the field of view may result in blacking out the second portion of the field of view and/or replacing the display of the second portion of the field of view with display of other content. It includes doing.

In some examples, the device magnifies the display of a first portion of the field of view of one or more cameras on the display (e.g., internal display portion 912) (1034). In some examples, magnifying the display of the first portion includes magnifying the display of some or all of the first portion of the field of view. In some examples, magnifying the display of the first portion of the field of view includes zooming in on the first portion of the field of view. In some examples, emphasizing a first portion of the field of view of one or more cameras (e.g., 912) relative to a second portion of the field of view of the one or more cameras (e.g., 914) reduces or hides some or all of the first portion. It includes In some examples, the device may be configured to zoom out the first portion before magnifying the display of the first portion (e.g., to provide zoom in and zoom out effects) and/or zoom out the first portion after magnifying the display of the first portion. I order it.

In some examples, the device modifies (1036) an alignment element (e.g., 910). For example, in some examples, modifying an alignment element includes removing the alignment element. In some examples, modifying an alignment element includes changing the shape and/or color of the alignment element (e.g., from 910 to 910-2 in FIGS. 9J and 9K). For example, the device modifies the shape of the alignment element from a first shape to a second shape (1038). In some examples, the first shape is substantially rectangular and the second shape is substantially circular (1040). Alternatively, the first shape and/or the second shape is optionally any other shape or portion of a shape. In some examples, the predetermined shape is optionally a segmented shape, such as a segmented rectangle (eg, a rectangle with part of one or more sides missing).

In some examples, after emphasizing a first portion of the field of view of one or more cameras (e.g., 912) relative to a second portion of the field of view of one or more cameras (e.g., 914), the device satisfies the alignment criteria. 1054 detects that each type of biometric feature (e.g., 917) is no longer in the field of view of one or more cameras. In response to detecting that the respective type of biometric feature that meets the alignment criteria is no longer detected in the field of view of one or more cameras, the device outputs an indication of an alignment error (e.g., 950). For example, for correctable errors, the device identifies the error and prompts the user to correct the error. For uncorrectable errors, the device only identifies the error. Errors are identified by text and tactile output (eg, 950, 924, 925). In some examples, errors are identified using auditory outputs, such as those provided for accessibility purposes. In some examples, the criteria for detecting that a biometric feature is no longer detected in the field of view of one or more cameras are the same as the criteria for determining that the biometric feature meets the alignment criteria. In some examples, the alignment criteria are different than the criteria for detecting that a biometric feature is no longer detected in the field of view of one or more cameras (e.g., once a biometric feature is aligned with one or more cameras, the biometric feature is , the device can be moved slightly out of alignment without terminating the biometric registration process and outputting an indication of an alignment error). Issuing an indication that the user's biometric features are no longer aligned with the biometric sensor quickly indicates that the user has moved away from a previously established alignment in the position and/or orientation of his or her biometric features. Provide feedback that makes you aware. This feedback prompts the user to quickly reposition his or her biometric features to re-establish proper alignment with the biometric sensor, thereby reducing the time that alignment user interfaces are displayed; reduces the number of inputs required in the field and improves the quality of biometric feature registration. Accordingly, providing improved audio, tactile and/or visual feedback to the user may be beneficial to the device's performance (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). It improves operability and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, outputting an indication of an alignment error includes outputting (1056) a second type of tactile output (e.g., 951). For example, tactile output is the output that corresponds to loss of successful alignment. In some examples, tactile outputs are error-specific, and in some examples audible output is additionally or alternatively provided.

In some examples, outputting an indication of an alignment error includes de-emphasizing a first portion of the field of view of one or more cameras (e.g., 912-2) relative to a second portion of the field of view of the one or more cameras (e.g., 914-2). Includes doing (1058).

For example, the device selectively brightens, blurs, and/or reveals a second portion of the field of view relative to the first portion of the field of view of one or more cameras. In some examples, the electronic device brightens and de-blurs the second portion of the field of view to de-emphasize the first portion relative to the second portion. In some examples, if a biometric feature (e.g., 917) is successfully aligned after receiving an alignment error, the device resumes the biometric registration process from where the registration process was prior to outputting an indication of the alignment error ( For example, registration progress up to the point when an alignment error was detected is preserved). In some examples, a progress indicator (e.g., 940) that indicated registration progress disappears when an indication of an alignment error is output, but is displayed again when the biometric feature is properly aligned with one or more biometric sensors (e.g., 940-2). In some cases, when the progress indicator is displayed again, it includes an indication of the progress made in registering the biometric feature before outputting an indication of an alignment error. Providing a visual effect that de-emphasizes a portion of the display upon detecting an alignment error allows the user to quickly notice that the location and/or orientation of his or her biometric features has deviated from a previously established alignment. Make them aware. This feedback prompts the user to quickly reposition his or her biometric features to re-establish proper alignment with the biometric sensor, which reduces the time the alignment user interfaces are displayed and allows the alignment user to Reduces the number of inputs required at interfaces and improves the quality of subsequent biometric feature registration (e.g., according to methods 1200 and/or 1400). Providing improved visual feedback when a set of conditions are met without requiring additional user input (e.g., helps the user provide appropriate inputs when operating/interacting with a device and reduces user errors) improves the device's operability and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, upon determining that the alignment error is a first type of alignment error (e.g., the biometric feature is too far from or too close to the electronic device) 1062, the device may be configured to correct for the first type of alignment error. Output (e.g., display) a prompt (e.g., 920) to move (e.g., 920) the biometric feature (1064). For example, the device prompts the user to move closer to or further away from the electronic device, respectively.

In some examples, upon determining that the alignment error is a second type of alignment error (e.g., the biometric feature is outside the first portion of the field of view) 1062, the device may perform a biometric measurement to correct for the second type of alignment error. Output (e.g., display) a prompt (e.g., 950) to move the feature (1064). For example, the device prompts the user to move a biometric feature into a first portion of the field of view. In this case, the device refrains from outputting a prompt (e.g., 926) to move the biometric feature to correct for the first type of alignment error (1068). In some examples, the second portion of the field of view (e.g., 914-2) is modified (e.g., blurred) in response to determining that the alignment error is a second type of alignment error.

For example, a first type of alignment error may be that a portion of the biometric feature (e.g., a portion of 917 shown at 918-2, 939) is oriented outside of the first portion of the field of view (e.g., 912, 912-2). It will be (1074). In this case, the device outputs a prompt (e.g., 950) to move a portion of the biometric feature into a first portion of the field of view (1076) to prompt the user to correct for the first type of alignment error. Providing a prompt with instructions on how to correct alignment errors allows the user to quickly recognize how to reposition his or her biometric features in order to re-establish proper alignment and proceed with the registration process. Provides feedback. This, in turn, reduces the time the device displays alignment interfaces and reduces the number of user inputs required in those alignment interfaces. Providing improved visual feedback when a set of conditions are met improves the operability of a device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors). and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In another example, the first type of alignment error is that the distance between a portion of the biometric feature (e.g., 917) and one or more biometric sensors (e.g., 903) is within a threshold distance (e.g., biometric feature is too close to one or more biometric sensors (1078). In this case, the device outputs a prompt (e.g., 920) to move the biometric feature away from the electronic device to prompt the user to correct for the first type of alignment error (1080).

In another example, a first type of alignment error is that the distance between a portion of the biometric feature (e.g., 917) and one or more biometric sensors (e.g., 903) exceeds a threshold distance (e.g., biometric The feature is too far from one or more biometric sensors (1082). In this case, the device outputs a prompt (e.g., 920) to move the biometric feature closer to the electronic device to prompt the user to correct for the first type of alignment error (1084).

In another example, a first type of alignment error occurs when the angle of a biometric feature (e.g., 917) relative to one or more biometric sensors (e.g., 903) is within a predefined range for the one or more biometric sensors. It is outside of angles (e.g. angles of elevation). For example, the biometric feature is too high in some situations. In another example, one or more biometric sensors are, in some situations, too low. In this case, the device outputs a prompt to move the biometric feature to adjust the angle (e.g., elevation angle) of the biometric feature relative to one or more biometric sensors.

In some examples, upon determining that the first type of error condition persists for a threshold period of time (1086), the device may configure an accessibility interface (e.g., 908) that allows the user to proceed with biometric registration without correcting the error condition. ) is displayed (1088). For example, in some examples, the device may be configured to move the biometric feature (e.g., 917) relative to the device so that an error condition is corrected or without tilting the biometric feature to capture images of a different side of the biometric feature. Allows the user to proceed with biometric registration. In some examples, the device allows a user to proceed with biometric registration in this manner if the biometric feature is improperly aligned for a predetermined amount of time and/or in response to a predetermined number of failed requests.

In some examples, after outputting an alignment error, upon determining that each type of biometric feature (e.g., 917) that meets the alignment criteria has been detected in the field of view of one or more cameras (e.g., 903), the device may: Re-emphasize (1070) a first portion of the field of view of one or more cameras (e.g., 912-2) relative to a second portion of the field of view of the one or more cameras (e.g., 914-2). For example, optimally, the device may display a second portion of the field of view of one or more cameras (e.g., 912-2) without darkening, blurring, or blacking out a first portion of the field of view of the one or more cameras (e.g., 912-2). For example, darken, dim and/or blacken 914-2).

In some examples, after outputting an alignment error, and upon determining that each type of biometric feature (e.g., 917) that meets the alignment criteria has been detected in the field of view of one or more cameras (e.g., 903), the device Outputs a first type of tactile output (e.g., 936) (1072). However, in some examples, the device outputs a third type of tactile output that is different from the first and second types.

Upon determining that each type of biometric feature (e.g., 917) that meets the alignment criteria was not detected in the field of view of one or more cameras (e.g., the user's face or eyes were not detected in the predetermined alignment), The device may be configured to display a second portion of the field of view of one or more cameras (e.g., 903) without emphasizing a first portion of the field of view of one or more cameras (e.g., 912, 912-2) relative to a second portion of the field of view of the one or more cameras (e.g., 903). Maintain the display of the digital viewfinder (1090).

In some examples, the device detects (1092) a change in the orientation and/or position of a biometric feature (e.g., 917) relative to one or more biometric sensors (e.g., 903). For example, the device optionally detects a change in position, a change in orientation, or both a change in orientation and position.

In some examples, in response to detecting (1094) a change in the orientation and/or position of a biometric feature (e.g., 917) relative to one or more biometric sensors (e.g., 903) and device movement criteria are met. Upon determining that this has occurred (e.g., the device is physically moving beyond a threshold amount in a way that prevents reliable alignment/registration), the device may prompt the electronic device to reduce movement (e.g., 946, 948, visually , tactile or auditory alarm) is output (1096). In some examples, the device detects reduced movement of the device, and in response to detecting reduced movement of the device, the device stops outputting the prompt. In some examples, movement of the device is determined based on one or more biometric sensors (eg, 903). For example, a change in the orientation and/or location of a biometric feature relative to one or more biometric sensors may result in movement of the device about the biometric feature rather than movement of the biometric feature relative to the one or more biometric sensors. It matches. In some examples, movement of the device is determined based on one or more orientation sensors of the device, such as an accelerometer (e.g., 168), magnetometer, inertial measurement unit, etc., that are separate from the one or more biometric sensors.

In some examples, while a biometric feature (e.g., 917) is within a first portion (e.g., 912) of the field of view of one or more biometric sensors (e.g., 903) and within a threshold distance of one or more biometric sensors, And upon determining that the biometric feature is within a predefined range of angles (e.g., elevation angles for one or more biometric sensors), the device (e.g., see method 1200 and FIGS. 11A-11E ) Displays an registration progress indicator (e.g., 940) for registration of biometric features (as described in more detail). Displaying the registration progress indicator optionally includes a first portion of the field of view of one or more cameras (e.g., 912, 912-) relative to a second portion of the field of view of the one or more cameras (e.g., 914, 914-2) as described above. This includes emphasizing 2) first. Displaying a progress indicator during enrollment in this way encourages the user to look at the display of the electronic device during enrollment, improving the ability to detect when gaze is directed toward the display and, therefore, whether the user is paying attention to the device. I order it. Encouraging a user to look at the display of an electronic device improves the operability of the device (e.g., by ensuring that the user's gaze is directed toward the display and thereby ensuring that the user's biometric characteristics are properly registered) and that the user- It makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, upon determining that a biometric feature (e.g., 917) is outside a predefined range of angles (e.g., elevation angles for one or more biometric sensors 903), the device may preview the image data. Obfuscate (e.g., blur, darken, or desaturate) at least a portion of (e.g., display portions 912, 912-2, 914, and/or 914-2). In some examples, the device may be configured to ensure that a portion of the preview of the image data is unambiguous when a user is actively shifting the orientation of the biometric feature relative to one or more biometric sensors (e.g., 903). Delaying obscuring a portion of the preview of the image data (for at least a predetermined period of time, such as 1 second, 2 seconds, or 5 seconds) after detecting that it is within the first portion of the field of view and within a threshold distance of one or more biometric sensors I order it. In some examples, obfuscation is delayed as long as the angle of the biometric feature is changing. In some examples, obfuscation is delayed until the angle of the biometric feature has been outside the predefined range of angles for at least a predetermined period of time. In some examples, only a portion of the preview (eg, 912 or 914, 912-2 or 914-2) is obscured. In some examples, the entire preview (eg, 912 and 914, 912-2 and 914-2) is obscured. Obscuring the digital viewfinder when the biometric sensors are positioned too high above or too low below the user's biometric features allows the user to quickly notice that his or her biometric features are out of alignment. Make them aware. This then prompts the user to change the elevation angle between the device and his or her biometric feature until proper alignment is established. Providing improved visual feedback when a set of conditions are met improves the operability of a device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors). and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the device determines the angle of a biometric feature (e.g., 917) relative to one or more biometric sensors (e.g., 903) while a portion of the preview of the image data (e.g., 912 or 914) is obscured. Detect changes. Responsive to detecting a change in the angle of the biometric feature relative to one or more biometric sensors, and wherein the change in angle moves the biometric feature to angles in the predefined range, Depending on the decision to move closer to the angles of Reduce the amount. In some examples, the amount by which the ambiguity of a portion of the preview of the image data is reduced depends on the amount of change in the angle of the biometric feature relative to one or more biometric sensors (e.g., the biometric feature moves more toward the one or more biometric sensors). The more you move, the greater the reduction in the amount of ambiguity). Upon determining that the change in angle moves the biometric feature to a predefined range of angles, the device stops obscuring a portion of the preview of the image data. In some examples, when a change in the angle of a biometric feature moves the biometric feature to a predefined range of angles, the device may inform the user that the angle of the biometric feature is within the predefined range of angles. generate tactile and/or audio output (e.g., 934, 936) for Reducing the ambiguity of the digital viewfinder as the user's biometric features move closer to the predefined angular range allows the user to quickly recognize the set of positions that correspond to successful alignment of the biometric features. This then prompts the user to change the elevation angle between the device and his or her biometric feature until proper alignment is established. Providing improved visual feedback when a set of conditions are met improves the operability of a device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors). and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, in response to detecting a change in the angle of a biometric feature (e.g., 917) relative to one or more biometric sensors (e.g., 903), and where the change in angle moves the biometric feature to a predefined range. Depending on the decision to move further away from the angles of , the device increases the amount of ambiguity of the portion of the preview of the image data (eg, 912 or 914, 912-2 or 914-2). In some examples, the amount by which the ambiguity of a portion of the preview of the image data is increased depends on the amount of change in the angle of the biometric feature relative to one or more biometric sensors (e.g., the amount of change in angle of the biometric feature relative to the one or more biometric sensors). The further you travel, the greater the increase in the amount of ambiguity).

In some examples, obfuscation includes blurring a preview of the image data, and reducing the amount of obscurity of a portion of the preview of the image data (e.g., by reducing the blur radius or other blur parameter) refers to blurring the preview of the image data. It includes reducing the amount of blur. In some examples, increasing the amount of obscurity of a portion of the preview of the image data includes increasing the blur radius or other blur parameter.

Note that the details of the processes described above with respect to method 1000 (e.g., FIGS. 10A-10F) are also applicable in a similar manner to the methods described below. For example, method 1000 optionally includes one or more of the features of various methods described below with reference to methods 800, 1200, 1400, 1600, 1800, 2000, 2200, 2400, 2700. do. For example, an registration process as described in method 1200 may be applied in connection with a facial registration interface (e.g., 954). For another example, hints as described in method 1400 may be applied in connection with a registration progress meter (e.g., 940). For another example, accessibility features as described in method 1400 may be applied in place of or in combination with accessibility options (e.g., 928-2). For brevity, these details are not repeated below.

Operations in the information processing methods described above optionally utilize one or more functional modules within an information processing device, such as general-purpose processors (e.g., as described with respect to FIGS. 1A, 3, and 5A) or application-specific chips. It is implemented by running it. Additionally, the operations described above with reference to FIGS. 9A-9I are optionally implemented by the components shown in FIGS. 1A and 1B. For example, display operation 1002, detection operation 1004, display operation 1006, highlight operation 1028, and sustain operation 1090 may optionally include event classifier 170 and event recognizer 180. , and is implemented by the event handler 190. Event monitor 171 in event sorter 170 detects contact on touch-sensitive surface 604, and event dispatcher module 174 passes event information to application 136-1. The respective event recognizer 180 of application 136-1 compares the event information to the respective event definition 186 and determines that the first contact at the first location on the touch-sensitive surface corresponds to the selection of an object on the user interface. Determines whether it corresponds to the same predefined event or subevent. When an individual predefined event or sub-event is detected, event recognizer 180 activates an event handler 190 associated with detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some examples, event handler 190 accesses the respective GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes may be implemented based on the components shown in FIGS. 1A and 1B.

Now, an example user interface for registering biometric features on an electronic device (e.g., device 100, device 300, device 500, device 700, or device 900), according to some examples. Note Figures 11A to 11L, which illustrate these. User interfaces in these figures are used to illustrate the processes described below, including the processes in FIG. 12.

FIG. 11A illustrates an electronic device 1100 (e.g., portable multifunction device 100, device 300, device 500, device 700, or device 900). In the illustrative example illustrated in FIGS. 11A-11L, electronic device 1100 is a smartphone. In other examples, electronic device 1100 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 1100 has a display 1102, one or more input devices (e.g., a touchscreen, buttons, microphone) of display 1102, and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the electronic device includes one or more biometric sensors (e.g., biometric sensor 1103), optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, one or more biometric sensors 1103 are one or more biometric sensors 703. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

As illustrated in FIG. 11A , device 1100 displays face registration user interface 1104 on display 1102 . In some examples, face registration user interface 1104 is displayed after device 1100 detects successful alignment of the user's face to its one or more cameras, as described above with respect to FIGS. 9A-9AE. . Face registration interface 1104 includes a user face image 1106. In some examples, user facial image 1106 is an image of a user captured by one or more cameras on device 1100. For example, the user facial image 1106 may optionally be a live image data captured by one or more cameras (e.g., a digital viewfinder) that is continuously updated as the camera's field of view and/or the contents of the field of view change. This is a preview. In some examples, background content is removed so that only the user's face is visible in facial image 1106. The facial registration interface also optionally includes an orientation guide 1108 that is superimposed (e.g., overlaid) on the user facial image 1106. As described above with respect to FIGS. 7I-7K , orientation guide 1108 optionally extends in the virtual z-dimension (e.g., along an axis perpendicular to the plane of the display) and above the center of user facial image 100. It is a set of intersecting curves. Accordingly, the curves of the orientation guide 1108 appear to bulge outward relative to the plane of the display 1102, providing a sense of the position of the user's head in three-dimensional space.

Facial enrollment user interface 1104 also includes an enrollment progress meter 1110. Enrollment progress meter 1110 includes a set of display elements (eg, progress elements 1110a, 1110b, 1110c) arranged around a user facial image 1106 and an orientation guide 1108. In the example of FIG. 11A , the progressive elements are a set of lines extending radially outward from the user face image 1106 arranged in a circular pattern. In some examples, progressive elements 1110a, 1110b, 1110c, etc. represent the orientation of the user's face needed to register corresponding facial features. For example, progressive elements within the upper portion of registration meter 1110 may optionally cause one or more cameras on device 1100 to capture image data of the underside of the user's face when the user's head is tilted upward. Moves and/or fills and/or extends and/or changes color. This process is described in more detail below. In the example of FIG. 9A , device 1110 displays progress elements in registered progress meter 1110 as unregistered (e.g., they are grayed out).

Face registration interface 1104 also includes text prompts 1112 instructing the user to move (e.g., rotate and/or tilt) their head in a circular motion during the registration process. In some examples, text prompt 1112 optionally is accompanied by a tactile and/or audible prompt depending on device settings and/or user selections. In some examples, device 1112 displays text prompt 1112 on enrollment interface 1104 through the facial enrollment process.

As illustrated in FIG. 11B, device 1100 detects movement of the user's face relative to its one or more cameras. The movement of the user's face is optionally a rotation and/or tilt motion relative to device 1100. In response, device 1100 continues to update user facial image 1106 to reflect changes in the orientation of the user's face (eg, display its movement). In some examples, orientation guide 1108 tracks (e.g., moves with) the movement of user face image 1106 to visually highlight tilt and rotation movements of the user's face in three dimensions. For example, the center (e.g., intersection) of the orientation guide 1108 is optionally positioned at a central point on the user's face image 1106 and moves with it. In some examples, device 1100 also adjusts the curvature of the lines comprising orientation guide 1108 to give the appearance of three-dimensional rotation (e.g., with respect to an axis perpendicular to display 1100). In some examples, device 1100 highlights orientation guide 1108 while it is moving (eg, while the orientation of the user's face is changing). For example, device 1100 optionally darkens orientation guide 1108 while it moves and/or displays a fading trail as it tracks the movement of the user's face. In this case, device 1100 optionally reduces this emphasis on orientation guide 1108 relative to user face image 1106 when the user's face is not moving.

As shown in FIG. 11B , in response to detecting that the user's face is oriented toward the progress meter portion 1114 (e.g., the image data captured by the biometric sensor 1103 provides an angular view of the user's face) (depending on the decision to include an angular view), the device 1110 updates the display of the progress elements within the meter portion 1114 to a “registering” state by changing the appearance of the progress elements within the meter portion 1114. For example, device 1100 optionally magnifies and/or changes the color of progressive elements within meter portion 1114 while the user's face is oriented toward meter portion 1114. In some examples, device 1100 extends progress ticks and changes their color from gray to blue when updating progress elements to a “registering” state. Changing the display of ongoing elements to the “enrolling” state in this manner indicates that device 1100 is capturing facial imaging data for an angular view corresponding to the current orientation of the user's face (e.g., registering). ). In the example of FIG. 11B , device 1100 leaves progressive elements in meter portion 1116 unregistered to indicate that the user has not yet oriented their face toward meter portion 1116 . In some examples, the display of meter portion 1114 is updated in this manner only when the user's face is sufficiently rotated toward meter portion 1114 (e.g., when the user's face is rotated at least a threshold amount or a threshold angle). do.

In some examples, a registered progress meter is comprised of a set of progress meter portions, such as meter portions 1114 and 1116. In some examples, each progress meter portion includes a predetermined number of progress elements (eg, 3, 5, or 8 progress elements) associated with each portion.

In some examples, as illustrated in FIG. 11C , device 1110 detects small rotations and/or tilts of the user's face and updates the digital viewfinder containing a graphic of the user's face 1106. For example, the user's face began to tilt downward and rotate to the right. However, in the example of Figure 11C, the user's face is still oriented toward the progress meter portion 1114. As a result, device 1100 continues to display the progress elements of meter portion 1114 in a registering state even if the user begins to rotate and/or tilt their head downward and to the right. In this case, device 1100 also maintains the display of progress elements proximate to meter portion 1114 because the user's head has not been rotated sufficiently to trigger registration of the corresponding orientations.

As illustrated in FIG. 11D , device 1110 detects that the user's face has been rotated and/or tilted toward meter portion 1118. In the example of FIG. 11D , the user's face continues the movement illustrated in FIG. 11C , tilting downward and rotating to the right through its initial position in FIG. 11A (e.g., the user's face is positioned at (Move so that it is not oriented towards other parts). In response to detecting a change in facial orientation, device 1100 moves orientation guide 1108 to track the movement of the user's facial image 1106 in the digital viewfinder. Upon determining that the user's face is oriented toward meter portion 1118 (e.g., the image data captured by biometric sensor 1103 includes a second angular view of the user's face), device 1100 Updates the progress elements within the meter portion 1118 to the “registering” status described above. For example, device 1100 stretches the progress ticks within meter portion 1118 and changes their color. In some examples, device 1100 may display a meter portion only if the corresponding portion of the user's face has not previously been registered (e.g., if ongoing elements within meter portion 1118 are in an “unregistered,” grayed-out state). Update the display of (1118). In some examples, device 1100 may determine whether a corresponding portion of the user's face has been previously registered (e.g., to provide a further indication of the orientation of the user's face relative to biometric sensor 1103). Update the display of portion 1118.

In the example of FIG. 11D , device 1100 also indicates that the user's face is no longer oriented toward progress meter portion 1114 (since the user's face is currently oriented toward meter portion 1118). detect. In response, device 1100 changes the appearance of the progress elements within meter portion 1114 back to the “registered” state. In the example of FIG. 11D , device 1100 updates the display of the progress ticks in portion 1114 from the extended “registering” state by again shortening the progress ticks and changing their color. For example, progression elements in the “registered” state are the same length and/or size as progression elements in the “unregistered” state, but do not cover a corresponding portion of the user's face (e.g., as described above with respect to FIG. 11B ). The angular view captured in 11b) is displayed in green to indicate that it has been successfully registered.

In the example of FIG. 11D , device 1100 leaves progress elements in meter portion 1116 unregistered to indicate that the user has not yet oriented their face toward meter portion 1116.

11E-11H illustrate the face enrollment interface 1104 as a user rotates and/or tilts their face in a counterclockwise motion through a series of orientations associated with the right side of the enrollment progress meter 1110. Starting with progress meter portion 1118, device 1100 rotates based on the user's facial orientation (e.g., in response to detecting that the user's face is oriented toward a corresponding portion of progress meter 1110). Progress elements in the path are sequentially changed to the aforementioned “registering” state. Once the user's face has been rotated past these progress elements (e.g., in response to detecting that the user's face is no longer oriented toward corresponding portions of the progress meter 1110), the device 1100 can Progress elements are updated to the "Registered" state to indicate successful registration of the corresponding parts of the face. This process is described in more detail below. In some examples, the visual characteristics of progressive elements in the “enrolling” state are based on the rate at which the user's facial orientation changes. For example, device 1100 may modify the color of progressive elements in the “enrolling” state in a first manner if the user's face is rotating at a first speed and rotate the user's face more slowly and/or faster. If it is rotating, the second method modifies the color of these progress elements.

As shown in FIG. 11E , device 1100 detects that the user's face has been rotated in a counterclockwise manner relative to the biometric sensor 1103 (e.g., the user's face is up relative to its position in FIG. 11D rotating and/or tilting to the left). As described above, device 1100 continuously updates user facial image 1106 to reflect changes in orientation and uses orientation guide 1108 to track the movement of user facial image 1106 in the digital viewfinder. Move it. As the user's face is rotated upward, device 1100 moves one or more progress elements (e.g., by stretching one or more progress elements and/or changing their color, as described above) within meter portion 1116. , 1116a) updates the display to “registering status”. As shown by the position of the user's face image 1106 in FIG. 11E, the rotation moves the user's face past an orientation corresponding to one or more progressive elements (e.g., 1118a) within the meter portion 1118 (e.g., move him away) In response to detecting that the user's face is no longer in this orientation, device 1100 may display one or more progressive elements (including 1118a) to indicate successful registration of these portions, as described above, as “registered.” "Update to status. In the example of FIG. 11E , device 1100 maintains one or more elements of progress meter portion 1118 (e.g., 1118b) in a “registering” state, meaning that the user's face has not yet been rotated out of the corresponding orientation. Because it wasn't. Likewise, device 1100 also continues to display one or more progress elements (e.g., 1116b) within meter portion 1116 in an initial “unregistered” state, meaning that the user's face has not yet been positioned in the corresponding orientation. Because.

FIG. 11F illustrates the face registration interface 1104 as counterclockwise rotation of the user's face relative to its position in FIG. 11E continues. Again, device 1100 continues to update user facial image 1106 to reflect changes in orientation and moves orientation guide 1108 to track the movement of user facial image 1106 in the digital viewfinder. As shown by the position of the user's face image 1106 in FIG. 11F, the rotation moves the user's face into an orientation corresponding to the progress meter portion 1116. In response to detecting the user's face in this orientation, device 1110 may move one or more advancing elements within meter portion 1116 (e.g., by stretching one or more advancing elements and/or changing their color, as described above). Change the display of the fields (e.g., 1116b) from the “unregistered” state to the “registering” state. As shown by the position of the user's face image 1106 in FIG. 11F, the rotation also moves the user's face past an orientation corresponding to the remaining elements of the meter portion 1118 (e.g., 1118b) (e.g., move him away) In response to detecting that the user's face is no longer in this orientation, device 1100 updates the display of these progress elements (including 1118b) to the "registered" state described above, thereby adjusting meter portion 1118b Indicates successful registration of the corresponding angular view of the user's face. In the example of FIG. 11F , device 1100 also continues to display progress elements within meter portion 1120 in the initial “unregistered” state described above because the user's face has not yet been positioned in the corresponding orientation. .

FIG. 11G illustrates the face registration interface 1104 as counterclockwise rotation of the user's face relative to its position in FIG. 11F continues. Again, device 1100 continues to update user facial image 1106 to reflect changes in orientation and moves orientation guide 1108 to track the movement of user facial image 1106 in the digital viewfinder. As shown by the position of the user's face image 1106 in FIG. 11G, the rotation moves the user's face into an orientation corresponding to the progress meter portion 1120. In response to detecting the user's face in this orientation, device 1110 may display the progress elements within meter portion 1120 (e.g., by stretching and/or changing the color of one or more progress elements as described above). Changes the status from “unregistered” to “registered.” As shown by the position of the user's face image 1106 in FIG. 11G, the rotation also moves the user's face past (eg, beyond) an orientation corresponding to the progress meter portion 1116. In response to detecting that the user's face is no longer in this orientation, device 1100 updates the display of progress elements within meter portion 1116 to a “registered” state, thereby updating the display of progress elements within meter portion 1116 to the corresponding Indicates successful registration of the angular view of the user's face. In the example of FIG. 11G , device 1100 continues to display progress elements within meter portion 1122 in an initial “unregistered” state because the user's face has not yet been positioned in the corresponding orientation.

FIG. 11H illustrates the face registration interface 1104 as counterclockwise rotation of the user's face relative to its position in FIG. 11G continues. Again, device 1100 continues to update user facial image 1106 to reflect changes in orientation and moves orientation guide 1108 to track the movement of user facial image 1106 in the digital viewfinder. As shown by the position of the user's face image 1106 in FIG. 11H, the rotation moves the user's face into an orientation corresponding to the progress meter portion 1122. In response to detecting the user's face in this orientation, device 1110 may display progress elements within meter portion 1122 (e.g., by stretching and/or changing the color of one or more progress elements as described above). Change from “unregistered” status to “registered” status. In some examples, this orientation may cause device 1100 to , causing the display of one or more progress elements within meter portion 1114 to change from the “registered” state shown in FIGS. 11D-11G back to the “registering” state based on the orientation of the user's face. In this case, device 1100 returns these elements of progress meter portion 1114 back to the “registered” state in response to detecting that the user's face is no longer oriented in that direction. As shown by the position of the user's face image 1106 in FIG. 11H, the rotation also moves the user's face past (eg, beyond) an orientation corresponding to the progress meter portion 1120. In response to detecting that the user's face is no longer in this orientation, device 1100 updates the display of progress elements within meter portion 1120 to a “registered” state, thereby updating the display of progress elements within meter portion 1120 to the corresponding Indicates successful registration of the angular view of the user's face. In the example of FIG. 11G , device 1100 initializes the remaining active elements of registered meter 1110 (e.g., active elements that are not within meter portions 1114, 1116, 1118, 1120, or 1122) to an initial “unregistered” status. " state continues to display, because the user's face has not yet been positioned in the corresponding orientation.

Registration and/or scanning of the user's facial features continues until all elements of the registration progress meter 1110 have transitioned to the registered state (e.g., image data of all corresponding angular views of the user's face has been collected from the biometric sensor ( It proceeds in this way until it is captured by 1103). Registration continues until the user's face returns to the orientation corresponding to the meter portion 1118, for example through counterclockwise rotation.

Displaying and updating a progress indicator during registration in this manner encourages the user to look at the display of device 1100 during registration, thereby detecting when gaze is directed at the display and therefore whether the user is paying attention to the device. improve the ability to Encouraging the user to look at the display of device 1100 improves the operability of the device (e.g., by ensuring that the user's gaze is directed toward the display and thereby ensuring that the user's biometric characteristics are properly registered) and It makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

It should be understood that the examples of FIGS. 11D-11H are illustrative only. In particular, registration of the user's facial features may begin at any portion of the progress meter 1110 (e.g., meter portion 1114). Similarly, angular views of the user's face corresponding to each meter portion or progress element may be registered in any order (eg, via clockwise rotation).

FIG. 11I illustrates the face registration interface 1104 after image data of all corresponding angular views of the user's face have been captured by the biometric sensor 1103. In the example of Figure 11I, device 1100 has transitioned the display of all progress elements in registration meter 1110 to a “registered” state (e.g., during the registration process described above with respect to Figures 11B-11H). For example, device 1100 changes the color of the progress element to green to indicate successful registration. In the example of FIG. 11I , device 1100 displays text prompt 1124 indicating that the first scan of the user's facial features is complete. In some examples, device 1110 issues an audio and/or tactile notification 1126 to provide additional indication that the first scan is complete. In some examples, the audio and/or tactile output indicating successful registration of the user's facial features is the same as the audio and/or tactile output used to indicate successful facial authentication in device 1100. In the example of Figure 11I, device 1100 continues to display user facial image 1106. In some examples, user facial image 1106 is still part of the digital viewfinder's live preview. In other examples, device 1100 displays a single (eg, still) user image captured during the registration process. In the example of Figure 11I, device 1100 stops displaying orientation guide 1108 once the scan is complete.

As illustrated in Figures 11J and 11K, in some examples, device 1100 displays an animation that transitions the display of registration progress meter 1110 to success status meter 1128 shown in Figure 11K. For example, device 1100 reduces the length of each progress tick mark and merges the display of previous individual progress elements into a continuous circle, as illustrated in FIG. 11J. In the example of FIG. 11K, after displaying the animation, device 1100 displays scan completion interface 1130. Scan completion interface 1130 includes a user facial image 1132 and a success status meter 1128. In the example of FIG. 11K, user facial image 1132 is blurred, faded, darkened, or otherwise obscured to indicate that additional image data is no longer being collected as part of the facial scan. In some examples, success status meter 1128 is a solid, continuous green circle surrounding user face image 1132, providing a visual indication that the first scan is complete. To provide additional visual notification, scan completion interface 1130 also includes a text prompt 1134 (e.g., a completion message). Scan completion interface 1130 also includes a continue affordance 1136. In some examples, while displaying registration completion interface 1130, device 1100 detects activation (e.g., selection) of continue affordance 1136 (e.g., via user input 1137). In some examples where the display is touch-sensitive, user input 1337 is essentially a tap, swipe, or other gesture on the display surface relative to the continue affordance 1136. In other examples, activation of the continue affordance 1136 is activation of the affordance by a focus selector (e.g., a mouse cursor) or keyboard input.

In some examples, after completion of the registration process described above with respect to FIGS. 11B-11G, a second iteration of face registration is performed. As shown in FIG. 11L , in response to detecting activation of continue affordance 1136, device 1100 displays second face registration interface 1138. In the example of FIG. 11L , the second facial registration indicator includes a second user facial image 1140 and a second registration progress meter 1142. In some examples, second user facial image 1140 is a representation of the field of view of biometric sensor 1103 with similar visual processing as user facial image 1106 (e.g., second user facial image 1140 is a digital view This is a live preview of the image data captured by the biometric sensor 1103 displayed as a finder). In some examples, device 1100 displays a second orientation guide 1144 that overlaps (e.g., overlays) the second user facial image 1140. In the example of FIG. 11L , second orientation guide 1144 has a similar visual treatment as orientation guide 1108 (e.g., second orientation guide 1144 extends out of the plane of display 1102 in the virtual z dimension). contains multiple visible curves). In some examples, the second registration progress meter 1142 is comprised of a set of progress elements (eg, 1142a, 1142b, 1142c) spaced around the second user facial image 1140. In some examples, portions of second enrollment progress meter 1142 (e.g., meter portions 1146, 1148) optionally display specific orientations or portions of the user's face relative to biometric sensor 1103. corresponds to In some examples, some or all of the meter portions optionally include a greater number of progress elements than individual portions of the registered progress meter 1110. By way of example, each portion of the second progress meter 1142 corresponds to the same facial orientation or angular view of the user's face as the corresponding portions of the progress meter 1140 (e.g., the meter portion 1146 is shown in FIG. corresponds to the same facial orientation as meter portion 1114 in Figures 11b-11h). In some examples, upon display of second registration interface 1138, device 1100 sets the visual state of progress elements within registration progress meter 1142 to the “unregistered” state described above (e.g., first registration scan registration progress from is reset). In the example of Figure 11L, the second face registration interface also includes a text prompt 1150 instructing the user to move (e.g., rotate and/or tilt) their head in a circular motion during the second registration process. .

In some examples, as the second iteration of facial registration is performed, device 1110 may generate a second user facial image 1140, a second image 1140 in response to a change in the orientation of the user's face relative to biometric sensor 1103. Updates the displays of the progress meter 1142 and orientation guide 1144. For example, the user repeats the same (or similar) motion of his or her face as was performed in the registration of the first repetition, and device 1100 repeats the motion in the manner described above with respect to FIGS. 11B-11H ( or in a similar manner) to update the display of these elements of the second user interface 1138.

11M illustrates the second face registration interface 1128 after the second iteration of registration is complete (e.g., after image data of multiple angular views of the user's face has been captured by the biometric sensor 1103). . In the example of FIG. 11M, device 1100 has transitioned the display of all progress elements in second registered meter 1142 to the “registered” state described above. For example, the color of each progress element was changed to green to indicate successful registration. In the example of Figure 11M, device 1100 displays text prompt 1152 indicating that the second scan of the user's facial features is complete. In some examples, device 1110 issues an audio and/or tactile notification 1154 to provide additional indication that the second scan is complete. In some examples, audio and/or tactile notification 1154 is the same as tactile notification 1126 issued to indicate completion of the first scan. In some examples, the audio and/or tactile output indicating a successful second scan of the user's facial features is the same audio and/or tactile output used to indicate successful facial authentication on the device. In the example of Figure 11M, device 1100 continues to display second user facial image 1140. In some examples, second user facial image 1140 is part of a live preview of the digital viewfinder. In other examples, device 1100 displays a single (eg, still) user image captured during the registration process. In the example of Figure 11M, device 1100 stops displaying second orientation guide 1144 once the scan is complete.

In the example of FIG. 11N, after issuing a notification indicating completion of the second scan, device 1100 displays a second scan completion interface 1156. The second scan completion interface 1156 includes a user facial image 1158 and a second success status meter 1160. In the example of FIG. 11N , user facial image 1158 is blurred, faded, darkened, or otherwise obscured to indicate that additional image data is no longer being collected as part of the second facial scan. In some examples, second success status meter 1160 includes a solid line surrounding user face image 1158 that provides a visual indication that the first scan is complete (e.g., similar to success status meter 1128). , is a continuous green circle. To provide additional visual notification, the second scan complete interface 1156 also includes a text prompt 1162 (eg, a second scan complete message). The second scan completion interface 1156 also includes a continue affordance 1164. In some examples, while displaying the second scan completion interface 1156, device 1100 detects activation (e.g., selection) of continue affordance 1164 (e.g., via user input 1165). In some examples where display 1102 is touch-sensitive, activation is essentially a tap, swipe, or other gesture on the display surface relative to the continue affordance 1164. In other examples, activation of the continue affordance 1164 is activation of the affordance by a focus selector (e.g., a mouse cursor) or keyboard input. In the example of FIG. 11N , user facial image 1158 is blurred, faded, darkened, or otherwise obscured to indicate that additional image data is no longer being collected during the second facial scan.

In the example of FIG. 11O, in response to detecting activation of continue affordance 1164, device 1100 displays registration completion interface 1166. As shown in FIG. 11O, the registration completion interface 1166 includes a biometric authentication glyph 1168. For example, the biometric authentication glyph is optionally a line drawing of all or part of a face (e.g., a stylized facial graphic). In the example of FIG. 11O , enrollment completion interface 1166 also includes text prompt 1170 indicating that the enrollment process is complete and facial authentication is set up and/or enabled on the device. In some examples, enrollment completion interface 1166 also includes completion affordance 1172, activation of which causes device 1100 to terminate facial authentication setup. In some examples, registration completion interface 1166 does not include facial image 1158.

12A and 12B are flow diagrams illustrating a method for registering a user's biometric characteristics on an electronic device, according to some examples. Method 1200 includes a device having a display, one or more input devices (e.g., a touchscreen, microphone, camera), and a wireless communication radio (e.g., a Bluetooth connection, a WiFi connection, a mobile broadband connection such as a 4G LTE connection). For example, 100, 300, 500, 1100). In some examples, the display is a touch-sensitive display. In some examples, the display is not a touch-sensitive display. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the device includes one or more biometric sensors, optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device, such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors. Some operations of method 2000 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 1200 provides an intuitive method for registering a user's biometric characteristics on an electronic device. This method reduces the cognitive burden on the user for registering biometric features on the device, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to register biometric features faster and more efficiently saves power and increases time between battery charges.

The device displays 1202 a biometric registration interface (e.g., 1104) on the display. Displaying the biometric registration interface includes displaying (1204) a representation of the biometric characteristic (e.g., 1106). For example, a representation of a biometric characteristic may optionally be a face, fingerprint, iris, handprint, or other physical feature that can be used to distinguish one person from another in the field of view of one or more cameras on the device. A representation of a biometric feature (e.g., a representation of the head of the user of the device). The representation of the biometric feature may include the user's head positioned in the field of view of one or more of the cameras based on the alignment of the biometric feature with respect to one or more biometric sensors (e.g., 1103) of the device. has an orientation that is determined (based on camera data).

In some examples, the device displays 1206 a digital viewfinder (e.g., a live preview of image data including 1106) that includes a representation of the field of view of one or more cameras (e.g., 1103). For example, in some examples, the device displays a live preview of image data captured by one or more cameras. In some examples, a representation of the field of view of one or more cameras has background content removed. The background is optionally determined based on depth information captured by one or more cameras (e.g., removing background content optionally includes removing any background or simply vignetting). . In some examples, the device does not perform any background removal.

Displaying the biometric registration interface may also include a first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) at a first location on the display relative to the representation of the biometric feature (e.g., 1106). A first set of objects spaced around a representation of the biometric feature, such as a set of tick marks (e.g., 1110a, 1110b and 1110c, or 1116a and 1116b, or 1118a and 1118b), and a representation of the biometric feature. A second progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) at a second location on the display for (e.g., 1106) around a representation of a biometric feature, such as a second set of tick marks. and simultaneously displaying (1208) a progress indicator (e.g., 1110) including a second set of spaced apart objects (e.g., 1110a, 1110b and 1110c, or 1116a and 1116b, or 1118a and 1118b). . A representation of the biometric characteristic (e.g., 1106) is displayed between a first location and a second location on the display. Displaying a progress indicator during enrollment in this way encourages the user to look at the display of the electronic device during enrollment, improving the ability to detect when gaze is directed toward the display and therefore whether the user is paying attention to the device. I order it. Encouraging a user to look at the display of an electronic device improves the operability of the device (e.g., by ensuring that the user's gaze is directed toward the display and thereby ensuring that the user's biometric characteristics are properly registered) and that the user- It makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, a progress indicator may include a plurality of progress indicator portions (e.g., 1114, 1116, 1118, 1120, 1122), each progress indicator portion containing one or more progress elements (e.g., 1110a, 1110b, 1110c, 1116a, includes 1116b, 1118a, 1118b)) (1210). In some examples, the plurality of progress indicator portions include a first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) and a second progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122), wherein the plurality of progress indicator portions surround at least a portion of the representation of the biometric characteristic (e.g., 1106). In some examples, one or more of the plurality of progress indicator portions includes a plurality of respective progress elements (e.g., 1110a, 1110b, 1110c, 1118a, 1118b) (1212). In some examples, the progress indicator optionally includes a set of one or more display elements (e.g., 1110a, 1110b, 1110c, 1116a, 1116b, 1118a, 1118b) arranged around a representation of the biometric feature (e.g., 1106). do. For example, these display elements are optionally a circle of radially extending lines (“grid marks”) representing the progression of the registration lines around the user's face. These lines optionally indicate a direction where each change in orientation of the biometric feature is sufficient for registration of the biometric feature (e.g., upward to move the upper lines, even if the bottom of the biometric feature is being scanned). In some examples, the first set of lines correspond to a first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) and the second set of lines correspond to a second progress indicator portion (e.g., 1114 , 1116, 1118, 1120, or 1122). For example, a predetermined number (e.g., eight) of tick marks is associated with each portion of the progress indicator.

In some examples, the first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) is configured such that the biometric feature (e.g., the user's face) is the first progress indicator portion (e.g., 1114, 1116). , 1118, 1120, or 1122) indicates registration status (1214) of a first portion of a biometric feature that is detected by (e.g., visible to) one or more biometric sensors (e.g., 1103). . For example, the upper right portion of the progress indicator (e.g., 1120) changes appearance when the user's face is turned to the upper right portion of the device to register the lower left portion of the user's face. Likewise, in some examples, the second progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) is configured such that the biometric feature (e.g., the user's face) is connected to the first progress indicator portion (e.g., 1114). , 1116, 1118, 1120, or 1122). Indicates the registration status of the second portion of the measurement feature. For example, the lower right portion of the progress indicator (e.g., 1118) changes appearance when the user's face is turned to the lower left portion of the device to register the upper right portion of the user's face.

In some examples, displaying 1202 a biometric registration interface (e.g., 1104) further includes displaying a prompt to move a biometric feature (e.g., 1112). In some examples, the displayed prompts optionally involve tactile and/or auditory prompts. In some examples, types of responses are provided based on settings of the electronic device and/or controlled manually by the user. Providing a prompt with instructions on how to properly move the biometric feature provides feedback to the user allowing them to quickly recognize and execute the required movements, reducing the time needed to complete the registration process. . Therefore, providing improved visual prompts regarding the appropriate inputs required for biometric registration (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) ) improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the device displays a prompt (eg, 1108) indicating the direction of movement. In some examples, the prompt is an orientation guide (e.g., 1108) overlaid on a biometric registration interface (e.g., 1104). In some examples, the prompt is overlaid on a representation of the biometric characteristic (e.g., 1106). In some examples, the device overlays a three-dimensional object (e.g., 1108) over a representation of biometric features (e.g., 1106). For example, the three-dimensional object is optionally an arc that moves as the user's head rotates and extends in a virtual z-dimension. In some examples, the three-dimensional object (eg, 1108) includes a plurality of arcs extending in a virtual z-dimension (eg, two arcs that intersect each other at a point in front of the user's face). In some examples, a three-dimensional object (e.g., 1108) is highlighted as the user is moving (e.g., the three-dimensional object darkens or displays a fading trace as it moves along with movement of the biometric features), which may Emphasizes the three-dimensional object when it moves, and reduces the emphasis on the three-dimensional object compared to a representation of the biometric feature when the biometric feature is not moving.

While simultaneously displaying a representation of the biometric feature (e.g., 1106) and a progress indicator (e.g., 1110), the device detects a change in the orientation of the biometric feature relative to one or more biometric sensors (e.g., 1103) (1216).

In some examples, in response to detecting a change in the orientation of a biometric feature relative to one or more biometric sensors (1218), the device determines a change in the orientation of the biometric feature relative to one or more biometric sensors (e.g., 1103). Rotate the prompt (e.g., 1108) according to the change. In some examples, rotating the prompt includes rotating a three-dimensional object (e.g., 1108) at least partially in a virtual z-dimension of the display. Rotating an orientation guide overlaid on a representation of a biometric feature provides the user with feedback regarding the orientation of his or her biometric features relative to the device's biometric sensors in three-dimensional space, allowing the user to Allows for more quickly placing his or her biometric features through the required orientation range during the registration process. Accordingly, providing the user with improved visual feedback regarding the orientation of the biometric feature of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device) It improves operability and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, in response to detecting a change in the orientation of the biometric feature relative to the one or more biometric sensors, the device may configure the device according to the change in the orientation of the biometric feature relative to the one or more biometric sensors (e.g., 1103). Update (1220) a representation of the biometric feature (e.g., 1106). For example, in some examples, the orientation of the representation of the biometric feature (e.g., 1106) is changed regardless of whether the registration criteria are met. In some examples, the orientation of a representation of a biometric feature (e.g., 1106) is changed only if registration criteria are met. Updating the orientation of the displayed representation of the biometric features provides the user with feedback regarding the orientation of his or her biometric features relative to the device's biometric sensors, allowing the user to determine the required orientation range during the registration process. allows biometric features to be moved more quickly. Accordingly, providing the user with improved visual feedback regarding the orientation of the biometric feature of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device) It improves operability and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

Upon determining that the change in orientation of the biometric feature meets registration criteria for the first portion of the biometric feature that corresponds to the first progress indicator portion (e.g., 1114, 1116, 1118), the device displays the first progress indicator portion. Update one or more visual characteristics of the ruler portion (1222). For example, determining a change in the orientation of a biometric feature that meets registration criteria may optionally cause the image data to change to a first perspective angle (e.g., when the user's face is tilted upward). based on determining that it includes data corresponding to a first angular view of the user's face from a bottom perspective. Updating the visual state of the portion of the progress meter that corresponds to the current orientation of the biometric feature allows the user to recognize that the portion of the biometric feature is properly oriented for registration. This then instructs the user how to change the orientation of the biometric feature to register different parts corresponding to different respective parts of the progress meter, reducing the time required to complete the registration process. Accordingly, providing users with improved visual feedback regarding the registration status of biometric features (e.g., by helping users provide appropriate inputs and reducing user errors when operating/interacting with a device) It improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the registration criteria for the first portion of the biometric feature that corresponds to the first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) may be such that the first portion of the biometric feature is predetermined. and a requirement that the user's face is oriented relative to one or more biometric sensors (e.g., 1103) in a manner (e.g., the user's face is looking toward the first progress indicator portion).

In some examples, the registration criteria for the first portion of the biometric feature that corresponds to the first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) are such that the first portion of the biometric feature is not registered. Includes the requirement that

In some examples, the registration criteria for a first portion of a biometric feature that corresponds to a first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) include that the first portion of the biometric feature has one or more Includes a requirement (1224) that the orientation of the biometric sensors (e.g., 1103) be changed (e.g., rotated) by at least a threshold amount. In some examples, registration of the first portion of the biometric feature may cause the biometric feature to move (rotate) sufficient to allow the first portion to be properly captured by one or more biometric sensors (e.g., 1103). ask for something

In some examples, updating one or more visual characteristics of the first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) in a first manner based on the registration status of the first portion of the biometric feature. updating one or more visual characteristics of the first progress indicator portion (e.g., color), and in a second manner based on alignment of the biometric feature with respect to one or more biometric sensors (e.g., 1103) of the device (e.g. , size or length of progress elements) and updating one or more visual characteristics of the first progress indicator portion. For example, the first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) changes from black to green when the portion of the biometric feature corresponding to the first progress indicator portion is registered, and 1 When a portion of the biometric feature corresponding to the progress indicator portion faces one or more biometric sensors (e.g., 1103), one or more lines (e.g., 1110a, 1110b, 1110c) corresponding to the first progress indicator portion , 1116a, 1116b, 1118a, 1118b) are stretched. In some examples, updating one or more visual characteristics of the first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) in the second manner may be performed on one or more biometric sensors (e.g., 1103). It is based on the direction of change in the orientation of the biometric feature. In some examples, updating in the second manner is additionally or alternatively performed based on a rate of change of the orientation of the biometric feature relative to one or more biometric sensors. Changing the portion of the progress meter that corresponds to the current orientation of the biometric feature from the first visual state to the second visual state allows the user to quickly recognize that the portion of the biometric feature is properly oriented for registration. This may then indicate to the user how to change the orientation of the biometric feature to register different parts corresponding to different respective parts of the progress meter, reducing the time needed to complete the registration process. Accordingly, providing the user with improved visual feedback regarding the registration status of the biometric feature may be beneficial to the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). It improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the device determines the rate of change of the orientation of the biometric feature relative to one or more biometric sensors (e.g., 1103) of the first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122). Updates one or more visual properties. In some examples, updating one or more visual characteristics in this manner includes modifying the color of the first progress indicator portion based on a rate of change in the orientation of the biometric feature.

In some examples, the first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) displays a plurality of display elements (e.g., 1110a, 1110b, 1110c, 1114a, 1116a, 1116b, 1118a) in each order. , 1118b). Upon determining that the change in orientation of the biometric feature relative to one or more biometric sensors (e.g., 1103) is a change in a first direction (e.g., clockwise rotation), the device optionally selects the first end of each sequence. Starting from (e.g., starting at 1118a), the appearance of the display elements is changed. For example, the device optionally begins to extend the lines in the first progress indicator portion starting from the first side of each sequence and moving to the second side of each sequence (to 1114a). Upon determining that the change in orientation of the biometric feature relative to the one or more biometric sensors is a change in a second direction (e.g., counterclockwise rotation), the device optionally causes the device to: Starting from the second end of each sequence (e.g., starting from 1114a), the appearance of the display elements (e.g., 1110a, 1110b, 1110c, 1114a, 1116a, 1116b, 1118a, 1118b) is changed. For example, the device optionally extends the lines in the first progress indicator portion starting from the second side of each sequence and moving to the first side of each sequence (e.g., to 1118a). In some examples, a similar approach is taken when changing the appearance of the second progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) or other progress indicator portions.

In some examples, the device determines one or more visual characteristics of a first progress indicator (e.g., 1114, 1116, 1118, 1120, or 1122) from a first state (e.g., “unregistered”) to a portion of the first progress indicator. Update to a second status (eg, “enrolling”) indicating that registration criteria are met. For example, the device may display display elements within the portion of the progress indicator at which the biometric feature is currently oriented (e.g., 1114), such as the portion of the progress indicator toward which the user's face is facing (e.g., orientation 1106 in FIG. 11B). Zoom in, enlarge or change its color.

In some examples, after updating one or more visual characteristics of the first progress indicator portion, the device optionally causes the biometric characteristic to be displayed in the first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122). Detect a change in the orientation of the biometric feature relative to the one or more biometric sensors such that it no longer meets registration criteria for the first portion of the corresponding biometric feature. In response to detecting a change in the orientation of the biometric feature relative to the one or more biometric sensors, the device optionally displays one or more visual characteristics of the first progress indicator portion in a second state (e.g., “enrolling”). Update 1226 to a third state (e.g., “registered”) indicating that the first portion of the biometric characteristic has been registered but no longer meets the registration criteria. For example, the device may optionally change the appearance (e.g., color or size) of the progress indicator portions again when the user's face moves away from them, and optionally, display the first portion (e.g., 1114a) of the progress indicator. , transitioning from a “tilted” appearance to a registered appearance when the user orients the biometric feature away from that portion of the progress indicator (e.g., orientation of 1106 in FIG. 11D ). One visual characteristic (e.g., color) of the progress indicator may optionally indicate registration status (e.g., blue for "tilted", green for "registered", gray for "unregistered"). Other visual characteristics (eg, length of lines) indicate the direction of orientation of the biometric feature. Progress is optionally advanced around a progress indicator (e.g., 1110) based on the speed and direction of change in tilt. For example, progress indicator lines (e.g., 1110a, 1110b, 1110c, 1114a, 1116a, 1116b, 1118a, 1118b) may optionally bulge and/or bulge based on the speed and direction of movement of the biometric feature. Changes color based on the speed and direction of movement of the measured feature. Changing the portion of the progress meter corresponding to the current orientation of the biometric feature from the second visual state to the third visual state allows the user to quickly recognize that the portion of the biometric feature has been successfully registered. This also indicates to the user that they no longer need to move the biometric feature in that orientation during the registration process, which directs the user's attention to registering other parts of the biometric feature as needed to complete the registration process. reduce time. Accordingly, providing users with improved visual feedback regarding the registration status of biometric features (e.g., by helping users provide appropriate inputs and reducing user errors when operating/interacting with a device) It improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

Upon determining that the change in orientation of the biometric feature satisfies registration criteria for the second portion of the biometric feature corresponding to the second progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122), the device Updates one or more visual characteristics of the second progress indicator portion (1228). For example, determining a change in the orientation of a biometric feature that meets registration criteria may optionally cause the image data to be viewed at a second different time (e.g., when the user's face is tilted to the right, such as when the user's face is tilted to the right). ). In some examples, updating the visual characteristics of the second progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) optionally involves updating the visual characteristics of the first progress indicator portion as described above. Includes some or all of the steps provided. Updating the visual state of the second portion of the progress meter corresponding to the current orientation of the biometric feature allows the user to recognize that the second portion of the biometric feature is properly oriented for registration. This then instructs the user how to change the orientation of the biometric feature to register different parts corresponding to different respective parts of the progress meter, reducing the time required to complete the registration process. Accordingly, providing users with improved visual feedback regarding the registration status of biometric features (e.g., by helping users provide appropriate inputs and reducing user errors when operating/interacting with a device) It improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, upon determining that registration completion criteria have been met (e.g., all portions of the biometric feature have been registered, 1110 in Figure 11E), the device may provide an indication that registration of the biometric feature is complete (e.g., 1124, 1126). ) is output.

For example, the device optionally updates one or more visual characteristics of a progress indicator (e.g., 1110) (e.g., a plurality of progress indicator display elements (e.g., 1110a, 1110b, 1110c, 1114a, 1116a, 1116b, 1118a, 1118b) are merged into a continuous shape such as a circle). In some examples, before detecting a change in the orientation of a biometric feature relative to one or more biometric sensors, a first progress indicator portion (e.g., 1114, 1116, 1118, 1120, or 1122) and a second progress indicator The portions (eg, 1114, 1116, 1118, 1120, or 1122) are visually separate. In this case, updating one or more visual characteristics of the progress indicator includes visually merging the first progress indicator portion and the second progress indicator portion.

In some examples, the device modifies the representation of the biometric characteristic (e.g., 1106, 1132). In some examples, the representation of the biometric feature is blurred, faded, and/or darkened to indicate that additional information about the biometric feature is no longer being collected as part of the registration process, and/or Otherwise it becomes ambiguous.

In some examples, the device displays a confirmation affordance (e.g., 1136, 1164), and selection of the confirmation affordance causes the electronic device to display a completion interface (e.g., 1166). In some examples, the device displays a simulation of a representation of the biometric characteristic (e.g., 1168). In some examples, the simulation of the representation of the biometric feature is two-dimensional. In some examples, the simulation of the representation of the biometric feature is three-dimensional.

In some examples, the device outputs an indication (e.g., 1126, 1154, 1122, 1162, 1170) (e.g., a tactile output) that the registration process is complete. In some examples, the device outputs a tactile output (e.g., 1126, 1154) indicating successful registration of the biometric feature. In some examples, the tactile output indicating successful registration of a biometric feature is the same as the tactile output used to indicate successful authentication by the biometric feature.

In some examples, after outputting an indication that registration of the biometric feature is complete, the device displays a second biometric registration interface (e.g., 1138). In some examples, after completion of the first registration, a second iteration of registration is performed. Registration of this second iteration is optionally performed in response to selection of an affordance (e.g., 1136). Performing a second scan of the user's biometric features allows the device to capture additional biometric data corresponding to different orientations or positions of the biometric features that may not have been recorded during the first iteration of registration. . Accordingly, performing a second scan of the user's biometric features may allow for more efficient and/or secure biometric authentication at the device (e.g., ensure that the user has appropriate information when operating/interacting with the device). improves the operability of the device (by helping provide inputs and reducing user errors) and makes the user-device interface more efficient, which additionally reduces the device's power by allowing users to use the device more quickly and efficiently. Reduces usage and improves battery life.

At the second biometric registration interface, the device displays a second representation 1140 of the biometric characteristics. The second representation of the biometric feature optionally has an orientation determined based on the alignment of the biometric feature with respect to one or more biometric sensors (e.g., 1103) of the device. In some examples, the second representation of the biometric feature is a representation of the field of view of one or more cameras with similar visual processing as the first representation of the biometric feature (e.g., 1106).

In the second biometric registration interface, the device displays a third progress indicator portion (e.g., around the representation of the biometric feature, such as 1146) at a first location on the display relative to the representation of the second biometric feature (e.g., 1140). a first set of objects spaced apart from each other), and a fourth progress indicator portion at a second location on the display relative to a second representation of the biometric feature (e.g., 1148). simultaneously displaying a second progress indicator (e.g., 1142) including a second set of peripherally spaced objects. The second representation of the biometric characteristic is displayed between the third and fourth positions on the display. In some examples, the third progress indicator portion corresponds to the same portion of the biometric feature as the first progress indicator portion (e.g., 1114). In some examples, the fourth progress indicator portion corresponds to the same portion of the biometric feature as the second progress indicator portion (e.g., 1118).

In some examples, the registration status of the first progress indicator portion (e.g., 1114, 1116, 1118, 1120 or 1122) does not correspond to the registration status of the third progress indicator portion (e.g., 1146 or 1148).

In some examples, the first progress indicator portion of the progress indicator includes a first number of progress elements (e.g., 1114a) and the third progress indicator portion of the second progress indicator includes a different than the first number (e.g., , larger) of the progression elements (e.g., 1142a, 1142b, 1142c). In some examples, a plurality (or all) of the progress indicator portions (e.g., 1146, 1148) in the second progress indicator (e.g., 1142) correspond to a corresponding progress indicator portion (e.g., 1110) in the first progress indicator (e.g., 1110). Contains more progression elements than parts (e.g., 1114, 1118).

In some examples, while simultaneously displaying a second representation of a biometric feature (e.g., 1140) and a second progress indicator (e.g., 1142), the device performs biometric measurement on one or more biometric sensors (e.g., 1103). A second change in orientation of the feature is detected. In response to detecting a second change in the orientation of the biometric feature relative to the one or more biometric sensors, and in accordance with a determination that the change in orientation of the biometric feature meets registration criteria for the first portion of the biometric feature , the device updates one or more visual characteristics of the third progress indicator portion (e.g., 1146 or 1148). Upon determining that the change in orientation of the biometric feature meets registration criteria for the second portion of the biometric feature, the device updates one or more visual characteristics of the fourth progress indicator portion (e.g., 1146 or 1148). For example, to advance through the second registration step of the registration process, the user may change the orientation of the biometric feature that was used to register the biometric feature in the first registration step of the registration process. Repeat step by step. Accordingly, updating the visual properties of the third and fourth progress indicator portions may optionally include those described above with respect to the first and second progress indicator portions (e.g., 1114, 1116, 1118, 1120 or 1122). It includes similar steps.

In some examples, after detecting a second change in the orientation of the biometric feature relative to one or more biometric sensors (e.g., 1103), and following a determination that the second set of registration completion criteria are met (e.g., biometric All portions of the feature have been registered), the device outputs a second indication (e.g., 1162, 1163) that registration of the biometric feature is complete. In some examples, registration does not actually occur; Rather, the process is visually emulated. In some examples, the second indication is a visual, auditory and/or tactile output (e.g., 1163) indicating that registration of the biometric feature is complete. In some examples, the second indication is the same as the indication provided in accordance with the determination that the first set of registration completion criteria has been met (e.g., 1126).

Note that the details of the processes described above with respect to method 1200 (e.g., FIGS. 12A and 12B) are also applicable in a similar manner to the methods described herein. For example, method 1200 optionally includes one or more of the features of the various methods described herein with reference to methods 800, 1000, 1400, 1600, 1800, 2000, 2200, 2500, and 2700. Includes. For example, a facial registration verification interface as described in method 1000 may be applied in conjunction with a facial registration user interface (e.g., 1104). For another example, hints as described in method 1400 may be applied in connection with a facial registration user interface (e.g., 1104). For another example, transitioning a progress meter as described in method 800 may be applied in conjunction with a registered progress meter (e.g., 1110). For brevity, these details are not repeated below.

Operations in the information processing methods described above optionally utilize one or more functional modules within an information processing device, such as general-purpose processors (e.g., as described with respect to FIGS. 1A, 3, and 5A) or application-specific chips. It is implemented by running it. Additionally, the operations described above with reference to FIGS. 12A and 12B are optionally implemented by components shown in FIGS. 1A and 1B. For example, display operation 1202, display operation 1208, detection operation 1216, update operation 1222, and update operation 1224 may optionally include event classifier 170, event recognizer 180, and , and is implemented by the event handler 190. Event monitor 171 in event sorter 170 detects contact on touch-sensitive surface 604, and event dispatcher module 174 passes event information to application 136-1. The respective event recognizer 180 of application 136-1 compares the event information to the respective event definition 186 and determines that the first contact at the first location on the touch-sensitive surface corresponds to the selection of an object on the user interface. Determines whether it corresponds to the same predefined event or subevent. When an individual predefined event or sub-event is detected, event recognizer 180 activates an event handler 190 associated with detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some examples, event handler 190 accesses the respective GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes may be implemented based on the components shown in FIGS. 1A and 1B.

13A-13R show on an electronic device (e.g., device 100, device 300, device 500, device 700, device 900, or device 1100), according to some examples. Illustrative example user interfaces for registering biometric features. User interfaces in these figures are used to illustrate the processes described below, including the processes in FIG. 14.

FIG. 13A illustrates an electronic device 1300 (e.g., portable multifunction device 100, device 300, device 500, device 700, device 900, or device 1100). In the illustrative example illustrated in FIGS. 13A-13R , electronic device 1300 is a smartphone. In other examples, electronic device 1300 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 1300 has a display 1302, one or more input devices (e.g., a touchscreen, buttons, microphone) of display 1302, and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the electronic device includes one or more biometric sensors (e.g., biometric sensor 903), optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, one or more biometric sensors 1303 are one or more biometric sensors 703. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

As illustrated in FIG. 13A , device 1300 displays face registration user interface 1304 on display 1302 . In some examples, face registration user interface 1304 is displayed after device 1300 detects successful alignment of the user's face to its one or more cameras, as described above with respect to FIGS. 9A-9Y. . In some examples, face registration interface 1304 has similar visual characteristics to face registration interface 1104 described above with respect to FIG. 11A. Face registration interface 1304 includes a user face image 1306. In some examples, user facial image 1306 is an image of a user captured by one or more cameras on device 1300 (eg, biometric sensor 1303). For example, the user facial image 1306 may optionally be a live image data captured by one or more cameras (e.g., a digital viewfinder) that is continuously updated as the camera's field of view and/or the contents of the field of view change. This is a preview. In some examples, background content is removed so that only the user's face is visible in facial image 1306. Facial registration interface 1304 also includes an orientation guide 1308 that is superimposed (e.g., overlaid) on user facial image 1106. As described above with respect to FIGS. 7I-7K , the orientation guide 1308 optionally extends in the virtual z-dimension (e.g., along an axis perpendicular to the plane of the display) and above the center of the user's facial image 1306. It is a set of intersecting curves (e.g., crosshairs). Accordingly, the curves of the orientation guide 1308 optionally appear to bulge outward relative to the plane of the display 1302, providing a sense of the position of the user's head in three-dimensional space.

Facial enrollment user interface 1304 includes an enrollment progress meter 1310. Enrollment progress meter 1310 includes a set of display elements (eg, progress elements 1310a, 1310b, 1310c) arranged around a user facial image 1306 and an orientation guide 1308. In the example of FIG. 13A , the progressive elements are a set of lines arranged in a circular pattern and extending radially outward from the user face image 1306. In some examples, progressive elements 1310a, 1310b, 1310c, etc. represent the orientation of the user's face needed to register corresponding facial features. For example, progressive elements within the upper portion of registration meter 1310 may optionally cause one or more cameras on device 1300 to capture image data of the underside of the user's face when the user's head is tilted upward. moves, fills, stretches, and/or changes color (e.g., in the manner of FIGS. 11B-11H). In the example of FIG. 13A , device 1310 displays progress elements in registered progress meter 1310 as unregistered (e.g., they are grayed out).

In the example of FIG. 13A , facial enrollment interface 1304 instructs the user to begin moving their face relative to the device to advance the enrollment progress meter 1310 (e.g., to enroll their facial features). Includes text prompt 1312. In some examples, device 1300 displays text prompt 1312 before any portion of the user's face is registered.

While displaying face registration interface 1304, device 1300 detects criteria (eg, hints) for displaying registration prompts. In some examples, the enrollment prompt criteria include a requirement that the user's face has moved less than a first threshold amount in a predetermined period of time, as determined by biometric sensor 1303.

In some examples, in response to detecting that these registration hint criteria are met, device 1300 displays audio hint enabling interface 1314, as shown in FIGS. 13B and 13C. In the example of FIG. 13B, hint enabling interface 1314 includes a text prompt 1316 informing the user of the option to enable or disable audio hints. Accordingly, in some examples, hint enabling interface 1314 includes YES affordance 1318 and NO affordance 1320. In some examples, in response to activation (e.g., selection) of No affordance 1320, device 1300 displays face registration interface 1304 again so that the user can access it without the prompts and/or hints described below. Proceed with registration of his or her facial features. However, in the example of Figure 13C, device 1300 detects activation (e.g., selection) of yes affordance 1310. In some examples, the activation is a user input (eg, a tap or swipe gesture) at the contact area 1322.

In response to detecting activation of yes affordance 1320, device 1300 displays hint-enabled registration interface 1324, for example, as illustrated in the example of FIG. 13D. In some examples, hint-enabled registration interface 1324 or one or more of the prompts described below may indicate whether the user has audio hints enabled (e.g., that the user's face has not moved sufficiently in a predetermined period of time). (in response to detecting something) is displayed. In the example of FIG. 13D , hint-enabled registration interface 1324 includes a user face image 1326 that has similar or identical visual characteristics as user face image 1306. For example, in some examples, user facial image 1326 is an image of a user captured by one or more cameras on device 1300 (eg, biometric sensor 1303). For example, the user facial image 1326 may optionally be image data captured by a biometric sensor 1303 that is continuously updated as the camera's field of view and/or the contents of the field of view change (e.g., a digital viewfinder). This is a live preview of . In some examples, hint-enabled registration interface 1324 includes a visual movement prompt 1328 that is optionally overlaid (e.g., overlapped) on the user facial image 1326. In the example of Figure 13D, visual movement prompt 1328 requires the user to move (e.g., rotate and/or tilt) his or her face to register a corresponding portion of the face (e.g., angular view). Contains an arrow element indicating the requested direction. In some examples, visual movement prompt 1328 is partially transparent so that underlying user face image 1326 is also visible. In the example of FIG. 13D , the arrow element of the visual movement prompt 1328 directs the user to move their face to the right (e.g., toward the right portion of the registration progress meter 1330 to register an angular view of the left side of the user's face). ) Instructs to move (e.g., rotate, tilt, or turn).

In some examples, device 1300 displays text prompts 1332 that provide written instructions to the user that match visual movement prompts 1328. In the example of Figure 13D, text prompt 1332 provides a written command to the user to turn their head to the right (e.g., in the same direction indicated by the arrow element in visual prompt 1328). In some examples, device 1300 also issues audio output 1334 that corresponds to visual movement prompt 1328 and/or text prompt 1328. For example, if a screen reader function is enabled, audio output 1334 is a verbal description of the requested movement (e.g., an audible mention of text prompt 1332). In some examples, audio output 1334 is issued instead of or in addition to visual movement prompt 1328 and/or text prompt 1332. In some examples, device 1300 also issues tactile output 1336 (e.g., vibration) (e.g., instead of or in addition to audio output 1334). In some examples, audio output 1334 and/or tactile output 1336 matches the movement (e.g., animation) of visual movement prompt 1328, as described in more detail below.

In some examples, device 1300 displays an animation of visual movement prompt 1328 to provide additional indication of the requested movement. In the examples of FIGS. 13D and 13E , device 1300 transitions the display of the arrow element of visual prompt 1328 in the requested direction of movement (e.g., to the right). In some examples, visual prompt 1328 also includes one or more lines (eg, arcs) extending across a central portion of user face image 1326. In some examples, these lines appear to extend out of the plane of display 1302 in an imaginary z-dimension (eg, perpendicular to the display). In the examples of FIGS. 13D and 13E , device 1300 rotates an arc in the requested direction of movement (e.g., to the right) to provide visual proof of the requested movement in three dimensions involving movement of the arrow element. I order it. In some examples, device 1300 continues to display text prompt 1332 while displaying the animation of visual movement prompt 1328. In some examples, device 1300 may provide audio output 1334 and/or tactile output 1336 while displaying an animation, such that these outputs correspond to movement of arrow and/or arc elements of visual prompt 1328. Ham - is issued.

In some examples, while displaying visual movement prompt 1328 and/or text prompt 1332, device 1300 determines (e.g., for a second time) the orientation of the user's face relative to biometric sensor 1303. Detect that it has not changed during this predetermined period of time. In response, device 1300 issues tactile output (e.g., tactile output 1338 shown in Figure 13E). In some examples, tactile output 1338 is generated as an error to indicate that face registration has stopped (e.g., because the user has not moved his or her face for a predetermined amount of time).

In the example of Figure 13F, in response to detecting that the orientation of the user's face has not changed for a predetermined period of time, device 1300 may perform a second set of prompts for the user to move his or her face in a different direction. Displays registration hints. In the example of Figure 13F, device 1300 displays a second visual movement prompt 1340. The second visual movement prompt 1340 has similar visual characteristics to the visual movement prompt 1328, but corresponds to a second requested direction of movement that is different from the visual prompt 1328 (e.g., up instead of to the right). For example, second visual movement prompt 1340 includes a second arrow element pointing in a different direction than the arrow element of visual movement prompt 1328 (eg, up instead of to the right). Additionally, in some examples, the second visual movement prompt 1340 includes an arc element similar to the arc element of visual prompt 1328, which is similar to the arc element of the requested second movement prompt 1328, as described below with respect to FIGS. 13F and 13G. Used to provide visual proof of direction of movement.

In some examples, the second set of registration hints includes a text prompt 1342 that provides the user with written instructions that match the visual movement prompt 1340. In the example of FIG. 13F , text prompt 1342 provides a written command to the user to tilt their head upward (e.g., in the second direction indicated by the arrow element of second visual prompt 1340). In the example of FIG. 13F , device 1300 also issues audio output 1344 corresponding to the second visual movement prompt 1340 and/or text prompt 1342. For example, if a screen reader function is enabled, audio output 1344 is a verbal description of the requested movement (e.g., an audible mention of text prompt 1342). In some examples, device 1300 issues tactile output 1346 (e.g., vibration) (e.g., instead of or in addition to audio output 1334).

As illustrated in FIGS. 13F and 13G , in some examples, device 1300 displays an animation of visual movement prompt 1340 to provide additional indication of the second requested direction of movement. In the examples of FIGS. 13F and 13G , device 1300 transitions the display of the arrow element of second visual prompt 1340 in the second requested direction of movement (e.g., upward). In the examples of FIGS. 13F and 13G , the animation also moves in a second requested direction of movement (e.g., upward into the plane of the display) to provide visual proof of the requested movement in three dimensions accompanying movement of the arrow element. ) Rotate the arc element of the second visual prompt 1340. In some examples, device 1300 continues to display text prompt 1340 while displaying an animation of visual movement prompt 1340. In some examples, device 1300 may provide audio output 1344 and/or tactile output 1346 while displaying an animation, such that these outputs correspond to movement of arrow and/or arc elements of visual prompt 1340. Ham - is issued.

Referring now to FIG. 13H , device 1300 may detect a change in the orientation of the user's face relative to biometric sensor 1303 (e.g., when the user moves his or her face upward, i.e., in the second requested direction of movement). Detects whether the device is tilted or tilted. In response to detecting a change in orientation, the device (e.g., for a second time) displays facial registration interface 1304, described above with respect to FIG. 13A. In the example of Figure 13H, device 1300 updated user facial image 1306 to reflect a change in orientation of the user's face (e.g., displayed its movement). In some examples, orientation guide 1308 tracks (e.g., moves with) the movement of user face image 1306 to visually highlight tilt and rotation movements of the user's face in three dimensions. For example, the center (e.g., intersection) of the orientation guide 1308 is optionally positioned at a central point on the user face image 1306 and moves with it. In some examples, device 1300 also adjusts the curvature of the lines comprising orientation guide 1308 to give the appearance of three-dimensional rotation (e.g., upward into the plane of the display). In some examples, device 1100 highlights orientation guide 1108 while it is moving (eg, while the orientation of the user's face is changing). For example, device 1300 optionally displays a fading trace as it tracks the movement of the user's face and/or darkens orientation guide 1308 while moving. In this case, device 1300 optionally reduces this emphasis on orientation guide 1308 relative to user face image 1306 when the user's face is not moving.

As shown in the example of Figure 13G, in response to detecting that the user's face is oriented toward the progress meter portion 1348 (e.g., a set of one or more progress elements, such as 1310a, 1310b, 1310c), the device ( 1300 updates the display of progress elements within meter portion 1348 to a “registering” state by changing the appearance of the progress elements within meter portion 1348. For example, device 1300 optionally magnifies and/or changes the color of progressive elements within meter portion 1348 while the user's face is oriented toward meter portion 1348. In some examples, device 1300 stretches the progress ticks and changes their color from gray to blue when updating progress elements to a “registering” state. In some examples, changing the display of ongoing elements to the “enrolling” state in this manner indicates that device 1300 is capturing facial imaging data for an angular view that corresponds to the current orientation of the user's face (e.g. , registered). In the example of FIG. 13G , device 1300 leaves progress elements within progress meter 1310 (e.g., progress elements that are not part of meter portion 1348) in an unregistered state (e.g., grayed out). , indicating that device 1300 has not yet detected the user's face in orientations corresponding to those progressive elements. In some examples, the display of meter portion 1348 is updated in this manner only when the user's face is sufficiently rotated toward meter portion 1348 (e.g., when the user's face is rotated at least a threshold amount or a threshold angle). do.

Referring now to the example of FIG. 13I , device 1300 determines that the user's face is no longer in an orientation corresponding to meter portion 1348 (e.g., the user has tilted their head downward back to a neutral position). ) is detected. In response, device 1300 changes the appearance of the progress elements within meter portion 1348 back to the “registered” state. In the example of Figure 13I, device 1300 updates the display of the progress ticks in portion 1348 from the extended “registering” state by again shortening the progress ticks and changing their color. For example, progression elements in the “registered” state are the same length and/or size as progression elements in the “unregistered” state, but do not cover a corresponding portion of the user's face (e.g., as described above with respect to Figure 11J). The angular view captured in 13j) is displayed in green to indicate that it has been successfully registered. In the example of Figure 13J, device 1300 leaves other progress elements in registration progress meter 1310 as unregistered, indicating that the device has not yet detected the user's face in the orientation corresponding to those progress elements. . In response to detecting a change in facial orientation, device 1300 also moves orientation guide 1308, causing it to track the movement of the user's facial image 1306 in the digital viewfinder.

Referring now to the example of FIG. 13J , after detecting the change in orientation shown in FIG. 13I , the device determines the orientation of the user's face relative to the biometric sensor 1303 for a predetermined period of time (e.g., for a second period of time). Detect that it has not changed for a while. In response, device 1300 displays a hint-enabled registration interface 1350. In some examples, hint-enabled registration interface 1350 is automatically displayed. In some examples, hint-enabled registration interface 1350 is responsive to detecting activation (e.g., selection) of an affordance (e.g., similar to example affordance 1318 on hint-enabled interface 1314). It is displayed. In some examples, hint-enabled registration interface 1350 and its components (e.g., user facial expression 1352, registration progress meter 1354, visual movement prompt 1356, and text prompt 1358) It has the same visual characteristics as described above with respect to the hint enabling interface 1324 in Figure 13D. However, in the example of FIG. 13J, device 1300 displays progress elements within meter portion 1360 of registered progress meter 1354 as “registered,” which corresponds to the face of the same portion of progress meter 1330. This is because the orientation has already been registered (e.g. in the manner of Figure 13h).

In the examples of FIGS. 13J and 13K , device 1300 displays an animation of a visual movement prompt 1356 that prompts the user to move his or her face to an orientation that has not yet been registered. For example, the animation of visual prompt 1356 prompts the user to move his or her face in a first requested direction (eg, to the right). The animation of visual movement prompt 1356 has similar or identical characteristics to the animation of visual movement prompt 1328 described above with respect to FIGS. 13D and 13E. For example, device 1300 transitions the display of the arrow element of visual prompt 1356 to the requested direction of movement (e.g., to the right) that corresponds to a facial orientation that has not yet been registered. In the examples of FIGS. 13J and 13K , the animation also includes a visual prompt 1356 in the requested direction of movement (e.g., to the right) to provide visual proof of the requested movement in three dimensions involving movement of the arrow element. Rotate the arc element of . In some examples, while displaying an animation of visual movement prompt 1356, device 1300 continues to display text prompt 1358, which provides a written description of the requested movement. In some examples, device 1300 may provide audio output 1362 and/or tactile output 1364 while displaying an animation, such that these outputs correspond to movement of arrow and/or arc elements of visual prompt 1340. Ham - is issued.

Referring to the example of FIG. 13L , device 1300 has detected that the orientation of the user's face relative to biometric sensor 1303 has not changed for a predetermined period of time (e.g., during a third period of time). In the example of FIG. 13L, in response to detecting little to no change in the user's facial orientation, device 1300 displays accessibility registration interface 1368. In some examples, the accessibility registration interface includes user facial image 1370, which optionally has similar or identical characteristics as user facial image 1308. In particular, user facial image 1370 is optionally a live preview of image data captured by biometric sensor 1303. In the example of FIG. 13L , accessibility registration interface 1368 includes registration progress meter 1372 , which is optionally displayed surrounding user facial image 1370 . In some examples, the display of meter portion 1370 (e.g., while device 1300 was displaying registration interface 1304 or hint-enabled registration interfaces 1324 and/or 1350 during a previous stage of registration) Indicates orientations and/or portions of the user's face that were previously registered. For example, device 1300 displays progress elements within portion 1374 of progress meter 1370 (which corresponds to meter portions 1348 and/or 1360). In the example of Figure 13L, accessibility registration interface 1368 also includes accessibility options affordance 1378. In some examples, activation of accessibility option affordance 1378 allows a user to set up biometric (e.g., facial) authentication with only a partial scan (e.g., after registering only a subset of facial orientations or portions that are registered during a full scan). Let's do it.

In the example of FIG. 13M , device 1300 detects activation (e.g., selection) of accessibility option affordance 1378 (e.g., by user input 1380). In response to detecting activation of accessibility option affordance 1378, device 1300 displays completion affordance 1382 on the accessibility registration interface, as illustrated in FIG. 13N. In some examples, activation of the completion affordance causes the device to proceed using only a partial scan of their facial features.

In the example of Figure 13O, device 1300 detects activation (e.g., selection) of completion affordance 1382 through user input 1384. In response to detecting activation of the completion affordance, device 1300 displays partial scan acknowledgment interface 1386, as illustrated in FIG. 13P. The partial scan acknowledgment interface includes a user facial image 1387, which optionally has some or all of the visual characteristics of the user facial image 1370. Because a portion of the user's face has been successfully registered, device 1300 also displays a registration success indicator 1388, for example, proximate to and/or surrounding user face image 1387. In the example of FIG. 13P, partial scan acknowledgment interface 1386 includes a text prompt 1389 that provides a written indication that image data corresponding to at least a portion of the user's face has been successfully captured and registered. In the example of FIG. 13P, device 1300 displays registration completion affordance 1390.

In the example of Figure 13q, device 1300 detects activation (e.g., selection) of registration completion affordance 1390 through user input 1392. In some examples, in response to detecting activation of the registration completion affordance, device 1300 registers image data of one or more angular views (e.g., orientations) of the user's face captured during the registration process described above. Optionally, device 1300 issues tactile output 1393 to acknowledge completion of the registration process. In some examples, tactile output 1393 is the same tactile output issued in response to a successful biometric authorization at device 1300. In the example of Figure 13q, device 1300 replaces the display of success indicator 1388 with a partial registration indicator 1391 that approximates facial image 1387, which visually reflects the orientations of the successfully registered user's face. It is expressed as In some examples, the size (e.g., arc length) and location of portion registration indicator 1391 corresponds to portions of the progress meter (e.g., 1310, 1354, 1372) that transitioned to the “registered” state during registration. In the example of FIG. 13Q , device 1300 displays a partial registration indicator 1391 at a similar location to meter portion 1374 to indicate that one or more facial orientations corresponding to meter portion 1374 have been successfully registered. do.

In the example of Figure 13R, in response to detecting activation (e.g., selection) of registration completion affordance 1390 (e.g., by user input 1392), device 1300 displays registration completion interface 1394. do. As shown in FIG. 13R, the registration completion interface 1394 includes a biometric authentication glyph 1395. For example, biometric authentication glyph 1395 is optionally a line drawing of all or part of a face (e.g., a stylized facial graphic). In the example of Figure 13R, enrollment completion interface 1394 also includes a text prompt 1396 indicating that the enrollment process is complete and facial authentication is set up and/or enabled on the device. In some examples, enrollment completion interface 1394 also includes completion affordance 1397, the activation of which causes device 1300 to terminate facial authentication setup. In some examples, registration completion interface 1394 does not include facial image 1387.

14 is a flow diagram illustrating a method for providing hints for effectively registering a biometric characteristic on an electronic device, according to some examples. Method 1400 includes a device having a display, one or more input devices (e.g., a touchscreen, microphone, camera), and a wireless communication radio (e.g., a Bluetooth connection, a WiFi connection, a mobile broadband connection such as a 4G LTE connection). For example, 100, 300, 500, 1300). In some examples, the display is a touch-sensitive display. In some examples, the display is not a touch-sensitive display. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the device includes one or more biometric sensors, optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device, such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors. Some operations of method 2000 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 1400 provides an intuitive method for providing hints for effectively registering biometric features on an electronic device. This method reduces the user's cognitive burden of registering biometric features on the device, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to register biometric features faster and more efficiently saves power and increases time between battery charges.

The device may be configured to register, on the display, a biometric feature (e.g., a user's face, fingerprint, iris, palm print, or other physical biometric feature that can be used to distinguish one person from another). Displays (1402) a registration user interface (e.g., 1304, 1324). Displaying the biometric registration user interface includes displaying a representation of the biometric feature (e.g., 1306, 1326, a representation of the head of a user of the device). The appearance of the representation of the biometric feature changes 1404 as the orientation of the biometric feature relative to one or more biometric sensors (e.g., 1303) changes. For example, the orientation of the biometric feature relative to one or more biometric sensors may optionally include alignment of the user's face in image data captured by one or more cameras (e.g., in the field of view of one or more of the cameras). based on camera data including the user's head being positioned. Displaying a preview of the image captured by the biometric sensors provides the user with feedback regarding the location and orientation of his or her biometric features relative to the device's biometric sensors, allowing the user to allowing him or her to properly align his or her biometric features with sensors more quickly and efficiently to properly register them. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

While displaying the biometric enrollment user interface, the device detects that enrollment prompt criteria have been met with respect to one or more portions of the biometric features (1406).

In some examples, the enrollment prompt criteria include a requirement that the biometric characteristic moves below a first threshold amount (as determined by one or more biometric sensors) for at least a first threshold period of time (1408). Automatically enabling registration hints after detecting little to no movement of the user's biometric features reduces the time required to complete the registration process, allowing the required movements to be performed quickly and automatically. This is because the struggling user receives instructions on how to proceed with the registration process. Performing an optimized set of actions when a set of conditions are met without requiring additional user input (e.g., helping the user provide appropriate inputs when operating/interacting with a device and preventing user mistakes) (by reducing the device's operability and making the user-device interface more efficient), which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently .

In response to detecting that the enrollment prompt criteria have been met with respect to one or more portions of the biometric feature, the device may display a respective prompt to move the biometric feature in a respective manner (e.g., 1328, 1332, 1334, 1336, 1340, 1342, 1344, 1346 (e.g., visual, auditory, and/or tactile prompts) are output (1410). Each prompt is selected 1412 based on the registration status of one or more portions of the biometric feature (e.g., whether the first portion and/or the second portion of the biometric feature is registered). In particular, upon determining that registration prompt criteria have been met with respect to a first portion of a biometric feature that can be enrolled by moving the biometric feature in a first manner, the device prompts the device to move the biometric feature in the first manner. (e.g., 1328, 1332, 1334, 1336) is output (1424). Upon determining that enrollment prompt criteria have been met with respect to a second portion of the biometric feature that can be enrolled by moving the biometric feature in a second manner that is different from the first manner, outputting the respective prompt may be performed in a second manner. and outputting (1426) a prompt to move the biometric feature (e.g., 1340, 1342, 1344, 1346). Providing visual and/or audible prompts to move a biometric feature in a specific direction allows the user to quickly and intuitively realize how to position the biometric feature so that the corresponding portion can be registered. These prompts allow the user to move the biometric feature through the range of orientations required for the registration process more quickly and efficiently than they otherwise would. Accordingly, providing improved visual and/or auditory feedback with instructions for appropriate movements of biometric features (e.g., assisting the user to provide appropriate inputs when operating/interacting with the device) (by reducing user errors) and making the user-device interface more efficient, which in addition reduces the device's power usage and battery life by enabling users to use the device more quickly and efficiently. improves

In some examples, upon determining that the audible prompt criteria are met (e.g., determining whether a screen reader feature of the device is enabled), the device may display (e.g., instead of or in addition to one or more visual prompts) Outputs an audible prompt (e.g., 1334) to move the biometric feature in manner 1. In some examples, upon determining that the auditory prompt criteria are not met, the device provides the user with an option to enable auditory prompts for biometric registration. For example, the device displays an affordance (e.g., 1318) that, when selected by the user, enables audible prompts, or describes steps for enabling audible prompts for biometric registration. An audio prompt (e.g., 1334, 1344) is provided. Providing audible commands to move a biometric feature in a specific direction allows the user to quickly and intuitively realize how to position the biometric feature so that the corresponding portion can be registered. These prompts allow the user to move the biometric feature through the series of orientations required for the registration process more quickly and efficiently than they otherwise would. Therefore, providing improved auditory feedback with instructions for appropriate movements of biometric features helps the user provide appropriate inputs (e.g., when operating/interacting with the device) and reduces user errors. improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the device may display each prompt (e.g., 1328, 1332, 1334, 1336, 1340, 1342, 1344, 1346, e.g., visual, auditory, and/or outputs a tactile prompt. For example, each prompt optionally indicates that the user must begin tilting their head to begin the registration process (1422).

In some examples, the device may display each prompt (e.g., 1328, 1332, 1334, 1336, 1340, 1342, 1344, 1346, e.g., visual, auditory, and/or tactile) after at least a portion of the biometric feature is registered. outputs an enemy prompt). For example, the prompt optionally indicates that the user must continue to tilt their head to continue the registration process. Automatically issuing prompts to move a biometric feature in a second direction after the user has moved the biometric feature in a first direction allows the user to continue moving the biometric feature to proceed with the enrollment process. Allows quick and intuitive understanding. Helping the user understand how to sequentially execute the required movements of biometric features reduces the time required to complete registration of biometric features. Accordingly, performing an optimized set of actions when a set of conditions are met without requiring additional user input (e.g., assists the user in providing appropriate inputs when operating/interacting with the device and allows the user to (by reducing mistakes) and making the user-device interface more efficient, which additionally reduces the device's power usage and battery life by enabling users to use the device more quickly and efficiently. improve it

In some examples, the device outputs a tactile output (eg, 1336, 1346). In some examples, tactile output is accompanied by auditory output (eg, 1334, 1344). In some examples, tactile output and/or audio output is generated to coincide with movement of a visual prompt (e.g., 1328, 1340). For example, the tactile outputs optionally correspond to movement of an arrow or arc (e.g., arrow elements and arc elements at 1328 and/or 1340) in the direction in which the user is being prompted to move the biometric feature. .

In some examples, each prompt includes a tactile output (eg, 1338, 1366) used to indicate failed biometric authentication by the device. For example, the tactile output generated as an error when biometric registration is stopped due to failure to change the orientation of a biometric feature with respect to one or more biometric sensors may be a tactile output used to indicate failed biometric authentication. Same as output.

In some examples, the device overlays a visual prompt (e.g., 1328, 1340, 1356) on the representation of the biometric characteristic. For example, the visual prompt is optionally an arrow indicating each way (direction) to move the biometric feature (such as up, down, left, right, or at a diagonal angle between those directions). In some examples, the visual prompt is partially transparent. Displaying a visual prompt, such as an arrow element, in the requested direction of movement allows the user to quickly understand how to move the biometric feature so that the portion of the feature corresponding to the requested direction can be properly registered. This allows the user to execute the requested move more quickly and efficiently, reducing the time required by the registration process. Accordingly, providing improved visual prompts that illustrate appropriate movements of biometric features can help the user provide appropriate inputs when operating/interacting with the device and reduce user errors. It improves operability and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, the device may display an animated prompt to move a biometric feature in a respective manner (e.g., 1328, 1340, or as described with respect to FIGS. 13D and 13E, 13F and 13G, or 13J and 13K). 1356 animation) is displayed (1414). For example, the device optionally displays an animation (e.g., the animation of 1328 shown in FIGS. 13D and 13E) prompting movement in a first manner relative to the first portion of the biometric feature and Display an animation (e.g., animation 1340 in FIGS. 13F and 13G) prompting movement in a second manner relative to the second portion of the unit. In some examples, displaying the animated prompt includes displaying 1416 an arrow element (e.g., an arrow element at 1328, 1340, or 1356) indicating a respective way to move the biometric feature. Displaying an animation that intuitively illustrates the requested direction of movement allows the user to quickly understand how to move the biometric feature so that the portion of the feature that corresponds to the requested direction can be properly registered. This allows the user to execute the requested move more quickly and efficiently, reducing the time required by the registration process. Accordingly, providing improved visual feedback with intuitive examples of appropriate movements of biometric features (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) ) improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the device outputs 1420 at least one of a tactile output (e.g., 1336, 1346, 1364) or an auditory output (e.g., 1334, 1344, or 1362) corresponding to the animation. For example, the animation selectively zooms in and out on biometric features. Alternatively and/or additionally, one or more elements of the registration user interface (e.g., 1324, 1350) optionally temporarily change state. Typically, the tactile output is synchronized with animation. Providing tactile and/or audio outputs accompanied by a visual example of the requested movement helps the user quickly understand how to move a biometric feature so that the portion of the feature corresponding to the requested direction can be properly registered. do. This allows the user to execute the requested move more quickly and efficiently, reducing the time required by the registration process. Accordingly, providing improved tactile and/or auditory feedback accompanying animation may be beneficial to the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). It improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the first manner of movement includes rotation about an axis parallel to the display (e.g., in the plane of display 1302) and the second way of movement includes rotation about an axis parallel to the display. Includes. In this case, the animated prompt (e.g., the animation of 1328, 1340, or 1356 described with respect to FIGS. 13D and 13E, 13F and 13G, or 13J and 13K) is centered on an axis parallel to the display. and simulated rotation of a user interface element (e.g., an arc element at 1328, 1340, or 1356) (1418). For example, if a user is being prompted to rotate a biometric feature clockwise about an axis parallel to the display, the animation optionally includes movement of a user interface element clockwise about an axis parallel to the display. do. Likewise, if the user is being prompted to rotate a biometric feature counterclockwise about an axis parallel to the display, the animation optionally includes movement of a user interface element counterclockwise about an axis parallel to the display. do. Displaying a simulated rotation of the orientation element to illustrate the requested movement allows the user to quickly understand how to move the biometric feature so that the portion of the feature corresponding to the requested direction can be properly registered. This allows the user to execute the requested move more quickly and efficiently, reducing the time required by the registration process. Accordingly, providing improved visual feedback with intuitive examples of appropriate movements of biometric features (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) ) improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, a biometric registration user interface is overlaid on a representation of the biometric feature (e.g., 1306) (e.g., as described above with reference to method 1200) and the representation of the biometric feature is positioned in different directions. It includes an orientation guide (eg, 1308) that tilts in different directions as it tilts. In this example, upon determining that enrollment prompt criteria have been met with respect to a first portion of a biometric feature that can be enrolled by moving the biometric feature in a first manner, an animated prompt (e.g., FIGS. 13D and 13E or 13J and the animation of 1356 described in connection with FIGS. 13K) includes movement of a portion of the orientation guide (e.g., the vertical component of 1308) in the direction in which the orientation guide would have moved if the biometric feature had been moved in the first manner. Includes. Displaying and/or rotating the orientation guide overlaid on the representation of the biometric features provides the user with feedback regarding the orientation of his or her biometric features relative to the device's biometric sensors in three-dimensional space. This allows the user to more quickly move the biometric feature through the required orientation range during the registration process. Accordingly, providing the user with improved visual feedback regarding the orientation of the biometric feature may improve the orientation of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). It improves operability and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

Likewise, upon determining that enrollment prompt criteria have been met with respect to a second portion of the biometric feature that can be enrolled by moving the biometric feature in a second manner, an animated prompt (e.g., as described with respect to FIGS. 13F and 13G The animation of 1340) includes movement of a portion of the orientation guide in a direction in which the orientation guide would have moved if the biometric feature had been moved in the second manner. In some examples, the orientation guide includes a first portion (e.g., a horizontal component of 1308, e.g., a first circular arc) and a second portion (e.g., a horizontal component of 1308, e.g., a second circular arc that intersects the first arc). and an animation prompt (e.g., the animation of 1340 shown in FIGS. 13F and 13G) may be directed to moving the first portion of the orientation guide without moving the second portion, or to move the second portion without moving the first portion. Includes moving. In some examples, when the first portion of the orientation guide is moving, the second portion stops being displayed. Similarly, if the second part is moving, the first part stops being displayed. In some examples, if registration is needed for a portion of a biometric feature that is not visible when the feature is tilted up, down, left, or right, the animation may move in a diagonal direction, allowing the user to view the biometric feature in a diagonal direction. Prompts you to tilt to .

In some examples, after outputting each prompt to move a biometric feature in a respective manner (e.g., 1328, 1332, 1334, 1336, 1340, 1342, 1344, 1346), and detecting movement of the biometric feature In response, the device registers each portion of the biometric characteristic. Optionally, the device updates progress indicators (e.g., 1310, 1330) as described in method 1200. While registering each portion of the biometric characteristic, the device optionally stops providing prompts. Updating the progress indicator during registration in this way encourages the user to look at the display of the electronic device during registration, improving the ability to detect when gaze is directed to the display and, therefore, whether the user is paying attention to the device. I order it. Encouraging a user to look at the display of an electronic device improves the operability of the device (e.g., by ensuring that the user's gaze is directed toward the display and thereby ensuring that the user's biometric characteristics are properly registered) and that the user- It makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, after registering each portion of the biometric characteristic, the device determines that registration prompt criteria have been met with respect to one or more portions of the biometric characteristic. In response to determining that enrollment prompt criteria have been met with respect to one or more portions of the biometric feature (e.g., the user stops responding during enrollment for a threshold period of time), the device determines that the biometric characteristic for which the enrollment prompt criteria have been met Outputs different respective prompts (e.g., 1356, 1358, 1362, 1364) to move the biometric feature in a respective manner determined based on one or more portions of the portion. For example, the device begins prompting the user to change the orientation of the biometric feature relative to one or more biometric sensors in order to register portions of the biometric feature that have not yet been registered. In some examples, the prompts have similar characteristics to other prompts described above. In some examples, the prompts proceed in a similar manner to the prompts described above. In some examples, a first prompt (e.g., 1356, 1358, 1362, 1364) is provided in a first direction after a first period of time with little or no movement of the biometric feature relative to one or more biometric sensors, and one A second prompt is provided in a second direction after a second period of time (longer than the first period) during which there is little or no movement of the biometric feature for the biometric sensors. An option is provided (e.g., 1382, 1390) to complete biometric registration without registering all portions of the biometric characteristic after a third period (longer than the first period) with little or no movement of the part. Automatically providing prompts to move a biometric feature in a different direction after detecting little to no movement of the biometric feature can be achieved by quickly and automatically providing instructions on how to proceed with the registration process. Helps users who are struggling or unable to perform directional movements. Performing an optimized set of actions when a set of conditions are met without requiring additional user input (e.g., helping the user provide appropriate inputs when operating/interacting with a device and preventing user mistakes) (by reducing the device's operability and making the user-device interface more efficient), which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently .

In some examples, after outputting a respective prompt (e.g., 1328, 1332, 1334, 1336, 1340, 1342, 1344, 1346, 1356, 1358, 1362, 1364) to move a biometric feature in a respective manner, Upon determining that the accessibility prompt criteria have been met, the device displays an option (e.g., 1378) to proceed with registration without further changes in the orientation of the biometric feature relative to one or more biometric sensors (e.g., 1303). Do (1428). The accessibility prompt criteria include a requirement that the orientation of the biometric feature relative to one or more biometric sensors has changed by less than a predetermined amount during each period (1430). For example, the user's biometric characteristic was not detected to be moving for a threshold period of time after providing the second prompt (e.g., 1340, 1342, 1344, 1346). In this case, accessibility affordances (e.g., 1378) are displayed and the user optionally selects an accessibility affordance (e.g., 1380). In other words, the user may authorize the use of biometric feature authentication for views of the biometric feature from a smaller orientation range than the available orientation range. In some examples, each period is greater than the second period. For example, after a delay with little or no movement of a biometric feature relative to one or more biometric sensors (e.g., 1303), the device may first cause movement of the biometric feature in a first direction (e.g., 1328, prompt (using 1332, 1334, 1336); Then, after a delay with little or no movement, the device prompts (e.g., using 1340, 1342, 1344, 1346) movement of the biometric feature in a second direction; Then, after an additional delay with little or no movement, the device provides an option (e.g., 1378) to proceed with registration without further movement of the biometric feature for one or more biometric sensors. In some examples, an accessibility prompt (e.g., 1378) is displayed after a sufficient amount of biometric features has been captured to ensure secure authentication by at least a portion of the biometric features (e.g., one angle of the face has been captured Upon registration and registration, users with limited mobility can select the accessibility option to register biometric features using only registered angles).

In some examples, the device detects selection of an option to proceed with registration without further changes in the orientation of the biometric feature relative to the one or more biometric sensors (1432). For example, in some examples, the device receives user input (e.g., 1382) indicating selection (e.g., 1380) of an affordance of an accessibility interface (e.g., 1368) to confirm registration of biometric data. In response to detecting 1434 selection of the option to proceed with registration without further changes in the orientation of the biometric feature relative to the one or more biometric sensors, the device suspends one or more steps in biometric registration (e.g. , skipped) (1436). For example, the device may display a second biometric registration user interface (e.g., second registration interface 1138 in FIG. 11H) to be displayed in a standard registration process in which the biometric features change orientation as prompted by the device. Skip the display of (e.g., as described in connection with the method of 1200, there is no second registration flow if the user registers through the accessibility interface).

In some examples, in response to selecting an option to proceed with registration without further changes in the orientation of the biometric feature relative to one or more biometric sensors (1434), the device determines which portions of the biometric feature were registered. An indication (e.g., 1391 and 1389) that registration of the biometric features containing the information is complete is displayed (1438). In some examples, the device displays an affordance (e.g., 1390) that, when selected, confirms partial registration of a biometric feature.

In some examples, in response to detecting selection of an option to proceed with registration without further changes in the orientation of the biometric feature relative to the one or more biometric sensors, the device may determine the biometric feature once the biometric feature has been registered. Outputs a tactile output (e.g., 1393) used to indicate successful biometric authentication by the user. For example, the tactile output generated when biometric registration is complete is optionally the same tactile output used to indicate successful authentication by biometric features.

Note that the details of the processes described above with respect to method 1400 (e.g., FIGS. 14A and 14B) are also applicable in a similar manner to the methods described herein. For example, method 1400 optionally includes one or more of the features of the various methods described herein with reference to methods 800, 1000, 1200, 1600, 1800, 2000, 2200, 2500, and 2700. Includes. For example, the accessibility interface described in method 1000 may be applied in conjunction with an accessibility registration interface (e.g., 1368). For another example, an orientation guide as described in method 1200 may be applied in conjunction with an orientation guide (e.g., 1308). For brevity, these details are not repeated below.

Operations in the information processing methods described above optionally utilize one or more functional modules within an information processing device, such as general-purpose processors (e.g., as described with respect to FIGS. 1A, 3, and 5A) or application-specific chips. It is implemented by running it. Additionally, the operations described above with reference to FIGS. 14A and 14B are optionally implemented by components shown in FIGS. 1A and 1B. For example, display operation 1402, detection operation 1406, output operation 1408, output operation 1412, and output operation 1414 may optionally include event classifier 170 and event recognizer 180. , and is implemented by the event handler 190. Event monitor 171 in event sorter 170 detects contact on touch-sensitive surface 604, and event dispatcher module 174 passes event information to application 136-1. The respective event recognizer 180 of application 136-1 compares the event information to the respective event definition 186 and determines that the first contact at the first location on the touch-sensitive surface corresponds to the selection of an object on the user interface. Determines whether it corresponds to the same predefined event or subevent. When an individual predefined event or sub-event is detected, event recognizer 180 activates an event handler 190 associated with detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some examples, event handler 190 accesses the respective GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes may be implemented based on the components shown in FIGS. 1A and 1B.

15A-15T illustrate example user interfaces for biometric authentication, according to some examples. As described in more detail below, the illustrative examples of user interfaces illustrated in Figures 15A-15T are used to illustrate the processes described below, including the processes in Figures 16A-16E.

FIG. 15A illustrates an electronic device 1500 (e.g., portable multifunction device 100, device 300, or device 500). In the illustrative example illustrated in FIGS. 15A-15T, electronic device 1500 is a smartphone. In other examples, electronic device 1500 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 1500 has a display 1502, one or more input devices (e.g., a touchscreen of display 1502, buttons 1504, a microphone (not shown)), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the electronic device includes one or more biometric sensors (e.g., biometric sensor 1503), optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, one or more biometric sensors 1503 are one or more biometric sensors 703. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In FIG. 15A , electronic device 1500 displays application interface 1506 including login affordance 1508 on display 1502 . In the example of Figure 15A, the application is a browser that displays a website (eg, onlinestore.com) in the browser's interface. 15B , while displaying application interface 1506, electronic device 1500 detects activation of login affordance 1508. As shown, the activation is a tap gesture 1510 on the login affordance 1508.

15C, in response to detecting activation of login affordance 1508, electronic device 1500 initiates biometric authentication. In some examples, initiating biometric authentication includes obtaining (e.g., capturing with one or more biometric sensors) data corresponding to at least a portion of the user's biometric features (e.g., the user's face). Includes. 15C , initiating biometric authentication further includes displaying a biometric authentication interface 1512 with a biometric authentication glyph 1514. In the example of Figure 15C, biometric authentication glyph 1514 is a simulation of a representation of a biometric feature (e.g., face). As can be seen in Figure 15C, biometric authentication interface 1512 is overlaid on at least a portion of application interface 1506. In some examples, the biometric authentication interface is an operating system level interface (e.g., an interface created by the device's operating system) and the application interface 1506 is an application level interface (e.g., a third party separate from the device's operating system). user interface created by the application).

In some examples, electronic device 1500 initiates biometric authentication in response to activation of the login affordance of the application, while in other examples, electronic device 1500 initiates biometric authentication in response to loading the application and/or application interface 1506. to initiate biometric authentication (e.g., start automatically). The application interface is displayed, for example, in response to loading an application (eg, by selecting an icon associated with the application on the home screen of electronic device 1500).

In some examples, including the example of Figure 15C, the biometric authentication interface is partially translucent. In some examples, the display (e.g., visual characteristics) of biometric authentication interface 1512 is based on application interface 1506. By way of example, one or more colors of biometric authentication interface 1512 are based on one or more colors of application interface 1506. 15C, the electronic device 1500 displays the application interface 1506 with a first color combination, and the biometric authentication interface 1512 is displayed based on the first color combination (e.g., the first color displayed using colors that contrast with the formulation). Referring to FIG. 15D , the electronic device 1500 displays the application interface 1507 with a second color combination that is different from the first color combination, and the biometric authentication interface 1512 is displayed based on that color combination. . Displaying the biometric authentication interface 1512 in this manner ensures that the biometric authentication interface 1512 is easily recognized and visible by the user when overlaid on an application interface.

In response to initiating biometric authentication, electronic device 1500 may capture and process (e.g., analyze) biometric data to determine whether the biometric feature (or portion thereof) is biometric. Determine whether authentication criteria are met (e.g., determine whether biometric data matches a biometric template, within a threshold). In some examples, in response to obtaining biometric data, electronic device 1500 displays a biometric authentication animation, including, for example, changing the size of a biometric authentication glyph. In some examples, while the electronic device is processing biometric data, the electronic device displays one or more biometric authentication glyphs and/or biometric authentication animations to indicate that biometric data is being processed (e.g., They replace the display of biometric authentication glyphs 1514).

As an example, in Figure 15E, the electronic device displays biometric authentication glyph 1514 in response to initiation of biometric authentication. 15F and 15G, once the electronic device 1500 has acquired biometric data (e.g., has acquired sufficient biometric data), the electronic device 1500 may use the biometric authentication glyphs (1515 in FIG. 15F). , 1516 in FIG. 15G ), wherein biometric authentication glyph 1514 is replaced by (e.g., transitions to) biometric authentication glyph 1517 ( FIG. 15H ). It serves as a part of Referring to Figure 15H, electronic device 1500 displays biometric authentication glyph 1517 to indicate that biometric data is being processed. In some examples, biometric authentication glyph 1517 includes a plurality of rings that rotate in a sphere, for example, while being displayed.

In Figure 15I, electronic device 1500 determines that the biometric characteristic satisfies biometric authentication criteria. In response, the electronic device displays biometric authentication glyph 1518 in biometric authentication interface 1512 indicating that biometric authentication was successful (e.g., replacing the display of biometric authentication glyph 1517 with it). do). Additionally or alternatively, the electronic device outputs a tactile output 1520 indicating that biometric authentication was successful. After indicating that the biometric authentication was successful, the electronic device 1500 provides authentication information to the application indicating that the biometric feature satisfies the biometric authentication criteria and, consequently, that the biometric authentication was successful.

As shown in FIG. 15J, in response to electronic device 1500 providing authentication information indicating that the biometric feature meets biometric authentication criteria, the application displays main interface 1522 (e.g., application replace the display of the interface 1506 with it). Referring to FIG. 15K, after a predetermined period of time, the electronic device 1500 stops displaying the biometric authentication interface. Thereafter, the user optionally uses the application as if the user had directly authenticated with the application (e.g., using a username and password for an account associated with the application). In some examples, the electronic device 1500 stops displaying the biometric authentication interface 1512 after biometric authentication is complete and a predetermined amount of time has passed. In other examples, the electronic device 1500 performs an operation such that an application displays an interface (e.g., main interface 1522) and stops displaying the biometric authentication interface 1512 after a predetermined period of time. .

Alternatively, in Figure 15L, the electronic device 1500 determines that the biometric feature does not meet the biometric authentication criteria (e.g., after displaying the biometric authentication glyph 1517 of Figure 15G). In response, the electronic device displays a biometric authentication glyph, such as biometric authentication glyph 1519, in biometric authentication interface 1512 to indicate that biometric authentication was not successful (e.g., failed). For example, replace the display of the biometric authentication glyph 1517 with it). In some examples, biometric authentication glyph 1519 is associated with a biometric authentication failure animation. 15L and 15M, in some examples, in response to an unsuccessful biometric authentication, the electronic device 1500 simulates a “head shake” effect and indicates that the biometric authentication was not successful. To illustrate, display a biometric authentication failure animation in which the biometric authentication glyph 1519 moves (e.g., rotates) side-to-side. Optionally, the electronic device 1500 outputs a tactile output 1526 indicating that biometric authentication was not successful. In some examples, tactile output 1526 is the same as tactile output 1520. In some examples, tactile output 1526 is different from tactile output 1520. In some examples, tactile output 1526 is synchronized with a biometric authentication failure animation.

15N and 15O show that, in response to an unsuccessful biometric authentication (as determined with respect to FIG. 15E), the electronic device 1500 creates a biometric authentication glyph 1514 within the biometric authentication interface 1512. Illustrative of an alternative biometric failure animation that displays (e.g., replaces the display of biometric authentication glyph 1517 (FIG. 15H) with it). In some examples, during display of the biometric authentication failure animation, the electronic device moves biometric authentication interface 1512 on display 1502. In some examples, electronic device 1500 moves biometric authentication interface 1512 left and right to simulate a “shake” effect and indicate that biometric authentication was not successful. In some examples, the electronic device moves only the biometric authentication glyph 1514 and does not move the biometric authentication interface 1512. In other examples, additional or alternative glyphs are used in the biometric authentication failure animation.

As illustrated in FIG. 15P , in some examples, after displaying one or more biometric authentication failure animations, the electronic device displays a biometric authentication interface 1512 with a biometric authentication glyph 1514. In this manner, the electronic device displays the initial biometric authentication glyph 1514 once more, indicating that the electronic device 1500 is enabled to perform additional biometric authentication. In some examples, the electronic device performs an additional iteration of biometric authentication, at least as described with respect to FIGS. 15E-15N.

15Q, in some examples, in response to unsuccessful biometric authentication, electronic device 1500 displays a failure interface 1540 (e.g., replaces the display of biometric authentication interface 1512 with it). do). In some examples, displaying failure interface 1540 includes maintaining display of biometric authentication interface 1512. In some examples, failure interface 1540 includes biometric authentication glyph 1514, alternative authentication affordance 1544, retry affordance 1546, and cancel affordance 1548. In some examples, activation of cancel affordance 1548 causes electronic device 1500 to stop displaying failure interface 1540.

Referring to Figure 15R, in some examples, in response to activation of a retry affordance 1546, such as a tap gesture 1550, electronic device 1500 performs another iteration of biometric authentication. In some examples, electronic device 1500 may display one or more biometric authentication glyphs (e.g., 1515-1517) and/or a biometric authentication animation within failure interface 1540 to indicate progress and/or results of biometric authentication. display them. Referring to Figure 15S, in some examples, the electronic device performs biometric authentication only if a threshold number (e.g., 5) failed biometric authentication attempts have not been made. In some examples, when a threshold number of failed biometric authentication attempts has been reached, the electronic device 1500 displays an indication 1560 that the threshold number has been reached and consequently biometric authentication is not available (e.g., Replace the display of the biometric authentication glyph (1514) with it).

In some examples, in response to activation of an alternative authentication affordance 1544, such as a tap gesture 1552, electronic device 1500 displays an alternative authentication interface 1562 (e.g., failure interface 1540). replacing the display with it), whereby the user authenticates using an alternative form of authentication (e.g., fingerprint authentication, password authentication) in addition to authentication associated with biometric features. As illustrated in Figure 15T, the user optionally authenticates by entering appropriate credentials in username field 1564 and password field 1566, respectively. In some examples, failure interface 1540 is an operating system level interface that allows a user to authenticate with the operating system of electronic device 1500, and alternative authentication interface 1562 is an application level interface that allows a user to authenticate with an application.

16A-16E are flow diagrams illustrating a method for managing peer-to-peer transmissions using an electronic device, according to some examples. Method 1600 includes a device having a display, one or more input devices (e.g., a touchscreen, microphone, camera), and a wireless communication radio (e.g., a Bluetooth connection, a WiFi connection, a mobile broadband connection such as a 4G LTE connection). For example, 100, 300, 500, 1500). In some examples, the display is a touch-sensitive display. In some examples, the display is not a touch-sensitive display. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the device includes one or more biometric sensors, optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device, such as an IR floodlight, a structured light projector, or a combination thereof. The light-emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric feature by one or more biometric sensors. Some operations of method 2000 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 1600 provides an intuitive method for managing authentication of biometric characteristics. This method reduces the user's cognitive burden for managing authentication of biometric features, thereby creating a more efficient human-machine interface and intuitive user experience. For battery-operated computing devices, allowing users to manage authentication of biometric features faster and more efficiently saves power and increases time between battery charges.

Before displaying the application interface (e.g., 1506) and the biometric authentication interface (e.g., 1512), the electronic device (e.g., 100, 300, 500, 1500) displays an application (e.g., a browser as discussed with respect to FIG. 15A). application) is loaded (1602). In some examples, the application interface (e.g., 1506) is not initially installed on the electronic device (e.g., 100, 300, 500, 1500) and/or operates the electronic device (e.g., 100, 300, 500, 1500). It is an interface to a third-party application that is not provided by the system manufacturer or the device manufacturer. In some examples, the biometric authentication interface (e.g., 1512) is an operating system-generated asset that is not under the control of the application that corresponds to (e.g., creates) the application interface (e.g., 1506).

The electronic device (e.g., 100, 300, 500, 1500) displays, on the display (e.g., 1502), an application interface (e.g., 1506) corresponding to the application and the electronic device (e.g., 100, 300, 500, 1500). Simultaneously displays (1604) a biometric authentication interface (e.g., 1512) controlled by the operating system. Simultaneously displaying the application interface and the biometric authentication interface allows the user to quickly recognize that the requested biometric authentication is associated with the application corresponding to the application interface, and further prevents the user from unintentionally using the application. It gives the user more control of the device by helping to avoid performing an action while at the same time making the user aware that authentication is required before the action can be performed. Providing additional control of the device in this way without cluttering the UI with additional displayed controls (e.g., helps the user provide appropriate inputs when operating/interacting with the device and reduces user errors) improves the device's operability and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the biometric authentication interface (e.g., 1512) is displayed (1606) over a portion of the application interface (e.g., 1506). In some examples, the biometric authentication interface (e.g., 1512) is not displayed entirely over the application interface (e.g., 1506), and at least a portion of the application interface (e.g., 1506) remains displayed without being overlaid. In some examples, the biometric authentication interface (e.g., 1512) is at least partially translucent. In some examples, the biometric authentication interface (e.g., 1512) is at least partially translucent (or transparent) such that the application interface (e.g., 1506) is at least partially visible through the biometric authentication interface (e.g., 1512). In some examples, the biometric authentication interface (e.g., 1512) blurs the underlying content such that the appearance of the biometric authentication interface (e.g., 1512) is based on the portion of the blurred content beneath the biometric authentication interface (e.g., 1512). Let's do it. In some examples, a biometric authentication interface (e.g., 1512) is displayed 1608 in response to loading of the application. In some examples, the biometric authentication interface (e.g., 1512) allows a user to load (e.g., launch or resume execution of) an application on an electronic device (e.g., 100, 300, 500, 1500). It is displayed in response. In some examples, the biometric authentication interface (eg, 1512) is loaded after the application is displayed. In some examples, the biometric authentication interface (eg, 1512) and the application interface (eg, 1506) are displayed simultaneously. In some examples, a biometric authentication interface (e.g., 1512) is displayed (1610) in response to detecting user interaction with an application interface (e.g., 1506) corresponding to a request to access content requiring authentication. In some examples, the request for authentication is selection of an authentication affordance (e.g., 1508) or performance of a gesture. In some examples, the application interface (eg, 1506) includes an authentication affordance 1506 (eg, a login affordance).

While displaying a biometric authentication interface (e.g., 1512), prior to acquiring biometric data corresponding to at least a portion of the biometric features, the electronic device (e.g., 100, 300, 500, 1500) displays one or more biometric sensors. Prepare to use the field (e.g., 1503). In some examples, in response to displaying a login affordance (e.g., 1508), the electronic device (e.g., 100, 300, 500, 1500) prepares (e.g., primes) one or more biometric sensors for use. In some examples, preparing one or more biometric sensors (e.g., 1503) for use may include moving the sensors (e.g., 1503) from a low power state (e.g., unpowered or sleep state) to a low latency state (e.g., partial power state or full power state, pre-warmed state). In this way, the electronic device (e.g., 100, 300, 500, 1500) optionally reduces the time required to perform biometric authentication when displaying a biometric authentication interface (e.g., 1512). In some examples, when one or more biometric sensors (e.g., 1503) is in a low power state, it takes a first time to attempt biometric authentication using the one or more biometric sensors, and When (eg, 1503) is in a low latency state, it takes a second time, less than the first time, to attempt biometric authentication using one or more biometric sensors (eg, 1503). While displaying a biometric authentication interface (e.g., 1512), an electronic device (e.g., 100, 300, 500, 1500) receives at least a portion of a biometric feature from one or more biometric sensors (e.g., 1503). Obtain biometric data (1612). In some examples, the biometric feature is a face, and the biometric data is data corresponding to a portion of the face.

In response to obtaining, from one or more biometric sensors, biometric data corresponding to at least a portion of the biometric feature, the electronic device (e.g., 100, 300, 500, 1500) determines, based on the biometric data, biometric data. Determine whether at least a portion of the measurement feature satisfies biometric authentication criteria (1614). Based on the acquired biometric data, determining whether at least a portion of the biometric features satisfies the biometric authentication criteria is a fast and efficient authentication process that allows the user to easily provide an authentication operation and proceed with it with minimal input. makes possible. Reducing the number of inputs required to perform an operation improves the operability of the device and improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). -Makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the electronic device (e.g., 100, 300, 500, 1500) may configure a facial (e.g., 100, 300, 500, 1500) device where the user's face or fingerprint is authorized for use in biometric authentication on the device (e.g., 100, 300, 500, 1500). s) and/or determine whether a match is made to stored information about the fingerprint(s). In some examples, based on the biometric data, determining whether at least a portion of the biometric feature satisfies biometric authentication criteria includes displaying a biometric authentication analysis animation (1616). In some examples, the biometric authentication animation includes displaying a sequence of interface objects (e.g., 1514, 1515, 1516, 1517, 1518, 1519) (e.g., glyphs). A first interface object (e.g., 1514) indicates that biometric authentication has been initiated, and a second interface object (e.g., 1517) indicates that the device (e.g., 100, 300, 500, 1500) is processing biometric data. , the third interface object (eg, 1518, 1519) indicates whether biometric authentication succeeded or failed. In some examples, the first interface object (e.g., 1514) is substantially square in shape and the second interface object (e.g., 1517) is substantially circular in shape. In some examples, displaying the biometric authentication analysis animation includes rotating one or more rings about an interface object (e.g., 1517) of the biometric authentication animation (e.g., a biometric authentication glyph). In some examples, one or more rings are rotated while the device (e.g., 100, 300, 500, 1500) is processing biometric data to determine whether the biometric data meets biometric authentication criteria. The rotation of the rings optionally simulates the rotation of the rings around a sphere. In some examples, once a device (eg, 100, 300, 500, 1500) has finished processing biometric data, one or more rings are overlaid on one another to verify that processing is complete. In some examples, displaying the biometric authentication analysis animation includes an animated object (e.g., 1514, 1515) on a platter (e.g., 1512) with an appearance based on underlying content (e.g., 1506, 1507, 1522). , 1516, 1517, 1518, 1519). In some examples, as the appearance of the animated object changes, the appearance of the platter changes. In some examples, when the animated object becomes darker, the platter becomes darker, and when the applied object becomes brighter, the platter becomes brighter. In some examples, the appearance of the platter (e.g., 1512) is based on an animated object (e.g., 1514, 1515, 1516, 1517, 1518, 1519) changes as its appearance changes. In some examples, one or more colors of the biometric authentication analysis animation are based on one or more colors of the application interface (e.g., 1506). In some examples, the colors of the animation are selected based on one or more colors of an application interface (eg, 1506, 1507, 1522) or another interface associated with the application. The colors are optionally derived, for example, based on the colors used for the application's controls and/or icons. In this way, the animation is optionally visually coordinated with the application interface (e.g., 1506, 1507, 1522) to provide a more robust user experience. In some examples, prior to displaying the biometric authentication analysis animation, the electronic device (e.g., 100, 300, 500, 1500) selects data corresponding to the application interface (e.g., 1506) or a color scheme of the application interface (e.g., 1506). Determine one or more colors of the animation based on the analysis. In some examples, further in response to obtaining biometric data corresponding to at least a portion of the biometric feature from one or more biometric sensors (e.g., 1503), the electronic device (e.g., 100, 300, 500, 1500) changes the size of an interface object (e.g., 1514) (e.g., a biometric authentication glyph) of a biometric authentication interface (e.g., 1512) from a first size to a second size, and Change the size from the second size to the first size. In some examples, once biometric data has been captured by one or more biometric sensors (e.g., 1503), an interface object (e.g., 1514) (e.g., a biometric authentication glyph) is incremented from an initial size and subsequently increased to the initial size. Returning to creates a “bounce” effect.

Based on the biometric data, upon determining that at least a portion of the biometric feature satisfies biometric authentication criteria (1636), the electronic device (e.g., 100, 300, 500, 1500) selects one or more portions of the biometric feature. Authentication information indicating that the biometric authentication standards have been satisfied is provided to the application (1620). Providing authentication information to an application following a determination that at least a portion of the biometric features meet biometric authentication criteria improves the security of the device and reduces the number of fraudulent transfers that can occur. Improving device security and reducing the number of fraudulent transmissions improves the operability of the device (e.g., by reducing fraud when operating/interacting with the device) and makes the user-device interface more secure.

In some examples, authentication information is provided by the operating system to an application that creates an application interface (e.g., 1506). In some examples, further, based on a determination that at least a portion of the biometric characteristic satisfies biometric authentication criteria, based on the biometric data, after providing authentication information to the application, the electronic device (e.g., 100, 300, 500, 1500) maintains the display of the biometric authentication interface (e.g., 1512) for a predetermined period of time (1624). In some examples, further, based on a determination that at least a portion of the biometric characteristic satisfies biometric authentication criteria, based on the biometric data, the electronic device (e.g., 100, 300, 500, 1500) is configured to: Display 1622 a biometric authentication success animation that includes a first representation of a simulation of biometric features (e.g., 1518) indicating that at least a portion of the portion satisfies the biometric authentication criteria.

In some examples, in response to successful biometric authentication, the device (e.g., 100, 300, 500, 1500) displays an animation including an interface object (e.g., 1518) indicating that biometric authentication was successful. In some examples, further, based on a determination that at least a portion of the biometric characteristic satisfies biometric authentication criteria, based on the biometric data, the electronic device (e.g., 100, 300, 500, 1500) is configured to: Provide a success tactile output (e.g., 1520) indicating that at least a portion of the unit meets biometric authentication criteria. Displaying an animation indicating that biometric authentication was successful can provide the user with visual feedback of the action being performed and allow the user to quickly recognize that the action was successful. Providing improved visual feedback to the user helps the user achieve the intended result (e.g., by providing feedback indicating input that will cause the device to produce the intended result when operating/interacting with the device). improves device operability and makes the user-device interface more efficient (by helping users use the device more quickly and efficiently), which in addition reduces the device's power usage and reduces battery life by allowing users to use the device more quickly and efficiently Improves lifespan.

After maintaining the display of the biometric authentication interface (e.g., 1512) for a predetermined period of time, the electronic device (e.g., 100, 300, 500, 1500) stops displaying the biometric authentication interface (e.g., 1512) ( 1626). In some examples, the application receives an indication of authentication before the device (e.g., 100, 300, 500, 1500) stops displaying the biometric authentication interface (e.g., 1512); This allows the application to provide (e.g., display) an interface (e.g., 1522) for the application, such as a “main application” interface or a post login interface, prior to transitioning from the biometric authentication interface (e.g., 1512). do it) In some examples, the biometric authentication interface (e.g., 1512) authenticates and stops being displayed after a predetermined time. In some examples, the biometric authentication interface (e.g., 1512) is stopped from being displayed (e.g., an unlocked user interface (e.g., 1522)) after a predetermined period of time after the application has performed an action in accordance with the biometric authentication. displayed).

Based on the biometric data, upon determining that at least a portion of the biometric feature does not meet biometric authentication criteria (1628), the electronic device (e.g., 100, 300, 500, 1500) determines that at least a portion of the biometric feature does not meet biometric authentication criteria (1628). Display 1630 a biometric authentication failure animation that includes a second representation of a simulation of biometric features (e.g., 1519) indicating that biometric authentication criteria are not met. Displaying a biometric authentication failure animation following a determination that at least a portion of the biometric feature does not meet biometric authentication criteria provides the user with visual feedback of a failure or error in the action being performed and allows the user to determine if the action was successful. You can quickly realize that you haven't done it. Providing improved visual feedback to the user helps the user achieve the intended result (e.g., by providing feedback indicating input that will cause the device to produce the intended result when operating/interacting with the device). improves device operability and makes the user-device interface more efficient (by helping users use the device more quickly and efficiently), which in addition reduces the device's power usage and reduces battery life by allowing users to use the device more quickly and efficiently. Improves lifespan.

In some examples, in response to unsuccessful biometric authentication, the device (e.g., 100, 300, 500, 1500) displays an animation including an interface object (e.g., 1519) indicating that biometric authentication was not successful. do. In response to an unsuccessful biometric authentication, displaying an animation that includes an interface object indicating that the biometric authentication was not successful provides the user with visual feedback of a failure or error in the action being performed and allows the user to perform the action. It can quickly make you realize that you haven't succeeded. Providing improved visual feedback to the user helps the user achieve the intended result (e.g., by providing feedback indicating input that will cause the device to produce the intended result when operating/interacting with the device). improves device operability and makes the user-device interface more efficient (by helping users use the device more quickly and efficiently), which in addition reduces the device's power usage and reduces battery life by allowing users to use the device more quickly and efficiently. Improves lifespan.

In some examples, during the animation, the interface object 1519 is moved (e.g., tilted and/or shifted) in a predetermined manner (e.g., side to side) to indicate a failure. In some examples, a device (e.g., 100, 300, 500, 1500) generates a tactile output (e.g., 1526) or a sequence of tactile outputs (e.g., 1526) corresponding to a biometric authentication failure animation (e.g., the tactile outputs correspond to a biometric authentication failure animation). generated as the simulation of the measurement feature moves back and forth). Outputting a tactile output or sequence of tactile outputs corresponding to a biometric authentication failure animation may additionally inform the user that authentication was not successful and quickly indicate that authentication is still required for the user to perform the action. Make it identifiable. Providing improved tactile feedback to the user improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power by allowing the user to use the device more quickly and efficiently. Reduces usage and improves battery life.

In some examples, the second representation of the simulation of the biometric feature (e.g., 1519) is a three-dimensional object. Displaying a three-dimensional object as a second representation of a simulation of biometric features provides the user with easily recognizable visual feedback regarding the status of the operation (e.g., whether the transmission was successful or unsuccessful), and Because it is three-dimensional, it additionally makes it easier for users to recognize objects. Providing improved visual feedback to the user allows the user to provide appropriate inputs when operating/interacting with the device (e.g., by improving the readability of user interface elements for the user while the device is at natural viewing angles). improves the operability of the device and makes the user-device interface more efficient (by helping users use the device more quickly and efficiently), which in addition reduces the device's power usage. and improve battery life.

In some examples, the second representation (e.g., 1519) is a three-dimensional face performing a head shake. In some examples, displaying the biometric authentication failure animation includes rotating the second representation (e.g., 1519) in a first direction about an axis parallel to the display and rotating the second representation (e.g., 1519) about an axis parallel to the display (e.g., 1502). and alternating between rotations about a second direction. In some examples, displaying the biometric authentication failure animation includes highlighting a boundary of the biometric authentication interface (e.g., 1512) relative to the application interface (e.g., 1506). In some examples, the biometric authentication interface (e.g., 1512), or its boundaries, shrinks and/or retracts to create a visual “bounce” effect. In some examples, further based on a determination, based on the biometric data, that at least a portion of the biometric characteristic does not meet biometric authentication criteria, the electronic device (e.g., 100, 300, 500, 1500) determines the success. It provides a failure tactile output (eg, 1526) that is different from the enemy output (eg, 1520). In some examples, further based on a determination, based on the biometric data, that at least a portion of the biometric feature does not meet biometric authentication criteria, the electronic device (e.g., 100, 300, 500, 1500) is configured to configure a failure interface ( For example, 1540) is displayed (1632). In some examples, the failure interface (e.g., 1540) includes a visual indication that biometric authentication has failed. In some examples, when biometric authentication fails, the application interface (e.g., 1506) is not changed (e.g., the application remains on the login interface (e.g., 1506) or authentication user interface). In some examples, when biometric authentication fails, the application user interface (e.g., 1506) is changed to indicate the failure of biometric authentication. In some examples, the failure interface (e.g., 1540) includes (1634) a retry affordance (e.g., 1546). In some examples, the failure interface (e.g., 1540) includes a cancel affordance (e.g., 1548) (1636). In some examples, the failure interface (e.g., 1540) includes (1638) an alternative authentication affordance (e.g., 1544).

An electronic device (e.g., 100, 300, 500, 1500) receives (1640) an input (e.g., 1550) corresponding to a selection of a retry affordance (e.g., 1546). In response to receiving an input (e.g., 1550) corresponding to selection of a retry affordance (e.g., 1546), the electronic device (e.g., 100, 300, 500, 1500) configures one or more biometric sensors (e.g., From 1503), second biometric data corresponding to at least a portion of the second biometric characteristic is obtained (1642). In some examples, the second biometric feature (eg, face) is the same biometric feature from which the initial biometric data was obtained. In some instances where the second biometric feature is the same biometric feature, a portion of the second biometric feature is a different portion of the same biometric feature from which the initial biometric data was obtained. In some examples, the portions are identical portions of the same biometric characteristic. In some examples, the second biometric characteristic is a different biometric characteristic than the initial biometric characteristic.

After obtaining second biometric data corresponding to at least a portion of the second biometric characteristic, based on the second biometric data, a determination that the at least a portion of the second biometric characteristic satisfies the second biometric authentication criteria. Accordingly, the electronic device (e.g., 100, 300, 500, 1500) provides second authentication information to the application indicating that second biometric authentication criteria have been satisfied with respect to one or more portions of the second biometric characteristic. (1646). In some examples, the second biometric authentication criteria are the same as the initial biometric authentication criteria. In some examples, the second biometric authentication criteria are different from the initial biometric authentication criteria. In some examples, the second authentication information is the same as the authentication information. In some examples, the second authentication information is different from the authentication information. In some examples, authentication information is provided by the operating system to an application that creates an application interface (e.g., 1506).

The electronic device (eg, 100, 300, 500, 1500) receives input corresponding to selection of the cancellation affordance (1646). In response to receiving input corresponding to selection of the cancel affordance, the electronic device (e.g., 100, 300, 500, 1500) stops displaying (1648) the biometric authentication interface (e.g., 1512). In some examples, selection of a cancel affordance releases the failure interface (e.g., 1540) while maintaining the application interface (e.g., 1506). In some examples, selection of a cancel affordance also causes the electronic device (e.g., 100, 300, 500, 1500) to provide information to the application indicating that the first and/or second biometric authentication criteria have not been met. .

An electronic device (e.g., 100, 300, 500, 1500) receives (1650) an input (e.g., 1548) corresponding to a selection of an alternative authentication affordance (e.g., 1544). Providing alternative authentication affordances (e.g., to provide alternative methods for providing authentication as an alternative to or in addition to biometric authentication) may be useful in cases where current authentication methods are not successful or continue to be unsuccessful. This allows users to easily provide authentication for their actions using different authentication methods. Providing additional control options (e.g., to provide authentication) in this manner without cluttering the UI with additional displayed controls allows the user to enter appropriate inputs (e.g., when operating/interacting with the device). improves device operability and makes the user-device interface more efficient (by helping provide and reduce user errors), which additionally reduces the device's power usage by allowing users to use the device more quickly and efficiently. Reduces battery life and improves battery life.

In response to receiving an input (e.g., 1548) corresponding to a selection of an alternative authentication affordance (e.g., 1544), the electronic device (e.g., 100, 300, 500, 1500) provides an alternative authentication interface (e.g., 1562). displays (1652). In some examples, an alternative authentication interface (e.g., 1562) is a non-biometric authentication interface (e.g., 1512). In some examples, an alternative authentication interface (e.g., 1562) allows a user to authenticate using a password and/or passcode. In some examples, the application determines what form of authentication is accepted by the alternative authentication interface (e.g., 1562). In some examples, one or more preferences of an application determine what form of authentication is accepted by the application. In some examples, alternative authentication affordances (e.g., 1562) may include consecutive failures of biometric authentication (e.g., 2 failed authentication attempts, 3 failed authentication attempts, 4 failed authentication failures) , etc.) are contained within a failure interface (e.g., 1540). In some examples, an alternative authentication interface (e.g., 1562) is an application level authentication interface (1654). In some examples, in response to receiving input corresponding to a selection of an alternative authentication affordance (e.g., 1544), an electronic device (e.g., 100, 300, 500, 1500) configures a biometric authentication interface (e.g., 1512). Stop displaying (1656). In some examples, selection of an alternative authentication affordance (e.g., 1544) causes the device (e.g., 100, 300, 500, 1500) to stop displaying the alternative authentication affordance (e.g., 1544) and at the application level. Transition to an operating alternative authentication interface (e.g., 1562). Accordingly, the user is optionally authenticated to the application using credentials associated with the application (eg, the user optionally logs in using a username and password for the application). In some examples, the application level alternative authentication interface (e.g., 1562) optionally includes an affordance to reinitiate biometric authentication. This will then cause the electronic device (eg, 100, 300, 500, 1500) to display the biometric authentication interface (eg, 1512) again and authenticate at the system or operating system level.

Note that the details of the processes described above with respect to method 1600 (e.g., FIGS. 16A-16E) are also applicable in a similar manner to the other methods described. For example, method 1600 optionally includes one or more of the features of the various methods described herein with reference to methods 800, 1000, 1200, 1400, 1800, 2000, 2200, 2500, 2700. Includes. For example, the registered biometric data described in method 1200 can be used to perform biometric authentication, such as the biometric authentication described with reference to FIGS. 15E-15I. As another example, a biometric authentication interface as described in method 1800 may be used to implement a biometric authentication interface (e.g., 1512). For brevity, these details are not repeated below.

Operations in the information processing methods described above optionally utilize one or more functional modules within an information processing device, such as general-purpose processors (e.g., as described with respect to FIGS. 1A, 3, and 5A) or application-specific chips. It is implemented by running it. Additionally, the operations described above with reference to FIGS. 16A-16E are optionally implemented by components shown in FIGS. 1A and 1B. For example, provision operation 1620 and maintenance operation 1624 are optionally implemented by event sorter 170, event recognizer 180, and event handler 190. Event monitor 171 in event sorter 170 detects contact on touch-sensitive surface 604, and event dispatcher module 174 passes event information to application 136-1. The respective event recognizer 180 of application 136-1 compares the event information to the respective event definition 186 and determines that the first contact at the first location on the touch-sensitive surface corresponds to the selection of an object on the user interface. Determines whether it corresponds to the same predefined event or subevent. When an individual predefined event or sub-event is detected, event recognizer 180 activates an event handler 190 associated with detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some examples, event handler 190 accesses the respective GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes may be implemented based on the components shown in FIGS. 1A and 1B.

17A-17J illustrate example user interfaces for biometric authentication, according to some examples. As described in more detail below, the illustrative examples of user interfaces illustrated in Figures 17A-17J are used to illustrate the processes described below, including the processes in Figures 18A-18D.

FIG. 17A illustrates an electronic device 1700 (e.g., portable multifunction device 100, device 300, or device 500). In the illustrative example illustrated in FIGS. 17A-17J , electronic device 1700 is a smartphone. In other examples, electronic device 1700 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 1700 has a display 1702, one or more input devices (e.g., a touchscreen of display 1702, a button 1704, a microphone), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the electronic device includes one or more biometric sensors (e.g., biometric sensor 1703), optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, one or more biometric sensors 1703 are one or more biometric sensors 703. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In FIG. 17A , electronic device 1700 displays the application's landing page interface, including login affordance 1706 , on display 1702 . As can be seen in FIG. 17A, the application is a browser or mobile application, and the interface corresponds to a website (onlinestore.com). While displaying the landing page interface, electronic device 1700 detects activation of login affordance 1706. As shown in Figure 17A, the activation is a tap gesture 1708 on the login affordance 1706.

17B, in response to detecting a tap gesture 1708 on login affordance 1706, electronic device 1700 populates non-secure data fillable field 1710 (labeled “Username”) with secure data. Displays the application interface of the application (e.g., replaces the display of the landing page interface with it), including the enablement field 1712 (labeled “Password”), and the submission affordance 1714. The electronic device further displays a biometric authentication glyph (e.g., icon) within the secure data fillable field 1712. As will be described in further detail, the biometric authentication glyph may indicate that the secure data fillable field 1712 is associated with secure data and/or that biometric authentication is required to automatically populate the secure data fillable field 1712. indicates that

While displaying the application interface, electronic device 1700 detects a request to automatically populate non-secure data fillable field 1710. For example, as shown in FIG. 17B, the request to automatically populate the non-secure data fillable field 1710 is a tap gesture 1718 indicating selection of the non-secure data fillable field 1710.

17C , in response to detecting a request to automatically fill in non-secure data fillable field 1710, electronic device 1700 uses a keyboard, such as a software keyboard, and/or keypad and auto-fill affordance 1722. Display (e.g., overlay on an application interface) the input interface 1720 that includes. While displaying input interface 1720, electronic device 1700 detects activation of autofill affordance 1722. For example, as shown in Figure 17A, the activation is a tap gesture 1724 on the autofill affordance 1722.

17D , in response to detecting a tap gesture 1724, the electronic device displays a plurality of candidate input affordances 1725 for automatically filling the non-secure data fillable field 1710 (e.g., auto-fill affordances) 1722 and/or one or more other affordances of input interface 1720). In the illustrated example, fillable field 1710 is associated with a user name. Accordingly, in some examples, each of candidate input affordances 1725 serves as a reference to a respective candidate user name.

While displaying candidate input affordances 1725 of input interface 1720, the electronic device detects activation of one candidate input affordance 1725. For example, as shown in Figure 17D, the activation is a tap gesture 1726 on one candidate input affordance 1725. 17E , in response to detecting a tap gesture 1726, the electronic device 1700 automatically populates the non-secure data fillable field with a candidate input 1728 that corresponds to the activated candidate input affordance 1725.

As described, in response to detecting the tap gesture 1724, the electronic device provides (eg, displays) candidate input affordances corresponding to each candidate input. In some examples, in response to detecting the tap gesture 1724, the electronic device determines whether a number of candidate inputs are available. If so, electronic device 1700 provides candidate input affordances as described. Any number of candidate input affordances are optionally provided in this manner. If not (e.g., when only a single candidate input is available), the electronic device optionally automatically fills the non-secure data fillable field 1710 without providing candidate inputs.

Referring to Figure 17F, while displaying an application interface, electronic device 1700 detects a request to automatically populate secure data fillable field 1712. For example, a request to automatically populate secure data fillable field 1712 is a tap gesture 1730 indicating selection of secure data fillable field 1712.

17G, in response to detecting a request to automatically populate secure data fillable field 1712, electronic device 1700 initiates biometric authentication. In some examples, initiating biometric authentication includes obtaining (eg, capturing with one or more biometric sensors) data corresponding to the user's biometric characteristics. In some examples, initiating biometric authentication further includes displaying biometric authentication interface 1732 with biometric authentication glyph 1734. Biometric authentication glyph 1734 is, in some examples, a simulation of a representation of a biometric feature. Biometric authentication interface 1732 is overlaid on at least a portion of the application interface in some examples.

Referring to FIG. 17H, in response to obtaining data, the electronic device processes the biometric data to determine whether the biometric characteristic satisfies biometric authentication criteria, e.g., based on the biometric data. (e.g., determine whether biometric data matches a biometric template, within a threshold). While the electronic device is processing biometric data, the electronic device optionally displays a biometric authentication glyph 1738 in biometric authentication interface 1732 indicating that biometric data is being processed (e.g., biometric authentication Replaces the display of the glyph (1734) with it).

In Figure 17I, electronic device 1700 determines that the biometric characteristic satisfies the biometric authentication criteria. In response, the electronic device displays biometric authentication glyph 1740 in biometric authentication interface 1732 indicating that biometric authentication was successful (e.g., replacing the display of biometric authentication glyph 1738 with it). do). Additionally or alternatively, the electronic device outputs a tactile output 1742 indicating that biometric authentication was successful. After indicating that biometric authentication is successful, the electronic device automatically populates the secure data fillable field with the appropriate password 1743, as shown in FIG. 17J. In some examples, the electronic device further automatically populates a second fillable field, such as non-secure fillable field 1710 (e.g., with username 1728), in response to successful biometric authentication. It will be appreciated that any number and/or type of fillable fields may optionally be automatically populated in response to successful biometric authentication.

While displaying an application interface with automatically populated fillable fields 1710 and 1720, the electronic device detects activation of the submission affordance 1714. As an example, as shown in Figure 17J, the activation is a tap gesture 1744 on the submit affordance 1714. In response, the user is optionally authenticated with the application, and the electronic device optionally presents a home interface, such as home interface 1782 in Figure 17S, referenced further below.

Alternatively, in Figure 17K, electronic device 1700 determines that the biometric characteristic does not meet the biometric authentication criteria. In response, the electronic device displays biometric authentication glyph 1746 within biometric authentication interface 1732 (e.g., biometric authentication glyph 1738) indicating that biometric authentication was not successful (e.g., failed). ) replaces the display with it). Optionally, the electronic device outputs a tactile output 1750 indicating that biometric authentication was not successful. In some examples, tactile output 1750 is the same as tactile output 1742. In some examples, tactile output 1750 is different from tactile output 1742. After indicating that biometric authentication was not successful, the electronic device stops displaying the biometric authentication interface, as illustrated in FIG. 17L.

In some examples, biometric authentication interface 1732 includes animation and/or one or more of the biometric authentication glyphs of biometric authentication interface 1732 are animated. As an example, biometric authentication glyph 1738 includes rings with a spherical rotation and/or biometric authentication glyph 1746 moves from side to side to simulate a “shake” movement.

Referring to FIG. 17M , in some examples, further in response to unsuccessful biometric authentication, electronic device 1700 displays a failure interface, such as failure interface 1752. The failure interface includes a biometric authentication glyph (1754), an alternative authentication affordance (1756), a retry affordance (1758), and a cancel affordance (1760). In some examples, activation of retry affordance 1758 causes the electronic device to initiate biometric authentication again, as described above. In some examples, the electronic device performs biometric authentication only if a threshold number of failed biometric authentication attempts have not been made. In some examples, activation of the cancel affordance causes electronic device 1700 to stop displaying failure interface 1752.

Referring to Figure 17N, in response to activation of an alternative authentication affordance 1756, such as a tap gesture 1762, electronic device 1700 displays an alternative authentication interface 1766 (Figure 17O) (e.g., replacing the display of the failure interface 1752 with it, which allows the user to use alternative forms of authentication (e.g., fingerprint authentication, password authentication, passcode authentication, pattern authentication, etc.) in addition to authentication associated with the biometric feature. Authentication may be performed using a method (including selection of a plurality of items in a predefined pattern or movement of a touch or other input in a predefined pattern). As shown in Figure 17O, the user optionally touches the electronic device's fingerprint sensor 1764 with a finger to authenticate.

17P illustrates another example failure interface 1766 that includes an alternative authentication affordance 1770. 17Q, while displaying failure interface 1766, electronic device 1766 detects activation of alternative authentication affordance 1770. As an example, the activation is a tap gesture 1776 on the login affordance 1770. In response to detecting the tap gesture 1776, the electronic device 1700 displays an alternative authentication interface 1778. In some examples, alternative authentication interface 1778 is a password (or passcode) interface that allows a user to provide a password (or passcode) to authenticate.

17R, in response to authentication (e.g., alternative authentication), the secure data fillable field is automatically populated with the password 1743 and, optionally, the non-secure data fillable field is automatically populated with the username 1728. . In this way, users can optionally build an auto-fill feature despite unsuccessful biometric authentication. While displaying an application interface with automatically populated fillable fields 1710 and 1720, the electronic device detects activation of the submission affordance 1714. As an example, the activation is a tap gesture 1780 on the submit affordance 1714. In response, the user is optionally authenticated with the application and the electronic device optionally presents a home interface, such as home interface 1782 in Figure 17S.

17T, electronic device 1700 displays application interface 1784 on display 1702, including a secure data fillable field 1786. In response to a request to automatically populate secure data fillable field 1786 (e.g., selection of secure data fillable field 1786), electronic device 1700 provides autofill affordance 1790, as illustrated. Displays an input interface 1788 including.

While displaying the auto-fill affordance 1790 of the input interface 1788, the electronic device 1700 detects activation of the auto-fill affordance 1790. For example, as shown in Figure 17U, the activation is a tap gesture 1792 on the autofill affordance 1792.

17V-17X, in response to detecting the tap gesture 1792, the electronic device 1700 initiates biometric authentication, as determined based on biometric data corresponding to the biometric characteristic. Likewise, determine whether at least a portion of the biometric features satisfies the biometric authentication criteria, at least described with reference to FIGS. 17G-17I.

17Z, in response to successful biometric authentication, the electronic device 1700 displays a candidate selection interface 1794 that includes a plurality of candidate input affordances 1792 to automatically populate the secure data fillable field 1786. Display (e.g., replace the display of the biometric authentication interface 1732 with it). In some examples, candidate selection interface 1794 is displayed without a keyboard. In the illustrated example, fillable field 1786 is associated with credit cards (e.g., fillable field 1786 is flagged as being associated with financial transactions). Accordingly, in some examples, each of the candidate input affordances 1792 serves as a reference to a respective credit card (e.g., a credit card number and/or one or more other respective candidate values associated with the credit card).

While displaying candidate input affordances 1792, electronic device 1700 detects activation of one candidate input affordance 1792. For example, as shown in Figure 17Z, the activation is a tap gesture 1795 on one candidate input affordance 1792. 17Z, in response to detecting a tap gesture 1795, the electronic device 1700 automatically populates the secure data fillable field with a candidate input 1796 that corresponds to the activated candidate input affordance 1792.

While displaying application interface 1784 with automatically populated fillable fields 1786, the electronic device detects activation of submission affordance 1798. As an example, the activation is a tap gesture 1702A on the submit affordance 1798. In response, an automatically populated credit card is optionally submitted using the application, for example, for authentication or payment purposes.

Although descriptions herein refer to performing biometric authentication prior to providing candidate input affordances when automatically populating secure data fillable fields, in some instances, candidate input affordances are provided prior to biometric authentication. You will understand. Referring to FIG. 17AA, for example, in response to a request to automatically populate secure data fillable field 1786, electronic device 1700 displays an input interface including a plurality of candidate input affordances 1704A. do. Each of the candidate inputs 1704A is, in some examples, a reference to (e.g., a representation of) a candidate input value.

As illustrated in Figure 17ab, while displaying an input interface that includes a plurality of candidate input affordances 1704A, the electronic device detects activation of one candidate input affordance 1704A. As an example, the activation is a tap gesture 1706A on one candidate input affordance 1704A. Referring to Figures 17ac-17ae, in response, the electronic device performs biometric authentication as described. 17AF, electronic device 1700 has determined that biometric authentication was successful and automatically populates secure data fillable field 1786 with the selected candidate input corresponding to selected candidate input affordance 1704A.

In Figure 17ag, the electronic device instead determines that biometric authentication was not successful. In response, electronic device 1700 stops displaying the biometric authentication interface, as illustrated in FIG. 17A.

As previously mentioned, the illustrative example of the user interfaces illustrated in FIGS. 17A-17AH described above is related to the illustrative example of the user interfaces illustrated in FIGS. 18A-18AH described below. Accordingly, the processes described above with respect to the example user interfaces illustrated in FIGS. 17A-17AF and the processes described below with respect to the example user interfaces illustrated in FIGS. 18A-18AH may be used with an electronic device (e.g. , 100, 300, 500, 700) are very similar processes that similarly involve initiating and managing transfers.

18A-18D are flow diagrams illustrating a method for performing biometric authentication using an electronic device, according to some examples. Method 1800 includes a device having a display, one or more input devices (e.g., a touchscreen, microphone, camera), and a wireless communication radio (e.g., a Bluetooth connection, a WiFi connection, a mobile broadband connection such as a 4G LTE connection). For example, 100, 300, 500, 1700). In some examples, the display is a touch-sensitive display. In some examples, the display is not a touch-sensitive display. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the device includes one or more biometric sensors, optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device, such as an IR floodlight, a structured light projector, or a combination thereof. The light-emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric feature by one or more biometric sensors. Some operations of method 1800 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 1800 provides an intuitive method for performing authentication of biometric characteristics. This method reduces the cognitive burden on the user for performing authentication of biometric features, thereby creating a more efficient human-machine interface and intuitive user experience. For battery-operated computing devices, allowing users to manage authentication of biometric features faster and more efficiently saves power and increases time between battery charges.

In some examples, the electronic device (e.g., 100, 300, 500, 1700) detects (1802) selection of a fillable field (e.g., 1710, 1712, 1786). In some examples, in response to detecting selection of a fillable field (e.g., 1710, 1712, 1786), the electronic device (e.g., 100, 300, 500, 1700) selects a fillable field (e.g., 1710, 1712, An input interface (e.g., 1720, 1788) including a plurality of user interface objects (e.g., 1725, 1793, 1704A) corresponding to candidate inputs (1786) is displayed (1804).

In some examples, before receiving a request (e.g., 1718, 1724, 1726, 1730, 1792, 1795, 1706A) to automatically fill at least one fillable field (e.g., 1710, 1712, 1786), the electronic device (e.g., E.g., 100, 300, 500, 1700) receives a selection (e.g., 1718, 1730) of a fillable field (e.g., 1710, 1712, 1786). In some examples, selection (e.g., 1718, 1730) of a fillable field (e.g., 1710, 1712, 1786) may be performed by selecting a fillable field (e.g., 1710, 1730) displayed within an application interface using an input device such as a mouse or button. 1712, 1786) is the user's choice. In some examples, in response to selection of a fillable field (e.g., 1710, 1712, 1786), the electronic device (e.g., 100, 300, 500, 1700) displays an auto-fill affordance (e.g., 1722, 1790) ( 1806). In some examples, the autofill affordance (e.g., 1722, 1790) is displayed in combination with a keyboard (or keypad).

In some examples, the electronic device (e.g., 100, 300, 500, 1700) displays (1808) an application interface that includes a fillable field (e.g., 1710, 1712, 1786) on the display. Displaying an application interface that includes fillable fields provides visual feedback to the user indicating that input can be made to a particular area of the application interface. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, displaying an application interface that includes a fillable field (e.g., 1710, 1712, 1786) on a display (e.g., 1702) may cause the fillable field (e.g., 1712, 1786) to be of a second type. and displaying (1810) a fillable field (e.g., 1712, 1786) as associated with data using a first visual process. Displaying the fillable field using a particular visual process (e.g., a first visual process) may cause the fillable field to be associated with a particular type of data (e.g., a second type) as the fillable field is associated with a particular type of data. Provides visual feedback that allows users to quickly and easily recognize that something is happening. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the second type of data includes data that requires authentication to be automatically populated, such as payment information, password, and/or user name. In some examples, the first visual treatment is a visual effect, such as a particular color scheme, highlighting, or animation. In some examples, the first visual processing includes a first color combination, such as a pattern of one or more colors. In some examples, the first visual processing includes a biometric authentication interface object (e.g., 1716) associated with (e.g., within or adjacent to) a fillable field (e.g., 1712, 1786).

In some examples, the electronic device (e.g., 100, 300, 500, 1700) displays a biometric authentication glyph (e.g., 1716) or icon within or near fields associated with biometric authentication (e.g., 1712, 1786). It is not displayed in or near fields not associated with biometric authentication (e.g., 1710). Displaying biometric authentication glyphs or icons in or near fields that are associated with biometric authentication and not displaying biometric glyphs or icons in or near fields that are not associated with biometric authentication determines which fields are Provides easily recognizable visual feedback on which fields are associated with and require biometric authentication and which fields are neither associated with nor require biometric authentication. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, displaying an application interface that includes a fillable field on a display may include creating a fillable field (e.g., 1710) as the fillable field (e.g., 1710) is associated with data of a first type. and displaying using a second visual process that is different from the first visual process (1812). In some examples, the first type of data includes data that does not require authentication to be automatically populated, such as contact information including name, address, phone number, zip code, etc. In some examples, the second visual processing is the absence of the first visual processing. In some examples, an electronic device (e.g., 100, 300, 500, 1700) may display a fillable field (e.g., 1712, 1786) with a different color, a biometric authentication glyph (e.g., 1716), and/or a fillable field (e.g., 1712, 1786). For example, 1712, 1786) are optionally highlighted using text indicating that they are automatically populated in response to successful biometric authentication. In some examples, the second visual processing includes a second color combination that is different than the first color combination. Accordingly, in some examples, an electronic device (e.g., 100, 300, 500, 1700) combines fields associated with biometric authentication (e.g., 1712, 1786) with fields not associated with biometric authentication (e.g., 1710 ) are displayed using different colors.

In some examples, displaying an application interface that includes a fillable field (e.g., 1710, 1712, 1786) on a display may include displaying a webpage that includes a fillable field (e.g., 1710, 1712, 1786). Includes (1814). In some examples, the application interface further includes a fillable field (e.g., 1710, 1712, 1786) and an associated submission affordance (e.g., 1714, 1798).

In some examples, while displaying an application interface, an electronic device (e.g., 100, 300, 500, 1700) may receive a request (e.g., 1718, 1724, 1726, 1730, 1792, 1795, 1706A) receives (1816). In some examples, the request may include selection of an autofill affordance (e.g., 1722, 1790) (e.g., 1724, 1792), selection of a field (e.g., 1718, 1730), selection of a candidate text input (e.g., 1726, 1795, 1706A). ), loading a web page, or any combination thereof. In some examples, receiving a request to automatically populate at least one fillable field (e.g., 1710, 1712, 1786) of an application interface may be performed on a display (e.g., , 1702) and receiving 1818 a selection of an autofill affordance (e.g., 1722, 1790). In some examples, in response to selection (e.g., 1710, 1712, 1786) of a field (e.g., 1710, 1712, 1786), the electronic device (e.g., 100, 300, 500, 1700) selects a fillable field (e.g., 1710). , 1712, 1786) displays a keyboard (or keypad) including an affordance (e.g., 1722, 1790) to automatically fill. In response to selection of the affordance, the electronic device (eg, 100, 300, 500, 1700) initiates biometric authentication. In some examples, receiving a request to automatically fill at least one fillable field (e.g., 1710, 1712, 1786) of an application interface may include selecting the fillable field (e.g., 1710, 1712, 1786) (e.g., 1718, 1730) and receiving (1820).

In some examples, in response to selection of a fillable field (e.g., 1710, 1712, 1786), an electronic device (e.g., 100, 300, 500, 1700) may select a biometric device without displaying an input interface (e.g., 1720, 1788). Measurement certification begins. Initiating biometric authentication without displaying an input interface in response to selection of a fillable field allows the user to quickly and efficiently initiate biometric authentication with minimal input. Reducing the number of inputs required to perform an operation improves the operability of the device and improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). -Makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, an input interface (e.g., 1720, 1788) is displayed in response to selection of a first type of field (e.g., 1786) (e.g., a credit card field) and a second type of field (e.g., 1712). ) (e.g., the password field) is not displayed in response to selection. In some examples, receiving a request to automatically populate at least one fillable field (e.g., 1710, 1712, 1786) of an application interface includes receiving a reference (e.g., 1725) corresponding to a candidate input associated with the second type of data. , 1793, 1704A) and receiving (1822) a selection (e.g., 1726, 1795, 1706A). In some examples, an electronic device (e.g., 100, 300, 500, 1700) provides one or more candidate inputs that can be used (e.g., upon selection) to automatically fill a fillable field (e.g., 1710, 1712, 1786). Provide one or more references (e.g., 1725, 1793, 1704A) corresponding to . In some examples, a reference is a reference to a credit card (eg, “CC1”) or a reference to a password (eg, “Facebook password”), for example. In some examples, the reference is the candidate itself (eg, an email address such as “[email protected]”). In some examples, selection of a reference (e.g., 1725, 1793, 1704A) for a candidate input (e.g., 1726, 1795, 1706A) is a selection of the affordances of a software keyboard. In some examples, the keyboard is a keypad. In some examples, receiving a request to automatically fill at least one fillable field of the application interface includes selecting 1824 a fillable field of the webpage. In some examples, receiving a request to automatically populate a fillable field of an application interface may include selecting a user interface object (e.g., 1725, 1793, 1704A) corresponding to each candidate input of the plurality of candidate inputs (e.g., , 1726, 1795, 1706A). In some examples, in response to selection of a fillable field, the electronic device (e.g., 100, 300, 500, 1700) provides candidate inputs (e.g., 1725, 1793, 1704A) for selection by the user. Afterwards, the electronic device (eg, 100, 300, 500, 1700) performs biometric authentication. In some examples, the electronic device (e.g., 100, 300, 500, 1700) identifies all fillable fields (e.g., 1710, 1712, 1786) and/or fills the fields (e.g., 1710) when the application interface is loaded. , 1712, 1786), determine candidate inputs for one or more fields. In some examples, automatically filling in this manner reduces the number of inputs required to automatically fill a fillable field (e.g., 1710, 1712, 1786). In some examples, the request to automatically fill a fillable field (e.g., 1710, 1712, 1786) is based on detection of loading a webpage that includes a fillable field (e.g., 1710, 1712, 1786).

In some examples, in response to receiving a request to automatically populate a fillable field (e.g., 1710, 1712, 1786) of the application interface (1828), a fillable field of the application interface (e.g., 1710, 1712, 1786) Upon determining that is associated with data of the first type, the electronic device (e.g., 100, 300, 500, 1700) automatically fills the fillable field (e.g., 1710, 1712, 1786) with data of the first type ( 1830). Automatically populating a fillable field with a particular type of data (e.g., a first type of data) based on a determination that the fillable field in the application interface is associated with a particular type of data (e.g., a first type of data) may cause the user to Avoids having to manually enter data into fillable fields in the application interface. Reducing the number of inputs required to perform an operation improves the operability of the device and improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). -Makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the first type of data includes unsecured or unsecured (eg, non-biometrically secure) data. In some examples, the non-secure data is the user's given name, nickname, publicly available phone number, or preferences associated with a particular field (e.g., shoe size for the shoe size field). In some examples, automatically populating a fillable field (e.g., 1710, 1712, 1786) may cause that field to be further authenticated in response to a request (e.g., 1718, 1724, 1726, 1730, 1792, 1795, 1706A). Accessible to an electronic device (e.g., 100, 300, 500, 1700) or populating with data stored by an electronic device (e.g., 100, 300, 500, 1700) without requiring additional biometric authentication Includes (populating).

In some examples, further in response to a request to automatically populate a fillable field (e.g., 1710, 1712, 1786) of the application interface, the fillable field (e.g., 1710, 1712, 1786) of the application is of a second type. While acquiring data corresponding to the biometric characteristic from one or more biometric sensors (e.g., 1703) (e.g., during at least a portion of the acquisition process), the electronic device (e.g., 100, 300, 500, 1700) displays (1834) a biometric authentication interface (e.g., 1732). Displaying a biometric authentication interface in accordance with a determination that a fillable field in the application is associated with a particular type of data (e.g., a second type of data) may cause the data to be of a particular type (e.g., a second type). In this case, device security is improved by requiring security verification measures. Improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally reduces the device's power usage by enabling users to use the device more efficiently. reduces and improves battery life.

In some examples, the second type of data is secure data (eg, biometric secure data). In some examples, secure data includes password information, credit card information, non-public user information such as unlisted phone numbers, or medical information. In some examples, an electronic device (e.g., 100, 300, 500, 1700) displays a biometric authentication interface (e.g., 1732) while performing biometric authentication. In some examples, the biometric authentication interface is displayed over at least a portion of the application interface. In some examples, displaying the biometric authentication interface includes displaying a biometric authentication animation. In some examples, the biometric authentication animation includes an initial animation (e.g., display of the first biometric authentication glyph (e.g., 1734)), a processing animation (e.g., rotating rings indicating that biometric data is being processed), and a success animation. Contains either animation or failure animation. In some examples, the failure animation is the same as the initial animation. These features are described in greater detail above with reference to FIGS. 15A-15T. In some examples, the biometric authentication interface includes 1836 a representation of a simulation of biometric characteristics (e.g., 1734, 1738, 1740, 1746). In some examples, the biometric authentication interface includes a representation of a simulation of biometric features representing the state of the biometric authentication sequence (e.g., 1734, 1738, 1740, 1746). In some examples, the biometric feature is a face and the representation (eg, 1734, 1738, 1740, 1746) is a simulation of a face.

In some examples, further in response to a request to automatically populate a fillable field and in response to a determination that the fillable field in the application is associated with a second type of data, the electronic device (e.g., 100, 300, 500, 1700) ) determines whether a number of candidate inputs (e.g., associated with the second type of data) are stored on the electronic device (e.g., 100, 300, 500, 1700). Additionally, in some examples, upon determining that a number of candidate inputs (e.g., 1793, 1704A) associated with a second type of data are stored on the electronic device (e.g., 100, 300, 500, 1700), the electronic device A device (eg, 100, 300, 500, 1700) displays a number of candidates. Additionally, in some examples, the electronic device (eg, 100, 300, 500, 1700) receives a selection of one candidate input from a plurality of displayed candidate inputs. Additionally, in some examples, in response to receiving a selection of candidate input (e.g., 1704A), the electronic device (e.g., 100, 300, 500, 1700) receives a signal from one or more biometric sensors (e.g., 1703). , obtain data corresponding to at least a portion of the biometric features. In some examples, automatically populating a fillable field (e.g., 1712, 1786) with data of a second type includes automatically populating a fillable field (e.g., 1712, 1786) with a selected candidate input (e.g., 1704A). Includes. In some examples, the electronic device (e.g., 100, 300, 500, 1700) determines whether a number of candidate inputs are stored on the electronic device (e.g., 100, 300, 500, 1700) prior to performing biometric authentication. Decide. In some examples, once the user has selected a candidate input (e.g., 1704A), the electronic device (e.g., 100, 300, 500, 1700) performs biometric authentication.

In some examples, further determine in response to a request to automatically populate a fillable field (e.g., 1710, 1712, 1786) and based on data obtained from one or more biometric sensors corresponding to a biometric characteristic. Upon determining that at least a portion of the biometric characteristic satisfies the biometric authentication criteria (1838), the electronic device (e.g., 100, 300, 500, 1700) generates a fillable field (e.g., 1710, 1712, 1786). Automatically fills with data of the second type (1840). Automatically populating a fillable field with data of a particular type (e.g., a second type) upon determining that at least a portion of the biometric characteristic satisfies biometric authentication criteria may require the user to manually enter data into the fillable field. Avoid doing what you need to do. Reducing the number of inputs required to perform an operation improves the operability of the device and improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). -Makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, if biometric authentication is successful, the electronic device (e.g., 100, 300, 500, 1700) automatically populates the fillable field (e.g., 1712, 1786) with information in response to the request. Automatically populating fillable fields avoids the user having to manually enter data into the fillable fields. Reducing the number of inputs required to perform an operation improves the operability of the device and improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). -Makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, in response to receiving a request (e.g., 1718, 1724, 1726, 1730, 1795, 1706A) to automatically populate a fillable field (e.g., 1712, 1786) of an application interface, an electronic device (e.g., 100, 300, 500, 1700) obtains data corresponding to at least a portion of a biometric characteristic from one or more biometric sensors. In some examples, data obtained from one or more biometric sensors is obtained prior to receiving a request to automatically populate fillable fields of the application interface. In some examples, data obtained from one or more biometric sensors is obtained in response to receiving a request to automatically populate a fillable field (e.g., 1712, 1786) of the application interface. In some examples, data obtained from one or more biometric sensors (e.g., 1703) is obtained in accordance with a determination that a fillable field of the application (e.g., 1712, 1786) is associated with a second type of data. In some examples, an electronic device (e.g., 100, 300, 500, 1700) may respond to a request to automatically fill a fillable field without displaying an input interface (e.g., 1720) (e.g., a keyboard or keypad). Automatically fills in fillable fields (e.g., 1712, 1786). In some examples, one or more biometric sensors (eg, 1703) include a camera (eg, an IR camera or a thermographic camera). In some examples, data obtained from one or more biometric sensors (e.g., 1703) corresponding to a biometric feature includes biometric data acquired using a camera. In some examples, the biometric feature is a face. In some examples, data obtained from one or more biometric sensors (e.g., 1703) corresponding to a biometric feature includes biometric data associated with a portion of the face, and the biometric authentication criteria include biometric authentication criteria. These include the requirement that the biometric data associated with the face match the biometric data associated with the authorized face in order to be met.

In some examples, upon determining that the electronic device (e.g., 100, 300, 500, 1700) has accessed a single candidate value of a second type for filling a fillable field (e.g., 1712, 1786), the electronic device (e.g., 100, 300, 500, 1700) automatically populates fillable fields (e.g., 1712, 1786) with data of the second type. In some examples, in accordance with a determination that the electronic device (e.g., 100, 300, 500, 1700) has accessed a plurality of candidate values of a second type for automatically populating a fillable field (e.g., 1712, 1786), the electronic device: (eg, 100, 300, 500, 1700) displays representations of a plurality of candidate values. In some examples, the candidate values are stored directly on the device and/or are stored directly on the electronic device (e.g., 100, 300, 500, 1700) from another electronic device (e.g., 100, 300, 500, 1700). (e.g., 100, 300, 500, 1700) are accessible. In some examples, while displaying representations of a plurality of candidate values (e.g., 1725, 1793, 1704A), the electronic device (e.g., 100, 300, 500, 1700) displays each candidate value among the plurality of candidate values. Receive a selection (e.g., 1726, 1795, 1706A) of a representation (e.g., 1725, 1793, 1704A) and, in some examples, automatically populate a fillable field (e.g., 1712, 1786) with the respective candidate value. In some examples, the electronic device (e.g., 100, 300, 500, 1700) determines whether the electronic device (e.g., 100, 300, 500, 1700) has accessed multiple instances of the second type of data. . In some examples, in response to successful biometric authentication, the electronic device (e.g., 100, 300, 500, 1700) receives a number of candidate inputs (e.g., candidate credit cards) of biometric security data, e.g. Determines whether or not data is stored on this device. If so, the electronic device (eg, 100, 300, 500, 1700) presents (eg, displays) each of the candidates (eg, 1725, 1793, 1704A) to the user. In response to a user selection (e.g., 1726, 1795, 1706A) of one of the candidates (e.g., 1725, 1793, 1704A), the electronic device (e.g., 100, 300, 500, 1700) selects a field using the selected candidate. (e.g., 1712, 1786) is automatically filled in.

In some examples, based on data obtained from one or more biometric sensors, the electronic device (e.g., 100, 300, 500, 1700) determines that at least a portion of the biometric characteristic does not meet biometric authentication criteria. suspends (1842) automatically populating fillable fields (e.g., 1712, 1786) with data of the second type. Suspending automatically populating a field using the selected candidate upon determining that at least a portion of the biometric feature does not meet the biometric authentication criteria provides visual feedback by allowing the user to recognize that authentication was not successful; , Additionally, provides improved device security by withholding automatic filling of fillable fields without successful authentication. Providing improved visual feedback to the user allows the user to provide appropriate inputs when operating/interacting with the device (e.g., by improving the readability of user interface elements for the user while the device is at natural viewing angles). improves the operability of the device and makes the user-device interface more efficient (by helping users use the device more quickly and efficiently), which in addition reduces the device's power usage. and improve battery life. Additionally, improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally allows users to use the device more efficiently. Reduces power usage and improves battery life.

In some examples, based on data obtained from one or more biometric sensors, the electronic device (e.g., 100, 300, 500, 1700) determines that at least a portion of the biometric characteristic does not meet biometric authentication criteria. displays an indication that at least a portion of the biometric feature does not meet biometric authentication criteria (1844). Displaying an indication that at least a portion of the biometric feature did not meet biometric authentication criteria provides visual feedback by allowing the user to quickly recognize that authentication was not successful. Providing improved visual feedback to the user allows the user to provide appropriate inputs when operating/interacting with the device (e.g., by improving the readability of user interface elements for the user while the device is at natural viewing angles). improves the operability of the device and makes the user-device interface more efficient (by helping users use the device more quickly and efficiently), which in addition reduces the device's power usage. and improve battery life. In some examples, in response to failed biometric authentication, the electronic device (eg, 100, 300, 500, 1700) provides an indication of the failure. In some examples, the electronic device (e.g., 100, 300, 500, 1700) indicates “biometric feature not recognized” or “biometric authentication inactive” when a threshold number of biometric attempts is reached. Displays a message indicating . In some examples, after a failure, the electronic device (e.g., 100, 300, 500, 1700) removes any biometric authentication interface displayed over the application interface and/or provides a biometric authentication retry affordance (e.g., 1758). (e.g., within a fillable field (e.g., 1712)), the selection of which reattempts biometric authentication. In some examples, in response to determining that at least a portion of the biometric feature does not meet biometric authentication criteria, the device may enter data in a fillable field (e.g., 1712, 1786) (e.g., user name, password, passcode, Displays a keypad or keyboard for entering contact information, credit card information, etc.

In some examples, based on data obtained from one or more biometric sensors, the electronic device (e.g., 100, 300, 500, 1700) determines that at least a portion of the biometric characteristic does not meet biometric authentication criteria. stops displaying the biometric authentication interface. In some examples, after a failed biometric authentication, the electronic device (eg, 100, 300, 500, 1700) stops displaying the biometric authentication. As a result, the electronic device (eg, 100, 300, 500, 1700) resumes display of an application interface, such as the application's login interface (eg, 1714).

In some examples, based on data obtained from one or more biometric sensors, the electronic device (e.g., 100, 300, 500, 1700) determines that at least a portion of the biometric characteristic does not meet biometric authentication criteria. Displays an input interface (e.g., 1720). In some examples, the input interface (e.g., 1720) includes a keypad or keyboard that includes character input keys for entering a password or passcode.

In some examples, upon determining that biometric authentication is not available, the electronic device (eg, 100, 300, 500, 1700) prompts the user for an alternative form of authentication. Prompting the user for an alternative form of authentication following a determination that biometric authentication is not available allows the user to easily provide authentication for the operation using a different authentication method. Providing additional control options (e.g., to provide authentication) in this manner without cluttering the UI with additional displayed controls allows the user to enter appropriate inputs (e.g., when operating/interacting with the device). improves device operability and makes the user-device interface more efficient (by helping provide and reduce user errors), which additionally reduces the device's power usage by allowing users to use the device more quickly and efficiently. Reduces battery life and improves battery life.

In some examples, a threshold number of failed biometric authentication attempts has been reached since the last successful authentication by the device, or the biometric sensor is unable to detect characteristics of the biometric characteristic (e.g., heat, cold, light, etc.). Biometric authentication fails because it cannot be used due to insufficient light (not enough light, too much light), or other environmental conditions. In some examples, after prompting the user for an alternative form of authentication (e.g., a passcode, password, or another form of biometric authentication such as a fingerprint), the electronic device (e.g., 100, 300, 500, 1700) Receive alternative forms of authentication. In some examples, additionally, after prompting the user for the alternative form of authentication, in response to receiving the alternative form of authentication, the alternative form of authentication may be performed using authorized authentication information (e.g., a previously stored fingerprint, Upon determining that it matches the password, or passcode), the electronic device (e.g., 100, 300, 500, 1700) automatically populates the fillable field (e.g., 1712, 1786). In some examples, further after prompting the user for the alternative form of authentication, and further in response to receiving the alternative form of authentication, a notification is made that the alternative form of authentication does not match authorized authentication information. Depending on the decision, the electronic device (e.g., 100, 300, 500, 1700) refrains from automatically filling the fillable field (e.g., 1712, 1786).

In some examples, after responding to a request to automatically populate a fillable field of an application interface, the electronic device (e.g., 100, 300, 500, 1700) receives a subsequent request to load a webpage. In some examples, additionally, after responding to a request to automatically populate a fillable field (e.g., 1710, 1712, 1786) of the application interface, in response to a subsequent request to load a webpage, Upon determining that the request meets authentication retry criteria, the electronic device (e.g., 100, 300, 500, 1700) performs biometric authentication to automatically populate fillable fields (e.g., 1710, 1712, 1786) in the application interface. Try . In some examples, further in response to a request to automatically populate a fillable field (e.g., 1710, 1712, 1786) of the application interface, and further in response to a subsequent request to load a webpage, Upon determining that a subsequent request to load does not meet the authentication retry criteria, the electronic device (e.g., 100, 300, 500, 1700) automatically populates a fillable field (e.g., 1710, 1712, 1786) in the application interface. Hold off on attempting biometric authentication to fill in. In some examples, loading a webpage conditionally triggers auto-population based on predetermined criteria. For example, loading a web page is treated as a request to automatically populate fillable fields on the web page when the web page is loaded for the first time, but when the web page is loaded within a predetermined time (e.g., 5 minutes, 1 hour). , or within 1 day), the second time the web page is loaded, it is not treated as such. In some examples, the authentication retry criteria include at least one of the following: a requirement that the webpage was not loaded within a predetermined time or a requirement that the webpage was not loaded during the same session. In some examples, the requirement is that the loading is the first instance of loading within a predetermined time and/or that the loading is the first instance of loading within a session.

In some examples, after automatically populating a fillable field (e.g., 1710, 1712, 1786) with the first type of data or the second type of data, the electronic device (e.g., 100, 300, 500, 1700) submits the submission affordance. (e.g., 1714, 1798). In some examples, in response to receiving a selection of a submission affordance, the electronic device (eg, 100, 300, 500, 1700) stops displaying the application interface. In some examples, further in response to receiving a selection of a submission affordance, the electronic device (e.g., 100, 300, 500, 1700) displays a second interface (e.g., 1782) created by the application. In some examples, displaying the second interface includes replacing the login user interface of the application with the user interface of the application that includes the protected information (e.g., 1782).

Note that the details of the processes described above with respect to method 1200 (e.g., FIGS. 18A-18D) are also applicable in a similar manner to the methods described herein. For example, method 1800 optionally includes one or more of the features of the various methods described herein with reference to methods 800, 1000, 1200, 1400, 1600, 2000, 2200, 2500, 2700. Includes. For example, the registered biometric data described in method 1200 may be used to perform biometric authentication as described with respect to FIGS. 17G-17K. For another example, one or more implantable interfaces as described in methods 2000 and 2700 are optionally displayed in response to receipt of input prior to completion of the biometric authentication process. For brevity, these details are not repeated herein.

Operations in the information processing methods described above optionally utilize one or more functional modules within an information processing device, such as general-purpose processors (e.g., as described with respect to FIGS. 1A, 3, and 5A) or application-specific chips. It is implemented by running it. Additionally, the operations described above with reference to FIGS. 18A-18D are optionally implemented by components shown in FIGS. 1A and 1B. For example, display operation 1808, receive operation 1816, and autofill operation 1830 are optionally implemented by event sorter 170, event recognizer 180, and event handler 190. Event monitor 171 in event sorter 170 detects contact on touch-sensitive surface 604, and event dispatcher module 174 passes event information to application 136-1. The respective event recognizer 180 of application 136-1 compares the event information to the respective event definition 186 and determines that the first contact at the first location on the touch-sensitive surface corresponds to the selection of an object on the user interface. Determines whether it corresponds to the same predefined event or subevent. When an individual predefined event or sub-event is detected, event recognizer 180 activates an event handler 190 associated with detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some examples, event handler 190 accesses the respective GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes may be implemented based on the components shown in FIGS. 1A and 1B.

19A-19AB illustrate example user interfaces for biometric authentication, according to some examples. As described in more detail below, the illustrative examples of user interfaces illustrated in Figures 19A-19AB are used to illustrate the processes described below, including the processes in Figures 20A-20F.

FIG. 19A illustrates an electronic device 1900 (e.g., portable multifunction device 100, device 300, or device 500). In the illustrative example illustrated in FIGS. 19A-19AB , electronic device 1900 is a smartphone. In other examples, electronic device 1900 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 1900 has a display 1902, one or more input devices (e.g., a touchscreen of display 1902, buttons 1904, a microphone (not displayed)), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the electronic device includes one or more biometric sensors (e.g., biometric sensor 1903), optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, one or more biometric sensors 1903 are one or more biometric sensors 703. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In Figure 19A, the electronic device wakes from a low-power (e.g., display-off) state. As illustrated, in some examples, electronic device 1900 wakes in response to a lift gesture 1906 performed by a user. 19B-19D, in response to the lift gesture 1906, the electronic device 1900 transitions from a low power state to a moderate power state (e.g., display-dimmed). For example, in Figure 19B, display 1902 of electronic device 1900 is disabled and, in response to lift gesture 1906, electronic device 1900 displays a predetermined display as shown in Figures 19C and 19D. Gradually increases the brightness of display 1902 over a period of time. In some examples, the brightness of display 1902 increases according to a function, such as a linear function. In some examples, when biometric authentication (e.g., facial recognition authentication) is enabled, the device is immediately locked when a hardware button (e.g., sleep/wake button) is pressed, and in some examples, the device is placed in sleep mode. It is locked every time it transitions.

19C and 19D , in some examples, while transitioning to and/or operating in a medium power state (e.g., display on, but not at full operating brightness), the electronic device is locked. Displays the interface 1910. The locked interface includes, for example, a locked status indicator 1912 and, optionally, one or more notifications 1914. As shown, notification 1914 is a message notification associated with a messaging application indicating that the electronic device has received a new message from a contact (“John Appleseed”) stored on the electronic device. In some examples, the medium power state is a locked state. Accordingly, while operating in a medium power state, electronic device 1900 operates in a secure manner. By way of example, while operating in a medium power state, the electronic device does not display the content of the message associated with notification 1914. In some examples, the locked state further corresponds to restrictions on access to other data (including other applications) and/or restrictions on permitted inputs.

In some examples, electronic device 1900 additionally displays a flashlight affordance 1907 and a camera affordance 1908. In some examples, activation of flashlight affordance 1907 causes the electronic device to load a flashlight application. In some examples, activation of camera affordance 1908 causes electronic device 1900 to load a camera application.

In some examples, after transitioning to (eg, in response to) a medium power state, electronic device 1900 initiates biometric authentication (eg, facial recognition authentication). In some examples, initiating biometric authentication includes obtaining (eg, capturing with one or more biometric sensors) data corresponding to at least a portion of the user's biometric characteristics. In some examples, when (the user's) face is detected, biometric authentication verifies (the user's) attention and intent to unlock by detecting that the user's eyes are open and facing the device.

19E-19G, when the electronic device 1900 determines that biometric authentication is successful, the electronic device transitions from a medium power state to a high power state (e.g., the display is not dimmed). For example, in Figure 19D, the display of electronic device 1900 is in a medium power state, and in response to successful biometric authentication, electronic device 1900 displays the Gradually increases the brightness of display 1902 over a determined period of time. In some examples, the brightness of display 1902 increases according to a function, such as a linear function.

In some examples, while transitioning from a medium power state to a high power state, electronic device 1900 displays unlocked interface 1920. In some examples, while displaying unlocked interface 1920, the electronic device displays an animation indicating that the electronic device is transitioning to a high power state. As illustrated in Figures 19E-19G, during the transition, the electronic device displays an animation of the locked state indicator 1912 transitioning to the unlocked state indicator 1922 (Figure 19G). In some examples, displaying the animation includes displacing and/or increasing the size of the locked status indicator 1912 to display an unlocked status indicator 1913 (FIG. 19E), and unlocking. and raising and rotating the latch of the unlocked status indicator to display the unlocked status indicators (1921 in FIG. 19F and 1922 in FIG. 19G), respectively. In some examples, the degree of blurriness of one or more objects in the locked state interface 1910 and/or the unlocked state interface 1920 changes during the animation. In some examples, electronic device 1900 further outputs tactile output 1926 during or in response to transitioning to a high power state (FIG. 19G).

In some examples, the high power state is an unlocked state. Accordingly, while operating in a high power state, electronic device 1900 operates in a non-secure manner (eg, secure data is accessible to authenticated users). For example, as illustrated in FIG. 19G, while operating in a high power state, the electronic device displays the content of a message associated with notification 1914.

In some examples, biometric authentication (e.g., facial recognition authentication) augments a user's stored mathematical representation over time to improve unlocking performance and follow natural changes in a user's face and appearance. In some examples, upon a successful unlock, the biometric authentication optionally uses a newly calculated mathematical expression - if its quality is sufficient - for a finite number of additional unlocks before the data is discarded. In some examples, biometric authentication fails to recognize the user, but the match quality is higher than a predetermined threshold and the user immediately resolves the failure (e.g., by entering an alternative authentication (e.g., passcode, password, pattern, fingerprint)). (within a predefined threshold time), the device takes another capture of the biometric data (e.g., through one or more cameras or other biometric sensors that capture facial recognition data) and converts it into a newly calculated mathematical expression. Augment his registered biometric authentication (eg, facial recognition authentication) data. In some examples, this new biometric authentication (eg, facial recognition authentication) data is optionally discarded after a finite number of unlocks and when the user stops matching against it. These augmentation processes allow biometric authentication (eg, facial recognition authentication) to follow dramatic changes in a user's facial hair or makeup usage, while minimizing false acceptance.

19E-19G, if electronic device 1900 determines that biometric authentication was not successful, electronic device 1900 does not transition to a high power state and, in some instances, remains in a medium power state. do. In some examples, electronic device 1900 remains locked while electronic device 1900 is maintained in a medium power state. To indicate that biometric authentication has failed, the electronic device 1900 displays a locked status indication, for example, by alternating the position of the locked status indicator 1912 between two positions on the locked status interface 1910. Simulates the shake of the ruler (1912). In some examples, electronic device 1900 further outputs tactile output 1918 to indicate that biometric authentication was not successful.

As described, while in a medium power state, electronic device 1900 is in a locked state and, as a result, secure data on the electronic device is not accessible while the electronic device is in a medium power state. As an example, in Figure 19I, the electronic device detects user input 1930 near the edge of display 1902. As illustrated in FIGS. 19I-19K , user input 1930 is, in some examples, a swipe gesture that is a request to access the home screen interface of electronic device 1900. However, because the electronic device 1900 is at medium power and in a locked state, in response to the swipe gesture, the electronic device 1900 slides the locked state interface 1910 in an upward direction to open the alternative authentication interface 1932. ), whereby the user is authenticated using an alternative form of authentication (e.g., password authentication) other than authentication associated with the biometric characteristic. The alternative authentication interface 1932 includes a locked status indicator 1934 and a prompt 1936 that tells the user that entering a valid passcode will unlock the electronic device 1900 (and optionally , which results in a transition to a high power state.

In some examples, an alternative form of authentication (eg, passcode, password, or pattern) is required to unlock the device in certain situations. In some examples, an alternative form of authentication is required when the device has just been turned on or restarted. In some examples, an alternative form of authentication is required if the device has not been unlocked for more than a predetermined time (eg, 48 hours). In some examples, an alternative form of authentication is required if the alternative form of authentication has not been used to unlock the device within a predetermined time (eg, 156 hours). In some examples, an alternative form of authentication may be used if the alternative form of authentication has not been used to unlock the device for a predetermined amount of time (e.g., 6.5 days) and biometric authentication (e.g., facial recognition authentication) has passed. Required if not used to unlock the device within a period of time (e.g., last 4 hours). In some examples, an alternative form of authentication is required when the device receives a remote lock command. In some examples, an alternative form of authentication is required after five unsuccessful attempts to match a face (via facial recognition authentication) on the device. In some examples, an alternative form of authentication is required after initiating a power off/emergency SOS on the device and then canceling the power off/emergency SOS.

19L and 19M, in response, at least in part, to a tap gesture 1938 (FIG. 19L) and, optionally, one or more other inputs representing additional alphanumeric digits of a valid passcode, electronic device 1900 A valid passcode (or password) is received by. As shown in FIG. 19N, once a valid passcode has been received, the electronic device is unlocked and displays home screen interface 1933 (e.g., replacing the display of an alternative authentication interface with it).

19O-19R, the device is operating in a high power (e.g., unlocked) state and receives input that is a request to access secure data on electronic device 1900. For example, as shown in Figure 19O, electronic device 1900 is operating in a high power state and a swipe gesture (as illustrated in Figure 19P) is a request to access the home screen interface of electronic device 1900. 1944). As further illustrated in FIGS. 19P-19R , in response to the swipe gesture 1944, the electronic device 1900 slides the unlocked status interface 1920 in an upward direction to display the home screen interface 1946. Display (e.g. reveal).

19S-19U illustrate various ways in which an electronic device transitions from a high power (eg, unlocked state) to a locked state, such as a medium power state or a low power state. In Figure 19S, while displaying unlocked status interface 1920 (at least as described with respect to Figure 19G), electronic device 1900 receives activation of unlocked status indicator 1922. Activation of the unlocked screen indicator 1922 is a tap gesture 1948 in some examples. As shown in FIG. 19V, in response to activation of the unlocked status indicator 1922, the electronic device transitions to a medium power state and, optionally, displays a locked status indicator 1912. Provides tactile output (1952). In some examples, while transitioning to the medium power state, the electronic device displays an animation indicating that the electronic device 1900 is transitioning to the medium power state (or low power state).

19T, while displaying home screen interface 1946 and while in a high-power unlocked state, electronic device 1900 receives activation of button 1904. Activation of button 1904 is, in some examples, pressing and/or pressing button 1904. In response to activation of button 1904, the electronic device transitions to a low power state (e.g., at least as described with reference to FIG. 19B). 19U, while displaying home screen interface 1946, electronic device 1900 receives activation of unlocked screen indicator 1950 of home screen interface 1946. Activation of the unlocked screen indicator 1922 is a tap gesture 1950 in some examples. In response to activation of the unlocked status indicator 1922, the electronic device transitions to a medium power state and, optionally, displays a locked status indicator 1910 (FIG. 19V).

In Figure 19W, electronic device 1900 displays device settings interface 1954. The device settings interface includes a gaze enablement setting 1955, which, when enabled, requires the user to be looking at the device for successful biometric authentication. When the setting is disabled, biometric authentication can be successful even if the authorized user is not looking at the device. Device settings interface 1954 further includes a biometric authentication enable setting 1956, which, when enabled, enables biometric authentication on electronic device 1900. When the biometric authentication enable setting 1956 is disabled, biometric authentication is not available on the electronic device 1900.

For example, in Figure 19W, electronic device 1900 receives activation of biometric authentication enable setting 1956. Activation of the biometric authentication enable setting 1956 is a tap gesture 1958 in some examples. Because the biometric authentication enable setting 1956 is enabled as shown in FIG. 19W, the biometric authentication enabling setting 1956 responds to the tap gesture 1958, as shown in FIG. 19X. It is disabled. In some examples, as a result, any request to access secure data on electronic device 1900 requires the user to authenticate using an alternative form of authentication. As an example, referring to FIGS. 19Y and 19Z, electronic device 1900 detects user input 1930 near the edge of display 1902. As illustrated in FIGS. 19I-19K , user input 1930 is, in some examples, a swipe gesture that is a request to access the home screen interface of electronic device 1900. 19AA, because the biometric authentication enabling setting 1956 is disabled, the electronic device 1900, in response to the swipe gesture 1930, slides the locked state interface 1910 in an upward direction. to display (e.g., reveal) an alternative authentication interface 1932, whereby a user may provide a passcode to unlock the electronic device 1900.

In some examples, one or more elements displayed by electronic device 1900 are context-based. As illustrated in FIG. 19AB , for example, the locked status indicator displayed by the electronic device is, in some cases, based on the location and/or type of the electronic device 1900.

20A-20F are flow diagrams illustrating a method for performing biometric authentication using an electronic device, according to some examples. Method 2000 includes a device (e.g., a device having a display, one or more input devices (e.g., touchscreen, microphone, camera), and a wireless communication radio (e.g., a Bluetooth connection, a WiFi connection, a mobile broadband connection such as a 4G LTE connection) For example, 100, 300, 500, 1900). In some examples, the display is a touch-sensitive display. In some examples, the display is not a touch-sensitive display. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the device includes one or more biometric sensors, optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device, such as an IR floodlight, a structured light projector, or a combination thereof. The light-emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric feature by one or more biometric sensors. Some operations of method 2000 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 2000 provides an intuitive method for performing authentication of biometric characteristics. This method reduces the cognitive burden on the user for performing authentication of biometric features, thereby creating a more efficient human-machine interface and intuitive user experience. For battery-operated computing devices, allowing users to manage authentication of biometric features faster and more efficiently saves power and increases time between battery charges.

In some examples, before detecting that device wake criteria have been met, the electronic device performs a biometric registration process. In some examples, during biometric registration, the device required that the face being enrolled include facial characteristics indicative of the face looking at the electronic device during registration of the face in order to proceed with biometric registration of the face. In some examples, the device outputs tactile, audio, and/or visual warnings during enrollment if the face is not looking at the electronic device during enrollment.

In some examples, the electronic device (eg, 100, 300, 500, 1900) detects that device wake criteria have been met (2002). In some examples, in response to detecting that device wake criteria have been met, the electronic device transitions the electronic device from a first visual state (e.g., a low power state) to a second visual state (e.g., a medium power state). Shida (2004). Transitioning from a first visual state (e.g., a low power state) to a second visual state (e.g., a medium power state) in response to detecting that device wake criteria have been met may involve the user manually providing one or more inputs. This avoids providing one or more inputs to transition the device from the first state to the second state. Performing an action (automatically) when a set of conditions is met without requiring additional user input helps the user provide appropriate input (e.g. when operating/interacting with a device) and prevents user mistakes. (by reducing the device's operability and making the user-device interface more efficient), which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently .

In some examples, wake criteria are met when the electronic device is lifted, upon pressing a button (e.g., 1904) and/or displaying a notification (e.g., 1914). In some examples, the first visual state is a display-off state, or a state in which the display of the electronic device is at 10% of maximum brightness state. In some examples, the second visual state is a display brightness state that is higher than the first visual state (e.g., 10% when the display is off in the first state; 20% when the display is at 10% of the first state). . In some examples, the second visual state includes a first introductory screen (e.g., 1910) displayed at a first brightness (2006). In some examples, while in the second visual state, the electronic device displays 2010 a fourth user interface object (e.g., 1912) representing the visual state of the electronic device. In some examples, while in the second visual state, the electronic device displays 2012 a fifth user interface object (e.g., 1912) representing the visual state of the electronic device. In some examples, one or more features of the electronic device (e.g., a display (e.g., 1902), one or more biometric sensors (e.g., 1903), a microphone, access to sensitive data such as the content of applications and messages, The ability to perform destructive actions, such as deleting communications or photos, and to perform communication operations, such as sending new messages and sharing content stored on the device) allows the electronic device to enter a first visual state. (e.g., while the device is in a locked state) and is disabled (e.g., powered off or operating with reduced functionality) (2008). In some examples, transitioning to the second visual state includes enabling one or more disabled functions of the electronic device. In some examples, transitioning to the second visual state includes bringing the device into a state in which one or more disabled components of the electronic device are enabled. In some examples, enabling one or more disabled functions includes enabling a display (e.g., 1902), one or more biometric sensors (e.g., 1903), and/or a microphone of the electronic device.

In some examples, after transitioning the device to a second visual state (2014), upon determining whether the biometric authentication criteria have been met, the electronic device determines that a selectable option (e.g., 1955) is enabled. The device uses the first set of criteria as biometric authentication criteria. When determining whether the biometric authentication criteria have been met, using the first set of criteria as biometric authentication criteria in accordance with a determination that a selectable option (e.g., 1955) of the device is enabled allows the user to require minimal input. Makes it easy to provide authentication information to devices. Performing an action (automatically) when a set of conditions is met without requiring additional user input helps the user provide appropriate input (e.g. when operating/interacting with a device) and prevents user mistakes. (by reducing the device's operability and making the user-device interface more efficient), which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently .

In some examples, the first set of criteria includes a requirement that the user's face was looking at the display of the electronic device (e.g., when unlocking the device and/or determining whether to transition from the second visual state to the third visual state) Includes. In some examples, further, after transitioning the device to the second visual state, upon determining whether the biometric authentication criteria have been met, upon determining that the electronic device's selectable option is not enabled, the electronic device The set of criteria is used as biometric authentication criteria. In some examples, the second set of criteria includes a requirement that the user's face was looking at the display of the electronic device (e.g., when unlocking the device and/or determining whether to transition from the second visual state to the third visual state) does not include In some situations, a user may, for example, use an accessibility option, require the user to look at the device during biometric authentication in order for the user's face to be recognized by the device, as required by the gaze detection requirement ( For example, 1955) is enabled.

In some examples, after transitioning to the second state, the electronic device determines, by one or more biometric sensors, whether biometric capture criteria are met (2016). In some examples, the electronic device determines whether a biometric feature is present, for example, in the field of view of one or more biometric sensors. In some examples, determining whether the biometric capture criteria are met includes transitioning to a second visual state and determining whether the biometric capture criteria are met after a first predetermined time (2018). In some examples, the electronic device detects the biometric characteristic immediately after transitioning to the second state. In some examples, the electronic device transitions to the second state and detects the biometric characteristic after a period of time. In some examples, upon determining that the biometric capture criteria are met, the electronic device provides, by one or more biometric sensors, biometric data associated with the biometric characteristic (2020). In some examples, once the electronic device has transitioned to the second visual state (recall that one or more biometric sensors are enabled prior to or during such transition), the electronic device uses one or more biometric sensors enabled. to capture biometric data.

In some examples, upon determining that biometric authentication criteria have been met based on biometric data provided by one or more biometric sensors (e.g., a biometric feature, such as a face, has been authenticated by the device), the electronic device , transitioning the electronic device from the second visual state to the third visual state (e.g., high power state) (2022). The device is moved from a second visual state (e.g., a medium power state) to a third visual state (e.g., a high power state) based on a determination that biometric authentication criteria have been met based on biometric data provided by one or more biometric sensors. state) allows the user to avoid providing one or more inputs to transition the device from the second state to the third state by manually providing one or more inputs. Performing an action (automatically) when a set of conditions is met without requiring additional user input helps the user provide appropriate input (e.g. when operating/interacting with a device) and prevents user mistakes. (by reducing the device's operability and making the user-device interface more efficient), which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently .

In some examples, while the electronic device is in the third visual state, the display of the electronic device is turned on at a second relatively high brightness. In some examples, the transition from the second visual state to the third visual state is a continuation of the transition from the first visual state to the second visual state (2024). In some examples, during transitions from the first visual state to the second visual state, and from the second visual state to the third visual state, the display changes from the off state to low brightness, and finally, in response to authentication, to high brightness. The brightness continues to brighten. In some examples, the transition to the second visual state is a transition at a particular brightness, and the transition from the second visual state to the third visual state is a transition starting from a particular brightness. In some examples, each increment is based on the same function. In some examples, transitioning to a second visual state includes magnifying at least each user interface element (e.g., 1912) displayed in the first visual state, and transitioning to a third visual state includes zooming in on each user interface element (e.g., 1912). Includes further expansion of elements (e.g., 1912, 1913, 1921). In some examples, the second visual state indicates that the device is in a locked state and the third visual state indicates that the device is in an unlocked state.

In some examples, further, upon determining that biometric authentication criteria have been met based on biometric data provided by one or more biometric sensors, the electronic device includes a fifth user interface object (e.g., 1912). Display the unlock animation (2026). displaying an unlock animation including a user interface object (e.g., fifth user interface object 1912) in accordance with a determination that biometric authentication criteria have been met based on biometric data provided by one or more biometric sensors. It provides visual feedback by allowing the user to quickly recognize that authentication has been successful and the device has therefore been unlocked. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the fifth user interface object is a lock. In some examples, the unlock animation is based on the context of the electronic device, such as location or type. In some examples, the fifth user interface object (e.g., 1912, 1922) has a first (e.g., locked) state when the electronic device is in the second visual state and a second state when the electronic device is in the third visual state. has a (e.g., unlocked) state (2028). In some examples, the visual state element transitions 2030 from a first state to a second state during an unlock animation (animations including 1912, 1913, 1921, 1922). In some examples, to demonstrate that biometric authentication was successful, the electronic device displays an animation of the lock being unlocked (an animation including 1912, 1913, 1921, and 1922).

In some examples, the third visual state includes a second entering screen at a second brightness that is higher than the first brightness (2032). In some examples, the first introductory screen (e.g., 1910) and the second introductory screen (e.g., 1920) are the same screen other than the brightness level of each screen.

In some examples, transitioning from the second visual state to the third visual state includes adjusting (e.g., increasing) the size of the first user interface object (e.g., 1912) displayed on the display of the electronic device. Hana (2034). In some examples, the electronic device resizes all of the displayed user interface objects. In some examples, the electronic device resizes fewer user interface elements than all of the displayed user interface elements. In some examples, the first user interface object (e.g., 1912) is a lock icon, and resizing the first user interface object includes increasing the size of the first user interface object (2036). . In some examples, transitioning from the second visual state to the third visual state includes changing the degree of blurriness of the second user interface object displayed on the display of the electronic device. In some examples, one or more blur parameters, such as blur radius and/or blur size, of one or more displayed user interface objects (e.g., desktop) are increased and/or decreased. In some examples, the blur parameters of all user interface objects are changed. In some examples, the blur parameters of fewer than all of the user interface objects are changed. In some examples, the first user interface object and the second user interface object are the same element. In some examples, transitioning from a second visual state to a third visual state includes translating the position of a third user interface object displayed on the display of the electronic device from a first position to a second position (e.g., shifting the position This includes shifting the position without rotating it. In some examples, the lock icon is moved closer to the edge of the display of the electronic devices before or during the unlock animation. In some examples, transitioning the device from the second visual state to the third visual state includes outputting a tactile output (e.g., 1926). In some examples, the electronic device outputs a tactile output indicating that biometric authentication criteria have been met while displaying an unlock animation.

In some examples, the third visual state corresponds to the unlocked state (2038). In some examples, while in the third visual state (e.g., while the device is unlocked), the electronic device receives 2040 a locking input (e.g., 1948, pressing 1952 of button 1904). In some examples, the lock input is a press of a button, such as a hardware button (e.g., 1904), or a selection of an affordance (e.g., 1922, 1950) that indicates the intent to lock the electronic device. Additionally, while in the third visual state, in response to receiving a locking input, the electronic device transitions from the third visual state to the locked state (2042). In some examples, the device locks in response to one or more specific inputs.

In some examples, while in the locked state, the device performs one or more operations available in the unlocked state (e.g., displaying the home screen, displaying the content of notifications, launching applications) , transmitting communications) is prevented. Preventing the device from performing one or more operations that are available in the unlocked state while in the locked state prevents certain functions or operations from being performed on the device when the device is in the locked state as opposed to being in the unlocked state. Improves device security by prohibiting execution. Improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally reduces the device's power usage by enabling users to use the device more efficiently. reduces and improves battery life.

In some examples, the electronic device displays 2044 a locking animation that includes a sixth user interface object (e.g., 1912, 1922) representing a visual state of the electronic device. Displaying a lock animation that includes a specific user interface object (eg, sixth user interface object 1912, 1922) provides visual feedback by allowing the user to quickly recognize that the device is in a locked state. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, the sixth user interface object is a lock. In some examples, the sixth user interface object has a first appearance (e.g., an open lock) when the electronic device is in a third visual state and a second appearance (e.g., a closed lock) when the electronic device is in a locked state. Have (2046). In some examples, the sixth user interface object transitions from the first appearance to the second appearance during the lock animation (2048). In some examples, to verify that the electronic device is locked, the electronic device displays an animation of the lock being locked. In some examples, transitioning the device from the third visual state to the locked state includes outputting 2050 a tactile output (e.g., 1952). In some examples, the tactile output includes a single tap. In some examples, the tactile output includes multiple tabs. In some examples, the tactile output is timed to synchronize with an animation of a sixth user interface object moving back and forth (eg, a lock shaking back and forth). In some examples, displaying the lock animation includes displaying the current time. In some examples, the electronic device displays the time when transitioning to the locked state.

In some examples, biometric authentication criteria include a requirement that the user was viewing the display of the electronic device with a face matching one or more authorized faces. Including a requirement that the user was viewing the display of the electronic device with a face matching one or more authorized faces against the biometric authentication criteria ensures that authentication is successful only by (the faces of) authorized users of the device. This improves device security. Improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally reduces the device's power usage by enabling users to use the device more efficiently. reduces and improves battery life. In some examples, unlocking an electronic device requires the user to be looking at the electronic device.

In some examples, upon determining that biometric authentication criteria have not been met based on biometric data provided by one or more biometric sensors (2052), the electronic device maintains the electronic device in a second visual state (2054) ). Maintaining the device in a second visual state upon determining that biometric authentication criteria have not been met based on biometric data provided by one or more biometric sensors may require authentication without the device meeting the appropriate authentication criteria. Improves device security by prohibiting transition to this state. Improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally reduces the device's power usage by enabling users to use the device more efficiently. reduces and improves battery life.

In some examples, if the biometric feature is not authenticated, the device's display does not brighten further as it responds to authentication of the biometric feature. In some examples, the device makes an explicit request (e.g., 1930) to unlock the device (e.g., a swipe gesture from a lower portion of the device, a press of the home button, or content not available when the device is in a locked state). When the biometric authentication criteria are not met prior to receiving the user's other input indicating that the user wants to see and/or interact with him/her), the device may perform biometric authentication, passcode authentication, password authentication, pattern authentication, etc. Displays an unlocking interface (e.g., 1932) while attempting to authenticate the user via one or more forms of authentication, such as: Examples of authenticating a user in response to a request to unlock the device through different forms of authentication are described in more detail with reference to FIGS. 26A-26AS. In some examples, further, upon determining that biometric authentication criteria have not been met based on biometric data provided by one or more biometric sensors, the electronic device determines the location of a fourth user interface object (e.g., 1912). Alternates between the first and second positions (2056). In some examples, to demonstrate that biometric authentication has failed, the electronic device shakes a lock icon displayed in the introduction interface. In some examples, a tactile output (e.g., 1918) is provided in combination with a shaking lock icon. In some examples, no tactile output is provided.

In some examples, while the device is in an unlocked state, the electronic device detects that a lock condition has been met. In some examples, in response to detecting that a lock condition has been met, the lock condition may be triggered by an explicit lock input (e.g., 1922, pressing button 1904, 1952) (e.g., pressing the power button, tapping the lock icon). Depending on the decision, the electronic device transitions the device from the unlocked state to the locked state and outputs a respective lock indication (e.g., 1912). In some examples, each lock indication includes visual, audio, and/or tactile output indicating that the device has transitioned from an unlocked state to a locked state. In some examples, further in response to detecting that a locking condition has been met, the electronic device determines that the locking condition is an implicit locking condition (e.g., covering a proximity sensor, an extended period of time without receiving input, etc.). Transitions the device from the unlocked state to the locked state without outputting a respective lock indication.

In some examples, after detecting that device wake criteria have been met, the electronic device detects a request to display a biometric authentication settings interface. In some examples, the request to display the biometric authentication settings interface may include a switch from an edge of the display to display a control panel user interface including a plurality of controls, including controls associated with enabling or disabling biometric authentication. A wipe, a long press of one or more hardware buttons that causes the device to display a configuration user interface that includes one or more controls, including controls associated with enabling or disabling biometric authentication, or and navigation through one or more menus in a settings app to a set of controls associated with biometric authentication, including one or more controls, including a control associated with enabling. In some examples, in response to a request to display a biometric authentication settings interface, the electronic device displays a biometric authentication settings interface (e.g., 1954). In some examples, while displaying a biometric authentication settings interface, the electronic device receives a first user input (e.g., 1958) corresponding to a request to disable biometric authentication. In some examples, in response to receiving the first user input, the electronic device disables biometric authentication. In some examples, while biometric authentication is disabled and the device is in a locked state, the electronic device receives a request to unlock the device. In some examples, in response to receiving a request to unlock the device, the electronic device may prompt for authentication using a form of authentication different from biometric authentication (e.g., "Pass to unlock" as shown in FIG. 19AA). Prints “ENTER PASSCODE TO UNLOCK”). In some examples, different forms of authentication are passcodes, passwords, fingerprints, etc.

In some examples, upon transitioning to a second visual state and determining that the biometric capture criteria are not met after the first predetermined time, the electronic device determines that after the first predetermined time has elapsed and after the second predetermined time the electronic device determines that the biometric capture criteria are not met. Determine whether capture criteria are met. In some examples, delays between attempts to detect a biometric characteristic become increasingly longer. In some examples, once a threshold of biometric authentication attempts has been reached, biometric authentication is disabled.

Note that the details of the processes described above with respect to method 1200 (e.g., FIGS. 20A-20F) are also applicable in a similar manner to the methods described herein. For example, method 2000 optionally includes one or more of the features of the various methods described herein with reference to methods 800, 1000, 1200, 1400, 1600, 1800, 2200, 2500, 2700. Includes. For example, the registered biometric data described in method 1200 may be used to perform biometric authentication as described with respect to FIGS. 19A-19H. For another example, one or more implantable interfaces described in methods 2700 are optionally displayed in response to receipt of input prior to completion of the biometric authentication process. For brevity, these details are not repeated herein.

Operations in the information processing methods described above optionally utilize one or more functional modules within an information processing device, such as general-purpose processors (e.g., as described with respect to FIGS. 1A, 3, and 5A) or application-specific chips. It is implemented by running it. Additionally, the operations described above with reference to FIGS. 20A-20F are optionally implemented by components shown in FIGS. 1A and 1B. For example, detection operation 2002, transition operation 2004, and transition operation 2022 are optionally implemented by event classifier 170, event recognizer 180, and event handler 190. Event monitor 171 in event sorter 170 detects contact on touch-sensitive surface 604, and event dispatcher module 174 passes event information to application 136-1. The respective event recognizer 180 of application 136-1 compares the event information to the respective event definition 186 and determines that the first contact at the first location on the touch-sensitive surface corresponds to the selection of an object on the user interface. Determines whether it corresponds to the same predefined event or subevent. When an individual predefined event or sub-event is detected, event recognizer 180 activates an event handler 190 associated with detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some examples, event handler 190 accesses the respective GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes may be implemented based on the components shown in FIGS. 1A and 1B.

21A-21aq illustrate example user interfaces for biometric authentication, according to some examples. As described in more detail below, the illustrative examples of user interfaces illustrated in Figures 21A-21aq are used to illustrate the processes described below, including the processes in Figures 22A-22F.

FIG. 21A illustrates an electronic device 2100 (e.g., portable multifunction device 100, device 300, or device 500). In the illustrative example illustrated in FIGS. 21A-21aq , electronic device 2100 is a smartphone. In other examples, electronic device 1500 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 2100 has a display 2102, one or more input devices (e.g., a touchscreen of display 2102, buttons 2104, a microphone), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the electronic device includes one or more biometric sensors (e.g., biometric sensor 2103), optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, one or more biometric sensors 2103 are one or more biometric sensors 703. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

21A to 21C, the electronic device 2100 performs biometric authentication with the user of the electronic device 2100. Referring to FIG. 21A, the electronic device 2100 is in a locked state. As illustrated, in some examples, while in the locked state, the display 2102 of the electronic device 2100 is disabled. In other examples, while in the locked state, the display 2102 of the electronic device 2100 is enabled and the electronic device 2100 displays a locked state interface (e.g., Displays the locked state interface 2110 of FIG. 21C. While device 2100 is in the locked state, the electronic device initiates biometric authentication. In Figure 21B, electronic device 2100 initiates biometric authentication in response to detecting a wake condition (e.g., the user moves the device in a predetermined manner). The electronic device performs biometric authentication in response to any number of wake conditions, including, but not limited to, moving the device (e.g., lifting), pressing a button on the device, or touching the display 2102. It will be understood that it starts.

In some examples, initiating biometric authentication includes obtaining (e.g., capturing with one or more biometric sensors 2103) data corresponding to at least a portion of the user's biometric characteristics. In response to initiating biometric authentication, electronic device 1500 may, for example, obtain (e.g., capture) and process (e.g., analyze) biometric data to Determine whether the measurement feature (or portion thereof) satisfies biometric authentication criteria (e.g., determines whether the biometric data matches a biometric template, within a threshold). In some examples, biometric authentication requires the user to be looking at the device during biometric authentication. Accordingly, as illustrated in FIG. 21B, the user's gaze 2106 is directed toward the electronic device when the user lifts the device 2100.

In FIG. 21C , electronic device 2100 displays a locked state interface 2110 that includes a locked state indicator 2112 in response to a wake condition. In some examples, while displaying the locked state interface 2110, the electronic device 2100 additionally displays a flashlight affordance 2107 and a camera affordance 2108. In some examples, activation of flashlight affordance 2107 causes the electronic device to load a flashlight application. In some examples, activation of camera affordance 2108 causes electronic device 2100 to load a camera application.

In FIG. 21D , electronic device 2100 determines that biometric authentication was successful and, in response, displays unlocked status interface 2120. In some examples, the display of unlocked status interface 2120 includes the display of an unlock animation, as described with reference to FIGS. 19D-19G. While displaying the unlocked status interface 2120 , the electronic device 2100 additionally displays (e.g., maintains its display) the flashlight affordance 2107 and the camera affordance 2108 . In some examples, electronic device 2100 outputs tactile output 2126 in response to determining that biometric authentication was successful.

In Figure 21E, electronic device 2100 determines that biometric authentication was not successful. In response, electronic device 2100 maintains display of locked status interface 2110. In some examples, the electronic device displays a shake animation in which the locked status indicator 2112 is moved from side to side to simulate a “shake” effect to indicate that biometric authentication was not successful. Electronic device 2100 further outputs tactile output 2118 to indicate that biometric authentication was not successful.

In some examples, one or more operations accessible during the display of locked state interface 2110 do not require authentication, and thus one or more operations may be performed while the electronic device is in a locked state. As an example, loading a flashlight application in response to activation of flashlight affordance 2107 does not require authentication. As another example, referring to Figure 21F, in some examples, electronic device 2100 detects activation of camera affordance 2108 while in a locked state. As shown, activation of camera affordance 2108 is a tap gesture 2130 on camera affordance 2108. 21G , in response to detecting activation of camera affordance 2108, electronic device 2100 displays, on display 2102, a camera application interface 2132 associated with a camera application (e.g., in a locked state). replaces the display of interface 2110).

Referring to Figure 21H, in some examples, while displaying unlocked status interface 2120, electronic device 2100 determines that the device is unlocked and/or uses a predetermined type of input (e.g., a swipe gesture). displays a prompt 2124 indicating that providing will allow the user to access secure content, such as a home screen interface (e.g., home screen interface 2129 in FIG. 21I). For example, as illustrated, electronic device 2100 detects user input 2128, such as near an edge of display 2102. User input 2128 is, in some examples, a swipe gesture that is a request to access the home screen interface of electronic device 2100, and in response to swipe input 2128, the electronic device displays the home screen interface of FIG. 21I ( 2129) (e.g., replaces the display of the unlocked interface 2120 with it). In some examples, displaying the home screen interface 2129 can be accomplished by sliding the unlocked state interface 2120 in an upward direction, as similarly described with reference to FIGS. 19P-19R. It includes displaying (e.g., revealing).

In FIG. 21J , the electronic device is in a locked state, e.g., in response to a failed biometric authentication (as described with reference to FIGS. 21A-21C and 21E), and is in a locked state interface while in the locked state. (2110) is displayed. While displaying locked status interface 2110, electronic device 2100 determines that the device is locked and/or providing a predetermined type of input (e.g., a swipe gesture) allows the user to authenticate with electronic device 2100. (and unlock it) displays a prompt 2133 indicating that it will do so. For example, as illustrated, electronic device 2100 detects user input 2134, such as near an edge of display 2102. User input 2134 is, in some examples, a swipe gesture that is a request to access the home screen interface of electronic device 2100. Because the device is in a locked state (e.g., the user is not authenticated with electronic device 2100), the electronic device may use alternative authentication interface 2140 in response to swipe input 2128, shown in FIG. 21K. Display (e.g., replace the display of locking interface 2120 with it). In some examples, alternative authentication interface 2140 includes a locked state indicator 2142 that indicates that electronic device 2100 is in a locked state.

21K-21M, electronic device 2100 performs biometric authentication while displaying alternative authentication interface 2140. In particular, while displaying alternative authentication interface 2140, electronic device 2100 detects and/or obtains facial biometric data. Electronic device 2100 then processes the biometric data to determine whether the biometric data satisfies biometric authentication criteria. As shown in FIG. 21L, electronic device 2100 displays biometric authentication processing glyph 2144 to indicate that the electronic device is processing biometric data (e.g., in locked status indicator 2142). replace the display with it). In FIG. 21M, electronic device 2100 determines that the biometric authentication performed during display of alternative authentication interface 2140 was successful. As a result, the electronic device 2100 displays the biometric authentication success glyph 2146 (e.g., replaces the display of the biometric authentication processing glyph 2144 with it) to indicate that biometric authentication was successful. . In some examples, electronic device 2100 further completes processing of a passcode progress indicator and, optionally, provides tactile output 2141 to indicate successful biometric authentication.

Alternatively, referring to FIGS. 21N-21P , the user authenticates with the electronic device 2100 by entering a passcode during the display of the alternative authentication interface 2140. As shown in Figures 21, the electronic device 2100 displays an alternative authentication interface 2140, at least in part in response to a tap gesture 2148, as shown in Figure 21O, to display a passcode, and Optionally, it receives one or more other inputs representing additional alphanumeric digits of the passcode. 21P, electronic device 2100 determines that the passcode is valid and, in response, displays notification 2150 indicating that the passcode is valid and the user is authenticated with electronic device 2100.

In some examples, in response to entering a valid passcode, electronic device 2100 optionally stores and/or updates biometric data. For example, in response to entering a valid passcode, electronic device 2100 obtains biometric data (e.g., facial biometric data) and compares the biometric data to biometric data stored on the electronic device. In some examples, if the acquired biometric data is sufficiently similar to stored biometric data, the electronic device stores the acquired biometric data and/or updates previously stored biometric data to improve biometric authentication. In FIG. 21P, the electronic device determines that the biometric data obtained in response to entering a valid passcode is sufficiently similar to the stored biometric data. In response, the electronic device stores the acquired biometric data and/or updates the stored biometric data and displays an indication 2152 that the biometric data has been updated. In this way, electronic device 2100 provides adaptive biometric authentication.

As described with reference to FIGS. 21A-21C , in some examples, electronic device 2100 performs biometric authentication in response to a wake condition. In some examples, the electronic device receives a request to access secure content (e.g., content requiring authentication to access), such as a swipe gesture requesting access to a home screen, before biometric authentication is complete. do. Accordingly, referring to FIGS. 21Q-21S, in response to receiving a request to access secure content, electronic device 2100 displays insertable interfaces to indicate that the electronic device has not yet completed biometric authentication. . 21Q, the electronic device displays an insertable interface 2154 that includes an alternative authentication affordance 2156 and a biometric authentication glyph 2160 indicating initiation of biometric authentication. Activation of alternative authentication affordance 2156 causes the electronic device to display (e.g., replace the display of insertable interface 2154 with) an alternative authentication interface (e.g., alternative authentication interface 2140 in FIG. 21K). do it) Biometric authentication glyph 2160 is, in some examples, a simulation of a representation of a biometric feature.

Once electronic device 2100 has obtained biometric data in response to initiating biometric authentication, the electronic device processes the biometric data, as described. In some examples, while the electronic device is processing biometric data, the electronic device displays biometric authentication glyph 2162 to indicate that biometric data is being processed (e.g., biometric authentication glyph 2160) replace the display with it). In some examples, biometric authentication glyph 2162 includes a plurality of rings that rotate in a sphere, for example, while being displayed.

In Figure 21S, electronic device 2100 determines that the biometric data meets biometric authentication criteria. In response, electronic device 2100 displays biometric authentication glyph 2163 in insertable interface 2154 indicating that biometric authentication was successful (e.g., by displaying biometric authentication glyph 2162 therewith). replace). In some examples, the electronic device stops displaying alternative authentication affordances 2156. Additionally or alternatively, the electronic device may display the unlocked status indicator 2122 (e.g., replace the display of the locked status indicator 2161 therewith) and/or indicate that biometric authentication was successful. A tactile output (2164) is output.

As described, in some cases, the electronic device receives a request to access secure content before biometric authentication is complete. In some examples, the electronic device receives the request after the electronic device begins processing biometric data but before completing biometric authentication. In such a case, the electronic device optionally displays the insertable interface 2154 with the biometric authentication glyph 2162 and omits displaying the biometric authentication glyph 2160 first.

In some examples, one or more functions of the electronic device are optionally enabled based on whether the user is viewing the electronic device 2100. 21T-21Y , in some examples, some functions are disabled when the user is not looking at the electronic device 2100 and enabled when the user is looking at the electronic device 2100. In FIG. 21T, the user's gaze 2165 is not directed toward the electronic device 2100. In response to determining that gaze 2165 is not directed toward electronic device 2100, electronic device 2100 performs the respective functions associated with flashlight affordance 2107 and camera affordance 2108, as shown in FIG. 21U. disable them. While the functions associated with flashlight affordance 2107 and camera affordance 2108 are disabled (e.g., while the user is not looking at device 2100), the electronic device receives activation of camera affordance 2108. As shown, the activation is a tap gesture 2166 on the camera affordance 2108. Because the functionality associated with that affordance is disabled, the electronic device withholds responding to the tap gesture 2166 (e.g., withholds loading the camera application).

In Figure 21V, the user's gaze 2168 is directed toward the electronic device. In response to determining that gaze 2168 is directed toward electronic device 2100, electronic device 2100 performs respective functions associated with flashlight affordance 2107 and camera affordance 2108, as shown in FIG. 21W. Enable. 21X, electronic device 2100 detects activation of camera affordance 2108. As shown, the activation is a tap gesture 2170 on the camera affordance 2108. In response to activation of the camera affordance 2108, the electronic device displays the camera application interface 2132 associated with the camera application on the display 2102 (Figure 21Y) (e.g., display of the locked status interface 2110) replaces ).

Referring to FIG. 21Z , in some examples, electronic device 2100, while in a locked state, displays a locked state interface 2110 that includes a notification affordance 2174. As described, in response to a wake condition, the electronic device initiates biometric authentication. While displaying the locked status interface 2110 and before completing biometric authentication, the electronic device 2100 receives a request to access secure content. As an example, in Figure 21AA, electronic device 2100 detects activation of notification affordance 2174. As shown, activation of notification affordance 2174 is a tap gesture 2176.

21AB, in response to activation of notification affordance 2174, electronic device 2100 displays an implantable biometric authentication interface 2178 having a biometric authentication progress indicator 2182 and an alternative authentication affordance 2180. ) (e.g., replaces the display of the locked state interface 2110 with it). In some examples, biometric authentication progress indicator 2182 includes a biometric authentication glyph, such as biometric authentication glyph 2183, indicating progress in biometric authentication. In some examples, the biometric authentication progress indicator further identifies secure content associated with a request to access secure content (eg, “messages”). Activation of alternative authentication affordances 2180 causes the electronic device to display an alternative authentication interface, examples of which are described further below.

While displaying the implantable biometric authentication interface 2178, the electronic device 2178 continues to perform biometric authentication initiated in response to a wake condition. In some examples, initiating biometric authentication includes obtaining (eg, capturing with one or more biometric sensors) data corresponding to the user's biometric characteristics. Referring to FIG. 21AC, in response to obtaining data, the electronic device processes the biometric data to determine whether the biometric characteristic satisfies biometric authentication criteria, e.g., based on the biometric data. (e.g., determine whether biometric data matches a biometric template, within a threshold). While electronic device 2100 is processing biometric data, electronic device 2100 optionally displays a biometric authentication glyph 2184 in insertable biometric authentication interface 2178 indicating that biometric data is being processed. (e.g., replace the display of the biometric authentication glyph 2183 with it).

In Figure 21A, electronic device 2100 determines that the biometric characteristic satisfies the biometric authentication criteria. In response, the electronic device displays biometric authentication glyph 2185 in insertable biometric authentication interface 2178 (e.g., displays biometric authentication glyph 2184 with it) indicating that biometric authentication was successful. replace). Additionally or alternatively, the electronic device may display the unlocked status indicator 2122 (e.g., replace the display of the locked status indicator 2112 therewith) and/or indicate that biometric authentication was successful. A tactile output (2164) is output.

As shown in FIGS. 21AE and 21AF , in response to determining that biometric authentication is successful, electronic device 2100 displays messaging application interface 2194 (e.g., of insertable biometric authentication interface 2178). replace the display with it). In some examples, displaying messaging application interface 2194 may include sliding biometric authentication interface 2178 in an upward direction to display messaging application interface 2194, as similarly described with reference to FIGS. 19P-19R. Includes displaying (e.g., revealing).

Figures 21A-21A depict alternative ways of displaying biometric authentication progress. As described with reference to FIG. 21AA (and as shown in FIG. 21Ag), the electronic device 2100, while displaying the locked status interface 2110, is prompted to access secure content before completing biometric authentication. Receive the request. The request is, in some examples, activation 2176 of notification affordance 2174. In response to activation 2176 of notification affordance 2174, the electronic device maintains display of the locked status interface 2110. Additionally, as shown in Figure 21Ah, electronic device 2100 displays biometric authentication glyph 2184 to indicate that biometric data is being processed (e.g., display of locked status indicator 2110). replace it with it). 21AI, the electronic device determines that biometric authentication is successful and, in response, displays an unlocked status indicator 2122 (e.g., replacing the display of biometric authentication glyph 2184 with it). . Optionally, the electronic device 2100 further outputs a tactile output 2193 indicating that biometric authentication is successful. In some examples, electronic device 2100 displays notification affordance 2175 as the electronic device transitions to an unlocked state in response to determining that biometric authentication was successful (e.g., notification affordance 2174 replace the display with it). In some examples, notification affordance 2174 identifies secure content (e.g., “John Appleseed…meeting where…”).

In Figure 21AJ, in response to processing biometric data (as described with reference to Figure 21AC), the electronic device determines that biometric authentication was not successful. In response, electronic device 2100 displays biometric authentication glyph 2189 in biometric authentication interface 2178 indicating that biometric authentication was not successful (e.g., display of biometric authentication glyph 2184 replace with it). Additionally or alternatively, the electronic device alternates the position of the locked status indicator 2112 to simulate a “shake effect” to indicate that biometric authentication was not successful and/or that biometric authentication was not successful. A tactile output (2193) indicating is output.

While displaying the implantable biometric authentication interface 2178, the electronic device detects activation of the alternative authentication affordance 2180. Activation of an alternative authentication affordance 2108 is a tap gesture 2192 in some examples. Referring to FIG. 21AK, in response to activation of alternative authentication affordance 2180, the electronic device displays alternative authentication affordance 2198. In some examples, alternative authentication affordances 2198 include an indicator 2199 that identifies secure content associated with a request to access secure content (e.g., “messages”).

21A and 21AM, in response, at least in part, to a tap gesture 2102A (FIG. 21L) and, optionally, one or more other inputs representing additional alphanumeric digits of a valid passcode, electronic device 2100 A valid passcode (or password) is received by. As shown in FIGS. 21N and 21O, once a valid passcode has been received, the electronic device is unlocked and displays messaging application interface 2194 (e.g., display of alternative authentication interface 2198 therewith). replace). In some examples, displaying messaging application interface 2194 may include sliding alternative authentication interface 2198 in an upward direction to display messaging application interface 2194, as similarly described with reference to FIGS. 19P-19R. Includes displaying (e.g., revealing).

In some examples, in response to determining that biometric authentication was not successful, the electronic device further determines that a threshold number of biometric authentication attempts has been reached. Accordingly, as shown in FIG. 21AP, electronic device 2100 uses biometric authentication progress indicator 2182 to indicate that a threshold has been reached (“face authentication is temporarily disabled”). As shown in FIG. 21AK, while displaying the implantable biometric authentication interface 2178, the electronic device detects activation of the alternative authentication affordance 2180 and, in response to activation of the alternative authentication affordance 2180, Displays alternative authentication affordances 2198. As shown in Figure 21aq, when the electronic device determines that the threshold number of biometric authentication attempts has been reached, indicator 2199 indicates that biometric authentication is re-enabled in response to entry of a valid passcode. Identify (“ENTER PASSCODE TO RE-ENABLE FACE AUTHENTICATION”).

22A-22F are flow diagrams illustrating a method for performing biometric authentication using an electronic device, according to some examples. Method 2200 includes a device having a display, one or more input devices (e.g., a touchscreen, microphone, camera), and a wireless communication radio (e.g., a Bluetooth connection, a WiFi connection, a mobile broadband connection such as a 4G LTE connection). For example, 100, 300, 500, 2100). In some examples, the display is a touch-sensitive display. In some examples, the display is not a touch-sensitive display. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the device includes one or more biometric sensors, optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device, such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric feature by one or more biometric sensors. Some operations of method 2200 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 2200 provides an intuitive method for performing authentication of biometric characteristics. This method reduces the cognitive burden on the user for performing authentication of biometric features, thereby creating a more efficient human-machine interface and intuitive user experience. For battery-operated computing devices, allowing users to manage authentication of biometric features faster and more efficiently saves power and increases time between battery charges.

In some examples, while the device is in a locked state, an electronic device (e.g., 2100) receives a request to perform an operation that does not require authentication (2202). In some examples, in response to a request to perform an operation that does not require authentication, the electronic device performs the operation without waiting for authentication. Performing operations that do not require authentication without waiting for authentication allows the user to access the operation more quickly without having to provide additional input (eg, input telling the device to proceed). Performing an action (automatically) when a set of conditions is met without requiring additional user input helps the user provide appropriate input (e.g. when operating/interacting with a device) and prevents user mistakes. (by reducing the device's operability and making the user-device interface more efficient), which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently .

In some examples, a request to perform an operation that does not require authentication may include a request to enable a camera of the electronic device (e.g., 2130) and/or a camera user interface (e.g., 2130) to capture images and/or videos with the device. , 2132). In some examples, an operation that does not require authentication includes one or more restricted features that are restricted without successful authentication (e.g., sharing captured photos or video, captured during previous use of the camera application while the device was unlocked). and displaying an application user interface, including viewing photos or videos that have been used, and the device attempts biometric authentication while displaying the application user interface. Displaying an application user interface that includes one or more restricted features that are restricted without successful authentication provides visual feedback by allowing the user to quickly see which features are currently restricted on the application without proper authentication. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. If biometric authentication is successful while displaying the application user interface, the restricted features are enabled, and if biometric authentication is not successful, the restricted features remain disabled (e.g., the user can view the captured photo prevents users from sharing photos or videos, viewing photos or videos that were captured during previous use of the camera application while the device was unlocked, and optionally authenticates in response to attempts to use any of the restricted features. you will be prompted to provide).

In some examples, while the electronic device is in a locked state, the electronic device displays one or more affordances (e.g., 2107, 2108) on a touch-sensitive display (e.g., 2102) for performing operations that do not require authentication. (e.g., a flashlight affordance to enable an illuminated flashlight operating mode on the device and/or to access the camera functionality of the device, such as displaying a camera user interface to capture images and/or videos with the device) camera affordance).

In some examples, while displaying one or more affordances for performing operations for which authentication is not required, the electronic device displays each of the one or more affordances for performing operations for which authentication is not required (e.g., 2107, 2108). Detect activation (e.g., 2130). In some examples, in response to detecting activation of each of the one or more affordances for performing actions for which authentication is not required, determining that the face was looking at a display of the electronic device when activation of the respective affordance was detected. Depending on the determination (e.g., a determination that there is a face within the field of view of one or more cameras or one or more biometric sensors of the device with facial characteristics indicative of a face looking at the electronic device), the electronic device performs an action associated with each affordance. do. Performing an action associated with each affordance based on a determination that (e.g., the user's) face was looking at the device's display includes performing the action when the device detects that the user is looking at the device (e.g., and the user Not performing an action when the device is not being viewed (which optionally indicates that the affordance has been selected unintentionally) reduces the device's power usage and improves battery life.

In some examples, when the flashlight affordance (e.g., 2107) is activated while the face is looking at the display of the electronic device, the electronic device enables a flashlight operating mode in which a light on the device is turned on and/or the face is looking at the display of the electronic device. If a camera affordance (e.g., 2108) is activated while viewing the display, the electronic device accesses the device's camera functionality, such as displaying a camera user interface for capturing images and/or videos with the device. In some examples, a determination that a face was not looking at the display of the electronic device when activation of the respective affordance was detected (e.g., a face was not detected, or a face was detected but one or more cameras on the device or one or more biometric measurements were used) Upon determining that there is a face in the sensors' view with facial characteristics indicative of a face looking away from the electronic device, the electronic device withholds performance of the action associated with the respective affordance. In some examples, if the flashlight affordance is activated while the face is not looking at the display of the electronic device, the electronic device withholds enabling a flashlight operating mode in which the light on the device turns on and/or the face is not looking at the display of the electronic device. If the camera affordance is activated while not being viewed, the electronic device withholds access to the device's camera functionality, such as displaying a camera user interface for capturing images and/or videos with the device.

In some examples, while the electronic device is in a locked state, the electronic device may be subject to conditions associated with performing a biometric authentication verification using a biometric sensor without explicit input from the user requesting biometric authentication (e.g., a user (elevating the device 2100 to the position shown in FIG. 21B) is detected. In some examples, conditions associated with performing a biometric authentication verification using a biometric sensor without explicit input from the user requesting biometric authentication include raising the device and/or using a display wake button (e.g., 2104 ) includes pressing.

In some examples, one or more biometric sensors may include a non-contact biometric sensor (e.g., 2103) configured to capture biometric data associated with biometric features located within a predetermined range of distances from the non-contact biometric sensor (e.g., 2103). 2103) (e.g., a facial recognition sensor) (2204). In some examples, the biometric sensor includes a camera. In some examples, the biometric sensor includes a light projector (eg, an IR-illuminated or structured light projector).

In some examples, the device is restricted from performing more than a predefined number of biometric authentication checks without successful authentication (2206). Restricting a device from performing more than a predefined number of biometric authentication checks without successful authentication improves device security by limiting fraudulent authentication attempts on the device. Improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally reduces the device's power usage by enabling users to use the device more efficiently. reduces and improves battery life.

In some examples, successful authentication includes successful authentication through biometric authentication, or any other form of authentication, such as using a passcode, password, or pattern. In some examples, the device performs less than a predefined number of biometric authentication checks in response to detecting a condition, thereby providing at least one biometric authentication for use in response to detecting a request to perform the respective action. Schedule confirmation. In some examples, the electronic device tracks multiple failed authentication attempts, such as multiple failed sequential failed attempts without an intervening successful authentication (eg, biometric authentication, or other authentication such as password authentication). In some such examples, when the maximum number of failed attempts is reached, the device does not perform biometric authentication until a successful non-biometric authentication is received. In some examples, a request to perform an action requiring authentication after the maximum number of failed biometric authentication confirmations has been reached may result in the display of an alternative authentication user interface (e.g., a password, passcode, pattern or other authentication interface). trigger.

In some examples, while the electronic device's display (e.g., 2102) is disabled, the electronic device detects a display wake condition (2208). In some examples, a display wake condition may include movement of the device in a predefined manner, such as movement of the device above a threshold amount, movement of the device in an orientation associated with waking the device, activation of a display wake button, or touch sensitivity. On the type surface, it includes gestures such as taps.

In some examples, in response to detecting the condition, the electronic device performs a first biometric authentication verification (2210). Performing a biometric authentication verification in response to detecting a condition (eg, a wake condition) allows a user to provide authentication information to the device in response to detecting a wake condition quickly and efficiently and with minimal input. Performing an action (automatically) when a set of conditions is met without requiring additional user input helps the user provide appropriate input (e.g. when operating/interacting with a device) and prevents user mistakes. (by reducing the device's operability and making the user-device interface more efficient), which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently .

In some examples, performing the first biometric authentication verification includes capturing 2212 first biometric data using one or more biometric sensors. In some examples, the electronic device initiates a first biometric authentication procedure that includes capturing first biometric data using one or more biometric sensors. In some examples, performing the first biometric authentication verification may occur after capturing 2214 the first biometric data (e.g., in response to capturing the first biometric data or in response to a request to unlock the device). Thus), upon determining that the first biometric data satisfies the biometric authentication criteria, the electronic device transitions the device from a locked state to an unlocked state (2216). Transitioning a device from a locked state to an unlocked state following a determination that the first biometric data satisfies the biometric authentication criteria can be accomplished by unlocking the device if the authentication process is successful (however, in some examples, authentication is successful). It improves device security (by preventing the device from being unlocked if it fails to do so). Improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally reduces the device's power usage by enabling users to use the device more efficiently. reduces and improves battery life.

In some examples, the electronic device determines whether the first biometric data meets biometric authentication criteria. In some examples, the biometric authentication criteria include criteria that are satisfied when the first biometric data matches biometric data (eg, facial feature data, fingerprint data, iris data) corresponding to the authorized user. In some examples, while in an unlocked state and prior to detecting a request to perform the respective action, the electronic device may, in accordance with the prompt criteria, provide instructions corresponding to the request to perform the respective action. Output a prompt (e.g., visual, audio, or tactile output) (2218). In some examples, the device is in an unlocked state after detecting the authorized user's face. In some examples, the electronic device displays instructions (e.g., 2124) to “swipe up” to access the home screen (e.g., 2129). In some examples, the prompt criteria include a requirement (2220) that the user's gaze (e.g., 2168) is directed to the electronic device. In some examples, the prompt criteria include a requirement that the device detect facial characteristics indicative of a face looking at the electronic device (e.g., detecting that the user's gaze is directed at the electronic device) for at least a predetermined amount of time (2222).

In some examples, upon determining that the first biometric data does not meet biometric authentication criteria, the electronic device keeps the device in a locked state (2224). Keeping the device locked following a determination that the first biometric data does not meet biometric authentication criteria improves device security by preventing fraudulent and/or unauthorized access to the device. Improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally reduces the device's power usage by enabling users to use the device more efficiently. reduces and improves battery life. In some examples, if biometric data corresponding to the biometric characteristic does not match the biometric authentication template, the device remains locked in response to failed authentication.

In some examples, after performing the first biometric authentication verification, the electronic device detects, via the device, a request to perform the respective action (e.g., 2134, 2176) without receiving additional authentication information from the user (e.g., 2134, 2176). 2226). In some examples, detecting, via the device, a request to perform a respective action without receiving additional authentication information from the user may result in a request to display content that is not available for display when the electronic device is in a locked state. Includes detecting (2228). In some examples, user input is input that requests access to secure data, such as a home screen or an application that presents secure data. In some examples, the request to perform each action includes a swipe gesture on the device, a swipe gesture from an edge of the device (e.g., 2134), or pressing a home button. In some examples, the request to perform each action may include a selection (e.g., 2176) of the notification (e.g., the request to perform each action may include a selection of the notification, such as an application corresponding to the notification or a long look). a request to display additional relevant information); A swipe gesture (e.g., the request to perform each action is an upward swipe on the display of the electronic device; in some examples, the request to perform each action is an upward swipe starting from an edge of the display of the electronic device. In some examples, a swipe is, among other things, a request to display the home screen of the electronic device); movement of the electronic device in a predetermined manner (eg, the request to perform each action includes lifting the electronic device); and at least one of a selection of affordances (e.g., the request to perform each action includes a selection of an affordance displayed by the electronic device, including a “lock” affordance displayed when the electronic device is in a locked state). Includes (2230).

In some examples, while performing the first biometric authentication verification, the electronic device receives 2232 a second request (e.g., 2134) to perform a second operation without receiving additional authentication information from the user. In some examples, the second request is a swipe gesture, selection of a notification, etc. In some examples, in response to receiving a second request to perform a second action (2234), a second request to perform a second action is received after determining that the first biometric data does not meet biometric authentication criteria. Depending on the determination, the electronic device displays 2236 a second alternative authentication interface (e.g., 2140). In some examples, the second alternative authentication interface is a passcode, password, pattern, or fingerprint authentication user interface and is displayed without performing a second action if biometric authentication fails at least once. In some examples, upon determining that a second request to perform a second action has been received prior to evaluating the first biometric data (e.g., prior to determining whether the first biometric data satisfies biometric authentication criteria) , the electronic device displays 2238 a biometric authentication indicator (e.g., 2156, 2162) including an indication that biometric authentication is being attempted without displaying a second alternative authentication interface. In some examples, the second alternative authentication interface is a passcode, password, pattern, or fingerprint authentication user interface and is not displayed and the second action is not performed if the device has not had time to complete the first biometric authentication attempt. . In some examples, if the user swipes up while the electronic device is performing a first iteration of biometric authentication, the electronic device displays an insertable interface (e.g., 2154) that indicates the processing status of the biometric authentication. In some examples, in response to receiving a second request to perform a second action, upon determining that the first biometric data satisfies biometric authentication criteria, a second request to perform the second action was received. , the electronic device performs the second operation without displaying an alternative authentication interface (e.g., displaying a passcode, password, pattern, or fingerprint authentication user interface if biometric authentication has failed at least once). In some examples, in response to receiving a second request to perform a second action, a biometric authentication display displayed in response to a determination that a second request to perform a second action was received prior to evaluating the first biometric data. The character (e.g., 2182) includes an indication of the application associated with the notification. In some examples, when a user selects a notification while the device is performing the first biometric authentication verification, the device presents an application associated with the notification. For example, when a user selects message notifications, the device may display information related to the messaging application, such as “Biometric Authentication for Messages” or “Face ID for Messages.” Displays the indication.

In some examples, in response to detecting a request to perform the respective operation (2240) and determining that the respective operation does not require authentication, the electronic device performs the respective operation (2242). By determining that each operation does not require authentication, performing each operation without successful authentication allows the user to more quickly access the operation without having to provide additional input (e.g., input telling the device to proceed). do. Performing an action (automatically) when a set of conditions is met without requiring additional user input helps the user provide appropriate input (e.g. when operating/interacting with a device) and prevents user mistakes. (by reducing the device's operability and making the user-device interface more efficient), which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently . In some examples, when an operation does not require authentication, the electronic device performs the operation regardless of whether the device is in a locked or unlocked state. In some examples, the device does not verify authentication if the respective operation does not require authentication, such as using a camera or making an emergency call.

In some examples, upon determining that each operation requires authentication and that the device is in an unlocked state, the electronic device performs the respective operation (2244). When the device is in an unlocked state, performing each action based on the determination that each action requires authentication occurs without the user having to provide additional input (e.g., input telling the device to proceed). Provides faster access to Performing an action (automatically) when a set of conditions is met without requiring additional user input helps the user provide appropriate input (e.g. when operating/interacting with a device) and prevents user mistakes. (by reducing the device's operability and making the user-device interface more efficient), which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently .

In some examples, upon determining that the respective operation requires authentication and the device is in a locked state (2246), the electronic device may deploy one or more biometric sensors without explicit input from the user requesting a second biometric authentication confirmation. Capture second biometric data using 2248. Capturing the second biometric data without explicit input from the user requesting a second biometric authentication confirmation in accordance with the determination that the respective operation requires authentication and that the device is in a locked state, while the device is in a locked state. Improves device security by requiring successful authentication and thereby preventing fraudulent and/or unauthorized access to the device. Improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally reduces the device's power usage by enabling users to use the device more efficiently. reduces and improves battery life.

In some examples, after capturing the second biometric data, the electronic device performs a second biometric authentication verification (2250). In some examples, the first biometric data and the second biometric data are compared to the same set of biometric criteria. In some examples, the first and second biometric data are compared to respective sets of biometric criteria. In some examples, performing the second biometric authentication verification includes performing 2252 a respective operation in accordance with a determination that the second biometric data satisfies biometric authentication criteria. In some examples, the electronic device optionally further transitions the device from a locked state to an unlocked state. In some examples, performing a second biometric authentication verification may include suspending performance of the respective operation (and optionally locking the device) based on a determination that the second biometric data does not meet biometric authentication criteria. Maintaining the condition) (2254). In some examples, suspending performance of the respective operation includes maintaining the device in a locked state (2256). In some examples, suspending performance of the respective action includes displaying 2258 an alternative authentication user interface (e.g., 2140, 2198). In some examples, the alternative authentication interface is a passcode, password, pattern, or fingerprint authentication user interface. In some examples, while displaying the alternative authentication user interface, the electronic device detects an alternative authentication attempt (e.g., entry of a passcode that includes a tap gesture 2102A) that corresponds to the alternative authentication user interface. In some examples, the alternative authentication challenge corresponding to the alternative authentication user interface is an input passcode, an input password, an input pattern, or a fingerprint detected on a fingerprint sensor. In some examples, in response to detecting an alternative authentication attempt corresponding to the alternative authentication user interface, the authentication attempt is successful (e.g., the provided authentication information includes stored passcode, stored password, stored pattern, or stored fingerprint information) and biometric data corresponding to alternative authentication attempts (e.g., the last character of a passcode or when the submit password button is selected) or biometric data captured while an authentication attempt is in progress or shortly thereafter; Depending on the determination that the measurement data or the second biometric data) meets first similarity criteria for stored biometric data corresponding to an authorized user of the device, the electronic device performs the respective operation. In some examples, the electronic device stores additional information based on biometric data corresponding to the alternative authentication attempt as biometric data that can be used in future biometric authentication attempts to identify an authorized user of the device. In some examples, the electronic device learns changes to the user's face for use in authenticating the user in future authentication attempts. In some examples, further in response to detecting an alternative authentication attempt corresponding to the alternative authentication user interface, if the authentication attempt is successful and biometric data corresponding to the alternative authentication attempt is generated to correspond to an authorized user of the device. Upon determining that the first similarity criteria for stored biometric data are met, the electronic device may modify information (e.g., a biometric template) that is used in future biometric authentication attempts to identify an authorized user of the device. Output (e.g., visual, audio, and/or tactile output) indicating that something has been done. In some examples, the electronic device displays an indication that the biometric data has been updated to better recognize the user's face.

In some examples, if an authentication attempt is successful (e.g., provided authentication information matches stored authentication information, such as a stored passcode, stored password, stored pattern, or stored fingerprint information) and biometric data corresponding to an alternative authentication attempt (e.g., biometric data or secondary biometric data captured during or immediately after an authentication attempt, such as the last character of a passcode or when a submit password button is selected), corresponding to the authorized user of the device. Depending on the determination that the first similarity criteria for the stored biometric data are not met, the electronic device performs the respective action. Performing each operation in accordance with a determination that the authentication attempt is successful and that the biometric data corresponding to the alternative authentication attempt does not meet the first similarity criteria for the stored biometric data includes: Provides users with an alternative method for accessing operations on the device that require successful authentication (eg, locked operations) when they do not correspond to data. Providing additional control options with additional displayed controls improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and allows the user to -Makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the electronic device does not store additional information based on biometric data corresponding to the alternative authentication attempt as biometric data that can be used in future biometric authentication attempts to identify an authorized user of the device. . In some examples, in response to detecting an alternative authentication attempt corresponding to an alternative authentication user interface, upon determining that the authentication attempt was not successful (e.g., the provided authentication information may be a stored passcode, a stored password, a stored pattern or does not match a stored authentication (matches stored authentication information, such as stored fingerprint information), the electronic device suspends performing the respective operation and retrieves additional information based on the biometric data corresponding to an alternative authentication attempt. Biometric data that can be used in future biometric authentication attempts to identify authorized users is not stored.

In some examples, in response to detecting a request to perform a respective operation and upon determining that the respective operation requires authentication and the device is in a locked state, the electronic device displays an alternative authentication interface. Providing alternative authentication interfaces (e.g., to provide alternative methods for providing authentication as an alternative to or in addition to biometric authentication) may be useful in cases where current authentication methods are not successful or continue to be unsuccessful. This allows users to easily provide authentication for their actions using different authentication methods. Providing additional control options (e.g., to provide authentication) in this manner without cluttering the UI with additional displayed controls allows the user to enter appropriate inputs (e.g., when operating/interacting with the device). improves device operability and makes the user-device interface more efficient (by helping provide and reduce user errors), which additionally reduces the device's power usage by allowing users to use the device more quickly and efficiently. Reduces battery life and improves battery life.

In some examples, the electronic device, in response to a user requesting access to secure data after failure of the first iteration of biometric authentication, provides an alternative authentication interface, such as a password or passcode interface (e.g., 2140, 2198). ) is displayed. In response to a user requesting access to secure data after a failure of a first iteration of biometric authentication, displaying an authentication interface, such as a password or passcode interface, may be used to determine successful authentication when biometric data is unsuccessful. Provides users with a quick alternative way to access required device operations (eg, locked operations). Providing additional control options with additional displayed controls improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and allows the user to -Makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, an alternative authentication interface is displayed after an additional time delay and/or after an additional biometric authentication attempt fails (e.g., as described in more detail with reference to FIGS. 26A-26AS). In some examples, the biometric authentication criteria include a requirement that authentication using an alternative authentication interface has not yet begun for the biometric authentication criteria to be met (2260). In some examples, biometric authentication is attempted at least partially while displaying an alternative authentication user interface (e.g., in response thereto): upon determining that biometric authentication was successful and authentication using the alternative authentication interface has not been initiated. , the electronic device performs each operation; Upon a determination that authentication using the alternative authentication interface has begun (e.g., a determination that at least a partial credential, such as a partial passcode, pattern, or password, has been received using the alternative authentication interface), the electronic device performs a biometric measurement. Suspend performing each operation based on authentication. In some examples, the electronic device waits to perform the second biometric authentication until the user finishes providing the passcode. In some examples, the device delays displaying the alternative authentication user interface until after the second biometric authentication verification fails (e.g., as described in more detail with reference to FIGS. 26A-26AS), and the device A third biometric authentication verification is performed after the user interface is displayed.

In some examples, a second biometric authentication verification is performed 2262 while displaying an alternative authentication interface (e.g., 2140). Performing a second biometric authentication verification while displaying the alternative authentication interface may, in some examples, include operating the device by completing the second biometric authentication verification prior to the user completing providing manual alternative authentication input. improves performance, thereby making the user-device interface more efficient.

In some examples, the alternative authentication interface is a passcode, password, pattern, or fingerprint authentication user interface. In some examples, performing at least a portion of the second biometric authentication verification includes performing at least a portion of the second biometric authentication verification while displaying an alternative authentication interface. In some examples, biometric authentication is performed during passcode entry. In some examples, biometric authentication UI is displayed on the passcode entry interface (e.g., biometric progress indicators 2142, 2144, 2146).

In some examples, while displaying an alternative authentication user interface (eg, a passcode, password, pattern, or fingerprint authentication user interface), the electronic device determines that biometric authentication criteria have been met. In some examples, in response to determining that biometric authentication criteria have been met, the electronic device performs the respective operation. In some examples, performing each action in response to determining that the biometric authentication criteria have been met while displaying the alternative authentication user interface may include performing a second action prior to the user completing providing manual alternative authentication input. Completing biometric authentication verification improves device operability, thereby making the user-device interface more efficient. In some examples, biometric authentication criteria include a requirement that the user did not enter at least a portion of the credential using an alternative authentication interface. In some examples, biometric authentication criteria include a requirement that the user did not enter at least a portion of the credential using an alternative authentication interface. In some examples, the respective action is not performed upon successful biometric authentication if the user has already begun entering an alternative form of authentication, such as a passcode, password, pattern, or fingerprint.

In some examples, in response to detecting a request to perform a respective operation and upon determining that the respective operation requires authentication and the device is in a locked state, the electronic device may have the option to proceed with an alternative form of authentication. Display an authentication indication for biometric authentication (e.g., a progress indicator or other indication that biometric authentication is being attempted). In some examples, the electronic device displays an authentication indication for biometric authentication without displaying an alternative authentication interface and/or without displaying a selectable option for displaying an alternative authentication interface. In some examples, while attempting biometric authentication in response to a request to perform a respective action, the device may provide options for alternative forms of authentication to indicate to the user that biometric authentication has not yet failed. (e.g., as described in more detail with reference to FIGS. 26A-26AS).

In some examples, while the device is in an unlocked state, the electronic device may be prompted to store additional information for use in biometric authentication (e.g., in the biometric registration user interface in the System Preferences user interface or the Device Settings user interface). Receive the request. In some examples, in response to a request to store additional information for use in biometric authentication, the electronic device captures third biometric data. In some examples, additionally, in response to a request to store additional information for use in biometric authentication, the third biometric data may be configured to generate second similarity criteria for the stored biometric data corresponding to the authorized user of the device. Following a determination of compliance, the electronic device stores additional information based on the third biometric data that can be used in future biometric authentication attempts to identify an authorized user of the device. storing additional information based on third-party biometric data that can be used in future biometric authentication attempts to identify the authorized user of the device while the device is in an unlocked state (e.g., prohibiting such operations while on the device) improves device security, preventing fraudulent and/or unauthorized attempts to access biometric authentication data stored on the device and thereby gain future access to the device. . Improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally reduces the device's power usage by enabling users to use the device more efficiently. reduces and improves battery life.

In some examples, the electronic device learns changes to the user's face for use in authenticating the user in future authentication attempts. In some examples, the second similarity criteria require that the similarity between the third biometric data and the stored biometric data be less than that required by the first similarity criteria. In some examples, when the device is in an unlocked state and is in the biometric data registration user interface, the device learns biometric characteristics that are detected when alternative authentication is successfully provided after biometric authentication fails. and configured to accept additional biometric data corresponding to biometric characteristics that are more different from the currently registered biometric characteristics than the current registered biometric characteristics. In some examples, upon determining that the third biometric data does not meet the second similarity criteria for stored biometric data corresponding to an authorized user of the device, the electronic device may be configured to identify the authorized user of the device. Stores additional information based on third party biometric data that can be used in future biometric authentication attempts. In some examples, the electronic device learns changes to the user's face for use in authenticating the user in future authentication attempts.

Note that the details of the processes described above with respect to method 1200 (e.g., FIGS. 22A-22F) are also applicable in a similar manner to the methods described herein. For example, method 2200 optionally includes one or more of the features of the various methods described herein with reference to methods 800, 1000, 1200, 1400, 1600, 1800, 2000, 2500, and 2700. Includes. For example, the registered biometric data described in method 1200 may be used to perform biometric authentication as described with respect to FIGS. 21C-21E. For another example, one or more implantable interfaces as described in methods 2000 and 2700 are optionally displayed in response to receipt of input prior to completion of the biometric authentication process. For brevity, these details are not repeated herein.

Operations in the information processing methods described above optionally utilize one or more functional modules within an information processing device, such as general-purpose processors (e.g., as described with respect to FIGS. 1A, 3, and 5A) or application-specific chips. It is implemented by running it. Additionally, the operations described above with reference to FIGS. 22A-22F are optionally implemented by the components shown in FIGS. 1A and 1B. For example, detect operation 2202, perform operation 2210, capture operation 2212, transition operation 2216, maintain operation 2224, detect operation 2226, perform operation 2250, and hold operation ( 2254) is optionally implemented by an event classifier 170, an event recognizer 180, and an event handler 190. Event monitor 171 in event sorter 170 detects contact on touch-sensitive surface 604, and event dispatcher module 174 passes event information to application 136-1. The respective event recognizer 180 of application 136-1 compares the event information to the respective event definition 186 and determines that the first contact at the first location on the touch-sensitive surface corresponds to the selection of an object on the user interface. Determines whether it corresponds to the same predefined event or subevent. When an individual predefined event or sub-event is detected, event recognizer 180 activates an event handler 190 associated with detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some examples, event handler 190 accesses the respective GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes may be implemented based on the components shown in FIGS. 1A and 1B.

23A-23Q illustrate example user interfaces for managing biometric authentication, according to some examples. As described in more detail below, the illustrative example of the user interfaces illustrated in FIGS. 23A-23Q relates to the illustrative example of the user interfaces illustrated in FIGS. 24A-24B, which in turn are illustrated in FIGS. 25A-25B. It is used to illustrate the processes described below, including the processes in 25c.

FIG. 23A illustrates an electronic device 2300 (e.g., portable multifunction device 100, device 300, or device 500). In the illustrative example illustrated in FIGS. 23A-23Q, electronic device 2300 is a smartphone. In other examples, electronic device 2300 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 2300 has a display 2302, one or more input devices (e.g., a touchscreen of display 2302, buttons 2304, a microphone (not displayed)), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the electronic device includes one or more biometric sensors (e.g., biometric sensor 2303), optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, one or more biometric sensors 2303 are one or more biometric sensors 703. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In FIG. 23A , electronic device 2300 displays photo gallery user interface 2310 on display 2302 . In some examples, photo gallery user interface 2310 slides into the display from an edge of the display (e.g., sliding up from the bottom edge of the display), replacing the display of a previous interface, application, and/or virtual keyboard. In some examples, photo gallery user interface 2310 slides up in response to a request to open the photo gallery application. In some examples, photo gallery user interface 2310 slides up in response to a request to send photos to a participant in a conversation in a messaging application.

In some examples, as shown in FIG. 23A, photo gallery user interface 2310 may display a plurality of selectable photos corresponding to photos stored on electronic device 2300 (or accessible by the device via a remote server). Includes preview images. In some examples, as also shown in Figure 23A, a plurality of selectable preview images are displayed based on time (e.g., the date the photo was taken) and/or based on location (e.g., where the photo was taken). Be organized. For example, the plurality of selectable preview images 2312A to 2312F shown below header 2312 correspond to photos taken on April 30 in Cupertino, California, USA, and shown below header 2314 A plurality of selectable preview images 2314A to 2314C correspond to photos taken yesterday in San Francisco, California, USA.

In some examples, upon launching a photo gallery application, electronic device 2300 displays selectable preview images of photo gallery user interface 2310 that may be selected (for transmission). For example, the plurality of selectable preview images include selectable preview images 2314-2314C. As shown in FIG. 23A, a plurality of selectable preview images 2314A-2314C may be selected by the user to be sent to the participant via one or more applications, such as a messaging application or an email application.

23B , while displaying photo gallery user interface 2310 with selectable preview images 2314A - 2314C (corresponding to photos selected for transfer), electronic device 2300 displays selectable preview images 2314A - 2314C. Detect user activation 2301 of a transmission affordance 2316 (e.g., a button) to initiate transmission of photos corresponding to images 2314A, 2314B. For example, user activation 2301 is a tap gesture on transfer affordance 2316.

23C, in response to detecting activation of transmission affordance 2316, electronic device 2300 provides prompt 2318. As illustrated in FIG. 23C , in some examples, prompt 2318 instructs the user to provide one or more activations of button 2304, such as double pressing button 2304. In some examples, prompt 2318 is emphasized relative to one or more other displayed objects. Highlighting a prompt in this manner includes, for example, darkening, blurring, and/or otherwise obscuring one or more portions of photo gallery user gallery 2310.

As also illustrated in FIG. 23C , further in response to detecting activation of transfer affordance 2316 , electronic device 2300 displays an application selection interface ( 2320) is displayed. In some examples, each of application affordances 2320A through 2320H is configured to transmit images (to a different device other than electronic device 2300), such as images corresponding to selectable preview images 2314A through 2314C. Corresponds to the application in which it can be used.

In some examples, button 2304 has a fixed location relative to display 2302 and/or one or more other components of electronic device 2300. In some examples, prompt 2318 is also displayed at a fixed location relative to display 2302 and/or one or more other components of the electronic device. In this way, prompt 2318 is displayed at a predetermined location relative to button 2304.

In Figure 23D, while displaying prompt 2318 (optionally overlaid on the photo gallery user interface), electronic device 2300 detects user activation 2306 of button 2304. In some examples, as shown in Figure 23D, user activation is a double press of button 2304. In some examples, a double press of button 2304 includes a first press of the button and a second press of the button occurring within a predetermined amount of time (eg, 1 second).

In response to detecting one or more activations of button 2304, electronic device 2300 initiates biometric authentication (e.g., facial recognition authentication) of the user's biometric feature (e.g., face). As illustrated in FIG. 23E , in some examples, upon initiation of biometric authentication, a biometric authentication interface 2322 is provided (e.g., displayed on display 2302). In some examples, the biometric authentication interface is overlaid on an application interface, such as photo gallery user interface 2310, during biometric authentication. In some examples, biometric authentication includes simulation of a representation of a biometric feature, such as glyph 2324. Additionally, in response to one or more activations of button 2304, one or more biometric sensors 2303 of electronic device 2300, such as one or more cameras or a facial recognition sensor (e.g., one or more biometric sensors (included in 2303) is activated.

In some examples, once one or more biometric sensors 2303 are activated, electronic device 2300 obtains (e.g., captures) biometric data corresponding to biometric characteristics associated with the user. In some examples, the biometric feature captures biometric data using one or more biometric sensors 2303 of the electronic device (and/or biometric sensors of one or more cameras). Optionally, a light-emitting device, such as an IR floodlight or structured light projector, is used to help illuminate the biometric features. In other examples, an electronic device receives biometric data from another device.

In some examples, once electronic device 2300 has acquired biometric data, the electronic device processes (e.g., analyzes) the biometric data to determine whether biometric authentication was successful. In some examples, this determination includes determining whether biometric data matches a biometric template associated with the user. The biometric template is optionally stored on electronic device 2300.

In some examples, while processing biometric data, as illustrated in Figure 23F, biometric authentication interface 2322 may be configured to use a biometric authentication interface, e.g., as described with reference to Figures 17A-17AI. Use to display one or more rotating rings 2326 to indicate that biometric data is being processed by the electronic device. In some examples, one or more rotating rings 2326 replace glyph 2324 in the biometric authentication interface.

If the electronic device 2300 determines that biometric authentication is successful (e.g., biometric data matches a biometric template associated with the user), the electronic device has the function (e.g., image transmission) disabled. transitions to a state in which the function is enabled. As an example, successful biometric authentication allows the electronic device to transmit (eg, share) images, such as images corresponding to selectable preview images 2314A-2314C. In some examples, the electronic device further indicates that biometric authentication was successful, such as by displaying a simulation of the representation of the biometric feature in a biometric authentication interface. As shown in Figure 23G, in some examples, biometric authentication interface 2322 includes a glyph 2328 that indicates (to the user) that biometric authentication was successful. In some examples, glyph 2328 replaces one or more rotating rings 2326 in biometric authentication interface 2322.

23H, after image transmission is enabled on the electronic device 2300 in response to successful biometric authentication, and while displaying the application selection interface 2320, the electronic device (to launch the corresponding application) ) Detect user activation 2305 of application affordances. For example, the activated affordance is application affordance 2320A. Activation of application affordance 2320A may cause the electronic device to launch an application 2330 (e.g., a messaging application) corresponding to application affordance 2320A (e.g., a messaging application), as shown in FIG. 23I. (simultaneously with startup) images corresponding to the selectable preview images 2314A to 2314C are transmitted using the application 2330.

If the electronic device 2300 determines that biometric authentication was not successful (e.g., the biometric data does not match the biometric template associated with the user), the electronic device may transfer images between states (e.g., It maintains the same state without transitioning from a state in which the same function as being applied is disabled to a state in which the function is enabled. In some examples, the electronic device further indicates (to the user) that biometric authentication was not successful, such as by displaying a simulation of the representation of the biometric features in a biometric authentication interface. As shown in Figure 23J, in some examples, biometric authentication interface 2322 includes a glyph 2332 indicating that biometric authentication was not successful. Glyph 2332 indicates, for example, that the biometric characteristic has not been recognized by the electronic device.

23K, after image transmission is not enabled on electronic device 2300 in response to unsuccessful biometric authentication, and while displaying application selection interface 2320, the electronic device selects the user of application affordance 2320A. Detect activation (2307). In some examples, as shown in Figure 23L, in response to detecting activation of application affordance 2320A of application selection user interface 2320, electronic device 2300 may select alternative authentication affordance 2334 (e.g., Password affordance, passcode affordance) are displayed. In some examples, while displaying alternative authentication affordances 2334, the electronic device detects user activation 2309 of alternative authentication affordances 2334. Activation of an alternative authentication affordance results in the display of an alternative authentication interface 2336 (e.g., password interface, passcode interface), as shown in FIG. 23M.

In some examples, electronic device 2300 performs biometric authentication during display of alternative authentication interface 2336. In some examples, the electronic device acquires and processes biometric data to determine whether the obtained biometric data matches a biometric template associated with the user. As such, in some examples, alternative authentication interface 2336 may simulate a representation of a biometric feature, such as glyph 2338 (e.g., corresponding to glyph 2324), as shown in FIG. 23M. Includes. In some examples, alternative authentication interface 2336 may include one or more rotating rings 2340 (e.g., as shown in Figure 23N and described with reference to Figures 17A-17AI, for example). By displaying one or more rotating rings (corresponding to 2326), it indicates that biometric data is being processed by the electronic device. In some examples, one or more rotating rings 2340 replace glyph 2338 in the alternative authentication interface.

If the electronic device 2300 determines that biometric authentication was not successful (e.g., the biometric data does not match the biometric template associated with the user), the electronic device disables the function (e.g., image transmission). It remains in its current state. In some examples, the electronic device further indicates that biometric authentication was not successful, such as by displaying a simulation of the representation of the biometric characteristic in alternative authentication interface 2336. As shown in FIG. 23O, in some examples, alternative authentication interface 2336 includes glyph 2342 (e.g., corresponding to glyph 2328) that indicates (to the user) that biometric authentication was not successful. do. In some examples, glyph 2342 replaces one or more rotating rings 2340 in an alternative authentication interface.

In some examples, in addition to or instead of biometric authentication, electronic device 2300 performs passcode authentication during display of alternative authentication interface 2336. Accordingly, the electronic device receives and processes the passcode data to determine whether the received passcode data matches a registered passcode associated with the user. As such, in some examples, alternative authentication interface 2336 includes an indication of a received passcode input, such as passcode indication 2344, as shown in FIG. 23P.

As previously discussed, if the electronic device 2300 determines that the biometric authentication and/or passcode authentication is successful, the electronic device moves the function (e.g., image transmission) from a disabled state to an enabled state. Transitions to the enabled state. For example, as shown in FIG. 23Q, successful biometric authentication and/or passcode authentication may require the electronic device, via application 2330 (e.g., a messaging application), to display selectable preview images 2314A-2314C. Allows transmission (e.g., sharing) of images, such as images corresponding to .

As previously mentioned, the illustrative example of the user interfaces illustrated in FIGS. 23A-23Q described above is related to the illustrative example of the user interfaces illustrated in FIGS. 24A-24B described below. Accordingly, the processes described above with respect to the example user interfaces illustrated in FIGS. 23A-23Q and the processes described below with respect to the example user interfaces illustrated in FIGS. 24A-24B may be used with an electronic device (e.g. , 100, 300, 500, 2300, or 2400) are very similar processes that similarly involve initiating and managing biometric authentication.

24A-24B illustrate example user interfaces for managing biometric authentication, according to some examples. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 25A-25C.

FIG. 24A illustrates an electronic device 2400 (e.g., portable multifunction device 100, device 300, or device 500). In the illustrative example illustrated in FIGS. 24A-24B, electronic device 2400 is a smartphone. In other examples, electronic device 2400 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 2400 has a display 2402, one or more input devices (e.g., a touchscreen of display 2402, buttons 2404, a microphone (not displayed)), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the electronic device includes one or more biometric sensors (e.g., biometric sensor 2403), optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In FIG. 24A , electronic device 2400 displays instructional user interface 2410 on display 2402 . In some examples, tutorial user interface 2410 slides into the display from an edge of the display (e.g., sliding up from the bottom edge of the display), replacing the display of a previous interface, application, and/or virtual keyboard. In some examples, tutorial user interface 2410 slides up in response to a request to proceed with a payment transaction (e.g., with a different device, such as a transaction terminal).

In some examples, as shown in FIG. 24A, instructional user interface 2410 may include a text display indicating to the user that an input (e.g., double pressing button 2404) may be performed to proceed with a payment transaction. Includes (2410A). In some examples, as also shown in FIG. 24A, the instructional user interface 2410 provides instructions (to the user) that input (e.g., double pressing button 2404) may be performed to proceed with a payment transaction. and a graphical representation 2410B, corresponding to a textual representation 2410A.

In some examples, in response to detecting user activation of an affordance on tutorial user interface 2410 (e.g., a “continue” affordance), electronic device 2400 displays display 2402, as shown in FIG. 24B. On top, the payment account currently selected for use in the payment transaction is overlaid by a prompt 2416 directing the user to provide one or more activations of button 2404 (e.g., double pressing button 2404). Displays a payment user interface 2412 including a representation of 2414. In some examples, prompt 2416 is emphasized relative to one or more other displayed objects (on payment user interface 2412). Highlighting a prompt in this manner includes, for example, darkening, blurring, and/or otherwise obscuring one or more portions of payment user interface 2412.

In some examples, the location of button 2404 that is requested by prompt 2416 to be activated (e.g., to proceed with a payment transaction) is highlighted by dynamic indication 2418. For example, as shown by the transition from Figure 24B to Figure 24C, dynamic display 2418 may continuously change size (e.g., becoming wider and wider) adjacent the position of button 2404 on the display. emphasizing the position of buttons 2404 on the device (by continuously alternating between narrowing, or otherwise continuously changing size), thereby making it easier for the user to find the button corresponding to the request of prompt 2416 make it possible

24D , while displaying prompt 2416, electronic device 2400 detects activation 2401 of button 2404. In some examples, as shown in Figure 24D, activation is a double press of button 2404. In some examples, a double press of button 2404 includes a first press of the button and a second press of the button occurring within a predetermined amount of time (eg, 1 second).

In response to one or more activations of button 2404, electronic device 2400 displays prompt 2416 (and any corresponding prompts) overlaid on payment user interface 2412, as shown in FIG. 24E. highlight) and display of dynamic indication 2418 and initiate biometric authentication (e.g., facial recognition) of the user's biometric features (e.g., face), as shown in FIG. 24F. In some examples, the biometric feature is at least a portion of a face (eg, the user's face), and biometric authentication involves facial recognition of at least a portion of (the user's) face.

As illustrated in FIG. 24F , in some examples, upon initiation of biometric authentication, a biometric authentication interface 2420 is provided. In some examples, the biometric authentication interface is overlaid on payment user interface 2412 during biometric authentication. In some examples, biometric authentication includes simulation of a representation of a biometric feature, such as glyph 2422. Additionally, in response to one or more activations of button 2404, one or more biometric sensors of electronic device 2400, such as one or more cameras or a facial recognition sensor (e.g., one or more biometric sensors 2403) included) is activated. In some examples, the electronic device displays biometric authentication interface 2420 in a central area of the display and displays (e.g., by displacing or moving up) a representation of payment account 2414 in an upper portion of the display.

In some examples, once one or more biometric sensors are activated, electronic device 2400 obtains (e.g., captures) biometric data corresponding to biometric characteristics associated with the user. In some examples, the biometric feature captures biometric data using one or more biometric sensors 2403 of the electronic device (and/or biometric sensors of one or more cameras). Optionally, a light-emitting device, such as an IR floodlight or structured light projector, is used to help illuminate the biometric features. In other examples, an electronic device receives biometric data from another device.

In some examples, once electronic device 2400 has acquired biometric data, the electronic device processes (e.g., analyzes) the biometric data to determine whether biometric authentication was successful. In some examples, this determination includes determining whether biometric data matches a biometric template associated with the user. The biometric template is optionally stored on electronic device 2400.

In some examples, as illustrated in FIG. 24G, biometric authentication interface 2420 may allow biometric data to be transmitted to an electronic device, for example, by displaying one or more rotating rings 2424 using the biometric authentication interface. Indicates that it is being processed by . In some examples, one or more rotating rings 2424 replace glyph 2422 in the biometric authentication interface.

If the electronic device 2400 determines that biometric authentication is successful (e.g., biometric data matches a biometric template associated with the user), the electronic device 2400 performs a function (e.g., authorization to send payment credentials). ) transitions from the first state in which the function is disabled to the second state in which the function is enabled. In some examples, the first state is a state in which the secure element of the device is disabled from emitting secure data (e.g., payment credentials of a payment account provisioned on the device), and the second state is a state in which the secure element of the device is disabled It is enabled to emit secure data.

In some examples, successful biometric authentication authorizes the electronic device to transmit account credentials in connection with a payment transaction. In some examples, the electronic device further indicates (to the user) that biometric authentication was successful, such as by displaying a simulation of the representation of the biometric features in a biometric authentication interface. As shown in Figure 24H, in some examples, biometric authentication interface 2420 includes a glyph 2426 that indicates (to the user) that biometric authentication was successful. In some examples, glyph 2426 replaces one or more rotating rings 2424 within the biometric authentication interface.

24I , following electronic device 2400 determining that biometric authentication was successful, the electronic device provides authorization to proceed with a payment transaction using the currently selected payment account (e.g., payment account 2414). It indicates (to the user) that the transaction has been completed and the payment transaction can be initiated accordingly. In some examples, electronic device 2400 displays textual representation 2428A and/or graphical representation 2428B to indicate that a payment transaction may be initiated. In some examples, textual representation 2428A and/or graphical representation 2428B replaces biometric authentication interface 2420 on payment user interface 2412, as shown in FIG. 24I. In some examples, graphical representation 2428B replaces glyph 2426 within payment user interface 2412.

24J , while displaying a payment user interface 2412 with a payment account 2414 selected and authorized for use in a payment transaction, electronic device 2400 (e.g., via the device's wireless transmitting radio) A second device 2430 (eg, a transaction terminal) that is different from the electronic device is detected. In response to detecting a second device (e.g., a transaction terminal), electronic device 2400 transmits payment credentials associated with payment account 2414 (e.g., via the device's wireless transmitting radio) to the second device. Complete the payment transaction.

In some examples, after successfully transmitting payment credentials to second device 2430, electronic device 2400 may indicate (to the user) that the payment transaction was successfully completed, as shown in FIG. 24K. Update the textual representation 2428A (e.g., “payment completed”) and/or the graphical representation 2428B (e.g., a check mark) within the payment user interface 2412.

In some examples, prior to using payment account 2414 to proceed with a payment transaction with a second device (e.g., a transaction terminal), a different payment account may be selected for use in the transaction. In some examples, as shown in FIG. 24L, electronic device 2400 may display within payment user interface 2412 (e.g., in a lower portion of the interface) a different payment account (e.g., payment account) than the currently selected payment account 2414. Display one or more representations of accounts 2432A through 2432C. In some examples, as shown in FIG. 24L, electronic device 2400 may be configured to enable user selection (e.g., payment account 2432A) of one or more payment accounts that are different from payment account 2414. 2406) (e.g., a tap gesture).

In some examples, when the second device is a transaction terminal in a store, to authorize an in-store payment using biometric authentication (e.g., facial recognition authentication), the user first presses a hardware button (e.g., button 2404). You must confirm your intention to pay by activating (double-clicking the sleep/wake button). In some examples, the user then authenticates using biometric authentication (eg, facial recognition authentication) before placing the device near a second device (eg, transaction terminal). In some examples, if the user wishes to select a different payment method after biometric authentication (e.g., facial recognition authentication), the device prompts the user to re-authenticate with biometric authentication, but does not require the user to use a hardware button (e.g., button (2404)) is not required to be activated (e.g., double-clicking the sleep/wake button again).

In some examples, upon user selection 2406 of payment account 2432A, as shown in FIG. 24M, a representation of payment account 2432A slides up within payment user interface 2412 and displays payment account 2414. ) slides down within the payment user interface 2412. In some examples, as shown in FIG. 24N, the representation of payment account 2432A slides up within the payment user interface 2412 into the position previously occupied by the representation of payment account 2414 (thereby making the payment). Indicating to the user that account 2432A is now selected for use in a payment transaction), a representation of payment account 2414 is configured to join one or more representations of payment accounts different from the currently selected payment user interface ( 2412) and slide down within. Once the currently selected payment account has been switched from payment account 2414 to payment account 2432A, the device proceeds with a payment transaction (e.g., as described with reference to FIG. 24J) using payment account 2432A to complete the transaction. can be completed.

In some examples, the technique described with reference to FIGS. 24B-24N allows electronic device 2400 to display an application 2434 on display 2402 that is different from the pointing user interface 2410, as shown in FIG. 24O. It may be initiated (e.g., as initiated by activation 2401 according to prompt 2416) while being displayed (e.g., a messaging application). For example, FIG. 24O illustrates electronic device 2400 displaying application 2434 (e.g., a messaging application) on display 2402. While displaying application 2434, electronic device 2400 receives user activation (e.g., double press 2405) of button 2404. In response to receiving user activation, the electronic device proceeds with the techniques described with reference to FIGS. 24B-24N to perform biometric authentication to proceed with a payment transaction (e.g., with a second device that is different from the electronic device). obtain.

In some examples, the technique described with reference to FIGS. 24B-24N may be performed while the display 2402 of the electronic device 2400 is in an off state (e.g., at a prompt 2416), as shown in FIG. 24P. (as initiated by activation 2401 while the application 2434 is displayed) or by activation 2405 while the application 2434 is displayed. While display 2402 is in the off state, electronic device 2400 receives user activation (e.g., double press 2407) of button 2404. In response to receiving user activation, the electronic device proceeds with the techniques described with reference to FIGS. 24B-24N to perform biometric authentication to proceed with a payment transaction (e.g., with a second device that is different from the electronic device). obtain.

FIG. 24Q illustrates electronic device 2400 with display 2402 in an off state. While display 2402 is in the off state, the electronic device detects (e.g., via the device's wireless communication radio) a second device 2430 (e.g., a transaction terminal). In some examples, in response to detecting the second device 2430 while the display 2402 is in an off state, the electronic device 2400 displays an image on the display 2402 to proceed with a payment transaction (e.g., Displays prompt 2416 and payment user interface 2412 (as illustrated in 24b).

If the electronic device 2400 determines that biometric authentication was not successful (e.g., the biometric data does not match the biometric template associated with the user), the device does not transition from the first state to the second state. , in some examples, the electronic device remains in the first state (e.g., authorization to process the payment transaction remains disabled). In some examples, the electronic device further indicates that biometric authentication was not successful, such as by displaying a simulation of the representation of the biometric features in biometric authentication interface 2420. As shown in Figure 24R, in some examples, biometric authentication interface 2420 includes glyph 2436 indicating that biometric authentication was not successful. Glyph 2436 indicates, for example, that the biometric characteristic has not been recognized by the electronic device. In some examples, in addition to glyphs 2436 in biometric authentication interface 2420, electronic device 2400 may display a tactile output 2438 (e.g., haptic) that further indicates (to the user) that biometric authentication was not successful. generate feedback).

In some examples, glyph 2436 is moved further (e.g., horizontally or vertically) within the area of biometric authentication interface 2420 to further indicate (to the user) that biometric authentication was not successful. do. For example, as shown in the transition from Figure 24R to Figure 24S to Figure 24T, glyph 2436 slides back and forth in the horizontal direction (e.g., repeating continuous sliding movement left and right) for a predetermined period of time (e.g., 3 seconds). box). In some examples, the device continues to generate tactile output 2438 (e.g., haptic feedback) for the duration of movement of glyph 2436 within biometric authentication interface 2420. In some examples, tactile output 2438 is synchronized with the movement of glyph 2436.

In some examples, as shown in FIG. 24U, while displaying glyphs 2436 within biometric authentication interface 2420 indicating to the user that biometric authentication was not successful, electronic device 2400 may Alternative authentication affordances 2440 are provided within payment user interface 2412 (e.g., below biometric authentication interface 2420) to provide alternative (e.g., passcode, password) authentication in addition to or instead of measurement authentication. ) Display it and proceed with the payment transaction. In some examples, while displaying an alternative authentication affordance 2440 to provide alternative authentication, the electronic device detects a user selection of the affordance 2411 (e.g., a tap gesture), as shown in Figure 24V. .

As shown in FIG. 24W, in response to detecting user selection of affordance 2440, electronic device 2400 displays alternative authentication interface 2442 on display 2402. In some examples, electronic device 2400 performs biometric authentication during display of alternative authentication interface 2442. In some examples, the electronic device acquires and processes biometric data to determine whether the obtained biometric data matches a biometric template associated with the user. As such, in some examples, alternative authentication interface 2442 may include a simulation of a representation of a biometric feature, such as glyph 2444 (e.g., corresponding to glyph 2422), as shown in FIG. 24W. Includes. In some examples, alternative authentication interface 2442 may be implemented by displaying one or more rotating rings 2446 (e.g., corresponding to one or more rotating rings 2424), e.g., as shown in FIG. 24X. , indicating that biometric data is being processed by an electronic device. In some examples, one or more rotating rings 2446 replace glyph 2444 in the alternative authentication interface.

If the electronic device 2400 determines that biometric authentication was successful (e.g., biometric data matches a biometric template associated with the user), the device performs a function (e.g., authorization to send payment credentials). The function transitions from the first state in which it is disabled to the second state in which the function is enabled. In some examples, successful biometric authentication allows the electronic device to transmit payment credentials (e.g., associated with payment account 2414), e.g., to a transaction terminal. In some examples, the electronic device further indicates that biometric authentication was successful, such as by displaying a simulation of the representation of the biometric characteristic in alternative authentication interface 2442. As shown in Figure 23Y, in some examples, alternative authentication interface 2442 includes a glyph 2448 (e.g., corresponding to glyph 2426) that indicates (to the user) that biometric authentication was successful. . In some examples, glyph 2448 replaces one or more rotating rings 2446 in an alternative authentication interface.

In some examples, in addition to or instead of biometric authentication, electronic device 2400 performs passcode authentication during display of alternative authentication interface 2442. Accordingly, the electronic device receives and processes the passcode data to determine whether the received passcode data matches a registered passcode associated with the user. As such, in some examples, alternative authentication interface 2442 includes an indication of a received passcode input, such as passcode indication 2450, as shown in Figure 23Y.

24Z, subsequent to electronic device 2400 determining that biometric authentication (and/or alternative authentication, such as passcode authentication) was successful, the electronic device determines that the currently selected payment account (e.g., payment account 2414 )) to indicate (to the user) that authorization has been provided to proceed with a payment transaction and that a payment transaction (e.g., with a transaction terminal) can be initiated accordingly. In some examples, electronic device 2400 displays textual representation 2452A (e.g., corresponding to textual representation 2428A) and/or graphical representation 2452B (e.g., graphical representation (e.g., graphical representation) to indicate that a payment transaction may be initiated. (corresponding to 2428B)) is displayed.

FIG. 24AA illustrates an electronic device 2400 displaying an alternative authentication interface 2442 similar to the alternative authentication interface of FIG. 24W on a display. As in Figure 24W, electronic device 2400 performs biometric authentication during display of alternative authentication interface 2442. Accordingly, the electronic device acquires and processes biometric data to determine whether the obtained biometric data matches a biometric template associated with the user. As such, alternative authentication interface 2442 includes glyphs 2444 that indicate to the user that biometric data has been (or is being acquired). In Figure 24ab, as in Figure 24x, alternative authentication interface 2442 indicates that biometric data is being processed by the electronic device, for example, by displaying one or more rotating rings 2446.

If the electronic device 2400 determines that biometric authentication was not successful (e.g., biometric data does not match a biometric template associated with the user), the electronic device performs a function (e.g., transmitting payment credentials). suspends the transition from a first state in which the function is disabled to a second state in which the function is enabled (thereby not allowing the device to proceed with a payment transaction). In some examples, electronic device 2400 further indicates (to the user) that biometric authentication was not successful, such as by displaying a simulation of the representation of the biometric characteristic in alternative authentication interface 2442. As shown in Figure 23AC, in some examples, alternative authentication interface 2442 includes a glyph 2454 (e.g., corresponding to glyph 2436) that indicates (to the user) that biometric authentication was not successful. do.

FIG. 24AD illustrates electronic device 2400 displaying an alternative authentication interface 2442 on display 2402 following unsuccessful biometric authentication. The electronic device displays an indication (e.g., via glyph 2454) that biometric authentication was not successful. In some examples, as shown in Figure 24AE, after unsuccessful biometric authentication, electronic device 2400 may display an alternative authentication interface 2442 (e.g., as described with reference to Figures 24B-24J). receive one or more activations of button 2404 (e.g., a double press of button 2404) to restart the biometric authentication process (as described). Accordingly, upon failure of the biometric authentication and/or alternative authentication process, the user may retry the process through one or more activations of button 2404. In some examples, if the biometric authentication process is not successfully attempted for a predetermined number of consecutive attempts, or is not attempted successfully for a predetermined number of consecutive attempts within a predetermined period of time, button 2404 One or more activations (e.g., double pressing button 2404) no longer enable restart of the biometric authentication process (e.g., as described with reference to FIGS. 24B-24J).

24AF, electronic device 2400 detects a second device 2430 (e.g., a transaction terminal) while displaying alternative authentication interface 2442 (e.g., via the device's wireless communication radio). . In some examples, in response to detecting the second device 2430 while displaying an alternative authentication interface, the electronic device 2400 may display a prompt to proceed with the payment transaction (e.g., a prompt as illustrated in FIG. 24B ). 2416) and displays a payment user interface.

24AG is authorized to initiate a transaction (e.g., after biometric authentication and/or alternative authentication, such as passcode authentication has been successfully provided) using the currently selected payment account (e.g., payment account 2414) and display ( illustrates an electronic device 2400 that displays a payment user interface 2412 on 2402). In some examples, payment user interface 2412 includes a menu tab 2456 (e.g., in the lower area of the interface, next to the lower edge of the display), as shown in Figure 24ag. In some examples, the electronic device detects a sliding gesture 2415 (e.g., in an upward direction) of the menu tab 2456. For example, sliding gesture 2415 corresponds to a touch-and-slide gesture (in the upward direction) of menu tab 2456.

In some examples, sliding gesture 2415 on menu tab 2456 expands the menu tab (e.g., over payment user interface 2412), as shown in Figure 24A. Once expanded, menu tab 2456 includes one or more application affordances (e.g., application affordances 2456A-2456D) that are accessible from the menu tab and correspond to applications installed on the device. For example, the menu tab 2456 has a first application affordance 2456A corresponding to a messaging application, a second application affordance 2456B corresponding to a voice call application, and a third application affordance 2456C corresponding to an email application. , and a fourth application affordance 2456D corresponding to a browsing application. In some examples, only first party applications (controlled only by the device's operating system) may be included within menu tab 2456.

FIG. 24AI illustrates an electronic device 2400 detecting a sliding gesture 2415 while the gesture slides in a downward direction across the display (and thereby collapses the expanded menu bar). As a result of the sliding gesture 2415 of the menu bar 2456 in a downward direction, the menu bar returns to its original size and position (e.g., at the bottom of the payment user interface 2412), as shown in Figure 24aj. Zoom out (or fold back into it). Once the menu bar is fully collapsed, the payment user interface is again fully visible on the display.

FIG. 24AK illustrates electronic device 2400 displaying a webpage 2458 of a browsing application on display 2402 . For example, the web page 2458 is a checkout page for an item 2460 that the user wants to purchase, and includes a purchase affordance 2462 for purchasing the item. In some examples, as shown in Figure 24AK, the electronic device detects user activation 2417 of purchase affordance 2462.

In some examples, upon detecting user activation of purchase affordance 2462, electronic device 2400 displays payment sheet interface 2464 on display 2402, as shown in FIG. 24A. In some examples, payment sheet interface 2464 (partially) overlays a browsing application and includes biometric authentication interface 2420. In some examples, as also shown in Figure 24A, in addition to the payment sheet interface, the device may further perform one or more activations (e.g., double pressing button 2404) to proceed with providing authorization for a purchase. Displays a prompt 2466 (e.g., corresponding to prompt 2416) indicating (to the user) to provide:

In some examples, payment sheet interface 2464 may display one or more details related to the proposed transaction (e.g., payment account, shipping method, billing address, shipping address, contact information), as shown in Figure 24A. Includes. In some examples, the one or more details include a selected payment account. In some examples, the user can change the selected payment account to a different payment account by selecting the details area 2464A (the area within or the icon therein) corresponding to the selected payment account. In some examples, one or more details include a selected shipping method. In some examples, the user can change the selected delivery method to a different delivery method by selecting the details area 2464B (an area within or an icon therein) that corresponds to the selected delivery method. In some examples, the one or more details include a selected address (eg, billing address, shipping address). In some examples, a user can change a selected address to a different address by selecting the details area 2464C (an area within or an icon within) that corresponds to the selected address. In some examples, the one or more details include selected contact information (eg, email, phone number). In some examples, a user can change selected contact information to different contact information by selecting the details area 2464D (an area within or an icon therein) that corresponds to the selected contact information.

24AM, after displaying payment sheet interface 2464 and prompt 2466 over webpage 2458 of a browsing application, electronic device 2400 selects an input (e.g., a button) corresponding to the request of prompt 2466. 2404) to detect a double press. In some examples, upon receiving input corresponding to the request of prompt 2466 (e.g., double pressing button 2404), a glyph in biometric authentication interface 2420 displayed within payment sheet interface 2464 2468 (e.g., corresponding to glyph 2422) is provided. Additionally, in response to the input, one or more biometric sensors of electronic device 2400, such as one or more cameras or a facial recognition sensor (e.g., included among one or more biometric sensors 2403) are activated.

In some examples, once one or more biometric sensors are activated, electronic device 2400 obtains (e.g., captures) biometric data corresponding to biometric characteristics associated with the user. In some examples, the biometric feature captures biometric data using one or more biometric sensors 2403 (and/or one or more cameras) of the electronic device. Optionally, a light-emitting device, such as an IR floodlight or structured light projector, is used to help illuminate the biometric features. In other examples, an electronic device receives biometric data from another device.

In some examples, once electronic device 2400 has acquired biometric data, the electronic device processes (e.g., analyzes) the biometric data to determine whether biometric authentication was successful. In some examples, this determination includes determining whether biometric data matches a biometric template associated with the user. The biometric template is optionally stored on electronic device 2400.

In some examples, as illustrated in FIG. 24ao , biometric authentication interface 2420 within payment sheet interface 2464 may include one or more rotating rings 2470 (e.g., using the biometric authentication interface). By displaying one or more rotating rings (corresponding to 2424), it indicates that biometric data is being processed by the electronic device. In some examples, one or more rotating rings 2470 replace glyph 2468 in the biometric authentication interface.

If the electronic device 2400 determines that biometric authentication is successful (e.g., biometric data matches a biometric template associated with the user), the electronic device 2400 performs a function (e.g., payment credentials for use in a payment transaction). It transitions from a first state in which the function is disabled to a second state in which the function is enabled. As such, if biometric authentication is successful, the device displays payment credentials (e.g., associated with payment account 2472) for use in a payment transaction (e.g., to make a purchase of item 2460). It is authorized to be transmitted (to a transaction terminal or to an external server). In some examples, the electronic device further displays glyph 2474 (e.g., corresponding to glyph 2426) indicating (to the user) that biometric authentication was successful, e.g., as shown in Figure 24AP. This indicates that biometric authentication was successful. In some examples, glyph 2474 replaces one or more rotating rings 2470 within the biometric authentication interface.

In some examples, in response to determining that biometric authentication was successful, electronic device 2400 processes the payment transaction (e.g., transmits the payment credentials to an external device, such as an external server, and verifies that the credentials have been successfully received). A response indicating this is received from the external device). In some examples, as shown in Figure 24aq, electronic device 2400 may also display a processing indication (e.g., having a similar or identical pattern as one or more rings 2470) indicating (to the user) that a payment transaction is being processed. 2476 is displayed within the payment sheet interface 2464. In some examples, once processing of a transaction is complete, electronic device 2400 may replace processing indication 2476 with a completed indication 2467 (e.g., a check mark to indicate completion, as shown in FIG. 24AR). (including ), thereby indicating (to the user) that the payment transaction has been successfully completed (and item 2460 has been successfully purchased).

In some examples, to make a payment within an application or on the web (e.g., webpage 2458), the electronic device requires the user to activate a hardware button (e.g., button 2404) (e.g., a sleep/wake button). Confirm your intention to pay (by double-clicking) and then authenticate using biometric authentication (e.g., facial recognition authentication) to authorize the payment. In some examples, if the payment transaction is not completed within a predetermined time threshold (e.g., 30 seconds) activating a hardware button (e.g., 2404) (e.g., double-clicking a sleep/wake button), the device may cause the user to reactivate the hardware button (e.g., 2404). Re-confirm your intention to pay by activating a button (e.g., button 2404) (e.g., by double-clicking the sleep/wake button).

24AS illustrates an electronic device 2480 (e.g., a laptop computer) displaying a webpage 2484 (e.g., similar to webpage 2458) of a browsing application on display 2482. For example, web page 2484 is a checkout page for an item 2486 that the user wishes to purchase. In some examples, the webpage 2484 of the browsing application may include a purchase affordance to provide authorization to proceed with a purchase of an item using a device different from electronic device 2480 (e.g., using electronic device 2400). Includes (2488).

In some examples, user activation of purchase affordance 2480 on electronic device 2480 causes electronic device 2400 to display remote payment user interface 2490, as shown in FIG. 24AT. In some examples, remote payment user interface 2490 may include a (graphical) representation 2492 of a device (e.g., electronic device 2480) requesting remote authentication to proceed with a payment transaction, for use in a payment transaction. Indication 2494 of the currently selected payment account, and allowing the user to provide one or more activations (e.g., double press) of button 2404 to proceed with providing authentication (e.g., biometric authentication) for the payment transaction. and a prompt 2496 (e.g., corresponding to prompt 2416) requesting that. In some examples, as also shown in FIG. 24AT, prompt 2496 is emphasized relative to one or more other displayed objects (e.g., relative to indication 2492 of electronic device 2480). Highlighting a prompt in this manner includes, for example, darkening, blurring, and/or otherwise obscuring one or more portions of remote payment user interface 2490.

In some examples, the indication 2494 of a payment account currently selected for use in a payment transaction includes affordance 2494A. In some examples, as shown in Figure 24AU, electronic device 2400 detects user activation 2421 (e.g., a tap gesture) of affordance 2494A. In some examples, in response to detecting a user selection of affordance 2494A, electronic device 2400 may, within remote payment user interface 2490, select a device provisioned on the device (and therefor, as shown in Figure 24AV). Accordingly, display representations of one or more payment accounts (e.g., payment account 2494, payment account 2498) available for use in a payment transaction. In some examples, as also shown in Figure 24av, the representation of the currently selected payment account also includes an indication 2494B (e.g., a check mark) indicating (to the user) that it corresponds to the currently selected account.

24AW, while displaying representations of payment accounts provisioned on the electronic device (e.g., payment account 2494, payment account 2498) within remote payment user interface 2490, electronic device 2400 is currently Detect user selection 2423 of a payment account (e.g., payment account 2498) that is different from the selected payment account (e.g., payment account 2494). In response to detecting user selection 2423 of payment account 2498, electronic device 2400 removes representations of available payment accounts from remote payment user interface 2490, as shown in FIG. 24ax. , displays payment account 2498 (instead of payment account 2494) as the currently selected payment account for use in the payment transaction.

As also shown in FIG. 24ax, after replacing payment account 2494 with payment account 2498, electronic device 2400 may perform one or more activations of button 2404 corresponding to the request of prompt 2496 ( 2425) (e.g., double press). 24ay , in response to detecting one or more activations 2425 of button 2404, electronic device 2400 displays biometric authentication interface 2420 within remote payment user interface 2490. In some examples, the electronic device further displays glyph 2499 (e.g., corresponding to glyph 2422) in biometric authentication interface 2420 that is displayed within remote payment user interface 2490. Additionally, in response to the input, one or more biometric sensors of electronic device 2400, such as one or more cameras or a facial recognition sensor (e.g., included among one or more biometric sensors 2403) are activated.

In some examples, once one or more biometric sensors are activated, electronic device 2400 obtains (e.g., captures) biometric data corresponding to biometric characteristics associated with the user. In some examples, the biometric feature captures biometric data using one or more biometric sensors 2403 (and/or one or more cameras) of the electronic device. Optionally, a light-emitting device, such as an IR floodlight or structured light projector, is used to help illuminate the biometric features. In other examples, an electronic device receives biometric data from another device.

In some examples, once electronic device 2400 has acquired biometric data, the electronic device processes (e.g., analyzes) the biometric data to determine whether biometric authentication was successful. In some examples, this determination includes determining whether biometric data matches a biometric template associated with the user. The biometric template is optionally stored on electronic device 2400.

In some examples, as illustrated in FIG. 24A-Z, biometric authentication interface 2420 within remote payment user interface 2490 may include one or more rotating rings 2497 (e.g., using the biometric authentication interface). , corresponding to one or more rotating rings 2424), thereby indicating that biometric data is being processed by the electronic device. In some examples, one or more rotating rings 2497 replace glyph 2499 in the biometric authentication interface.

If the electronic device 2400 determines that biometric authentication is successful (e.g., biometric data matches a biometric template associated with the user), the electronic device 2400 performs a function (e.g., payment credentials for use in a payment transaction). It transitions from a first state in which the function is disabled to a second state in which the function is enabled. As such, if biometric authentication is successful, the device displays payment credentials (e.g., associated with payment account 2498) for use in a payment transaction (e.g., to make a purchase of item 2486). It is authorized to be transmitted (to a transaction terminal or to an external server). In some examples, the electronic device further displays glyph 2495 (e.g., corresponding to glyph 2426) indicating (to the user) that biometric authentication was successful, e.g., as shown in Figure 24B. This indicates that biometric authentication was successful. In some examples, glyph 2495 replaces one or more rotating rings 2497 in biometric authentication interface 2420 of remote payment user interface 2490.

In some examples, in response to determining that biometric authentication was successful, electronic device 2400 processes the payment transaction (e.g., transmits the payment credentials to an external device, such as an external server, and verifies that the credentials have been successfully received). A response indicating this is received from the external device). In some examples, as shown in FIG. 24BB, electronic device 2400 may also display a processing indication (e.g., having a similar or identical pattern as one or more rings 2497) indicating (to the user) that a payment transaction is being processed. 2493 is displayed within the remote payment user interface 2490. In some examples, once processing of a transaction is complete, electronic device 2400 may replace processing indication 2493 with a completed indication 2491 (e.g., a check mark to indicate completion), as shown in Figure 24BC. (including ), thereby indicating (to the user) that the payment transaction has been successfully completed (and item 2486 has been successfully purchased).

25A-25C are flow diagrams illustrating a method for performing biometric authentication using an electronic device, according to some examples. Method 2500 includes a device having a display, one or more input devices (e.g., a touchscreen, microphone, camera), and a wireless communication radio (e.g., a Bluetooth connection, a WiFi connection, a mobile broadband connection such as a 4G LTE connection). For example, 100, 300, 500, 1900). In some examples, the display is a touch-sensitive display. In some examples, the display is not a touch-sensitive display. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the device includes one or more biometric sensors, optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device, such as an IR floodlight, a structured light projector, or a combination thereof. The light-emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric feature by one or more biometric sensors. Some operations of method 2000 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 2500 provides an intuitive method for managing biometric authentication. This method reduces the user's cognitive burden for managing biometric authentication, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to manage biometric authentication faster and more efficiently saves power and increases time between battery charges.

In some examples, an electronic device (e.g., 2300, 2400) receives a request from a second device to proceed with an action, where the request includes information associated with one or more options selected at the second device (2502) . In some examples, the electronic device receives a request to proceed with the transaction and further receives information regarding details of the transaction from the second device. In some examples, an action involves a transaction. In some examples, prior to receiving user input corresponding to a request to engage in a transaction, the electronic device receives input corresponding to the transaction from a second device, wherein the input includes one or more details of the transaction, Displays one or more transaction details associated with the transaction along with a request for authorization to proceed with the transaction.

The electronic device (e.g., 2300, 2400) performs one or more activations of a button (e.g., 2304, 2404) (e.g., a hardware button) while the electronic device is in a first state in which respective functions of the device are disabled. or a double press of a button, such as a mechanical button, a double press of button 2304 or button 2404) is detected (2508). In some examples, each function is associated with a financial transaction, such as payment for goods or services. In some examples, if the feature is disabled, the device cannot participate in a transaction.

In some examples, the function of each electronic device (e.g., 2300, 2400) is participation in a transaction (2510). In some examples, participating in a transaction includes transmission of secure data from an electronic device. In some examples, the security data is financial data. In some examples, a transaction additionally or alternatively includes transmission of non-secure data. In some examples, the information that enables a device to engage in a transaction is stored in a secure element (e.g., a physically and/or logically separate memory that stores credentials in a manner that prevents them from being maliciously accessed). It is stored safely. In some examples, the electronic device is not enabled to engage in a transaction (eg, a financial transaction, such as payment for goods or services) when in the first state. In some examples, when the device is in the first state, information that enables the device to engage in a transaction is not accessible to the device outside of the secure element (e.g., payment credentials for use in a wireless payment transaction). not available for).

In some examples, each function of the electronic device (eg, 2300, 2400) is a function for providing information related to an action through the short-range communication wireless device of the electronic device (2512). In some examples, the electronic device displays a first display while in an inactive state in which the display (e.g., 2302, 2402), biometric sensor(s) (e.g., 2303, 2403), and/or one or more other components of the device are inactive. is in a state In some examples, because a respective function of the device is the ability to provide information (e.g., payment information) associated with an action (e.g., a transaction) via a near-field communication radio (e.g., NFC transmitter) of the device, the electronic device It is not configured to participate in transactions when in the first state. For example, the device may be configured to: Requests for payment information are not responded to with payment information.

In some examples, one or more activations of a button (e.g., 2304, 2404) may cause the display of the electronic device (e.g., 2300, 2400) to be turned off or to display a non-payment-related user interface (e.g., a lock screen user interface, a plurality of A cover sheet user interface containing recent notifications, a home screen user interface containing application icons for a plurality of different applications and optionally one or more widgets, or a payment application, such as an email application, a phone application, a messaging application, or a camera application. Occurs at least in part during display of a non-application (application user interface for an application) (2514).

In some examples, while the display of the electronic device (e.g., 2300, 2400) is off or displaying a user interface not related to payment, the electronic device may It is not configured to transmit payment information to a payment terminal (e.g., 2430) in response to a request for payment information (to prevent payment information from being inadvertently provided). In some examples, in at least some situations when the display of the device is off or displays a user interface not related to payment, the electronic device listens for requests for payment information, informs the user that payment information has been requested, and prompts the user to make payment. Respond to at least some requests for payment information by displaying a payment user interface (e.g., a virtual wallet) that prompts you to provide authorization to provide the information.

In some examples, one or more activations of a (hardware) button include double pressing the button (e.g., a first pressing and a second pressing of the button within a predetermined period of time) (2516). In some examples, a hardware button (eg, 2304, 2404) is located on the side of the electronic device (eg, 2300, 2400). In some examples, the hardware button is a mechanical button. In some examples, activation of a button without a second activation of the button within a predetermined period of time performs a different function (eg, turning a display of the device on or off). In some examples, different activations of a button (e.g., pressing and holding a button for different periods of time) cause different functions to occur (e.g., powering off the device or entering a user interface to invoke a virtual assistant).

In some examples, one or more activations of a (hardware) button (e.g., 2304, 2404) are detected 2518 while the first application is active on the electronic device. In some examples, detecting one or more activations of a button occurs 2520 while displaying a tutorial interface (e.g., 2410). In some examples, the electronic device (e.g., 2300, 2400) does not display a tutorial interface, but rather displays a biometric authentication interface (e.g., 2322, 2420) and performs biometric authentication. Performing biometric authentication (e.g., as opposed to other forms of authentication, such as password authentication) allows users to more quickly and easily authenticate to perform specific actions (e.g., transactions) using a device (e.g., (without any input and within a shorter period of time). Reducing the number of inputs required to perform an operation improves the operability of the device and improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). -Makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, a button (eg, 2304, 2404) has a fixed location relative to the display of the electronic device (eg, 2300, 2400) (eg, the button is not a purely software button). In some examples, the button is a hardware button, such as a mechanical button or a solid state button. In some examples, the button is a switch or any other type of toggle. In some examples, the button has a fixed position relative to the electronic device, particularly relative to the display of the electronic device, such that the electronic device can display prompts based on the position of the button.

In some examples, the button (e.g., 2304, 2404) is a mechanical button (e.g., a hardware button such as a push button). In some examples, the button is not a software button, such as a button on a touchscreen of an electronic device (eg, 2300, 2400). In some examples, the button is a solid state button. In some examples, the button is a solid state button that operates upon capacitive and/or resistive touch and/or responds to changes in the intensity of the input without having a mechanical switch pressed to activate the button, and instead Monitor whether the intensity exceeds an intensity threshold corresponding to activation of the solid-state button.

In some examples, prior to detecting 2508 one or more activations of a button (e.g., 2304, 2404), the electronic device (e.g., 2300, 2400) may display a prompt requesting that one or more activations of the button be provided (e.g., 2318 , 2416) is output (e.g., by displaying it on a display) (2504). In some examples, the electronic device prompts the user by displaying “DOUBLE CLICK FOR PAY.” In some examples, the prompt is displayed on a display (eg, 2302, 2404) of the electronic device. In some examples, the prompt is displayed adjacent to the button. In some examples, the prompts are audible and/or tactile prompts. In some examples, the prompt is displayed when the device is displaying a transaction user interface but has not received any indication that a transaction terminal is nearby and requesting transaction credentials (e.g., requesting payment information). A prompt is displayed to provide one or more activations of the button before the device is placed within the NFC field of the NFC reader). In some examples, before outputting a prompt, the electronic device displays an instructional interface that includes an affordance.

In some examples, outputting a prompt (e.g., 2318, 2416) occurs in response to selection of an affordance. In some examples, a tutorial interface (e.g., 2410) is displayed the first time a user attempts to implement a respective function without providing one or more activations of a button. In some examples, the instructional interface may be animated at a location based on the location of a button (e.g., 2304, 2404) on the device (e.g., the animation may be a user interface in a direction in which a button may be pressed at a location adjacent or proximate to the button on the device). includes movement of objects).

In some examples, outputting a prompt to the user to provide one or more activations of a button (e.g., 2318, 2416) may be compared to one or more elements displayed on the display of the electronic device (e.g., 2300, 2400). It includes emphasizing (2506). In some examples, highlighting a prompt includes blurring, dimming, and/or stopping displaying at least a portion of the display of the electronic device. In some examples, highlighting a prompt includes brightening the prompt, flashing the prompt, or otherwise drawing attention to the prompt. In some examples, emphasizing a prompt relative to one or more elements displayed on a display of an electronic device includes dimming one or more elements. In some examples, all elements displayed on the display (e.g., 2302, 2402) (except prompts) are blurred. In some examples, only elements adjacent to the prompt are blurred. In some examples, emphasizing a prompt relative to one or more elements displayed on a display of the electronic device includes dimming the display of the one or more elements. In some examples, all elements displayed on the display (except prompts) are dimmed. In some examples, only elements adjacent to the prompt are dimmed. Dimming in this manner optionally includes reducing brightness and/or darkening displayed colors.

In some examples, outputting a prompt (e.g., 2318, 2416) occurs in response to detecting an external signal of a predetermined type. In some examples, an electronic device (e.g., 2300, 2400) detects a signal, such as an NFC field, from an NFC reader, such as a payment terminal (e.g., 2430), and causes the device to transmit payment credentials to the NFC reader. Prompts the user to provide input to initiate a biometric authentication process to authorize making it available.

In some examples, outputting a prompt (e.g., 2318, 2416) includes displaying an instructional interface that includes a prompt element adjacent to a button. In some examples, the electronic device (e.g., 2300, 2400) provides one or more activations of a button (e.g., 2304, 2404) via an interface where a prompt is displayed near the location of the button and/or indicates the location of the button. Prompt the user. In some examples, the educational interface attempts to enable the user to implement a respective function of the electronic device, but does not provide one or more activations necessary to initiate biometric authentication to enable the electronic device to implement the respective function. If not, it is displayed.

In some examples, the electronic device (e.g., 2300, 2400) is configured to provide, on a display, one or more activations of a button (e.g., 2304, 2404) at a first location in a biometric authentication interface (e.g., 2322, 2420). Display a prompt. Displaying a prompt requesting that one or more activations of a button be provided provides feedback to the user regarding the current state of the device and provides visual feedback indicating what steps the user must take to proceed with a particular function using the device. is provided to the user. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the electronic device (e.g., 2300, 2400) detects activation (e.g., selection) of an affordance of a first application (e.g., activation of an affordance of an application requesting a transaction for a good or service). In some examples, the first application is a communications application. In some examples, the device displays details of the transaction, and one or more of an affordance to initiate the transaction and a prompt to trigger biometric authentication. In some examples, the details of the transaction are optionally modified after biometric authentication is triggered via user input (eg, a double tap) but prior to detection of a biometric feature or prior to activation of an affordance.

In some examples, in response to detecting activation of an affordance of a first application (2522), the electronic device (e.g., 2300, 2400) provides information associated with an action from the first application to the second application (e.g., send). In some examples, an action from a first application to a second application involves a transaction. In some examples, prior to receiving user input corresponding to a request to engage in a transaction, the electronic device detects activation of an affordance of the first application and, in response to detecting activation of the affordance of the first application, performs the first Provide information about the transaction from the application to the second application, and proceed with the transaction using the second application.

In some examples, the electronic device may also display transaction information (e.g., separate from the first application and not shared with the first application, such as a credit card number, billing address) on the display (e.g., 2302, 2402). (in a payment user interface comprising) simultaneously displaying at least a portion of information associated with an action at a first location, and a second prompt to provide one or more activations of a button at a second location, wherein the second location is at the first location. It's closer to a button.

In some examples, prior to receiving activation of a button (e.g., 2304, 2404), the details of the transaction are confined to a specific portion of the display such that a prompt to provide activation of the button may be displayed proximate the button. Confining transaction details to a specific portion of the display so that the prompt can be displayed proximate the relevant button provides visual feedback to the user that allows the user to follow the prompt's request more quickly and easily. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, details are limited to a certain height of the display. In some examples, if there is more information than can be displayed in the available area below the second prompt, the information associated with the action may display additional information that was hidden (e.g., from the screen) before the scrollable area was scrolled. It is displayed in a scrollable area that scrolls in response to user inputs for display. In some examples, providing information regarding a transaction from a first application to a second application includes providing information that can be displayed in an available area and additional information that has been hidden.

In some examples, after displaying a prompt (e.g., 2318, 2416), the electronic device (e.g., 2300, 2400) moves a representation of the user credentials from a second location on the display (e.g., 2302, 2402) to a first location on the display. Move to location. In some examples, the representation of the user credential may include the user credential covering a prompt (e.g., a double click) to press a button (e.g., 2304, 2404) and/or a biometric authentication glyph (e.g., 2324, 2422). moved to reveal In some examples, moving the representation of the user credential from the second location to the first location includes displaying the biometric authentication glyph on a portion of the display that was occupied by the user credential when the user credential was displayed at the second location. It includes doing.

In response to detecting (2522) one or more activations of the button, the electronic device (e.g., 2300, 2400) may detect one or more biometric sensors (e.g., 2303, 2403) that are separate from the button (e.g., 2304, 2404). ) to capture biometric data (2524). In some examples, in response to a double press of a hardware button, the device receives biometric data, such as data about the user's face. In some examples, the one or more biometric sensors include a facial recognition sensor, and the biometric data corresponds to at least a portion of the face.

In some examples, capturing biometric data includes capturing the biometric data using a camera. In some examples, biometric data is captured using a camera and/or facial recognition sensor (eg, 2303, 2403). In some examples, a camera is used to ensure that the user is looking at the device, and a facial recognition sensor is used to authenticate the user's face.

In some examples, capturing biometric data with one or more biometric sensors includes activating one or more biometric sensors (e.g., 2303, 2403) for a second predetermined period of time. For example, in response to pressing a button (e.g., 2304, 2404), the electronic device (e.g., 2300, 2400) may display one or more biometric sensors (e.g., 2303, 2403), such as a facial recognition sensor or a camera. Activate (e.g., transition the biometric sensors from an inactive state to an active state) and capture biometric data using the activated one or more biometric sensors. In some examples, one or more biometric sensors are activated for a period of time, and if biometric data has not been captured for a period of time, the biometric authentication process fails. In some examples, the second predetermined period begins when one or more activations of the button are detected. In some examples, a period of time begins when a button is pressed. In some examples, capturing biometric data in this manner includes illuminating a biometric feature and capturing data corresponding to the illumination. In some examples, biometric features are illuminated using IR flood illumination or a structured light projector.

In some examples, in response to detecting (2522) one or more activations of a button, the electronic device (e.g., 2300, 2400) may also display a biometric device containing a representation of user credentials that are restricted from use without appropriate biometric authentication. Displays (2526) a measurement authentication interface (e.g., 2322, 2420). In some examples, the biometric authentication interface may include authentication glyphs (e.g., 2324, 2422) and/or one or more representations of user credentials (e.g., an image of a credit card, an image of a bank account, an image of a business card) and images representing data to be used by the functions of the electronic device. Providing authentication glyphs provides the user with easily visible and recognizable visual feedback regarding the current status or progress of the authentication process. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, a function performs a transaction using the represented data. In some examples, biometric authentication is performed on a specific credential, and that credential is highlighted relative to other credentials. In some examples, the biometric authentication interface (e.g., 2322, 2420) includes an animation (e.g., a card carousel).

Upon determining that the biometric data satisfies the biometric authentication criteria (e.g., the biometric criteria match a biometric template stored on the device), the electronic device (e.g., 2300, 2400) determines the device's respective functionality. Transition to the enabled second state (2528). In some examples, if the device is enabled, the device can participate in a transaction. In some examples, the electronic device determines whether biometric data meets biometric authentication criteria. Limiting a device's ability to engage in transactions to cases where the device is enabled helps users avoid executing transactions unintentionally, while also providing improved device security, allowing users to Gives you more control of your device. Providing additional control of a device without cluttering the UI with additional displayed controls (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device) It improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the electronic device (eg, 2300, 2400) is enabled to engage in a transaction when in the second state. In some examples, when the device is in the second state, information that enables the device to engage in a transaction is made temporarily accessible by the secure element to the device external to the secure element (e.g., payment credentials available for use in wireless payment transactions).

In some examples, after transitioning to the second state, the electronic device (e.g., 2300, 2400) maintains the device in the second state for a first predetermined period of time (e.g., 60 seconds) (2530). In some examples, functionality of the electronic device is enabled even when user credentials change (eg, as a result of a card switch). In some examples, the first predetermined period begins when one or more activations of a button are detected (eg, when a button is pressed). In some examples, after transitioning to the second state, the electronic device transitions from the second state to the first state after a first predetermined period of time has elapsed (2532). In some examples, after transitioning to the second state, the electronic device performs an action based on the information (e.g., sending credentials to a remote server to process a transaction) using respective functionality of the electronic device. (2534). In some examples, the electronic device uses credentials provided in the request to proceed with the transaction. In some examples, credentials are provided to a remote server for processing the transaction. In some examples, the electronic device causes the display of an indication at the other device indicating whether authentication was successful. In some examples, the electronic device causes display of an indication at another device (eg, another device or a second device) indicating whether the transaction was successful. In some examples, a prompt is output while details of the transaction are displayed.

In some examples, while the device is in the second state, the electronic device (e.g., 2300, 2400) detects user input corresponding to a request to terminate the first application (2536). In some examples, in response to detecting a user input corresponding to a request to terminate the first application, the electronic device terminates the first application and transitions to the first state (2538). In some examples, when a device is enabled to engage in a transaction, terminating the application whose input was received would have enabled the device to engage in the transaction causes the device to stop being enabled to engage in the transaction. . In some examples, when a device is authorized to provide payment credentials for a payment transaction in a wallet application and the device switches from the wallet application to a different application, the device may be required to provide payment credentials (e.g., using biometric authentication). ) disables the ability to provide payment credentials until re-authorized by the user. Accordingly, unintentional transmission of payment information is prevented by disabling transmission of payment information when the device is not displaying a user interface indicating that the device is configured to provide payment credentials.

Upon determining that the biometric data does not meet biometric authentication criteria (e.g., the biometric data does not match a biometric template stored on the device), the electronic device (e.g., 2300, 2400) maintains the first state. (2540), and an indication that biometric authentication has failed is displayed on the display (2540). Maintaining a first state (eg, the device is not authorized to proceed with the transaction) when authentication fails provides the user with improved control and security of the device. Providing additional control and improved security of the device without cluttering the UI with additional displayed controls (e.g., helping the user provide appropriate inputs when operating/interacting with the device and preventing user mistakes) (by reducing the device's operability and making the user-device interface more efficient), which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently .

In some examples, the electronic device (e.g., 2300, 2400) also displays (2542) alternative authentication affordances (e.g., 2334, 2440). In some examples, alternative authentication affordances, when selected, cause an electronic device to provide an alternative form of authentication (e.g., a non-biometric form of authentication), such as allowing a user to enter a password, passcode, or pattern. It is an affordance that displays an interface (e.g., 2336, 2442) that can be used. In some examples, successful authentication via alternative authentication causes the electronic device to transition to a second state. In some examples, a first failure results in display of a “TRY AGAIN” affordance and a second failure results in display of an alternative authentication affordance (e.g., 2336, 2442), such as a “passcode” affordance. do. In some examples, the electronic device receives user input, such as a double press of a button. In some examples, if a threshold number of biometric authentication attempts has been reached, the electronic device may provide an affordance for entering a passcode (or password) and, optionally, a notification that biometric authentication is not available and/or a threshold number has been reached. Displays an indication (e.g., “Passcode required to enable FaceID”).

In some examples, after determining that biometric data does not meet biometric authentication criteria (e.g., while displaying an alternative authentication interface or alternative authentication affordance), the electronic device (e.g., 2300, 2400) may perform alternative authentication. A selection of affordances (e.g., 2334, 2440) is detected (2544). In some examples, in response to detecting the selection of an alternative authentication affordance, the electronic device (e.g., 2300, 2400) displays, on the display, an alternative authentication interface, such as a password or passcode interface, in response to the selection of the affordance. (e.g., 2336, 2442) is displayed (2546).

In some examples, the electronic device (e.g., 2300, 2400) also captures second biometric data with one or more biometric sensors (e.g., 2303, 2403) (2548). In some examples, upon determining that the second biometric data satisfies biometric authentication criteria, the electronic device transitions to a second state (2550). In some examples, in response to selection of the affordance, the electronic device performs a second iteration of biometric authentication.

In some examples, biometric authentication is performed during display of or transition to an alternative authentication interface (e.g., 2336, 2442). In some examples, an alternative authentication interface includes display of a biometric authentication glyph sequence such that a user is notified that biometric authentication is occurring. In some instances, successful biometric authentication avoids the need for alternative authentication. As a result, the electronic device (e.g., 2300, 2400) stops displaying the alternative authentication interface (e.g., 2336, 2442) and proceeds as if the user was successfully authenticated on the first attempt.

In some examples, upon determining that the second biometric data does not meet biometric authentication criteria, the electronic device (e.g., 2300, 2400) maintains the first state and displays an alternative display on the display (e.g., 2302, 2402). Display 2552 an authentication interface (e.g., 2336, 2442). In some examples, in the event of a failure, display of an alternative authentication interface is maintained to allow the user to optionally provide alternative authentication. Maintaining the display of alternative authentication interfaces so that the user can provide alternative authentication (in case of failure) provides the user with multiple different ways to provide authentication (for specific operations performed by the device). This gives users more control over their devices. Providing additional control of a device without cluttering the UI with additional displayed controls (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) It improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the electronic device (e.g., 2300, 2400) detects 2554 respective user input corresponding to a request to retry biometric authentication. In some examples, the electronic device detects user input, such as pressing a button or moving the electronic device (e.g., raising and/or lowering the device) or selecting an alternative authentication affordance (e.g., 2334, 2440). In some examples, user input corresponding to a request to retry biometric authentication includes one or more activations of a button. In some examples, user input includes one or more activations of a button used to initiate the first iteration of biometric authentication. In some examples, user input corresponding to a request to retry biometric authentication includes movement of the electronic device. In some examples, the predetermined type of user input is input that is different from activation of a button. In some examples, the predetermined type of user input is raising and/or lowering the electronic device (e.g., the electronic device is lowered near another electronic device, such as an NFC-compatible device, and raised again to the user's eye level) ).

In some examples, in response to detecting user input corresponding to a request to retry biometric authentication, the electronic device (e.g., 2300, 2400) captures third biometric data with one or more biometric sensors ( 2556). In some examples, in response to user input, the device performs additional repeat biometric authentication. In some examples, upon determining that the third biometric data satisfies biometric authentication criteria, the electronic device transitions to a second state in which a respective function of the device is enabled (2558). In some examples, upon determining that the third biometric data does not meet the biometric authentication criteria, the electronic device maintains the first state (and, optionally, displays on the display an indication that biometric authentication has failed). (2560).

In some examples, the electronic device (e.g., 2300, 2400) detects one or more other activations of a button (e.g., 2304, 2404). In some examples, upon determining that the biometric capture criteria are met, the electronic device captures second biometric data with one or more biometric sensors (e.g., 2303, 2403) separate from the button (e.g., 2304, 2404). Capture. In some examples, upon determining that biometric capture criteria are not met, the electronic device withholds capturing the second biometric data. In some examples, the number of biometric authentication attempts is limited to a predetermined number (eg, 5). In some examples, this number is reset in response to successful authentication. In some examples, this number is reset after a set amount of time.

Note that the details of the processes described above with respect to method 1200 (e.g., FIGS. 25A-25C) are also applicable in a similar manner to the methods described herein. For example, method 2500 optionally includes one or more of the features of the various methods described herein with reference to methods 800, 1000, 1200, 1400, 1600, 1800, 2000, 2200, 2700. Includes. For example, the registered biometric data described in method 1200 may be used to perform biometric authentication as described with respect to FIGS. 24F and 24G. For another example, one or more implantable interfaces as described in methods 2000 and 2700 are optionally displayed in response to receipt of input prior to completion of the biometric authentication process. For brevity, these details are not repeated herein.

Operations in the information processing methods described above optionally utilize one or more functional modules within an information processing device, such as general-purpose processors (e.g., as described with respect to FIGS. 1A, 3, and 5A) or application-specific chips. It is implemented by running it. Additionally, the operations described above with reference to FIGS. 25A-25C are optionally implemented by the components shown in FIGS. 1A and 1B. For example, detection operation 2508, transition operation 2528, and sustain operation 2540 are optionally implemented by event sorter 170, event recognizer 180, and event handler 190. Event monitor 171 in event sorter 170 detects contact on touch-sensitive surface 604, and event dispatcher module 174 passes event information to application 136-1. The respective event recognizer 180 of application 136-1 compares the event information to the respective event definition 186 and determines that the first contact at the first location on the touch-sensitive surface corresponds to the selection of an object on the user interface. Determines whether it corresponds to the same predefined event or subevent. When an individual predefined event or sub-event is detected, event recognizer 180 activates an event handler 190 associated with detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some examples, event handler 190 accesses the respective GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes may be implemented based on the components shown in FIGS. 1A and 1B.

26A-26AS illustrate example user interfaces for biometric authentication, according to some examples. As described in more detail below, the illustrative examples of user interfaces illustrated in Figures 26A-26As are used to illustrate the processes described below, including the processes in Figures 27A-27E.

FIG. 26A illustrates electronic device 2600 (e.g., portable multifunction device 100, device 300, or device 500). In the illustrative example illustrated in FIGS. 26A-26AS , electronic device 1900 is a smartphone. In other examples, electronic device 1500 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 1900 has a display 2602, one or more input devices (e.g., a touchscreen of display 2602, buttons 2604, a microphone (not displayed)), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the electronic device includes one or more biometric sensors (e.g., biometric sensor 2603), optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, one or more biometric sensors 2603 are one or more biometric sensors 703. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In Figure 26A, the electronic device displays an unlocked interface 2606 while in an unlocked state. Unlocked interface 2606 includes notification affordance 2608 and unlocked status indicator 2610. In some examples, notification affordance 2608 includes an indication of security content associated with notification affordance 2608 because electronic device 2600 is in an unlocked state. For example, as shown, a notification affordance is associated with a messaging application and includes at least a portion of a message received by the electronic device.

26B-26D , while displaying unlocked interface 2606, electronic device 2100 detects user input 2612 (FIG. 26B), for example near an edge of display 2602. do. User input 2612 is, in some examples, a swipe gesture that is a request to access the home screen interface of electronic device 2600, and in response to swipe input 2612, the electronic device displays the home screen interface of FIG. 26D ( 2614) (e.g., replaces the display of the unlocked interface 2606 with it). In some examples, displaying home screen interface 2129 may include sliding unlocked interface 2606 in an upward direction to display home screen interface 2614, as similarly described with reference to FIGS. 19P-19R. Includes displaying (e.g., revealing).

In Figure 26E, the electronic device displays an unlocked interface 2606 while in an unlocked state. Unlocked interface 2606 includes notification affordance 2608 and unlocked status indicator 2610. In some examples, notification affordance 2608 includes an indication of security content associated with notification affordance 2608 because electronic device 2600 is in an unlocked state. For example, as shown, a notification affordance is associated with a messaging application and includes at least a portion of a message received by the electronic device.

While displaying unlocked interface 2606, the electronic device detects activation of notification affordance 2608. Activation of notification affordance 2608 is a tap gesture 2615 in some examples. In response to activation of notification affordance 2608, the electronic device displays messaging application interface 2616 of Figure 26G (e.g., replacing the display of unlocked interface 2606 therewith). Referring to FIGS. 21F and 21G , in some examples, displaying messaging application interface 2616 may include viewing unlocked interface 2606 in an upward direction, as similarly described with reference to FIGS. 19P-19R . and sliding to display (e.g., reveal) the messaging application interface 2616.

In Figure 26H, the electronic device displays a locking interface 2620 while in a locked state. Lock interface 2620 includes notification affordance 2622 and lock status indicator 2624. In some examples, because the electronic device 2600 is in a locked state, notification affordance 2622 does not include an indication of secure content associated with notification affordance 2622.

26I-26K, while displaying the locking interface 2620, the electronic device 2600 detects user input 2628 (FIG. 26I), for example near an edge of the display 2602. User input 2628 is, in some examples, a swipe gesture that is a request to access the home screen interface of electronic device 2600. In some examples, electronic device 2600 may request user input 2628 before completing initial biometric authentication (e.g., biometric authentication performed in response to a wake condition, as described with reference to FIGS. 21A-21C). receives. Accordingly, in response to swipe input 2628, the electronic device displays insertable interface 2630 of Figure 26K to indicate that the electronic device has not yet completed biometric authentication (e.g., locked interface 2620). replace the display with it). In some examples, displaying insertable interface 2630 may include sliding locking interface 2620 in an upward direction to display insertable interface 2630, as similarly described with reference to FIGS. 19P-19R (e.g., , revealing). Embedded interface 2630 includes a locked status indicator 2624 in some examples.

Alternatively, in some examples, the electronic device determines that a threshold number (eg, 5) of biometric authentication attempts has been reached. Thereafter, in response to user input 2628, electronic device 2600 displays insertable interface 2632. The embeddable interface includes a biometric authentication enabled indicator indicating that biometric authentication is disabled (e.g., because the number of attempts has been reached). Embedded interface 2632 further includes alternative authentication affordances 2636 and 2638. Activation of alternative authentication affordances 2636 causes the electronic device to display a first alternative authentication interface, such as a fingerprint authentication interface, and activation of alternative authentication affordances 2638 causes the electronic device to display a first alternative authentication interface, such as a fingerprint authentication interface. Display a second alternative authentication interface such as .

In some examples, while displaying insertable interface 2630, the electronic device detects biometric data (e.g., facial biometric data) and, in response, performs biometric authentication. Referring to FIG. 26M, electronic device 2600 displays biometric progress indicator 2625 to indicate that biometric data is being processed.

In Figure 26N, electronic device 2600 determines that biometric authentication was successful. In response, electronic device 2600 displays an unlocked status indicator 2626 and, optionally, outputs tactile output 2640. After indicating that biometric authentication was successful (e.g., after a predetermined period of time), the electronic device displays home screen interface 2614 of FIG. 26P (e.g., replacing the display of insertable interface 2630 therewith). . 21O and 21P, in some examples, displaying home screen interface 2614 includes sliding home screen interface 2614, as similarly described with reference to FIGS. 19P-19R. do.

Alternatively, in Figure 26Q, electronic device 2600 determines that biometric authentication was not successful. In response, electronic device 2600 alternates the position of locked status indicator 2627 to simulate a “shake” effect. Electronic device 2600 further outputs tactile output 2644 to indicate that biometric authentication was not successful. In some examples, tactile output 2644 is the same as tactile output 2640. In some examples, tactile output 2644 is different from tactile output 2640. In some examples, in response to determining that biometric authentication was not successful, the electronic device displays alternative authentication affordances 2642.

Referring to Figure 26R, the electronic device receives activation of the locked status indicator 2624. In some examples, activation of the locked status indicator is a tap gesture 2650 on the locked status indicator 2624. In response, as shown in Figure 26S, electronic device 2600 initiates biometric authentication. In some examples, initiating biometric authentication includes obtaining (e.g., capturing with one or more biometric sensors 2603) data corresponding to at least a portion of the biometric characteristic, and processing the biometric data. Thus, determining whether the biometric characteristic (or portion thereof) satisfies biometric authentication criteria (e.g., determining whether the biometric data matches, within a threshold, a biometric template). do. While processing biometric data, the electronic device displays a biometric progress indicator 2625 (e.g., a locked status indicator 2624), indicating that the electronic device 2600 is processing biometric data. replace the display with it). If the electronic device 2600 determines that the biometric authentication was successful, the electronic device indicates success, as described with respect to FIGS. 26N-26P.

26T, electronic device 2600 determines that biometric authentication (e.g., as described with reference to FIG. 26S) was not successful and, in response, adjusts the position of the locked status indicator to simulate a “shake” effect. alternating, outputting tactile output 2652, and displaying alternative authentication affordances 2642.

26U, the electronic device detects activation of alternative authentication affordance 2642. In some examples, activation of the alternative authentication affordance is a tap gesture 2654 on the alternative authentication affordance 2642. 26V, in response to activation of alternative authentication affordance 2642, the electronic device displays (e.g., replaces the display of insertable interface 2630 therewith) an alternative authentication interface 2656, which The user authenticates with the electronic device when entering a valid passcode (or password).

Referring to Figures 26W-26Y, in some examples, the electronic device displays one or more interfaces and/or enters a low power state if the electronic device fails to detect a biometric characteristic for a predetermined period of time. 26W, the electronic device displays an implantable interface 2630 (recall that the electronic device displays the implantable interface 2630 in response to a request for secure content received prior to completion of biometric authentication). If, for a predetermined period of time, the electronic device 2600 does not detect a biometric characteristic, the electronic device displays an alternative authentication interface 2657 (e.g., replaces the display of insertable interface 2630 therewith). In some examples, alternative authentication interface 2657 includes an indicator directing the user to provide alternative authentication, such as a passcode. In other examples, as shown in Figure 26X, alternative authentication interface 2657 does not include an indicator directing that alternative authentication be provided to the user.

During display of alternative authentication interface 2657, if biometric characteristics are not detected for a predetermined period of time and alternative authentication is not provided, electronic device 2600 is placed in a low power state (as shown in FIG. 26Y). For example, transition to a display-disabled state).

During display of alternative authentication interface 2657, if biometric characteristics are detected, electronic device 2600 performs biometric authentication, as described. As shown in FIG. 26Z, the electronic device displays a biometric progress indicator 2625 to indicate that the electronic device is processing biometric data (e.g., displays a locked status indicator 2624 thereon). (replaced with). 26AA, electronic device 2600 determines that biometric authentication was successful. In response, the electronic device displays an unlocked status indicator 2610 (replacing the display of biometric progress indicator 2625 with it) and, optionally, provides a tactile signal to indicate that biometric authentication was successful. Outputs red output (2658). In some examples, electronic device 2600 subsequently displays home screen interface 2614, as shown in Figure 26ab.

26ac-26ae, during the display of alternative authentication interface 2657, if biometric authentication fails and at least a portion of the alternative authentication is provided, electronic device 2600 does not provide tactile output. Otherwise, biometric authentication will not be successful. As shown in Figure 26ac, the electronic device may, while performing biometric authentication (as indicated by biometric progress indicator 2625), use user input (e.g., a tap gesture) 2660 to provide alternative Receive a portion of the authentication (e.g., passcode). 26A, the electronic device determines that biometric authentication was not successful and, in response, displays a locked status indicator 2627 and further alternates the position of the locked status indicator to simulate a “shake” effect. . In some examples, electronic device 2600 does not output tactile output and further maintains a display of alternative authentication interface 2657, as shown in Figure 26ae.

In Figure 26AF, electronic device 2600 displays a locking interface 2620 while in a locked state. As described, locking interface 2620 includes notification affordance 2622 and locking status indicator 2624. In some examples, the electronic device receives a request for secure content (e.g., a message associated with notification affordance 2622) on the electronic device. Electronic device 2600 detects activation of notification affordance 2622, for example. In some examples, activation of notification affordance 2622 is a tap gesture 2662.

In some examples, activation of notification affordance 2622 is received prior to completion of biometric authentication. Accordingly, as shown in Figure 26ag, in response to activation of notification affordance 2622, electronic device 2600 displays insertable interface 2629 that includes biometric indicator 2666. Biometric indicator 2666, in some examples, identifies secure content associated with a received request for secure content.

As shown in FIG. 26Ah, if the electronic device 2600 does not detect a biometric characteristic while displaying the insertable interface 2629, the electronic device displays an alternative authentication affordance 2668. In some examples, activation of alternative authentication affordance 2668 causes the electronic device to display an alternative authentication interface (e.g., alternative authentication interface 2657 in FIG. 26X).

During the display of the alternative authentication interface, if the biometric characteristic is not detected for a predetermined period of time and no alternative authentication is provided, the electronic device 2600 enters a low power state (e.g., displays -transitions to a disabled state).

As described in relation to Figure 26Ah, the electronic device displays an alternative authentication affordance 2668 if the biometric characteristic is not detected. In some examples, a biometric characteristic is detected following the display of alternative authentication affordances 2668, and in response, the electronic device performs biometric authentication, as described above. As shown in FIG. 26AJ , electronic device 2600 displays biometric progress indicator 2625 to indicate that biometric data is being processed. In Figure 26AK, electronic device 2600 determines that biometric authentication was successful. In response, electronic device 2600 displays an unlocked status indicator 2610 and, optionally, provides tactile output 2670 to indicate that biometric authentication was successful. In some examples, electronic device 2600 subsequently displays messaging application interface 2616, as shown in FIG. 26AM. 26A and 26AM, in some examples, displaying messaging application interface 2616 can be accomplished by sliding insertable interface 2629 in an upward direction, as similarly described with reference to FIGS. 19P-19R. and displaying (e.g., revealing) a messaging application interface 2616.

In Figure 26A, electronic device 2600 displays an insertable interface 2629 with alternative authentication affordances 2668. While displaying insertable interface 2629, the electronic device detects activation of alternative authentication affordance 2668. In some examples, activation of alternative authentication affordance 2668 is a tap gesture 2674 on alternative authentication affordance 2668.

Referring to Figure 26ao, in response to activation of alternative authentication affordance 2668, electronic device 2600 displays alternative authentication interface 2631. In some examples, alternative authentication interface 2631 identifies the requested secure content (“ENTER PASSCODE FOR MESSAGES”).

26AP and 26aq, in response, at least in part, to a tap gesture 2076 (FIG. 26AP) and, optionally, one or more other inputs representing additional alphanumeric digits of a valid passcode, electronic device 2600 A valid passcode (or password) is received by. 26AR and 26AS, once a valid passcode has been received, the electronic device unlocks and displays messaging application interface 2616 (e.g., display of alternative authentication interface 2631 therewith). replace). In some examples, displaying messaging application interface 2616 may include sliding alternative authentication interface 2631 in an upward direction to display messaging application interface 2616, as similarly described with reference to FIGS. 19P-19R. Includes displaying (e.g., revealing).

27A-27E are flow diagrams illustrating a method for performing biometric authentication using an electronic device, according to some examples. Method 2700 includes a device having a display, one or more input devices (e.g., a touchscreen, microphone, camera), and a wireless communication radio (e.g., a Bluetooth connection, a WiFi connection, a mobile broadband connection such as a 4G LTE connection). For example, 100, 300, 500, 2600). In some examples, the display is a touch-sensitive display. In some examples, the display is not a touch-sensitive display. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In some examples, the device includes one or more biometric sensors, optionally including a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, the device further includes a light emitting device, such as an IR floodlight, a structured light projector, or a combination thereof. The light-emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric feature by one or more biometric sensors. Some operations of method 2700 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 2700 provides an intuitive method for performing biometric authentication. This method reduces the user's cognitive burden of authenticating biometric features, thereby creating a more efficient human-machine interface and intuitive user experience. For battery-operated computing devices, allowing users to perform authentication of biometric features faster and more efficiently saves power and increases time between battery charges.

In some examples, while the electronic device (e.g., 2700) is in a locked state in which the device is not authorized to perform the respective operation, the electronic device displays a first graphical indication (e.g., 2624) indicating that the device is in the locked state. (e.g., a closed lock icon). Displaying a first graphical indication that the device is in a locked state provides the user with a readily available indication of the status of the device. The user is then informed about what features of the device are enabled and/or available, thereby helping the user provide appropriate inputs and reducing user errors when operating and/or interacting with the device. ) makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, while the device is in an unlocked state in which the device is authorized to perform the respective operation, the electronic device displays a second graphical display (e.g., a second graphical display (e.g., , 2610) (e.g., an open lock icon). Displaying a second graphical indication that the device is in an unlocked state provides the user with a readily available indication of the status of the device. The user is then informed about what features of the device are enabled and/or available, thereby helping the user provide appropriate inputs and reducing user errors when operating and/or interacting with the device. ) makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, the first graphical representation and the second graphical representation are displayed at respective locations in the user interface (e.g., 2606, 2620) (e.g., near substantially the top center of display 2602).

In some examples, the electronic device detects a request to perform a respective operation requiring authentication (2702). In some examples, the request to perform each action requiring authentication may include a request (e.g., 2612) to display a home screen (e.g., 2614) with a plurality of application open icons that, when selected, open corresponding applications; or a request to display an application user interface corresponding to the selected notification. In some examples, the request to perform each action includes a home input (e.g., 2612). In some examples, the home input is a home gesture, such as a button on the display, or a selection of a home button, such as an upward swipe from each edge of the display. In some examples, the request to perform each action includes selection (e.g., 2615) of a notification (e.g., 2608). In some examples, selection of a notification is a tap, long press, hard press, or swipe on the notification user interface object. In some examples, each action includes displaying a home screen that includes a plurality of application icons for opening different applications. In some examples, the plurality of application icons for opening different applications are application icons that, when selected, cause a corresponding application to open. In some examples, the home screen also includes one or more widgets, system status indicators, device controls, etc. In some examples, each operation includes displaying an application user interface (e.g., 2616) for the application corresponding to the notification. In some examples, the application interface includes information specifically related to the notification (e.g., the notification is a notification of an electronic communication and the application user interface includes a representation of the notification or electronic communication).

In response to detecting a request to perform a respective operation requiring authentication (2704), and in accordance with a determination that the device is unlocked, the electronic device performs the respective operation (2706). Additionally, upon determining that the device is locked and that a first form of authentication is available for use (2708), the electronic device displays, on a display (e.g., 2602), one or more images for using the second form of authentication. Display 2712 an authentication indicator (e.g., 2625) for the first form of authentication without displaying affordances (e.g., 2636, 2638). Displaying an authentication indicator without displaying affordances for using the second form of authentication provides the user with an intuitive interface that withholds the device from providing additional options while biometric authentication is performed. Providing an intuitive interface in this way improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and (e.g., by This makes the user-device interface more efficient (avoiding the user attempting alternative authentication prior to completion of measurement authentication), which additionally reduces the device's power usage by allowing the user to use the device more quickly and efficiently. Reduces battery life and improves battery life.

In some examples, the authentication indicator may be a visual indication that a first form of authentication is being attempted by the device, such as text describing the first form of authentication (e.g., a password, passcode, fingerprint, or other form of authentication) or These are graphic elements. In some examples, the first form of authentication is a form of biometric authentication based on data acquired by one or more biometric sensors (e.g., a non-contact form of biometric authentication such as facial recognition or iris recognition) (2710). In some examples, the authentication indicator includes information indicating that the device is attempting to use a first form of authentication (2714). In some examples, the authentication indicator includes a graphic or text description indicating that facial biometric authentication is available, such as “Face ID” or “Face ID to open Messages.” In some examples, an authentication indicator is displayed along with an option to revoke authentication. In some examples, the authentication indicator is displayed with an option to display emergency information (eg, emergency call user interface and/or emergency medical information) without unlocking the device. In some examples, the authentication indicator includes information about the progress of the attempt in the first form of authentication (2716), such as a progress indicator as described in more detail with respect to FIGS. 11A-11O.

In some examples, while displaying an authentication indicator for a first form of authentication without displaying affordances for using a second form of authentication, the electronic device may Process the data (2718). In some examples, each of the data from one or more biometric sensors is processed while displaying a biometric authentication indicator for a first form of biometric authentication without displaying one or more affordances for using a second form of authentication. At least a portion of the data was acquired by one or more biometric sensors prior to displaying an authentication indicator for the first form of authentication (2720). In some examples, each of the biometric sensors from one or more biometric sensors is processed while displaying a biometric authentication indicator for a first form of biometric authentication without displaying one or more affordances for using a second form of authentication. At least a portion of the data was obtained by one or more biometric sensors after displaying an authentication indicator for the first form of authentication (2722).

In some examples, after processing 2724 each data from one or more biometric sensors, a determination is made that each data from one or more biometric sensors matches the biometric information authorized to perform the respective operation. Accordingly (e.g., the device detects an authorized face in each biometric data), the electronic device performs each operation (2726). Performing each action in response to determining that the respective data from one or more biometric sensors matches the biometric information authorized to perform the respective action may, in some instances, require the user to manually authenticate. It improves the operability of the device by allowing authentication with the device without requiring authentication, thereby making the user-device interface more efficient.

In some examples, further, after processing each data from one or more biometric sensors, upon determining that each data does not match the biometric information authorized to perform the respective operation (e.g., the device may (not detecting any face or detecting a face that does not match the authorized faces) 2728, the electronic device is configured to display a second form of data that was not displayed prior to processing the respective data from the one or more biometric sensors. Display 2730 one or more affordances (e.g., 2636, 2638) for using authentication. Displaying one or more affordances for using a second form of authentication that was not displayed prior to processing the respective data from the one or more biometric sensors may be used to enable the device to request successful authentication when the biometric data is not successful. Provides the user with a quick alternative way to access operations (eg, locked operations). Providing additional control options with additional displayed controls improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and allows the user to -Makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, one or more affordances include displaying a “Use Passcode” button, a “Use Password” button, or a keypad/keyboard for entering a passcode/password. In some examples, the one or more affordances for using the second form of authentication may be displayed after each delay in which the authentication indicator for the first form of authentication is displayed without displaying the one or more affordances for using the second form of authentication. is displayed in

In some examples, displaying one or more affordances may be performed after a first period of time has elapsed (e.g., an authentication indication) upon a determination that a biometric characteristic corresponding to a first form of authentication is detected by the one or more biometric sensors. displaying one or more affordances for using a second form of authentication (after the self is displayed); and a second form of authentication after a second period of time has elapsed (e.g., since the authentication indicator has been displayed) in accordance with a determination that no biometric characteristic corresponding to the first form of authentication is detected by the one or more biometric sensors. and displaying one or more affordances for using authentication. In some examples, the second period is different (eg, longer or shorter) than the first period.

In some examples, displaying one or more affordances may be used to authenticate using a second form of authentication, depending on a determination that a biometric characteristic that could be used in a first form of authentication but does not match authorized biometric characteristics has been detected. Displaying a user interface (e.g., 2656) for a second form of authentication (e.g., a sequence of characters for authentication (e.g., a password or passcode)) simultaneously with respective instructions to provide one or more inputs to displaying a plurality of character input keys (e.g., a keyboard or keypad) for input (e.g., displaying commands to input a sequence of characters for authentication using one or more character input keys (e.g., a passcode) The keypad is displayed with the commands “Enter passcode to unlock”)); and respective instructions to provide one or more inputs to authenticate using the second form of authentication in response to a determination that no biometric characteristic corresponding to the first form of authentication has been detected by the one or more biometric sensors. displaying a user interface (e.g., 2657) for a second form of authentication (e.g., a plurality of character input keys (e.g., a password or passcode) for entering a sequence of characters for authentication (e.g., a password or passcode) including displaying a keyboard or keypad). In some examples, the user interface for the second form of authentication is displayed without displaying respective instructions to enter a sequence of characters for authentication using one or more character input keys (e.g., a passcode keypad may display a "lock lock" keypad). (displayed without “Enter passcode to unlock” commands). In some examples, the plurality of character input keys initially (e.g., while the device is attempting to use a first form of authentication) each command to enter a sequence of characters for authentication using one or more character input keys. and then, when using the first form of authentication fails, the device displays explicit instructions to enter a sequence of characters for authentication using one or more character input keys.

In some examples, displaying one or more affordances may include entering a plurality of characters for entering a sequence of characters for authentication (e.g., a password or passcode), depending on a determination that the request to perform the respective action includes entering a home. displaying keys (eg, a keyboard or keypad); and a plurality of character input keys (e.g., on a keyboard or and displaying a passcode affordance that causes the display of a keypad. In some examples, the passcode affordance is limited to activation in response to tap inputs, and is responsive to one or more other types of inputs, including additional input requirements other than touch input. In some examples, one or more additional input requirements may include a requirement that the input be a hard press input (e.g., a requirement that the input reach a characteristic intensity that exceeds a respective intensity threshold), a requirement that the input be a long press input (e.g., a requirement that the input be a long press input), , a requirement that the input include contact maintained on the touch-sensitive surface for a predetermined period of time without moving beyond a predetermined distance), and/or a requirement that the input be a swipe input (e.g., a requirement that the input be a swipe input in each direction). (requirement that the movement of the contact exceeds the threshold amount of movement). Limiting activation in response to tap inputs in this manner provides improved control and usability of the electronic device by avoiding spurious (e.g., accidental and/or unintentional) activation of the passcode affordance, thereby It reduces user errors when operating/interacting with the device, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, after displaying one or more affordances for using a second form of authentication that were not displayed prior to processing the respective data from the one or more biometric sensors, the request to perform the respective action is prompted by a home input. Depending on the decision to include, the electronic device may (automatically, without further user input) stop displaying (e.g., turn off the display) one or more affordances for using the second form of authentication, such as a first delay period. while waiting for further input; Upon determining that the request to perform the respective action includes selection of the notification, the electronic device may (automatically, without further user input) stop displaying one or more affordances for using the second form of authentication. Wait for additional input for a second delay period before (e.g., turning off the display). In some examples, the second delay period is different (eg, shorter or longer) than the first delay period.

In some examples, while the device is locked and a first form of authentication is available for use, the electronic device attempts biometric authentication using the first form of authentication (2732). In some examples, in response to a request to perform a respective action, in response to an attempt to use a second form of authentication, or in response to an input requesting authentication, such as lifting the device, a button on the device (e.g., 2604 ), tapping a lock icon on the device, or tapping the touch-sensitive display of the device, a first form of authentication is available for use and/or the device is locked. In some examples, while attempting biometric authentication using a first form of authentication, the electronic device displays a progress indicator (e.g., progress indicator) that changes appearance to indicate progress toward biometric authentication using the first form of authentication. display 2734 (as indicated by indicators 2624 and/or 2625). In some examples, the electronic progress indicator is an icon or progress bar that changes from a “face detection” icon or animation to a “face analysis” icon or animation. In some examples, while attempting biometric authentication using the first form of authentication, the device replaces the first graphical display with a progress indicator. In some examples, after completing an attempt at a first form of authentication, upon a determination that authentication was not successful, the electronic device replaces the progress indicator with a first graphical indication (eg, a closed lock icon); Upon determining that authentication was successful, the electronic device replaces the progress indicator with a second graphical representation (eg, an open lock icon).

In some examples, after attempting biometric authentication using the first form of authentication (2736), upon determining that biometric authentication using the first form of authentication was successful, the electronic device determines that the electronic device successfully authenticated using the first form of authentication. update the progress indicator (and, optionally, perform the respective action) in a first manner (e.g., by displaying a check mark or an open lock icon) to indicate (2738); and/or generate a second tactile output indicating authentication success (eg, different from the first tactile output indicating authentication failure) (eg, a single tap).

In some examples, after attempting biometric authentication using a first form of authentication, biometric authentication using the first form of authentication may be performed on a biometric device that may be used in the first form of authentication but does not match authorized biometric characteristics. Depending on the determination of detecting the measurement feature, the electronic device updates the progress indicator in a second manner to indicate unsuccessful authentication (e.g., shaking the lock icon or face icon left or right to indicate that authentication was unsuccessful). (2740). In some examples, the second way of updating the progress indicator is different than the first way of updating the progress indicator (2742). In some examples, the electronic device generates a first tactile output (eg, a triple tap) indicating authentication failure.

In some examples, after attempting biometric authentication using the first form of authentication, upon determining that biometric authentication using the first form of authentication does not detect a biometric characteristic that can be used in the first form of authentication, The electronic device updates the progress indicator in a third way that is different from the first way and the second way (e.g., displaying a closed lock icon without shaking the lock icon from side to side to indicate that authentication failed) (2744) .

In some examples, after attempting biometric authentication using the first form of authentication, upon determining that biometric authentication using the first form of authentication does not detect a biometric characteristic that can be used in the first form of authentication, The electronic device displays a graphical indication that successful authentication did not occur without generating tactile output. In some examples, the device does not generate the first tactile output or other tactile output indicating authentication failure because the device did not recognize any biometric characteristic that can be used for the first form of authentication.

In some examples, after attempting biometric authentication using the first form of authentication for a first time, the electronic device displays a plurality of character input keys (e.g., a plurality of character input keys) for entering a sequence of characters for authentication (e.g., a password or passcode). Displays a user interface (e.g., 2654) for a second form of authentication that includes a keyboard or keypad. Additionally, the electronic device attempts biometric authentication using the first form of authentication for a second time while displaying a user interface for the second form of authentication. Additionally, upon determining that biometric authentication using the first form of authentication is successful, the electronic device performs the respective operation (and, optionally, performs a first form of authentication to indicate successful authentication using the first form of authentication). updates the progress indicator in a manner (e.g., by displaying a check mark or an open lock icon (e.g., 2610)). Additionally, before receiving input inputting less than a threshold number of characters (e.g., less than 1, 2, 3, 4, or 5 characters) through one or more character input keys of the plurality of character input keys, Upon determining that biometric authentication using a first form of authentication detects biometric characteristics that may be used in the first form of authentication but do not match authorized biometric characteristics, the electronic device performs the respective actions: suspend doing so and generate a first tactile output (e.g., a triple tap) indicating authentication failure (and, optionally, display a graphical indication that successful authentication did not occur). Additionally, after receiving input of entering at least a threshold number of characters (e.g., less than 1, 2, 3, 4, or 5 characters) through one or more character input keys of the plurality of character input keys, Upon determining that biometric authentication using a first form of authentication detects biometric characteristics that may be used in the first form of authentication but do not match authorized biometric characteristics, the electronic device performs the respective actions: and displays a graphical indication (e.g., a closed lock icon) that successful authentication did not occur without generating tactile output.

In some examples, in response to detecting a request to perform a respective action requiring authentication, the device is locked (e.g., when a request to perform the respective action is received) and a first form of biometric authentication is used. Upon determining that the electronic device is not available to use one or more affordances for using a second form of authentication (e.g., a passcode or password entry user interface, or a second form of biometric authentication, such as fingerprint authentication) A prompt to use is displayed (2746). In some examples, the first form of authentication is not available for use because it has been disabled (2748). In some examples, the first form of authentication may be performed due to more than a threshold number of failed biometric authentication attempts using the first form of biometric authentication, due to a restart of the device, or disabling the first form of biometric authentication. Not available for use due to user request. In some examples, the first form of authentication is disabled in response to user input to the emergency options user interface without selecting an option corresponding to a request to access additional information on the device (e.g., the user exceeds a threshold time). triggering the display of the Emergency Options User Interface by pressing two or more buttons simultaneously, and then stopping operation of the device or accessing the Emergency Options User Interface, rather than selecting the option to display medical information or the Emergency Dialing Interface. Select the option to cancel the display of). In some examples, the first form of authentication is disabled in response to user selection of an option to disable the first form of biometric authentication (e.g., via biometric authentication settings in a settings user interface). In some examples, the first form of authentication may be where operation of the one or more biometric sensors is determined by current environmental and/or device conditions that reduce the ability of the one or more biometric sensors to operate within predefined parameters. Not available for use because it is restricted (2750). In some examples, the device is too hot, the device is too cold, there is too much light in the device's environment, there is too little light in the device's environment, and/or the device's battery is insufficient to power one or more biometric sensors. It is not charged sufficiently.

In some examples, the electronic device detects the first input (eg, 2650) (eg, a tap input) at a location that corresponds to the respective position in the user interface. In some examples, in response to detecting a first input at a location corresponding to the respective location in the user interface, determining that the device is in a locked state (e.g., a tap input was detected on a closed lock icon), electronically The device attempts a first form of authentication. Attempting the first form of authentication in response to detecting a first input at a location corresponding to the respective location in the user interface and in accordance with a determination that the device is in a locked state may be an intuitive process in which the first form of authentication is initiated. and provides convenient features, thereby making the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling the user to use the device more quickly and efficiently.

In some examples, the electronic device detects the second input (eg, a tap input) at a location that corresponds to the respective location in the user interface. In some examples, in response to detecting a second input at a location corresponding to the respective location in the user interface, determining that the device is in an unlocked state (e.g., a tap input was detected on an open lock icon) , the electronic device transitions the device from an unlocked state to a locked state. In some examples, each location is on a cover sheet user interface that is displayed when the device screen is initially turned on, and a second graphical indication (e.g., an open lock icon) is displayed on the cover sheet while the device is still in an unlocked state. While the user interface is displayed on the device, a first graphical indication (eg, a lock icon) is displayed when the cover sheet user interface is displayed on the device while the device is in a locked state.

Note that the details of the processes described above with respect to method 1200 (e.g., FIGS. 27A-27E) are also applicable in a similar manner to the methods described herein. For example, method 2700 optionally includes one or more of the features of the various methods described herein with reference to methods 800, 1000, 1200, 1400, 1600, 1800, 2000, 2200, 2500. Includes. For example, the registered biometric data described in method 1200 may be used to perform biometric authentication as described with respect to FIGS. 26L-26N. For another example, one or more implantable interfaces as described in methods 2000 and 2700 are optionally displayed in response to receipt of input prior to completion of the biometric authentication process. For brevity, these details are not repeated herein.

Operations in the information processing methods described above optionally utilize one or more functional modules within an information processing device, such as general-purpose processors (e.g., as described with respect to FIGS. 1A, 3, and 5A) or application-specific chips. It is implemented by running it. Additionally, the operations described above with reference to FIGS. 27A-27E are optionally implemented by the components shown in FIGS. 1A and 1B. For example, detect operation 2702, perform operation 2706, display operation 2712, and display operation 2746 may optionally be performed by event classifier 170, event recognizer 180, and event handler 190. It is implemented. Event monitor 171 in event sorter 170 detects contact on touch-sensitive surface 604, and event dispatcher module 174 passes event information to application 136-1. The respective event recognizer 180 of application 136-1 compares the event information to the respective event definition 186 and determines that the first contact at the first location on the touch-sensitive surface corresponds to the selection of an object on the user interface. Determines whether it corresponds to the same predefined event or subevent. When an individual predefined event or sub-event is detected, event recognizer 180 activates an event handler 190 associated with detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some examples, event handler 190 accesses the respective GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes may be implemented based on the components shown in FIGS. 1A and 1B.

28A-28AA illustrate example user interfaces for preventing retry of biometric authentication, according to some examples. As described in more detail below, illustrative examples of the user interfaces illustrated in Figures 28A-28AA are used to illustrate the processes described below, including the processes in Figures 29A and 29B.

FIG. 28A illustrates an electronic device 2800 (e.g., portable multifunction device 100, device 300, device 500, or device 1700). In the illustrative examples illustrated in FIGS. 28A-28AA, electronic device 2800 is a smartphone. In other examples, electronic device 2800 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 2800 includes a display 2802, one or more input devices (e.g., a touchscreen and microphone of display 2802), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In Figure 28A, the electronic device includes a biometric sensor 2803. In some examples, the biometric sensor is one or more biometric sensors that may include a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, biometric sensor 2803 is biometric sensor 703. In some examples, the one or more biometric sensors include one or more fingerprint sensors (eg, a fingerprint sensor integrated into a button). In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In FIG. 28A , electronic device 2800 displays an application's user interface 2804 on display 2802 . The application is a mobile browser application, and the user interface 2804 corresponds to a website (onlinestore.com). In Figure 28B, electronic device 2800 detects a tap gesture 2806 on login affordance 2808. Electronic device 2800 identifies tap gesture 2806 as a request to load login user interface 2810 (shown in FIG. 28C). Electronic device 2800 may also use a tap gesture 2806 to enter fillable fields within login user interface 2810, namely username field 2812 and password field 2814, with credential information (e.g., Identifies you as a request to automatically fill in your username and password to enable you to successfully log in to your account. Auto-fill requests require biometric authentication to proceed with auto-filling fillable fields. In some examples, the request also includes a request to automatically log the user in so that the user does not have to tap a submit affordance (e.g., 2860 in FIG. 28Z) to submit credentials and log in.

28C, in response to a tap gesture 2806 (e.g., a request to automatically fill fillable fields), electronic device 2800 uses biometric sensor 2803 to determine whether certain biometric authentication criteria have been met. Decide whether Electronic device 2800 may capture and process (e.g., analyze) biometric data from biometric sensor 2803 to determine, based on the biometric data, biometric features (or portions thereof) based on biometric authentication criteria. determine whether the biometric data is satisfied (e.g., determine whether the biometric data matches the biometric template, within a threshold). The biometric sensor 2803 is non-contact, allowing the sensor to be configured to perform biometric authentication without physical input from the user (e.g., without any additional gestures after the tap gesture 2806). As a result, electronic device 2800 initiates biometric authentication using biometric sensor 2803 without having to receive an explicit request from the user to initiate biometric authentication.

Performing biometric authentication includes displaying a biometric authentication interface 2816 with a biometric authentication glyph 2818. Biometric authentication glyph 2818 is a simulation of a representation of a biometric feature (e.g., face). As shown in FIG. 28C, biometric authentication interface 2816 is overlaid on at least a portion of login user interface 2810. Biometric authentication interface 2816 is optionally an operating system level interface (e.g., an interface created by the device's operating system) and login user interface 2810 is an application level interface (e.g., an interface that is separate from the device's operating system). a user interface created by a separate third-party application). In some examples, the displayed biometric authentication interface is approximately centered along the horizontal and/or vertical axis, as in Figures 28B-28E. In some examples, electronic device 2800 displays a biometric authentication interface on the top, bottom, side, or corner of display 2802. For example, electronic device 2800 displays a biometric authentication interface near the top of display 2802, for example in Figure 30A. In some examples, electronic device 3000 does not display a biometric authentication interface while biometric authentication is being performed.

28D and 28E, electronic device 2800 displays a biometric authentication animation including biometric authentication glyph 2820 in FIG. 28D and biometric authentication glyph 2822 in FIG. 28E, which glyphs are: Biometric sensor 2803 serves as part of an animation attempting to acquire biometric data.

In Figure 28F, a determination is made that biometric authentication has failed (e.g., biometric authentication criteria were not met). As a result, electronic device 2800 refrains from automatically populating username field 2812 and password field 2814. Additionally, electronic device 2800 does not display an indication to the user that the user should retry biometric authentication. In some examples, a determination is made that biometric authentication was successful (e.g., biometric authentication criteria were met). As a result, in such examples, electronic device 2800 automatically populates username field 2812 and password field 2814.

In Figure 28G, electronic device 2800 detects a tap gesture 2824 on a reload affordance 2826 after determining that biometric authentication failed. Electronic device 2800 identifies tap gesture 2824 as a request to reload login user interface 2810. Electronic device 2800 may also perform a tap gesture 2824 as a request to automatically fill one or more fillable fields within login user interface 2810 (e.g., username field 2812 and password field 2814). It is identified as. As previously described, the auto-fill request requires biometric authentication to proceed with auto-filling the fillable fields.

In response to the request to automatically populate the fillable fields, a determination is made that the failure of biometric authentication in Figure 28F was due to failure to detect the presence of a face for a predetermined period of time. As a result, the electronic device 2800 performs biometric authentication again using the biometric sensor 2803, as shown in FIG. 28H. The electronic device 2800 automatically performs biometric authentication again without the user having to provide input to initiate authentication.

28H and 28I , electronic device 2800 performs biometric authentication, including displaying a biometric authentication interface and biometric authentication glyphs, as described with respect to FIGS. 28C and 28D. Once the electronic device 2800 has acquired biometric data (e.g., has acquired sufficient biometric data), the electronic device transitions to displaying the biometric authentication glyph 2828. Electronic device 2800 displays biometric authentication glyph 2828 to indicate that biometric data is being processed. In some examples, biometric authentication glyph 2828 includes a plurality of rings that rotate in a sphere, for example, while being displayed.

In Figure 28K, a determination is made that biometric authentication has again failed. As a result, the electronic device 2800 displays a failed biometric authentication interface 2830 with a failed biometric authentication glyph 2832. As a result, electronic device 2800 refrains from automatically populating username field 2812 and password field 2814. In some examples, a determination is made that biometric authentication was successful. As a result, in such examples, electronic device 2800 automatically populates username field 2812 and password field 2814.

In Figure 28L, electronic device 2800 detects a tap gesture 2824 on reload affordance 2826 after determining that biometric authentication failed a second time. Electronic device 2800 identifies tap gesture 2826 as a request to reload login user interface 2810. Electronic device 2800 may also perform a tap gesture 2826 as a request to automatically fill one or more fillable fields within login user interface 2810 (e.g., username field 2812 and password field 2814). It is identified as. As previously described, the auto-fill request requires biometric authentication to proceed with auto-filling the fillable fields.

In response to a request to automatically populate fillable fields, a determination is made that the failure of biometric authentication in Figure 28K was due to detecting a face that does not match the authorized face (e.g., the biometric data is within a threshold). does not match the biometric template). As a result, electronic device 2800 holds off on performing biometric authentication, as illustrated by FIG. 28M.

In FIG. 28N , electronic device 2800 detects a tap gesture 2834 on username field 2812 after pending performing biometric authentication in response to tap gesture 2826. As a result, the electronic device 2800 displays a cursor 2836 within the user name field 2812, and also displays a virtual keyboard 2838 and a password affordance 2840, as shown in FIG. 28O. In Figure 28P, electronic device 2800 detects a tap gesture 2842 on password affordance 2840. As a result, electronic device 2800 displays a list of candidate input affordances (e.g., 2844, 2846, 2848), as shown in FIG. 28Q. In some examples, in response to detecting a tap gesture 2834 on username field 2812, electronic device 2800 displays an affordance labeled “usernames” instead of password affordance 2840. do.

In Figure 28R, electronic device 2800 detects a tap gesture 2850 on candidate input affordance 2848 (labeled “[email protected]”). Electronic device 2800 identifies tap gesture 2850 as a request to automatically populate username field 2812 and password field 2814 with credential information corresponding to candidate input affordance 2848. This request to automatically fill fillable fields is a different type of request to automatically fill than those resulting from a request to load or reload the login user interface 2810. The request to auto-fill through a request to load the login user interface is an implicit request because the request to auto-fill is performed as part of the request to load the login user interface. In contrast, the request to automatically fill in FIG. 28R is an explicit request by the user to automatically fill the username field 2812 and password field 2814 with credential information corresponding to the candidate input affordance 2848. am. In response to the request to automatically populate fillable fields through the explicit request in Figure 28R, electronic device 2800 initiates biometric authentication, as shown in Figure 28S.

28S-28U, electronic device 2800 performs biometric authentication, including displaying a biometric authentication interface and biometric authentication glyphs, as described with respect to FIGS. 28H-28J.

In Figure 28v, a determination is made that biometric authentication was successful. As a result, electronic device 2800 displays successful biometric authentication glyph 2852 indicating that biometric authentication was successful.

28W, because biometric authentication was successful, electronic device 2800 automatically populates username field 2812 and password field 2814 with credential information corresponding to candidate input affordance 2848. In some examples, a determination is made that biometric authentication has failed. As a result, in such examples, electronic device 2800 refrains from automatically populating username field 2812 and password field 2814 with credential information. In some examples, upon failure of biometric authentication, electronic device 2800 displays failure interface 2854 in Figure 28X, as described with respect to Figure 17M. Alternatively, electronic device 2800 may display failure interface 2856 in FIG. 28Y, as described with respect to FIG. 15S. Failure interface 2854 may be displayed when the user has not reached the maximum number of failed biometric authentication attempts (e.g., the maximum number of failed attempts without an intervening successful authentication attempt). When the maximum number of failed biometric authentication attempts has been reached, a failure interface 2856 may be displayed instead.

After the electronic device 2800 automatically populates the username field 2812 and password field 2814 in Figure 28W, the electronic device performs a tap gesture on the submit affordance 2860, as illustrated by Figure 28Z. 2858) is detected. Electronic device 2800 identifies tap gesture 2858 as a request to submit credential information in username field 2812 and password field 2814 for user authentication. Upon successful user authentication, electronic device 2800 provides access to restricted content (e.g., content that can only be viewed when the user is logged in) within user interface 2862 of Figure 28aa.

29A and 29B are flow diagrams illustrating a method for re-performing biometric authentication after an initial unsuccessful biometric authentication attempt using an electronic device, according to some examples. Method 2900 includes one or more biometric sensors (e.g., 2803) (e.g., a fingerprint sensor, a non-contact biometric sensor (e.g., a biometric sensor that does not require physical contact, such as a thermal or optical facial recognition sensor), iris). It is performed on a device (eg, 100, 300, 500, 1700, 2800) with a scanner. In some examples, one or more biometric sensors (eg, 2803) include one or more cameras. The electronic device (eg, 100, 300, 500, 1700, 2800) optionally includes a display (eg, 2802). In some examples, the display (eg, 2802) is a touch-sensitive display. In some examples, the display (eg, 2802) is not a touch-sensitive display.

Some operations of method 2900 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted. As described below, method 2900 provides an intuitive method for re-performing biometric authentication after an initial unsuccessful biometric authentication attempt. This method reduces the user's cognitive burden of authenticating on an electronic device, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to authenticate faster and more efficiently saves power and increases time between battery charges.

An electronic device (e.g., 100, 300, 500, 1700, 2800) receives a first request (e.g., 2806) to perform each operation requiring authentication (e.g., autofill, unlock device, make payment). Do (2902). In some examples, a first request (eg, 2806) is associated with performance of a respective operation. In some examples, the first request (e.g., 2806) may also be a request to perform a second operation that is different from the respective operation (e.g., a request to display a web page (e.g., 2810) or load content that requires authentication )am. In some examples, the second operation does not require authentication.

According to some examples, the first request (eg, 2806) is also a request to perform an operation that does not require biometric authentication. In response to receiving the first request (e.g., 2806), the electronic device (e.g., 100, 300, 500, 1700, 2800) performs an operation that does not require biometric authentication.

According to some examples, the first request (eg, 2806) is a request to open a webpage (eg, 2810).

In response to receiving a first request (e.g., 2806) to perform a respective operation (2904), the electronic device (e.g., 100, 300, 500, 1700, 2800) proceeds to blocks 2906 through 2910. Proceed.

The electronic device (e.g., 100, 300, 500, 1700, 2800) uses one or more biometric sensors (e.g., 2803) to determine whether biometric authentication criteria are met, wherein the biometric authentication criteria are: Includes a requirement (2906) that each type of biometric feature (e.g., face or fingerprint) that is authorized to perform the operation of is detected by biometric sensors (e.g., 2803). In some examples, biometric authentication criteria include a requirement that an authorized biometric characteristic be detected by one or more biometric sensors (e.g., 2803).

Upon determining that the biometric authentication criteria are met, the electronic device (e.g., 100, 300, 500, 1700, 2800) performs the respective operation (2908).

Upon determining that the biometric authentication criteria are not met (2910), the electronic device (e.g., 100, 300, 500, 1700, 2800) withholds (2912) from performing the respective operation. Suspending (or performing) each operation based on not meeting biometric authentication criteria can provide security and prevent unauthorized users from initiating sensitive operations. Providing improved security improves the operability of the device (e.g., by restricting unauthorized access) and makes the user-device interface more efficient, which additionally reduces the power of the device by limiting performance of restricted operations. Reduces usage and improves battery life.

According to some examples, further in response to receiving a first request (e.g., 2806) to perform the respective operation (2904) and in accordance with a determination that biometric authentication criteria are not met (2910), the electronic device ( For example, 100, 300, 500, 1700, 2800) may indicate on the display (e.g., 2802) an indication to retry authentication using one or more biometric sensors (e.g., 2803) (e.g., to provide biometric authentication to the user). Suspend displaying (2914) visually presented commands that prompt to retry. In some examples, the electronic device (eg, 100, 300, 500, 1700, 2800) also refrains from displaying an indication to re-request the respective operation.

According to some examples, determining whether biometric authentication criteria are met may include determining whether biometric authentication criteria are met based on data obtained from one or more biometric sensors (e.g., 2803) corresponding to the biometric characteristic. and determining whether at least a portion satisfies biometric authentication criteria. In some examples, when the request (e.g., 2806) is also a request to perform a second action that does not require authentication and is different from the respective action, the second action may even be dependent on a determination that the biometric authentication criteria are not met. is also performed. For example, a first request (e.g., 2806) (e.g., entry of a URL address) may be performed for each operation requiring authentication (e.g., a username and/or password for a web page (e.g., 2810) associated with the URL address. It is a request to perform a second operation (display of a web page (eg, 2810) associated with a URL address) that does not require authentication. Even when the biometric authentication criteria are not met, performing a second action that does not require authentication may provide feedback to the user about the request even if the biometric security action is not performed. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In response to receiving the first request (e.g., 2806) and subsequently determining that the biometric authentication criteria have not been met (e.g., no face was detected or the face that was detected does not match the authorized face), The electronic device (e.g., 100, 300, 500, 1700, 2800) receives (2916) a second request (e.g., 2824) to perform the respective operation. In some examples, the non-user request to reload a webpage (e.g., 2810) is not a request associated with retrying biometric authentication.

In response to receiving a second request (e.g., 2824) to perform a respective operation (2918), the electronic device (e.g., 100, 300, 500, 1700, 2800) proceeds to blocks 2920 and 2922. Proceed.

Upon determining that the biometric authentication criteria have not been met in response to the first request (e.g., 2806) due to one or more biometric sensors (e.g., 2803) failing to detect the presence of the respective type of biometric characteristic, The electronic device uses one or more biometric sensors (e.g., 2803) to determine (2920) whether biometric authentication criteria are met in response to the second request (e.g., 2824). Re-performing biometric authentication when a previous authentication failure was due to failure to detect the presence of a biometric characteristic, without requiring additional inputs from the user and cluttering the user interface with additional displayed controls. Provides the ability to retry authentication without Providing the ability to retry authentication without additional input and without cluttering the UI with additional displayed controls helps the user provide appropriate inputs when operating/interacting with the device (e.g., (by reducing mistakes) and making the user-device interface more efficient, which additionally reduces the device's power usage and battery life by enabling users to use the device more quickly and efficiently. improve it

In some examples, the biometric feature is a face, and data from biometric sensors (e.g., 2803) does not include data indicating that a face was detected. In some examples, a determination that the biometric authentication criteria have not been met in response to the first request (e.g., 2806) may include detecting one or more biometric sensors (e.g., 2803) of each type of biometric characteristic for a predetermined period of time. Occurs when its presence cannot be detected.

According to some examples, biometric authentication criteria are not met in response to a first request (e.g., 2806) due to one or more biometric sensors (e.g., 2803) failing to detect the presence of a respective type of biometric characteristic. The determination is that one or more biometric sensors (e.g., 2803) has not been activated at least a predetermined amount of time (e.g., after biometric authentication is triggered, e.g., by a first request to perform the respective action (e.g., 2806)). A determination is made in response to the first request (e.g., 2806) that the biometric authentication criteria have not been met due to a failure to detect the presence of the respective type of biometric characteristic for a determined period of time.

Biometric authentication criteria are generated in response to a first request (e.g., 2806) where one or more biometric sensors (e.g., 2803) detect a respective type of biometric feature that does not correspond to an authorized biometric feature. Depending on the determination that this is not met (e.g., the face that was detected does not match the authorized face), the electronic device (e.g., 100, 300, 500, 1700, 2800) responds to a second request (e.g., 2824). Pending determining whether biometric authentication criteria are met using one or more biometric sensors (e.g., 2803) (e.g., the device performs biometric authentication in response to reloading a webpage (e.g., 2810). does not automatically retry) (2922). In some examples, withholding from performing biometric authentication again further includes withholding from performing an action that would be performed if the biometric authentication criteria are met. Deferring retrying biometric authentication when a previous authentication failure was due to detection of unauthorized biometric characteristics improves security and reduces the number of resource-intensive retries for users who appear to be unauthorized. reduce cases. Providing improved security improves the operability of the device (e.g., by restricting unauthorized access) and makes the user-device interface more efficient, which additionally reduces the power of the device by limiting performance of restricted operations. Reduces usage and improves battery life.

According to some examples, in response to receiving the first request (e.g., 2806) and subsequent to determining that biometric authentication criteria have not been met, the electronic device (e.g., 100, 300, 500, 1700, 2800) may each Receives a third request (e.g., 2850) to perform an action (e.g., tapping the secure password field and selecting a password to automatically fill, tapping the non-secure username field and selecting a username to automatically fill), where: The third request is a different type of request than the first request (e.g., 2806) and the second request (e.g., 2824) (e.g., the third request is a different type of request than the one used to make the first and second requests. Made using a selection of affordances, the third type of request is also not a request to perform a second action (e.g. loading a web page), whereas the first and second requests are also a request to perform a second action. requests). In response to receiving a third request (e.g., 2850) to perform the respective operation, the electronic device (e.g., 100, 300, 500, 1700, 2800) performs one or more operations in response to the third request (e.g., 2850). Use biometric sensors (e.g., 2803) to determine whether the biometric authentication criteria are met (e.g., regardless of the reason the biometric authentication criteria were not met in response to the first request (e.g., 2806) For example, the biometric authentication criteria may be due to one or more biometric sensors detecting a respective type of biometric characteristic that does not correspond to an authorized biometric characteristic or one or more biometric sensors detecting a respective type of biometric characteristic that does not correspond to an authorized biometric characteristic. Determine whether biometric authentication criteria are met using one or more biometric sensors (regardless of whether they are not met due to failure to detect the presence of a wealth). Regardless of the reason for a previous authentication failure, performing biometric authentication again after receiving a different type of request (e.g., an explicit request) provides the user with the ability to explicitly request re-authentication, Provides additional control options for the user. Providing additional control options to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

According to some examples, further in response to receiving a second request (e.g., 2824) to perform the respective operation and upon determining that biometric authentication is not available (e.g., maximum number of failed biometric authentication Reached attempts, attempts since last successful authentication have exceeded a predefined number of allowed attempts), the electronic device (e.g., 100, 300, 500, 1700, 2800) may require an alternative form of authentication (e.g., prompt for a non-biometric form of authentication, such as a password or passcode (e.g., 2854, 2856). Providing a prompt for alternative authentication (e.g., displaying a notification) when biometric authentication is no longer available provides the user with feedback regarding the current state of the device and what is needed to authenticate. Provides indicating feedback to the user. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

According to some examples, the electronic device (e.g., 100, 300, 500, 1700, 2800) imposes a respective limit on the number of unsuccessful biometric authentication attempts allowed before an alternative form of authentication is required. . The electronic device (e.g., 100, 300, 500, 1700, 2800) is prompted to perform each operation after a predetermined number of requests to perform each operation (e.g., within a threshold period) result in failed biometric authentication attempts. Stop using one or more biometric sensors (e.g., 2803) in response to requests to determine whether biometric authentication criteria are met, where a predefined number of requests is less than a respective limit.

In some examples, in response to detecting a respective request to perform a respective action, the device (e.g., 100, 300, 500, 1700, 2800) determines whether a predetermined number of requests to perform the respective action have failed. Determine whether measurement authentication attempts have occurred. Upon determining that a predetermined number of requests to perform each operation have resulted in failed biometric authentication attempts, the electronic device (e.g., 100, 300, 500, 1700, 2800) withholds from attempting biometric authentication. Upon determining that the predetermined number of requests to perform each operation did not result in failed biometric authentication attempts, the electronic device (e.g., 100, 300, 500, 1700, 2800) proceeds with additional biometric authentication attempts.

In some examples, the number of biometric authentication attempts or retries that can be made (e.g., that can be made without success) is a predetermined number of unsuccessful attempts before alternative authentication (e.g., a password or passcode) is required. limited. In such examples, the electronic device (e.g., 100, 300, 500, 1700, 2800) may attempt to detect the presence of each type of biometric characteristic so as not to exceed a predetermined number of allowable attempts (e.g., by failing to detect the presence of each type of biometric characteristic). refrain from retrying biometric authentication after a predetermined number of attempts, even under conditions where biometric authentication is otherwise attempted (after a previous failure due to a previous failure). Stopping use of a biometric sensor (e.g., withholding biometric authentication) before the allowed number of attempts after repeated requests is exhausted will prevent the user from making repeated requests (e.g., repeats of the same type). requests), thereby preserving at least one attempt for requests for other operations requiring biometric authentication. Preserving at least one attempt improves the operability of the device (e.g., by avoiding exhaustion of authentication attempts for repeated similar requests) and makes the user-device interface more efficient, which additionally allows the user to access the device. Reduces device power usage and improves battery life by enabling faster and more efficient use of devices.

According to some examples, determining whether biometric authentication criteria are met using one or more biometric sensors (e.g., 2803) in response to a second request (e.g., 2824) may include a request to perform the respective action. 2 Occurs automatically (eg, without the need for input from the user) in response to receiving a request (eg, 2824).

According to some examples, one or more biometric sensors (e.g., 2803) may include non-contact biometric sensors (e.g., 2803) configured to perform biometric authentication without physical contact from the user (e.g., an infrared camera, a visible light camera, or combinations thereof).

According to some examples, in response to the second request (e.g., 2824) and in response to a determination that biometric authentication criteria are met in response to the second request (e.g., 2824), the electronic device (e.g., 100, 300, 500, 1700, 2800) perform respective operations (e.g., operations include auto-filling, accessing data, unlocking the device, and/or making a payment).

According to some examples, each operation automatically populates one or more fillable fields (e.g., 2812, 2814) with credential information (e.g., credit card information or login information). In some examples, credit card information includes information associated with payment account information (eg, credit card, bank account, or payment service information). In some examples, login information includes information needed to log in to an application, account, or website (e.g., 2862). Automatically populating credential information upon request and successful authentication provides the user with the ability to populate credentials without requiring additional input (besides the request). Performing an action when a set of conditions is met without requiring additional user input (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device) It improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

According to some examples, each operation may include providing access to restricted content (e.g., logging into a webpage (e.g., 2862), displaying a list of passwords associated with the user, displaying credit card information, etc. will be.

According to some examples, each operation transitions the electronic device (eg, 100, 300, 500, 1700, 2800) from a locked state to an unlocked state. In some examples, transitioning an electronic device (e.g., 100, 300, 500, 1700, 2800) to an unlocked state includes the electronic device's display (e.g., 2802) and one or more biometric sensors (e.g., 2803). , and/or enabling a microphone.

According to some examples, each operation is to enable an electronic device (eg, 100, 300, 500, 1700, 2800) to engage in a transaction (eg, a financial transaction, such as payment for goods or services).

According to some examples, while determining whether biometric authentication criteria are met using one or more biometric sensors (e.g., 2803), the electronic device (e.g., 100, 300, 500, 1700, 2800) displays On (e.g., 2802), display an indication that biometric authentication is being performed (e.g., a small indicator is displayed on the top, bottom, side, or corner). In some examples, no indicator is displayed during biometric authentication. In some examples, an electronic device (e.g., 100, 300, 500, 1700, 2800) uses one or more biometric sensors (e.g., 2803) to identify a biometric device while determining whether biometric authentication criteria are met. Reserve displaying an indication that measurement authentication is being performed. In some examples, biometric authentication criteria are not met in response to a first request (e.g., 2806) due to one or more biometric sensors (e.g., 2803) failing to detect the presence of a respective type of biometric characteristic. When a first visual indication (e.g., 2822) is displayed. In some examples, one or more biometric sensors (e.g., 2803) respond to a first request (e.g., 2806) by detecting a respective type of biometric feature that does not correspond to an authorized biometric feature. A second visual indication (e.g., 2832) (e.g., the same or different from the first visual indication) is displayed if the measurement authentication criteria are not met. In some examples, a third visual indication (e.g., 2852) (e.g., a third visual indication that is different from the first visual indication and/or the second visual indication) is displayed if the biometric authentication criteria are met.

Note that the details of the processes described above with respect to method 2900 (e.g., FIGS. 29A and 29B) are also applicable in a similar manner to the methods described below and above. For example, method 2900 may optionally have features of various methods described above with reference to methods 800, 1000, 1200, 1400, 1600, 2000, 2200, 2500, 2700, 3100, 3300, 3500. Contains one or more of For example, registered biometric data described in method 1200 may be used to perform biometric authentication as described in connection with method 2900. As a further example, authentication caching of method 3100 may be based on a successful authentication performed pursuant to a re-performed biometric authentication as described with respect to method 2900. For brevity, these details are not repeated below.

30A-30A illustrate example user interfaces for cached biometric authentication, according to some examples. As described in more detail below, illustrative examples of the user interfaces illustrated in Figures 30A-30A are used to illustrate the processes described below, including the processes in Figures 31A and 31B.

FIG. 30A illustrates an electronic device 3000 (e.g., portable multifunction device 100, device 300, device 500, or device 1700). In the illustrative examples illustrated in FIGS. 30A-30A, electronic device 3000 is a smartphone. In other examples, electronic device 3000 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 3000 includes a display 3002, one or more input devices (e.g., a touchscreen of display 3002, buttons 3004, and microphone), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In Figure 30A, the electronic device includes a biometric sensor 3003. In some examples, the biometric sensor is one or more biometric sensors that may include a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, biometric sensor 3003 is biometric sensor 703. In some examples, the one or more biometric sensors include one or more fingerprint sensors (eg, a fingerprint sensor integrated into a button). In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In FIG. 30A , electronic device 3000 displays a user interface 3006 of an application on display 3002 . The application is a mobile browser application, and the interface corresponds to a website (online.com). In some examples, website online.com is the root domain of the online.com domain. Electronic device 3000 detects tap gesture 3008 on login affordance 3010. Electronic device 3000 identifies tap gesture 3008 as a request to load login user interface 3012 (shown in FIG. 30B). The user interface 3012 is the web page id.online.com. In some examples, id.online.com is a subdomain of the online.com domain that requires successful authentication before access to restricted content is granted. Electronic device 3000 also identifies tap gesture 3008 as a request to automatically fill fillable fields within login user interface 3012, namely username field 3014 and password field 3016. . Auto-fill requests require biometric authentication to proceed with auto-filling fillable fields. In some examples, the request also includes a request to automatically log the user in so that the user does not have to tap a submit affordance (e.g., 3030, 3046) to submit credentials and log in.

30B, in response to a tap gesture 3008 (e.g., a request to automatically fill fillable fields), electronic device 3000 uses biometric sensor 3003 to determine whether certain biometric authentication criteria have been met. Decide whether Electronic device 3000 may capture and process (e.g., analyze) biometric data from biometric sensor 3003 to determine, based on the biometric data, biometric features (or portions thereof) based on biometric authentication criteria. determine whether the biometric data is satisfied (e.g., determine whether the biometric data matches the biometric template, within a threshold). The biometric sensor 3003 is non-contact, allowing the sensor to be configured to perform biometric authentication without physical input from the user (e.g., without any additional gestures after the tap gesture 3008). As a result, the electronic device 3000 initiates biometric authentication using biometric sensor 3003 without having to receive an explicit request from the user to initiate biometric authentication.

Performing biometric authentication includes displaying a biometric authentication interface 3018 with a biometric authentication glyph 3020. Biometric authentication glyph 3020 is a simulation of a representation of a biometric feature (e.g., face). As shown in FIG. 30B, biometric authentication interface 3018 is overlaid on at least a portion of login user interface 3012. Biometric authentication interface 3018 is optionally an operating system level interface (e.g., an interface created by the device's operating system) and login user interface 3012 is an application level interface (e.g., an interface that is different from the device's operating system). a user interface created by a separate third-party application).

30C, electronic device 3000 displays a portion of a biometric authentication animation that includes biometric authentication glyph 3022, which is a portion of an animation in which biometric sensor 3003 acquires biometric data. It plays a role as In some examples, the animation of which glyphs 3020 and 3022 are a part indicates that the electronic device is attempting to identify a particular type of biometric feature (eg, identify a face). Once electronic device 3000 has acquired biometric data (e.g., has acquired sufficient biometric data), electronic device 3000 displays biometric authentication glyph 3024, as shown in FIG. 30D. transition to Electronic device 3000 displays biometric authentication glyph 3024 to indicate that biometric data is being processed. In some examples, biometric authentication glyph 3024 includes a plurality of rings that rotate in a sphere, for example, while being displayed.

In Figure 30E, a determination is made that biometric authentication was successful. As a result, the electronic device 3000 displays a successful biometric authentication glyph 3026 indicating that biometric authentication was successful.

30F, because biometric authentication was successful, electronic device 3000 stores username field 3014 and password field 3016 with credential information (e.g., a username that allows the user to successfully log into the account). and password) are automatically filled in. Electronic device 3000 automatically fills in the fields while the device is in an unlocked state. In some examples, a determination is made that biometric authentication has failed (e.g., biometric authentication criteria have not been met). As a result, in such examples, electronic device 3000 refrains from automatically filling one or more fillable fields (e.g., username field 3014 and password field 3016).

30G, electronic device 3000 detects a tap gesture 3028 on submission affordance 3030. In response to detecting the tap gesture 3028, the electronic device 3000 submits credential information in the username field 3014 and password field 3016 for user authentication. Upon successful user authentication, electronic device 3000 provides access to restricted content (e.g., content that can only be viewed when the user is logged in) within account user interface 3032 of Figure 30H.

In Figure 30I, electronic device 3000 detects a tap gesture 3034 on a shop affordance 3036. Electronic device 3000 identifies tap gesture 3034 as a request to load login user interface 3038 (shown in FIG. 30J). User interface 3038 is the web page shop.online.com. In some examples, shop.online.com is a subdomain of the online.com domain that requires successful authentication before access to restricted content is granted. In some examples, such as the example of Figure 30J, the subdomain shop.online.com is one or more other subdomains of the online.com domain, such as subdomain id.online.com, discussed in connection with Figures 30B-30I. Requires separate authentication from that required by the domains. Electronic device 3000 may also perform a tap gesture 3034 as a request to automatically fill one or more fillable fields within login user interface 3038 (e.g., username field 3040 and password field 3042). It is identified as. As described previously, in some examples, the request also includes a request to automatically log the user in so that the user does not have to tap a submit affordance (e.g., 3030, 3046) to submit credentials and log in. .

In response to the request to automatically populate the fillable fields, a determination is made that the cached authentication from the successful authentication that occurred in Figure 30E is available for use. As a result, the electronic device 3000 holds off on performing biometric authentication again, as shown in FIG. 30J, and proceeds to automatically fill in the user name field 3040 and password field 3042. Electronic device 3000 automatically populates the fields regardless of whether biometric sensor 3003 detects a biometric feature (eg, face or finger). 30K, electronic device 3000 detects a tap gesture 3044 on submission affordance 3046. In response to detecting the tap gesture 3044, the electronic device 3000 submits credential information in the username field 3040 and password field 3042 for user authentication. Upon successful user authentication, electronic device 3000 provides access to restricted content (e.g., content that can only be viewed when the user is logged in) within store user interface 3048 of FIG. 30L.

Alternatively, in response to a request to automatically populate fillable fields, a determination is made that a cached authentication is not available for use. 30N-30V illustrate various examples that cause cached authentication to be unavailable for use by electronic device 3000, as described below. 30W-30Y illustrate that biometric authentication must be performed when cached authentication is not available for use, as described below.

30M, electronic device 3000 displays store user interface 3048 and detects a tap gesture 3050 on link affordance 3052. In response to detecting tap gesture 3050, electronic device 3000 displays account user interface 3032, as shown in FIG. 30N.

30N and 30O illustrate an example that causes cached authentication to be unavailable for use by electronic device 3000. In Figure 30N, electronic device 3000 detects input (e.g., a single press) by finger 3054 on home button 3056. In response to detecting input, electronic device 3000 displays home screen 3058, as shown in Figure 30O, and causes the application with accounting user interface 3032 to enter an inactive state (e.g., on hold). , a dormant state, a background state, and/or a non-active state). The application will receive a request to automatically fill the fillable fields in the login user interface 3038 (e.g., Figure 30J) and when the fillable fields in the login user interface 3012 (e.g., Figure 30F) are automatically filled. If there is inactivity for more than a threshold time (e.g., 2 minutes and 40 seconds) between times, the cached authentication is not available for use by the electronic device 3000.

30P-30T illustrate an example that causes cached authentication to be unavailable for use by electronic device 3000. In FIG. 30P , electronic device 3000 detects an input (e.g., double-pressing) by finger 3054 on home button 3056. In response to detecting input, electronic device 3000 displays recently used apps view 3060, as shown in FIG. 30Q. 30R and 30S, the electronic device 3000 detects a swipe gesture 3062, which causes the application with the account user interface 3032 to close (e.g., terminate). As a result, the electronic device 3000 displays a recently used apps view 3060 that no longer includes applications with an account user interface 3032, as illustrated by FIG. 30T. Once the application receives a request to automatically populate fillable fields in login user interface 3038 (e.g., FIG. 30J), the fillable fields in login user interface 3012 (e.g., FIG. 30F) are automatically populated. If it is closed in between, the cached authentication is not available for use by the electronic device 3000.

30U and 30V illustrate an example that causes cached authentication to be unavailable for use by electronic device 3000. 30U, electronic device 3000 detects input (e.g., a single press) by finger 3064 on button 3004. In response to detecting input, electronic device 3000 transitions the device from an unlocked state to a locked state. Transitioning the device to a locked state includes deactivating (e.g., disabling) the display 3002, one or more biometric sensors 3003, and/or microphone of the electronic device 3000. In Figure 30V, electronic device 3000 is in a locked state and displays nothing on display 3002. Once the electronic device 3000 automatically fills the fillable fields in the login user interface 3038 (e.g., Figure 30J) and when the fillable fields in the login user interface 3012 (e.g., Figure 30F) are automatically filled. If the server transitions to a locked state between the time a request is received, the cached authentication is not available for use by the electronic device 3000.

In FIG. 30W , electronic device 3000 is configured to retrieve the event information after the cached authentication is no longer available (e.g., events discussed with respect to FIGS. 30N and 30O, 30P to 30T, or 30U and 30V). After one or more of the sequence), the account user interface 3032 is displayed. Electronic device 3000 detects a tap gesture 3034 on store affordance 3036. Electronic device 3000 identifies tap gesture 3034 as a request to load login user interface 3038 (shown in FIG. 30X). Electronic device 3000 may also perform a tap gesture 3034 as a request to automatically fill one or more fillable fields within login user interface 3038 (e.g., username field 3040 and password field 3042). It is identified as. As described previously, in some examples, the request also includes a request to automatically log the user in so that the user does not have to tap the submit affordance to submit credentials and log in.

In response to a request to automatically populate one or more fillable fields, a determination is made that a cached authentication is not available for use by electronic device 3000. In Figure 30y, the electronic device 3000 again performs biometric authentication using one or more biometric sensors 3003. Biometric authentication occurs automatically in response to receiving a request to automatically populate fillable fields so that no intermediate input from the user is required to initiate biometric authentication. If biometric authentication is successful (e.g., biometric authentication criteria are met), the electronic device automatically populates the fillable fields. If biometric authentication is not successful (eg, biometric authentication criteria are not met), electronic device 3000 withholds automatically populating fillable fields.

In Figure 30z, the user is not logged in and a user interface similar to Figure 30a is presented. Electronic device 3000 displays a user interface 3006 of the application on display 3002 . Electronic device 3000 detects tap gesture 3008 on login affordance 3010. Electronic device 3000 identifies tap gesture 3008 as a request to load login user interface 3012 (shown in Figure 30aa). Electronic device 3000 may also perform a tap gesture 3008 as a request to automatically fill one or more fillable fields within login user interface 3012 (e.g., username field 3014 and password field 3016). It is identified as. The auto-fill request requires biometric authentication to proceed with auto-filling the fillable fields. As described previously, in some examples, the request also includes a request to automatically log the user in so that the user does not have to tap the submit affordance to submit credentials and log in.

30AA, in response to a request to automatically populate fillable fields, electronic device 3000 uses biometric sensor 3003 to determine whether certain biometric authentication criteria have been met. Biometric sensor 3003 is non-contact, allowing the sensor to be configured to perform biometric authentication without physical input from the user. As a result, the electronic device 3000 initiates biometric authentication using biometric sensor 3003 without having to receive an explicit request from the user to initiate biometric authentication. Biometric authentication is not successful (eg, biometric authentication criteria are not met). As a result, the electronic device 3000 displays the failure interface 3066 in Figure 30ab upon failed biometric authentication, as described with respect to Figure 17m. Failure interface 3066 may be displayed when the user has not reached the maximum number of failed biometric authentication attempts.

In Figure 30ac, electronic device 3000 detects a tap gesture 3068 on cancel affordance 3070. In response to detecting tap gesture 3068, electronic device 3000 displays login user interface 3012 (shown in Figure 30A). 30A, in response to detecting a selection of username field 3014, electronic device 3000 displays cursor 3072 within username field 3014 and also displays virtual keyboard 3074. The electronic device 3000 receives input for entering one or more characters corresponding to credential information in the user name field 3014 and the password field 3016. In Figure 30ae, electronic device 3000 detects a tap gesture 3028 on submission affordance 3030. As a result, the electronic device 3000 submits credential information to the user name field 3014 and password field 3016 for user authentication. Upon successful user authentication, electronic device 3000 provides access to restricted content (e.g., content that can only be viewed when the user is logged in) within account user interface 3032 of Figure 30AF.

In Figure 30ag, electronic device 3000 detects a tap gesture 3034 on store affordance 3036. Electronic device 3000 identifies tap gesture 3034 as a request to load login user interface 3038 (shown in Figure 30A). Electronic device 3000 may also perform a tap gesture 3034 as a request to automatically fill one or more fillable fields within login user interface 3038 (e.g., username field 3040 and password field 3042). It is identified as. As described previously, in some examples, the request also includes a request to automatically log the user in so that the user does not have to tap the submit affordance to submit credentials and log in.

In response to a request to automatically populate one or more fillable fields, a determination is made that a cached authentication is not available for use by electronic device 3000. In Figure 30ah, the electronic device 3000 performs biometric authentication using the biometric sensor 3003. Biometric authentication occurs automatically in response to receiving a request to automatically populate fillable fields so that no intermediate input from the user is required to initiate biometric authentication.

In Figure 30AI, a determination is made that biometric authentication is successful (e.g., biometric authentication criteria are met). As a result, the electronic device 3000 displays the biometric authentication interface 3018 with a successful biometric authentication glyph 3026 indicating that biometric authentication was successful. Upon successful biometric authentication, electronic device 3000 automatically populates username field 3040 and password field 3042 with credential information, as shown in FIG. 30aj.

30aj, electronic device 3000 detects a tap gesture 3044 on submission affordance 3046. In response to detecting the tap gesture 3044, the electronic device 3000 submits credential information in the username field 3040 and password field 3042 for user authentication. Upon successful user authentication, electronic device 3000 provides access to restricted content (e.g., content that can only be viewed when the user is logged in) within store user interface 3048 of Figure 30AK.

In some examples, electronic device 3000 displays a biometric authentication interface with a biometric authentication glyph indicating that biometric authentication is being performed. In some examples, the displayed biometric authentication interface is approximately centered along the horizontal and/or vertical axis, as in Figures 30B-30E. In other examples, electronic device 3000 displays a biometric authentication interface on the top, bottom, side, or corner of display 3002. For example, electronic device 3000 displays biometric authentication interface 3076 near the top of display 3002, as shown in FIG. 30A. In some examples, electronic device 3000 does not display a biometric authentication interface while biometric authentication is being performed.

31A and 31B are flow diagrams illustrating a method for determining whether biometric re-authentication is required or a cached authentication is available for use using an electronic device, according to some examples. Method 3100 includes one or more biometric sensors (e.g., 3003) (e.g., a fingerprint sensor, a non-contact biometric sensor (e.g., a biometric sensor that does not require physical contact, such as a thermal or optical facial recognition sensor), iris). It is performed on a device (eg, 100, 300, 500, 1700, 3000) with a scanner. In some examples, one or more biometric sensors (eg, 3003) include one or more cameras. The electronic device (eg, 100, 300, 500, 1700, 3000) optionally includes a display (eg, 3002). In some examples, the display (eg, 3002) is a touch-sensitive display. In some examples, the display (eg, 3002) is not a touch-sensitive display.

Some operations of method 3100 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted. As described below, method 3100 provides an intuitive method for determining whether biometric re-authentication is required or whether cached authentication is available for use. This method reduces the user's cognitive burden of authenticating on an electronic device, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to authenticate faster and more efficiently saves power and increases time between battery charges.

The electronic device (e.g., 100, 300, 500, 1700, 3000) makes a first request (e.g., 3008) to perform a first action requiring authentication (e.g., select a password to automatically fill, unlock the device, make a payment) below) is received (3102).

In response to receiving a first request (e.g., 3008) to perform a first operation (3104), the electronic device (e.g., 100, 300, 500, 1700, 3000) proceeds to blocks 3106 through 3110. Proceed.

The electronic device (e.g., 100, 300, 500, 1700, 3000) uses one or more biometric sensors (e.g., 3003) to determine whether first biometric authentication criteria are met (3106). First biometric authentication criteria include a requirement that each type of biometric feature (eg, face or fingerprint) that is authorized to perform a first operation is detected by biometric sensors (eg, 3003).

Upon determining that the first biometric authentication criteria are met (e.g., at least a portion of the biometric characteristic, as determined based on data obtained from one or more biometric sensors (e.g., 3003) corresponding to the biometric characteristic Upon meeting the biometric authentication criteria (e.g., the detected face matches the authorized face), the electronic device (e.g., 100, 300, 500, 1700, 3000) performs a first operation (3108). Performing the first action upon request and successful authentication provides the user with the ability to perform the first action without requiring additional inputs (other than the request). Performing an action when a set of conditions is met without requiring additional user input (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device) It improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

Upon determining that the biometric authentication criteria are not met, the electronic device (e.g., 100, 300, 500, 1700, 3000) refrains from performing the first operation (3110). Suspending (or performing) each operation based on not meeting biometric authentication criteria can provide security and prevent unauthorized users from initiating sensitive operations. Providing improved security improves the operability of the device (e.g., by restricting unauthorized access) and makes the user-device interface more efficient, which additionally reduces the power of the device by limiting performance of restricted operations. Reduces usage and improves battery life.

After performing the first operation, the electronic device (e.g., 100, 300, 500, 1700, 3000) makes a second request to perform a second operation (e.g., the same or different from the first operation) requiring authentication. For example, 3034) (e.g., selecting a password to be automatically filled, unlocking the device, making a payment) is received (3112).

In response to receiving the second request (e.g., 3034) (3114), the electronic device (e.g., 100, 300, 500, 1700, 3000) proceeds to blocks 3116 and 3118.

Depending on the determination that the re-authentication criteria have been met (e.g., the cached authentication is not permitted for the second operation or the cached authentication is not available for use), the electronic device (e.g., 100, 300, 500, 1700) , 3000) uses one or more biometric sensors (e.g., 3003) to determine whether the second biometric authentication criteria are met (3116). The second biometric authentication criteria include a requirement that each type of biometric feature (eg, face or fingerprint) that is authorized to perform the second action is detected by biometric sensors (eg, 3003). In some examples, the first and second biometric authentication criteria are the same. In some examples, the first and second biometric authentication criteria are different. Performing biometric authentication when cached authentication is not available can provide security and prevent unauthorized users from initiating sensitive operations. Providing improved security improves the operability of the device (e.g., by restricting unauthorized access) and makes the user-device interface more efficient, which additionally reduces the power of the device by limiting performance of restricted operations. Reduces usage and improves battery life.

Upon determining that re-authentication criteria have not been met (e.g., a cached authentication is available for use), the electronic device (e.g., 100, 300, 500, 1700, 3000) performs a second operation without performing biometric authentication. Perform 3118 and withhold determining whether the second biometric authentication criteria are met using one or more biometric sensors (e.g., 3003). Performing the second action upon request without requiring re-authentication provides the user with the ability to perform the action without requiring additional inputs (besides the request). Performing an action when a set of conditions is met without requiring additional user input or re-authentication helps the user provide appropriate inputs (e.g., when operating/interacting with a device) and reduces user errors improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

According to some examples, the first operation and the second operation occur while the electronic device (eg, 100, 300, 500, 1700, 3000) is in an unlocked state. In some examples, determining whether second biometric authentication criteria are met using one or more biometric sensors (e.g., 3003) may be performed when the electronic device (e.g., 100, 300, 500, 1700, 3000) is locked. Occurs while in the released state. Performing biometric authentication while the device is in an unlocked state allows the device to provide feedback by displaying an indication of the progress of the biometric authentication. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

According to some examples, in response to receiving the second request (e.g., 3034) and determining that the second biometric authentication criteria are not met, the electronic device (e.g., 100, 300, 500, 1700, 3000) Hold off on performing the second operation.

According to some examples, performing the second action may include selecting each type of biometric feature authorized to perform the second action in response to receiving the second request (e.g., 3034) using biometric sensors (e.g., 3003) occurs regardless of whether it is detected or not.

According to some examples, the first action is logging on to a first web domain (e.g., 3028), and the second action is logging on to a second web domain corresponding to the first web domain (e.g., 3044). In some examples, the second web domain is the same as the first web domain. In some examples, the second web domain is a subdomain of the first web domain.

According to some examples, the re-authentication criteria are such that the device (e.g., 100, 300, 500, 1700, 3000) is in a locked state between when the first operation is performed and the second request (e.g., 3034) is received. (e.g., the cached authentication is not available for use) (e.g., the device remains unlocked between when the first action is performed and when the second request is received). Recertification criteria are not met and biometric verification is not required again).

According to some examples, a first operation is performed in an application, and the re-authentication criteria are such that the application has been closed (e.g., terminated) between when the first operation is performed and when the second request (e.g., 3034) is received. ) requirements (e.g., cached authentication is not available for use) (e.g., re-authentication criteria when the application remains open between when the first operation is performed and when the second request is received) are not met and biometric authentication is not required again).

According to some examples, the re-authentication criteria include that the application is inactive (e.g., for more than a threshold time (e.g., 2 minutes and 40 seconds) between when the first operation is performed and when the second request (e.g., 3034) is received. , pending state, sleeping state, background state, not active state) (e.g., cached authentication is not available for use) (e.g., when the first operation is performed and the second request If the application remains active between the time the re-authentication criteria are met and biometric authentication is not required again).

According to some examples, determining whether second biometric authentication criteria are met using one or more biometric sensors (e.g., 3003) may include receiving a second request to perform a second action requiring authentication (e.g., 3034) occurs automatically (e.g., without the need for intermediate input from the user).

According to some examples, one or more biometric sensors (e.g., 3003) may be non-contact biometric sensors (e.g., infrared cameras, visible light cameras, or combinations thereof) configured to perform biometric authentication without physical contact from the user. ) (e.g., one or more biometric sensors (e.g., 3003) may perform biometric authentication without physical input (e.g., touch or button press) from the user).

According to some examples, an electronic device (e.g., 100, 300, 500, 1700, 3000) displays an indication (e.g., 3076) on a display (e.g., 3002) that biometric authentication is being performed (e.g., a small indicator is displayed on the top, bottom, side, or corner). Displaying a small indicator away from the center of the display provides the user with an indication of the progress of the biometric authentication without diverting the user's focus and obstructing or confusing the display, thereby providing improved visual feedback during authentication. do. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, no indicator is displayed during biometric authentication. In some examples, an electronic device (e.g., 100, 300, 500, 1700, 3000) uses one or more biometric sensors (e.g., 3003) to identify a biometric device while determining whether biometric authentication criteria are met. Reserve displaying an indication that measurement authentication is being performed.

Note that the details of the processes described above with respect to method 3100 (e.g., FIGS. 31A and 31B) are also applicable in a similar manner to the methods described below and above. For example, method 3100 may optionally have features of various methods described above with reference to methods 800, 1000, 1200, 1400, 1600, 2000, 2200, 2500, 2700, 2900, 3300, 3500. Contains one or more of For example, registered biometric data described in method 1200 may be used to perform biometric authentication as described in connection with method 3100. As a further example, the visibility criteria of method 3300 can be used in conjunction with method 3100 to control when biometric authentication should be performed (or performed again). For brevity, these details are not repeated below.

32A-32W illustrate example user interfaces for automatically populating fillable fields based on visibility criteria, according to some examples. As described in more detail below, illustrative examples of the user interfaces illustrated in Figures 32A-32W are used to illustrate the processes described below, including the processes in Figure 33.

FIG. 32A illustrates electronic device 3200 (e.g., portable multifunction device 100, device 300, or device 500). In the illustrative examples illustrated in FIGS. 32A-32W, electronic device 3200 is a smartphone. In other examples, electronic device 3200 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 3200 includes a display 3202, one or more input devices (e.g., a touchscreen and microphone of display 3202), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In Figure 32A, the electronic device includes a biometric sensor 3203. In some examples, the biometric sensor is one or more biometric sensors that may include a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, biometric sensor 3203 is biometric sensor 703. In some examples, the one or more biometric sensors include one or more fingerprint sensors (eg, a fingerprint sensor integrated into a button). In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In FIG. 32A , electronic device 3200 displays a user interface 3204 of an application on display 3202 . The application is a mobile browser application, and the interface corresponds to a website (airline.com). 32B-32D, electronic device 3200 does not initiate biometric authentication because the login user interface does not meet certain visibility criteria. For example, visibility criteria may include whether a critical mass of one or more fillable fields (e.g., corresponding to credential information) are displayed within the visible area of the user interface.

32B and 32C, electronic device 3200 detects a scroll gesture 3206 in upward motion. In response to detecting scroll gesture 3206, electronic device 3200 causes user interface 3204 to scroll down. In Figure 32D, following scroll gesture 3206, electronic device 3200 displays scrolled user interface 3208. Electronic device 3200 did not initiate biometric authentication due to determining that visibility criteria were still not met.

In Figure 32E, electronic device 3200 detects a tap gesture 3210 on hidden menu affordance 3212. In response to detecting the tap gesture 3210, the electronic device 3200 displays a hidden menu 3214 that contains one or more fillable fields (e.g., a user name), as shown in Figure 32F. field 3216 and password field 3218). The electronic device 3200 displays a hidden menu 3214 that is overlaid on the scrolled user interface 3208 such that the covered portion of the scrolled user interface 3208 is no longer displayed. A determination is made as to whether the fillable fields meet certain visibility criteria.

If it is determined that the fillable fields meet predetermined visibility criteria, the electronic device 3200 may display the fillable fields within hidden menu 3214 with credential information (e.g., a user name that allows the user to successfully log in to the account). and password) are automatically filled in. Auto-fill requests require biometric authentication to proceed with auto-filling fillable fields. In some examples, the request also includes a request to automatically log the user in so that the user does not have to tap a submission affordance (e.g., submission affordance 3232 in Figure 32L) to submit credentials and log in.

32G, upon determining that one or more fillable fields meet certain visibility criteria, electronic device 3200 uses biometric sensor 3203 to determine whether certain biometric authentication criteria have been met. Electronic device 3200 may capture and process (e.g., analyze) biometric data from biometric sensor 3203 to determine, based on the biometric data, biometric features (or portions thereof) based on biometric authentication criteria. determine whether the biometric data is satisfied (e.g., determine whether the biometric data matches the biometric template, within a threshold). Biometric authentication occurs automatically once a determination is made that the fillable fields meet visibility criteria. Biometric sensor 3203 is non-contact, allowing the sensor to be configured to perform biometric authentication without physical contact from the user. As a result, electronic device 3200 initiates biometric authentication using biometric sensor 3203 without receiving an explicit request from the user to initiate biometric authentication. In some examples, initiating biometric authentication includes detecting contact with one or more fingerprint sensors and determining whether the contact meets certain fingerprint authentication criteria (e.g., secure element 115 and determining whether the fingerprint matches an enrolled fingerprint as previously discussed in connection therewith; matching the fingerprint to stored information as previously discussed in connection with methods 1600, 1800, 2200, and FIGS. 17O and 21. determines whether or not it is possible). In some examples, determining whether a fingerprint matches an enrolled fingerprint may be performed according to one or more of the methods discussed in US Patent Application Publication No. 2015/0146945 (e.g., in paragraphs [0119] to [0121]). It is carried out. US Patent Application Publication No. 2015/0146945 is incorporated herein by reference, particularly with respect to its disclosure of methods for determining whether a fingerprint matches an enrolled fingerprint.

Alternatively, a determination may be made that one or more fillable fields do not meet certain visibility criteria. If the fillable fields do not meet visibility criteria, electronic device 3200 withholds from initiating biometric authentication.

Performing biometric authentication includes displaying a biometric authentication interface 3220 with a biometric authentication glyph 3222. Biometric authentication glyph 3222 is a simulation of a representation of a biometric feature (e.g., face). As shown in FIG. 32G, biometric authentication interface 3220 is overlaid on at least a portion of hidden menu 3214. Biometric authentication interface 3220 is optionally an operating system level interface (e.g., an interface created by the device's operating system) and hidden menu 3214 is an application level interface (e.g., separate from the device's operating system). a user interface created by a private third-party application). In some examples, the displayed biometric authentication interface is approximately centered along the horizontal and/or vertical axis, as in Figures 32G-32J. In some examples, electronic device 3200 displays a biometric authentication interface on the top, bottom, side, or corner of display 3202. For example, electronic device 3200 displays a biometric authentication interface near the top of display 3202, for example in Figure 30A. In some examples, electronic device 3200 does not display a biometric authentication interface while biometric authentication is being performed.

32H, electronic device 3200 displays a portion of a biometric authentication animation that includes biometric authentication glyph 3224, which is a portion of an animation in which biometric sensor 3203 acquires biometric data. It plays a role as Once electronic device 3200 has acquired biometric data (e.g., has acquired sufficient biometric data), electronic device 3200 displays biometric authentication glyph 3226, as shown in Figure 32I. transition to Electronic device 3200 displays biometric authentication glyph 3226 to indicate that biometric data is being processed. In some examples, biometric authentication glyph 3226 includes a plurality of rings that rotate in a sphere, for example, while being displayed.

In Figure 32J, a determination is made that biometric authentication was successful. As a result, electronic device 3200 displays successful biometric authentication glyph 3228 indicating that biometric authentication was successful.

32K, because the biometric authentication was successful, the electronic device 3200 fills one or more fillable fields (e.g., username field 3216 and password field 3218) with credential information (e.g., the user's account Automatically fills in your login information (such as username and password) to enable you to log in successfully. In some examples, electronic device 3200 automatically populates fillable fields with credit card information (eg, information associated with payment account information).

Alternatively, a determination may be made that biometric authentication criteria have not been met. If biometric authentication fails, electronic device 3200 refrains from automatically filling one or more fillable fields with credential information (eg, login information or credit card information). Withholding automatically populating one or more fillable fields optionally includes displaying a failure interface, such as failure interface 2854 in Figure 28X or failure interface 2856 in Figure 28Y.

32L, the electronic device detects a tap gesture 3230 on the submission affordance 3232. In response to detecting tap gesture 3230, electronic device 3200 submits credential information in username field 3216 and password field 3218 for user authentication. Upon successful authentication, electronic device 3200 provides access to restricted content (e.g., content that can only be viewed when the user is logged in) within member user interface 3234 of FIG. 32M.

Figures 32N-32W illustrate various scenarios where certain visibility criteria are not initially met and then, following user input, the visibility criteria are met.

In Figure 32N, electronic device 3200 displays a user interface 3236 of an application on display 3202. The application is a mobile browser application, and the interface corresponds to a website (restaurant.com). User interface 3236 includes one or more fillable fields (e.g., username field 3238 and password field 3240). Fillable fields are displayed within the viewable area of user interface 3236 at a size that is less than a threshold size (e.g., a threshold size that must be met or exceeded to meet certain visibility criteria).

32O, while displaying user interface 3236, electronic device 3200 detects zoom gesture 3242. In response to detecting zoom gesture 3242, electronic device 3200 displays an enlarged user interface 3244, as shown in FIG. 32P. The expanded user interface 3244 includes an expanded username field 3246 and an expanded password field 3248. A determination is made that the zoom gesture 3242 did not cause the fillable fields to meet certain visibility criteria. For example, fillable fields are still displayed at a size that is less than the threshold size within the viewable area of the enlarged user interface 3244. Upon determining that the visibility criteria are still not met, electronic device 3200 withholds from initiating biometric authentication.

In FIG. 32Q, while displaying zoomed-in user interface 3244, electronic device 3200 detects zoom gesture 3250. In response to detecting the zoom gesture 3250, the electronic device 3200 displays an enlarged user interface 3252, as shown in Figure 32R. Expanded user interface 3252 includes an expanded username field 3254 and an expanded password field 3256. A determination is made that the zoom gesture 3250 caused the fillable fields to meet visibility criteria. For example, fillable fields are now displayed at a size that is greater than or equal to a threshold size within the viewable area of the enlarged user interface 3252. Upon determining that the visibility criteria are met, electronic device 3200 initiates biometric authentication and displays biometric authentication interface 3220 with biometric authentication glyph 3222, as described with respect to FIG. 32G. do.

In Figure 32S, electronic device 3200 displays an application's user interface 3258 on display 3202. The application is a mobile browser application, and the interface corresponds to a website (newsfeed.com). User interface 3258 is a user interface area corresponding to a portion of an electronic document (eg, HTML document). The electronic document includes one or more fillable fields that are outside the viewable area of user interface 3258 (e.g., username field 3268 and password field 3270 in Figure 32W).

32T, while displaying user interface 3258, electronic device 3200 detects a scroll gesture 3260 in upward motion. In response to detecting scroll gesture 3260, electronic device 3200 causes user interface 3258 to scroll down. 32U, following scroll gesture 3260, electronic device 3200 displays a scrolled user interface 3262, which displays a portion of one or more fillable fields (e.g., user name field 3268). It includes doing. A determination is made that scroll gesture 3260 did not cause one or more fillable fields to meet certain visibility criteria. For example, visibility criteria include whether a critical mass of one or more fillable fields are displayed within the visible area of scrolled user interface 3262. Upon determining that the visibility criteria are still not met, electronic device 3200 withholds from initiating biometric authentication.

32V, while displaying scrolled user interface 3262, electronic device 3200 detects scroll gesture 3264. In response to detecting scroll gesture 3264, electronic device 3200 causes scrolled user interface 3262 to scroll further down. 32W, following the scroll gesture 3264, the electronic device 3200 displays a scrolled user interface 3266, which includes one or more fillable fields (e.g., a username field 3268 and a password field ( 3270)). A determination is made that scroll gesture 3264 caused the fillable fields to meet visibility criteria. For example, a threshold amount of one or more fillable fields are now displayed within the visible area of scrolled user interface 3262. Upon determining that the visibility criteria are met, electronic device 3200 initiates biometric authentication and displays biometric authentication interface 3220 with biometric authentication glyph 3222, as described with respect to FIG. 32G. do.

33 is a flow diagram illustrating a method for determining when to perform an authentication operation using an electronic device, according to some examples. Method 3300 is performed at a device (eg, 100, 300, 500, 1700, 3200) having a display (eg, 3202). In some examples, the display (eg, 3202) is a touch-sensitive display. In some examples, the display (eg, 3202) is not a touch-sensitive display.

Some operations of method 3300 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted. As described below, method 3300 provides an intuitive method for determining when to perform an authentication operation. This method reduces the user's cognitive burden for performing authentication operations, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to authenticate faster and more efficiently saves power and increases time between battery charges.

An electronic device (e.g., 100, 300, 500, 1700, 3200) receives a request (e.g., 3210) to display a first portion (e.g., 3214) of respective content (e.g., 3208) (e.g., loading a web page) , a request to scroll the web page, or zoom the web page) is received (3302).

In response 3304 to a request (e.g., 3210) to display a first portion (e.g., 3214) of the respective content (e.g., 3208), the electronic device (e.g., 100, 300, 500, 1700, 3200) displays a block Proceeds from block 3306 to block 3310.

The electronic device (e.g., 100, 300, 500, 1700, 3200) displays at least a first portion (e.g., 3214) of each content (e.g., 3208) on a display (e.g., 3202) (e.g., a user-interactive A region of an electronic document (eg, HTML document) having elements is displayed (3306). Each content (e.g., 3208) optionally includes elements (e.g., 3216, 3218) associated with an authentication operation (e.g., one or more fillable fields, such as a credit card entry field, username and password fields for logging into the service). (including login user interface elements).

Upon determining that the element associated with the authentication operation (e.g., 3216, 3218) meets visibility criteria (e.g., the element associated with the authentication operation is entirely outside the visible area of the content, and at least a critical mass of the element associated with the authentication operation is within the content) is outside the visible area of the content, such that the element associated with the authentication operation is displayed at a size that is less than a threshold size within the visible area of the content, and/or the element associated with the operation is contained within a folded menu area or other hidden element. contained within a portion of content that is hidden from view) (e.g., at least a portion of one or more fillable fields is displayed, one or more fillable fields are fully displayed, and/or one or more fillable fields are larger than a threshold size) , the electronic device (e.g., 100, 300, 500, 1700, 3200) initiates biometric authentication (e.g., as described with reference to FIGS. 17G and 17H) (3308). In some examples, a region corresponds to a portion of an electronic document (e.g., an HTML document), and one or more fillable fields are one or more fillable fields of the electronic document that have properties that cause one or more elements of the electronic document to be rendered in a visible state. elements (e.g., HTML elements with the "style.display" attribute of "inline"). Initiating biometric authentication upon meeting the visibility criteria provides the user with the ability to perform biometric authentication without requiring additional inputs (other than the input that causes the visibility criteria to be met). Performing biometric authentication when a set of conditions is met without requiring additional user input (e.g., by helping the user provide appropriate inputs when operating/interacting with a device and reducing user errors) ) improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

Upon determining that an element associated with the authentication operation (e.g., 3216, 3218) does not meet the visibility criteria (e.g., the element associated with the authentication operation is displayed entirely within the visible area of the content, and at least a critical mass of the element associated with the authentication operation is within the content) is displayed within the visible area of the content, the element associated with the authentication operation is displayed at a size that exceeds a threshold size within the visible area of the content, and/or the element associated with the operation is contained within a folded menu area or other hidden element. Likewise contained within a portion of content that is not otherwise hidden from view), the electronic device (e.g., 100, 300, 500, 1700, 3200) refrains from initiating biometric authentication (3310). Withholding from initiating biometric authentication based on not meeting visibility criteria prevents biometric authentication from occurring when the user did not intend for the device to initiate biometric authentication. Preventing unintentional biometric authentication improves the operability of the device (e.g., by limiting unauthorized access) and makes the user-device interface more efficient, which additionally improves the device's performance by restricting performance of restricted operations. Reduces power usage and improves battery life.

According to some examples, a first portion (e.g., 3214) of each content (e.g., 3208) is displayed without displaying a second portion of each content on a display (e.g., 3202). In some examples, the second portion is displayed prior to displaying the first portion.

According to some examples, biometric authentication may be performed automatically (e.g., without requiring intermediate input to initiate biometric authentication) based on a determination that an element associated with the authentication operation (e.g., 3216, 3218) meets visibility criteria. Occurs.

According to some examples, while displaying a first portion of respective content (e.g., 3236, 3258), the electronic device (e.g., 100, 300, 500, 1700, 3200) receives an input (e.g., 3242, 3250, 3260, 3264) (e.g., zoom, scroll, menu display). In response to detecting input (e.g., 3250, 3264) and determining that the input causes an element associated with an authentication operation (e.g., 3246, 3248, 3268, 3270) to meet visibility criteria, the electronic device (e.g. , 100, 300, 500, 1700, 3200) initiates biometric authentication. Initiating biometric authentication in response to input and upon meeting visibility criteria provides the user with the ability to perform biometric authentication without requiring additional inputs (other than the input that causes the visibility criteria to be met). Performing biometric authentication when a set of conditions is met without requiring additional user input (e.g., by helping the user provide appropriate inputs when operating/interacting with a device and reducing user errors) ) improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In response to detecting the input (e.g., 3242, 3260) and determining that the input does not cause an element associated with the authentication operation (e.g., 3238, 3240, 3268, 3270) to meet visibility criteria, the electronic device ( For example, 100, 300, 500, 1700, 3200) suspends initiating biometric authentication. In some examples, input (e.g., 3210, 3242, 3250, 3260, 3264) (e.g., selection of an affordance, request to resize) may be combined with an element associated with an authentication operation (e.g., 3216, 3218, 3246, 3248, 3268, 3270). ), causing the element to transition from not meeting the visibility criteria to meeting the visibility criteria. Withholding from initiating biometric authentication based on not meeting visibility criteria prevents biometric authentication from occurring when the user did not intend for the device to initiate biometric authentication. Preventing unintentional biometric authentication improves the operability of the device (e.g., by limiting unauthorized access) and makes the user-device interface more efficient, which additionally improves the device's performance by restricting performance of restricted operations. Reduces power usage and improves battery life.

According to some examples, the input (e.g., 3242, 3250) is a request to perform a zoom operation, and the visibility criteria are such that the element associated with the authentication operation (3238, 3240, 3246, 3248, 3254, 3256) is larger than a threshold size. Includes the requirement that it has size.

According to some examples, the input 3260, 3264 is a request to perform a scroll operation, and the visibility criteria are such that at least a predetermined amount of the element 3268, 3270 associated with the authentication operation is displayed on the display (e.g., 3202). Includes requirements.

According to some examples, the input (e.g., 3210) is a request to perform a hidden interface area display operation (e.g., a request to display a hidden menu or other hidden interface area) and the visibility criteria are an element associated with the authentication operation (e.g., 3216, 3218) is not specified for display in a hidden interface area.

According to some examples, the electronic device (e.g., 100, 300, 500, 1700, 3200) further includes one or more biometric sensors (e.g., 3203), and initiating biometric authentication includes one or more biometric authentication. and initiating biometric authentication using sensors (e.g., 3203).

According to some examples, one or more biometric sensors (e.g., 3203) may include one or more non-contact biometric sensors (e.g., an infrared camera, a visible light camera, or a combination thereof) configured to perform biometric authentication without physical contact from the user. combinations) (e.g., one or more biometric sensors (e.g., 3203) may perform biometric authentication without physical input (e.g., touch or button press) from the user. Initiating biometric authentication occurs without receiving an explicit request to initiate biometric authentication.

According to some examples, one or more biometric sensors (eg, 3203) include one or more facial recognition sensors. Initiating biometric authentication includes determining whether facial authentication criteria have been met using one or more facial recognition sensors (e.g., as described with respect to FIGS. 23D-23F).

According to some examples, one or more biometric sensors (eg, 3203) include one or more fingerprint sensors. Initiating biometric authentication involves detecting contact with one or more fingerprint sensors and determining whether the contact meets fingerprint authentication criteria (e.g., the fingerprint matches an enrolled or authorized fingerprint). Includes.

According to some examples, initiating biometric authentication includes displaying a progress indicator (e.g., 3222, 3224, 3226, 3228) on a display (e.g., 3202) indicating the status of the biometric authentication process. . In some examples, the progress indicator is a simulated progress indicator (e.g., a portion of the features of the progress indicator that is displayed surrounding a simulation of the biometric feature, such as a plurality of progress elements distributed around a representation of the user's biometric feature). or a progress indicator with all). In some examples, a small progress indicator is displayed on the top, bottom, side, or corner. Displaying a small indicator away from the center of the display provides the user with an indication of the progress of the biometric authentication without diverting the user's focus and obstructing or confusing the display, thereby providing improved visual feedback during authentication. do. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, no indicator is displayed during biometric authentication. In some examples, no progress indicator is displayed during biometric authentication. In some examples, the electronic device (eg, 100, 300, 500, 1700, 3200) refrains from displaying a progress indicator indicating the status of the biometric authentication process.

According to some examples, the element associated with the authentication operation is a fillable field (e.g., 3216, 3218, 3254, 3256, 3268, 3270) (e.g., a username, password, credential, or payment information input field). In response to initiating biometric authentication and upon determining that the biometric authentication criteria have been met, the electronic device (e.g., 100, 300, 500, 1700, 3200) enters a fillable field (e.g., 3216, 3218, 3254, 3256, 3268, 3270) automatically populates fields with credential information (e.g., fields such as user name, password, credit card information, or other sensitive information) on an electronic device (e.g., 100, 300, 500, 1700, 3200) Populating with data accessible by or stored by an electronic device (e.g., 100, 300, 500, 1700, 3200). In response to initiating biometric authentication and upon determining that biometric authentication criteria have not been met, the electronic device (e.g., 100, 300, 500, 1700, 3200) may populate a fillable field (e.g., 3216, 3218, 3254, 3256, 3268, 3270) are automatically filled with credential information.

According to some examples, in response to initiating biometric authentication and upon determining that biometric authentication criteria have been met, the electronic device (e.g., 100, 300, 500, 1700, 3200) may provide access to restricted content (e.g., logging into a web page (e.g., 3234), displaying a list of passwords associated with the user, displaying credit card information). In response to initiating biometric authentication and determining that biometric authentication criteria have not been met, the electronic device (e.g., 100, 300, 500, 1700, 3200) withholds providing access to the restricted content.

According to some examples, credential information includes login information (eg, information needed to log in to an application, account, or website).

According to some examples, credential information includes information associated with payment account information (eg, credit card, bank account, or payment service information).

Note that the details of the processes described above with respect to method 3300 (e.g., FIG. 33) are also applicable in a similar manner to the methods described below and above. For example, method 3300 may optionally have features of various methods described above with reference to methods 800, 1000, 1200, 1400, 1600, 2000, 2200, 2500, 2700, 2900, 3100, 3500. Contains one or more of For example, registered biometric data described in method 1200 may be used to perform biometric authentication as described in connection with method 3300. As a further example, the recertification criteria of method 3100 can be used in conjunction with method 3300 to control when biometric authentication should be performed (or performed again). For brevity, these details are not repeated below.

Figures 34A-34N illustrate example user interfaces for automatic login using biometric authentication, according to some examples. As described in more detail below, illustrative examples of the user interfaces illustrated in Figures 34A-34N are used to illustrate the processes described below, including the processes in Figure 35.

FIG. 34A illustrates electronic device 3400 (e.g., portable multifunction device 100, device 300, device 500, or device 1700). In the illustrative examples illustrated in FIGS. 34A-34N, electronic device 3400 is a smartphone. In other examples, electronic device 3400 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 3400 includes a display 3402, one or more input devices (e.g., a touchscreen and microphone of display 3402), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In Figure 34A, the electronic device includes a biometric sensor 3403. In some examples, the biometric sensor is one or more biometric sensors that may include a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, a biometric sensor may be a depth camera or device 100 having one or more features and/or the functions of a depth camera as described with respect to certain examples of biometric sensor 703 and device 700. ) is the depth camera 175. In some examples, biometric sensor 3403 may be used in conjunction with a visible light camera to determine a depth map of different portions of an object captured by the visible light camera, as described above with respect to biometric sensor 703. It's a camera.

As can be seen in FIG. 34A , electronic device 3400 also includes a fingerprint sensor 3414 (e.g., a biometric sensor) integrated into the button. In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In some examples, electronic device 3400 is one of devices 100, 300, 500, 700, 900, 1100, 1300, 1500, 1700, 1900, 2100, 2300, 2400, 2600, 2800, 3000, 3200, 3600, 38. 00 , 4000, 4200), each of which shares one or more features, elements, and/or components with another of the devices. may (e.g., device 700 may include components of device 3200 and vice versa). For example, biometric sensor 3403 may be biometric sensor 903, or biometric sensor 1103 may be biometric sensor 1303. As another example, button-integrated fingerprint sensor 3414 may be fingerprint sensor 1764. For another example, display 3402 may be display 1302, or display 1502 may be display 2102.

Before displaying login user interface 3404 in Figure 34A, electronic device 3400 detects a request to load login user interface 3404. In response to detecting the request, a determination is made as to whether biometric authentication using fingerprint sensor 3414 is available. Upon determining that biometric authentication is available, the electronic device 3400 may submit credential information to a submission affordance 3420 (e.g., upon selection, one or more fillable fields (e.g., a username field or a password field)). Displays a login user interface 3404 with a prompt 3406 (“SCAN FINGER TO LOGIN”) located within the affordance of submitting). Prompt 3406 directs the user to use the fingerprint sensor ( Placing their finger on 3414 causes credential information (e.g., a username and password that allows the user to successfully log in to the account) to be submitted through username field 3408 and password field 3410. Additionally, username field 3408 is pre-populated with a default username (e.g., [email protected]), as shown in Figure 34A. In some examples, the username field is populated with the username. It is not pre-filled.

34B, while displaying login user interface 3404, electronic device 3400 uses fingerprint sensor 3414 to detect finger 3412. In response to detecting finger 3412, a determination is made that the finger meets certain biometric authentication criteria (e.g., the fingerprint matches an enrolled fingerprint). Upon successful authentication, in Figure 34C, electronic device 3400 automatically enters credential information in user name field 3408 and/or password field 3410 and submits the credential information for user authentication ( For example, submitting the information without requiring further input from the user). Upon successful user authentication, electronic device 3400 provides access to restricted content (e.g., content that can only be viewed when the user is logged in) within account user interface 3416 of FIG. 34D.

Alternatively, if a determination is made that the finger does not meet certain biometric authentication criteria, the electronic device 3400 may withhold entering and submitting credential information and return to the failure interface, as shown in FIG. 34E. 3418) is displayed. Failure interface 3418 may be displayed when the user has reached a maximum number of failed biometric authentication attempts. If the maximum number of failed biometric authentication attempts has been reached, fingerprint sensor 3414 is not available for biometric authentication.

34F shows the login displayed when biometric authentication using fingerprint sensor 3414 is not available (e.g., when such authentication is disabled through a user-selectable setting or when the maximum number of attempts has been exceeded) Illustrative user interface 3404-1. In response to detecting a request to display a login user interface, a determination is made that biometric authentication using fingerprint sensor 3414 is not available. Upon making this determination, electronic device 3400 displays login user interface 3404-1 without prompt 3406. Electronic device 3400 displays submission affordance 3420-1 in its original, unmodified state, where text 3422 (“LOGIN”) is displayed within submission affordance 3420-1.

In some examples, electronic device 3400 does not immediately display prompt 3406 in response to a request to load the login user interface. Instead, the electronic device 3400 displays a prompt 3406 after receiving a selection of a fillable field to enter text. In Figure 34G, electronic device 3400 initially displays login user interface 3404-1 without prompt 3406. Electronic device 3400 detects a tap gesture 3424 on password field 3410. In response to detecting the tap gesture 3424, the electronic device 3400 displays a virtual keyboard 3426 (e.g., a keyboard for entering one or more characters) and a password field 3410, as shown in Figure 34H. Displays the cursor 3428 within. Additionally, in response to the tap gesture 3424, the electronic device 3400 displays a login user interface 3404 with a prompt 3406 positioned within the submission affordance 3420.

In FIG. 34I , the electronic device 3400 receives an input corresponding to input of one or more characters (eg, character 3430) through the virtual keyboard 3426. In response to receiving input of text 3430, electronic device 3400 again displays login user interface 3404-1 without prompt 3406. Electronic device 3400 displays submission affordance 3420-1 in its original, unmodified state, where text 3422 is displayed within submission affordance 3420-1.

34J, after receiving input via virtual keyboard 3426, electronic device 3400 detects a tap gesture 3432 on submission affordance 3420-1. In response to detecting tap gesture 3432, electronic device 3400 submits one or more characters to username field 3408 and password field 3410 for user authentication. Upon successful user authentication, electronic device 3400 provides access to restricted content (e.g., content that can only be viewed when the user is logged in) within account user interface 3416 of Figure 34K.

In some examples, successful user authentication via fingerprint authentication (as described with respect to FIGS. 34A-34D) results in certain consequences (eg, access to restricted applications, web pages, or accounts). In some examples, successful user authentication by typing and submitting credential information (as described with respect to FIGS. 34F-34K) may result in the same result (e.g., access to a restricted application, web page, or account). cause

FIG. 34L illustrates that prompt 3406 may be displayed in different locations on the login user interface and some elements discussed with respect to interfaces 3404 and 3404-1 may be omitted. 34L, in response to detecting a request to display a login user interface and if biometric authentication is available, electronic device 3400 displays a prompt 3406 within password field 3410 and submits a submission affordance (e.g. , 3420, 3420-1) displays the non-displayed login user interface 3404-2.

34M, while displaying login user interface 3404-2, electronic device 3400 uses fingerprint sensor 3414 to detect finger 3412. In response to detecting finger 3412, a determination is made that the finger meets certain biometric authentication criteria (e.g., the fingerprint matches an enrolled fingerprint). Upon successful authentication, electronic device 3400 automatically submits credential information for user authentication. Upon successful user authentication, electronic device 3400 provides access to restricted content (eg, content that can only be viewed when the user is logged in).

Alternatively, if a determination is made that the finger does not meet certain biometric authentication criteria (e.g., the fingerprint does not match an enrolled fingerprint), the electronic device 3400 withholds submitting credential information. Additionally, upon failed biometric authentication, electronic device 3400 displays login user interface 3404-1 including previously hidden submission affordances (e.g., 3420, 3420-1) (shown in FIG. 34N). do. Additionally, upon failed biometric authentication, electronic device 3400 prompts the user for manual input by displaying a cursor 28 within a fillable field, such as password field 3410.

Figure 35 is a flow diagram illustrating a method for indicating availability of biometric authentication using an electronic device, according to some examples. Method 3500 includes a display (e.g., 3402) and one or more biometric sensors (e.g., 3403, 3414) (e.g., a fingerprint sensor, a non-contact biometric sensor (e.g., a thermal or optical facial recognition sensor) that requires physical contact. It is performed on devices (e.g., 100, 300, 500, 1700, 3400) that do not require biometric sensors) and iris scanners. In some examples, one or more biometric sensors (eg, 3403) include one or more cameras. In some examples, the display (eg, 3402) is a touch-sensitive display. In some examples, the display (eg, 3402) is not a touch-sensitive display.

Some operations of method 3500 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted. As described below, method 3500 provides an intuitive way to indicate the availability of biometric authentication. This method reduces the cognitive burden on the user for determining the availability of biometric authentication, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to identify the availability of biometric authentication faster and more efficiently saves power and increases time between battery charges.

The electronic device (e.g., 100, 300, 500, 1700, 3400) has a credential submission (e.g., login) user interface element (e.g., 3408, 3410) (e.g., a fillable field such as a username or password). Detect 3502 a predefined action (e.g., a request to load and/or display a user interface, selection of a particular element of the user interface) corresponding to a credential submission (e.g., login) user interface (e.g., 3404). .

In response to detecting the predefined operation 3504, the electronic device (e.g., 100, 300, 500, 1700, 3400) proceeds to blocks 3506 through 3516.

In response to detecting a predefined action (3504) and upon determining that biometric authentication (e.g., Touch ID, Face ID) via one or more biometric sensors (e.g., 3403, 3414) is available (3506) ), the electronic device (e.g., 100, 300, 500, 1700, 3400) displays, on a display (e.g., 3402), one or more biometric sensors to display biometric features (e.g., 3412) that meet biometric authentication criteria. Credential submission (e.g., 3406) with a visual indication (e.g., 3406) that presenting credentials (e.g., 3403, 3414) will cause the credentials to be submitted via a credential submission user interface element (e.g., 3408, 3410) Display 3508 a user interface (e.g., 3404) (e.g., login).

In some examples, the credential submission user interface is generated based on an electronic document (eg, an HTML document), and the credential submission user interface element is an input element (eg, a login button) for submitting credentials. In some examples, when biometric authentication is available, the electronic device (e.g., 100, 300, 500, 1700, 3400) places the credential input element in a first state (e.g., default state, biometric authentication is not available). Rather than rendering and displaying a biometric authentication element (e.g., instructions to provide the required biometric authentication input (e.g., a fingerprint) instead of a credential submission user interface element), Display. Displaying a prompt indicating to the user that placing their finger on the sensor will automatically cause them to log in provides the user with feedback regarding the current state of the device (e.g., biometric authentication is available), Provide feedback to users indicating efficient options for logging in. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

According to some examples, in response to detecting a predefined action (3504) and in response to determining that biometric authentication via one or more biometric sensors (e.g., 3403, 3414) is available (3506), the electronic device (e.g., 100, 300, 500, 1700, 3400) withholds from displaying the credential submission affordance (e.g., 3420) on the display (e.g., 3402) (e.g., does not display a login button) (3516) . Omitting the display of a login button encourages users to seek efficient ways to log in, thereby providing improved feedback. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

According to some examples, in response to detecting a predefined action (3504) and in response to determining that biometric authentication via one or more biometric sensors (e.g., 3403, 3414) is not available (3510), the electronic A device (e.g., 100, 300, 500, 1700, 3400) may display a credential submission (e.g., login) user interface (e.g., 3404- 1) is displayed (3512). Withholding the display of a prompt to log in via biometric authentication provides the user with feedback regarding the current state of the device because it indicates to the user that log in via biometric authentication is not available. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

According to some examples, further in response to detecting a predefined action (3504) and in response to a determination (3510) that biometric authentication via one or more biometric sensors (e.g., 3403, 3414) is not available. ), an electronic device (e.g., 100, 300, 500, 1700, 3400) displays a credential submission affordance (e.g., 3420) (e.g., a touch-activated device associated with one or more fillable fields) on a display (e.g., 3402). Login button) is displayed (3514). Receiving input (e.g., 3432) corresponding to selection of a credential submission affordance (e.g., 3420) causes credentials to be submitted via a credential submission user interface element (e.g., 3408, 3410) (e.g. , allows credentials to be submitted without using biometric authentication).

According to some examples, while displaying a credential submission user interface (e.g., 3404), an electronic device (e.g., 100, 300, 500, 1700, 3400) may use one or more biometric sensors (e.g., 3403, 3414). Through, each type of biometric feature (e.g., 3412) is detected. In response to detecting each type of biometric feature (e.g., 3412), and upon determining that the biometric feature (e.g., 3412) meets biometric authentication criteria, the electronic device (e.g., 100, 300) , 500, 1700, 3400) submit credentials (e.g., successful authentication results in submission of credentials) through a credential submission user interface element (e.g., 3408, 3410). In response to detecting each type of biometric feature (e.g., 3412), and upon determining that the biometric feature (e.g., 3412) does not meet biometric authentication criteria, the electronic device (e.g., 100, 300, 500, 1700, 3400) refrain from submitting credentials through a credential submission user interface element (e.g., 3408, 3410). Withholding submission of credentials based on not meeting biometric authentication criteria can provide security and prevent unauthorized users from initiating sensitive operations. Providing improved security improves the operability of the device (e.g., by restricting unauthorized access) and makes the user-device interface more efficient, which additionally reduces the power of the device by limiting performance of restricted operations. Reduces usage and improves battery life.

According to some examples, the credential submission user interface element (e.g., 3408, 3410) includes one or more fillable fields.

According to some examples, displaying a credential submission user interface (e.g., 3404, 3404-1, 3404-2) may pre-screen credentials with credentials to be submitted through a credential submission user interface element (e.g., 3408, 3410). and displaying a credential submission user interface element populated (e.g., pre-populated with a default user name). Pre-populating a default username gives users the ability to log in using fewer inputs. Performing operations with a reduced number of inputs improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device) and user- It makes the device interface more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

According to some examples, an electronic device (e.g., 100, 300, 500, 1700, 3400) receives a selection of a fillable field (e.g., 3408, 3410) among one or more fillable fields. In response to receiving a selection of a fillable field, the electronic device (e.g., 100, 300, 500, 1700, 3400) displays a character input interface (e.g., 3426) (e.g., a password or pass) on the display (e.g., 3402). Displays a keypad or keyboard containing character input keys for entering a code.

According to some examples, an electronic device (e.g., 100, 300, 500, 1700, 3400) may be configured to input one or more characters through a character input interface (e.g., 3426) (e.g., through character input keys) in a fillable field. An input corresponding to (e.g., 3430) is received. In some examples, the text input interface includes text input keys. Subsequent to receiving input, the electronic device (e.g., 100, 300, 500, 1700, 3400) receives selection of a second credential submission affordance (e.g., 3420) (e.g., a login button). In response to receiving a selection of a second credential submission affordance, the electronic device (e.g., 100, 300, 500, 1700, 3400) enters one or more characters in a fillable field (e.g., 3408, 3410) for credential verification. submit them.

According to some examples, presenting a biometric feature (e.g., 3412) that meets biometric authentication criteria to one or more biometric sensors (e.g., 3403, 3414) causes the credentials to be displayed in the credential submission user interface. A visual indication (e.g., 3406) that an element (e.g., 3408, 3410) is to be submitted via one of one or more fillable fields (e.g., username field, password field) (e.g., 3410) is displayed in Displaying a prompt indicating to the user that placing their finger on the sensor will automatically cause them to log in provides the user with feedback regarding the current state of the device (e.g., biometric authentication is available), Provide feedback to users indicating efficient options for logging in. Displaying a prompt in the password field can provide feedback on the action to be performed upon successful authentication (eg, automatically populating the password field). Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

According to some examples, the predefined operation may be to display a credential submission interface (e.g., 3404, 3404-1, 3404-2) on a display (e.g., 3402) (e.g., to load a credential submission user interface). , scroll the credential submission user interface into view, zoom in on the credential submission user interface, and reveal the credential submission user interface from hidden user interface elements. In some examples, the predefined action, which is a request to display a credential submission interface, may also include: It is a request to display .

According to some examples, the predefined action is detected while displaying a credential submission interface (e.g., 3404, 3404-1), and the predefined action is detected upon input to a portion of the credential submission user interface (e.g., 3424). (e.g., user input (e.g., a tab) on a credential submission user interface or user input on a fillable field such as a username field or password field).

According to some examples, in response to submitting one or more characters to a fillable field for credential verification (e.g., 3408, 3410) and in response to receiving a selection of a second credential submission affordance (e.g., 3420), the electronic A device (eg, 100, 300, 500, 1700, 3400) provides a first result. Upon submission of credentials via a credential submission user interface element (e.g., 3408, 3410) and via one or more biometric sensors (e.g., 3403, 3414), a biometric characteristic (e.g., 3412) is identified as a biometric feature. In response to determining that measurement verification criteria are met, the electronic device (eg, 100, 300, 500, 1700, 3400) provides a first result. In some examples, submitting (e.g., successfully submitting) credentials (e.g., valid credentials) via a password or passcode entered through a credential submission user interface element may require biometric authentication. results in the same results as successful authentication (e.g., access to restricted applications, web pages, or accounts).

Note that the details of the processes described above with respect to method 3500 (e.g., FIG. 33) are also applicable in a similar manner to the methods described below and above. For example, method 3500 may optionally have features of various methods described above with reference to methods 800, 1000, 1200, 1400, 1600, 2000, 2200, 2500, 2700, 2900, 3100, 3300. Contains one or more of For example, registered biometric data described in method 1200 may be used to perform biometric authentication as described in connection with method 3500. As a further example, the recertification criteria of method 3100 can be used in conjunction with method 3500 to control when biometric measurements are available. For brevity, these details are not repeated below.

In some examples (e.g., in some examples of methods 1600, 1800, 2000, 2200, 2900, 3100, 3300, 3500), the electronic device can perform (brute force attempts that bypass security) biometric authentication attempts that can be made before biometric authentication is disabled (e.g., until successful authentication occurs through alternative means) to provide improved security and conserve device resources by preventing Limit the number allowed. In some such examples, stopping use of a biometric sensor before the allowed/limited number of attempts is exhausted (e.g., pending further biometric authentication retries) may prevent the user from making repeated requests (e.g. , repeated requests of the same type) thereby avoiding consuming the allowed number of attempts (e.g., requests for other operations requiring biometric authentication) (e.g., requests for other more important operations). Preserve at least one attempt to Preserving at least one attempt improves the operability of the device (e.g., by avoiding exhaustion of authentication attempts for repeated similar requests) and makes the user-device interface more efficient, which additionally allows the user to access the device. Reduces device power usage and improves battery life by enabling faster and more efficient use of devices. Additionally, preserving at least one biometric authentication attempt may reduce the instances where a user must provide alternative non-biometric authentication (e.g., such as a password or passcode authentication), which in turn may: May promote the use of more secure (e.g., more complex) passwords/passcodes, which may encourage users to frequently use such passwords/passcodes when biometric authentication is disabled due to exhaustion of allowed attempts. This is because using more secure passwords/passcodes is not discouraged by necessity. Promoting the use of more secure passwords/passcodes improves the operability of the device by reducing the risk of unauthorized access.

36A-36L illustrate example user interfaces for retrying biometric authentication in a credential entry user interface, according to some examples. The user interfaces in these figures are used to illustrate the processes described below, including the processes in FIGS. 37A and 37B.

36A illustrates an electronic device 3600 (e.g., portable multifunction device 100, device 300, device 500). In the illustrative examples illustrated in FIGS. 36A-36L, electronic device 3600 is a smartphone. In other examples, electronic device 3600 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 3600 includes a display 3602, one or more input devices (e.g., a touchscreen of display 3002, buttons 3604, and a microphone), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In Figure 36A, the electronic device includes a biometric sensor 3603. In some examples, the biometric sensor is one or more biometric sensors that may include a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, biometric sensor 3603 is biometric sensor 703. In some examples, the one or more biometric sensors include one or more fingerprint sensors (eg, a fingerprint sensor integrated into a button). In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

36A , electronic device 3600 displays a locked state user interface (UI) 3606 that includes a lock icon 3608, providing an indication to the user that electronic device 3600 is in a locked state. Because electronic device 3600 is in a locked state, the user cannot view the limited content of notification 3610A (eg, a message from John Appleseed).

In Figure 36B, the user wishes to unlock electronic device 3600 to access restricted content on the device (e.g., messages from John Appleseed, home screen 3628 in Figure 36L, most recently used applications). . Unlocking the device requires successful authentication of the user. To request to unlock the device, the user performs an upward swipe starting within area 3612A, a predefined area adjacent the bottom edge of display 3602.

While displaying locked state UI 3606, electronic device 3600 receives input 3614A (e.g., an upward swipe). While displaying locked status UI 3606, electronic device 3600 requires an upward swipe to start from within area 3612A to trigger a request to unlock the device. In response to receiving input 3614A, electronic device 3600 determines whether input 3614A begins within area 3612A.

Upon detecting input 3614A (e.g., an input starting within area 3612A), electronic device 3600 initiates biometric authentication and determines if biometric authentication is currently enabled (or enabled) on the device. determine whether it is available for use. Biometric authentication may be unavailable for a variety of reasons, including, for example, that biometric authentication has failed more than a predetermined number of times (e.g., 5, 10, 15) since the last successful authentication by the device. there is.

In some examples, instead of receiving input 3614A, electronic device 3600 may receive any one of inputs 3614B-3614E (e.g., swipe inputs moving a similar distance as input 3614A). Receive. Like input 3614A, inputs 3614B through 3614E are each an upward swipe. In some examples, electronic device 3600 treats input 3614A and input 3614B as the same because input 3614B also begins within region 3614A (e.g., electronic device 3600 treats both inputs 3614A and 3614B as the same). have the same response to inputs). In contrast, in some examples, the response of electronic device 3600 to inputs 3614C - 3614E is different than the response to inputs 3614A and 3614B. Specifically, in some examples, in response to receiving any one of inputs 3614C-3614E, electronic device 3600 does not initiate biometric authentication, as described below with respect to FIG. 36C. No. Instead, the electronic device 3600 may, as shown in Figure 36K, (e.g., by scrolling a portion of the wake screen user interface (e.g., 3606) to display one or more notifications that were not previously visible on the display - where Display one or more notifications (eg, 3610A-3610E) (eg, unread notifications) while keeping the device locked (the amount of scrolling is optionally determined based on the speed or magnitude of movement of the contact).

36C, in response to input 3614 and upon determining that biometric authentication is currently enabled, electronic device 3600 attempts to authenticate the user biometrically (e.g., using biometric sensor 3603). attempts to match the information about the user's face obtained by matching the stored authorized credentials). While attempting to biometrically authenticate a user, the electronic device 3600 displays an insertable interface 3616 with an authentication glyph 3618 (e.g., replacing the display of the locked state UI 3606 with it). Authentication glyph 3618, which includes a plurality of rings rotating about an axis parallel to the display such that it appears to rotate off the z-axis of the device, provides an indication to the user that biometric authentication is being performed.

While attempting to biometrically authenticate a user, electronic device 3600 uses biometric sensor 3603 to determine whether certain biometric authentication criteria have been met. More specifically, electronic device 3600 captures and processes (e.g., analyzes) biometric data from biometric sensor 3603 to create biometric features (or portions thereof) based on the biometric data. Determine whether biometric authentication criteria are met (e.g., determine whether biometric data matches a biometric template (e.g., stored authorized credentials) within a threshold). After initiating biometric authentication, electronic device 3600 determines that biometric authentication has failed (e.g., biometric data from biometric sensor 3603 does not match stored authorized credentials).

36D , upon determining that biometric authentication has failed, electronic device 3600 displays an animation of a lock icon 3608 alternating between different positions to simulate a “shake” effect. This “shake” animation provides an indication to the user that electronic device 3600 was unable to biometrically authenticate the user.

As shown in FIG. 36E, further in response to determining that biometric authentication has failed, electronic device 3600 displays passcode entry UI 3620 (e.g., display of insertable interface 3616). alternatively), which provides an alternative (e.g., non-biometric) method of authenticating a user at electronic device 3600. The passcode input UI 3620 includes a lock icon 3608 and a plurality of input keys for entering (or writing) a password or passcode. The passcode entry UI 3620 also prompts the user to swipe up to retry biometric authentication or to enter a passcode (or password) to authenticate the user (e.g., with a non-biometric authentication). Includes a prompt 3622A: Electronic device 3600 displays prompt 3622A upon determining that biometric authentication is currently enabled on the device. Additionally, upon determining that biometric authentication is currently enabled, the electronic device 3600 displays an unlock indication 3624, which is a display on which the user can initiate an upward swipe to retry biometric authentication ( 3602) Provides an indication of the approximate location of the image.

In Figure 36E, the user wishes to retry biometric authentication rather than entering a password or passcode to authenticate. While displaying passcode input UI 3620, electronic device 3600 receives input 3614C, starting outside area 3612A in FIG. 36B. Nonetheless, input 3614C triggers a retry of biometric authentication because input 3614C starts from within area 3612B. In particular, area 3612B is larger than area 3612A because the parameters for where an upward swipe should begin to initiate biometric authentication are greater than those in passcode entry UI 3620 compared to locked state UI 3606. ) is alleviated.

In some examples, instead of receiving input 3614C, the electronic device receives any one of inputs 3614A, 3614B, 3614D, and 3614E. In some examples, given the relaxed parameters for passcode input UI 3620, anything other than input 3614E will trigger retrying biometric authentication. In some examples, in response to receiving input 3614E, electronic device 3600 determines that input 3614E does not originate within region 3612B and, in response, does not retry biometric authentication. .

36F, in response to receiving input 3614C, electronic device 3600 determines that input 3614 begins within area 3612B. Upon determining that input 3614 begins within area 3612B, electronic device 3600 reattempts biometric authentication. While retrying biometric authentication, electronic device 3600 displays authentication glyph 3618 (e.g., replaces the display of lock icon 3608 with it). Authentication glyph 3618 provides an indication to the user that biometric authentication is being performed.

While retrying biometric authentication, electronic device 3600 determines that biometric authentication was successful (e.g., biometric data obtained using biometric sensor 3603 matches stored authorized credentials ).

36G and 36H, upon determining that biometric authentication is successful, electronic device 3600 transitions from a locked state to an unlocked state. The electronic device 3600 provides an indication of this transition by displaying an animation of the lock icon 3608 transitioning to the unlock icon 3626 in Figure 36H, which causes the electronic device 3600 to transition to the unlocked state. Provides an indication that it has been done. Additionally, upon determining that biometric authentication is successful and after displaying unlock icon 3626, electronic device 3600 provides access to the restricted content. For example, electronic device 3600 may display home screen 3628 of Figure 36L or the user interface of a most recently used application (e.g., messaging application interface 2616 of Figure 26G). displays.

In some examples, instead of determining that biometric authentication failed in Figure 36C, electronic device 3600 determines that biometric authentication was successful. In some examples, upon determining that biometric authentication was successful, electronic device 3600 transitions to the unlocked state and displays unlock icon 3626, as described above with respect to FIGS. 36G and 36H. . Additionally, in some examples, upon determining that biometric authentication is successful and after displaying the unlock icon in Figure 36H, electronic device 3600 displays home screen 3628. (Home screen 3628 may include some or all of the features of home screen interface 2614 of Figure 26D.) In some examples, upon determining that biometric authentication was successful, and unlock icon 3626 After displaying, electronic device 3600 displays the most recently used application (e.g., messaging application interface 2616 in FIG. 26G).

In some examples, instead of determining that biometric authentication is enabled on the device as described above with respect to FIGS. 36B and 36C, electronic device 3600 may allow biometric authentication to be enabled (e.g., manually) by the user. It is determined that biometric authentication is not currently enabled (either because it has been disabled or because predetermined biometric authentication has been attempted without success). In some examples, upon determining that biometric authentication is not currently enabled, electronic device 3600 displays prompt 3622B in Figure 36I instead of prompt 3622A in Figure 36E. In contrast to prompt 3622A, prompt 3622B only prompts the user to enter a passcode (or password) to authenticate the user without prompting the user to swipe to retry biometric authentication. do. Additionally, in some examples, upon determining that biometric authentication is not currently enabled, electronic device 3600 does not display unlock indication 3624, as shown in FIG. 36I. As described above, unlock indication 3624 provides an indication of the approximate location on display 3602 where the user can initiate an upward swipe to retry biometric authentication. In some examples, electronic device 3600 does not display unlock indication 3624 because biometric authentication is not currently enabled.

In some examples, instead of determining that biometric authentication was successful in Figure 36F, electronic device 3600 determines that biometric authentication failed. In some examples, upon determining that biometric authentication has failed, electronic device 3600 may display a “shake” effect (instead of displaying a transition to the unlocked state as described above with respect to FIGS. 36G and 36H). Displays an animation of the lock icon 3608 alternating between different positions to simulate . As described above, this “shake” animation provides an indication to the user that electronic device 3600 was unable to biometrically authenticate the user.

37A and 37B are flow diagrams illustrating a method for reattempting biometric authentication in a credential entry user interface using an electronic device, according to some examples. Method 3700 includes a touch-sensitive display (e.g., 3602) and one or more biometric sensors (e.g., 3603) (e.g., a first biometric sensor of a device with a plurality of biometric sensors) (e.g., a fingerprint sensor, It is performed on an electronic device (e.g., 100, 300, 500, 3600) with a non-contact biometric sensor (e.g., a biometric sensor that does not require physical contact, such as a thermal or optical facial recognition sensor), an iris scanner). In some examples, one or more biometric sensors include one or more cameras. Some operations of method 3700 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 3700 provides an intuitive method for reattempting biometric authentication in a credential entry user interface using an electronic device. This method reduces the user's cognitive burden of retrying biometric authentication, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to retry biometric authentication faster and more efficiently saves power and increases time between battery charges.

The electronic device (e.g., 100, 300, 500, 3600) includes a credential input (e.g., passcode, password, pattern) user interface (e.g., passcode, password, pattern) having a plurality of character input keys on a touch-sensitive display (e.g., 3602). For example, 3620) is displayed (3706). In some examples, the credential entry user interface includes a virtual keypad or virtual keyboard. In some examples, the virtual keypad or virtual keyboard includes a plurality of character input keys. In some examples, while displaying a credential input user interface (e.g., 3620), the electronic device receives input corresponding to one or more character input keys of a plurality of character input keys. In some examples, in response to (or subsequent to) receiving input corresponding to a plurality of character input keys, the electronic device determines that the received input corresponds to authorized credentials (e.g., a stored passcode or password). Depending on the decision to do (or match), there is a transition from the locked state (e.g., corresponding to 3606) to the unlocked state (e.g., corresponding to 3628).

While displaying a credential input user interface (e.g., 3620), the electronic device may, through a touch-sensitive display (e.g., 3602), perform a touch gesture input (e.g., 3614A, including movement of contact on the touch-sensitive display). 3614B) (e.g., a swipe at a predefined location) is received (3708).

In response to receiving a touch gesture input 3712 comprising movement of contact on the touch-sensitive display and in accordance with a determination that one or more criteria of the first set are met—one or more criteria of the first set include: biometric authentication Including a requirement that the electronic device is currently enabled on the electronic device, the electronic device attempts to biometrically authenticate the user of the electronic device based on biometric information captured using one or more biometric sensors (3714). In some examples, the one or more criteria in the first set include only one criterion. In some examples, biometric authentication may become unavailable (or not enabled on an electronic device) when one or more of the following conditions are met: The electronic device is being turned on or restarted successfully. not certified; The electronic device has not been unlocked for more than a predetermined period of time (e.g., 48 hours); The passcode has not been used to unlock the device for more than a predetermined amount of time (e.g., 156 hours); Biometric authentication using biometric features (e.g., face, fingerprint) was not used to unlock the device for more than a predetermined time (e.g., 4 hours); The electronic device has received a remote lock command; biometric authentication has failed more than a predetermined number of times (e.g., 5, 10, 15) since the last successful authentication by the device; The electronic device then received a power-off and/or emergency SOS command. In some examples, the touch gesture input is a request to unlock electronic device 3600. Providing users with the ability to retry biometric authentication by performing touch gesture input in the credential entry user interface improves the operability of the device by providing additional control of the device without cluttering the UI with additional displayed controls. improves Providing these capabilities improves the operability of the device (e.g., by helping users provide appropriate inputs and reducing user errors when operating/interacting with the device) and makes the user-device interface more efficient. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, in response to receiving 3712 a touch gesture input comprising movement of contact on the touch-sensitive display and (e.g., because biometric authentication is not currently enabled on the device) one or more of the first set of Upon determining that the criteria are not met, the electronic device withholds attempting to biometrically authenticate a user of the electronic device based on biometric information captured using one or more biometric sensors (e.g., 3603). 3716). Deferring attempts to biometrically authenticate a user when biometric authentication is not enabled improves the security of the device by preventing fraudulent use of the device.

In some examples, displaying a credential input user interface (e.g., 3620) may include a request to perform an action requiring authentication (e.g., a request to unlock an electronic device (e.g., 3614A, 3614B) (e.g., a request to perform an action requiring authentication) swipes starting from an edge (e.g., bottom edge) or within a predefined area (e.g., bottom portion) of the display), and biometric information captured using one or more biometric sensors. Occurs 3702 in response to a failure to biometrically authenticate a user of an electronic device based on . In some examples, an electronic device (e.g., 100, 300, 500, 3600) is configured to store biometric information captured using one or more biometric sensors using authorized credentials (e.g., authorized for use in biometric authentication). Biometric authentication of a user of an electronic device fails when it does not correspond (or match) to biometric features (e.g., stored information about the face, fingerprint). Displaying a credential entry user interface when a set of conditions are met provides the user with the ability to authenticate via an alternative method without requiring the user to explicitly request that the credential entry user interface be displayed. Performing an action when a set of conditions is met without requiring additional user input (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device) It improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, a request to perform an action requiring authentication (e.g., 3614A, 3614B) may include a second touch gesture input comprising movement of a contact on a touch-sensitive display (e.g., an edge (e.g., bottom edge) of the display). 3704), or a swipe starting from within a predefined area (e.g., 3612A) of the display (e.g., the lower portion). In some examples, the set of one or more criteria may be such that the touch gesture input begins in a first area (e.g., 3612A) of the display (e.g., an area along the bottom edge of the display) and begins in a second area of the display (e.g., an area along the bottom edge of the display). and the requirement that it terminate in (or proceed through) an area along the bottom edge (e.g., an area above 3612A).

In some examples, upon determining that one or more criteria of the second set are met, wherein the one or more criteria of the second set include a requirement that biometric authentication is currently enabled on the electronic device, a credential entry user interface ( For example, displaying 3620) may include, on a touch-sensitive display (e.g., 3602), an indication (e.g., 3622A) to perform a gesture on the touch-sensitive display (e.g., text) to attempt to biometrically authenticate a user. , graphics, and icons). In some examples, upon determining that one or more criteria of the second set are not met, the electronic device withholds displaying an indication to perform a gesture on the touch-sensitive display to attempt to biometrically authenticate the user. In some examples, one or more criteria in the second set are the same as one or more criteria in the first set. In some examples, the one or more criteria in the second set include only one criterion. Displaying an indication to perform a gesture when biometric authentication is available provides the user with feedback about the current state of the device (e.g., biometric authentication is available) and prompts the user to perform the gesture associated with the indication. Prompt. Additionally, not displaying an indication to perform a gesture when biometric authentication is not available also provides feedback to the user about the current state of the device (eg, biometric authentication is not available). Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, upon determining that one or more criteria of the third set are met, wherein the one or more criteria of the third set includes a requirement that biometric authentication is currently enabled on the electronic device, a credential entry user interface ( For example, displaying 3620 may include a user interface element (e.g., 3624) (e.g., a graphical element) on a touch-sensitive display (e.g., 3602) adjacent an edge of the touch-sensitive display (e.g., a bottom edge of the display). , a horizontal bar, a home affordance, and an indication of where the user must swipe to attempt biometric authentication). In some examples, one or more criteria in the third set are the same as one or more criteria in the first set. In some examples, the one or more criteria in the third set include only one criterion. In some examples, upon determining that one or more criteria of the third set are not met, the electronic device (eg, 100, 300, 500, 3600) withholds displaying the user interface element. In some examples, the set of one or more criteria may be such that the touch gesture input begins in a first area (e.g., 3612A) of the display (e.g., an area along the bottom edge of the display) and begins in a second area of the display (e.g., an area along the bottom edge of the display). It includes the requirement that it terminates in (or proceeds through) the area above the area along the bottom edge. In some examples, a user interface element (e.g., 3624) is displayed at a location corresponding to (e.g., within) a first area of the display. Displaying user interface elements adjacent to the edge of the display when biometric authentication is available provides feedback to the user regarding the current state of the device and how the user can request to unlock the device. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, while attempting to biometrically authenticate a user of the electronic device based on biometric information captured using one or more biometric sensors, the electronic device (e.g., 100, 300, 500, 3600) may touch On the responsive display, display an animation (e.g., 3618) with one or more moving elements (e.g., an animation (e.g., a graphic icon) that includes one or more rings moving on the display) indicating that biometric authentication is occurring. displays). In some examples, an animation provides an indication that biometric information is being processed. In some examples, the animation includes rings rotating around an object (eg, a sphere). In some examples, a sphere is visible. In some examples, the sphere is not visible. Displaying one or more moving elements that indicate that biometric authentication is occurring provides feedback to the user regarding the current state of the device (e.g., that biometric authentication is occurring) and that the user does not need to take any action at this time. provides. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, displaying an animation with one or more moving elements includes transitioning from the animation with one or more moving elements to a lock icon (e.g., 3608) (e.g., an icon indicating a locked state). In some examples, after transitioning from an animation with one or more moving elements (e.g., 3618) to a lock icon and using one or more biometric sensors, biometric information is captured using authorized credentials (e.g., for biometric authentication). Upon determining that it corresponds (e.g., matches) to a biometric feature (e.g., stored information about a face, fingerprint) that is authorized for use, the electronic device transitions from a locked state to an unlocked state. In some examples, transitioning the electronic device from a locked state to an unlocked state includes displaying an animation of a lock icon transitioning to an unlock icon (e.g., 3626) (e.g., an icon representing an unlocked state). do. In some examples, a determination that biometric information captured after transitioning from an animation with one or more moving elements to a lock icon and using one or more biometric sensors does not correspond to (e.g., does not match) authorized credentials. Accordingly, the electronic device displays an animation on a touch-sensitive display (e.g., 3602) with a lock icon (e.g., 3608) alternating between a first position and a second position, the second position being the first position. It is different from In some examples, the animation with the lock icon is an animation of the lock icon shaking (e.g., rotating left and right, back and forth). In some examples, the electronic device displays an animation accompanying a lock icon to indicate that biometric authentication has failed. In some examples, tactile output is provided in combination with a shaking lock icon. In some examples, no tactile output is provided. In some examples, upon determining that biometric information captured using one or more biometric sensors does not correspond or does not match the authorization credentials, the electronic device (e.g., 100, 300, 500, 3600) Maintains the locked state. Displaying an animation of the lock icon transitioning to the unlock icon informs the user of the current state of the device (e.g., biometric authentication was successful, the device is unlocked, and the user can now access restricted content). Provide feedback on Additionally, displaying an animation of a shaking lock icon provides feedback to the user regarding the current state of the device (eg, biometric authentication has failed) and prompts the user to take further action. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, a touch gesture input involving movement of contact on a touch-sensitive display starts from a location that is far from (e.g., not substantially proximate) an edge of the touch-sensitive display (e.g., a bottom edge of the display). Do (3710). In some examples, a location away from an edge of a touch-sensitive display (e.g., 3602) includes a location closer to the center of the display than the edge. In some examples, a location away from the edge of the touch-sensitive display includes a location that exceeds a threshold distance from the edge of the display (or is outside a predefined area (e.g., 3612A)). In some examples, a threshold distance (or predefined area) is used on the wake screen to determine whether to disarm the wake screen when a swipe input starts less than the threshold distance away from an edge of the display. In some examples, the threshold distance (or predefined area (e.g., 3612A)) occurs when the swipe input begins beyond the threshold distance (or outside the predefined area (e.g., 3612A)) away from an edge of the display. Used to perform different operations on the wake screen, such as scrolling content on the wake screen (eg, 3610A through 3610E). In some examples, parameters for where touch gesture input should begin are relaxed for the credential entry user interface (e.g., 3620). Relaxing parameters in the credential entry user interface for where touch gesture input should initiate improves the operability of the device by allowing less precise gestures to initiate biometric authentication. The parameters are relaxed because the user has already communicated the desire to initiate biometric authentication. As a result, in the credential entry user interface, there is less risk that the user will not intend to initiate biometric authentication. Allowing less precise gestures to initiate biometric authentication after the user has already communicated a desire to initiate biometric authentication improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the need for limited operations. By limiting their performance, they reduce the device's power usage and improve battery life.

In some examples, in response to receiving a touch gesture input (e.g., 3614C-3614E) comprising movement of contact on the touch-sensitive display starting from a position distal to an edge of the touch-sensitive display and one or more biometric sensors that the biometric information captured using the ) Depending on the decision, the electronic device transitions from the locked state to the unlocked state (3718). In some examples, transitioning the electronic device to an unlocked state includes an unlocked user interface (e.g., a user interface indicating the unlocked state, such as a home screen (e.g., 3628) or most recently used application). It includes displaying .

In some examples, prior to displaying the credential entry user interface, the electronic device (e.g., 100, 300, 500, 3600) displays a locking user interface (e.g., 3606) on the touch-sensitive display (e.g., 3602) (e.g., Displays a user interface (user interface, wake screen, lock screen) indicating the locked state of the electronic device. In some examples, the locking user interface is different from the credential entry user interface (e.g., 3620). In some examples, while displaying a locking user interface, the electronic device receives, via a touch-sensitive display (e.g., 3602), a touch gesture input (e.g., 3614A, 3614B) (e.g., an upward swipe) with a starting position. do. In response to receiving the touch gesture input having a starting location and in accordance with a determination that one or more criteria of the fourth set are met - one or more criteria of the fourth set are provided such that the starting location of the touch gesture input is substantially a touch-sensitive display. including a requirement that the electronic device be positioned near the edge (e.g., within a predefined area (e.g., 3612A), within a predefined distance from the edge; closer to the edge than to the center of the display) -, the electronic device may be configured to perform biometric authentication. commences. In some examples, the one or more criteria in the fourth set include only one criterion. In some examples, initiating biometric authentication includes attempting to biometrically authenticate a user of an electronic device based on biometric information captured using one or more biometric sensors. In some examples, in response to receiving a touch gesture input with a starting location and in response to a determination that one or more criteria of the fourth set are not met, the electronic device withholds from initiating biometric authentication.

Note that the details of the processes described above with respect to method 3700 (e.g., FIGS. 37A and 37B) are also applicable in a similar manner to the methods described below. For example, method 3900, method 4100, and/or method 4300 optionally include one or more of the features of the various methods described above with respect to method 3700. For example, processes for retrying biometric authentication, as described above with respect to method 3700, may include biometric authentication to authorize payment for goods, as described with respect to method 4100. Can be used to retry. Similarly, a process for reattempting biometric authentication may be used to retry biometric authentication in the processes described in connection with method 3900. For brevity, these details are not repeated below.

38A-38A illustrate example user interfaces for providing indications of error conditions during biometric authentication, according to some examples. The user interfaces in these figures are used to illustrate the processes described below, including the processes in FIGS. 39A and 39B.

38A illustrates an electronic device 3800 (e.g., portable multifunction device 100, device 300, device 500). In the illustrative examples illustrated in Figures 38A-38AD, electronic device 3800 is a smartphone. In other examples, electronic device 3600 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 3600 includes a display 3802, one or more input devices (e.g., a touchscreen of display 3802, buttons 3804, and microphone), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In Figure 38A, the electronic device includes a biometric sensor 3803. In some examples, the biometric sensor is one or more biometric sensors that may include a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, biometric sensor 3803 is biometric sensor 703. In some examples, the one or more biometric sensors include one or more fingerprint sensors (eg, a fingerprint sensor integrated into a button). In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In Figure 38A, the user learns from notification 3808 that she has received a message from John Appleseed. The user would like to view the limited content of notification 3808 (e.g., a message from John Appleseed), but is unable to do so because electronic device 3800 is currently locked. Electronic device 3800 displays a locked state user interface (UI) with a lock icon 3806, providing an indication that electronic device 3800 is in a locked state. Viewing the limited content of notification 3808 indicates that successful authentication (e.g., corresponds to (or matches) authorized credentials stored with information (or data) regarding biometric characteristics obtained using biometric sensor 3803 It asks you to decide) To view the limited content of notification 3808, the user lifts (or elevates) the electronic device 3800 (e.g., from a substantially horizontal orientation of the device to that of the device as shown in the user's hand in Figure 38A). The electronic device 3800 detects a change in the orientation of the electronic device 3800 and, in response, initiates biometric authentication. In some examples, after initiating biometric authentication, electronic device 3800 determines that biometric authentication was successful. In some examples, upon determining that biometric authentication was successful, electronic device 3800 transitions from a locked state to an unlocked state and displays the restricted content of notification 3808.

After initiating biometric authentication (e.g., prior to successful authentication), electronic device 3800 determines whether a face is detected by biometric sensor 3803. 38B, upon determining that a face has been detected, the electronic device 3803 displays an authentication glyph 3810, comprising a plurality of rings rotating into a sphere. Authentication glyph 3810 provides an indication that biometric authentication is being performed. In some examples, electronic device 3800 displays an animation of lock icon 3806 morphing into an authentication glyph 3810. In some examples, upon determining that no face is detected using biometric sensor 3803, electronic device 3803 remains locked and does not display authentication glyph 3810.

After detecting the presence of a face, the electronic device 3800 determines that authentication was not successful due to failure to obtain sufficient information about the user's face using the biometric sensor 3803. Specifically, as shown by FIG. 38B, biometric sensor 3803 is positioned outside the acceptable distance range 3812 (e.g., beyond the maximum threshold range) to obtain sufficient information about the user's face. causes failure. Upon determining that biometric authentication is not successful due to the user's face being outside the acceptable distance range 3812, electronic device 3800 keeps the device locked and does not display the restricted content of notification 3808. No. In some examples, electronic device 3800, upon determining that authentication is not successful and that an error condition does not exist, keeps the device locked and does not display the restricted content of notification 3808. In some examples, authentication is not successful (e.g., due to the captured biometric information not matching an authorized biometric information profile (e.g., stored authorized credentials)) and an error condition does not exist (e.g., Upon determining that there are no conditions preventing the capture of sufficient biometric information, the electronic device 3800 remains locked and automatically re-attempts biometric authentication. In some examples, while retrying biometric authentication, electronic device 3800 continues to display authentication glyph 3810 in FIG. 38B.

As shown in FIGS. 38C-38G , upon determining that biometric authentication is not successful due to the user's face being outside the allowable distance range 3812, the electronic device 3800 displays an error indication 3814A. displays an animation of authentication glyph 3810 that morphs into an error indication 3814A to replace the display of authentication glyph 3810. 38G, electronic device 3800 displays error indication 3814A, which prompts the user to take action to correct the error condition underlying error indication 3814A. Specifically, error indication 3814A prompts the user to move her face closer to biometric sensor 3803. Error indication 3814A also suggests to the user that the user's face is too far from biometric sensor 3803, which is the cause of error indication 3814A. As long as the user's face is outside the acceptable distance range 3812, the electronic device 3800 will continue to determine that an error indication 3814A is present. Upon determining that error indication 3814A still exists, electronic device 3800 does not attempt to retry biometric authentication. Note that electronic device 3800 displays error indication 3814A at a location that matches the location of lock icon 3806 in Figure 38A. Additionally, electronic device 3800 displays a portion of display 3802 adjacent biometric sensor 3803 to suggest to the user that error indication 3814A is associated with (or corresponds to) biometric sensor 3803. Displays an error indication (3814A) on the screen.

As shown in Figure 38H, after being prompted to correct the error indication 3814A, the user moves her face closer to the biometric sensor 3803 so that the user's face is within the acceptable distance range 3812. . In Figure 38H, the electronic device determines that error indication 3814A is no longer present. Upon determining that error indication 3814A is no longer present, electronic device 3800 enables biometric authentication on the device and automatically reattempts biometric authentication using biometric sensor 3803.

In response to automatically retrying biometric authentication, the electronic device 3800 may display a shimmer effect (e.g., an error message) to indicate that the electronic device 3800 is attempting to biometrically authenticate the user again. Displaying an error indication 3814A with animating the error indication such that one or more portions of the indication are moved from side to side to create an effect in which the error indication appears to flash. Figures 38H-38L show an animation of error indication 3814A with a shimmer effect. In some examples, instead of displaying the error indication 3814A with a shimmer effect, the electronic device 3800 displays an authentication glyph ( 3810) (e.g., replaces the display of error indication 3814A with it). Accordingly, in some examples, electronic device 3800 displays an animation of authentication glyph 3814A morphing into lock icon 3806 instead of error indication 3814A morphing into lock icon 3806.

38L, after retrying biometric authentication, electronic device 3800 successfully biometrically authenticates the user. In response to successful biometric authentication, electronic device 3800 transitions the device from a locked state to an unlocked state. While transitioning from the locked state to the unlocked state, the electronic device 3800 displays an animation of an error indication 3814A morphing into a lock icon 3806, as shown in FIGS. 38L-38N. After displaying the animation of error indication 3814A morphing into lock icon 3806, electronic device 3800 displays lock icon 3806 transitioning to unlock icon 3816, as shown in FIGS. 38N and 38O. ) displays the animation. Unlock icon 3816 provides an indication that electronic device 3800 is in an unlocked state. Additionally, as shown in FIG. 38O, electronic device 3800 displays limited content of notification 3808 (e.g., “Hey, is our meeting still on?”) in response to biometric authentication being successful.

In Figure 38P, instead of determining that the user's face is outside the acceptable distance range 3812 as discussed previously with respect to Figure 38B, electronic device 3800 determines that biometric authentication is not available on the device. . Upon determining that biometric authentication is not available, electronic device 3800 displays error indication 3814B in Figure 38P, which provides an indication that biometric authentication is not currently available on the device. Biometric authentication may be unavailable for a variety of reasons, including that biometric authentication has failed more than a predefined number of times (e.g., 5, 10, 15) since the last successful authentication.

Due to the unavailability of biometric authentication, the user must use an alternative method to authenticate the user. For example, the user can authenticate by entering a passcode on the electronic device 3800. While displaying error indication 3814B in FIG. 38P, electronic device 3800 receives input 3820 at error indication 3814B.

38Q, in response to receiving input 3820 in error indication 3814B, electronic device 3800 displays a passcode input UI (UI) having a plurality of input affordances for entering a passcode (or password). 3822A) is displayed.

In some examples, instead of determining that authentication was successful as a result of retrying biometric authentication, electronic device 3800 determines that authentication was not successful, as discussed above with respect to FIGS. 38L-38O. In some examples, upon determining that authentication was not successful, the electronic device 3800 remains locked and displays the lock icon 3806 in FIG. 38R alternating between different positions to simulate a “shake” effect. Display animation. The shake animation provides an indication to the user that biometric authentication has failed and electronic device 3800 remains locked.

After determining that authentication was not successful, the user may perform an action at electronic device 3800 to trigger retrying biometric authentication. In Figure 38S, the user triggers a retry of biometric authentication by starting from an area near the bottom edge of display 3802 and swiping upward. Electronic device 3800 receives input 3824 and, in response, reattempts biometric authentication. In some examples, after reattempting biometric authentication, electronic device 3800 determines that authentication was successful. In some examples, upon determining that authentication was successful as a result of retrying biometric authentication, electronic device 3800 transitions from a locked state to an unlocked state.

38S and 38T, the electronic device, in response to input 3824, determines that authentication was not successful as a result of reattempting biometric authentication. Upon determining that authentication was not successful as a result of retrying biometric authentication, electronic device 3800 displays passcode input UI 3822B in FIG. 38T. As described in connection with the processes in FIG. 37, the user may retry biometric authentication again at passcode entry UI 3822B by performing an upward swipe (e.g., input 3826). (Passcode entry UI 3822B includes some or all of the features of passcode entry UI 3620, including relaxed parameters for where an upward swipe should begin to initiate biometric authentication. .)

In Figure 38U, the electronic device determines that authentication is successful as a result of retrying biometric authentication in the passcode input UI 3822B. Upon determining that authentication was successful, the electronic device transitions from the locked state to the unlocked state, as shown in Figures 38U-38W. In some examples, in Figure 38U, the electronic device determines that authentication was not successful as a result of retrying biometric authentication at passcode entry UI 3822B. In some examples, when making this determination, the electronic device remains locked.

38X-38AD illustrate various error conditions that electronic device 3800 can detect while attempting to biometrically authenticate a user. Instead of displaying error indication 3814A as described above with respect to FIG. 38G, electronic device 3800 may display any one of the error indications described below (e.g., error indications 3814C through 3814I). can do. 38X-38A also provide instructions to take action (e.g., error indications 3814C-3814I) to correct a detected error condition so that the electronic device 3800 can retry biometrically authenticating the user. (via) shows an electronic device 3800 that provides coaching to a user.

In Figure 38x, the user's face is positioned too close to the biometric sensor 3803. As a result, the electronic device 3800 determines that the user's face is positioned outside the acceptable distance range 3812 (e.g., below the minimum threshold range). Upon determining that the user's face is positioned outside the acceptable distance range 3812, the electronic device 3800 displays an error indication 3814C, which prompts the user to place her face further away from the biometric sensor 3803. Prompts you to move. Error indication 3814C also provides an indication of the cause of the error condition (e.g., that the user's face is too close to biometric sensor 3803).

In Figure 38y, the user's hand covers the biometric sensor 3803. As a result, the electronic device 3800 determines that an object (e.g., the user's hand) is covering the biometric sensor 3803 such that the sensor cannot obtain any information about the user's face. Upon determining that an object is covering biometric sensor 3803, electronic device 3800 displays an error indication 3814D, which prompts the user to move her hand away from biometric sensor 3803. do. Error indication 3814D also provides an indication of the cause of the error condition (e.g., an indication that biometric sensor 3803 is covered).

In Figure 38Z, the user is not looking at electronic device 3800. As a result, electronic device 3800 determines that the user's eyes are not looking at the device. Upon determining that the user's eyes are not looking at the device, electronic device 3800 displays an error indication 3814E, which prompts the user to look at the device to correct the error condition. Error indication 3814E also provides an indication of the cause of the error condition (eg, that the user is not looking at the device).

In Figure 38aa, the user's face is within field of view 3828, but the user is wearing a hat. As a result, the electronic device 3800 determines that a portion of the user's face is hidden (or obscured). For example, electronic device 3800 obtains partial information about the user's face using biometric sensor 3803, where the partial information is below a threshold amount needed for comparison with stored authorized credentials. Upon determining that a portion of the user's stoop is hidden, electronic device 3800 displays error indication 3814F, which prompts the user to remove the hat. Error indication 3814F also provides an indication of the cause of the error condition (e.g., that a portion of the user's face is hidden).

In Figure 38ab, the user's face is outside the field of view 3828 of the biometric sensor 3803. As a result, electronic device 3800 determines that the user's face is outside the field of view 3828 of biometric sensor 3803. In some examples, the user's face is outside of field of view 3828 when more than a critical portion of the face is outside of field of view. In some examples, the user's face is outside the field of view 3828 when no faces are detected within the field of view. Upon determining that the user's face is outside the field of view 3828, the electronic device 3800 displays an error indication 3814G, which prompts the user to move her face within the field of view 3828. Error indication 3814G also provides an indication of the cause of the error condition (e.g., indication 3814G that the user's face is out of view).

In Figure 38ac, the user's face is within field of view 3828, but turned away from biometric sensor 3803. As a result, electronic device 3800 determines that the user's face is turned away from biometric sensor 3803. Upon determining that the user's face is turned away from the biometric sensor 3803, the electronic device 3800 displays an error indication 3814H, which prompts the user to turn her face toward the sensor. Error indication 3814H also provides an indication of the cause of the error condition (e.g., that the user's face is turned away from biometric sensor 3803).

In Figure 38A, the user's face is positioned approximately within the acceptable distance range of the field of view and biometric sensor 3803. However, the lighting conditions of the environment in which the user is located are not suitable for performing biometric authentication. Specifically, the amount of light is so high that it prevents performing biometric authentication. As a result, the electronic device 3800 determines (eg, via one or more ambient light sensors) that the amount of light exceeds a predefined threshold. Upon determining that the light quantity exceeds the threshold, electronic device 3800 displays an error indication 3814I, which prompts the user to seek improved lighting conditions with lower light quantity. Error indication 3814I also provides an indication of the cause of the error condition (eg, that light conditions are not suitable for performing biometric authentication).

39A and 39B are flow diagrams illustrating a method for providing indications of error conditions during biometric authentication, according to some examples. Method 3900 includes an electronic device (e.g., 100, 300, 500, 3800). In some examples, the electronic device may include one or more biometric sensors (e.g., a fingerprint sensor, a non-contact biometric sensor (e.g., a biometric sensor that does not require physical contact, such as a thermal or optical facial recognition sensor), an iris scanner). Includes. In some examples, one or more biometric sensors include one or more cameras. Some operations of method 3900 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 3900 provides an intuitive method for providing indications of error conditions during biometric authentication. This method reduces the user's cognitive burden for performing biometric authentication, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to perform biometric authentication faster and more efficiently saves power and increases time between battery charges.

The electronic device (e.g., 100, 300, 500, 3800) performs authentication (e.g., biometric authentication) via one or more input devices (e.g., an accelerometer (e.g., 168), a touchscreen of a display (e.g., 3802)) A request to perform an operation request is received (3902). In some examples, the request to perform an action requiring authentication includes a request to unlock the device (eg, swipe at a predefined location). In some examples, the request is triggered by lifting the device from a substantially horizontal position.

In response to a request to perform an action requiring authentication (e.g., biometric authentication) (3904) and upon a determination that authentication (e.g., biometric authentication) was successful, the electronic device performs the action (3906). In some examples, user input (e.g., data obtained from one or more biometric sensors corresponding to a user's biometric features (e.g., face, finger), passcode) is combined with an authorized credential (e.g., an enrolled fingerprint). Authentication is successful when it matches (e.g., matches) a face, face, or passcode). In some examples, user input corresponds to an authorized credential when the user input matches an authorized credential.

In response to a request to perform an operation 3904 that requires authentication (e.g., biometric authentication) and that authentication (e.g., biometric authentication) is not successful and a set of error condition criteria is met (e.g., an error condition exists) Depending on the determination 3908, the electronic device (e.g., 100, 300, 500, 3800) displays an indication of an error condition (e.g., of a set of error condition criteria) on a display (e.g., 3802) (e.g., 3814A through 3814I) and suspend performing the operation (3916). The indication includes information regarding the cause of the error condition (3912). In some examples, user input (e.g., data obtained from one or more biometric sensors corresponding to a user's biometric features (e.g., face, finger), passcode) is combined with an authorized credential (e.g., an enrolled fingerprint). , face, or passcode), authentication is not successful. In some examples, user input does not correspond to an authorized credential when the user input does not match an authorized credential. In some examples, the set of error condition criteria includes only one criterion. Displaying an indication of an error condition provides feedback to the user regarding the current state of the device (e.g., that an error condition is preventing successful biometric authentication) and allows the user to take further action to correct the error condition. prompt. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently. Additionally, suspending performing an action when biometric authentication fails and an error condition is detected improves security and reduces the risk of multiple, resource-intensive retries of biometric authentication likely to fail due to an error condition. reduce them. Providing improved security improves the operability of the device (e.g., by restricting unauthorized access) and makes the user-device interface more efficient, which additionally reduces the power of the device by limiting performance of restricted operations. Reduces usage and improves battery life.

In some examples, in response to a request to perform an operation requiring authentication (3904) and in response to a determination (3926) that authentication (e.g., biometric authentication) is not successful and a set of error condition criteria is not met, the electronic device (e.g., 100, 300, 500, 3800) withholds from displaying an indication of an error condition on a display (e.g., 3802) (3928) and withholds from performing an action (3930).

In some examples, an indication of an error condition (e.g., 3814A-3814I) may be an indication (e.g., a visual display (e.g., graphic or text) of a user action that can be performed to correct the error condition (e.g., for a subsequent authentication attempt). )) includes (3914). In some examples, the indication of a user action indicates a way to correct an error condition for a subsequent authentication attempt. Displaying an indication of user actions that can be performed to correct the error condition provides feedback to the user on what course of action to take so that the user can be biometrically authenticated on a subsequent authentication attempt. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, no indicator is displayed during biometric authentication.

In some examples, an indication of an error condition (e.g., 3814A-3814I) includes information regarding the cause of the error condition (e.g., an indication of a user action and/or device condition, a visual representation (e.g., graphic or text)). Displaying an indication of the cause of the error condition provides feedback to the user on what course of action to take so that the user can be biometrically authenticated on subsequent authentication attempts. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, no indicator is displayed during biometric authentication.

In some examples, the set of error condition criteria is such that a biometric feature (e.g., fingerprint, face) of a first type (e.g., a type corresponding to the authorized biometric features) is detected by one or more biometric sensors of the electronic device. (e.g., 3803). In some examples, an indication of an error condition (e.g., 3814A-3814I) is not displayed if a potentially valid biometric characteristic is not detected (e.g., indicating that the user is not currently associated with the device). Withholding display of an indication of an error condition when no biometric features are detected is potentially confusing to the user because it is unlikely that the user intended to perform biometric authentication if no biometric features were detected. prevent it from happening. Accordingly, withholding display of indications in these scenarios makes the user-device interface more efficient (e.g., by helping users provide appropriate inputs and reducing user errors when operating/interacting with the device). This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, upon determining that authentication (e.g., biometric authentication) was successful, the electronic device (e.g., 100, 300, 500, 3800) displays an indication of an error condition (e.g., 3814A through 3802) on the display (e.g., 3802). 3814I) is reserved for display.

In some examples, following displaying an indication of an error condition (e.g., 3814A-3814I) and upon determining that the set of error condition criteria continues to be met, the electronic device (e.g., 100, 300, 500, 3800) , suspend attempts at biometric authentication on the electronic device (and, optionally, disable further attempts at biometric authentication) (e.g., the biometric authentication function may not available) (3918). In some examples, following displaying an indication of an error condition and determining that the set of error condition criteria is no longer met, the electronic device enables reattempting biometric authentication on the electronic device (e.g., The error condition no longer exists (e.g., has been corrected (e.g., due to a user taking action to correct the error condition)) 3922. Automatically retrying biometric authentication when a set of error condition criteria is no longer met allows users to quickly attempt to biometrically authenticate themselves without requiring the user to explicitly request biometric authentication. do. Performing optimized behavior when a set of conditions is met without requiring additional user input (e.g., by helping the user provide appropriate inputs when operating/interacting with a device and reducing user errors) ) improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, following displaying an indication of an error condition and in response to determining that the set of error condition criteria is no longer met, the electronic device reattempts authentication (e.g., biometric authentication) (e.g., authentication automatically retried) (3924). In some examples, reattempting authentication may involve attempting to match biometric information obtained by one or more biometric sensors with authorized credentials (e.g., stored data that has been authorized for use in biometric authentication). It includes In some examples, the determination that the error condition is not met occurs subsequent to (eg, in response to) receiving input to correct the error condition. In some examples, retrying authentication occurs (or only occurs) upon a determination that an error condition has not been met due to detecting user input that causes the error condition to not be met.

In some examples, following a determination that a set of error condition criteria is no longer met (e.g., detecting that the error condition has been corrected), the electronic device (e.g., 100, 300, 500, 3800) may: Through the devices, input corresponding to a request to retry authentication (e.g., 3824, 3826) is received. In some examples, the input is a touch gesture input (eg, tap, swipe (eg, up swipe)) or activation of a hardware button (eg, power button). In some examples, in response to receiving input corresponding to a request to retry authentication, the electronic device retries authentication (eg, biometric authentication) (eg, automatically retries authentication). In some examples, reattempting authentication may involve attempting to match biometric information obtained by one or more biometric sensors with authorized credentials (e.g., stored data that has been authorized for use in biometric authentication). It includes In some examples, reattempting authentication includes obtaining data of the user's biometric characteristics (eg, face, fingerprint) using one or more biometric sensors.

In some examples, displaying an indication of an error condition (e.g., 3814A-3814I) includes an animation (e.g., shimmering) indicating that an attempt to authenticate is in progress. In some examples, attempting to authenticate includes attempting to detect biometric information using one or more biometric sensors. Displaying a shimmering animation indicating that an authentication attempt is in progress provides the user with feedback about the current state of the device and that no further action is required at this time. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, no indicator is displayed during biometric authentication.

In some examples, subsequent to (or in response to) receiving a request to perform an operation requiring authentication and prior to displaying an indication of an error condition (e.g., 3814A through 3814I), the electronic device (e.g., 100, 300) , 500, 3800) performs authentication. In some examples, while performing authentication, the electronic device 3800 displays a biometric feature on a display (e.g., 3802) using one or more biometric sensors (e.g., 3803) of the electronic device. A first indication that information is being obtained (e.g., 3810, 3814A-3814I) (e.g., rings rotating about a sphere, shimmering user interface objects, where the user interface object includes an indication of an error condition) displays. In some examples, displaying an indication of an error condition includes replacing display of the first indication with a display of an indication of an error condition. Displaying an indication that biometric authentication is occurring provides feedback to the user regarding the current state of the device (e.g., that biometric authentication is being performed) and that the user does not need to take any action at this time. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, while performing authentication, the electronic device (e.g., 100, 300, 500, 3800) displays a first lock icon (e.g., 3806) on the display (e.g., 3802) (e.g., a locked state of the electronic device). icon representing) and a first animation transitioning from the first lock icon to the first display. In some examples, following displaying an indication of an error condition (e.g., and following a determination that authentication was successful) and following displaying the first animation, the electronic device displays an error message on the display (e.g., 3802). A second animation is displayed that transitions from displaying the condition to an unlock icon (e.g., 3816) (e.g., an icon representing a locked state of the electronic device). In some examples, the first animation and the second animation represent morphing from one object to the next object. In some examples, the second animation includes displaying the first lock icon subsequent to the indication of the error condition and before the unlock icon.

In some examples, following displaying an indication of an error condition, the electronic device displays a second lock icon (e.g., 3806) on the display (e.g., 3802), or the electronic device displays a lock icon (e.g., 3806). From a second indication (e.g., 3810, 3814A through 3814I) (e.g., rings rotating about a sphere) that information about a biometric characteristic is being obtained using one or more biometric sensors, a second lock icon (e.g., Displays an animation that transitions to an icon representing the locked state of the electronic device. In some examples, the second lock icon is the first lock icon. In some examples, the second indication is the first indication.

In some examples, while retrying authentication and subsequent to displaying an indication of an error condition and upon determining that there is no error condition, the electronic device may use one or more biometric sensors of the electronic device on the display. a third indication that information about a biometric feature is being obtained (e.g., 3810, 3814A-3814I) (e.g., rings rotating about a sphere, shimmering user interface objects, where the user interface object is an error condition) (including the indication of) is displayed. In some examples, the third indication is the first indication.

In some examples, prior to displaying an indication of an error condition, the electronic device (e.g., 100, 300, 500, 3800) displays on the display (e.g., 3802) a third lock icon (e.g., 3806) at a location on the display. ) (e.g., an icon indicating the locked state of the electronic device) is displayed. In some examples, an indication of the error condition (e.g., 3814A-3814I) is displayed proximate (e.g., near, adjacent, at, within a predetermined distance therefrom) the location on the display. In some examples, the third lock icon is the first lock icon and/or the second lock icon.

In some examples, when the electronic device is in a locked state while receiving a request to perform an operation requiring authentication and upon a determination that authentication is successful, the electronic device (e.g., 100, 300, 500, 3800) is in a locked state. Transition to unlocked state. In some examples, the act of requiring authentication is to transition the electronic device from a locked state to an unlocked state. In some examples, when an electronic device is in a locked state while receiving a request to perform an operation requiring authentication and upon a determination that authentication was not successful, the electronic device remains locked. Keeping the device locked when authentication is not successful improves device security by preventing fraudulent and/or unauthorized access to the device. Improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally reduces the device's power usage by enabling users to use the device more efficiently. reduces and improves battery life.

In some examples, when the electronic device is in a locked state while receiving a request to perform an action requiring authentication and upon a determination that authentication was not successful, the electronic device (e.g., 100, 300, 500, 3800) is locked. Maintain the state and retry authentication (e.g., biometric authentication) (e.g., automatically retry authentication). In some examples, reattempting authentication includes attempting to obtain information regarding a biometric feature (eg, face, fingerprint) using one or more biometric sensors of the electronic device. In some examples, reattempting authentication may involve attempting to match biometric information obtained by one or more biometric sensors with authorized credentials (e.g., stored data that has been authorized for use in biometric authentication). It includes In some examples, after reattempting authentication and following a determination that authentication was successful resulting from retrying authentication, the electronic device transitions from a locked state to an unlocked state. In some examples, after reattempting authentication and following a determination that authentication resulting from retrying authentication was not successful, the electronic device remains locked.

In some examples, subsequent to (or in response to) receiving a request to perform an operation requiring authentication, the electronic device (e.g., 100, 300, 500, 3800) attempts authentication (e.g., biometric authentication). do. In some examples, while attempting authentication, the electronic device displays information on a display (e.g., 3802) regarding a biometric characteristic (e.g., face, fingerprint) using one or more biometric sensors of the electronic device. Displays a third indication (eg, 3810, 3814A to 3814I) (eg, rings rotating about a sphere) that is obtaining. In some examples, the display is a scanning animation. In some examples, the third indication is the first indication and/or the second indication. In some examples, while retrying authentication, the electronic device maintains display of the third indication on the display (e.g., 3802).

In some examples, upon a determination that authentication was not successful resulting from retrying authentication, the electronic device displays a lock icon on the display (e.g., 3802) that alternates between a first position and a second position (e.g., 3806) (e.g., an icon representing a locked state of the electronic device), where the second position is different from the first position. In some examples, the animation with the lock icon is an animation of the lock icon shaking (e.g., rotating left and right, back and forth). In some examples, the electronic device displays an animation accompanying a lock icon to indicate that biometric authentication has failed. In some examples, tactile output is provided in combination with a shaking lock icon. In some examples, no tactile output is provided. In some examples, upon determining that biometric information captured using one or more biometric sensors does not correspond or does not match the authorization credentials, the electronic device (e.g., 100, 300, 500, 3800) Maintains the locked state. Displaying an animation of a shaking lock icon provides feedback to the user regarding the current state of the device (eg, biometric authentication has failed) and prompts the user to take further action. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, the electronic device (e.g., 100, 300, 500, 3800) includes a biometric sensor (e.g., 3803), and the set of error condition criteria includes one or more of the following error condition criteria:

The distance of the biometric feature from the biometric sensor exceeds a first predetermined threshold distance (e.g., the biometric feature (e.g., face) is too far from the biometric sensor) or the maximum of the distance range (e.g., 3812). exceeds In some examples, exceeding the first predetermined threshold or maximum of the distance range is highly correlated with reduced accuracy or degradation of information regarding the biometric feature acquired by the biometric sensor. In some examples, a user can compensate for this error condition by moving the user's face closer to the biometric sensor.

The distance of the biometric feature from the biometric sensor is less than a second predetermined threshold distance (e.g., the biometric feature (e.g., face) is too close to the biometric sensor) or less than the minimum of the distance range (e.g., 3812). falls to In some examples, falling below the second predetermined threshold or minimum of the distance range is highly correlated with reduced accuracy or degradation of information regarding the biometric characteristic acquired by the biometric sensor. In some examples, a user can compensate for this error condition by moving the user's face further away from the biometric sensor.

The biometric sensor (e.g., 3803) is occluded (e.g., partially occluded, fully occluded, occluded sufficiently to inhibit operation of the sensor) (e.g., while interacting with the electronic device, a portion of the user (e.g., a hand) is occluded. ) obscured by ). In some examples, the user can correct for this error condition by moving the user's hand away from the biometric sensor.

A subportion of the detected biometric feature (e.g., an eye of the detected face) is not oriented toward the biometric sensor (e.g., one or more eyes are not focused on the electronic device (e.g., biometric sensor)) . In some examples, a user can correct for this error condition by opening the user's eyes or looking at an electronic device (eg, a biometric sensor).

At least a portion of the detected biometric feature is occluded (e.g., partially occluded, fully occluded, occluded to a sufficient extent to result in incomplete information about the biometric feature). In some examples, a user can correct for this error condition by removing an accessory (eg, sunglasses) or clothing (eg, scarf, hat) that is blocking the user's face.

No biometric features are detected within the field of view (e.g., 3828) of the biometric sensor.

The pose (e.g., orientation relative to the biometric sensor) of the detected biometric feature exceeds a threshold range (e.g., the biometric feature (e.g., face) is turned away from the biometric sensor). In some examples, exceeding the threshold range is highly correlated with reduced accuracy or degradation of information about the biometric characteristic acquired by the biometric sensor. In some examples, a user can correct for this error condition by turning the user's face toward the electronic device (eg, a biometric sensor).

The electronic device detects (e.g., via one or more ambient light sensors) an amount of light (e.g., ambient light) that exceeds a predetermined light threshold (e.g., that exceeding the predetermined light threshold indicates that the biometric sensor captures the biometric sensor). highly correlated with reduced accuracy or degradation of information about the measurement feature). In some examples, a user may correct this error condition by turning the user's back toward the sun to reduce the amount of light detected by the electronic device or by moving to a new location (e.g., indoors) with less ambient light. .

In some examples, the set of error condition criteria may be a first subset of the error conditions listed above. For example, the first subset may include: a distance of the biometric feature is greater than a first predetermined threshold distance, a distance of the biometric feature is less than a second predetermined threshold distance, and the biometric feature is a biometric sensor. is outside the field of view, and the pose of the biometric feature exceeds a threshold range. The first subset focuses on guiding the user to correct error conditions involving orientation and/or positioning of the face. As a further example, the second subset may include one or more error condition criteria selected from the group consisting of: the biometric sensor is occluded, and no biometric feature is detected within the field of view of the biometric sensor. The second subset focuses on guiding the user to correct for error conditions where the biometric sensor cannot obtain any information about the user's biometric characteristics. As another example, the third subset may include one or more error condition criteria selected from the group consisting of the pose of the detected biometric feature exceeding a threshold range and the biometric sensor is occluded. The third subset focuses on error conditions that are likely to occur for devices of a given form factor/size (eg, tablet device (eg, iPad)).

In some examples, the electronic device (e.g., 100, 300, 500, 3800) includes a biometric sensor (e.g., 3803) in a portion (e.g., location) of the electronic device (e.g., a portion that is not on the display). In some examples, in response to a request to perform an action requiring authentication, the electronic device displays a display (e.g., 3802) proximate to (e.g., adjacent to, near, at, or at a predetermined distance from) a portion of the electronic device. within), and display progress indicators (e.g., 3814A through 3814I) that include an indication of an error condition. Displaying a progress indicator near the biometric sensor provides the user with feedback about the correlation of the biometric sensor with processes occurring on the device (eg, attempted authentication). Specifically, the user may notice the biometric sensor during biometric authentication, such that the user is less likely to perform an action that would interfere with the biometric sensor, or alternatively, the user may be prompted to take corrective action. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, the indication of an error condition (e.g., 3814B) includes an indication that biometric authentication is not currently enabled on the electronic device in accordance with a determination that biometric authentication is not currently enabled on the electronic device. In some examples, biometric authentication may become unavailable (or not enabled on an electronic device) when one or more of the following conditions are met: The electronic device is being turned on or restarted successfully. not certified; The electronic device has not been unlocked for more than a predetermined period of time (e.g., 48 hours); The passcode has not been used to unlock the device for more than a predetermined amount of time (e.g., 156 hours); Biometric authentication using biometric features (e.g., face, fingerprint) was not used to unlock the device for more than a predetermined time (e.g., 4 hours); The electronic device has received a remote lock command; biometric authentication has failed more than a predetermined number of times (e.g., 5, 10, 15) since the last successful authentication by the device; The electronic device has received a power-off and/or emergency SOS command; Then, an explicit request by the user to disable biometric authentication was detected. Displaying an indication that biometric authentication is not currently enabled provides the user with feedback of the current state of the device and prompts the user to pursue alternative methods to authenticate themselves. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, the indication that biometric authentication is not currently enabled includes an affordance (e.g., 3814B) (e.g., the indication is an affordance). In some examples, the electronic device (e.g., 100, 300, 500, 3800) receives an input corresponding to an affordance (e.g., 3820), and in response to receiving the input corresponding to the affordance, the electronic device (e.g., 100) , 300, 500, 3800) displays a credential entry user interface (e.g., 3822A) on a display (e.g., 3802) with a plurality of character input keys. In some examples, the credential entry user interface includes a virtual keypad or virtual keyboard. In some examples, the virtual keypad or virtual keyboard includes a plurality of character input keys.

In some examples, the electronic device (eg, 100, 300, 500, 3800) detects a condition that triggers attempting authentication (eg, biometric authentication). In some examples, the request to perform an action requiring authentication includes a request to unlock the device (eg, swipe at a predefined location). In some examples, in response to detecting a condition that triggers attempting authentication (e.g., biometric authentication) while the biometric feature is available for detection by one or more biometric sensors ( e.g., a face is detected in the field of view of one or more face detection sensors, such as a depth camera) an alert generated by a device without user input to the device (e.g., based on satisfaction of criteria other than detection of user input) (e.g., 3808 In accordance with a determination that corresponds to rings rotating around the center). In some examples, the condition is such that the biometric feature is not available for detection by one or more biometric sensors (e.g., no face is detected in the field of view of one or more face detection sensors, such as a depth camera) ( In accordance with a determination to respond to an alert generated by the device without user input to the device (e.g., based on satisfaction of criteria other than detection of user input), the electronic device may use one or more biometric sensors of the electronic device. thus withholding display of a fifth indication (eg, rings rotating about a sphere) that information about the biometric characteristic is being obtained. In some examples, the condition is a user input to the device (e.g., a request not associated with a notification; a touch gesture input (e.g., a tap, a swipe (e.g., 3824) (e.g., an upward swipe) or a hardware button (e.g., a power Upon determining that the electronic device responds to a request that is a button) or activation of sensor data indicative of movement (e.g., lifting) of the device, the electronic device may use one or more biometric sensors of the electronic device (e.g., Displays a fifth indication that information is being obtained regarding the biometric characteristic (regardless of whether the biometric characteristic is available for detection by one or more biometric sensors). Withholding display of an indication when no face is detected avoids potentially confusing the user, as the user is unlikely to intend to initiate biometric authentication if no biometric feature is detected. . Accordingly, withholding display of indications in these scenarios makes the user-device interface more efficient (e.g., by helping users provide appropriate inputs and reducing user errors when operating/interacting with the device). This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

Note that the details of the processes described above with respect to method 3900 (e.g., FIGS. 39A and 39B) are also applicable in a similar manner to the methods described below/above. For example, method 3700, method 4100, and/or method 4300 optionally include one or more of the features of the various methods described above with respect to method 3900. For example, error indications (e.g., 3814A-3814I) as described in connection with method 3900 may occur during biometric authentication performed in the processes described in connection with method 3700 and method 4100. Can be used to provide indications of error conditions. For brevity, these details are not repeated below.

40A-40U illustrate example user interfaces for providing indications regarding a biometric sensor during biometric authentication, according to some examples. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 41A-41C.

FIG. 40A illustrates an electronic device 4000 (e.g., portable multifunction device 100, device 300, device 500). In the illustrative examples illustrated in FIGS. 40A-40U, electronic device 4000 is a tablet computer. In other examples, electronic device 4000 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 4000 includes a display 4002, one or more input devices (e.g., a touchscreen of display 4002, buttons 4004, and microphone), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In Figure 40A, the electronic device includes a biometric sensor 4003. In some examples, the biometric sensor is one or more biometric sensors that may include a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, biometric sensor 4003 is biometric sensor 703. In some examples, the one or more biometric sensors include one or more fingerprint sensors (eg, a fingerprint sensor integrated into a button). In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In Figure 40A, the user wants to purchase products using payment information stored on the electronic device 4000. As shown in FIG. 40A, the electronic device 4000 is in split screen (eg, multitasking) mode. While in split screen mode, the electronic device 4000 displays an app store user interface (UI) 4006 on the left area 4007 of the display 4002 and a browser UI (4006) on the right area 4009 of the display 4002. 4008) are displayed simultaneously. While simultaneously displaying app store UI 4006 and browser UI 4008, electronic device 4000 receives input 4010 at purchase affordance 4012.

40B, in response to receiving input 4010 at purchase affordance 4012, electronic device 4000 selects the applications being displayed in left region 4007 and right region 4009 of display 4002. Swapping. Specifically, the electronic device 4000 displays a browser UI 4008 in the left area 4007 and an app store UI 4006 in the right area 4009. Electronic device 4000 swaps applications to place applications associated with products being purchased in areas closer to biometric sensor 4003. By placing the browser UI 4008 in the left area 4007, the electronic device 4000 provides the user with an indication of the location of the biometric sensor 4003, which allows the user to It is used for authentication. As shown in Figure 40B, swapping applications also places the application associated with the products being purchased in an area closer to button 4004. In some examples, if button 4004 and biometric sensor 4003 are not in close proximity (e.g., on the same side), electronic device 4000 may swap applications, if necessary, to determine the product being purchased. Place the applications associated with them in an area closer to the biometric sensor 4003. In some examples, if button 4004 and biometric sensor 4003 are not in close proximity (e.g., on the same side), electronic device 4000 may swap applications, if necessary, to determine the product being purchased. Applications associated with them are placed in an area closer to the button 4004.

Additionally, as shown in FIG. 40B, in response to receiving input 4010 at purchase affordance 4012, electronic device 4000 darkens browser UI 4008 while displaying app store UI 4006. ) to a greater extent than that of the browser UI 4008. By making the browser UI 4008 less dark than the app store UI 4006, the electronic device 4000 indicates to the user which application is associated with the payment sheet interface 4014 and which products the user wishes to purchase.

Additionally, in response to receiving input 4010 at purchase affordance 4012, electronic device 4000 displays a payment sheet interface 4014 with information about the products being purchased and a payment sheet interface 4014 with information about the products being purchased. It simultaneously displays a prompt 4016 prompting you to double-click button 4004 to initiate the process to authorize the payment. Additionally, in response to receiving input 4010 at purchase affordance 4012, electronic device 4000 displays dynamic indication 4018 to highlight the location of button 4004. While displaying payment sheet interface 4014, the electronic device receives input 4020 at button 4004 (e.g., double pressing button 4004). In some examples, prompt 4016 includes some or all of the features of prompt 2416. In some examples, dynamic indicia 4018 includes some or all of the features of dynamic indicia 2418. In some examples, payment sheet interface 1014 includes the name of the application to which it corresponds (eg, the name of the application with which the user initiated the process to authorize the payment).

In Figure 40C, in response to receiving input 4020 at button 4004, electronic device 4000 initiates a process to authorize payment for products. Authorizing payment for products requires successfully authenticating the user. As a result, in response to receiving input 4020, electronic device 4000 initiates biometric authentication using biometric sensor 4003. After initiating biometric authentication, electronic device 4000 displays facial glyph 4022, which indicates that electronic device 4000 is attempting to biometrically authenticate a user (e.g., using biometric sensor 4003). Provides an indication that an attempt is being made to obtain biometric information about the user. In some examples, facial glyph 4022 includes some or all of the features of glyph 2468 in Figure 24F. In some examples, in response to receiving input 4020 at button 4004, the electronic device selects face glyph 4022 from the location of prompt 4016, as shown in FIG. 40C. ) displays an animation moving to the position. In some examples, the animation is such that the face glyph 4022 appears to slide out of the prompt 4016.

40D, after displaying facial glyph 4022, the electronic device transitions to displaying authentication glyph 4024, which indicates that electronic device 4000 is attempting to biometrically authenticate the user (e.g., Provides an indication that ongoing attempts are being made to obtain measurement information and to match the obtained information with stored authorized credentials. Authentication glyph 4024 includes a plurality of rings rotating into a sphere. In some examples, authentication glyph 4024 includes some or all of the features of one or more rings 2470 of FIG. 24G.

While displaying authentication glyph 4024, electronic device 4000 detects that an error condition exists (e.g., a condition that prevents biometric sensor 4003 from obtaining sufficient information about the user's face). Specifically, the electronic device 4000 detects that the biometric sensor 4003 is covered by a physical object (eg, the user's hand). In some examples, electronic device 4000 may obtain sufficient information about the user's face without detecting an error condition. In some examples, after obtaining sufficient information about the user's face, and while displaying authentication glyph 4024, electronic device 4000 determines whether the obtained information satisfies biometric authentication criteria ( For example, determine whether the obtained biometric information matches, within a threshold, a biometric template associated with the user (e.g., stored authorized credentials). In some examples, upon determining that biometric authentication is successful (e.g., biometric authentication criteria are met), electronic device 4000 transitions to an unlocked state.

40E, in response to detecting that an error condition exists, electronic device 4000 displays an error indication 4026 at a location on top of display 4002 (e.g., relative to the ground, relative to the user). ) is displayed. Error indication 4026 provides an indication of the error condition currently present. Additionally, in response to detecting that an error condition exists, the electronic device 4000 displays an error icon 4028 at a location on the display 4002 adjacent the biometric sensor 4003, thereby Provides an indication of the location of sensor 4003. By providing an indication of the location of biometric sensor 4003, error icon 4028 suggests to the user the cause of the error condition. In some examples, in response to detecting that an error condition exists, electronic device 4000 displays an error indication 4026 at a location adjacent biometric sensor 4003. In some examples, error indication 4026 includes some or all of the features of error indication 3814A, including a shimmer effect.

40F, further in response to detecting that an error condition exists, the electronics 4000 moves the payment sheet interface 4014 closer to the biometric sensor 4003 from its initial position in FIG. 40E. Displays an animation moving to the position at 40f.

. By moving the payment sheet interface toward the biometric sensor 4003, the electronic device 4000 indicates to the user the location of the biometric sensor 4003 as well as the presence of the error icon 4028 (and thus , suggests to the user the cause of the error condition).

In some examples, error icon 4028 is displayed at different locations on display 4002 depending on the positioning of the user's hand on display 4002. As illustrated in Figure 40F, the user's hand covers a portion of the display 4002 adjacent to the biometric sensor 4003. While the user's hand is in contact with the display 4002, the electronic device 4000 detects input as a result of the contact from the user's hand. In response to detecting this input, the electronic device 4000 displays an error icon 4028 at the location where the input is not detected. As another example, in Figure 40G, the user's hand covers a smaller portion of the display 4002 than the user's hand in Figure 40F. In some examples, in response to detecting input of the user's hand in Figure 40G, electronic device 4000 displays error icon 4028 at a different location than in Figure 40F, where the error icon 4028 is displayed in Figure 40G. The location is closer to the biometric sensor 4003 than in Figure 40F. As another example, in Figure 40H, the user's hand covers much of the upper left side of display 4002. In some examples, in response to detecting input of the user's hand in Figure 40H, electronic device 4000 displays error icon 4028 at a different location than the positions in Figures 40F and 40G. Specifically, in some examples, the electronic device 4000 displays an error icon 4028 at a location close to (or substantially near) the biometric sensor 4003 if it is not at a location where input from the user's hand is detected. Display.

In Figure 40I, the user moves her hand away so that it no longer covers the biometric sensor 4003. Error icon 4028 and while displaying error icon 4028, electronic device 4000 detects that the error condition no longer exists.

40J, in response to detecting that an error condition no longer exists, electronic device 4000 automatically reattempts biometric authentication. While retrying biometric authentication, electronic device 4000 displays authentication glyph 4024. While displaying authentication glyph 4024, electronic device 4000 attempts to biometrically authenticate the user. Specifically, the electronic device 4000 uses the biometric sensor 4003 to obtain information about the user's face and determines whether biometric authentication is successful (e.g., the obtained information matches stored authorized credentials). ) determine whether

While retrying biometric authentication, the electronic device 4000 determines that biometric authentication was successful. 40K, upon determining that biometric authentication was successful, electronic device 4000 displays success glyph 4030, which provides an indication that biometric authentication was successful. In some examples, success glyph 4030 includes some or all of the features of glyph 2474.

40L , further in response to determining that biometric authentication is successful, electronic device 4000 displays a processing indicator 4032, which provides an indication that a payment transaction is being processed (e.g., electronic device 4000 (4000) is transmitting payment information (e.g., credentials) to authorize payment to an external device (e.g., server). In some examples, processing indicator 4032 has some or all of the features of processing indicator 2476.

40M, upon receiving an indication that a payment has been completed (e.g., authorized), electronic device 4000 displays a completed indication 4034, which provides an indication that the payment has been completed. In some examples, done indication 4034 has some or all of the features of done indication 2478 of Figure 24ar.

40N-40S illustrate a technique for displaying error indication 4026 and error icon 4028 when error indication 4026 and error icon 4028 should be displayed at approximately the same location. In Figure 40N, the user wishes to unlock the device to access restricted content (eg, home screen, most recently used applications). Figure 40N shows the electronic device 4000 in a vertical orientation relative to the ground, with the user covering the biometric sensor 4003 with her hand. Additionally, the electronic device 4000 displays a locked status UI 4036 with a lock icon 4038. Lock icon 4038 provides an indication that electronic device 4000 is in a locked state.

While displaying the locked state UI 4036, the electronic device 4000 receives a request to unlock the device. For example, electronic device 4000 detects that a user lifts the device from a substantially horizontal position.

40O, in response to receiving a request to unlock the device, electronic device 4000 attempts to biometrically authenticate the user. While attempting to biometrically authenticate a user, electronic device 4000 displays authentication glyph 4024. Additionally, while attempting to biometrically authenticate a user, electronic device 4000 determines that an error condition exists (e.g., a condition that prevents biometric sensor 4003 from obtaining sufficient information about the user's face). detect that Specifically, the electronic device 4000 detects that the biometric sensor 4003 is covered by a physical object (eg, the user's hand).

40P, in response to detecting that an error condition exists, electronic device 4000 places an error message at a location on display 4002 near biometric sensor 4003 (e.g., on top of display 4002). Display icon 4028. Additionally, in response to detecting that an error condition exists, the electronic device 4000 determines that the error indication 4026 should be displayed at approximately the same location as the error icon 4028. Upon determining that the error indication 4026 should be displayed at approximately the same location, the electronic device 4000 does not immediately display the error indication 4028, as described below with respect to FIGS. 40Q and 40R. Instead, it displays the error indication 4026 as part of an animation that transitions from the error icon 4028 to the lock icon 4038 for the error indication 4026.

40Q, after displaying error icon 4028, electronic device 4000 displays error indication 4026 (e.g., replacing the display of error icon 4028 with it), as discussed above. Likewise, it provides an indication of the cause of the error condition.

While displaying the error indication 4026, the user removes her hand from the biometric sensor 4003 so that it no longer covers the biometric sensor 4003. In response to detecting that the error condition no longer exists, electronic device 4000 automatically reattempts biometric authentication.

40R and 40S, when it is determined that authentication is successful as a result of retrying biometric authentication, the electronic device 4000 transitions from the locked state to the unlocked state. Specifically, the electronic device 4000 displays an animation of the lock icon 4038 transitioning to the unlock icon 4040 (e.g., replacing the display of the error indication 4026 therewith), which allows the user to use the electronic device. (4000) provides an indication that it has transitioned to the unlocked state. In some examples, instead of successful biometric authentication, electronic device 4000 determines that authentication was not successful as a result of retrying biometric authentication. In some examples, upon determining that authentication was not successful, electronic device 4000 displays a passcode entry UI that, when activated, has the affordance of triggering retrying biometric authentication. In some examples, while retrying biometric authentication, electronic device 4000 darkens all portions of display 4002 except the user interface associated with retrying biometric authentication.

FIG. 40T illustrates a technique for displaying an error icon 4028 when the error icon 4028 should be displayed at approximately the same location as one of the notifications being displayed (e.g., 4044A through 4044D). In some examples, a user wishes to view limited content of one or more of the notifications displayed while electronic device 4000 is in a locked state (eg, 4044A-4044D). As shown in Figure 40T, the user is covering the biometric sensor 4003 with her hand when the electronic device is in a portrait orientation, where the biometric sensor 4003 is located near the bottom of the device. In some examples, while attempting to biometrically authenticate a user to access restricted content of notifications, electronic device 4000 may detect an error condition as a result of the user covering biometric sensor 4003 with her hand. detect that it exists. In response to detecting that an error condition exists, the electronic device 4000 determines that the error icon 4028 should be displayed at approximately the same location as one of the notifications (e.g., 4044A through 4044D). Upon making this determination, and in response to detecting that an error condition exists, the electronic device 4000 displays a UI element 4042 (e.g., a background) concurrently with the error icon 4028, such that the error icon 4028 ) provides a background to overlay the display. As shown in Figure 40T, UI element 4042 is opaque, so that the notification overlaid with error icon 4028 (e.g., 4042D) is not visible to the user. In some examples, UI element 4042 is transparent, allowing the notification overlaid with error icon 4028 to be visible to the user.

Figure 40T also illustrates a technique for hiding the unlock indication 4044 of Figure 40U when the error icon 4028 should be displayed at approximately the same location as the unlock indication 4044. In some examples, electronic device 4000 displays unlock indication 4044, which provides an indication of the approximate location on display 4002 where a user can begin swiping upward to initiate biometric authentication. . In some examples, while displaying the unlock indication 4044, the electronic device 4000 detects that an error condition exists as a result of the user covering the biometric sensor 4003 with her hand. In some examples, in response to detecting that an error condition exists, the electronic device 4000 determines that the error icon 4028 should be displayed at approximately the same location as the unlock indication 4044. In some examples, when making this determination, and in response to detecting that an error condition exists, the electronic device 4000 stops displaying the unlock indication 4044 and displays the unlock indication 4044. An error icon 4028 is displayed at approximately the same location as the error icon 4028.

While displaying the error icon 4028, the electronic device 4000 detects that an error condition no longer exists (e.g., due to the user removing her hand from the biometric sensor 4003). As shown in Figure 40U, the user removed her hand from biometric sensor 4003. 40U, upon detecting that an error condition no longer exists, the electronic device 4000 stops displaying the error icon 4028 and resumes the unlock indication 4044 in the location where it was previously displayed. Display.

41A-41C are flow diagrams illustrating a method for providing indications regarding a biometric sensor during biometric authentication, according to some examples. Method 4100 includes a device having a display (e.g., 4002) and a biometric sensor (e.g., 4003) (e.g., a plurality of biometric sensors) in a first portion (e.g., a portion that is not part of the display) of the electronic device. 1 An electronic device (e.g., 100, 300, 500, 4000). In some examples, the biometric sensor includes one or more cameras. Some operations of method 4100 are optionally combined, the order of some operations is optionally varied, and some operations are optionally omitted.

As described below, method 4100 provides an intuitive method for providing indications regarding a biometric sensor during biometric authentication. This method reduces the user's cognitive burden for performing biometric authentication, thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to perform biometric authentication faster and more efficiently saves power and increases time between battery charges.

An error condition that prevents an electronic device (e.g., 100, 300, 500, 4000) from obtaining biometric information about a user of the device (e.g., a non-contact biometric sensor such as a thermal or optical facial recognition sensor). presence is obscured (e.g., partially obscured, completely obscured, obscured sufficiently to inhibit operation of the sensor) (e.g., by a portion of the user (e.g., hands) while interacting with the electronic device). occlusion) is detected 4102 (e.g., detected in response to a request to perform an operation requiring authentication).

In response to detecting the presence of an error condition (4104), the electronic device (e.g., 100, 300, 500, 4000) displays an error indication (e.g., 4028) (e.g., a graphic icon) on the display (e.g., 4002). ) is displayed. In some examples, the error indication includes text (eg, indicating that the sensor is occluded). In some examples, the error indication does not include text. An error indication is displayed 4106 at a location proximate the first portion of the electronic device. In some examples, the location is at or near the portion of the display closest to the location of the biometric sensor (eg, 4003). Displaying an error indication provides feedback to the user regarding the current state of the device (e.g., that an error condition is preventing successful biometric authentication) and prompts the user to take further action to correct the error condition. . Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently. Displaying an error indication near a biometric sensor provides feedback to the user about the association of the biometric sensor with a process occurring on the device (eg, attempted authentication). Specifically, the user may notice the biometric sensor during biometric authentication, such that the user is less likely to perform an action that would interfere with the biometric sensor, or alternatively, the user may be prompted to take corrective action. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, the error indication (e.g., 4028) includes a biometric sensor occluded icon and a reticle, and the error indication provides an indication that the biometric sensor is occluded (4108). (ISE, error indication is associated with an electronic device that performs biometric authentication (e.g., using a biometric sensor to obtain biometric information about biometric features (e.g., face, fingerprint)).) Biometric Providing an indication that a sensor is occluded provides feedback to the user regarding the current state of the device (e.g., biometric sensor is occluded) and prompts the user to take further action to correct the error condition. Accordingly, providing improved feedback with instructions for appropriate movements of biometric features (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device) It improves the operability of the device and makes the user-device interface more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

Upon determining that the user interface of the electronic device is in a first orientation relative to the biometric sensor (4110), the electronic device (e.g., 100, 300, 500, 4000) is proximate (e.g., adjacent to) the first portion of the electronic device. , displays an error indication at a first location in the user interface (near it, within its predetermined distance).

Upon determining that the user interface of the electronic device is in a second orientation relative to the biometric sensor (4112), the electronic device (e.g., 100, 300, 500, 4000) is proximate (e.g., adjacent to) the first portion of the electronic device. , displaying an error indication (e.g., 4028) at a second location in the user interface (proximate thereto, within a predetermined distance thereof), the first orientation being different from the second orientation.

In some examples, while attempting to obtain 4114 biometric information using a biometric sensor (e.g., 4003), an electronic device (e.g., 100, 300, 500, 4000) displays on a display (e.g., 4002) Display 4116 a first progress indicator (e.g., 4024, 4026, 4038, 4040). In some examples, the first progress indicator provides an indication of a current state of the electronic device (e.g., locked, unlocked, performing biometric authentication, error state, error condition). In some examples, upon determining that the user interface of the electronic device (e.g., 4006, 4008) is in a third orientation relative to the biometric sensor—the user interface in the third orientation has a first upper side—4118; The electronic device displays the first progress indicator proximate (eg, adjacent to, near, within a predetermined distance thereof) a first upper side of the user interface in the third orientation. In some examples, upon determining that the user interface of the electronic device is in a fourth orientation relative to the biometric sensor—the user interface in the fourth orientation has a second top side—(4120), the electronic device is in the fourth orientation. displays a first progress indicator proximately (e.g., adjacent to, near, within a predetermined distance thereof) a second upper side of the user interface, and the third orientation is different from the fourth orientation. In some examples, the first progress indicator is displayed on the display at a location closest to or proximate to (eg, adjacent to, proximate to, within a predetermined distance thereof) the biometric sensor. Displaying the first progress indicator near the top of the display, regardless of orientation, ensures that the user is more likely to recognize the feedback provided to the user (eg, the progress indicator). Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, no indicator is displayed during biometric authentication.

In some examples, the electronic device (e.g., 100, 300, 500, 4000) displays its second progress indicator (e.g., 4024, 4026, 4038, 4040) on the display (e.g., 4002). . In some examples, the second progress indicator provides an indication of the current state of the electronic device (e.g., locked, unlocked, performing biometric authentication, error state). In some examples, the first progress indicator is a second progress indicator. In some examples, the second progress indicator may include a first portion (e.g., an indication (e.g., rotating rings) that the electronic device is performing biometric authentication using a biometric sensor (e.g., 4024)) and a first portion. A different second portion (e.g., an error condition or indication of an error state (e.g., 4026), an indication of the current locked or unlocked state of the electronic device (e.g., a lock icon (e.g., 4038), an unlock icon (e.g., 4040) It is an animation with ))). In some examples, upon determining that the second progress indicator is displayed at a location proximate to the first portion of the electronic device, the electronic device displays an error as part of the animation after the first portion and before the second portion (e.g., 4028). displays.

In some examples, the electronic device (e.g., 100, 300, 500, 4000) is displayed on the display (e.g., 4002) at a third location within the user interface (e.g., proximate a side (e.g., lower side) of the user interface). a home affordance (e.g., 4044) (e.g., an indication of the location of a gesture that, when performed, results in displaying the home screen, such as a swipe upward gesture from an edge of the display or a tap gesture on an affordance) Display. In some examples, upon determining that an error indication (e.g., 4028) is to be displayed at the third location, the electronic device stops displaying the home affordance (e.g., 4044) while displaying the error indication at the third location. Suspending display of the home affordance while displaying an error indication allows the user to quickly recognize that the home affordance is not accessible because there is an error and prompts the user to take further action to correct the error condition. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, no indicator is displayed during biometric authentication.

In some examples, after stopping displaying a home affordance (e.g., 4044), the electronic device (e.g., 100, 300, 500, 4000) may detect a biometric sensor (e.g., 4003) to collect biometric information about a user of the device. Detect correction of error conditions that prevent obtaining. In some examples, the electronic device detects the absence of an error condition after displaying an error indication (e.g., 4028) in the third location. In some examples, in response to detecting correction of the error condition, the electronic device displays the home affordance on a display (e.g., 4002), in a third location within the user interface (e.g., and an error indication (e.g., 4028) ) stops displaying ).

In some examples, the electronic device (e.g., 100, 300, 500, 4000) receives input (e.g., , palm, and fingers) are detected. In some examples, in response to detecting input at a location proximate to the first portion of the electronic device, the electronic device displays an error indication at a different location (e.g., 4028) on the display. In some examples, the different location is a location where input is not detected. In some examples, before displaying an error indication at a new location, the electronic device determines a different location based on the location of the input to the display. In some examples, the different location is proximate to a location proximate to the first portion of the electronic device. In some examples, the error indication is initially displayed in a first location proximate the first portion of the electronic device and then moved to a different location. In some examples, the error indication is initially displayed at a location chosen to be away from any areas of the display that are known to be occluded (eg, occluded by the detected touch input). Displaying error indications at different locations depending on the location of the input (e.g., the user's hand) provides feedback to the user regarding the current state of the device (e.g., that an error condition is preventing successful biometric authentication) and Prompts the user to take additional action to correct the error condition. Additionally, by adjusting the position, the device ensures that error indications are visible to the user, and thus the user is more likely to take corrective action at the device. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, the electronic device (e.g., 100, 300, 500, 4000) displays its predetermined display proximate (e.g., adjacent to, proximate to) a first portion of the electronic device, on a display (e.g., 4002). A first transaction interface (e.g., 4014) (e.g., separate from (or overlaid on top of) a user interface) at a location (within distance) of the transaction (or payment) containing transaction information such as a credit card number, billing address, etc. interface). In some examples, the first transaction interface is configured to receive input (e.g., 4010) corresponding to an affordance (e.g., 4012) of a user interface (e.g., 4008) (e.g., an affordance for making a payment or completing a transaction). It is displayed in response.

In some examples, displaying the first transaction interface (e.g., 4014). and displaying an animation in which the first transaction interface transitions (eg, translates) from an initial position substantially centered with respect to the display to a position proximate to the first portion of the electronic device. In some examples, the animation includes displaying (e.g., maintaining display thereof) the first transaction interface while the first transaction interface transitions (e.g., translates) from an initial position to a position proximate to the first portion of the electronic device. Includes. In some examples, the animation includes a visual effect that causes the first transaction interface to appear to float during the transition.

In some examples, the electronic device (e.g., 100, 300, 500, 4000) displays a prompt (e.g., 4016) on the display (e.g., 4002) to provide one or more activations of a hardware button (e.g., 4004) of the electronic device. ) is displayed. In some examples, the electronic device prompts the user by displaying “double click for Apple Pay.” In some examples, the prompt is displayed adjacent to the button. In some examples, the prompt is displayed when the device is displaying a transaction user interface area (e.g., 4014) but is not receiving any indication that a transaction terminal is nearby and requesting transaction credentials (e.g., A prompt is displayed to provide one or more activations of a button before the device is placed within the NFC field of the NFC reader requesting payment information). In some examples, the hardware button is a mechanical button or solid state button. In some examples, the button is a switch or any other type of toggle. In some examples, the button has a fixed position relative to the electronic device, particularly relative to the display of the electronic device, such that the electronic device can display prompts based on the position of the button. In some examples, the button is a solid state button that operates upon capacitive and/or resistive touch and/or responds to changes in the intensity of the input without having a mechanical switch pressed to activate the button, and instead Monitor whether the intensity exceeds an intensity threshold corresponding to activation of the solid-state button. In some examples, an electronic device (e.g., 100, 300, 500, 4000) receives one or more activations (e.g., 4020) of a hardware button of the electronic device and in response to receiving one or more activations of a hardware button, The electronic device displays an authentication progress indicator (eg, 4022, 4024, 4030, 4032, 4034) on a display (eg, 4002). In some examples, displaying the authentication progress indicator includes displaying an animation of the authentication progress indicator transitioning from the position of the prompt (e.g., 4016) to the final position of the authentication progress indicator. In some examples, the authentication indicator provides the status of authentication (eg, in progress, successful, not successful). In some examples, the animation includes displaying (eg, maintaining its display) the authentication progress indicator while the authentication progress indicator transitions (eg, translates) from the position of the prompt to the final position. In some examples, the animation includes a visual effect in which the authentication progress indicator appears to slide out of the prompt. In some examples, the authentication progress indicator is displayed with (or overlaid on) a user interface (e.g., 4014) (or transaction user interface area). Prompting the user to activate a hardware button guides the user to perform an action to complete the transaction on the device. Prompting the user in this way improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and makes the user-device interface more efficient. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. Displaying an authentication progress indicator provides feedback to the user regarding the status of authentication. Improved feedback improves the operability of the device (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device) and makes the user-device interface more efficient, This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently.

In some examples, an electronic device (e.g., 100, 300, 500, 4000) corresponds to a first application (e.g., 4006, 4008) in a first area (e.g., 4007, 4009) on a display (e.g., 4002). ) and simultaneously displaying a second application (e.g., corresponding to 4006, 4008) in a second area (e.g., 4007, 4009), the second application being adjacent to (e.g., next to) the first application. close to him, within his predetermined distance) (4122). In some examples, the electronic device displays 4124 a second transaction interface (e.g., 4014) on the display. In some examples, the second transaction interface is the first transaction interface. In some examples, the second transaction interface is displayed overlaid on the first application and/or the second application. In some examples, upon determining that the second transaction interface corresponds to the first application (4126), the electronic device modifies a first visual characteristic (e.g., blurry, dark, blurry) of the first application. In some examples, the second transaction interface is configured to connect a first application to a first application when the first application includes information regarding a product or service (or transaction) that is being purchased (or completed) using (or through) the second transaction interface. corresponds to In some examples, this determination is made while displaying the second transaction interface. In some examples, upon determining that the second transaction interface corresponds to a second application (4130), the electronic device (e.g., 100, 300, 500, 4000) determines a first visual characteristic (e.g., blurry, dark) of the second application. , blurry). In some examples, the second transaction interface is configured to connect a first application to a first application when the first application includes information regarding a product or service (or transaction) that is being purchased (or completed) using (or through) the second transaction interface. corresponds to In some examples, this determination is made while displaying the second transaction interface.

In some examples, modifying the first visual characteristic of the first application includes modifying the second visual characteristic of the second application. In some examples, modifying the second visual characteristic of the second application increases the darkening and/or causes the blur radius of the blur effect applied to the second application to a greater degree (or amount) than that for the first application. Includes increasing. In some examples, modifying the first visual characteristic of the second application includes modifying the second visual characteristic of the first application. In some examples, modifying the second visual characteristic of the first application increases the darkening and/or causes the blur radius of the blur effect applied to the first application to be greater (or amount) than that for the second application. Includes increasing. Modifying the visual characteristics of one application to a greater extent than that of another provides feedback to the user as to which application is more relevant at that time. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, no indicator is displayed during biometric authentication.

In some examples, modifying the first visual characteristic of the first application may cause the second area to be closer to (e.g., closer to) the first portion of the electronic device (e.g., a biometric sensor) than the first area. and displaying the first application in the second area according to the determination (4128). In some examples, displaying the first application in the second area includes ceasing to display the first application in the first area. In some examples, modifying the first visual characteristic of the second application may cause the first area to be closer to (e.g., closer to) the first portion of the electronic device (e.g., a biometric sensor) than the second area. and displaying the second application in the first area according to the determination (4132). In some examples, displaying the second application in the first area includes ceasing to display the second application in the second area. In some examples, the electronic device displays an animation of a first application swapping locations with a second application.

In some examples, upon determining that the second transaction interface (e.g., 4014) corresponds to the first application, the second transaction interface includes an indication of the first application (e.g., a name of the first application). In some examples, upon determining that the second transaction interface corresponds to a second application, the second transaction interface includes an indication of the second application (eg, a name of the second application).

Note that the details of the processes described above with respect to method 4100 (e.g., FIGS. 41A-41C) are also applicable in a similar manner to the methods described above/below. For example, method 3700, method 3900, and/or method 4300 optionally include one or more of the features of the various methods described above with respect to method 4100. For example, as described in method 4100, error icon 4028 indicates that a biometric sensor is It can be used to indicate something that is disturbed. For brevity, these details are not repeated below.

42A-42P illustrate example user interfaces for orienting a device to register a biometric feature (e.g., a face for subsequent use in biometric authentication), according to some examples. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 43A-43C.

FIG. 42A illustrates an electronic device 4200 (e.g., portable multifunction device 100, device 300, device 500). In the illustrative examples illustrated in FIGS. 42A-42P, electronic device 4200 is a tablet computer. In other examples, electronic device 4200 may be a different type of electronic device, such as a wearable device (eg, smartwatch). Electronic device 4200 includes a display 4202, one or more input devices (e.g., a touchscreen, buttons, and microphone of display 4202), and a wireless communication radio. In some examples, the electronic device includes a plurality of cameras. In some examples, the electronic device includes only one camera. In Figure 42A, the electronic device includes a biometric sensor 4203. In some examples, the biometric sensor is one or more biometric sensors that may include a camera, such as an infrared camera, a thermographic camera, or a combination thereof. In some examples, biometric sensor 4203 is biometric sensor 703. In some examples, the one or more biometric sensors include one or more fingerprint sensors (eg, a fingerprint sensor integrated into a button). In some examples, the device further includes a light emitting device (eg, a light projector), such as an IR floodlight, a structured light projector, or a combination thereof. The light emitting device is optionally used to illuminate a biometric feature (eg, face) during capture of biometric data of the biometric features by one or more biometric sensors.

In Figure 42A, a user wishes to set up biometric (e.g., facial) authentication on electronic device 4200. Successfully setting up biometric authentication on a device allows a user to perform operations on the device that require authentication (such as unlocking the device) by presenting the user's face for biometric authentication. To set up biometric authentication on an electronic device, a user must first register her face. The process for registering a face may include some or all of the features (or processes) of FIGS. 11A-11O.

As illustrated in FIG. 42A , electronic device 4200 displays an introductory user interface (UI) 4206 with an initiation affordance 4208 . Electronic device 4200 receives input 4210 at initiation affordance 4208 to begin the process of registering the user's face for biometric authentication.

42B, in response to receiving input 4210 at initiation affordance 4208, electronic device 4200 determines that the orientation of the device is not appropriate for registering the user's face. In some examples, a suitable orientation for registering a user's face is an upright (e.g., vertical) portrait orientation, where the biometric sensor 4203 is positioned at the top of the device (e.g., on the side of the device furthest from the ground). ) so that it is in. In response to determining that the orientation of the device is not appropriate for registering the user's face, electronic device 4200 displays one or more prompts prompting the user to orient electronic device 4200 in a suitable orientation (e.g. , replacing the display of the introduced UI 4206 with it). More specifically, electronic device 4200 determines that electronic device 4200 is in a substantially horizontal orientation (eg, approximately parallel to the ground). As a result, as shown in FIG. 42B, electronic device 4200 displays prompt 4212A prompting the user to lift electronic device 4200 to an upright position.

In some examples, in response to receiving input 4210 in initiation affordance 4208, the electronic device determines that the orientation of the device is appropriate for registering the user's face. In some examples, upon determining that the orientation is appropriate for registering the user's face, electronic device 4200 automatically initiates a process to register the user's face, as described below with respect to FIG. 42D. do.

42C, in response to determining that electronic device 4200 is in an upright position but not in a portrait orientation (e.g., the user has lifted the device off a table in response to prompt 4212A), electronic device 4200 Display a prompt 4212B (e.g., replace the display of prompt 4212A with it) prompting the user to rotate the electronic device 4200 to a portrait orientation (e.g., with biometric sensor 4203 on top). box). Specifically, prompt 4212B prompts the user to rotate in a specific direction such that the minimum rotation is required to achieve the desired (or suitable) orientation. For example, rotating electronic device 4200 clockwise requires less rotation to achieve the desired orientation than rotating the device counterclockwise.

42D, in response to determining that electronic device 4200 is in the appropriate orientation, electronic device 4200 automatically initiates a process to register the user's face. As illustrated in FIGS. 42D-42F , after initiating the process for registering the user's face, the electronic device 4200 displays the face registration UI 4214. Face registration UI 4214 includes some or all of the features of face registration UI 1104.

42G, after successfully completing registration of the user's face, electronic device 4200 displays a scan completion interface 4216 (e.g., replacing the display of face registration UI 4214 with it), which Includes the continue affordance (4218). Scan completion interface 4216 includes some or all of the features of scan completion interface 1130.

After completing registration of the user's face, a second iteration of the registration process is performed without requiring the user to reorient the device. As shown in Figure 42G, while displaying the scan completion interface 4216, the electronic device 4200 receives input 4220 at the continue affordance 4218 to initiate a second iteration of the registration process. .

42H , in response to receiving input 4220 at continue affordance 4218, electronic device 4200 performs a second iteration of a registration process similar to the processes described above with respect to FIGS. 42D-42F. Begin. Electronic device 4200 initiates the second iteration without prompting the user to reorient the device to a different orientation than its current orientation. Initiating the second iteration of the registration process includes displaying a second face registration UI 4222. The second face registration UI 4222 includes some or all of the features of the second face registration UI 1138.

42I , after successfully completing the second iteration of the registration process, the electronic device 4200 displays a second scan completion interface 4224 (e.g., displays a second face registration UI 4222 therewith). replaced), which includes a continue affordance (4226). Second scan completion interface 4224 includes some or all of the features of second scan completion interface 1156. As illustrated in Figure 42I, electronic device 4200 receives input 4228 at continue affordance 4226.

42J , in response to receiving input 4228 at continue affordance 4226, electronic device 4200 displays registration completion interface 4230 (e.g., of second scan completion interface 4224). replace the display with it), providing the user with an indication that biometric authentication has been successfully set up on electronic device 4200. Registration completion interface 4230 includes some or all of the features of registration completion interface 1166.

42K, after biometric authentication is set up on electronic device 4200, a user can use biometric authentication to unlock electronic device 4200 by presenting the user's face to biometric sensor 4203. (e.g., transitioning the device from a locked state to an unlocked state). In some examples, a user initiates biometric authentication to unlock the electronic device 4200 by lifting (or elevating) the electronic device 4200 (e.g., from a substantially horizontal orientation). While electronic device 4200 is being lifted, electronic device 4200 detects a change in the orientation of the device and, in response, initiates biometric authentication to unlock the device. While electronic device 4200 is in a locked state, electronic device 4200 displays a biometric sensor indicator 4234 that provides the user with an indication of the location of biometric sensor 4203, and electronic device 4200 is locked. Note that it displays a locked state interface 4232 that includes a lock icon 4236 providing an indication that it is in the state. In some examples, electronic device 4200 does not display biometric sensor indicator 4234 while electronic device 4200 is in a locked state.

As shown in FIG. 42L , when electronic device 4200 initiates biometric authentication, the user is holding electronic device 4200 so that the user's face is outside the field of view 4238 of biometric sensor 4203. . In some examples, the user's face is outside of field of view 4238 when more than a critical portion of the face is outside of field of view. In some examples, the user's face is outside the field of view 4238 when no faces are detected within the field of view. While attempting to biometrically authenticate a user's face, the electronic device 4200 may not be able to obtain sufficient information about the user's face using the biometric sensor 4203. As a result, electronic device 4200 does not have sufficient information for comparison with stored authorized credentials that were generated from the registration process described above with respect to FIGS. 42D-42J.

42M, upon determining that the user's face is outside of field of view 4238, electronic device 4200 displays an error indication 4240, which provides the user with an indication that the user's face is outside of field of view 4238. . (Error indication 4240 includes some or all of the features of error indication 3814G.) Additionally, upon determining that the user's face is outside of field of view 4238, electronic device 4200 automatically performs authentication. Do not retry. In some examples, electronic device 4200 also displays biometric sensor indicator 4234. In some examples, if sufficient information was obtained, but authentication nonetheless failed (e.g., if the obtained information did not match stored authorized credentials), electronic device 4200 automatically retries biometric authentication. do.

As shown in FIG. 42N, after learning from the error indication 4240 that the user's face is outside the field of view 4238 of the biometric sensor 4203, the user can Move the face into the field of view (4238). In response to detecting that the cause of the error indication 4240 has been corrected (e.g., detecting more than a threshold amount of the user's face), the electronic device 4200 automatically reattempts biometric authentication. When determining that authentication is successful (e.g., information obtained using biometric sensor 4203 matches stored authorized credentials) as a result of retrying biometric authentication, electronic device 4200 is in a locked state. Transition to unlocked state. After transitioning to the unlocked state, the electronic device displays an unlocked state interface 4242.

In some examples, while displaying unlocked status interface 4242, electronic device 4200 may request access to restricted content on the device (e.g., home screen 4244 in Figure 42O, most recently used applications). (e.g., an upward swipe starting within an area adjacent the bottom edge of display 4202). In response to receiving a request to access restricted content, electronic device 4200 displays a home screen 4244 that includes a plurality of icons that, when activated, launch an application corresponding to the activated icon. In some examples, instead of displaying home screen 4244, electronic device 4200 displays the most recently used application (e.g., the application's user interface). Note that the above processes described above with respect to FIGS. 42K-42O are performed when the electronic device 4200 is in a landscape orientation. However, in some examples, some or all of the processes described above with respect to FIGS. 42K-42N may be performed when electronic device 4200 is in a portrait orientation.

In some examples, instead of transitioning to an unlocked state as described with respect to FIG. 42N, electronic device 4200 remains locked if the information obtained does not match stored authorized credentials. . In some examples, as shown in FIG. 42P, upon determining that the obtained information does not match stored authorized credentials, electronic device 4200 causes locked state interface 4232 to simulate a “shake” effect. The locked state interface is displayed while alternating the position of the lock icon 4236, thereby providing the user with an indication that the electronic device 4200 remains locked.

43A-43C are flow diagrams illustrating a method for orienting a device to register a biometric feature (e.g., a face for subsequent use in biometric authentication), according to some examples. Method 4300 includes a display (e.g., 4202) and one or more biometric sensors (e.g., 4203) (e.g., a first biometric sensor of a device having a plurality of biometric sensors) (e.g., a fingerprint sensor, a contactless biometric sensor) (e.g., biometric sensors that do not require physical contact, such as thermal or optical facial recognition sensors), iris scanners) (e.g., 100, 300, 500, 4200). In some embodiments, one or more biometric sensors include one or more cameras. Some operations of method 4300 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

As described below, method 4300 provides an intuitive way to prompt a user to orient the device to register biometric features. This method reduces the cognitive burden on the user for registering biometric features (e.g., the face for subsequent use in biometric authentication), thereby creating a more efficient human-machine interface. For battery-operated computing devices, allowing users to register biometric features faster and more efficiently saves power and increases time between battery charges.

The electronic device (e.g., 100, 300, 500, 4200) displays, on a display (e.g., 4202), a biometric registration user interface (e.g., 4206) for initiating biometric registration with one or more biometric sensors. Do (4302).

While displaying the biometric registration user interface 4304, the electronic device may receive input (e.g., 4210) (e.g., a touch gesture (e.g., tap), spoken user input) corresponding to a request to initiate biometric registration. ) is received.

In response to receiving 4306 an input (e.g., 4210) and determining that the orientation of the electronic device (e.g., current orientation, orientation of the electronic device at (or near) the time of the input) satisfies a set of registration criteria. Following the determination to do so (4308), the electronic device initiates a process to register the biometric characteristic with one or more biometric sensors (e.g., 4203). In some examples, the set of registration criteria may include whether the electronic device is oriented in a vertical orientation relative to a frame of reference (e.g., the Earth, the ground), and whether one or more biometric sensors are positioned on a particular side of the electronic device in a vertical orientation (e.g., the Earth). whether the electronic device is oriented (or positioned) on the side furthest from the ground, or whether the electronic device is oriented such that it is approximately subparallel to the ground. In some examples, the set of registration criteria determines whether the electronic device is in a given (e.g., appropriate) orientation relative to a biometric feature (e.g., the face) (e.g., defined by the major plane of the device (e.g., the display of the device) plane) is facing the biometric feature. In some examples, initiating a process to register a biometric feature includes capturing data corresponding to the user's face using one or more biometric sensors.

In response to receiving input (e.g., 4210) (4306), and upon determining that the orientation of the electronic device does not meet the set of registration criteria (4322), the electronic device determines the orientation of the electronic device, Output one or more prompts (e.g., 4212A, 4212B) (e.g., visual, audio, and/or tactile prompts) to change to a different orientation that satisfies the set. Outputting one or more prompts if the set of registration criteria is not met provides feedback to the user on corrective actions to take to continue registering the biometric characteristic. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently. In some examples, no indicator is displayed during biometric authentication.

In some examples, outputting one or more prompts includes outputting a first prompt (e.g., 4212A) to orient the electronic device to an initial orientation (4324). In some examples, the initial orientation is such that the electronic device is not approximately parallel to the ground. In some examples, the initial orientation is such that the electronic device is approximately parallel to the force of gravity. In some examples, the set of registration criteria includes a requirement that the major plane of the device be substantially aligned with a predetermined plane (e.g., a plane substantially normal to the ground) such that the display of the device is substantially vertical. In some examples, the set of registration criteria may be such that the device's major plane is a (second) predetermined plane (e.g., a plane substantially parallel to the ground) such that the device is not resting on a horizontal surface while attempting to register a biometric feature. ) includes the requirement that it is not substantially aligned with In some examples, outputting one or more prompts may include outputting a first prompt (e.g., 4212A) followed by a second prompt (e.g., 4212A) to orient the electronic device to a different orientation that satisfies a set of registration criteria. and outputting 4212B), where the first prompt is different from the second prompt (4326). In some examples, the electronic device outputs the first prompt without outputting the second prompt. In some examples, the electronic device stops outputting the first prompt when the orientation of the electronic device changes to the initial orientation. In some examples, the electronic device outputs a second prompt when the orientation of the electronic device changes to the initial orientation. In some examples, the electronic device outputs the second prompt without outputting the first prompt (eg, when the electronic device is already in an initial orientation). In some examples, the set of registration criteria may include whether the electronic device is oriented in a vertical orientation relative to a frame of reference (e.g., the Earth, the ground), and whether one or more biometric sensors are positioned on a particular side of the electronic device in a vertical orientation (e.g., the Earth). whether the electronic device is oriented (or positioned) on the side furthest from the ground, or whether the electronic device is oriented such that it is approximately subparallel to the ground. Outputting a first prompt without outputting a second prompt provides improved feedback to the user by reducing the opportunity for confusion when the user is taking corrective actions to trigger registration of the biometric characteristic. Providing improved visual feedback to the user improves the operability of the device (e.g., by helping the user provide appropriate inputs when operating/interacting with the device and reducing user errors) and improves the user-device interface. This additionally reduces the device's power usage and improves battery life by enabling users to use the device more quickly and efficiently. In some examples, no indicator is displayed during biometric authentication.

In some examples, outputting one or more prompts outputs a third prompt (e.g., 4212B) to rotate the electronic device to a different orientation (e.g., about an axis perpendicular to the electronic device) that satisfies the set of registration criteria. wherein the third prompt is based on the orientation of the electronic device while receiving the input. In some examples, the third prompt is a second prompt. In some examples, upon determining that the orientation of the electronic device is a first orientation, the electronic device outputs a first rotation prompt directing the electronic device to rotate to a different orientation that satisfies the set of registration criteria. In some examples, upon determining that the orientation of the electronic device is a second orientation that is different from the first orientation, the electronic device outputs a second rotation prompt directing the electronic device to rotate to a different orientation that satisfies the set of registration criteria; , the second rotation prompt is different from the first rotation prompt. In some examples, the first rotation prompt or the second rotation prompt is a second prompt. In some examples, the set of registration criteria may include whether the electronic device is oriented in a vertical orientation relative to a frame of reference (e.g., the Earth, the ground), and whether one or more biometric sensors are positioned on a particular side of the electronic device in a vertical orientation (e.g., the Earth). whether the electronic device is oriented (or positioned) on the side furthest from the ground, or whether the electronic device is oriented such that it is approximately subparallel to the ground. Outputting a prompt based on the orientation of the device provides feedback to the user on an efficient process for achieving appropriate orientation of the device to register biometric features. Providing improved feedback to the user improves the operability of the device and improves the user-device interface (e.g., by helping the user provide appropriate inputs and reducing user errors when operating/interacting with the device). This makes it more efficient, which additionally reduces the device's power usage and improves battery life by allowing users to use the device more quickly and efficiently.

In some examples, outputting one or more prompts may cause the electronic device to be positioned (e.g., about an axis perpendicular to the electronic device) in a different orientation (e.g., in the main plane of the device (e.g., about an axis perpendicular to the electronic device) that satisfies a set of registration criteria. outputting a fourth prompt (e.g., 4212B) to rotate (along an axis parallel to a plane defined by the display), wherein the third prompt is a predetermined plane (e.g., a plane substantially normal to the ground; It is based on the alignment of the main plane of the device (e.g., the plane defined by the display of the device) relative to the plane substantially parallel to the. In some examples, the electronic device outputs a fourth prompt in response to a determination that the electronic device is oriented substantially parallel to the ground. In some examples, the set of registration criteria includes a requirement that the major plane of the device be substantially aligned with a predetermined plane (e.g., a plane substantially normal to the ground) such that the display of the device is substantially vertical. In some examples, the set of registration criteria may be such that the device's major plane is a (second) predetermined plane (e.g., a plane substantially parallel to the ground) such that the device is not resting on a horizontal surface while attempting to register a biometric feature. ) includes the requirement that it is not substantially aligned with

In some examples, following initiating a process to register a biometric characteristic (e.g., following successfully registering a biometric characteristic), the electronic device (e.g., 100, 300, 500, 4200) requires authentication. Receives a request to perform an action (e.g., a request to unlock the device (e.g., perform a swipe at a predefined location)). In some examples, the electronic device receives a request to perform an action requiring authentication subsequent to performing (or completing) biometric registration. In some examples, the electronic device may receive one or more prompts (e.g., 4212A, 4212B) (e.g., visual, audio, and/or tactile prompts) to change the orientation of the electronic device to a different orientation that satisfies a set of registration criteria. Following the output, a request is received to perform an operation requiring authentication. In some examples, in response to receiving a request to perform an operation requiring authentication, the electronic device uses one or more biometric sensors (e.g., 4203) to authenticate (e.g., retrieve data by one or more biometric sensors). (including obtaining). In some examples, after attempting authentication using one or more biometric sensors (e.g., unsuccessfully), and then authenticating using one or more biometric sensors (e.g., due to the face being outside of the field of view (e.g., 4238)) Upon determining that the data obtained by the device corresponds to less than a threshold amount of biometric features (e.g., a portion of the face/fingerprint rather than the entire face/fingerprint), the electronic device withholds from retrying authentication. In some examples, the electronic device forgoes automatically retrying authentication. In some examples, after attempting authentication using one or more biometric sensors, the electronic device causes biometric authentication to fail more than a predetermined number of times (e.g., 5, 10, 15) since the last successful authentication by the device. Because of this, retrying authentication is on hold. In some examples, the electronic device is prohibited from retrying authentication without an explicit request to perform an action requiring authentication (e.g., a request to unlock the device (e.g., perform a swipe at a predefined location)). put on hold In some examples, after an initial attempt at authentication is not successful, the electronic device re-establishes biometric authentication unless a determination is made that data acquired by one or more biometric sensors corresponds to only a portion of the biometric characteristics. Try it. Withholding from retrying authentication when less than a threshold amount of biometric characteristics are obtained avoids the user exhausting the allowed number of attempts for repeated requests (e.g., repeated requests of the same type). , thereby preserving at least one attempt for requests for other operations requiring biometric authentication. Preserving at least one attempt improves the operability of the device (e.g., by avoiding exhaustion of authentication attempts for repeated similar requests) and makes the user-device interface more efficient, which additionally allows the user to access the device. Reduces device power usage and improves battery life by enabling faster and more efficient use of devices.

In some examples, after attempting authentication using one or more biometric sensors (e.g., unsuccessfully), and the data obtained by the one or more biometric sensors exceeding a threshold amount (e.g., exceeding) the biometric characteristic Depending on the determination that it corresponds to the negative, the electronic device reattempts authentication. Automatically retrying authentication when a critical mass of biometric characteristics is obtained provides the user with the ability to attempt authentication when conditions are appropriate without requiring the user to explicitly request to retry authentication. Performing an action when a set of conditions is met without requiring additional user input (e.g., by helping the user provide appropriate input and reducing user errors when operating/interacting with a device) It improves the operability of the device (e.g., increases the chances of successful authentication) and makes the user-device interface more efficient, which additionally reduces the device's power usage by allowing users to use the device more quickly and efficiently. reduces and improves battery life.

In some examples, upon a determination that authentication was successful resulting from retrying authentication, the electronic device (eg, 100, 300, 500, 4200) performs an action corresponding to the request. In some examples, upon a determination that authentication was not successful resulting from reattempting authentication, the electronic device withholds performing an action corresponding to the request. In some examples, authentication may be performed using biometric information captured using one or more biometric sensors to display authorized credentials (e.g., biometric features (e.g., face, fingerprint) authorized for use in biometric authentication). is successful when responding to (or matching) stored information about. In some examples, authentication may be performed using biometric information captured using one or more biometric sensors to display authorized credentials (e.g., biometric features (e.g., face, fingerprint) authorized for use in biometric authentication). is not successful when it does not correspond to (or does not match) stored information about. Suspending performing an operation when authentication is not successful improves device security by preventing fraudulent and/or unauthorized access to the device. Improving a device's security measures improves the device's operability by preventing unauthorized access to content and operations, and additionally reduces the device's power usage by enabling users to use the device more efficiently. reduces and improves battery life.

In some examples, output one or more prompts (e.g., 4212A, 4212B) (e.g., a visual, audio, and/or tactile prompt) to change the orientation of the electronic device to a different orientation that satisfies a set of registration criteria. Subsequently, the electronic device (e.g., 100, 300, 500, 4200) detects (4328) that the current orientation of the electronic device satisfies the set of registration criteria. In some examples, in response to determining that the current orientation of the electronic device satisfies the set of registration criteria (4330), the electronic device initiates a process to register the biometric feature with one or more biometric sensors. In some examples, the set of registration criteria may include whether the electronic device is oriented in a vertical orientation relative to a frame of reference (e.g., the Earth, the ground), and whether one or more biometric sensors are positioned on a particular side of the electronic device in a vertical orientation (e.g., the Earth). whether the electronic device is oriented (or positioned) on the side furthest from the ground, or whether the electronic device is oriented such that it is approximately subparallel to the ground. In some examples, the set of registration criteria includes a requirement that the major plane of the device be substantially aligned with a predetermined plane (e.g., a plane substantially normal to the ground) such that the display of the device is substantially vertical. In some examples, the set of registration criteria may be such that the device's major plane is a (second) predetermined plane (e.g., a plane substantially parallel to the ground) such that the device is not resting on a horizontal surface while attempting to register a biometric feature. ) includes the requirement that it is not substantially aligned with In some examples, the set of enrollment criteria determines whether the electronic device is in a given (e.g., appropriate) orientation relative to a biometric feature (e.g., the face) (e.g., defined by the major plane of the device (e.g., the display of the device) plane) is facing the biometric feature.

In some examples, initiating a process to register a biometric feature with one or more biometric sensors includes successfully registering the biometric feature. In some examples, following successfully registering a biometric feature, the electronic device (e.g., 100, 300, 500, 4200) prompts to register the biometric feature for a second time with one or more biometric sensors (e.g., (corresponds to 4222) is output (4312). In some examples, the electronic device prompts to register the biometric characteristic without prompting to change the orientation of the electronic device.

In some examples, initiating a process to register a biometric feature with one or more biometric sensors includes successfully registering the biometric feature (4310). In some examples, following successfully registering a biometric characteristic, the electronic device (e.g., 100, 300, 500, 4200) receives a request to perform an operation requiring authentication (e.g., unlock the device (e.g., A request to perform a swipe at a defined location and a request to access a home screen (e.g., 4244) are received (4314). In some examples, in response to receiving 4316 a request to perform an operation requiring authentication, and data obtained by one or more biometric sensors correspond to a registered biometric characteristic (e.g., According to the determination (matching), the electronic device performs an operation requesting authentication (4318). In some examples, in response to receiving a request to perform an action requiring authentication, the electronic device performs (or attempts to authenticate) using one or more biometric sensors (e.g., 4203). In some examples, in response to receiving 4316 a request to perform an action requiring authentication and that data obtained by one or more biometric sensors does not correspond to a registered biometric characteristic (e.g., According to the determination of "does not match", the electronic device suspends performing an operation requiring authentication (4320).

Note that the details of the processes described above with respect to method 4300 (e.g., FIGS. 43A-43C) are also applicable in a similar manner to the methods described above/below. For example, method 3700, method 3900, and/or method 4100 optionally include one or more of the features of the various methods described above with respect to method 4300. For example, the processes described in method 4300 for registering biometric features may include biometric authentication, such as retrying biometric authentication in a passcode entry user interface, as described in method 3700. It can be used to register a face for later use. As another example, the registered face may be used to authorize payment for products, as described in method 4100. For brevity, these details are not repeated below.

The foregoing description, for purposes of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were selected and described to best illustrate the principles of the techniques and their practical applications. Accordingly, those skilled in the art will be able to best utilize the techniques and various embodiments using various modifications as appropriate for the particular use contemplated.

Although the present disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will be apparent to those skilled in the art. It is to be understood that such changes and modifications are included within the scope of the disclosure and examples as defined by the claims.

As described above, one aspect of the present technology is the collection and use of data available from various sources to enhance the delivery to users of invited content or any other content that may be of interest to users. . This disclosure contemplates that, in some cases, such collected data may include personal information data that may uniquely identify or be used to contact or locate a particular individual. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, home addresses, or any other identifying information.

This disclosure recognizes that the use of such personal information data in the present technology can be used to benefit users. For example, personal information data may be used to deliver targeted content of greater interest to users. Accordingly, the use of such personal information data enables calculated control of the delivered content. Additionally, other uses for personal information data that benefit the user are also contemplated by this disclosure.

This disclosure further contemplates that entities responsible for the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will comply with well-established privacy policies and/or privacy practices. In particular, such entities must implement and continue to use privacy policies and practices that are generally recognized as meeting or exceeding industry or administrative requirements for keeping personal information data private and secure. For example, personal information from users must be collected for lawful and appropriate uses of the entity, and must not be shared or sold beyond those lawful uses. Additionally, such collection should only occur after receiving the informed consent of users. Additionally, such entities will take whatever steps are necessary to protect and secure access to such Personal Information Data and to ensure that others who have access to Personal Information Data adhere to their privacy policies and procedures. . In addition, such entities may themselves be evaluated by third parties to demonstrate their adherence to widely accepted privacy policies and practices.

Notwithstanding the foregoing, this disclosure also contemplates embodiments in which a user selectively blocks use of, or access to, personal information data. That is, the present disclosure contemplates that hardware and/or software elements may be provided to prevent or block access to such personal information data. For example, in the case of advertising delivery services, the technology may be configured to allow users to select “agree” or “disagree” to participate in the collection of personal information data during registration for the service. . In another example, users may choose not to provide location information for targeted content delivery services. In another example, users may choose not to provide precise location information but allow delivery of location zone information.

Accordingly, although this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also provides that various embodiments may be implemented without the need for access to such personal information data. Also consider: That is, various embodiments of the present technology are not rendered inoperable by the absence of all or part of such personal information data. For example, content may contain a minimal amount of personal information or non-personal information, such as content requested by a device associated with a user, other non-personal information available to content delivery services, or publicly available information. Based on preferences, they can be selected and communicated to users by inferring preferences.

Claims (37)

As a method,
In an electronic device having a display, a button, and one or more biometric sensors separate from the button,
detecting activation of one or more buttons while the electronic device is in a first state in which respective functions of the device are disabled; and
In response to detecting activation of one or more of the buttons:
capturing biometric data with the one or more biometric sensors separate from the button;
Upon determining that the biometric data satisfies biometric authentication criteria, transitioning the electronic device to a second state in which each function of the device is enabled; and
In response to a determination that the biometric data does not meet the biometric authentication criteria, maintaining the electronic device in the first state and displaying an indication on the display that biometric authentication has failed.
How to include . The method of claim 1, wherein the one or more biometric sensors comprise a facial recognition sensor, and the biometric data corresponds to at least a portion of the face. According to paragraph 1,
Each function of the electronic device is to participate in a transaction,
the electronic device is not capable of participating in the transaction when in the first state, and
wherein the electronic device is capable of participating in the transaction when in the second state. 4. The method of claim 3, wherein participating in the transaction includes transmission of secure data from the electronic device. According to paragraph 3,
Information that allows the device to participate in the transaction is stored securely in a secure element;
When the device is in the first state, information that allows the device to participate in the transaction is not accessible to the device outside the secure element;
When the device is in the second state, information enabling the device to participate in the transaction is temporarily made accessible by the secure element to the device outside the secure element.
method. According to paragraph 1,
The one or more biometric sensors include a camera,
Capturing biometric data includes capturing the biometric data using a camera.
How to include . The method of claim 1, wherein activating one or more of the buttons comprises a double press of the button. The method of claim 1, wherein the button has a fixed position relative to the display of the electronic device. The method of claim 1, wherein the button is a mechanical button. The method of claim 1, wherein the button is a solid state button. According to paragraph 1,
Before detecting the one or more activations of the buttons, outputting a prompt to provide one or more activations of the buttons.
How to further include . 12. The method of claim 11, wherein the prompt is displayed on the display of the electronic device, and outputting the prompt to a user to provide activation of one or more buttons includes one displayed on the display of the electronic device. A method comprising emphasizing the prompt relative to the above elements. According to clause 12,
In response to detecting activation of one or more of the buttons:
displaying a biometric authentication interface including a representation of user credentials restricted from use without appropriate biometric authentication;
How to further include . According to clause 13,
prior to detecting one or more activations of the buttons, displaying, on the display, the prompt for providing one or more activations of the buttons at a first location within the biometric authentication interface; and
After displaying the prompt, moving a representation of the user credentials from a second location on the display to a first location on the display.
How to further include . 15. The method of claim 14, wherein moving the representation of the user credential from a second location to the first location comprises: moving the display that was occupied by the user credential when the user credential was displayed at the second location. A method comprising displaying a biometric authentication glyph on a portion of a method. According to paragraph 1,
After transitioning the electronic device to the second state,
maintaining the device in the second state for a first predetermined period of time; and
After the first predetermined period of time has elapsed, transitioning the electronic device from the second state to the first state.
How to further include . 2. The method of claim 1, wherein activation of one or more buttons is detected while a first application is active on the electronic device, the method comprising:
While the device is in the second state, detecting user input corresponding to a request to terminate the first application; and
In response to detecting user input corresponding to a request to terminate the first application:
terminating the first application; and
Transitioning to the first state
How to further include . The method of claim 1, wherein capturing biometric data with the one or more biometric sensors comprises activating the one or more biometric sensors for a second predetermined period of time. According to paragraph 1,
In response to a determination that the biometric data does not meet the biometric authentication criteria, displaying alternative authentication affordances.
How to further include . According to clause 19,
After determining that the biometric data does not meet the biometric authentication criteria, detecting selection of the alternative authentication affordance; and
In response to detecting selection of the alternative authentication affordance, displaying an alternative authentication interface on the display.
How to further include . The method of claim 20, wherein the biometric data is first biometric data, and the method comprises:
In response to detecting selection of the alternative authentication affordance:
capturing second biometric data with the one or more biometric sensors;
transitioning the electronic device to the second state in response to a determination that the second biometric data satisfies the biometric authentication criteria; and
In response to a determination that the second biometric data does not meet the biometric authentication criteria, maintaining the electronic device in the first state and displaying the alternative authentication interface on the display.
How to further include . According to clause 21,
The biometric data is first biometric data,
The method, after determining that the first biometric data does not meet the biometric authentication criteria:
detecting each user input corresponding to a request to retry biometric authentication; and
In response to detecting said respective user input corresponding to said request to retry biometric authentication:
capturing, with the one or more biometric sensors, third biometric data;
Upon determining that the third biometric data satisfies the biometric authentication criteria, transitioning the electronic device to the second state in which respective functions of the device are enabled; and
maintaining the electronic device in the first state in accordance with a determination that the third biometric data does not meet the biometric authentication criteria.
How to include . 23. The method of claim 22, wherein each user input corresponding to the request to retry biometric authentication includes one or more activations of the button. 23. The method of claim 22, wherein each user input corresponding to the request to retry biometric authentication includes movement of the electronic device. According to paragraph 1,
detecting activation of one or more other buttons;
in response to a determination that biometric capture criteria are met, capturing second biometric data with the one or more biometric sensors separate from the button; and
withholding capture of the second biometric data upon determining that the biometric capture criteria are not met.
How to further include . 12. The method of claim 11, wherein outputting the prompt occurs in response to detecting an external signal of a predetermined type. 27. The method of claim 26, wherein outputting the prompt includes displaying an instructional interface including a prompt element adjacent the button. ◈Claim 28 was abandoned upon payment of the setup registration fee.◈ According to clause 27,
Before the step of outputting the prompt, it further includes the step of displaying an instruction manual interface including an affordance,
Wherein outputting the prompt occurs in response to selection of the affordance. ◈Claim 29 was abandoned upon payment of the setup registration fee.◈ 29. The method of claim 28, wherein detecting activation of one or more buttons occurs while displaying the instruction manual interface. The method of claim 1, wherein each function of the device is a function of providing information associated with an action through a short-range communication wireless device of the electronic device. The method of claim 1 , wherein activation of the one or more buttons occurs at least partially while the display of the device is turned off or while displaying a user interface not related to payment. According to paragraph 1,
Prior to detecting activation of one or more of said buttons:
detecting activation of an affordance of a first application;
In response to detecting activation of an affordance of the first application, providing information associated with an action from the first application to a second application; and
After transitioning the electronic device to the second state in which each function of the device is enabled, performing the action based on the information using each function of the electronic device.
How to further include . ◈Claim 33 was abandoned upon payment of the setup registration fee.◈ According to clause 32,
In response to detecting selection of an affordance of the first application, on the display:
at least some of the information associated with the action in a first location, and
a second prompt for providing one or more activations of the button at a second location, the second location being closer to the button than the first location;
A method further comprising the step of simultaneously displaying. According to paragraph 1,
Receiving, from a second device, a request to proceed with an action, the request including information associated with one or more options selected at the second device; and
After transitioning the electronic device to the second state in which each function of the device is enabled, performing the action based on the information using each function of the device.
How to further include . A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having a display, buttons, and one or more biometric sensors separate from the buttons, the one or more programs comprising: A computer-readable storage medium comprising instructions for performing the method of any one of claims 34. As an electronic device,
display;
button;
one or more biometric sensors separate from the button;
One or more processors; and
An electronic device comprising: a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs comprising instructions for performing the method of any one of claims 1 to 34. As an electronic device,
display;
button;
one or more biometric sensors separate from the button; and
Means for carrying out the method of any one of paragraphs 1 to 34
An electronic device containing a.