WO2024037379A1 - Notification checking method and system, and related apparatus - Google Patents
Notification checking method and system, and related apparatus Download PDFInfo
- Publication number
- WO2024037379A1 WO2024037379A1 PCT/CN2023/111722 CN2023111722W WO2024037379A1 WO 2024037379 A1 WO2024037379 A1 WO 2024037379A1 CN 2023111722 W CN2023111722 W CN 2023111722W WO 2024037379 A1 WO2024037379 A1 WO 2024037379A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- electronic device
- area
- user
- notification
- gaze
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 109
- 230000004424 eye movement Effects 0.000 claims abstract description 18
- 230000001815 facial effect Effects 0.000 claims abstract description 12
- 238000007667 floating Methods 0.000 claims description 64
- 230000004044 response Effects 0.000 claims description 43
- 230000015654 memory Effects 0.000 claims description 24
- 230000033001 locomotion Effects 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 10
- 238000003860 storage Methods 0.000 claims description 6
- 238000004891 communication Methods 0.000 description 28
- 210000001508 eye Anatomy 0.000 description 21
- 238000010586 diagram Methods 0.000 description 20
- 230000008569 process Effects 0.000 description 16
- 238000007726 management method Methods 0.000 description 14
- 230000008859 change Effects 0.000 description 13
- 238000012545 processing Methods 0.000 description 13
- 238000010295 mobile communication Methods 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 8
- 230000003287 optical effect Effects 0.000 description 8
- 210000003128 head Anatomy 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 229920001621 AMOLED Polymers 0.000 description 4
- 230000009471 action Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000003190 augmentative effect Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000014509 gene expression Effects 0.000 description 3
- 230000005855 radiation Effects 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 210000005252 bulbus oculi Anatomy 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 2
- 230000003862 health status Effects 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 238000013529 biological neural network Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000002096 quantum dot Substances 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
Definitions
- the present application relates to the field of terminal technology, and in particular, to a notification viewing method, system and related devices.
- a notification prompt window When an electronic device receives information or notifications from other applications, a notification prompt window will be displayed at the top. Whether the user needs to view a notification or quickly close the notification prompt window, he or she needs to operate the notification prompt window on the top of the electronic device. After receiving the user's operation on the notification prompt window, the electronic device will display the specific content of the notification or close the notification prompt window according to the user's operation.
- This application provides a notification viewing method, system and related devices.
- the electronic device displays a notification prompt window in the first area of the touch screen
- the electronic device can obtain any of the user's facial image, eye image, eye movement information, and electrooculogram information, and determine the user accordingly. Whether the gaze area is the first area.
- the electronic device can receive and respond to the user's operation on an area other than the first area on the touch screen, view the content of the notification or cancel the display of the notification prompt window. This enables users to conveniently view notification content when holding the electronic device with one hand.
- this application provides a notification viewing method, including: a first electronic device displays a first interface; the first electronic device obtains a first notification of a first application; in response to the first notification, the first electronic device touches The first area of the control screen displays a first notification prompt window, and the first notification prompt window includes part or all of the content of the first notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device Receive the user's first operation on the second area, which is an area on the touch screen other than the first area; after the first electronic device detects the first operation, the first electronic device overlays and displays the first operation on the first interface.
- a floating window, the first floating window includes an interface of the first application, and the interface of the first application includes the content of the first notification.
- the method further includes: the first electronic device outputs gaze feedback, and the gaze feedback is used to prompt that the user's gaze area is located in the first region. .
- outputting gaze feedback includes any one or more of the following: changing the display color of the first interface, changing the display color of the first notification prompt window, changing the color of the controls in the first notification prompt window, Light up the indicator light, vibrate and play a tone of the first electronic device.
- the first electronic device outputs gaze feedback, which specifically includes: when the first electronic device detects that the user gazes at the first area for longer than the first duration, the first electronic device outputs gaze feedback.
- the first electronic device can prompt the user's gaze area to be located in the first area through gaze feedback.
- the first operation is a sliding operation; after the first electronic device receives the user's first operation on the second area, the method further includes: when the movement distance of the first operation on the touch screen is greater than When the first movement distance is reached, the first electronic device moves the display position of the first notification prompt window; when the movement distance of the first operation on the touch screen is greater than the second movement distance, the first electronic device expands the first notification prompt window. to form a first floating window; wherein the second movement distance is greater than the first movement distance.
- the first operation is a sliding operation; after the first electronic device receives the user's first operation on the second area, the method further includes: when the movement distance of the first operation on the touch screen When the first movement distance is greater than the first movement distance, the first electronic device moves the display position of the first notification prompt window; when the movement distance of the first operation on the touch screen is greater than the second movement distance, the first electronic device displays the first floating window, And the display of the first notification prompt window is cancelled, and the second moving distance is greater than the first moving distance.
- the first electronic device can control the first notification prompt window to perform different functions based on the movement distance of the first operation on the touch screen.
- the first operation is a sliding operation; after the first electronic device receives the user's first operation for the second area, the method further includes: when the duration of the first operation is greater than the second duration, The first electronic device moves the display position of the first notification prompt window; when the duration of the first operation is greater than the third duration, the first electronic device expands the first notification prompt window into a first floating window, and the third duration is longer than the second duration. duration.
- the first operation is a sliding operation; after the first electronic device receives the user's first operation on the second area, the method further includes: when the duration of the first operation is greater than the second duration , the first electronic device moves the display position of the first notification prompt window; when the duration of the first operation is greater than the third duration, the first electronic device displays the first floating window, and cancels the display of the first notification prompt window, and the third duration greater than the second duration.
- the first electronic device can control the first notification prompt window to perform different functions based on the duration of the first operation.
- the method further includes: after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives a second operation from the user for the second area; in response to the second operation, The first electronic device displays an interface of the first application.
- the method further includes: the first electronic device obtains a second notification of the first application; in response to the second notification, the first electronic device displays a second notification prompt window in the first area of the touch screen , the second notification prompt window includes part or all of the content of the second notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's second operation for the second area; responds In the second operation, the first electronic device displays the interface of the first application, and the interface of the first application includes the content of the second notification.
- the method further includes: the first electronic device obtains a third notification from the second application; in response to the third notification, the first electronic device displays a third notification prompt window in the first area of the touch screen , the third notification prompt window includes part or all of the content of the third notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's second operation for the second area; responds In the second operation, the first electronic device displays the interface of the second application, and the interface of the second application includes the content of the third notification.
- the method further includes: after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives a third operation from the user for the second area; in response to the third operation, The first electronic device cancels display of the first notification prompt window.
- the method further includes: the first electronic device obtains a second notification of the first application; in response to the second notification, the first electronic device displays a second notification prompt window in the first area of the touch screen , the second notification prompt window includes part or all of the content of the second notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's third operation for the second area; responds In the third operation, the first electronic device cancels display of the second notification prompt window.
- the method further includes: the first electronic device obtains a third notification from the second application; in response to the third notification, the first electronic device displays a third notification prompt window in the first area of the touch screen , the third notification prompt window includes part or all of the content of the third notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's third operation for the second area; responds In the third operation, the first electronic device cancels display of the third notification prompt window.
- the method further includes: the first electronic device redisplays the first notification prompt window after a fourth period of time.
- the method further includes: the first electronic device displays the floating ball; the first electronic device receives the user's input for the floating ball; and responds to the user In response to input from the floating ball, the first electronic device displays a first notification prompt window or a first floating window.
- the method further includes: when the first electronic device receives the user's fourth operation for the second area, the first electronic device The first function corresponding to the first interface is executed, and the fourth operation is different from the first operation.
- the method further includes: when the first electronic device determines that the user's gaze area changes to the second area, and the first notification prompts When the display duration of the window is longer than the fifth duration, the first electronic device cancels display of the first notification prompt window.
- the method further includes: the first electronic device displays an operation prompt, and the operation prompt is used to prompt the user to perform an operation on the first area in the second area. Notification prompt window to control.
- the first area is a top area of the touch screen.
- the second area is a bottom area of the touch screen. In another possible implementation, the second area may also be the middle area of the touch screen.
- the method further includes: the first electronic device obtains gaze information, and the gaze information includes any one or more of the following: the user's eye image, the user's facial image, eye movement information, and electrooculogram information. ; The first electronic device determines that the user's gaze area is located in the first area based on the gaze information.
- the first electronic device obtains the gaze information, which specifically includes: the first electronic device receives the gaze information sent by the second electronic device.
- the method further includes: the first electronic device receives a line of sight estimation result sent by the second electronic device, the line of sight estimation result includes that the user's gaze area is located in the first area; the first electronic device determines based on the line of sight estimation result The user's gaze area is located in the first area.
- the method further includes: the first electronic device receiving the user's gaze direction sent by the second electronic device; and the first electronic device determining that the user's gaze area is located in the first area based on the user's gaze direction.
- this application provides an electronic device, which is a first electronic device including: one or more processors and one or more memories.
- the one or more memories are coupled to one or more processors, and the one or more memories are used to store computer program codes.
- the computer program codes include computer instructions.
- the one or more processors execute the computer instructions, the first electronic The device performs the notification viewing method in any of the possible implementations of any of the above aspects.
- the present application provides a computer storage medium that includes computer instructions.
- the computer instructions When the computer instructions are run on an electronic device, the electronic device causes the electronic device to execute any of the above aspects in a possible implementation manner. How to view notifications.
- this application provides a computer program product.
- the computer program product When the computer program product is run on a computer, it causes the computer to execute the notification viewing method in any of the possible implementations of any of the above aspects.
- Figure 1 is a schematic diagram of the one-hand operation range when an electronic device is held with one hand according to an embodiment of the present application
- FIG. 2A is a schematic diagram of an electronic device 100 provided by an embodiment of the present application.
- Figure 2B is a schematic structural diagram of a system 10 provided by an embodiment of the present application.
- Figure 3 is a schematic structural diagram of an electronic device 100 provided by an embodiment of the present application.
- Figure 4 is a schematic structural diagram of an electronic device 200 provided by an embodiment of the present application.
- Figures 5A-5H are schematic interface diagrams of a notification viewing method provided by an embodiment of the present application.
- Figures 6A-6B are schematic interface diagrams of another notification viewing method provided by an embodiment of the present application.
- Figures 7A-7D are schematic interface diagrams of a method for canceling the display of a notification prompt window provided by an embodiment of the present application.
- Figures 8A-8C are schematic interface diagrams of another method for canceling the display of a notification prompt window provided by an embodiment of the present application.
- 9A-9B are schematic interface diagrams of an electronic device responding to operations in a non-gaze operation set according to an embodiment of the present application.
- Figure 10 is a schematic interface diagram of another electronic device outputting gaze feedback provided by an embodiment of the present application.
- Figure 11A is a schematic flowchart of a notification viewing method provided by an embodiment of the present application.
- FIGS 11B to 11D show three different situations in which the gaze area is located in the first area provided by the embodiment of the present application.
- Figure 12 is a schematic flowchart of a method for an electronic device according to an embodiment of the present application to execute a response corresponding to the operation based on the relationship between the received operation and the set of gaze operations;
- Figure 13 is a schematic flowchart of another notification viewing method provided by an embodiment of the present application.
- FIG. 14 is a schematic diagram of functional modules of an electronic device 100 provided by an embodiment of the present application.
- first and second are used for descriptive purposes only and shall not be understood as implying or implying relative importance or implicitly specifying the quantity of indicated technical features. Therefore, the features defined as “first” and “second” may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, “plurality” The meaning is two or more.
- GUI graphical user interface
- Figure 1 shows a schematic diagram of the one-hand operation range when the electronic device is held with one hand.
- the touch screen of the electronic device can be divided into the following three areas according to the difficulty of the user's one-hand operation: area 1001, area 1002 and area 1003 .
- area 1001 is an area that is difficult to operate when the user holds it with one hand
- area 1003 is an area that is inconvenient to operate when the user holds it with one hand
- area 1002 is an area that is easy to operate when the user holds it with one hand.
- the electronic device When the electronic device obtains a new notification, the electronic device displays a notification prompt window on the top of the touch screen.
- the display area of the notification prompt window is located in the area 1001 shown in Figure 1 . Therefore, when the user holds the electronic device with one hand, it is inconvenient for the user to operate the notification prompt window with one hand.
- the execution subject of the notification viewing method provided by the embodiment of the present application may be the electronic device 100 .
- FIG. 2A shows a schematic diagram of an electronic device 100 provided by an embodiment of the present application.
- the electronic device 100 includes a front camera 101.
- the front camera 101 can be used to obtain the user's face or eye image.
- the face or eye image can be used to estimate the user's line of sight and determine the relationship between the user's line of sight and the electronic device. 100 touch screen intersection.
- the electronic device 100 displays the top notification prompt window, the electronic device 100 can view the notification content corresponding to the notification prompt window or close the notification prompt window based on the result of the line of sight estimation and the user's operation on the touch screen.
- the electronic device 100 may be a mobile phone as shown in FIG. 2A or other electronic devices such as a tablet computer.
- the embodiment of the present application also provides a system that can execute a notification viewing method.
- the following introduces the architecture of a system provided by embodiments of this application.
- system 10 may include electronic device 100 and electronic device 200 .
- the electronic device 100 may be a mobile phone as shown in FIG. 2B , a tablet computer, or the like.
- the electronic device 200 may be smart glasses as shown in Figure 2B, or it may be an augmented reality (Augmented Reality, AR) device, a Virtual Reality (VR) device, or the like.
- AR Augmented Reality
- VR Virtual Reality
- Electronic device 200 may be used to obtain images of the user's face or eyes.
- the electronic device 200 may send the user's face or eye image to the electronic device 100 for the electronic device 100 to estimate the user's gaze.
- the electronic device 200 can also obtain the positional relationship between the electronic device 100 and the electronic device 200, and estimate the user's line of sight based on the image of the user's face or eyes and the positional relationship between the electronic device 100 and the electronic device 200.
- the intersection point of the user's line of sight and the touch screen of the electronic device 100 is determined, and the result of the line of sight estimation is sent to the electronic device 100 .
- the electronic device 100 can view the notification content corresponding to the notification prompt window or close the notification prompt window based on the result of the line of sight estimation and the user's operation on the touch screen.
- the electronic device 100 and the electronic device 200 may communicate through Bluetooth technology (including basic rate (BR)/enhanced data rate (EDR) Bluetooth and Bluetooth low energy (BLE)).
- Bluetooth technology including basic rate (BR)/enhanced data rate (EDR) Bluetooth and Bluetooth low energy (BLE)
- the following describes a notification viewing method based on line of sight estimation and inertial measurement unit (IMU) tracking provided by embodiments of the present application.
- IMU inertial measurement unit
- the electronic device 100 can obtain the user's facial image through the front camera, and determine whether the area the user is looking at is the area displayed by the notification prompt window based on the user's facial image. When it is determined that the user's gaze area is within the area displayed in the notification prompt window, the electronic device 100 may perform corresponding operations in conjunction with the motion of the electronic device 100 measured by the IMU. For example, when the electronic device 100 detects that the electronic device 100 is approaching When the user moves, the electronic device 100 may display a floating window, and display the specific content of the notification corresponding to the notification prompt window in the floating window.
- the electronic device 100 determines to ignore the notification and moves the notification prompt window to the right until it disappears completely.
- the electronic device 100 may determine to display the notification prompt window later, and move the notification prompt window to the left until it disappears completely, etc. wait.
- users can view the specific content of the notification through one-handed operation, or quickly cancel the notification prompt window at the top of the display, etc.
- the electronic device 100 may not be accurately recognized as the user's hand swings or approaches, resulting in the electronic device 100 being unable to perform corresponding operations.
- the actions in the above method are also inconvenient for the user to operate.
- An embodiment of the present application provides a notification viewing method.
- the electronic device 100 displays a notification prompt window in the first area of the touch screen
- the electronic device 100 can obtain the gaze information, and determine whether the user's gaze area is located in the first area based on it. After determining that the user's gaze area is located in the first area, the electronic device 100 may receive and respond to the user's operation on the second area on the electronic device 100, view the content of the notification or cancel the display of the notification prompt window, etc.
- the second area is the electronic device 100 for areas outside the first area. In this way, when the user holds the electronic device 100 with one hand, the user can conveniently operate the notification prompt window (for example, view the specific content of the notification or quickly cancel the display of the notification prompt window, etc.).
- FIG. 3 shows a schematic structural diagram of the electronic device 100.
- the electronic device 100 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (personal digital assistant) digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, vehicle-mounted device, smart home device and/or Smart city equipment, the embodiment of this application does not place special restrictions on the specific type of electronic equipment.
- PDA personal digital assistant
- AR augmented reality
- VR virtual reality
- AI artificial intelligence
- wearable device wearable device
- vehicle-mounted device smart home device and/or Smart city equipment
- the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, sensor module 180, button 190, camera 193, display screen 194, and subscriber identification module (subscriber identification module, SIM) card interface 195, etc.
- the sensor module 180 may include a touch sensor 180K and the like.
- the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100 .
- the electronic device 100 may include more or fewer components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently.
- the components illustrated may be implemented in hardware, software, or a combination of software and hardware.
- the processor 110 may include one or more processing units.
- the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc.
- application processor application processor, AP
- modem processor graphics processing unit
- GPU graphics processing unit
- image signal processor image signal processor
- ISP image signal processor
- controller video codec
- digital signal processor digital signal processor
- DSP digital signal processor
- baseband processor baseband processor
- neural network processor neural-network processing unit
- the controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.
- the processor 110 may also be provided with a memory for storing instructions and data.
- the memory in processor 110 is cache memory. This memory may hold instructions or data that have been recently used or recycled by processor 110 . If the processor 110 needs to use the instructions or data again, it can be called directly from the memory. Repeated access is avoided and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.
- the charging management module 140 is used to receive charging input from the charger.
- the power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.
- the power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, the wireless communication module 160, and the like.
- the power management module 141 can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, resistance resistance) and other parameters.
- the power management module 141 may also be provided in the processor 110 .
- the power management module 141 and the charging management module 140 may also be provided in the same device.
- the wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
- Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
- Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example: Antenna 1 can be reused as a diversity antenna for a wireless LAN. In other embodiments, antennas may be used in conjunction with tuning switches.
- the mobile communication module 150 can provide solutions for wireless communication including 2G/3G/4G/5G applied on the electronic device 100 .
- the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc.
- the mobile communication module 150 can receive electromagnetic waves through the antenna 1, perform filtering, amplification and other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation.
- the mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves through the antenna 1 for radiation.
- at least part of the functional modules of the mobile communication module 150 may be disposed in the processor 110 .
- at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
- a modem processor may include a modulator and a demodulator.
- the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal.
- the demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal.
- the demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
- the application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194.
- the modem processor may be a stand-alone device.
- the modem processor may be independent of the processor 110 and may be provided in the same device as the mobile communication module 150 or other functional modules.
- the wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (bluetooth, BT), and global navigation satellites.
- WLAN wireless local area networks
- System global navigation satellite system, GNSS
- frequency modulation frequency modulation, FM
- near field communication technology near field communication, NFC
- infrared technology infrared, IR
- the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
- the wireless communication module 160 receives electromagnetic waves via the antenna 2 , demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
- the wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
- the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
- the wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc.
- the GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (quasi) -zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).
- GPS global positioning system
- GLONASS global navigation satellite system
- BDS Beidou navigation satellite system
- QZSS quasi-zenith satellite system
- SBAS satellite based augmentation systems
- the electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like.
- the GPU is an image processing microprocessor and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
- Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
- the display screen 194 is used to display images, videos, etc.
- Display 194 includes a display panel.
- the display panel can use a liquid crystal display (LCD).
- the display panel can also use an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (OLED).
- OLED organic light-emitting diode
- OLED active matrix organic light-emitting diode
- OLED active matrix organic light-emitting diode
- the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
- the electronic device 100 can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.
- the ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera sensor through the lens, the optical signal is converted into an electrical signal, and the camera sensor passes the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise and brightness. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193.
- Camera 193 is used to capture still images or video.
- the object passes through the lens to produce an optical image that is projected onto the photosensitive element.
- the photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
- CMOS complementary metal-oxide-semiconductor
- the photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal.
- ISP outputs digital image signals to DSP for processing.
- DSP converts digital image signals into standard RGB, YUV and other format image signals.
- the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
- Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
- Video codecs are used to compress or decompress digital video.
- Electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
- MPEG moving picture experts group
- MPEG2 MPEG2, MPEG3, MPEG4, etc.
- NPU is a neural network (NN) computing processor.
- NN neural network
- Intelligent cognitive applications of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, etc.
- the internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).
- RAM random access memories
- NVM non-volatile memories
- Touch sensor 180K also known as "touch device”.
- the touch sensor 180K can be disposed on the display screen 194.
- the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen”.
- the touch sensor 180K is used to detect a touch operation on or near the touch sensor 180K.
- the touch sensor can pass the detected touch operation to the application processor to determine the touch event type.
- Visual output related to the touch operation may be provided through display screen 194 .
- the touch sensor 180K may also be disposed on the surface of the electronic device 100 at a location different from that of the display screen 194 .
- the buttons 190 include a power button, a volume button, etc.
- Key 190 may be a mechanical key. It can also be a touch button.
- the electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .
- the electronic device 100 may also include a motor 191, and the motor 191 may generate a vibration prompt.
- the motor 191 can be used for vibration prompts for incoming calls and can also be used for touch vibration feedback.
- touch operations acting on different applications can correspond to different vibration feedback effects.
- the motor 191 can also correspond to different vibration feedback effects for touch operations in different areas of the display screen 194 .
- Different application scenarios such as time reminders, receiving information, alarm clocks, games, etc.
- the touch vibration feedback effect can also be customized.
- the electronic device 100 may also include an indicator 192.
- the indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, or may be used to indicate messages, missed calls, notifications, etc.
- Figure 4 shows a schematic structural diagram of an electronic device 200.
- the electronic device 200 may include: a processor 201 , a memory 202 , a Bluetooth communication module 203 , a camera module 204 and a power module 205 .
- the electronic device 200 may also include a button 206 . in:
- Processor 201 may be used to read and execute computer readable instructions.
- the processor 201 may mainly include a controller, arithmetic unit, and registers.
- the controller is mainly responsible for decoding instructions and issuing control signals for operations corresponding to the instructions.
- the arithmetic unit is mainly responsible for saving register operands and intermediate operation results temporarily stored during instruction execution.
- the hardware architecture of the processor 201 may be an application specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture, etc.
- ASIC application specific integrated circuit
- Memory 202 is coupled to processor 201 for storing various software programs and/or sets of instructions.
- the memory 202 may include high-speed random access memory.
- the memory 202 may also include non-volatile memory.
- Memory 202 may store communication programs that may be used to communicate with electronic device 100 or other devices.
- the memory 202 can also store operating systems, such as uCOS, VxWorks, RTLinux and other embedded operating systems.
- the Bluetooth communication module 203 can be used to establish a communication connection with the electronic device 100 .
- the Bluetooth communication module 203 may include a Bluetooth chip.
- the electronic device 200 can pair with the Bluetooth chip of the electronic device 100 and establish a Bluetooth connection through the Bluetooth chip, so as to realize wireless communication and business processing between the electronic device 200 and the electronic device 100 through the Bluetooth connection.
- the Bluetooth chip can support BR/EDR Bluetooth and BLE, for example, it can receive/method paging (page) information, receive/send BLE broadcast messages, etc.
- the Bluetooth communication module 203 may also include an antenna.
- the Bluetooth communication module 203 receives electromagnetic waves via the antenna, frequency modulates and filters the electromagnetic wave signals, and processes the processed electromagnetic waves.
- the signal is sent to processor 201.
- the Bluetooth communication module 203 can also receive the signal to be sent from the processor 201, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna for radiation.
- the camera module 204 may be used to obtain images of the user's eyes and/or face.
- the camera module 204 may include 1 or N cameras, where N is a positive integer greater than 1. Cameras are used to capture still images or videos. The object passes through the lens to produce an optical image that is projected onto the photosensitive element.
- the photosensitive element may be a charge coupled device or a complementary metal oxide semiconductor phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal.
- ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other format image signals.
- the camera module 204 may also be used to acquire images of the electronic device 100 .
- the power module 205 can be used to provide system power for the electronic device 200, power each module of the electronic device 200, and support the electronic device 200 to receive charging input, etc.
- the power module 205 may include a power management unit (PMU) and a battery.
- PMU power management unit
- the power management unit can receive external charging input; provide the electrical signal input by the charging circuit to charge the battery; and can also provide the electrical signal provided by the battery to other modules in the electronic device 200 to prevent the battery from overcharging and over-discharging. , short circuit or overcurrent, etc.
- the power module 205 may also include a wireless charging coil for wirelessly charging the electronic device 200 .
- the power management unit can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, impedance) and other parameters.
- the electronic device 200 may also include buttons and/or sensors (not shown in FIG. 4 ) and the like.
- the buttons can be physical buttons or touch buttons (used in conjunction with touch sensors), etc., used to trigger operations such as power on, power off, pause, play, record, start pairing, reset, etc.
- the touch sensor can detect the user's touch operations such as single click, double click, multiple clicks, long press, heavy pressure, etc. It can also perform user fingerprint recognition to authenticate the user's identity in security-sensitive business scenarios.
- the electronic device 200 shown in FIG. 4 is only an example, and the electronic device 200 may have more or fewer components than shown in FIG. 4 , two or more components may be combined, or Can have different component configurations.
- the various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.
- the electronic device 100 may obtain the first notification of the first application (for example, information, notification of application software, etc.) when displaying the first interface (for example, desktop, interface in application software, etc.). etc).
- the electronic device 100 may display a notification prompt window in the first area of the touch screen, and the notification prompt window includes part or all of the content of the first notification.
- the electronic device 100 may determine the user's gaze area based on gaze information (eg, the user's facial image, the user's eye image, the user's electrooculogram information, the user's eye movement information, etc.).
- the electronic device 100 may receive and respond to the user's request for a second area of the electronic device 100 (ie, an area other than the first area on the touch screen of the electronic device 100).
- Operation 1 (for example, sliding operation), display a floating window in the first interface.
- the floating window is used to display the interface of the first application.
- the interface of the first application includes the specific content of the first notification.
- the electronic device 100 can also output gaze feedback.
- the gaze feedback can be changing the display color of the first interface, changing the color of the notification prompt window, or changing the color of the horizontal bar below the notification message, etc.
- the electronic device 100 may also determine whether to display the interface of the first application in the floating window based on the distance moved on the touch screen in operation 1.
- the electronic device 100 displays a desktop interface 500 .
- the desktop interface 500 displays a page on which application icons are placed.
- the page includes multiple application icons (for example, a stock application icon, a calculator application icon, a browser application icon, a social application icon 501, etc.).
- a page indicator 502 is also displayed below the page where the application icon is placed to indicate the total number of pages on the desktop interface 500 and the positional relationship between the currently displayed page and other pages.
- the desktop interface 500 may include three pages, and the white dot in the page indicator is located at the third position from the left, which may indicate that the currently displayed page is the third page from the left among the three pages.
- a status bar 503 is also displayed above the page where the application icon is placed.
- the status bar 503 may include information such as a strength indicator of the communication signal, a power value, and time.
- a dock area 504 below the page indicator, and the dock area 504 may include one or more tray icons (for example, a dialing application icon, a messaging application icon, a contact application icon, a camera application icons, etc.), one or more tray icons in the dock area can remain displayed when switching pages.
- the electronic device 100 may receive and respond to the user's click operation on the social application icon 501, open the social application software, and display the social application interface 510.
- the social application interface 510 includes one or more graphic and text contents (for example, graphic and text content 511 ), a range selection bar 512 and an auxiliary control bar 513 .
- Graphic content 511 may include user avatar 511a, user name 511b, picture 511c, forwarding control 511d, comment control 511e and like control 511f.
- the user avatar 511a may be used to indicate the avatar of the user who posted the graphic content 511
- the user name 511b may be used to indicate the nickname of the user who posted the graphic content 511, and so on.
- the forwarding control 511d can be used to trigger the electronic device 100 to forward the graphic content 511
- the comment control 511e can be used to trigger the electronic device 100 to display a comment input interface for the graphic content 511
- the like control 511f can be used to trigger the electronic device 100 to change the graphic content 511.
- the like status of 511 changes the color of the like control 511f.
- the electronic device 100 can respond to the user's click operation on the like control 511f, change the color of the like control 511f from white to black, that is, as shown in FIG.
- the like status of article content 511 was changed from unliked to liked.
- the graphic content 511 may also include more or less pictures, text, etc., which is not limited in this application.
- the range selection column 512 may include one or more range options, such as a focus option, a recommendation option 512a, and a local option.
- the range option can be used to trigger the electronic device 100 to display one or more graphic contents corresponding to the range option.
- the selected range option may be displayed in bold font, text in different colors, different background colors, etc.
- the recommended option 512a is displayed in bold font, that is, the recommended option 512a is in a selected state, indicating that one or more graphic and text contents displayed in the social application interface 510 are graphic and text contents recommended by the social application software.
- the range selection bar 512 may also include one or more controls, such as a photo control 512b and an add control 512c.
- the camera control 512b can be used to trigger the electronic device 100 to display a shooting interface
- the add control 512c can be used to trigger the electronic device 100 to add one or more range options.
- the auxiliary control bar 513 may include one or more auxiliary controls, such as a homepage control 513a, a search control 513b, a message control 513c, and a personal homepage control 513d.
- the homepage control 513a can be used to trigger the electronic device 100 to display the homepage of the social application software.
- the search control 513b may be used to trigger the electronic device 100 to display a search interface.
- the message control 513c can be used to trigger the electronic device 100 to display a message interface in the social application software.
- the personal homepage control 513d may be used to trigger the electronic device 100 to display the user's personal homepage in the social application software.
- the electronic device 100 can obtain the notification of the Changlian application. While the electronic device 100 can obtain the notification of the Changlian application, the notification prompt window 520 as shown in FIG. 5C is displayed in the first area of the touch screen. The notification prompt window 520 is used to prompt the user that the electronic device 100 has received the Changlian application. Even app notifications.
- the notification prompt window 520 may include an icon 520a.
- the notification prompt window 520 may also include any one or more of the following: sender 520b, notification brief description 520c, and read mark control 520d. , application name 520e, notification time 520f, horizontal bar 520g.
- the icon 520a can be used to prompt the application software corresponding to the notification, that is, the Changlian application software.
- the application software name 520e can also be used to prompt that the application software corresponding to the notification is the Changlian application software.
- the sender 520b can be used to prompt the sender of the notification, that is, Xiaohong.
- Notification brief description 520c can be used to prompt the content of the notification, that is, "There is a good exhibition on Saturday, do you want to go see it?". It can be understood that when the number of words in the notification content exceeds a certain number of words (for example, 20 words), the notification summary 520c may only display a part of the text in the notification content, for example, only the first 15 words in the notification content may be displayed. Text, etc. In some embodiments, notification content may also be hidden. Mark as read control 520d may be used to trigger the electronic device 100 to mark the notification as read. The notification time 520f may be used to prompt the electronic device 100 to receive the notification, such as "just now".
- the horizontal bar 520g can be used to trigger the electronic device 100 to cancel the display of the notification prompt window 520 .
- the notification prompt window 520 may also include more or less content than the notification prompt window 520 shown in FIG. 5C , which is not limited in this application.
- the electronic device 100 may obtain images of the user's eyes and/or face. As shown in FIG. 5D , the electronic device 100 can acquire an image of the user's eyes and/or face while displaying the notification prompt window. The electronic device 100 can determine the direction of the user's gaze based on the image of the user's eyes or face, and further determine whether the area the user is looking at is within the area (first area) where the notification prompt window 520 is located.
- the electronic device 100 when the electronic device 100 detects that the user's gaze area is located in the first area, the electronic device 100 can change the display color of the social application interface 510 in the background layer of the notification prompt window 520 . In some embodiments, the electronic device 100 may also change the display color of the social application interface 510 in the background layer of the notification prompt window 520 when detecting that the user's gaze time on the first area exceeds a first time threshold (eg, 500 milliseconds).
- a first time threshold eg, 500 milliseconds
- the electronic device 100 may receive the user's response to an area other than the first area on the touch screen of the electronic device 100 (in this application It can also be called the sliding operation of the second area).
- the electronic device 100 can move The notification prompt window 520 in the mobile social application interface 510 moves downward.
- the notification prompt window 520 may be displayed in a blurred manner as it moves downward along with the user's sliding operation.
- the first distance may also be written as the first movement distance.
- the electronic device 100 when the electronic device 100 detects that the duration of the user's sliding operation for the second area on the touch screen is not less than the first duration (for example, 1 microsecond or 1 millisecond, etc.), as shown in FIG. As shown in 5F, the electronic device 100 can move the notification prompt window 520 in the mobile social application interface 510 downward. In the embodiment of this application, the first duration can also be written as the second duration.
- the first duration for example, 1 microsecond or 1 millisecond, etc.
- the notification prompt window 520 in the social application interface 510 can gradually become larger and expanded as the user slides down to form a floating window 530 as shown in Figure 5G.
- the second distance is greater than the first distance
- the second duration is greater than the first duration.
- the second distance can also be written as the second moving distance
- the second duration can also be written as the third duration.
- the floating window 530 may include an icon 531, and the icon 531 may be an application software icon corresponding to the notification, that is, a Changlian application software icon.
- the electronic device 100 may display a larger floating window than the floating window 530 shown in FIG. 5G in response to the sliding operation.
- the position of the floating window 530 can also move downward as the user continues to slide downward.
- the electronic device 100 when the electronic device 100 detects that the sliding distance moved on the touch screen is not less than a second distance (for example, 200 pixels), or when the electronic device 100 detects that the sliding operation moves When the duration is not less than the second duration (for example, 0.1 seconds), the electronic device 100 can cancel the display of the notification prompt window 520 and display the floating window 530 as shown in FIG. 5G.
- a second distance for example, 200 pixels
- the electronic device 100 can cancel the display of the notification prompt window 520 and display the floating window 530 as shown in FIG. 5G.
- the electronic device 100 When the electronic device 100 detects that the distance moved by the slide operation on the touch screen is not less than a third distance (for example, 500 pixels), or when the electronic device 100 detects that the duration of the slide operation is not less than a third duration. (for example, 1 second), the electronic device 100 can display the Changlian application interface 541 as shown in FIG. 5H in the floating window 530, where the third distance is greater than the second distance, and the third duration is greater than the second duration.
- a third distance for example, 500 pixels
- a third duration for example, 1 second
- the Changlian application interface 541 can be displayed in the floating window 530.
- the Changlian application interface 541 may include one or more messages, a sender identification 541b, and one or more controls for replying to messages (for example, a recording control 541c, a sending control 541d, a switching input control 541e, etc.).
- the one or more messages include message 541a, which is used to prompt the specific content of the notification.
- the message 541a may include one or more of text, pictures, and voice.
- the message 541a may be "There is a good exhibition on Saturday, do you want to go see it?"
- the sender identification 541b is used to prompt the sender of the message 541a.
- the sender identification 541b may be the sender's avatar, or may be the sender's mobile phone number, user nickname, or other information that can prompt the sender's identity.
- the sender in the embodiment of this application refers to the electronic device that sends the message 541a.
- the one or more controls for replying to a message may be used to reply to message 541a, that is, to send information to the sender of message 541a.
- the recording control 541c is used to trigger the electronic device 100 to record voice information.
- the sending control 541d is used to trigger the electronic device 100 to send the recorded voice message to the sender of the message 541a.
- the switching input control 541e is used to trigger the electronic device 100 to switch the current input mode from voice input to keyboard input, and display the keyboard and input field.
- the one or more controls for replying to messages may also include a picture control 541f, an envelope control 541g, an expression control 541h, a position control 541i, and an add control 541j.
- the picture control 541f can be used to send picture information.
- the envelope control 541g can be used to trigger the electronic device 100 to send the information to be sent in the form of envelope information.
- Expression control 541h can be used to send expressions.
- Location control 541i may be used to send location information of electronic device 100.
- Add Control 541j can be used to add additional controls.
- the floating window 530 may also include one or more auxiliary controls, such as a full screen control 542, a minimize control 543, a drag control 544, and a close control 545.
- the full-screen control 542 can be used to trigger the electronic device 100 to open the Changlian application software and display the Changlian application interface 541 displayed in the floating window 530 in full screen.
- the minimization control 543 can be used to trigger the electronic device 100 to cancel the display of the floating window 530 and display a floating ball in the social application interface 510.
- the display size of the floating ball is much smaller than the display size of the floating window 530, and there can be a smooth display on the floating ball. Even application software icons.
- the drag control 544 can be used to trigger the electronic device 100 to move the floating window 530 along the user's sliding trajectory.
- the close control 545 can be used to cancel display of the floating window 530 .
- the electronic device 100 may respond to the user's release operation and determine that the display position of the floating window 530 when the user releases the operation is the display position of the floating window 530.
- the electronic device 100 can also be based on the user's gaze area and the user's second area of the electronic device 100 For domain operations, check the notification content corresponding to the notification prompt window at the top. Since there is no need to operate the notification prompt window on the top of the electronic device 100, the user can easily view the notification content even in a scenario where the user operates with one hand.
- the electronic device 100 when the electronic device 100 displays gaze feedback, the electronic device 100 can also receive and respond to the user's operation 2 (for example, click operation, long press operation) on the second area on the electronic device 100 , double-click operation or long-press operation with two fingers, etc.), open the application software corresponding to the notification prompt window, and display the interface of the application software.
- the user's operation 2 for example, click operation, long press operation
- the electronic device 100 can also receive and respond to the user's operation 2 (for example, click operation, long press operation) on the second area on the electronic device 100 , double-click operation or long-press operation with two fingers, etc.), open the application software corresponding to the notification prompt window, and display the interface of the application software.
- the electronic device 100 100 may display social application interface 510 as shown in Figure 6A.
- the social application interface 510 is the same as the aforementioned social application interface 510 shown in FIG. 5E.
- the electronic device 100 can receive and respond to the user's long press operation on the second area, open the Changlian application software, and display the Changlian application interface 600 as shown in Figure 6B.
- the specific content of the Changlian application interface 600 can be referred to the relevant description of the Changlian application interface 541 displayed in the floating window 530 shown in Figure 5H, and will not be described again here.
- the application software corresponding to the notification prompt window can be directly opened through a simpler operation, and the application software interface can be displayed.
- the time required to display the interface of the application software is shortened.
- the electronic device 100 may also receive and respond to the user's operation 3 on the second area on the electronic device 100 (for example, a double-click operation, a long press operation, or a two-finger long press operation). Press operation, etc.) to display the floating window 530 as shown in Figure 5H. It should be noted that operation 2 is different from operation 3.
- the electronic device 100 can also respond to the user's operation 4 on the second area on the electronic device 100 (for example, , right swipe operation), cancel the display of the notification prompt window, for example, drag the notification prompt window to the right until it disappears completely, or cancel the display of the notification prompt window and display a floating ball, which can be used to trigger the electronic device 100 to display a notification A prompt window, or an interface used to trigger the electronic device 100 to display the first application.
- the electronic device 100 can also mark the notification corresponding to the notification prompt window as read in response to operation 4.
- the electronic device 100 100 may display social application interface 510 as shown in Figure 7A.
- the social application interface 510 is the same as the aforementioned social application interface 510 shown in FIG. 5E.
- the electronic device 100 may cancel the display of the notification prompt window 520 in response to the user's right sliding operation on the second area, and display the notification prompt window 701 as shown in FIG. 7B on the social application interface 510 .
- the notification prompt window 701 is a part of the notification prompt window 520 shown in FIG. 7A , and the notification prompt window 701 is located farther in the social application interface 510 than the notification prompt window 520 is in the social application interface 510 . Keep Right.
- the electronic device 100 displays the social application interface 510 shown in FIG. 7B
- the electronic device 100 can receive and respond to the user's right sliding operation and continue to slide to the right. Drag the notification prompt window 701 until the notification prompt window 701 disappears completely. At this time, the electronic device 100 can cancel the display of the notification prompt window 701 and display the social application interface 510 shown in FIG. 7C.
- the electronic device 100 may also change the notification prompt window 701 into a floating ball in response to the user's right sliding operation.
- the electronic device 100 can cancel the display of the gaze feedback in the social application interface 510, change the display color of the social application interface 510, and display the social application interface 510 as shown in Figure 7D .
- the electronic device 100 when the electronic device 100 displays the social application interface 510 as shown in Figure 7C, the electronic device 100 can also cancel output of gaze feedback after the second time threshold (for example, 10 milliseconds) has passed, as shown in Figure 7D Social application interface 510 is shown.
- the second time threshold for example, 10 milliseconds
- the electronic device 100 can also swipe away the notification prompt window based on the user's gaze area and the user's operation on the second area of the electronic device 100 . Similarly, since there is no need to operate the notification prompt window on the top of the electronic device 100, the user can easily perform the above operations even in a scenario where the user operates with one hand.
- the electronic device 100 can also respond to the user's operation 5 on the second area on the electronic device 100 (for example, swipe left), mark the notification as a reminder later, cancel the display of the notification prompt window (for example, drag the notification prompt window to the left until it disappears completely), and after The notification prompt window is redisplayed after a period of time (for example, the fourth period of time).
- a period of time for example, the fourth period of time.
- the electronic device 100 may also determine whether to cancel the display of the notification prompt window based on the duration of the user's gaze on the second area. In some embodiments, the electronic device 100 may also determine whether to cancel the display of the notification prompt window based on the display duration of the notification prompt window when detecting that the user is gazing at the second area.
- the electronic device 100 after receiving the notification from the Changlian application software, the electronic device 100 detects that the area the user is looking at is within the first area, and the time the user is looking at the first area is not less than the first time threshold. At this time, the electronic device 100 The device 100 may display a social application interface 510 as shown in FIG. 8A , which is the same as the social application interface 510 shown in FIG. 5E .
- the electronic device 100 when the electronic device 100 detects that the user is gazing at the second area, and the gazing time is not less than the third time threshold, as shown in FIG. 8B , the electronic device 100 can cancel output of gaze feedback and change the social application interface 510 display color.
- the electronic device 100 may cancel the display of the notification prompt window 520 .
- the fourth time threshold is greater than the third time threshold.
- the electronic device 100 may also use the moment when the electronic device 100 displays the social application interface 510 as shown in FIG. 8B as a starting point, when it is determined that the time the user looks at the second area is greater than the fifth time threshold. , the notification prompt window 520 is cancelled, and the social application interface 510 shown in FIG. 8C is displayed.
- the social application interface 510 as shown in FIG. 8C is displayed.
- the electronic device 100 can determine whether to cancel the display of the notification prompt window by detecting the duration of the user's gaze on the second area or by detecting the display duration of the notification prompt window when the user is gazing at the second area, without requiring the user to perform additional operations, simplifying the operating steps.
- the electronic device 100 may also perform the function of the first interface in response to the user's operation 6 on the electronic device 100 (eg, slide-up operation, etc.). It should be noted that this operation is different from any of the above operations (for example, operation 1, operation 2, operation 3, etc.).
- the electronic device 100 may display a social application interface 510 with gaze feedback.
- the social application interface 510 is the same as the aforementioned social application interface 510 shown in FIG. 5E .
- the electronic device 100 may, in response to the user's upward sliding operation on the electronic device 100, slide down one or more graphic content in the social application interface 510 to display the graphic content 901 as shown in FIG. 9B.
- the graphic and text content 901 is different from the graphic and text content in the social application interface 510 shown in FIG. 9A .
- the social application interface 510 shown in FIG. 9B may also include a sliding bar 902, which may be used to trigger the electronic device 100 to slide and display the graphic content in the social application interface 510.
- FIG. 9A to FIG. 9B are only exemplary illustrations.
- operation for example, operation 6
- the electronic device 100 can receive and respond to the operation 6, and perform a response corresponding to the operation 6.
- This application does not limit the specific form of operation 6 and the response corresponding to operation 6.
- the area on the touch screen of the electronic device 100 other than the first area may include a second area and a third area.
- the electronic device 100 may control the notification prompt window.
- the electronic device 100 may perform the function of the first interface.
- the second area may be the area 1002 shown in the above-mentioned FIG. 1
- the third area may be the area 1003 shown in the above-mentioned FIG. 1.
- the first area may be the top area of the touch screen
- the second area may be the bottom area of the touch screen
- the third area may be the middle area of the touch screen, and so on. This application does not limit the specific division of areas.
- the electronic device 100 can also change the color of the notification prompt window, change the color of the horizontal bar under the notification message in the notification prompt window, flash the indicator light, light up the indicator light, play a prompt sound or vibrate. and other forms to output gaze feedback. In another possible implementation, the electronic device 100 may not output gaze feedback.
- the electronic device 100 can change the color of the notification prompt window 520, as shown in Figure The social application interface 510 shown in 10.
- the interface for displaying the notification prompt window can also be the desktop, a negative screen interface or an interface in other application software.
- the notification prompt window The corresponding notification can also be a notification from other application software. This application is not limited here.
- the specific process of the notification viewing method may include the following steps:
- the electronic device 100 When the electronic device 100 displays the first interface, the electronic device 100 obtains the first notification of the first application and displays a notification prompt window in the first area of the touch screen.
- the first interface may be an interface in application software, such as the social application interface 510 in the embodiment shown in FIG. 5B , or it may be a desktop or a one-screen interface displaying multiple application software icons, and so on.
- the first notification may be a notification of the first application, such as a notification of the Changlian application software in the embodiment shown in FIG. 5C.
- the first notification may also be a text message, a notification from a chat application software, a notification from a video application software, and so on.
- the first area may be a top area on the touch screen of the electronic device 100 . In some embodiments, the first area may also be other areas on the touch screen.
- the electronic device 100 may obtain the first notification by receiving the first notification sent by other electronic devices, or by obtaining the first notification pushed by the first application, and so on.
- the electronic device 100 may display a notification prompt window on the first interface, and the notification prompt window includes part or all of the content of the first notification.
- the notification prompt window may be the notification prompt window 520 in the social application interface 510 shown in FIG. 5C .
- the electronic device 100 obtains gaze information.
- the gaze information includes any one or more of the user's facial image, the user's eye image, eye movement information, and electrooculogram information.
- the electronic device 100 may obtain the user's image information through a camera.
- the user's image information may include the user's facial image information and/or the user's eye image information.
- the electronic device 100 may respond to the first notification, turn on the front camera of the electronic device 100, and continuously obtain the user's image information through the front camera of the electronic device 100.
- the electronic device 100 may respond to the first notification, at a fixed time Continuously obtain the user's image information at intervals.
- the front camera of the electronic device 100 can always remain on, that is, the electronic device 100 does not need to respond to the first notification and can continue to obtain the user's image information.
- the electronic device 100 can also obtain the user's eye movement information or electrooculogram information.
- S1103 The electronic device 100 determines whether the user's gaze area is located in the first area based on the gaze information.
- the gaze area being located in the first area may include the following three situations: the gaze area completely coincides with the first region, the gaze area is within the first region, and the gaze area and the first region exist intersection.
- the relationship between the gaze area and the first area will be described below with the help of diagrams.
- the first area may be area 2001 on the touch screen of the electronic device 100
- the user's gaze area may also be area 2001 , that is, the first area completely overlaps with the user's gaze area. In this case, it can be determined that the user's gaze area is located in the first area.
- the first area may be area 2001 on the touch screen of the electronic device 100
- the user's gaze area may be area 2002 , that is, the user's gaze area may be within the first area. In this case, it can be determined that the user's gaze area is located in the first area.
- the first area may be area 2001 on the touch screen of the electronic device 100
- the user's gaze area may be area 2003 , that is, the user's gaze area intersects with the first area, and the intersection is area 2004 .
- the gaze area may also be a rounded rectangle, a circle, or other shapes
- the first area may also be a rounded rectangle, a circle, or the like. This application does not limit the specific shapes of the first area and the gaze area.
- the specific process for the electronic device 100 to determine whether the user's gaze area is located in the first area based on the user's image information may include the following steps:
- the electronic device 100 determines the gaze direction of the user's line of sight in a first coordinate system based on the user's image information.
- the first coordinate system is a coordinate system with the center point of the user's face or eyes as the origin.
- the electronic device 100 determines the position and posture of the user's head relative to the front camera of the electronic device 100 based on the user's image information.
- the electronic device 100 determines the gaze of the user's gaze in the second coordinate system based on the gaze direction of the user's gaze in the first coordinate system and the position and posture of the user's head relative to the front camera of the electronic device 100 direction, the second coordinate system is a coordinate system with the light point of the front camera of the electronic device 100 as the origin and the optical axis of the front camera as the Z axis.
- the electronic device 100 determines the intersection point of the user's line of sight and the touch screen as the first position based on the gaze direction of the user's line of sight in the second coordinate system and the coordinates of the touch screen in the second coordinate system.
- the coordinates of the touch screen in the second coordinate system may be data pre-stored in the electronic device 100 .
- the electronic device 100 may also determine the coordinates of the touch screen in the second coordinate system based on the size of the touch screen of the electronic device 100 and the position of the front camera relative to the touch screen.
- the electronic device 100 determines the user's gaze area based on the first location.
- the electronic device 100 may use the first position as the center of the gaze area, and determine the gaze area as the length and width of the rectangular area with a specific length (such as the width of the touch screen, eye distance or viewing distance, etc.). It can be understood that the gaze area can also be a circular area or an area of other shapes, which is not limited in this application.
- the electronic device 100 determines whether the user's gaze area is located in the first area.
- the electronic device 100 may determine that the user's gaze area is located in the first area.
- the electronic device 100 may also determine whether the user's gaze area is located in the first area based on the eye movement information or electrooculogram information.
- eye movement information is image information of eye movement or potential information generated by eye movement.
- electrooculogram eog
- electrooculogram information By analyzing and calculating eye movement information or electrooculogram information, the eye movement trajectory can be obtained. Based on the movement trajectory of the eyeballs, the user's line of sight direction relative to the electronic device 100 can be calculated, and then the user's gaze area on the electronic device 100 can be determined.
- step S1104 When the electronic device 100 determines that the user is gazing at the first area, the electronic device 100 performs the following step S1104 or step S1105; when the electronic device 100 determines that the user is not gazing at the first area, the electronic device 100 performs step S1108 or step S1109.
- the electronic device 100 outputs gaze feedback.
- Step S1104 is an optional step.
- the electronic device 100 may determine whether to output gaze feedback based on a positional relationship between the user's gaze area and the first region. Gaze feedback is used to prompt the user that the gaze area is located in the first area.
- Gaze feedback may be changing the display color of the first interface, such as the social application interface 510 shown in FIG. 5E.
- the gaze feedback may also be to change the color of the notification prompt window in the first interface, such as the notification prompt window 520 in the embodiment shown in FIG. 10 .
- the gaze feedback may also be a change in the controls in the notification prompt window, for example, changing the color of the horizontal bar below the notification message in the notification prompt window, and so on.
- the gaze feedback can also be the brightness of the indicator light, the vibration of the electronic device 100, the playing of prompt sounds, etc., which is not limited in this application.
- the electronic device 100 may also determine whether to output gaze feedback based on the relationship between the duration of the user's gaze on the first area and the first time threshold.
- the electronic device 100 determines whether the time the user looks at the first area is less than a first time threshold (in the embodiment of the present application, the first time threshold can also be written as a first duration). . That is, taking the moment when the electronic device 100 determines that the user is gazing at the first area as a starting point, it is determined whether the user continues to gazing at the first area within the time period corresponding to the first time threshold. If the user continues to gaze at the first area during this time period, it is determined that the time for which the user gazes at the first area is greater than or equal to the first time threshold. If the user's gaze leaves the first area within this time period, it is determined that the time the user looks at the first area is less than the first time threshold.
- a first time threshold in the embodiment of the present application, the first time threshold can also be written as a first duration.
- step S1108 When the time the user looks at the first area is less than the first time threshold, it means that the user is not interested in the first notification prompted by the notification prompt window, and the electronic device 100 executes step S1108 or step S1109.
- the electronic device 100 may also display an operation prompt in the first interface.
- the operation prompt can be used to prompt the user.
- the operation prompt can be any one or more of text prompts, picture prompts, animation prompts, sound prompts, etc.
- the operation prompt is used to prompt the user to control the notification prompt window in the second area, that is, to prompt the user to move the notification prompt window and display the notification content through operations in the second area (such as operation 1, operation 2, operation 3, etc.) Or cancel the display of the notification prompt window, and other functions.
- the electronic device 100 receives the user's operation 1 for the second area on the electronic device 100.
- the second area is an area on the touch screen of the electronic device 100 other than the first area.
- the second area may include area 1002 and/or area 1003 in the embodiment shown in FIG. 1 .
- the second area may be a middle area on the touch screen of the electronic device 100 or a bottom area on the touch screen.
- operation 1 may be the sliding operation in the embodiment shown in FIG. 5E. In the embodiment of this application, operation 1 can also be written as the first operation.
- the electronic device 100 moves the display position of the notification prompt window downward in the first interface.
- Step S1106 is an optional step.
- the electronic device 100 may detect the distance moved by user operation 1 in the first interface or the duration of operation 1, and based on the distance moved by user operation 1 on the touch screen or the duration of operation 1 The duration determines whether the notification prompt window is moved. For example, referring to the embodiment shown in Figure 5G, when the movement distance of user operation 1 on the touch screen is not less than the first distance (in the embodiment of this application, the first distance can also be written as the first movement distance) , the electronic device 100 can move the display position of the notification prompt window downward in the first interface. As another example, when the duration of operation 1 is not less than the first duration (for example, 1 microsecond or 1 millisecond), the electronic device 100 can move the display position of the notification prompt window downward in the first interface. It should be noted that in the embodiment of the present application, the first duration can also be written as the second duration.
- the notification prompt window can be prevented from being dragged downward due to the user's accidental touch, thereby improving the accuracy of the method and reducing unnecessary energy consumption.
- the electronic device 100 displays a floating window.
- the floating window is used to display the interface of the first application.
- the interface of the first application includes the specific content of the first notification.
- the electronic device 100 may display the Changlian application interface 541 in the floating window 530 .
- the specific content of the first notification is displayed in the Changlian application interface 541.
- the user can view the specific content of the first notification and process the first notification in the interface displayed in the floating window.
- the electronic device 100 can also expand the notification prompt window into a floating window based on the movement distance of user operation 1 in the first interface or the duration of operation 1, or cancel the display of the notification prompt window and display the floating window.
- the display size of the floating window may also increase as the movement distance or duration increases.
- the electronic device 100 may also determine whether to display the interface of the first application in the floating window based on the distance moved by operation 1 on the touch screen or the duration of operation 1, such as Changlian shown in Figure 5H Application interface 541.
- S1108 The electronic device 100 determines whether the time period in which the user has not looked at the first area is greater than or equal to the fourth time threshold.
- Step S1108 is an optional step.
- the fact that the user does not focus on the first area means that the user's focus area does not intersect with the first area, that is, the user's focus area is not located in the first area.
- the electronic device 100 After the electronic device 100 determines that the user has not looked at the first area, the electronic device 100 can use the display time of the notification prompt window as a starting point to determine whether the user has never looked at the first area within the time period corresponding to the fourth time threshold.
- the electronic device 100 may use the moment when the user's sight leaves the first area as a starting point to determine the fourth time threshold. Whether the user has never looked at the first area during the corresponding time period.
- step S1110 When the electronic device 100 determines that the user has never looked at the first area during the time period, that is, the time that the user has not looked at the first area is greater than or equal to the fourth time threshold, the electronic device 100 executes step S1110.
- step S1109 When the electronic device 100 determines that the time during which the user does not look at the first area is less than the fourth time threshold within the time period, the electronic device 100 executes step S1109.
- S1109 The electronic device 100 determines whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.
- the electronic device 100 may determine whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.
- the electronic device 100 may perform step S1110.
- the electronic device 100 may perform step S1103.
- the sixth time threshold can also be written as the fifth duration.
- the electronic device 100 cancels display of the notification prompt window.
- the interface after the electronic device 100 cancels display of the notification prompt window can refer to the social application interface 510 shown in FIG. 7D and FIG. 8C.
- the electronic device 100 displays gaze feedback
- the electronic device 100 detects that the user's gaze leaves the first area (or detects that the user's gaze area is different from the first area).
- the electronic device 100 determines the length of time that the user continues not to look at the first area.
- the electronic device 100 cancels outputting gaze feedback.
- the electronic device 100 cancels display of the notification prompt window.
- the electronic device 100 may also use the moment when gaze feedback is turned off as a starting point to determine that the length of time the user has not gazed at the first area is not less than the fifth time threshold. Cancels the notification window.
- the electronic device 100 may also use the moment when gaze feedback is turned off as a starting point to determine that the length of time the user has not gazed at the first area is not less than the fifth time threshold. Cancels the notification window.
- the electronic device 100 can receive and respond to the user's operation 2 (such as a single click operation, a double-click operation, a long press) on the second area. operation, etc.), open the application software corresponding to the first notification, and display the interface of the application software, such as the Changlian application interface 600 shown in Figure 6B.
- the above operation 2 can also be written as the second operation.
- operation 3 in the embodiment shown in FIG. 6A and FIG. 6B can also be written as the second operation.
- the electronic device 100 may also receive and respond to the user's operation 3 on the second area (such as a double-click operation, a long-press operation, etc.), The floating window in step S1107 shown in FIG. 11A is displayed.
- the floating window can be quickly displayed through simpler operations, or the application software corresponding to the notification prompt window can be quickly opened and the interface of the application software can be displayed.
- the time required to display a floating window or display an application software interface is shortened.
- the electronic device 100 can also cancel the display of the notification prompt window in response to the user's operation 4 on the second area, such as a right sliding operation.
- the relevant content in the embodiment shown in FIGS. 7A to 7D please refer to the relevant content in the embodiment shown in FIGS. 7A to 7D .
- the electronic device when the user wants to view the specific content of the first notification later, can also cancel the display of the notification prompt window in response to the user's operation 5 on the second area, such as a left sliding operation. , and then display the notification prompt window after a period of time.
- the relevant content in the embodiment shown in FIGS. 7A to 7D please refer to the relevant content in the embodiment shown in FIGS. 7A to 7D .
- operation 4 can also be written as the third operation, and in other embodiments, operation 5 can also be written as the third operation.
- the electronic device 100 when the electronic device 100 outputs gaze feedback, the electronic device 100 can only respond to operations in the gaze operation set.
- the gaze operation set includes one or more operations defined in the gaze feedback state (for example, the above-mentioned operations 1 to Operation 5).
- the electronic device 100 when the electronic device 100 outputs gaze feedback, the electronic device 100 may also respond to an operation outside the gaze operation set and perform a response corresponding to the operation. For example, in the embodiment shown in FIGS. 9A to 9B , the electronic device 100 may slide down one or more graphic contents in the social application interface 510 in response to the user's upward sliding operation on the electronic device 100 , and so on.
- the specific process of the electronic device 100 responding to operation 7 when gaze feedback is displayed may include the following steps:
- the electronic device 100 When the electronic device 100 displays the first interface, the electronic device 100 obtains the first notification of the first application and displays a notification prompt window in the first area of the touch screen.
- the electronic device 100 obtains gaze information, which includes any one or more of the user's facial image, the user's eye image, eye movement information, and electrooculogram information.
- S1203 The electronic device 100 determines whether the user's gaze area is located in the first area based on the gaze information.
- step S1204 When the user's gaze area is located in the first area, the electronic device 100 executes step S1204 or step S1205.
- the electronic device 100 may also determine steps to be performed later based on the relationship between the duration of the user's gaze on the first area and the first time threshold.
- the electronic device 100 executes the following step S1205.
- the electronic device 100 executes the following step S1210.
- the electronic device 100 outputs gaze feedback.
- Step S1204 is an optional step.
- S1205 The electronic device 100 receives the user's operation 7 on the second area on the electronic device 100.
- S1206 The electronic device 100 determines whether operation 7 belongs to the gaze operation set.
- the set of gaze operations may include valid operations for the notification prompt window when the electronic device 100 outputs gaze feedback, such as one of operations 1, 2, 3, 4, and 5 in the above embodiment, or Multiple items. It can be understood that the set of gaze operations may be stored in the electronic device 100 in advance.
- the electronic device 100 determines that operation 7 does not belong to the gaze operation set, the electronic device 100 performs the following step S1207.
- the electronic device 100 determines that operation 7 belongs to the gaze operation set, the electronic device 100 performs the following step S1208.
- S1207 The electronic device 100 determines that the action object of operation 7 is the first interface, and executes a response corresponding to operation 7.
- the electronic device 100 may perform the function of the first interface (eg, the first function of the first interface) in response to operation 7 .
- the function of the first interface eg, the first function of the first interface
- the electronic device 100 may perform the function of the first interface (eg, the first function of the first interface) in response to operation 7 .
- S1208 The electronic device 100 determines that the action object of operation 7 is the notification prompt window, and executes a response corresponding to operation 7.
- the electronic device 100 may respond to operation 7 by displaying the interface of the first application, displaying the floating window, or canceling the display of the notification prompt window, and so on.
- S1209 The electronic device 100 determines whether the time period in which the user has not looked at the first area is greater than or equal to the fourth time threshold.
- Step S1209 is an optional step.
- the fact that the user does not focus on the first area means that the user's focus area does not intersect with the first area, that is, the user's focus area is not located in the first area.
- the electronic device 100 executes the following step S1211.
- the electronic device 100 When the time period in which the user does not look at the first area is less than the fourth time threshold, the electronic device 100 performs the following step S1210.
- S1210 The electronic device 100 determines whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.
- the electronic device 100 executes the following step S1211.
- the electronic device 100 executes the above step S1203.
- the electronic device 100 cancels display of the notification prompt window.
- steps S1201 to S1204 and steps S1209 to S1211 shown in FIG. 12 please refer to the relevant content in the embodiment shown in FIG. 11A , and will not be described again here.
- operation 7 when operation 7 does not belong to the gaze operation set, operation 7 can also be written as the fourth operation.
- the fourth operation may also include operation 6 in the embodiment shown in FIG. 9A.
- System 10 may include electronic device 100 and electronic device 200 .
- the specific process of the system 10 executing the notification viewing method may include the following steps:
- the electronic device 100 When the electronic device 100 displays the first interface, the electronic device 100 obtains the first notification of the first application and displays a notification prompt window in the first area of the touch screen.
- S1302 The electronic device 100 sends request 1 to the electronic device 200, requesting the electronic device 200 to send gaze information.
- Step S1302 is an optional step.
- the electronic device 100 may also send a request 1 to the electronic device 200, requesting the electronic device 200 to send a request to the electronic device 100.
- Send gaze messages In response to the first notification, the electronic device 100 may also send a request 1 to the electronic device 200, requesting the electronic device 200 to send a request to the electronic device 100. Send gaze messages.
- the electronic device 200 obtains the user's gaze information.
- the gaze information includes any one or more of the user's facial image, the user's eye image, eye movement information, and electrooculogram information.
- the electronic device 200 may continuously obtain the gaze information in response to the request 1 sent by the electronic device 100 .
- the electronic device 200 may also continue to obtain the gaze information before receiving the request 1 sent by the electronic device 100 .
- S1304 The electronic device 200 sends the gaze information to the electronic device 100.
- the electronic device 200 may send the gaze information to the electronic device 100 after acquiring the gaze information.
- steps S1303 and S1304 are both continuous steps, that is, during the entire notification viewing process, the electronic device 200 continues to obtain gaze information and continues to send gaze information to the electronic device 100 .
- the electronic device 200 may also send the latest collected gaze information obtained by the electronic device 200 to the electronic device 100 at fixed time intervals.
- the electronic device 100 obtains the positional relationship between the electronic device 100 and the electronic device 200.
- the electronic device 100 can acquire an image of the electronic device 200 through a front-facing camera, and determine the positional relationship between the electronic device 200 and the electronic device 100 based on the image of the electronic device 200 .
- the electronic device 100 may also send request 2 to the electronic device 200 , where the request 2 is used to request the electronic device 200 to send the image information of the electronic device 100 to the electronic device 100 .
- the electronic device 100 After receiving the image information of the electronic device 100 sent by the electronic device 200, the electronic device 100 can determine the positional relationship between the electronic device 100 and the electronic device 200 based on the image information of the electronic device 100.
- the electronic device 100 can also measure the positional relationship between the electronic device 100 and the electronic device 200 based on Bluetooth, ultrasound, wifi, etc.
- the electronic device 100 determines whether the user's gaze area is located in the first area based on the gaze information and the positional relationship between the electronic device 100 and the electronic device 200.
- the specific process for the electronic device 100 to determine whether the user's gaze area is located in the first area may include the following steps:
- the electronic device 100 determines the gaze direction of the user's line of sight in a first coordinate system based on the user's image information.
- the first coordinate system is a coordinate system with the center point of the user's face or eyes as the origin.
- the electronic device 100 determines the position and posture of the user's head relative to the electronic device 200 based on the user's image information.
- the electronic device 100 determines the position and posture of the user's head relative to the electronic device 100 based on the position and posture of the user's head relative to the electronic device 200 and the positional relationship between the electronic device 100 and the electronic device 200 .
- the electronic device 100 determines the gaze of the user's gaze in the second coordinate system based on the gaze direction of the user's gaze in the first coordinate system and the position and posture of the user's head relative to the front camera of the electronic device 100 direction, the second coordinate system is a coordinate system with the center of the touch screen of the electronic device 100 as the origin and the normal vector of the touch screen as the Z axis.
- the electronic device 100 determines the intersection point of the user's line of sight and the touch screen as the first position based on the gaze direction of the user's line of sight in the second coordinate system and the coordinates of the touch screen in the second coordinate system.
- the electronic device 100 determines the user's gaze area based on the first location.
- the electronic device 100 determines whether the user's gaze area is located in the first area.
- gaze estimation algorithms can also be used to determine whether the user's gaze area is located in the first area, or the user can be determined based on eye movement information or electrooculogram information. Whether the gaze area is located in the first area is not limited in this application.
- the electronic device 200 may also determine whether the user's gaze area is located in the first area, and send the determination result to the electronic device 100 .
- the electronic device 200 can determine the intersection point of the user's line of sight and the touch screen of the electronic device 100 based on the user's image information and the image information of the electronic device 100, and then determine whether the user's gaze area is located in the first area. Specifically, Reference may be made to the relevant content in the above embodiments.
- the electronic device 200 may determine the user's gaze direction based on the gaze information, and send the user's gaze direction to the electronic device 100 .
- the electronic device 100 may The line direction and the positional relationship of the screen of the electronic device 100 relative to the electronic device 200 determine whether the user's gaze area is located in the first area.
- the electronic device 100 may also determine steps to be performed later based on the relationship between the duration of the user's gaze on the first area and the first time threshold.
- the electronic device 100 when the duration of the user's gaze on the first area is greater than or equal to the first time threshold, the electronic device 100 performs the following step S1307 or step S1308.
- the electronic device 100 executes the following step S1311 or step S1312.
- S1307 The electronic device 100 outputs gaze feedback.
- Step S1307 is an optional step.
- S1308 The electronic device 100 receives the user's operation 1 for an area other than the first area on the electronic device 100.
- the electronic device 100 moves the display position of the notification prompt window downward in the first interface.
- Step S1309 is an optional step.
- the electronic device 100 displays a floating window.
- the floating window is used to display the interface of the first application.
- the interface of the first application includes the specific content of the first notification.
- the electronic device 100 determines whether the time period in which the user has not looked at the first area is greater than or equal to the fourth time threshold.
- Step S1311 is an optional step.
- the fact that the user does not focus on the first area means that the user's focus area does not intersect with the first area, that is, the user's focus area is not located in the first area.
- the electronic device 100 executes the following step S1313.
- the electronic device 100 executes the following step S1312.
- S1312 The electronic device 100 determines whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.
- the electronic device 100 executes the following step S1313.
- the electronic device 100 executes the above step S1306.
- the electronic device 100 cancels display of the notification prompt window.
- step S1307 to step S1313 shown in Figure 13 reference can be made to the relevant content in the above embodiment shown in Figure 11A, and will not be described again here.
- FIG. 11A, FIG. 12 and FIG. 13 only illustrate the notification viewing method by taking the electronic device 100 receiving the first notification of the first application as an example.
- the electronic device 100 receives the first notification of the first application as an example.
- the device 100 may also receive other notifications from the first application (for example, the second notification) or notifications from other applications (for example, the third notification from the second application), and use any one of the above notification viewing methods to view the notification, or Cancel the notification prompt window, etc.
- This application is not limited here.
- the notification prompt window may be a first notification prompt window, a second notification prompt window, a third notification prompt window, etc.
- the floating window may refer to the first floating window. It can also be a second floating window or a third floating window, and so on.
- the electronic device 100 may also be referred to as a first electronic device, and the electronic device 200 may also be referred to as a second electronic device.
- the electronic device 100 may include a notification prompt module 1401, a display module 1402, a gaze information acquisition module 1403, a line of sight estimation module 1404, and a gesture recognition module 1405.
- the notification prompt module 1401 can receive the first notification and send a notification reminder to the display module 1402 and the gaze information acquisition module 1403.
- the display module 1402 may display the notification prompt window in response to the notification reminder sent by the notification prompt module 1401.
- the display module 1402 can also display or close gaze feedback based on the gaze estimation result sent by the gaze estimation module 1404, or close the notification prompt window, and so on.
- the display module 1402 can output gaze feedback and change the background color of the interface or the color of the notification prompt window.
- the display module 1402 can cancel the display of the notification prompt window, and so on.
- the display module 1402 may also execute a corresponding response based on the operation recognition result (eg, operation 1, operation 2, etc.) sent by the gesture recognition module 1405.
- the gaze information acquisition module 1403 can acquire gaze information, such as the user's image information, eye movement information, or electrooculogram information.
- the gaze information acquisition module 1403 may respond to the notification reminder sent by the notification reminder module 1401 and continue to obtain gaze information.
- the gaze information acquisition module 1403 may also send gaze information to the gaze estimation module 1404.
- the gaze information acquisition module 1403 may also acquire image information of the electronic device 100 or image information of the electronic device 200 .
- the gaze estimation module 1404 may determine whether the user's gaze area is located in the first area based on the gaze information. In some embodiments, the gaze estimation module 1404 may also be used to determine whether the time the user looks at the first area is not less than a first time threshold. In other embodiments, the gaze estimation module 1404 can also be used to determine whether the time when the user's gaze leaves the first area is not less than a third time threshold, and can also be used to determine whether the time when the user does not look at the first area is not less than a fourth time threshold. time threshold, etc.
- the gaze estimation module 1404 may send the above gaze estimation results (for example, the user gazes at the first area, the user's gaze area is not located in the first area, the user gazes at the first area for longer than the first time threshold, etc.) to the display module 1402 .
- the gesture recognition module 1405 can receive and recognize the user's operation. For example, the gesture recognition module 1405 can receive and recognize the user's operation 1, and send the operation recognition result to the display module 1402, instructing the display module 1402 to perform a corresponding response.
- the functional module diagram of the system 10 shown in FIG. 2B may also refer to the functional module diagram of the electronic device 100 shown in FIG. 14 .
- the above-mentioned gaze information acquisition module 1403 in FIG. 14 may be located in the electronic device 200 , and the remaining functional modules are located in the electronic device 100 .
- gaze estimation module 1404 may also be located in electronic device 200.
- the functional module structure of the electronic device 100 shown in FIG. 14 is just an example.
- the electronic device 100 may also include more or fewer functional modules than shown in FIG. 14 , which is not limited in this application. .
- the computer program product includes one or more computer instructions.
- the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
- the computer instructions may be stored in or transmitted from one computer-readable storage medium to another, e.g., the computer instructions may be transferred from a website, computer, server, or data center Transmission to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) means.
- the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more available media integrated.
- the available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state disk (SSD)), etc.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Dermatology (AREA)
- Neurosurgery (AREA)
- Neurology (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Disclosed in the present application are a notification checking method and system, and a related apparatus. When an electronic device displays a notification prompt window in a first area of a touch-control screen, the electronic device can acquire any one of a facial image, an eye image, eye movement information and electrooculogram information of a user, and according to same, determines whether a gaze area of the user is the first area. When the gaze area of the user is located in the first area, the electronic device can receive and respond to an operation of the user for an area on the touch-control screen other than the first area, and the electronic device checks the content of a notification or cancels the display of the notification prompt window. In this way, when a user holds an electronic device in a single hand, the user can also conveniently check the specific content of a notification.
Description
本申请要求于2022年08月18日提交中国专利局、申请号为202210995037.7、申请名称为“一种通知查看方法、系统及相关装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims priority to the Chinese patent application submitted to the China Patent Office on August 18, 2022, with application number 202210995037.7 and the application title "A notification viewing method, system and related devices", the entire content of which is incorporated by reference. in this application.
本申请涉及终端技术领域,尤其涉及一种通知查看方法、系统及相关装置。The present application relates to the field of terminal technology, and in particular, to a notification viewing method, system and related devices.
随着终端技术的不断发展,智能手机、平板电脑等电子设备与日常生活的联系越发紧密,人们常常通过电子设备互相发送信息,实现高效沟通。With the continuous development of terminal technology, electronic devices such as smartphones and tablets have become more and more closely connected with daily life. People often send information to each other through electronic devices to achieve efficient communication.
电子设备在接收到信息或者其他应用软件的通知时,会在顶部显示通知提示窗。无论用户需要查看通知还是要快速关闭通知提示窗,都需要针对电子设备顶部的通知提示窗进行操作。电子设备在接收到用户针对通知提示窗的操作之后,才会根据用户的操作,显示通知的具体内容或将关闭通知提示窗。When an electronic device receives information or notifications from other applications, a notification prompt window will be displayed at the top. Whether the user needs to view a notification or quickly close the notification prompt window, he or she needs to operate the notification prompt window on the top of the electronic device. After receiving the user's operation on the notification prompt window, the electronic device will display the specific content of the notification or close the notification prompt window according to the user's operation.
但是,采用上述方法,在用户单手持握电子设备的情况下,用户不方便单手操作顶部的通知提示窗。However, using the above method, when the user holds the electronic device with one hand, it is inconvenient for the user to operate the notification prompt window on the top with one hand.
发明内容Contents of the invention
本申请提供了一种通知查看方法、系统及相关装置。电子设备在触控屏的第一区域显示通知提示窗的情况下,电子设备可以获取用户的面部图像、眼部图像、眼动信息和眼电信息中的任一项,并据此确定出用户注视区域是否为第一区域。当用户的注视区域位于第一区域时,电子设备可以接收并响应于用户针对触控屏上第一区域以外区域的操作,查看通知的内容或取消显示通知提示窗。实现了在用户单手持握电子设备的情况下,通知内容的便捷查看。This application provides a notification viewing method, system and related devices. When the electronic device displays a notification prompt window in the first area of the touch screen, the electronic device can obtain any of the user's facial image, eye image, eye movement information, and electrooculogram information, and determine the user accordingly. Whether the gaze area is the first area. When the user's gaze area is located in the first area, the electronic device can receive and respond to the user's operation on an area other than the first area on the touch screen, view the content of the notification or cancel the display of the notification prompt window. This enables users to conveniently view notification content when holding the electronic device with one hand.
第一方面,本申请提供了一种通知查看方法,包括:第一电子设备显示第一界面;第一电子设备获取第一应用的第一通知;响应于第一通知,第一电子设备在触控屏的第一区域显示第一通知提示窗,第一通知提示窗中包括第一通知的部分内容或全部内容;在第一电子设备确定用户的注视区域位于第一区域之后,第一电子设备接收用户针对第二区域的第一操作,第二区域为触控屏上第一区域以外的区域;在第一电子设备检测到第一操作之后,第一电子设备在第一界面上覆盖显示第一悬浮窗,第一悬浮窗中包括第一应用的界面,第一应用的界面中包括第一通知的内容。In a first aspect, this application provides a notification viewing method, including: a first electronic device displays a first interface; the first electronic device obtains a first notification of a first application; in response to the first notification, the first electronic device touches The first area of the control screen displays a first notification prompt window, and the first notification prompt window includes part or all of the content of the first notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device Receive the user's first operation on the second area, which is an area on the touch screen other than the first area; after the first electronic device detects the first operation, the first electronic device overlays and displays the first operation on the first interface. A floating window, the first floating window includes an interface of the first application, and the interface of the first application includes the content of the first notification.
这样,在用户单手持握电子设备的情况下,用户可以方便地查看通知的具体内容。In this way, when the user holds the electronic device with one hand, the user can conveniently view the specific content of the notification.
在一种可能的实现方式中,在第一电子设备确定用户的注视区域位于第一区域之后,方法还包括:第一电子设备输出注视反馈,注视反馈用于提示用户的注视区域位于第一区域。In a possible implementation, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes: the first electronic device outputs gaze feedback, and the gaze feedback is used to prompt that the user's gaze area is located in the first region. .
在一种可能的实现方式中,输出注视反馈包括以下任一项或多项:改变第一界面的显示颜色、改变第一通知提示窗的显示颜色、改变第一通知提示窗中控件的颜色、点亮第一电子设备的指示灯、振动和播放提示音。In a possible implementation, outputting gaze feedback includes any one or more of the following: changing the display color of the first interface, changing the display color of the first notification prompt window, changing the color of the controls in the first notification prompt window, Light up the indicator light, vibrate and play a tone of the first electronic device.
在一种可能的实现方式中,第一电子设备输出注视反馈,具体包括:当第一电子设备检测到用户注视第一区域的时长大于第一时长时,第一电子设备输出注视反馈。In a possible implementation, the first electronic device outputs gaze feedback, which specifically includes: when the first electronic device detects that the user gazes at the first area for longer than the first duration, the first electronic device outputs gaze feedback.
这样,第一电子设备可以通过注视反馈提示用户的注视区域位于第一区域。In this way, the first electronic device can prompt the user's gaze area to be located in the first area through gaze feedback.
在一种可能的实现方式中,第一操作为滑动操作;在第一电子设备接收用户针对第二区域的第一操作之后,方法还包括:当第一操作在触控屏上的移动距离大于第一移动距离时,第一电子设备移动第一通知提示窗的显示位置;当第一操作在触控屏上的移动距离大于第二移动距离时,第一电子设备将第一通知提示窗展开成第一悬浮窗;其中,第二移动距离大于第一移动距离。In a possible implementation, the first operation is a sliding operation; after the first electronic device receives the user's first operation on the second area, the method further includes: when the movement distance of the first operation on the touch screen is greater than When the first movement distance is reached, the first electronic device moves the display position of the first notification prompt window; when the movement distance of the first operation on the touch screen is greater than the second movement distance, the first electronic device expands the first notification prompt window. to form a first floating window; wherein the second movement distance is greater than the first movement distance.
在另一种可能的实现方式中,第一操作为滑动操作;在第一电子设备接收用户针对第二区域的第一操作之后,方法还包括:当第一操作在触控屏上的移动距离大于第一移动距离时,第一电子设备移动第一通知提示窗的显示位置;当第一操作在触控屏上的移动距离大于第二移动距离时,第一电子设备显示第一悬浮窗,且取消显示第一通知提示窗,第二移动距离大于第一移动距离。
In another possible implementation, the first operation is a sliding operation; after the first electronic device receives the user's first operation on the second area, the method further includes: when the movement distance of the first operation on the touch screen When the first movement distance is greater than the first movement distance, the first electronic device moves the display position of the first notification prompt window; when the movement distance of the first operation on the touch screen is greater than the second movement distance, the first electronic device displays the first floating window, And the display of the first notification prompt window is cancelled, and the second moving distance is greater than the first moving distance.
这样,第一电子设备可以基于第一操作在触控屏上的移动距离控制第一通知提示窗执行不同的功能。In this way, the first electronic device can control the first notification prompt window to perform different functions based on the movement distance of the first operation on the touch screen.
在一种可能的实现方式中,第一操作为滑动操作;在第一电子设备接收用户针对第二区域的第一操作之后,方法还包括:当第一操作的持续时间大于第二时长时,第一电子设备移动第一通知提示窗的显示位置;当第一操作的持续时间大于第三时长时,第一电子设备将第一通知提示窗展开成第一悬浮窗,第三时长大于第二时长。In a possible implementation, the first operation is a sliding operation; after the first electronic device receives the user's first operation for the second area, the method further includes: when the duration of the first operation is greater than the second duration, The first electronic device moves the display position of the first notification prompt window; when the duration of the first operation is greater than the third duration, the first electronic device expands the first notification prompt window into a first floating window, and the third duration is longer than the second duration. duration.
在另一种可能的实现方式中,第一操作为滑动操作;在第一电子设备接收用户针对第二区域的第一操作之后,方法还包括:当第一操作的持续时间大于第二时长时,第一电子设备移动第一通知提示窗的显示位置;当第一操作的持续时间大于第三时长时,第一电子设备显示第一悬浮窗,且取消显示第一通知提示窗,第三时长大于第二时长。In another possible implementation, the first operation is a sliding operation; after the first electronic device receives the user's first operation on the second area, the method further includes: when the duration of the first operation is greater than the second duration , the first electronic device moves the display position of the first notification prompt window; when the duration of the first operation is greater than the third duration, the first electronic device displays the first floating window, and cancels the display of the first notification prompt window, and the third duration greater than the second duration.
这样,第一电子设备可以基于第一操作的持续时长控制第一通知提示窗执行不同的功能。In this way, the first electronic device can control the first notification prompt window to perform different functions based on the duration of the first operation.
在一种可能的实现方式中,方法还包括:在第一电子设备确定用户的注视区域位于第一区域之后,第一电子设备接收用户针对第二区域的第二操作;响应于第二操作,第一电子设备显示第一应用的界面。In a possible implementation, the method further includes: after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives a second operation from the user for the second area; in response to the second operation, The first electronic device displays an interface of the first application.
在一种可能的实现方式中,方法还包括:第一电子设备获取第一应用的第二通知;响应于第二通知,第一电子设备在触控屏的第一区域显示第二通知提示窗,第二通知提示窗中包括第二通知的部分内容或全部内容;在第一电子设备确定用户的注视区域位于第一区域之后,第一电子设备接收用户针对第二区域的第二操作;响应于第二操作,第一电子设备显示第一应用的界面,第一应用的界面中包括第二通知的内容。In a possible implementation, the method further includes: the first electronic device obtains a second notification of the first application; in response to the second notification, the first electronic device displays a second notification prompt window in the first area of the touch screen , the second notification prompt window includes part or all of the content of the second notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's second operation for the second area; responds In the second operation, the first electronic device displays the interface of the first application, and the interface of the first application includes the content of the second notification.
在一种可能的实现方式中,方法还包括:第一电子设备获取第二应用的第三通知;响应于第三通知,第一电子设备在触控屏的第一区域显示第三通知提示窗,第三通知提示窗中包括第三通知的部分内容或全部内容;在第一电子设备确定用户的注视区域位于第一区域之后,第一电子设备接收用户针对第二区域的第二操作;响应于第二操作,第一电子设备显示第二应用的界面,第二应用的界面中包括第三通知的内容。In a possible implementation, the method further includes: the first electronic device obtains a third notification from the second application; in response to the third notification, the first electronic device displays a third notification prompt window in the first area of the touch screen , the third notification prompt window includes part or all of the content of the third notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's second operation for the second area; responds In the second operation, the first electronic device displays the interface of the second application, and the interface of the second application includes the content of the third notification.
在一种可能的实现方式中,方法还包括:在第一电子设备确定用户的注视区域位于第一区域之后,第一电子设备接收用户针对第二区域的第三操作;响应于第三操作,第一电子设备取消显示第一通知提示窗。In a possible implementation, the method further includes: after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives a third operation from the user for the second area; in response to the third operation, The first electronic device cancels display of the first notification prompt window.
在一种可能的实现方式中,方法还包括:第一电子设备获取第一应用的第二通知;响应于第二通知,第一电子设备在触控屏的第一区域显示第二通知提示窗,第二通知提示窗中包括第二通知的部分内容或全部内容;在第一电子设备确定用户的注视区域位于第一区域之后,第一电子设备接收用户针对第二区域的第三操作;响应于第三操作,第一电子设备取消显示第二通知提示窗。In a possible implementation, the method further includes: the first electronic device obtains a second notification of the first application; in response to the second notification, the first electronic device displays a second notification prompt window in the first area of the touch screen , the second notification prompt window includes part or all of the content of the second notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's third operation for the second area; responds In the third operation, the first electronic device cancels display of the second notification prompt window.
在一种可能的实现方式中,方法还包括:第一电子设备获取第二应用的第三通知;响应于第三通知,第一电子设备在触控屏的第一区域显示第三通知提示窗,第三通知提示窗中包括第三通知的部分内容或全部内容;在第一电子设备确定用户的注视区域位于第一区域之后,第一电子设备接收用户针对第二区域的第三操作;响应于第三操作,第一电子设备取消显示第三通知提示窗。In a possible implementation, the method further includes: the first electronic device obtains a third notification from the second application; in response to the third notification, the first electronic device displays a third notification prompt window in the first area of the touch screen , the third notification prompt window includes part or all of the content of the third notification; after the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's third operation for the second area; responds In the third operation, the first electronic device cancels display of the third notification prompt window.
在一种可能的实现方式中,在第一电子设备取消显示第一通知提示窗之后,方法还包括:第一电子设备在经过第四时长之后重新显示第一通知提示窗。In a possible implementation, after the first electronic device cancels displaying the first notification prompt window, the method further includes: the first electronic device redisplays the first notification prompt window after a fourth period of time.
在一种可能的实现方式中,在第一电子设备取消显示第一通知提示窗之后,方法还包括:第一电子设备显示悬浮球;第一电子设备接收用户针对悬浮球的输入;响应于用户针对悬浮球的输入,第一电子设备显示第一通知提示窗或第一悬浮窗。In a possible implementation, after the first electronic device cancels display of the first notification prompt window, the method further includes: the first electronic device displays the floating ball; the first electronic device receives the user's input for the floating ball; and responds to the user In response to input from the floating ball, the first electronic device displays a first notification prompt window or a first floating window.
在一种可能的实现方式中,在第一电子设备确定用户的注视区域位于第一区域之后,方法还包括:当第一电子设备接收用户针对第二区域的第四操作时,第一电子设备执行第一界面对应的第一功能,第四操作与第一操作不同。In a possible implementation, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes: when the first electronic device receives the user's fourth operation for the second area, the first electronic device The first function corresponding to the first interface is executed, and the fourth operation is different from the first operation.
在一种可能的实现方式中,在第一电子设备确定用户的注视区域位于第一区域之后,方法还包括:当第一电子设备确定用户的注视区域变为第二区域,且第一通知提示窗的显示时长大于第五时长时,第一电子设备取消显示第一通知提示窗。
In a possible implementation, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes: when the first electronic device determines that the user's gaze area changes to the second area, and the first notification prompts When the display duration of the window is longer than the fifth duration, the first electronic device cancels display of the first notification prompt window.
在一种可能的实现方式中,在第一电子设备确定用户的注视区域位于第一区域之后,方法还包括:第一电子设备显示操作提示,操作提示用于提示用户在第二区域对第一通知提示窗进行控制。In a possible implementation, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes: the first electronic device displays an operation prompt, and the operation prompt is used to prompt the user to perform an operation on the first area in the second area. Notification prompt window to control.
在一种可能的实现方式中,第一区域为触控屏的顶部区域。In a possible implementation, the first area is a top area of the touch screen.
在一种可能的实现方式中,第二区域为触控屏的底部区域。在另一种可能的实现方式中,第二区域也可以为触控屏的中部区域。In a possible implementation, the second area is a bottom area of the touch screen. In another possible implementation, the second area may also be the middle area of the touch screen.
在一种可能的实现方式中,方法还包括:第一电子设备获取注视信息,注视信息包括以下任一项或多项:用户的眼部图像、用户的面部图像、眼动信息和眼电信息;第一电子设备基于注视信息确定用户的注视区域位于第一区域。In a possible implementation, the method further includes: the first electronic device obtains gaze information, and the gaze information includes any one or more of the following: the user's eye image, the user's facial image, eye movement information, and electrooculogram information. ; The first electronic device determines that the user's gaze area is located in the first area based on the gaze information.
在一种可能的实现方式中,第一电子设备获取注视信息,具体包括:第一电子设备接收第二电子设备发送的注视信息。In a possible implementation, the first electronic device obtains the gaze information, which specifically includes: the first electronic device receives the gaze information sent by the second electronic device.
在一种可能的实现方式中,方法还包括:第一电子设备接收第二电子设备发送的视线估计结果,视线估计结果包括用户的注视区域位于第一区域;第一电子设备基于视线估计结果确定用户的注视区域位于第一区域。In a possible implementation, the method further includes: the first electronic device receives a line of sight estimation result sent by the second electronic device, the line of sight estimation result includes that the user's gaze area is located in the first area; the first electronic device determines based on the line of sight estimation result The user's gaze area is located in the first area.
在一种可能的实现方式中,方法还包括:第一电子设备接收第二电子设备发送的用户的视线方向;第一电子设备基于用户的视线方向确定用户的注视区域位于第一区域。In a possible implementation, the method further includes: the first electronic device receiving the user's gaze direction sent by the second electronic device; and the first electronic device determining that the user's gaze area is located in the first area based on the user's gaze direction.
第二方面,本申请提供了一种电子设备,为第一电子设备,包括:一个或多个处理器和一个或多个存储器。该一个或多个存储器与一个或多个处理器耦合,一个或多个存储器用于存储计算机程序代码,计算机程序代码包括计算机指令,当一个或多个处理器执行计算机指令时,使得第一电子设备执行上述任一方面任一项可能的实现方式中的通知查看方法。In a second aspect, this application provides an electronic device, which is a first electronic device including: one or more processors and one or more memories. The one or more memories are coupled to one or more processors, and the one or more memories are used to store computer program codes. The computer program codes include computer instructions. When the one or more processors execute the computer instructions, the first electronic The device performs the notification viewing method in any of the possible implementations of any of the above aspects.
第三方面,本申请提供了本申请实施例提供了一种计算机存储介质,包括计算机指令,当计算机指令在电子设备上运行时,使得电子设备执行上述任一方面任一项可能的实现方式中的通知查看方法。In a third aspect, the present application provides a computer storage medium that includes computer instructions. When the computer instructions are run on an electronic device, the electronic device causes the electronic device to execute any of the above aspects in a possible implementation manner. How to view notifications.
第四方面,本申请提供了一种计算机程序产品,当计算机程序产品在计算机上运行时,使得计算机执行上述任一方面任一项可能的实现方式中的通知查看方法。In a fourth aspect, this application provides a computer program product. When the computer program product is run on a computer, it causes the computer to execute the notification viewing method in any of the possible implementations of any of the above aspects.
图1为本申请实施例提供的单手持握电子设备的情况下单手操作范围的示意图;Figure 1 is a schematic diagram of the one-hand operation range when an electronic device is held with one hand according to an embodiment of the present application;
图2A为本申请实施例提供的一种电子设备100的形态示意图;FIG. 2A is a schematic diagram of an electronic device 100 provided by an embodiment of the present application;
图2B为本申请实施例提供的一种系统10是结构示意图;Figure 2B is a schematic structural diagram of a system 10 provided by an embodiment of the present application;
图3为本申请实施例提供的一种电子设备100的结构示意图;Figure 3 is a schematic structural diagram of an electronic device 100 provided by an embodiment of the present application;
图4为本申请实施例提供的一种电子设备200的结构示意图;Figure 4 is a schematic structural diagram of an electronic device 200 provided by an embodiment of the present application;
图5A-图5H为本申请实施例提供的一种通知查看方法的界面示意图;Figures 5A-5H are schematic interface diagrams of a notification viewing method provided by an embodiment of the present application;
图6A-图6B为本申请实施例提供的另一种通知查看方法的界面示意图;Figures 6A-6B are schematic interface diagrams of another notification viewing method provided by an embodiment of the present application;
图7A-图7D为本申请实施例提供的一种取消显示通知提示窗的方法的界面示意图;Figures 7A-7D are schematic interface diagrams of a method for canceling the display of a notification prompt window provided by an embodiment of the present application;
图8A-图8C为本申请实施例提供的另一种取消显示通知提示窗的方法的界面示意图;Figures 8A-8C are schematic interface diagrams of another method for canceling the display of a notification prompt window provided by an embodiment of the present application;
图9A-图9B为本申请实施例提供的一种电子设备响应于非注视操作集合中的操作的界面示意图;9A-9B are schematic interface diagrams of an electronic device responding to operations in a non-gaze operation set according to an embodiment of the present application;
图10为本申请实施例提供的另一种电子设备输出注视反馈的界面示意图;Figure 10 is a schematic interface diagram of another electronic device outputting gaze feedback provided by an embodiment of the present application;
图11A为本申请实施例提供的一种通知查看方法的流程示意图;Figure 11A is a schematic flowchart of a notification viewing method provided by an embodiment of the present application;
图11B-图11D为本申请实施例提供的注视区域位于第一区域的三种不同的情况;Figures 11B to 11D show three different situations in which the gaze area is located in the first area provided by the embodiment of the present application;
图12为本申请实施例提供的一种电子设备基于接收的操作与注视操作集合的关系执行该操作对应响应的方法流程示意图;Figure 12 is a schematic flowchart of a method for an electronic device according to an embodiment of the present application to execute a response corresponding to the operation based on the relationship between the received operation and the set of gaze operations;
图13为本申请实施例提供的另一种通知查看方法的流程示意图;
Figure 13 is a schematic flowchart of another notification viewing method provided by an embodiment of the present application;
图14为本申请实施例提供的一种电子设备100的功能模块示意图。FIG. 14 is a schematic diagram of functional modules of an electronic device 100 provided by an embodiment of the present application.
下面将结合附图对本申请实施例中的技术方案进行清楚、详尽地描述。其中,在本申请实施例的描述中,除非另有说明,“/”表示或的意思,例如,A/B可以表示A或B;文本中的“和/或”仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况,另外,在本申请实施例的描述中,“多个”是指两个或多于两个。The technical solutions in the embodiments of the present application will be described clearly and in detail below with reference to the accompanying drawings. Among them, in the description of the embodiments of this application, unless otherwise stated, "/" means or, for example, A/B can mean A or B; "and/or" in the text is only a way to describe related objects. The association relationship means that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiment of the present application , "plurality" means two or more than two.
以下,术语“第一”、“第二”仅用于描述目的,而不能理解为暗示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征,在本申请实施例的描述中,除非另有说明,“多个”的含义是两个或两个以上。Hereinafter, the terms “first” and “second” are used for descriptive purposes only and shall not be understood as implying or implying relative importance or implicitly specifying the quantity of indicated technical features. Therefore, the features defined as “first” and “second” may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, “plurality” The meaning is two or more.
本申请以下实施例中的术语“用户界面(user interface,UI)”,是应用程序或操作系统与用户之间进行交互和信息交换的介质接口,它实现信息的内部形式与用户可以接受形式之间的转换。用户界面是通过java、可扩展标记语言(extensible markup language,XML)等特定计算机语言编写的源代码,界面源代码在电子设备上经过解析,渲染,最终呈现为用户可以识别的内容。用户界面常用的表现形式是图形用户界面(graphic user interface,GUI),是指采用图形方式显示的与计算机操作相关的用户界面。它可以是在电子设备的显示屏中显示的文本、图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、Widget等可视的界面元素。The term "user interface (UI)" in the following embodiments of this application is a media interface for interaction and information exchange between an application or operating system and a user. It realizes the difference between the internal form of information and the form acceptable to the user. conversion between. The user interface is source code written in specific computer languages such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed graphically. It can be text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets and other visual interface elements displayed on the display screen of an electronic device.
图1示出了单手持握电子设备的情况下,单手操作范围的示意图。Figure 1 shows a schematic diagram of the one-hand operation range when the electronic device is held with one hand.
如图1所示,在用户单手持握电子设备的情况下,根据用户单手操作的难易程度,可以将电子设备的触控屏划分为以下三个区域:区域1001、区域1002和区域1003。其中,区域1001为用户单手持握状态下操作困难的区域,区域1003为用户单手持握状态下操作不便的区域,区域1002为用户单手持握状态下操作便利的区域。As shown in Figure 1, when the user holds the electronic device with one hand, the touch screen of the electronic device can be divided into the following three areas according to the difficulty of the user's one-hand operation: area 1001, area 1002 and area 1003 . Among them, area 1001 is an area that is difficult to operate when the user holds it with one hand, area 1003 is an area that is inconvenient to operate when the user holds it with one hand, and area 1002 is an area that is easy to operate when the user holds it with one hand.
当电子设备获取到新的通知时,电子设备会在触控屏的顶部显示通知提示窗,通知提示窗的显示区域位于图1所示的区域1001中。因此,在用户单手持握电子设备的情况下,用户不方便通过单手实现对通知提示窗的操作。When the electronic device obtains a new notification, the electronic device displays a notification prompt window on the top of the touch screen. The display area of the notification prompt window is located in the area 1001 shown in Figure 1 . Therefore, when the user holds the electronic device with one hand, it is inconvenient for the user to operate the notification prompt window with one hand.
本申请实施例提供的通知查看方法的执行主体可以是电子设备100。The execution subject of the notification viewing method provided by the embodiment of the present application may be the electronic device 100 .
图2A示出了本申请实施例提供的一种电子设备100的形态示意图。FIG. 2A shows a schematic diagram of an electronic device 100 provided by an embodiment of the present application.
如图2A所示,电子设备100包括前置摄像头101,前置摄像头101可用于获取用户的面部或眼部图像,面部或眼部图像可用于进行用户的视线估计,确定用户的视线与电子设备100触控屏的交点。在电子设备100显示顶部通知提示窗的情况下,电子设备100可以基于视线估计的结果以及用户针对触控屏的操作,查看通知提示窗对应的通知内容或关闭通知提示窗。As shown in Figure 2A, the electronic device 100 includes a front camera 101. The front camera 101 can be used to obtain the user's face or eye image. The face or eye image can be used to estimate the user's line of sight and determine the relationship between the user's line of sight and the electronic device. 100 touch screen intersection. When the electronic device 100 displays the top notification prompt window, the electronic device 100 can view the notification content corresponding to the notification prompt window or close the notification prompt window based on the result of the line of sight estimation and the user's operation on the touch screen.
需要说明的是,电子设备100可以是如图2A所示的手机,也可以是平板电脑等其他电子设备。It should be noted that the electronic device 100 may be a mobile phone as shown in FIG. 2A or other electronic devices such as a tablet computer.
本申请实施例还提供了一种可以执行通知查看方法的系统。下面介绍本申请实施例提供的一种系统的架构。The embodiment of the present application also provides a system that can execute a notification viewing method. The following introduces the architecture of a system provided by embodiments of this application.
如图2B所示,系统10可以包括电子设备100和电子设备200。As shown in FIG. 2B , system 10 may include electronic device 100 and electronic device 200 .
其中,电子设备100可以是如图2B所示的手机,也可以是平板电脑,等等。电子设备200可以是如图2B所示的智能眼镜,也可以是增强现实(augmented reality,AR)设备、虚拟现实(virtual reality,VR)设备,等等。The electronic device 100 may be a mobile phone as shown in FIG. 2B , a tablet computer, or the like. The electronic device 200 may be smart glasses as shown in Figure 2B, or it may be an augmented reality (Augmented Reality, AR) device, a Virtual Reality (VR) device, or the like.
电子设备200可用于获取用户的面部或眼部的图像。电子设备200可以将用户的面部或眼部图像发送给电子设备100,用于电子设备100进行用户的视线估计。在一些实施例中,电子设备200也可以获取电子设备100与电子设备200的位置关系,并基于用户的面部或眼部的图像以及电子设备100与电子设备200的位置关系进行用户的视线估计,确定用户视线与电子设备100触控屏的交点,并将视线估计的结果发送至电子设备100。电子设备100可以基于视线估计的结果以及用户针对触控屏的操作,查看通知提示窗对应的通知内容或关闭通知提示窗。Electronic device 200 may be used to obtain images of the user's face or eyes. The electronic device 200 may send the user's face or eye image to the electronic device 100 for the electronic device 100 to estimate the user's gaze. In some embodiments, the electronic device 200 can also obtain the positional relationship between the electronic device 100 and the electronic device 200, and estimate the user's line of sight based on the image of the user's face or eyes and the positional relationship between the electronic device 100 and the electronic device 200. The intersection point of the user's line of sight and the touch screen of the electronic device 100 is determined, and the result of the line of sight estimation is sent to the electronic device 100 . The electronic device 100 can view the notification content corresponding to the notification prompt window or close the notification prompt window based on the result of the line of sight estimation and the user's operation on the touch screen.
电子设备100和电子设备200可以通过蓝牙技术(包括基础速率(basic rate,BR)/增强速率(enhanced data rate,EDR)蓝牙和低功耗蓝牙(bluetooth low energy,BLE))进行通信。
The electronic device 100 and the electronic device 200 may communicate through Bluetooth technology (including basic rate (BR)/enhanced data rate (EDR) Bluetooth and Bluetooth low energy (BLE)).
下面介绍本申请实施例提供的一种基于视线估计和惯性测量单元(Inertial Measurement Unit,IMU)跟踪的通知查看方法。The following describes a notification viewing method based on line of sight estimation and inertial measurement unit (IMU) tracking provided by embodiments of the present application.
电子设备100可以通过前置摄像头获取用户的面部图像,并基于用户的面部图像确定用户注视的区域是否为通知提示窗显示的区域。在确定用户注视区域在通知提示窗显示的区域之内的情况下,电子设备100可以结合IMU测量的电子设备100的运动执行相应的操作,例如,当电子设备100检测到电子设备100在做靠近用户的运动时,电子设备100可以显示悬浮窗,并在悬浮窗中显示通知提示窗对应的通知的具体内容。又例如当电子设备100确定电子设备100随着手做向右甩动的动作时,电子设备100确定忽略通知,将通知提示窗向右移动直至完全消失。又例如,当电子设备100确定电子设备100随着手做向左甩动的动作时,电子设备100可以确定稍后再显示该通知提示窗,并将该通知提示窗向左移动直至完全消失,等等。The electronic device 100 can obtain the user's facial image through the front camera, and determine whether the area the user is looking at is the area displayed by the notification prompt window based on the user's facial image. When it is determined that the user's gaze area is within the area displayed in the notification prompt window, the electronic device 100 may perform corresponding operations in conjunction with the motion of the electronic device 100 measured by the IMU. For example, when the electronic device 100 detects that the electronic device 100 is approaching When the user moves, the electronic device 100 may display a floating window, and display the specific content of the notification corresponding to the notification prompt window in the floating window. For another example, when the electronic device 100 determines that the electronic device 100 swings to the right with the hand, the electronic device 100 determines to ignore the notification and moves the notification prompt window to the right until it disappears completely. For another example, when the electronic device 100 determines that the electronic device 100 swings to the left with the hand, the electronic device 100 may determine to display the notification prompt window later, and move the notification prompt window to the left until it disappears completely, etc. wait.
采用上述方法,用户可以通过单手操作来查看通知的具体内容,或是快速取消显示顶部的通知提示窗,等等。但是,IMU识别动作时存在误判的情况,电子设备100随着用户手部甩动或靠近的动作有时不能被准确识别,从而导致电子设备100无法执行相应的操作。此外,在一些特殊场景中,例如,在用户双手提拿重物的情况下,上述方法中的动作(例如甩动、靠近用户等动作)也不方便用户操作。Using the above method, users can view the specific content of the notification through one-handed operation, or quickly cancel the notification prompt window at the top of the display, etc. However, there may be misjudgments when the IMU recognizes actions. The electronic device 100 may not be accurately recognized as the user's hand swings or approaches, resulting in the electronic device 100 being unable to perform corresponding operations. In addition, in some special scenarios, for example, when the user carries heavy objects with both hands, the actions in the above method (such as swinging, approaching the user, etc.) are also inconvenient for the user to operate.
本申请实施例提供的一种通知查看方法。电子设备100在触控屏的第一区域显示通知提示窗的情况下,电子设备100可以获取注视信息,并据此确定出用户注视区域是否位于第一区域。在确定用户注视区域位于第一区域之后,电子设备100可以接收并响应于用户针对电子设备100上第二区域的操作,查看通知的内容或取消显示通知提示窗等等,第二区域为电子设备100上第一区域以外的区域。这样,在用户单手持握电子设备100的情况下,用户可以方便地对通知提示窗进行操作(例如,查看通知的具体内容或快速取消显示通知提示窗,等等)。An embodiment of the present application provides a notification viewing method. When the electronic device 100 displays a notification prompt window in the first area of the touch screen, the electronic device 100 can obtain the gaze information, and determine whether the user's gaze area is located in the first area based on it. After determining that the user's gaze area is located in the first area, the electronic device 100 may receive and respond to the user's operation on the second area on the electronic device 100, view the content of the notification or cancel the display of the notification prompt window, etc. The second area is the electronic device 100 for areas outside the first area. In this way, when the user holds the electronic device 100 with one hand, the user can conveniently operate the notification prompt window (for example, view the specific content of the notification or quickly cancel the display of the notification prompt window, etc.).
图3示出了电子设备100的结构示意图。FIG. 3 shows a schematic structural diagram of the electronic device 100.
电子设备100可以是手机、平板电脑、桌面型计算机、膝上型计算机、手持计算机、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer,UMPC)、上网本,以及蜂窝电话、个人数字助理(personal digital assistant,PDA)、增强现实(augmented reality,AR)设备、虚拟现实(virtual reality,VR)设备、人工智能(artificial intelligence,AI)设备、可穿戴式设备、车载设备、智能家居设备和/或智慧城市设备,本申请实施例对该电子设备的具体类型不作特殊限制。The electronic device 100 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (personal digital assistant) digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, vehicle-mounted device, smart home device and/or Smart city equipment, the embodiment of this application does not place special restrictions on the specific type of electronic equipment.
电子设备100可以包括处理器110,外部存储器接口120,内部存储器121,通用串行总线(universal serial bus,USB)接口130,充电管理模块140,电源管理模块141,电池142,天线1,天线2,移动通信模块150,无线通信模块160,传感器模块180,按键190,摄像头193,显示屏194,以及用户标识模块(subscriber identification module,SIM)卡接口195等。其中传感器模块180可以包括触摸传感器180K等。The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, sensor module 180, button 190, camera 193, display screen 194, and subscriber identification module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a touch sensor 180K and the like.
可以理解的是,本发明实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中,电子设备100可以包括比图示更多或更少的部件,或者组合某些部件,或者拆分某些部件,或者不同的部件布置。图示的部件可以以硬件,软件或软件和硬件的组合实现。It can be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.
处理器110可以包括一个或多个处理单元,例如:处理器110可以包括应用处理器(application processor,AP),调制解调处理器,图形处理器(graphics processing unit,GPU),图像信号处理器(image signal processor,ISP),控制器,视频编解码器,数字信号处理器(digital signal processor,DSP),基带处理器,和/或神经网络处理器(neural-network processing unit,NPU)等。其中,不同的处理单元可以是独立的器件,也可以集成在一个或多个处理器中。The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors.
控制器可以根据指令操作码和时序信号,产生操作控制信号,完成取指令和执行指令的控制。The controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.
处理器110中还可以设置存储器,用于存储指令和数据。在一些实施例中,处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据,可从所述存储器中直接调用。避免了重复存取,减少了处理器110的等待时间,因而提高了系统的效率。The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have been recently used or recycled by processor 110 . If the processor 110 needs to use the instructions or data again, it can be called directly from the memory. Repeated access is avoided and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.
充电管理模块140用于从充电器接收充电输入。The charging management module 140 is used to receive charging input from the charger.
电源管理模块141用于连接电池142,充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入,为处理器110,内部存储器121,显示屏194,摄像头193,和无线通信模块160等供电。电源管理模块141还可以用于监测电池容量,电池循环次数,电池健康状态(漏电,阻
抗)等参数。在其他一些实施例中,电源管理模块141也可以设置于处理器110中。在另一些实施例中,电源管理模块141和充电管理模块140也可以设置于同一个器件中。The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, resistance resistance) and other parameters. In some other embodiments, the power management module 141 may also be provided in the processor 110 . In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.
电子设备100的无线通信功能可以通过天线1,天线2,移动通信模块150,无线通信模块160,调制解调处理器以及基带处理器等实现。The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
天线1和天线2用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用,以提高天线的利用率。例如:可以将天线1复用为无线局域网的分集天线。在另外一些实施例中,天线可以和调谐开关结合使用。Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example: Antenna 1 can be reused as a diversity antenna for a wireless LAN. In other embodiments, antennas may be used in conjunction with tuning switches.
移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器,开关,功率放大器,低噪声放大器(low noise amplifier,LNA)等。移动通信模块150可以由天线1接收电磁波,并对接收的电磁波进行滤波,放大等处理,传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大,经天线1转为电磁波辐射出去。在一些实施例中,移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中,移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。The mobile communication module 150 can provide solutions for wireless communication including 2G/3G/4G/5G applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, perform filtering, amplification and other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be disposed in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
调制解调处理器可以包括调制器和解调器。其中,调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后,被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A,受话器170B等)输出声音信号,或通过显示屏194显示图像或视频。在一些实施例中,调制解调处理器可以是独立的器件。在另一些实施例中,调制解调处理器可以独立于处理器110,与移动通信模块150或其他功能模块设置在同一个器件中。A modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110 and may be provided in the same device as the mobile communication module 150 or other functional modules.
无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wireless local area networks,WLAN)(如无线保真(wireless fidelity,Wi-Fi)网络),蓝牙(bluetooth,BT),全球导航卫星系统(global navigation satellite system,GNSS),调频(frequency modulation,FM),近距离无线通信技术(near field communication,NFC),红外技术(infrared,IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波,将电磁波信号解调以及滤波处理,将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号,对其进行调频,放大,经天线2转为电磁波辐射出去。The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (bluetooth, BT), and global navigation satellites. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
在一些实施例中,电子设备100的天线1和移动通信模块150耦合,天线2和无线通信模块160耦合,使得电子设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯系统(global system for mobile communications,GSM),通用分组无线服务(general packet radio service,GPRS),码分多址接入(code division multiple access,CDMA),宽带码分多址(wideband code division multiple access,WCDMA),时分码分多址(time-division code division multiple access,TD-SCDMA),长期演进(long term evolution,LTE),BT,GNSS,WLAN,NFC,FM,和/或IR技术等。所述GNSS可以包括全球卫星定位系统(global positioning system,GPS),全球导航卫星系统(global navigation satellite system,GLONASS),北斗卫星导航系统(beidou navigation satellite system,BDS),准天顶卫星系统(quasi-zenith satellite system,QZSS)和/或星基增强系统(satellite based augmentation systems,SBAS)。In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (quasi) -zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).
电子设备100通过GPU,显示屏194,以及应用处理器等实现显示功能。GPU为图像处理的微处理器,连接显示屏194和应用处理器。GPU用于执行数学和几何计算,用于图形渲染。处理器110可包括一个或多个GPU,其执行程序指令以生成或改变显示信息。The electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is an image processing microprocessor and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
显示屏194用于显示图像,视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display,LCD),显示面板还可以采用有机发光二极管(organic light-emitting diode,OLED),有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的,AMOLED),柔性发光二极管(flex light-emitting diode,FLED),Miniled,MicroLed,Micro-oLed,量子点发光二极管(quantum dot light emitting diodes,QLED)等制造。在一些实施例中,电子设备100可以包括1个或N个显示屏194,N为大于1的正整数。The display screen 194 is used to display images, videos, etc. Display 194 includes a display panel. The display panel can use a liquid crystal display (LCD). The display panel can also use an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (OLED). Manufacturing of active-matrix organic light emitting diodes (AMOLED), flexible light-emitting diodes (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
电子设备100可以通过ISP,摄像头193,视频编解码器,GPU,显示屏194以及应用处理器等实现拍摄功能。The electronic device 100 can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.
ISP用于处理摄像头193反馈的数据。例如,拍照时,打开快门,光线通过镜头被传递到摄像头感光元件上,光信号转换为电信号,摄像头感光元件将所述电信号传递给ISP处理,转化为肉眼可见的图像。
ISP还可以对图像的噪点,亮度进行算法优化。ISP还可以对拍摄场景的曝光,色温等参数优化。在一些实施例中,ISP可以设置在摄像头193中。The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera sensor through the lens, the optical signal is converted into an electrical signal, and the camera sensor passes the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise and brightness. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193.
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device,CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor,CMOS)光电晶体管。感光元件把光信号转换成电信号,之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB,YUV等格式的图像信号。在一些实施例中,电子设备100可以包括1个或N个摄像头193,N为大于1的正整数。Camera 193 is used to capture still images or video. The object passes through the lens to produce an optical image that is projected onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other format image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
数字信号处理器用于处理数字信号,除了可以处理数字图像信号,还可以处理其他数字信号。例如,当电子设备100在频点选择时,数字信号处理器用于对频点能量进行傅里叶变换等。Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样,电子设备100可以播放或录制多种编码格式的视频,例如:动态图像专家组(moving picture experts group,MPEG)1,MPEG2,MPEG3,MPEG4等。Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
NPU为神经网络(neural-network,NN)计算处理器,通过借鉴生物神经网络结构,例如借鉴人脑神经元之间传递模式,对输入信息快速处理,还可以不断的自学习。通过NPU可以实现电子设备100的智能认知等应用,例如:图像识别,人脸识别,语音识别,文本理解等。NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can continuously learn by itself. Intelligent cognitive applications of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, etc.
内部存储器121可以包括一个或多个随机存取存储器(random access memory,RAM)和一个或多个非易失性存储器(non-volatile memory,NVM)。The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).
触摸传感器180K,也称“触控器件”。触摸传感器180K可以设置于显示屏194,由触摸传感器180K与显示屏194组成触摸屏,也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器,以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中,触摸传感器180K也可以设置于电子设备100的表面,与显示屏194所处的位置不同。Touch sensor 180K, also known as "touch device". The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is used to detect a touch operation on or near the touch sensor 180K. The touch sensor can pass the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation may be provided through display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 at a location different from that of the display screen 194 .
按键190包括开机键,音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备100可以接收按键输入,产生与电子设备100的用户设置以及功能控制有关的键信号输入。The buttons 190 include a power button, a volume button, etc. Key 190 may be a mechanical key. It can also be a touch button. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .
可选的,电子设备100还可以包括马达191,马达191可以产生振动提示。马达191可以用于来电振动提示,也可以用于触摸振动反馈。例如,作用于不同应用(例如拍照,音频播放等)的触摸操作,可以对应不同的振动反馈效果。作用于显示屏194不同区域的触摸操作,马达191也可对应不同的振动反馈效果。不同的应用场景(例如:时间提醒,接收信息,闹钟,游戏等)也可以对应不同的振动反馈效果。触摸振动反馈效果还可以支持自定义。Optionally, the electronic device 100 may also include a motor 191, and the motor 191 may generate a vibration prompt. The motor 191 can be used for vibration prompts for incoming calls and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. The motor 191 can also correspond to different vibration feedback effects for touch operations in different areas of the display screen 194 . Different application scenarios (such as time reminders, receiving information, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.
可选的,电子设备100还可以包括指示器192,指示器192可以是指示灯,可以用于指示充电状态,电量变化,也可以用于指示消息,未接来电,通知等。Optionally, the electronic device 100 may also include an indicator 192. The indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, or may be used to indicate messages, missed calls, notifications, etc.
图4示出了一种电子设备200的结构示意图。Figure 4 shows a schematic structural diagram of an electronic device 200.
如图4所示,电子设备200可以包括:处理器201,存储器202,蓝牙通信模块203,摄像模块204和电源模块205,可选的,电子设备200还可以包括按键206。其中:As shown in FIG. 4 , the electronic device 200 may include: a processor 201 , a memory 202 , a Bluetooth communication module 203 , a camera module 204 and a power module 205 . Optionally, the electronic device 200 may also include a button 206 . in:
处理器201可用于读取和执行计算机可读指令。具体实现中,处理器201可主要包括控制器、运算器和寄存器。其中,控制器主要负责指令译码,并为指令对应的操作发出控制信号。运算器主要负责保存指令执行过程中临时存放的寄存器操作数和中间操作结果等。具体实现中,处理器201的硬件架构可以是专用集成电路(ASIC)架构、MIPS架构、ARM架构或者NP架构等等。Processor 201 may be used to read and execute computer readable instructions. In specific implementation, the processor 201 may mainly include a controller, arithmetic unit, and registers. Among them, the controller is mainly responsible for decoding instructions and issuing control signals for operations corresponding to the instructions. The arithmetic unit is mainly responsible for saving register operands and intermediate operation results temporarily stored during instruction execution. In specific implementation, the hardware architecture of the processor 201 may be an application specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture, etc.
存储器202与处理器201耦合,用于存储各种软件程序和/或多组指令。具体实现中,存储器202可包括高速随机存取的存储器,在一些实施例中,存储器202也可包括非易失性存储器。存储器202可以存储通信程序,该通信程序可用于与电子设备100或其他设备进行通信。在另一些实施例中,存储器202也可以存储操作系统,例如uCOS,VxWorks、RTLinux等嵌入式操作系统。Memory 202 is coupled to processor 201 for storing various software programs and/or sets of instructions. In specific implementations, the memory 202 may include high-speed random access memory. In some embodiments, the memory 202 may also include non-volatile memory. Memory 202 may store communication programs that may be used to communicate with electronic device 100 or other devices. In other embodiments, the memory 202 can also store operating systems, such as uCOS, VxWorks, RTLinux and other embedded operating systems.
蓝牙通信模块203可用于与电子设备100建立通信连接。蓝牙通信模块203可以包括有蓝牙芯片。电子设备200可以通过该蓝牙芯片与电子设备100的蓝牙芯片之间进行配对并建立蓝牙连接,以通过该蓝牙连接实现电子设备200和电子设备100之间的无线通信和业务处理。通常,蓝牙芯片可以支持BR/EDR蓝牙和BLE,例如可以收/法寻呼(page)信息,收/发BLE广播消息等。在一些实施例中,蓝牙通信模块203还可以包括天线,蓝牙通信模块203经由天线接收电磁波,将电磁波信号调频以及滤波处理,将处理后的
信号发送到处理器201。蓝牙通信模块203还可以从处理器201接收待发送的信号,对其进行调频,放大,经天线转为电磁波辐射出去。The Bluetooth communication module 203 can be used to establish a communication connection with the electronic device 100 . The Bluetooth communication module 203 may include a Bluetooth chip. The electronic device 200 can pair with the Bluetooth chip of the electronic device 100 and establish a Bluetooth connection through the Bluetooth chip, so as to realize wireless communication and business processing between the electronic device 200 and the electronic device 100 through the Bluetooth connection. Usually, the Bluetooth chip can support BR/EDR Bluetooth and BLE, for example, it can receive/method paging (page) information, receive/send BLE broadcast messages, etc. In some embodiments, the Bluetooth communication module 203 may also include an antenna. The Bluetooth communication module 203 receives electromagnetic waves via the antenna, frequency modulates and filters the electromagnetic wave signals, and processes the processed electromagnetic waves. The signal is sent to processor 201. The Bluetooth communication module 203 can also receive the signal to be sent from the processor 201, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna for radiation.
摄像模块204可用于获取用户的眼部和/或面部图像。具体实现中,摄像模块204可以包括1个或N个摄像头,N为大于1的正整数。摄像头用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件或互补金属氧化物半导体光电晶体管。感光元件把光信号转换成电信号,之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB,YUV等格式的图像信号。在一些实施例中,摄像模块204还可用于获取电子设备100的图像。The camera module 204 may be used to obtain images of the user's eyes and/or face. In specific implementation, the camera module 204 may include 1 or N cameras, where N is a positive integer greater than 1. Cameras are used to capture still images or videos. The object passes through the lens to produce an optical image that is projected onto the photosensitive element. The photosensitive element may be a charge coupled device or a complementary metal oxide semiconductor phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other format image signals. In some embodiments, the camera module 204 may also be used to acquire images of the electronic device 100 .
电源模块205,可以用于提供电子设备200的系统电源,为电子设备200各个模块供电;支持电子设备200接收充电输入等。电源模块205可以包括电源管理单元(power management unit,PMU)和电池。其中,电源管理单元可以接收外部的充电输入;将充电电路输入的电信号提供给电池充电,还可以将电池提供的电信号提供给电子设备200中的其他模块,以防止电池过充、过放、短路或过流等。在一些实施例中,电源模块205还可以包括无线充电线圈,用于对电子设备200进行无线充电。另外,电源管理单元还可以用于监测电池容量,电池循环次数,电池健康状态(漏电,阻抗)等参数。The power module 205 can be used to provide system power for the electronic device 200, power each module of the electronic device 200, and support the electronic device 200 to receive charging input, etc. The power module 205 may include a power management unit (PMU) and a battery. Among them, the power management unit can receive external charging input; provide the electrical signal input by the charging circuit to charge the battery; and can also provide the electrical signal provided by the battery to other modules in the electronic device 200 to prevent the battery from overcharging and over-discharging. , short circuit or overcurrent, etc. In some embodiments, the power module 205 may also include a wireless charging coil for wirelessly charging the electronic device 200 . In addition, the power management unit can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, impedance) and other parameters.
可选的,在一些实施例中,电子设备200还可以包括按键和/或传感器(图4中未示出)等等。按键可以是物理按键或触摸按键(与触摸传感器配合使用)等,用于触发开机、关机、暂停、播放、录音、开始配对、重置等操作。触摸传感器可以检测用户的单击、双击、多次点击、长按、重压等触摸操作,还可以进行用户指纹识别,以在安全敏感等业务场景中对用户身份进行鉴权。Optionally, in some embodiments, the electronic device 200 may also include buttons and/or sensors (not shown in FIG. 4 ) and the like. The buttons can be physical buttons or touch buttons (used in conjunction with touch sensors), etc., used to trigger operations such as power on, power off, pause, play, record, start pairing, reset, etc. The touch sensor can detect the user's touch operations such as single click, double click, multiple clicks, long press, heavy pressure, etc. It can also perform user fingerprint recognition to authenticate the user's identity in security-sensitive business scenarios.
可以理解的是,图4所示电子设备200仅是一个范例,并且电子设备200可以具有比图4中所示的更多的或者更少的部件,可以组合两个或多个的部件,或者可以具有不同的部件配置。图中所示出的各种部件可以在包括一个或多个信号处理和/或专用集成电路在内的硬件、软件、或硬件和软件的组合中实现。It is understood that the electronic device 200 shown in FIG. 4 is only an example, and the electronic device 200 may have more or fewer components than shown in FIG. 4 , two or more components may be combined, or Can have different component configurations. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.
下面结合应用场景,具体介绍本申请实施例提供的一种通知查看方法。The following is a detailed introduction to a notification viewing method provided by the embodiment of this application based on application scenarios.
在一些应用场景下,电子设备100可以在显示第一界面(例如,桌面、应用软件中的界面,等等)时,获取到第一应用的第一通知(例如,信息、应用软件的通知,等等)。响应于第一通知,电子设备100可以在触控屏的第一区域显示通知提示窗,该通知提示窗中包括第一通知的部分内容或全部内容。电子设备100可以基于注视信息(例如用户的面部图像、用户的眼部图像、用户的眼电信息、用户的眼动信息等等)确定用户的注视区域。在电子设备100确定用户的注视区域位于第一区域之后,电子设备100可以接收并响应于用户针对电子设备100的第二区域(即电子设备100的触控屏上第一区域以外的区域)的操作1(例如,下滑操作),在第一界面中显示悬浮窗,该悬浮窗用于显示第一应用的界面,该第一应用的界面包括第一通知的具体内容。可选的,在确定用户的注视区域位于第一区域之后(例如,电子设备100检测到用户的注视区域位于第一区域时,或,电子设备100检测到用户注视第一区域的持续时间大于等于第一时间阈值,等等),电子设备100还可以输出注视反馈,注视反馈可以是改变第一界面的显示颜色、改变通知提示窗的颜色或改变通知消息下方横条的颜色,等等。可选的,电子设备100也可以基于操作1在触控屏上移动的距离确定是否在悬浮窗中显示第一应用的界面。In some application scenarios, the electronic device 100 may obtain the first notification of the first application (for example, information, notification of application software, etc.) when displaying the first interface (for example, desktop, interface in application software, etc.). etc). In response to the first notification, the electronic device 100 may display a notification prompt window in the first area of the touch screen, and the notification prompt window includes part or all of the content of the first notification. The electronic device 100 may determine the user's gaze area based on gaze information (eg, the user's facial image, the user's eye image, the user's electrooculogram information, the user's eye movement information, etc.). After the electronic device 100 determines that the user's gaze area is located in the first area, the electronic device 100 may receive and respond to the user's request for a second area of the electronic device 100 (ie, an area other than the first area on the touch screen of the electronic device 100). Operation 1 (for example, sliding operation), display a floating window in the first interface. The floating window is used to display the interface of the first application. The interface of the first application includes the specific content of the first notification. Optionally, after determining that the user's gaze area is located in the first area (for example, the electronic device 100 detects that the user's gaze area is located in the first area, or the electronic device 100 detects that the duration of the user's gaze at the first area is greater than or equal to first time threshold, etc.), the electronic device 100 can also output gaze feedback. The gaze feedback can be changing the display color of the first interface, changing the color of the notification prompt window, or changing the color of the horizontal bar below the notification message, etc. Optionally, the electronic device 100 may also determine whether to display the interface of the first application in the floating window based on the distance moved on the touch screen in operation 1.
示例性的,如图5A所示,电子设备100显示桌面界面500。其中,桌面界面500中显示了一个放置有应用图标的页面,该页面包括多个应用图标(例如,股票应用图标,计算器应用图标,浏览器应用图标,社交应用图标501,等等),该放置有应用图标的页面的下方还显示包括有页面指示符502,以表明桌面界面500上页面总数,以及当前显示的页面与其他页面的位置关系。例如,桌面界面500可以包括有三个页面,该页面指示符中的白点位于左起第三个位置,可以表示当前显示页面为三个页面中左起的第三个页面。可选的,该放置有应用图标的页面的上方还显示包括有状态栏503,该状态栏503中可以包括有通信信号的强度指示符、电量值、时间等信息。进一步可选的,页面指示符的下方可以有托盘(dock)区域504,该dock区域504中可以包括有一个或多个托盘图标(例如,拨号应用图标、信息应用图标、联系人应用图标、相机应用图标等等),该dock区域中的一个或多个托盘图标可以在页面切换时保持显示。Exemplarily, as shown in FIG. 5A , the electronic device 100 displays a desktop interface 500 . The desktop interface 500 displays a page on which application icons are placed. The page includes multiple application icons (for example, a stock application icon, a calculator application icon, a browser application icon, a social application icon 501, etc.). A page indicator 502 is also displayed below the page where the application icon is placed to indicate the total number of pages on the desktop interface 500 and the positional relationship between the currently displayed page and other pages. For example, the desktop interface 500 may include three pages, and the white dot in the page indicator is located at the third position from the left, which may indicate that the currently displayed page is the third page from the left among the three pages. Optionally, a status bar 503 is also displayed above the page where the application icon is placed. The status bar 503 may include information such as a strength indicator of the communication signal, a power value, and time. Further optionally, there may be a dock area 504 below the page indicator, and the dock area 504 may include one or more tray icons (for example, a dialing application icon, a messaging application icon, a contact application icon, a camera application icons, etc.), one or more tray icons in the dock area can remain displayed when switching pages.
电子设备100可以接收并响应于用户针对社交应用图标501的点击操作,打开社交应用软件,显示社交应用界面510。The electronic device 100 may receive and respond to the user's click operation on the social application icon 501, open the social application software, and display the social application interface 510.
如图5B所示,社交应用界面510包括一个或多个图文内容(例如,图文内容511)、范围选择栏512和辅助控件栏513。
As shown in FIG. 5B , the social application interface 510 includes one or more graphic and text contents (for example, graphic and text content 511 ), a range selection bar 512 and an auxiliary control bar 513 .
图文内容511可以包括用户头像511a、用户名称511b、图片511c、转发控件511d、评论控件511e和点赞控件511f。用户头像511a可用于指示发布该图文内容511的用户的头像,用户名称511b可用于指示发布该图文内容511的用户的昵称,等等。转发控件511d可用于触发电子设备100转发图文内容511,评论控件511e可用于触发电子设备100显示针对图文内容511的评论输入界面,点赞控件511f可用于触发电子设备100改变对图文内容511的点赞状态,改变点赞控件511f的颜色。例如,当点赞控件511f的颜色为如图5B所示的白色时,电子设备100可以响应于用户针对点赞控件511f的点击操作,将点赞控件511f的颜色由白色改变为黑色,即将图文内容511的点赞状态由未点赞状态更改为已点赞状态。在一些实施例中,图文内容511还可以包括更多或更少的图片,也可以包括文字,等等,本申请在此不做限定。Graphic content 511 may include user avatar 511a, user name 511b, picture 511c, forwarding control 511d, comment control 511e and like control 511f. The user avatar 511a may be used to indicate the avatar of the user who posted the graphic content 511, and the user name 511b may be used to indicate the nickname of the user who posted the graphic content 511, and so on. The forwarding control 511d can be used to trigger the electronic device 100 to forward the graphic content 511, the comment control 511e can be used to trigger the electronic device 100 to display a comment input interface for the graphic content 511, and the like control 511f can be used to trigger the electronic device 100 to change the graphic content 511. The like status of 511 changes the color of the like control 511f. For example, when the color of the like control 511f is white as shown in FIG. 5B, the electronic device 100 can respond to the user's click operation on the like control 511f, change the color of the like control 511f from white to black, that is, as shown in FIG. The like status of article content 511 was changed from unliked to liked. In some embodiments, the graphic content 511 may also include more or less pictures, text, etc., which is not limited in this application.
范围选择栏512可以包括一个或多个范围选项,例如关注选项、推荐选项512a和同城选项。范围选项可用于触发电子设备100显示该范围选项对应的一个或多个图文内容。在一些实施例中,处于选中状态的范围选项可以采用加粗的字体显示,也可以采用不同颜色的文字、或是不同的背景颜色显示,等等。以图5B为例,推荐选项512a采用加粗字体显示,即推荐选项512a处于选中状态,说明社交应用界面510中显示的一个或多个图文内容为社交应用软件推荐的图文内容。在一些实施例中,范围选择栏512中还可以包括一个或多个控件,例如拍照控件512b、添加控件512c。拍照控件512b可用于触发电子设备100显示拍摄界面,添加控件512c可用于触发电子设备100添加一个或多个范围选项。The range selection column 512 may include one or more range options, such as a focus option, a recommendation option 512a, and a local option. The range option can be used to trigger the electronic device 100 to display one or more graphic contents corresponding to the range option. In some embodiments, the selected range option may be displayed in bold font, text in different colors, different background colors, etc. Taking FIG. 5B as an example, the recommended option 512a is displayed in bold font, that is, the recommended option 512a is in a selected state, indicating that one or more graphic and text contents displayed in the social application interface 510 are graphic and text contents recommended by the social application software. In some embodiments, the range selection bar 512 may also include one or more controls, such as a photo control 512b and an add control 512c. The camera control 512b can be used to trigger the electronic device 100 to display a shooting interface, and the add control 512c can be used to trigger the electronic device 100 to add one or more range options.
辅助控件栏513可以包括一个或多个辅助控件,例如,主页控件513a、搜索控件513b、消息控件513c和个人主页控件513d。其中,主页控件513a可用于触发电子设备100显示社交应用软件的主页。搜索控件513b可用于触发电子设备100显示搜索界面。消息控件513c可用于触发电子设备100显示社交应用软件中的消息界面。个人主页控件513d可用于触发电子设备100显示用户在社交应用软件中的个人主页。The auxiliary control bar 513 may include one or more auxiliary controls, such as a homepage control 513a, a search control 513b, a message control 513c, and a personal homepage control 513d. Among them, the homepage control 513a can be used to trigger the electronic device 100 to display the homepage of the social application software. The search control 513b may be used to trigger the electronic device 100 to display a search interface. The message control 513c can be used to trigger the electronic device 100 to display a message interface in the social application software. The personal homepage control 513d may be used to trigger the electronic device 100 to display the user's personal homepage in the social application software.
在电子设备100显示社交应用界面510的情况下,电子设备100可以获取到畅连应用的通知。电子设备100可以获取到畅连应用的通知的同时,在触控屏的第一区域显示如图5C所示的通知提示窗520,该通知提示窗520用于提示用户电子设备100接收到该畅连应用的通知。When the electronic device 100 displays the social application interface 510, the electronic device 100 can obtain the notification of the Changlian application. While the electronic device 100 can obtain the notification of the Changlian application, the notification prompt window 520 as shown in FIG. 5C is displayed in the first area of the touch screen. The notification prompt window 520 is used to prompt the user that the electronic device 100 has received the Changlian application. Even app notifications.
如图5C所示,该通知提示窗520可以包括图标520a,可选的,该通知提示窗520还可以包括以下任一项或多项:发送人520b、通知简述520c、已读标记控件520d、应用软件名称520e、通知时间520f、横条520g。其中,图标520a可用于提示该通知对应的应用软件,即畅连应用软件。应用软件名称520e也可以用于提示该通知对应的应用软件为畅连应用软件。发送人520b可用于提示该通知的发送人,即小红。通知简述520c可用于提示该通知的内容,即“周六有个不错的展览,去看吗?”。可以理解的是,当该通知内容中的文字字数超过一定字数(例如20个字)时,通知简述520c中可以只显示通知内容中的一部分文字,例如,只显示通知内容中的前15个文字,等等。在一些实施例中,通知内容也可以隐藏。已读标记控件520d可用于触发电子设备100将该通知标记为已读。通知时间520f可用于提示电子设备100接收到该通知的时间,例如“刚刚”。横条520g可用于触发电子设备100取消显示通知提示窗520。在一些实施例中,通知提示窗520还可以包括比图5C所示的通知提示窗520中更多或更少的内容,本申请在此不做限定。As shown in Figure 5C, the notification prompt window 520 may include an icon 520a. Optionally, the notification prompt window 520 may also include any one or more of the following: sender 520b, notification brief description 520c, and read mark control 520d. , application name 520e, notification time 520f, horizontal bar 520g. Among them, the icon 520a can be used to prompt the application software corresponding to the notification, that is, the Changlian application software. The application software name 520e can also be used to prompt that the application software corresponding to the notification is the Changlian application software. The sender 520b can be used to prompt the sender of the notification, that is, Xiaohong. Notification brief description 520c can be used to prompt the content of the notification, that is, "There is a good exhibition on Saturday, do you want to go see it?". It can be understood that when the number of words in the notification content exceeds a certain number of words (for example, 20 words), the notification summary 520c may only display a part of the text in the notification content, for example, only the first 15 words in the notification content may be displayed. Text, etc. In some embodiments, notification content may also be hidden. Mark as read control 520d may be used to trigger the electronic device 100 to mark the notification as read. The notification time 520f may be used to prompt the electronic device 100 to receive the notification, such as "just now". The horizontal bar 520g can be used to trigger the electronic device 100 to cancel the display of the notification prompt window 520 . In some embodiments, the notification prompt window 520 may also include more or less content than the notification prompt window 520 shown in FIG. 5C , which is not limited in this application.
当电子设备100获取到畅连应用的通知时,电子设备100可以获取用户眼部和/或面部的图像。如图5D所示,电子设备100可以在显示通知提示窗的同时获取用户的眼部和/或面部的图像。电子设备100可以根据用户眼部或面部的图像确定用户的视线方向,进而确定用户注视的区域是否位于通知提示窗520所在区域(第一区域)之内。When the electronic device 100 obtains the notification from the Changlian application, the electronic device 100 may obtain images of the user's eyes and/or face. As shown in FIG. 5D , the electronic device 100 can acquire an image of the user's eyes and/or face while displaying the notification prompt window. The electronic device 100 can determine the direction of the user's gaze based on the image of the user's eyes or face, and further determine whether the area the user is looking at is within the area (first area) where the notification prompt window 520 is located.
如图5E所示,当电子设备100检测到用户的注视区域位于第一区域时,电子设备100可以改变处于通知提示窗520背景层的社交应用界面510的显示颜色。在一些实施例中,电子设备100也可以在检测到用户注视第一区域的时间超过第一时间阈值(例如500毫秒)时,改变处于通知提示窗520背景层的社交应用界面510的显示颜色。As shown in FIG. 5E , when the electronic device 100 detects that the user's gaze area is located in the first area, the electronic device 100 can change the display color of the social application interface 510 in the background layer of the notification prompt window 520 . In some embodiments, the electronic device 100 may also change the display color of the social application interface 510 in the background layer of the notification prompt window 520 when detecting that the user's gaze time on the first area exceeds a first time threshold (eg, 500 milliseconds).
电子设备100在检测到用户的注视区域位于第一区域之后,或,当电子设备100输出注视反馈时,电子设备100可以接收用户针对电子设备100触控屏上第一区域以外区域(本申请中也可以称作第二区域)的下滑操作。在一些实施例中,当电子设备100检测到用户针对触控屏上第二区域的下滑操作在触控屏上的移动距离不小于第一距离(例如,0像素或50像素,等等)时,如图5F所示,电子设备100可以将移
动社交应用界面510中的通知提示窗520向下移动。可选的,如图5F所示,在随着用户的下滑操作向下移动的过程中,通知提示窗520可以模糊显示。需要说明的是,在本申请实施例中,第一距离也可以写作第一移动距离。After the electronic device 100 detects that the user's gaze area is located in the first area, or when the electronic device 100 outputs gaze feedback, the electronic device 100 may receive the user's response to an area other than the first area on the touch screen of the electronic device 100 (in this application It can also be called the sliding operation of the second area). In some embodiments, when the electronic device 100 detects that the user's sliding operation for the second area on the touch screen moves the distance on the touch screen not less than the first distance (for example, 0 pixels or 50 pixels, etc.) , as shown in Figure 5F, the electronic device 100 can move The notification prompt window 520 in the mobile social application interface 510 moves downward. Optionally, as shown in FIG. 5F , the notification prompt window 520 may be displayed in a blurred manner as it moves downward along with the user's sliding operation. It should be noted that in the embodiment of the present application, the first distance may also be written as the first movement distance.
在一些实施例中,当电子设备100检测到用户针对触控屏上第二区域的下滑操作的持续时间不小于第一持续时间(例如,1微秒或1毫秒,等等)时,如图5F所示,电子设备100可以将移动社交应用界面510中的通知提示窗520向下移动。在本申请实施例中,第一持续时间也可以写作第二时长。In some embodiments, when the electronic device 100 detects that the duration of the user's sliding operation for the second area on the touch screen is not less than the first duration (for example, 1 microsecond or 1 millisecond, etc.), as shown in FIG. As shown in 5F, the electronic device 100 can move the notification prompt window 520 in the mobile social application interface 510 downward. In the embodiment of this application, the first duration can also be written as the second duration.
在一些实施例中,当电子设备100检测到上述下滑操作在触控屏上移动的距离不小于第二距离(例如,200像素)时,或是,当电子设备100检测到上述下滑操作的持续时间不小于第二持续时间(例如,0.1秒)时,社交应用界面510中的通知提示窗520可以随着用户的下滑操作逐渐变大展开形成如图5G所示悬浮窗530。其中,第二距离大于第一距离,第二持续时间大于第一持续时间。在本申请实施例中,第二距离也可以写作第二移动距离,第二持续时间也可以写作第三时长。In some embodiments, when the electronic device 100 detects that the sliding distance moved on the touch screen is not less than a second distance (for example, 200 pixels), or when the electronic device 100 detects that the sliding operation continues, When the time is not less than the second duration (for example, 0.1 seconds), the notification prompt window 520 in the social application interface 510 can gradually become larger and expanded as the user slides down to form a floating window 530 as shown in Figure 5G. Wherein, the second distance is greater than the first distance, and the second duration is greater than the first duration. In the embodiment of the present application, the second distance can also be written as the second moving distance, and the second duration can also be written as the third duration.
如图5G所示,悬浮窗530可以包括图标531,图标531可以是通知对应的应用软件图标,即畅连应用软件图标。可选的,当用户继续向下滑动时,电子设备100可以响应于该下滑操作,显示比图5G所示悬浮窗530更大的悬浮窗。此外,可选的,悬浮窗530的位置也可以随着用户继续向下滑动的操作向下移动。As shown in Figure 5G, the floating window 530 may include an icon 531, and the icon 531 may be an application software icon corresponding to the notification, that is, a Changlian application software icon. Optionally, when the user continues to slide downward, the electronic device 100 may display a larger floating window than the floating window 530 shown in FIG. 5G in response to the sliding operation. In addition, optionally, the position of the floating window 530 can also move downward as the user continues to slide downward.
在另一些实施例中,当电子设备100检测到上述下滑操作在触控屏上移动的距离不小于第二距离(例如,200像素)时,或是,当电子设备100检测到上述下滑操作的持续时间不小于第二持续时间(例如,0.1秒)时,电子设备100可以取消显示通知提示窗520,并显示如图5G所示的悬浮窗530。In other embodiments, when the electronic device 100 detects that the sliding distance moved on the touch screen is not less than a second distance (for example, 200 pixels), or when the electronic device 100 detects that the sliding operation moves When the duration is not less than the second duration (for example, 0.1 seconds), the electronic device 100 can cancel the display of the notification prompt window 520 and display the floating window 530 as shown in FIG. 5G.
当电子设备100检测到上述下滑操作在触控屏上移动的距离不小于第三距离(例如,500像素)时,或者,当电子设备100检测到上述下滑操作的持续时间不小于第三持续时间(例如,1秒)时,电子设备100可以在悬浮窗530中显示如图5H所示的畅连应用界面541,其中,第三距离大于第二距离,第三持续时间大于第二持续时间。When the electronic device 100 detects that the distance moved by the slide operation on the touch screen is not less than a third distance (for example, 500 pixels), or when the electronic device 100 detects that the duration of the slide operation is not less than a third duration. (for example, 1 second), the electronic device 100 can display the Changlian application interface 541 as shown in FIG. 5H in the floating window 530, where the third distance is greater than the second distance, and the third duration is greater than the second duration.
如图5H所示,悬浮窗530中可以显示畅连应用界面541。其中,畅连应用界面541可以包括一条或多条消息、发送人标识541b、一个或多个用于回复消息的控件(例如,录音控件541c、发送控件541d和切换输入控件541e,等等)。As shown in Figure 5H, the Changlian application interface 541 can be displayed in the floating window 530. The Changlian application interface 541 may include one or more messages, a sender identification 541b, and one or more controls for replying to messages (for example, a recording control 541c, a sending control 541d, a switching input control 541e, etc.).
其中,该一条或多条消息包括消息541a,用于提示该通知的具体内容。消息541a可以包括文字、图片和语音中的一项或多项,例如,消息541a可以是“周六有个不错的展览,去看吗?”。The one or more messages include message 541a, which is used to prompt the specific content of the notification. The message 541a may include one or more of text, pictures, and voice. For example, the message 541a may be "There is a good exhibition on Saturday, do you want to go see it?"
发送人标识541b用于提示消息541a的发送人,发送人标识541b可以是发送人的头像,也可以是发送人的手机号码、用户昵称等可以提示发送人身份的信息。本申请实施例中的发送人指的是发送消息541a的电子设备。The sender identification 541b is used to prompt the sender of the message 541a. The sender identification 541b may be the sender's avatar, or may be the sender's mobile phone number, user nickname, or other information that can prompt the sender's identity. The sender in the embodiment of this application refers to the electronic device that sends the message 541a.
该一个或多个用于回复消息的控件可用于回复消息541a,即将信息发送给消息541a的发送人。录音控件541c用于触发电子设备100录制语音信息。发送控件541d用于触发电子设备100向消息541a的发送人发送录制好的语音信息。切换输入控件541e用于触发电子设备100将当前输入方式由语音输入切换为键盘输入,并显示键盘和输入栏。该一个或多个用于回复消息的控件还可以包括图片控件541f、信封控件541g、表情控件541h、位置控件541i和添加控件541j。其中,图片控件541f可用于发送图片信息。信封控件541g可用于触发电子设备100将待发送的信息以信封信息形式发送。表情控件541h可用于发送表情。位置控件541i可用于发送电子设备100的位置信息。添加控件541j可用于添加其他控件。The one or more controls for replying to a message may be used to reply to message 541a, that is, to send information to the sender of message 541a. The recording control 541c is used to trigger the electronic device 100 to record voice information. The sending control 541d is used to trigger the electronic device 100 to send the recorded voice message to the sender of the message 541a. The switching input control 541e is used to trigger the electronic device 100 to switch the current input mode from voice input to keyboard input, and display the keyboard and input field. The one or more controls for replying to messages may also include a picture control 541f, an envelope control 541g, an expression control 541h, a position control 541i, and an add control 541j. Among them, the picture control 541f can be used to send picture information. The envelope control 541g can be used to trigger the electronic device 100 to send the information to be sent in the form of envelope information. Expression control 541h can be used to send expressions. Location control 541i may be used to send location information of electronic device 100. Add Control 541j can be used to add additional controls.
悬浮窗530还可以包括一个或多个辅助控件,例如,全屏控件542、最小化控件543、拖动控件544和关闭控件545。其中,全屏控件542可用于触发电子设备100打开畅连应用软件,将悬浮窗530中显示的畅连应用界面541全屏显示,具体可以参考下述图6B所示界面600的相关描述,本申请在此暂不详述。最小化控件543可用于触发电子设备100取消显示悬浮窗530,并在社交应用界面510中显示悬浮球,悬浮球的显示尺寸远小于悬浮窗530的显示尺寸,且该悬浮球上可以显示有畅连应用软件图标。拖动控件544可用于触发电子设备100使悬浮窗530随着用户的滑动轨迹移动。关闭控件545可用于取消显示该悬浮窗530。The floating window 530 may also include one or more auxiliary controls, such as a full screen control 542, a minimize control 543, a drag control 544, and a close control 545. Among them, the full-screen control 542 can be used to trigger the electronic device 100 to open the Changlian application software and display the Changlian application interface 541 displayed in the floating window 530 in full screen. For details, please refer to the following description of the interface 600 shown in Figure 6B. This application is in This will not be described in detail yet. The minimization control 543 can be used to trigger the electronic device 100 to cancel the display of the floating window 530 and display a floating ball in the social application interface 510. The display size of the floating ball is much smaller than the display size of the floating window 530, and there can be a smooth display on the floating ball. Even application software icons. The drag control 544 can be used to trigger the electronic device 100 to move the floating window 530 along the user's sliding trajectory. The close control 545 can be used to cancel display of the floating window 530 .
需要说明的是,在电子设备100显示悬浮窗530之后,电子设备100可以响应于用户的松开操作,确定用户松开操作时悬浮窗530的显示位置为悬浮窗530的显示位置。It should be noted that after the electronic device 100 displays the floating window 530, the electronic device 100 may respond to the user's release operation and determine that the display position of the floating window 530 when the user releases the operation is the display position of the floating window 530.
采用上述通知查看方法,电子设备100也可以基于用户的注视区域以及用户针对电子设备100第二区
域的操作,查看顶部的通知提示窗对应的通知内容。由于不需要针对电子设备100顶部的通知提示窗进行操作,即使是在用户单手操作的场景下,用户也可以很轻易地查看通知内容。Using the above notification viewing method, the electronic device 100 can also be based on the user's gaze area and the user's second area of the electronic device 100 For domain operations, check the notification content corresponding to the notification prompt window at the top. Since there is no need to operate the notification prompt window on the top of the electronic device 100, the user can easily view the notification content even in a scenario where the user operates with one hand.
在一种可能的实现方式中,当电子设备100显示有注视反馈时,电子设备100也可以接收并响应于用户针对电子设备100上第二区域的操作2(例如,单击操作、长按操作、双击操作或两指长按操作,等等),打开通知提示窗对应的应用软件,并显示该应用软件的界面。In a possible implementation, when the electronic device 100 displays gaze feedback, the electronic device 100 can also receive and respond to the user's operation 2 (for example, click operation, long press operation) on the second area on the electronic device 100 , double-click operation or long-press operation with two fingers, etc.), open the application software corresponding to the notification prompt window, and display the interface of the application software.
示例性的,在接收到畅连应用软件的通知之后,当电子设备100检测到用户注视的区域位于第一区域之内,且用户注视第一区域的时间不小于第一时间阈值时,电子设备100可以显示如图6A所示的社交应用界面510。该社交应用界面510与前述图5E所示社交应用界面510相同。Exemplarily, after receiving the notification from the Changlian application software, when the electronic device 100 detects that the area the user is looking at is within the first area, and the time the user is looking at the first area is not less than the first time threshold, the electronic device 100 100 may display social application interface 510 as shown in Figure 6A. The social application interface 510 is the same as the aforementioned social application interface 510 shown in FIG. 5E.
如图6A所示,电子设备100可以接收并响应于用户针对第二区域的长按操作,打开畅连应用软件,显示如图6B所示的畅连应用界面600。As shown in Figure 6A, the electronic device 100 can receive and respond to the user's long press operation on the second area, open the Changlian application software, and display the Changlian application interface 600 as shown in Figure 6B.
如图6B所示,畅连应用界面600的具体内容可以参考上述图5H所示的悬浮窗530中显示的畅连应用界面541的相关描述,此处不再赘述。As shown in Figure 6B, the specific content of the Changlian application interface 600 can be referred to the relevant description of the Changlian application interface 541 displayed in the floating window 530 shown in Figure 5H, and will not be described again here.
这样,可以通过更为简便的操作,直接打开通知提示窗对应的应用软件,并显示应用软件的界面。缩短了显示应用软件的界面所需的时间。In this way, the application software corresponding to the notification prompt window can be directly opened through a simpler operation, and the application software interface can be displayed. The time required to display the interface of the application software is shortened.
在一些实施例中,当电子设备100显示有注视反馈时,电子设备100也可以接收并响应于用户针对电子设备100上第二区域的操作3(例如,双击操作、长按操作或两指长按操作,等等),显示如图5H所示的悬浮窗530。需要说明的是,操作2与操作3不同。In some embodiments, when the electronic device 100 displays gaze feedback, the electronic device 100 may also receive and respond to the user's operation 3 on the second area on the electronic device 100 (for example, a double-click operation, a long press operation, or a two-finger long press operation). Press operation, etc.) to display the floating window 530 as shown in Figure 5H. It should be noted that operation 2 is different from operation 3.
这样,也可以通过定义操作3,简化查看通知的步骤,节省时间。In this way, you can also simplify the steps of viewing notifications and save time by defining operation 3.
在一些应用场景中,在电子设备100输出注视反馈之后,如果用户不想查看通知提示窗对应的通知的具体内容,电子设备100也可以响应于用户针对电子设备100上第二区域的操作4(例如,右滑操作),取消显示该通知提示窗,例如将该通知提示窗向右拖动直至完全消失,或者将取消显示通知提示窗,并显示悬浮球,悬浮球可用于触发电子设备100显示通知提示窗,或用于触发电子设备100显示第一应用的界面。可选的,在一些实施例中,电子设备100还可以响应于该操作4,将该通知提示窗对应的通知标记为已读。In some application scenarios, after the electronic device 100 outputs gaze feedback, if the user does not want to view the specific content of the notification corresponding to the notification prompt window, the electronic device 100 can also respond to the user's operation 4 on the second area on the electronic device 100 (for example, , right swipe operation), cancel the display of the notification prompt window, for example, drag the notification prompt window to the right until it disappears completely, or cancel the display of the notification prompt window and display a floating ball, which can be used to trigger the electronic device 100 to display a notification A prompt window, or an interface used to trigger the electronic device 100 to display the first application. Optionally, in some embodiments, the electronic device 100 can also mark the notification corresponding to the notification prompt window as read in response to operation 4.
示例性的,在接收到畅连应用软件的通知之后,当电子设备100检测到用户注视的区域位于第一区域之内,且用户注视第一区域的时间不小于第一时间阈值时,电子设备100可以显示如图7A所示的社交应用界面510。该社交应用界面510与前述图5E所示社交应用界面510相同。Exemplarily, after receiving the notification from the Changlian application software, when the electronic device 100 detects that the area the user is looking at is within the first area, and the time the user is looking at the first area is not less than the first time threshold, the electronic device 100 100 may display social application interface 510 as shown in Figure 7A. The social application interface 510 is the same as the aforementioned social application interface 510 shown in FIG. 5E.
如图7A所示,电子设备100可以响应于用户针对第二区域的右滑操作,取消显示通知提示窗520,并在社交应用界面510上显示如图7B所示的通知提示窗701。As shown in FIG. 7A , the electronic device 100 may cancel the display of the notification prompt window 520 in response to the user's right sliding operation on the second area, and display the notification prompt window 701 as shown in FIG. 7B on the social application interface 510 .
如图7B所示,通知提示窗701为图7A所示的通知提示窗520的一部分,且通知提示窗701在社交应用界面510中的位置比通知提示窗520在社交应用界面510中的位置更靠右。As shown in FIG. 7B , the notification prompt window 701 is a part of the notification prompt window 520 shown in FIG. 7A , and the notification prompt window 701 is located farther in the social application interface 510 than the notification prompt window 520 is in the social application interface 510 . Keep Right.
在电子设备100显示图7B所示社交应用界面510的情况下,用户如果继续执行右滑操作,即用户继续向右滑动时,电子设备100可以接收并响应于用户的右滑操作,继续向右拖动通知提示窗701,直至该通知提示窗701完全消失。此时,电子设备100可以取消显示通知提示窗701,显示图7C所示的社交应用界面510。在一些实施例中,电子设备100也可以响应于用户的右滑操作,将通知提示窗701变为悬浮球。When the electronic device 100 displays the social application interface 510 shown in FIG. 7B, if the user continues to perform a right sliding operation, that is, when the user continues to slide to the right, the electronic device 100 can receive and respond to the user's right sliding operation and continue to slide to the right. Drag the notification prompt window 701 until the notification prompt window 701 disappears completely. At this time, the electronic device 100 can cancel the display of the notification prompt window 701 and display the social application interface 510 shown in FIG. 7C. In some embodiments, the electronic device 100 may also change the notification prompt window 701 into a floating ball in response to the user's right sliding operation.
当检测到用户针对电子设备100的右滑操作结束后,电子设备100可以取消显示社交应用界面510中的注视反馈,改变社交应用界面510的显示颜色,显示如图7D所示的社交应用界面510。After detecting that the user's right-swipe operation on the electronic device 100 is completed, the electronic device 100 can cancel the display of the gaze feedback in the social application interface 510, change the display color of the social application interface 510, and display the social application interface 510 as shown in Figure 7D .
在一些实施例中,当电子设备100显示如图7C所示的社交应用界面510时,电子设备100也可以在经过第二时间阈值(例如10毫秒)之后,取消输出注视反馈,显示如图7D所示的社交应用界面510。In some embodiments, when the electronic device 100 displays the social application interface 510 as shown in Figure 7C, the electronic device 100 can also cancel output of gaze feedback after the second time threshold (for example, 10 milliseconds) has passed, as shown in Figure 7D Social application interface 510 is shown.
采用上述方法,当用户不想查看通知的具体内容时,电子设备100也可以基于用户的注视区域以及用户针对电子设备100第二区域的操作,划走通知提示窗。同样的,由于不需要针对电子设备100顶部的通知提示窗进行操作,即使是在用户单手操作的场景下,用户也可以很轻易实现上述操作。Using the above method, when the user does not want to view the specific content of the notification, the electronic device 100 can also swipe away the notification prompt window based on the user's gaze area and the user's operation on the second area of the electronic device 100 . Similarly, since there is no need to operate the notification prompt window on the top of the electronic device 100, the user can easily perform the above operations even in a scenario where the user operates with one hand.
在一些实施例中,在电子设备100输出注视反馈之后,如果用户想稍后查看通知提示窗对应的通知的具体内容,电子设备100也可以响应于用户针对电子设备100上第二区域的操作5(例如,左滑操作),将该通知标记为稍后提醒,取消显示该通知提示窗(例如将该通知提示窗向左拖动直至完全消失),并在经
过一段时间(例如第四时长)之后重新显示该通知提示窗。具体的实施方式可以参数考图7A至图7D所示实施例中的相关内容。In some embodiments, after the electronic device 100 outputs the gaze feedback, if the user wants to view the specific content of the notification corresponding to the notification prompt window later, the electronic device 100 can also respond to the user's operation 5 on the second area on the electronic device 100 (for example, swipe left), mark the notification as a reminder later, cancel the display of the notification prompt window (for example, drag the notification prompt window to the left until it disappears completely), and after The notification prompt window is redisplayed after a period of time (for example, the fourth period of time). For specific implementation methods, reference may be made to the relevant content in the embodiments shown in FIGS. 7A to 7D .
在一种可能的实现方式中,当电子设备100显示有通知提示窗时,电子设备100也可以通过用户注视第二区域的时长确定是否取消显示通知提示窗。在一些实施例中,电子设备100也可以在检测到用户注视第二区域时,基于通知提示窗的显示时长确定是否取消显示通知提示窗。In a possible implementation, when the electronic device 100 displays a notification prompt window, the electronic device 100 may also determine whether to cancel the display of the notification prompt window based on the duration of the user's gaze on the second area. In some embodiments, the electronic device 100 may also determine whether to cancel the display of the notification prompt window based on the display duration of the notification prompt window when detecting that the user is gazing at the second area.
示例性的,电子设备100在接收到畅连应用软件的通知之后,检测到用户注视的区域位于第一区域之内,且用户注视第一区域的时间不小于第一时间阈值,此时,电子设备100可以显示如图8A所示的社交应用界面510,图8A所示的社交应用界面510与图5E所示社交应用界面510相同。Exemplarily, after receiving the notification from the Changlian application software, the electronic device 100 detects that the area the user is looking at is within the first area, and the time the user is looking at the first area is not less than the first time threshold. At this time, the electronic device 100 The device 100 may display a social application interface 510 as shown in FIG. 8A , which is the same as the social application interface 510 shown in FIG. 5E .
如图8A所示,当电子设备100检测到用户注视第二区域,且注视的时间不小于第三时间阈值时,如图8B所示,电子设备100可以取消输出注视反馈,改变社交应用界面510的显示颜色。As shown in FIG. 8A , when the electronic device 100 detects that the user is gazing at the second area, and the gazing time is not less than the third time threshold, as shown in FIG. 8B , the electronic device 100 can cancel output of gaze feedback and change the social application interface 510 display color.
当电子设备100检测到用户注视第二区域,且注视的时间不小于第四时间阈值时,如图8C所示,电子设备100可以取消显示通知提示窗520。在这种情况下,第四时间阈值大于第三时间阈值。When the electronic device 100 detects that the user is gazing at the second area, and the gazing time is not less than the fourth time threshold, as shown in FIG. 8C , the electronic device 100 may cancel the display of the notification prompt window 520 . In this case, the fourth time threshold is greater than the third time threshold.
在一种可能的实现方式中,电子设备100也可以以电子设备100显示如图8B所示的社交应用界面510的时刻为起点,在确定用户注视第二区域的时间大于第五时间阈值的情况下,取消显示通知提示窗520,显示如图8C所示的社交应用界面510。In a possible implementation, the electronic device 100 may also use the moment when the electronic device 100 displays the social application interface 510 as shown in FIG. 8B as a starting point, when it is determined that the time the user looks at the second area is greater than the fifth time threshold. , the notification prompt window 520 is cancelled, and the social application interface 510 shown in FIG. 8C is displayed.
在另一种可能的实现方式中,当电子设备100检测到用户注视第二区域,且通知提示窗520的显示时长不小于第六时间阈值时,显示如图8C所示的社交应用界面510。In another possible implementation, when the electronic device 100 detects that the user is gazing at the second area and the display duration of the notification prompt window 520 is not less than the sixth time threshold, the social application interface 510 as shown in FIG. 8C is displayed.
这样,电子设备100可以通过检测用户注视第二区域的时长或通过检测用户注视第二区域的情况下通知提示窗的显示时长来确定是否取消显示通知提示窗,不需要用户进行额外的操作,简化了操作步骤。In this way, the electronic device 100 can determine whether to cancel the display of the notification prompt window by detecting the duration of the user's gaze on the second area or by detecting the display duration of the notification prompt window when the user is gazing at the second area, without requiring the user to perform additional operations, simplifying the operating steps.
在一些实施例中,在电子设备100输出注视反馈之后,电子设备100还可以响应于用户针对电子设备100的操作6(例如,上滑操作等等),执行第一界面的功能。需要说明的是,该操作与上述任一操作(例如,操作1、操作2、操作3,等等)均不相同。In some embodiments, after the electronic device 100 outputs the gaze feedback, the electronic device 100 may also perform the function of the first interface in response to the user's operation 6 on the electronic device 100 (eg, slide-up operation, etc.). It should be noted that this operation is different from any of the above operations (for example, operation 1, operation 2, operation 3, etc.).
示例性的,如图9A所示,电子设备100可以显示有注视反馈的社交应用界面510,该社交应用界面510与前述图5E所示社交应用界面510相同。For example, as shown in FIG. 9A , the electronic device 100 may display a social application interface 510 with gaze feedback. The social application interface 510 is the same as the aforementioned social application interface 510 shown in FIG. 5E .
电子设备100可以响应于用户针对电子设备100的上滑操作,向下滑动社交应用界面510中的一个或多个图文内容,显示如图9B所示的图文内容901。The electronic device 100 may, in response to the user's upward sliding operation on the electronic device 100, slide down one or more graphic content in the social application interface 510 to display the graphic content 901 as shown in FIG. 9B.
如图9B所示,图文内容901与图9A所示社交应用界面510中的图文内容不同。可选的,如图9B所示的社交应用界面510还可以包括滑动条902,滑动条902可用于触发电子设备100滑动显示社交应用界面510中的图文内容。As shown in FIG. 9B , the graphic and text content 901 is different from the graphic and text content in the social application interface 510 shown in FIG. 9A . Optionally, the social application interface 510 shown in FIG. 9B may also include a sliding bar 902, which may be used to trigger the electronic device 100 to slide and display the graphic content in the social application interface 510.
可以理解的是,图9A至图9B只是示例性的说明,当电子设备100接收的操作(例如操作6)与前述操作1至操作5中任一操作不同时,无论用户的注视区域是否位于第一区域,电子设备100可以接收并响应于该操作6,执行该操作6对应的响应。本申请在此对该操作6以及该操作6对应的响应的具体形式不做限定。It can be understood that FIG. 9A to FIG. 9B are only exemplary illustrations. When the operation (for example, operation 6) received by the electronic device 100 is different from any of the aforementioned operations 1 to 5, no matter whether the user's gaze area is located in the third In a region, the electronic device 100 can receive and respond to the operation 6, and perform a response corresponding to the operation 6. This application does not limit the specific form of operation 6 and the response corresponding to operation 6.
在一种可能的实现方式中,电子设备100的触控屏上第一区域以外的区域可以包括第二区域和第三区域。其中,响应于用户针对触控屏上第二区域的操作(例如,操作1、操作2、操作3等等),电子设备100可以控制通知提示窗。响应于用户针对第三区域的操作(例如操作6),电子设备100可以执行第一界面的功能。示例性的,第二区域可以是上述图1所示的区域1002,第三区域可以是上述图1所示的区域1003。又示例性的,第一区域可以是触控屏的顶部区域,第二区域可以是触控屏的底部区域,第三区域可以是触控屏的中部区域,等等。本申请对区域的具体划分方式不做限定。In a possible implementation, the area on the touch screen of the electronic device 100 other than the first area may include a second area and a third area. Wherein, in response to the user's operation on the second area on the touch screen (for example, operation 1, operation 2, operation 3, etc.), the electronic device 100 may control the notification prompt window. In response to the user's operation on the third area (for example, operation 6), the electronic device 100 may perform the function of the first interface. For example, the second area may be the area 1002 shown in the above-mentioned FIG. 1, and the third area may be the area 1003 shown in the above-mentioned FIG. 1. As another example, the first area may be the top area of the touch screen, the second area may be the bottom area of the touch screen, the third area may be the middle area of the touch screen, and so on. This application does not limit the specific division of areas.
在一种可能的实现方式中,电子设备100也可以通过改变通知提示窗的颜色、改变通知提示窗中通知消息下方的横条的颜色、指示灯闪烁、点亮指示灯、播放提示音或振动等其他形式来输出注视反馈。在另一种可能的实现方式中,电子设备100也可以不输出注视反馈。In a possible implementation, the electronic device 100 can also change the color of the notification prompt window, change the color of the horizontal bar under the notification message in the notification prompt window, flash the indicator light, light up the indicator light, play a prompt sound or vibrate. and other forms to output gaze feedback. In another possible implementation, the electronic device 100 may not output gaze feedback.
示例性的,当电子设备100检测到用户注视的区域位于第一区域之内且用户注视该区域的时间不小于第一时间阈值时,电子设备100可以改变通知提示窗520的颜色,显示如图10所示的社交应用界面510。For example, when the electronic device 100 detects that the area the user is looking at is within the first area and the time the user looks at the area is not less than the first time threshold, the electronic device 100 can change the color of the notification prompt window 520, as shown in Figure The social application interface 510 shown in 10.
需要说明的是,图5A至图10所示实施例只是范例,在本申请实施例中,显示通知提示窗的界面也可以是桌面、负一屏界面或其他应用软件中的界面,通知提示窗对应的通知也可以是其他应用软件的通知,
本申请在此不做限定。It should be noted that the embodiments shown in Figures 5A to 10 are only examples. In the embodiments of the present application, the interface for displaying the notification prompt window can also be the desktop, a negative screen interface or an interface in other application software. The notification prompt window The corresponding notification can also be a notification from other application software. This application is not limited here.
下面介绍本申请实施例提供的一种通知查看方法的流程示意图。The following is a schematic flowchart of a notification viewing method provided by an embodiment of the present application.
如图11A所示,通知查看方法的具体流程可以包括以下步骤:As shown in Figure 11A, the specific process of the notification viewing method may include the following steps:
S1101,在电子设备100显示第一界面的情况下,电子设备100获取第一应用的第一通知,在触控屏的第一区域显示通知提示窗。S1101. When the electronic device 100 displays the first interface, the electronic device 100 obtains the first notification of the first application and displays a notification prompt window in the first area of the touch screen.
第一界面可以是应用软件中的界面,例如上述图5B所示实施例中的社交应用界面510,也可以是显示有多个应用软件图标的桌面或负一屏界面,等等。The first interface may be an interface in application software, such as the social application interface 510 in the embodiment shown in FIG. 5B , or it may be a desktop or a one-screen interface displaying multiple application software icons, and so on.
第一通知可以是第一应用的通知,例如图5C所示实施例中畅连应用软件的通知。在一些实施例中,第一通知也可以是短信、聊天应用软件的通知或视频应用软件的通知,等等。The first notification may be a notification of the first application, such as a notification of the Changlian application software in the embodiment shown in FIG. 5C. In some embodiments, the first notification may also be a text message, a notification from a chat application software, a notification from a video application software, and so on.
第一区域可以是电子设备100的触控屏上的顶部区域。在一些实施例中,第一区域也可以是触控屏上的其他区域。The first area may be a top area on the touch screen of the electronic device 100 . In some embodiments, the first area may also be other areas on the touch screen.
电子设备100获取第一通知的方式可以是接收其他电子设备发送的第一通知,也可以是获取第一应用推送的第一通知,等等。The electronic device 100 may obtain the first notification by receiving the first notification sent by other electronic devices, or by obtaining the first notification pushed by the first application, and so on.
电子设备100可以响应于上述第一通知,在第一界面上显示通知提示窗,该通知提示窗中包括第一通知的部分内容或全部内容。示例性的,当第一通知为畅连应用软件的通知时,通知提示窗可以是上述图5C所示社交应用界面510中的通知提示窗520。In response to the above-mentioned first notification, the electronic device 100 may display a notification prompt window on the first interface, and the notification prompt window includes part or all of the content of the first notification. For example, when the first notification is a notification from the Changlian application software, the notification prompt window may be the notification prompt window 520 in the social application interface 510 shown in FIG. 5C .
S1102,电子设备100获取注视信息,注视信息包括用户的面部图像、用户的眼部图像、眼动信息、眼电信息中的任一项或多项。S1102. The electronic device 100 obtains gaze information. The gaze information includes any one or more of the user's facial image, the user's eye image, eye movement information, and electrooculogram information.
在一些实施例中,当电子设备100获取到第一通知时,电子设备100可以通过摄像头获取用户的图像信息。用户的图像信息可以包括用户的面部图像信息和/或用户的眼部图像信息。In some embodiments, when the electronic device 100 obtains the first notification, the electronic device 100 may obtain the user's image information through a camera. The user's image information may include the user's facial image information and/or the user's eye image information.
电子设备100可以响应于第一通知,开启电子设备100的前置摄像头,并通过电子设备100的前置摄像头持续获取用户的图像信息,例如,电子设备100可以响应于第一通知,以固定时间间隔持续获取用户的图像信息。The electronic device 100 may respond to the first notification, turn on the front camera of the electronic device 100, and continuously obtain the user's image information through the front camera of the electronic device 100. For example, the electronic device 100 may respond to the first notification, at a fixed time Continuously obtain the user's image information at intervals.
在一些实施例中,电子设备100的前置摄像头可以始终保持开启状态,即电子设备100不需要响应于第一通知,也可以持续获取用户的图像信息。In some embodiments, the front camera of the electronic device 100 can always remain on, that is, the electronic device 100 does not need to respond to the first notification and can continue to obtain the user's image information.
在另一些实施例中,电子设备100也可以获取用户的眼动信息或眼电信息。In other embodiments, the electronic device 100 can also obtain the user's eye movement information or electrooculogram information.
S1103,电子设备100基于注视信息判断用户的注视区域是否位于第一区域。S1103: The electronic device 100 determines whether the user's gaze area is located in the first area based on the gaze information.
需要说明的是,在本申请实施例中,注视区域位于第一区域可以包括以下三种情况:注视区域与第一区域完全重合、注视区域在第一区域之内、注视区域与第一区域存在交集。下面结合图示说明注视区域与第一区域的关系。It should be noted that in the embodiment of the present application, the gaze area being located in the first area may include the following three situations: the gaze area completely coincides with the first region, the gaze area is within the first region, and the gaze area and the first region exist intersection. The relationship between the gaze area and the first area will be described below with the help of diagrams.
如图11B所示,第一区域可以是电子设备100触控屏上的区域2001,用户的注视区域也可以是区域2001,即第一区域与用户的注视区域完全重合。在这种情况下,可以确定用户的注视区域位于第一区域。As shown in FIG. 11B , the first area may be area 2001 on the touch screen of the electronic device 100 , and the user's gaze area may also be area 2001 , that is, the first area completely overlaps with the user's gaze area. In this case, it can be determined that the user's gaze area is located in the first area.
如图11C所示,第一区域可以是电子设备100触控屏上的区域2001,用户的注视区域可以是区域2002,即用户的注视区域可以在第一区域之内。在这种情况下,可以确定用户的注视区域位于第一区域。As shown in FIG. 11C , the first area may be area 2001 on the touch screen of the electronic device 100 , and the user's gaze area may be area 2002 , that is, the user's gaze area may be within the first area. In this case, it can be determined that the user's gaze area is located in the first area.
如图11D所示,第一区域可以是电子设备100触控屏上的区域2001,用户的注视区域可以是区域2003,即用户的注视区域与第一区域存在交集,且该交集为区域2004。在这种情况下,也可以确定用户的注视区域位于第一区域。As shown in FIG. 11D , the first area may be area 2001 on the touch screen of the electronic device 100 , and the user's gaze area may be area 2003 , that is, the user's gaze area intersects with the first area, and the intersection is area 2004 . In this case, it may also be determined that the user's gaze area is located in the first area.
可以理解的是,上述图11B至图11D所示实施例只是范例,在一些实施例中,注视区域也可以是圆角矩形、圆形等形状区域,第一区域也可以是圆角矩形、圆形等形状的区域,本申请在此对第一区域以及注视区域的具体形状不做限定。It can be understood that the above-mentioned embodiments shown in FIGS. 11B to 11D are only examples. In some embodiments, the gaze area may also be a rounded rectangle, a circle, or other shapes, and the first area may also be a rounded rectangle, a circle, or the like. This application does not limit the specific shapes of the first area and the gaze area.
示例性的,在获取用户的图像信息之后,电子设备100基于用户的图像信息判断用户的注视区域是否位于第一区域的具体流程可以包括以下步骤:Exemplarily, after acquiring the user's image information, the specific process for the electronic device 100 to determine whether the user's gaze area is located in the first area based on the user's image information may include the following steps:
1.电子设备100基于用户的图像信息确定用户的视线在第一坐标系下的注视方向,第一坐标系是以用户的面部或眼部的中心点为原点的坐标系。1. The electronic device 100 determines the gaze direction of the user's line of sight in a first coordinate system based on the user's image information. The first coordinate system is a coordinate system with the center point of the user's face or eyes as the origin.
2.电子设备100基于用户的图像信息确定用户的头部相对于电子设备100的前置摄像头的位置与姿态。
2. The electronic device 100 determines the position and posture of the user's head relative to the front camera of the electronic device 100 based on the user's image information.
3.电子设备100基于用户的视线在第一坐标系下的注视方向以及用户的头部相对于电子设备100的前置摄像头的位置与姿态,确定出用户的视线在第二坐标系下的注视方向,第二坐标系是以电子设备100的前置摄像头的光点为原点,以该前置摄像头的光轴为Z轴的坐标系。3. The electronic device 100 determines the gaze of the user's gaze in the second coordinate system based on the gaze direction of the user's gaze in the first coordinate system and the position and posture of the user's head relative to the front camera of the electronic device 100 direction, the second coordinate system is a coordinate system with the light point of the front camera of the electronic device 100 as the origin and the optical axis of the front camera as the Z axis.
4.电子设备100基于用户的视线在第二坐标系下的注视方向与触控屏在第二坐标系下的坐标,确定用户的视线与触控屏的交点为第一位置。4. The electronic device 100 determines the intersection point of the user's line of sight and the touch screen as the first position based on the gaze direction of the user's line of sight in the second coordinate system and the coordinates of the touch screen in the second coordinate system.
其中,触控屏在第二坐标系下的坐标可以是预先存储在电子设备100中的数据。在一些实施例中,电子设备100也可以基于电子设备100的触控屏的尺寸以及前置摄像头相对于触控屏的位置,确定触控屏在第二坐标系下的坐标。The coordinates of the touch screen in the second coordinate system may be data pre-stored in the electronic device 100 . In some embodiments, the electronic device 100 may also determine the coordinates of the touch screen in the second coordinate system based on the size of the touch screen of the electronic device 100 and the position of the front camera relative to the touch screen.
5.电子设备100基于第一位置确定用户的注视区域。5. The electronic device 100 determines the user's gaze area based on the first location.
电子设备100可以以第一位置为注视区域的中心,以特定长度(例如触控屏的宽度、眼距或视距等等)为矩形区域的长和宽确定出注视区域。可以理解的是,注视区域也可以是圆形区域或其他形状的区域,本申请在此不做限定。The electronic device 100 may use the first position as the center of the gaze area, and determine the gaze area as the length and width of the rectangular area with a specific length (such as the width of the touch screen, eye distance or viewing distance, etc.). It can be understood that the gaze area can also be a circular area or an area of other shapes, which is not limited in this application.
6.电子设备100判断用户的注视区域是否位于第一区域。6. The electronic device 100 determines whether the user's gaze area is located in the first area.
当用户的注视区域与第一区域的位置关系满足图11B至图11D所示实施例中的任一种情况时,电子设备100可以确定用户的注视区域位于第一区域。When the positional relationship between the user's gaze area and the first area satisfies any of the conditions in the embodiments shown in FIGS. 11B to 11D , the electronic device 100 may determine that the user's gaze area is located in the first area.
需要说明的是,上述方式只是一个范例,在本申请实施例中,还可以采用其他视线估计算法来确定用户的注视区域,本申请在此不做限定。It should be noted that the above method is only an example. In the embodiment of the present application, other line of sight estimation algorithms can also be used to determine the user's gaze area, which is not limited in this application.
当电子设备100获取的注视信息为眼动信息或眼电信息时,电子设备100也可以基于眼动信息或眼电信息确定用户的注视区域是否位于第一区域。其中,眼动信息是眼球运动的图像信息或眼球运动产生的电位信息。当眼球运动时,会产生可测量的电位变化,称为眼电信号(electro-oculogram,eog),也可以称作眼电信息。通过对眼动信息或眼电信息进行分析计算,可以得到眼球运动的轨迹。基于眼球的运动轨迹,就能够计算出用户相对于电子设备100的视线方向,进而确定出用户在电子设备100上的注视区域。When the gaze information obtained by the electronic device 100 is eye movement information or electrooculogram information, the electronic device 100 may also determine whether the user's gaze area is located in the first area based on the eye movement information or electrooculogram information. Among them, eye movement information is image information of eye movement or potential information generated by eye movement. When the eyeballs move, measurable potential changes are produced, called electrooculogram (eog), which can also be called electrooculogram information. By analyzing and calculating eye movement information or electrooculogram information, the eye movement trajectory can be obtained. Based on the movement trajectory of the eyeballs, the user's line of sight direction relative to the electronic device 100 can be calculated, and then the user's gaze area on the electronic device 100 can be determined.
当电子设备100确定用户注视第一区域时,电子设备100执行下述步骤S1104或步骤S1105;当电子设备100确定用户没有注视第一区域时,电子设备100执行步骤S1108或步骤S1109。When the electronic device 100 determines that the user is gazing at the first area, the electronic device 100 performs the following step S1104 or step S1105; when the electronic device 100 determines that the user is not gazing at the first area, the electronic device 100 performs step S1108 or step S1109.
S1104,电子设备100输出注视反馈。S1104. The electronic device 100 outputs gaze feedback.
步骤S1104为可选步骤。Step S1104 is an optional step.
在一些实施例中,电子设备100可以基于用户的注视区域与第一区域的位置关系确定是否输出注视反馈。注视反馈用于提示用户的注视区域位于第一区域。In some embodiments, the electronic device 100 may determine whether to output gaze feedback based on a positional relationship between the user's gaze area and the first region. Gaze feedback is used to prompt the user that the gaze area is located in the first area.
注视反馈可以是改变第一界面的显示颜色,例如图5E所示的社交应用界面510。在一些实施例中,注视反馈也可以是改变第一界面中通知提示窗的颜色,例如图10所示的实施例中的通知提示窗520。在一些实施例中,注视反馈还可以是通知提示窗内的控件变化,例如,改变通知提示窗内通知消息下方的横条的颜色,等等。在另一些实施例中,注视反馈还可以是点亮指示灯亮度、电子设备100振动、播放提示音等等,本申请在此不做限定。Gaze feedback may be changing the display color of the first interface, such as the social application interface 510 shown in FIG. 5E. In some embodiments, the gaze feedback may also be to change the color of the notification prompt window in the first interface, such as the notification prompt window 520 in the embodiment shown in FIG. 10 . In some embodiments, the gaze feedback may also be a change in the controls in the notification prompt window, for example, changing the color of the horizontal bar below the notification message in the notification prompt window, and so on. In other embodiments, the gaze feedback can also be the brightness of the indicator light, the vibration of the electronic device 100, the playing of prompt sounds, etc., which is not limited in this application.
在另一些实施例中,电子设备100也可以基于用户注视第一区域的时长与第一时间阈值的关系确定是否输出注视反馈。In other embodiments, the electronic device 100 may also determine whether to output gaze feedback based on the relationship between the duration of the user's gaze on the first area and the first time threshold.
示例性的,在用户注视第一区域的情况下,电子设备100判断用户注视第一区域的时间是否小于第一时间阈值(在本申请实施例中,第一时间阈值也可以写作第一时长)。即,以电子设备100确定用户注视第一区域的时刻为起点,判断在第一时间阈值对应的时间段内,用户是否持续注视第一区域。若用户在该时间段内,持续注视第一区域,则确定用户注视第一区域的时间大于等于第一时间阈值。若用户在该时间段内,视线离开第一区域,则确定用户注视第一区域的时间小于第一时间阈值。For example, when the user looks at the first area, the electronic device 100 determines whether the time the user looks at the first area is less than a first time threshold (in the embodiment of the present application, the first time threshold can also be written as a first duration). . That is, taking the moment when the electronic device 100 determines that the user is gazing at the first area as a starting point, it is determined whether the user continues to gazing at the first area within the time period corresponding to the first time threshold. If the user continues to gaze at the first area during this time period, it is determined that the time for which the user gazes at the first area is greater than or equal to the first time threshold. If the user's gaze leaves the first area within this time period, it is determined that the time the user looks at the first area is less than the first time threshold.
当用户注视第一区域的时间小于第一时间阈值时,说明用户对通知提示窗提示的第一通知没有兴趣,电子设备100执行步骤S1108或步骤S1109。When the time the user looks at the first area is less than the first time threshold, it means that the user is not interested in the first notification prompted by the notification prompt window, and the electronic device 100 executes step S1108 or step S1109.
当用户注视第一区域的时间大于等于第一时间阈值时,说明用户对通知提示窗提示的第一通知有兴趣,电子设备100执行步骤S1105。When the time the user looks at the first area is greater than or equal to the first time threshold, it indicates that the user is interested in the first notification prompted by the notification prompt window, and the electronic device 100 executes step S1105.
在一些实施例中,例如,当电子设备100首次检测到用户注视区域位于第一区域时、当用户首次触发电子设备100输出注视反馈,或是当电子设备100在较长时间内(例如一个月)未输出注视反馈后,重新
输出注视反馈时,等等,在上述这些情况下,电子设备100也可以在第一界面中显示操作提示。操作提示可用于提示用户操作提示可以是文字提示、图片提示、动画提示、声音提示等提示中的任一种或多种。操作提示用于提示用户在第二区域对通知提示窗进行控制,即提示用户可以通过针对第二区域的操作(例如操作1、操作2、操作3等等)实现移动通知提示窗、显示通知内容或取消显示通知提示窗,等等功能。In some embodiments, for example, when the electronic device 100 detects that the user's gaze area is located in the first area for the first time, when the user triggers the electronic device 100 to output gaze feedback for the first time, or when the electronic device 100 responds within a longer period of time (eg, one month). ) after gaze feedback is not output, re- When outputting gaze feedback, etc., in the above situations, the electronic device 100 may also display an operation prompt in the first interface. The operation prompt can be used to prompt the user. The operation prompt can be any one or more of text prompts, picture prompts, animation prompts, sound prompts, etc. The operation prompt is used to prompt the user to control the notification prompt window in the second area, that is, to prompt the user to move the notification prompt window and display the notification content through operations in the second area (such as operation 1, operation 2, operation 3, etc.) Or cancel the display of the notification prompt window, and other functions.
S1105,电子设备100接收用户针对电子设备100上第二区域的操作1。S1105. The electronic device 100 receives the user's operation 1 for the second area on the electronic device 100.
第二区域为电子设备100的触控屏上第一区域以外的区域。第二区域可以包括上述图1所示实施例中的区域1002和/或区域1003。在一些实施例中,第二区域可以是电子设备100的触控屏上的中部区域,或是触控屏上的底部区域。The second area is an area on the touch screen of the electronic device 100 other than the first area. The second area may include area 1002 and/or area 1003 in the embodiment shown in FIG. 1 . In some embodiments, the second area may be a middle area on the touch screen of the electronic device 100 or a bottom area on the touch screen.
在一些实施例中,操作1可以是图5E所示实施例中的下滑操作。在本申请实施例中,操作1也可以写作第一操作。In some embodiments, operation 1 may be the sliding operation in the embodiment shown in FIG. 5E. In the embodiment of this application, operation 1 can also be written as the first operation.
S1106,响应于操作1,电子设备100将通知提示窗在第一界面中的显示位置向下移动。S1106. In response to operation 1, the electronic device 100 moves the display position of the notification prompt window downward in the first interface.
步骤S1106为可选步骤。Step S1106 is an optional step.
在一些实施例中,响应于操作1,电子设备100可以检测用户操作1在第一界面中移动的距离或操作1的持续时间,并基于用户操作1在触控屏上的移动距离或操作1的持续时间确定是否移动通知提示窗。示例性的,可以参考图5G所示实施例,当用户操作1在触控屏上的移动距离不小于第一距离(在本申请实施例中,第一距离也可以写作第一移动距离)时,电子设备100可以将通知提示窗在第一界面中的显示位置向下移动。又示例性的,当操作1的持续时间不小于第一持续时间(例如1微秒或1毫秒)时,电子设备100可以将通知提示窗在第一界面中的显示位置向下移动。需要说明的是,在本申请实施例中,第一持续时间也可以写作第二时长。In some embodiments, in response to operation 1, the electronic device 100 may detect the distance moved by user operation 1 in the first interface or the duration of operation 1, and based on the distance moved by user operation 1 on the touch screen or the duration of operation 1 The duration determines whether the notification prompt window is moved. For example, referring to the embodiment shown in Figure 5G, when the movement distance of user operation 1 on the touch screen is not less than the first distance (in the embodiment of this application, the first distance can also be written as the first movement distance) , the electronic device 100 can move the display position of the notification prompt window downward in the first interface. As another example, when the duration of operation 1 is not less than the first duration (for example, 1 microsecond or 1 millisecond), the electronic device 100 can move the display position of the notification prompt window downward in the first interface. It should be noted that in the embodiment of the present application, the first duration can also be written as the second duration.
这样,可以通过设置第一距离或第一持续时间避免由于用户的误触而向下拖动通知提示窗,提高方法的准确性,还可以减少不必要的能耗。In this way, by setting the first distance or the first duration, the notification prompt window can be prevented from being dragged downward due to the user's accidental touch, thereby improving the accuracy of the method and reducing unnecessary energy consumption.
S1107,响应于操作1,电子设备100显示悬浮窗,悬浮窗用于显示第一应用的界面,该第一应用的界面包括第一通知的具体内容。S1107. In response to operation 1, the electronic device 100 displays a floating window. The floating window is used to display the interface of the first application. The interface of the first application includes the specific content of the first notification.
如图5H所示,当第一通知为畅连应用软件的通知时,如图5H所示,电子设备100可以在悬浮窗530中显示的畅连应用界面541。畅连应用界面541中显示有第一通知的具体内容,具体可以参考图5H所示实施例中的相关描述。As shown in FIG. 5H , when the first notification is a notification from the Changlian application software, as shown in FIG. 5H , the electronic device 100 may display the Changlian application interface 541 in the floating window 530 . The specific content of the first notification is displayed in the Changlian application interface 541. For details, please refer to the relevant description in the embodiment shown in Figure 5H.
用户可以在悬浮窗显示的界面中,查看第一通知的具体内容,并对第一通知进行处理。The user can view the specific content of the first notification and process the first notification in the interface displayed in the floating window.
可选的,电子设备100还可以基于用户操作1在第一界面中的移动距离或操作1的持续时间将通知提示窗展开成悬浮窗,或是取消显示通知提示窗并显示悬浮窗。在上述任一情况下,可选的,悬浮窗的显示尺寸还可以随着该移动距离或持续时间的增大而增大。关于悬浮窗的具体内容可以参考上述图5G所示实施例中的相关描述。Optionally, the electronic device 100 can also expand the notification prompt window into a floating window based on the movement distance of user operation 1 in the first interface or the duration of operation 1, or cancel the display of the notification prompt window and display the floating window. In any of the above situations, optionally, the display size of the floating window may also increase as the movement distance or duration increases. Regarding the specific content of the floating window, please refer to the relevant description in the embodiment shown in FIG. 5G.
在一些实施例中,电子设备100也可以基于操作1在触控屏上移动的距离或操作1的持续时间,确定是否在悬浮窗中显示第一应用的界面,例如图5H所示的畅连应用界面541。In some embodiments, the electronic device 100 may also determine whether to display the interface of the first application in the floating window based on the distance moved by operation 1 on the touch screen or the duration of operation 1, such as Changlian shown in Figure 5H Application interface 541.
S1108,电子设备100判断用户未注视第一区域的时长是否大于等于第四时间阈值。S1108: The electronic device 100 determines whether the time period in which the user has not looked at the first area is greater than or equal to the fourth time threshold.
步骤S1108为可选步骤。在本申请实施例中,用户未注视第一区域是指用户的注视区域与第一区域没有交集,即用户的注视区域不位于第一区域。Step S1108 is an optional step. In this embodiment of the present application, the fact that the user does not focus on the first area means that the user's focus area does not intersect with the first area, that is, the user's focus area is not located in the first area.
在电子设备100确定用户没有注视第一区域之后,电子设备100可以以通知提示窗的显示时刻为起点,确定在第四时间阈值对应的时间段内,用户是否从未注视第一区域。After the electronic device 100 determines that the user has not looked at the first area, the electronic device 100 can use the display time of the notification prompt window as a starting point to determine whether the user has never looked at the first area within the time period corresponding to the fourth time threshold.
在一些实施例中,在用户短暂注视第一区域,且电子设备100确定用户的实现离开第一区域时,电子设备100可以以用户视线离开第一区域的时刻为起点,确定在第四时间阈值对应的时间段内,用户是否从未注视第一区域。In some embodiments, when the user briefly gazes at the first area and the electronic device 100 determines that the user has left the first area, the electronic device 100 may use the moment when the user's sight leaves the first area as a starting point to determine the fourth time threshold. Whether the user has never looked at the first area during the corresponding time period.
当电子设备100确定在该时间段内,用户从未注视第一区域,即用户未注视第一区域的时间大于等于第四时间阈值,电子设备100执行步骤S1110。When the electronic device 100 determines that the user has never looked at the first area during the time period, that is, the time that the user has not looked at the first area is greater than or equal to the fourth time threshold, the electronic device 100 executes step S1110.
当电子设备100确定在该时间段内,用户未注视第一区域的时间小于第四时间阈值,电子设备100执行步骤S1109。
When the electronic device 100 determines that the time during which the user does not look at the first area is less than the fourth time threshold within the time period, the electronic device 100 executes step S1109.
S1109,电子设备100判断通知提示窗的显示时长是否大于等于第六时间阈值。S1109: The electronic device 100 determines whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.
当用户未注视第一区域时,或是当用户未注视第一区域的时间小于第四时间阈值时,电子设备100可以判断通知提示窗的显示时长是否大于等于第六时间阈值。When the user does not look at the first area, or when the time for which the user does not look at the first area is less than the fourth time threshold, the electronic device 100 may determine whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.
当电子设备100确定通知提示窗的显示时长大于等于第六时间阈值时,电子设备100可以执行步骤S1110。When the electronic device 100 determines that the display duration of the notification prompt window is greater than or equal to the sixth time threshold, the electronic device 100 may perform step S1110.
当电子设备100确定通知提示窗的显示时长小于第六时间阈值时,电子设备100可以执行步骤S1103。When the electronic device 100 determines that the display duration of the notification prompt window is less than the sixth time threshold, the electronic device 100 may perform step S1103.
需要说明的是,在本申请实施例中,第六时间阈值也可以写作第五时长。It should be noted that in the embodiment of the present application, the sixth time threshold can also be written as the fifth duration.
S1110,电子设备100取消显示通知提示窗。S1110. The electronic device 100 cancels display of the notification prompt window.
示例性的,电子设备100取消显示通知提示窗之后的界面,可以参考上述图7D和图8C所示的社交应用界面510。For example, the interface after the electronic device 100 cancels display of the notification prompt window can refer to the social application interface 510 shown in FIG. 7D and FIG. 8C.
在一些实施例中,在电子设备100显示有注视反馈之后,当电子设备100未接收到用户的任何操作,且电子设备100检测到用户的视线离开第一区域(或检测到用户的注视区域与第一区域不存在交集)时,电子设备100确定用户持续未注视第一区域的时间长度。当用户持续未注视第一区域的时间长度大于等于第三时间阈值时,电子设备100取消输出注视反馈。当用户持续未注视第一区域的时间长度大于等于第四时间阈值时,电子设备100取消显示通知提示窗。可选的,在电子设备100取消输出注视反馈之后,电子设备100也可以以关闭注视反馈的时刻为起点,确定用户持续未注视第一区域的时间长度不小于第五时间阈值时,电子设备100取消显示通知提示窗。具体方式和界面可以参考上述图8A至图8C所示实施例中的相关内容。In some embodiments, after the electronic device 100 displays gaze feedback, when the electronic device 100 does not receive any operation from the user, and the electronic device 100 detects that the user's gaze leaves the first area (or detects that the user's gaze area is different from the first area). When there is no intersection in the first area), the electronic device 100 determines the length of time that the user continues not to look at the first area. When the user continues not to gaze at the first area for a length of time greater than or equal to the third time threshold, the electronic device 100 cancels outputting gaze feedback. When the length of time that the user continues not to look at the first area is greater than or equal to the fourth time threshold, the electronic device 100 cancels display of the notification prompt window. Optionally, after the electronic device 100 cancels the output of gaze feedback, the electronic device 100 may also use the moment when gaze feedback is turned off as a starting point to determine that the length of time the user has not gazed at the first area is not less than the fifth time threshold. Cancels the notification window. For specific methods and interfaces, please refer to the relevant content in the embodiment shown in FIGS. 8A to 8C.
在一种可能的实现方式中,在上述图11A所示的步骤S1103或步骤S1104之后,电子设备100可以接收并响应于用户针对第二区域的操作2(例如单击操作、双击操作、长按操作,等等),打开第一通知对应的应用软件,并显示该应用软件的界面,例如图6B所示的畅连应用界面600。在本申请的一些实施例中,上述操作2也可以写作第二操作。在另一些实施例中,上述图6A和图6B所示实施例中的操作3也可以写作第二操作。In a possible implementation, after step S1103 or step S1104 shown in FIG. 11A , the electronic device 100 can receive and respond to the user's operation 2 (such as a single click operation, a double-click operation, a long press) on the second area. operation, etc.), open the application software corresponding to the first notification, and display the interface of the application software, such as the Changlian application interface 600 shown in Figure 6B. In some embodiments of the present application, the above operation 2 can also be written as the second operation. In other embodiments, operation 3 in the embodiment shown in FIG. 6A and FIG. 6B can also be written as the second operation.
在一些实施例中,在上述图11A所示的步骤S1103或步骤S1104之后,电子设备100也可以接收并响应于用户针对第二区域的操作3(例如双击操作、长按操作,等等),显示上述图11A所示步骤S1107中的悬浮窗。In some embodiments, after step S1103 or step S1104 shown in FIG. 11A above, the electronic device 100 may also receive and respond to the user's operation 3 on the second area (such as a double-click operation, a long-press operation, etc.), The floating window in step S1107 shown in FIG. 11A is displayed.
这样,可以通过更为简便的操作,快速显示悬浮窗,或者快速打开通知提示窗对应的应用软件,并显示应用软件的界面。缩短了显示悬浮窗或显示应用软件的界面所需的时间。In this way, the floating window can be quickly displayed through simpler operations, or the application software corresponding to the notification prompt window can be quickly opened and the interface of the application software can be displayed. The time required to display a floating window or display an application software interface is shortened.
在一种可能的实现方式中,当用户不想查看第一通知的具体内容时,电子设备100也可以响应于用户针对第二区域的操作4,例如右滑操作,取消显示通知提示窗。具体的操作方式以及界面可以参考上述图7A至图7D所示实施例中的相关内容。In a possible implementation, when the user does not want to view the specific content of the first notification, the electronic device 100 can also cancel the display of the notification prompt window in response to the user's operation 4 on the second area, such as a right sliding operation. For specific operation methods and interfaces, please refer to the relevant content in the embodiment shown in FIGS. 7A to 7D .
在另一种可能的实现方式中,当用户想要稍后查看第一通知的具体内容时,电子设备也可以响应于用户针对第二区域的操作5,例如左滑操作,取消显示通知提示窗,并在一段时间之后再显示该通知提示窗。具体的操作方式以及界面也可以参考上述图7A至图7D所示实施例中的相关内容。In another possible implementation, when the user wants to view the specific content of the first notification later, the electronic device can also cancel the display of the notification prompt window in response to the user's operation 5 on the second area, such as a left sliding operation. , and then display the notification prompt window after a period of time. For specific operation methods and interfaces, please refer to the relevant content in the embodiment shown in FIGS. 7A to 7D .
需要说明的是,在本申请的一些实施例中,操作4也可以写作第三操作,在另一些实施例中,操作5也可以写作第三操作。It should be noted that in some embodiments of the present application, operation 4 can also be written as the third operation, and in other embodiments, operation 5 can also be written as the third operation.
在一些实施例中,当电子设备100输出注视反馈时,电子设备100只能响应于注视操作集合中的操作,注视操作集合包括注视反馈状态下定义的一个或多个操作(例如上述操作1至操作5)。In some embodiments, when the electronic device 100 outputs gaze feedback, the electronic device 100 can only respond to operations in the gaze operation set. The gaze operation set includes one or more operations defined in the gaze feedback state (for example, the above-mentioned operations 1 to Operation 5).
在另一些实施例中,当电子设备100输出注视反馈时,电子设备100也可以响应于注视操作集合之外的操作,执行该操作对应的响应。例如在上述图9A至图9B所示实施例中,电子设备100可以响应于用户针对电子设备100的上滑操作,向下滑动社交应用界面510中的一个或多个图文内容,等等。In other embodiments, when the electronic device 100 outputs gaze feedback, the electronic device 100 may also respond to an operation outside the gaze operation set and perform a response corresponding to the operation. For example, in the embodiment shown in FIGS. 9A to 9B , the electronic device 100 may slide down one or more graphic contents in the social application interface 510 in response to the user's upward sliding operation on the electronic device 100 , and so on.
如图12所示,电子设备100在显示有注视反馈的情况下响应于操作7的具体流程可以包括以下步骤:As shown in Figure 12, the specific process of the electronic device 100 responding to operation 7 when gaze feedback is displayed may include the following steps:
S1201,在电子设备100显示第一界面的情况下,电子设备100获取第一应用的第一通知,在触控屏的第一区域显示通知提示窗。
S1201. When the electronic device 100 displays the first interface, the electronic device 100 obtains the first notification of the first application and displays a notification prompt window in the first area of the touch screen.
S1202,电子设备100获取注视信息,注视信息包括用户的面部图像、用户的眼部图像、眼动信息、眼电信息中的任一项或多项。S1202. The electronic device 100 obtains gaze information, which includes any one or more of the user's facial image, the user's eye image, eye movement information, and electrooculogram information.
S1203,电子设备100基于注视信息判断用户的注视区域是否位于第一区域。S1203: The electronic device 100 determines whether the user's gaze area is located in the first area based on the gaze information.
当用户的注视区域位于第一区域时,电子设备100执行步骤S1204或步骤S1205。When the user's gaze area is located in the first area, the electronic device 100 executes step S1204 or step S1205.
当用户的注视区域不位于第一区域时,电子设备100执行步骤S1209或步骤S1210。When the user's gaze area is not located in the first area, the electronic device 100 executes step S1209 or step S1210.
可选的,电子设备100还可以基于用户注视第一区域的时长与第一时间阈值的关系确定之后执行的步骤。Optionally, the electronic device 100 may also determine steps to be performed later based on the relationship between the duration of the user's gaze on the first area and the first time threshold.
当用户注视第一区域的时间大于等于第一时间阈值时,电子设备100执行下述步骤S1205。When the time the user looks at the first area is greater than or equal to the first time threshold, the electronic device 100 executes the following step S1205.
当用户注视第一区域的时间小于第一时间阈值时,电子设备100执行下述步骤S1210。When the time the user looks at the first area is less than the first time threshold, the electronic device 100 executes the following step S1210.
S1204,电子设备100输出注视反馈。S1204. The electronic device 100 outputs gaze feedback.
步骤S1204为可选步骤。Step S1204 is an optional step.
S1205,电子设备100接收用户针对电子设备100上第二区域的操作7。S1205: The electronic device 100 receives the user's operation 7 on the second area on the electronic device 100.
S1206,电子设备100判断操作7是否属于注视操作集合。S1206: The electronic device 100 determines whether operation 7 belongs to the gaze operation set.
注视操作集合可以包括在电子设备100输出注视反馈的情况下针对通知提示窗的有效操作,例如上述实施例中的操作1、操作2、操作3、操作4和操作5等操作中的一项或多项。可以理解的是,注视操作集合可以预先存储在电子设备100中。The set of gaze operations may include valid operations for the notification prompt window when the electronic device 100 outputs gaze feedback, such as one of operations 1, 2, 3, 4, and 5 in the above embodiment, or Multiple items. It can be understood that the set of gaze operations may be stored in the electronic device 100 in advance.
当电子设备100确定操作7不属于注视操作集合时,电子设备100执行下述步骤S1207。When the electronic device 100 determines that operation 7 does not belong to the gaze operation set, the electronic device 100 performs the following step S1207.
当电子设备100确定操作7属于注视操作集合时,电子设备100执行下述步骤S1208。When the electronic device 100 determines that operation 7 belongs to the gaze operation set, the electronic device 100 performs the following step S1208.
S1207,电子设备100确定操作7的作用对象为第一界面,执行操作7对应的响应。S1207: The electronic device 100 determines that the action object of operation 7 is the first interface, and executes a response corresponding to operation 7.
在这种情况下,电子设备100可以响应于操作7,执行第一界面的功能(例如第一界面的第一功能)。其中,操作7以及第一界面的功能的具体内容可以参考图9A至图9B所示实施例中操作6以及第一界面的功能的相关描述,此处不再赘述。In this case, the electronic device 100 may perform the function of the first interface (eg, the first function of the first interface) in response to operation 7 . For the specific content of operation 7 and the functions of the first interface, reference can be made to the relevant descriptions of operation 6 and the functions of the first interface in the embodiment shown in FIG. 9A to FIG. 9B , which will not be described again here.
S1208,电子设备100确定操作7的作用对象为通知提示窗,并执行操作7对应的响应。S1208: The electronic device 100 determines that the action object of operation 7 is the notification prompt window, and executes a response corresponding to operation 7.
在这种情况下,电子设备100可以响应于操作7,显示第一应用的界面,显示悬浮窗或取消显示通知提示窗,等等。In this case, the electronic device 100 may respond to operation 7 by displaying the interface of the first application, displaying the floating window, or canceling the display of the notification prompt window, and so on.
S1209,电子设备100判断用户未注视第一区域的时长是否大于等于第四时间阈值。S1209: The electronic device 100 determines whether the time period in which the user has not looked at the first area is greater than or equal to the fourth time threshold.
步骤S1209为可选步骤。在本申请实施例中,用户未注视第一区域是指用户的注视区域与第一区域没有交集,即用户的注视区域不位于第一区域。Step S1209 is an optional step. In this embodiment of the present application, the fact that the user does not focus on the first area means that the user's focus area does not intersect with the first area, that is, the user's focus area is not located in the first area.
当用户未注视第一区域的时长大于等于第四时间阈值时,电子设备100执行下述步骤S1211。When the user does not gaze at the first area for longer than or equal to the fourth time threshold, the electronic device 100 executes the following step S1211.
当用户未注视第一区域的时长小于第四时间阈值时,电子设备100执行下述步骤S1210。When the time period in which the user does not look at the first area is less than the fourth time threshold, the electronic device 100 performs the following step S1210.
S1210,电子设备100判断通知提示窗的显示时长是否大于等于第六时间阈值。S1210: The electronic device 100 determines whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.
当通知提示窗的显示时长大于等于第六时间阈值时,电子设备100执行下述步骤S1211。When the display duration of the notification prompt window is greater than or equal to the sixth time threshold, the electronic device 100 executes the following step S1211.
当通知提示窗的显示时长小于第六时间阈值时,电子设备100执行上述步骤S1203。When the display duration of the notification prompt window is less than the sixth time threshold, the electronic device 100 executes the above step S1203.
S1211,电子设备100取消显示通知提示窗。S1211. The electronic device 100 cancels display of the notification prompt window.
图12所示步骤S1201至步骤S1204、步骤S1209至步骤S1211的详细内容可以参考上述图11A所示实施例中的相关内容,此处不再赘述。For details of steps S1201 to S1204 and steps S1209 to S1211 shown in FIG. 12 , please refer to the relevant content in the embodiment shown in FIG. 11A , and will not be described again here.
这样,即使电子设备100输出注视反馈,也不影响电子设备100执行的其他功能。而且,由于操作7不属于注视反馈状态下定义的一个或多个操作,电子设备100不会发生误判。In this way, even if the electronic device 100 outputs gaze feedback, other functions performed by the electronic device 100 will not be affected. Moreover, since operation 7 does not belong to one or more operations defined in the gaze feedback state, the electronic device 100 will not cause misjudgment.
需要说明的是,在一些实施例中,例如当操作7不属于注视操作集合时,操作7也可以写作第四操作。在一些实施例中,第四操作也可以包括上述图9A所示实施例中的操作6。It should be noted that in some embodiments, for example, when operation 7 does not belong to the gaze operation set, operation 7 can also be written as the fourth operation. In some embodiments, the fourth operation may also include operation 6 in the embodiment shown in FIG. 9A.
本申请实施例中还提供可一种由上述图2B所示的系统10执行的通知查看方法。系统10可以包括电子设备100和电子设备200。The embodiment of the present application also provides a notification viewing method that can be executed by the system 10 shown in FIG. 2B. System 10 may include electronic device 100 and electronic device 200 .
如图13所示,系统10执行通知查看方法的具体流程可以包括以下步骤:As shown in Figure 13, the specific process of the system 10 executing the notification viewing method may include the following steps:
S1301,在电子设备100显示第一界面的情况下,电子设备100获取第一应用的第一通知,在触控屏的第一区域显示通知提示窗。S1301. When the electronic device 100 displays the first interface, the electronic device 100 obtains the first notification of the first application and displays a notification prompt window in the first area of the touch screen.
S1302,电子设备100向电子设备200发送请求1,请求电子设备200发送注视信息。S1302: The electronic device 100 sends request 1 to the electronic device 200, requesting the electronic device 200 to send gaze information.
步骤S1302为可选步骤。Step S1302 is an optional step.
响应于第一通知,电子设备100还可以向电子设备200发送请求1,请求电子设备200向电子设备100
发送注视信息。In response to the first notification, the electronic device 100 may also send a request 1 to the electronic device 200, requesting the electronic device 200 to send a request to the electronic device 100. Send gaze messages.
S1303,电子设备200获取用户的注视信息,注视信息包括用户的面部图像、用户的眼部图像、眼动信息、眼电信息中的任一项或多项。S1303. The electronic device 200 obtains the user's gaze information. The gaze information includes any one or more of the user's facial image, the user's eye image, eye movement information, and electrooculogram information.
电子设备200可以响应于电子设备100发送的请求1,持续获取注视信息。The electronic device 200 may continuously obtain the gaze information in response to the request 1 sent by the electronic device 100 .
在一些实施例中,电子设备200也可以在接收到电子设备100发送的请求1之前,持续获取注视信息。In some embodiments, the electronic device 200 may also continue to obtain the gaze information before receiving the request 1 sent by the electronic device 100 .
S1304,电子设备200向电子设备100发送注视信息。S1304: The electronic device 200 sends the gaze information to the electronic device 100.
电子设备200可以在获取注视信息之后向电子设备100发送注视信息。The electronic device 200 may send the gaze information to the electronic device 100 after acquiring the gaze information.
需要说明的是,步骤S1303和步骤S1304都是持续性步骤,即在整个通知查看过程中,电子设备200持续获取注视信息,并持续向电子设备100发送注视信息。It should be noted that steps S1303 and S1304 are both continuous steps, that is, during the entire notification viewing process, the electronic device 200 continues to obtain gaze information and continues to send gaze information to the electronic device 100 .
在一些实施例中,电子设备200也可以以固定的时间间隔向电子设备100发送电子设备200获取的最新采集的注视信息。In some embodiments, the electronic device 200 may also send the latest collected gaze information obtained by the electronic device 200 to the electronic device 100 at fixed time intervals.
S1305,电子设备100获取电子设备100与电子设备200之间的位置关系。S1305: The electronic device 100 obtains the positional relationship between the electronic device 100 and the electronic device 200.
在一些实施例中,电子设备100可以通过前置摄像头获取电子设备200的图像,并基于电子设备200的图像确定出电子设备200与电子设备100之间的位置关系。In some embodiments, the electronic device 100 can acquire an image of the electronic device 200 through a front-facing camera, and determine the positional relationship between the electronic device 200 and the electronic device 100 based on the image of the electronic device 200 .
在一种可能的实现方式中,电子设备100也可以向电子设备200发送请求2,请求2用于请求电子设备200向电子设备100发送电子设备100的图像信息。电子设备100在接收到电子设备200发送的电子设备100的图像信息之后,可以基于电子设备100的图像信息,确定出电子设备100与电子设备200之间的位置关系。In a possible implementation, the electronic device 100 may also send request 2 to the electronic device 200 , where the request 2 is used to request the electronic device 200 to send the image information of the electronic device 100 to the electronic device 100 . After receiving the image information of the electronic device 100 sent by the electronic device 200, the electronic device 100 can determine the positional relationship between the electronic device 100 and the electronic device 200 based on the image information of the electronic device 100.
在另一种可能的实现方式中,电子设备100也可以基于蓝牙、超声波、wifi等方式测量电子设备100与电子设备200之间的位置关系。In another possible implementation, the electronic device 100 can also measure the positional relationship between the electronic device 100 and the electronic device 200 based on Bluetooth, ultrasound, wifi, etc.
S1306,电子设备100基于注视信息、电子设备100与电子设备200之间的位置关系判断用户的注视区域是否位于第一区域。S1306: The electronic device 100 determines whether the user's gaze area is located in the first area based on the gaze information and the positional relationship between the electronic device 100 and the electronic device 200.
示例性的,以注视信息包括用户的图像信息(用户的面部图像和/或用户的眼部图像)为例,电子设备100判断用户的注视区域是否位于第一区域的具体流程可以包括以下步骤:For example, taking the gaze information including the user's image information (the user's facial image and/or the user's eye image) as an example, the specific process for the electronic device 100 to determine whether the user's gaze area is located in the first area may include the following steps:
1.电子设备100基于用户的图像信息确定用户的视线在第一坐标系下的注视方向,第一坐标系是以用户的面部或眼部的中心点为原点的坐标系。1. The electronic device 100 determines the gaze direction of the user's line of sight in a first coordinate system based on the user's image information. The first coordinate system is a coordinate system with the center point of the user's face or eyes as the origin.
2.电子设备100基于用户的图像信息确定用户的头部相对于电子设备200的位置与姿态。2. The electronic device 100 determines the position and posture of the user's head relative to the electronic device 200 based on the user's image information.
3.电子设备100基于用户的头部相对于电子设备200的位置与姿态、电子设备100与电子设备200之间的位置关系确定用户的头部相对于电子设备100的位置与姿态。3. The electronic device 100 determines the position and posture of the user's head relative to the electronic device 100 based on the position and posture of the user's head relative to the electronic device 200 and the positional relationship between the electronic device 100 and the electronic device 200 .
4.电子设备100基于用户的视线在第一坐标系下的注视方向以及用户的头部相对于电子设备100的前置摄像头的位置与姿态,确定出用户的视线在第二坐标系下的注视方向,第二坐标系是以电子设备100触控屏的中心为原点,以该触控屏的法向量为Z轴的坐标系。4. The electronic device 100 determines the gaze of the user's gaze in the second coordinate system based on the gaze direction of the user's gaze in the first coordinate system and the position and posture of the user's head relative to the front camera of the electronic device 100 direction, the second coordinate system is a coordinate system with the center of the touch screen of the electronic device 100 as the origin and the normal vector of the touch screen as the Z axis.
5.电子设备100基于用户的视线在第二坐标系下的注视方向与触控屏在第二坐标系下的坐标,确定用户的视线与触控屏的交点为第一位置。5. The electronic device 100 determines the intersection point of the user's line of sight and the touch screen as the first position based on the gaze direction of the user's line of sight in the second coordinate system and the coordinates of the touch screen in the second coordinate system.
6.电子设备100基于第一位置确定用户的注视区域。6. The electronic device 100 determines the user's gaze area based on the first location.
7.电子设备100判断用户的注视区域是否位于第一区域。7. The electronic device 100 determines whether the user's gaze area is located in the first area.
需要说明的是,上述方式只是一个范例,在本申请实施例中,还可以采用其他视线估计算法来判断用户的注视区域是否位于第一区域,也可以基于眼动信息或眼电信息来判断用户的注视区域是否位于第一区域,本申请在此不做限定。It should be noted that the above method is only an example. In the embodiment of the present application, other gaze estimation algorithms can also be used to determine whether the user's gaze area is located in the first area, or the user can be determined based on eye movement information or electrooculogram information. Whether the gaze area is located in the first area is not limited in this application.
在一些实施例中,也可以由电子设备200判断用户的注视区域是否位于第一区域,并将判决结果发送给电子设备100。在这种情况下,电子设备200可以基于用户的图像信息以及电子设备100的图像信息确定用户视线与电子设备100触控屏的交点,进而确定出用户的注视区域是否位于第一区域,具体的可以参考上述实施例中的相关内容。In some embodiments, the electronic device 200 may also determine whether the user's gaze area is located in the first area, and send the determination result to the electronic device 100 . In this case, the electronic device 200 can determine the intersection point of the user's line of sight and the touch screen of the electronic device 100 based on the user's image information and the image information of the electronic device 100, and then determine whether the user's gaze area is located in the first area. Specifically, Reference may be made to the relevant content in the above embodiments.
在另一些实施例中,电子设备200在获取注视信息之后,可以基于注视信息确定用户的视线方向,并将用户的视线方向发送给电子设备100。电子设备100在接收到用户的视线方向之后,可以基于用户的视
线方向以及电子设备100的屏幕相对于电子设备200的位置关系确定用户的注视区域是否位于第一区域。In other embodiments, after acquiring the gaze information, the electronic device 200 may determine the user's gaze direction based on the gaze information, and send the user's gaze direction to the electronic device 100 . After receiving the user's line of sight direction, the electronic device 100 may The line direction and the positional relationship of the screen of the electronic device 100 relative to the electronic device 200 determine whether the user's gaze area is located in the first area.
可选的,电子设备100还可以基于用户注视第一区域的时长与第一时间阈值的关系确定之后执行的步骤。Optionally, the electronic device 100 may also determine steps to be performed later based on the relationship between the duration of the user's gaze on the first area and the first time threshold.
示例性的,当用户注视第一区域的时长大于等于第一时间阈值时,电子设备100执行下述步骤S1307或步骤S1308。For example, when the duration of the user's gaze on the first area is greater than or equal to the first time threshold, the electronic device 100 performs the following step S1307 or step S1308.
当用户注视第一区域的时长小于第一时间阈值时,电子设备100执行下述步骤S1311或步骤S1312。When the duration of the user's gaze on the first area is less than the first time threshold, the electronic device 100 executes the following step S1311 or step S1312.
S1307,电子设备100输出注视反馈。S1307: The electronic device 100 outputs gaze feedback.
步骤S1307为可选步骤。Step S1307 is an optional step.
S1308,电子设备100接收用户针对电子设备100上第一区域以外区域的操作1。S1308: The electronic device 100 receives the user's operation 1 for an area other than the first area on the electronic device 100.
S1309,响应于操作1,电子设备100将通知提示窗在第一界面中的显示位置向下移动。S1309, in response to operation 1, the electronic device 100 moves the display position of the notification prompt window downward in the first interface.
步骤S1309为可选步骤。Step S1309 is an optional step.
S1310,响应于操作1,电子设备100显示悬浮窗,悬浮窗用于显示第一应用的界面,该第一应用的界面包括第一通知的具体内容。S1310. In response to operation 1, the electronic device 100 displays a floating window. The floating window is used to display the interface of the first application. The interface of the first application includes the specific content of the first notification.
S1311,电子设备100判断用户未注视第一区域的时长是否大于等于第四时间阈值。S1311. The electronic device 100 determines whether the time period in which the user has not looked at the first area is greater than or equal to the fourth time threshold.
步骤S1311为可选步骤。在本申请实施例中,用户未注视第一区域是指用户的注视区域与第一区域没有交集,即用户的注视区域不位于第一区域。Step S1311 is an optional step. In this embodiment of the present application, the fact that the user does not focus on the first area means that the user's focus area does not intersect with the first area, that is, the user's focus area is not located in the first area.
当用户未注视第一区域的时长大于等于第四时间阈值时,电子设备100执行下述步骤S1313。When the user does not gaze at the first area for longer than or equal to the fourth time threshold, the electronic device 100 executes the following step S1313.
当用户未注视第一区域的时长小于第四时间阈值时,电子设备100执行下述步骤S1312。When the time period in which the user does not look at the first area is less than the fourth time threshold, the electronic device 100 executes the following step S1312.
S1312,电子设备100判断通知提示窗的显示时长是否大于等于第六时间阈值。S1312: The electronic device 100 determines whether the display duration of the notification prompt window is greater than or equal to the sixth time threshold.
当通知提示窗的显示时长大于等于第六时间阈值时,电子设备100执行下述步骤S1313。When the display duration of the notification prompt window is greater than or equal to the sixth time threshold, the electronic device 100 executes the following step S1313.
当通知提示窗的显示时长小于第六时间阈值时,电子设备100执行上述步骤S1306。When the display duration of the notification prompt window is less than the sixth time threshold, the electronic device 100 executes the above step S1306.
S1313,电子设备100取消显示通知提示窗。S1313. The electronic device 100 cancels display of the notification prompt window.
图13所示步骤S1301、步骤S1307至步骤S1313的详细内容可以参考上述图11A所示实施例中的相关内容,此处不再赘述。For details of step S1301, step S1307 to step S1313 shown in Figure 13, reference can be made to the relevant content in the above embodiment shown in Figure 11A, and will not be described again here.
可以理解的是,图11A、图12以及图13所示实施例只是以电子设备100接收到第一应用的第一通知为例对通知查看方法进行示例性说明,在本申请实施例中,电子设备100也可以接收到第一应用的其他通知(例如第二通知)或是其他应用的通知(例如第二应用的第三通知),并采用上述通知查看方法中的任一项查看通知、或取消通知提示窗,等等。本申请在此不做限定。It can be understood that the embodiments shown in FIG. 11A, FIG. 12 and FIG. 13 only illustrate the notification viewing method by taking the electronic device 100 receiving the first notification of the first application as an example. In the embodiment of the present application, the electronic device 100 receives the first notification of the first application as an example. The device 100 may also receive other notifications from the first application (for example, the second notification) or notifications from other applications (for example, the third notification from the second application), and use any one of the above notification viewing methods to view the notification, or Cancel the notification prompt window, etc. This application is not limited here.
此外,需要说明的是,本申请实施例中,通知提示窗可以是第一通知提示窗,也可以是第二通知提示窗或第三通知提示窗等等,悬浮窗可以指第一悬浮窗,也可以是第二悬浮窗或第三悬浮窗,等等。电子设备100也可以写作第一电子设备,电子设备200也可以写作第二电子设备。In addition, it should be noted that in the embodiment of the present application, the notification prompt window may be a first notification prompt window, a second notification prompt window, a third notification prompt window, etc., and the floating window may refer to the first floating window. It can also be a second floating window or a third floating window, and so on. The electronic device 100 may also be referred to as a first electronic device, and the electronic device 200 may also be referred to as a second electronic device.
下面介绍本申请实施例中提供的一种电子设备100的功能模块图。The following introduces the functional module diagram of an electronic device 100 provided in the embodiment of the present application.
如图14所示,电子设备100可以包括通知提示模块1401、显示模块1402、注视信息获取模块1403、视线估计模块1404以及手势识别模块1405。As shown in FIG. 14 , the electronic device 100 may include a notification prompt module 1401, a display module 1402, a gaze information acquisition module 1403, a line of sight estimation module 1404, and a gesture recognition module 1405.
其中,通知提示模块1401可以接收第一通知,并向显示模块1402、注视信息获取模块1403发送通知提醒。Among them, the notification prompt module 1401 can receive the first notification and send a notification reminder to the display module 1402 and the gaze information acquisition module 1403.
显示模块1402可以响应于通知提示模块1401发送的通知提醒,显示通知提示窗。显示模块1402还可以基于视线估计模块1404发送的视线估计结果,显示或关闭注视反馈,亦或是关闭通知提示窗,等等。例如,当显示模块1402接收到的视线估计结果为用户注视第一区域的时间不小于第一时间阈值时,显示模块1402可以输出注视反馈,改变界面的背景颜色或通知提示窗的颜色。又例如,当显示模块1402接收到的视线估计结果为用户未注视第一区域的时间不小于第四时间阈值时,显示模块1402可以取消显示通知提示窗,等等。显示模块1402还可以基于手势识别模块1405发送的操作识别结果(例如,操作1、操作2,等等),执行对应响应。The display module 1402 may display the notification prompt window in response to the notification reminder sent by the notification prompt module 1401. The display module 1402 can also display or close gaze feedback based on the gaze estimation result sent by the gaze estimation module 1404, or close the notification prompt window, and so on. For example, when the gaze estimation result received by the display module 1402 is that the time the user gazes at the first area is not less than the first time threshold, the display module 1402 can output gaze feedback and change the background color of the interface or the color of the notification prompt window. For another example, when the gaze estimation result received by the display module 1402 is that the time the user has not looked at the first area is not less than the fourth time threshold, the display module 1402 can cancel the display of the notification prompt window, and so on. The display module 1402 may also execute a corresponding response based on the operation recognition result (eg, operation 1, operation 2, etc.) sent by the gesture recognition module 1405.
注视信息获取模块1403可获取注视信息,例如用户的图像信息、眼动信息或眼电信息。注视信息获取模块1403可以响应于通知提示模块1401发送的通知提醒,持续获取注视信息。注视信息获取模块1403还可以向视线估计模块1404发送注视信息。The gaze information acquisition module 1403 can acquire gaze information, such as the user's image information, eye movement information, or electrooculogram information. The gaze information acquisition module 1403 may respond to the notification reminder sent by the notification reminder module 1401 and continue to obtain gaze information. The gaze information acquisition module 1403 may also send gaze information to the gaze estimation module 1404.
在一些实施例中,注视信息获取模块1403还可以获取电子设备100的图像信息或者电子设备200的图像信息。
In some embodiments, the gaze information acquisition module 1403 may also acquire image information of the electronic device 100 or image information of the electronic device 200 .
视线估计模块1404可以基于注视信息确定用户的注视区域是否位于第一区域。在一些实施例中,视线估计模块1404也可以用于确定用户注视第一区域的时间是否不小于第一时间阈值。在另一些实施例中,视线估计模块1404也可以用于确定用户视线离开第一区域的时间是否不小于第三时间阈值,还可以用于确定用户未注视第一区域的时间是否不小于第四时间阈值,等等。视线估计模块1404可以将上述视线估计结果(例如,用户注视第一区域,用户的注视区域不位于第一区域,用户注视第一区域的时间大于第一时间阈值,等等)发送给显示模块1402。The gaze estimation module 1404 may determine whether the user's gaze area is located in the first area based on the gaze information. In some embodiments, the gaze estimation module 1404 may also be used to determine whether the time the user looks at the first area is not less than a first time threshold. In other embodiments, the gaze estimation module 1404 can also be used to determine whether the time when the user's gaze leaves the first area is not less than a third time threshold, and can also be used to determine whether the time when the user does not look at the first area is not less than a fourth time threshold. time threshold, etc. The gaze estimation module 1404 may send the above gaze estimation results (for example, the user gazes at the first area, the user's gaze area is not located in the first area, the user gazes at the first area for longer than the first time threshold, etc.) to the display module 1402 .
手势识别模块1405可以接收并识别用户的操作。例如,手势识别模块1405可以接收并识别用户的操作1,并将操作识别结果发送至显示模块1402,指示显示模块1402执行对应的响应。The gesture recognition module 1405 can receive and recognize the user's operation. For example, the gesture recognition module 1405 can receive and recognize the user's operation 1, and send the operation recognition result to the display module 1402, instructing the display module 1402 to perform a corresponding response.
需要说明的是,上述图2B所示系统10的功能模块图也可以参考图14所示电子设备100的功能模块图。在系统10中,上述图14中的注视信息获取模块1403可以位于电子设备200中,其余功能模块则位于电子设备100中。在一些实施例中,视线估计模块1404也可以位于电子设备200中。It should be noted that the functional module diagram of the system 10 shown in FIG. 2B may also refer to the functional module diagram of the electronic device 100 shown in FIG. 14 . In the system 10 , the above-mentioned gaze information acquisition module 1403 in FIG. 14 may be located in the electronic device 200 , and the remaining functional modules are located in the electronic device 100 . In some embodiments, gaze estimation module 1404 may also be located in electronic device 200.
可以理解的是,上述图14所示的电子设备100的功能模块结构只是一个范例,电子设备100还可以包括比图14所示的更多或更少的功能模块,本申请在此不做限定。It can be understood that the functional module structure of the electronic device 100 shown in FIG. 14 is just an example. The electronic device 100 may also include more or fewer functional modules than shown in FIG. 14 , which is not limited in this application. .
本申请的各实施方式可以任意进行组合,以实现不同的技术效果。The various embodiments of the present application can be combined arbitrarily to achieve different technical effects.
在上述实施例中,可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时,可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时,全部或部分地产生按照本申请所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一个计算机可读存储介质传输,例如,所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如,DVD)、或者半导体介质(例如固态硬盘(solid state disk,SSD))等。In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in this application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, e.g., the computer instructions may be transferred from a website, computer, server, or data center Transmission to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state disk (SSD)), etc.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,该流程可以由计算机程序来指令相关的硬件完成,该程序可存储于计算机可读取存储介质中,该程序在执行时,可包括如上述各方法实施例的流程。而前述的存储介质包括:ROM或随机存储记忆体RAM、磁碟或者光盘等各种可存储程序代码的介质。Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments are implemented. This process can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, , may include the processes of the above method embodiments. The aforementioned storage media include: ROM, random access memory (RAM), magnetic disks, optical disks and other media that can store program codes.
总之,以上所述仅为本发明技术方案的实施例而已,并非用于限定本发明的保护范围。凡根据本发明的揭露,所作的任何修改、等同替换、改进等,均应包含在本发明的保护范围之内。
In short, the above descriptions are only examples of the technical solutions of the present invention and are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made based on the disclosure of the present invention shall be included in the protection scope of the present invention.
Claims (20)
- 一种通知查看方法,其特征在于,包括:A notification viewing method is characterized by including:第一电子设备显示第一界面;The first electronic device displays the first interface;所述第一电子设备获取第一应用的第一通知;The first electronic device obtains a first notification of the first application;响应于所述第一通知,所述第一电子设备在触控屏的第一区域显示第一通知提示窗,所述第一通知提示窗中包括所述第一通知的部分内容或全部内容;In response to the first notification, the first electronic device displays a first notification prompt window in the first area of the touch screen, and the first notification prompt window includes part or all of the content of the first notification;在所述第一电子设备确定用户的注视区域位于所述第一区域之后,所述第一电子设备接收用户针对第二区域的第一操作,所述第二区域为所述触控屏上所述第一区域以外的区域;After the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's first operation on a second area, and the second area is located on the touch screen. Areas other than the first area mentioned above;在所述第一电子设备检测到所述第一操作之后,所述第一电子设备在所述第一界面上覆盖显示第一悬浮窗,所述第一悬浮窗中包括所述第一应用的界面,所述第一应用的界面中包括所述第一通知的内容。After the first electronic device detects the first operation, the first electronic device overlays and displays a first floating window on the first interface, and the first floating window includes the first application. An interface, the interface of the first application includes the content of the first notification.
- 根据权利要求1所述的方法,其特征在于,在所述第一电子设备确定用户的注视区域位于所述第一区域之后,所述方法还包括:The method of claim 1, wherein after the first electronic device determines that the user's gaze area is located in the first area, the method further includes:所述第一电子设备输出注视反馈,所述注视反馈用于提示用户的注视区域位于所述第一区域。The first electronic device outputs gaze feedback, and the gaze feedback is used to prompt the user that the gaze area is located in the first area.
- 根据权利要求2所述的方法,其特征在于,所述输出注视反馈包括以下任一项或多项:改变所述第一界面的显示颜色、改变所述第一通知提示窗的显示颜色、改变所述第一通知提示窗中控件的颜色、点亮所述第一电子设备的指示灯、振动和播放提示音。The method of claim 2, wherein the outputting gaze feedback includes any one or more of the following: changing the display color of the first interface, changing the display color of the first notification prompt window, changing The color of the control in the first notification prompt window, lighting of the indicator light of the first electronic device, vibration and playing of prompt sounds.
- 根据权利要求2或3所述的方法,其特征在于,所述第一电子设备输出注视反馈,具体包括:The method according to claim 2 or 3, characterized in that the first electronic device outputs gaze feedback, specifically including:当所述第一电子设备检测到用户注视所述第一区域的时长大于第一时长时,所述第一电子设备输出所述注视反馈。When the first electronic device detects that the user gazes at the first area for longer than a first duration, the first electronic device outputs the gaze feedback.
- 根据权利要求1-4中任一项所述的方法,其特征在于,所述第一操作为滑动操作;The method according to any one of claims 1-4, characterized in that the first operation is a sliding operation;在所述第一电子设备接收用户针对第二区域的第一操作之后,所述方法还包括:After the first electronic device receives the user's first operation on the second area, the method further includes:当所述第一操作在所述触控屏上的移动距离大于第一移动距离时,所述第一电子设备移动所述第一通知提示窗的显示位置;When the movement distance of the first operation on the touch screen is greater than the first movement distance, the first electronic device moves the display position of the first notification prompt window;当所述第一操作在所述触控屏上的移动距离大于第二移动距离时,所述第一电子设备将所述第一通知提示窗展开成所述第一通知提示窗,所述第二移动距离大于所述第一移动距离。When the movement distance of the first operation on the touch screen is greater than the second movement distance, the first electronic device expands the first notification prompt window into the first notification prompt window. The second moving distance is greater than the first moving distance.
- 根据权利要求1-4中任一项所述的方法,其特征在于,所述第一操作为滑动操作;The method according to any one of claims 1-4, characterized in that the first operation is a sliding operation;在所述第一电子设备接收用户针对第二区域的第一操作之后,所述方法还包括:After the first electronic device receives the user's first operation on the second area, the method further includes:当所述第一操作的持续时间大于第二时长时,所述第一电子设备移动所述第一通知提示窗的显示位置;When the duration of the first operation is greater than the second duration, the first electronic device moves the display position of the first notification prompt window;当所述第一操作的持续时间大于第三时长时,所述第一电子设备将所述第一通知提示窗展开成所述第一通知提示窗,所述第三时长大于所述第二时长。When the duration of the first operation is greater than a third duration, the first electronic device expands the first notification prompt window into the first notification prompt window, and the third duration is greater than the second duration. .
- 根据权利要求1-6中任一项所述的方法,其特征在于,所述方法还包括:The method according to any one of claims 1-6, characterized in that the method further includes:在所述第一电子设备确定用户的注视区域位于所述第一区域之后,所述第一电子设备接收用户针对第二区域的第二操作;After the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives a second operation from the user for the second area;响应于所述第二操作,所述第一电子设备显示所述第一应用的界面。In response to the second operation, the first electronic device displays an interface of the first application.
- 根据权利要求1-6中任一项所述的方法,其特征在于,所述方法还包括:The method according to any one of claims 1-6, characterized in that the method further includes:在所述第一电子设备确定用户的注视区域位于所述第一区域之后,所述第一电子设备接收用户针对第二区域的第三操作;After the first electronic device determines that the user's gaze area is located in the first area, the first electronic device receives the user's third operation for the second area;响应于所述第三操作,所述第一电子设备取消显示所述第一通知提示窗。In response to the third operation, the first electronic device cancels display of the first notification prompt window.
- 根据权利要求8所述的方法,其特征在于,在所述第一电子设备取消显示所述第一通知提示窗之后,所述方法还包括:The method according to claim 8, characterized in that, after the first electronic device cancels display of the first notification prompt window, the method further includes:所述第一电子设备在经过第四时长之后重新显示所述第一通知提示窗; The first electronic device redisplays the first notification prompt window after a fourth period of time;或,or,所述第一电子设备显示悬浮球;The first electronic device displays a floating ball;所述第一电子设备接收用户针对悬浮球的输入;The first electronic device receives a user's input regarding the floating ball;响应于用户针对悬浮球的输入,所述第一电子设备显示所述第一通知提示窗或所述第一悬浮窗。In response to the user's input on the floating ball, the first electronic device displays the first notification prompt window or the first floating window.
- 根据权利要求1-9中任一项所述的方法,其特征在于,在所述第一电子设备确定用户的注视区域位于所述第一区域之后,所述方法还包括:The method according to any one of claims 1 to 9, characterized in that, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes:当所述第一电子设备接收用户针对第二区域的第四操作时,所述第一电子设备执行所述第一界面对应的第一功能,所述第四操作与所述第一操作不同。When the first electronic device receives the user's fourth operation on the second area, the first electronic device performs the first function corresponding to the first interface, and the fourth operation is different from the first operation.
- 根据权利要求1-10中任一项所述的方法,其特征在于,在所述第一电子设备确定用户的注视区域位于所述第一区域之后,所述方法还包括:The method according to any one of claims 1 to 10, characterized in that, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes:当所述第一电子设备确定用户的注视区域变为所述第二区域,且所述第一通知提示窗的显示时长大于第五时长时,所述第一电子设备取消显示所述第一通知提示窗。When the first electronic device determines that the user's gaze area changes to the second area and the display duration of the first notification prompt window is longer than the fifth duration, the first electronic device cancels display of the first notification. Prompt window.
- 根据权利要求1-11中任一项所述的方法,其特征在于,在所述第一电子设备确定用户的注视区域位于所述第一区域之后,所述方法还包括:The method according to any one of claims 1-11, characterized in that, after the first electronic device determines that the user's gaze area is located in the first area, the method further includes:所述第一电子设备显示操作提示,所述操作提示用于提示用户在所述第二区域对所述第一通知提示窗进行控制。The first electronic device displays an operation prompt, and the operation prompt is used to prompt the user to control the first notification prompt window in the second area.
- 根据权利要求1-12中任一项所述的方法,其特征在于,所述第一区域为所述触控屏的顶部区域。The method according to any one of claims 1-12, wherein the first area is a top area of the touch screen.
- 根据权利要求13所述的方法,其特征在于,所述第二区域为所述触控屏的底部区域。The method of claim 13, wherein the second area is a bottom area of the touch screen.
- 根据权利要求1-14所述的方法,其特征在于,所述方法还包括:The method according to claims 1-14, characterized in that the method further includes:所述第一电子设备获取注视信息,注视信息包括以下任一项或多项:用户的眼部图像、用户的面部图像、眼动信息和眼电信息;The first electronic device obtains gaze information, and the gaze information includes any one or more of the following: the user's eye image, the user's facial image, eye movement information, and electrooculogram information;所述第一电子设备基于注视信息确定用户的注视区域位于所述第一区域。The first electronic device determines that the user's gaze area is located in the first area based on the gaze information.
- 根据权利要求15所述的方法,其特征在于,所述第一电子设备获取注视信息,具体包括:The method according to claim 15, wherein the first electronic device obtains gaze information, specifically including:所述第一电子设备接收第二电子设备发送的所述注视信息。The first electronic device receives the gaze information sent by the second electronic device.
- 根据权利要求1-14所述的方法,其特征在于,所述方法还包括:The method according to claims 1-14, characterized in that the method further includes:所述第一电子设备接收第二电子设备发送的视线估计结果,所述视线估计结果包括所述用户的注视区域位于所述第一区域;The first electronic device receives a line of sight estimation result sent by the second electronic device, where the line of sight estimation result includes that the user's gaze area is located in the first area;所述第一电子设备基于所述视线估计结果确定用户的注视区域位于所述第一区域。The first electronic device determines that the user's gaze area is located in the first area based on the line of sight estimation result.
- 根据权利要求1-14所述的方法,其特征在于,所述方法还包括:The method according to claims 1-14, characterized in that the method further includes:所述第一电子设备接收第二电子设备发送的用户的视线方向;The first electronic device receives the user's line of sight direction sent by the second electronic device;所述第一电子设备基于所述用户的视线方向确定用户的注视区域位于所述第一区域。The first electronic device determines that the user's gaze area is located in the first area based on the user's line of sight direction.
- 一种电子设备,为第一电子设备,其特征在于,包括:一个或多个处理器、一个或多个存储器;所述一个或多个存储器与所述一个或多个处理器耦合,所述一个或多个存储器用于存储计算机程序代码,所述计算机程序代码包括计算机指令,当所述一个或多个处理器执行所述计算机指令时,使得所述终端执行上述权利要求1-18中任一项所述的方法。An electronic device, which is a first electronic device, is characterized in that it includes: one or more processors, and one or more memories; the one or more memories are coupled to the one or more processors, and the One or more memories are used to store computer program codes. The computer program codes include computer instructions. When the one or more processors execute the computer instructions, the terminal performs any of the above claims 1-18. The method described in one item.
- 一种计算机可读存储介质,包括计算机指令,其特征在于,当所述计算机指令在第一电子设备上运行时,使得所述第一电子设备执行上述权利要求1-18中任一项所述的方法。 A computer-readable storage medium including computer instructions, characterized in that when the computer instructions are run on a first electronic device, the first electronic device is caused to execute any one of the above claims 1-18. Methods.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210995037.7 | 2022-08-18 | ||
CN202210995037.7A CN117632326A (en) | 2022-08-18 | 2022-08-18 | Notification viewing method, system and related device |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2024037379A1 true WO2024037379A1 (en) | 2024-02-22 |
Family
ID=89940650
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2023/111722 WO2024037379A1 (en) | 2022-08-18 | 2023-08-08 | Notification checking method and system, and related apparatus |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN117632326A (en) |
WO (1) | WO2024037379A1 (en) |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20150093016A (en) * | 2014-02-06 | 2015-08-17 | 삼성전자주식회사 | Dispaly apparatus and controlling method thereof |
US20150348513A1 (en) * | 2014-05-27 | 2015-12-03 | Lenovo (Singapore) Pte. Ltd. | Gaze based notification placement |
CN105739700A (en) * | 2016-01-29 | 2016-07-06 | 珠海市魅族科技有限公司 | Notice opening method and apparatus |
CN106951093A (en) * | 2017-03-31 | 2017-07-14 | 联想(北京)有限公司 | A kind of data processing method and device |
CN107608514A (en) * | 2017-09-20 | 2018-01-19 | 维沃移动通信有限公司 | Information processing method and mobile terminal |
CN109032444A (en) * | 2018-07-13 | 2018-12-18 | 维沃移动通信有限公司 | A kind of notification message display methods and terminal device |
CN109960412A (en) * | 2019-03-22 | 2019-07-02 | 北京七鑫易维信息技术有限公司 | A kind of method and terminal device based on touch-control adjustment watching area |
WO2022089060A1 (en) * | 2020-10-29 | 2022-05-05 | 华为技术有限公司 | Interface display method and electronic device |
-
2022
- 2022-08-18 CN CN202210995037.7A patent/CN117632326A/en active Pending
-
2023
- 2023-08-08 WO PCT/CN2023/111722 patent/WO2024037379A1/en unknown
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20150093016A (en) * | 2014-02-06 | 2015-08-17 | 삼성전자주식회사 | Dispaly apparatus and controlling method thereof |
US20150348513A1 (en) * | 2014-05-27 | 2015-12-03 | Lenovo (Singapore) Pte. Ltd. | Gaze based notification placement |
CN105739700A (en) * | 2016-01-29 | 2016-07-06 | 珠海市魅族科技有限公司 | Notice opening method and apparatus |
CN106951093A (en) * | 2017-03-31 | 2017-07-14 | 联想(北京)有限公司 | A kind of data processing method and device |
CN107608514A (en) * | 2017-09-20 | 2018-01-19 | 维沃移动通信有限公司 | Information processing method and mobile terminal |
CN109032444A (en) * | 2018-07-13 | 2018-12-18 | 维沃移动通信有限公司 | A kind of notification message display methods and terminal device |
CN109960412A (en) * | 2019-03-22 | 2019-07-02 | 北京七鑫易维信息技术有限公司 | A kind of method and terminal device based on touch-control adjustment watching area |
WO2022089060A1 (en) * | 2020-10-29 | 2022-05-05 | 华为技术有限公司 | Interface display method and electronic device |
Also Published As
Publication number | Publication date |
---|---|
CN117632326A (en) | 2024-03-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11722449B2 (en) | Notification message preview method and electronic device | |
US20220197584A1 (en) | Foldable screen display method and electronic device | |
US11921987B2 (en) | System navigation bar display method, system navigation bar control method, graphical user interface, and electronic device | |
WO2021213164A1 (en) | Application interface interaction method, electronic device, and computer readable storage medium | |
CN109753159B (en) | Method and apparatus for controlling electronic device | |
US20240179237A1 (en) | Screenshot Generating Method, Control Method, and Electronic Device | |
CN108234875B (en) | Shooting display method and device, mobile terminal and storage medium | |
WO2021000881A1 (en) | Screen splitting method and electronic device | |
US20140354538A1 (en) | Method and apparatus for operating notification function in user device | |
US20220150403A1 (en) | Input Method and Electronic Device | |
WO2021063237A1 (en) | Control method for electronic device, and electronic device | |
WO2022053006A1 (en) | Proximity light sensor control method and related device | |
WO2020103091A1 (en) | Touch operation locking method and electronic device | |
WO2022042766A1 (en) | Information display method, terminal device, and computer readable storage medium | |
US20180088664A1 (en) | Method and device for switching environment picture | |
CN111984347A (en) | Interaction processing method, device, equipment and storage medium | |
WO2023222130A1 (en) | Display method and electronic device | |
CN111325746A (en) | Skin detection method and electronic equipment | |
WO2023222128A9 (en) | Display method and electronic device | |
WO2024037379A1 (en) | Notification checking method and system, and related apparatus | |
CN111341317A (en) | Method and device for evaluating awakening audio data, electronic equipment and medium | |
CN114690985B (en) | Display method and electronic equipment | |
US20240288990A1 (en) | Application interface display method and electronic device | |
WO2024037384A1 (en) | Display method for electronic device and electronic device | |
WO2023179259A1 (en) | Information sharing method and related device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23854282 Country of ref document: EP Kind code of ref document: A1 |