CN111553220A - Intelligent device and data processing method - Google Patents
Intelligent device and data processing method Download PDFInfo
- Publication number
- CN111553220A CN111553220A CN202010315153.0A CN202010315153A CN111553220A CN 111553220 A CN111553220 A CN 111553220A CN 202010315153 A CN202010315153 A CN 202010315153A CN 111553220 A CN111553220 A CN 111553220A
- Authority
- CN
- China
- Prior art keywords
- user
- area
- auxiliary
- function
- display
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 12
- 230000001815 facial effect Effects 0.000 claims abstract description 37
- 230000006870 function Effects 0.000 claims description 117
- 239000002537 cosmetic Substances 0.000 claims description 25
- 239000000047 product Substances 0.000 claims description 17
- 238000004891 communication Methods 0.000 claims description 13
- 238000011156 evaluation Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 9
- 230000001502 supplementing effect Effects 0.000 claims description 4
- 239000013589 supplement Substances 0.000 claims description 2
- 238000004458 analytical method Methods 0.000 abstract description 3
- 210000004709 eyebrow Anatomy 0.000 description 40
- 238000000034 method Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 9
- 230000001360 synchronised effect Effects 0.000 description 8
- 210000001508 eye Anatomy 0.000 description 6
- 238000013461 design Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000003796 beauty Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 210000000214 mouth Anatomy 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses an intelligent device and a data processing method, wherein key point information of a face can be determined through analysis of a reference image, so that the facial features of a user can be determined, when the user starts an auxiliary function, auxiliary information which is in line with the characteristics of the user and corresponds to the auxiliary function can be determined according to the facial features of the user, and the auxiliary information is displayed on a display; therefore, the user can quickly and accurately finish makeup, the display has a mirror function, the user can make up according with the characteristics of the user according to the auxiliary information displayed by the display while looking into the mirror, the user demand is met, and the user experience is improved.
Description
Technical Field
The invention relates to the technical field of smart home, in particular to an intelligent device and a data processing method.
Background
In daily life, when a user who needs makeup makes up by facing a mirror, if the makeup technique is not high, makeup according with the characteristics of the user cannot be made up according to the characteristics of the user, and the user who wants to learn makeup also has the problem of difficulty in handling.
Therefore, the technical problem to be solved by the technical staff in the field is how to help users to make up according with their own features during makeup.
Disclosure of Invention
The embodiment of the invention provides intelligent equipment and a data processing method, which are used for helping a user to make up according with the characteristics of the user during makeup.
In a first aspect, an embodiment of the present invention provides an intelligent device, including:
a display configured to display a screen, the display having a mirror function;
an image collector configured to: collecting face information of a user to obtain a reference image and transmitting the reference image to a processor;
the processor configured to:
determining key point information of the face according to the reference image;
determining the facial features according to the key point information;
and when the user starts a preset auxiliary function for providing auxiliary operation for the facial makeup, determining auxiliary information corresponding to the auxiliary function according to the facial features, and displaying the auxiliary information on the display.
In a second aspect, an embodiment of the present invention provides a data processing method, including:
determining key point information of a face according to an acquired reference image comprising face information of a user;
determining the facial features according to the key point information;
and when the auxiliary operation needs to be provided for the facial makeup, determining auxiliary information corresponding to the auxiliary operation according to the facial features.
The invention has the following beneficial effects:
according to the intelligent device and the data processing method provided by the embodiment of the invention, the key point information of the face can be determined through analyzing the reference image, so that the facial features of the user can be determined, when the user starts the auxiliary function, the auxiliary information which is in line with the characteristics of the user and corresponds to the auxiliary function can be determined according to the facial features of the user, and the auxiliary information is displayed on the display; therefore, the user can quickly and accurately finish makeup, the display has a mirror function, the user can make up according with the characteristics of the user according to the auxiliary information displayed by the display while looking into the mirror, the user demand is met, and the user experience is improved.
Drawings
Fig. 1 is a schematic structural diagram of an intelligent device provided in an embodiment of the present invention;
fig. 2 is a schematic diagram of another smart device provided in an embodiment of the present invention;
fig. 3 is a schematic diagram of collected key point information provided in an embodiment of the present invention;
FIG. 4 is a diagram illustrating a display interface provided in an embodiment of the invention;
FIG. 5 is a schematic diagram of another display interface provided in embodiments of the present invention;
FIG. 6 is a schematic diagram of yet another display interface provided in an embodiment of the present invention;
FIG. 7 is a schematic diagram of yet another display interface provided in an embodiment of the invention;
FIG. 8 is a schematic diagram of yet another display interface provided in an embodiment of the invention;
fig. 9 is a schematic structural diagram of another intelligent device provided in the embodiment of the present invention;
fig. 10 is a flowchart of a data processing method provided in an embodiment of the present invention;
FIG. 11 is a flow chart of one embodiment provided in embodiments of the present invention;
FIG. 12 is a flow chart of another embodiment provided in embodiments of the present invention;
fig. 13 is a flowchart of yet another embodiment provided in embodiments of the present invention.
Detailed Description
The following describes in detail a specific implementation of an intelligent device and a data processing method according to an embodiment of the present invention with reference to the accompanying drawings. It should be noted that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
An embodiment of the present invention provides an intelligent device, as shown in fig. 1, including:
a display 10 configured to display a screen, and the display 10 having a mirror function;
an image acquirer 20 configured to: collecting face information of a user to obtain a reference image and transmitting the reference image to the processor 30;
a processor 30 configured to:
determining key point information of the face according to the reference image;
determining facial features according to the key point information;
when the user turns on a preset auxiliary function for providing auxiliary operations for making up the face, auxiliary information corresponding to the auxiliary function is determined according to the facial features and displayed on the display 10.
In the embodiment of the invention, the key point information of the face can be determined through the analysis of the reference image, so that the facial features of the user can be determined, when the user starts the auxiliary function, the auxiliary information which is in line with the characteristics of the user and corresponds to the auxiliary function can be determined according to the facial features of the user, and the auxiliary information is displayed on the display; therefore, the user can quickly and accurately finish makeup, the display has a mirror function, the user can make up according with the characteristics of the user according to the auxiliary information displayed by the display while looking into the mirror, the user demand is met, and the user experience is improved.
It should be noted that, in the embodiment of the present invention, the display needs to have a mirror function, so that a user can realize a mirror function through the display, so as to finish makeup through a mirror; meanwhile, auxiliary information can be displayed through the display, so that the user can be assisted in making up.
Furthermore, the intelligent device in the embodiment of the present invention may be a mirror with a display function (as shown in fig. 2, and a processor is not shown in the figure), or a display device with a mirror function, and the display device may be a device with a display function, such as a mobile terminal or a television, and is not limited herein.
When the intelligent device is a device which has a display function and a mirror function and is not a mobile terminal, taking the cosmetic mirror as an example, due to the fact that the scene applied by the cosmetic mirror has a fixity factor, shopping (introduction in contents later), sharing communication (introduction in contents later), personal information (introduction in contents later) and the like cannot be consulted in real time. Therefore, in order to enable the user not to be limited by the geographic location, the module for implementing shopping, the module for implementing the communication sharing function, and the module for implementing the personal information setting in the processor may be disposed on one side of the mobile terminal (for example, but not limited to, a mobile phone, a tablet computer, and the like), and the display interface of the mobile terminal may be designed similarly to the display interface of the cosmetic mirror, so that the mobile terminal and the cosmetic mirror are adapted, and thus the data of the mobile terminal may be associated with the background data of the cosmetic mirror through a network, and synchronous content updating is implemented.
In addition, the image collector may be an image collecting structure such as a camera, as long as image collection can be achieved, and the specific implementation structure is not specifically limited herein.
In addition, the intelligent device can also comprise a voice collector for collecting voice information sent by a user so as to realize the functions of man-machine interaction, voice control and the like. The voice collector may be, but is not limited to, a microphone array.
Optionally, in the embodiment of the present invention, when determining the key point information of the face according to the reference image, the following manner may be adopted:
the reference image is analyzed and processed by using a PFLD (a Practical Facial Landmark Detector, a face key point detection algorithm).
The algorithm has high accuracy under the complex conditions of unconstrained posture, expression, illumination and shielding, and can effectively solve the problems of geometric constraint and data imbalance; moreover, the receptive field can be enlarged, the global structure of the human face can be better captured, the key points in the image can be accurately positioned, and the image with the key points is shown in fig. 3.
Of course, in practical cases, when determining the key point information of the face according to the reference image, the algorithm is not limited to the above-mentioned PFLD algorithm, and may be other algorithms that can determine the key point information of the face, and is not limited herein.
In specific implementation, in the embodiment of the present invention, the setting may be performed as follows for the display interface of the display:
the display includes a display area including: a mirror area and a regular display area;
the conventional display area is used for displaying function buttons, and the function buttons comprise auxiliary function buttons corresponding to auxiliary functions;
the mirror area is used for providing a mirror function for a user;
wherein the auxiliary information is displayed in the mirror area or the regular display area.
For example, as shown in FIG. 4, in the mirror area (e.g., Q1), the user may look in the area and make up against the mirror; in a normal display area (e.g., Q2, and only a portion of the normal display area is indicated by a dashed box in fig. 4), a screen may be displayed, including function buttons and the like.
It is noted that the area ratio of the mirror area and the normal display area is adjustable, and can be achieved by manually dragging the dashed line L0 (as shown in fig. 4) or by voice control, so as to adapt to the requirements of different functions.
Therefore, the mirror can be seen, the picture can be displayed simultaneously, and auxiliary information is provided for a user during mirror-looking makeup so as to assist in guiding the user to makeup, so that makeup according with the characteristics of the user is formed, and the requirements of different customers are met.
Of course, the design of the display interface of the display is not limited to that shown in fig. 4, and other design manners capable of implementing the above functions may also be adopted, and the design is not limited herein.
Optionally, in this embodiment of the present invention, as shown in fig. 4, the intelligent device further includes: a light supplement structure 40 for supplementing light to make up on the face;
the mirror area Q1 and the normal display area Q2 are aligned in a first direction (e.g., Y direction);
the display area has two first sides that are disposed oppositely and parallel to the first direction, and the light filling structure 40 is disposed near at least one of the first sides.
So, look into the mirror through the mirror surface region at the user, and be in when the darker region of ambient light, can carry out the light filling or polish for the user through the light filling structure, not only can make the user can see more clearly when looking into the mirror, can also obtain better shooting effect when shooing (the content has the introduction later).
Optionally, in the embodiment of the present invention, as shown in fig. 4, the display further includes a frame region Q3 surrounding the display region (e.g., the regions where Q1 and Q2 are located), and the light filling structure 40 is located in the frame region Q3.
Therefore, the light supplementing structure can be prevented from influencing the structure in the display area, and the display effect of the display area is improved.
The functions of the intelligent device are described in a classification manner.
1. An auxiliary function, or so-called makeup operation guide function.
Specifically, in the embodiment of the present invention, the auxiliary operation is: the method comprises the following steps of assisting a user in making up the face, wherein auxiliary information comprises auxiliary lines, auxiliary points and/or character guide information, and an auxiliary function comprises displaying the auxiliary information;
the auxiliary information is displayed in the mirror area;
the area of the mirror area is larger than that of the conventional display area.
For example, as shown in fig. 5, taking the user to make eyebrow makeup as an example, the user may click an eyebrow shaping auxiliary button (e.g., G1), and the processor may determine characteristics of the user's eyebrow based on the captured reference image, and then determine an auxiliary line (not shown) or an auxiliary point (as shown in F1) corresponding to the characteristics of the user's eyebrow, where the user's eyebrow is located, i.e., the auxiliary line or the auxiliary point is superimposed on the user's eyebrow, so that the user may finish making up the eyebrow based on the displayed auxiliary line or the auxiliary point.
Since the auxiliary information is used for assisting and guiding the user to apply makeup, the auxiliary information needs to be in the current mirror surface area and is superposed on the part needing makeup so as to provide assistance and guidance for the user.
Further, as shown in fig. 5, if a represents the display area, Q1 represents the mirror area, and the area of the display area a except the mirror area Q1 is the normal display area, in which case the area of the mirror area Q1 is larger than that of the normal display area, so as to provide a larger field of view for the user.
Therefore, for users with low makeup technology, makeup can be completed quickly and accurately, the makeup can better meet the characteristics of the users, and the requirements of customers can be met; and by increasing the area of the mirror surface area, a user can look into the mirror more conveniently, and a larger makeup career is provided for the user, so that the user can finish makeup conveniently.
In addition, as shown in fig. 5, taking eyebrow transformation as an example, the processor may further recommend an eyebrow type (as shown in G2) corresponding to the user feature for the user according to the determined eyebrow feature of the user, and display the eyebrow type in the regular display area, where the user may select an eyebrow type that the user likes or that corresponds to the current occasion of presence in the regular display area, and after determining the result selected by the user, the processor determines auxiliary information according to the selection result, and displays the auxiliary information in the mirror area.
Thus, optionally, in an embodiment of the invention, the auxiliary function comprises: the sub-functions corresponding to different parts are provided with related recommendation information corresponding to specific parts in the conventional display area;
at this point, the processor is configured to:
determining a plurality of pieces of recommendation information corresponding to the specific parts according to the characteristics of the specific parts;
when the user selects the specific recommendation information from the plurality of recommendation information, the auxiliary information corresponding to the specific recommendation information is determined.
Optionally, in the embodiment of the present invention, a zoom button for performing zoom processing on a specific portion is disposed in the regular display area;
a processor configured to:
when a user selects a zooming button and selects a specific part needing zooming, zooming the specific part;
a display configured to:
in the zoom area in the mirror area, the specific portion after the zoom processing is displayed.
For example, taking the example shown in fig. 6, the normal display area is provided with a zoom button (as shown in G3), and the zoom button may include a zoom-in button and a zoom-out button, only the zoom-in button (as shown in G3) being shown in the figure; after the user clicks the zoom-in button, the user may select a region to be zoomed in, where:
if the intelligent device has the touch screen function, a user can select a part to be amplified through a mirror area in a touch screen, if the user selects eyebrows, the amplified eyebrows are displayed in a single zooming area (as shown in Q4);
or, if the smart device has a voice control function, the user may select the portion to be enlarged through voice control, for example, if the user selects an eyebrow, the enlarged eyebrow is displayed in a separate zoom area (as shown in Q4);
to illustrate, the zoom region (as shown in Q4) may be set separately to avoid interference between the zoom region Q4 and the region Q5, and thus avoid affecting the user's mirror-viewing process in the region Q5.
In this way, the specific part after the zooming processing is displayed in the zooming area, which is helpful for assisting the user to finish the makeup of the specific part, so as to facilitate the makeup of finer makeup.
2. Auxiliary functions, otherwise known as synchronous teaching functions.
Specifically, in the embodiment of the present invention, the auxiliary operation is: providing teaching video operation for a user to make up on the face, wherein the auxiliary information comprises the teaching video, and the auxiliary function comprises displaying the teaching video;
the conventional display area comprises a button area and a video playing area, the function buttons are displayed in the button area, and the teaching video is displayed in the video playing area.
For example, as shown in FIG. 7, the button zones may be represented by the zones indicated by Q21a, Q21b, and Q21c, the video playback zone may be represented by the zone indicated by Q22, and the processor may determine and display a teaching video associated with the facial features of the user in the video playback zone Q22; at this time, the user performs synchronous learning in the mirror surface region Q1 while watching the teaching video, thereby achieving the effect of "learning while watching".
Therefore, teaching guidance can be provided for the user to learn makeup, the effect of learning while seeing is achieved, and the requirement of learning makeup is met.
In practical cases, the recommended teaching video may be a video for full-face makeup, but sometimes the user does not need to learn the technique of full-face makeup, and may be interested only in certain parts, such as eyebrows, lips, eyes, and so on, when:
optionally, in the embodiment of the present invention, a selection button for different parts of the face is provided in the button area, as shown in G4 in fig. 7;
accordingly, a processor configured to:
after the auxiliary information corresponding to the auxiliary function is determined, and when a user selects a specific part from each part, analyzing the determined teaching video, and determining a sub-video corresponding to the specific part in the teaching video;
a display configured to:
and displaying the sub-video in the video playing area.
That is, taking the specific portion selected by the user as an eyebrow as an example, the processor may determine a portion of video content (i.e., a sub-video) related to the eyebrow in the recommended teaching video, and then only the portion of video content related to the eyebrow is displayed in the video playing area, while the other portion of video content is not required to be displayed.
When determining the sub-video, there may be the following ways:
mode 1:
and deleting the video content irrelevant to the specific part, and only keeping the video content relevant to the specific part to form the sub-video.
Mode 2:
and determining the time stamp information of the video content related to the specific part, and directly positioning to the time of the video content related to the specific part when the display displays, and directly skipping or skipping the video content not related to the specific part.
For the above mode 1 and mode 2, selection can be performed according to actual conditions to meet the requirements of different application scenarios, and the flexibility of design is improved.
In this case, of course, when the user selects a specific portion, the user may also refer to the touch screen or the voice control mentioned above for selection, which may specifically refer to the above, and repeated details are not repeated.
Therefore, through the function setting, when the user has insufficient time or does not need to watch the complete teaching video, the video content related to the specific part can be played in a targeted manner according to the selection of the user, and the requirements of the user are met.
Optionally, in this case, the auxiliary information may include auxiliary lines, auxiliary points, and/or text information mentioned in the foregoing, in addition to the teaching video displayed in the video playing area, so that the user can be guided while watching while learning.
For example, for the eyebrow-shaped part, dotted line area assistance can be superposed, so that eyebrow-shaped symmetry and position symmetry can be realized for the eyebrow makeup;
for the eye shadow part, auxiliary dotted line guidance can be added, and the degree of the skill is explained by characters, so that the eye makeup can be accurately positioned by the user;
for the blush part, an auxiliary dotted line can be added to guide the realization of makeup;
and the makeup aid of shadow and highlight can be realized by means of dotted line area division and detailed character identification.
Therefore, the user can be guided and helped when watching and learning, the learning efficiency and speed can be improved, the user can quickly get on the hand, and the experience of the user is improved.
When assisting and guiding the user to make up, the following settings may be performed in addition to displaying the auxiliary lines, auxiliary points, or character information in the mirror area:
the smart device may further include:
a player configured to play voice information;
at this time, the processor may be further configured to:
and according to the determined facial characteristics, determining voice prompt information for performing auxiliary operation on facial makeup, and playing the voice prompt information through a player.
That is to say, except that the auxiliary line, the auxiliary point and the corresponding characters can be displayed on the display, the auxiliary function can be provided in a voice mode, so that the user can be conveniently provided with multi-directional and comprehensive assistance and guidance, and the experience of the user is improved.
Furthermore, it should be further noted that, optionally, in an embodiment of the present invention, the processor is configured to:
and adjusting the display position of the auxiliary information in the display according to the position of the specific part in the reference image acquired by the image acquirer, so that the position of the auxiliary information displayed on the display corresponds to the area where the specific part is located.
In practical situations, for example, when a user applies eyebrow makeup, the body of the user may move, such as moving left and right or moving up and down, so that the position of the eyebrow in the display may change.
Therefore, the face information of the user can be collected in real time (or according to a certain collection period) through the image collector, the position of the eyebrow in the reference image is obtained after the reference image is analyzed and processed, and the position of the auxiliary information to be displayed is adjusted according to the position, so that the display position of the auxiliary information can always correspond to the area of the eyebrow, and accurate and effective assistance and guidance are provided for the user.
3. And (4) a shopping function.
Specifically, in the embodiment of the present invention, when the shopping function is implemented, there may be the following modes:
mode 1: and (4) a scene shopping mode.
Optionally, in an embodiment of the present invention, as shown in fig. 7, the conventional display area further includes a shopping area (shown as Q23), the shopping area is used for: displaying the corresponding cosmetic product in the teaching video;
a processor configured to:
after the auxiliary information corresponding to the auxiliary function is determined, and when the user selects a specific product from the cosmetic products, the transaction information for purchasing the specific product is determined, and the transaction information is subjected to transaction processing.
That is, based on the aforementioned 2 nd function (i.e., the synchronous teaching function), a cosmetic product associated with the cosmetic product involved in the teaching video may be provided on the basis of the aforementioned 2 nd function, as shown in fig. 7, wherein the cosmetic product may include, but is not limited to: cosmetics and cosmetic tools can be directly selected in a shopping area based on the requirements of a user, and are added into a shopping cart for transaction processing.
So, not only can combine together through shopping function and synchronous teaching function, provide abundanter function for the user, can also make shopping function and synchronous teaching function perfect more, provide abundanter and relevant peripheral information of makeup for the user.
Mode 2: shopping mode of the shopping mall.
In the mode 2, the shopping area does not only display the corresponding cosmetic product in the teaching video, but displays various cosmetic products related to the cosmetic, and is not limited by the teaching video, so the shopping function can be understood as a "cosmetic mall" function, as shown in G5 of fig. 7, when the user selects a function button G5 of the "cosmetic mall", the conventional display area can be switched from the current display interface to an interface (not shown) of the "cosmetic mall", and at the interface of the "cosmetic mall", related goods such as cosmetic, skin care, beauty body, and color cosmetic tools can be provided for the user, the user can select from the goods as required, and then the processor can perform transaction processing according to the goods of the user.
Thus, optionally, in an embodiment of the present invention, the regular display area further comprises a shopping area, the shopping area being for: displaying the cosmetic product;
a processor configured to:
after the auxiliary information corresponding to the auxiliary function is determined, and when the user selects a specific product from the cosmetic products, the transaction information for purchasing the specific product is determined, and the transaction information is subjected to transaction processing.
Therefore, the shopping function can meet the requirement of providing richer shopping for the user, and richer information on the periphery related to makeup is provided for the user.
In practical cases, of course, when the shopping function is implemented, the shopping function is not limited to the two modes, and other modes capable of implementing shopping may also be included, and are not limited specifically herein.
4. The function is evaluated.
Optionally, in an embodiment of the present invention, the function buttons further include an evaluation function button (as shown in G6 in fig. 5);
a processor configured to:
when the user selects the evaluation function button, the makeup result of the face is evaluated.
That is, after the user finishes makeup, the user can evaluate the finished makeup of the face according to a preset evaluation rule by selecting an evaluation function button, so as to give an evaluation result to the makeup of the user and provide reference for the user.
The evaluation rule may be set according to actual needs, fashion trends, and other factors, for example, but not limited to, symmetry of makeup such as eyebrows, eyeliners, and the like, and is not limited herein.
Moreover, the evaluation function can be combined with the synchronous teaching function and the makeup operation guidance function, so that richer functions are provided for the user; or the evaluation function may be set separately, and may be specifically set according to actual needs, which is not limited herein.
Therefore, by setting the evaluation function, the user can evaluate the makeup of the user so as to provide reference for subsequently improving the makeup technology.
5. The makeup shares the function of communication.
Specifically, in the embodiment of the present invention, when the makeup sharing communication function is implemented, the following modes may be adopted:
mode 1: static mode.
Optionally, in an embodiment of the present invention, the function buttons further include a sharing communication function button;
a display configured to:
when the user selects the share communication function button, the area ratio of the normal display area and the mirror area is adjusted so that the display area includes only the normal display area.
For example, in the display interface shown in fig. 4, if the user selects the "beauty map" button (as shown in G7), it indicates that the user selects the sharing communication function button, at this time, the display interface of the display adjusts the area of the mirror surface region, so that the entire display region a only includes the normal display region G2, as shown in fig. 8, where the normal display region Q2 may include the sharing communication region (as Q24), and in the sharing communication region Q24, the user may browse the content shared by other users and himself, and at the same time, may perform operations such as approval, comment, or forwarding on the shared content, thereby achieving sharing and communication between different users.
Mode 2: dynamic mode.
Optionally, in an embodiment of the present invention, the function buttons may further include a live function button (not shown), where the display may also be configured to:
when the user selects the live function button, the area ratio of the normal display area and the mirror area is adjusted so that the display area includes only the normal display area.
Moreover, during live broadcasting, operations such as comments given by other users, praise, gift delivery and the like can be displayed, and live broadcasters can also communicate and communicate with other users, so that communication and sharing among different users are realized.
In addition, no matter the mode 1 or the mode 2, when shooting, the intelligent device can also provide video and picture editing functions so as to meet the requirements of users in the aspects of beauty, video clips and the like and provide richer functional experience for the users.
6. And logging in a registration function.
In order to improve the use safety of the intelligent device, a login account can be set, so that each user can only log in the own account and can only see the content in the own account, and the privacy safety of the user is protected.
Therefore, optionally, in the embodiment of the present invention, the normal display area further includes a user display area, as shown in Q25 in fig. 4, and in the user display area Q25, user information (such as an avatar, a user name, a user nickname, a mobile phone number, a mailbox, or other information indicating the identity of the user) that has been currently logged in may be displayed.
When the user logs in the intelligent device for the first time, in the registration process, the reference image comprising the face information of the user can be collected through the image collector, the key point information of the face is determined, then the face feature of the user is determined, and the determined face feature is stored.
After the registration is completed and the user logs in the intelligent device later, the image collector continuously collects the face information of the current user, carries out face recognition, determines whether the face information is matched with the stored user or not, and if the face information is matched with the stored user, the user can directly log in automatically, and the process of filling in an account and a password is skipped, so that the face recognition login is realized.
Certainly, during registration and login, besides face recognition login, fingerprint recognition login, account password login and other modes can be realized, the specific implementation process is similar to the face recognition login process, the registration information is stored after registration, and automatic recognition is performed during subsequent login, so that the login efficiency is improved, and the login time is reduced.
To illustrate the specific registration and registration process, reference may be made to the prior art and will not be described in detail herein.
The above is only described for six functions, and in specific implementation, the smart device may further include other functions related to makeup besides the above six functions.
For example, referring to fig. 4, a skin care reminding function is further included, that is, an information prompting area (as shown in Q26) may be included in the normal display area, and weather information, skin care information, address information (not shown) and the like may be displayed in the information prompting area Q26, so as to provide assistance and guidance for the user in skin care.
For another example, referring to fig. 4, a music playing function is further included, that is, a music playing area (as shown in Q27) may be included in the normal display area, and in the music playing area Q27, information related to music control and display, such as a name of music currently being played, a volume control key, next and previous control keys, and a pause key and a start key, may be displayed, so that a user may play music during the process of practicing makeup, to relieve the user from boring.
For another example, the mobile terminal may further include a connection function, where the connection module may be connected to a wireless network, or the connection module may implement bluetooth connection with the mobile terminal, so as to implement uploading and sharing of data.
For another example, a time display function can be further included, so that a time reference is provided for the user, the user can conveniently give a time prompt when making up, and the experience of the user is improved.
Therefore, the requirements of the user on various aspects can be met, and the requirements of different users can be met, so that the intelligent equipment has wide applicability.
In summary, in combination with the above description of the six functions, in a specific implementation, in an embodiment of the present invention, to implement the above functions, as shown in fig. 9, a smart device may include the following parts:
firstly, inputting a sensor part.
Specifically, the portion includes: a display with a mirror function and a display function, an image collector (such as but not limited to a camera), a voice collector (such as but not limited to a player, a microphone array).
The specific working principle of the display, the image collector and the voice collector can be referred to in the prior art, and detailed description is omitted here.
And II, identifying a part.
Specifically, the portion may include: the device comprises a voice recognition module, a gesture recognition module, a face recognition module and a video content analysis and recognition module.
Wherein the functions of the modules in the identification part can be realized by a processor.
And thirdly, a data part.
Specifically, the portion may include: at least part of the data, database, synchronized with the mobile terminal.
Wherein, the database can include: face data, personal information data, teaching video data, family account data, and the like.
And when the processor works, the processor can read the related data in the database according to the requirement and then correspondingly process the read related data.
Based on the same inventive concept, an embodiment of the present invention provides a data processing method, as shown in fig. 10, including:
s1001, determining key point information of a face according to an acquired reference image comprising face information of a user;
s1002, determining facial features according to the key point information;
when the facial features are determined, the facial features can be determined according to the characteristics of organs such as eyebrows, eyes and mouths, because the face comprises the organs.
Taking the eyebrow as an example, the characteristics of the eyebrow can be reflected by the eyebrow, the eyebrow peak and the eyebrow tail.
S1003, when the auxiliary operation needs to be provided for the face makeup, the auxiliary information corresponding to the auxiliary operation is determined according to the face features.
For specific implementation of the above steps, reference may be made to the above contents, and repeated details are not described herein.
Therefore, by analyzing the reference image, the key point information of the face can be determined, the face characteristics of the user can be further determined, and when auxiliary operation needs to be provided for the user, the auxiliary information which accords with the characteristics of the user and corresponds to the auxiliary operation can be determined according to the face characteristics of the user; therefore, the user can finish makeup quickly and accurately, makeup which better meets the characteristics of the user is formed according to the auxiliary information displayed by the display, the requirements of the user are met, and the user experience is improved.
The data processing method provided by the embodiment of the invention is explained and explained by the specific embodiment.
Take the assisting operation as an example to assist and guide the user to make up the eyebrow, with reference to the schematic diagram shown in fig. 5 and the flowchart shown in fig. 11.
S1101, determining key point information of a face according to the acquired reference image comprising the face information of the user;
s1102, determining facial features according to the key point information;
among them, facial features may include: face shape, eye shape, eyebrow shape, mouth shape, etc.
S1103, when the user selects a function button for assisting the eyebrow shape, determining auxiliary information corresponding to the eyebrow shape according to the eyebrow shape, and displaying the auxiliary information;
and S1104, when the user selects an evaluation function button for evaluating the makeup, evaluating the makeup of the face according to the collected current face information of the user, and displaying an evaluation result.
The auxiliary operation is taken as an example to provide teaching video, and the schematic diagram shown in fig. 7 and the flowchart shown in fig. 12 are combined.
S1201, determining key point information of the face according to the acquired reference image comprising the face information of the user;
s1202, determining facial features according to the key point information;
among them, facial features may include: face shape, eye shape, eyebrow shape, mouth shape, etc.
S1203, when the user selects a function button for providing the teaching video, determining the teaching video corresponding to the facial features according to the facial features;
and S1204, when the user selects the eyebrow button, analyzing the determined teaching video, determining a sub-video related to the eyebrow, and displaying the sub-video.
Taking the login and registration process of the user as an example, the flowchart shown in fig. 13 is combined.
S1301, determining key point information of the face according to the acquired reference image comprising the face information of the user;
s1302, determining facial features according to the key point information;
s1303, judging whether the determined facial features are stored or not; if yes, go to S1304; if not, executing S1305;
s1304, automatic login;
and S1305, storing the determined facial features, prompting the user to fill in account information, and finishing registration.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. A smart device, comprising:
a display configured to display a screen, the display having a mirror function;
an image collector configured to: collecting face information of a user to obtain a reference image and transmitting the reference image to a processor;
the processor configured to:
determining key point information of the face according to the reference image;
determining the facial features according to the key point information;
and when the user starts a preset auxiliary function for providing auxiliary operation for the facial makeup, determining auxiliary information corresponding to the auxiliary function according to the facial features, and displaying the auxiliary information on the display.
2. The smart device of claim 1, wherein the display comprises a display area, the display area comprising: a mirror area and a regular display area;
the normal display area is used for displaying function buttons, and the function buttons comprise auxiliary function buttons corresponding to the auxiliary functions;
the mirror area is used for providing a mirror function for the user;
wherein the auxiliary information is displayed in the mirror area or the normal display area.
3. The smart device of claim 2, wherein the smart device further comprises: a light supplement structure for supplementing light to the facial makeup;
the mirror area and the normal display area are arranged along a first direction;
the display area is provided with two first side edges which are oppositely arranged and parallel to the first direction, and the light supplementing structure is arranged close to at least one first side edge.
4. The smart device of claim 3, wherein the display further comprises a bezel area surrounding the display area, the fill light structure being located within the bezel area.
5. The smart device of claim 2, wherein the secondary operation is to: assisting the user in making up the face, wherein the auxiliary information comprises auxiliary lines, auxiliary points and/or character guide information, and the auxiliary function comprises displaying the auxiliary information;
the auxiliary information is displayed in the mirror area;
the area of the mirror region is larger than that of the normal display region.
6. The smart device as claimed in claim 5, wherein a zoom button for zooming a specific portion is provided in the regular display area;
the processor configured to:
when the user selects the zooming button and selects a specific part needing zooming, zooming the specific part;
the display configured to:
and displaying the specific part after the zooming processing in a zooming area in the mirror area.
7. The smart device of claim 2, wherein the secondary operation is to: providing a teaching video operation for the user to make up the face, wherein the auxiliary information comprises the teaching video, and the auxiliary function comprises displaying the teaching video;
the conventional display area comprises a button area and a video playing area, the function buttons are displayed in the button area, and the teaching video is displayed in the video playing area.
8. The smart device of claim 7 wherein selection buttons for different parts of the face are provided in the button area;
the processor configured to:
after the auxiliary information corresponding to the auxiliary function is determined, and when the user selects a specific part from each part, analyzing the determined teaching video, and determining a sub-video corresponding to the specific part in the teaching video;
the display configured to:
displaying the sub-video in the video playing area;
and/or, the regular display area further comprises a shopping area for: displaying a corresponding cosmetic product in the teaching video;
the processor configured to:
after the auxiliary information corresponding to the auxiliary function is determined, and when the user selects a specific product from the cosmetic products, determining transaction information for purchasing the specific product, and performing transaction processing on the transaction information.
9. The smart device of claim 2 wherein the function buttons further comprise an evaluate function button;
the processor configured to:
evaluating a makeup result of the face when the user selects the evaluation function button;
and/or the function buttons further comprise a sharing communication function button;
the display configured to:
when the user selects the sharing communication function button, adjusting the area ratio of the conventional display area and the mirror area, so that the display area only comprises the conventional display area.
10. A data processing method, comprising:
determining key point information of a face according to an acquired reference image comprising face information of a user;
determining the facial features according to the key point information;
and when the auxiliary operation needs to be provided for the facial makeup, determining auxiliary information corresponding to the auxiliary operation according to the facial features.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010315153.0A CN111553220A (en) | 2020-04-21 | 2020-04-21 | Intelligent device and data processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010315153.0A CN111553220A (en) | 2020-04-21 | 2020-04-21 | Intelligent device and data processing method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111553220A true CN111553220A (en) | 2020-08-18 |
Family
ID=72005663
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010315153.0A Pending CN111553220A (en) | 2020-04-21 | 2020-04-21 | Intelligent device and data processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111553220A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112019931A (en) * | 2020-08-27 | 2020-12-01 | 深圳创维－Rgb电子有限公司 | Man-machine interaction method and device of smart television, smart television and storage medium |
| CN112946251A (en) * | 2021-01-27 | 2021-06-11 | 广州微框物联科技有限公司 | Intelligent mirror urine test control and display method and system |
| CN114115617A (en) * | 2020-08-27 | 2022-03-01 | 华为技术有限公司 | Display method applied to electronic equipment and electronic equipment |
| WO2022042195A1 (en) * | 2020-08-24 | 2022-03-03 | 华为技术有限公司 | Skin care aiding method, device, and storage medium |
| CN115120077A (en) * | 2021-03-20 | 2022-09-30 | 海信集团控股股份有限公司 | Cosmetic mirror and method for assisting make-up |
| CN117495664A (en) * | 2023-12-25 | 2024-02-02 | 成都白泽智汇科技有限公司 | Intelligent auxiliary cosmetic system |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009213751A (en) * | 2008-03-12 | 2009-09-24 | Sony Ericsson Mobilecommunications Japan Inc | Program, method, and device for makeup evaluation |
| CN102197918A (en) * | 2010-03-26 | 2011-09-28 | 鸿富锦精密工业（深圳）有限公司 | System and method for adjusting cosmetic mirror, and cosmetic mirror with the adjusting system |
| JP2014149678A (en) * | 2013-02-01 | 2014-08-21 | Panasonic Corp | Makeup support apparatus, makeup support system, makeup support method and makeup support program |
| CN105447125A (en) * | 2015-11-17 | 2016-03-30 | 上海斐讯数据通信技术有限公司 | Electronic equipment and makeup assisting method |
| CN106108523A (en) * | 2016-08-22 | 2016-11-16 | 陈云 | A kind of portable intelligent cosmetic mirror, cosmetic aid system and method |
| CN108062400A (en) * | 2017-12-25 | 2018-05-22 | 深圳市美丽控电子商务有限公司 | Examination cosmetic method, smart mirror and storage medium based on smart mirror |
| CN108345849A (en) * | 2018-01-31 | 2018-07-31 | 深圳港云科技有限公司 | A kind of face recognition method and its equipment |
| CN109419140A (en) * | 2017-08-31 | 2019-03-05 | 丽宝大数据股份有限公司 | Recommend eyebrow shape display methods and electronic device |
| KR20190037050A (en) * | 2017-09-28 | 2019-04-05 | 칼-콤프 빅 데이터, 인크. | Body Information Analysis Apparatus and Method of Auxiliary Comparison of Eyebrow Shapes thereof |
| CN109671142A (en) * | 2018-11-23 | 2019-04-23 | 南京图玩智能科技有限公司 | A kind of intelligence makeups method and intelligent makeups mirror |
| CN110246206A (en) * | 2019-06-21 | 2019-09-17 | 北京新氧科技有限公司 | Assist the method, apparatus and system of thrush |
| US20190362134A1 (en) * | 2017-02-01 | 2019-11-28 | Lg Household & Health Care Ltd. | Makeup evaluation system and operating method thereof |
| CN110866139A (en) * | 2019-08-22 | 2020-03-06 | 北京新氧科技有限公司 | Cosmetic treatment method, device and equipment |
-
2020
- 2020-04-21 CN CN202010315153.0A patent/CN111553220A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009213751A (en) * | 2008-03-12 | 2009-09-24 | Sony Ericsson Mobilecommunications Japan Inc | Program, method, and device for makeup evaluation |
| CN102197918A (en) * | 2010-03-26 | 2011-09-28 | 鸿富锦精密工业（深圳）有限公司 | System and method for adjusting cosmetic mirror, and cosmetic mirror with the adjusting system |
| JP2014149678A (en) * | 2013-02-01 | 2014-08-21 | Panasonic Corp | Makeup support apparatus, makeup support system, makeup support method and makeup support program |
| CN105447125A (en) * | 2015-11-17 | 2016-03-30 | 上海斐讯数据通信技术有限公司 | Electronic equipment and makeup assisting method |
| CN106108523A (en) * | 2016-08-22 | 2016-11-16 | 陈云 | A kind of portable intelligent cosmetic mirror, cosmetic aid system and method |
| US20190362134A1 (en) * | 2017-02-01 | 2019-11-28 | Lg Household & Health Care Ltd. | Makeup evaluation system and operating method thereof |
| CN109419140A (en) * | 2017-08-31 | 2019-03-05 | 丽宝大数据股份有限公司 | Recommend eyebrow shape display methods and electronic device |
| KR20190037050A (en) * | 2017-09-28 | 2019-04-05 | 칼-콤프 빅 데이터, 인크. | Body Information Analysis Apparatus and Method of Auxiliary Comparison of Eyebrow Shapes thereof |
| CN108062400A (en) * | 2017-12-25 | 2018-05-22 | 深圳市美丽控电子商务有限公司 | Examination cosmetic method, smart mirror and storage medium based on smart mirror |
| CN108345849A (en) * | 2018-01-31 | 2018-07-31 | 深圳港云科技有限公司 | A kind of face recognition method and its equipment |
| CN109671142A (en) * | 2018-11-23 | 2019-04-23 | 南京图玩智能科技有限公司 | A kind of intelligence makeups method and intelligent makeups mirror |
| CN110246206A (en) * | 2019-06-21 | 2019-09-17 | 北京新氧科技有限公司 | Assist the method, apparatus and system of thrush |
| CN110866139A (en) * | 2019-08-22 | 2020-03-06 | 北京新氧科技有限公司 | Cosmetic treatment method, device and equipment |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022042195A1 (en) * | 2020-08-24 | 2022-03-03 | 华为技术有限公司 | Skin care aiding method, device, and storage medium |
| CN114187649A (en) * | 2020-08-24 | 2022-03-15 | 华为技术有限公司 | Skin care assisting method, device and storage medium |
| CN112019931A (en) * | 2020-08-27 | 2020-12-01 | 深圳创维－Rgb电子有限公司 | Man-machine interaction method and device of smart television, smart television and storage medium |
| CN112019931B (en) * | 2020-08-27 | 2021-12-31 | 深圳创维－Rgb电子有限公司 | Man-machine interaction method and device of smart television, smart television and storage medium |
| CN114115617A (en) * | 2020-08-27 | 2022-03-01 | 华为技术有限公司 | Display method applied to electronic equipment and electronic equipment |
| WO2022042163A1 (en) * | 2020-08-27 | 2022-03-03 | 华为技术有限公司 | Display method applied to electronic device, and electronic device |
| CN114115617B (en) * | 2020-08-27 | 2024-04-12 | 华为技术有限公司 | Display method applied to electronic equipment and electronic equipment |
| CN112946251A (en) * | 2021-01-27 | 2021-06-11 | 广州微框物联科技有限公司 | Intelligent mirror urine test control and display method and system |
| CN115120077A (en) * | 2021-03-20 | 2022-09-30 | 海信集团控股股份有限公司 | Cosmetic mirror and method for assisting make-up |
| CN117495664A (en) * | 2023-12-25 | 2024-02-02 | 成都白泽智汇科技有限公司 | Intelligent auxiliary cosmetic system |
| CN117495664B (en) * | 2023-12-25 | 2024-04-09 | 成都白泽智汇科技有限公司 | Intelligent auxiliary cosmetic system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111553220A (en) | Intelligent device and data processing method | |
| US20240193833A1 (en) | System and method for digital makeup mirror | |
| CN101055647B (en) | Method and device for processing image | |
| US9900498B2 (en) | Glass-type terminal and method for controlling the same | |
| CN113298585B (en) | Method and device for providing commodity object information and electronic equipment | |
| JP2019145108A (en) | Electronic device for generating image including 3d avatar with facial movements reflected thereon, using 3d avatar for face | |
| CN104133956B (en) | Handle the method and device of picture | |
| WO2022227393A1 (en) | Image photographing method and apparatus, electronic device, and computer readable storage medium | |
| CN108712603B (en) | Image processing method and mobile terminal | |
| CN104813258A (en) | Data input device | |
| CN108932654B (en) | Virtual makeup trial guidance method and device | |
| KR101987748B1 (en) | Emoticon Service System And Emoticon Service providing Method thereof | |
| CN110663063B (en) | Method and device for evaluating facial makeup | |
| WO2021063096A1 (en) | Video synthesis method, apparatus, electronic device, and storage medium | |
| CN107798654A (en) | Image mill skin method and device, storage medium | |
| CN107832746A (en) | Expression recognition method and device | |
| WO2024114459A1 (en) | Method and apparatus for generating 3d hand model, and electronic device | |
| US10810647B2 (en) | Hybrid virtual and physical jewelry shopping experience | |
| CN109074680A (en) | Realtime graphic and signal processing method and system in augmented reality based on communication | |
| CN116744820A (en) | Digital color cosmetic teacher | |
| CN109523461A (en) | Method, apparatus, terminal and the storage medium of displaying target image | |
| CN109658167A (en) | Try adornment mirror device and its control method, device | |
| US20200250498A1 (en) | Information processing apparatus, information processing method, and program | |
| KR102340904B1 (en) | System and method for virtual fitting based on augument reality | |
| CN112184359A (en) | Guided consumer experience |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200818 |
|
| RJ01 | Rejection of invention patent application after publication |