CN118466798A

CN118466798A - Icon display method, electronic device and computer readable medium

Info

Publication number: CN118466798A
Application number: CN202310715021.0A
Authority: CN
Inventors: 张祎楠; 李琼琼
Original assignee: Beijing Thinking Magic Education Technology Co ltd
Current assignee: Beijing Thinking Magic Education Technology Co ltd
Priority date: 2023-06-15
Filing date: 2023-06-15
Publication date: 2024-08-09

Abstract

Embodiments of the present disclosure disclose icon display methods, electronic devices, and computer-readable media. One embodiment of the method comprises the following steps: responding to the receiving of the selection operation of a user acting on a start control in a test main interface, displaying a test page, and acquiring a sequence of intelligent board images through a camera device; determining a target puzzle image from the puzzle image sequence; mirror image processing is carried out on the target intelligent board image to obtain a mirror image intelligent board image; correcting the mirror image intelligent board image to obtain a corrected intelligent board image; performing button detection processing on the corrected intelligent board image to obtain a button information set; for each button information included in the button information set, the following steps are performed: generating matching result information according to the button information and preset button information; and displaying a preset button icon at a preset position of the test page corresponding to the button information. According to the embodiment, the number of times of displaying the error icons can be reduced, and the learning effect of children is improved.

Description

Icon display method, electronic device and computer readable medium

Technical Field

Embodiments of the present disclosure relate to the field of computer technology, and in particular, to an icon display method, an electronic device, and a computer readable medium.

Background

Children can help the children verify correct errors by themselves by displaying icons corresponding to the operations of the children on a computing device (such as a mobile phone) when the children operate the intelligent board with the card to learn logic, mathematics, chinese, colors and shapes, and the children are promoted to learn. Currently, when displaying icons, the following methods are generally adopted: firstly, the shooting angle of a camera device is adjusted through a reflecting mirror surface fixed at a terminal to acquire an intelligent board image, then, button detection is directly carried out on the intelligent board image acquired by a camera through a target detection model based on a convolutional neural network to obtain a detection result, and finally, a corresponding icon is displayed according to the detection result.

However, the inventors have found that when the icons are displayed in the above manner, there are often the following technical problems:

first, through fixing the shooting angle at terminal mirror face adjustment camera device in order to gather the intelligent board image, the intelligent board image that obtains can have the slope of certain angle, directly detects the intelligent board image of the slope that obtains gathering, and the accuracy of the testing result that obtains is lower, leads to the error number of times of the icon according to the testing result show more to cause children's study effect relatively poor.

Second, projection-based methods perform tilt correction on the image, which is relatively complex, resulting in long time spent correcting the image.

Thirdly, because the proportion of the buttons in the intelligent board is smaller, the target detection model based on the convolutional neural network is used for directly detecting and identifying the buttons in the intelligent board image, so that the accuracy of a detection result is lower, more error times of icons displayed according to the detection result are further caused, and the learning effect of children is poor.

The above information disclosed in this background section is only for enhancement of understanding of the background of the inventive concept and, therefore, may contain information that does not form the prior art that is already known to those of ordinary skill in the art in this country.

Disclosure of Invention

The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

Some embodiments of the present disclosure propose icon presentation methods, electronic devices, and computer-readable media to address one or more of the technical problems mentioned in the background section above.

In a first aspect, some embodiments of the present disclosure provide an icon display method, the method including: responding to the receiving of the selection operation of a user acting on a start control in a test main interface, displaying a test page, and acquiring a sequence of intelligent board images corresponding to a preset interval duration through a camera device; determining a target puzzle image according to the puzzle image sequence; mirror image processing is carried out on the target intelligent board image to obtain a mirror image intelligent board image; correcting the mirror image intelligent board image to obtain a corrected intelligent board image; performing button detection processing on the correction intelligent board image to obtain a button information set; for each piece of button information included in the button information set, the following steps are performed: generating matching result information according to the button information and preset button information corresponding to the button information; and displaying a preset button icon corresponding to the button information at a preset position of the test page corresponding to the button information.

In a second aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors causes the one or more processors to implement the method described in any of the implementations of the first aspect above.

In a third aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any of the implementations of the first aspect.

The above embodiments of the present disclosure have the following advantageous effects: the icon display method of some embodiments of the present disclosure can reduce the number of times of displaying the wrong icon and improve the learning effect of children. Specifically, the reason for the poor learning effect of children is that: the shooting angle of the camera device is adjusted through the reflecting mirror surface fixed at the terminal so as to acquire the intelligent board image, the acquired intelligent board image has inclination of a certain angle, the acquired inclined intelligent board image is directly detected, the accuracy of the acquired detection result is lower, the number of errors of the icon displayed according to the detection result is more, and therefore the learning effect of children is poor. Based on this, the icon display method of some embodiments of the present disclosure first displays a test page in response to receiving a selection operation of a user for a start control in a test main interface, and acquires a sequence of puzzle images corresponding to a preset interval duration through a camera device. Thus, a test page can be displayed and a sequence of puzzle images obtained, which can be used to present icons. And secondly, determining a target intelligent board image according to the intelligent board image sequence. Thus, a puzzle image with relatively high image quality can be obtained, and thus can be used for detecting buttons in the puzzle image. And then, carrying out mirror image processing on the target intelligent board image to obtain a mirror image intelligent board image. Therefore, the angle of the shot intelligent board image can be adjusted to be the angle corresponding to the actual intelligent board placing position, so that the intelligent board image can be conveniently corrected and detected subsequently. And then, correcting the mirror image intelligent board image to obtain a corrected intelligent board image. Thus, a positive puzzle image can be obtained, and the positive puzzle image can be used for detecting the puzzle image so as to improve the accuracy of a detection result. Then, the button detection processing is performed on the corrected mental plate image to obtain a button information set. Therefore, the more accurate button information of each button included in the intelligent board image can be obtained, and the intelligent board image can be used for matching with preset button icons. Next, for each piece of button information included in the above-described button information set, the following steps are performed: generating matching result information according to the button information and preset button information corresponding to the button information; and displaying a preset button icon corresponding to the button information at a preset position of the test page corresponding to the button information. Therefore, each preset button icon corresponding to each button information can be displayed, and the user can be helped to verify the error by himself. Also, since after the sequence of the puzzle images is collected, firstly, the puzzle image with relatively high image quality is selected from the sequence of the puzzle images, then the selected puzzle image is subjected to angle correction, and finally the corrected puzzle image is detected, thereby improving the accuracy of the detection result, further improving the accuracy of the preset button icons displayed according to the detection result, thereby reducing the number of times of displaying the wrong icons and improving the learning effect of children.

Drawings

The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.

FIG. 1 is a flow chart of some embodiments of an icon presentation method according to the present disclosure;

Fig. 2 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.

It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings. Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.

It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.

It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.

The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Fig. 1 illustrates a flow 100 of some embodiments of an icon presentation method according to the present disclosure. The icon display method comprises the following steps:

step 101, responding to the receiving of the selection operation of a user on a start control in a test main interface, displaying a test page, and acquiring a sequence of intelligent board images corresponding to a preset interval duration through a camera device.

In some embodiments, an execution body (e.g., a computing device) of the icon presentation method may display a test page in response to receiving a user-initiated selection of a start control in a test main interface, and acquire a sequence of puzzle images corresponding to a preset interval duration via a camera device. The test main interface may be an interface for a child to enter into a test. The start control may be a control for jumping to a test page to characterize the start of the test by the child. The test page may be a page for self-verification of errors after the child has operated the puzzle. The selection operation may include, but is not limited to, at least one of: click, slide, hover. The image pickup device may be a device for capturing an image. For example, the imaging device may be a camera provided on the execution body. The execution body may be a tablet or a mobile phone. The preset interval duration may be a preset interval duration. For example, the preset interval duration may be 1 second. The sequence of the puzzle images may be a sequence in which the puzzle images collected by the camera device are arranged in ascending order of time within a period corresponding to a preset interval duration. The puzzle image in the puzzle image sequence may be a child operated puzzle image captured by a camera device.

Step 102, determining a target puzzle image according to the puzzle image sequence.

In some embodiments, the executing subject may determine the target puzzle image from the sequence of puzzle images. Wherein the target puzzle image may be a puzzle image for detecting a button. In practice, the executing subject may determine the target puzzle image from the sequence of puzzle images in various ways.

In some alternative implementations of some embodiments, the executing entity may determine the target puzzle image from the sequence of puzzle images by:

First, for each puzzle image in the sequence of puzzle images, the following determination step is performed:

A first determining step of determining the contrast of the puzzle image.

And a second determining step, wherein card area detection is carried out on the intelligent board image, and information of a rectangular frame circumscribed by the card area is obtained. In practice, the executing body can detect the card area of the intelligent board image by presetting a card area detection model to obtain information of a circumscribed rectangular frame of the card area. The preset card area detection model can be a neural network which is trained in advance, takes an intelligent board image as input and takes information of a rectangular frame circumscribed by a card area as output. The neural network may be a convolutional neural network. The information of the circumscribed rectangular frame of the card area may be information of the circumscribed rectangular frame of the card area. The card area may be an area of the puzzle where a card is placed. The card area circumscribed rectangular frame information may include, but is not limited to, circumscribed rectangular frame width, circumscribed rectangular frame height, circumscribed rectangular frame center point coordinates. The width of the circumscribed rectangular frame may be the width of the circumscribed rectangular frame of the card area. The height of the circumscribed rectangular frame can be the height of the circumscribed rectangular frame of the card area. The coordinates of the center point of the circumscribed rectangular frame may be coordinates of the center point of the circumscribed rectangular frame of the card area in the image coordinate system. The image coordinate system may be a planar coordinate system with the upper left corner of the puzzle image as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis.

And a third determining step, detecting the operation area of the intelligent board graph to obtain information of a rectangular frame circumscribed by the operation area. In practice, the execution subject may perform operation region detection on the puzzle image by presetting an operation region detection model, so as to obtain information of an operation region circumscribed rectangular frame. The preset operation area detection model may be a neural network which is trained in advance and takes an intelligent board image as input and information of an external rectangular frame of an operation area as output. The neural network may be a convolutional neural network. The operation area circumscribed rectangular frame information may be information of a circumscribed rectangular frame of the operation area. The operation area may be an area of a child operation button in the puzzle. The operation area circumscribed rectangular frame information may include, but is not limited to, an operation area circumscribed rectangular frame width, an operation area circumscribed rectangular frame height, and an operation area circumscribed rectangular frame center point coordinate. The width of the circumscribed rectangular frame of the operation area may be the width of the circumscribed rectangular frame of the operation area. The height of the circumscribed rectangular frame of the operation area can be the height of the circumscribed rectangular frame of the operation area. The coordinates of the center point of the circumscribed rectangular frame of the operation area may be coordinates of the center point of the circumscribed rectangular frame of the operation area in the image coordinate system.

And a fourth determining step, namely generating first image integrity information according to the information of the circumscribed rectangular frame of the card area. In practice, first, the executing body may determine, as the first ratio, a ratio of a width of the circumscribed rectangular frame included in the circumscribed rectangular frame information of the card area to a preset first width. The preset first width may be 0.5 times of the width of the circumscribed rectangular frame of the preset complete card area. And secondly, determining the ratio of the height of the circumscribed rectangular frame included in the circumscribed rectangular frame information of the card area to the preset first height as a second ratio. The preset first height may be 0.5 times the height of the circumscribed rectangular frame of the preset complete card area. Then, the product of the first ratio and a preset first ratio weight is determined as a width ratio. The preset first ratio weight may be a preset weight. And then, determining the product of the second ratio and the preset second ratio weight as a height ratio. The preset second ratio weight may be a preset weight. It should be noted that, the sum of the preset first ratio weight and the preset second ratio weight is 1. For example, the preset first ratio weight and the preset second ratio weight may be both 0.5. And then determining the sum of the width ratio and the height ratio as first image integrity information.

And fifth determining, namely generating second image integrity information according to the information of the circumscribed rectangular frame of the operation area. In practice, first, the execution body may determine, as the operation area width ratio, a ratio of the width of the operation area circumscribed rectangular frame included in the operation area circumscribed rectangular frame information to the preset second width. The preset second width may be a width of a circumscribed rectangular frame of the preset complete operation area. And secondly, determining the ratio of the height of the operation area circumscribed rectangular frame included in the operation area circumscribed rectangular frame information to the preset second height as the height ratio of the operation area. The preset second height may be a height of a circumscribed rectangular frame of the preset complete operation area. Then, the product of the operation area width ratio and the preset third ratio weight is determined as the operation area width ratio. The preset third ratio weight may be a preset weight. And then, determining the product of the operation area height ratio and the preset fourth ratio weight as the operation area height ratio. The preset fourth ratio weight may be a preset weight. It should be noted that, the sum of the preset third ratio weight and the preset fourth ratio weight is 1. For example, the preset third ratio weight and the preset fourth ratio weight may be both 0.5. And then determining the sum of the width ratio of the operation area and the height ratio of the operation area as second image integrity information.

And a sixth determining step of generating image quality score information according to the contrast ratio, the first image integrity information and the second image integrity information. In practice, first, the execution subject may determine the contrast as a contrast score. And secondly, determining the first image integrity information as a first integrity score. Then, the second image integrity information is determined as a second integrity score. And finally, determining the sum of the product of the contrast score and a preset contrast weight coefficient, the product of the first integrity score and a preset first integrity weight coefficient, and the product of the second integrity score and a preset second integrity weight coefficient as image quality score information. The preset contrast weight coefficient may be a preset weight coefficient of a contrast dimension of the corresponding image. For example, the preset contrast weight coefficient may be 0.3. The preset first integrity weight coefficient may be a preset weight coefficient corresponding to the first image integrity information dimension. For example, the preset first integrity weight coefficient may be 0.3. The preset second integrity weight coefficient may be a preset weight coefficient corresponding to the second image integrity information dimension. For example, the preset second integrity weight coefficient may be 0.4.

And a second step of determining a target puzzle image according to the generated quality score information of each image and the puzzle image sequence. In practice, the execution subject may determine the target puzzle image from the generated individual image quality score information and the sequence of puzzle images in various ways.

In some optional implementations of some embodiments, the executing entity may determine the target puzzle image from the generated respective image quality score information and the sequence of puzzle images by:

And determining each piece of image quality score information meeting the preset first quality score condition in the image quality score information as a target image quality score information group in response to determining that at least one piece of image quality score information in the generated image quality score information meets the preset first quality score condition. The preset first quality score condition may be that the image quality score information is greater than or equal to a preset image quality score. The predetermined image quality score may be a predetermined score indicative of the qualification of the puzzle image.

And a second step of determining the target image quality score information meeting the preset second quality score condition in the target image quality score information group as matching image quality score information. The preset second quality score condition may be that the target image quality score information is a maximum value in the target image quality score information set.

And thirdly, determining a puzzle image corresponding to the matching image quality score information in the puzzle image sequence as a target puzzle image.

Optionally, the above execution body may further execute the following steps:

In the first step, in response to determining that each of the generated image quality score information does not meet the preset first quality score condition, preset image re-acquisition information is displayed, and a puzzle image sequence corresponding to the preset interval duration is acquired as an updated puzzle image sequence by the image pickup device. The preset image re-acquisition information can be preset information for prompting a user to re-acquire the intelligent board image. For example, the preset image re-acquisition information may be "the image needs to be re-acquired, ask to adjust the position of the mental retardation".

And a second step of continuing the determining step by using the updated puzzle image as a puzzle image for each updated puzzle image in the sequence of updated puzzle images.

And 103, mirror image processing is carried out on the target intelligent board image, and a mirror image intelligent board image is obtained.

In some embodiments, the executing body may mirror the target puzzle image to obtain a mirrored puzzle image. In practice, the executing body may perform horizontal mirroring on the target puzzle image to obtain a mirrored puzzle image.

And 104, correcting the mirror image intelligent board image to obtain a corrected intelligent board image.

In some embodiments, the execution subject may perform a correction process on the mirrored puzzle image to obtain a corrected puzzle image. In practice, the execution subject may perform correction processing on the mirrored puzzle image in various ways to obtain a corrected puzzle image.

In some optional implementations of some embodiments, the executing entity may perform a correction process on the mirrored puzzle image to obtain a corrected puzzle image by:

first, the height of the mirrored puzzle image is determined as image height information.

And secondly, determining the collected image angle information. The collected image angle information may be an angle of the image pickup device that is adjusted by the reflecting mirror when the intelligent board image is collected. In practice, the executing body may determine the preset angle as the acquired image angle information. The preset angle may be a preset angle of the image capturing device that is adjusted by the reflecting mirror. For example, the predetermined angle may be 40 degrees. The reflecting mirror may be a mirror surface engaged with the image pickup device of the execution body and used in cooperation with the image pickup device. The reflecting mirror has a predetermined inclination angle so that the imaging device can capture the intelligent board. As an example, when the computing device is a mobile phone, the reflecting mirror is clamped at the front camera position of the mobile phone, and when the mobile phone is erected on a desktop, the intelligent board horizontally placed on the desktop can be shot through the reflecting mirror.

Optionally, the executing body may further acquire acquired image angle information. In practice, the executing body may acquire the collected image angle information from the mirror angle sensor through a wired connection manner or a wireless connection manner. The mirror angle sensor may be an angle sensor provided on the mirror. It should be noted that the wireless connection may include, but is not limited to, 3G/4G connection, wiFi connection, bluetooth connection, wiMAX connection, zigbee connection, UWB (ultra wideband) connection, and other now known or later developed wireless connection.

And thirdly, determining the width of the mirror image intelligent board image as image width information.

Fourth, generating image bottom width information according to the image height information, the image width information and the collected image angle information. In practice, first, the execution subject may determine the product of the image height information and the tangent value corresponding to the acquired image angle information as the one-side increase width value. Then, the product of the one-side increase width value and 2 is determined as the increase width value. Finally, the sum of the image width information and the increasing width value is determined as image bottom width information.

And fifthly, carrying out transverse stretching treatment on the mirror image intelligent board image according to the image bottom width information to obtain a first stretched image. In practice, the executing body may transversely stretch the width of the bottom of the mirrored puzzle image from left to right to the bottom width information of the image, and the width of the top is unchanged, so as to obtain a first stretched image.

And a sixth step of determining a ratio of the image bottom width information to the image width information as a stretch ratio.

Seventh, determining the product of the image height information and the stretch ratio as the image stretch height information.

And eighth, carrying out longitudinal stretching treatment on the first stretched image according to the image stretching height information to obtain the corrected intelligent board image. In practice, the executing body may stretch the height of the first stretched image from top to bottom to the image stretching height information, so as to obtain the corrected intellectual panel image.

The above technical solution and the related content thereof are taken as an invention point of the embodiments of the present disclosure, and solve the second technical problem mentioned in the background art, namely that the projection-based method corrects the image in an oblique manner, and the algorithm is relatively complex, which results in a long time for correcting the image. Factors that cause the time taken to correct the image to be longer tend to be as follows: the projection-based method corrects the image in an inclined manner, and the algorithm is relatively complex, so that the time consumed for correcting the image is long. If the above factors are solved, the effect of shortening the time taken for image correction can be achieved. To achieve this effect, the icon presentation method of some embodiments of the present disclosure first determines the height of the mirrored puzzle image described above as image height information. Thereby, the height of the mirror image puzzle image can be obtained. Next, the acquired image angle information is determined. Thus, the angle of inclination of the reflecting mirror when the image is acquired can be obtained, and the image correction device can be used for correcting the intelligent board image. Then, the width of the mirrored puzzle image described above is determined as image width information. Thus, the width of the mirrored puzzle image can be obtained. And generating image bottom width information according to the image height information, the image width information and the acquired image angle information. Thus, the width of the mirror image bottom to be stretched can be determined according to the angle of inclination of the reflecting mirror, and thus can be used for stretching the mirror image bottom laterally. And then, carrying out transverse stretching treatment on the mirror image intelligent board image according to the image bottom width information to obtain a first stretched image. Thus, a mirrored puzzle image with the bottom stretched can be obtained. Next, the ratio of the image bottom width information to the image width information is determined as a stretch ratio. And determining the product of the image height information and the stretching ratio as the image stretching height information. Thus, the height of the longitudinal stretching of the mirror image puzzle image can be obtained under the condition that the same transverse and longitudinal stretching ratio of the mirror image puzzle image is ensured, and deformation of the mirror image puzzle image caused by stretching treatment can be reduced. And finally, carrying out longitudinal stretching treatment on the first stretched image according to the image stretching height information to obtain the corrected intelligent board image. Thus, a corrected puzzle image can be obtained, which can be used for detecting the buttons. Also, since the image is corrected not according to the projection-based method but according to the inclination angle of the mirror, the height and width of the image stretching are determined by basic mathematical calculation, and then the image is stretched to complete the image correction, the complexity of the algorithm is reduced, and thus, the time consumed for the image correction can be shortened.

And 105, performing button detection processing on the corrected intelligent board image to obtain a button information set.

In some embodiments, the execution body may perform a button detection process on the corrected puzzle image to obtain a button information set. In practice, the execution subject may perform button detection processing on the corrected puzzle image in various ways to obtain a button information set.

In some optional implementations of some embodiments, the executing body may perform a button detection process on the corrected puzzle image to obtain a button information set by:

And firstly, performing button detection processing on the corrected intelligent board image to obtain a button image set. In practice, the execution subject may perform button detection processing on the correction puzzle image in various ways to obtain a button image set.

A second step of, for each button image in the set of button images, performing the sub-steps of:

And a first sub-step of determining button rectangular frame information corresponding to the button image as target rectangular frame information.

And a second sub-step of determining coordinates of a central point of the button rectangular frame included in the target rectangular frame information as button position information.

And a third sub-step, carrying out color feature extraction processing on the button image to obtain a color feature vector. In practice, the execution body may perform color feature extraction processing on the button image through a preset color feature extraction algorithm, so as to obtain a color feature vector. The preset color feature extraction algorithm may be a preset color feature extraction algorithm. For example, the preset color feature extraction algorithm may be a color moment.

And a fourth sub-step of inputting the color feature vector into a pre-trained color class information generation model to obtain color class information. The color class information generation model is a classification model which takes a color feature vector as an input and color class information as an output. The color class information may be a type of color. The color category information may be, but is not limited to, one of red, yellow, green, orange, violet, blue. The classification model may be a neural network or a support vector machine.

And a fifth sub-step of determining the color category information as button color information.

And a sixth sub-step of determining the button position information and the button color information as button information corresponding to the button image.

And thirdly, determining each piece of determined button information as a button information set.

In some optional implementations of some embodiments, the executing body may perform a button detection process on the correction puzzle image to obtain a button image set by:

step one, cutting the corrected intelligent board image to obtain a cut image. In practice, first, the execution subject may determine the width of the top of the corrected puzzle image as a clipping height. Then, a coordinate point corresponding to the upper left corner of the top of the corrected puzzle image is determined as a left start point. Then, a coordinate point corresponding to the upper right corner of the top of the corrected puzzle image is determined as a right start point. Then, starting from the left starting point and the right starting point, the correction intelligent board image is cut from top to bottom to the cutting height, and a cutting image is obtained.

And secondly, performing mark detection processing on the cut image to obtain mark rectangular frame information. Wherein the above-mentioned logo rectangular frame information may include, but is not limited to, rectangular frame height, rectangular frame width and rectangular frame center point coordinates. The height of the rectangular frame may be the height of the rectangular frame of the detected mark. The width of the rectangular frame may be the width of the rectangular frame of the detected mark. The coordinates of the center point of the rectangular frame may be coordinates of the center point of the rectangular frame of the mark detected under the clipping image coordinate system. The clipping image coordinate system may be a planar coordinate system with the upper left corner of the clipping image as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis. The sign may be a sign of a puzzle. In practice, the execution subject may perform the mark detection processing on the clipping image by presetting a mark detection model, so as to obtain mark rectangular frame information. The preset mark detection model can be a neural network which is trained in advance, takes a cut image as input and takes mark rectangular frame information as output. The neural network may be a convolutional neural network. The information of the marked rectangular frame may be information of a rectangular frame of the mark of the detected puzzle.

And thirdly, cutting the correction intelligent board image according to the height of the rectangular frame and the width of the rectangular frame to obtain a cut intelligent board image. In practice, first, the execution body may determine the product of the rectangular frame height and the preset height multiple as the image clipping height. The preset height multiple may be a preset cutting height multiple of the height of the rectangular frame. It should be noted that the preset height multiple may enable the image obtained after clipping to include each button in the puzzle. For example, the preset height multiple may be 8.5 times. Then, the product of the width of the rectangular frame and the multiple of the preset width is determined as the image clipping width. The preset width multiple may be a preset cutting width multiple of the width of the rectangular frame. For example, the preset width multiple may be 2.2 times. Then, the coordinates of the midpoint of the upper edge of the rectangular frame corresponding to the above-described sign rectangular frame information are determined as target coordinates. The coordinates of the midpoint may be coordinates in a coordinate system having a center point of a rectangular frame corresponding to the marker rectangular frame information as an origin, a horizontal direction as an x-axis, and a vertical direction as a y-axis. Next, the sum of the abscissa of the target coordinate and 0.5 times the image clipping width is determined as the abscissa of the right clipping point, and the ordinate of the target coordinate is determined as the ordinate of the right clipping point. Next, a difference between the abscissa of the target coordinate and 0.5 times the clipping width of the image is determined as the abscissa of the left clipping point, and the ordinate of the target coordinate is determined as the ordinate of the left clipping point. And finally, taking the left cutting point and the right cutting point as the cutting starting points of the left side and the right side, and cutting the corrected intelligent board image from top to bottom to the image cutting height to obtain a cut intelligent board image.

And step four, performing first cutting processing on the cut intelligent board image according to the height of the cut intelligent board image to obtain a first button area image. In practice, first, the execution subject may determine the product of the height of the cropped puzzle image and a preset first cropping height multiple as the first target cropping height. The preset first clipping height multiple may be a preset height to be clipped, which is a multiple of the height of clipping the intelligent board image. For example, the preset first clipping height multiple may be 0.6. Then, the width of the cutting intelligent board image is kept unchanged, and the cutting intelligent board image is cut to a first target cutting height from top to bottom from the upper edge of the cutting intelligent board image, so that a first button area image is obtained.

And fifthly, performing second cutting processing on the cut intelligent board image according to the height of the cut intelligent board image to obtain a second button area image. In practice, first, the execution subject may determine the product of the height of the cropped puzzle image and a preset second cropping height multiple as the second target cropping height. The preset second clipping height multiple may be a preset height to be clipped, which is a multiple of the height of clipping the intelligent board image. It should be noted that the preset first clipping height multiple may be the same as or different from the preset second clipping height multiple. Then, the width of the cutting intelligent board image is kept unchanged, the cutting intelligent board image is cut to a second target cutting height from bottom to top from the lower edge of the cutting intelligent board image, and a second button area image is obtained.

Step six, the first button area image and the second button area image are spliced to obtain the button area image. In practice, the execution subject may perform the stitching process on the first button area image and the second button area image in order from left to right, so as to obtain a button area image.

And step seven, performing button detection processing on the button area image to obtain a button image set. In practice, the execution body may perform button detection processing on the button area image according to a preset target detection algorithm, so as to obtain a button image set. The preset target detection algorithm may be, but is not limited to, one of the following: target detection algorithm based on convolutional neural network, SSD (Single Shot MultiBox Detector) algorithm. Each button image included in the button image set corresponds to button rectangular frame information. The button rectangle frame information may include, but is not limited to, button rectangle frame center point coordinates. The coordinates of the center point of the rectangular button frame may be coordinates of the center point of the circumscribed rectangular button frame corresponding to the button image in the button image coordinate system. The button image coordinate system may be a planar coordinate system having the coordinates of the lower left corner of the button area image as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis.

The technical scheme and the related content are taken as an invention point of the embodiment of the disclosure, and the technical problem mentioned in the background art is solved, namely the target detection model based on the convolutional neural network is obtained by directly detecting and identifying the buttons in the image of the intelligent board, so that the accuracy of a detection result is lower, more error times of icons displayed according to the detection result are further caused, and the learning effect of children is poor. Factors that cause the displayed icons to have a large number of errors and the learning effect of children is poor tend to be as follows: because the proportion of the buttons in the intelligent board is smaller, the target detection model based on the convolutional neural network is used for directly detecting and identifying the buttons in the intelligent board image, so that the accuracy of a detection result is lower, more error times of icons displayed according to the detection result are further caused, and the learning effect of children is poor. If the above factors are solved, the effects of reducing the times of displaying the wrong icons and improving the learning effect of children can be achieved. To achieve this effect, the icon display method according to some embodiments of the present disclosure first performs clipping processing on the corrected puzzle image to obtain a clipped image. Thus, an image containing the sign of the puzzle can be obtained, which can be used for detecting the sign of the puzzle. And secondly, performing mark detection processing on the cut image to obtain mark rectangular frame information. Wherein the mark rectangular frame information comprises a rectangular frame height and a rectangular frame width. Thereby, the position of the logo of the puzzle can be obtained and thus can be used to determine the button area of the puzzle. Then, the correction intelligent board image is cut according to the height of the rectangular frame and the width of the rectangular frame, and a cut intelligent board image is obtained. Therefore, the area irrelevant to the button area in the corrected intelligent board image can be cut off, and the image which is cut and contains the complete button area is obtained, so that the influence of other image characteristics when the button is detected can be reduced. And then, according to the height of the intelligent board image, performing first cutting processing on the intelligent board image to obtain a first button area image. And performing second cutting processing on the cut intelligent board image according to the height of the cut intelligent board image to obtain a second button area image. Therefore, the button area can be divided into an upper part and a lower part in a clipping mode, so that the proportion of the buttons in the clipped image can be increased. And then, performing stitching processing on the first button area image and the second button area image to obtain a button area image. Therefore, a complete button area image with a large proportion of buttons in the image can be obtained, and the button area image can be used for detecting the buttons so as to improve the accuracy of detection results. And finally, carrying out button detection processing on the button area image to obtain a button image set. Thus, a more accurate button image set can be obtained. And when the button detection processing is carried out on the corrected intelligent board image, the mark of the intelligent board is used as a calibration position, and the corrected intelligent board image is cut and spliced for a plurality of times to obtain a complete button area image, so that the proportion of each button in the button area image is improved, the accuracy of the recognition result for recognizing the button area image can be improved, the frequency for displaying the wrong icon can be further reduced, and the learning effect of children is improved.

Step 106, for each piece of button information included in the button information set, executing the following steps:

step 1061, generating matching result information according to the button information and the preset button information corresponding to the button information.

In some embodiments, the execution body may generate the matching result information according to the button information and preset button information corresponding to the button information. The preset button information may be preset button information. The corresponding button information may be that a linear distance between button position information included in the button information and button position information included in the preset button information is within a preset distance range. The preset distance range may be a preset distance range. In practice, the execution subject may determine the preset matching success information as the matching result information in response to determining that the button color information included in the button information is the same as the color information included in the preset button information. The preset matching success information may be preset information indicating that the matching is successful. For example, the preset matching success information may be "color matching success". And in response to determining that the button color information included in the button information is different from the color information included in the preset button information, determining preset matching failure information as matching result information. The preset matching failure information may be preset information indicating matching failure. For example, the preset matching failure information may be "color matching failure".

Step 1062, displaying a preset button icon corresponding to the button information at a preset position of the test page corresponding to the button information.

In some embodiments, the execution body may display a preset button icon corresponding to the button information at a preset position of the test page corresponding to the button information. The preset position may be a preset position of the corresponding button. The preset button icon may be a preset icon representing a matching result, which is the same as button color information included in the button information. For example, when the button information includes yellow button color information and the matching result information indicates that the matching is successful, the preset button icon may be a preset yellow icon with a corresponding number on the upper surface of the button.

The above embodiments of the present disclosure have the following advantageous effects: the icon display method of some embodiments of the present disclosure can reduce the number of times of displaying the wrong icon and improve the learning effect of children. Specifically, the reason for the poor learning effect of children is that: the shooting angle of the camera device is adjusted through the reflecting mirror surface fixed at the terminal so as to acquire the intelligent board image, the acquired intelligent board image has inclination of a certain angle, the acquired inclined intelligent board image is directly detected, the accuracy of the acquired detection result is lower, the number of errors of the icon displayed according to the detection result is more, and therefore the learning effect of children is poor. Based on this, the icon display method of some embodiments of the present disclosure first displays a test page in response to receiving a selection operation of a user for a start control in a test main interface, and acquires a sequence of puzzle images corresponding to a preset interval duration through a camera device. Thus, a test page can be displayed and a sequence of puzzle images obtained, which can be used to present icons. And secondly, determining a target intelligent board image according to the intelligent board image sequence. Thus, a puzzle image with relatively high image quality can be obtained, and thus can be used for detecting buttons in the puzzle image. And then, carrying out mirror image processing on the target intelligent board image to obtain a mirror image intelligent board image. Therefore, the angle of the shot intelligent board image can be adjusted to be the angle corresponding to the actual intelligent board placing position, so that the intelligent board image can be conveniently corrected and detected subsequently. And then, correcting the mirror image intelligent board image to obtain a corrected intelligent board image. Thus, a positive puzzle image can be obtained, and the positive puzzle image can be used for detecting the puzzle image so as to improve the accuracy of a detection result. Then, the button detection processing is performed on the corrected mental plate image to obtain a button information set. Therefore, the more accurate button information of each button included in the intelligent board image can be obtained, and the intelligent board image can be used for matching with preset button icons. Next, for each piece of button information included in the above-described button information set, the following steps are performed: generating matching result information according to the button information and preset button information corresponding to the button information; and displaying a preset button icon corresponding to the button information at a preset position of the test page corresponding to the button information. Therefore, each preset button icon corresponding to each button information can be displayed, and the user can be helped to verify the error by himself. Also, since after the sequence of puzzle images is collected, a puzzle image having a relatively high image quality is first selected from the sequence of puzzle images, then the selected puzzle image is subjected to angle correction, and finally the corrected puzzle image is detected, thereby improving the accuracy of the detection result and the accuracy of the displayed preset button icons, thereby reducing the number of times of displaying the wrong icons and improving the learning effect of children.

Referring now to fig. 2, a schematic diagram of an electronic device 200 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic device shown in fig. 2 is merely an example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.

As shown in fig. 2, the electronic device 200 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 201, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 202 or a program loaded from a storage means 308 into a Random Access Memory (RAM) 203. In the RAM 203, various programs and table data necessary for the operation of the electronic apparatus 200 are also stored. The processing device 201, ROM 202, and RAM 203 are connected to each other through a bus 204. An input/output (I/O) interface 205 is also connected to bus 204.

In general, the following devices may be connected to the I/O interface 205: input devices 206 including camera devices and, for example, touch screens, touch pads, keyboards, mice, microphones, accelerometers, gyroscopes, etc.; an output device 207 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 208 including, for example, magnetic tape, hard disk, etc.; and a communication device 209. The communication means 209 may allow the electronic device 200 to communicate with other devices wirelessly or by wire to exchange form data. While fig. 2 shows an electronic device 200 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 2 may represent one device or a plurality of devices as needed.

In particular, according to some embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via the communication device 209, or from the storage device 208, or from the ROM 202. The above-described functions defined in the methods of some embodiments of the present disclosure are performed when the computer program is executed by the processing device 201.

It should be noted that, the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, the computer readable signal medium may comprise a tabular data signal propagated in baseband or as part of a carrier wave, with the computer readable program code embodied therein. Such a propagated tabular data signal may take a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol ), and may be interconnected with any form or medium of digital form data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: responding to the receiving of the selection operation of a user acting on a start control in a test main interface, displaying a test page, and acquiring a sequence of intelligent board images corresponding to a preset interval duration through a camera device; determining a target puzzle image according to the puzzle image sequence; mirror image processing is carried out on the target intelligent board image to obtain a mirror image intelligent board image; correcting the mirror image intelligent board image to obtain a corrected intelligent board image; performing button detection processing on the correction intelligent board image to obtain a button information set; for each piece of button information included in the button information set, the following steps are performed: generating matching result information according to the button information and preset button information corresponding to the button information; and displaying a preset button icon corresponding to the button information at a preset position of the test page corresponding to the button information.

Computer program code for carrying out operations for some embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in some embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be provided in a processor, for example, described as: a processor includes a display and acquisition unit, a determination unit, a mirror image processing unit, a correction processing unit, a button detection unit, and an execution unit. The names of these units do not limit the units themselves in some cases, for example, the display and acquisition unit may also be described as "a unit that displays a test page in response to receiving a selection operation of a user acting on a start control in a test main interface, and acquires a sequence of puzzle images corresponding to a preset interval duration by a camera device".

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.

The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the invention. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.

Claims

1. An icon display method, comprising:

Responding to the receiving of the selection operation of a user acting on a start control in a test main interface, displaying a test page, and acquiring a sequence of intelligent board images corresponding to a preset interval duration through a camera device;

Determining a target puzzle image from the sequence of puzzle images;

mirror image processing is carried out on the target intelligent board image to obtain a mirror image intelligent board image;

Correcting the mirror image intelligent board image to obtain a corrected intelligent board image;

Performing button detection processing on the correction intelligent board image to obtain a button information set;

For each button information included in the button information set, performing the steps of:

generating matching result information according to the button information and preset button information corresponding to the button information;

And displaying a preset button icon corresponding to the button information at a preset position of the test page corresponding to the button information.

2. The method of claim 1, wherein the determining a target puzzle image from the sequence of puzzle images comprises:

for each puzzle image in the sequence of puzzle images, performing the following determining steps:

Determining a contrast of the puzzle image;

detecting the card area of the intelligent board image to obtain information of a circumscribed rectangular frame of the card area;

detecting an operation area of the intelligent board graph to obtain information of an external rectangular frame of the operation area;

generating first image integrity information according to the information of the circumscribed rectangular frame of the card area;

generating second image integrity information according to the information of the external rectangular frame of the operation area;

generating image quality score information according to the illuminance, the first image integrity information and the second image integrity information;

And determining a target intelligent board image according to the generated quality score information of each image and the intelligent board image sequence.

3. The method of claim 2, wherein said determining a target puzzle image from the generated respective image quality score information and the sequence of puzzle images comprises:

in response to determining that at least one of the generated respective image quality score information satisfies a preset first quality score condition, determining respective ones of the respective image quality score information satisfying the preset first quality score condition as a target image quality score information group;

determining target image quality score information meeting a preset second quality score condition in the target image quality score information group as matching image quality score information;

And determining a puzzle image corresponding to the matching image quality score information in the sequence of puzzle images as a target puzzle image.

4. A method according to claim 3, wherein the method further comprises:

In response to determining that none of the generated image quality score information satisfies the preset first quality score condition, displaying preset image re-acquisition information, and acquiring a puzzle image sequence corresponding to the preset interval duration as an updated puzzle image sequence by the camera device;

for each updated puzzle image in the sequence of updated puzzle images, continuing to perform the determining step with the updated puzzle image as a puzzle image.

5. The method of claim 1, wherein the performing a button detection process on the rectified puzzle image to obtain a button information set includes:

Performing button detection processing on the correction intelligent board image to obtain a button image set, wherein each button image included in the button image set corresponds to button rectangular frame information, and the button rectangular frame information comprises coordinates of a central point of a button rectangular frame;

for each button image in the set of button images, performing the steps of:

determining button rectangular frame information corresponding to the button image as target rectangular frame information;

determining coordinates of a central point of a button rectangular frame included in the target rectangular frame information as button position information;

Performing color feature extraction processing on the button image to obtain a color feature vector;

Inputting the color feature vector into a pre-trained color class information generation model to obtain color class information, wherein the color class information generation model is a classification model taking the color feature vector as input and the color class information as output;

determining the color category information as button color information;

Determining the button position information and the button color information as button information corresponding to the button image;

the determined individual button information is determined as a button information set.

6. An electronic device, comprising:

one or more processors;

a camera device configured to acquire a sequence of puzzle images;

a storage device having one or more programs stored thereon,

When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-5.

7. A computer readable medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the method of any of claims 1-5.