WO2024179319A1

WO2024179319A1 - Image processing method and apparatus, and electronic device

Info

Publication number: WO2024179319A1
Application number: PCT/CN2024/077550
Authority: WO
Inventors: 刘文焘; 萧文鹏
Original assignee: 北京字跳网络技术有限公司
Priority date: 2023-02-28
Filing date: 2024-02-19
Publication date: 2024-09-06
Also published as: CN116168123A

Abstract

The present disclosure provides an image processing method and apparatus, and an electronic device. A specific embodiment of the method comprises: obtaining an image to be processed, the image to be processed comprising an object to undergo fluttering; determining the object to undergo fluttering and a reference object in the image to be processed; on the basis of the object to undergo fluttering and the reference object, determining a plurality of layers to be processed, different layers to be processed corresponding to different blocking relationships; and on the basis of the plurality of layers to be processed, generating a fluttering effect of the object to undergo fluttering.

Description

Image processing method, device and electronic equipment

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on an application with CN application number 202310215668.7 and filing date February 28, 2023, and claims priority. The disclosed content of the CN application is hereby introduced as a whole into this application.

Technical Field

The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device and electronic device.

Background Art

With the continuous development of digital image technology, people can use digital cameras to capture digital images to record beautiful moments in life, or draw digital cartoon images, etc. At present, in order to make the image more vivid, a solution is needed to make some objects (such as hair, dress, scarf, etc.) in a static image flutter, thereby obtaining a dynamic image.

Summary of the invention

The present disclosure provides an image processing method, an image processing device and an electronic device.

According to a first aspect, there is provided an image processing method, comprising:

Acquire an image to be processed; the image to be processed includes an object to be moved;

Determining the object to be moved and a reference object in the image to be processed;

Based on the object to be floated and the reference object, a plurality of layers to be processed are determined; wherein different layers to be processed correspond to different occlusion relationships;

Based on the multiple layers to be processed, a fluttering effect of the object to be fluttered is generated.

According to a second aspect, there is provided an image processing apparatus, comprising:

An acquisition module, used for acquiring an image to be processed; the image to be processed includes an object to be moved;

A determination module, used for determining the object to be moved and the reference object in the image to be processed;

A layering module, used to determine a plurality of layers to be processed based on the object to be floated and the reference object; wherein different layers to be processed correspond to different occlusion relationships;

A generating module is used to generate a fluttering effect of the object to be fluttered based on the multiple layers to be processed.

According to a third aspect, a computer-readable storage medium is provided, wherein the storage medium stores a computer program. When the computer program is executed by a processor, the image processing method described in any one of the first aspects is implemented.

According to a fourth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image processing method described in any one of the first aspects is implemented.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

FIG1 is a schematic diagram of an image processing scenario according to an exemplary embodiment of the present disclosure;

FIG2 is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure;

FIG3 is a flow chart of another image processing method according to an exemplary embodiment of the present disclosure;

FIG4 is a block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure;

FIG5 is a schematic block diagram of an electronic device provided by some embodiments of the present disclosure;

FIG6 is a schematic block diagram of another electronic device provided by some embodiments of the present disclosure;

FIG. 7 is a schematic diagram of a storage medium provided by some embodiments of the present disclosure.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be described clearly and completely below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are only examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

The terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. The singular forms of "a", "the" and "the" used in this disclosure are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and/or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

With the continuous development of digital image technology, people can use digital cameras to shoot digital images to record the beautiful moments in life, or draw digital cartoon images. In order to make the image more vivid, some objects in the static image (such as hair, dress, etc.) can be made to flutter, thereby obtaining a dynamic image. In the related art, professionals are usually required to use professional image processing tools to manually process the static image, and render the image data obtained after manual processing to obtain a dynamic image, which is difficult for ordinary non-professionals to achieve. In addition, the obtained dynamic image has a poor dynamic display effect and lacks a sense of hierarchy and space.

The present disclosure provides an image processing method, which determines an object to be floated and a reference object in an image to be processed, and based on the object to be floated and the reference object, determines a plurality of layers to be processed corresponding to different occlusion relationships, and generates a floating effect of the object to be floated according to the plurality of layers to be processed. Since the present embodiment takes into account the occlusion relationship in the image to be processed, and generates a floating effect of the object to be floated according to the layers to be processed with different occlusion relationships, the object to be floated in the animated image or video after the floating process can float in an interlaced manner by layer based on a certain occlusion relationship. Thus, without the need for manual operation, the floating effect of some objects in a static image is achieved, and the display effect of the image is made more realistic and vivid, with a stronger sense of layering and space, and the flexibility and interest of image application are also improved.

Referring to Fig. 1, a schematic diagram of an image processing scenario according to an exemplary embodiment is shown. Referring to Fig. 1, the solution of the present disclosure is schematically described in combination with a complete and specific application example. The application example describes a specific image processing process.

As shown in FIG1 , the image A to be processed may include an object to be fluttered that can add a fluttering special effect. For example, the image A to be processed may be a face image of a person, and the object to be fluttered may be the hair of the person. For another example, the image A to be processed may be a full-body image of a person, and the object to be fluttered may be the hem of the person's dress. For another example, the image A to be processed may be an image of interior decoration, and the object to be fluttered may be a curtain.

In some embodiments, first, the object to be floated in the image to be processed A can be identified to determine the target area corresponding to the object to be floated, and a mask image B of the object to be floated can be obtained based on the target area. For example, a pre-trained recognition model can be used to identify the object to be floated in the image to be processed A, and the pixel points of the target area in the image to be processed A are assigned 0, and the pixel points of other areas are assigned 1, so as to obtain the mask image B of the object to be floated.

Then, based on the mask image B, the target area corresponding to the fluttering object is segmented to obtain a mask image C of multiple sub-areas. The sub-area may be an area obtained by vertically segmenting the target area corresponding to the fluttering object. For example, if the fluttering object is hair, the sub-area may be an area corresponding to a bunch of hair. For another example, if the fluttering object is a skirt of a person's clothing, the sub-area may be an area corresponding to a wisp of the skirt. For another example, if the fluttering object is a curtain, the sub-area may be an area corresponding to a wisp of the curtain.

In some embodiments, a reference object in the image A to be processed may be identified to determine the area corresponding to the reference object, wherein there is an occlusion relationship between the reference object and the object to be moved. For example, if the object to be moved is hair, the reference object may be a face. For another example, if the object to be moved is a curtain, the reference object may be a window, etc.

In some embodiments, multiple layers to be processed with different occlusion relationships can be determined based on the mask image B of the object to be floated and the region corresponding to the reference object. And based on the mask image B of the object to be floated, the mask image C of the sub-region and the multiple layers to be processed, gridding processing is performed to obtain multiple grid layers, wherein each grid layer corresponds to a layer to be processed.

In some embodiments, anchor points for constraining the floating object to float can also be extracted. Then, based on the image A to be processed, the anchor points and the multiple grid layers, rendering processing is performed to obtain multiple frames of images for displaying animated images or playing videos, and the multiple frames of images can achieve the floating effect of the floating object.

Specifically, the trajectory information corresponding to each anchor point can be determined, and the position points of each anchor point in each frame of the multi-frame image to be generated can be determined based on the trajectory information corresponding to each anchor point. Then, the floating data corresponding to each layer of grid can be determined based on the position points of each anchor point in each frame of the image. Based on the information of the occlusion relationship of different layers to be processed, the floating data corresponding to each layer of grid is occluded. According to the pixel value of each pixel point in the image to be processed and the data after occlusion processing, the pixel value of each pixel point in each frame of the image to be generated is determined, so as to perform rendering processing and obtain multiple frames of images. The multiple frames of images can constitute a dynamic image or video. If the multiple frames of images are displayed, the floating effect of the object to be floated can be displayed.

The present disclosure will be described in detail below in conjunction with specific embodiments.

FIG2 is a flow chart of an image processing method according to some embodiments. The method can be implemented as any device, platform, server or device cluster with computing and processing capabilities, and includes the following steps: 201-204.

As shown in FIG. 2 , in step 201 , an image to be processed is acquired, and in step 202 , an object to be moved and a reference object in the image to be processed are determined.

In some embodiments, the image to be processed may be an image in RGB color mode, and the image to be processed may include an object to be fluttered that can achieve a fluttering special effect. Specifically, the object to be fluttered may be an object with a fixed upper end and a free lower end that can flutter in the wind. For example, the object to be fluttered may be the hair of a person, or the hem of a person's dress, or the tassel on a decoration, or a scarf worn by a person, or a curtain, etc. It can be understood that the present embodiment does not limit the specific category of the fluttering object.

The object to be floated in the image to be processed can be determined, that is, the region information corresponding to the object to be floated in the image to be processed can be obtained. Specifically, the object to be floated can be identified using a pre-trained recognition model, so as to determine the region information corresponding to the object to be floated in the image to be processed. Among them, the recognition model can be obtained by training based on a large number of sample images including the object to be floated. Optionally, after determining the region information corresponding to the object to be floated in the image to be processed, the mask image corresponding to the object to be floated can be further obtained based on the region information corresponding to the object to be floated. For example, the pixel points of the area corresponding to the object to be floated in the image to be processed can be assigned 0, and the pixel points of other areas can be assigned 1, so as to obtain the mask image of the object to be floated. The mask image of the object to be floated can be used in the subsequent gridding process and other processes.

In some embodiments, the image to be processed may further include a reference object, and the reference object may be an object that has an occlusion relationship with the object to be fluttered. When the object to be fluttered flutters, it will occlude the reference object, or the reference object will occlude the object to be fluttered. For example, if the object to be fluttered is the hair of a person, the reference object may be the face of the person. For another example, if the object to be fluttered is the hem of a person's skirt, the reference object may be the legs of the person, etc.

A preset algorithm may be used to determine the region information corresponding to the reference object in the image to be processed. For example, taking the case where the object to be fluttered is the hair of a person and the reference object is a face, a preset face key point detection algorithm may be used to detect the face key points in the image to be processed, wherein the face key points may be points in key regions of the face. For example, face key points may include but are not limited to multiple points on regions such as eyebrows, eyes, nose, and mouth. Then, the face contour is determined based on the face key points, and the information of the region surrounded by the face contour is obtained as the region information corresponding to the reference object.

In step 203, a plurality of layers to be processed are determined based on the object to be floated and the reference object.

In some embodiments, the multiple layers to be processed may be multiple regional layers corresponding to different occlusion relationships. By obtaining the area corresponding to any layer to be processed, the layer to be processed can be determined. The multiple layers to be processed may include at least a first layer to be processed and a second layer to be processed. The area where the object to be floated overlaps with the reference object can be obtained as the area corresponding to the first layer to be processed, and the area in the object to be floated that does not overlap with the reference object can be obtained as the area corresponding to the second layer to be processed. The first layer to be processed has occlusion on the reference object and the second layer to be processed, and the second layer to be processed has no occlusion on the reference object.

In some embodiments, the plurality of layers to be processed may include a third layer to be processed in addition to the first layer to be processed and the second layer to be processed. The area corresponding to the object to be floated may be obtained as the area corresponding to the third layer to be processed. The first layer to be processed has occlusion on the reference object, the second layer to be processed and the third layer to be processed; the second layer to be processed has occlusion on the third layer to be processed and has no occlusion on the reference object; the reference object has occlusion on the third layer to be processed.

For example, if the object to be fluttered is the hair of a person and the reference object is a face, the area where the hair area overlaps with the face area can be obtained as the area corresponding to the first layer to be processed. The area in the hair area that does not overlap with the face area is obtained as the area corresponding to the second layer to be processed. The hair area is obtained as the area corresponding to the third layer to be processed. Among them, the first layer to be processed corresponds to the bangs part of the hair, the second layer to be processed corresponds to the hair bundle part other than the bangs, and the third layer to be processed corresponds to the entire part of the hair. Therefore, the first layer to be processed corresponding to the bangs part is in the uppermost layer, and has occlusion on the face and the other two layers to be processed. The second layer to be processed corresponding to the hair bundle part is in the middle layer, has no occlusion on the face and the first layer to be processed, and has occlusion on the third layer to be processed. The third layer to be processed corresponding to the entire part of the hair is in the lowermost layer as the background layer, and the face and the other two layers to be processed have occlusion on the third layer to be processed.

In step 204, a fluttering effect of the object to be fluttered is generated based on the multiple layers to be processed.

In some embodiments, the grid corresponding to each layer to be processed can be obtained to obtain a multi-layer grid, and then the occlusion relationship information corresponding to each layer to be processed can be obtained. According to the multi-layer grid and the occlusion relationship information, a multi-frame image is generated, and the floating effect of the object to be floated can be displayed by playing the multi-frame image.

The present disclosure provides an image processing method, which determines an object to be floated and a reference object in an image to be processed, and based on the object to be floated and the reference object, determines a plurality of layers to be processed corresponding to different occlusion relationships, and generates a floating effect of the object to be floated according to the plurality of layers to be processed. Since the present embodiment takes into account the occlusion relationship in the image to be processed, and generates a floating effect of the object to be floated according to the layers to be processed with different occlusion relationships, the object to be floated in the animated image or video after the floating process can float in an interlaced manner by layer based on a certain occlusion relationship. Thereby, without the need for manual operation, the floating effect of some objects in a static image is achieved, and the display effect of the image is made more realistic and vivid, with a stronger sense of layering and space, and the flexibility and interest of image application are also improved. Taste.

FIG. 3 is a flow chart of another image processing method according to some embodiments. The embodiment describes a process of generating a floating effect of an object to be floated, including the following steps: 301 - 303 .

As shown in FIG. 3 , in step 301 , a grid corresponding to each layer to be processed is obtained to obtain a multi-layer grid.

In some embodiments, the area corresponding to each layer to be processed can be gridded to obtain a multi-layer grid. Specifically, the area corresponding to each layer to be processed can be obtained respectively, and the area corresponding to each layer to be processed can be gridded to obtain a grid corresponding to each layer to be processed.

In some embodiments, the area corresponding to the object to be fluttered can be divided into multiple sub-areas vertically, each sub-area can be a fluttering unit, and grid processing can be performed according to the multiple sub-areas. For example, if the object to be fluttered is the hair of a person and the reference object is a face, the hair of the person can be divided into multiple hair bundles. Then, a preset algorithm (such as a triangulation algorithm, etc.) can be used to perform grid processing on each hair bundle in units of hair bundles to obtain a grid corresponding to each hair bundle. For example, multiple points can be randomly selected in the area corresponding to the hair bundle, and the selected multiple points can be connected using a preset algorithm to generate a grid composed of multiple triangles.

Next, a plurality of first hair bundles included in the region corresponding to the first to-be-processed layer may be determined, and a plurality of second hair bundles included in the region corresponding to the second to-be-processed layer may be determined. Then, a layer of grids corresponding to the first to-be-processed layer may be formed by a plurality of grids corresponding to the plurality of first hair bundles, and a layer of grids corresponding to the second to-be-processed layer may be formed by a plurality of grids corresponding to the plurality of second hair bundles. In addition, a preset algorithm may be used to perform grid processing on the entire hair region to obtain a layer of grids corresponding to the third to-be-processed layer.

In step 302, the information of the occlusion relationship corresponding to each layer to be processed is obtained, and in step 303, the floating effect of the object to be floated is generated based on the information of the multi-layer grid and the occlusion relationship.

In some embodiments, multiple anchor points for constraining the floating of the object to be floated can be determined, and floating effect processing can be performed on each layer of grids based on the multiple anchor points to obtain floating data corresponding to each layer of grids. Specifically, first, the anchor points for the object to be floated can be determined, and the anchor points for the object to be floated can include a first anchor point for fixing the object to be floated and a second anchor point for controlling the floating of the object to be floated. In the case where the area corresponding to the object to be floated is divided into multiple sub-areas, the anchor points corresponding to each sub-area can also be determined. The anchor points corresponding to any sub-area can include a third anchor point for fixing the sub-area and a fourth anchor point for controlling the floating of the sub-area.

For any sub-region, the highest point corresponding to the sub-region and the midline passing through the highest point can be determined, wherein the highest point corresponding to the sub-region can be determined based on the coordinates of the sub-region in the image, and the highest point can be the point with the largest ordinate in the sub-region. The midline passing through the highest point also passes through the lowest point corresponding to the sub-region, and the midline can divide the sub-region into two parts. The midline can be determined by any reasonable algorithm. The middle line, this embodiment does not limit the determination method of the middle line. The highest point can be determined as the third anchor point for fixing the sub-area, and the points on the middle line equidistantly selected according to the preset distance can be determined as the fourth anchor point for controlling the movement of the sub-area.

For the object to be floated, the key points of the reference object can be detected, and a fixed edge line for fixing the object to be floated can be determined based on the key points, and a point covered by the object to be floated is selected from the fixed edge line as the first anchor point for fixing the object to be floated. For example, taking the object to be floated as the hair of a person and the reference object as a face, the key points of the face are determined, and the forehead contour edge line for fixing the hair is determined based on the key points of the face, and the point covered by the hair is selected from the forehead contour edge line as the first anchor point for the entire hair. In addition, multiple points can be randomly selected from multiple fourth anchor points used to control the movement of the sub-region as the second anchor point for controlling the floating of the object to be floated.

In some embodiments, after determining multiple anchor points, the trajectory points corresponding to each anchor point can also be determined according to the desired floating effect. For the first anchor point used to fix the object to be floated and the third anchor point used to fix the sub-area, their corresponding trajectory points are fixed points. For the second anchor point used to control the floating of the object to be floated and the fourth anchor point used to control the floating of the sub-area, the trajectory information corresponding to each anchor point can be calculated using a preset algorithm according to the desired floating effect. And according to the trajectory information corresponding to each anchor point, the position point of each anchor point in each frame of the multiple frames of images to be generated is determined.

In some embodiments, the floating data corresponding to each layer of grid can be determined based on preset constraints (for example, the constraints can indicate the distance relationship between the anchor point and other points in the area, etc.) and the position points of each anchor point in each frame of the image. Among them, the floating data corresponding to any layer of grids can include the grid area information corresponding to the layer of grids. Finally, based on the information of the above-mentioned occlusion relationship, the floating data corresponding to each layer of grids can be occluded, and the pixel values of each pixel point in each frame of the image to be generated can be determined based on the pixel values of each pixel point in the image to be processed and the data after occlusion processing, so as to perform rendering processing and obtain multiple frames of images. The multiple frames of images can constitute a dynamic image or video. If the multiple frames of images are displayed, the floating effect of the object to be floated can be displayed.

Since this embodiment generates the floating effect of the object to be floated through the information of the grids corresponding to each layer to be processed and the occlusion relationship corresponding to each layer to be processed, the floating effect of the object to be floated in the animated image or video after the floating processing can be made more realistic by staggered floating by layers based on certain occlusion relationships, further improving the floating effect of the object to be floated in the animated image or video after the floating processing.

It should be noted that although the operations of the method of the embodiment of the present disclosure are described in a specific order in the above embodiments, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flowchart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and/or one step can be performed in a different order. The step is broken down into multiple steps for execution.

Corresponding to the aforementioned image processing method embodiment, the present disclosure also provides an embodiment of an image processing device.

As shown in FIG. 4 , FIG. 4 is a block diagram of an image processing device according to an exemplary embodiment of the present disclosure. The device may include: an acquisition module 401 , a determination module 402 , a layering module 403 and a generation module 404 .

The acquisition module 401 is used to acquire an image to be processed, where the image to be processed includes an object to be floating.

The determination module 402 is used to determine the object to be moved and the reference object in the image to be processed.

The layering module 403 is used to determine a plurality of layers to be processed based on the object to be floated and the reference object, wherein different layers to be processed correspond to different occlusion relationships.

The generating module 404 is used to generate a fluttering effect of the object to be fluttered based on the multiple layers to be processed.

In some embodiments, the multiple layers to be processed may include a first layer to be processed and a second layer to be processed, wherein the layering module 403 is configured to: determine an area where the object to be floated overlaps with the reference object as an area corresponding to the first layer to be processed; and determine an area in the object to be floated that does not overlap with the reference object as an area corresponding to the second layer to be processed.

In some implementations, the multiple layers to be processed may further include a third layer to be processed, wherein the layering module 403 is further configured to: determine the area corresponding to the object to be fluttered as the area corresponding to the third layer to be processed.

In some embodiments, different layers to be processed correspond to different occlusion relationships, including: the first layer to be processed has occlusion on the reference object, the second layer to be processed and the third layer to be processed, the second layer to be processed has occlusion on the third layer to be processed, and the reference object has occlusion on the third layer to be processed.

In some implementations, the generation module 404 may include: a first acquisition submodule, a second acquisition submodule, and a generation submodule (not shown in the figure).

The first acquisition submodule is used to acquire the grid corresponding to each layer to be processed to obtain a multi-layer grid.

The second acquisition submodule is used to obtain the occlusion relationship information corresponding to each layer to be processed.

The generation submodule is used to generate the floating effect of the object to be floated based on the information of the multi-layer grid and the occlusion relationship.

In some implementations, the first acquisition submodule is configured to: perform gridding processing on the area corresponding to each layer to be processed to obtain a multi-layer grid.

In some embodiments, the generation submodule is configured to: perform floating effect processing on each layer of grids to obtain floating data corresponding to each layer of grids, and based on the occlusion relationship information, perform occlusion processing on the floating data corresponding to each layer of grids to obtain multiple frames of images for displaying the floating effect.

For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only illustrative, and the separation The units described by the components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure. Those of ordinary skill in the art may understand and implement them without creative work.

FIG5 is a schematic block diagram of an electronic device provided by some embodiments of the present disclosure. As shown in FIG5 , the electronic device 910 includes a processor 911 and a memory 912, which can be used to implement a client or a server. The memory 912 is used to store computer executable instructions (e.g., one or more computer program modules) non-transiently. The processor 911 is used to run the computer executable instructions, and when the computer executable instructions are run by the processor 911, one or more steps in the image processing method described above can be executed, thereby implementing the image processing method described above. The memory 912 and the processor 911 can be interconnected via a bus system and/or other forms of connection mechanisms (not shown).

For example, the processor 911 may be a central processing unit (CPU), a graphics processing unit (GPU), or other forms of processing units having data processing capabilities and/or program execution capabilities. For example, the central processing unit (CPU) may be an X86 or ARM architecture, etc. The processor 911 may be a general-purpose processor or a dedicated processor, and may control other components in the electronic device 910 to perform desired functions.

For example, the memory 912 may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor 911 may run one or more computer program modules to implement various functions of the electronic device 910. Various applications and various data, as well as various data used and/or generated by the application, etc. may also be stored in the computer-readable storage medium.

It should be noted that, in the embodiment of the present disclosure, the specific functions and technical effects of the electronic device 910 can refer to the above description of the image processing method, which will not be repeated here.

FIG6 is a schematic block diagram of another electronic device provided by some embodiments of the present disclosure. The electronic device 920 is suitable for implementing the image processing method provided by the embodiments of the present disclosure. The electronic device 920 may be a terminal device, etc., and may be used to implement a client or a server. The electronic device 920 may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable electronic devices, etc., and devices such as Fixed terminals of digital TV, desktop computers, smart home devices, etc. It should be noted that the electronic device 920 shown in FIG6 is only an example, which does not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG6 , the electronic device 920 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 921, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 922 or a program loaded from a storage device 928 to a random access memory (RAM) 923. In the RAM 923, various programs and data required for the operation of the electronic device 920 are also stored. The processing device 921, the ROM 922, and the RAM 923 are connected to each other via a bus 924. An input/output (I/O) interface 925 is also connected to the bus 924.

Typically, the following devices may be connected to the I/O interface 925: input devices 926 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 927 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 928 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 929. The communication devices 929 may allow the electronic device 920 to communicate with other electronic devices wirelessly or by wire to exchange data. Although FIG. 6 shows an electronic device 920 with various devices, it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 920 may alternatively implement or have more or fewer devices.

For example, according to an embodiment of the present disclosure, the above-mentioned image processing method can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the above-mentioned image processing method. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 929, or installed from the storage device 928, or installed from the ROM 922. When the computer program is executed by the processing device 921, the functions defined in the image processing method provided by the embodiment of the present disclosure can be implemented.

FIG7 is a schematic diagram of a storage medium provided by some embodiments of the present disclosure. For example, as shown in FIG7, the storage medium 930 may be a non-transitory computer-readable storage medium for storing a non-transitory computer-executable instruction 931. When the non-transitory computer-executable instruction 931 is executed by a processor, the image processing method described in the embodiment of the present disclosure may be implemented. For example, when the non-transitory computer-executable instruction 931 is executed by a processor, one or more steps in the image processing method described above may be executed.

For example, the storage medium 930 may be applied to the above-mentioned electronic device. For example, the storage medium 930 may include a memory in the electronic device.

For example, the storage medium may include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a flash memory, or any combination of the above storage media. It can also be other applicable storage media.

For example, the description of the storage medium 930 can refer to the description of the memory in the embodiment of the electronic device, and the repeated parts are not repeated. The specific functions and technical effects of the storage medium 930 can refer to the description of the image processing method above, and are not repeated here.

It should be noted that in the context of the present disclosure, a computer-readable medium may be a tangible medium that may contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer-readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used by an instruction execution system, device or device or used in combination with it. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

The present disclosure also provides a computer program product, comprising instructions, which, when executed by a processor, enable the processor to implement the image processing method of any of the aforementioned embodiments.

The present disclosure also provides a computer program, comprising instructions, which, when executed by a processor, enable the processor to implement the image processing method of any of the aforementioned embodiments.

Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

An image processing method, comprising:

Acquire an image to be processed, wherein the image to be processed includes an object to be moved;

Determining the object to be moved and a reference object in the image to be processed;

Based on the object to be floated and the reference object, determining a plurality of layers to be processed, wherein different layers to be processed correspond to different occlusion relationships;

Based on the multiple layers to be processed, a fluttering effect of the object to be fluttered is generated.
The image processing method according to claim 1, wherein the plurality of layers to be processed include a first layer to be processed and a second layer to be processed, and the determining the plurality of layers to be processed based on the object to be floated and the reference object includes:

Determine an area where the object to be floated overlaps with the reference object as an area corresponding to the first layer to be processed;

An area of the object to be floated that does not overlap with the reference object is determined as an area corresponding to the second layer to be processed.
The image processing method according to claim 2, wherein the plurality of layers to be processed further include a third layer to be processed, and the determining the plurality of layers to be processed based on the object to be floated and the reference object further includes:

The area corresponding to the object to be floated is determined as the area corresponding to the third layer to be processed.
According to the image processing method according to claim 3, wherein the different layers to be processed correspond to different occlusion relationships, including: the first layer to be processed has occlusion on the reference object, the second layer to be processed and the third layer to be processed; the second layer to be processed has occlusion on the third layer to be processed; the reference object has occlusion on the third layer to be processed.
According to the image processing method according to any one of claims 1 to 4, wherein generating the floating effect of the object to be floated based on the multiple layers to be processed comprises:

Obtaining the grid corresponding to each of the layers to be processed to obtain a multi-layer grid;

Obtaining information about the occlusion relationship corresponding to each of the layers to be processed;

Based on the information of the multi-layer grids and the occlusion relationship, a floating effect of the object to be floated is generated.
According to the image processing method of claim 5, wherein the obtaining of the grid corresponding to each of the layers to be processed to obtain the multi-layer grid comprises: performing gridding processing on the area corresponding to each of the layers to be processed to obtain the multi-layer grid.
According to the image processing method of claim 5 or 6, wherein the generating the floating effect of the object to be floated based on the information of the multi-layer grid and the occlusion relationship comprises:

Performing floating effect processing on the grids of each layer to obtain floating data corresponding to the grids of each layer;

Based on the information of the occlusion relationship, occlusion processing is performed on the floating data corresponding to each layer of the grid to obtain multiple frames of images for displaying the floating effect.
According to the image processing method of claim 7, wherein the performing floating effect processing on each layer of the grid to obtain floating data corresponding to each layer of the grid comprises:

Determining a plurality of anchor points for constraining the floating of the object to be floated;

According to the fluttering effect, determining the trajectory point corresponding to each anchor point;

Determine the position of each anchor point in each frame of the multiple frames of images to be generated according to the trajectory information corresponding to each anchor point;

According to the preset constraints and the position points of each anchor point in each frame image, the floating data corresponding to each layer of grid is determined.
According to the image processing method of claim 8, wherein the multiple anchor points include a first anchor point for fixing the object to be floated and a second anchor point for controlling the floating of the object to be floated, and when the area corresponding to the object to be floated is divided into multiple sub-areas, the multiple anchor points also include anchor points corresponding to each sub-area, and the anchor points corresponding to each sub-area include a third anchor point for fixing the sub-area and a fourth anchor point for controlling the floating of the sub-area.
An image processing device, comprising:

An acquisition module, used for acquiring an image to be processed; the image to be processed includes an object to be moved;

A determination module, used for determining the object to be moved and the reference object in the image to be processed;

A layering module, used to determine a plurality of layers to be processed based on the object to be floated and the reference object; wherein different layers to be processed correspond to different occlusion relationships;

A generating module is used to generate a fluttering effect of the object to be fluttered based on the multiple layers to be processed.
A computer-readable storage medium stores a computer program, which, when executed in a computer, causes the computer to execute the image processing method according to any one of claims 1 to 9.
An electronic device comprises a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the image processing method according to any one of claims 1 to 9 is implemented.
A computer program product comprises instructions, and when the instructions are executed by a processor, the processor is enabled to implement the image processing method according to any one of claims 1 to 9.
A computer program comprises instructions, and when the instructions are executed by a processor, the processor is enabled to implement the image processing method according to any one of claims 1 to 9.