US20230118322A1

US20230118322A1 - Systems and methods for using artificial intelligence for improved rinse cycles in a washing machine appliance

Info

Publication number: US20230118322A1
Application number: US17/501,204
Authority: US
Inventors: Je Kwon Yoon; Myunggeon Chung; Hoyoung Lee; Hyeonsoo Moon; Khalid Jamal Mashal; JaeHyo Lee
Original assignee: Haier US Appliance Solutions Inc
Current assignee: Haier US Appliance Solutions Inc
Priority date: 2021-10-14
Filing date: 2021-10-14
Publication date: 2023-04-20

Abstract

A washing machine appliance includes a wash basket that is rotatably mounted within a wash tub and that defines a wash chamber for receiving a load of clothes. A dispensing assembly selectively dispenses wash fluid through a discharge nozzle into the wash tub and a camera assembly is used to monitor the load of clothes and diagnose the presence of excessive bubbles after a wash cycle. Specifically, a controller of the washing machine appliance uses the camera assembly to obtain one or more images of the load of clothes after the drain phase of the wash cycle and analyzes the one or more images using a machine learning image recognition process to determine a bubble level within the wash tub. If the bubble level is greater than a predetermined bubble threshold, corrective action is taken to reduce the bubbles and detergent concentration.

Description

FIELD OF THE INVENTION

The present subject matter relates generally to washing machine appliances, or more specifically, to systems and methods for using image recognition processes to improve rinse performance in a washing machine appliance.

BACKGROUND OF THE INVENTION

Washing machine appliances generally include a cabinet which receives a wash tub for containing water or wash fluid (e.g., water and detergent, bleach, or other wash additives). The wash tub may be suspended within the cabinet by a suspension system to allow some movement relative to the cabinet during operation. A wash basket is rotatably mounted within the wash tub and defines a wash chamber for receipt of articles for washing. A drive assembly is coupled to the wash tub and is configured to selectively rotate the wash basket within the wash tub.
Detergent and other additives are commonly added to the wash tub to form a wash fluid that facilitates a wash cycle. For example, a user may add detergent directly into the wash tub or into a dispenser reservoir where it is flushed with water into the wash tub. By contrast, certain conventional washing machine appliances are equipped with a bulk dispensing detergent system that includes a bulk reservoir for storing a large amount of detergent. A detergent dispenser can inject a particular amount of detergent based on the load size, water level, and load type to facilitate a wash operation.
Whether the detergent is supplied manually or through a smart dispense system, it is possible that too much detergent gets added into the wash tub. For example, a user might not have a good idea on the optimal detergent amount to add to the tub. Alternatively, a smart dispense feature may be available, but the user may add additional detergent without knowing the feature is available. In addition, the wrong type of detergent may be inadvertently added to the wash tub, e.g., accidental use of dish soap, use of concentrated detergent instead of regular, etc. If too much detergent or the wrong type of detergent is added for a wash cycle, conventional rinse cycles may fail to remove all the detergent from the load of clothes, which may cause some issues on the consumer side (e.g., skin irritation).
Accordingly, a washing machine appliance with improved rinse performance is desirable. More specifically, a washing machine appliance that is capable of detecting situations where excess detergent remains after a wash cycle would be particularly beneficial.

BRIEF DESCRIPTION OF THE INVENTION

Advantages of the invention will be set forth in part in the following description, or may be apparent from the description, or may be learned through practice of the invention.
In one exemplary embodiment, a washing machine appliance is provided including a wash tub positioned within a cabinet, a wash basket rotatably mounted within the wash tub and defining a wash chamber configured for receiving a load of clothes, a dispensing assembly configured to selectively dispense wash fluid through a discharge nozzle into the wash tub, a drain pump assembly fluidly coupled to the wash tub for selectively draining the wash fluid from the wash tub, a camera assembly mounted within the cabinet in view of the wash basket, and a controller operably coupled to the dispensing assembly, the drain pump assembly, and the camera assembly. The controller is configured to operate the drain pump assembly to drain the wash fluid from the wash tub after a wash cycle, obtain one or more images of the wash chamber using the camera assembly, analyze the one or more images using a machine learning image recognition process to determine a bubble level within the wash tub, determine that the bubble level is greater than a predetermined bubble threshold, and implement a responsive action in response to determining that the bubble level is greater than the predetermined bubble threshold.
In another exemplary embodiment, a method of operating a washing machine appliance is provided. The washing machine appliance includes a wash basket rotatably mounted within a wash tub and defining a wash chamber configured for receiving a load of clothes, a dispensing assembly configured to selectively dispense wash fluid through a discharge nozzle into the wash tub, a drain pump assembly fluidly coupled to the wash tub for selectively draining the wash fluid from the wash tub, and a camera assembly mounted within in view of the wash basket. The method includes operating the drain pump assembly to drain the wash fluid from the wash tub after a wash cycle, obtaining one or more images of the wash chamber using the camera assembly, analyzing the one or more images using a machine learning image recognition process to determine a bubble level within the wash tub, determining that the bubble level is greater than a predetermined bubble threshold, and implementing a responsive action in response to determining that the bubble level is greater than the predetermined bubble threshold.
These and other features, aspects and advantages of the present invention will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

A full and enabling disclosure of the present invention, including the best mode thereof, directed to one of ordinary skill in the art, is set forth in the specification, which makes reference to the appended figures.

FIG. 1 provides a perspective view of an exemplary washing machine appliance according to an exemplary embodiment of the present subject matter.

FIG. 2 provides a side cross-sectional view of the exemplary washing machine appliance of FIG. 1 .

FIG. 3 provides a cross-sectional view of the exemplary washing machine appliance of FIG. 1 with a camera assembly mounted on a door according to an exemplary embodiment of the present subject matter.

FIG. 4 provides a schematic view of a door and gasket sealed against a cabinet of the exemplary washing machine of FIG. 1 , along with a camera mounted within the gasket according to an exemplary embodiment of the present subject matter.

FIG. 5 illustrates a method for operating a washing machine appliance in accordance with one embodiment of the present disclosure.

FIG. 6 provides an image captured by the exemplary camera assembly of FIG. 3 with excess remaining detergent according to an exemplary embodiment of the present disclosure.

FIG. 7 provides a flow diagram illustrating an exemplary process for implementing an adaptive rinse cycle according to an exemplary embodiment of the present subject matter.

Repeat use of reference characters in the present specification and drawings is intended to represent the same or analogous features or elements of the present invention.

DETAILED DESCRIPTION

Reference now will be made in detail to embodiments of the invention, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the invention, not limitation of the invention. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope or spirit of the invention. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present invention covers such modifications and variations as come within the scope of the appended claims and their equivalents.
As used herein, the terms “first,” “second,” and “third” may be used interchangeably to distinguish one component from another and are not intended to signify location or importance of the individual components. The terms “includes” and “including” are intended to be inclusive in a manner similar to the term “comprising.” Similarly, the term “or” is generally intended to be inclusive (i.e., “A or B” is intended to mean “A or B or both”). In addition, here and throughout the specification and claims, range limitations may be combined and/or interchanged. Such ranges are identified and include all the sub-ranges contained therein unless context or language indicates otherwise. For example, all ranges disclosed herein are inclusive of the endpoints, and the endpoints are independently combinable with each other. The singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
Approximating language, as used herein throughout the specification and claims, may be applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term or terms, such as “generally,” “about,” “approximately,” and “substantially,” are not to be limited to the precise value specified. In at least some instances, the approximating language may correspond to the precision of an instrument for measuring the value, or the precision of the methods or machines for constructing or manufacturing the components and/or systems. For example, the approximating language may refer to being within a 10 percent margin, i.e., including values within ten percent greater or less than the stated value. In this regard, for example, when used in the context of an angle or direction, such terms include within ten degrees greater or less than the stated angle or direction, e.g., “generally vertical” includes forming an angle of up to ten degrees in any direction, e.g., clockwise or counterclockwise, with the vertical direction V.
The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” In addition, references to “an embodiment” or “one embodiment” does not necessarily refer to the same embodiment, although it may. Any implementation described herein as “exemplary” or “an embodiment” is not necessarily to be construed as preferred or advantageous over other implementations. Moreover, each example is provided by way of explanation of the invention, not limitation of the invention. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope of the invention. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present invention covers such modifications and variations as come within the scope of the appended claims and their equivalents.
The terms “wash fluid” and the like may be used herein to generally refer to a liquid used for washing and/or rinsing clothing or other articles. For example, the wash fluid is typically made up of water that may include other additives such as detergent, fabric softener, bleach, or other suitable treatments (including combinations thereof). By contrast, the term “water” is intended to refer to water only with no detergent, additives, etc. According to exemplary embodiments, the wash fluid for a wash cycle may be a mixture of water, detergent, and/or other additives, while the wash fluid for a rinse cycle may be water only.
Referring now to the figures, an exemplary laundry appliance that may be used to implement aspects of the present subject matter will be described. Specifically, FIG. 1 is a perspective view of an exemplary horizontal axis washing machine appliance 100 and FIG. 2 is a side cross-sectional view of washing machine appliance 100. As illustrated, washing machine appliance 100 generally defines a vertical direction V, a lateral direction L, and a transverse direction T, each of which is mutually perpendicular, such that an orthogonal coordinate system is generally defined. Washing machine appliance 100 includes a cabinet 102 that extends between a top 104 and a bottom 106 along the vertical direction V, between a left side 108 and a right side 110 along the lateral direction, and between a front 112 and a rear 114 along the transverse direction T.
Referring to FIG. 2 , a wash basket 120 is rotatably mounted within cabinet 102 such that it is rotatable about an axis of rotation A. A motor 122, e.g., such as a pancake motor, is in mechanical communication with wash basket 120 to selectively rotate wash basket 120 (e.g., during an agitation or a rinse cycle of washing machine appliance 100). Wash basket 120 is received within a wash tub 124 and defines a wash chamber 126 that is configured for receipt of articles for washing. The wash tub 124 holds wash and rinse fluids for agitation in wash basket 120 within wash tub 124.
Wash basket 120 may define one or more agitator features that extend into wash chamber 126 to assist in agitation and cleaning articles disposed within wash chamber 126 during operation of washing machine appliance 100. For example, as illustrated in FIG. 2 , a plurality of ribs 128 extends from basket 120 into wash chamber 126. In this manner, for example, ribs 128 may lift articles disposed in wash basket 120 during rotation of wash basket 120.
Referring generally to FIGS. 1 and 2 , cabinet 102 also includes a front panel 130 which defines an opening 132 that permits user access to wash basket 120 of wash tub 124. More specifically, washing machine appliance 100 includes a door 134 that is positioned over opening 132 and is rotatably mounted to front panel 130. In this manner, door 134 permits selective access to opening 132 by being movable between an open position (not shown) facilitating access to a wash tub 124 and a closed position (FIG. 1 ) prohibiting access to wash tub 124.
A window 136 in door 134 permits viewing of wash basket 120 when door 134 is in the closed position, e.g., during operation of washing machine appliance 100. Door 134 also includes a handle (not shown) that, e.g., a user may pull when opening and closing door 134. Further, although door 134 is illustrated as mounted to front panel 130, it should be appreciated that door 134 may be mounted to another side of cabinet 102 or any other suitable support according to alternative embodiments. Washing machine appliance 100 may further include a latch assembly 138 (see FIG. 1 ) that is mounted to cabinet 102 and/or door 134 for selectively locking door 134 in the closed position and/or confirming that the door is in the closed position. Latch assembly 138 may be desirable, for example, to ensure only secured access to wash chamber 126 or to otherwise ensure and verify that door 134 is closed during certain operating cycles or events.
Referring again to FIG. 2 , wash basket 120 also defines a plurality of perforations 140 in order to facilitate fluid communication between an interior of basket 120 and wash tub 124. A sump 142 is defined by wash tub 124 at a bottom of wash tub 124 along the vertical direction V. Thus, sump 142 is configured for receipt of and generally collects wash fluid during operation of washing machine appliance 100. For example, during operation of washing machine appliance 100, wash fluid may be urged by gravity from basket 120 to sump 142 through plurality of perforations 140.
A drain pump assembly 144 is located beneath wash tub 124 and is in fluid communication with sump 142 for periodically discharging soiled wash fluid from washing machine appliance 100. Drain pump assembly 144 may generally include a drain pump 146 which is in fluid communication with sump 142 and with an external drain 148 through a drain hose 150. During a drain cycle, drain pump 146 urges a flow of wash fluid from sump 142, through drain hose 150, and to external drain 148. More specifically, drain pump 146 includes a motor (not shown) which is energized during a drain cycle such that drain pump 146 draws wash fluid from sump 142 and urges it through drain hose 150 to external drain 148.
A spout 152 is configured for directing a flow of fluid into wash tub 124. For example, spout 152 may be in fluid communication with a water supply 154 (FIG. 2 ) in order to direct fluid (e.g., clean water or wash fluid) into wash tub 124. Spout 152 may also be in fluid communication with the sump 142. For example, pump assembly 144 may direct wash fluid disposed in sump 142 to spout 152 in order to circulate wash fluid in wash tub 124.
As illustrated in FIG. 2 , a detergent drawer 156 is slidably mounted within front panel 130. Detergent drawer 156 receives a wash additive (e.g., detergent, fabric softener, bleach, or any other suitable liquid or powder) and directs the fluid additive to wash tub 124 during operation of washing machine appliance 100. According to the illustrated embodiment, detergent drawer 156 may also be fluidly coupled to spout 152 to facilitate the complete and accurate dispensing of wash additive. It should be appreciated that according to alternative embodiments, these wash additives could be dispensed automatically via a bulk dispensing unit (not shown). Other systems and methods for providing wash additives are possible and within the scope of the present subject matter.
In addition, a water supply valve 158 may provide a flow of water from a water supply source (such as a municipal water supply 154) into detergent dispenser 156 and into wash tub 124. In this manner, water supply valve 158 may generally be operable to supply water into detergent dispenser 156 to generate a wash fluid, e.g., for use in a wash cycle, or a flow of fresh water, e.g., for a rinse cycle. It should be appreciated that water supply valve 158 may be positioned at any other suitable location within cabinet 102. In addition, although water supply valve 158 is described herein as regulating the flow of “wash fluid,” it should be appreciated that this term includes, water, detergent, other additives, or some mixture thereof.
A control panel 160 including a plurality of input selectors 162 is coupled to front panel 130. Control panel 160 and input selectors 162 collectively form a user interface input for operator selection of machine cycles and features. For example, in one embodiment, a display 164 indicates selected features, a countdown timer, and/or other items of interest to machine users. Operation of washing machine appliance 100 is controlled by a controller or processing device 166 (FIG. 1 ) that is operatively coupled to control panel 160 for user manipulation to select washing machine cycles and features. In response to user manipulation of control panel 160, controller 166 operates the various components of washing machine appliance 100 to execute selected machine cycles and features.
Controller 166 may include a memory and microprocessor, such as a general or special purpose microprocessor operable to execute programming instructions or micro-control code associated with a cleaning cycle. The memory may represent random access memory such as DRAM, or read only memory such as ROM or FLASH. In one embodiment, the processor executes programming instructions stored in memory. The memory may be a separate component from the processor or may be included onboard within the processor. Alternatively, controller 166 may be constructed without using a microprocessor, e.g., using a combination of discrete analog and/or digital logic circuitry (such as switches, amplifiers, integrators, comparators, flip-flops, AND gates, and the like) to perform control functionality instead of relying upon software. Control panel 160 and other components of washing machine appliance 100 may be in communication with controller 166 via one or more signal lines or shared communication busses.
During operation of washing machine appliance 100, laundry items are loaded into wash basket 120 through opening 132, and washing operation is initiated through operator manipulation of input selectors 162. Wash tub 124 is filled with water, detergent, and/or other fluid additives, e.g., via spout 152 and/or detergent drawer 156. One or more valves (e.g., water supply valve 158) can be controlled by washing machine appliance 100 to provide for filling wash basket 120 to the appropriate level for the amount of articles being washed and/or rinsed. By way of example for a wash mode, once wash basket 120 is properly filled with fluid, the contents of wash basket 120 can be agitated (e.g., with ribs 128) for washing of laundry items in wash basket 120.
After the agitation phase of the wash cycle is completed, wash tub 124 can be drained. Laundry articles can then be rinsed by again adding fluid to wash tub 124, depending on the particulars of the cleaning cycle selected by a user. Ribs 128 may again provide agitation within wash basket 120. One or more spin cycles may also be used. In particular, a spin cycle may be applied after the wash cycle and/or after the rinse cycle in order to wring wash fluid from the articles being washed. During a final spin cycle, basket 120 is rotated at relatively high speeds and drain assembly 144 may discharge wash fluid from sump 142. After articles disposed in wash basket 120 are cleaned, washed, and/or rinsed, the user can remove the articles from wash basket 120, e.g., by opening door 134 and reaching into wash basket 120 through opening 132.
Referring now specifically to FIGS. 2 and 3 , washing machine appliance 100 may further include a camera assembly 170 that is generally positioned and configured for obtaining images of wash chamber 126 or a load of clothes (e.g., as identified schematically by reference numeral 172) within wash chamber 126 of washing machine appliance 100. Specifically, according to the illustrated embodiment, door 134 of washing machine appliance 100 comprises and inner window 174 that partially defines wash chamber 126 and an outer window 176 that is exposed to the ambient environment. According to the illustrated exemplary embodiment, camera assembly 170 includes a camera 178 that is mounted to inner window 174. Specifically, camera 178 is mounted such that is faces toward a bottom side of wash tub 124. In this manner, camera 178 can take images or video of an inside of wash chamber 126 and remains unobstructed by windows that may obscure or distort such images.
Referring now briefly to FIG. 4 , another installation of camera assembly 170 will be described according to an exemplary embodiment of the present subject matter. Due to the similarity between this and other embodiments, like reference numerals may be used to refer to the same or similar features. According to this exemplary embodiment, camera assembly 170 is mounted within a gasket 180 that is positioned between a front panel 130 of cabinet 102 and door 134. Although exemplary camera assemblies 170 are illustrated and described herein, it should be appreciated that according to alternative embodiments, washing machine appliance 100 may include any other camera or system of imaging devices for obtaining images of the load of clothes 172 or wash chamber 126.
It should be appreciated that camera assembly 170 may include any suitable number, type, size, and configuration of camera(s) 178 for obtaining images of wash chamber 126. In general, cameras 178 may include a lens 182 that is constructed from a clear hydrophobic material or which may otherwise be positioned behind a hydrophobic clear lens. So positioned, camera assembly 170 may obtain one or more images or videos of clothes 172 within wash chamber 126, as described in more detail below. Referring still to FIGS. 2 through 4 , washing machine appliance 100 may further include a tub light 184 that is positioned within cabinet 102 or wash chamber 126 for selectively illuminating wash chamber 126 and/or the load of clothes 172 positioned therein.
According to exemplary embodiments of the present subject matter, washing machine appliance 100 may further include a basket speed sensor 186 (FIG. 2 ) that is generally configured for determining a basket speed of wash basket 120. In this regard, for example, basket speed sensor 186 may be an optical, tactile, or electromagnetic speed sensor that measures a motor shaft speed (e.g., such as a tachometer, hall-effect sensor, etc.). According to still other embodiments, basket speeds may be determined by measuring a motor frequency, a back electromotive force (EMF) on motor 122, or a motor shaft speed in any other suitable manner. Accordingly, it should be appreciated that according to exemplary embodiments, a physical basket speed sensor 186 is not needed, as electromotive force and motor frequency may be determined by controller 166 without needing a physical speed sensor. It should be appreciated that other systems and methods for monitoring basket speeds may be used while remaining within the scope of the present subject matter.
Notably, controller 166 of washing machine appliance 100 (or any other suitable dedicated controller) may be communicatively coupled to camera assembly 170, tub light 184, latch assembly 138, and other components of washing machine appliance 100. As explained in more detail below, controller 166 may be programmed or configured for obtaining images using camera assembly 170, e.g., in order to detect certain operating conditions and improve the performance of washing machine appliance, e.g., such as initiating an adaptive rinse cycle upon analyzing images of the load of clothes after the wash cycle using a machine learning image recognition process.
Referring still to FIG. 1 , a schematic diagram of an external communication system 190 will be described according to an exemplary embodiment of the present subject matter. In general, external communication system 190 is configured for permitting interaction, data transfer, and other communications with washing machine appliance 100. For example, this communication may be used to provide and receive operating parameters, cycle settings, performance characteristics, user preferences, user notifications, or any other suitable information for improved performance of washing machine appliance 100.
External communication system 190 permits controller 166 of washing machine appliance 100 to communicate with external devices either directly or through a network 192. For example, a consumer may use a consumer device 194 to communicate directly with washing machine appliance 100. For example, consumer devices 194 may be in direct or indirect communication with washing machine appliance 100, e.g., directly through a local area network (LAN), Wi-Fi, Bluetooth, Zigbee, etc. or indirectly through network 192. In general, consumer device 194 may be any suitable device for providing and/or receiving communications or commands from a user. In this regard, consumer device 194 may include, for example, a personal phone, a tablet, a laptop computer, or another mobile device.
In addition, a remote server 196 may be in communication with washing machine appliance 100 and/or consumer device 194 through network 192. In this regard, for example, remote server 196 may be a cloud-based server 196, and is thus located at a distant location, such as in a separate state, country, etc. In general, communication between the remote server 196 and the client devices may be carried via a network interface using any type of wireless connection, using a variety of communication protocols (e.g., TCP/IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and/or protection schemes (e.g., VPN, secure HTTP, SSL).
In general, network 192 can be any type of communication network. For example, network 192 can include one or more of a wireless network, a wired network, a personal area network, a local area network, a wide area network, the internet, a cellular network, etc. According to an exemplary embodiment, consumer device 194 may communicate with a remote server 196 over network 192, such as the internet, to provide user inputs, transfer operating parameters or performance characteristics, receive user notifications or instructions, etc. In addition, consumer device 194 and remote server 196 may communicate with washing machine appliance 100 to communicate similar information.
External communication system 190 is described herein according to an exemplary embodiment of the present subject matter. However, it should be appreciated that the exemplary functions and configurations of external communication system 190 provided herein are used only as examples to facilitate description of aspects of the present subject matter. System configurations may vary, other communication devices may be used to communicate directly or indirectly with one or more laundry appliances, other communication protocols and steps may be implemented, etc. These variations and modifications are contemplated as within the scope of the present subject matter.
While described in the context of a specific embodiment of horizontal axis washing machine appliance 100, using the teachings disclosed herein it will be understood that horizontal axis washing machine appliance 100 is provided by way of example only. Other washing machine appliances having different configurations, different appearances, and/or different features may also be utilized with the present subject matter as well, e.g., vertical axis washing machine appliances. In addition, aspects of the present subject matter may be utilized in a combination washer/dryer appliance.
Now that the construction of washing machine appliance 100 and the configuration of controller 166 according to exemplary embodiments have been presented, an exemplary method 200 of operating a washing machine appliance will be described. Although the discussion below refers to the exemplary method 200 of operating washing machine appliance 100, one skilled in the art will appreciate that the exemplary method 200 is applicable to the operation of a variety of other washing machine appliances, such as vertical axis washing machine appliances. In exemplary embodiments, the various method steps as disclosed herein may be performed by controller 166 or a separate, dedicated controller.
Referring now to FIG. 5 , method 200 includes, at step 210, operating a drain pump assembly to drain wash fluid from a wash tub after a wash cycle. In this regard, using washing machine appliance 100 as an example, a typical wash cycle may include filling wash tub 124 with wash fluid that contains both water and detergent. The wash cycle may further include agitating the load of clothes 172 within wash basket 120, e.g., by operating drive motor to rotate wash basket 120, thereby facilitating an agitation and cleaning process. After the agitation and cleaning portion of the wash cycle is complete, the wash cycle proceeds to drain the wash fluid from wash tub 124 to complete the wash cycle. Subsequently, a rinse cycle may be performed to remove residual detergent from the load of clothes 172.
However, as explained above, if the wrong type or amount of detergent is added to the wash fluid during the wash phase, an undesirable amount of bubbles may be generated that may be difficult or impossible to remove with a conventional rinse cycle. In addition, these bubbles may be a visual indicator as to the presence of an excessive amount of detergent within the load of clothes 172 or within wash tub 124 after the wash cycle has been completed. Aspects of the present subject matter are directed to detecting such a condition in implementing corrective action.
Specifically, step 220 may include obtaining one or more images of the wash chamber using a camera assembly. In this regard, for example, camera 178 of camera assembly 170 may be oriented such that it has a field of view that encompasses the load of clothes 184 positioned within wash basket 120 (e.g., as shown for example in FIG. 6 ). As explained in more detail below, these images may be used to determine whether excessive bubbles or detergent are present in the wash tub 124 and/or on the load of clothes 172. By identifying excessive bubbles at this stage, the rinse portion(s) of the appliance operating cycle may be modified for improved detergent removal.
Thus, step 220 includes obtaining one or more images, a series of frames, a video, or any other visual representation of the load of clothes 172 within wash basket 120. For example, camera assembly 170 may obtain a video clip of the load of clothes 172, take a still image from the video clip, or otherwise obtain a still representation or photo from the video clip. It should be appreciated that the images obtained by camera assembly 170 may vary in number, frequency, angle, resolution, detail, etc. in order to improve the clarity of the load of clothes. In addition, according to exemplary embodiments, controller 166 may be configured for illuminating the tub using tub light 184 just prior to obtaining images.
Referring still to FIG. 5 , method 200 may include, at step 230, analyzing the one or more images using a machine learning image recognition process to determine whether excessive bubbles or detergent are present in the wash tub. It should be appreciated that any suitable image processing or recognition method may be used to analyze the images obtained at step 220 and facilitate determination of the actual bubble/detergent content. In addition, it should be appreciated that this image analysis or processing may be performed locally (e.g., by controller 166) or remotely (e.g., by a remote server).
According to exemplary embodiments of the present subject matter, step 230 of analyzing the one or more images may include analyzing the image(s) of the wash fluid using a neural network classification module and/or a machine learning image recognition process. In this regard, for example, controller 166 may be programmed to implement the machine learning image recognition process that includes a neural network trained with a plurality of images of a load of clothes after being washed with wash fluid with different detergent concentrations or volumes, images of wash basket 120 containing different amounts of bubbles, etc. By analyzing the image(s) obtained at step 220 using this machine learning image recognition process, controller 156 may determine or approximate the actual bubble or detergent content within the load of clothes or wash tub, e.g., by identifying the trained image that is closest to the obtained image.
As used herein, the terms image recognition process and similar terms may be used generally to refer to any suitable method of observation, analysis, image decomposition, feature extraction, image classification, etc. of one or more images or videos taken within a washing machine appliance. In this regard, the image recognition process may use any suitable artificial intelligence (AI) technique, for example, any suitable machine learning technique, or for example, any suitable deep learning technique. It should be appreciated that any suitable image recognition software or process may be used to analyze images taken by camera assembly 170 and controller 166 may be programmed to perform such processes and take corrective action.
According to an exemplary embodiment, controller may implement a form of image recognition called region based convolutional neural network (“R-CNN”) image recognition. Generally speaking, R-CNN may include taking an input image and extracting region proposals that include a potential object, such as a particular region containing a load of clothes that is covered in bubbles or where the water is particularly hazy or cloudy. In this regard, a “region proposal” may be regions in an image that could belong to a particular object, such as a particular grouping of detergent bubbles. A convolutional neural network is then used to compute features from the regions proposals and the extracted features will then be used to determine a classification for each particular region.
According to still other embodiments, an image segmentation process may be used along with the R-CNN image recognition. In general, image segmentation creates a pixel-based mask for each object in an image and provides a more detailed or granular understanding of the various objects within a given image. In this regard, instead of processing an entire image—i.e., a large collection of pixels, many of which might not contain useful information—image segmentation may involve dividing an image into segments (e.g., into groups of pixels containing similar attributes) that may be analyzed independently or in parallel to obtain a more detailed representation of the object or objects in an image. This may be referred to herein as “mask R-CNN” and the like.
According to still other embodiments, the image recognition process may use any other suitable neural network process. For example, step 230 may include using Mask R-CNN instead of a regular R-CNN architecture. In this regard, Mask R-CNN is based on Fast R-CNN which is slightly different than R-CNN. For example, R-CNN first applies CNN and then allocates it to zone recommendations on the covn5 property map instead of the initially split into zone recommendations. In addition, according to exemplary embodiments standard CNN may be used to analyze the image determine an actual bubble level within the wash tub. In addition, a K-means algorithm may be used. Other image recognition processes are possible and within the scope of the present subject matter.
It should be appreciated that any other suitable image recognition process may be used while remaining within the scope of the present subject matter. For example, step 230 may include using a deep belief network (“DBN”) image recognition process. A DBN image recognition process may generally include stacking many individual unsupervised networks that use each network's hidden layer as the input for the next layer. According to still other embodiments, step 230 may include the implementation of a deep neural network (“DNN”) image recognition process, which generally includes the use of a neural network (computing systems inspired by the biological neural networks) with multiple layers between input and output. Other suitable image recognition processes, neural network processes, artificial intelligence (“AI”) analysis techniques, and combinations of the above described or other known methods may be used while remaining within the scope of the present subject matter.
Referring now briefly to FIG. 6 , an exemplary image obtained by camera assembly 170 will be provided according to an exemplary embodiment. For example, FIG. 6 illustrates load of clothes 172 positioned within wash basket 120 after a wash cycle. Specifically, the image shown is exemplary of a wash cycle that was performed with too much detergent or the wrong type of detergent, such that an excessive amount of detergent bubbles (e.g., identified herein generally by reference numeral 198) remain after completion of a wash cycle. Notably, if this operating cycle proceeded directly to a conventional rinse cycle, it is possible that the remaining detergent would not be properly removed from the load of clothes 172, which may result in skin irritation and general dissatisfaction of the washing machine performance.
Accordingly, step 230 may include an artificial intelligence or machine learning image classification process that classifies the load of clothes 172 and/or the wash basket 120 as containing no detergent bubbles, excessive detergent bubbles, or some quantification there between. For example, the bubble level might be low when the detergent used in the wash cycle was appropriate, such that the image contains minimal bubbles or distortion. By contrast, FIG. 6 illustrates a relatively large amount of bubbles (e.g., or otherwise includes hazy, cloudy, bubbly, or opaque portions), indicating that the user has added too much detergent or the wrong kind of detergent prior to initiating the wash cycle.
Although the bubble content is described above as being a classification of an image as either showing the absence or presence of bubbles, it should be appreciated that the determination of the bubble content may vary while remaining within the scope of the present subject matter. For example, some bubbles might be expected or okay after a wash cycle. Accordingly, controller 166 may be programmed with a predetermined bubble threshold that may be used to determine whether too much residual detergent or bubbles are present.
In this regard, step 240 may include determining that the bubble level detected at step 230 is greater than a predetermined bubble threshold. As noted above, this condition may be associated with the presence of too much residual detergent after the wash cycle. Although the analysis described in steps 230 and 240 is described above as being performed with respect to a single image, it should be appreciated that this analysis may be extended to a video stream, a plurality of images, or any other visual representation of the of the load of clothes 172. Notably, if too many bubbles are present, it may be desirable to take corrective action.
Accordingly, step 250 may include implementing a responsive action in response to determining that the bubble level is greater than the predetermined bubble threshold. In general, the responsive action may be intended to reduce the amount of residual detergent or otherwise facilitate a rinse cycle where the remaining detergent falls below a suitable level. Although exemplary responsive actions are described below, it should be appreciated that variations and modifications may be made to step 250 while remaining within scope the present subject matter.
For example, according to an exemplary embodiment, implementing the responsive action may include performing a plurality of rinse cycles (e.g., as opposed to the conventional single rinse cycle). In addition, each of these rinse cycles may include adding a flow of fresh water into the wash tub, agitating the load of clothes by rotating the wash basket, and subsequently draining the tub. Alternatively, each of these steps may be performed simultaneously. In this regard, the responsive action may include simultaneously operating the dispensing assembly to provide a flow of water into the wash tub, agitating the load of clothes by rotating the wash basket, and draining the wash tub using the drain pump assembly.
According to still other embodiments, implementing the responsive action may include adjusting one or more operating parameters of the rinse cycle. For example, the temperature of the fresh water supply during the rinse cycle may be adjusted (e.g., increased or decreased relative to a conventional rinse cycle) to facilitate detergent breakdown and removal. In addition, or alternatively, the volume of water dispensed during a rinse cycle may be adjusted for example, it may be desirable to supply more fresh water into the wash tub during a rinse cycle to better dilute and remove the residual detergent.
According to still other embodiments, implementing the responsive action may be implementing a time delay to permit the bubbles within the wash tub to pop and/or otherwise dissipate on their own. In this regard, the time delay may be implemented after draining the wash fluid after the wash cycle before performing the rinse cycle (e.g., before injecting additional water). According to exemplary embodiments, the time delay after the draining phase of the wash cycle may be between about 10 seconds and 1 hour, between about 30 seconds and 30 minutes, between about 1 minute and 20 minutes, between about 2 minutes and 10 minutes, or about three minutes. Other time delays are possible and within scope the present subject matter.
In addition, step 250 of implementing a responsive action may further include providing a user notification that that the bubble level is greater than the predetermined bubble threshold. In addition, this user notification may include useful information such as estimated bubble levels, fault diagnosis, and/or suggested responsive action. It should be appreciated that the user notification may be provided to the user from any suitable source and in any suitable manner. For example, according to exemplary embodiments, the user notification may be provided through control panel 160 so that the user may be aware of the issue (e.g., such as via an illuminated warning indicator, an image displayed on a screen, etc.). In addition, or alternatively, controller 166 may be configured to provide a user notification to a remote device, such as remote device 194 via a network 192. Whether provided via control panel 160, remote device 194, or by other means, this user notification may include useful information regarding the presence or absence of detergent bubbles or other useful information. For example, the user notification may include a pop-up notification on a user's cell phone or other remote device and may include a display of the one or more images with categorization of the bubble level in preset bubble threshold.
Notably, method 200 may further include taking no action when the bubble level is as desired. In this regard, method 200 may include determining that the bubble level is less than the predetermined bubble threshold and continuing with a normal operating cycle, e.g., such as initiating a standard rinse cycle. In addition, method 200 may include verifying that the responsive action was effective. In this regard, after implementing the responsive action, such as repeated rinse cycles or rinse cycles with different operating parameters, camera assembly may obtain one or more additional images that may be analyzed in the same manner described above to determine the bubble level within the wash tub. If the bubble level still remains above the predetermined bubble threshold, additional action may be taken. Otherwise, the operating cycle may proceed to the rinse cycle as usual.
Referring now briefly to FIG. 7 , an exemplary flow diagram of a bubble detection or adaptive rinse method 300 that may be implemented by washing machine appliance 100 will be described according to an exemplary embodiment of the present subject matter. According to exemplary embodiments, method 300 may be similar to or interchangeable with method 200 and may be implemented by controller 166 of washing machine appliance 100. As shown, at step 302, controller 166 may first start an operating cycle of a washing machine appliance.
Step 304 includes filling the wash tub with a flow of wash fluid containing both water and detergent. Step 306 generally includes performing an agitation phase of the wash cycle, e.g., by using the drive motor to rotate wash basket 120 and tumble the load of clothes contained therein. After the agitation cycle, step 308 may include operating a drain pump assembly to drain the wash fluid from the wash tub. Notably, as explained above, if an excessive amount of detergent or the wrong type of detergent were added to the wash tub during the wash cycle, there may be excessive residual detergent and detergent bubbles within the load of clothes 172 after step 308. Accordingly, method 300 is generally directed to a process of detecting such a condition and facilitating an adaptive rinse cycle to correct this condition.
Specifically step 310 may include obtaining one or more images within the wash basket using a camera assembly and analyzing those images using a machine learning image recognition process. This analysis may conclude with a classification as to the bubble level within the load of clothes 172. If the bubble level determined at step 310 exceeds a predetermined bubble threshold, responsive action may be taken at step 312. As explained above, this responsive action may include performing repeated rinse cycles, performing rinse cycles with the adjusted operating parameters (e.g., such as increased water volumes or temperatures), implementing a time delay, notifying the user, etc.
If step 310 results in a determination that the bubble level does not exceed the bubble threshold, the operating cycle may proceed as usual toward the rinse cycle. In this regard, the rinse cycle may generally include step 314 of filling the wash tub with fresh water and subsequently draining that water and step 316. According to exemplary embodiments, step 318 may include a post-rinse verification that any remaining bubbles fall below a predetermined threshold. If the bubble level is still too high, step 320 may once again involve taking responsive actions (e.g., as described above). By contrast, if the post-rinse verification at step 318 results in a determination that the bubble level is below the threshold, the operating cycle may complete by performing a spin cycle at step 322.
FIGS. 5 and 7 depict steps performed in a particular order for purposes of illustration and discussion. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the steps of any of the methods discussed herein can be adapted, rearranged, expanded, omitted, or modified in various ways without deviating from the scope of the present disclosure. Moreover, although aspects of method 200 and method 300 are explained using washing machine appliance 100 as an example, it should be appreciated that this method may be applied to the operation of any suitable laundry appliance, such as another washing machine appliance.
The systems and methods described herein may utilize adaptive rinse cycles when excessive detergent is detected by obtaining images with a camera and analyzing those images using an artificial intelligence or machine learning classification technique. Specifically, a classification artificial intelligence technique is used to identify existing bubbles through the images acquired by the camera. For example, after a wash cycle, if residual detergent is recognized, responsive actions are taken, e.g., such as increasing the number of rinse cycles, performing drum agitation and draining simultaneously, increasing the temperature of the water to generate fewer bubbles or waiting until the bubbles have popped and then running the drain cycle. If no residual detergent is detected, the spinning operation may be performed and the cycle may be completed.
This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.

Claims

1. A washing machine appliance, comprising:

a wash tub positioned within a cabinet;

a wash basket rotatably mounted within the wash tub and defining a wash chamber configured for receiving a load of clothes;

a dispensing assembly configured to selectively dispense wash fluid through a discharge nozzle into the wash tub;

a drain pump assembly fluidly coupled to the wash tub for selectively draining the wash fluid from the wash tub;

a camera assembly mounted within the cabinet in view of the wash basket; and

a controller operably coupled to the dispensing assembly, the drain pump assembly, and the camera assembly, the controller being configured to:

operate the drain pump assembly to drain the wash fluid from the wash tub after a wash cycle;

obtain one or more images of the wash chamber using the camera assembly after operating the drain pump assembly to drain the wash fluid;

analyze the one or more images using a machine learning image recognition process to determine a bubble level within the wash tub;

determine that the bubble level is greater than a predetermined bubble threshold; and

implement a responsive action in response to determining that the bubble level is greater than the predetermined bubble threshold.

2. The washing machine appliance of claim 1, wherein the machine learning image recognition process comprises at least one of a convolution neural network (“CNN”), a region-based convolution neural network (“R-CNN”), a deep belief network (“DBN”), or a deep neural network (“DNN”) image recognition process.

3. The washing machine appliance of claim 1, wherein implementing the responsive action comprises:

performing a plurality of rinse cycles, wherein each of the plurality of rinse cycles comprises adding a flow of water into the wash tub and at least one of agitating the load of clothes by rotating the wash basket, spinning the wash basket, or draining the wash tub.

4. The washing machine appliance of claim 1, wherein implementing the responsive action comprises:

adjusting a water temperature of a flow of water dispensed during a rinse cycle.

5. The washing machine appliance of claim 1, wherein implementing the responsive action comprises:

adjusting a volume of water dispensed during a rinse cycle.

6. The washing machine appliance of claim 1, wherein implementing the responsive action comprises:

implementing a time delay after draining the wash fluid and before performing a rinse cycle.

7. The washing machine appliance of claim 6, wherein the time delay is between 1 minute and 30 minutes.

8. The washing machine appliance of claim 1, wherein implementing the responsive action comprises:

simultaneously operating the dispensing assembly to provide a flow of water into the wash tub, agitate the load of clothes by rotating the wash basket, and draining the wash tub using the drain pump assembly.

9. The washing machine appliance of claim 1, wherein implementing the responsive action comprises:

providing a user notification that the bubble level is greater than the predetermined bubble threshold.

10. The washing machine appliance of claim 9, further comprising:

a user interface panel, wherein the user notification is provided through the user interface panel.

11. The washing machine appliance of claim 9, wherein the controller is in operative communication with a remote device through an external network, and wherein the user notification is provided through the remote device.

12. The washing machine appliance of claim 1, wherein the controller is further configured to:

determine that the bubble level is less than the predetermined bubble threshold; and

perform a rinse cycle.

13. The washing machine appliance of claim 1, wherein the controller is further configured to:

obtain one or more additional images after implementing the responsive action;

analyze the one or more additional images using the machine learning image recognition process to determine a bubble level within the wash tub;

determine that the bubble level is greater than the predetermined bubble threshold; and

implement another responsive action in response to determining that the bubble level is greater than the predetermined bubble threshold.

14. The washing machine appliance of claim 1, wherein the camera assembly has a field of view oriented toward a bottom of the wash basket where the load of clothes collects.

15. The washing machine appliance of claim 1, wherein the camera assembly includes a light source for illuminating the wash chamber while obtaining the one or more images.

16. The washing machine appliance of claim 1, wherein the washing machine appliance is a horizontal axis washing machine appliance.

17. A method of operating a washing machine appliance, the washing machine appliance comprising a wash basket rotatably mounted within a wash tub and defining a wash chamber configured for receiving a load of clothes, a dispensing assembly configured to selectively dispense wash fluid through a discharge nozzle into the wash tub, a drain pump assembly fluidly coupled to the wash tub for selectively draining the wash fluid from the wash tub, and a camera assembly mounted within in view of the wash basket, the method comprising:

operating the drain pump assembly to drain the wash fluid from the wash tub after a wash cycle;

obtaining one or more images of the wash chamber using the camera assembly after operating the drain pump assembly to drain the wash fluid;

analyzing the one or more images using a machine learning image recognition process to determine a bubble level within the wash tub;

determining that the bubble level is greater than a predetermined bubble threshold; and

implementing a responsive action in response to determining that the bubble level is greater than the predetermined bubble threshold.

18. The method of claim 17, wherein implementing the responsive action comprises:

19. The method of claim 17, wherein implementing the responsive action comprises:

20. The method of claim 17, further comprising:

determining that the bubble level is less than the predetermined bubble threshold; and

performing a rinse cycle.