US10871480B2 - Multi-sensor platform for crop health monitoring - Google Patents

Multi-sensor platform for crop health monitoring Download PDF

Info

Publication number
US10871480B2
US10871480B2 US16/268,744 US201916268744A US10871480B2 US 10871480 B2 US10871480 B2 US 10871480B2 US 201916268744 A US201916268744 A US 201916268744A US 10871480 B2 US10871480 B2 US 10871480B2
Authority
US
United States
Prior art keywords
sensor
plant
crop
mobile sensory
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US16/268,744
Other versions
US20190170718A1 (en
Inventor
Saber Miresmailli
Murray B. ISMAN
Maryam ANTIKCHI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ecoation Innovative Solutions Inc
Original Assignee
Ecoation Innovative Solutions Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ecoation Innovative Solutions Inc filed Critical Ecoation Innovative Solutions Inc
Priority to US16/268,744 priority Critical patent/US10871480B2/en
Publication of US20190170718A1 publication Critical patent/US20190170718A1/en
Priority to US17/099,422 priority patent/US11499955B2/en
Application granted granted Critical
Publication of US10871480B2 publication Critical patent/US10871480B2/en
Priority to US18/054,437 priority patent/US11965870B2/en
Priority to US18/056,598 priority patent/US12038426B2/en
Priority to US18/056,610 priority patent/US11874265B2/en
Priority to US18/056,604 priority patent/US12111301B2/en
Priority to US18/185,211 priority patent/US11867680B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N33/00Investigating or analysing materials by specific methods not covered by groups G01N1/00 - G01N31/00
    • G01N33/0098Plants or trees
    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01GHORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G13/00Protecting plants
    • A01G13/06Devices for generating heat, smoke or fog in gardens, orchards or forests, e.g. to prevent damage by frost
    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01GHORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G22/00Cultivation of specific crops or plants not otherwise provided for
    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01GHORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G25/00Watering gardens, fields, sports grounds or the like
    • A01G25/16Control of watering
    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01GHORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G7/00Botany in general
    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01GHORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G9/00Cultivation in receptacles, forcing-frames or greenhouses; Edging for beds, lawn or the like
    • A01G9/24Devices or systems for heating, ventilating, regulating temperature, illuminating, or watering, in greenhouses, forcing-frames, or the like
    • A01G9/26Electric devices
    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01MCATCHING, TRAPPING OR SCARING OF ANIMALS; APPARATUS FOR THE DESTRUCTION OF NOXIOUS ANIMALS OR NOXIOUS PLANTS
    • A01M21/00Apparatus for the destruction of unwanted vegetation, e.g. weeds
    • A01M21/04Apparatus for destruction by steam, chemicals, burning, or electricity
    • A01M21/043Apparatus for destruction by steam, chemicals, burning, or electricity by chemicals
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01DMEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
    • G01D11/00Component parts of measuring arrangements not specially adapted for a specific variable
    • G01D11/30Supports specially adapted for an instrument; Supports specially adapted for a set of instruments
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B15/00Systems controlled by a computer
    • G05B15/02Systems controlled by a computer electric
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00Computing arrangements using knowledge-based models
    • G06N5/04Inference or reasoning models
    • GPHYSICS
    • G08SIGNALLING
    • G08CTRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C17/00Arrangements for transmitting signals characterised by the use of a wireless electrical link
    • G08C17/02Arrangements for transmitting signals characterised by the use of a wireless electrical link using a radio link
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning

Definitions

  • the present invention relates to systems and methods for crop monitoring and assessment. Some embodiments relate to multi-sensory devices and platforms for crop monitoring, and methods for operating the same.
  • growers face several challenges. For example, it is generally difficult for a grower to predict the quality and yield of the crop at a stage in crop development when intervention will still be feasible and useful. Also it can be difficult for a grower to know if, where and when the crop has a problem (such as related to a pest, disease, water, other abiotic stress or nutritional deficit), and the extent of the problem, until it is readily visible to human scouts. Often by that stage it may require expensive and extensive intervention. Crop yield is affected by the physiological performance of the crop throughout its development cycle, which is in turn dependent on external environmental factors among other things.
  • a problem such as related to a pest, disease, water, other abiotic stress or nutritional deficit
  • Sensor systems have been developed for crop monitoring, but many of these systems have limitations and shortcomings. For example, some systems use a grid of sensors suspended above the crop (in a zone, usually about an acre in greenhouses) or that fly over the crops. Such sensory grids can be used to monitor environmental conditions or general responses from plants, but generally this is on a course-grained scale. Handheld devices can be used to capture data from individual plants, but these devices tend to be cumbersome to use, and data is captured only from plants that the operator of the handheld device interacts with directly. It is generally not feasible to use a handheld device to capture data from all plants within an area being managed. Often such devices are used by clipping them to the plant or otherwise contacting the plant with the sensing device.
  • Some sensory devices/systems are geared toward specific indicators (presence of disease, anthocyanin content, emergence of adult pests, etc.) with narrow spectra of responses, often using sensors that function during daylight hours. These technologies are generally large and expensive, and require a human operator, and some of them are time-consuming.
  • fluorescent measurement systems have been used to detect far red spectra produced by plants when exposed to blue or red light.
  • Conventional fluorescent measurement requires complex equipment, and typically a single assessment takes several minutes and sometimes up to about 15 minutes to complete.
  • Other sensory systems can collect very general information (temperature, humidity) that cannot accurately pinpoint problems at the level of individual plants, or at levels of sensitivity that convey timely information in real time.
  • the housing also contains a control unit operable to control operation of the plurality of sensors, a location tracking system connected to the control unit for tracking the location of the multi-sensor device, and a communications interface connected to the control unit for transmitting and receiving information.
  • the communication interface is operable to transmit data, for example wirelessly, from the plurality of sensor modules.
  • a user-interface is connected to the control unit to allow a user to interact with the device, for example, via a touch screen.
  • the device further includes an electrical power connector for connecting the multi-sensor device to a power source.
  • the electrical power connector can be integral with one or more of the mounts, or can be separate from the mounts.
  • the housing can be mounted to a handle via the mount for use of the multi-sensor device as a hand-held device.
  • the device can be mounted to a mobile platform via the mount.
  • the mobile platform can comprise a propulsion system, a power supply, and a control system for controlling movement of the mobile sensory platform.
  • the sensor modules are each coupled to a universal adapter.
  • the universal adapters are connectable to provide power, data and control signal connectivity between one of the sensor modules and the control unit.
  • the sensor modules can comprise a physiological sensor module, a surface analysis sensor module, and a chemical sensor module.
  • the physiological sensor module can comprise configurable optical probes and tunable detectors.
  • the surface analysis sensor module can comprise a full spectrum light source and spectroscopic detectors.
  • the chemical analysis sensor module can be, for example, a photo-ionization detector, a surface acoustic wave sensor or a quartz crystal microbalance sensor.
  • the sensor modules can further comprise a thermal imaging sensor module, and/or a vision system module comprising a pair of high resolution RGB-IR cameras for stereo-imaging, and/or a temperature and humidity sensor module.
  • Each of the plurality of sensor modules is preferably operable to capture data pertaining to at least one plant-related parameter without physical contact with the plant.
  • the communications interface is operable to provide two-way communication between the multi-sensor device and a data processing unit over a wireless network, and/or is operable to receive control commands from a wireless mobile device.
  • a mobile sensory platform for capturing and transmitting sensor data for plants in a crop comprises a propulsion system, and a plurality of sensors mounted to a support structure. Each of the plurality of sensors is for capturing data pertaining to at least one plant-related parameter when the sensor is positioned proximate to a plant.
  • the platform also comprises a communications interface for two-way communication with a data processing unit over a wireless network, an electrical power supply; and a control system for controlling movement of the mobile sensory platform and operation of the plurality of sensors.
  • each of the plurality of sensors is operable to capture data pertaining to at least one plant-related parameter without physical contact with the plant.
  • the plurality of sensors can comprise a physiological sensor and/or a surface analysis sensor and/or a chemical sensor, and optionally microclimate sensors, and/or canopy screening sensors.
  • the sensors of the mobile sensory platform comprise a physiological sensor comprising configurable optical probes and tunable detectors, a surface analysis sensor comprising a full spectrum light source and spectroscopic detectors, and a chemical analysis sensor selected from the group consisting of a photo-ionization detector, a surface acoustic wave sensor and a quartz crystal microbalance sensor.
  • the position of at least one of the sensors on the support structure is adjustable during operation of the mobile sensory platform.
  • the control system may be operable to control the position of the at least one of the plurality of sensors.
  • the mobile sensory platform of claim further comprises an intervention module for administering remediation to plants in the crop.
  • the mobile sensory platform can be a ground-based platform or an air-borne platform.
  • FIG. 1 is a schematic illustration of an embodiment of an automated crop monitoring system comprising a mobile sensory platform.
  • FIG. 2 is a block diagram illustrating components of an embodiment of a mobile sensory platform.
  • FIG. 3 is a block diagram illustrating components of a data processing unit (DPU).
  • DPU data processing unit
  • FIG. 4 illustrates an embodiment of a method of operating an automated crop monitoring system, the method involving two phases of data collection and analysis.
  • FIG. 5 is a schematic illustration of an embodiment of an automated crop monitoring system that captures and uses expert knowledge and comprises a mobile sensory platform.
  • FIG. 6 illustrates an embodiment of a method for crop monitoring and assessment comprising four activities.
  • FIG. 7A illustrates an expert knowledge capture and system training activity that is a component of the method illustrated in FIG. 6 .
  • FIG. 7B is a screen-shot from a system training app that can be used by an expert to capture and convey their assessment of plants in a crop.
  • FIG. 8 illustrates a non-expert sensor data acquisition activity that is a component of the method illustrated in FIG. 6 .
  • FIG. 9 illustrates a mobile sensory platform data acquisition and analysis activity that is a component of the method illustrated in FIG. 6 .
  • FIG. 10A illustrates an information dissemination activity that is a component of the method illustrated in FIG. 6 .
  • FIG. 10B is a screen-shot from a reporting app that can be used to convey information relating to crop health to a person, such as a grower.
  • FIG. 11 is a schematic illustration showing various sources of information that may provide inputs to a DPU in embodiments of systems and methods for monitoring and assessing crop health.
  • FIG. 12 illustrates an embodiment of a method involving correlation of crop performance data with historical data captured during the lifecycle of the crop.
  • FIG. 13A is a simplified drawing showing a front view and a side view of an embodiment of a mobile sensory platform.
  • FIG. 13B is a simplified drawing showing a front view and a side view of another embodiment of a mobile sensory platform.
  • FIG. 13C is a simplified drawing showing a front view and a side view of yet another embodiment of a mobile sensory platform.
  • FIG. 14A is a simplified drawing showing two views of an embodiment of an air-borne mobile sensory platform.
  • FIG. 14B is a simplified drawing showing two views of an embodiment of an air-borne mobile sensory platform.
  • FIG. 14C is a simplified drawing showing embodiments of a landing pad where the air-borne mobile sensory platforms of FIGS. 14A and 14B can land for recharging.
  • FIG. 15A is a simplified drawing of an embodiment of a hand-held device comprising a portable sensory platform connected to a smartphone.
  • FIG. 1513 is a simplified drawing of another embodiment of a hand-held device comprising a portable sensory platform connected to a smartphone.
  • FIG. 15C is a simplified drawing showing three views of yet another embodiment of a hand-held device comprising a portable sensory platform connected to a smartphone.
  • FIG. 16A illustrates an embodiment of a modular multi-sensor device, for crop monitoring, that can be used as a hand-held device or can be mounted to a mobile platform.
  • FIG. 16B illustrates a housing that can accommodate various sensor modules and other components
  • FIG. 16C shows the multi-sensor device of FIG. 16A with some of the sensor modules partially removed by sliding out of housing.
  • FIG. 16D shows a housing with a sensor module installed in the housing position, and two alternative sensor modules which can replace the installed sensor module.
  • FIG. 16E shows a rear perspective view of the housing shown in FIG. 16B , and shows separately a bundle of components that can be installed in the housing.
  • FIG. 17A shows an example of handle that can be fastened to the multi-sensor device of FIG. 16A .
  • FIG. 17B illustrates how the handle of FIG. 17A can be fastened to the multi-sensor device of FIG. 16A .
  • FIG. 17C illustrates a mounting connector that can be removably fastened to a mount on the exterior of a multi-sensor device.
  • FIG. 17D illustrates a mounting bracket via which a multi-sensor device can be mounted to a mobile platform.
  • FIG. 17E illustrates how the multi-sensor device housing of FIG. 16B can be mounted to the bracket of FIG. 17D .
  • the systems and methods described herein for monitoring and assessing crop health can provide rapid and sensitive screening of individual plant health with reduced human labor, and at a far greater speed than can be accomplished by human scouts.
  • the systems and methods described herein can be deployed outdoors, such as in a field or orchard, or indoors such as in a greenhouse.
  • the systems and methods have an automated component, but are flexible and can be repeatedly modified to enhance the crop-related information that they provide.
  • Some embodiments also capture and integrate knowledge from human experts into automated crop monitoring systems and methods. Their expertise can be effectively and efficiently captured and applied via an automated system that acts as a proxy, making the technology an extension of the grower.
  • embodiments of the systems and methods described herein can provide predictive models that can be used to predict future health of plants in a crop based on their current sensor data, so that steps can be taken to try to avoid deterioration in the health of the plant.
  • a predictive model will provide the capability to identify a potential issue with a plant, before any single sensor or a human expert could detect a problem.
  • Embodiments of the systems and methods described herein rely primarily on the detection (through sensors) and interpretation (through data analysis) of plant-based signals to provide information about crop health.
  • Monitoring and assessing crop or plant health as described herein can include monitoring and assessing performance of the crop or plant. Performance is generally related to the health of the crop or plant.
  • Crop monitoring system 100 comprises a mobile sensory platform 110 comprising a plurality of sensors mounted on a vehicle, cart or drone for example.
  • Mobile sensory platform 110 captures sensor data related to plants in the crop and transmits it to a data processing unit (DPU) 140 via a network 160 .
  • DPU data processing unit
  • mobile sensory platform 110 comprises more than one mobile sensory platform, and the platforms may communicate and exchange information with one another, as well as with DPU 140 .
  • DPU 140 analyzes the sensor data and sends information regarding the crop to an individual 180 , such as a grower and or/other parties via one or more end-user devices, such as a smart phone 170 a and/or a computer 170 b. DPU 140 may also send commands to mobile sensory platform 110 . Grower 180 or other parties may also send information to DPU 140 and/or send commands to mobile sensory platform 110 via network 160 .
  • system 100 is a web-based and/or cloud-based system, and communication between mobile sensory platform 110 , DPU 140 , and grower 180 and/or other parties or devices, is primarily or entirely wireless communication.
  • the mobile sensory platform is designed to operate in the dark, for example, at night. This can be beneficial as it can reduce interference with other greenhouse or field operations.
  • the monitoring system and method may operate with greater sensitivity at night as plants tend to be dormant during periods of darkness. During the daytime, normal practices of staff tending to the crop might temporarily stress the plants, for example moving plant heads, removing shoots, picking fruits, and the like.
  • the sensors on the mobile sensory platform are proximate to the plant during sensing and data capture, but do not touch the plants or soil. Such non-contact monitoring can help to reduce the spread of pests and diseases.
  • the mobile sensory platform is configured to move autonomously among the plants, or in response to commands from a controller, which in some embodiments is a component of the data processing unit.
  • FIG. 2 is a block diagram illustrating components of an embodiment of a mobile sensory platform, such as platform 110 of FIG. 1 .
  • Mobile sensory platform 110 comprises a plurality of sensors 112 (as described in further detail below) and a data storage device 114 for storing data captured from sensors 112 .
  • Mobile sensory platform 110 also comprises a CPU and control system 116 with associated software 118 , and a communications system/interface 120 .
  • an external serial connection is provided to allow a user to connect a PC or other device, for example, to modify software on the platform 110 .
  • Mobile sensory platform 110 further comprises a propulsion system 122 (for example this could comprise an electric motor, wheels, propellers etc.), a power supply 124 (for example, a battery or other energy storage device and associated recharging equipment and/or a power management system that, in some embodiments, provides a switching between multiple energy sources. Power management system may also incorporate safety and protection devices.
  • Mobile sensory platform 110 may also comprise a GPS and positioning system 126 or similar location tracking system, and a display and/or user interface 128 . In some embodiments GPS and positioning system 126 produces and associates a location tag with the sensor measurements; location tags can be stored in data storage device 114 and also transmitted, along with sensor data, via communications system/interface 120 .
  • Mobile sensory platform 110 may also comprise an on-board data processing unit, and other components (not shown in FIG. 2 ).
  • FIG. 3 is a block diagram illustrating components of a data processing unit (DPU), such as DPU 140 of FIG. 1 .
  • DPU 140 comprises at least one CPU 142 , a controller 144 , software 146 , a power supply 148 , a communications system/interface 150 , and a user interface 152 .
  • DPU can also comprise one or more databases 154 for storing raw and/or processed sensor data and/or other information.
  • Method 400 is a two-phase data collection and analysis method.
  • the first phase is a rapid pre-screening phase 405 to identify plants that may have a problem, for example by identifying plants that exhibit a variance based on analysis of their sensor data.
  • data processing system (DPU) 140 activates mobile sensory platform 110 (of FIG. 1 ) for pre-screening.
  • mobile sensory platform performs pre-screening, moving between the plants and capturing sensor data relating to some or all of the plants in the crop.
  • pre-screening phase 405 Only one sensor or a sub-set of the plurality of sensors 112 is used in pre-screening phase 405 .
  • sensor data from the rapid pre-screening is transmitted from mobile sensory platform 110 via network 160 to DPU 140 , where the pre-screening data is processed and analyzed at 430 . Plants that may have a problem are identified and tagged for further inspection.
  • each row of crops may be identified using a digital beacon or an RFID tag to signify the beginning of the row.
  • the position of each plant in that particular row can be calculated based on its distance from the beacon or tag.
  • DPU 140 develops a secondary screening assignment for mobile sensory platform 110 .
  • the secondary screening assignment can be used to gather more detailed information about the crop or specific plants in the crop.
  • the second phase of method 400 is a secondary screening phase 435 which begins at 440 .
  • Commands related to the secondary screening assignment are transmitted to mobile sensory platform 110 via network 160 . These commands can include commands to activate certain sensors or to cause the mobile sensory platform to move to capture data from particular plants or regions or the crop, and/or the route that should be taken for the secondary screening.
  • Mobile sensory platform 110 performs the secondary screening at 445 in accordance with these commands.
  • mobile sensory platform 110 may move to potential problem plants identified in pre-screening phase 405 , and capture further sensor data from those plants using additional ones of the plurality of sensors 112 .
  • the particular sensors used in the secondary screening are selected based on the analysis of the pre-screening sensor data captured during the pre-screening phase.
  • Sensor data gathered during the secondary screening phase is transmitted from mobile sensory platform 110 via network 160 to DPU 140 at 450 .
  • the sensor data is analyzed at 460 to provide information related to the crop.
  • the mobile sensory platform used for secondary screening (for example, at 445 ) is different from the mobile sensory platform used for pre-screening (for example, at 415 ).
  • a multi-sensor device is mounted to a mobility platform such as an all-terrain rover robot with the capability to go to specific areas of a farm or greenhouse.
  • the rover robot carries a drone that is equipped with one or more sensors; for example, these can be a subset of the sensory devices that are on the multi-sensor device that is mounted to the rover robot.
  • the drone flies over a specific area of the crop and performs pre-screening.
  • the drone lands or docks on the rover robot, where it may be re-charged for example.
  • Sensor data from the pre-screening phase may be transmitted from the drone to DPU 140 during the pre-screening phase, or may be transmitted from the drone to DPU 140 (directly or via the rover robot) upon completion of the pre-screening phase.
  • DPU 140 develops a secondary screening assignment for the rover robot, and commands related to the secondary screening assignment are transmitted to the rover robot via network 160 (and optionally via the drone).
  • the rover robot then performs a secondary screening phase, typically in closer proximity to the plants than the drone, and transmits data for analysis by DPU 140 as described above.
  • DPU 140 transmits information to one or more end-user devices at 470 .
  • information may be sent to the grower 180 or others via one or more end-user devices, such as smart phone 170 a and/or computer 170 b.
  • Such information can include, for example: information about the condition of the crop or individual plants, diagnostic information, alerts, action plans, suggested treatments or interventions and the like.
  • DPU 140 may also send commands to mobile sensory platform 110 to implement one or more interventions in order to attempt to remediate an adverse condition affecting one or more of the plants, as indicated at 480 .
  • DPU 140 could command mobile sensory platform 110 to disperse a bio-control agent.
  • DPU 140 could also activate other systems at the crop-site to implement one or more interventions in order to try to remediate an adverse condition affecting one or more of the plants. For example, it could activate an irrigation system, adjust a temperature control system or cause nutrients or pesticides to be automatically administered to certain plants.
  • Events in method 400 can be sequential (with one event being completed before the next begins), or in some cases they can overlap.
  • information relating to one portion of the crop may be being transmitted from the DPU to the one or more end-user devices at the same time as mobile sensory platform 110 is still capturing sensor data for another portion of the crop.
  • two different mobile sensory platforms are used for pre-screening and secondary screening (for example, a drone and a rover robot), one platform can be performing pre-screening at the same time the other is performing secondary screening, for example.
  • the sensor data is analyzed by using pre-established scientific models and/or comparing the sensor data to known indices or standards.
  • DPU 140 may draw on information stored in other databases for this purpose.
  • particular sensors can be used to look for particular issues with the plants. Data can be collected and compared to a catalogue of plant stress signatures to identify particular problems with the plant or to determine whether the plant is healthy. Some sensors are used to look for specific chemical volatile indicators of a particular pest infestation.
  • Such conventional sensors usually function during daylight hours, are generally large and expensive, and require a human operator. Also it can be difficult for conventional sensors to pinpoint specific causal factors in a real crop growing situation where plants are exposed to many potential stressors.
  • sensor data is compared to a profile or signature known to be indicative of a healthy plant.
  • the DPU 140 looks for deviations from this healthy plant profile. This can be done by classifying the sensor data against a set of “trained data” or a model derived therefrom.
  • the trained data can be derived from expert assessments as described in further detail below.
  • crop monitoring systems and methods similar to those described above are based upon or enhanced by the utilization of human expertise, for example, from an expert such as an experienced grower, farmer or professional crop advisor assessing the same crop or a similar crop in a similar environment.
  • Embodiments of such systems and methods can capture and integrate this expert knowledge.
  • the expert knowledge is captured from a human expert in a way that allows it to be re-applied later or elsewhere by an automated system and/or used to teach a non-expert.
  • the system is “trained” based on correlating an assessment of the health of individual plants, as inputted by an expert, with sensor data captured for the same plants.
  • training refers to a process of establishing a repeatable, statistically meaningful relationship between observed data (such as sensor data) and “truth” data—in this case the assessment of an expert—to give “trained data” that can be used as a reference for the classification of new data.
  • This is a form of supervised learning.
  • the trained data can be used in evaluating plant-related sensor data subsequently received from a mobile sensory platform (e.g. in unsupervised learning). Thresholds for different classifications can be established through this process, and data-derived models that can be used for classification and analysis of new data can be developed.
  • a crop monitoring and assessment system can apply the expert knowledge on an on-going basis during automated monitoring and assessment of the crop, without the need for a human expert to inspect the entire crop.
  • This can make the technology effectively an extension of an expert, such as an experienced grower, farmer or professional crop advisor.
  • FIG. 5 An embodiment of a crop monitoring system 500 that captures and uses expert knowledge is illustrated in FIG. 5 .
  • crop monitoring system 500 is similar to crop monitoring system 100 of FIG. 1 .
  • System 500 comprises a data processing unit (DPU) 540 that receives data from a plurality of sources via network 560 .
  • DPU 540 can be similar to DPU 140 illustrated in FIG. 1 and FIG. 3 .
  • Handheld mobile devices 570 a and 570 b comprise a plurality of sensors for capturing sensor data from plants. The handheld devices also provide a way of tagging, locating or identifying the plant associated with the sensor data.
  • Handheld device 570 a also allows expert 580 to enter an assessment of plant health.
  • Information and sensor data can be transmitted from handheld devices 570 a and 570 b to the DPU 540 via network 560 .
  • Another source of sensor data is a mobile sensory platform 510 comprising a plurality of sensors mounted on a vehicle, cart or drone for example. Mobile sensory platform 510 can be similar to platform 110 illustrated in FIG. 1 and FIG. 2 .
  • Other electronic devices 565 may be used to enter and transmit information and data about the crop to DPU 540 via network 560 , for example crop conditions, planting times, source of seeds, environmental factors and the like.
  • Other data sources 575 that can transmit data to DPU 540 via network 560 may include, for example, fixed sensors located around the crop-site.
  • Software managing the DPU can be based on an open application program interface (API) structure that allows information exchange and integration from and to multiple external data sources.
  • API application program interface
  • CDSI Cross-domain semantic interoperability
  • DPU 540 stores sensor data and information that it receives in one or more databases 545 . It also performs data correlation (correlating an assessment of the health of individual plants as inputted by the expert with sensor data captured for the same plants) and stores the resultant “trained data” in one or more databases 545 . DPU 540 can then analyze plant-based sensor data using the trained data and/or one or more models derived therefrom, as described in further detail below. In some embodiments, DPU performs more complex analysis of current and historical sensor data, for example, using data mining and machine learning techniques to generate predictive models. In performing data analysis, DPU 540 may supplement the analysis of sensor data using information stored in other databases, such as pre-established scientific models and known indices or standards.
  • DPU 540 can also transmit information regarding the condition of the crop to expert 580 , non-expert 585 and or/other parties such as a grower or crop-site manager 590 via one or more end-user devices, such as a smart phones, computers, handheld devices such as 570 a, 570 b, 570 c and other electronic devices. These can be located at the crop-site or remotely. DPU 540 may also send commands to mobile sensory platform 510 .
  • arrows are used to indicate transmission of sensor data and/or other information.
  • the system is a web-based and/or cloud-based system, and communication between mobile sensory platform, data processing unit, and the expert, non-expert and or/other parties or devices, and is primarily or entirely wireless communication
  • System 500 is further described in reference also to FIGS. 6 through 10 which describe a method 600 for crop monitoring and assessment.
  • Method 600 comprises four activities: expert knowledge capture and system training 700 , non-expert data acquisition 800 , mobile sensory platform data acquisition and analysis 900 , and information dissemination 1000 .
  • the four activities can, for example, occur repeatedly in series, can be performed in different sequences over time, can overlap, or can occur at least to some extent in parallel.
  • Method 600 of FIG. 6 starts at 610 , for example, when crop monitoring begins.
  • method 600 branches into the four activities: expert knowledge capture and system training 700 , non-expert data acquisition 800 , mobile sensory platform data acquisition and analysis 900 , and information dissemination 1000 . Each of the activities are described in further detail in the following paragraphs.
  • Expert knowledge capture and system training 700 is a fundamental basis of the overall method. As shown in FIG. 7A , activity 700 commences at 710 when an expert enters the crop-site (e.g. field or greenhouse) to assess plants in the crop. At 720 the expert (such as expert 580 of FIG. 5 ) evaluates individual plants and makes an assessment of their condition or health. The expert also captures sensor data from the plants. Location information or plant tagging information is also captured so that the expert assessment and sensor data can be associated with each particular plant. In some embodiments, the expert can be equipped with a handheld or wearable device (e.g. 570 a of FIG. 5 ) comprising one or more sensors for sensing the plants and capturing and transmitting sensor data via a network (such as 560 shown in FIG.
  • a handheld or wearable device e.g. 570 a of FIG. 5
  • a network such as 560 shown in FIG.
  • sensor data and expert assessments are transmitted to the DPU. This can be done in real-time plant-by-plant or once the expert has completed that particular visit to the crop. Preferably it is transmitted wirelessly and in real-time.
  • Activity 700 is typically performed during the day. Multiple experts may perform expert knowledge capture and system training activity 700 simultaneously or at different times for a given crop.
  • the DPU correlates the assessments of the health of individual plants (as inputted by the expert) with the sensor data captured for the same plants to generate trained data and, in some cases, one or more data-derived models.
  • the correlation can involve the use of machine learning and classification, and the development of pattern recognition models.
  • the trained data resulting from the correlation is stored. Once there is sufficient trained data to give a reasonable level of accuracy, the trained data, or models derived therefrom, can be used as described in reference to activities 800 , 900 and 1000 below.
  • additional expert assessments and associated sensor data can be added, processed and included in the stored trained data and/or used to further enhance the models. This accumulation of expert knowledge will generally improve the accuracy of the crop monitoring and assessment system over time.
  • FIG. 7B shows a screen-shot 700 B from a system training app (i.e., a software program or application that can be used on a mobile electronic device, such as a smartphone or tablet) that can be used by an expert to capture and convey to a DPU their assessment of plants in a crop.
  • the expert can assess the plant as healthy or can select the level of different problems; in the illustrated example these are diseases, pests and deficiencies.
  • the custom options allow the expert to assess the plants with respect to other positive or negative attributes of their choosing. For example, the expert may choose a custom profile or feature that they prefer to see in their crop based on their knowledge and experience. This way, growers can create a profile of a healthy plant, an unhealthy plant and/or in some cases, a custom attribute that they want to track. Corresponding sensor data for each plant is also transmitted to the DPU.
  • an expert may look for plants that are healthy and for plants that are exhibiting a particular problem (e.g. a specific disease or pest) and capture and transmit sensor data along with their expert assessment for just those plants.
  • a particular problem e.g. a specific disease or pest
  • unsupervised learning processes can be used to test the resulting data-derived models for accuracy on a new set of unclassified or unlabeled data.
  • Non-expert sensor data acquisition activity 800 is illustrated in the flow chart of FIG. 8 , and involves the collection of additional plant-based sensor data by a human who is a non-expert.
  • the non-expert is not as skilled or experienced as the expert at accurately assessing the health of plants in the crop. They might be a worker who works in the greenhouse attending to the plants (feeding, pruning, harvesting etc.).
  • Non-expert sensor data acquisition can provide useful data from additional plants in the crop that can be used to develop assignments for automated crop monitoring activities that may happen overnight, for example. It can also be used so that the non-expert can learn how to assess plant health “like an expert”—this learning aspect is described in further detail below.
  • Non-expert sensor data acquisition activity 800 may be performed on a daily basis, frequently or not at all, as part of overall method 600 .
  • activity 800 commences at 810 when a non-expert enters the crop-site to capture sensor data for plants in the crop.
  • the non-expert (such as non-expert 585 of FIG. 5 ) captures sensor data for a plant, along with location information or plant tagging information so that sensor data can be associated with the particular plant.
  • the non-expert is equipped with a handheld or wearable device (e.g. 570 b of FIG. 5 ) comprising one or more sensors for sensing plants and capturing and transmitting sensor data to a data processing unit (such as DPU 540 shown FIG. 5 ).
  • the handheld sensor device can be similar to or the same as the one used by the expert in activity 700 .
  • Sensor data capture may happen passively as the non-expert moves from plant to plant performing other tasks, or may require the non-expert to activate the sensors and capture sensor data, for example, by pressing a button.
  • sensor data is transmitted to the DPU. This can be done in real-time plant-by-plant (as shown in FIG. 8 ), or for multiple plants once the non-expert has completed that particular visit to the crop. Preferably it is transmitted wirelessly and in real-time.
  • Activity 800 is also typically performed during the day. Multiple non-experts may be performing sensor data acquisition activity simultaneously or at different times for a given crop. For example, a large crew of workers could be equipped with handheld or wearable sensor devices in order to capture plant-based sensor information while they are busy performing other tasks related to the crop.
  • the DPU classifies the condition of each plant by passing the sensor data through a model derived from the trained data (generated from expert knowledge capture and system training activity 700 ). Plant health information based on this classification can be disseminated for various purposes as described below in reference to activity 1000 of FIG. 10A . In some embodiments, such as at 1040 of FIG. 10A , information is sent back to the non-expert. For example, referring again to FIG.
  • handheld sensor data captured by the non-expert for each plant is analyzed in real-time by the DPU and, at 850 , the non-expert may receive an immediate assessment of the condition of the plant from DPU 540 via their handheld device 570 b. For example, this could be a simple ranking of the plant's condition (e.g. red, orange, green for poor, moderate, healthy respectively).
  • the real-time assessment delivered to the non-expert is based on a model derived from trained data that was derived from expert assessments in activity 700 . In this way the non-expert can inspect the plant and learn how it would have been assessed by an expert, without the expert needing to be present to teach the non-expert. Once the non-expert has received feedback on a particular plant at 850 they can move on to another plant at 860 if desired.
  • Mobile sensory platform data acquisition and analysis activity 900 is illustrated in the flow chart of FIG. 9 , and is an important aspect of method 600 .
  • This activity can be performed during the day, or at night when there might be reduced interference with other greenhouse or field operations. At night the sensing may be more sensitive to the presence of disease or pests, as plants tend to be dormant and less stressed by other external factors during periods of darkness. Multiple mobile sensory platforms may be used simultaneously to cover different regions of the crop, for example, to allow the whole crop to be assessed in a shorter time-frame.
  • Activity 900 starts when a mobile sensory platform (such as mobile sensory platform 510 of FIG. 5 ) is activated to move between the plants and capture data relating to some or all of the plants in the crop.
  • the mobile sensory platform may have a different number of sensors than the handheld devices used by the expert and non-expert in activities 700 and 800 . In some cases, it will have a greater number of sensors, but not always. Also, it may have a different set of sensors usually, but not always, with some sensor types in common with the handheld devices.
  • the mobile sensory platform can be operated, for example, in a similar manner to that described in reference to FIG. 4 , with a pre-screening phase and a more detailed secondary screening phase. In other embodiments mobile sensory platform can be operated with a single-pass screening operation.
  • sensor data is captured at 920 .
  • Location information or plant tagging information is also captured at 920 , so that sensor data can be associated with each particular plant.
  • data is transmitted to the DPU.
  • the sensor data from the mobile sensory platform can be transmitted to the DPU in real-time or once a portion or all of the screening session is completed. Preferably it is transmitted wirelessly.
  • the raw information (including plant identifier/locator and sensor data for each plant that was evaluated) can be stored.
  • further correlation can be performed at 940 to generate additional trained data and/or to enhance data-derived models. For example if data has been captured from sensor-types on the mobile sensory platform that are not on the handheld devices, this data may be correlated with the expert assessments obtained for the same plants during activity 700 to provide further trained data and/or enhance models that can be stored and used for classification of plant health.
  • the DPU classifies the condition of each plant by applying a model, derived from the trained data, to the sensor data received from mobile sensory platform at 930 .
  • Plant health information based on this classification can be disseminated for various purposes as described below in reference to activity 1000 of FIG. 10A .
  • commands are sent to the mobile sensory platform based on the analysis of sensor data received from the mobile sensory platform. For example, at 960 (see FIG. 9 ), commands are transmitted to the mobile sensory platform to cause it to implement one or more interventions in order to attempt to remediate an adverse condition affecting one or more of the plants, for example, at 960 DPU 540 could command mobile sensory platform 510 to disperse a bio-control agent.
  • Information dissemination activity 1000 is illustrated in FIG. 10A which shows some non-limiting examples of how information can be delivered and used.
  • Activity 1000 starts at 1010 , for example, when there is updated information about the crop available to disseminate or when an end-user desires or requests information.
  • activity 1000 branches into the four exemplary information dissemination activities which can occur asynchronously. For example, these can each occur repeatedly, can occur at different times, or can overlap, or occur simultaneously.
  • information about the crop is transmitted to an expert, such as grower 580 in FIG. 5 .
  • This information could be delivered each morning, for example, based on analysis and classification of sensor data captured overnight for the entire crop by mobile sensory platform 510 performing activity 900 .
  • it could be delivered in real-time as the DPU analyzes data received from the mobile sensory platform in real-time.
  • DPU could provide a grower with an alert, an alert plus a diagnosis of the problem, or an alert plus a diagnosis plus a suggested intervention plan, for specific plants or regions of the crop that are not healthy.
  • a grower might use an interactive map of a farm or greenhouse where problematic areas identified by the DPU are marked by dots on the map.
  • the grower clicks on each dot specific information about the type or severity of the issue at that location may be displayed, along with a suggested intervention plan.
  • the information delivery will be done via a wearable device, which can be used by growers, experts and/or non-experts.
  • the DPU may generate a notification, for example, in the form of an audible alarm or a haptic vibration that occurs when the wearer of the device comes in close proximity to a problematic area.
  • the intervention plan may be communicated as part of the notification.
  • information about the crop is transmitted to a non-expert, such as 585 in FIG. 5 .
  • this information could be delivered to a worker each morning to guide the worker to specific areas of the crop that need intervention based on analysis and classification of sensor data gathered overnight for the entire crop by mobile sensory platform 510 performing activity 900 . This could allow the worker to apply interventions only to those specific areas, as opposed to general broad-based applications, thereby reducing costs and exposure risk.
  • information is delivered in real-time to a non-expert performing activity 800 of FIG. 8 so that the non-expert can learn on a plant-by-plant basis how an expert would assess plant health. This is described above in reference to 850 of FIG. 8 .
  • information and/or commands are transmitted to a mobile sensory platform (such as platform 510 of FIG. 5 ) based on analysis and classification of sensor data received from the mobile sensory platform.
  • a mobile sensory platform such as platform 510 of FIG. 5
  • these could be further screening assignments or commands to perform interventions based on analysis and classification of sensor data received from the mobile sensory platform. The latter is described in more detail above in reference to 960 of FIG. 9 .
  • At 1060 information about the crop is transmitted to other parties or devices either at the crop-site or at other locations.
  • information can be pushed from the DPU or can be pulled upon request by the end-user or device.
  • FIG. 10B shows a screen-shot 1000 B from a reporting app that can be used to convey information about the crop to a person, such as a grower.
  • the growers may define number of phases they have in their greenhouses and number of bays in each phase as well as the number of rows in each bay, thereby creating a map of the greenhouse.
  • the dynamic mapping panel on the left alerts users to the location of problems in their crops—it indicates the phase and bay of the greenhouse for which information is being reported. In the top right quadrant the rows of that particular bay are shown.
  • the dots indicate plants identified by the system as deviating from a healthy profile.
  • the user can click on a dot and the location of the plant is more precisely identified, and a description of the problem and an indication of the probability of the problem is displayed, as shown in the lower right quadrant.
  • a plant at post 11, in row 3 of bay 14B in phase 1 of the greenhouse is indicated as having a high probability that it is suffering from a sucking pest infestation and a lower probability that the problem is a bacterial disease.
  • the knowledge of an expert can be captured and then extended and applied at future times and/or at other locations without the expert being physically present at those times or in those locations. It is extremely valuable to be able to harness an expert's knowledge and experience in this way, both for teaching other people how to assess crops and for actual crop assessment.
  • sensor data from similar crops in other (remote) locations can be captured via mobile sensory platforms and/or handheld devices and then one or more data-derived models in the DPU can be applied to the sensor data to provide crop health assessment information about that crop without an expert needing to be there at all.
  • Another advantage of the present approach is that the machine-based analysis of the data by the DPU will provide a more consistent and accurate assessment of plants than a human. Generally, even an expert will not always provide a consistent assessment through the day or from one day to the next, due to a variety of factors such as fatigue, distractions or challenging environmental conditions, for example.
  • FIG. 11 shows a DPU 1100 receiving inputs including expert assessments 1110 , mobile sensor data 1120 , and handheld sensor data 1130 as described above.
  • DPU 1100 can also receive human input 1140 or input derived from other sources via other devices—for example, personal observations, information about other events that may have affected the crop such as planting, watering and harvesting schedule information.
  • DPU can also receive other crop-site sensor data 1150 , for example, from fixed sensors located around the crop-site such as temperature, moisture, light, and air-flow sensors and cameras, and/or from secondary mobile sensors such as drone-mounted sensors.
  • DPU 1100 may also draw on information stored in other databases 1160 , such as pre-established scientific models and known indices and standards, or trained data from other crop-sites. This additional input can also be correlated with the expert assessment as described above to generate enhanced trained data.
  • DPU 1100 analyzes incoming data and information and provides crop-related information as output 1170 .
  • sensor data can be collected and analyzed, for example, in real-time to classify the current health of a plant. It can also be useful to store and re-analyze such data at a future time. For example, once sensor data is collected for the same plant over a period of time, historical spatiotemporal data can be reverse-engineered or re-analyzed in the context of data that is collected later. Once it is known that a plant is suffering from a problem, it may be possible to retroactively derive patterns or trends from the earlier data for that plant that provided indications that the plant was beginning to deviate from a healthy profile. These clues may exist in the data, even before a less sophisticated data-derived model or an expert would be able to discern a problem. This type of analysis of historical data can be used in the development of predictive models which can then be used to predict health issues and allow intervention before the plant exhibits visible symptoms.
  • the historical data can include:
  • plant-related sensor data e.g. from handheld devices and mobile sensory platforms
  • information about how the crop was managed for example information about seed source, planting time, irrigation, nutrition, pruning, spraying, harvesting;
  • Information relating to the actual performance of the crop can also be gathered (for example yield, quality, appearance, taste, shelf-life etc.). For example, this can be based on information provided by the grower or other humans (e.g. feedback from customer) and/or data that is captured automatically. Using predictive analytics, this performance information and data can be correlated with data gathered during the lifecycle of the crop to look for patterns and indicators earlier in the crop's lifecycle that are indicative of future performance. For example, by looking at portions of the crop (e.g. specific plants or groups of plants) that performed particularly well or particularly poorly, and analyzing past data for these portions of the crop it may be possible to correlate performance with particular growing conditions (e.g. based on the crop management information and environmental data) and or plant-based sensor data.
  • portions of the crop e.g. specific plants or groups of plants
  • particular growing conditions e.g. based on the crop management information and environmental data
  • This information can then be used in the future to try to re-create desirable growing conditions and achieve these over a larger portion of the crop, thereby enhancing performance of the crop in subsequent plantings. Similarly it can be used to identify and try to avoid adverse growing conditions, or to alert the grower when a region of the crop is exhibiting characteristics (e.g. based on monitored sensor data) indicative of future poor performance, so that remedial action can be taken. It can also be used to evaluate the effect of interventions that were performed in trying to mitigate problems with the crop, so that the effectiveness of the interventions can be improved.
  • FIG. 12 is a flow chart illustrating an embodiment of such a process 1200 .
  • a first phase 1205 of process 1200 information relating to the actual performance of a crop is correlated with data gathered during the lifecycle of the crop to identify patterns and indicators.
  • these patterns are then used to attempt to improve the performance of a future crop.
  • desired attributes are identified and are assessed for plants in the crop. For example, information relating to yield, taste and other attributes can be collected.
  • the plants are classified into groups based on their performance against one or more of these attributes (e.g. high, medium and low performance).
  • performance information for the classified groups of plants is correlated with historical data gathered during the lifecycle of the plants to identify patterns. These patterns can be developed into weekly trends for various parameters, at 1240 . The trends are associated with the performance level of each group of plants.
  • a new crop is planted and information is captured for plants in the new crop.
  • parameters for plants in the new crop are compared with the historical weekly trends for those parameters that were obtained for the previous crop at 1240 .
  • attempts can be made to correct those deviations through various interventions, as shown at 1260 .
  • Activities 1250 and 1260 can continue through the growing season for the new crop.
  • the performance of plants in the new crop is assessed with respect to one or more of the desired attributes, and again the plants are classified into groups, as shown at 1270 .
  • this new performance information is correlated with data gathered during the lifecycle of the plants.
  • this information is used to update and improve the patterns and weekly trends that can be used to try to improve the performance of the next crop.
  • multi-dimensional data points are collected non-invasively from an individual plant in just a few seconds, allowing physiological, chemical, biological and/or physical changes inside, on the surface and/or around each plant to be detected.
  • the technology described herein has the potential to capture massive volumes of spatiotemporal data relating to one or more crops over one or more growing seasons.
  • the DPU can develop specific performance patterns and data-derived models that can be used for classification and analysis of new data. Predictive models can also be developed, that can be used to predict future health or performance of plants in a crop based on their current sensor data.
  • plants that are on a path towards deteriorating health can be identified based on early clues that can be derived from their current multi-sensor data, in some cases before any single sensor or an expert could detect symptoms of problem. With this early-stage detection, preventative measures can then be taken to try to avoid deterioration in the health and/or improve the performance of the plant.
  • Deep learning techniques can be used, for example, for feature selection.
  • Feature selection methods can identify attributes in the data that will enhance accuracy of a predictive model, and identify and remove irrelevant and redundant attributes in the data that do not contribute to the accuracy of a predictive model or that may even decrease the accuracy of the model.
  • the system can pool the assessments from multiple experts from different farms or sites with respect to a particular type of crop, for example, and then use this combined digitized expertise to enhance the quality and consistency of the data-derived models.
  • Growers who are generating and providing crop-related data for development of data-derived models, as well as for automated assessment of their own crops, can then become data entrepreneurs. This is a potential source of revenue generation for growers who opt to sell their generic and non-proprietary crop-related information (such as trends and statistics), for example, to other growers or to the provider of a platform that provides data processing of crop-related data for multiple growers.
  • growers may contribute data or statistics to a centralized or shared data-derived model, and then receive a revenue stream based on the amount of their contribution and/or based on the extent to which the model is used for analysis of third party crop-related data.
  • At least some processing and/or analysis of certain types of sensor data is performed on the mobile sensory platform or sensor device itself instead of at a remote DPU.
  • Statistics or information derived from the sensor data, rather than the sensor data itself, is then transmitted to the DPU. This can reduce the volume of data that needs to be transmitted and can increase the overall speed and efficiency of the process.
  • data that is gathered by optical sensors or stereo cameras for the purposes of disparity and depth analysis or verification purposes could be processed on the mobile sensory platform, and then the relevant information could be transmitted to the DPU along with data from other sensors.
  • NLP Natural Language Processing
  • the NLP capability can allow experts to describe the condition of a crop verbally while capturing a sensory profile. The same terminology can be used for the reporting app.
  • the NLP may receive and deliver information in various languages.
  • the mobile sensory platform employed in the systems and methods described above generally comprises more than one type of sensor mounted on a mobile platform.
  • the mobile platform can be a vehicle or cart, such as an automated robot that can patrol between rows of crops, or a drone that can fly over or in between rows of crops.
  • the mobile sensory platform will include a mounting structure such as a scaffold or rack that supports a plurality of sensors and optionally additional probes and/or devices.
  • the mobile sensory platform can comprise a mast with attachable, extendable arms, or a column that houses fixed sensors and probes, or a dome that mounts on or under a mobile platform.
  • the mobile sensory platform can comprise some or all of the following types of sensors:
  • Physiological sensors these include sensors and probes that can measure physiological performance of crops and/or detect minute changes inside the plant caused by biotic and/or abiotic stressors. For example, chlorophyll fluorescence emitted from the leaves can provide insight into the health of the photosynthetic systems within the plant. Sensors can be used to sense the level of chlorophyll in leaves, and/or photosynthetic efficiency, and/or changes in internal chemical composition related to stress. These sensors can include pulse-modulated optical probes and detectors that stimulate the plant to give a physiological response and then detect that response. The probes might consist of LEDs with specific spectral bands that are used to excite plants and generate various physiological responses that can be correlated to photosynthetic activity or defensive chemicals inside plant foliage.
  • the detectors may be tuned to be responsive to a narrow spectral band that corresponds with specific light that is reflected from or emitted by plants.
  • these sensors will provide the earliest clues that the plant is developing a problem, whereas some of the other sensor types described below will detect changes that occur as a disease, pest or other problem becomes further developed.
  • the reaction of plants to stress typically begins with internal changes in the physiology and chemistry of the plant. This family of sensors can detect those early stage changes and prime the system to conduct further analysis to verify and identify the source of stress.
  • Surface analysis sensors include sensors and probes that can detect changes on the surface of the leaves and other parts of plants, for example, changes in color related to water stress, changes in surface chemistry related to biotic and abiotic stress, physical attributes of leaf surface.
  • Such sensors generally involve spectral detection to detect certain wavelengths of visible (RGB) and near infra-red (NIR) light reflected by the plant.
  • the probes used with these sensors may consist of full spectrum light sources, such as halogen lamps, or probes with narrow spectral bands such as ultraviolet (UV) or near infra-red (NIR).
  • UV ultraviolet
  • NIR near infra-red
  • Chemical sensors include sensors and probes that can detect changes in plant-emitted volatile chemicals (e.g. volatile organic compounds, known as VOCs), for example, detecting herbivore-induced volatile compounds emitted by plants while under pest attack.
  • volatile chemicals e.g. volatile organic compounds, known as VOCs
  • PIDs photo-ionization detectors
  • SAW surface acoustic wave
  • QMB quartz crystal microbalance
  • Thermal sensors these may include thermal imaging sensors that can give information about surface damage to the foliage or fruit. For example, tiny holes that could be caused by a pest will tend to increase moisture loss and evaporation, resulting in localized lower surface temperatures that can be detected by thermal imaging.
  • Microclimate sensors these include sensors and probes that can monitor changes in the microclimate around individual plants, for example, temperature and relative humidity.
  • Canopy scanning sensors include sensors and probes that can detect changes in canopy structure, for example, changes in leaf angle in response to water stress or viral infection. These can include ultrasound and/or LiDaR (light detecting and ranging) type sensors, or stereo-imaging (visible RGB and IR) sensors, for example. Such sensors may be used, for example, to generate disparity maps (providing depth measurement and information about the 3D structure of the plant canopy) which can give information about plant growth. Also they may be used to provide various vegetation indices.
  • the crop monitoring systems and methods described herein can function with little or no reliance on visual sensors or imaging.
  • the mobile sensory platform does not comprise cameras or other imaging devices.
  • one or more cameras or imaging devices are used primarily for verification purposes (e.g. so that a grower can inspect a photographic or video image of a plant that has been assessed by the automated system as having a problem, without having to physically go to the plant to visually inspect it).
  • the imaging devices might be installed on a drone or other flying platforms.
  • the position of some or all of the sensors is adjustable so that they can be positioned appropriately depending on the size (e.g. height and volume) of the plant and which region of the plant is to be sensed.
  • the sensors can be moved and re-positioned automatically (rather than manually) based on commands from a control system that is responsive to inputs indicative of where the sensors should be positioned.
  • the mobile sensory platform can further comprise one or more intervention modules for administering remediation to selected plants.
  • Such modules may be mounted to the mounting scaffold to disperse bio-control agents or other pest and disease management products where and when they are needed.
  • the mobile sensory platform will be charged daily via a stationary charging station installed inside a greenhouse or at the farm.
  • the charging station can be powered by AC electricity or via solar panels.
  • the mobile sensory platform can move among the rows of crops.
  • the mobile sensory platform moves on rails, such as rails that are sometimes installed in greenhouses for other purposes.
  • the platform may detect a rail adjacent to a first row of the plants using one or more sensors and then position itself to move along the rail adjacent to the first row, or may be placed by a staff member at the beginning of a first row within a desired zone.
  • the mobile sensory platform may then move down and back between each pair of rows of plants (assuming they are dead-ended rows) until it covers all the rows in the zone.
  • Specific rail detecting sensors or positioning beacons can be used to guide the mobile sensory platform from one row to another.
  • the platform may move itself to the charging station following a pre-programmed route or may remain at the end of its path to be moved by a staff member in the morning.
  • FIGS. 13A, 13B and 13C show a front view and a side view of a mobile sensory platform.
  • FIG. 13A is a simplified drawing showing two orthogonal views of a mobile sensory platform 1300 A having a base 1310 and wheels or rollers 1320 that can move around on the ground and/or on rails.
  • Mobile sensory platform 1300 A includes a mounting scaffold 1330 to which a plurality of sensors 1350 a - e can be attached.
  • Mounting scaffold 1330 is equipped with a data transmission mechanism 1340 that can be placed in various locations on mounting scaffold 1330 .
  • a variety of sensor types are attached at various locations on mounting scaffold 1330 : physiological sensors 1350 a, chemical sensors 1350 b, microclimate sensors 1350 c, surface analysis sensors 1350 d and a canopy scanning sensor 1350 e.
  • Physiological sensors 1350 a and chemical sensors 1350 b can be placed on a rotating arm 1360 that moves both vertically and horizontally on an anchor.
  • Physiological sensors 1350 a include excitation probes 1370 and a signal detector 1375 .
  • Mobile sensory platform can be automated and self-powered so that it moves around the greenhouse or field under the control of a control system.
  • FIG. 13B and FIG. 13C illustrate mobile sensory platforms 1300 B and 13000 , respectively, that are similar to mobile sensory platform 1300 A that is illustrated in FIG. 13A , but with different mounting structures.
  • FIG. 13B and FIG. 13C the same numbers are used to label elements that are the same as or similar to those referred to in the description of FIG. 13A .
  • a sliding actuator 1380 is attached to mounting scaffold 1330 and moves up and down vertically (shown with dashed lines in a lower position).
  • the positioning of actuator 1380 can be based on input from one or more of the sensors (e.g. indicative of the height of the plant or the location of the region of interest on or around the plant).
  • the sliding actuator 1380 carries physiological sensors 1350 a, chemical sensors 1350 b, microclimate sensors 1350 c and surface analysis sensors 1350 d.
  • Mobile sensory platform 1300 C of FIG. 13C comprises a cylindrical mounting scaffold 1390 that houses the physiological sensors 1350 a, chemical sensors 1350 b, microclimate sensors 1350 c, surface analysis sensors 1350 d, and canopy scanning sensor 1350 e.
  • Cylindrical mounting scaffold 1390 is attached to the ground mobility platform 1310 . Cylindrical mounting scaffold 1390 protects the sensors that are placed in various locations inside it.
  • FIG. 14A is a simplified drawing showing two views of an air-borne mobile sensory platform 1400 A that carries a suspended sensory scaffold 1410 .
  • Various sensors 1450 are attached to suspended sensory scaffold, similar to sensors 1350 a - d.
  • Sensory platform 1400 A also includes data transmission mechanism 1440 .
  • Housing 1460 accommodates a propulsion mechanism (not visible) which can include one or more propellers and a motor.
  • FIG. 14B is a simplified drawing showing two views of another air-borne mobile sensory platform 1400 B comprising a dome 1415 positioned underneath housing 1465 .
  • Dome 1415 houses various sensors similar to those described above, and housing 1465 accommodates a propulsion system including four propellers 1490 .
  • the mobile sensory platform further comprises a docking station or similar device where the device can be re-charged.
  • FIG. 14C shows a landing pad 1425 A and 1425 B where an airborne mobile sensory platform, such as 1400 A of FIG. 14A or 1400B of FIG. 14B , respectively, can land and charge its batteries.
  • the landing pads are 1425 A and 1425 B are each fitted with a solar panel 1435 A and 1435 B respectively, that harvests solar energy and turns it in to electrical power that is used to charge the airborne mobile sensory platform.
  • an air-borne mobile sensory platform (such as a drone) can dock with another mobile sensory platform (such as a cart or rover robot) for re-charging and/or data transfer purposes.
  • the landing pad could be on another mobile sensory platform.
  • hand-held devices can be employed in the systems and methods described herein in order to capture human knowledge.
  • Some example embodiments of hand-held devices that can be employed in systems and methods as described herein are illustrated in FIGS. 15A, 15B and 15C .
  • a hand-held multi-sensor device is also mountable to a mobile platform so that it can be used by a person or as part of an automated crop monitoring system (see FIGS. 16 and 17 below, for example).
  • FIG. 15A shows a simplified drawing of an embodiment of a hand-held device 1500 A comprising a portable sensory platform 1510 A that houses various sensors 1550 .
  • Portable sensory platform 1510 A connects to a smartphone or tablet 1520 either wirelessly or by wire 1525 .
  • FIGS. 15B and 15C are simplified drawings of various views and configurations of another embodiment of a hand-held device 1500 B comprising a multi-sensor module 1560 B that houses various sensors 1550 connects to a smartphone or tablet 1520 .
  • Multi-sensor module 1560 B is equipped with a set of sensors that can be positioned in two configurations.
  • sensors 1550 are oriented in-line with the smartphone 1520 (i.e. directed in the plane of the phone). This allows the user to point the sensors on device 1500 B toward a plant and enter their assessment of the health of the plant using smart phone 1520 , based on their expert knowledge. For example, in an assessment mode, an app on the phone may instruct an expert user to point the sensors toward the plant, click a run button to record sensor data, and immediately or simultaneously enter their expert assessment in response to multiple options related to the health of the plant.
  • the multiple choices may be depicted by colored circles, for example, red for unhealthy, orange for moderately healthy and green for healthy.
  • detailed multiple-choice questions may guide an expert user to assess the health of plant.
  • FIG. 15C A second configuration of handheld device 1500 B is shown in FIG. 15C where sensors 1550 on multi-sensor module 1560 E are oriented in a perpendicular position relative to the plane of smartphone 1520 .
  • Multi-sensor module 1560 B includes a pivot mechanism to allow this change in configuration.
  • handheld device 1500 B can be inserted into or clipped to a shirt pocket, for example.
  • a non-expert can carry module 1560 B in this way so that sensors 1550 is directed toward the plants and can capture sensor data as the non-expert performs routine tasks.
  • the data may be processed by a DPU and information or alerts sent back to the non-expert in real-time via device 1560 B as described above.
  • FIG. 16A shows an embodiment of a multi-sensor device 1600 , for crop monitoring, that can be used as a hand-held device or can be mounted to a mobile platform.
  • Multi-sensor device 1600 comprises a housing 1610 and removable sensor modules 1620 a - g that are slotted into housing 1610 .
  • FIG. 16B shows housing 1610 with the sensor modules removed, and shows interior slots or cavities 1615 a - g that can accommodate various sensor modules.
  • Housing 1610 has a mount 1612 a on its upper exterior surface and a similar mount 1612 b (not visible) on its lower exterior surface.
  • FIG. 16C shows multi-sensor device 1600 with sensor modules 1620 a and 1620 b partially removed by sliding out of housing 1610 .
  • Housing 1610 can be constructed from any suitable material or materials including for example, various plastics and/or metals. If the device is to be used outdoors the materials are preferably waterproof, weatherproof and heat-resistant to withstand harsh environmental conditions.
  • FIG. 16D shows housing 1610 with sensor module 1620 d installed, and two alternative sensor modules 1620 h and 1620 i which can replace sensor module 1620 d.
  • Multi-sensor device 1600 incorporates a variety of sensors in a space-efficient manner into a compact, portable device.
  • Sensor modules 1620 a - g can be different types of sensors that can be used to sense and provide sensor data for plant-related parameters in a crop monitoring system.
  • the sensor modules may screw, snap, lock, or be otherwise secured in the housing. Preferably they can be removed and replaced manually or via a simple mechanism.
  • the sensor modules are connected to a universal adapter at the rear of the housing.
  • FIG. 16E shows a rear perspective view of housing 1610 shown in FIG. 16B , and shows separately the bundle of components that is housed in housing 1610 when the multi-sensor device 1600 is assembled.
  • the components include a communications system/interface module 1660 comprising an LCD touch-screen display 1650 (for use as a user interface), and a master control unit (MCU) 1670 comprising a processor and associated software.
  • Multi-sensor device 1600 comprises a universal adaptor (not shown) to which all of the sensor modules 1620 a - g can connect.
  • the universal adapter provides power, data and control signal connectivity between the sensor modules and MCU 1670 , regardless of the connector or port on the sensor.
  • On MCU 1670 there is a device driver associated with each sensor.
  • Multi-sensor device 1600 can be mounted to a handle for use as a hand-held device, or can be mounted to a mobile platform such as a vehicle, cart, robot, drone, or zip-line harness for automated use.
  • FIG. 17A shows an example of handle 1710 having a mounting connector 1715 which can be snapped into or otherwise removably fastened to mount 1612 b on housing 1610 of multi-sensor device 1600 , as illustrated in FIG. 17B .
  • FIG. 17C shows a mounting connector 1720 that can be snapped into or otherwise removably fastened to mount 1612 a or 1612 b on housing 1610 via portion 1725 .
  • Mounting connector 1720 includes tongue 1730 that can be slide-mounted into any of four corresponding grooves 1735 a - d on a mounting bracket 1740 shown in FIGS. 17D and 17E .
  • Bracket 1740 can be connected to or integrated into a mobile platform.
  • Multi-sensor device can be mounted so that is side-facing, upward-facing, downward-facing or at a desired angle relative to the plants.
  • Housing 1610 may have additional mounts on it.
  • Mounting connectors 1715 or 1720 , or bracket 1725 may include a pivot mechanism, so that the angle of orientation of the multi-sensor platform relative to the plants can be adjusted.
  • Mounts 1612 a and 1612 b also serve as power connectors via which an external power source can be connected to provide electrical power to the multi-sensor device 1600 .
  • handle 1710 can comprise a rechargeable battery or, if the device is mounted to a mobile platform, it can be connected to receive power from a power source on-board the device.
  • Housing 1610 can be populated with different sensor modules. This modularity and flexibility makes multi-sensor platform versatile easily adaptable for different crop-sensing applications.
  • a multi-sensor device comprises the following modules contained within a single housing:
  • a single plant can be assessed using all of the sensor modules in the multi-sensor device within a few seconds or less.
  • the optical probes can comprise LEDs with specific spectral bands that are used to excite plants and generate various physiological responses.
  • An optical detector array can incorporate multiple-wavelength optical detectors selected or tuned to be responsive to a narrow spectral band that corresponds with specific light that is reflected from or emitted by the plants.
  • FIG. 16D shows three physiological sensor modules 1620 d, 1620 h and 1620 i with different configurations of LEDs and detectors.
  • Module 1620 d has four central detectors (these could be blue, yellow, red and far red detectors, for example) surrounded by a ring of LED probes, with another array of LED probes on each side.
  • the specific LEDs and detectors can be selected and changed to suit a specific crop or to obtain information about a specific condition. Again, this provides flexibility and versatility to the device.
  • the MCU controls the sequence and spacing of the LED pulses, pulse duration, pulse intensity and the number of LED pulses that are used to stimulate the plant in a particular assessment.
  • the optical detectors receive reflected and emitted light from the plant to excitation light and generate a low-level current.
  • the current is significantly amplified (for example, up to 10 7 times), and filtered before being digitized.
  • the plant is stimulated with a series of one hundred 100 microsecond pulses of RGB-UV-NIR light separated by 25 millisecond gaps, and at least 4 detectors are used to measure light reflected and emitted from plants.
  • Analog front-end (AFE) circuitry that allows the user to switch between “day” and “night” modes can be incorporated into the MCU of the multi-sensor device.
  • the AFE filters out ambient light, thereby enabling detection of extremely low levels of response light (signal) in the presence of high intensity ambient light (noise).
  • the night mode is adapted or optimized for measurements in low ambient light (noise) intensity.
  • the processed AFE response (amplified and filtered) is digitized by a multi-channel high-speed (e.g. 2 mega samples per second) high-resolution (e.g. 16-bit) analog-to-digital converter (ADC).
  • ADC analog-to-digital converter
  • Optical readings and digitized data are transmitted via the communications system/interface.
  • mathematical manipulation can be used to eliminate distance and uneven surfaces as factors in the response data. For example, if the detectors and light probes are in the same plain, a ratio of measurements in correlation to each other can be used to overcome the issue of the distance. For example, instead of comparing a sensor A reading from plant 1 to plant 2, the ratio of a pair of sensor readings (sensor A reading /sensor B reading) for each of the two plants can be compared. This way, although the individual sensor readings might vary from plant to plant based on the distance between the plant and the sensor, comparison of the ratios will provide meaningful comparison between plants.
  • the full spectrum light source is also activated during a light emission period, and spectroscopic detectors measure the entire wavelength from NIR to UV at high resolution, but at lower speed than for the physiological sensing. Spectroscopy readings are transmitted via the communications system/interface.
  • the chemical sensor detects VOCs produced by the plant down to sub ppb levels, and produces a voltage in response to the compound concentration level.
  • the output voltage is digitized by a high-resolution analog-to-digital converter (ADC) and the data is transmitted via the communications system/interface.
  • ADC analog-to-digital converter
  • Embodiments of the technology, devices, systems and methods described herein can be used separately or can be used in various combinations as desired.

Landscapes

  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Environmental Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Wood Science & Technology (AREA)
  • Chemical & Material Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Botany (AREA)
  • Toxicology (AREA)
  • Forests & Forestry (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Food Science & Technology (AREA)
  • Immunology (AREA)
  • Insects & Arthropods (AREA)
  • Pest Control & Pesticides (AREA)
  • Zoology (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Biochemistry (AREA)
  • Medicinal Chemistry (AREA)
  • Analytical Chemistry (AREA)
  • Pathology (AREA)
  • Automation & Control Theory (AREA)
  • Computational Linguistics (AREA)
  • Water Supply & Treatment (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Ecology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

A multi-sensor device comprises a housing containing multiple sensor modules for capturing and transmitting sensor data for plants in a crop. A control unit within the housing is operable to control the sensor modules, and a communications interface is connected to the control unit for transmitting data from said plurality of sensor modules. The sensor modules can include a physiological sensor, a surface analysis sensor, and chemical sensor. The multi-sensor device can be used as a hand-held device or mounted to a mobile platform for use in an automated crop monitoring system.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims priority benefits from U.S. provisional patent application No. 62/198,761 filed on Jul. 30, 2015, entitled “Systems and Methods for Crop Monitoring and Assessment.” The '761 application is incorporated by reference herein in its entirety.
FIELD OF THE INVENTION
The present invention relates to systems and methods for crop monitoring and assessment. Some embodiments relate to multi-sensory devices and platforms for crop monitoring, and methods for operating the same.
BACKGROUND OF THE INVENTION
When food and other crops are grown on a large scale, either in protected cultivation (such as in a greenhouse) or outdoors, growers face several challenges. For example, it is generally difficult for a grower to predict the quality and yield of the crop at a stage in crop development when intervention will still be feasible and useful. Also it can be difficult for a grower to know if, where and when the crop has a problem (such as related to a pest, disease, water, other abiotic stress or nutritional deficit), and the extent of the problem, until it is readily visible to human scouts. Often by that stage it may require expensive and extensive intervention. Crop yield is affected by the physiological performance of the crop throughout its development cycle, which is in turn dependent on external environmental factors among other things. Precise intervention at critical developmental stages, can allow growers to achieve high or optimum yields of the crop. Pest and disease problems are often exacerbated by the large scale on which some crops are grown, the costs for labor, and the speed and ease with which pests and diseases can spread, especially in protected cultivation. When it comes to monitoring crops for pests, diseases and other deleterious conditions, a common approach has been the use of human scouts who visually inspect the crop. However, human scouts whose role it is to locate plants with pests, diseases or other problems, can themselves facilitate the spread of those pests and diseases, for example, through their physical contact with multiple plants and the resulting transfer of pests or diseases from plant to plant. Other limitations of using human scouts for crop monitoring include the speed with which they can cover a large area, and variation in interpretation among individual humans. They also require specific training, and performance of even a diligent employee will be subjective and vary over time.
Many crop management practices are employed prophylactically or simply based on past practices and customs. A common underlying assumption is that crops are uniform and perform evenly which is not necessarily the case, for example, because plants respond to differences in microclimate on a finer scale.
Sensor systems have been developed for crop monitoring, but many of these systems have limitations and shortcomings. For example, some systems use a grid of sensors suspended above the crop (in a zone, usually about an acre in greenhouses) or that fly over the crops. Such sensory grids can be used to monitor environmental conditions or general responses from plants, but generally this is on a course-grained scale. Handheld devices can be used to capture data from individual plants, but these devices tend to be cumbersome to use, and data is captured only from plants that the operator of the handheld device interacts with directly. It is generally not feasible to use a handheld device to capture data from all plants within an area being managed. Often such devices are used by clipping them to the plant or otherwise contacting the plant with the sensing device. Other systems rely on visual detection of causal factors (e.g. pests or disease) by means of motion detection or visual pattern recognition. Visual detection devices can be technologically taxing and economically unfeasible. Additionally, in certain cases, significant damage has already been done to the crop by the time the causal factor is visually identified.
Some sensory devices/systems are geared toward specific indicators (presence of disease, anthocyanin content, emergence of adult pests, etc.) with narrow spectra of responses, often using sensors that function during daylight hours. These technologies are generally large and expensive, and require a human operator, and some of them are time-consuming. For example, fluorescent measurement systems have been used to detect far red spectra produced by plants when exposed to blue or red light. Conventional fluorescent measurement requires complex equipment, and typically a single assessment takes several minutes and sometimes up to about 15 minutes to complete. Other sensory systems can collect very general information (temperature, humidity) that cannot accurately pinpoint problems at the level of individual plants, or at levels of sensitivity that convey timely information in real time.
Expert growers develop a wealth of knowledge and experience by working their crop for multiple years. When currently available, highly automated sensor-based crop monitoring systems are used, the valuable expertise and insight of an experienced grower is no longer effectively harnessed. Furthermore, although humans and existing sensory systems for crop monitoring may, to some degree, be able to identify problems with a crop, they are not capable of predicting the future health of a crop or plant.
SUMMARY OF THE INVENTION
In one aspect, a multi-sensor device for capturing and transmitting sensor data for plants in a crop comprises a housing comprising a plurality of interior cavities and at least one mount on an exterior surface thereof. A plurality of exchangeable sensor modules, occupying the cavities. Each sensor module is for sensing at least one plant-related parameter when the sensor module is positioned proximate to a plant. The housing also contains a control unit operable to control operation of the plurality of sensors, a location tracking system connected to the control unit for tracking the location of the multi-sensor device, and a communications interface connected to the control unit for transmitting and receiving information. The communication interface is operable to transmit data, for example wirelessly, from the plurality of sensor modules. A user-interface is connected to the control unit to allow a user to interact with the device, for example, via a touch screen. The device further includes an electrical power connector for connecting the multi-sensor device to a power source. The electrical power connector can be integral with one or more of the mounts, or can be separate from the mounts.
The housing can be mounted to a handle via the mount for use of the multi-sensor device as a hand-held device. Alternatively the device can be mounted to a mobile platform via the mount. The mobile platform can comprise a propulsion system, a power supply, and a control system for controlling movement of the mobile sensory platform.
In some embodiments of the device, the sensor modules are each coupled to a universal adapter. The universal adapters are connectable to provide power, data and control signal connectivity between one of the sensor modules and the control unit.
The sensor modules can comprise a physiological sensor module, a surface analysis sensor module, and a chemical sensor module. The physiological sensor module can comprise configurable optical probes and tunable detectors. The surface analysis sensor module can comprise a full spectrum light source and spectroscopic detectors. The chemical analysis sensor module can be, for example, a photo-ionization detector, a surface acoustic wave sensor or a quartz crystal microbalance sensor.
The sensor modules can further comprise a thermal imaging sensor module, and/or a vision system module comprising a pair of high resolution RGB-IR cameras for stereo-imaging, and/or a temperature and humidity sensor module.
Each of the plurality of sensor modules is preferably operable to capture data pertaining to at least one plant-related parameter without physical contact with the plant.
In some embodiments of the multi-sensor device, the communications interface is operable to provide two-way communication between the multi-sensor device and a data processing unit over a wireless network, and/or is operable to receive control commands from a wireless mobile device.
In another aspect, a mobile sensory platform for capturing and transmitting sensor data for plants in a crop comprises a propulsion system, and a plurality of sensors mounted to a support structure. Each of the plurality of sensors is for capturing data pertaining to at least one plant-related parameter when the sensor is positioned proximate to a plant. The platform also comprises a communications interface for two-way communication with a data processing unit over a wireless network, an electrical power supply; and a control system for controlling movement of the mobile sensory platform and operation of the plurality of sensors.
Preferably each of the plurality of sensors is operable to capture data pertaining to at least one plant-related parameter without physical contact with the plant.
The plurality of sensors can comprise a physiological sensor and/or a surface analysis sensor and/or a chemical sensor, and optionally microclimate sensors, and/or canopy screening sensors. In some embodiments the sensors of the mobile sensory platform comprise a physiological sensor comprising configurable optical probes and tunable detectors, a surface analysis sensor comprising a full spectrum light source and spectroscopic detectors, and a chemical analysis sensor selected from the group consisting of a photo-ionization detector, a surface acoustic wave sensor and a quartz crystal microbalance sensor.
In some embodiments of the mobile sensory platform the position of at least one of the sensors on the support structure is adjustable during operation of the mobile sensory platform. The control system may be operable to control the position of the at least one of the plurality of sensors.
In some embodiments, the mobile sensory platform of claim further comprises an intervention module for administering remediation to plants in the crop.
The mobile sensory platform can be a ground-based platform or an air-borne platform.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a schematic illustration of an embodiment of an automated crop monitoring system comprising a mobile sensory platform.
FIG. 2 is a block diagram illustrating components of an embodiment of a mobile sensory platform.
FIG. 3 is a block diagram illustrating components of a data processing unit (DPU).
FIG. 4 illustrates an embodiment of a method of operating an automated crop monitoring system, the method involving two phases of data collection and analysis.
FIG. 5 is a schematic illustration of an embodiment of an automated crop monitoring system that captures and uses expert knowledge and comprises a mobile sensory platform.
FIG. 6 illustrates an embodiment of a method for crop monitoring and assessment comprising four activities.
FIG. 7A illustrates an expert knowledge capture and system training activity that is a component of the method illustrated in FIG. 6.
FIG. 7B is a screen-shot from a system training app that can be used by an expert to capture and convey their assessment of plants in a crop.
FIG. 8 illustrates a non-expert sensor data acquisition activity that is a component of the method illustrated in FIG. 6.
FIG. 9 illustrates a mobile sensory platform data acquisition and analysis activity that is a component of the method illustrated in FIG. 6.
FIG. 10A illustrates an information dissemination activity that is a component of the method illustrated in FIG. 6.
FIG. 10B is a screen-shot from a reporting app that can be used to convey information relating to crop health to a person, such as a grower.
FIG. 11 is a schematic illustration showing various sources of information that may provide inputs to a DPU in embodiments of systems and methods for monitoring and assessing crop health.
FIG. 12 illustrates an embodiment of a method involving correlation of crop performance data with historical data captured during the lifecycle of the crop.
FIG. 13A is a simplified drawing showing a front view and a side view of an embodiment of a mobile sensory platform.
FIG. 13B is a simplified drawing showing a front view and a side view of another embodiment of a mobile sensory platform.
FIG. 13C is a simplified drawing showing a front view and a side view of yet another embodiment of a mobile sensory platform.
FIG. 14A is a simplified drawing showing two views of an embodiment of an air-borne mobile sensory platform.
FIG. 14B is a simplified drawing showing two views of an embodiment of an air-borne mobile sensory platform.
FIG. 14C is a simplified drawing showing embodiments of a landing pad where the air-borne mobile sensory platforms of FIGS. 14A and 14B can land for recharging.
FIG. 15A is a simplified drawing of an embodiment of a hand-held device comprising a portable sensory platform connected to a smartphone.
FIG. 1513 is a simplified drawing of another embodiment of a hand-held device comprising a portable sensory platform connected to a smartphone.
FIG. 15C is a simplified drawing showing three views of yet another embodiment of a hand-held device comprising a portable sensory platform connected to a smartphone.
FIG. 16A illustrates an embodiment of a modular multi-sensor device, for crop monitoring, that can be used as a hand-held device or can be mounted to a mobile platform.
FIG. 16B illustrates a housing that can accommodate various sensor modules and other components
FIG. 16C shows the multi-sensor device of FIG. 16A with some of the sensor modules partially removed by sliding out of housing.
FIG. 16D shows a housing with a sensor module installed in the housing position, and two alternative sensor modules which can replace the installed sensor module.
FIG. 16E shows a rear perspective view of the housing shown in FIG. 16B, and shows separately a bundle of components that can be installed in the housing.
FIG. 17A shows an example of handle that can be fastened to the multi-sensor device of FIG. 16A.
FIG. 17B illustrates how the handle of FIG. 17A can be fastened to the multi-sensor device of FIG. 16A.
FIG. 17C illustrates a mounting connector that can be removably fastened to a mount on the exterior of a multi-sensor device.
FIG. 17D illustrates a mounting bracket via which a multi-sensor device can be mounted to a mobile platform.
FIG. 17E illustrates how the multi-sensor device housing of FIG. 16B can be mounted to the bracket of FIG. 17D.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENT(S)
The systems and methods described herein for monitoring and assessing crop health can provide rapid and sensitive screening of individual plant health with reduced human labor, and at a far greater speed than can be accomplished by human scouts. The systems and methods described herein can be deployed outdoors, such as in a field or orchard, or indoors such as in a greenhouse. The systems and methods have an automated component, but are flexible and can be repeatedly modified to enhance the crop-related information that they provide.
Some embodiments also capture and integrate knowledge from human experts into automated crop monitoring systems and methods. Their expertise can be effectively and efficiently captured and applied via an automated system that acts as a proxy, making the technology an extension of the grower.
Furthermore, embodiments of the systems and methods described herein can provide predictive models that can be used to predict future health of plants in a crop based on their current sensor data, so that steps can be taken to try to avoid deterioration in the health of the plant. In some cases a predictive model will provide the capability to identify a potential issue with a plant, before any single sensor or a human expert could detect a problem.
Embodiments of the systems and methods described herein rely primarily on the detection (through sensors) and interpretation (through data analysis) of plant-based signals to provide information about crop health.
Monitoring and assessing crop or plant health as described herein, can include monitoring and assessing performance of the crop or plant. Performance is generally related to the health of the crop or plant.
Automated Crop Monitoring Systems & Methods
A first aspect of the technology relates to an automated crop monitoring system and method. An embodiment of such a crop monitoring system 100 is illustrated in FIG. 1. Crop monitoring system 100 comprises a mobile sensory platform 110 comprising a plurality of sensors mounted on a vehicle, cart or drone for example. Mobile sensory platform 110 captures sensor data related to plants in the crop and transmits it to a data processing unit (DPU) 140 via a network 160. In some embodiments, mobile sensory platform 110 comprises more than one mobile sensory platform, and the platforms may communicate and exchange information with one another, as well as with DPU 140. DPU 140 analyzes the sensor data and sends information regarding the crop to an individual 180, such as a grower and or/other parties via one or more end-user devices, such as a smart phone 170 a and/or a computer 170 b. DPU 140 may also send commands to mobile sensory platform 110. Grower 180 or other parties may also send information to DPU 140 and/or send commands to mobile sensory platform 110 via network 160.
In FIG. 1 arrows are used to indicate transmission of sensor data and/or other information. Preferably system 100 is a web-based and/or cloud-based system, and communication between mobile sensory platform 110, DPU 140, and grower 180 and/or other parties or devices, is primarily or entirely wireless communication.
In some embodiments the mobile sensory platform is designed to operate in the dark, for example, at night. This can be beneficial as it can reduce interference with other greenhouse or field operations. Furthermore, the monitoring system and method may operate with greater sensitivity at night as plants tend to be dormant during periods of darkness. During the daytime, normal practices of staff tending to the crop might temporarily stress the plants, for example moving plant heads, removing shoots, picking fruits, and the like.
In some embodiments the sensors on the mobile sensory platform are proximate to the plant during sensing and data capture, but do not touch the plants or soil. Such non-contact monitoring can help to reduce the spread of pests and diseases.
Preferably the mobile sensory platform is configured to move autonomously among the plants, or in response to commands from a controller, which in some embodiments is a component of the data processing unit.
FIG. 2 is a block diagram illustrating components of an embodiment of a mobile sensory platform, such as platform 110 of FIG. 1. Mobile sensory platform 110 comprises a plurality of sensors 112 (as described in further detail below) and a data storage device 114 for storing data captured from sensors 112. Mobile sensory platform 110 also comprises a CPU and control system 116 with associated software 118, and a communications system/interface 120. In some embodiments an external serial connection is provided to allow a user to connect a PC or other device, for example, to modify software on the platform 110. Mobile sensory platform 110 further comprises a propulsion system 122 (for example this could comprise an electric motor, wheels, propellers etc.), a power supply 124 (for example, a battery or other energy storage device and associated recharging equipment and/or a power management system that, in some embodiments, provides a switching between multiple energy sources. Power management system may also incorporate safety and protection devices. Mobile sensory platform 110 may also comprise a GPS and positioning system 126 or similar location tracking system, and a display and/or user interface 128. In some embodiments GPS and positioning system 126 produces and associates a location tag with the sensor measurements; location tags can be stored in data storage device 114 and also transmitted, along with sensor data, via communications system/interface 120. Mobile sensory platform 110 may also comprise an on-board data processing unit, and other components (not shown in FIG. 2).
FIG. 3 is a block diagram illustrating components of a data processing unit (DPU), such as DPU 140 of FIG. 1. DPU 140 comprises at least one CPU 142, a controller 144, software 146, a power supply 148, a communications system/interface 150, and a user interface 152. DPU can also comprise one or more databases 154 for storing raw and/or processed sensor data and/or other information.
One method of operating an automated crop monitoring system, such as system 100 of FIG. 1, is described in reference to FIG. 4 which illustrates a method 400. Method 400 is a two-phase data collection and analysis method. The first phase is a rapid pre-screening phase 405 to identify plants that may have a problem, for example by identifying plants that exhibit a variance based on analysis of their sensor data. In this phase, at 410 data processing system (DPU) 140 activates mobile sensory platform 110 (of FIG. 1) for pre-screening. At 415 mobile sensory platform performs pre-screening, moving between the plants and capturing sensor data relating to some or all of the plants in the crop. Only one sensor or a sub-set of the plurality of sensors 112 is used in pre-screening phase 405. At 420 sensor data from the rapid pre-screening is transmitted from mobile sensory platform 110 via network 160 to DPU 140, where the pre-screening data is processed and analyzed at 430. Plants that may have a problem are identified and tagged for further inspection.
Various known methods can be used to tag the plants or otherwise capture location information, so that the sensor data that is captured for a particular plant can be associated with that plant. In one non-limiting example, each row of crops may be identified using a digital beacon or an RFID tag to signify the beginning of the row. The position of each plant in that particular row can be calculated based on its distance from the beacon or tag.
During 430 DPU 140 develops a secondary screening assignment for mobile sensory platform 110. The secondary screening assignment can be used to gather more detailed information about the crop or specific plants in the crop. The second phase of method 400 is a secondary screening phase 435 which begins at 440. Commands related to the secondary screening assignment are transmitted to mobile sensory platform 110 via network 160. These commands can include commands to activate certain sensors or to cause the mobile sensory platform to move to capture data from particular plants or regions or the crop, and/or the route that should be taken for the secondary screening. Mobile sensory platform 110 performs the secondary screening at 445 in accordance with these commands. For example, during secondary screening 445 mobile sensory platform 110 may move to potential problem plants identified in pre-screening phase 405, and capture further sensor data from those plants using additional ones of the plurality of sensors 112. In some embodiments the particular sensors used in the secondary screening are selected based on the analysis of the pre-screening sensor data captured during the pre-screening phase. Sensor data gathered during the secondary screening phase is transmitted from mobile sensory platform 110 via network 160 to DPU 140 at 450. The sensor data is analyzed at 460 to provide information related to the crop.
In some embodiments of the method, the mobile sensory platform used for secondary screening (for example, at 445) is different from the mobile sensory platform used for pre-screening (for example, at 415). In such one implementation, a multi-sensor device is mounted to a mobility platform such as an all-terrain rover robot with the capability to go to specific areas of a farm or greenhouse. The rover robot carries a drone that is equipped with one or more sensors; for example, these can be a subset of the sensory devices that are on the multi-sensor device that is mounted to the rover robot. During a two-phase mission, the drone flies over a specific area of the crop and performs pre-screening. Once desired pre-screening phase is complete, the drone lands or docks on the rover robot, where it may be re-charged for example. Sensor data from the pre-screening phase may be transmitted from the drone to DPU 140 during the pre-screening phase, or may be transmitted from the drone to DPU 140 (directly or via the rover robot) upon completion of the pre-screening phase. DPU 140 develops a secondary screening assignment for the rover robot, and commands related to the secondary screening assignment are transmitted to the rover robot via network 160 (and optionally via the drone). The rover robot then performs a secondary screening phase, typically in closer proximity to the plants than the drone, and transmits data for analysis by DPU 140 as described above.
Following the sensor data analysis at 460, DPU 140 then transmits information to one or more end-user devices at 470. For example information may be sent to the grower 180 or others via one or more end-user devices, such as smart phone 170 a and/or computer 170 b. Such information can include, for example: information about the condition of the crop or individual plants, diagnostic information, alerts, action plans, suggested treatments or interventions and the like. In some embodiments, DPU 140 may also send commands to mobile sensory platform 110 to implement one or more interventions in order to attempt to remediate an adverse condition affecting one or more of the plants, as indicated at 480. For example DPU 140 could command mobile sensory platform 110 to disperse a bio-control agent. DPU 140 could also activate other systems at the crop-site to implement one or more interventions in order to try to remediate an adverse condition affecting one or more of the plants. For example, it could activate an irrigation system, adjust a temperature control system or cause nutrients or pesticides to be automatically administered to certain plants.
Events in method 400 can be sequential (with one event being completed before the next begins), or in some cases they can overlap. For example, during the secondary screening phase 435, information relating to one portion of the crop (for which secondary screening sensor data has been captured and analyzed) may be being transmitted from the DPU to the one or more end-user devices at the same time as mobile sensory platform 110 is still capturing sensor data for another portion of the crop. If two different mobile sensory platforms are used for pre-screening and secondary screening (for example, a drone and a rover robot), one platform can be performing pre-screening at the same time the other is performing secondary screening, for example.
There are different approaches that can be used for the analysis of sensor data at 430 and 460.
In one more conventional approach the sensor data is analyzed by using pre-established scientific models and/or comparing the sensor data to known indices or standards. DPU 140 may draw on information stored in other databases for this purpose. For example, particular sensors can be used to look for particular issues with the plants. Data can be collected and compared to a catalogue of plant stress signatures to identify particular problems with the plant or to determine whether the plant is healthy. Some sensors are used to look for specific chemical volatile indicators of a particular pest infestation. Clearly with this approach, it would be necessary to look for many different signatures indicative of many possible stressors. This would typically require use of many different sensors, each geared toward specific indicators (presence of disease, anthocyanin content, emergence of adult pests, etc.) and generally having a narrow spectrum of responses. Such conventional sensors usually function during daylight hours, are generally large and expensive, and require a human operator. Also it can be difficult for conventional sensors to pinpoint specific causal factors in a real crop growing situation where plants are exposed to many potential stressors.
In another approach, rather than analyzing the sensor data to look for specific plant problems or stressors, sensor data is compared to a profile or signature known to be indicative of a healthy plant. In analyzing the sensor data the DPU 140 looks for deviations from this healthy plant profile. This can be done by classifying the sensor data against a set of “trained data” or a model derived therefrom. The trained data can be derived from expert assessments as described in further detail below.
Automated Crop Monitoring Systems and Methods that Capture and Harness Expert Knowledge
In this aspect, crop monitoring systems and methods similar to those described above are based upon or enhanced by the utilization of human expertise, for example, from an expert such as an experienced grower, farmer or professional crop advisor assessing the same crop or a similar crop in a similar environment. Embodiments of such systems and methods can capture and integrate this expert knowledge. The expert knowledge is captured from a human expert in a way that allows it to be re-applied later or elsewhere by an automated system and/or used to teach a non-expert.
The system is “trained” based on correlating an assessment of the health of individual plants, as inputted by an expert, with sensor data captured for the same plants. In this context training refers to a process of establishing a repeatable, statistically meaningful relationship between observed data (such as sensor data) and “truth” data—in this case the assessment of an expert—to give “trained data” that can be used as a reference for the classification of new data. This is a form of supervised learning. For example, the trained data can be used in evaluating plant-related sensor data subsequently received from a mobile sensory platform (e.g. in unsupervised learning). Thresholds for different classifications can be established through this process, and data-derived models that can be used for classification and analysis of new data can be developed.
Using this integrated approach, a crop monitoring and assessment system can apply the expert knowledge on an on-going basis during automated monitoring and assessment of the crop, without the need for a human expert to inspect the entire crop. This can make the technology effectively an extension of an expert, such as an experienced grower, farmer or professional crop advisor.
An embodiment of a crop monitoring system 500 that captures and uses expert knowledge is illustrated in FIG. 5. In some respects crop monitoring system 500 is similar to crop monitoring system 100 of FIG. 1. System 500 comprises a data processing unit (DPU) 540 that receives data from a plurality of sources via network 560. DPU 540 can be similar to DPU 140 illustrated in FIG. 1 and FIG. 3.
One source of data is a handheld mobile device 570 a operated by an expert 580 (such as an experienced grower, farmer or professional crop advisor). Another source of data is a handheld mobile device 570 b operated by a non-expert 585. A “non-expert” in this context refers to someone who is not as skilled or experienced as an expert at accurately assessing the health of plants in the crop. For example, it might be a worker who works in the greenhouse attending to the plants (feeding, pruning, harvesting etc.). Handheld mobile devices 570 a and 570 b comprise a plurality of sensors for capturing sensor data from plants. The handheld devices also provide a way of tagging, locating or identifying the plant associated with the sensor data. Handheld device 570 a also allows expert 580 to enter an assessment of plant health. Information and sensor data can be transmitted from handheld devices 570 a and 570 b to the DPU 540 via network 560. Another source of sensor data is a mobile sensory platform 510 comprising a plurality of sensors mounted on a vehicle, cart or drone for example. Mobile sensory platform 510 can be similar to platform 110 illustrated in FIG. 1 and FIG. 2. Other electronic devices 565 may be used to enter and transmit information and data about the crop to DPU 540 via network 560, for example crop conditions, planting times, source of seeds, environmental factors and the like. Other data sources 575 that can transmit data to DPU 540 via network 560 may include, for example, fixed sensors located around the crop-site.
Software managing the DPU can be based on an open application program interface (API) structure that allows information exchange and integration from and to multiple external data sources. Cross-domain semantic interoperability (CDSI) software allows the DPU to exchange information and commands with other devices and agricultural machines and initiate mutually agreeable tasks.
DPU 540 stores sensor data and information that it receives in one or more databases 545. It also performs data correlation (correlating an assessment of the health of individual plants as inputted by the expert with sensor data captured for the same plants) and stores the resultant “trained data” in one or more databases 545. DPU 540 can then analyze plant-based sensor data using the trained data and/or one or more models derived therefrom, as described in further detail below. In some embodiments, DPU performs more complex analysis of current and historical sensor data, for example, using data mining and machine learning techniques to generate predictive models. In performing data analysis, DPU 540 may supplement the analysis of sensor data using information stored in other databases, such as pre-established scientific models and known indices or standards. DPU 540 can also transmit information regarding the condition of the crop to expert 580, non-expert 585 and or/other parties such as a grower or crop-site manager 590 via one or more end-user devices, such as a smart phones, computers, handheld devices such as 570 a, 570 b, 570 c and other electronic devices. These can be located at the crop-site or remotely. DPU 540 may also send commands to mobile sensory platform 510.
In FIG. 5 arrows are used to indicate transmission of sensor data and/or other information. Preferably the system is a web-based and/or cloud-based system, and communication between mobile sensory platform, data processing unit, and the expert, non-expert and or/other parties or devices, and is primarily or entirely wireless communication
System 500 is further described in reference also to FIGS. 6 through 10 which describe a method 600 for crop monitoring and assessment. Method 600 comprises four activities: expert knowledge capture and system training 700, non-expert data acquisition 800, mobile sensory platform data acquisition and analysis 900, and information dissemination 1000. The four activities can, for example, occur repeatedly in series, can be performed in different sequences over time, can overlap, or can occur at least to some extent in parallel.
Method 600 of FIG. 6 starts at 610, for example, when crop monitoring begins. At 620 method 600 branches into the four activities: expert knowledge capture and system training 700, non-expert data acquisition 800, mobile sensory platform data acquisition and analysis 900, and information dissemination 1000. Each of the activities are described in further detail in the following paragraphs.
Expert knowledge capture and system training 700 is a fundamental basis of the overall method. As shown in FIG. 7A, activity 700 commences at 710 when an expert enters the crop-site (e.g. field or greenhouse) to assess plants in the crop. At 720 the expert (such as expert 580 of FIG. 5) evaluates individual plants and makes an assessment of their condition or health. The expert also captures sensor data from the plants. Location information or plant tagging information is also captured so that the expert assessment and sensor data can be associated with each particular plant. In some embodiments, the expert can be equipped with a handheld or wearable device (e.g. 570 a of FIG. 5) comprising one or more sensors for sensing the plants and capturing and transmitting sensor data via a network (such as 560 shown in FIG. 5) to a data processing unit (such as DPU 540 shown FIG. 5). Preferably the sensors capture plant-based information by sensing characteristics of the plant non-invasively without direct contact with plants. The expert can also input his personal assessment of individual plants using the handheld device, for example, via an app. The assessment can involve a ranking of the plant's condition (e.g. red, orange, green for poor, moderate, healthy respectively), or it can involve a more granular or detailed quantitative or qualitative assessment. The expert repeats the assessment and the capture of sensor data for multiple plants in the crop (although generally this will only be a small fraction of the total crop). At 730 sensor data and expert assessments are transmitted to the DPU. This can be done in real-time plant-by-plant or once the expert has completed that particular visit to the crop. Preferably it is transmitted wirelessly and in real-time.
Activity 700 is typically performed during the day. Multiple experts may perform expert knowledge capture and system training activity 700 simultaneously or at different times for a given crop.
Once the assessments and sensor data are transmitted to the DPU at 730, the raw information (including the plant identifier/locator, and the expert assessment and sensor data for each plant that was evaluated) can be stored. At 740 the DPU correlates the assessments of the health of individual plants (as inputted by the expert) with the sensor data captured for the same plants to generate trained data and, in some cases, one or more data-derived models. The correlation can involve the use of machine learning and classification, and the development of pattern recognition models. The trained data resulting from the correlation is stored. Once there is sufficient trained data to give a reasonable level of accuracy, the trained data, or models derived therefrom, can be used as described in reference to activities 800, 900 and 1000 below. As and when activity 700 is repeated, additional expert assessments and associated sensor data can be added, processed and included in the stored trained data and/or used to further enhance the models. This accumulation of expert knowledge will generally improve the accuracy of the crop monitoring and assessment system over time.
FIG. 7B shows a screen-shot 700B from a system training app (i.e., a software program or application that can be used on a mobile electronic device, such as a smartphone or tablet) that can be used by an expert to capture and convey to a DPU their assessment of plants in a crop. The expert can assess the plant as healthy or can select the level of different problems; in the illustrated example these are diseases, pests and deficiencies. The custom options allow the expert to assess the plants with respect to other positive or negative attributes of their choosing. For example, the expert may choose a custom profile or feature that they prefer to see in their crop based on their knowledge and experience. This way, growers can create a profile of a healthy plant, an unhealthy plant and/or in some cases, a custom attribute that they want to track. Corresponding sensor data for each plant is also transmitted to the DPU.
In order to quickly build a data-derived model for a particular type of crop and a particular disease or condition, during a training phase an expert may look for plants that are healthy and for plants that are exhibiting a particular problem (e.g. a specific disease or pest) and capture and transmit sensor data along with their expert assessment for just those plants.
Following a training phase that involves supervised learning, for example as described above, unsupervised learning processes can be used to test the resulting data-derived models for accuracy on a new set of unclassified or unlabeled data.
Non-expert sensor data acquisition activity 800 is illustrated in the flow chart of FIG. 8, and involves the collection of additional plant-based sensor data by a human who is a non-expert. The non-expert is not as skilled or experienced as the expert at accurately assessing the health of plants in the crop. They might be a worker who works in the greenhouse attending to the plants (feeding, pruning, harvesting etc.). Non-expert sensor data acquisition can provide useful data from additional plants in the crop that can be used to develop assignments for automated crop monitoring activities that may happen overnight, for example. It can also be used so that the non-expert can learn how to assess plant health “like an expert”—this learning aspect is described in further detail below. Non-expert sensor data acquisition activity 800 may be performed on a daily basis, frequently or not at all, as part of overall method 600.
Referring to FIG. 8, activity 800 commences at 810 when a non-expert enters the crop-site to capture sensor data for plants in the crop. At 820 the non-expert (such as non-expert 585 of FIG. 5) captures sensor data for a plant, along with location information or plant tagging information so that sensor data can be associated with the particular plant. In some embodiments, the non-expert is equipped with a handheld or wearable device (e.g. 570 b of FIG. 5) comprising one or more sensors for sensing plants and capturing and transmitting sensor data to a data processing unit (such as DPU 540 shown FIG. 5). For example, the handheld sensor device can be similar to or the same as the one used by the expert in activity 700. Sensor data capture may happen passively as the non-expert moves from plant to plant performing other tasks, or may require the non-expert to activate the sensors and capture sensor data, for example, by pressing a button. At 830 sensor data is transmitted to the DPU. This can be done in real-time plant-by-plant (as shown in FIG. 8), or for multiple plants once the non-expert has completed that particular visit to the crop. Preferably it is transmitted wirelessly and in real-time.
Activity 800 is also typically performed during the day. Multiple non-experts may be performing sensor data acquisition activity simultaneously or at different times for a given crop. For example, a large crew of workers could be equipped with handheld or wearable sensor devices in order to capture plant-based sensor information while they are busy performing other tasks related to the crop.
Once sensor data is transmitted to the DPU at 830, the raw information (including plant identifier/locator and sensor data for each plant that was evaluated) can be stored. At 840 the DPU classifies the condition of each plant by passing the sensor data through a model derived from the trained data (generated from expert knowledge capture and system training activity 700). Plant health information based on this classification can be disseminated for various purposes as described below in reference to activity 1000 of FIG. 10A. In some embodiments, such as at 1040 of FIG. 10A, information is sent back to the non-expert. For example, referring again to FIG. 8, handheld sensor data captured by the non-expert for each plant is analyzed in real-time by the DPU and, at 850, the non-expert may receive an immediate assessment of the condition of the plant from DPU 540 via their handheld device 570 b. For example, this could be a simple ranking of the plant's condition (e.g. red, orange, green for poor, moderate, healthy respectively). The real-time assessment delivered to the non-expert is based on a model derived from trained data that was derived from expert assessments in activity 700. In this way the non-expert can inspect the plant and learn how it would have been assessed by an expert, without the expert needing to be present to teach the non-expert. Once the non-expert has received feedback on a particular plant at 850 they can move on to another plant at 860 if desired.
Mobile sensory platform data acquisition and analysis activity 900 is illustrated in the flow chart of FIG. 9, and is an important aspect of method 600. This activity can be performed during the day, or at night when there might be reduced interference with other greenhouse or field operations. At night the sensing may be more sensitive to the presence of disease or pests, as plants tend to be dormant and less stressed by other external factors during periods of darkness. Multiple mobile sensory platforms may be used simultaneously to cover different regions of the crop, for example, to allow the whole crop to be assessed in a shorter time-frame.
Activity 900 starts when a mobile sensory platform (such as mobile sensory platform 510 of FIG. 5) is activated to move between the plants and capture data relating to some or all of the plants in the crop. The mobile sensory platform may have a different number of sensors than the handheld devices used by the expert and non-expert in activities 700 and 800. In some cases, it will have a greater number of sensors, but not always. Also, it may have a different set of sensors usually, but not always, with some sensor types in common with the handheld devices.
The mobile sensory platform can be operated, for example, in a similar manner to that described in reference to FIG. 4, with a pre-screening phase and a more detailed secondary screening phase. In other embodiments mobile sensory platform can be operated with a single-pass screening operation.
With these, or other methods of operating the mobile sensory platform, sensor data is captured at 920. Location information or plant tagging information is also captured at 920, so that sensor data can be associated with each particular plant. At 930 data is transmitted to the DPU. The sensor data from the mobile sensory platform can be transmitted to the DPU in real-time or once a portion or all of the screening session is completed. Preferably it is transmitted wirelessly.
Once sensor data from the mobile platform is transmitted to the DPU at 930, the raw information (including plant identifier/locator and sensor data for each plant that was evaluated) can be stored. In some embodiments, further correlation can be performed at 940 to generate additional trained data and/or to enhance data-derived models. For example if data has been captured from sensor-types on the mobile sensory platform that are not on the handheld devices, this data may be correlated with the expert assessments obtained for the same plants during activity 700 to provide further trained data and/or enhance models that can be stored and used for classification of plant health. At 950 the DPU classifies the condition of each plant by applying a model, derived from the trained data, to the sensor data received from mobile sensory platform at 930. Plant health information based on this classification can be disseminated for various purposes as described below in reference to activity 1000 of FIG. 10A. In some embodiments, such as at 1050 of FIG. 10A, commands are sent to the mobile sensory platform based on the analysis of sensor data received from the mobile sensory platform. For example, at 960 (see FIG. 9), commands are transmitted to the mobile sensory platform to cause it to implement one or more interventions in order to attempt to remediate an adverse condition affecting one or more of the plants, for example, at 960 DPU 540 could command mobile sensory platform 510 to disperse a bio-control agent.
Information dissemination activity 1000 is illustrated in FIG. 10A which shows some non-limiting examples of how information can be delivered and used. Activity 1000 starts at 1010, for example, when there is updated information about the crop available to disseminate or when an end-user desires or requests information. At 1020 activity 1000 branches into the four exemplary information dissemination activities which can occur asynchronously. For example, these can each occur repeatedly, can occur at different times, or can overlap, or occur simultaneously.
At 1030 information about the crop is transmitted to an expert, such as grower 580 in FIG. 5. This information could be delivered each morning, for example, based on analysis and classification of sensor data captured overnight for the entire crop by mobile sensory platform 510 performing activity 900. Or it could be delivered in real-time as the DPU analyzes data received from the mobile sensory platform in real-time. For example, DPU could provide a grower with an alert, an alert plus a diagnosis of the problem, or an alert plus a diagnosis plus a suggested intervention plan, for specific plants or regions of the crop that are not healthy. In a non-limiting example of an implementation of the information delivery, a grower might use an interactive map of a farm or greenhouse where problematic areas identified by the DPU are marked by dots on the map. Once the grower clicks on each dot, specific information about the type or severity of the issue at that location may be displayed, along with a suggested intervention plan. In yet another non-limiting example, the information delivery will be done via a wearable device, which can be used by growers, experts and/or non-experts. The DPU may generate a notification, for example, in the form of an audible alarm or a haptic vibration that occurs when the wearer of the device comes in close proximity to a problematic area. The intervention plan may be communicated as part of the notification.
At 1040 information about the crop is transmitted to a non-expert, such as 585 in FIG. 5. For example, this information could be delivered to a worker each morning to guide the worker to specific areas of the crop that need intervention based on analysis and classification of sensor data gathered overnight for the entire crop by mobile sensory platform 510 performing activity 900. This could allow the worker to apply interventions only to those specific areas, as opposed to general broad-based applications, thereby reducing costs and exposure risk. In other example, information is delivered in real-time to a non-expert performing activity 800 of FIG. 8 so that the non-expert can learn on a plant-by-plant basis how an expert would assess plant health. This is described above in reference to 850 of FIG. 8.
At 1050 information and/or commands are transmitted to a mobile sensory platform (such as platform 510 of FIG. 5) based on analysis and classification of sensor data received from the mobile sensory platform. For example, these could be further screening assignments or commands to perform interventions based on analysis and classification of sensor data received from the mobile sensory platform. The latter is described in more detail above in reference to 960 of FIG. 9.
At 1060 information about the crop is transmitted to other parties or devices either at the crop-site or at other locations.
In the above examples, during information dissemination activities 1030, 1040, 1050 and 1060, information can be pushed from the DPU or can be pulled upon request by the end-user or device.
FIG. 10B shows a screen-shot 1000B from a reporting app that can be used to convey information about the crop to a person, such as a grower. During a setup process, the growers may define number of phases they have in their greenhouses and number of bays in each phase as well as the number of rows in each bay, thereby creating a map of the greenhouse. The dynamic mapping panel on the left alerts users to the location of problems in their crops—it indicates the phase and bay of the greenhouse for which information is being reported. In the top right quadrant the rows of that particular bay are shown. The dots indicate plants identified by the system as deviating from a healthy profile. The user can click on a dot and the location of the plant is more precisely identified, and a description of the problem and an indication of the probability of the problem is displayed, as shown in the lower right quadrant. In the illustrated example, a plant at post 11, in row 3 of bay 14B in phase 1 of the greenhouse is indicated as having a high probability that it is suffering from a sucking pest infestation and a lower probability that the problem is a bacterial disease.
Using the approach described above, the knowledge of an expert can be captured and then extended and applied at future times and/or at other locations without the expert being physically present at those times or in those locations. It is extremely valuable to be able to harness an expert's knowledge and experience in this way, both for teaching other people how to assess crops and for actual crop assessment. For example, sensor data from similar crops in other (remote) locations can be captured via mobile sensory platforms and/or handheld devices and then one or more data-derived models in the DPU can be applied to the sensor data to provide crop health assessment information about that crop without an expert needing to be there at all.
Another advantage of the present approach is that the machine-based analysis of the data by the DPU will provide a more consistent and accurate assessment of plants than a human. Generally, even an expert will not always provide a consistent assessment through the day or from one day to the next, due to a variety of factors such as fatigue, distractions or challenging environmental conditions, for example.
As illustrated in FIG. 11, information from other sources can also be employed in embodiments of the systems and methods for monitoring and assessing crop health that are described herein. FIG. 11 shows a DPU 1100 receiving inputs including expert assessments 1110, mobile sensor data 1120, and handheld sensor data 1130 as described above.
DPU 1100 can also receive human input 1140 or input derived from other sources via other devices—for example, personal observations, information about other events that may have affected the crop such as planting, watering and harvesting schedule information. DPU can also receive other crop-site sensor data 1150, for example, from fixed sensors located around the crop-site such as temperature, moisture, light, and air-flow sensors and cameras, and/or from secondary mobile sensors such as drone-mounted sensors. DPU 1100 may also draw on information stored in other databases 1160, such as pre-established scientific models and known indices and standards, or trained data from other crop-sites. This additional input can also be correlated with the expert assessment as described above to generate enhanced trained data.
As described above, DPU 1100 analyzes incoming data and information and provides crop-related information as output 1170.
Learning from Correlation of Future Crop Performance with Past Data
As described above, sensor data can be collected and analyzed, for example, in real-time to classify the current health of a plant. It can also be useful to store and re-analyze such data at a future time. For example, once sensor data is collected for the same plant over a period of time, historical spatiotemporal data can be reverse-engineered or re-analyzed in the context of data that is collected later. Once it is known that a plant is suffering from a problem, it may be possible to retroactively derive patterns or trends from the earlier data for that plant that provided indications that the plant was beginning to deviate from a healthy profile. These clues may exist in the data, even before a less sophisticated data-derived model or an expert would be able to discern a problem. This type of analysis of historical data can be used in the development of predictive models which can then be used to predict health issues and allow intervention before the plant exhibits visible symptoms.
Similarly, over the lifecycle of a crop, a large amount of sensor data and other information is typically gathered and can be stored and reverse-engineered or re-analyzed to provide useful information. The historical data can include:
expert grower assessments;
plant-related sensor data, e.g. from handheld devices and mobile sensory platforms;
data from other sensors monitoring conditions at various locations around the crop-site (for example environmental data such as temperature, light, humidity, wind);
information about how the crop was managed (for example information about seed source, planting time, irrigation, nutrition, pruning, spraying, harvesting);
information about specific interventions that were performed in response to crop monitoring.
Information relating to the actual performance of the crop can also be gathered (for example yield, quality, appearance, taste, shelf-life etc.). For example, this can be based on information provided by the grower or other humans (e.g. feedback from customer) and/or data that is captured automatically. Using predictive analytics, this performance information and data can be correlated with data gathered during the lifecycle of the crop to look for patterns and indicators earlier in the crop's lifecycle that are indicative of future performance. For example, by looking at portions of the crop (e.g. specific plants or groups of plants) that performed particularly well or particularly poorly, and analyzing past data for these portions of the crop it may be possible to correlate performance with particular growing conditions (e.g. based on the crop management information and environmental data) and or plant-based sensor data. This information can then be used in the future to try to re-create desirable growing conditions and achieve these over a larger portion of the crop, thereby enhancing performance of the crop in subsequent plantings. Similarly it can be used to identify and try to avoid adverse growing conditions, or to alert the grower when a region of the crop is exhibiting characteristics (e.g. based on monitored sensor data) indicative of future poor performance, so that remedial action can be taken. It can also be used to evaluate the effect of interventions that were performed in trying to mitigate problems with the crop, so that the effectiveness of the interventions can be improved.
FIG. 12 is a flow chart illustrating an embodiment of such a process 1200. In a first phase 1205 of process 1200, information relating to the actual performance of a crop is correlated with data gathered during the lifecycle of the crop to identify patterns and indicators. In a second phase 1245, these patterns are then used to attempt to improve the performance of a future crop. At 1210, at the end of a growing season for a particular crop, desired attributes are identified and are assessed for plants in the crop. For example, information relating to yield, taste and other attributes can be collected. At 1220 the plants are classified into groups based on their performance against one or more of these attributes (e.g. high, medium and low performance). At 1230, performance information for the classified groups of plants is correlated with historical data gathered during the lifecycle of the plants to identify patterns. These patterns can be developed into weekly trends for various parameters, at 1240. The trends are associated with the performance level of each group of plants. In the second phase 1245, a new crop is planted and information is captured for plants in the new crop. At 1250, on a weekly basis parameters for plants in the new crop are compared with the historical weekly trends for those parameters that were obtained for the previous crop at 1240. When parameters for particular plants in the new crop begin to show deviation from trends that were previously associated with high performance (desirable attributes), attempts can be made to correct those deviations through various interventions, as shown at 1260. Activities 1250 and 1260 can continue through the growing season for the new crop. At the end of the growing season, the performance of plants in the new crop is assessed with respect to one or more of the desired attributes, and again the plants are classified into groups, as shown at 1270. At 1280, this new performance information is correlated with data gathered during the lifecycle of the plants. At 1290 this information is used to update and improve the patterns and weekly trends that can be used to try to improve the performance of the next crop.
In some implementations of the present technology over 50,000 multi-dimensional data points are collected non-invasively from an individual plant in just a few seconds, allowing physiological, chemical, biological and/or physical changes inside, on the surface and/or around each plant to be detected. Thus, the technology described herein has the potential to capture massive volumes of spatiotemporal data relating to one or more crops over one or more growing seasons. Over time, through machine learning, data mining and/or pattern recognition processes, the DPU can develop specific performance patterns and data-derived models that can be used for classification and analysis of new data. Predictive models can also be developed, that can be used to predict future health or performance of plants in a crop based on their current sensor data. Using predictive models, plants that are on a path towards deteriorating health can be identified based on early clues that can be derived from their current multi-sensor data, in some cases before any single sensor or an expert could detect symptoms of problem. With this early-stage detection, preventative measures can then be taken to try to avoid deterioration in the health and/or improve the performance of the plant.
Deep learning techniques can be used, for example, for feature selection. Feature selection methods can identify attributes in the data that will enhance accuracy of a predictive model, and identify and remove irrelevant and redundant attributes in the data that do not contribute to the accuracy of a predictive model or that may even decrease the accuracy of the model. The large volumes of diverse data that can be generated through crop monitoring, and the potential value of being able to use predictive models for prophylactic intervention to maintain healthy crops, make this application particularly suitable for the application of deep learning techniques.
Generally, the greater the volume of data that is processed, the more robust and accurate the resulting data-derived models and patterns will be. In some aspects, the system can pool the assessments from multiple experts from different farms or sites with respect to a particular type of crop, for example, and then use this combined digitized expertise to enhance the quality and consistency of the data-derived models.
Growers, who are generating and providing crop-related data for development of data-derived models, as well as for automated assessment of their own crops, can then become data entrepreneurs. This is a potential source of revenue generation for growers who opt to sell their generic and non-proprietary crop-related information (such as trends and statistics), for example, to other growers or to the provider of a platform that provides data processing of crop-related data for multiple growers. In one business model, for example, growers may contribute data or statistics to a centralized or shared data-derived model, and then receive a revenue stream based on the amount of their contribution and/or based on the extent to which the model is used for analysis of third party crop-related data.
In some embodiments of the systems and methods described herein, at least some processing and/or analysis of certain types of sensor data is performed on the mobile sensory platform or sensor device itself instead of at a remote DPU. Statistics or information derived from the sensor data, rather than the sensor data itself, is then transmitted to the DPU. This can reduce the volume of data that needs to be transmitted and can increase the overall speed and efficiency of the process. For example, data that is gathered by optical sensors or stereo cameras for the purposes of disparity and depth analysis or verification purposes, could be processed on the mobile sensory platform, and then the relevant information could be transmitted to the DPU along with data from other sensors.
Natural Language Processing
Natural Language Processing (NLP) can be employed in embodiments of the systems and methods described herein, for example, NLP can be incorporated into expert knowledge capture and system training activities and/or information dissemination activities. During expert knowledge capture, verbal assessment of the plants by the expert may be captured and correlated with other input from the expert and sensor data. Different experts might use different words to describe the same situation. For example, the common name of a pest might vary in different locations, yet the terms used may all refer to the same problem. A library of terms and synonyms may be developed and then used. The language and terminology used in disseminating information about plant health maybe automatically adapted based on the geo-location and/or profile of the recipient. The NLP capability can allow experts to describe the condition of a crop verbally while capturing a sensory profile. The same terminology can be used for the reporting app. The NLP may receive and deliver information in various languages.
Mobile Sensory Platform
The mobile sensory platform employed in the systems and methods described above generally comprises more than one type of sensor mounted on a mobile platform. For example, the mobile platform can be a vehicle or cart, such as an automated robot that can patrol between rows of crops, or a drone that can fly over or in between rows of crops. Generally the mobile sensory platform will include a mounting structure such as a scaffold or rack that supports a plurality of sensors and optionally additional probes and/or devices. For example the mobile sensory platform can comprise a mast with attachable, extendable arms, or a column that houses fixed sensors and probes, or a dome that mounts on or under a mobile platform.
Most plants are highly responsive to changes in their surroundings and can convey precise information about their overall health status through those responses. At least some of the sensors that are employed in the mobile sensory platform rely on plant-generated signals or the plants' responses to stimuli to provide indicators of crop health issues. Sensors can be used to obtain information from the plants, and then trained data and associated models generated as described above, can be used to assess and/or predict plant health based on new sensor data.
The mobile sensory platform can comprise some or all of the following types of sensors:
Physiological sensors: these include sensors and probes that can measure physiological performance of crops and/or detect minute changes inside the plant caused by biotic and/or abiotic stressors. For example, chlorophyll fluorescence emitted from the leaves can provide insight into the health of the photosynthetic systems within the plant. Sensors can be used to sense the level of chlorophyll in leaves, and/or photosynthetic efficiency, and/or changes in internal chemical composition related to stress. These sensors can include pulse-modulated optical probes and detectors that stimulate the plant to give a physiological response and then detect that response. The probes might consist of LEDs with specific spectral bands that are used to excite plants and generate various physiological responses that can be correlated to photosynthetic activity or defensive chemicals inside plant foliage. The detectors may be tuned to be responsive to a narrow spectral band that corresponds with specific light that is reflected from or emitted by plants. Generally these sensors will provide the earliest clues that the plant is developing a problem, whereas some of the other sensor types described below will detect changes that occur as a disease, pest or other problem becomes further developed. The reaction of plants to stress typically begins with internal changes in the physiology and chemistry of the plant. This family of sensors can detect those early stage changes and prime the system to conduct further analysis to verify and identify the source of stress.
Surface analysis sensors: these include sensors and probes that can detect changes on the surface of the leaves and other parts of plants, for example, changes in color related to water stress, changes in surface chemistry related to biotic and abiotic stress, physical attributes of leaf surface. Such sensors generally involve spectral detection to detect certain wavelengths of visible (RGB) and near infra-red (NIR) light reflected by the plant. The probes used with these sensors may consist of full spectrum light sources, such as halogen lamps, or probes with narrow spectral bands such as ultraviolet (UV) or near infra-red (NIR). These sensors generally detect secondary stages of changes in plants, caused by stress, that occur on the surface of the foliage.
Chemical sensors: these include sensors and probes that can detect changes in plant-emitted volatile chemicals (e.g. volatile organic compounds, known as VOCs), for example, detecting herbivore-induced volatile compounds emitted by plants while under pest attack. These include photo-ionization detectors (PIDs), surface acoustic wave (SAW) sensors, quartz crystal microbalance (QMB) sensors or other types of chemical sensors that can detect certain compounds down to sub parts per billion concentrations. The chemical volatiles emitted by plants generally convey information about specific biotic stressors.
Thermal sensors: these may include thermal imaging sensors that can give information about surface damage to the foliage or fruit. For example, tiny holes that could be caused by a pest will tend to increase moisture loss and evaporation, resulting in localized lower surface temperatures that can be detected by thermal imaging.
Microclimate sensors: these include sensors and probes that can monitor changes in the microclimate around individual plants, for example, temperature and relative humidity.
Canopy scanning sensors: these include sensors and probes that can detect changes in canopy structure, for example, changes in leaf angle in response to water stress or viral infection. These can include ultrasound and/or LiDaR (light detecting and ranging) type sensors, or stereo-imaging (visible RGB and IR) sensors, for example. Such sensors may be used, for example, to generate disparity maps (providing depth measurement and information about the 3D structure of the plant canopy) which can give information about plant growth. Also they may be used to provide various vegetation indices.
The crop monitoring systems and methods described herein can function with little or no reliance on visual sensors or imaging. In some embodiments, the mobile sensory platform does not comprise cameras or other imaging devices. In other embodiments, one or more cameras or imaging devices are used primarily for verification purposes (e.g. so that a grower can inspect a photographic or video image of a plant that has been assessed by the automated system as having a problem, without having to physically go to the plant to visually inspect it). The imaging devices might be installed on a drone or other flying platforms.
In some embodiments of a mobile sensory platform, the position of some or all of the sensors is adjustable so that they can be positioned appropriately depending on the size (e.g. height and volume) of the plant and which region of the plant is to be sensed. Preferably the sensors can be moved and re-positioned automatically (rather than manually) based on commands from a control system that is responsive to inputs indicative of where the sensors should be positioned.
In some embodiments the mobile sensory platform can further comprise one or more intervention modules for administering remediation to selected plants. Such modules may be mounted to the mounting scaffold to disperse bio-control agents or other pest and disease management products where and when they are needed.
In some applications, the mobile sensory platform will be charged daily via a stationary charging station installed inside a greenhouse or at the farm. In some cases the charging station can be powered by AC electricity or via solar panels.
The mobile sensory platform can move among the rows of crops. In some embodiments, the mobile sensory platform moves on rails, such as rails that are sometimes installed in greenhouses for other purposes. The platform may detect a rail adjacent to a first row of the plants using one or more sensors and then position itself to move along the rail adjacent to the first row, or may be placed by a staff member at the beginning of a first row within a desired zone. The mobile sensory platform may then move down and back between each pair of rows of plants (assuming they are dead-ended rows) until it covers all the rows in the zone. Specific rail detecting sensors or positioning beacons can be used to guide the mobile sensory platform from one row to another. At the end of the mission, the platform may move itself to the charging station following a pre-programmed route or may remain at the end of its path to be moved by a staff member in the morning.
Some example embodiments of mobile sensory platforms that can be employed in the systems and methods described herein are illustrated in FIGS. 13A, 13B and 13C, each of which shows a front view and a side view of a mobile sensory platform.
FIG. 13A is a simplified drawing showing two orthogonal views of a mobile sensory platform 1300A having a base 1310 and wheels or rollers 1320 that can move around on the ground and/or on rails. Mobile sensory platform 1300A includes a mounting scaffold 1330 to which a plurality of sensors 1350 a-e can be attached. Mounting scaffold 1330 is equipped with a data transmission mechanism 1340 that can be placed in various locations on mounting scaffold 1330. In the illustrated embodiment a variety of sensor types are attached at various locations on mounting scaffold 1330: physiological sensors 1350 a, chemical sensors 1350 b, microclimate sensors 1350 c, surface analysis sensors 1350 d and a canopy scanning sensor 1350 e. Physiological sensors 1350 a and chemical sensors 1350 b can be placed on a rotating arm 1360 that moves both vertically and horizontally on an anchor. Physiological sensors 1350 a include excitation probes 1370 and a signal detector 1375. Mobile sensory platform can be automated and self-powered so that it moves around the greenhouse or field under the control of a control system.
FIG. 13B and FIG. 13C illustrate mobile sensory platforms 1300B and 13000, respectively, that are similar to mobile sensory platform 1300A that is illustrated in FIG. 13A, but with different mounting structures. In FIG. 13B and FIG. 13C the same numbers are used to label elements that are the same as or similar to those referred to in the description of FIG. 13A.
In mobile sensory platform 1300B of FIG. 13B a sliding actuator 1380 is attached to mounting scaffold 1330 and moves up and down vertically (shown with dashed lines in a lower position). The positioning of actuator 1380 can be based on input from one or more of the sensors (e.g. indicative of the height of the plant or the location of the region of interest on or around the plant). The sliding actuator 1380 carries physiological sensors 1350 a, chemical sensors 1350 b, microclimate sensors 1350 c and surface analysis sensors 1350 d.
Mobile sensory platform 1300C of FIG. 13C comprises a cylindrical mounting scaffold 1390 that houses the physiological sensors 1350 a, chemical sensors 1350 b, microclimate sensors 1350 c, surface analysis sensors 1350 d, and canopy scanning sensor 1350 e. Cylindrical mounting scaffold 1390 is attached to the ground mobility platform 1310. Cylindrical mounting scaffold 1390 protects the sensors that are placed in various locations inside it.
FIG. 14A is a simplified drawing showing two views of an air-borne mobile sensory platform 1400A that carries a suspended sensory scaffold 1410. Various sensors 1450 are attached to suspended sensory scaffold, similar to sensors 1350 a-d. Sensory platform 1400A also includes data transmission mechanism 1440. Housing 1460 accommodates a propulsion mechanism (not visible) which can include one or more propellers and a motor.
FIG. 14B is a simplified drawing showing two views of another air-borne mobile sensory platform 1400B comprising a dome 1415 positioned underneath housing 1465. Dome 1415 houses various sensors similar to those described above, and housing 1465 accommodates a propulsion system including four propellers 1490.
In some embodiments the mobile sensory platform further comprises a docking station or similar device where the device can be re-charged. For example, FIG. 14C shows a landing pad 1425A and 1425B where an airborne mobile sensory platform, such as 1400A of FIG. 14A or 1400B of FIG. 14B, respectively, can land and charge its batteries. The landing pads are 1425A and 1425B are each fitted with a solar panel 1435A and 1435B respectively, that harvests solar energy and turns it in to electrical power that is used to charge the airborne mobile sensory platform. In some embodiments an air-borne mobile sensory platform (such as a drone) can dock with another mobile sensory platform (such as a cart or rover robot) for re-charging and/or data transfer purposes. For example, the landing pad could be on another mobile sensory platform.
Hand-Held Device for Expert Knowledge Capture
In addition to a mobile sensory platform, hand-held devices can be employed in the systems and methods described herein in order to capture human knowledge. Some example embodiments of hand-held devices that can be employed in systems and methods as described herein are illustrated in FIGS. 15A, 15B and 15C. In some systems a hand-held multi-sensor device is also mountable to a mobile platform so that it can be used by a person or as part of an automated crop monitoring system (see FIGS. 16 and 17 below, for example).
FIG. 15A shows a simplified drawing of an embodiment of a hand-held device 1500A comprising a portable sensory platform 1510A that houses various sensors 1550. Portable sensory platform 1510A connects to a smartphone or tablet 1520 either wirelessly or by wire 1525.
FIGS. 15B and 15C are simplified drawings of various views and configurations of another embodiment of a hand-held device 1500B comprising a multi-sensor module 1560B that houses various sensors 1550 connects to a smartphone or tablet 1520.
Multi-sensor module 1560B is equipped with a set of sensors that can be positioned in two configurations. In the first configuration, shown in FIG. 15B sensors 1550 are oriented in-line with the smartphone 1520 (i.e. directed in the plane of the phone). This allows the user to point the sensors on device 1500B toward a plant and enter their assessment of the health of the plant using smart phone 1520, based on their expert knowledge. For example, in an assessment mode, an app on the phone may instruct an expert user to point the sensors toward the plant, click a run button to record sensor data, and immediately or simultaneously enter their expert assessment in response to multiple options related to the health of the plant. In one non-limiting example of the functionality of app, the multiple choices may be depicted by colored circles, for example, red for unhealthy, orange for moderately healthy and green for healthy. In another non-limiting example of the functionality of app, detailed multiple-choice questions may guide an expert user to assess the health of plant.
A second configuration of handheld device 1500B is shown in FIG. 15C where sensors 1550 on multi-sensor module 1560E are oriented in a perpendicular position relative to the plane of smartphone 1520. Multi-sensor module 1560B includes a pivot mechanism to allow this change in configuration. In this configuration handheld device 1500B can be inserted into or clipped to a shirt pocket, for example. A non-expert can carry module 1560B in this way so that sensors 1550 is directed toward the plants and can capture sensor data as the non-expert performs routine tasks. The data may be processed by a DPU and information or alerts sent back to the non-expert in real-time via device 1560B as described above.
Modular Multi-Sensor Device
FIG. 16A shows an embodiment of a multi-sensor device 1600, for crop monitoring, that can be used as a hand-held device or can be mounted to a mobile platform. Multi-sensor device 1600 comprises a housing 1610 and removable sensor modules 1620 a-g that are slotted into housing 1610. FIG. 16B shows housing 1610 with the sensor modules removed, and shows interior slots or cavities 1615 a-g that can accommodate various sensor modules. Housing 1610 has a mount 1612 a on its upper exterior surface and a similar mount 1612 b (not visible) on its lower exterior surface. FIG. 16C shows multi-sensor device 1600 with sensor modules 1620 a and 1620 b partially removed by sliding out of housing 1610. Each sensor module can be replaced with a different module having the same form factor, or optionally with a blank or cover to close off the cavity. Housing 1610 can be constructed from any suitable material or materials including for example, various plastics and/or metals. If the device is to be used outdoors the materials are preferably waterproof, weatherproof and heat-resistant to withstand harsh environmental conditions. FIG. 16D shows housing 1610 with sensor module 1620 d installed, and two alternative sensor modules 1620 h and 1620 i which can replace sensor module 1620 d. Multi-sensor device 1600 incorporates a variety of sensors in a space-efficient manner into a compact, portable device.
Sensor modules 1620 a-g can be different types of sensors that can be used to sense and provide sensor data for plant-related parameters in a crop monitoring system. The sensor modules may screw, snap, lock, or be otherwise secured in the housing. Preferably they can be removed and replaced manually or via a simple mechanism. The sensor modules are connected to a universal adapter at the rear of the housing.
FIG. 16E shows a rear perspective view of housing 1610 shown in FIG. 16B, and shows separately the bundle of components that is housed in housing 1610 when the multi-sensor device 1600 is assembled. As well as sensor modules, the components include a communications system/interface module 1660 comprising an LCD touch-screen display 1650 (for use as a user interface), and a master control unit (MCU) 1670 comprising a processor and associated software. Multi-sensor device 1600 comprises a universal adaptor (not shown) to which all of the sensor modules 1620 a-g can connect. The universal adapter provides power, data and control signal connectivity between the sensor modules and MCU 1670, regardless of the connector or port on the sensor. On MCU 1670 there is a device driver associated with each sensor.
Multi-sensor device 1600 can be mounted to a handle for use as a hand-held device, or can be mounted to a mobile platform such as a vehicle, cart, robot, drone, or zip-line harness for automated use. FIG. 17A shows an example of handle 1710 having a mounting connector 1715 which can be snapped into or otherwise removably fastened to mount 1612 b on housing 1610 of multi-sensor device 1600, as illustrated in FIG. 17B. FIG. 17C shows a mounting connector 1720 that can be snapped into or otherwise removably fastened to mount 1612 a or 1612 b on housing 1610 via portion 1725. Mounting connector 1720 includes tongue 1730 that can be slide-mounted into any of four corresponding grooves 1735 a-d on a mounting bracket 1740 shown in FIGS. 17D and 17E. Bracket 1740 can be connected to or integrated into a mobile platform. Multi-sensor device can be mounted so that is side-facing, upward-facing, downward-facing or at a desired angle relative to the plants. Housing 1610 may have additional mounts on it. Mounting connectors 1715 or 1720, or bracket 1725 may include a pivot mechanism, so that the angle of orientation of the multi-sensor platform relative to the plants can be adjusted.
Mounts 1612 a and 1612 b also serve as power connectors via which an external power source can be connected to provide electrical power to the multi-sensor device 1600. For example, handle 1710 can comprise a rechargeable battery or, if the device is mounted to a mobile platform, it can be connected to receive power from a power source on-board the device.
Housing 1610 can be populated with different sensor modules. This modularity and flexibility makes multi-sensor platform versatile easily adaptable for different crop-sensing applications. In one implementation, a multi-sensor device comprises the following modules contained within a single housing:
    • (a) a physiological sensor module comprising configurable optical probes and tunable detectors;
    • (b) a surface analysis sensor module comprising a full spectrum light source (for example, a halogen lamp) as the probe, and spectroscopic detectors;
    • (c) a chemical analysis sensor module comprising a photo-ionization detector (PID), or a surface acoustic wave (SAW) sensor, or a quartz microbalance crystal (QMB) sensor;
    • (d) a thermal imaging sensor module which can produce a graphical heat map of the surface of a plant;
    • (e) a vision system module comprising a pair of high resolution ROB-IR cameras for stereo-imaging, and infra-red distance sensors the output of which can be used to calibrate the stereo images; and
    • (f) an environmental measurement module comprising temperature and humidity sensors to evaluate the microclimate of the area surrounding a plant.
A single plant can be assessed using all of the sensor modules in the multi-sensor device within a few seconds or less.
In the physiological sensor module, the optical probes can comprise LEDs with specific spectral bands that are used to excite plants and generate various physiological responses. An optical detector array can incorporate multiple-wavelength optical detectors selected or tuned to be responsive to a narrow spectral band that corresponds with specific light that is reflected from or emitted by the plants.
FIG. 16D shows three physiological sensor modules 1620 d, 1620 h and 1620 i with different configurations of LEDs and detectors. Module 1620 d has four central detectors (these could be blue, yellow, red and far red detectors, for example) surrounded by a ring of LED probes, with another array of LED probes on each side. The specific LEDs and detectors can be selected and changed to suit a specific crop or to obtain information about a specific condition. Again, this provides flexibility and versatility to the device.
The MCU controls the sequence and spacing of the LED pulses, pulse duration, pulse intensity and the number of LED pulses that are used to stimulate the plant in a particular assessment. The optical detectors receive reflected and emitted light from the plant to excitation light and generate a low-level current. The current is significantly amplified (for example, up to 107 times), and filtered before being digitized. In one example, the plant is stimulated with a series of one hundred 100 microsecond pulses of RGB-UV-NIR light separated by 25 millisecond gaps, and at least 4 detectors are used to measure light reflected and emitted from plants.
Analog front-end (AFE) circuitry that allows the user to switch between “day” and “night” modes can be incorporated into the MCU of the multi-sensor device. Once set in day mode, the AFE filters out ambient light, thereby enabling detection of extremely low levels of response light (signal) in the presence of high intensity ambient light (noise). The night mode is adapted or optimized for measurements in low ambient light (noise) intensity. The processed AFE response (amplified and filtered) is digitized by a multi-channel high-speed (e.g. 2 mega samples per second) high-resolution (e.g. 16-bit) analog-to-digital converter (ADC). The optical detector system measures discreet wavelength bands at very high speeds. Optical readings and digitized data are transmitted via the communications system/interface. During further processing of the data, mathematical manipulation can be used to eliminate distance and uneven surfaces as factors in the response data. For example, if the detectors and light probes are in the same plain, a ratio of measurements in correlation to each other can be used to overcome the issue of the distance. For example, instead of comparing a sensor A reading from plant 1 to plant 2, the ratio of a pair of sensor readings (sensor A reading /sensor B reading) for each of the two plants can be compared. This way, although the individual sensor readings might vary from plant to plant based on the distance between the plant and the sensor, comparison of the ratios will provide meaningful comparison between plants.
During operation of the surface analysis sensor module, the full spectrum light source is also activated during a light emission period, and spectroscopic detectors measure the entire wavelength from NIR to UV at high resolution, but at lower speed than for the physiological sensing. Spectroscopy readings are transmitted via the communications system/interface.
During operation of the chemical analysis sensor module, the chemical sensor detects VOCs produced by the plant down to sub ppb levels, and produces a voltage in response to the compound concentration level. The output voltage is digitized by a high-resolution analog-to-digital converter (ADC) and the data is transmitted via the communications system/interface.
Embodiments of the technology, devices, systems and methods described herein can be used separately or can be used in various combinations as desired.
While particular elements, embodiments and applications of the present invention have been shown and described, it will be understood, that the invention is not limited thereto since modifications can be made by those skilled in the art without departing from the scope of the present disclosure, particularly in light of the foregoing teachings.

Claims (21)

What is claimed is:
1. A mobile sensory platform for capturing and transmitting sensor data for individual plants in a crop, the mobile sensory platform comprising:
a propulsion system;
a plurality of sensors adjustably mounted to a support structure so that, during operation of the mobile sensory platform, at least one of the plurality of sensors is configured to be positioned proximate to a plant and to capture data pertaining to at least one plant-related parameter of the plant;
a communications interface configured to engage in two-way communication over a wireless network;
an electrical power supply; and
a control system configured to control movement of the mobile sensory platform and operation of the plurality of sensors;
wherein the plurality of sensors comprises an optical probe configured to stimulate the plant with excitation light and a tunable detector configured to receive light reflected and/or emitted from the plant in response to the excitation light; and
wherein the control system is configured to control operation of the optical probe based on one or more input signals from at least one of the plurality of sensors.
2. The mobile sensory platform of claim 1, wherein the optical probe is configured to transmit the excitation light selectively in one or more spectral bands.
3. The mobile sensory platform of claim 2, wherein:
the optical probe comprises a plurality of light-emitting diodes (LEDs);
a first subset of the plurality of LEDs is configured to transmit light in a first spectral band; and
a second subset of the plurality of LEDs is configured to transmit light in a second spectral band.
4. The mobile sensory platform of claim 3, wherein the control system is configured to cause at least some of the plurality of LEDs to emit a sequence of pulses of excitation light.
5. The mobile sensory platform of claim 4, wherein the control system is configured to control a spacing of the pulses, a duration of the pulses, an intensity of the pulses, and a number of the pulses within the sequence of pulses of excitation light.
6. The mobile sensory platform of claim 4, wherein the control system is configured to vary a spacing of the pulses, a duration of the pulses, and an intensity of the pulses within the sequence of pulses of excitation light.
7. The mobile sensory platform of claim 4, wherein the tunable detector is configured to be tunable responsive to a spectral band that corresponds to the light reflected and/or emitted from the plant in response to the excitation light.
8. The mobile sensory platform of claim 4, wherein the tunable detector comprises one or more multiple-wavelength optical detectors.
9. The mobile sensory platform of claim 4, wherein the control system is configured to tune the tunable detector based on one or more input signals from at least one of the plurality of sensors.
10. The mobile sensory platform of claim 4, wherein:
the tunable detector is configured to generate an electrical current output signal in response to receiving the light reflected and/or emitted from the plant in response to the excitation light; and
the mobile sensory platform further comprises an amplifier configured to amplify the output signal, a filter configured to filter the amplified output signal, and an analog-to-digital converter configured to digitize the amplified and filtered output signal.
11. The mobile sensory platform of claim 1, wherein the plurality of sensors comprises:
a physiological sensor comprising the optical probe and the tunable detector;
a surface analysis sensor comprising a full-spectrum light source and one or more spectroscopic detectors; and
a chemical analysis sensor comprising at least one of: a photo-ionization detector, a surface acoustic wave sensor, or a quartz crystal microbalance sensor.
12. The mobile sensory platform of claim 1, wherein the mobile sensory platform is a ground-based platform.
13. The mobile sensory platform of claim 1, wherein the mobile sensory platform is an air-borne platform.
14. A multi-sensor device for capturing and transmitting sensor data for individual plants in a crop, the multi-sensor device comprising:
a housing comprising a plurality of interior cavities;
a plurality of sensor modules, each sensor module occupying one of the plurality of cavities, each sensor module configured to sense at least one plant-related parameter when the multi-sensor device is positioned proximate to a plant;
a control unit configured to control operation of the plurality of sensor modules;
a communications interface connected to the control unit and configured to transmit and receive information, the communications interface configured to transmit data from the plurality of sensor modules;
a user interface connected to the control unit; and
an electrical power connector configured to connect the multi-sensor device to a power source;
wherein the plurality of sensor modules comprises a physiological sensor module comprising an optical excitation probe and a tunable detector, the control unit configured to control operation of the optical excitation probe based on one or more input signals from at least one of the plurality of sensor modules.
15. The multi-sensor device of claim 14, wherein:
the multi-sensor device further comprises a handle; and
the multi-sensor device is a portable hand-held device.
16. The multi-sensor device of claim 14, wherein:
the optical excitation probe is configured to transmit excitation light selectively in one or more spectral bands;
the optical excitation probe comprises a plurality of light-emitting diodes (LEDs);
a first subset of the plurality of LEDs is configured to transmit light in a first spectral band; and
a second subset of the plurality of LEDs is configured to transmit light in a second spectral band.
17. The multi-sensor device of claim 16, wherein the control unit is configured to cause at least some of the plurality of LEDs to emit a sequence of pulses of excitation light.
18. The multi-sensor device of claim 17, wherein the control unit is configured to control a spacing of the pulses, a duration of the pulses, an intensity of the pulses, and a number of the pulses within the sequence of pulses of excitation light.
19. The multi-sensor device of claim 17, wherein the control unit is configured to vary a spacing of the pulses, a duration of the pulses, and an intensity of the pulses within the sequence of pulses of excitation light.
20. The multi-sensor device of claim 14, wherein the tunable detector is configured to be tunable responsive to a spectral band that corresponds to light reflected and/or emitted from the plant in response to excitation light.
21. The multi-sensor device of claim 14, wherein the control unit is configured to tune the tunable detector based on one or more input signals from at least one of the plurality of sensor modules.
US16/268,744 2015-07-30 2019-02-06 Multi-sensor platform for crop health monitoring Active US10871480B2 (en)

Priority Applications (7)

Application Number Priority Date Filing Date Title
US16/268,744 US10871480B2 (en) 2015-07-30 2019-02-06 Multi-sensor platform for crop health monitoring
US17/099,422 US11499955B2 (en) 2015-07-30 2020-11-16 Crop health monitoring using predictive modeling
US18/054,437 US11965870B2 (en) 2015-07-30 2022-11-10 Multi-sensor platform for crop health monitoring
US18/056,598 US12038426B2 (en) 2015-07-30 2022-11-17 Multi-sensor platform for crop health monitoring
US18/056,610 US11874265B2 (en) 2015-07-30 2022-11-17 Multi-sensor platform for crop health monitoring
US18/056,604 US12111301B2 (en) 2015-07-30 2022-11-17 Multi-sensor platform for crop health monitoring
US18/185,211 US11867680B2 (en) 2015-07-30 2023-03-16 Multi-sensor platform for crop health monitoring

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201562198761P 2015-07-30 2015-07-30
US15/219,320 US10241097B2 (en) 2015-07-30 2016-07-26 Multi-sensor platform for crop health monitoring
US16/268,744 US10871480B2 (en) 2015-07-30 2019-02-06 Multi-sensor platform for crop health monitoring

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US15/219,320 Continuation US10241097B2 (en) 2015-07-30 2016-07-26 Multi-sensor platform for crop health monitoring

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/099,422 Continuation US11499955B2 (en) 2015-07-30 2020-11-16 Crop health monitoring using predictive modeling

Publications (2)

Publication Number Publication Date
US20190170718A1 US20190170718A1 (en) 2019-06-06
US10871480B2 true US10871480B2 (en) 2020-12-22

Family

ID=57885958

Family Applications (10)

Application Number Title Priority Date Filing Date
US15/219,320 Active 2037-05-02 US10241097B2 (en) 2015-07-30 2016-07-26 Multi-sensor platform for crop health monitoring
US15/219,328 Active 2037-12-22 US11287411B2 (en) 2015-07-30 2016-07-26 Systems and methods for crop health monitoring, assessment and prediction
US16/268,744 Active US10871480B2 (en) 2015-07-30 2019-02-06 Multi-sensor platform for crop health monitoring
US17/099,422 Active 2036-07-31 US11499955B2 (en) 2015-07-30 2020-11-16 Crop health monitoring using predictive modeling
US17/553,332 Pending US20220107298A1 (en) 2015-07-30 2021-12-16 Systems and methods for crop health monitoring, assessment and prediction
US18/054,437 Active US11965870B2 (en) 2015-07-30 2022-11-10 Multi-sensor platform for crop health monitoring
US18/056,604 Active US12111301B2 (en) 2015-07-30 2022-11-17 Multi-sensor platform for crop health monitoring
US18/056,598 Active US12038426B2 (en) 2015-07-30 2022-11-17 Multi-sensor platform for crop health monitoring
US18/056,610 Active US11874265B2 (en) 2015-07-30 2022-11-17 Multi-sensor platform for crop health monitoring
US18/185,211 Active US11867680B2 (en) 2015-07-30 2023-03-16 Multi-sensor platform for crop health monitoring

Family Applications Before (2)

Application Number Title Priority Date Filing Date
US15/219,320 Active 2037-05-02 US10241097B2 (en) 2015-07-30 2016-07-26 Multi-sensor platform for crop health monitoring
US15/219,328 Active 2037-12-22 US11287411B2 (en) 2015-07-30 2016-07-26 Systems and methods for crop health monitoring, assessment and prediction

Family Applications After (7)

Application Number Title Priority Date Filing Date
US17/099,422 Active 2036-07-31 US11499955B2 (en) 2015-07-30 2020-11-16 Crop health monitoring using predictive modeling
US17/553,332 Pending US20220107298A1 (en) 2015-07-30 2021-12-16 Systems and methods for crop health monitoring, assessment and prediction
US18/054,437 Active US11965870B2 (en) 2015-07-30 2022-11-10 Multi-sensor platform for crop health monitoring
US18/056,604 Active US12111301B2 (en) 2015-07-30 2022-11-17 Multi-sensor platform for crop health monitoring
US18/056,598 Active US12038426B2 (en) 2015-07-30 2022-11-17 Multi-sensor platform for crop health monitoring
US18/056,610 Active US11874265B2 (en) 2015-07-30 2022-11-17 Multi-sensor platform for crop health monitoring
US18/185,211 Active US11867680B2 (en) 2015-07-30 2023-03-16 Multi-sensor platform for crop health monitoring

Country Status (2)

Country Link
US (10) US10241097B2 (en)
CA (2) CA2937574A1 (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210315150A1 (en) * 2018-08-16 2021-10-14 Mahindra & Mahindra Limited Method and systems for generating prescription plans for a region under cultivation
US20220104437A1 (en) * 2020-10-02 2022-04-07 Ecoation Innovative Solutions Inc. Reduction of time of day variations in plant-related data measurements
US11499955B2 (en) 2015-07-30 2022-11-15 Ecoation Innovative Solutions Inc. Crop health monitoring using predictive modeling
US20220377991A1 (en) * 2019-11-01 2022-12-01 Omron Corporation Plant biosensor
US11555690B2 (en) 2020-11-13 2023-01-17 Ecoation Innovative Solutions Inc. Generation of stereo-spatio-temporal crop condition measurements based on human observations and height measurements
US20230074663A1 (en) * 2021-09-07 2023-03-09 X Development Llc Auditing task performance
US11666004B2 (en) 2020-10-02 2023-06-06 Ecoation Innovative Solutions Inc. System and method for testing plant genotype and phenotype expressions under varying growing and environmental conditions
WO2024025527A1 (en) * 2022-07-27 2024-02-01 Neatleaf, Inc. Aerial sensor and manipulation platform for farming and method of using same
US11925151B2 (en) 2020-11-13 2024-03-12 Ecoation Innovative Solutions Inc. Stereo-spatial-temporal crop condition measurements for plant growth and health optimization
US12067711B2 (en) 2020-11-13 2024-08-20 Ecoation Innovative Solutions Inc. Data processing platform for analyzing stereo-spatio-temporal crop condition measurements to support plant growth and health optimization
US12131393B2 (en) 2020-10-02 2024-10-29 Ecoation Innovative Solutions Inc. Platform for real-time identification and resolution of spatial production anomalies in agriculture

Families Citing this family (119)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9804097B1 (en) * 2014-06-02 2017-10-31 Iowa State University Research Foundation, Inc. Crop stand analyzer using reflective laser proximity sensors
WO2017040492A1 (en) * 2015-09-01 2017-03-09 Wal-Mart Stores, Inc. Method and apparatus to automatically facilitate changes to a fresh produce display
US11430023B2 (en) 2015-09-08 2022-08-30 Agrisync, Inc. Integrated customer support system for field-centric business sectors
US11612092B1 (en) 2015-12-18 2023-03-28 Realmfive, Inc. Agricultural system having actuatable work tool assemblies
CA3029134A1 (en) * 2015-12-18 2017-06-22 Realmfive, Inc. Autonomous integrated farming system
US11774944B2 (en) 2016-05-09 2023-10-03 Strong Force Iot Portfolio 2016, Llc Methods and systems for the industrial internet of things
US20180284753A1 (en) * 2016-05-09 2018-10-04 StrongForce IoT Portfolio 2016, LLC Methods and systems for data storage and communication in an internet of things chemical production process
US11327475B2 (en) 2016-05-09 2022-05-10 Strong Force Iot Portfolio 2016, Llc Methods and systems for intelligent collection and analysis of vehicle data
US9898688B2 (en) * 2016-06-01 2018-02-20 Intel Corporation Vision enhanced drones for precision farming
IL264830B (en) * 2016-08-18 2022-07-01 Tevel Aerobotics Tech Ltd System and method for drone fleet management for harvesting and dilution
US11538099B2 (en) 2016-09-21 2022-12-27 Iunu, Inc. Online data market for automated plant growth input curve scripts
US11244398B2 (en) 2016-09-21 2022-02-08 Iunu, Inc. Plant provenance and data products from computer object recognition driven tracking
US10791037B2 (en) 2016-09-21 2020-09-29 Iunu, Inc. Reliable transfer of numerous geographically distributed large files to a centralized store
US10635274B2 (en) * 2016-09-21 2020-04-28 Iunu, Inc. Horticultural care tracking, validation and verification
US10627386B2 (en) 2016-10-12 2020-04-21 Aker Technologies, Inc. System for monitoring crops and soil conditions
DE102016220222A1 (en) * 2016-10-17 2018-04-19 Robert Bosch Gmbh Diagnostic device and method for state detection of an object by means of the diagnostic device
US9842449B1 (en) 2016-11-18 2017-12-12 Motogo, Llc Secure transport container
US20200383313A1 (en) * 2016-11-28 2020-12-10 Zasso Gmbh Method and device for treating grounds and weed killing device
JP6605523B2 (en) * 2017-03-10 2019-11-13 ヤンマー株式会社 Field information display system
JP6873542B2 (en) * 2017-03-13 2021-05-19 大崎電気工業株式会社 Cultivation environment monitoring system
US11328614B1 (en) * 2017-03-30 2022-05-10 Alarm.Com Incorporated System and method for returning a drone to a dock after flight
SE540910C2 (en) * 2017-05-02 2018-12-18 Gorzsas Andras Spectroscopic method and device for determining the characteristics of a tree
GB2562275A (en) * 2017-05-10 2018-11-14 Univ Warwick Monitoring of an agricultural product and/or the environment in the vicinity of the agricultural product
JP7162016B2 (en) * 2017-05-22 2022-10-27 コルテバ アグリサイエンス エルエルシー Selective detection of bed bugs
US10921303B1 (en) 2017-05-31 2021-02-16 Iowa State University Research Foundation, Inc. Miniature sensors with probe insertable into and for obtaining measurements from plants and a variety of other mediums
JOP20190048A1 (en) * 2017-06-14 2019-03-19 Grow Solutions Tech Llc Systems and methods for self-learning in a grow pod
CA3068855A1 (en) * 2017-07-06 2019-01-10 Bayer Aktiengesellschaft Apparatus for weed control
DE102017116658A1 (en) 2017-07-24 2019-01-24 Vorwerk & Co. Interholding Gmbh Automatically movable device with a monitoring module outdoors
DE102017116661A1 (en) * 2017-07-24 2019-01-24 Vorwerk & Co. Interholding Gmbh System for recording three-dimensional environmental data, in particular for plant care and sensor module
US11536660B2 (en) 2017-07-28 2022-12-27 Cornell University In situ sensing of water potential
US11331019B2 (en) 2017-08-07 2022-05-17 The Research Foundation For The State University Of New York Nanoparticle sensor having a nanofibrous membrane scaffold
US11537107B2 (en) * 2017-10-09 2022-12-27 Siemens Aktiengesellschaft Autonomous mobile robots for movable production systems
DK3476188T3 (en) * 2017-10-30 2022-08-08 Kverneland Group Les Landes Genusson METHOD AND SYSTEM FOR DETERMINING AND STORING SURFACE CONDITIONS OF A FIELD
US11275068B2 (en) * 2017-10-31 2022-03-15 Honeywell International Inc. Remote gas sensing using UAVs
AR113330A1 (en) * 2017-10-31 2020-04-08 Univ La Republica EQUIPMENT AND METHOD TO DETERMINE THE ELASTICITY OF SOFT SOLIDS
US11690369B2 (en) * 2017-11-29 2023-07-04 Troy Benjegerdes Method and apparatus for weed control using a high intensity light source
US11657488B2 (en) * 2017-12-15 2023-05-23 Vineland Research And Innovation Centre Methods and systems related to mushroom ripeness determination
US10638667B2 (en) * 2017-12-26 2020-05-05 X Development Llc Augmented-human field inspection tools for automated phenotyping systems and agronomy tools
CN108362326B (en) 2018-01-03 2020-12-18 江苏大学 Suspension rail type greenhouse comprehensive information automatic cruise monitoring device
CN108387262B (en) * 2018-01-03 2020-05-01 江苏大学 Greenhouse information automatic monitoring method based on suspension type sliding rail platform
US10591352B2 (en) * 2018-01-10 2020-03-17 X Development Llc Sensor and system for plant canopy measurement
US10809745B2 (en) 2018-01-15 2020-10-20 Motogo, Llc System and method of last mile delivery
US11062516B2 (en) 2018-02-07 2021-07-13 Iunu, Inc. Augmented reality based horticultural care tracking
WO2019173876A1 (en) * 2018-03-16 2019-09-19 Alinda Chandra Mondal Soil ecosystem management and intelligent farming arrangement
US11116125B2 (en) * 2018-03-26 2021-09-14 Steve M. Murphy Greenhouse mobile sensor vehicle
CN108275282B (en) * 2018-03-29 2024-03-08 江苏建筑职业技术学院 Ground-air complementary remote sensing measurement device
EP3782001A4 (en) * 2018-04-16 2021-06-02 Growponics Greenhouse Technology Ltd. Greenhouse control system
CN108594906A (en) * 2018-04-18 2018-09-28 河南工业大学 A kind of wisdom agricultural environment information detecting system
US11022511B2 (en) 2018-04-18 2021-06-01 Aron Kain Sensor commonality platform using multi-discipline adaptable sensors for customizable applications
US10755367B2 (en) 2018-05-10 2020-08-25 The Climate Corporation Analysis and presentation of agricultural data
US10748021B2 (en) * 2018-05-11 2020-08-18 Samsung Electronics Co., Ltd. Method of analyzing objects in images recorded by a camera of a head mounted device
CN108764544A (en) * 2018-05-17 2018-11-06 广东电网有限责任公司 Circuit hidden danger prediction technique and device
WO2019222860A1 (en) * 2018-05-25 2019-11-28 Greenearth Automation Inc. System, method and/or computer readable medium for growing plants in an autonomous green house
WO2020005317A1 (en) * 2018-06-26 2020-01-02 Just Greens, Llc Controlling plant growth conditions
US10936870B2 (en) * 2018-07-05 2021-03-02 Iron Ox, Inc. Method for selectively deploying sensors within an agricultural facility
WO2020039434A1 (en) * 2018-08-20 2020-02-27 Ramot At Tel-Aviv University Ltd., Plant-monitor
US10524461B1 (en) * 2018-09-25 2020-01-07 Jace W. Files Pest detector to identify a type of pest using machine learning
US11058074B2 (en) * 2018-10-08 2021-07-13 Taiwan Semiconductor Manufacturing Co., Ltd. Apparatus, systems and methods for irrigating lands
CN112823229B (en) 2018-10-09 2023-02-03 天马迪克斯有限公司 Diagnosis support system and method thereof
SG11202105191VA (en) * 2018-11-21 2021-06-29 Dow Agrosciences Llc Selective detection of bed bugs
US20210321553A1 (en) * 2018-11-27 2021-10-21 Mahindra & Mahindra Limited Methods and systems for applying a remedy for a region under cultivation
AU2018282338A1 (en) 2018-12-19 2020-07-09 Canon Kabushiki Kaisha Method, system and apparatus for managing crops
EP3897107A4 (en) * 2018-12-19 2022-10-05 Innerplant, Inc. Sensor plant and method for identifying stressors in crops based on characteristics of sensor plants
US11861544B2 (en) 2018-12-21 2024-01-02 Motogo, Llc System and method for conditional delivery of a transport container
CN109828623B (en) * 2018-12-28 2021-03-12 北京农业信息技术研究中心 Production management method and device for greenhouse crop context awareness
EP3693735A1 (en) * 2019-02-06 2020-08-12 SpexAI GmbH Method and device for analysing plants
US11120552B2 (en) * 2019-02-27 2021-09-14 International Business Machines Corporation Crop grading via deep learning
CN109813852B (en) * 2019-03-08 2024-03-05 山东农业大学 High-flux phenotype information acquisition device for field wheat and control method thereof
GB2582547B (en) * 2019-03-18 2022-08-10 Vivent Sa Apparatus and method for assessing a characteristic of a plant
US11610158B2 (en) * 2019-05-02 2023-03-21 Mjnn Llc Automated placement of plant varieties for optimum performance within a grow space subject to environmental condition variability
DE102019207242A1 (en) * 2019-05-17 2020-11-19 Zf Friedrichshafen Ag Agricultural vehicle location
US10957036B2 (en) * 2019-05-17 2021-03-23 Ceres Imaging, Inc. Methods and systems for crop pest management utilizing geospatial images and microclimate data
DE102019207245A1 (en) * 2019-05-17 2020-11-19 Zf Friedrichshafen Ag Agricultural land classification
KR102251798B1 (en) * 2019-05-24 2021-05-13 홍익대학교세종캠퍼스산학협력단 System for sprinkling agricultural chemicals and fertilizer using agriculture solar power generation structure
WO2020237395A1 (en) * 2019-05-31 2020-12-03 Argus Control Systems Ltd. Plant health monitoring apparatus and methods
US10986827B2 (en) 2019-06-20 2021-04-27 InnerPlant, Inc. Methods for deploying biosentinels to agricultural fields and monitoring biotic and abiotic stresses in crops remotely
CN114026420A (en) * 2019-06-24 2022-02-08 巴斯夫农化商标有限公司 Decision system for crop enhancement product application using remote sensing based soil parameters
WO2021005589A1 (en) * 2019-07-05 2021-01-14 Arugga A.I Farming Ltd Automated plant monitoring systems and methods
US11665992B2 (en) * 2019-07-30 2023-06-06 Root Applied Sciences Inc. Predictive agricultural management system and method
CN110440855A (en) * 2019-08-02 2019-11-12 邢台职业技术学院 A kind of intelligent environment monitoring information collecting device and its application method
US20210048822A1 (en) * 2019-08-12 2021-02-18 Ecoation Innovative Solutions Inc. Mobile platform for crop monitoring and treatment
JP6742491B2 (en) * 2019-10-16 2020-08-19 ヤンマーパワーテクノロジー株式会社 Field information display system
CN110679440A (en) * 2019-10-29 2020-01-14 广东食品药品职业学院 Precise irrigation system and irrigation method applying Beidou short message communication function
KR102336828B1 (en) * 2019-11-06 2021-12-09 대한민국 Smart control device capable of selective control by image or lidar based fruit tree shape and presence
JP2021073925A (en) * 2019-11-11 2021-05-20 直政 鈴木 Management and checking method on growth of field crop
CA3163802A1 (en) * 2020-01-06 2021-07-15 Thomas James MATARAZZO Mobile sensing system for crop monitoring
US20210270791A1 (en) * 2020-02-27 2021-09-02 Agco Corporation Crop monitoring system and method
CN111400534B (en) * 2020-03-05 2023-09-19 杭州海康威视系统技术有限公司 Cover determination method and device for image data and computer storage medium
CN111239015A (en) * 2020-03-08 2020-06-05 瑞德(新乡)路业有限公司 Catering oil smoke intelligence supervision device easy to assemble and disassemble
US11720980B2 (en) 2020-03-25 2023-08-08 Iunu, Inc. Crowdsourced informatics for horticultural workflow and exchange
US20210318292A1 (en) * 2020-04-13 2021-10-14 Sensorium Bio, Inc. System and Method to Detect Small Molecules
CN111598181B (en) * 2020-05-21 2023-04-07 中国热带农业科学院热带生物技术研究所 Banana flower and leaf heart rot APP identification method and system
US11631475B2 (en) * 2020-05-26 2023-04-18 Ecoation Innovative Solutions Inc. Real-time projections and estimated distributions of agricultural pests, diseases, and biocontrol agents
US11794782B2 (en) * 2020-06-26 2023-10-24 Gm Cruise Holdings Llc Sensor aggregation module
CN112051361A (en) * 2020-08-20 2020-12-08 北京农业信息技术研究中心 Multifunctional field plant growth data acquisition vehicle
US11076589B1 (en) * 2020-10-16 2021-08-03 Verdant Robotics, Inc. Autonomous agricultural treatment system using map based targeting of agricultural objects
US11425852B2 (en) 2020-10-16 2022-08-30 Verdant Robotics, Inc. Autonomous detection and control of vegetation
US12004441B2 (en) 2020-10-28 2024-06-11 International Business Machines Corporation Soil density prediction for subsoil crops
US11879751B2 (en) * 2020-11-03 2024-01-23 Ford Global Technologies, Llc Stand-alone inspection apparatus for use in a manufacturing facility
US20220414795A1 (en) * 2020-11-18 2022-12-29 Hijo Ip Pte Ltd. Crop disease prediction and treatment based on artificial intelligence (ai) and machine learning (ml) models
US11158006B1 (en) 2020-11-24 2021-10-26 Edible Garden Ag Incorporated Greenhouse agriculture system
IT202000030038A1 (en) * 2020-12-11 2021-03-11 Impattozero S R L Device for monitoring and managing the environmental conditions of an ecosystem
WO2022150328A1 (en) * 2021-01-06 2022-07-14 Matergenics, Inc. Electrochemical plant activity monitor
CA3208111A1 (en) * 2021-01-11 2022-07-14 Agtonomy Crop view and irrigation monitoring
CN112800929B (en) * 2021-01-25 2022-05-31 安徽农业大学 Bamboo shoot quantity and high growth rate online monitoring method based on deep learning
US11895942B2 (en) 2021-03-10 2024-02-13 Earth Scout GBC Plant growth platform
KR20240011715A (en) * 2021-04-27 2024-01-26 조르디, 인크. Autonomous Greenhouse Control System
CA3117042A1 (en) 2021-05-03 2022-11-03 AIRM Consulting Ltd. Computer vision system and method for agriculture
TWI799850B (en) * 2021-05-05 2023-04-21 嶺東科技大學 Farmland monitoring system
US20230025714A1 (en) * 2021-07-16 2023-01-26 Bovi, Inc. Swarm Based Orchard Management
CN113741597B (en) * 2021-09-03 2022-04-12 安徽中昆绿色防控科技有限公司 Intelligent control system for insect trapping, measuring and reporting in agriculture and forestry
US11399531B1 (en) 2021-10-20 2022-08-02 Verdant Robotics, Inc. Precision detection and control of vegetation with real time pose estimation
EP4430391A1 (en) 2021-11-09 2024-09-18 Signify Holding B.V. Systems and methods for determining a location of a pest infested and/or diseased plant
CN114330785B (en) * 2022-03-14 2022-06-28 北京绿佳杰园林工程有限公司 Intelligent saving type maintenance system and method for garden vegetation
CN114431126B (en) * 2022-04-11 2022-08-26 天地智控(天津)科技有限公司 Well and canal double-irrigation intelligent management and control system
WO2023212494A2 (en) * 2022-04-26 2023-11-02 Pioneer Hi-Bred International, Inc. Predictive pesticide resistance information generation and use
CN114868640B (en) * 2022-07-12 2022-11-11 成都秦川物联网科技股份有限公司 Smart city green space irrigation water distribution method and system based on Internet of things
EP4316236A1 (en) * 2022-08-02 2024-02-07 Source.ag International B.V. Greenhouse environment optimisation
CN115759665B (en) * 2022-11-24 2024-04-09 安徽科技学院 Method for constructing intelligent field planting target crop growth information model

Citations (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4755942A (en) 1985-05-17 1988-07-05 The Standard Oil Company System for indicating water stress in crops which inhibits data collection if solar insolation exceeds a range from an initial measured value
US4876647A (en) 1985-05-17 1989-10-24 The Standard Oil Company Apparatus for determining water stress in crops
US5130545A (en) 1991-04-05 1992-07-14 Lussier Robert R Video imaging plant management system
US5839106A (en) 1996-12-17 1998-11-17 Apple Computer, Inc. Large-vocabulary speech recognition using an integrated syntactic and semantic statistical language model
US6397162B1 (en) 1999-03-29 2002-05-28 Phytech Ltd. System, device and method for estimating evapo-transpiration in plants
US20020167587A1 (en) 2001-05-10 2002-11-14 E.C.R Corporation Monitoring system
US20020170229A1 (en) 2001-04-13 2002-11-21 Phytech Ltd. System and method for phytomonitoring
DE10148747A1 (en) 2001-09-26 2003-04-17 Norsk Hydro As Non-contact monitoring crop and other vegetation condition, for determining fertilizer requirements, uses flash lights for the reflected light to be received by a single photo-diode giving digitized signals for processing
US6573512B1 (en) 1997-12-22 2003-06-03 Communaute Europeene Analysing device non-destructive of plants and vehicle comprising such device on board
US6657117B2 (en) 2000-07-14 2003-12-02 Microsoft Corporation System and methods for providing automatic classification of media entities according to tempo properties
US20030229497A1 (en) 2000-04-21 2003-12-11 Lessac Technology Inc. Speech recognition method
US6701665B1 (en) 2000-10-23 2004-03-09 Phytech Ltd. Remote phytomonitoring
US20040241635A1 (en) 2001-04-18 2004-12-02 Buckley Wayne T. Assay for seed vigour
US7112806B2 (en) 2001-09-27 2006-09-26 Robert Lussier Bio-imaging and information system for scanning, detecting, diagnosing and optimizing plant health
US7412330B2 (en) 2005-08-01 2008-08-12 Pioneer Hi-Bred International, Inc. Sensor system, method, and computer program product for plant phenotype measurement in agricultural environments
US7487925B2 (en) 2001-05-31 2009-02-10 Skinner Paul W Conduit including sensors for monitoring and maintenance of vegetation
US7617057B2 (en) 2005-12-21 2009-11-10 Inst Technology Development Expert system for controlling plant growth in a contained environment
WO2009141465A1 (en) 2008-05-19 2009-11-26 Otri-Universidad De Almeria Multipurpose autonomous vehicle for work in a greenhouse
US7715013B2 (en) 2005-09-16 2010-05-11 The United States Of America As Represented By The Administrator Of The United States Environmental Protection Agency Optical system for plant characterization
US20110101239A1 (en) * 2008-05-08 2011-05-05 Iain Woodhouse Remote sensing system
US8028470B2 (en) 2009-04-21 2011-10-04 Deere & Company Robotic watering unit
US20110261355A1 (en) 2010-04-22 2011-10-27 Thaddaeus Hannel Portable device and method for spectroscopic analysis
US8061080B2 (en) 2006-12-07 2011-11-22 Koninklijke Philips Electronics N.V. Controlling device for a greenhouse
US20120046837A1 (en) 2010-08-20 2012-02-23 Noel Wayne Anderson Automated plant problem resolution
US20120101861A1 (en) 2010-10-25 2012-04-26 Lindores Robert J Wide-area agricultural monitoring and prediction
US20120109387A1 (en) 2009-04-06 2012-05-03 Smartfield, Inc. Remote analysis and correction of crop condition
US20120114187A1 (en) 2009-07-08 2012-05-10 Basf Se System for diagnosis of plant anomalies
US20120150355A1 (en) 2010-08-20 2012-06-14 Deere & Company Networked Chemical Dispersion System
US8249308B2 (en) 2008-09-26 2012-08-21 Robert Lussier Portable intelligent fluorescence and transmittance imaging spectroscopy system
US8437879B2 (en) 2009-04-21 2013-05-07 Deere & Company System and method for providing prescribed resources to plants
US8437498B2 (en) 2010-08-31 2013-05-07 Valmont Industries Inc. Self propelled mechanized irrigation system with a remote visual monitoring system
US8476603B2 (en) 2007-06-01 2013-07-02 Force-A In-situ plant analysis apparatus, method for tracking the state or evolution of a culture and method for managing vegetable processing
US8504234B2 (en) 2010-08-20 2013-08-06 Deere & Company Robotic pesticide application
US20140035752A1 (en) 2012-08-06 2014-02-06 Jerome Dale Johnson Methods, apparatus, and systems for determining in-season crop status in an agricultural crop and alerting users
US20140059722A1 (en) 2012-05-15 2014-02-27 Bioglow Llc Biosensors
US20140064568A1 (en) 2012-09-03 2014-03-06 Electronics And Telecommunications Research Institute Apparatus and method for non-destructively diagnosing crop growth using terahertz waves
US20140222374A1 (en) 2011-09-20 2014-08-07 Monford Ag Systems Limited System and Method for Measuring Parameters Relating to Agriculture
US8836504B2 (en) 2009-02-25 2014-09-16 Eidgenössische Technische Hochschule Zürich System and method for the remote monitoring of potted plants
US20170032258A1 (en) 2015-07-30 2017-02-02 Ecoation Innovative Solutions Inc. Systems and methods for crop health monitoring, assessment and prediction
US20170176595A1 (en) 2012-06-01 2017-06-22 Agerpoint, Inc. Modular systems and methods for determining crop yields with high resolution geo-referenced sensors

Family Cites Families (49)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008083062A1 (en) * 2006-12-29 2008-07-10 Pioneer Hi-Bred International, Inc. Automated location-based information recall
US9538714B2 (en) 2009-04-21 2017-01-10 Deere & Company Managing resource prescriptions of botanical plants
BRPI1012177A2 (en) * 2009-05-14 2016-04-05 Pioneer Hi Bred Int methods and system for estimating a plant trait, drought tolerance prediction methods in a plant, prediction of target analyte content in a plant, prediction of genome introgression content of a backcross experiment.
US10115158B2 (en) 2010-10-25 2018-10-30 Trimble Inc. Generating a crop recommendation
US20120101784A1 (en) 2010-10-25 2012-04-26 Trimble Navigation Limited Wide-area agricultural monitoring and prediction
US20140180549A1 (en) 2011-01-07 2014-06-26 The Arizona Board Of Regents On Behalf Of The University Of Arizona Automated machine for selective in situ manipulation of plants
SG2013065057A (en) 2012-08-27 2014-03-28 Agency Science Tech & Res Quantifying curvature of biological structures from imaging data
WO2014052712A2 (en) 2012-09-28 2014-04-03 Agco Corporation Windrow relative yield determination through stereo imaging
CA2840436C (en) 2013-01-22 2021-02-09 Vale S.A. System for mapping and identification of plants using digital image processing and route generation
US10219449B2 (en) 2013-03-07 2019-03-05 Blue River Technology Inc. System and method for plant dislodgement
WO2015025252A1 (en) 2013-08-21 2015-02-26 Basf Agro B.V., Arnhem (Nl), Zurich Branch Apparatus and method for applying a soil treatment beneath the surface of the ground
US9532411B2 (en) 2014-04-04 2016-12-27 iUNU, LLC Lighting fixture with application controller
US9939132B2 (en) 2014-07-18 2018-04-10 Iunu, Inc. RFI shielding for luminaires using reflection optics
US10627785B2 (en) 2014-07-29 2020-04-21 Iunu, Inc. Plant production feedback control loop optimization
US9576786B2 (en) 2014-09-02 2017-02-21 iUNU, LLC Intelligent radio-controlled plasma light
US10772253B2 (en) 2014-12-10 2020-09-15 The University Of Sydney Automatic target recognition and dispensing system
US9638678B2 (en) 2015-01-30 2017-05-02 AgriSight, Inc. System and method for crop health monitoring
US10021837B2 (en) 2015-01-30 2018-07-17 iUNU, LLC Radio-controlled luminaire with integrated sensors
CN108135122B (en) 2015-06-05 2021-09-03 阿格里斯私人有限公司 Device for controlling weeds
US20170332544A1 (en) 2015-10-05 2017-11-23 iUNU, LLC Data driven indoor farming optimization
US10219448B2 (en) 2015-12-21 2019-03-05 International Business Machines Corporation Autonomous mobile platform and variable rate irrigation method for preventing frost damage
US11166415B2 (en) 2016-07-26 2021-11-09 Sensor Electronic Technology, Inc. Plant growth with radiation-based mildew and/or bacteria control
US11244398B2 (en) 2016-09-21 2022-02-08 Iunu, Inc. Plant provenance and data products from computer object recognition driven tracking
WO2018057799A1 (en) 2016-09-21 2018-03-29 iUNU, LLC Horticultural care tracking, validation and verification
US10791037B2 (en) 2016-09-21 2020-09-29 Iunu, Inc. Reliable transfer of numerous geographically distributed large files to a centralized store
US11538099B2 (en) 2016-09-21 2022-12-27 Iunu, Inc. Online data market for automated plant growth input curve scripts
US10339380B2 (en) 2016-09-21 2019-07-02 Iunu, Inc. Hi-fidelity computer object recognition based horticultural feedback loop
US10635274B2 (en) 2016-09-21 2020-04-28 Iunu, Inc. Horticultural care tracking, validation and verification
KR20200003877A (en) 2017-05-04 2020-01-10 아루까 에이. 아이 파밍 엘티디 Plant processing system and method
US11370108B2 (en) 2017-10-10 2022-06-28 Iunu, Inc. Intelligent track and navigation for sensor carrier
CN108106673A (en) 2017-12-26 2018-06-01 苏州爱达荷农业信息科技有限公司 A kind of seedling growth recording device and its control method
CN108362326B (en) 2018-01-03 2020-12-18 江苏大学 Suspension rail type greenhouse comprehensive information automatic cruise monitoring device
WO2019141465A1 (en) 2018-01-18 2019-07-25 Dupont Nutrition Biosciences Aps Probiotics for cognitive and mental health
CA3089518A1 (en) 2018-01-25 2019-08-01 Eleos Robotics Inc. Autonomous unmanned ground vehicle for pest control
US11062516B2 (en) 2018-02-07 2021-07-13 Iunu, Inc. Augmented reality based horticultural care tracking
US11003456B2 (en) 2018-03-30 2021-05-11 Iunu, Inc. Pipelined processing of plant images for monitoring horticultural grow operations
US10929664B2 (en) 2018-03-30 2021-02-23 Iunu, Inc. Visual observer of unmanned aerial vehicle for monitoring horticultural grow operations
US10891482B2 (en) 2018-07-10 2021-01-12 Adroit Robotics Systems, devices, and methods for in-field diagnosis of growth stage and crop yield estimation in a plant area
CN209181816U (en) 2018-07-27 2019-07-30 成都感智索蓝科技有限公司 The device of real-time herborization growing environment data
US20220130036A1 (en) 2019-01-21 2022-04-28 Sony Group Corporation Information processing apparatus, electronic device and method
US10949974B2 (en) 2019-02-28 2021-03-16 Iunu, Inc. Automated plant disease detection
US11232526B2 (en) 2019-05-31 2022-01-25 Iunu, Inc. Centralized governance regulatory compliance (C-GRC) system
US20210048822A1 (en) 2019-08-12 2021-02-18 Ecoation Innovative Solutions Inc. Mobile platform for crop monitoring and treatment
CN111089829A (en) 2020-02-10 2020-05-01 山东农业大学 Multi-sensor fusion-based device and method for measuring porosity of canopy layering
IN202041012385A (en) 2020-03-22 2020-05-15
US11694212B2 (en) 2020-03-25 2023-07-04 Iunu, Inc. Decentralized governance regulatory compliance (D-GRC) controller
US11656624B2 (en) 2020-03-25 2023-05-23 Iunu, Inc. Horticulture aided by autonomous systems
US11720980B2 (en) 2020-03-25 2023-08-08 Iunu, Inc. Crowdsourced informatics for horticultural workflow and exchange
US11730089B2 (en) 2020-03-25 2023-08-22 Iunu, Inc. Horticulture aided by autonomous systems

Patent Citations (42)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4876647A (en) 1985-05-17 1989-10-24 The Standard Oil Company Apparatus for determining water stress in crops
US4755942A (en) 1985-05-17 1988-07-05 The Standard Oil Company System for indicating water stress in crops which inhibits data collection if solar insolation exceeds a range from an initial measured value
US5130545A (en) 1991-04-05 1992-07-14 Lussier Robert R Video imaging plant management system
US5839106A (en) 1996-12-17 1998-11-17 Apple Computer, Inc. Large-vocabulary speech recognition using an integrated syntactic and semantic statistical language model
US6573512B1 (en) 1997-12-22 2003-06-03 Communaute Europeene Analysing device non-destructive of plants and vehicle comprising such device on board
US6397162B1 (en) 1999-03-29 2002-05-28 Phytech Ltd. System, device and method for estimating evapo-transpiration in plants
US20030229497A1 (en) 2000-04-21 2003-12-11 Lessac Technology Inc. Speech recognition method
US6657117B2 (en) 2000-07-14 2003-12-02 Microsoft Corporation System and methods for providing automatic classification of media entities according to tempo properties
US6701665B1 (en) 2000-10-23 2004-03-09 Phytech Ltd. Remote phytomonitoring
US20020170229A1 (en) 2001-04-13 2002-11-21 Phytech Ltd. System and method for phytomonitoring
US20040241635A1 (en) 2001-04-18 2004-12-02 Buckley Wayne T. Assay for seed vigour
US20020167587A1 (en) 2001-05-10 2002-11-14 E.C.R Corporation Monitoring system
US7487925B2 (en) 2001-05-31 2009-02-10 Skinner Paul W Conduit including sensors for monitoring and maintenance of vegetation
DE10148747A1 (en) 2001-09-26 2003-04-17 Norsk Hydro As Non-contact monitoring crop and other vegetation condition, for determining fertilizer requirements, uses flash lights for the reflected light to be received by a single photo-diode giving digitized signals for processing
US7112806B2 (en) 2001-09-27 2006-09-26 Robert Lussier Bio-imaging and information system for scanning, detecting, diagnosing and optimizing plant health
US7412330B2 (en) 2005-08-01 2008-08-12 Pioneer Hi-Bred International, Inc. Sensor system, method, and computer program product for plant phenotype measurement in agricultural environments
US7715013B2 (en) 2005-09-16 2010-05-11 The United States Of America As Represented By The Administrator Of The United States Environmental Protection Agency Optical system for plant characterization
US7987632B2 (en) 2005-12-21 2011-08-02 The Institute For Technology Development Expert system for controlling plant growth in a contained environment
US7617057B2 (en) 2005-12-21 2009-11-10 Inst Technology Development Expert system for controlling plant growth in a contained environment
US8061080B2 (en) 2006-12-07 2011-11-22 Koninklijke Philips Electronics N.V. Controlling device for a greenhouse
US8476603B2 (en) 2007-06-01 2013-07-02 Force-A In-situ plant analysis apparatus, method for tracking the state or evolution of a culture and method for managing vegetable processing
US20110101239A1 (en) * 2008-05-08 2011-05-05 Iain Woodhouse Remote sensing system
WO2009141465A1 (en) 2008-05-19 2009-11-26 Otri-Universidad De Almeria Multipurpose autonomous vehicle for work in a greenhouse
US8249308B2 (en) 2008-09-26 2012-08-21 Robert Lussier Portable intelligent fluorescence and transmittance imaging spectroscopy system
US8836504B2 (en) 2009-02-25 2014-09-16 Eidgenössische Technische Hochschule Zürich System and method for the remote monitoring of potted plants
US20120109387A1 (en) 2009-04-06 2012-05-03 Smartfield, Inc. Remote analysis and correction of crop condition
US8437879B2 (en) 2009-04-21 2013-05-07 Deere & Company System and method for providing prescribed resources to plants
US8028470B2 (en) 2009-04-21 2011-10-04 Deere & Company Robotic watering unit
US20120114187A1 (en) 2009-07-08 2012-05-10 Basf Se System for diagnosis of plant anomalies
US20110261355A1 (en) 2010-04-22 2011-10-27 Thaddaeus Hannel Portable device and method for spectroscopic analysis
US20120046837A1 (en) 2010-08-20 2012-02-23 Noel Wayne Anderson Automated plant problem resolution
US20120150355A1 (en) 2010-08-20 2012-06-14 Deere & Company Networked Chemical Dispersion System
US8504234B2 (en) 2010-08-20 2013-08-06 Deere & Company Robotic pesticide application
US8437498B2 (en) 2010-08-31 2013-05-07 Valmont Industries Inc. Self propelled mechanized irrigation system with a remote visual monitoring system
US20120101861A1 (en) 2010-10-25 2012-04-26 Lindores Robert J Wide-area agricultural monitoring and prediction
US20140222374A1 (en) 2011-09-20 2014-08-07 Monford Ag Systems Limited System and Method for Measuring Parameters Relating to Agriculture
US20140059722A1 (en) 2012-05-15 2014-02-27 Bioglow Llc Biosensors
US20170176595A1 (en) 2012-06-01 2017-06-22 Agerpoint, Inc. Modular systems and methods for determining crop yields with high resolution geo-referenced sensors
US20140035752A1 (en) 2012-08-06 2014-02-06 Jerome Dale Johnson Methods, apparatus, and systems for determining in-season crop status in an agricultural crop and alerting users
US20140064568A1 (en) 2012-09-03 2014-03-06 Electronics And Telecommunications Research Institute Apparatus and method for non-destructively diagnosing crop growth using terahertz waves
US20170032258A1 (en) 2015-07-30 2017-02-02 Ecoation Innovative Solutions Inc. Systems and methods for crop health monitoring, assessment and prediction
US10241097B2 (en) * 2015-07-30 2019-03-26 Ecoation Innovative Solutions Inc. Multi-sensor platform for crop health monitoring

Non-Patent Citations (9)

* Cited by examiner, † Cited by third party
Title
"The Autonomous . . . Greenhouse Operation", IEEE Robotics and Automation Magazine, Dec. 1996.
Bart M. Nicolai et al., "Nondestructive measurement of fruit . . . NIR spectroscopy: A review", Science Direct, Postharvest Biology and Technology 46 (2007), 99-118.
Jansen et al., "Induced plant . . . status in greenhouses", Plant signaling & Behavior, 824-829, 2009.
M. Ruiz-Altisent et al., "Sensors for product characterization and quality of specialty crops-A review", Computers and Electronics in Agriculture 74 (2010), 176-194.
M. Ruiz-Altisent et al., "Sensors for product characterization and quality of specialty crops—A review", Computers and Electronics in Agriculture 74 (2010), 176-194.
Sankaran et al., "A review of advanced techniques for detecting plant diseases", Computer and Electronics in Agriculture, vol. 72, Jun. 2010, pp. 1-13.
Story, D. and Kacira, M. (2014). Automated Machine Vision Guided Plant Monitoring System for Greenhouse Crop Diagnostics. Acta Hortic. 1037, 635-641-abstract only available via this link: https://www.actahort.org/books/1037/1037_81.htm.
Story, D. and Kacira, M. (2014). Automated Machine Vision Guided Plant Monitoring System for Greenhouse Crop Diagnostics. Acta Hortic. 1037, 635-641—abstract only available via this link: https://www.actahort.org/books/1037/1037_81.htm.
Ton et al., "Phytomonitoring: A Bridge . . . For Greenhouse Control", Phytech Ltd., 2003.

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11965870B2 (en) 2015-07-30 2024-04-23 Ecoation Innovative Solutions Inc. Multi-sensor platform for crop health monitoring
US11874265B2 (en) 2015-07-30 2024-01-16 Ecoation Innovative Solutions Inc. Multi-sensor platform for crop health monitoring
US12111301B2 (en) 2015-07-30 2024-10-08 Ecoation Innovative Solutions Inc. Multi-sensor platform for crop health monitoring
US12038426B2 (en) 2015-07-30 2024-07-16 Ecoation Innovative Solutions Inc. Multi-sensor platform for crop health monitoring
US11499955B2 (en) 2015-07-30 2022-11-15 Ecoation Innovative Solutions Inc. Crop health monitoring using predictive modeling
US11867680B2 (en) 2015-07-30 2024-01-09 Ecoation Innovative Solutions Inc. Multi-sensor platform for crop health monitoring
US20210315150A1 (en) * 2018-08-16 2021-10-14 Mahindra & Mahindra Limited Method and systems for generating prescription plans for a region under cultivation
US20220377991A1 (en) * 2019-11-01 2022-12-01 Omron Corporation Plant biosensor
US12131393B2 (en) 2020-10-02 2024-10-29 Ecoation Innovative Solutions Inc. Platform for real-time identification and resolution of spatial production anomalies in agriculture
US11666004B2 (en) 2020-10-02 2023-06-06 Ecoation Innovative Solutions Inc. System and method for testing plant genotype and phenotype expressions under varying growing and environmental conditions
US20220104437A1 (en) * 2020-10-02 2022-04-07 Ecoation Innovative Solutions Inc. Reduction of time of day variations in plant-related data measurements
US11555690B2 (en) 2020-11-13 2023-01-17 Ecoation Innovative Solutions Inc. Generation of stereo-spatio-temporal crop condition measurements based on human observations and height measurements
US11925151B2 (en) 2020-11-13 2024-03-12 Ecoation Innovative Solutions Inc. Stereo-spatial-temporal crop condition measurements for plant growth and health optimization
US12067711B2 (en) 2020-11-13 2024-08-20 Ecoation Innovative Solutions Inc. Data processing platform for analyzing stereo-spatio-temporal crop condition measurements to support plant growth and health optimization
US11915421B2 (en) * 2021-09-07 2024-02-27 Mineral Earth Sciences Llc Auditing task performance
US20230074663A1 (en) * 2021-09-07 2023-03-09 X Development Llc Auditing task performance
WO2024025527A1 (en) * 2022-07-27 2024-02-01 Neatleaf, Inc. Aerial sensor and manipulation platform for farming and method of using same

Also Published As

Publication number Publication date
US11867680B2 (en) 2024-01-09
US12111301B2 (en) 2024-10-08
US20230084925A1 (en) 2023-03-16
CA2937571A1 (en) 2017-01-30
US10241097B2 (en) 2019-03-26
US20230228725A1 (en) 2023-07-20
US11287411B2 (en) 2022-03-29
US20210072210A1 (en) 2021-03-11
US20170032258A1 (en) 2017-02-02
US11499955B2 (en) 2022-11-15
US11874265B2 (en) 2024-01-16
US11965870B2 (en) 2024-04-23
US12038426B2 (en) 2024-07-16
US20220107298A1 (en) 2022-04-07
US20190170718A1 (en) 2019-06-06
US20230070689A1 (en) 2023-03-09
US20170030877A1 (en) 2017-02-02
CA2937574A1 (en) 2017-01-30
US20230081523A1 (en) 2023-03-16
US20230086875A1 (en) 2023-03-23

Similar Documents

Publication Publication Date Title
US11874265B2 (en) Multi-sensor platform for crop health monitoring
US11744189B2 (en) Plant treatment based on morphological and physiological measurements
US20140168412A1 (en) Methods and systems for automated micro farming
US20180373937A1 (en) Methods and systems for automated micro farming
CA3139447A1 (en) Automated plant monitoring systems and methods
US11666004B2 (en) System and method for testing plant genotype and phenotype expressions under varying growing and environmental conditions
US12131393B2 (en) Platform for real-time identification and resolution of spatial production anomalies in agriculture
US20220104437A1 (en) Reduction of time of day variations in plant-related data measurements
Mustafa et al. Precision Agriculture and Unmanned Aerial Vehicles (UAVs)

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

Year of fee payment: 4