BR112020017937B1 - SEED IMAGING SYSTEM AND METHOD - Google Patents

Abstract

A seed imaging system for imaging seeds includes a seed transfer station configured to move seeds through the system. An imaging assembly includes a first camera mounted relative to the seed transfer station and configured to acquire images of the seeds as the seeds move through the system. A second camera is mounted relative to the seed transfer station and is configured to acquire images of the seeds as the seeds move through the system. The second camera has an imaging modality different from an imaging modality of the first camera. The first and second cameras may be disposed above and below the seed transfer stations, such as a transparent belt.

Description

[001] The present disclosure relates generally to a system and method for processing seeds, and more specifically, a seed imaging system and method for imaging and storing seeds.

FUNDAMENTALS

[002] In the agricultural industry, and more specifically in the seed breeding industry, it is important for scientists to be able to analyze seeds with high throughput. By this we mean that seed analysis preferably occurs not only quickly, but also reliably and with high total volume. Historically, seeds have been categorized by size using mechanical equipment containing screens with holes corresponding to predetermined sizes. Seed categorization has also been conducted using image analysis of seeds to detect certain appearance characteristics of the seeds. However, previous seed analysis systems are limited in their ability to detect seed size, shape, and appearance. As a result, previous image analysis systems have limited capabilities to characterize seed shape and defects. Additionally, previous image analysis systems do not allow for automated collection of statistically significant amounts of data for the development of robust data models to determine correlations between batches of seeds using seed quality metrics.

SUMMARY

[003] In one aspect, a seed imaging system for imaging seeds generally comprises a seed transfer station configured to move seeds through the system. An imaging assembly comprises a first camera mounted relative to the seed transfer station and configured to acquire images of the seeds as the seeds move through the system. A second camera is mounted relative to the seed transfer station and is configured to acquire images of the seeds as the seeds move through the system. The second camera has an imaging modality different from an imaging modality of the first camera.

[004] In another aspect, a method for imaging seeds generally comprises moving seeds through the system using a seed transfer station; acquiring, using a first camera mounted relative to the seed transfer station, images of the seeds as the seeds move through the system via the seed transfer station; and acquiring, using a second camera mounted relative to the seed transfer station, images of the seeds as the seeds move through the system via the seed transfer station, an imaging modality of the second camera being different from an imaging modality of the first camera.

[005] In yet another aspect, a seed imaging system for imaging seeds generally comprises a seed transfer station configured to move seeds through the system. An imaging assembly comprises a first camera mounted above the seed transfer station and configured to acquire images of the seeds as the seeds move through the system. A second camera is mounted below the seed transfer station and configured to acquire images of the seeds as the seeds move through the system.

BRIEF DESCRIPTION OF THE DRAWINGS

[006] Fig. 1 is a block diagram of an automated seed imaging system;

[007] Fig. 2 is a perspective view of the seed imaging system with one imaging and analysis assembly of the system removed;

[008] Fig. 2A is another perspective of the seed imaging system showing the imaging and analysis assembly;

[009] Fig. 2B is an enlarged fragmentary perspective view of Fig. 2;

[0010] Fig. 3 is a front view of the seed imaging system with the imaging and analysis assembly removed;

[0011] Fig. 4 is an enlarged fragmentary view of Fig. 3 showing schematically the imaging and analysis assembly;

[0012] Fig. 5 is a schematic illustration of the seed imaging system;

[0013] Fig. 6 is another schematic illustration of the seed imaging system;

[0014] Fig. 7 is an enlarged fragmentary view of Fig. 2;

[0015] Fig. 8 is a perspective view of a seed imaging system of another embodiment;

[0016] Fig. 9 is a front view of a seed imaging system of another embodiment;

[0017] Fig. 10 is another embodiment of a seed imaging system; and

[0018] Fig. 11 refers to exemplary data from a seed imaging system.

[0019] Corresponding reference characters indicate corresponding parts throughout the drawings.

DETAILED DESCRIPTION

[0020] Referring to Figs. 1-2A and 3-6, a seed imaging system is generally indicated at 10. The system is configured to receive, analyze, and store a plurality of seeds for further processing, evaluation, and/or analysis. The system 10 comprises a loading and transfer assembly 12 configured to receive and distribute the seeds through the system, an imaging and analysis assembly 14 for collecting image data of the seeds as they are distributed through the system by the loading and transfer assembly, and a weighing assembly 15 for weighing the seeds and a storage assembly 16 configured to store the seeds for further processing. A controller 18 (e.g., a processor and suitable memory) is programmed to operate system 10. Imaging and analysis assembly 14 acquires image data and incorporates optimized image analysis algorithms to provide rapid and highly accurate seed characteristics, including one or more of the color, size, shape, texture, internal composition, mass, volume, moisture content, and chemical composition data of the seeds that provide a complete picture of the appearance and condition of the seeds. Being able to capture the complete internal and external image of the seed allows system 10 to reliably detect defects in the seeds.

[0021] The imaging and analysis assembly 14 combines multiple imaging modalities to measure the color, size, shape, texture, and internal characteristics of seeds, for example, which provides a more accurate indication of their appearance and condition. The storage assembly 16 is configured to store the seeds in microplates for later processing, evaluation, and/or analysis. In one or more other examples, the storage assembly may be configured to sort individual seeds into two or more bulk fractions, such as at the end of the loading and transfer assembly 12. For example, pulses of air may direct the seeds, depending on the analyses performed on the seeds, into one or more bulk containers. Additionally, the system 10 is designed for high-content data acquisition. Image data (e.g., color, internal characteristics, shape, etc.) will be mined from the high-content data to extract the best predictors of quality. A high-throughput belt system may be incorporated later to collect these characteristics using faster imaging devices.

[0022] Referring to Figs. 2-6, the loading and transfer assembly 12 comprises a hopper (broadly, a seed loading station) 20 including an inlet 22 for receiving seeds into the hopper and an outlet 24 for dispensing seeds from the hopper, a vibratory feed 25 at the outlet for individualizing seeds as they are dispensed from the outlet, and a conveyor 26 (broadly, a seed transfer station) at an outlet of the vibratory feed. The vibratory feed 25 comprises a pair of vibratory feeders 27 and a pair of vibratory channels 29 associated with a respective vibratory feeder. The vibratory feeders 27 use vibratory energy to transport seeds along the vibratory channels 29 and arrange the seeds in a single row. The vibratory energy also separates the seeds from each other within the row so that each seed can be imaged individually by the imaging and analysis assembly 14 as the seed is transported to the conveyor 26. Vibratory feed rates can be controlled by the controller 18. Although a vibratory feed 25 is shown, it is anticipated that other methods for individualizing seeds can be used. In one embodiment, an individualization wheel (not shown) can be used. Additionally, a tracking sensor 33 is located at an outlet of the vibratory feed 25. The tracking sensor 33 matches each seed as it exits the vibratory feed 25. The tracking sensor 33 allows the system 10 to track each seed prior to being imaged in order to momentarily stop the vibratory load, if necessary, to allow for ample spacing between the seeds prior to imaging. Additionally, the tracking sensor 33 allows the imaging data collected by the imaging and analysis assembly 14 to be properly associated with the correct seed.

[0023] In the illustrated embodiment, conveyor 26 comprises a belt 28 defining a flat horizontal conveyor transport surface. Conveyor 26 provides a flat surface for seeds to rest on as they are distributed through system 10. As a result, system 10 is able to better control the travel of each seed through the system and therefore better track the position of the seeds as they move on conveyor 26, because the seeds will remain in a substantially fixed orientation and position on the conveyor. In one embodiment, a high precision encoder (not shown) is incorporated into system 10 to track the position of the seeds on conveyor 26. The encoder may work in combination with, or include, tracking sensor 33. The encoder may also act as a master timing device to trigger different imaging modalities to acquire their images. As will be explained in more detail below, the flat surface allows for more accurate measurements to be acquired by the imaging and analysis assembly 14. Furthermore, in one example, being able to accurately track the position and location of the seeds as they travel on the conveyor 26 allows the system 10 to locate each seed for placement in a microplate of the storage assembly 16 for later evaluation. The seeds may be loaded onto the conveyor from multiple supply channels, whereby the seeds are individually spaced in parallel along the length of the belt, for example. In another example, the seeds may be loaded into trays (e.g., each cell received in a cell of a "scanning tray" and the tray with loaded seeds may be transported along the conveyor. Thus, each seed within a corresponding cell or position on the tray is imaged and analyzed, such that the acquired data is associated with the location of the seed on the tray (i.e., the seeds are tracked by their corresponding cell or position on the tray).

[0024] Conveyor 26 may be a low speed conveyor operating at speeds of about 5 to 10 seeds/minute. In one or more other embodiments, the conveyor may operate at speeds of about 30 to 100 seeds/minute or other rates. The speed of belt 28 may be controlled by controller 18. In one embodiment, conveyor 26 is transparent. The transparent nature of conveyor 26 allows imaging from beneath the conveyor to be performed, as will be explained in more detail below. However, the conveyor may be translucent or semi-transparent without departing from the scope of the disclosure. In one embodiment, belt 28 is formed of Mylar. Other materials, including optically and X-ray transmissive materials are contemplated without departing from the scope of the disclosure. A coating may also be applied to the belt 28 of the conveyor 26. The coating may be configured to repel dust and/or have scratch-resistant properties, which may help to keep the belt 28 clean and free of marks that may impair the ability of the imaging and analysis assembly 14 to acquire clear images. Additionally or alternatively, a plurality of ionizers (not shown) may be provided to dissipate static charges in the system 10 to reduce adhesion of fine particulate matter to the conveyor 26. In one or more other embodiments, seeds may be loaded into “scanning trays,” which are placed on the conveyor, to image and track individual seeds across multiple modalities. The collected data may be used for data fusion and multimodal classifier training.

[0025] Referring to Figs. 2-6, the imaging and analysis assembly 14 comprises a first hyperspectral reflectance camera 30 mounted above the carrier surface to collect and process image data across the electromagnetic spectrum. In one embodiment, the first hyperspectral reflectance camera 30 obtains image data across the visible light spectrum. The first hyperspectral reflectance camera 30 may have a spectral range of about 400 nm to about 900 nm. However, a different spectral range is envisioned without departing from the scope of the disclosure. A second hyperspectral reflectance camera 32 is mounted above the carrier surface to collect and process image data across the electromagnetic spectrum. In one embodiment, the second hyperspectral reflectance camera 32 obtains image data across the near-infrared spectrum. The second hyperspectral reflectance camera 32 may have a spectral range of about 1,000 nm to about 1,700 nm. However, a different spectral range is contemplated without departing from the scope of the disclosure. Hyperspectral cameras look at objects using a broad range of the electromagnetic spectrum. This is in contrast to the human eye, which sees only visible light in the red, green, and blue spectrum. However, certain objects may leave unique fingerprints in the electromagnetic spectrum. These fingerprints may help identify materials that make up a scanned object. In the present case, seeds imaged by the hyperspectral reflectance cameras 30, 32 may leave fingerprints that may indicate certain conditions of the seed. The imaging and analysis assembly 14 also includes a processor and memory for processing (i.e., analyzing) the image data, although in other embodiments, the controller 18 may be used for such processing. Hyperspectral cameras can also be added below the belt to image the bottom of the seed for interrogation of more than 90% of the total seed surface area. Belt materials can be chosen to be broadly transparent across the spectral range of interest. In all cases, samples are illuminated with line source light arrays that have spectral outputs covering the range of the hyperspectral cameras. For example, quartz tungsten halogen lamps and the like can be used.

[0026] A first 2D line scan red-green-blue (RGB) camera (broadly, a first 2D camera) 34 is mounted above the conveyor surface to acquire image data of the seeds to measure the color, size, shape, and appearance of the seeds in two dimensions, and a second 2D line scan RGB camera (broadly, a second 2D camera) 36 is mounted below the conveyor surface to acquire image data of the seeds to measure the color, size, shape, and appearance of the seeds in two dimensions. In one embodiment, the upper 2D camera 34 is mounted above the conveyor 26 in a substantially vertical orientation such that a focal axis of the camera extends perpendicular to a horizontal plane of the conveyor, and the lower 2D camera 36 is mounted below the conveyor in a substantially vertical orientation such that a focal axis of the camera extends perpendicular to a horizontal plane of the conveyor. The length and width dimensions of the seeds can be calculated using an image processing routine executed by the controller 18. With the length and width dimensions of the seeds, the areas of each seed can be calculated. Each 2D camera 34, 36 is configured to image a 150 mm strip on the belt 28 of the conveyor 26 with a spatial resolution of about 0.14 mm. An example of a suitable 2D camera is the JAI model CV-L107CL.

[0027] Additionally, each 2D camera 34, 36 has an associated light assembly 37 for illuminating the fields of view of the cameras 34, 36 to aid in producing bright, clear images. Each light assembly 37 comprises a pair of top white lights 37A and a rear blue light 37B. The light assemblies 37 provide illumination that complements the light carrier 28 so that the images from the cameras 34, 36 are clear and bright. The field of view for the top 2D camera 34 is illuminated by the white lights 37A mounted above the carrier surface and by the blue light 37B mounted below the carrier surface. Conversely, the field of view for the bottom 2D camera 36 is illuminated by the white lights 37A mounted below the carrier surface and by the blue light 37B mounted above the carrier surface. Using only the upper and lower 2D cameras, the imaging assembly 14 is capable of imaging over 90% of the surface of each seed. In a similar embodiment, additional upper and/or lower cameras may be added in orientations off-perpendicular to the conveyor 26. These cameras may be used in conjunction with the upper 2D camera 34 and/or the lower 2D camera 36 for detailed inspection of defects across a larger portion of the seed surface area.

[0028] Although the illustrated embodiment shows the hyperspectral cameras 30, 32 mounted upstream of the 2D cameras 34, 36, it is contemplated that at least one of the 2D cameras may be mounted upstream of the hyperspectral cameras such that the 2D camera is the first imaging device passed by the seeds (see Fig. 9). In this embodiment (Fig. 9), the top 2D camera 134 is mounted upstream of the hyperspectral cameras 130 and 132 and may be used to locate a seed to predict when the seed will arrive at the hyperspectral cameras to predict when to trigger the hyperspectral cameras so that only data from the seed region is acquired and/or saved. This has the benefit of dramatically reducing file size and may avoid computer memory issues. In this embodiment, the lower 2D camera 136 is disposed between the two hyperspectral cameras 130, 132 along the conveyor path. Furthermore, in addition to the upper 3D camera 140 (which may be the same or similar to the 3D camera 40), a second 3D camera 142 (which may be the same or similar to the upper 3D camera 140) is mounted below the conveyor belt 128. Although not shown in Fig. 9, an x-ray camera, such as the x-ray camera 38 in Fig. 2A and described below, may also be incorporated into the imaging assembly.

[0029] The imaging and analysis assembly 14 further comprises an X-ray camera 38 mounted above the conveyor surface and an X-ray source below the belt for producing radiation detected by the X-ray camera to acquire X-ray images of the seeds. The X-ray camera 38 is housed within an X-ray housing 39 that also allows passage of the conveyor belt 28 through the housing. In particular, the housing 39 includes a passageway (Fig. 3) through which the belt 28 passes. An opening in the passageway provides a window for the X-ray camera 38 to view the belt 28 so that seeds traveling on the belt can be imaged by the X-ray camera 38. In one embodiment, the X-ray camera 38 comprises a low-energy X-ray TDI (time delay and integration) camera. TDI technology is based on the concept of accumulating multiple exposures of a moving object, effectively increasing the integration time available to collect incident light. Indeed, to accommodate the seeds on the moving conveyor 26, the preferred imaging technique for all imaging embodiments of the system 10 is a pressure broom line scanning method in which the moving seeds are imaged one line at a time. The imaged lines may be accumulated at a frame rate referenced to the speed of the belt 28. In the embodiment shown in FIG. 10, the inner and outer housings for the X-ray source are shown. The inner housing 139A inhibits most X-ray emissions and shields the other imaging equipment, while allowing the seeds and belt to pass through. The outer housing 139B prevents any X-rays exiting the inner housing 139A (primarily the openings where the seeds enter and exit) from exiting the device (e.g., to a laboratory).

[0030] A 3D line laser profiler (broadly, a 3D camera) 40 is also mounted above the conveyor surface to acquire 3D image data of the seeds to measure the size and shape of the seeds in three dimensions. In one embodiment, the 3D camera 40 is mounted above the conveyor in a substantially vertical orientation, such that a laser from the camera projects substantially perpendicular to a horizontal plane of the conveyor 26, and a focal axis of the camera extends at a slightly inclined angle to the vertical, such that a focal axis of the 3D camera extends at an angle non-orthogonal to the plane of the conveyor. The 3D camera 40 projects a line laser to create a line profile of the seed surface. The 3D camera 40 measures the line profile to determine displacement which is represented by an image of the seed showing varying pixel intensities corresponding to differences in height. A thickness dimension is obtained from the pixel intensity of the 3D image produced by the 3D camera 40. For example, a maximum pixel intensity may be interpreted as a marker of seed thickness. Thus, as seeds pass through the focal window of the 3D camera 40, a thickness of each seed is recorded as the maximum pixel intensity detected by the 3D camera for each seed. In order to acquire an accurate thickness measurement, it may be necessary to calibrate the distance measurement of the 3D camera 40 based on objects of known height. Using the length and width dimensions acquired from the 2D cameras 34, 36 and the thickness dimensions acquired from the 3D camera 40, the system 10 may obtain volume estimates for each seed. In another embodiment, more sophisticated image processing may be used to estimate volume from a detailed contour map of the top half of each seed. Furthermore, a second, bottom-facing 3D camera (not shown) could generate a detailed contour map of the bottom half of the seed. The contour maps from the top and bottom 3D cameras can be combined to provide a more complete estimate of the overall seed volume. In any case, for a known or estimated weight (as measured by seed weighing mechanism 15) of the seed, the volume data can be used to estimate seed density. An example of a suitable 3D camera is the Cognex DS1101R model.

[0031] Additional imaging devices may also be mounted on system 10 to acquire additional image data. For example, additional hyperspectral cameras, including optical fluorescence, optical polarization imaging, a 1D NMR spectroscopy device, and/or a microwave measurement system may be mounted on system 10 to provide additional data for the seeds. In one example, as shown in Fig. 10 , one or more of the following embodiments are included: Upper RGB 100; Lower RGB 102; Upper Visible Hyperspectral 104; Lower Visible Hyperspectral 106; Upper NIR Hyperspectral 108; Upper Laser Profilometer 110; Lower Laser Profilometer 112; X-Ray Absorption Imaging 114; X-Ray Fluorescence Spectroscopy 116; Laser Fluorescence Spectroscopy 118; Lower NIR hyperspectral 120; NIR backlight using polarized light 122; and Mass measurement 124 (seed weighing).

[0032] Generally, combining data from two or more imaging modalities can provide improved prediction compared to a single method. For example, as shown in FIG. 10 , identification of defective seeds can be improved by supplementing external appearance images from RGB cameras 34, 36 with images from X-ray camera 38 for inspection of naturally occurring abnormalities of internal seed structure or damage resulting from seed processing steps. Prediction can be further enhanced by adding data from 3D camera 40 for detailed characterization of single seed shape, surface area, volume, and density, with the latter derivative combining volume with the seed mass measured at weighing assembly 15. Optical hyperspectral VIS-NIR reflectance 30, 32 and fluorescence can also be added to provide additional information about near-surface contamination, disease, and composition, which can affect seed viability. 1D-NMR and X-ray Fluorescence (XRF) can provide additional information about the bulk composition of the seed, including internal oil, water, and other constituents. Collectively, these techniques form a detailed set of external appearance, internal structure, and bulk features at a single seed level for a more complete characterization of defective seed relative to viable seed. The selection of techniques and predictive features of each modality may be counterintuitive and revealed only with machine learning and other combinatorial data modeling methods.

[0033] Preservation of the seed's orientation in each upper or lower imaging modality as it travels along belt 28 allows alignment of images from different techniques using coincidence methods in post-acquisition software processing. The combination of all imaging techniques applied to spatially localized regions, possibly as small as a single pixel, can produce fingerprints of defects and physiological structures that would be difficult to classify using a simple method. For example, the combination of spectral signatures from hyperspectral cameras 30, 32 with local height data from 3D camera 40 and attenuation changes in images from X-ray camera 38 may indicate an abnormality at or near the surface of the seed, which may be difficult to identify by current manual inspection methods or classify using features derived from analysis of the entire seed area. A similar approach can be employed to identify small physiological structures in the seed, such as the embryo region.

[0034] Colocalization of defects with physiological seed structures can improve quality prediction, as the effect of the defect may depend on the location in the seed. For example, disease or damage in the embryo region of the seed may have a more pronounced impact on seed vitality compared to defects in other parts of the seed. In this example, after using a combination of imaging techniques to accurately identify the embryo region, a similar procedure can be used to identify localized abnormalities, possibly with a different combination of techniques. Single seed quality score data can then be used as a response variable to examine the effect of defect location relative to the embryo or other critical seed structures. Additionally or alternatively, the score data can be used to define a localized combinatorial signature characteristic of an embryo or other region that predicts poor seed quality. These approaches would not be possible if seeds were transported to different instruments for imaging data acquisition, since seed orientation could not be reliably maintained for each technique.

[0035] Using thresholding methods based on real-time image processing, unique seeds can be discriminated from the belt background and isolated in the field of view for each imaging modality. This ensures that only seed data is retained, reducing the size of image files. Furthermore, because the vibratory feed 25 produces a defined spacing between seeds and the belt speed can be adjusted by the controller 18, images produced from each imaging modality can be associated with a given seed. Additionally or alternatively, seed detection from one imaging technique can be used to anticipate seed arrival in another technique and trigger camera acquisition accordingly. Image files from each imaging modality can be saved independently or combined into a single image file with multiple data layers consisting of the separate image data from all modalities. In either case, the identity of each seed is maintained across all modalities, which prevents labeling errors. Furthermore, since the orientation of the seeds is the same for all imaging modes, correlations can be made between the different modes. For example, defects visibly apparent on the exterior of the seed and imaged by optical cameras can correlate with internal structures observed in the X-ray images. By maintaining seed identity in the single seed identity storage mechanism 16 for post-imaging quality assays (e.g., RET (radicle emergence test), germination, and vigor tests), single seed imaging data can be mined to extract relevant spatial and/or structural features from one or more imaging modalities that provide intrinsic seed quality data. Additionally, the source of seed damage can be determined by analyzing seeds at different stages during seed processing (e.g., harvesting, transporting, processing, or grading). To this end, process-induced mechanical damage and the effects of formulation and application rate of seed treatments can be determined.

[0036] The imaging and analysis assembly 14 is configured to determine circularity, solidity, and smoothness from the produced images. It will be understood by those skilled in the art that the system 10 may include image analysis software to process the images to obtain color, mean and variance in spectrum, size, shape, texture, and internal composition information for the seeds for any and all of the individual modalities or specific wavelengths in the hyperspectral data. For example, the software may incorporate machine learning analytics that facilitate the production of detailed image data. Typical modeling methods include Partial Least Squares discriminate analysis (PLS-DA), neural networks, Support Vector Machines (SVM), logistic regression, and other methods. Because the imaging and analysis assembly 14 acquires images using multiple imaging modalities, including hyperspectral imaging, 2D imaging, 3D imaging, and X-ray imaging, and because images are obtained from the top and bottom of the seeds, the assembly can acquire a complete picture of the condition of the seeds in three dimensions. This figurative image includes data involving the length, width, thickness (or roundness), solidity, smoothness dimensions, and internal composition of the seeds. The various imaging modalities produce image data that is analyzed by the controller 18. Data extracted from the imaging modalities include mean and variance of optical spectra from the image data produced by the hyperspectral cameras 30, 32 and attenuation characteristics in the image data obtained by the X-ray camera 38. These characteristics can include internal structural features and external cracks. In addition, general size, shape, and color data from the 2D cameras 34, 36 and the 3D camera 40 are extracted and analyzed by the controller 18.

[0037] Based on measurement data from cameras 30-40, controller 18 may identify and categorize each seed according to its appearance. For example, a quality score may be assigned to each seed based on data from imaging and analysis set 14. The quality score may be used as the response variable for a prediction model used to classify other seeds processed in system 10. Furthermore, being able to acquire image data from multiple imaging modalities allows the system to tailor the image analysis process for a particular use. For example, image data from each imaging modality may be compared with each other to determine which imaging modality provides the most reliable indication of the condition of the seed. Furthermore, correlations between the different imaging modalities may be formed. Thus, characteristics of a seed determined by one imaging mode, embodied in the image data from the imaging mode, may be compared with the image data from another mode to find correlations in the two data sets. This may allow one imaging mode to function as a check on the image data acquired by another imaging mode. Furthermore, one imaging mode may be used in place of another imaging mode if their image data are found to correlate. This could serve as a cost savings in the case where image data from the 2D cameras 34, 36 are found to correlate with the image data from the significantly more expensive X-ray camera 38. Thus, the 2D camera 34, 36 would be used instead of the more expensive X-ray camera 38. Conversely, the image data from the separate imaging modalities could be combined to provide a more robust modeling tool. Combining the image data may enhance the overall predictive power of the system compared to using each imaging modality separately.

[0038] Similarly, predetermined appearance categories may be stored in the controller 18. The appearance categories may be based on measurement thresholds or ranges for each of the color, spectral characteristics, length, width, roundness, solidity, smoothness, and internal composition data. Based on these thresholds/ranges, at least two categories may be defined. For example, the measurement data may be used to provide thresholds or ranges that indicate the seed as either healthy or defective. When each seed is analyzed, the seed is associated with one of the categories. For example, a seed having one or more dimensions that are outside a range of values or above/below a threshold value is categorized into a first defective category; and seeds having one or more dimensions that are within a range of values or above/below a threshold value are categorized into a second healthy category. Multiple range/threshold values may be established to further categorize the seeds into more than two categories.

[0039] To maintain accuracy and repeatability of all imaging modes, provisions for verifying proper functionality and calibration of the cameras may be added to the system 10. For example, time-stable reference samples may be designed to include spectral, spatial, and X-ray transmission standards. Such standards may include reflectance and fluorescence targets, spatial calibration targets (e.g., line pairs or similar geometric patterns), height references, and X-ray targets (e.g., sample machines of varying thickness of synthetic material). The standards may be imaged at periodic intervals (e.g., beginning of each batch), and image processing methods will be used to verify the state of the imaging hardware and perform any necessary recalibrations.

[0040] Referring to Figs. 2, 3, and 5-7, weighing assembly 15 is located at a dispensing end of conveyor 26 for receiving and weighing each seed individually. In the illustrated embodiment, weighing assembly 15 comprises a collection mechanism 50 for receiving seeds as they are expelled from conveyor 26, a scale 52 at an outlet of the collection mechanism for weighing the seeds, and a transport mechanism 54 for transporting seeds from weighing assembly 15 to storage assembly 16. Collection mechanism 50 comprises a hopper that collects seeds from the end of conveyor 26 and drops the seeds onto scale 52. Scale 52 comprises a static load cell configured to measure the weight of each seed individually. Once a seed has been weighed, the transport mechanism 54 delivers the seed to the storage assembly 16. The transport mechanism 54 comprises a transport tube 56 and a vacuum 58 attached to the transport tube. The vacuum 58 transports each seed through the transport tube 54 to a seed collector 60, where each seed is held before being stored in the storage assembly 16.

[0041] Referring to Figs. 2, 3 and 5-7, the storage assembly 16 comprises the seed collector 60 and a collection bank including a plurality of wells 62 arranged in an x-y grid and preloaded with microplates 64. The seed collector 60 drops each seed into a dedicated microplate well 64 position. The seeds may be allowed to grow within the microplates 64 and seed growth is monitored. Having the imaging data saved for each seed allows correlations to be made between the imaging data and seed germination. As a result, the system 10 can determine which imaging modalities provide the best predictive capabilities for a given batch of seeds. Rather than growing the seeds in the microplates, the seeds may be transferred to other seed quality measurement techniques, as long as seed identity is maintained. In the illustrated embodiment of Fig. 2, storage assembly 16 is shown in a folded configuration that reduces the overall footprint of system 10. However, storage assembly 16 could be arranged in a generally collinear configuration (Fig. 8) with the rest of system 10. The collinear configuration provides greater accessibility to components of system 10 for repair and replacement.

[0042] In the illustrated embodiment, storage assembly 16 includes microplate storage. However, other storage methods are envisioned. For example, a gel-based format storage method may be used when it is desirable to perform radicle emergence testing. Additionally or alternatively, a wet towel storage method may be used for germination assays. Additionally or alternatively, a soil-based storage format may be used for greenhouse testing and/or field transplantation.

[0043] In the illustrated embodiment, the conveyor 26 is mounted to a support wall 70. The imaging and analysis assembly 14 could also be mounted to the support wall. However, the components of the system 10 could be located in a different manner without departing from the scope of the disclosure.

[0044] Referring to Figures 2 through 6, seeds are first placed in hopper 20 in preparation for being transported by conveyor 26 through system 10. As the seeds leave outlet 24 of hopper 20, vibratory feed 25 singularizes the seeds by spacing the seeds into a single row. Vibratory feed 25 then delivers the row of seeds to conveyor 26 which carries the seeds for viewing by cameras. Tracking sensor 33 matches each seed as it exits vibratory feed 25. Because the seeds travel along clear flat conveyor 26, clear image data can be acquired from both the upper and lower cameras. Additionally, the seeds remain in a known location and fixed orientation on conveyor 26, which allows each seed to be tracked with a high level of accuracy by the precision encoder.

[0045] The seeds first pass through the focal view of the first hyperspectral reflectance camera 30, which acquires image data across the visible light spectrum. An encoder reading may also be recorded as the seed is imaged by the hyperspectral reflectance camera 30 to track the position of the seed on the conveyor 26. The seeds then pass through the focal view of the second hyperspectral reflectance camera 32, which acquires image data across the near-infrared spectrum. An encoder reading may also be recorded as the seed is imaged by the hyperspectral reflectance camera 32 to track the position of the seed on the conveyor 26.

[0046] The seeds then pass under the focal view of the lower 2D camera 36. The lower 2D camera 36 acquires a two-dimensional image of each seed that is processed by the controller 18 to produce length and width data for each seed. In one embodiment, the value associated with maximum length and width measurements is recorded as the length and width values for the seed. An encoder reading may also be recorded when the seed is imaged by the lower 2D camera 36 to track the position of the seed on the conveyor 26. Shortly thereafter, the seeds pass under the focal view of the upper 2D camera 34. The upper 2D camera 34 acquires a two-dimensional image of each seed that is processed by the controller 18 to produce length and width data for each seed. In one embodiment, the values associated with a maximum length and width measurement are recorded as the length and width values for the seed. An encoder reading may also be recorded when the seed is imaged by the upper 2D camera 34 to track the position of the seed on the conveyor 26. As explained above, in a preferred embodiment, seeds may pass through the focal view of a 2D camera 34 before passing through the focal view of the hyperspectral reflectance cameras 30, 32.

[0047] The seeds then pass through passage 41 in housing 39 and into opening 43 under the view of X-ray camera 38 which takes an X-ray of the seeds. X-ray camera 38 acquires an image of the internal construction of each seed which is processed by controller 18. An encoder reading may also be recorded when the seed is imaged by X-ray camera 38 to track the position of the seed on conveyor 26. Finally, the seeds pass under the focal view of 3D camera 40. 3D camera 40 acquires a three-dimensional image of each seed which is processed by controller 18 to produce thickness data for each seed. An encoder reading may also be recorded when the seed is imaged by 3D camera 40 to track the position of the seed on conveyor 26. It is understood that the order of the instruments is not limited to the embodiments described herein.

[0048] Once a seed reaches the conveyor 26, the seeds are captured by the collection mechanism 50 of the weighing assembly 15 which then delivers the seeds individually to the scale 52 for weighing. After the seeds are weighed, the transport mechanism 54 may transport the seeds to the storage assembly 16 where the seed collector 60 places the seeds individually into a microplate 64. Within the microplates 64, the seeds are allowed to grow. The data acquired for each seed is linked to the microplate 64 in which the seed is stored. Thus, the seed may be further analyzed by referencing the image data acquired by the system 10 to make various determinations and correlations between seed quality and the associated seed image data. Alternatively, the seeds may be transferred to other trays, tubes, etc. for quality assessment while still maintaining the identity of each seed. In one embodiment, the transport mechanism 54 deposits the seeds directly into the growth medium.

[0049] Information obtained using imaging and analysis assembly 14 may be useful in subsequent processing, evaluation, or analysis of seeds. Generally, an attempt is made to correlate color, mean and variation in spectrum, size, shape, texture, and internal composition information for seeds with quality attributes, including germination. These correlations suggest preferred imaging techniques on an application-specific basis. For example, in seed production plants, data generated by system 10 may be used to predict an overall distribution of defective seeds in a seed inventory and to determine the distribution of defective seeds from a subsample of seeds that may then be extrapolated to predict the overall condition of the seed inventory. This distribution information may also be used to estimate seed quantities by commercial size categories and to slightly adjust sizing thresholds in cases where seed quantities are limited.

[0050] In addition to the above description, other embodiments may include a lower NIR hyperspectral camera and an NIR backlight using polarized light. Additionally, the X-ray housing may also consist of an inner and outer housing, where the inner housing is intended to block the majority of X-rays from the X-ray source and prevent X-ray interference or damage to other imaging equipment, while the outer X-ray housing is intended to prevent any remaining X-rays from leaving the housing pursuant to the presence of a human operator. Examples:

[0051] Example 1: Using the seed imager shown in FIG. 10. Figure 11 shows spatially aligned multimodal images of four types of corn seeds, including good, discolored, damaged, and inert seeds, each of which is identified by a trained human inspector. First, the good seed does not exhibit any damaged structure, abnormal color, and disease from the external surface view. However, X-ray imaging captured internal crack damage through the endosperm and embryo, affecting the physiological potential of the seed. Second, the discolored seed exhibits dark spots on the surface that can be seen by multiple optical cameras but not X-rays. Third and fourth, the damage and inertness may only occur in a local area, meaning that nothing abnormal is shown in the top (or bottom) view. Therefore, the combination of the top and bottom cameras is necessary to capture defects from a nearly 360-degree viewing range. Overall, this multimodal imaging invention enables capturing external appearance, internal structure, three-dimensional geometry and wide range of spectral information. The necessity of such data fusion in seed quality measurement can be demonstrated from this.

[0052] Example 2: A method of using embodiments of the invention for the purpose of making breeding determinations.

[0053] Example 3: A method of using data toward simple seed correlation for imaging data for quality metrics, including correlating pixel with defect class.

[0054] Example 4: A method of using the output data for big data and potentially machine learning.

[0055] Example 5: A method of using embodiments of the invention in combination with other seed categorization systems, e.g., genotyping, single seed identity entering the system, e.g., seed chopper HD trays, and identity maintained during the process.

[0056] Example 6: Embodiments where each of the imaging systems is used in part, such as when they are not used on a single sequential machine, but are separate and the system provides placement of the seeds such that they do not move from their location for imaging purposes. Furthermore, this would also include embodiments where the imaging systems are used in different sequential order. For example, particular applications use different groupings of imaging options. Seed identity/orientation is maintained.

[0057] Having described the invention in detail, it will be evident that modifications and variations are possible without departing from the scope of the invention defined in the appended claims.

[0058] When presenting elements of the present invention or preferred embodiment(s) thereof, the articles “a,” “an,” “the,” and “referred to” are intended to mean that there is one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that there may be additional elements other than the elements listed.

[0059] In view of the above, it will be seen that the various objects of the invention are achieved and other advantageous results attained. As various changes could be made in the above constructions and methods without departing from the scope of the invention, it is intended that all matter contained in the above description and shown in the accompanying drawings be construed as illustrative and not in a limiting sense.

Claims (19)

1. A seed imaging system for imaging seeds, comprising: a seed transfer station configured to move seeds through the system, the seed transfer station including a conveyor belt, wherein the seed transfer station is configured to substantially fix at least one of the position and orientation of each of the seeds on the conveyor belt as the seeds move through the system; a tracking sensor configured to record at least one of the position and orientation of each of the seeds on the conveyor belt; an imaging assembly comprising: at least a first camera mounted relative to the conveyor belt of the seed transfer station and configured to acquire 2D images of the seeds as the seeds move through the system; at least a second camera mounted relative to the conveyor belt of the seed transfer station and configured to acquire 3D images of the seeds as the seeds move through the system; at least a third camera mounted relative to the conveyor belt of the seed transfer station and configured to acquire X-ray images of the seeds as the seeds move through the system; and at least a fourth camera mounted relative to the conveyor belt of the seed transfer station and configured to acquire hyperspectral images of the seeds as the seeds move through the system; wherein the 2D images, the 3D images, the X-ray images, and the hyperspectral images include images of each of the seeds in at least one of a substantially fixed position and orientation on the conveyor belt; and a controller configured to: (i) align the acquired 2D images, the 3D images, the X-ray images, and the hyperspectral images for each of the seeds based, at least in part, on at least one of a substantially fixed position and orientation of each of the seeds recorded by the tracking sensor, and (ii) analyze the aligned images of the seeds for one or more characteristics. 2. The seed imaging system of claim 1, wherein the conveyor comprises a belt that is transparent or semi-transparent. 3. The seed imaging system of claim 1, further comprising a weighing assembly located at one end of the conveyor belt of the seed transfer station and configured to weigh individual seeds; wherein the weighing assembly includes: a hopper configured to receive seeds from the end of the conveyor belt; a scale disposed adjacent an outlet of the hopper, wherein the scale is configured to receive an individual seed from the hopper and weigh the individual seed; and a transport mechanism configured to move the weighed seed away from the scale. 4. The seed imaging system of claim 1, further comprising a storage assembly configured to automatically store seeds in separate storage containers based on analysis of seed images by the controller. 5. Seed imaging system according to claim 1, characterized in that the one or more characteristics include at least one characteristic among color, size, shape, texture, internal composition, mass, volume, moisture content and chemical composition. 6. The seed imaging system of claim 1, further comprising a seed collector configured to receive seeds from the conveyor belt and position the seeds in one or more storage containers; wherein the controller is configured to associate one or more characteristics of the seeds with the one or more storage containers in which the seeds are positioned. 7. Seed imaging system according to claim 1, characterized in that it further comprises multiple storage containers, each configured to receive at least one of the seeds. 8. The seed imaging system of claim 7, wherein each of the multiple storage containers includes a growth medium configured to facilitate growth of at least one of the seeds received in said storage container. 9. A method for imaging seeds, comprising the steps of: aligning individual seeds in at least one row in a seed loading assembly; transferring the individual seeds in at least one row from the seed loading assembly to a conveyor belt of a seed transfer station; recording, via a tracking sensor, at least one of the position and orientation of each of the individual seeds transferred to the conveyor belt; moving the seeds through a seed imaging system in at least one row using the conveyor belt of the seed transfer station, wherein at least one of the position and orientation of each of the seeds is substantially fixed on the conveyor belt as the seeds move past the first camera and the second camera of the seed imaging system; acquiring, using the first camera mounted relative to the conveyor belt of the seed transfer station, a second group of images of the seeds as the seeds move through the system via the conveyor belt acquiring, using the second camera mounted relative to the conveyor belt of the seed transfer station, a second group of images of the seeds as the seeds move through the system via the conveyor belt, wherein an imaging technology of the second camera is different from an imaging technology of the first camera; and aligning, by a controller, the first group of images of the seeds from the first camera and the second group of images of the seeds from the second camera based, at least in part, on at least one of the recorded position and orientation of each of the seeds. 10. The method of claim 9, further comprising analyzing the aligned images using the controller. 11. The method of claim 9, further comprising acquiring, using a third camera mounted relative to the conveyor belt of the seed transfer station, a third group of images of the seeds as the seeds move through the system via the conveyor belt, wherein an imaging technology of the third camera is different from the imaging technology of the first and second cameras. 12. The method of claim 11, further comprising acquiring, using a fourth camera mounted relative to the conveyor belt of the seed transfer station, a fourth group of images of the seeds as the seeds move across the conveyor belt, wherein an imaging technology of the fourth camera is different from the imaging technology of the first, second, and third cameras. 13. The method of claim 9, wherein the imaging technology of the first and second cameras is one of 2D imaging, 3D imaging, X-ray imaging, and hyperspectral imaging; and wherein the imaging technology of the second camera is one of 2D imaging, 3D imaging, X-ray imaging, and hyperspectral imaging. 14. The method of claim 9, further comprising: receiving individual seeds from the conveyor belt of the seed transfer station onto the scale of a weighing assembly, weighing the individual seeds received onto the scale using the weighing assembly, and then transporting the weighed individual seeds from the scale to a storage assembly. 15. The method of claim 9, further comprising storing the seeds using a storage assembly. 16. The method of claim 15, wherein storing the seeds comprises storing the seeds individually in microplates. 17. A seed imaging system for imaging seeds, comprising: a seed transfer station configured to move seeds through the system; a seed loading station configured to transfer seeds to the seed transfer station; a tracking sensor disposed at the seed loading station, wherein the tracking sensor is configured to record in at least one of the position and orientation of each of the seeds transferred to the seed transfer station; and an imaging assembly comprising a first camera mounted above the seed transfer station and configured to acquire a first group of images of the seeds as the seeds move through the system and a second camera mounted below the seed transfer station and configured to acquire a second group of images of the seeds as the seeds move through the system; wherein the transfer station is configured to substantially fix in at least one of the position and orientation of the seeds in the seed transfer station as the seeds move at least past the first and second cameras. 18. The seed imaging system of claim 17, wherein the first and second cameras have the same imaging technology. 19. Seed imaging system according to claim 17, characterized by the fact that the imaging technology of the first and second cameras is a technology between 2D imaging, 3D imaging, X-ray imaging and hyperspectral imaging.