EP3364228A1 - Forward-on-forward high dynamic range architecture for digital micromirror devices - Google Patents

Forward-on-forward high dynamic range architecture for digital micromirror devices Download PDF

Info

Publication number
EP3364228A1
EP3364228A1 EP18152342.4A EP18152342A EP3364228A1 EP 3364228 A1 EP3364228 A1 EP 3364228A1 EP 18152342 A EP18152342 A EP 18152342A EP 3364228 A1 EP3364228 A1 EP 3364228A1
Authority
EP
European Patent Office
Prior art keywords
dmd
plane
image
optics
subject plane
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP18152342.4A
Other languages
German (de)
French (fr)
Other versions
EP3364228B1 (en
Inventor
John David Jackson
Michael Perkins
Mark Lamm
Joseph Ma
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Christie Digital Systems USA Inc
Original Assignee
Christie Digital Systems USA Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Christie Digital Systems USA Inc filed Critical Christie Digital Systems USA Inc
Publication of EP3364228A1 publication Critical patent/EP3364228A1/en
Application granted granted Critical
Publication of EP3364228B1 publication Critical patent/EP3364228B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/10Beam splitting or combining systems
    • G02B27/1066Beam splitting or combining systems for enhancing image performance, like resolution, pixel numbers, dual magnifications or dynamic range, by tiling, slicing or overlapping fields of view
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/12Picture reproducers
    • H04N9/31Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3102Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM] using two-dimensional electronic spatial light modulators
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B26/00Optical devices or arrangements for the control of light using movable or deformable optical elements
    • G02B26/08Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
    • G02B26/0816Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light by means of one or more reflecting elements
    • G02B26/0833Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light by means of one or more reflecting elements the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B21/00Projectors or projection-type viewers; Accessories therefor
    • G03B21/005Projectors using an electronic spatial light modulator but not peculiar thereto
    • G03B21/008Projectors using an electronic spatial light modulator but not peculiar thereto using micromirror devices
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B21/00Projectors or projection-type viewers; Accessories therefor
    • G03B21/14Details
    • G03B21/147Optical correction of image distortions, e.g. keystone
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/001Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/12Picture reproducers
    • H04N9/31Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3102Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM] using two-dimensional electronic spatial light modulators
    • H04N9/312Driving therefor
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/12Picture reproducers
    • H04N9/31Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3102Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM] using two-dimensional electronic spatial light modulators
    • H04N9/312Driving therefor
    • H04N9/3126Driving therefor for spatial light modulators in series
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/12Picture reproducers
    • H04N9/31Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3141Constructional details thereof
    • H04N9/315Modulator illumination systems
    • H04N9/3152Modulator illumination systems for shaping the light beam

Definitions

  • the specification relates generally to digital micromirror devices, and specifically to a forward-on-forward high dynamic range architecture for digital micromirror devices.
  • High dynamic range and/or high contrast, projectors attempt to maximize contrast between black portions of projected images and white portions of projected images.
  • HDR projectors are based on digital micromirror devices ("DMD", such as Digital Light Processing or DLPTM), and the high dynamic range is achieved with two stages of imaging: a first pre-modulator DMD that creates an initial "normal” contrast version of an image and a second prime imaging DMD that re-images the initial "normal” contrast version of the image from the first DMD, a second time (which can be at a higher resolution), to create blacker blacks in the final image and thus increasing the final image contrast.
  • DMD digital micromirror devices
  • Existing architectures of such HDR projectors use the first DMD in a reverse configuration and the second DMD in a forward configuration.
  • the forward configuration which is the configuration in which most DMDs are designed to operate
  • a DMD is illuminated at an angle to a plane of the DMD and an image formed by the DMD is reflected normal (perpendicular) to the plane of the DMD.
  • the reverse configuration a plane of the DMD is illuminated perpendicularly, and an image formed by the DMD is reflected at an angle to the plane.
  • this disclosure is directed to a system and/or device for a high dynamic range (“HDR") projector that includes a first pre-modulator digital micromirror device (“DMD”) and a second prime-modulator DMD, each operated in a forward configuration.
  • the system includes optics which convey light from the first DMD to the second DMD, and at least one optical device which tilt a subject plane of the first DMD to an equivalent tilted subject plane, an equivalent lens plane of the optics, the equivalent tilted subject plane and an image plane of the second DMD all intersecting at a Scheimpflug intersection.
  • elements may be described as “configured to” perform one or more functions or “configured for” such functions.
  • an element that is configured to perform or configured for performing a function is enabled to perform the function, or is suitable for performing the function, or is adapted to perform the function, or is operable to perform the function, or is otherwise capable of performing the function.
  • An aspect of the specification provides a device comprising: a first digital micromirror device (DMD) and a second DMD each operated in a forward configuration, such that each is illuminated at a respective non-normal angle and a respective output image is reflected at a normal angle, a subject plane of the first DMD being parallel to the first DMD; optics between the first DMD and the second DMD, the optics configured to convey light reflected from the first DMD to illuminate an image plane at the second DMD in the forward configuration, the optics including an equivalent lens plane; and at least one optical device between the first DMD and the second DMD, the at least one optical device configured to: tilt the subject plane of the first DMD to an equivalent tilted subject plane, the equivalent lens plane, the equivalent tilted subject plane and the image plane all intersecting at a Scheimpflug intersection.
  • DMD digital micromirror device
  • second DMD each operated in a forward configuration, such that each is illuminated at a respective non-normal angle and a respective output image is reflected
  • the at least one optical device is further configured to: tilt the subject plane of the first DMD to the equivalent tilted subject plane such that the image plane at the second DMD is in uniform focus at the respective non-normal angle.
  • the at least one optical device includes a prism with an exit face at an angle tilted relative to the subject plane of the first DMD.
  • the at least one optical device includes a plurality of wedge prisms distributed between the first DMD and the second DMD.
  • the at least one optical device includes a prism configured to convey illumination light to the subject plane of the first DMD at the respective non-normal angle and convey the respective output image through an exit face of the prism, the exit face at an angle relative to the subject plane, the angle being greater than zero, and the angle selected to optically tilt the subject plane of the first DMD to the equivalent tilted subject plane.
  • the optics includes a prism configured to convey illumination light from the first DMD to the image plane of the second DMD at the respective non-normal angle and convey the respective output image of the second DMD through a respective exit face.
  • the optics includes one or more lenses.
  • the optics includes one or more apertures.
  • the first DMD is further operated in a pre-modulator mode and the second DMD is further operated in a prime-modulator mode.
  • the first DMD is modulated using first image data
  • the second DMD is modulated using second image determined from one or more of the first image data, a target projected image and a pixel spread function of the device.
  • Fig. 1 depicts a device 10 comprising: a first digital micromirror device (DMD) 11 and a second DMD 12 each operated in a forward configuration, such that each is illuminated at a respective non-normal angle and a respective output image is reflected at a normal angle, a subject plane 13 of the first DMD 11 being parallel to the first DMD 11; optics 14 between the first DMD 11 and the second DMD 12, the optics 14 configured to convey light reflected from the first DMD 11 to illuminate an image plane 15 at the second DMD 12 in the forward configuration, the optics 14 including an equivalent lens plane 16; and at least one optical device 17 between the first DMD 11 and the second DMD 12, the at least one optical device 17 configured to: tilt the subject plane 13 of the first DMD 11 to an equivalent tilted subject plane 18, the equivalent lens plane 16, the equivalent tilted subject plane 18 and the image plane 15 all intersecting at a Scheimpflug intersection 19.
  • DMD digital micromirror device
  • the equivalent lens plane 16, the equivalent tilted subject plane 18 and the image plane 15 meet a Scheimpflug condition such that light from the first DMD 11 is in uniform focus and/or is in substantially uniform focus at the second DMD 12 and/or at the image plane 15.
  • the equivalent lens plane 16, the equivalent tilted subject plane 18 and the image plane 15 meet a Scheimpflug condition such that light imaged onto the second DMD 12, from the first DMD 11, has a uniform pixel spread function (PSF) across the second DMD 12.
  • PSF uniform pixel spread function
  • the at least one optical device 17 has the effect of tilting the subject plane 13 of the first DMD 11 to appear tilted with respect to the optics 14 and/or the second DMD 12 such that the Scheimpflug condition can be met. Indeed, optically, the first DMD 11 "appears" to be located at the equivalent tilted subject plane 18, as indicated by virtual DMD 21 located inside the at least one optical device 17.
  • the at least one optical device 17 includes a prism with an exit face at an angle tilted relative to the subject plane 13 of the first DMD 11.
  • the at least one optical device 17 tilts the subject plane 13 of the first DMD 11 to the equivalent tilted subject plane 18 such that the image plane 15 at the second DMD 12 is in uniform focus and/or is in substantially uniform focus at at the respective non-normal angle (e.g. a PSF across the second DMD 12 is uniform).
  • the subject plane 13 of the first DMD 11 could not be uniformly focussed onto the second DMD 12 (e.g. nonuniform PSF) as, with the subject plane 13 being about parallel to the equivalent lens plane 16 of the optics 14, the Scheimpflug condition is impossible to meet.
  • the subject plane 13 of the first DMD 11 in order to operate the first DMD 11 in the forward configuration, the subject plane 13 of the first DMD 11 must be about parallel to the equivalent lens plane 16 of the optics 14.
  • the at least one optical device 17 enables the use of both DMDs 11, 12 in the forward configuration such that the Scheimpflug condition can be met.
  • the optics 14 includes a single lens, however, it is appreciated that such a depiction represent a simplification of the optics 14, and furthermore, device 10 is represents a simplification of devices used in projectors.
  • Fig. 2 and Fig. 3 depict a device 100 that can be used with, and/or incorporated into, a high dynamic range ("HDR") projector.
  • Fig. 2 depicts a schematic side view of the device 100, including example optics
  • Fig. 3 depicts a schematic side view of the device 100 with the optics of the device 100 unfolded (e.g. reflections from mirrors and/or reflective surfaces are removed for clarity with the optical path through the device 100 depicted without reflections), as well as a ray-trace diagram for three points at the first DMD 101, through the device 100.
  • the device 100 is not depicted to scale in either of Fig. 2 or Fig. 3 . While device 100 is generally more complex than device 10, device 100 generally operates according to the same principles of device 10, as described hereafter.
  • FIG. 2 an optical path 125 through the device 100 is depicted, and in Fig. 3 an unfolded optical path 225 between the DMDs 101, 102 is depicted schematically.
  • the device 100 generally comprises: a first DMD 101, a second DMD 102, each operated in a forward configuration, such that each is illuminated at a respective non-normal angle and a respective output image is reflected at a normal angle, a subject plane 113 of the first DMD 101 being parallel to the first DMD 101; optics 114 between the first DMD 101 and the second DMD 102, the optics 114 configured to convey light reflected from the first DMD 101 to illuminate an image plane 115 at the second DMD 102 in the forward configuration, the optics 114 including an equivalent lens plane 116; and at least one optical device 117 between the first DMD 101 and the second DMD 102, the at least one optical device 117 configured to: tilt the subject plane 113 of the first DMD 101 to an equivalent tilted subject plane 118, the equivalent lens plane 116, the equivalent tilted subject plane 118 and the image plane 115 all intersecting at a Scheimpflug intersection (not depicted
  • the relative locations of the subject plane 113, the equivalent lens plane 116, the equivalent tilted subject plane 118 and the image plane 115 are depicted in Fig. 3 without intervening reflecting surfaces (as in Fig. 2 ).
  • the relative angles between the subject plane 113, the equivalent lens plane 116, the equivalent tilted subject plane 118 and the image plane 115 are clearer as reflections due to mirrors and/or reflecting surfaces are removed.
  • the equivalent lens plane 116 is depicted in Fig. 3 , but not Fig. 2 , for clarity.
  • At least one optical device 117 is further configured to: tilt the subject plane 113 of the first DMD 101 to the equivalent tilted subject plane 118 such that the image plane 115 at the second DMD 102 is uniformly focussed at the respective non-normal angle.
  • the at least one optical device 117 includes a prism with an exit face 129 at an angle 130 tilted relative to the subject plane of the first DMD 101
  • the first DMD 101 is illuminated by respective illumination light as indicated by path 125 incident on the first DMD 101, for example from a light source (not depicted) of a projector (not depicted), and the like.
  • the illumination light generally comprises one or more of white light, red light, green light, blue light, infrared light, ultraviolet light and the like, having an aspect ratio similar to an aspect ratio of the first DMD 101.
  • the illumination light comprises a series of colors of illumination light, for example two or more of a red light, a green light and a blue light that illuminates the first DMD 101 in a sequence.
  • the device 100 can be a component of a single-chip imaging projector (e.g. one each of a pre-modulator DMD (e.g. the first DMD 101) and a prime-modulator DMD (e.g. the second DMD 102)), a two-chip imaging projector (e.g. two each of a pre-modulator DMD and a prime modulator DMD), and a three-chip imaging projector (e.g. three each of a pre-modulator DMD and a prime modulator DMD).
  • a single-chip imaging projector e.g. one each of a pre-modulator DMD (e.g. the first DMD 101) and a prime-modulator DMD (e.g. the second DMD 102)
  • a two-chip imaging projector e.g. two each of a pre-modulator DMD and a prime modulator DMD
  • a three-chip imaging projector e.g. three each of a pre-modulator DMD and a prime modul
  • an aspect ratio of each of the first DMD 101 and the second DMD 102 can be similar or different.
  • different aspect ratios can be used when the first DMD 101 is under filled.
  • each of the first DMD 101 and the second DMD 102 can be modulated using similar image data.
  • the first DMD 101 can be modulated using first image data
  • the second DMD 102 can be modulated using second image data which can comprise the first image such that regions of the second DMD 102 that correspond to dark areas of the image being generated for projection are illuminated by light from the first DMD 101 having a similar level of brightness, in order to achieve a high dynamic range.
  • DMDs 101, 102 are operated in order to illuminate non-white pixels of the second DMD 102 with corresponding non-white light from the first DMD 101 and/or to not illuminate black pixels of the second DMD 102.
  • the image data used to modulate the second DMD 102 operated in a prime-modulator mode can be calculated using a target image to be projected as well as data defining the first DMD 101 operated in a pre-modulator mode (which effectively is a light source for illuminating the second DMD 102).
  • a pre-modulator mode which effectively is a light source for illuminating the second DMD 102
  • non-HDR projection there is no pre-modulator (e.g. no first DMD 101) and illumination light on a prime-modulator (e.g. the second DMD 102) is assumed to be uniform.
  • the input light to prime-modulator e.g. the second DMD 102
  • the image data, used to modulate the first DMD 101 operated in a pre-modulator mode is determined to be whatever is needed to yield the target image to be projected (assuming illumination light is uniformly focussed and/or is substantially uniformly focussed at the first DMD 101).
  • OutImage is the target image to be projected
  • PremodImage is the image to be provided at the first DMD 101 operated in the pre-modulator mode
  • PSF is the pixel spread function of device 100
  • PrimeImage is the image to be provided at the second DMD 102 operated in the prime-modulator mode.
  • the first DMD 101 is operated in a forward configuration, such that illumination light is received at a subject plane 113 of the first DMD 101 at a non-normal angle and a respective output image is reflected from the first DMD 101 at a normal angle (e.g. optical path 125 is normal and/or perpendicular to a plane of the first DMD 101, as represented by arrow 138, and/or subject plane 113 is parallel to the plane of the first DMD 101).
  • a prism e.g.
  • a total internal reflection (TIR) prism) of the at least one optical device 117 is configured to convey the illumination light to the subject plane 113 of the first DMD 101 at the non-normal angle and convey the respective output image through an exit face 129 of the prism of the at least one optical device 117.
  • TIR total internal reflection
  • a prism e.g. a TIR prism
  • device 100 can include a TIR prism with an exit face parallel to the subject plane 113, and a separate wedge prism located after the exit face configured to tilt the subject plane 113 of the first DMD 101 to the equivalent tilted subject plane 118.
  • wedge prism can be located within optics 114, as long as the equivalent lens plane 116, the equivalent tilted subject plane 118 and the image plane 115 all intersect at a Scheimpflug intersection.
  • the exit face 129 is at an angle 130 relative to the subject plane 113, the angle 130 being greater than zero (e.g. not parallel to the subject plane 113 of the first DMD 101).
  • the angle 130 is selected to tilt the subject plane 113 of the first DMD 101 to the equivalent tilted subject plane 118, similar to the operation of device 10.
  • the second DMD 102 is also operated in the forward configuration, such that the image plane 115 intersects the equivalent tilted subject plane 118 and the equivalent lens plane 116 of the optics 114 to meet the Scheimpflug condition.
  • the optics 114 include a prism 142 configured to convey illumination light from the first DMD 101 (e.g. from the equivalent tilted subject plane 118) to the image plane 115 of the second DMD 102 at the respective non-normal angle (as best seen in Fig. 3 ) and convey the respective output image of the second DMD 102 through a respective exit face 149 of the prism 142 (as best seen in Fig. 2 , and represented by optical path 125 reflected away from the second DMD 102 through prism 142.
  • a prism 142 configured to convey illumination light from the first DMD 101 (e.g. from the equivalent tilted subject plane 118) to the image plane 115 of the second DMD 102 at the respective non-normal angle (as best seen in Fig. 3 ) and convey the respective output image of the second DMD 102 through a respective exit face 149 of the prism 142 (as best seen in Fig. 2 , and represented by optical path 125 reflected away from the second DMD
  • light along each of paths 125, 225 is for on-state light and that off-state light is reflected out of device 100 and/or into respective light dump devices (e.g. one for each of DMDs 101, 102).
  • respective light dump devices e.g. one for each of DMDs 101, 102
  • device 100 can include such light dump devices and at least one optical device 117 and prism 142 can be adapted to convey respective off-state light from each of DMDs 101, 102 to respective light dump devices.
  • Optics 114 between the DMDs 101, 102 can comprise one or more lenses, apertures and the like. As depicted, the optics 114 comprises five lenses and one aperture. Regardless, it is appreciated that optics 114 can be effectively modeled with a lens similar to the optics 14 depicted in Fig. 1 , with an equivalent lens plane 116.
  • the lenses generally image the equivalent tilted subject plane 118 onto the image plane 115 such that the PSF of device 100 at the image plane 115 is generally uniform.
  • the optics 114 can collimate the light at the equivalent tilted subject plane.
  • angle 130 and the optics 114 are further selected to both meet the Scheimpflug condition and to meet physical constraints of a projector into which device 100 is integrated.
  • DMDs are generally designed to be operated in a forward configuration with illumination light at a given angle.
  • DLPTM based DMDs have illumination angles of about 24°
  • other DLPTM based DMDs have illumination angles of about 34°, though other given angles are within the scope of present implementations.
  • a given illumination angle of a DMD selected for use with device 100 is generally used to selecting the angle 130, as well as in selecting the optics 114.
  • each of the DMDs 101, 102 can have the same illumination angle, or each of the DMDs 101, 102 can have illumination angle different from one another.
  • the positions of each of the DMDs, 101, 102, the position of the at least one optical device 117, including the angle 130, and the optics 114 are selected such that the Scheimpflug condition is met
  • the parameters of device 100 can further be selected using optical modelling software, using a desired physical configuration of the components of the device 100.
  • a certain configuration of the optics 114 may be desired such that the device 100 has a given physical footprint and/or each of the DMDs 101, 102 can have a given set of operational conditions (including, but not limited to an angle of illumination in a forward configuration).
  • such a configuration of device 100 allows both DMDs 101, 102 to be operated in a forward configuration, which generally allows a more efficient operation of a projector into which device 100 is incorporated, as compared to similar projectors that use a first DMD in a reverse configuration.
  • devices 10, 100 include only one optical device configured to: tilt a subject plane of a first DMD to an equivalent tilted subject plane, in yet further implementations, more than one optical device can be used to tilt the subject plane of the first DMD to an equivalent tilted subject plane.
  • Fig. 4 and Fig. 5 depict a device 400 that can be used with, and/or incorporated into, a high dynamic range ("HDR") projector.
  • Fig. 4 depicts a schematic side view of the device 400, including example optics
  • Fig. 5 depicts a schematic side view of the device 400 with the optics of the device 400 unfolded (e.g. reflections from mirrors and/or reflective surfaces are removed for clarity with the optical path through the device 400 depicted without reflections).
  • the device 400 is not depicted to scale in either of Fig. 4 or Fig. 5 . While device 400 is generally more complex than device 10, device 400 generally operates according to the same principles of device 10, as described hereafter.
  • the device 400 generally comprises: a first DMD 401, a second DMD 402, each operated in a forward configuration, such that each is illuminated at a respective non-normal angle and a respective output image is reflected at a normal angle, a subject plane 413 of the first DMD 401 being parallel to the first DMD 401; optics 414 between the first DMD 401 and the second DMD 402, the optics 414 configured to convey light reflected from the first DMD 401 to illuminate an image plane 415 at the second DMD 402 in the forward configuration, the optics 414 including an equivalent lens plane 416; and a plurality of optical devices 417-1, 417-2 between the first DMD 401 and the second DMD 402, the plurality of optical devices 417-1, 417-2 configured to: tilt the subject plane 413 of the first DMD 401 to an equivalent tilted subject plane 418, the equivalent lens plane 416, the equivalent tilted subject plane 418 and the image plane 415
  • Device 400 further comprises a respective TIR prism 450-1, 450-2 at each of the DMDs 401, 402 configured to convey illumination light to a respective DMD 401, 402 and convey a respective output image away from the respective DMD 401, 402 (e.g. the illumination light for the second DMD 402 comprising the respective output image of the first DMD 401).
  • Each of the TIR prism 450-1. 450-2 can be a component of the optics 414.
  • the optics 414 otherwise comprises eight lenses symmetrically arranged along an optical path between the DMDs 401, 402; again, it is appreciated that the optics 414 can be modelled using a single lens as in device 10, having the equivalent lens plane 416.
  • the subject plane 413, the equivalent lens plane 416, the equivalent tilted subject plane 418 and the image plane 415 are depicted in Fig. 5 , but not Fig. 4 , for clarity.
  • the relative angles between the subject plane 413, the equivalent lens plane 416, the equivalent tilted subject plane 418 and the image plane 415 are clearer as reflections due to mirrors and/or reflecting surfaces are removed.
  • the plurality of optical devices 417-1, 417-2 are further configured to: tilt the subject plane 413 of the first DMD 401 to the equivalent tilted subject plane 418 such that the image plane 415 at the second DMD 402 is in uniform focus and/or is in substantially uniform focus at the respective non-normal angle.
  • the plurality of optical devices 417-1, 417-2 include a plurality of wedge prisms distributed between the first DMD 401 and the second DMD 402.
  • optical device 417-1 comprises a first wedge prism having an exit face 429-1 angled with respect to the subject plane 418
  • optical device 417-2 comprises a second wedge prism having an exit face 429-2 also angled with respect to the subject plane 418.
  • first wedge prism e.g. optical device 417-1
  • the second wedge prism e.g.
  • optical device 417-2) is located after optics 114 (and before the TIR prism 450-2), it is appreciated that the total effect of the two wedge prisms on the subject plane 413, with respect to the image plane 415, is to tilt the subject plane 413 of the first DMD 401 to the equivalent tilted subject plane 418.
  • device 400 demonstrates that any number of optical devices can be used to tilt the subject plane of a first DMD, operated in a forward configuration, to an equivalent tilted subject plane such that the equivalent tilted subject plane, an equivalent lens plane and an image plane intersect at a Scheimpflug intersection and/or meet the Scheimpflug condition.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Optics & Photonics (AREA)
  • Computer Hardware Design (AREA)
  • Theoretical Computer Science (AREA)
  • Projection Apparatus (AREA)
  • Mechanical Light Control Or Optical Switches (AREA)
  • Transforming Electric Information Into Light Information (AREA)

Abstract

A forward-on-forward high dynamic range architecture for digital micromirror devices device is provided. In particular, provided herein is a device that includes two digital micromirror devices (DMDs), each operated in a forward configuration, such that each is illuminated at a respective non-normal angle and a respective output image is reflected at a normal angle, a subject plane of a first DMD being parallel to the first DMD. Optics between the DMDs are configured to convey light reflected from the first DMD to illuminate an image plane at a second DMD in the forward configuration, the optics including an equivalent lens plane. At least one optical device between the DMDs is configured to: tilt the subject plane of the first DMD to an equivalent tilted subject plane, the equivalent lens plane, the equivalent tilted subject plane and the image plane all intersecting at a Scheimpflug intersection.

Description

  • The specification relates generally to digital micromirror devices, and specifically to a forward-on-forward high dynamic range architecture for digital micromirror devices.
  • BACKGROUND
  • High dynamic range ("HDR"), and/or high contrast, projectors attempt to maximize contrast between black portions of projected images and white portions of projected images. Some HDR projectors are based on digital micromirror devices ("DMD", such as Digital Light Processing or DLP™), and the high dynamic range is achieved with two stages of imaging: a first pre-modulator DMD that creates an initial "normal" contrast version of an image and a second prime imaging DMD that re-images the initial "normal" contrast version of the image from the first DMD, a second time (which can be at a higher resolution), to create blacker blacks in the final image and thus increasing the final image contrast.
  • Existing architectures of such HDR projectors use the first DMD in a reverse configuration and the second DMD in a forward configuration. In the forward configuration, which is the configuration in which most DMDs are designed to operate, a DMD is illuminated at an angle to a plane of the DMD and an image formed by the DMD is reflected normal (perpendicular) to the plane of the DMD. However, in the reverse configuration, a plane of the DMD is illuminated perpendicularly, and an image formed by the DMD is reflected at an angle to the plane.
  • Operating the first DMD in the reverse configuration in HDR projectors has been necessary when using traditional projection optics because of the Scheimpflug principle, which describes under what conditions it is possible to focus two optical planes onto each other to achieve acceptable focus of light from the first DMD onto the second DMD. Indeed, if the first DMD were operated in the forward configuration using traditional optics, a subject plane of the first DMD could not be focussed on the second DMD, other than in a narrow region. As a result, the first DMD in existing-architecture HDR projectors is operated in the reverse configuration. However, operating the first DMD in the reverse configuration results in a system that is significantly less optically efficient, due to diffractive effects of the DMD mirror array, which acts as a blaze grating, and which can lead to increases in projector cost, size, noise and power consumption.
  • SUMMARY
  • In general, this disclosure is directed to a system and/or device for a high dynamic range ("HDR") projector that includes a first pre-modulator digital micromirror device ("DMD") and a second prime-modulator DMD, each operated in a forward configuration. The system includes optics which convey light from the first DMD to the second DMD, and at least one optical device which tilt a subject plane of the first DMD to an equivalent tilted subject plane, an equivalent lens plane of the optics, the equivalent tilted subject plane and an image plane of the second DMD all intersecting at a Scheimpflug intersection.
  • In this specification, elements may be described as "configured to" perform one or more functions or "configured for" such functions. In general, an element that is configured to perform or configured for performing a function is enabled to perform the function, or is suitable for performing the function, or is adapted to perform the function, or is operable to perform the function, or is otherwise capable of performing the function.
  • It is understood that for the purpose of this specification, language of "at least one of X, Y, and Z" and "one or more of X, Y and Z" can be construed as X only, Y only, Z only, or any combination of two or more items X, Y, and Z (e.g., XYZ, XY, YZ, XZ, and the like). Similar logic can be applied for two or more items in any occurrence of "at least one ..." and "one or more..." language.
  • An aspect of the specification provides a device comprising: a first digital micromirror device (DMD) and a second DMD each operated in a forward configuration, such that each is illuminated at a respective non-normal angle and a respective output image is reflected at a normal angle, a subject plane of the first DMD being parallel to the first DMD; optics between the first DMD and the second DMD, the optics configured to convey light reflected from the first DMD to illuminate an image plane at the second DMD in the forward configuration, the optics including an equivalent lens plane; and at least one optical device between the first DMD and the second DMD, the at least one optical device configured to: tilt the subject plane of the first DMD to an equivalent tilted subject plane, the equivalent lens plane, the equivalent tilted subject plane and the image plane all intersecting at a Scheimpflug intersection.
  • In some implementations, the at least one optical device is further configured to: tilt the subject plane of the first DMD to the equivalent tilted subject plane such that the image plane at the second DMD is in uniform focus at the respective non-normal angle.
  • In some implementations, the at least one optical device includes a prism with an exit face at an angle tilted relative to the subject plane of the first DMD.
  • In some implementations, the at least one optical device includes a plurality of wedge prisms distributed between the first DMD and the second DMD.
  • In some implementations, the at least one optical device includes a prism configured to convey illumination light to the subject plane of the first DMD at the respective non-normal angle and convey the respective output image through an exit face of the prism, the exit face at an angle relative to the subject plane, the angle being greater than zero, and the angle selected to optically tilt the subject plane of the first DMD to the equivalent tilted subject plane.
  • In some implementations, the optics includes a prism configured to convey illumination light from the first DMD to the image plane of the second DMD at the respective non-normal angle and convey the respective output image of the second DMD through a respective exit face.
  • In some implementations, the optics includes one or more lenses.
  • In some implementations, the optics includes one or more apertures.
  • In some implementations, the first DMD is further operated in a pre-modulator mode and the second DMD is further operated in a prime-modulator mode. In some these implementations, the first DMD is modulated using first image data, and the second DMD is modulated using second image determined from one or more of the first image data, a target projected image and a pixel spread function of the device.
  • BRIEF DESCRIPTIONS OF THE DRAWINGS
  • For a better understanding of the various implementations described herein and to show more clearly how they may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings in which:
    • Fig. 1 depicts a schematic side view of a device for use with a high dynamic range projector, the device including two DMDs, each operated in a forward configuration, optics which convey light reflected from the first DMD to illuminate an image plane at the second DMD in the forward configuration, and at least one optical device between the first DMD and the second DMD, the at least one optical device configured to: tilt a subject plane of the first DMD to an equivalent tilted subject plane, an equivalent lens plane of the optics, the equivalent tilted subject plane and the image plane all intersecting at a Scheimpflug intersection, according to non-limiting implementations
    • Fig. 2 depicts a specific implementation of the device of Fig. 1, according to non-limiting implementations.
    • Fig. 3 depicts an unfolded version of the optics of the device of Fig. 2, along with a ray-trace diagram through the device, according to non-limiting implementations.
    • Fig. 4 depicts another specific implementation of the device of Fig. 1, according to non-limiting implementations.
    • Fig. 5 depicts an unfolded version of the optics of the device of Fig. 4, according to non-limiting implementations.
    DETAILED DESCRIPTION
  • Fig. 1 depicts a device 10 comprising: a first digital micromirror device (DMD) 11 and a second DMD 12 each operated in a forward configuration, such that each is illuminated at a respective non-normal angle and a respective output image is reflected at a normal angle, a subject plane 13 of the first DMD 11 being parallel to the first DMD 11; optics 14 between the first DMD 11 and the second DMD 12, the optics 14 configured to convey light reflected from the first DMD 11 to illuminate an image plane 15 at the second DMD 12 in the forward configuration, the optics 14 including an equivalent lens plane 16; and at least one optical device 17 between the first DMD 11 and the second DMD 12, the at least one optical device 17 configured to: tilt the subject plane 13 of the first DMD 11 to an equivalent tilted subject plane 18, the equivalent lens plane 16, the equivalent tilted subject plane 18 and the image plane 15 all intersecting at a Scheimpflug intersection 19.
  • In other words, the equivalent lens plane 16, the equivalent tilted subject plane 18 and the image plane 15 meet a Scheimpflug condition such that light from the first DMD 11 is in uniform focus and/or is in substantially uniform focus at the second DMD 12 and/or at the image plane 15. Put another way, the equivalent lens plane 16, the equivalent tilted subject plane 18 and the image plane 15 meet a Scheimpflug condition such that light imaged onto the second DMD 12, from the first DMD 11, has a uniform pixel spread function (PSF) across the second DMD 12.
  • The at least one optical device 17 has the effect of tilting the subject plane 13 of the first DMD 11 to appear tilted with respect to the optics 14 and/or the second DMD 12 such that the Scheimpflug condition can be met. Indeed, optically, the first DMD 11 "appears" to be located at the equivalent tilted subject plane 18, as indicated by virtual DMD 21 located inside the at least one optical device 17.
  • As depicted, the at least one optical device 17 includes a prism with an exit face at an angle tilted relative to the subject plane 13 of the first DMD 11. The result is that the at least one optical device 17 tilts the subject plane 13 of the first DMD 11 to the equivalent tilted subject plane 18 such that the image plane 15 at the second DMD 12 is in uniform focus and/or is in substantially uniform focus at at the respective non-normal angle (e.g. a PSF across the second DMD 12 is uniform).
  • Put yet another way, without the at least one optical device 17 the subject plane 13 of the first DMD 11 could not be uniformly focussed onto the second DMD 12 (e.g. nonuniform PSF) as, with the subject plane 13 being about parallel to the equivalent lens plane 16 of the optics 14, the Scheimpflug condition is impossible to meet. However, in order to operate the first DMD 11 in the forward configuration, the subject plane 13 of the first DMD 11 must be about parallel to the equivalent lens plane 16 of the optics 14. Hence, the at least one optical device 17 enables the use of both DMDs 11, 12 in the forward configuration such that the Scheimpflug condition can be met.
  • As depicted, the optics 14 includes a single lens, however, it is appreciated that such a depiction represent a simplification of the optics 14, and furthermore, device 10 is represents a simplification of devices used in projectors.
  • Hence, attention is next directed to Fig. 2 and Fig. 3, each of which depict a device 100 that can be used with, and/or incorporated into, a high dynamic range ("HDR") projector. Fig. 2 depicts a schematic side view of the device 100, including example optics, and Fig. 3 depicts a schematic side view of the device 100 with the optics of the device 100 unfolded (e.g. reflections from mirrors and/or reflective surfaces are removed for clarity with the optical path through the device 100 depicted without reflections), as well as a ray-trace diagram for three points at the first DMD 101, through the device 100. Furthermore, it is understood that the device 100 is not depicted to scale in either of Fig. 2 or Fig. 3. While device 100 is generally more complex than device 10, device 100 generally operates according to the same principles of device 10, as described hereafter.
  • Furthermore, in Fig. 2, an optical path 125 through the device 100 is depicted, and in Fig. 3 an unfolded optical path 225 between the DMDs 101, 102 is depicted schematically.
  • With reference to both Fig. 2 and Fig. 3, the device 100 generally comprises: a first DMD 101, a second DMD 102, each operated in a forward configuration, such that each is illuminated at a respective non-normal angle and a respective output image is reflected at a normal angle, a subject plane 113 of the first DMD 101 being parallel to the first DMD 101; optics 114 between the first DMD 101 and the second DMD 102, the optics 114 configured to convey light reflected from the first DMD 101 to illuminate an image plane 115 at the second DMD 102 in the forward configuration, the optics 114 including an equivalent lens plane 116; and at least one optical device 117 between the first DMD 101 and the second DMD 102, the at least one optical device 117 configured to: tilt the subject plane 113 of the first DMD 101 to an equivalent tilted subject plane 118, the equivalent lens plane 116, the equivalent tilted subject plane 118 and the image plane 115 all intersecting at a Scheimpflug intersection (not depicted, but assumed to be nonetheless present, for example off the page of Fig. 3).
  • It is appreciated that the relative locations of the subject plane 113, the equivalent lens plane 116, the equivalent tilted subject plane 118 and the image plane 115 are depicted in Fig. 3 without intervening reflecting surfaces (as in Fig. 2). For example, in the unfolded view of device 100 in Fig. 3, the relative angles between the subject plane 113, the equivalent lens plane 116, the equivalent tilted subject plane 118 and the image plane 115 are clearer as reflections due to mirrors and/or reflecting surfaces are removed. The equivalent lens plane 116 is depicted in Fig. 3, but not Fig. 2, for clarity.
  • As with at least one optical device 17, at least one optical device 117 is further configured to: tilt the subject plane 113 of the first DMD 101 to the equivalent tilted subject plane 118 such that the image plane 115 at the second DMD 102 is uniformly focussed at the respective non-normal angle.
  • As depicted, the at least one optical device 117 includes a prism with an exit face 129 at an angle 130 tilted relative to the subject plane of the first DMD 101
  • It is further assumed in Fig. 2 and Fig. 3 that the first DMD 101 is illuminated by respective illumination light as indicated by path 125 incident on the first DMD 101, for example from a light source (not depicted) of a projector (not depicted), and the like. The illumination light generally comprises one or more of white light, red light, green light, blue light, infrared light, ultraviolet light and the like, having an aspect ratio similar to an aspect ratio of the first DMD 101. In some implementations, the illumination light comprises a series of colors of illumination light, for example two or more of a red light, a green light and a blue light that illuminates the first DMD 101 in a sequence. For example, the device 100 can be a component of a single-chip imaging projector (e.g. one each of a pre-modulator DMD (e.g. the first DMD 101) and a prime-modulator DMD (e.g. the second DMD 102)), a two-chip imaging projector (e.g. two each of a pre-modulator DMD and a prime modulator DMD), and a three-chip imaging projector (e.g. three each of a pre-modulator DMD and a prime modulator DMD).
  • Furthermore, an aspect ratio of each of the first DMD 101 and the second DMD 102 can be similar or different. For example, different aspect ratios can be used when the first DMD 101 is under filled.
  • Either way, it is assumed that the first DMD 101 can further operate in a pre-modulator mode and the second DMD 102 can further operate in a prime-modulator mode. In other words, each of the first DMD 101 and the second DMD 102 can be modulated using similar image data. For example, the first DMD 101 can be modulated using first image data, and the second DMD 102 can be modulated using second image data which can comprise the first image such that regions of the second DMD 102 that correspond to dark areas of the image being generated for projection are illuminated by light from the first DMD 101 having a similar level of brightness, in order to achieve a high dynamic range. For example, DMDs 101, 102 are operated in order to illuminate non-white pixels of the second DMD 102 with corresponding non-white light from the first DMD 101 and/or to not illuminate black pixels of the second DMD 102. The image data used to modulate the second DMD 102 operated in a prime-modulator mode can be calculated using a target image to be projected as well as data defining the first DMD 101 operated in a pre-modulator mode (which effectively is a light source for illuminating the second DMD 102). In non-HDR projection there is no pre-modulator (e.g. no first DMD 101) and illumination light on a prime-modulator (e.g. the second DMD 102) is assumed to be uniform. In HDR projection the input light to prime-modulator (e.g. the second DMD 102) is intentionally not uniform to achieve very black HDR blacks. So, given this non-uniformity, the image data, used to modulate the first DMD 101 operated in a pre-modulator mode, is determined to be whatever is needed to yield the target image to be projected (assuming illumination light is uniformly focussed and/or is substantially uniformly focussed at the first DMD 101). Mathematically, the target image is divided by the pre-image data to yield the primeimage data, which can be mathematically expressed as: Out Image = PremodImage PSF × PrimeImage .
    Figure imgb0001

    where OutImage is the target image to be projected, PremodImage is the image to be provided at the first DMD 101 operated in the pre-modulator mode, PSF is the pixel spread function of device 100, and PrimeImage is the image to be provided at the second DMD 102 operated in the prime-modulator mode. Hence, given a target image (e.g. OutImage), and the PSF, the prime and premod images can be calculated to minimize the errors between the target and a source image.
  • With further reference to Fig. 2 and Fig. 3, the first DMD 101 is operated in a forward configuration, such that illumination light is received at a subject plane 113 of the first DMD 101 at a non-normal angle and a respective output image is reflected from the first DMD 101 at a normal angle (e.g. optical path 125 is normal and/or perpendicular to a plane of the first DMD 101, as represented by arrow 138, and/or subject plane 113 is parallel to the plane of the first DMD 101). A prism (e.g. a total internal reflection (TIR) prism) of the at least one optical device 117 is configured to convey the illumination light to the subject plane 113 of the first DMD 101 at the non-normal angle and convey the respective output image through an exit face 129 of the prism of the at least one optical device 117.
  • It is further appreciated that while the prism of the at least one optical device 117 is configured to both convey light to the first DMD 101 and also to tilt the subject plane 113 of the first DMD 101 to the equivalent tilted subject plane 118, in other implementations a prism (e.g. a TIR prism) can be used to convey light to, and away from, the first DMD 101, and at least one optical device 117 can be a component separate from such a prism. For example, device 100 can include a TIR prism with an exit face parallel to the subject plane 113, and a separate wedge prism located after the exit face configured to tilt the subject plane 113 of the first DMD 101 to the equivalent tilted subject plane 118. Alternatively, such wedge prism can be located within optics 114, as long as the equivalent lens plane 116, the equivalent tilted subject plane 118 and the image plane 115 all intersect at a Scheimpflug intersection.
  • The exit face 129 is at an angle 130 relative to the subject plane 113, the angle 130 being greater than zero (e.g. not parallel to the subject plane 113 of the first DMD 101). The angle 130 is selected to tilt the subject plane 113 of the first DMD 101 to the equivalent tilted subject plane 118, similar to the operation of device 10. The second DMD 102 is also operated in the forward configuration, such that the image plane 115 intersects the equivalent tilted subject plane 118 and the equivalent lens plane 116 of the optics 114 to meet the Scheimpflug condition.
  • As depicted, device 100, the optics 114 include a prism 142 configured to convey illumination light from the first DMD 101 (e.g. from the equivalent tilted subject plane 118) to the image plane 115 of the second DMD 102 at the respective non-normal angle (as best seen in Fig. 3) and convey the respective output image of the second DMD 102 through a respective exit face 149 of the prism 142 (as best seen in Fig. 2, and represented by optical path 125 reflected away from the second DMD 102 through prism 142.
  • While not depicted, it is assumed that light along each of paths 125, 225 is for on-state light and that off-state light is reflected out of device 100 and/or into respective light dump devices (e.g. one for each of DMDs 101, 102). In other words, while not depicted, it is assumed that device 100 can include such light dump devices and at least one optical device 117 and prism 142 can be adapted to convey respective off-state light from each of DMDs 101, 102 to respective light dump devices.
  • Optics 114 between the DMDs 101, 102 can comprise one or more lenses, apertures and the like. As depicted, the optics 114 comprises five lenses and one aperture. Regardless, it is appreciated that optics 114 can be effectively modeled with a lens similar to the optics 14 depicted in Fig. 1, with an equivalent lens plane 116.
  • Furthermore, the lenses generally image the equivalent tilted subject plane 118 onto the image plane 115 such that the PSF of device 100 at the image plane 115 is generally uniform. Optionally, the optics 114 can collimate the light at the equivalent tilted subject plane.
  • In general, the angle 130 and the optics 114 are further selected to both meet the Scheimpflug condition and to meet physical constraints of a projector into which device 100 is integrated.
  • It is further appreciated that in selecting the angle 130 and the optics 114 the illumination angle of the second DMD 102 is considered. In other words, DMDs are generally designed to be operated in a forward configuration with illumination light at a given angle. For example, DLP™ based DMDs have illumination angles of about 24°, while other DLP™ based DMDs have illumination angles of about 34°, though other given angles are within the scope of present implementations. Indeed, a given illumination angle of a DMD selected for use with device 100 is generally used to selecting the angle 130, as well as in selecting the optics 114. Indeed, each of the DMDs 101, 102 can have the same illumination angle, or each of the DMDs 101, 102 can have illumination angle different from one another. However, regardless of the illumination angle of each of the DMDs 101, 102, the positions of each of the DMDs, 101, 102, the position of the at least one optical device 117, including the angle 130, and the optics 114 are selected such that the Scheimpflug condition is met
  • Indeed, the parameters of device 100, including the angle 130 can further be selected using optical modelling software, using a desired physical configuration of the components of the device 100. For example, a certain configuration of the optics 114 may be desired such that the device 100 has a given physical footprint and/or each of the DMDs 101, 102 can have a given set of operational conditions (including, but not limited to an angle of illumination in a forward configuration).
  • In any event, such a configuration of device 100 allows both DMDs 101, 102 to be operated in a forward configuration, which generally allows a more efficient operation of a projector into which device 100 is incorporated, as compared to similar projectors that use a first DMD in a reverse configuration.
  • It is yet further appreciated that while devices 10, 100 include only one optical device configured to: tilt a subject plane of a first DMD to an equivalent tilted subject plane, in yet further implementations, more than one optical device can be used to tilt the subject plane of the first DMD to an equivalent tilted subject plane.
  • For example, attention is next directed to Fig. 4 and Fig. 5, each of which depict a device 400 that can be used with, and/or incorporated into, a high dynamic range ("HDR") projector. Fig. 4 depicts a schematic side view of the device 400, including example optics, and Fig. 5 depicts a schematic side view of the device 400 with the optics of the device 400 unfolded (e.g. reflections from mirrors and/or reflective surfaces are removed for clarity with the optical path through the device 400 depicted without reflections). Furthermore, it is understood that the device 400 is not depicted to scale in either of Fig. 4 or Fig. 5. While device 400 is generally more complex than device 10, device 400 generally operates according to the same principles of device 10, as described hereafter.
  • With reference to both Fig. 4 and Fig. 5, the device 400 generally comprises: a first DMD 401, a second DMD 402, each operated in a forward configuration, such that each is illuminated at a respective non-normal angle and a respective output image is reflected at a normal angle, a subject plane 413 of the first DMD 401 being parallel to the first DMD 401; optics 414 between the first DMD 401 and the second DMD 402, the optics 414 configured to convey light reflected from the first DMD 401 to illuminate an image plane 415 at the second DMD 402 in the forward configuration, the optics 414 including an equivalent lens plane 416; and a plurality of optical devices 417-1, 417-2 between the first DMD 401 and the second DMD 402, the plurality of optical devices 417-1, 417-2 configured to: tilt the subject plane 413 of the first DMD 401 to an equivalent tilted subject plane 418, the equivalent lens plane 416, the equivalent tilted subject plane 418 and the image plane 415 all intersecting at a Scheimpflug intersection (not depicted, but assumed to be nonetheless present, for example off the page of Fig. 5).
  • Device 400 further comprises a respective TIR prism 450-1, 450-2 at each of the DMDs 401, 402 configured to convey illumination light to a respective DMD 401, 402 and convey a respective output image away from the respective DMD 401, 402 (e.g. the illumination light for the second DMD 402 comprising the respective output image of the first DMD 401). Each of the TIR prism 450-1. 450-2 can be a component of the optics 414.
  • Furthermore, the optics 414 otherwise comprises eight lenses symmetrically arranged along an optical path between the DMDs 401, 402; again, it is appreciated that the optics 414 can be modelled using a single lens as in device 10, having the equivalent lens plane 416.
  • It is appreciated that the subject plane 413, the equivalent lens plane 416, the equivalent tilted subject plane 418 and the image plane 415 are depicted in Fig. 5, but not Fig. 4, for clarity. For example, in the unfolded view of device 400, the relative angles between the subject plane 413, the equivalent lens plane 416, the equivalent tilted subject plane 418 and the image plane 415 are clearer as reflections due to mirrors and/or reflecting surfaces are removed.
  • As with at least one optical device 17, the plurality of optical devices 417-1, 417-2 are further configured to: tilt the subject plane 413 of the first DMD 401 to the equivalent tilted subject plane 418 such that the image plane 415 at the second DMD 402 is in uniform focus and/or is in substantially uniform focus at the respective non-normal angle.
  • As depicted, the plurality of optical devices 417-1, 417-2 include a plurality of wedge prisms distributed between the first DMD 401 and the second DMD 402. For example, optical device 417-1 comprises a first wedge prism having an exit face 429-1 angled with respect to the subject plane 418, and optical device 417-2 comprises a second wedge prism having an exit face 429-2 also angled with respect to the subject plane 418. While the first wedge prism (e.g. optical device 417-1) is located before optics 414 (and after the TIR prism 450-1), and the second wedge prism (e.g. optical device 417-2) is located after optics 114 (and before the TIR prism 450-2), it is appreciated that the total effect of the two wedge prisms on the subject plane 413, with respect to the image plane 415, is to tilt the subject plane 413 of the first DMD 401 to the equivalent tilted subject plane 418.
  • Indeed, device 400 demonstrates that any number of optical devices can be used to tilt the subject plane of a first DMD, operated in a forward configuration, to an equivalent tilted subject plane such that the equivalent tilted subject plane, an equivalent lens plane and an image plane intersect at a Scheimpflug intersection and/or meet the Scheimpflug condition.
  • Persons skilled in the art will appreciate that there are yet more alternative implementations and modifications possible, and that the above examples are only illustrations of one or more implementations. The scope, therefore, is only to be limited by the claims appended hereto.

Claims (10)

  1. A device comprising:
    a first digital micromirror device (DMD) (11) and a second DMD (12) each operated in a forward configuration, such that each is illuminated at a respective non-normal angle and a respective output image is reflected at a normal angle, a subject plane (13) of the first DMD (11) being parallel to the first DMD (11);
    optics (14) between the first DMD (11) and the second DMD (12), the optics (14) configured to convey light reflected from the first DMD (11) to illuminate an image plane (15) at the second DMD (12) in the forward configuration, the optics (14) including an equivalent lens plane (16); and
    at least one optical device (17) between the first DMD (11) and the second DMD (12), the at least one optical device (17) configured to: tilt the subject plane (13) of the first DMD (11) to an equivalent tilted subject plane (18), the equivalent lens plane (16), the equivalent tilted subject plane (18) and the image plane (15) all intersecting at a Scheimpflug intersection.
  2. The device of claim 1, wherein the at least one optical device (17) is further configured to: tilt the subject plane (13) of the first DMD (11) to the equivalent tilted subject plane (18) such that the image plane (15) at the second DMD (12) is in uniform focus at the respective non-normal angle.
  3. The device of claim 1, wherein the at least one optical device (17) includes a prism (17) with an exit face at an angle tilted relative to the subject plane (13) of the first DMD (11).
  4. The device of claim 1, wherein the at least one optical device (17) includes a plurality of wedge prisms (417) distributed between the first DMD (11) and the second DMD (12).
  5. The device of claim 1, wherein the at least one optical device (17) includes a prism (117) configured to convey illumination light to the subject plane (13) of the first DMD (11) at the respective non-normal angle and convey the respective output image through an exit face of the prism, the exit face at an angle relative to the subject plane (13), the angle being greater than zero, and the angle selected to optically tilt the subject plane (13) of the first DMD (11) to the equivalent tilted subject plane (18).
  6. The device of any of claims 1 to 5, wherein the optics (14) includes a prism (142) configured to convey illumination light from the first DMD (11) to the image plane (15) of the second DMD (12) at the respective non-normal angle and convey the respective output image of the second DMD (12) through a respective exit face.
  7. The device of any of claims 1 to 6, wherein the optics (14) includes one or more lenses (114).
  8. The device of any of claims 1 to 7, wherein the optics (14) includes one or more apertures (114).
  9. The device of any of claims 1 to 8, wherein the first DMD (11) is further operated in a pre-modulator mode and the second DMD (12) is further operated in a prime-modulator mode.
  10. The device of claim 9, wherein the first DMD (11) is modulated using first image data, and the second DMD (12) is modulated using second image determined from one or more of the first image data, a target projected image and a pixel spread function of the device.
EP18152342.4A 2017-02-16 2018-01-18 Forward-on-forward high dynamic range architecture for digital micromirror devices Active EP3364228B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US15/434,635 US9983402B1 (en) 2017-02-16 2017-02-16 Forward-on-forward high dynamic range architecture for digital micromirror devices

Publications (2)

Publication Number Publication Date
EP3364228A1 true EP3364228A1 (en) 2018-08-22
EP3364228B1 EP3364228B1 (en) 2020-12-16

Family

ID=61024574

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18152342.4A Active EP3364228B1 (en) 2017-02-16 2018-01-18 Forward-on-forward high dynamic range architecture for digital micromirror devices

Country Status (5)

Country Link
US (1) US9983402B1 (en)
EP (1) EP3364228B1 (en)
JP (1) JP7140452B2 (en)
CN (1) CN108449587B (en)
HK (1) HK1252470A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110716372B (en) * 2018-07-12 2021-08-03 深圳光峰科技股份有限公司 DMD adjusting device, adjusting system and multi-DMD optical-mechanical system
US11503274B2 (en) 2019-05-02 2022-11-15 Disney Enterprises, Inc. High speed binary compressive light field projection system
EP4062219A1 (en) * 2019-11-18 2022-09-28 Ricoh Company, Ltd. Projection lens system and image projection apparatus
JP7427926B2 (en) * 2019-11-19 2024-02-06 株式会社リコー Optical system and image projection device
CN115209061B (en) * 2022-09-15 2022-12-16 中国人民解放军国防科技大学 Real-time high-dynamic imaging method and system based on DMD

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3972584A (en) * 1971-09-07 1976-08-03 Redifon Limited Compound optical system with image tilt compensation
US20050185249A1 (en) * 2004-02-12 2005-08-25 Seiko Epson Cororation Light modulation device and optical display device, and light modulation method and image display method
US20070057164A1 (en) * 2003-07-02 2007-03-15 David Vaughnn Scheimpflug normalizer
EP2670144A1 (en) * 2012-05-30 2013-12-04 Christie Digital Systems USA, Inc. Zonal illumination for high dynamic range projection
US20150146175A1 (en) * 2013-11-28 2015-05-28 Christie Digital Systems Canada Inc. Light modulator system including relay optics for correcting optical distortions
US20160073100A1 (en) * 2014-09-10 2016-03-10 Delta Electronics, Inc. Coded illuminator and light field projection device using the same

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4436392A (en) 1983-01-03 1984-03-13 Minnesota Mining And Manufacturing Company Distortion correction for an overhead projector system
GB9001509D0 (en) 1990-01-23 1990-03-21 Davy Mckee Poole An optical imaging device for guiding a robot
US6406150B1 (en) 1999-07-09 2002-06-18 Sarnoff Corporation Compact rear projections system
DE20109394U1 (en) 2001-06-06 2001-08-16 Carl Zeiss Jena Gmbh, 07745 Jena Projection arrangement
JP2006504116A (en) 2001-12-14 2006-02-02 ディジタル・オプティクス・インターナショナル・コーポレイション Uniform lighting system
CN1711778A (en) 2002-12-04 2005-12-21 汤姆森许可贸易公司 Two-stage projector architecture
JP4207797B2 (en) 2004-02-24 2009-01-14 セイコーエプソン株式会社 LIGHT MODULATION DEVICE, OPTICAL DISPLAY DEVICE, LIGHT MODULATION METHOD, AND IMAGE DISPLAY METHOD
JP2007163547A (en) 2005-12-09 2007-06-28 Toppan Printing Co Ltd Projection display device
JP4645486B2 (en) 2006-03-13 2011-03-09 セイコーエプソン株式会社 Image display device and projector
TWI436150B (en) * 2011-02-15 2014-05-01 Asia Optical Co Inc Projector with dual projection function and its external kit
GB2505708B (en) * 2012-09-11 2015-02-25 Barco Nv Projection system with safety detection
US9188767B2 (en) * 2013-11-04 2015-11-17 Christie Digital Systems Usa, Inc. Relay lens system for a high dynamic range projector
US9232172B2 (en) 2013-11-04 2016-01-05 Christie Digital Systems Usa, Inc. Two-stage light modulation for high dynamic range
EP3779593A1 (en) 2014-12-31 2021-02-17 Dolby Laboratories Licensing Corp. Methods and systems for high dynamic range image projectors
JP2017032631A (en) 2015-07-29 2017-02-09 Necディスプレイソリューションズ株式会社 Projector

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3972584A (en) * 1971-09-07 1976-08-03 Redifon Limited Compound optical system with image tilt compensation
US20070057164A1 (en) * 2003-07-02 2007-03-15 David Vaughnn Scheimpflug normalizer
US20050185249A1 (en) * 2004-02-12 2005-08-25 Seiko Epson Cororation Light modulation device and optical display device, and light modulation method and image display method
EP2670144A1 (en) * 2012-05-30 2013-12-04 Christie Digital Systems USA, Inc. Zonal illumination for high dynamic range projection
US20150146175A1 (en) * 2013-11-28 2015-05-28 Christie Digital Systems Canada Inc. Light modulator system including relay optics for correcting optical distortions
US20160073100A1 (en) * 2014-09-10 2016-03-10 Delta Electronics, Inc. Coded illuminator and light field projection device using the same

Also Published As

Publication number Publication date
EP3364228B1 (en) 2020-12-16
CN108449587A (en) 2018-08-24
JP2018156065A (en) 2018-10-04
CN108449587B (en) 2021-03-19
HK1252470A1 (en) 2019-05-24
US9983402B1 (en) 2018-05-29
JP7140452B2 (en) 2022-09-21

Similar Documents

Publication Publication Date Title
EP3364228A1 (en) Forward-on-forward high dynamic range architecture for digital micromirror devices
CN107111216B (en) High contrast discrete input prism for image projector
EP1411733A3 (en) Broad gamut color display apparatus using an electromechanical grating device
US20150042564A1 (en) Projector
JP2003295110A (en) Image display device
US20190166348A1 (en) Optically offset three-dimensional imager
WO2006088735A2 (en) Method and apparatus for combining light paths of like-colored light sources
JP2006065202A (en) Projection type picture display device
GB2567524A (en) Image projection apparatus
TWI427347B (en) Optical imaging system
US20020051094A1 (en) Aperture element for video projector and video projector using the same aperture element
CN110876045B (en) Projection method and projector
US20060092390A1 (en) Projector
US20230388460A1 (en) Digital display system and method
CN110873995B (en) Projection method, projection system and projector
US20090290128A1 (en) Tiled color filter for a projection system
KR20010080590A (en) Image projection system
CN114175624A (en) Control device, projection system, control method, and control program
JP2021081564A (en) Optical system and image projection device
US10798350B2 (en) Split aperture projector/camera
US20230362338A1 (en) Optically reduced display pixel fill factor for increased display resolution
CN111433661B (en) Compact multi-modulator projector
KR20080053792A (en) Projector with laser source
JP2006078645A (en) Optical engine for rear projection type monitor
KR20030048862A (en) Laser display device

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20190131

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602018010725

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G02B0026080000

Ipc: G03B0021140000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 9/31 20060101ALI20200415BHEP

Ipc: G03B 21/00 20060101ALI20200415BHEP

Ipc: G02B 27/10 20060101ALI20200415BHEP

Ipc: G03B 21/14 20060101AFI20200415BHEP

INTG Intention to grant announced

Effective date: 20200514

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

INTC Intention to grant announced (deleted)
GRAR Information related to intention to grant a patent recorded

Free format text: ORIGINAL CODE: EPIDOSNIGR71

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

INTG Intention to grant announced

Effective date: 20201104

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602018010725

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1346140

Country of ref document: AT

Kind code of ref document: T

Effective date: 20210115

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210316

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210317

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1346140

Country of ref document: AT

Kind code of ref document: T

Effective date: 20201216

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20201216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210316

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210416

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602018010725

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210118

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210416

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20210131

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

26N No opposition filed

Effective date: 20210917

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210131

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210118

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210416

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210131

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230516

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20180118

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231221

Year of fee payment: 7

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20231222

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20231220

Year of fee payment: 7

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20231228

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201216