CN110278420B

CN110278420B - Image processing device, display device, image processing and display device and method

Info

Publication number: CN110278420B
Application number: CN201810219380.6A
Authority: CN
Inventors: 李屹; 余新; 胡飞; 郭祖强
Original assignee: Appotronics Corp Ltd
Current assignee: Shenzhen Appotronics Corp Ltd
Priority date: 2018-03-16
Filing date: 2018-03-16
Publication date: 2022-10-04
Anticipated expiration: 2038-03-16
Also published as: CN110278420A; WO2019174273A1

Abstract

The invention relates to an image processing device, a display device, an image processing and displaying device and a method. The image processing device receives original image data of an image to be displayed, acquires a color gamut range and a brightness value of each pixel of the image to be displayed according to the original image data of the image to be displayed, determines a current color gamut range corresponding to the image to be displayed according to the color gamut range of the image to be displayed, calculates a light quantity control signal corresponding to the current color gamut range according to the current color gamut range and the brightness value of each pixel, generates correction image data according to the original image data and the current color gamut range, and jointly encodes the light quantity control signal and the correction image data to generate pre-processing image data.

Description

Image processing device, display device, image processing and display device and method

Technical Field

The present invention relates to the field of display technologies, and in particular, to an image processing apparatus, a display device, an image processing and display apparatus and method.

Background

Light sources of display devices such as laser projectors are generally classified into three major categories, one of which is to excite phosphors of different colors by short-wavelength laser to generate primary colors of red, green and blue. Another type directly uses red, green and blue lasers as the three primary color light sources. The third is the combination of the first two, and a general blue laser light source is used as an excitation light source with short wavelength to excite the fluorescent powder to generate red-green primary color light and also used as blue primary color light. Each of these three different implementation techniques has advantages and disadvantages. For the scheme of exciting the fluorescent powder or mixing the laser fluorescence by the laser, because the semiconductor blue laser with the gallium nitride substrate has the characteristics of high efficiency, long service life and stable work, the scheme of exciting the fluorescent powder color wheel by the blue semiconductor laser has the characteristics of long service life, high efficiency, stable equipment and low cost. However, the relatively narrow color gamut of this approach is due to the broad spectrum of phosphor-excited fluorescence (Laser phosphor). Generally, a display device using this technology can cover the full sRGB color gamut, and can enhance the color gamut to the DCI-P3 color gamut by some enhancement processes, such as adding narrow-band optical filters to remove the yellow light spectrum in the green and red light. But the narrow-band filtering loses considerable luminance of the light, so that the efficiency of the display device is greatly reduced. A display device using pure RGB lasers has a very wide color gamut because RGB lasers have very good monochromaticity. A display device (e.g., a projection system) using RGB lasers can easily meet the REC2020 color gamut standard, and please refer to fig. 1 for a comparison of the color gamuts of the display devices.

However, RGB laser display devices (such as projectors) also have a number of disadvantages. The first is speckle. Speckle is caused by the coherence of laser light, and the light reflected on a display plane interferes due to a phase difference caused by the fluctuation of the plane, resulting in uneven brightness distribution on the display screen. Although there have been many inventions that attempt to solve the problem of laser speckle, none are ideal. Secondly, the cost of the RGB laser display device is high. This is because the red and green lasers in RGB laser display devices are not mature under the current technology. The efficiency of semiconductor green laser can only be below 20%, which is far lower than blue laser of gallium nitride substrate and red laser of ternary substrate, and the cost is very high. While the red laser has almost the same efficiency as the blue laser, the red laser has poor temperature stability, and not only the efficiency is significantly reduced with the increase of temperature, but also the center wavelength is shifted. The two points make the RGB laser display device have color cast along with temperature change. This requires the addition of a thermostat to the red laser to stabilize the operating state of the red laser, which also means that a powerful cooling device is required to ensure that the operating temperature of the red laser is stable, thereby greatly increasing the cost of the RGB laser display device.

A basic light source 200 for laser-excited phosphor wheel is shown in fig. 2 (as disclosed in chinese patent application CN 201110424486.8), and a short-wavelength visible light emitted from an excitation light source 210 excites a phosphor on a color wheel 220 to generate a time-sequential primary color light or white light. The wide spectrum of fluorescence makes the gamut coverage based on this system relatively narrow. An improved method for enhancing color gamut is shown in fig. 3 (as disclosed in chinese patent application CN 201110191454.8). The short wavelength visible light emitted from the excitation light source 310 is converted into primary light by the color wheel 320 and filtered by the synchronization filter 330 to obtain primary light with higher narrow band purity to expand the color gamut of the laser fluorescence. The filter device causes additional optical power loss and reduces the efficiency of the display device.

The color gamut of the light source can be expanded by doping pure red and green laser into laser fluorescence. Optical systems for incorporating a pure color laser into laser fluorescence have been reported, such as an implementation scheme of incorporating a pure color laser into a laser fluorescence system proposed in a prior art (such as that disclosed in U.S. patent application No. US20150316775 A1), and an implementation scheme of incorporating one or two optical paths proposed in another prior art (such as that disclosed in chinese patent application No. CN 201110191454.8). Although the color gamut of laser fluorescence can be expanded by doping pure-color laser, the color gamut which can be enhanced by the pure-color laser is limited due to no modulation of the light source ratio according to the display content. As shown in fig. 4, on the basis of the mixed light (mix gamit) added with the pure color laser (shown in fig. 4 a) with 20% fluorescence brightness, if the color gamut of the laser fluorescence needs to be expanded to the DCI-P3 standard, the mixed light is formed by adding the pure color laser (shown in fig. 4 b) with 40% fluorescence brightness. The display device of this scheme is more efficient than the phosphor plus filter scheme, but the need to add a powerful red-green laser leads to increased system cost.

Disclosure of Invention

In order to solve the technical problem that the light source cost of the existing wide color gamut display equipment is high, the invention provides an image processing device, display equipment, an image processing and display device and method which can realize a wide color gamut and are low in light source cost.

An image processing apparatus, the image processing apparatus comprising:

the data receiving module is used for receiving original image data of an image to be displayed;

the first calculation module is used for acquiring the color gamut range of the image to be displayed and the brightness value of each pixel according to the original image data of the image to be displayed;

the second calculation module is used for determining the current color gamut range corresponding to the image to be displayed according to the color gamut range of the image to be displayed;

the light quantity signal generating module is used for calculating a light quantity control signal corresponding to the current color gamut range according to the current color gamut range and the brightness value of each pixel;

the data conversion module is used for generating corrected image data according to the original image data and the current color gamut range; and

and the coding module is used for coding the light quantity control signal and the corrected image data together to generate preprocessed image data.

An image processing method, comprising the steps of:

receiving original image data of an image to be displayed;

acquiring the color gamut range and the brightness value of each pixel of the image to be displayed according to the original image data of the image to be displayed;

determining a current color gamut range corresponding to the image to be displayed according to the color gamut range of the image to be displayed;

calculating a light quantity control signal corresponding to the current color gamut range according to the current color gamut range and the brightness value of each pixel;

generating corrected image data according to the original image data and the current color gamut range; and

the light amount control signal and the correction image data are collectively encoded to generate pre-processing image data.

A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned image processing method.

A display method, comprising the steps of:

receiving pre-processing image data and decoding the pre-processing image data to obtain corrected image data and a light quantity control signal;

emitting a first light and a second light according to the light quantity control signal, wherein the light quantities of the first light and the second light are controlled by the light quantity control signal, the first light is used for modulating an image in a first color gamut range, the second light is used for modulating an image out of the first color gamut range separately or together with the first light, the color gamut range of the second light is a second color gamut range, and the second color gamut range covers the first color gamut range and has a part out of the first color gamut range; and

and modulating the first light and the second light according to the corrected image data to generate image light of an image to be displayed.

A display apparatus, comprising:

an image decoder for receiving the pre-processed image data and decoding the pre-processed image data to obtain corrected image data and a light quantity control signal;

a light source device for emitting a first light and a second light according to the light quantity control signal, wherein the light quantities of the first light and the second light are controlled by the light quantity control signal, the first light is used for modulating an image in a first color gamut range, the second light is used for modulating an image outside the first color gamut range separately or together with the first light, the color gamut range of the second light is a second color gamut range, and the second color gamut range covers the first color gamut range and has a portion beyond the first color gamut range;

and the light modulation device is used for modulating the light emitted by the light source device according to the corrected image data so as to generate image light of an image to be displayed.

An image processing and display method, comprising the steps of:

receiving original image data of an image to be displayed;

generating corrected image data according to the original image data and the current color gamut range;

jointly encoding the light quantity control signal and the correction image data to generate pre-processed image data;

receiving the pre-processing image data and decoding the pre-processing image data to obtain the correction image data and the light quantity control signal;

emitting a first light and a second light according to the light quantity control signal, wherein the light quantities of the first light and the second light are controlled by the light quantity control signal, the first light is used for modulating an image in a first color gamut range, the second light is used for modulating an image out of the first color gamut range separately or together with the first light, the color gamut range of the second light is a second color gamut range, and the second color gamut range covers the first color gamut range and has a part beyond the first color gamut range; and

An image processing and display device, the image processing and display device comprising:

the image processing device is used for receiving original image data of an image to be displayed, acquiring a color gamut range and a brightness value of each pixel of the image to be displayed according to the original image data of the image to be displayed, determining a current color gamut range corresponding to the image to be displayed according to the color gamut range of the image to be displayed, calculating a light quantity control signal corresponding to the current color gamut range according to the current color gamut range and the brightness value of each pixel, generating correction image data according to the original image data and the current color gamut range, and encoding the light quantity control signal and the correction image data together to generate pre-processing image data;

an image decoder for receiving the pre-processed image data and decoding the pre-processed image data to obtain the corrected image data and the light quantity control signal;

a light source device for emitting a first light and a second light according to the light quantity control signal, wherein the light quantities of the first light and the second light are controlled by the light quantity control signal, the first light is used for modulating an image in a first color gamut range, the second light is used for modulating an image outside the first color gamut range separately or together with the first light, the color gamut range of the second light is a second color gamut range, and the second color gamut range covers the first color gamut range and has a portion beyond the first color gamut range; and

Compared with the prior art, in the image processing device, the display device, the image processing and displaying device and the image processing and displaying method, the current color gamut range and the light quantities of the first light and the second light corresponding to the current color gamut range are determined according to the color gamut range of the image to be displayed and the brightness value of each pixel, and the first light and the second light with the light quantities corresponding to the light quantities are modulated according to the correction image data, so that not only can the display of the image data of a wide color gamut be realized, but also the wide color gamut light (namely the second light) can be used to the minimum extent by adjusting the light quantities of the first light and the second light according to the current color gamut range, the use of the second light is reduced, and further, the light source cost is reduced. In addition, the use of the second light is reduced, the power and the heat dissipation requirements of the light source device are reduced to a certain extent, a complex heat dissipation system is not needed, and the cost can be reduced.

Furthermore, in the image processing apparatus, the display device, the image processing and displaying apparatus and method of the present invention, the corrected image data and the light quantity control signal are jointly encoded and transmitted, and then decoded and used after being received, so that the image processing apparatus, the display device, the image processing and displaying apparatus and method of the present invention are easily applicable to the existing system, and the applicability of the image processing apparatus, the display device, the image processing and displaying apparatus and method of the present invention is improved.

Drawings

Fig. 1 is a comparison of the gamut ranges of several display devices using different light sources.

Fig. 2 is a schematic diagram of a light source structure of a related art display device.

Fig. 3 is a schematic view of a light source structure of another prior art display device.

Fig. 4a and 4b are schematic diagrams of the color gamut of the display device shown in fig. 2 and 3 with different proportions of pure color laser added.

FIG. 5 is a schematic structural diagram of an image processing and display device according to a preferred embodiment of the invention.

Fig. 6 is a schematic diagram of the current gamut range of the display device shown in fig. 5.

Fig. 7 is a schematic diagram of the operation of the image processing apparatus of the display device shown in fig. 5.

FIG. 8 is a flowchart illustrating an image processing and displaying method according to a preferred embodiment of the present invention.

FIG. 9 is a block diagram of a preferred embodiment of the display device shown in FIG. 5.

Description of the main elements

Image processing and display device 500

Light source device 510

Light source driving circuit 520

Gamma correction circuit 530

Image processing apparatus 540

Light modulation device 550

Image synthesizing apparatus 560

Excitation light source 511

Color wheel 512

Supplemental light source 513

Light splitting and combining device 514

Dodging device 515

Relay lens 516

Decoding circuit 570

Display device 580

Gamut ranges F1, F2, F3, F4

Control chip 551

Modulator 552

Beam splitting module 561

Projection lens 562

Steps S1-S9

Data receiving module 541

First calculation module 542

Second computing module 543

Light quantity signal generation module 544

Data conversion module 545

Encoding module 546

The following detailed description will further illustrate the invention in conjunction with the above-described figures.

Detailed Description

Based on the adoption of the wide color gamut light source (such as an R, G and B three-color pure laser light source) or the mixed light source of the wide color gamut light source and the fluorescent light source, the display device can realize the color gamut standard of Rec.2020, but has the technical problems of higher cost and the like (such as high price of red laser and green laser and lower electro-optical conversion efficiency), and the invention provides the display device and the display method which can reduce the use of the wide color gamut light source.

In one embodiment, the present invention provides a method for dynamically enhancing color gamut by adding laser light emitted from a wide color gamut light source to fluorescent light, so that the display device can reach and exceed the DCI-P3 color gamut standard while maintaining high efficiency. Furthermore, the scheme of dynamically adjusting the brightness of the first light (such as fluorescent light) and the second light (such as red laser and green laser) according to the content of the displayed image, provided by the invention, can greatly enhance the display color gamut of the system without changing the second light, and can greatly reduce the required power of the second light, reduce the use of a wide color gamut light source and reduce the cost of the display device under the condition of keeping the color gamut of the display device not to change much. Specifically, the brightness and color gamut of light emitted by the light source device of the display device are changed along with image data to be displayed, so that the light source device does not always work at the maximum power, the energy consumption and the heat dissipation burden of the device are reduced, and the cost of the heat dissipation device and the constant temperature device is reduced.

The following describes the detailed structure and principle of the display device and the display method of the present invention in detail with reference to the accompanying drawings. Referring to fig. 5, fig. 5 is a schematic structural diagram of an image processing and display device 500 according to a preferred embodiment of the invention. The image processing and display apparatus 500 includes an image processing apparatus 540 and a display device 580. The image processing apparatus 540 may be a device disposed outside the display device 580, such as a computer or a computer image device disposed outside the display device 580, and is configured to receive the original image data and output the preprocessed image data. The display device 580 may be a projector for projecting an image, and the display device 580 may be in data communication with the image processing apparatus 540 in a wired or wireless manner, may receive the pre-processed image data output by the image processing apparatus 540, and may display an image according to the projection. The display apparatus 580 includes a decoding circuit 570, a light source device 510, a light source driving circuit 520, a gamma correction circuit 530, a light modulation device 550, and an image synthesizing device 560.

The light source driving circuit 520 is electrically connected to the light source device 510, and is configured to send a driving signal to drive the light source device 510 to emit light. Specifically, the light source device 510 is configured to emit light source light, which may include a first light and a second light, wherein the first light is used to modulate an image in a first color gamut range F1, a color gamut range of the second light is wider than that of the first light, the second light is used to modulate an image outside the first color gamut range F1 alone or together with the first light, a color gamut range of the second light is a second color gamut range F2, and the second color gamut range F2 covers the first color gamut range F1 and has a portion beyond the first color gamut range F1. Specifically, referring to fig. 6, fig. 6 is a schematic diagram of the gamut ranges of the image processing and display apparatus 500 shown in fig. 5, where the first gamut range F1 is a gamut range that can be exhibited by the first light, such as a DCI gamut range, for example, DCI-P3, and the second gamut range F2 is a REC gamut range, for example, a gamut range rec.2020.

It is understood that the first light and the second light may each include at least two colors of light, such as red, green and blue. Specifically, the light source device 510 includes an excitation light source 511, a color wheel 512, a supplementary light source 513, and a light splitting and combining device 514, the excitation light source 511 emits excitation light, which may be a first color light (e.g., blue light), the color wheel 512 has a fluorescent material and receives the excitation light to generate fluorescent light, the color wheel 512 is configured to receive the excitation light and emit a first light, and the first light emitted by the color wheel 512 includes the first color light and the fluorescent light. The supplemental light source 513 is configured to emit supplemental light, which may include laser light, fluorescent light containing the same primary color component as the supplemental light, the supplemental light being the second light as at least a portion of the excitation light emitted by the excitation light source 511 or the supplemental light alone. It is understood that, in the present embodiment, the light source driving circuit 520 can control the first light and the second light emitted by the light source device 510 by emitting driving signals to the excitation light source 511 and the supplemental light source 513.

Further, the fluorescent light includes a second color fluorescent light (e.g., red fluorescent light) and a third color fluorescent light (e.g., green fluorescent light) or the fluorescent light includes a fourth color fluorescent light (e.g., yellow fluorescent light) in which the second color and the third color (e.g., red and green) are mixed, and the supplementary light includes a second color laser light (e.g., red laser light) and a third color laser light (e.g., green laser light), and the first color, the second color and the third color are three primary colors.

In this embodiment, the excitation light source 511 may be a blue laser light source for emitting blue excitation light, the color wheel 512 may include at least two segment regions sequentially arranged along a circumferential direction, such as a blue segment region, a yellow segment region, or a blue segment region, a red segment region, and a green segment region, wherein the blue segment region may be provided with a scattering material, at least one segment region is provided with a fluorescent material, for example, the yellow segment region is provided with a yellow fluorescent material, or the red and green segment regions are respectively provided with a red and green fluorescent materials, and each segment region emits one color light, for example, the blue segment region scatters the blue excitation light to emit blue light, the yellow segment region emits yellow light, or the red and green segment regions respectively emit red and green lights, and further the at least two segment regions may emit at least two color lights, such as blue and yellow lights, or blue, red and green lights. Therein, it is understood that the yellow light contains components of red and green light, i.e. the first light may comprise the three primary colors red, green and blue.

Further, the excitation light emitted from the excitation light source 511 enters the color wheel 512 through the light splitting and combining device 514 (for example, through transmission), the color wheel 512 rotates along the circumferential direction during operation, so that each of the segment regions is sequentially located on the light path where the excitation light is located, one segment region of the color wheel 512 guides (for example, reflects) the received excitation light to the light splitting and combining device 514 in the first period, and the other segment region or two segment regions of the color wheel 512 also receives the excitation light to generate fluorescence and reflects the fluorescence to the light splitting and combining device 514 in the other period, the excitation light and the fluorescence emitted from the color wheel 512 to the light splitting and combining device 514 serve as first light, and the light splitting and combining device 514 further provides (for example, reflects) the first light to the light modulation device 550. The color wheel 512 may be a reflective color wheel, and the light splitting and combining device 514 may include a light splitting and combining membrane.

Further, in an alternative embodiment, the excitation light source 511 and the color wheel 512 may also be replaced by light emitting diodes, that is, the light emitting diodes emit first light having fluorescence, and specifically, the first light may include at least two color lights, such as blue light and yellow light or blue light, red light and green light.

The supplemental light may include laser light, and specifically, the supplemental light source 513 may include at least two color lasers, such as a red laser and a green laser, for emitting at least two color lasers, such as a red laser and a green laser, when the supplemental light (such as the red laser and the green laser) and at least a portion of the excitation light (blue excitation light) emitted by the excitation light source 511 are used as the second light. In a modified embodiment, the supplemental light also includes laser light, and the supplemental light source 513 may include lasers of at least two colors, such as a blue laser, a red laser, and a green laser, for emitting laser light of at least two colors, such as a blue laser, a red laser, and a green laser, in which case the supplemental light (such as a blue laser, a red laser, and a green laser) is used as the second light. It is understood that, in a modified embodiment, the color of the supplemental light emitted by the supplemental light source can be selected according to actual needs, and specifically, the supplemental light source can also include a laser emitting a color, such as a red laser emitting red light, a green laser emitting green light, or a yellow light emitting yellow light.

In this embodiment, the supplemental light source 513 further emits supplemental light to the color wheel 512, one of the segment regions of the color wheel 512 further receives excitation light in a second time period different from the first time period, the color wheel 512 further guides the supplemental light and the excitation light received in the second time period to the light splitting and combining device 514 as second light, and the light splitting and combining device 514 guides (e.g., reflects) the second light to the light modulation device 550. Specifically, the color wheel 512 may transmit the complementary light to the light splitting and combining device 514, and the color wheel 512 reflects the excitation light to the light splitting and combining device 514 in the second period.

In a modified embodiment, when the supplemental light source 513 emits the supplemental light as the second light, one of the segment regions of the color wheel 512 may transmit the supplemental light to the light splitting and combining device 514, and the light splitting and combining device 514 may guide the second light to the light modulation device 550.

Specifically, the light source device 510 may further include a light homogenizing device 515 (such as a light homogenizing square bar) and a relay lens 516, the relay lens 516 may be disposed between the supplemental light source 513 and the color wheel 512, between the color wheel 512 and the light splitting and combining device 514, and the light homogenizing device 515 may be disposed between the light splitting and combining device 514 and the light modulation device 550, and is configured to homogenize the first light and the second light and guide the homogenized first light and second light to the light modulation device 550.

The image processing apparatus 540 is configured to receive original image data of an image to be displayed, obtain a color gamut range and a brightness value of each pixel of the image to be displayed according to the original image data of the image to be displayed, determine a current color gamut range corresponding to the image to be displayed according to the color gamut range of the image to be displayed, calculate a light quantity control signal corresponding to the current color gamut range according to the current color gamut range and the brightness value of each pixel, generate corrected image data according to the original image data and the current color gamut range, encode the light quantity control signal and the corrected image data together to generate pre-processed image data, and provide the pre-processed image data to the display device 580. The preprocessed image data may have a preset format, and the preset format may be VGA, DP or DVI format.

After the display device 580 receives the preprocessed image data, the decoding circuit 570 decodes the preprocessed image data to obtain the light amount control signal and the corrected image data. The light quantity control signal is used for controlling the light quantities of the first light and the second light emitted by the light source device 510, and specifically, the light quantity control signal is provided to the gamma correction circuit 530, the gamma correction circuit 530 outputs a correction signal to the light source driving circuit 520 according to the light quantity control signal, and the light source driving circuit 520 emits corresponding driving signals to the light source device 510 according to the correction signal to control the light quantities of the first light and the second light emitted. The decoding circuit 570 further provides the corrected image data obtained by decoding to the light modulation device 550, the light modulation device 550 modulates the first light and the second light emitted from the light source device 510 according to the corrected image data to generate image light, and the image synthesis device 560 projects the image light generated by the light modulation device 550 to display an image.

It is to be understood that, in this embodiment, the display device 580 may further include an interface circuit, and the interface circuit (e.g., VGA interface circuit, HDMI interface circuit, DP interface circuit, or DVI interface circuit, etc.) may receive the pre-processed image data in a specific format (e.g., VGA, DP, or DVI) from the image processing apparatus; the display device may further include a data processing circuit, which may be disposed between the decoding circuit and the light modulation device, and the data processing circuit may perform the steps of keystone correction, edge blending, uniformity correction, and the like on the decoded corrected image data and then supply the corrected image data to the light modulation device 540. Of course, in the modified embodiment, the data processing circuit may be omitted as needed, and the decoding circuit 570 may directly supply the decoded corrected image data to the image processing apparatus 540.

Specifically, the light amount control signal may include a first control signal for controlling the light amount of the first light and a second control signal for controlling the light amount of the second light. The image processing apparatus 540 may obtain a brightness value of each pixel of the image to be displayed according to the original image data of the image to be displayed, generate a first control signal to control the light amount of the first light according to the maximum brightness value of each pixel of the image to be displayed, and generate a second control signal according to the maximum brightness value of each pixel of the image to be displayed and the current color gamut range. However, since the first control signal is obtained according to the maximum brightness value of each pixel of the image to be displayed, in an alternative embodiment, the image processing apparatus 540 may also generate the second control signal according to the first control signal and the current color gamut range.

The light quantity control signal is used for controlling the excitation light emitted from the excitation light source 511 and the supplement light emitted from the supplement light source 513 to control the light quantities of the first light and the second light. It is understood that the light amount may refer to the light amount within the image modulation time (i.e., the image display time) of the image to be displayed, and that, in the case where the supply time of the first light and the second light is determined, the control of the light amount may be achieved by controlling the brightness (i.e., the light intensity) of the first light and the second light.

Further, in the present embodiment, the raw image data of each pixel of the image to be displayed is in the RGB encoding format, but it is understood that in the modified embodiment, the raw image data of each pixel of the image to be displayed is not limited to the RGB encoding format, and may be in the YUV encoding format, for example. Further, the original image data of each pixel of the image to be displayed may include three primary color original image data, such as red original image data r, green original image data g, and blue original image data b, where in one embodiment, r, g, and b may be represented by gray scale values, such as the original image data r, g, and b of any one pixel may be gray scale values 100, 120, and 150, respectively.

Further, the original image data of each pixel of the image to be displayed has a color gamut range to which the original image data of each pixel of the image to be displayed belongs, and the color gamut range information to which the original image data of each pixel of the image to be displayed belongs is known or can be known. In this embodiment, the original image data of each pixel of the image to be displayed may be image data of a wider color gamut range, such as image data of a second color gamut range, that is, image data of a REC color gamut range.

Wherein, the color gamut range of the original image data of each pixel of the image to be displayed is three vertexes r under the xyz coordinate system ₀ 、g ₀ 、b ₀ Color coordinate (x) of _r ,y _r ,Y _r )、(x _g ,y _g ,Y _g )、(x _b ,y _b ,Y _b ) Can be expressed using the following equation 1:

it is understood that the xyz coordinate system can be defined in the CIE 1931 standard, and CIE 1931 defines the absolute color and the brightness of the color that any human eye can distinguish in a three-dimensional vector, which does not change with the change of the color gamut. As described above, the gamut range information to which the original image data of each pixel of the image to be displayed belongs is known or can be known, that is, the gamut ranges to which the original image data of each pixel of the image to be displayed belongs are determined by three vertexes r under the xyz coordinate ₀ 、g ₀ 、b ₀ Color coordinate (x) _r ,y _r ,Y _r )、(x _g ,y _g ,Y _g )、(x _b ,y _b ,Y _b ) Are known or may be known. For example, if the original image data of each pixel of the image to be displayed is the image data of the rec.2020 color gamut range, the three vertices r are determined according to the standard of the rec.2020 color gamut range ₀ 、g ₀ 、b ₀ Coordinate (x) of _r ,y _r ,Y _r )、(x _g ,y _g ,Y _g )、(x _b ,y _b ,Y _b ) Are respectively provided withAre (0.708, 0.292, 0.2627), (0.17, 0.797, 0.6780), (0.131, 0.046, 0.0593).

Further, the tristimulus values X, Y, Z of the pixels calculated according to the original image data r, g, b of any one pixel of the image to be displayed are as shown in formula 2:

the conversion matrix C is a conversion matrix required for calculating the tristimulus values X, Y, and Z according to the original image data of each pixel of the image to be displayed and the color gamut range information to which the original image data belongs, and it conforms to the following formula 3:

specifically, in one embodiment, the color gamut range information based on which the original image data of the image to be displayed is based may include the conversion matrix C, that is, in addition to the original image data of three primary colors, the original image data of the image to be displayed may store the conversion matrix C as the color gamut range information based on which the original image data of the image to be displayed is based, but in a modified embodiment, the color gamut range information based on which the original image data of the image to be displayed is based may also be three vertexes r ₀ 、g ₀ 、b ₀ Color coordinate (x) _r ,y _r ,Y _r )、(x _g ,y _g ,Y _g )、(x _b ,y _b ,Y _b ) The information or the specific character or code representing the gamut range information, etc. is not limited to the above.

Further, according to the above formulas 1, 2 and 3, the original image data r, g and b of any pixel of the image to be displayed and the color gamut range information thereof, i.e. the three vertexes r ₀ 、g ₀ 、b ₀ Color coordinate (x) of _r ,y _r ,Y _r )、(x _g ,y _g ,Y _g )、(x _b ,y _b ,Y _b ) The tristimulus values X, Y, Z of the pixel may be calculated, and in the tristimulus values X, Y, Z, Y represents the luminance value of the pixel, and the relationship between the tristimulus values X, Y, Z and the color coordinate xy conforms to the following formula 4:

further, according to the above formulas 1-4, the original image data r, g, b of any pixel of the image to be displayed and the color gamut range information (x) thereof _r ,y _r ,Y _r )、(x _g ,y _g ,Y _g )、(x _b ,y _b ,Y _b ) CIE xyY chromaticity value data of the pixels, i.e. the color coordinates x, Y and luminance value Y of each pixel, may be obtained.

Further, for the image processing and displaying apparatus 500 (or for the display device 580), it has a default color gamut range, that is, the light modulation apparatus 550 generally needs to store a color gamut conversion formula T corresponding to the default color gamut range, where the default color gamut range of the image processing and displaying apparatus 500 generally matches with the color gamut range of the default light output of the light source apparatus 510, when the light modulation apparatus 550 operates, the received image data is converted into image data for modulation by using the color gamut conversion formula T, and the default light output emitted by the light source apparatus 510 is further modulated according to the image data for modulation, so that an image can be accurately displayed, and the displayed image meets the color gamut requirement. Generally, the color gamut conversion formula T stored in the light modulation device 550 is fixed, for example, the color gamut conversion formula T may be stored in the light modulation device 550 in advance during the manufacturing process of the image processing and display device 500 (e.g., the display device 580), so that the image processing and display device 500 (e.g., the display device 580) may generate image data for modulation using the color gamut conversion formula T when operating normally. Let three vertexes r of the gamut range of the light supplied from the light source device 510 to the light modulation device 550 ₀ ’、g ₀ ’、b ₀ ' the color coordinates are respectively (x) _r ’,y _r ’,Y _r ’)、(x _g ’,y _g ’,Y _g ’)、(x _b ’,y _b ’,Y _b '), it is understood that the default gamut of the image processing and display apparatus 500 is related to the gamut of the tricolor light emitted from the light source device 510, i.e. when the tricolor light emitted from the light source device is fixed for a display device, the default gamut of the display device is known, i.e. the gamut of the tricolor light emitted by the light source device by default, so the three vertexes r are all the same ₀ ’、g ₀ ’、b ₀ ' color coordinate (x) _r ’,y _r ’,Y _r ’)、(x _g ’,y _g ’,Y _g ’)、(x _b ’,y _b ’,Y _b ') is also the vertex of the default gamut range of the display device and can be obtained by measuring the gamut range of the tricolor light emitted by its light source means. For example, if the default gamut range of the tricolor light emitted by the light source device 510 of the image processing and display device 500 is the rec.2020 gamut range, the default gamut range of the image processing and display device 500 is the rec.2020 gamut range, and further, according to the standard of the rec.2020 gamut range, the three vertices r are r ₀ ’、g ₀ ’、b ₀ ' coordinate (x) _r ’,y _r ’,Y _r ’)、(x _g ’,y _g ’,Y _g ’)、(x _b ’,y _b ’,Y _b ') are (0.708, 0.292, 0.2627), (0.17, 0.797, 0.6780), (0.131, 0.046, 0.0593), respectively.

Further, the image processing and display device 500 has three vertices r of the default gamut range ₀ ’、g ₀ ’、b ₀ ' color coordinate (x) _r ’,y _r ’,Y _r ’)、(x _g ’,y _g ’,Y _g ’)、(x _b ’,y _b ’,Y _b ') can be expressed using the following equation 5:

further, the tristimulus values X, Y, Z of the pixels calculated from the image data r ', g ', b ' of any one of the pixels corresponding to the default gamut range of the display device 500 by the image processing are as shown in formula 6:

the conversion matrix C' is a conversion matrix required for calculating the corresponding tristimulus values X, Y, Z according to the image data of any one pixel corresponding to the default color gamut range, which is recorded by the light modulation device 550, and it conforms to the following formula 7:

since the tristimulus values X, Y, Z of the pixels remain unchanged regardless of the gamut range corresponding to the image data of any one pixel, the relationship between the original image data r, g, b of any one pixel and the image data r ', g ', b ' of the pixels corresponding to the default gamut range of the image processing and display device 500 satisfies the following formula 8 according to the above formulas 1 to 6:

as can be seen from the above description, for a display device, it is generally necessary to convert the image data (e.g. the original image data r, g, b) received by the light modulation device 550 into the image data r ', g', b 'and further modulate the light emitted from the light source device according to the image data r', g ', b' to accurately generate the image light. It follows that the color gamut conversion formula T stored in the light modulation device 550 to convert the received image data of any one pixel into image data whose image processing corresponds to the default color gamut range of the display device 500 may conform to the following formula:

T＝C′ ^-1 c (formula 9).

According to the above formulas 8 and 9, after the original image data r, g, b is outputted to the light modulation device, the light modulation device 550 can calculate the image data r ', g ', b ' according to the color gamut conversion formula T, and further modulate the corresponding light source light to generate accurate image light. In other words, since the light modulation device 550 needs to store a fixed color gamut conversion formula T for the default color gamut range of the image processing and display device 500, the image processing device 540 needs to take the influence of the color gamut conversion formula T stored in the light modulation device 550 into consideration when converting the original image data into the corrected image data. How the image processing apparatus 540 obtains the corrected image data, the light quantity control signal and the pre-processed image data is further described below with reference to fig. 7.

Referring to fig. 7, fig. 7 is a schematic diagram illustrating an operation of the image processing apparatus 540 of the image processing and display apparatus 500 shown in fig. 5. The image processing apparatus 540 may convert the raw image data (e.g., r, g, b) of each pixel of the image to be displayed into CIE xyY chromaticity value data using the above equations 1, 2, and 3, wherein the CIE xyY chromaticity value data of each pixel includes color coordinates x, Y and a luminance value Y. According to the CIE xyY chromaticity value data of each pixel, i.e. the color coordinates x, Y and the luminance value Y, the image processing apparatus 540 obtains the color coordinates (i.e. the color coordinates x, Y) of each pixel of the image to be displayed, and further obtains the range defined by the color coordinates of each pixel of the image to be displayed, i.e. the color gamut range of the image to be displayed. Further, the image processing apparatus 540 further obtains the luminance value Y of each pixel of the image to be displayed according to the CIE xyY chromaticity value data of each pixel, so that the image processing apparatus 540 can generate the first control signal to control the luminance of the first light emitted by the light source apparatus 510 according to the maximum luminance value among the luminance values Y of each pixel of the image to be displayed, so as to control the light amount of the first light.

Specifically, the image processing apparatus 540 calculates a brightness value of the first light according to a maximum brightness value in each pixel of the image to be displayed, and generates a first control signal for controlling the light amount of the first light according to the brightness value of the first light. It is understood that, in one embodiment, the larger the maximum brightness value is, the larger the light quantity of the first light may be, i.e. the two may be in a proportional relationship.

Further, the image processing apparatus 540 determines the current color gamut range according to the color gamut range of the image to be displayed, where the current color gamut range is a triangular region that covers the color gamut range of the image to be displayed, that is, the current color gamut range covers the color coordinates of each pixel of the image to be displayed, and specifically, the current color gamut range may be a color gamut region that just covers the color coordinates of each pixel of the image to be displayed and has the smallest area. It can be understood that, since the content of each image to be displayed is different, the color gamut of each image to be displayed (e.g., a frame of image to be displayed) may also be different, and thus the current color gamut determined by the image processing apparatus 540 according to each image to be displayed may also be different. Setting the three vertexes r of the current color gamut range determined by the image processing device 540 according to the original image data of the image to be displayed ₀ ”、g ₀ ”、b ₀ "has color coordinates of (x) _r ”,y _r ”,Y _r ”)、(x _g ”,y _g ”,Y _g ”)、(x _b ”,y _b ”,Y _b ") and three vertices r of the current gamut range ₀ ”、g ₀ ”、b ₀ "color coordinate (x) _r ”,y _r ”,Y _r ”)、(x _g ”,y _g ”,Y _g ”)、(x _b ”,y _b ”,Y _b ") can be expressed using the following equation 10:

further, the tristimulus values X, Y, Z of the pixels calculated according to the image data r ", g", b "of any one pixel corresponding to the current gamut range are shown in formula 11:

wherein, the conversion matrix C ″ is a conversion matrix required for calculating the corresponding tristimulus values X, Y, Z according to the image data of any one pixel corresponding to the current color gamut range, and it conforms to the following formula 12:

specifically, the image processing apparatus may determine the current color gamut range, that is, may determine the current color gamut range information corresponding to the current color gamut range, and in an embodiment, the current color gamut range information may include a conversion matrix C ″ required for calculating the tristimulus values X, Y, Z corresponding to the image data of any one pixel corresponding to the current color gamut range, but in an alternative embodiment, the color gamut range information based on which the original image data of the image to be displayed is based may also be three vertices r ₀ ’、g ₀ ’、b ₀ ' color coordinate (x) _r ’,y _r ’,Y _r ’)、(x _g ’,y _g ’,Y _g ’)、(x _b ’,y _b ’,Y _b ') information, or specific characters or codes representing the color gamut range information, etc., are not limited to the above.

Specifically, according to formula 2, calculating a conversion matrix C required by the corresponding tristimulus values X, Y, Z according to the original image data r, g, b of each pixel of the image to be displayed and the color gamut range information to which the original image data r, g, b belongs; according to formula 6, calculating a conversion matrix required by the corresponding tristimulus values X, Y and Z according to the image data r ', g ' and b ' of any pixel corresponding to the default color gamut range; according to the formula 11, the tristimulus values X, Y, Z of the pixels calculated according to the image data r ", g", b "of any one pixel of the current color gamut range are required to be the transformation matrix C"; the image processing device 540 converts the original image data r, g, b of the pixels into the corresponding corrected image data r ' ", g '", b ' "according to the following formula 13:

further, after the light modulation device 550 receives the corrected image data r ' ", g '", b ' ", the image data r", g ", b" (where r ", g", b "are also the image data for modulation by the light modulation device 55) corresponding to the current gamut range calculated according to the stored gamut conversion matrix T will conform to the following formula 14:

furthermore, according to the formula 14, the light modulation device 550 receives the corrected image data r ' ", g '", b ' "and further calculates to obtain the image data r", g ", b" corresponding to the current color gamut range by using the internally stored fixed color gamut conversion formula T, and the light modulation device 550 further modulates the required light source light according to the image data r ", g", b "to accurately restore the pixels of the image to be displayed.

According to the above-described principles, in this embodiment, the image processing apparatus 540 may calculate the correction conversion formula based on the determined current color gamut range information (e.g. the conversion matrix C), the color gamut range information (e.g. the conversion matrix C) based on the original image data of the image to be displayed, and the color gamut conversion formula T of the image processing and display apparatus 500, wherein the correction conversion formula is C' C ″, according to formula 13 ^-1 The image processing device 540 further converts the original image data r, g, b of the image to be displayed into corrected image data r '", g'", b '"according to a correction conversion formula, and the light modulation device 550 converts the corrected image data r'", g '", b'" into image data r ", g", b "corresponding to the current color gamut range according to a color gamut conversion formula T.

Further, according to the formula 13, the image processing apparatus 540 needs to know the transformation matrix C, the transformation matrix C' and the transformation matrix C ″ according to the received original image data of the image to be displayed, wherein according to the analysis, the transformation matrix C is determined by the color gamut range to which the original image data of the image to be displayed belongs, and since the original image data of the image to be displayed is known, the color gamut range to which the original image data of the image to be displayed belongs is also known, so that the transformation matrix C is known. The conversion matrix C' is determined by the default gamut of the image processing and display device 500, i.e., the conversion formula T stored inside the light modulation device 550, and thus is also known. Further, when the image processing apparatus 540 determines the current color gamut range according to the original image data of the image to be displayed, i.e. knows three vertices of the current color gamut range, the transformation matrix C ″ is also known, and when C, C ', and C ″ are known, the corrected image data r' ", g '", b' "can be calculated and known according to the original image data r, g, b.

In the present embodiment, the first color gamut range, the second color gamut range, and the current color gamut range are all triangular regions. As to how to know the current color gamut, according to the foregoing analysis, the image processing apparatus 540 may calculate the color coordinates of each pixel of the image to be displayed so as to know the color gamut of the image to be displayed (as shown in F3 in fig. 6, it is understood that F3 refers to an irregular shaded area formed by the color coordinates of each pixel), and in general, the current color gamut (as shown in F4 in fig. 6) may be a color gamut area that just covers the color coordinates of each pixel of the image to be displayed and has the smallest area. Referring to fig. 6, let the three vertexes of the first color gamut range respectively corresponding to the three primary colors be R1, G1, B1, and the three vertexes of the second color gamut range respectively corresponding to the three primary colors be R2, G2, B2, and the three vertexes of the current color gamut range respectively corresponding to the three primary colors may be located on the connection line between R1 and R2, the connection line between G1 and G2, and the connection line between B1 and B2. Specifically, after the color gamut range of the image to be displayed is obtained, vertexes R0, G0 and B0 are respectively selected on a connecting line between R1 and R2, a connecting line between G1 and G2 and a connecting line between B1 and B2, so that a region surrounded by the vertexes R0, G0 and B0 just covers the color coordinates of each pixel of the image to be displayed, and the area of the region surrounded by the vertexes R0, G0 and B0 is the smallest compared with a region surrounded by any other three vertexes on the three connecting lines, or in the case that the region surrounded by the vertexes R0, G0 and B0 can cover the color coordinates of each pixel of the image to be displayed, a point on the three connecting lines with the closest distance from R1, G1 and B1 is selected as the vertex three connecting lines. Of course, it can be understood that if the color gamut of the image to be displayed has a portion exceeding the second color gamut F2 (i.e., a portion outside the triangular region surrounded by the vertices R2, G2, and B2), since the light source device 510 cannot generate light exceeding the second color gamut F2 according to the capability of the current light source device 510, the current color gamut may be determined as the maximum color gamut (i.e., the second color gamut) that the light source device 510 can display. It is understood that in the present embodiment, the vertices B1 and B2 overlap.

Therefore, according to the above principle, the image processing apparatus 540 can determine the current color gamut range by knowing the color coordinates of each pixel of the image to be displayed, that is, three vertices R0, G0, and B0 of the current color gamut range are known, so as to obtain the conversion matrix C ″.

Further, according to the analysis, the image processing apparatus 540 obtains the current color gamut of the image to be displayed, and in order to accurately restore the image to be displayed, the color gamut of the mixed light of the first light and the second light emitted by the light source apparatus 510 needs to be consistent with the current color gamut, specifically, according to the analysis, the image processing apparatus 540 generates a first control signal to control the brightness of the first light emitted by the light source apparatus 510 according to the maximum brightness value among the brightness values Y of the pixels of the image to be displayed so as to control the light quantity of the first light, and further according to the current color gamut to which the mixed light of the first light and the second light needs to reach, the image processing apparatus 540 can calculate the light quantity of the second light of the first light corresponding to the intensity, and generate a second control signal according to the calculation result of the light quantity of the second light, for controlling the light quantity of the second light emitted by the light source apparatus 510, so that the color gamut of the mixed light of the first light and the second light can be consistent with the current range. Specifically, the light amount of the second light may be controlled by controlling the brightness of the second light supplied from the light source device 510 to the light modulation device 550 during the modulation time of the image to be displayed.

Specifically, if the image processing device 540 obtains that the color coordinates of each pixel of the image to be displayed are located within the first color gamut range F1, that is, the color gamut range of the image to be displayed is located within the first color gamut range F1, the image processing device 540 may determine the first color gamut range F1 as the current color gamut range, the image processing device 540 may send a light quantity control signal (including a first control signal and a second control signal), and by using the light quantity control signal, at this time, the second light may be turned off for the light source device 510, the light source device 510 sends the first light, and at this time, the light quantity of the first light may reach a maximum value, the image processing device 540 calculates the corrected image data r '", g'", and b '"according to the formula 12, and the light modulation device 550 calculates the image data r", g ", and b" of the current color gamut range according to the corrected image data r' ", g '", and b' ", and then modulates the first light according to the image data of the current color gamut range, so as to obtain the image light that the image can be accurately restored.

If the image processing device 540 obtains that some color coordinates of the pixels of the image to be displayed exceed the second color gamut F2 or are located on a partial boundary line of the second color gamut F2, that is, the color gamut F3 of the image to be displayed has a portion exceeding the second color gamut F2 or a partial boundary line including the second color gamut F2, the image processing device 540 may send a light quantity control signal (including the first control signal and the second control signal) by using the second color gamut F2 as the current color gamut, the image processing device 540 may send a light quantity control signal, and the light quantity control signal may be used to turn off the first light to the light source device 510, send the second light to the light source device 510, and the light quantity of the second light may reach a maximum value at this time, the image processing device 540 may calculate the corrected image data r '", g'", b '"according to formula 12, and the light modulation device may calculate the image data r", g ", b" for modulation according to the corrected image data r' ", g", b ", and then modulate the second light according to the image data r", b "for modulation, thereby obtaining the accurate image light for restoring.

It can be understood that if the image processing apparatus 540 obtains the boundary line that the color coordinates of each pixel of the image to be displayed have partial color coordinates exceeding the first color gamut F1 but not exceeding the second color gamut F2, that is, the color gamut F3 of the image to be displayed exceeds the first color gamut F1 but not exceeding the second color gamut F2, the image processing apparatus 540 selects the region enclosed by the vertexes R0, G0 and B0 as the current color gamut F4, and at this time, the current color gamut F4 is located between the first color gamut F1 and the second color gamut F2 and covers the color gamut F3 of the image to be displayed, the first control signal in the light quantity control signals is obtained according to the maximum brightness value in each pixel of the image to be displayed, so that the first control signal controls the light source device to emit the first light corresponding to the maximum brightness value of each pixel of the image to be displayed, and the second control signal of the light quantity control signal can be calculated according to the light quantity of the first light (specifically, the brightness of the first light) and the current color gamut F4, so as to control the light quantity of the second light emitted by the light source device 510, the image processing device 540 calculates the corrected image data R '", G'", B '"according to the formula 12, and the light modulation device 550 calculates the image data R", G ", B" (or the image data R ", G", B "for modulation) corresponding to the current color gamut range according to the corrected image data R'", G '", B'", and modulates the first light and the second light according to the image data R ", G", B "to obtain the image light for accurately restoring the image.

It should be noted that, since the relationship between the light quantity control signal and the brightness of the first light and the second light emitted by the light source device 510 is not linear, the light quantity control signal provided to the light source device 510 needs to be gamma (gamma) compensated to achieve a linear mapping between the light quantity control signal and the brightness of the first light and the second light. Therefore, the image processing and display apparatus 500 further includes a gamma correction circuit 530, the gamma correction circuit 530 is configured to receive the light quantity control signal and send a correction signal to the light source driving circuit 520 to control the driving signal sent by the light source driving circuit 520 based on the light quantity control signal, and the light source driving circuit 520 controls the light quantities of the first light and the second light sent by the light source apparatus 510 according to the driving signal, the gamma correction circuit 530 stores a plurality of light quantity control signals and correction signals corresponding to the plurality of light quantity control signals one to one, and the gamma correction circuit 530 searches the correction signal corresponding to the light quantity control signal one to one according to the light quantity control signal after receiving the light quantity control signal output by the image processing apparatus 540 and provides the correction signal to the light source driving circuit 520. It is understood that the gamma correction circuit 530 may take the form of a look-up table.

The light modulation device 550 may include a control chip 551 and a modulator 552. The control chip 551 stores therein a color gamut conversion formula T for receiving the correction image data and calculating image data for modulation in accordance with the color gamut conversion formula T. The modulator 552 may be a DMD modulator, but is not limited to a DMD modulator, such as an LCOS modulator, the modulator 552 modulates the light emitted from the light source device 510 according to the image data for modulation to generate image light, and the modulator 552 may include a plurality of modulation units (e.g., mirror units), each of which may correspond to the image data for modulation of one pixel and modulate (e.g., reflect) the corresponding light source light under the control of the image data for modulation to generate corresponding image light.

Further, the number of the modulators 552 is not limited, and may be one, two or more, for example, one modulator is used to sequentially modulate the first light and the second light, two modulators are used to respectively modulate the first light and the second light, three modulators are used to respectively modulate the light of three different primary colors in the mixed light of the first light and the second light, two modulators are used to respectively modulate the light of three different primary colors in the mixed light of the first light and the second light (one modulator modulates the light of two primary colors, and the other modulates the light of one primary color), six modulators are used to respectively modulate the light of three different primary colors of the first light and the light of three different primary colors of the second light \8230, 8230, and all embodiments are difficult to be exhaustive, and thus will not be described herein again.

The image synthesizing device 560 is used for projecting the image light generated by the light modulation device 550 to display an image. Specifically, the image synthesis apparatus 560 may include a light splitting module 561 and a projection lens 562, in one embodiment, the light splitting module 561 may be located between the light source apparatus and the light modulation apparatus 550, that is, the first light and the second light emitted from the light source apparatus 510 may be provided to the light modulation apparatus 550 through the light splitting module 561, the image light generated by the light modulation apparatus 550 may also be further guided to the projection lens 562 through the light splitting module 561, and the projection lens 562 projects the image light to a predetermined area or object (such as a projection screen, a wall, or a specific position in a space) to display an image.

Compared with the prior art, in the image processing and display device 500 of the present invention, the current color gamut range and the light quantities of the first light and the second light corresponding to the current color gamut range are determined according to the color gamut range of the image to be displayed and the brightness value of each pixel, and the first light and the second light having the light quantities corresponding to the current color gamut range are modulated according to the correction image data, so that not only can the image data of the wide color gamut be displayed, but also the wide color gamut light (i.e. the second light) can be used to the minimum extent by adjusting the light quantities of the first light and the second light according to the current color gamut range, the use of the second light is reduced, and further the light source cost is reduced. In addition, the use of the second light is reduced, and the power and heat dissipation requirements of the light source device 510 are reduced to a certain extent, so that a complex heat dissipation system is not required, and the cost can also be reduced.

Furthermore, in the image processing and display apparatus 500 and method of the present invention, the applicability of the image processing apparatus 540, the display device 580, and the image processing and display apparatus 500 of the present invention is improved by jointly encoding and transmitting the corrected image data and the light quantity control signal, and decoding and using the corrected image data and the light quantity control signal after receiving the corrected image data and the light quantity control signal, so that the image processing apparatus 540, the display device 580, and the image processing and display apparatus 500 of the present invention can be easily applied to the existing system.

Further, according to the above embodiment, since the light modulation device 550 needs to pre-store the color gamut conversion formula T, the image processing device 540 needs to consider the existence of the color gamut conversion formula T, and need to convert the original image data r, g, b into the corrected image data r ' ", g '", b ' "according to the formula 12, specifically, the image processing device 540 can know the conversion matrix C ' (specifically, known according to the color gamut conversion formula T pre-stored by the light modulation device 550), and further calculate the current color gamut range F4 according to the original image data of the image to be displayed, so as to know the conversion matrix C" according to the formula 11, and further calculate according to the original image data r, g, b (such as the original image data r, g, b matrix and C ' C ″) ^-1 Multiplication) to obtain corrected image data r ' ", g '", b ' ". Further, the optical modulation device 550 receives the calibrationAfter the image data r '", g'", b '"are corrected, the corrected image data r'", g '", b'" can be further converted into image data r ", g", b "for modulation by using the pre-stored gamut conversion formula T, where the image data r", g ", b" for modulation are image data based on the current gamut range F4, and the light modulation device 550 modulates the light source light of the corresponding current gamut range F4 based on the image data of the current gamut range F4, so as to accurately restore the image to be displayed.

However, according to the above principle, in a modified embodiment, if the light modulation device 550 does not need to pre-store the color gamut conversion formula T, but directly receives the image data output by the image processing device 540 and directly modulates the light according to the received image data (i.e. does not need to convert into image data for modulation), then the image processing device 540 may generate the image data r ", g", b "based on the current color gamut range F4 as the corrected image data, and further in this modified embodiment, the image processing device 540 may directly calculate r", g ", b" as the corrected image data according to the original image data r, g, b, and further compress and encode the corrected image data of r ", g", b "and the light quantity control signal into the pre-processed data and provide the pre-processed data to the display device 580, according to the above formula 14:

since the transformation matrix C is known, the image processing apparatus 540 may determine the current color gamut range F4 after analyzing the original image data of the image to be displayed, so as to obtain the transformation matrix C ″, that is, the image data r ", g", and b ″ can be obtained by calculation and provided to the light modulation apparatus 550 as the output image data, without further data transformation by the control chip 551 of the light modulation apparatus 550. It is to be understood that in this modified embodiment, the principles and steps of how the image processing apparatus 540 generates the light quantity control signal, determines the current color gamut range F4, calculates the corrected image data, and encodes the generated pre-processed data may be substantially the same as those of the previous embodiments, and thus will not be described herein again.

It is to be understood that in the above embodiments, the principle of the primary color conversion is illustrated by taking a three-primary color display device as an example. For a display device using four or five primary colors, C 'may be a 4 × 3 or 5 × 3 matrix whose determinant value of the pseudo-inverse matrix is zero, and thus there is an infinite solution for the conversion from the XYZ space to the primary color space of the corrected image data (r', g ', b'). Wherein C' ^-1 Should be writable as C' ^* Which is some conversion matrix of XYZ to the primary color space of the corrected image data (r ', g ', b '). This transformation matrix can be solved by adding some constraint, such as maximizing white light in RGBW system, distributing the primary luminance as evenly as possible, etc.

Further, the calibration for the color of the display device, i.e., the calibration for C'. By accurately measuring the color coordinates and the luminance values of the tricolor light of the light source device of the display device, the three vertexes r of the color gamut range of the display device can be obtained ₀ ’、g ₀ ’、b ₀ ' the color coordinates are respectively (x) _r ’,y _r ’,Y _r ’)、(x _g ’,y _g ’,Y _g ’)、(x _b ’,y _b ’,Y _b ') to generate accurate C', thereby ensuring accurate color display of the display device. The existing display device, no matter a flat panel display or a projection display device, no matter a three-primary color display device or a multi-primary color display device, the three-primary color light of the display device is fixed and unchanged, and the light quantity proportion (such as the brightness proportion) of the three-primary color light does not dynamically change along with the content of a picture. Therefore, the color calibration of the traditional display equipment can be performed only once or a limited number of times. However, for the image processing and display device 500 of the present invention, the light quantities of the first light and the second light are finally calculated and determined by the image processing device 540 according to the original image data of the image to be displayed, and the light quantity control signals for controlling the light quantities of the first light and the second light are generated to control the light quantities of the first light and the second light emitted by the light source device 550, so that the light quantity ratio of the three primary colors of the image processing and display device 500 is dynamically changed, i.e. the light source device 510 emits lightThe light quantity ratio of the output three primary colors may vary according to the content of each image to be displayed, so that the conversion matrix C ″ of the image processing corresponding to the current color gamut range F4 of the display device 500 may vary according to the content variation of each image to be displayed in each frame of image. Existing fixed color conversion display devices do not meet this requirement. In the image processing and display device 500 provided by the present invention, the conversion matrix C ″ can be dynamically calculated according to the original image data of each image to be displayed, the light quantities of the first light and the second light emitted by the light source device 510 are controlled to control the light quantity ratio of the three primary colors, the image data r ", g", b "are further calculated according to the light quantities of the first light and the second light, and the image is further displayed by modulating the first light and the second light according to the image data r", g ", b", so that the image data of each image and the corresponding light source light are both adapted to the current color gamut range F4 of the image, i.e. the display according to the dynamic color gamut of each image.

Further, referring to fig. 8, fig. 8 is a flowchart of an image processing and displaying method according to a preferred embodiment of the invention. The image processing and displaying method may be classified into an image processing method and a displaying method. Therein, the image processing method may comprise the following steps S1-S6, and the steps S1-S6 of the image processing method may be performed by the image processing apparatus 540. Specifically, referring to fig. 9, the image processing apparatus 540 may include a data receiving module 541, a first calculating module 542, a second calculating module 543, a light quantity signal generating module 544, a data converting module 545 and an encoding module 546, which execute steps S1 to S6, respectively. The display method may include the following steps S7-S9, and steps S7-S9 of the display method may be performed by the display device 580.

Step S1, receiving original image data of an image to be displayed. Wherein, step S1 may be executed by the data receiving module 541 of the image processing apparatus 540.

And S2, acquiring the color gamut range and the brightness value of each pixel of the image to be displayed according to the original image data of the image to be displayed. Wherein step S2 may be performed by the first calculation module 542 of the image processing apparatus 540.

And S3, determining the current color gamut range corresponding to the image to be displayed according to the color gamut range of the image to be displayed. Wherein, step S3 can be executed by the second computing module 543 of the image processing apparatus 540.

And S4, calculating a light quantity control signal corresponding to the current color gamut range according to the current color gamut range and the brightness value of each pixel. Here, step S4 may be performed by the light quantity signal generation module 544 of the image processing apparatus 540.

And S5, generating corrected image data according to the original image data and the current color gamut range. Wherein, step S5 may be performed by the data conversion module 545 of the image processing apparatus 540.

And S6, jointly encoding the light quantity control signal and the correction image data to generate pre-processing image data. Wherein step S6 may be performed by the encoding module 546 of the image processing apparatus 540.

It is understood that the specific principles regarding the data receiving module 541, the first calculating module 542, the second calculating module 543, the light quantity signal generating module 544, the data converting module 545 and the encoding module 546 can be directly determined according to the above description of the operation principle (the principle shown in fig. 7) of the image processing apparatus 540, for example: the light quantity signal generating module can obtain the brightness value of each pixel of the image to be displayed according to the original image data of the image to be displayed, generate a first control signal to control the light quantity of the first light according to the maximum brightness value in each pixel of the image to be displayed, and generate a second control signal according to the maximum brightness value in each pixel of the image to be displayed and the current color gamut range or according to the first control signal and the current color gamut range; in one embodiment, the data conversion module may calculate a correction conversion formula based on the determined current color gamut range information, the color gamut range information based on the original image data of the image to be displayed, and the color gamut conversion formula, and convert the original image data of the image to be displayed into the correction image data according to the correction conversion formula; in another embodiment, the data conversion module may convert the original image data of the image to be displayed into the image data of the current color gamut range based on the determined current color gamut range information and the color gamut range information based on the original image data of the image to be displayed, and use the image data of the current color gamut range as the corrected image data; therefore, the detailed functions and operation principles of the modules are not described herein again.

Further, it is understood that steps S1-S6 of the image processing method may be implemented by a computer program stored in the image processing apparatus 540 when executed by a processor. In one embodiment, the modules of the image processing apparatus 540, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer readable storage medium. Based on such understanding, the image processing method of the present invention implements all or part of the flow of the method in the above embodiments, and can also be implemented by a computer program instructing related hardware, where the computer program can be stored in a computer readable storage medium, and when the computer program is executed by a processor, the steps of the above embodiments of the method can be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like.

And S7, receiving the pre-processing image data and decoding the pre-processing image data to obtain corrected image data and a light quantity control signal. Wherein step S7 may be performed by the image decoder 570 of the display device 580.

Step S8, emitting a first light and a second light according to the light quantity control signal, wherein the light quantities of the first light and the second light are controlled by the light quantity control signal, the first light is used for modulating an image in a first color gamut range, the second light is used for modulating an image outside the first color gamut range separately or together with the first light, the color gamut range of the second light is a second color gamut range, and the second color gamut range covers the first color gamut range and has a portion beyond the first color gamut range. The step S8 can be executed by the light source device 510, the gamma correction circuit 530, and the light source driving circuit 520 of the display device 580. It is understood that the light quantity control signal can be converted into a correction signal by the gamma correction circuit 530 and then provided to the light source driving circuit 520, and the light source driving circuit 520 generates a driving signal according to the correction signal to drive the light source device 510 to emit the corresponding first light and the second light. How the light source device 510, the gamma correction circuit 530 and the light source driving circuit 520 implement step S8 has been described in detail in the foregoing description of the principles of the image processing and display device 500, and will not be described again here.

Step S9, modulating the first light and the second light according to the corrected image data to generate image light of the image to be displayed. The step S8 can be executed by the optical modulation device 550, and the detailed structure and operation principle of the optical modulation device 550 have been described in detail in the foregoing, and are not repeated here.

Compared with the prior art, in the image processing device, the display device, the image processing and displaying device and the image processing and displaying method, the current color gamut range and the light quantities of the first light and the second light corresponding to the current color gamut range are determined according to the color gamut range of the image to be displayed and the brightness value of each pixel, and the first light and the second light with the light quantities corresponding to the light quantities are modulated according to the corrected image data, so that the display of the image data of the wide color gamut can be realized, the wide color gamut light (namely, the second light) can be used to the minimum extent by adjusting the light quantities of the first light and the second light according to the current color gamut range, the use of the second light is reduced, and the light source cost is further reduced. In addition, the use of the second light is reduced, the power and the heat dissipation requirements of the light source device are reduced to a certain degree, a complex heat dissipation system is not needed, and the cost can be reduced.

Compared with the prior art, in the image processing method, the display method, the image processing method and the display method, the current color gamut range and the light quantities of the first light and the second light corresponding to the current color gamut range are determined according to the color gamut range of the image to be displayed and the brightness value of each pixel, and the first light and the second light with the light quantities corresponding to the light quantities are modulated according to the correction image data, so that the display of the image data of the wide color gamut can be realized, the wide color gamut light (namely the second light) can be used to the minimum extent by adjusting the light quantities of the first light and the second light according to the current color gamut range, the use of the second light is reduced, and the light source cost is further reduced. In addition, the use of the second light is reduced, the power and the heat dissipation requirements of the light source device are reduced to a certain degree, a complex heat dissipation system is not needed, and the cost can be reduced.

Furthermore, in the image processing and displaying method of the present invention, the corrected image data and the light quantity control signal are jointly encoded and transmitted, and then decoded and used after being received, so that the image processing method, the image displaying method, the image processing and displaying method of the present invention are easily applicable to the existing system, and the applicability of the image processing method, the image displaying method, the image processing and displaying method of the present invention is improved.

Claims

1. An image processing apparatus characterized by comprising:

a light quantity signal generating module, configured to calculate a light quantity control signal corresponding to the current color gamut according to the current color gamut and the luminance value of each pixel, where the light quantity control signal includes a first control signal for controlling a light quantity of a first light and a second control signal for controlling a light quantity of a second light, and if the color gamut of the image to be displayed exceeds the first color gamut but does not exceed a boundary line of the second color gamut, the light quantity signal generating module is configured to generate the first control signal to control the light quantity of the first light according to a maximum luminance value in each pixel of the image to be displayed, and generate the second control signal according to the maximum luminance value in each pixel of the image to be displayed and the current color gamut or according to the first control signal and the current color gamut, where the current color gamut is located between the first color gamut and the second color gamut and covers the range of the image to be displayed;

2. The image processing apparatus according to claim 1, characterized in that: the data conversion module is used for calculating a correction conversion formula based on the determined current color gamut range information, the color gamut range information based on the original image data of the image to be displayed and the color gamut conversion formula, and converting the original image data of the image to be displayed into the correction image data according to the correction conversion formula.

3. The image processing apparatus according to claim 2, characterized in that: the gamut range information based on the original image data of the image to be displayed comprises a conversion matrix C required by calculating corresponding tristimulus values X, Y and Z according to the original image data of each pixel of the image to be displayed and the gamut range information thereof, wherein the gamut conversion formula T =

C' is the calculated correspondence of the image data of any one pixel recorded by the light modulation device corresponding to the default color gamut rangeThe current color gamut range information includes a conversion matrix C ″ required for calculating the corresponding tristimulus values X, Y, Z according to the image data of any pixel corresponding to the current color gamut range, and the corrected image data r ', g ', b ' corresponding to the original image data r, g, b of any pixel of the image to be displayed conforms to the following formula:

。

4. the image processing apparatus according to claim 1, characterized in that: the data conversion module is configured to convert, based on the determined current color gamut range information and the color gamut range information based on which the original image data of the image to be displayed is based, the original image data of the image to be displayed into image data of the current color gamut range, and use the image data of the current color gamut range as the corrected image data, the color gamut range information based on which the original image data of the image to be displayed is based includes a conversion matrix C required to calculate a tristimulus value X, Y, Z according to the original image data r, g, b of each pixel of the image to be displayed and the color gamut range information to which the original image data r, g, b of any pixel of the image to be displayed corresponds, the current color gamut range information includes a conversion matrix C "required to calculate a tristimulus value X, Y, Z according to the image data r", g, b "of the current range corresponding to the original image data r, g, b" of any pixel of the image to be displayed conforms to the following formulas:

。

5. an image processing method, comprising the steps of:

receiving original image data of an image to be displayed;

calculating a light quantity control signal corresponding to the current color gamut range according to the current color gamut range and the brightness value of each pixel, wherein the light quantity control signal comprises a first control signal for controlling the light quantity of the first light and a second control signal for controlling the light quantity of the second light;

the step of calculating the light quantity control signal corresponding to the current color gamut range according to the current color gamut range and the brightness value of each pixel comprises: if the color gamut range of the image to be displayed exceeds a first color gamut range but does not exceed a boundary line of a second color gamut range, generating the first control signal to control the light quantity of the first light according to the maximum brightness value in each pixel of the image to be displayed, and generating the second control signal according to the maximum brightness value in each pixel of the image to be displayed and the current color gamut range or according to the first control signal and the current color gamut range;

6. A display device, characterized in that the display device comprises:

an image decoder for receiving and decoding the pre-processed image data to obtain corrected image data and light quantity control signals, the light quantity control signals including a first control signal for controlling a light quantity of the first light and a second control signal for controlling a light quantity of the second light;

a light source device, configured to emit first light and second light according to the light quantity control signal if a color gamut of an image to be displayed exceeds a first color gamut but does not exceed a boundary line of a second color gamut, wherein a light quantity of the first light is controlled by the first control signal and a light quantity of the second light is controlled by the second control signal, the first light is used for modulating an image in the first color gamut, the second light is used for modulating an image outside the first color gamut alone or in cooperation with the first light, a color gamut of the second light is a second color gamut, and the second color gamut covers the first color gamut and has a portion exceeding the first color gamut;

7. The display device of claim 6, wherein: the light source device comprises an excitation light source, a color wheel and a supplement light source, wherein the excitation light source emits excitation light, the color wheel is used for receiving the excitation light and emitting first light, the supplement light source is used for emitting supplement light, the supplement light and at least part of the excitation light emitted by the excitation light source serve as second light or the supplement light serves as the second light, the first light and the second light respectively comprise at least two color lights, and the light quantity control signal is used for controlling the excitation light and the supplement light to control the light quantities of the first light and the second light.

8. The display device of claim 7, wherein: the exciting light is first color light, the color wheel is provided with a fluorescent material and receives the exciting light to generate fluorescence, the first light emitted by the color wheel comprises the first color light and the fluorescence, the supplementary light comprises laser, the fluorescence and the supplementary light comprise the same primary color components, the fluorescence comprises second color fluorescence and third color fluorescence or the fluorescence comprises fourth color fluorescence mixed with the second color and the third color, the supplementary light comprises the second color laser and the third color laser, and the first color, the second color and the third color are three primary colors.

9. The display device of claim 6, wherein: the display equipment also comprises a light source driving circuit and a gamma correction circuit, wherein the light source driving circuit is used for sending a driving signal to drive the light source device to emit light;

the gamma correction circuit is used for receiving the light quantity control signal and sending a correction signal to the light source drive circuit to control the drive signal sent by the light source drive circuit based on the light quantity control signal, the light source drive circuit further controls the light quantities of the first light and the second light sent by the light source device according to the drive signal, the gamma correction circuit stores a plurality of light quantity control signals and correction signals corresponding to the light quantity control signals one to one, the gamma correction circuit searches the correction signals corresponding to the light quantity control signals one to one according to the light quantity control signals after receiving the light quantity control signals, and the correction signals are provided to the light source drive circuit.

10. An image processing and display device, comprising:

an image decoder for receiving the pre-processed image data and decoding the pre-processed image data to obtain the corrected image data and the light amount control signal;

a light modulation device for modulating the light emitted from the light source device according to the corrected image data to generate image light of an image to be displayed;

if the color gamut range of the image to be displayed exceeds the first color gamut range but does not exceed the boundary line of the second color gamut range, the light quantity control signal controls the light source device to emit the first light and the second light to the light modulation device, and the current color gamut range is located between the first color gamut range and the second color gamut range; the current color gamut range is larger than or equal to the color gamut range of the image to be displayed and smaller than the second color gamut range, and the image processing device further determines the light quantity of the second light according to the determined light quantity of the first light and the current color gamut range so that the mixed light of the first light and the second light reaches the current color gamut range.

11. The image processing and display apparatus of claim 10, wherein: the image processing apparatus generates the first control signal to control the light amount of the first light according to a maximum brightness value in each pixel of the image to be displayed, and generates the second control signal according to the maximum brightness value in each pixel of the image to be displayed and the current color gamut range or according to the first control signal and the current color gamut range.

12. The image processing and display apparatus of claim 10, wherein: the image processing and displaying device is provided with a default color gamut range, the light modulation device is stored with a color gamut conversion formula corresponding to the default color gamut range, the image processing device calculates a correction conversion formula based on determined current color gamut range information, color gamut range information based on original image data of an image to be displayed and the color gamut conversion formula, the image processing device also converts the original image data of the image to be displayed into the correction image data according to the correction conversion formula, and the light modulation device converts the correction image data into image data corresponding to the current color gamut range according to the color gamut conversion formula and modulates light emitted by the light source device according to the image data corresponding to the current color gamut range to generate image light.

13. The image processing and display apparatus of claim 12, wherein: the gamut range information based on the original image data of the image to be displayed comprises a conversion matrix C required by calculating corresponding tristimulus values X, Y and Z according to the original image data of each pixel of the image to be displayed and the gamut range information thereof, wherein the gamut conversion formula T =

C 'calculates a conversion matrix required by the corresponding tristimulus values X, Y, Z for the image data of any one pixel corresponding to the default color gamut range recorded by the light modulation device, the current color gamut range information includes a conversion matrix C ″ required by the corresponding tristimulus values X, Y, Z calculated according to the image data of any one pixel corresponding to the current color gamut range, and the corrected image data r', g ', b' corresponding to the original image data r, g, b of any one pixel of the image to be displayed conform to the following formula:

，

the correction conversion formula is

The corrected image data r ' ' ', g ' ' ', b ' ' ' converted into the image data r ' ', g ' ', b ' ' of the current color gamut range conforms to the following formula:

。

14. the image processing and display apparatus of claim 10, wherein: the image processing apparatus converts the original image data of the image to be displayed into image data of a current color gamut range based on the determined current color gamut range information and the color gamut range information on which the original image data of the image to be displayed is based, and takes the image data of the current color gamut range as the corrected image data.

15. The image processing and display apparatus of claim 14, wherein: the color gamut range information based on the original image data of the image to be displayed includes a conversion matrix C required for calculating the tristimulus values X, Y, Z according to the original image data r, g, b of each pixel of the image to be displayed and the color gamut range information thereof, the current color gamut range information includes a conversion matrix C ″ required for calculating the tristimulus values X, Y, Z according to the image data of any one pixel of the current color gamut range, and the image data r', g ", b ″ of the current color gamut range corresponding to the original image data r, g, b of any one pixel of the image to be displayed conforms to the following formula:

。

16. the image processing and display apparatus of claim 10, wherein: if the color gamut range of the image to be displayed is within the first color gamut range, the light quantity control signal controls the second light to be turned off, the light source device emits the first light, and the current color gamut range is the first color gamut range.

17. The image processing and display apparatus of claim 10, wherein: and if the color gamut range of the image to be displayed has a part exceeding the second color gamut range or a part of boundary line containing the second color gamut range, the light source device emits the second light, and the current color gamut range is the second color gamut range.