BR112016027461B1 - METHOD OF CODING A HIGH DYNAMIC RANGE IMAGE, IMAGE ENCODER ARRANGED TO ENCODE A HIGH DYNAMIC RANGE IMAGE, IMAGE DECODER AROUND TO RECEIVE A HIGH DYNAMIC RANGE IMAGE SIGNAL, AND, METHOD OF DECODING A HIGH DYNAMIC RANGE IMAGE SIGNAL HIGH DYNAMIC RANGE - Google Patents

Abstract

METHOD OF CODING A HIGH DYNAMIC RANGE IMAGE, IMAGE ENCODER ARRANGED TO ENCODE A HIGH DYNAMIC RANGE IMAGE, IMAGE DECODER AROUND TO RECEIVE A HIGH DYNAMIC RANGE IMAGE SIGNAL, AND, METHOD OF DECODING A HIGH DYNAMIC RANGE IMAGE SIGNAL HIGH DYNAMIC RANGE. The present invention relates to enabling a good HDR image or video coding technology, which has the ability to produce high dynamic range images, as well as short dynamic range images; a method of encoding a high dynamic range image (M_HDR) was invented, which comprises the steps of: converting the high dynamic range image into a shorter luminance dynamic range image (LDR_o) by applying) scaling of the high dynamic range image to a predetermined scale of the luma axis such as [0, 1], b) a sensitivity tone mapping that changes the brightness of pixel colors that are covered by at least one subrange comprising the most dark areas in the high dynamic range image, c) a gamma function, and d) an arbitrary monotonically increasing function that maps the lumas resulting from performing steps b (...).

Description

FIELD OF INVENTION

[001] The invention relates to encoding an image (i.e., a static image), but preferably more High Dynamic Range images (i.e., video), and which corresponds to systems and methods of the art for conveying the necessary encoded image information to a receiving side, and decoders for decoding the encoded images and ultimately making the same available for display.

[002] An HDR image is an image that encodes the textures of an HDR scene (which can generally contain both very bright and dark regions), in such a way, with sufficient information for high-quality encoding of the textures of color of the various objects captured in the scene, that a visually good quality rendering of the HDR scene can be performed on an HDR display with a high peak brightness, such as 5,000 cd/m2 (5,000 nit). In the present HDR encoding framework developed over the past years, it is also desired to simultaneously encode several dynamic subviews covered in that scene, which can serve as good trigger images for multiple screens of successively reduced dynamic range, at least an LDR image appearance suitable for driving, for example, a 100 cd/m2 (100 nit) peak brightness display.

[003] Furthermore, an HDR encoding is preferably generally designed so that it not only correlates relatively well with existing video encoding technologies, but can even fit into existing image or video encoding structures. video of existing technology, such as, for example, blu-ray disc storage (and its limitations such as the amount of storage memory, as well as all substantially already standardized aspects), or HDMI cable connections, or other transmission systems or image storage etc. Specifically, HDR coding systems that use two encoded images per unit time (e.g., an LDR image, and an image that has local lighting intensities to enhance each pixel to an HDR image pixel) are seen, especially, by device manufacturers as unnecessarily complex.

[004] HDR video (or even still image) encoding has only recently been attempted tentatively and has been a daunting task to date, and the characteristic belief is that it needs significantly more bits, to encode brightnesses above the LDR range of objects in the scene (for example, encodings that encode scene luminances directly), or it needs some two-layer approach, for example, in addition to a reflectance image of the object there is a lighting reinforcement image, or similar decomposition strategies. Alternative approaches, such as Dolby's HDR encoding system, generally always use such a two-layer HDR encoding technology (see US8248486B1). WO2010/105036 is another example of such a Dolby 2-picture-per-unit-time coding system. Some simple TM fixed tone mapping (for example, which emulates the behavior of classic analog celluloid photography) can be used to map an LDR image to an HDR rating that matches it, or vice versa. This could be any mapping function that is not quite correct (due to the fact that the classifier may have used complicated optimization decisions, i.e. its LDR classification), so there could be a considerable difference between the original image prediction, say up, HDR image, and the original HDR image itself, but this is not a problem due to the fact that the system can send the differences as a second correction image (residual bitstream 742).

[005] Another example of such a two-image coding system is the Technicolor HDR coding system (document JCTVC-P0159r1, from the 16th meeting of the Cooperative Board Team on Video Coding of ITU-T SG 16 WP3, San José January 9-17, 2014) which encodes as a first image a low-resolution signal that is local lighting as would normally be generated in a 2D modulated LED backlight, and the second image is a textured image of typically shorter dynamic range modulations after that low frequency signal, which will typically be generated after display via suitable LCD valve drive signals.

[006] Philips has recently invented a much simpler single-image approach (not yet widely published, but some aspects of it can be found in documents WO2012/147022 and WO2012/153224), which is a new direction in image coding and of video, and not only, a priori, not easy to imagine, but also when, in doing so, it leads to many new technical issues to be solved, which, however, works well in practice.

[007] By the term “high dynamic range” (HDR) it is meant that any image (or images) as captured from the capture side has a high luminance and contrast ratio compared to legacy LDR encoding (i.e. Object luminance and contrast ratios of 10,000:1 or more can be manipulated by encoding, and all components of the image manipulation chain through rendering; and captured object luminances can be above 1,000 cd/m2 (1,000 nit ), or more specifically, can typically be played/rendered above 1,000 cd/m2 (1,000 nit) to, due to the playback environment, generate some desired appearance of, say, a lit lamp or sunny exterior). And generally, the rendering of such an image (or images) is HDR (that is, the images must be suitable in the sense that they contain sufficient information for high-quality HDR rendering, and preferably in a manner technically easy to use), which means that the image (or images) is rendered or is intended to be rendered on screens with peak brightness of at least 1,000 cd/m2 (1,000 nit) (which does not imply that the same should not or need not be alternatively rendered on LDR displays of, for example, 100 cd/m2 (100 nit) peak brightness, typically after appropriate color mapping).

BACKGROUND OF THE INVENTION

[008] Recently, numerous HDR encoding technologies have been proposed, such as Dolby's dual layer method (WO2005/1040035). However, currently, the industry is still looking for a pragmatic HDR video (/image) coding technology that fits all requirements (a balance of them), such as the very important factors such as quantity of data, but also computational complexity (price of ICs), ease of introduction, versatility for artists to create whatever they want, etc. Specifically, a dual-layer approach is seen as unnecessarily complex. One would ideally like to have the ability to design an encoding technology that suits legacy encoding, such as DCT-based MPEG HEVC encoding. One problem is that this is somewhat counterintuitive: how can you encode an HDR image, which should by definition be something different from an LDR image, which generally has a greater number of interesting brightness/luminance ranges, instead of a technology optimized to contain LDR images? These legacy LDR image manipulation/encoding systems have been designed and optimized to work with typical LDR imaging scenarios, which are typically well lit with, for example, a 4:1 studio lighting ratio(or, for example, example, 10:1), providing for most objects (which can vary in reflectance between, say, 85% for white and 5% for black) in the view, a total contrast ratio of about 68:1 ( respectively, 170:1). If one observes the relative rendering (i.e., mapping the white image to the available screen white) of object luminances that start from a white peak, a typical earlier LCD monitor without local light intensity regulation would have had something such as 100 cd/m2 (100 nit) white and 1 cd/m2 (1 nit) black, which would correlate to the image contrast ratio and generally one would think that on average CRT systems that may have been observed also during the day they would have something like a 40:1 capacity. The fact that there is a standard legacy luminance code allocation gamma function of 2.2 in these systems seemed satisfactory for most even higher scene contrast scenarios. Although some errors currently considered acceptable were made, such errors in rendering poorly encoded high luminance scene regions (e.g., severely limiting bright exteriors behind a window) were also acceptable due to the fact that screens of LDR could not, in any case, render these physically accurate object luminances.

[009] However, there are scenarios for which there is currently a desire to improve rendering, for example, an indoor scene in which an individual may simultaneously view sunny outdoor environments, in which case there may be a reason to illumination of 100:1 or even more. With linear relative rendering (i.e., focusing on the first brightest encoded regions, or equivalently, the middle gray of the brightest scene regions, and mapping the white of the image to the white of the screen), the white of the inner parts would map it to psychovisual black for the viewer! So in LDR, these sunny regions will normally appear as (moderately) limited (usually already in the encoded image which has difficulty discriminating codes around 255 at most for these pixels). However, on an HDR screen, you want to show them both bright and colorful. This would provide a much more naturalistic and spectacular rendering of such scenes (as if you were actually on vacation in Italy), but even scenes where the higher brightness content is just made up of some specular reflections already show a greater visual quality improvement. . If artifacts such as limiting or quantization errors do not appear to be inconvenient, for example on a 5,000 or 10,000 cd/m2 (5,000 or 10,000 nit) display, one would at least want the ability to drive such HDR displays with the right type of image, so that the rendered images are as beautiful as the HDR screen allows.

[0010] The classical thinking was, however, that to encode additional excess brightness ranges, one would need to have (many) more bits, which are the upper bits that encode object luminances above an LDR range. This could happen through natural encoding into larger single code words (such as OpenEXR with 16 bits of which a sign bit, 5-bit exponent and 10-bit mantissa, or Ward's LogLuv encoding, which rigorously mathematically attempts to capture all possible object luminances with high precision), or using a first layer with standard LDR range codes (for example, a classic JPEG approximation of the HDR image), and a second layer to enhance such luminances of pixel for higher brightness (e.g., a boost image to boost each pixel, if necessary, to higher luminance, i.e., a multiplication of two such 8-bit images equivalent to a single 16-bit linear code).

[0011] A major practical problem to be solved when designing a practical HDR encoding technology, besides the fact that it logically needs to have the ability to handle a wide range of images of different HDR, is that hardware manufacturers desire smaller amounts of bits per codeword (channel, i.e. the luma, and two chromatic channels), however, and although the present technology proposed below can also work with words with more bits, a solution is proposed that works well under a limitation of 10 bits for at least one luminance channel (or, more precisely, one luma) (note that, although modalities with a luminance channel are elucidated, the present concepts can, with due modifications, be realized as functioning in RGB color representations (linear or non-linear) etc.). Furthermore, a framework has been developed that can perform, in a dual philosophy, both color pixel encoding (from HDR appearance through an LDR image) and color appearance conversion for various rendering scenarios (i.e. is, the ideal appearances needed to render a scene on multiple screens with different peak brightness, e.g. PB = 800 nit) in a functional manner, meaning that only the functions need to be co-coded when encoding the appearance of at least one additional rating, and specifically an HDR appearance in addition to an LDR appearance, rather than, for each figure, at least a second figure.

[0012] There are currently two categories of HDR coding systems, as the market would like such versatility in a coding system, due to the various players and their different needs. In the i-mode system (or HDR appearance encoded as a single definition image, for example, on a BD disc, or a stream of AVC or HEVC images over a network connection), an image is used of HDR appearance as the single pixel image, which is used to encode the color textures and shapes of the object (see applicant's document WO2015007505 how such a single HDR image can be sent to a receiver to define the pixel colors of at least the HDR appearance, and as with suitable reclassification functions the receiver can calculate, by processing the colors in this image, other image appearances). This means taking the original HDR top-rated image, that is, an ideally color-rated image looks best on a reference HDR display, for example, typically a 5,000 peak brightness display. cd/m2 (5,000 nit), and that transforms it only minimally: apply only basically a code allocation function or OETF Optoelectronic transfer function (note that although this OETF defines how the luminances of the scene as captured, for example, by a camera are transferred to luma codes, television engineers instead like to specify the inverse function which is the EOTF electro-optical transfer function to go from luma codes to reference screen rendered luminances) with Using the OETF ideally allocates the 10 bits of codes available, for example, to the luma channel Y' across all brightness values, an individual needs to be able to perform on a reference screen [0 to 5,000] cd/ m2 ([0 to 5,000] nit). Other desired ratings for different brightness peaks can then be accomplished by transforming this HDR appearance image. In the present structure, this adjustment of screen appearance is allowed by generally performing only a second classification that is at the other end of the range of possible screens to be served, that is, an appearance that is ideal or reasonable according to the content creator/color classifier for a 100 cd/m2 (100 nit) peak brightness display (which is typically the reference display for the LDR display category). Note that this is a co-coding of an additional appearance rather than a mere recolor step on the creation side. This required color transformation is determined by applying mapping functions such as gamma functions that perform a global brightness readjustment (e.g., lightens the darkest colors in the image), arbitrary S-shaped curves, or inverse S-shaped curves. to adjust local contrast, color saturation processing functions to adjust, for example, saturation for the corresponding brightness of some objects or regions in the image, etc. One can liberally co-code these functions (whatever functions are needed, as long as they belong to a limited set of base functions that the receiver can understand in a standardized way) as metadata associated with the appearance image of Pixelated HDR, in which case you parametrically DEFINE the second LDR appearance classification image from the HDR appearance image (i.e., it is no longer necessary to encode this LDR appearance image as a pixel image). Note carefully the difference with two-layer coding systems: in the present system, the color transformation functions are all that are encoded on the second appearance with the ability to reclassify the second appearance at the receiver, so instead From the approximate functions of 2-image technologies, the present functions contain the total intelligent knowledge of how the illuminations of the various objects should behave in various rendering appearances according to the content creator. Given this knowledge of how creative artists want to appear to transform, from the first appearance for screens with a first level of color rendering capabilities to a second appearance for screens with a second level of color rendering capabilities (specifically, the screen brightness peak), a screen with intermediate capabilities (e.g. 1200 cd/m2 (1200 nit) peak brightness) can then automatically obtain a more ideal drive image for that rendering situation using knowledge in the two classifications and interpolation (for example, the screen may perform an asymmetric blend of the two pixelated images of the HDR appearance and the LDR appearance image derived from the HDR appearance image and the functional transformations, in which the multiplicative blend percentages are determined by how close the HDR or LDR screen is to the current screen on a non-linear psychovisual scale), which will be better than triggering the screen with the HDR appearance image or the original LDR appearance image.

[0013] This is a powerful yet simple definition of not just a single image appearance (HDR) in a scene (e.g., a 5,000 cd/m2 (5,000 nit) render), but a total framework for triggering reasonable renders of the scene to various possible screens in the field such as in a consumer home (and further, the potential adaptation to viewing the environment, for example, applying post-gamma modeling to the altered contrast sensitivity of human vision under various surrounding illuminances ). It is mainly useful, for example, for applications/scenarios where a creator has made a good HDR version of their content and wants to have that HDR look in advance in the actual encoding sent to the receivers (e.g. on a BD disc of HDR, or by ordering an HDR movie online via the internet, or an HDR television broadcast, etc.). It is not necessary for a customer who purchases this version of content to actually have an HDR screen, as they can purchase the same for later, when they have an HDR screen and can now use the image conversion. HDR-2-LDR, but would be the preferred option when the customer wants content for their HDR screen.

[0014] While the way of encoding HDR appearance HDR scenes (as explained, the i mode which is that mode of at least HDR appearance images encoded as a pixel image, but in fact also promotes appearance in that same scene that are then encoded parametrically with color transformation functions, such as, for example, a limiting modality, in which the LDR appearance isolates a subrange of the HDR image and limits the rest) already imposes significant technical challenges to obtain a new pragmatic technical system for future imaging, but mainly also for video coding (taking into account such factors as simplicity of IC (integrated circuit) design for hardware manufacturers, while still allowing , for content developers to create whatever beautiful HDR content they want to create like sci-fi movies, spectacular television shows or nature documentaries, etc., with lots of creative HDR effects like light bulbs that actually appear to be lit), the market wants yet another layer of complexity, which will be taught in this patent report.

[0015] That is, for some applications (which will be called mode-ii), it may be desirable to have an LDR appearance image as the only pixelated image encoding the objects in the scene, which is, for example, recorded as the Single image on a Blu-ray disc. Although the content creator also cares a lot about the quality of the HDR appearance, he pays a lot of attention to the LDR appearance, which is similar to what it would be like with legacy technologies. There will then generally be function parameters co-coded into bindable metadata to derive an HDR appearance image by updating the LDR appearance image that was communicated in the S_im image signal. There may be several reasons for choosing this mode-ii variant (or LDR appearance), which may, for example, be for legacy systems that do not have the capability to perform any processing (e.g., if an individual prefers to encode the single image in a specific modality that encodes the colors as Y'uv colors rather than a YCrCb encoding, an individual could still encode this in a legacy HEVC structure, pretending that the Y'uv image is a YCrCb image with unusual coloring and with the additional use of legacy DCT-based coding schemes, as standardized in one of the members of the MPEG codec family), but also for applications that need an LDR appearance (e.g. , when watching a movie on a portable screen with low brightness) and may not want to do much processing. Or perhaps the creator doesn't want to invest a lot of time in creating a perfect HDR look (but, for example, just a quick minimal fine-tune adjustment of an LDR-2-HDR autoconversion that, for example, isolates regions and enhances them in a non-linear fashion, for example, for a remaster of an old “The Fat and the Skinny” film), and considers its LDR appearance to be the most important master rating of LDR and HDR appearances, which should be directly coded without needing any color transformation, with possible color errors. For example, a television broadcaster may choose this option, especially for broadcasts about real life (for example, the news may not need to be in the most spectacular HDR).

[0016] This LDR appearance coding (mode ii) has, however, additional complexity due to the mathematical nature of the problem and the mathematics of coding on the one hand, versus desires for liberal artistic classification on the other hand, which makes this a daunting task to produce a good technical structure. To be more precise, on the one hand, one needs functions that first decrease the rating of a desired master HDR image, and in the receiver with these received functions (or the inverse functions of the decrease, in fact) the receiver can update to at least a close approximation of the original HDR image again, that is, in the metadata function, the parameter data will be parameters to functions (derived by the encoder from the functions the classifier used in reducing the HDR master) that can map the single LDR image for sufficiently close HDR prediction Rec_HDR. But on the other hand, the LDR image must, when directly rendered on a +- 100 cd/m2 (100 nit) screen, i.e. without further color transformation, look good enough according to the color classifier as well. . So there will be a balance between the selection of functions, and how they will influence the LDR Rec_HDR appearances, and that, also taking into account other issues, such as IC or device manufacturers would like to see a limited set of standard functions that are useful for reclassifying appearances, and content creators want these functions to quickly specify which appearances they want, since classification time is expensive and the timing of film releases can be critical. In the description below, a practical system for handling this mode II variant of HDR scene encoding will be described.

[0017] The following documents are tangentially related to the present invention, but, nevertheless, it is of interest to mention and distinguish them.

[0018] During the Sapporo MPEG2014/M34274 meeting, R. van de Vleuten et al. of Philips published (July 2014, then, after the priority date of this request) “Proposed electro-optical transfer function (EOTF) for HDR video delivery”. This document contains the EOTF which is also used for the present embodiments to perform a pre-mapping for the LDR dynamic range. However, this document only teaches that this EOTF has good correlation properties with the appearance of human lighting, and therefore, it could be useful as a luma code allocation function for HDR image coding. Ergo, clearly, what, at most, can be derived from this document, is a teaching or an inspiration that if it is desired to encode a single master HDR image or video, that is, images with pixel lumas encoded to, e.g. example, a 5,000 nit screen, then this EOTF is useful. Encoding only an HDR video means that there are only images defined with reference to the peak brightness of, for example, 1000 nit or 5000 nit that need to be communicated to the receivers, and there is nothing else, in particular, no other appearance classified in the same scene images as LDR images with 100 nit peak brightness, which also need to be reported. This unique HDR video encoding is much simpler technically, and a different view on HDR encoding is not what the present invention teaches. Such HDR-only images are not suitable for direct display on legacy LDR displays, which typically have a peak brightness of about 100 nits, as, for example, dark regions will appear much darker on such a display that is 50 nits. times darker than the 5,000 nit screen the HDR images were created for. As clearly mentioned above, the present application provides technologies for scenarios in which it is not desired to convey the HDR information of an original HDR scene to a receiver, but one may wish to in fact communicate an LDR version of such image (in a way that, however, allows reconstructed imaging of the HDR image in the receiver), which provides a suitable appearance when rendered directly on a legacy screen, that is, with the dark regions not being too dark. In other words, a very different technical requirement from HDR image-only encoding and, therefore, this request teaches additional well-considered factors.

[0019] Document m34165 from the same Sapporo meeting by Tourapis et al. “Report on the XYZ/HDR exploratory experiment 1: EOTFs for Wrong EOTF can lead to banding on the receiving side), that is, without any consideration, or even mentioning a need for an LDR image, leave the visual or artistic quality of that LDR image.

[0020] Document WO2013/046095 teaches general technical principles necessary to have the ability to perform any HDR decoding, in particular, again decoding HDR scene data that has been encoded as single HDR images, but which does not need necessarily be rendered on a screen with the same peak screen brightness (PB_D) as the white point luminance of the received HDR image (for example, consider a signal decoder that receives specified HDR content, i.e. with color graded image object pixel brightnesses are correct to be rendered on a 5,000 nit PB_D reference screen, however, it has a fixed screen of, say, 2,000 nit PB_D, or 7,000 nit, so it must perform an optimal dynamic range transformation to get correct-looking images to output to your current connected screen). One needs to know several things, such as what the code's white point luminance was (e.g., R=G=B=1023 meant a 1000 nit white, or 5000 nit, as would seriously change the transformation needed for some renderable white on the connected screen), and that it is the EOTF used that actually defines the RGB or luma codes, that is, that links the luma codes to the luminances that they are supposed to represent, that is, that should be rendered on the connected screen. There is also some general teaching that some dynamic range transformation data may be co-communicated with the (single) communicated HDR image/video (or images/videos), or perhaps communicated LDR image (or images). This teaching could, at most, inspire a reader skilled in the art to co-communicate some desired appearance change with the image (e.g., HDR can on purpose be darkened a little for whatever reason, and then lightened again on the side). of the receiver communicating a brightening dynamic range/tone mapping transformation, or perhaps the transformation could occur on the receiving side independent of what the encoding side would have done with the image before transmitting it; this tone mapping function tone must be contrasted with EOTF, which is a pure technical allocation of luma codes to luminances, but tone mapping can perform some additional changes of pixel brightness, for example, technically performed as a mapping from input_luma to output_luma, e.g. example, for artistic reasons). For this patent application the teaching about the main data manipulation structure to manage the understanding of the meaning of some type of HDR image coding (defined by what the maximum of the peak brightness of the codes corresponds to, e.g. R=G =B=1023 needs to be rendered as 5000 nit white; and that EOTF was used to allocate all RGB luma codes to in fact be RGB values and screen-rendered linear luminances), which cannot be derived from this , is the construction of a specific new and different technical system that communicates HDR image data as LDR images as in the present application, leaving the specific necessary components that accomplishes this in a pragmatic way as claimed.

[0021] Doc. JVT-S087 of the 19th Geneva MPEG Meeting: Segall et al. “Tone mapping SEI message” just teaches, in a generic way, that if someone has a desire for “some” tone mapping to be applied to pixel luminances, then this can be communicated in a dedicated metadata SEI message. That is, the receiving side could then receive, for example, a 10-bit encoding of an HDR image, and, for example, use the SEI image to convert to a standard 8-bit LDR image that matches at the same, that is, with the object's pixel brightnesses best suited to rendering on a 100 nit PB_C screen (rather than, for example, the 1000 nit or 5000 nit screen for which the HDR image was made). For example, a sigmoid shape can be used. This is, again, an example of communicating only the pixel color data of an HDR video of images, so this document again contains nothing sufficient to inspire towards some component of an LDR-compatible dual coding system for images of HDR as it is.

[0022] Document JCTVC-P0159r1/m32076 by Lasserre et al. “High dynamic range video coding” is an example of a 2-layer system (i.e., a basic image stream and a correction image stream) (see Figure 1), consequently, quite far from the technology presented, and which it does not contain any useful inspiration, or even likely to be consulted for useful inspiration. Two-layer systems are without their technical philosophy, hence construction and the technical components used, very different from single image encodings plus color function transformations to derive other dynamic range images.

[0023] Documents EP2406959 or WO2010/105036 are also just a two-layer HDR encoding (see Figure 7, residual bit stream 742), too different and irrelevant to sufficiently absorb lessons learned in the concepts of the present application. It provides an interesting insight into some aspects of HDR image coding as background information.

[0024] Paper M34355 from the July Sapporo conference (again, after the priority date) by Stessen et al of Philips “Chromaticity-based color signals” teaches that when an individual wishes to encode (only) HDR image (or images) , you may want to use another color space in addition to the legacy LDR encoding Y'CbCr color space. It mentions again that the “brightness” axis of this color space could be defined with the Philips EOTF which, unsurprisingly, is also used in some other disclosures and teachings, such as in the present patent application. The document then further teaches that if one wishes to perform color processing in a space that is determined by, for example, such a highly biased EOTF and two chrominances, for example, if a receiver would like to perform its dynamic range transform of any mode or other color optimization on its end, for example, whenever it has automatically decided that it could be a reasonable transform to convert the received HDR image(s) to LDR image(s), then this space may not be as useful, and color errors of a magnitude greater than desirable could occur. Therefore, the authors introduced two-color channel chromaticities that are independent of dynamic range. Little value could be derived from the teachings of the present application, much less that these teachings would be covered by the standard of technology necessary for encoding or decoding LDR-based HDR video.

[0025] Philips' generic proposal of TM-AVC0671 presented at the DVB standardization forum, is merely a high-level black box system teaching in the business sense that was developed from a (mode-ii) possibility to encode images of HDR communicating, in fact, LDR images (+ the right type of functional metadata, one must research more about an invention, for this to suddenly work). It, however, does not provide any specifications about the necessary components, as presented in this patent application. It only says that there is a box “dynamic range converter”, which a person sufficiently versed in the art would understand what it would do in such a system topology, but no details of the internals were revealed. Furthermore, a screenshot of the sorting tool is shown, which a human grader on the creation side could use to specify all the necessary functions, as also in the present invention, but the applicant was careful not to show any technical details that would would be derivable from that simple figuration.

SUMMARY OF THE INVENTION

[0026] There is a need to have improved encoding of HDR images, and specifically, it started with the philosophy that especially at the current time, when there are many legacy LDR systems in the field, there is a need for some levels of compatibility. This means, on the one hand, that one wants to maintain the use of existing (de)coder ICs that implement functionality as (I)DCT [= first level of compatibility with image communication technologies]. But in addition there must be the second level of compatibility with displays that, because of their low peak brightness, need LDR images (i.e. an LDR look, not an HDR look with e.g. very dark in the darkest parts of the image, but instead with darker colors that have been lightened for better visibility on LDR screens) due to the fact that they can only render LDR images (i.e. the appearance of correct LDR under such screen dynamic range capability). This is due to the fact that, in addition to the legacy TVs currently employed, in the future there will be a spectrum of screens ranging from small portable screens with low brightness capabilities to laptop-type computers or pad-type computers or even mobile phones in which a consumer would also want to see some HDR film rendering, up to the most advanced HDR displays, which in the future may have a peak brightness of, for example, 10,000 cd/m2 (10,000 nit), and all displays in between or around them. So, although the display is still legacy and simple, it could be served by a color mapping and new decoding IC with high complexity in, for example, a future signal decoder or computer that delivers the HDR content through e.g. of an HDMI or other connection, with the signal decoder offering any combination of the invented and described options. It is noted that a legacy LDR image would need to be some optimization between intra-object and inter-object contrast. It would be desirable to see the object's interior textures clearly, yet one would also like to have, in the LDR image, an impression of the perhaps high-contrast HDR appearance of the original scene. That is, the difference between a high and low bright region may not be perfectly renderable with the LDR image, however, there should still be a remnant of it, making the lighting changes in the scene as ideally transportable as possible in LDR by human classifier.

[0027] These requirements have been converted into an approach in which an individual would need, in the ideal scenario, (at least) two gradations for the same film or the same pictures from the content provider, which will be called an LDR image (to be used for LDR screen scenarios, for example with screens with peak brightness around 100 cd/m2 (100 nit)) and an HDR image (for the brightest screens, for example a reference peak brightness of 5,000 cd/m2 (5,000 nit)).

[0028] So, for several practical example scenarios, there is a starting point for innovative HDR coding modalities as input from a master HDR classified image (that is, it is classified of its own accord, according to any that is to the creator's taste with whatever color processing software and, for example, encoded in a starting color encoding like OpenEXR, and can even be an update to an HDR look of an image that was originally captured as LDR, for example, by adding computer graphics effects). One then needs to encode this master HDR (M_HDR) in a way that is practically useful for current video and image coding technologies (that is, only minimally modified from the normal way of using such coding technologies that may involve a redefinition of the code meanings, i.e. the respective luminances encoded by various luma codes, but not that, for example, all buses need to be changed to 12 bit, i.e. the present methods must work with 12 bit hardware. bits, but also, if only 10 bits per component are available, or if lower quality is supported even on 8-bit systems), for, for example, a new BD disc player, or television IC that receives video streaming Internet, or any receiver connected to any image source that is broadly compatible with a variant of current image/video coding technologies.

[0029] An important understanding has been reached that an HDR image can be encoded as an LDR-looking image (that is, an image that, with little or no colorimetric processing - can be a conversion into another image space). color, but without any or much tone mapping to convert the brightness of image objects to be more suitable for a screen with a different dynamic range of luminance - can be directly used to screen with good quality on an LDR screen) if only if you add the parameters of the color mapping functions that can convert this LDR appearance into an HDR appearance image (the present mode ii). The reader should consider that this is not a trivial thing to do, even theoretically, certainly not a priori, but even after having defined the technical task (due to the fact that without the correct further development it would seem somewhat contradictory to code one appearance through another appearance that is supposedly different). Specifically, since many of the present embodiments start from an existing M_HDR, the functions can map the pixelated LDR appearance image into a nearby Rec_HDR reconstruction of the M_HDR. But logically this, in general, cannot just be accomplished in any specific ad hoc manner, i.e. a specific technical coding chain is required.

[0030] The present invention can be carried out, for example, in at least the following ways: a method of encoding a high dynamic range (M_HDR) image, which comprises the steps of: - converting the high dynamic range image into a luminance shortest dynamic range image (LDR_o) by applying: a) normalizing the wide dynamic range image to a luma axis scale that is [0, 1] producing a normalized wide dynamic range image with normalized colors that have normalized luminances (Yn_HDR), b) calculation of a gamma function on the normalized luminances that produce gamma-converted luminances (xg), c) a first tone mapping that reinforces these gamma-converted luminances of pixels of the high dynamic range image normalized value that lies below 0.1 with a predetermined quantity that lies between 1.5 and 5.0, producing lumas (v), and d) an arbitrary monotonically increasing tone mapping function that maps the lumas that results from performing of steps b and c to output lumas (Yn_LDR) from the shortest dynamic range image (LDR_o); and - output in an image signal (S_im) an encoding of the pixel colors of the image of shorter luminance dynamic range (LDR_o), and - output the image signal values (S_im) that encode the function formats of the conversions of color above b to d as metadata, or values for their inverse functions, which metadata allows a receiver to reconstruct a reconstructed high dynamic range image (Rec_HDR) from the shorter luminance dynamic range image (LDR_o).

[0031] This particular combination of luma conversion functions has proven to be a very good way of handling the encoding of HDR images in the mode ii mindset, that is, in particular, in the dual mode of encoding HDR images as LDR appearance derived by these conversion functions from an HDR master, whose LDR images serve the dual purpose of accurately encoding the HDR appearance as well as being very usable LDR appearance images.

[0032] It is noted that any coding of pixel colors, i.e. representing the colors of pixels in some color space coding system can be used, and some will be more pragmatic than others. For example, LDR_o could be output as a (R’G’B’) image [where the dashes indicate some non-linear mappings of linear RGB components]. An example is elucidated that has the ability to encode the LDR image to be communicated to a receiver in a Yu'v' manner and then also processing such as tone mapping can be performed on a Yxy representation with coordinates of chromaticity of xy as, for example, then u'v', but the same principles of the invention below can also be incorporated into other color representations, for example, linear RGB representations (i.e., calculations are then made with RGB components directly instead of Y components), etc. Also, those skilled in the art will understand which of the various ways can be used to co-code the parameters characterizing the functional mappings (which may be, for example, a multilinear function defined by its segment change points), which will generally be , as metadata in, or associable with, the S_im image signal (e.g., a SEI message structure or similar), meaning that at the time a receiver needs the parameters to make sense of the color data of coded pixels to transform them into, for example, a linear RGB output image for rendering on a connected screen, it must have obtained these appearance definition parameters through some data communication technology, for example, a memory location connectable through some communication link.

[0033] This specific combination of a “sensitivity” mapping that lightens at least the darkest object colors in an image (in a subrange of darker pixel colors, which can be defined, in practice, to be the 10 % of lowest luminances of a normalized linear image, which can be, for example, a Y channel, or the corresponding lowest linear RGB values) and a gamma/power function works well for manipulating the colorimetric characteristics of images. HDR and in particular its typical discrepancy with LDR rendering. Many LDR renderings could logically be considered, for example, a trivial rendering just ignores all colors that are considered too bright or too dark through limiting, but this is not necessarily the best LDR appearance, and certainly not It is useful for good quality HDR appearance reconstruction.

[0034] Since both an HDR image (directly in mode i) and an LDR image (in particular, an LDR image that is, in fact, encoding an HDR image) can, in fact, be encoded in a similar container, for example, 3x 10-bit HEVC container, one may wonder what is the difference between an HDR and LDR image. This difference is a calorimetric difference, which is very important when a screen, a viewing environment, and a viewer are involved. Mathematically, one can measure this from typical pixel values in the normalized image (i.e., a normalized LDR or HDR image) and, in particular, the histogram. If it is a regular LDR image, there will usually not be as high a contrast between the darkest and brightest pixel colors. Logically, in LDR images there can also be values that are white, as well as values that are black (zero), but these will correspond to the actual luminances to be rendered differently, due to the fact that there is a different code allocation function defining the same. Other innovative receivers/decoders will recognize the situation and apply the appropriate decoding in each case. When it is said that the LDR image needs to be directly usable on a legacy screen, it means that the receiver providing decoded images to the receiver will understand/decode the luma values with a legacy code allocation function, i.e. generally , the 2.2 gamma of Rec. 709. Now a master_HDR (i-mode) image can be encoded by an entirely different code allocation function, which means that a black luma of, say, 0.05 corresponds to a much darker black than for an LDR image, and 0.96 corresponds to a much brighter color. In addition to this, I introduce a newer concept that luma codes can now be related to HDR or can be lumas rendered by another code allocation, and that can even be variable depending on the choices made by the classifier (in particular, the customer curve)! Specifically, an i-mode image will not generally have a relatively uniform (brightly lit) histogram as in legacy LDR encoding, but will generally have a bimodal histogram, in which there are numerous darker objects, and numerous much brighter pixels ( for example, the outer pixels that could be 100x brighter in a linear luminance representation, but can be, for example, 3x brighter when using a specific code allocation function, i.e. in the ultimately used luma representation) . In some HDR images, the brightest pixels may also be fewer in number, for example, a pair of light bulbs in a night scene. In a mode ii image, the relationship will again be different. There will still be some sufficient difference between the regions (assuming, in the simple elucidation of this document, that HDR images are then formed), not only because relatively simple functions can map Rec_HDR, but also due to the fact that even direct LDR rendering may be desirable to retain some of the appearance of contrast. But on the other hand, the two luminance ranges may have been shrunk towards each other to a certain degree due to the limitations of the LDR scale. But what's important in all of this is that you can still see some signature of an image whether or not it comes from an LDR or HDR scene. Not only can mathematical image analysis algorithms analyze dynamic range appearances as encoded in images (e.g., for real-time television production, where the final quality of appearances is less important than, e.g., costs). production), for which an image analysis unit 177 can be used. But in general, present encoding technologies in their highest quality format will be used with a human color classifier at the end of creation, who can generally see on an HDR and LDR screen how the system behaves (i.e. is, as LDR and HDR actually appear to be), turn the knobs on his sorting keyboard and ultimately encode an LDR image and the HDR reconstruction functions he is satisfied with. Note that normally receivers do not need to carry out a total analysis of the situation. They do not, in themselves, need to worry about the possibility of the normalized image they received being an HDR image or an LDR image, and with which LDR image variant. They just need to “blindly” apply the functions received. The only thing they need to know, generally, is what the functions define and/or what the single image defines. So, generally, the signal will contain an indicator (IND) of what type of signal it is. Logically, many things may have been communicated about the signal, for example to televisions wanting to perform their own intelligent additional picture enhancement, but generally what a receiver needs to know minimally is that this HDR encoding signal S_im is from type that contains an image directly useful for LDR rendering (regardless of whether its appearance can be adjusted to a better LDR image by receivers that can tone map the received LDR image LDR_t with more tone mapping functions [parameterized with Ff1 , Ff2, etc.] or not). With this information, the receiver knows that if a connected LDR display is to be served with suitable images, the LDR appearance images can be directly sent to it, and if it is an HDR display, color transformations first. will be applied to obtain the correct HDR images for rendering. The skilled reader will understand that this can be indicated in several ways, for example, with a keyword such as “DIRECTLDR”, or with a set of numbers “100/5,000”, which indicates that the only image is an image intended to a 100 cd/m2 (100 nit) screen (real or reference screen) and was derived from and is mappable to a 5,000 cd/m2 (5,000 nit) HDR image (which does not mean that any other image for another brightness peak can be derived from the LDR image with the information in the parameters that define the color transformation functions) etc.

[0035] If one looks in a little more detail at what an HDR image might typically look like (when normalized and to be classified to an ideal mode II LDR image), one should understand how various scenes will typically be classified in a environment defined by HDR reference screen with a peak brightness of, for example, 5,000 or 10,000 cd/m2 (5,000 or 10,000 nit).

[0036] Again, taking the example of elucidating an indoor scene with a bright sunny outdoor environment, one may wish to classify the outdoor colors in M_HDR to approximately HDR mid-gray, so about 20% of 5,000 cd/ m2 (5,000 nit), that is, +1,000 cd/m2 (1,000 nit). Indoor colors should not be rendered with luminances typical of real indoor environments, as the film is viewed in another environment, for example, a normal darkened television viewing environment. So indoor colors should definitely not be rendered at 1/100° of the pixel luminances of sunny outdoor environments, since those are not also rendered exactly, just an exact copy on either receiving side of which the rating reference master on the reference screen would have been made. Consideration should be given to the appearance for the average adapted viewer, in particular that in the HDR appearance indoor environments should not appear unrealistically dark. You can classify these colors at, for example, 1/10° of the average luminance of the “sunny outer space” image region colors, so around +- 100 cd/m2 (100 nit). However, now, naively mapping these lumas onto a 100 cd/m2 (100 nit) LDR reference screen (with a color mapping that is, let's say, close to a linear extent at least conceptually), the colors of sunny outdoor spaces in LDR would appear perfect, at about 20 cd/m2 (20 nit) and up to white, but colors of indoor spaces would render at around 2 cd/m2 (2 nit), which would appear like psychovisual blacks . This is why “some” optimization must be performed, which can be quite complex depending on the complexity of a specific HDR scene, which is why, ideally, you have a human color classifier involved in the content creation side, for aspects of this coding framework. To make these indoor colors also reasonably viewable, you can make them somewhat darker than medium gray (18%), but not much more in optimization. So, one may wish to enhance these darker colors with a factor generally between 5 and 7 (depending on what is in the respectively bright dark regions, logically, the optimization may be different for a dark base where one should hardly see tools on the wall). and can limit the light from the lamp that illuminates them), keeping the brightest colors above them. Figure 5 shows two example scenarios of the present HDR/LDR coding chain. Curves 501 and 502 show only the first characteristic tone mapping (“sensitivity”) curve, i.e. before gamma. They are defined by , with possible normalized values for the log(7?/70)' input xg between zero and 1.0, and an ideal RHO value in case the HDR master is set with a reference screen brightness peak of f 1,000 cd /m2 (1,000 nit) for curve 501 (which means that whatever the content in the captured scene, object luminances in M_HDR are set between zero and maximally 1,000 cd/m2 (1,000 nit), the value for which, for example, a welding spark or the sun can be graded), and 5,000 cd/m2 (5,000 nit) for curve 502. The ideal RHO value can be determined in several ways, as the reader skilled in the art will understand. For example, the classifier may choose what it considers a good LDR appearance given the specific M_HDR image. Or, a device on the creation side can calculate it automatically, for example, according to the following equation:

[0037] In this equation PBHDR is the peak brightness of the reference screen associated with the M_HDR rating (that is, which defines the range of possible values, and typically corresponds to the PB of a real screen on which the grader studied and created his master HDR appearance), for example, 1,000 or 5,000 cd/m2 (1,000 or 5,000 nit) as in Figure 5, and GAM is a gamma value, which can typically be, for example, 2.4. Logically, the device (or classifier) can deviate from these values by any other algorithm or heuristic, for example, in case a somewhat brighter, or flat, appearance is required, etc.

[0038] Now, it can be seen in Figure 5 that if the reinforcement factors are determined (compared to the diagonal, with the normalized HDR luma being on the x-axis, and the normalized LDR luma on the y-axis) for the part of the first tone mapping/sensitivity tone mapping only for a value between +- 1.5 and 4.0, one has, after applying the gamma mapping with a gamma of 2.4 also, the gamma factors boost of about 6 to 7 for the darkest 10% of colors (curves 503 respectively 504 being the combined log and gamma mapping), which is roughly what is needed (the classifier can later adjust as desired with its arbitrary tone mapping curve, but it is a good strategy for, for example, autoconverting devices, which minimally involve the classifier just in case tuning is necessary or desirable). In general, an individual would generally like to have a boost of +- 4 to 8 for the combined log/gamma tone mapping operations (i.e., unit 602 and 603), which would mean that a boost value between 1.5 and 5.0 would be suitable for the sensitivity part based on RHO only (unit 603). Any tone mapping function for unit 603 that has such behavior for the darkest colors would satisfy what is desired for the present invention, but the above log-based equation is a simple pragmatic way of accomplishing this. The behavior for the lighter colors above will generally be a smooth compression, that is, with a function format that maps generally non-linearly to the lighter luminances above the range taken by the boosted darker colors. Now, one can have very complex HDR images, which may want other values, but such extreme situations can be handled by a suitable arbitrary curve definition by the classifier (or an automatic classification algorithm). Note that on the decoding side, the processing chain needs to be substantially invertible, to be able to calculate Rec_HDR from the single communicated LDR image (or images). “Substantially invertible” means that one does not necessarily have to obtain exactly the same color component values in Rec_HDR as in the original M_HDR, the color differences must be within a tolerance limit. Therefore, the receiver must ultimately have the ability to obtain the color transformation functions necessary to update the appearance of HDR Rec_HDR, regardless of whether it calculates them by inverting the drawdown functions originally used on the receiver side when causes the LDR_o (or LDR_i) to form the M_HDR and receive the format information from these functions, or by directly receiving the inverse functions necessary to perform the update to Rec_HDR. This, inter alia, will generally mean that for the arbitrary tone mapping function that the classifier can define to adjust the LDR appearance to its critical preferences, it will need to define a monotonically increasing function related to the LDR and HDR lumas normalized as per the skilled in the technique will understand.

[0039] The basic mode ii technical chain can work in a simple way. For example, for some less critical scenes, the classifier may occupy the arbitrary function with default values that are an identity transformation. It is also noted that although the basic technical components needed in the chain are described, in practical implementations, one or more of these blocks can be grouped into real units that perform the functions. For example, in some applications, it may be desirable to send a total LUT of all color mapping functions together, while in other applications, it may be advantageous to send the functions separately, due to the fact that the television (automatically, e.g. after analysis of the scene, or under user interface control by the viewer) may, for example, wish to additionally adjust, for example, the first function which brightens the image somewhat in comparison to the sensitivity or RHO value received from the image technology. image/video communication. More advanced versions may use some additional processing steps, for example the encoding method may determine a gain value (gai) to map the maximum luma of the shortest dynamic range image (LDR_o) to a specific value from the possible values in the reconstructed high dynamic range image (Rec_HDR), and to encode this gain value into the image signal (S_im), which should not be confused with the final scaling shape normalized colors relative to the peak brightness of the connected display ( for example, Lm=5,000 cd/m2 (5,000 nit)). This gain allows for more versatile classification and/or coding.

[0040] A very useful improved method of encoding a high dynamic range (M_HDR) image comprises: after applying any of the above color mappings to determine the shortest dynamic range image (LDR_o), applying a tone mapping additional technical skill (301) to determine a second shorter dynamic range image (LDR_i) that can be used to drive LDR displays as an alternative drive image alternative to the luminance shorter dynamic range image (LDR_o), which Technical tone mapping is determined by: a) determining a first geometric region of the shortest luminance dynamic range image (LDR_o) for which the image banding visibility in the corresponding reconstructed high dynamic range image (Rec_HDR) is above an acceptable level, b) a range of lumas (L_u) is determined for that region, c) a second pixel luma range (L_uu) is determined adjacent on the luma axis to the luma range (L_u), being that the second strip is identified to satisfy the conditions that it has several lumas above a minimum number (MIN), and corresponds to a second geometric image region that contains a texture that can be represented using less than the minimum number of codes in an LDR image (LDR_i) to which functions are applied that produce a reconstructed high dynamic range (Rec_HDR) image of sufficient visual quality for that second region, and d) a redistribution mapping function is determined that redistributes the lumas of the first and second luma tracks, so that additional codes are available for the first track, and outputs in the image signal values (S_im) that encode the function format of the redistribution mapping function, or otherwise preference, its inverse.

[0041] One method is somewhat limited with respect to the tradeoff between fully or sufficiently accurate reconstruction of the Rec_HDR, and the appearance of the LDR LDR_o image, in particular, if the hardware (and classification costs) dictate that relatively limited amounts of sorting functions need to be used. Some HDR scenes may not be that difficult (for example, the average viewer may not be very critical about whether the shadows on a shaded side of a sunny street are a little darker, or a little more light gray, as long as the deviations from the ideal appearance are not too excessive), but some HDR scenes may be more critical (for example, somewhere in the HDR luminance range there may be a boy partially hidden in the luminous haze, and if the existing local contrast is very high, it can be very visible, but if the contrast is too low, it can be invisible, changing the story). It would be advantageous to have another rating dimension possible, at least for non-legacy receivers (and don't know how to do any HDR processing), and can do some additional tone mapping. A legacy display could then get the “best effort” LDR image, which will be the only image transmitted, but future smart receivers could perform some clever technical tricks to further optimize the LDR appearance so that it approaches the that the classifier wants (perhaps even limits some values in the final LDR appearance, which would be incongruous with the HDR reconstruction if this happened in the only transmitted LDR image). If such a possibility exists, some coding methods or encoders must satisfy this need. Compress a very complex very high contrast ratio HDR image into a single LDR image (for example, an HDR image that has several important regions with many gray values, for example, a room with no dark lighting, a second room relatively well-lit, and, at the same time, an external space illuminated with colored sunlight, and with these 3 regions also containing important gradients that cover many gray values, for example, of a white table under the lamp in the well-lit room), It could occur that one or more regions become unacceptable, due to the limited word length (e.g. 10 bits) for the color components, anywhere (depending on the formats of the color mapping functions) there is banding in the image, which is considered very serious. This region can be identified, for example, by the classifier that observes it (and it can be helped by receiving indication of possible critical areas by the HDR image analysis software in the classification device). Image banding detectors can calculate, for example, that for an extended region (which potentially also takes into account what luminances that region has, and estimated JNDs) there are jumps at a time of countless successively identical colors, and can define an acceptable level based on the values from such a calculation (and typical factory experiments). The classification apparatus, after having found such a region (for example, through finer segmentation of what the classifier has approximately selected), can then determine approximately the range L_u of luminances that correspond to it. For example, there may be banding in the image in a blue sky, whose colors have luminances between L_sky_low and L_sky_high. The problem would be mitigated if the LDR encoding had more values to encode the image, and it must be understood that on the encoding side, M_HDR and any transformations can still have very high precision. But these codes do not exist: we only have 10 bits available for all the necessary luminances, and we also need to sufficiently encode all other image regions with different lighting. But a trick can be used if some codes can be borrowed from regions of the image that have luminances adjacent to L_u, especially if the visual quality of these regions worsens a little, taking some codes out of their code range (which the classifier usually will judge, through a simple operation of accepting the result, or disagreeing in relation to which case another attempt will be made, which is more aggressive in case the creation of bands in the image is still large for the originally banded area and the region adjacent region may still be more deteriorated, or a little less aggressive if the classifier indicates that the adjacent region begins to deteriorate a lot). A simple way to redistribute the codes is, for example, a linear or non-linear modification of the local function part. Now, the issue with the single image transmitted LDR_o, is that the sky may, for example, have been made a little too dark and perhaps too contrasty by this operation (and also the adjacent regions may be a little too dark). too dark, and its texture appearance may have changed, etc.). This may not be too much of an issue in the case of small changes and less critical scenes, and a bit more inconvenient for difficult scenes. This is the price that legacy systems may have to pay, due to the fact that they cannot do absolutely anything with any of the received data except for directly rendering the LDR_o, but new receivers can apply the inverse of the transformations used to redistribute the lumas, to create an LDR appearance very close to that originally intended (i.e. with the appropriate sky luminances, etc.), but now with less banding in the image. A receiver does not need to do much intelligent analysis, it only needs to see that such a technical tone mapping function is available, and it needs to apply the same to the reconstruction of the single transmitted LDR image LDR_t to obtain the best LDR appearance image LDR_ul. Numerous methods can also be applied to classification devices to provide good suggestions for the adjacent region, for example, a region with a sufficient amount of lumas (e.g., equal to the amount in the sky) and with some complex texture can be determined. Simple embodiments may, for example, use all codes below the range of banded regions, down to the darkest black.

[0042] The number of additional codes for the first stripe is determined based on a visibility criterion for creating stripes in the image for the first geometric region. An automatic algorithm might be at a ratio, for example 20% additional codes, and usually the human classifier will recognize this. The algorithm can also highlight the regions that it had to worsen, for example, leaving a coloration of these regions flashing, so that the classifier can quickly verify the possibility of these having sufficient visual quality also in the reconstructed HDR Rec_HDR.

[0043] In most practical embodiments, identification of the first geometric region that shows excessive banding in the image is typically performed ultimately by a human classifier via a user interface unit (105), e.g., by scribbling a wavy line along the banded region, and the amount of banding in the image of the first geometric region in the reconstructed high dynamic range (Rec_HDR) image and the visual quality of reconstruction of the second geometric region in the high dynamic range image reconstructed dynamics (Rec_HDR) be judged by the human classifier as acceptable or not acceptable, and in the case of acceptable judgment, the values that encode the function format of the redistribution mapping function or its inverse are encoded in the image signal, or in the case of unacceptable judgment, steps are performed with different parameters to result in an alternative redistribution mapping function. For example, 10% more code may be allocated to the banded region, perhaps at the expense of an enlarged adjacent luma band L_uu.

[0044] An interesting embodiment of the method of encoding a large dynamic range (M_HDR) image has the pixel colors of the shorter luminance dynamic range (LDR_o) image that are encoded as a luma channel and color coordinates u ' and v' which are calculated device-independent CIE 1931 color coordinates that are derivable for any RGB representation (i.e., the CIE 1976 chromaticity representation (u',v')). Normally, according to the legacy philosophy, images (especially LDR images) would be encoded as YCrCb images. But, if any codec is changed (for example for internet transmission), one can also encode the color components as Yuv component planes, which has some advantages both in the image quality of the transmitted images and in the ease to apply the various color transformations of the present system (logically, legacy televisions will then not have the capacity to produce good-looking figurations).

[0045] It was concluded that the chosen luma definition in general, (defined by the total tone mapping strategy chosen from the steps above, which ultimately obtains the lumas in LDR_o or LDR_i) which, together with two chromaticity coordinates luma-independent, in particular, the CIE-standardized u',v' coordinates, will be a good encoding to use in standard video or image compression technologies. Similar compression technologies, for example HEVC, will typically apply at least spatial compression by making DCTs of sample blocks, but for video, they can also do compression based on motion estimation, etc.

[0046] In simple embodiments, the coding of the functional color transformation behavior of color conversions can be communicated with just a few simple parameters by storing in the metadata associated or associable with the single image (or images): a) a sensitivity value (e.g., RHO, or an equivalent parameter that defines RHO called SENS and defined herein below, or any function or correlate of RHO that allows a value of RHO to be determined), b) a gamma value (GAM), and c) numerous values that characterize the functional format of the arbitrary function that maps the lumas.

[0047] A method of encoding a high dynamic range (M_HDR) image comprising determining a gain value to map the maximum luma of the shortest dynamic range image (LDR_o) to a specific value from the values possible in the high dynamic range image. reconstructed dynamic range (Rec_HDR), and encode this gain value into the image signal (S_im). This is useful for further scaling of the Rec_HDR image that can be obtained from the LDR image, for example if the LDR image of a relatively dark footage is represented relatively brightly, i.e. representing in lumas , a relatively high value in the range of possible LDR lumas, yet the HDR image must be rendered in a not too bright way, which is best handled by decoding it with a not too high maximum luma.

[0048] A method of encoding a high dynamic range (M_HDR) image comprising determining a saturation modification strategy from the colors of the high dynamic range image (M_HDR) to the colors in the shorter dynamic range image (LDR_o) , or vice versa, and encode this saturation modification strategy as parametric values in metadata in the signal (S_im). Generally, classifiers will also want to affect the saturation of an image, for example, they can change the saturation of LDR_o obtained from M_HDR with some saturation mapping strategy, and/or the saturation of Rec_HDR from LDR_o (e.g. example, first, a tone mapping that leaves the chromaticities of u,v of the Rec_HDR colors obtained at the values they had in LDR_o, and then changing the saturations of these Rec_HDR colors).

[0049] Some embodiments of the method of encoding a high dynamic range (M_HDR) image comprise: after applying any of the above color mappings to determine the shortest dynamic range image (LDR_o), applying a technical tone mapping ( 301) additional to determine a second shorter dynamic range image with redistributed lumas, i.e., lumas generally of slightly altered values for at least one geometric region and luma underrange of the image (i.e., a short dynamic range image luma redistributed LDR_i), which ensures that at least for the classifier, in the most important regions, of the second shortest dynamic range image (LDR_i), for example, which are carefully observed by the typical intended viewer, due to the fact that they are , for example, large and bright, and prone to image banding, sufficient luma codes can be allocated to encode the textures in these regions with sufficient precision to enable reconstruction of the reconstructed high dynamic range (Rec_HDR) image with errors below a predetermined error criterion (a minimum amount of banding in the image).

[0050] It is important that in some embodiments no unusual tone mapping strategy is chosen. Specifically, if it is desired to be able to obtain Rec_HDR with good quality, that is, close in mathematical pixel color values to M_HDR, then it is necessary to ensure that there are no textures not adequately sampled in LDR_i, which happens if it is guaranteed that the final mapping before uniform quantization is now very flat.

[0051] Generally, this can be done through luma counting strategies in the LDR figuration and/or luma counting strategies such as, for example, an image banding detector in the HDR image, or any such criterion of default HDR reconstruction error. In some embodiments, the criteria may be performed by a human classifier. Its presence could have been seen by having a technical remapping strategy co-coded in S_im, to be applied by smarter future generation receivers.

[0052] The method may be incorporated into an image encoder (100) arranged to encode a high dynamic range (M_HDR) image, which comprises: - a dynamic range conversion unit (104) arranged to convert the high dynamic range (M_HDR) image dynamic range into a shorter luminance dynamic range image (LDR_o), the dynamic range conversion unit (104) comprising, connected in processing order: a) a normalizer (601) arranged to normalize the high luminance image. dynamic range with respect to a luma axis that is in the range of [0,1] and to output normalized luminances (Yn_HDR), b) a gamma conversion unit (602) arranged to apply a gamma function to the normalized luminances and to output gamma-converted luminances (xg), c) a first tone mapping unit (603) arranged to apply a first tone mapping that enhances those gamma-converted luminances that fall below 0.1 with a predetermined amount that is lies between 1.5 and 5.0, producing lumas (v), d) an arbitrary tone mapping unit (604) arranged to d) apply an arbitrary function that maps the lumas (v) to output lumas (Yn_LDR) of the shorter dynamic range image (LDR_o); wherein the image encoder (100) further comprises: - an image compressor (108) arranged to apply a data reduction transformation to the colors of the shortest dynamic range image (LDR_o), the colors of which are organized into component images , and wherein the reduction transformation involves at least applying a DCT transformation to blocks of adjacent color component values, producing a compressed encoding (LDR_c) of the pixel colors of the shorter luminance dynamic range image; and - a formatter (110) arranged to output, in an image signal (S_im), the compressed coding (LDR_c), and arranged to, additionally, output in the image signal values (S_im), encode the function format of the color conversions as metadata, or values for their inverse functions, which metadata allows a receiver to reconstruct a high dynamic range (Rec_HDR) image based on the shorter luminance dynamic range (LDR_o) image.

[0053] A pragmatic embodiment of such an encoder is one in which the gamma conversion unit (602) uses a gamma value equal to 1/(2,4), and/or the first tone mapping unit (603) uses a tone mapping where RHO has a predetermined value, which value is generally a function of the peak brightness of the target served screen and/or the reference screen associated with the M_HDR master HDR encoding.

[0054] An image encoder (100) arranged to specify a gain that allows mapping the maximum of the luma codes in the shortest dynamic range image (LDR_o) to a luma value chosen from the reconstructed high dynamic range image (Rec_HDR), and which has the formatter (110) arranged to produce this gain as a value in metadata in the image signal (S_im).

[0055] An image encoder (100) as claimed in any one of the above encoder claims comprising a technical tone mapping unit (106), arranged to automatically or human-guidedly determine the texture and statistical information of the shortest dynamic range image (LDR_o) and in particular, at least one critical geometric region that is prone to reconstruction errors, in particular banding the image in Rec_HDR, and based on the same calculate a second tone mapping (Ff1 , Ff2, ...) to be applied as a transformation to the shortest dynamic range image (LDR_o) to produce a second shortest dynamic range image (LDR_i) that has a minimum number of luma codes (e.g. 1 ,3*L_u) that characterizes the textures of at least some important error-prone regions of the second shortest dynamic range image (LDR_i), thereby allowing reconstruction of the reconstructed high dynamic range image (Rec_HDR) with errors below of a predetermined error criterion. To enable communication of the necessary information that allows, after encoding, a receiver to mirror-implement the present mode II system, it is useful to transmit (or store, for later transmission) a high dynamic range image signal ( S_im) comprising: - a pixelated shorter dynamic range image (LDR_o) with encoded pixel colors; and additionally: - a sensitivity value (RHO); e - a gamma value (GAM); and - a gain value (GAI); and - a set of values that specify an arbitrary tone mapping function format (P_CC).

[0056] From these values, the receiver can then determine the function formats of all functions to be applied to the single communicated LDR image (LDR_o, or LDR_i), if any image of greater dynamic range than the image of LDR 100 cd/m2 (100 nit) is necessary and calculated.

[0057] Specifically, the S_im may also comprise values 207 that encode a technical remapping strategy (Ff1, Ff2, ...) for mapping between an artistic LDR classification as desired by the human content creator/classifier, and a technical LDR which, when sampled, has enough lumas for all regions of the image for good Rec_HDR reconstruction, or at least those regions determined to be most critical by an automatic image analysis unit and/or a human.

[0058] Specifically, it is useful, although very pragmatic, for receivers to quickly determine which of several (many) different possible current HDR image coding mechanisms is used, in particular, by understanding, in the image signal S_im, a indicator (IND) that specifies that a high dynamic range image has been encoded in it, and with a method that encodes it as a short dynamic range image, that is directly useful, without a need for additional tone mapping, for rendering in an LDR screen. Various coding modes can be devised and agreed upon, as long as any receiver understands this.

[0059] A memory product such as a Blu-ray disc stores any modality of the present high dynamic range image signal (S_im).

[0060] To have an image communication chain, at the receiving end there may be several embodiments of apparatus that are or comprise an image decoder (150) arranged to receive a high dynamic range image signal (S_im) and comprising: - a deformatter (151) arranged to obtain a compressed pixelated shorter dynamic range image (LDR_c) and parameter data (P) out of the image signal (S_im); and - a decompressor (152) arranged to apply at least one inverse DCT transformation to the compressed pixelated shorter dynamic range image (LDR_c) to obtain a pixelated shorter dynamic range image (LDR_t); and a dynamic range conversion unit (153) arranged to transform the shorter dynamic range image (LDR_t) into a reconstructed high dynamic range image (Rec_HDR), the dynamic range conversion unit (153) comprising, in the processing order: a) an arbitrary tone mapping unit (402) arranged to apply an arbitrary tone mapping, the parameters defining the same (P_CC) being received in the parameter data (P), b) a first tone mapping unit (403) arranged to apply a mapping as defined by at least one received parameter (RHO) that defines the first tone mapping that was previously determined by any of the encoder or encoding method embodiments, and c) a gamma conversion unit (404) arranged to apply a gamma mapping to a received gamma value (GAM).

[0061] This decoder will first undo all typical legacy compression encodings, e.g. HEVC or similar, and then apply the various mappings in reverse order (note that not everything needs to be, in all embodiments, exactly in the order inverse; for example, in Yu'v' one may choose to do orthogonal luma and saturation processing in a reverse order, perhaps with slightly different mathematical functions, as long as the end result is exactly or approximately the intended color). It should also be noted that there may be additional processing steps, which may exist only at the receiving end (for example, an image may be encoded in some RGB representation such as Rec. 2020, but may need to be converted to another format as per understood by a television, e.g. DCI-P3, and additionally converted to the TV's actual primary colors).

[0062] Then, the image decoder (150) will comprise a dynamic range conversion unit (153) arranged to transform the shorter dynamic range image (LDR_t) into a reconstructed high dynamic range image (Rec_HDR), and may there will normally be logical units and more color processing functions that define at least when to do what is desired (for example, depending on which screen or screens are currently connected and served).

[0063] A pragmatic embodiment of the image decoder has the first tone mapping unit (403) arranged to apply a function of the formula: , where v is a pixel luma, and RHO is a (RHO-1) real-valued parameter or integer received in the parameter data (P).

[0064] A useful embodiment of the image decoder (150) comprises a tone remapping unit (159) arranged to apply an additional tone mapping (Ff1, Ff2, ...) received in the image signal (S_im) to the image of shorter dynamic range (LDR_t) to obtain a second shorter dynamic range image (LDR_ul) which reverses a code redistribution action applied by any of encoder methods 5 to 7 producing a second short dynamic range image ( LDR_i) with redistributed lumas to achieve banding in the reduced image in at least one region of the reconstructed high dynamic range (Rec_HDR) image. In fact, the coder does not necessarily need to know exactly how any coder arrives at a specific transformation function that redistributes the lumas, he only needs to apply the inverse functions to substantially obtain the intended LDR appearance (LDR_ul).

[0065] Another useful embodiment of the decoder may comprise Yu'v' encodings of the LDR image, and in relation thereto, comprises a color transformation unit (155) arranged to convert a color representation of Yu'v' in an RGB color representation. Tone mapping can be performed before the conversion is done, therefore leaving the conversion to RGB for the last part of the processing chain, or alternatively, the conversion can be performed first, and equivalent color processing can be done on the signals. of RGB.

[0066] In correspondence with any of the decoders, the corresponding methods of decoding a high dynamic range image signal (S_im) comprise obtaining a reconstructed high dynamic range image (Rec_HDR) by applying the color conversions encoded in the parameter data (P) for the shorter dynamic range image (LDR_t), in particular, a method of decoding a high dynamic range image signal (S_im) comprises: - obtaining a compressed pixelated shorter dynamic range image (LDR_c) and parameter data (P) outside the image signal (S_im); decompressing the compressed pixelated shorter dynamic range image (LDR_c) by applying at least one inverse DCT transformation to the compressed pixelized shorter dynamic range image (LDR_c) to obtain a pixelized shorter dynamic range image (LDR_t); and transform the shortest dynamic range image (LDR_t) into a reconstructed high dynamic range image (Rec_HDR), by: a) applying an arbitrary tone mapping, with the parameters that define it (P_CC) being received on the parameter data (P), b) applying a mapping as defined by at least one received parameter (RHO) that defines the first tone mapping that was previously determined by any of the encoder or encoding method embodiments , and c) applying a gamma mapping with a received gamma value (GAM) which is preferably equal to 2.4. A system is described that allows the classifier to simply powerfully optimize the appearance of an LDR appearance in an HDR image of an HDR scene. Preferably, as little sacrifice in visual quality as possible is made, but since LDR may need some optimization due to its dynamic range limitations, the system allows the classifier to adjust the microcontrasts of scene objects. interesting for the same specific, that is, objects with a generally important characteristic in this scene and, therefore, if any sacrifice of brightness quality needs to be made, the precise appearance of some less important objects such as a wall in the background, instead of the main object in the scene. The invention can be carried out in many other (practical) ways such as with intermediaries containing the main technical requirements of the various embodiments such as the definition parameters incorporated in the signals, and many applications thereof can result, such as various ways of communicating, using, transforming color etc. the various possible signals, and various ways of incorporating the various hardware components, or of using the various methods, in consumer or professional systems. Any component can logically be realized in or as a small component, or vice versa as a key core of a large apparatus or system that functions predominantly due to that component.

BRIEF DESCRIPTION OF THE DRAWINGS

[0067] These and other aspects of the method and apparatus according to the invention will be evident and will be elucidated with reference to the implementations and embodiments hereinafter described, and with reference to the accompanying drawings, which serve merely as specific non-limiting illustrations that exemplify the broader concept.

[0068] Figure 1 schematically shows an example of an embodiment of an encoder and a decoder according to the present invention, in an image communication technology;

[0069] Figure 2 schematically shows an embodiment of what an HDR S_sim image signal according to the present invention may look like;

[0070] Figure 3 schematically elucidates, through a species, how one can obtain, in a generic way, a technical LDR classification, which could even occur below coverage automatically without disturbing the classifier or content creator in some modalities, which allows for better sampling of object lumas and therefore better quality Rec_HDR reconstruction;

[0071] Figure 4 is a simple schematic elucidation of a possible decoder in accordance with the present invention;

[0072] Figure 5 shows possible two x two modality curves to perform sensitivity mapping that significantly brightens colors, or combined initial LDR classification, in addition to gamma behavior; It is

[0073] Figure 6 is a simple schematic elucidation of a possible dynamic range conversion unit of an encoder.

DETAILED DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 depicts a typical exemplary system incorporating the present invention, with an image (or video) encoder 100 on one side of the creation, and an image decoder 150. It is assumed that there is a memory 101 in a system grading file that contains a master graded HDR appearance (M_HDR) image that has been graded as desired by the content creator according to currently known color grading techniques for, say, a movie in color grading software such as , for example, from Da Vinci (similar to other systems can benefit from the techniques in the present application, for example, M_HDR can be sourced directly from a camera, after, for example, an adjustment of a camera appearance curve in the camera dials, etc.). In this M_HDR, for example, the brightness of the light shining through windows can be chosen to provide a more pleasing appearance on the [0.5000] cd/m2 ([0.5000] nit) reference screen when provided with these pixels an intended luminance to be rendered L_out in a corresponding luma code v_HDR, and many additional light effects may have been designed, as well as other color optimizations. The M_HDR is input via image input 115 in the present encoder 100, and can also be viewed on an HDR reference screen 102 (which has exact characteristics of the theoretical reference screen, e.g., [0 to 5,000] cd/m2 ([0 to 5,000] nit) are proposed for HDR encoding). This means that when the classifier wants to perform an LDR appearance (it must not only encode the object textures sufficiently precisely so that on the receiving side a reasonably accurate Rec_HDR reconstruction of the M_HDR can be obtained, but also that LDR appearance must be suitable to ideally render the HDR scene encoded on an LDR screen), the classifier can at the same time compare how the LDR appearance, given technical limitations, looks on the LDR screen. LDR 103 similar to M_HDR, and can optimize by changing the color mapping functions to get the same from M_HDR as desired according to your taste. The two screens may be in different ideal viewing environments and the classifier may be looking at both separated by, for example, a wall (e.g., in two confined reference environments with their respective window opening to simultaneously look at them, and with curtains that can be closed if the classifier wishes to see only one of them during some period of time). The grader may also check the reconstructed rating of the HDR appearance on the HDR screen 102 (e.g., switching Rec_HDR and M_HDR alternatively).

[0075] By means of a user interface unit 105 that provides the classifier with classical controls such as, for example, rotating wheels or similarly slides to set values such as a gamma or sensitivity value, the classifier can perform colorimetric transformations that define how the M_HDR is to be mapped to the LDR appearance image, with the parameters of the transformations to be output into an image signal S_im via an encoder output 116 that may be connectable to any image transmission medium 140, for example, a communication network, or a physical carrier memory like a BD or solid state memory etc.

[0076] The LDR appearance is generated by means of a dynamic range conversion unit 104, which is arranged to apply colorimetric transformations at least to the pixel color lumas, but also generally to the chromaticity coordinates. By “lumas” we mean any encoding that is ultimately convertible into a physical luminance, or even through psychovisual models, a glow (which is the final appearance a viewer will see when the image is rendered in a screen). Note that, by equivalent mathematics, luma transformations can be applied as corresponding transformations to RGB components directly. Although the ultimate goal is the correct brightness of the object (appearances) in appearance, one may limit the present technical discussion to determining luminances in the reference range, e.g., [0 to 5,000], or a device-independent color space. as XYZ defined by that range. Furthermore, it is assumed that any chromatic transformations of the colors are made on the UCS plane of 1976 CIE Luv space, however, one skilled in the art can understand how similarly other second and third color components can be used, the basic components being of the present invention are generally applicable.

[0077] CIELuv defines ueva from XYZ (similarly, it can be transformed from some RGB) as:

[0078] It is assumed, for the sake of simplicity, that the HDR and LDR scales (i.e., the theoretical screen scales associated with the coding mathematics of the two images) have the same three (or more) R, G, B , primary colors, and can therefore, scaling the respective maximum of, say, 5,000 and 100 cd/m2 (5,000 and 100 nit) to 1.0, be colocalized as exactly overlapping. So a tone mapping from HDR to LDR then becomes a relative transformation along the luma direction normalized to this single device-dependent RGB scale. For example, if it is desired to make the darker colors in the HDR appearance be the same on the LDR and HDR screen, this becomes, as a relative transformation on the same scale, the following: due to the fact that in a color definition set to 5,000 cd/m2 (5,000 nit) such colors in the HDR image will have small codes (for example, below 0.1), they need to be brightened to become sufficiently visible on an LDR screen of 100 cd/m2 (100 nit), for example, with values around 0.3. The exact mapping will depend on the luma definition for both the LDR and HDR image, due to the fact that as a generalization of the “2.2 gamma” definitions of the legacy LDR image and video encoding, it may We now define the arbitrary code allocation function that maps from physical luminances to luma codes (or, conversely, due to the fact that TV engineers start by defining a reference screen that, in addition to addition from a reference range of [0 to 5000] cd/m2 ( [0 to 5000] nit) has some reference screen EOTF behavior that indicates how, for example, lumas 1024 map to renderable luminances over that range of reference). Not only can one use a power of 1/(7.0) gamma as OETF, but one can also use the discontinuous code allocation functions when shooting images where there are no luminances present within a lower range of luminances. and a superior range of luminances. It is also noted that working on a representation of Y’uv with independent chromaticities of luma (u,v) allows us to work in a totally independent and free way in the achromatic and chromatic directions of color space.

[0079] Limiting the present elucidation to the reader versed in relation to HDR-2-LDR achromatic mappings only, these can be generally formulated as, in principle, an arbitrary tone mapping function of the lumas [0, 1] of the HDR appearance image to the lumas [0, 1] of the LDR appearance image, as can be seen with an example in Figure 2a.

[0080] By specifying such a function, it can be assumed that the mapping in all colors (Y_M_HDR, u,v) is carried out so that for a non-achromatic color (u<>u_wp, v<>v_wp) where ( u_wp, v_wp) are the chromaticity coordinates of a white point chosen as D65, the determined tone mapping function 210 is scaled non-linearly to a maximum luminance L_max(u,v) capable of being obtained for that color, as taught in more detail in document WO2014056679. The skilled reader can understand how such processing instead of being applied in a Y'uv color coding can also be similarly carried out in an RGB color coding.

[0081] Once the classifier specifies a tone mapping behavior, the encoders have sufficient information for a brightness dynamic range transformation to be applied to any possible color in M_HDR, producing an original LDR appearance (uncompressed, possibly not yet quantized into a fluctuation representation). From this, any exact or approximate mathematical transformation can be determined by the encoder, which allows a receiver to perform the prediction in another way, from LDR_o to Rec_HDR. The grader may check via an image output 111 what such an image (after sufficiently formatting into an image signal that can be communicated via an image communication link such as, for example, HDMI) would look like on an LDR screen 103 reference (say, 100 cd/m2 (100 nit), or in the future, perhaps, 500 cd/m2 (500 nit)).

[0082] However, it will be taught in the present invention that it is useful when the tone mapping is not just constructed in any generic way, but in a specific way, and the (few) corresponding parameters are usefully encoded as separate metadata in the image signal S_im, due to the fact that they can then be advantageously used on a receiving side, for example during tuning to derive an ideal drive image for a specific screen of X cd/m2 (X nit).

[0083] As a first parameter, the classifier will choose, for example, a sensitivity parameter SENS, or RHO directly. This will be a value that is intuitively similar to known ASA or ISO values in photography, and generally determines how bright the LDR image will appear (inter alia, how much the dark M_HDR object's colors are produced).

[0084] According to a preferred embodiment, the encoder can use an EOTF/OETF function that already provides a good initial LDR appearance,

[0085] whose EOTF function is defined as follows:

[0086] This equation defines the HDR luminances L to be rendered corresponding to the luma codes v in [0, 1] dispersed equidistantly based on the amount of bits available for the luma codeword of the pixel colors, such as, say, 1,024 possible values. Lm is a chooseable variable that indicates the peak brightness of the reference screen from the linear color/luminance representation of M_HDR or Rec HDR, which can, for example, be set to 5,000. For example, the classifier will have dials to choose sensitivity which can generally be reported for rho as:

[0087] Along with the SENS value (RHO) that determines the behavior of dark colors and some appearance of general brightness, the classifier can co-adjust the gamma (GAM) as some bending parameter that relocates the brightness of the object/region to the over the range of possible LDR lumas. Of course, when mapping the luminances L onto a reference XYZ space representation of the M_HDR classification (which can be a useful intermediate representation), to luma v values of the LDR appearance, the classifier will define the inverse function.

[0088] When carrying out elementary mathematical calculations in the division of RHO, it can be seen that the inverse function (OETF) is: first applied to a production of and then calculate:

[0089] Generally, in the encoder there may be one of several possible embodiments of an image analysis unit 177. This unit may be arranged with artificial intelligence to analyze regions in the image, and each of these regions could produce particular problems in HDR coding. , in particular, mode type ii Specifically, it can identify regions that could be prone to banding in the image, and regions that are sufficiently textured, so that they can be encoded with a smaller amount of image component codes. color and/or luma. In some applications, this unit can automatically arrive at a final coding proposition (e.g., a transcoder) without any human classifier involvement, but in other applications it can, for example, place regions under the classifier's attention, so that the even examine them. Of course, there may be an interaction with the user interface, for example, the classifier could indicate that it wants to mitigate banding in the image with a particular region, or with a particular texture, and then the unit 177 may extract such a region and your luma range etc.

[0090] As can be seen in Figure 2, although one may decide to encode a final tone mapping function in generally a reserved LUT space in metadata 205, one will generally encode a sensitivity parameter (e.g., ISO 200) or a RHO value, and a gamma value (e.g., 2.8) in respectively sensitivity metadata field 202 and gamma metadata field 203. Figure 2 schematically shows how an image or video signal S_im (200) looks, and one skilled in the art will certainly know that it can be defined in practice, but many digital variants, due to existing image data containers etc. The encoder embodiments use a classical 3-component encoding of the pixel color image (201), which will be the classifier-optimized LDR appearance image. This LDR LDR_o image will generally be classically DCT encoded, run encoded, formatted, etc. according to a standardized image coding format similar to JPEG, or a standardized video coding format similar to MPEG-HEVC, VP1, etc. The reader skilled in the art will understand that colorimetric reformatting to be able to reuse legacy (or similar future) encoding technologies as a generic concept is part of this invention, but it is not so important that such encodings are, in fact, used. And another part of this invention is the metadata needed to be coherent with the data, for example, at least when retrieving the HDR Rec_appearance of the scene (due to the fact that the LDR appearance can, in theory, be used directly to trigger a screen of LDR, with no additional dynamic range processing, but just colorimetric reset mapping of Y'uv to some device dependent RGB space coding).

[0091] Additionally, the classifier may use a gain value (co-encoded in a gain metadata field 204) so that the functions do not need to map 1.0 to 1.0 by themselves. For example, the gain can indicate how an LDR image that is defined across the full [0.1] range should be mapped from, say, just a [0.1500] subrange of the [0.5000] screen range. of HDR. The other way of limiting the LDR range used is also in principle possible, although less likely to be used. This gain can be used to produce some not very bright images, as one might imagine if the scene is, for example, a hazy scene, or a dark image that is reasonably brightened in LDR but needs to remain dark in HDR.

[0092] These three parameters (RHO, GAM, GAI) already provide a very useful first mapping of an M_HDR image to a corresponding LDR appearance image, with a practically global illumination or brightness adjustment. This may, for example, be sufficient for broadcasting real-life shows, where the ideal parameters are determined just before broadcasting begins. More critical users, such as film producers, may want fine-tuned control over appearance. They may wish to specify a tone mapping function broader than the “loggama” above, with well-placed folds in the curve that can increase, for example, the average local brightness or contrast of a particular object (e.g., a face). for a desired underreach of all renderable LDR luminances (or more precisely, their corresponding lumas). Or a specification of a local slope may specify the desired contrast in some interesting BL subrange of an important region in the image, at the cost of positions and brightness contrasts of other regions/objects in the LDR appearance image.

[0093] Now, an important issue that one must understand is that with the i-mode system (HDR appearance), the classifier can define such mappings arbitrarily, due to the fact that one only needs to derive an image from LDR appearance (which is not reconstruction, but can be done by data destruction, if desired by the classifier), due to the fact that in this coding approach we have the HDR appearance image already encoded as a single image in the signal of S-im image. In mode-ii systems, however, it is necessary to fulfill a double criterion: on the one hand, one must have the ability to reconstruct the Rec_HDR image with good quality, but, on the other hand, one wants sufficient freedom to create most if not all of the appearances of LDR that a classifier might desire (and then can be quite creative at times, as an individual can see, for example, in the movie Sin City 2).

[0094] But it must be understood that whatever the LDR_o classification, the classifier was made with its preferred tone mapping 210, in a legacy encoding these output LDR lumas will be subjected to classical uniform quantization (and even doing DCT). So, one must be careful not to create mappings that are too flat in some parts of their range (i.e., the local delta_LDR_out/delta_HDR-in derivative must not be too small, so that a minimum required amount of LDR luma codes is allocated to that range delta_HDR-in or the corresponding delta_LDR_out), due to the fact that otherwise, when boosting that range in LDR-2-HDR tone mapping, artifacts will be seen, such as banding in the image or artifacts of visible DCT with excessive contrast.

[0095] One could have a control mechanism with a rigidity of local control points that the user uses to change the format of the arbitrary tone mapping, but which is unpleasant for the user, especially if implemented in a very adverse way (certainly, the system can warn if the classifier is waiting to perform very unusual mapping curves, for example, inversions such as an N-curve should not be done).

[0096] A useful embodiment is shown in Figure 3, which elucidates the behavior of a technical tone mapping unit 106, which can be used to determine a second LDR appearance, alternatively useful for LDR_o by more intelligent receivers that need to assist a LDR screen.

[0097] It is assumed that the classifier has chosen its desired curve that provides the appropriate LDR appearance, which is the solid curve in Figure 3. If the tone mapping curve is not good, this will mean that there is at least one range that is very flat, which is assumed in this document to be the R-u part of the HDR with the brightest LDR pixels, in other words the sky of the scene. It needs to be possible to extend this L_u range in LDR, so that a little more luma codes can be allocated, and in a manner both non-destructive (little change in their appearance) and where possible for the classifier.

[0098] This can be done when there is an adjacent strip L_uu that contains more textured object.

[0099] This is a way out of the conundrum that the appearance curve for acquiring a desired LDR appearance at the same time determines the quantization or number of luma codes available to reliably encode the various HDR region textures (the faithful characterization enough of all the textures that are in the scene which are the primary objective of encoding quality in HDR encoding). Having 1024 different luma/gray levels (and millions of codes) should be enough to encode all textures well for human vision, if done right. Complex objects can be encoded with relatively less code, as the eye sees the rough texture pattern first and then not so much the precise pixel color values. Only in particular unfavorable situations may there be an issue, if there are brightness gradients for which very few codes were used.

[00100] So, there are two things when adapting a curve: the technical tone mapping unit 106 generally keeps the adaptation when necessary sufficiently local on the luma axis, so that there is no disorientation of the lumas of many object colors ( for example, again avoid darkening critical dark regions too much). A quality criterion for this example scene may be that it is necessary to lighten the dark colors to achieve a good LDR appearance, so a local change in the light colors will not disorient it in any way. Then, the unit tone mapping 106 will generally redistribute the codes in the same local luma subrange around the problem area and determine a corresponding adaptation curve for this, which is the dotted line (this curve may somewhat follow the shape of the original curve, in its two image region coding parts, that is, if there was a parabolic slant location format for the sky lumas, can generally use a similarly larger slant parabolic segment, scaled to the air , but that is not absolutely necessary, since only coding accuracy is the criterion).

[00101] Therefore, one needs to extend the brightness range of the sky region in some way, to have enough codes to faithfully encode a Rec_HDR blue sky gradient. But how much is needed and how much can the R_Adj adjustment range be extended?

[00102] This depends on several things. Of course, R_adj must cover the region in which there is a problem, which will generally be a relatively simple-looking region, such as relatively uniform regions like a gradient in the sky (this blue gradient will exist in some form throughout the LDR luma range). ). On the other hand, you must need an adjacent region that is sufficiently textured. In the unlikely situation that the adjacent region is yet another smooth gradient (which could occur in synthetic images, such as artificial gradient test images, which will have to be satisfied with whatever optimal luma allocation there is, but this does not generally occur in natural images), R_adj can become relatively large. In the normal situation you will find a textured strip that can extend L_u with a strip L_uu of a size that depends on how many codes were added and the complexity of the texture pattern. If you only need to add 3 codes for the sky, you need to save 3 luma codes in L_uu and if it is textured enough it could be done beyond a range of y 10 to 15 lumas depending on what the classifier or viewer thinks /may find it acceptable.

[00103] The device may contain tables for this.

[00104] So the nasty problem with appearance curve-dependent luma coding is now largely solved. On the one hand, it does not darken the adjacent darker objects too severely, as it only changes the colors of L_uu a little in the upper band expanding the L_u sky band, but largely keeps the lower part L_uu's the same, just sampled a little less, which isn't a visually notable issue anyway, due to the fact that textures don't need as much coding anyway. The expanded skyband may be a little sub-optimal, but it shouldn't really be a problem, and you get an improved Rec_HDR quality in return. However, this only occurs if no contrary action is taken at the receiving end, for example by a receiver that cannot perform any processing. Due to the fact that in the decoder a pre-compensation strategy can be carried out in the tone remapping unit 159. This will then make luma allocation a purely technical matter that is not of interest to the artistic intentions of the classifier. Due to the fact that the tone resurfacing unit 159 will apply the correction for local extension in a compression again, before using the resulting intended LDR appearance (LDR_ul) to, for example, trigger an LDR screen. So, in the sky example, where by extending the lower sky boundary of L_u downward into the brightness of objects in the adjacent range L_uu (thereby dimming those objects), the tone resurfacing unit 159 of a 150 decoder will apply the inverse mapping of 301 as a correction. This means that, visually, the sky strip will have its original luma range L_u again and, when rendered on an LDR screen, the correct luminance range, although it will have more precision due to the fact that more luma codes have been allocated. of texture encoding. Similarly, in LDR_ul appearance, the object with adjacent brightness in L_uu will also have the correct unregulated brightness, and only differs in accuracy due to the reduced number of codes. And one skilled in the art can understand how this technique can always, in many other possible situations, improve the coding accuracy in those regions of an image where necessary, while maintaining the LDR appearance predicted by the classifier LDR_ul. The only thing that the tone resurfacing unit 159 needs to be able to do is apply a tone mapping strategy to the encoded technical LDR_t, for example, through a LUT, which can be co-coded into the S_im signal (or partially encoded if the tone mapping can be derived from, for example, a limited set of control points, e.g. delimiting linear segments) and consequently it should be clear why it is advantageous to encode this technical adjustment function separately (Ff1, Ff2,...) in S_im, due to the fact that the same can be used by the decoder even to acquire a more desirable LDR appearance LDR_ul, since the same has been determined on the creation side and accepted by the classifier and communicated to a receiving side.

[00105] There will be broadly two categories of encoder modalities that will enable the above. The first largely does all processing automatically, and does not need to involve the user. The smoothness and texture detectors will automatically categorize the various regions and then identify the gradient pattern in the sky and the objects located adjacently (i.e. in the luma range located below and/or above L_u) other textured objects. Various texture characterizers can be built in to determine texture complexity (e.g. no fine granules, amount of interlocking gray values, etc.) and determine from this how visually noticeable interferences that lead to fewer encoding lumas will be , and the resulting required range L_uu. As mentioned, these preferences can be pre-built into formulas that determine the functionality of L_uu or with LUTs. Additionally, in some embodiments, DCT or other compression emulators may be present, for example, which calculate the resulting decompressed images from LDR LDR_d through various choices for R_adj and the functional tone mapping disturbance format 301, and calculate a measurement Severe to typical visibility (at normal viewing range, screen size, surrounding brightness, etc.) of image banding and/or other compression artifacts. The texture analysis unit 117 may be present for this, which is generally arranged to analyze textures and in particular their appearance impact, both in the original (LDR_o) and in the encoded LDR_c, or in the fact that the decoding of the same LDR_d which will ultimately be present at the receiving end. Specifically, HDR remaps per LDR-2-HDR 118 color mapping unit can be used to enable the classifier to check appearance impact if necessary. If the classifier wishes to check the ability to reconstruct this M_HDR as Rec_HDR, it can, for example, switch them at the same time on its HDR screen 102, through HDR image output 119. In fact, the decoder can have several outputs (which were shown separately, but they can certainly be routed internally to just one output) 111, 112, 113, 114 to be able to check the different LDR versions.

[00106] A second category of encoders with technical reclassification may directly involve the human classifier. If he is already checking the quality of automatic algorithms, he may have an option to influence the results (i.e., usually semi-automatically). This should be simple for the classifier, as he may wish to be more involved with the artistic determination of appearance, i.e. the placement of object lumas, rather than technical issues such as compression artifacts (if he is already waiting to look at the same, and although it will verify one or more typical and approved scenarios, the image communication line can certainly be additional compressions that could have more severe artifacts).

[00107] In these encoder embodiments, the user interface unit 105 will generally allow the classifier to specify geometric image areas that, according to it, are particularly problematic areas. For example, it can scribble across the sky, and the histogram analysis and texture analysis units will then focus on that part of the image when they perform their analysis and technical update partial tone mapping curve determination. For example, they can successively propose a strategy that adds a few more luma codes at a time to the sky, until the classifier is satisfied. For example, a modality algorithm of the tone mapping unit 106 may multiply this range of the gradient object (sensitive to banding in the image) by k = e.g., 1.5 and select a range close to an image region textured and compact it to L_uu-1.5*L_u. That is, any curvilinear or linear redistribution of the codes in the two regions can be used. The L_uu can be selected to be at least, for example, 3*L_u, in which the values are generally optimized by a device projector based on a set of representative images. If the proposition by the device is good, the classifier accepts it, making the encoder store the corresponding parameters in S_im or, otherwise, a new iteration is started, for example, with k=1.1*1.5.

[00108] Disturbance 301 will lead to a final tone mapping, which corresponds to a final technical classification LDR_i, which will be the appearance of LDR that is sent in the communication system after additional formatting according to the HDR coding system of mode-ii and which broadly corresponds to what the classifier wants the LDR to look like.

[00109] The advantage of classifier involvement lies in the fact that it can indicate - at least with a minimum of involvement - which regions are semantically most relevant. The statistical texture analyzer may determine that few lumas (i.e., few pixels) actually exist in a region between, for example, the dark lumas of indoor environments, and the bright lumas of sunny outdoor environments, and accordingly decide to apply a remapping strategy that applies few codes to it (in this case, the decoder remapper 159 can arbitrarily reconstruct the desired LDR appearance, one can even use a strong technical deformation curve that almost cuts out the entire underrange used so limited LDR_i encoding, thereby making internal environments and external subranges immediately adjacent in LDR_i value). However, if in this small region there is an important object, such as someone's face or an object that has been emphasized by someone as an object appearing, the classifier may counteract it. Several practical modalities are possible, for example, one can scribble a rectangle around that region on the drawing and then rotate a dial that increases the number of luma codes to be used for that region. The reader skilled in the art will understand that there are several other ways for the user interface to select an important region or object in the image or photo, and to indicate how they should be encoded with lumas, even down to the classifier that draws or influences the format of the image itself. modification curve 301.

[00110] The rest of the mode-ii system is as follows:

[00111] Optionally, the dynamic range conversion unit may perform some color saturation processing (e.g., since coloration decreases with darkening and vice versa, the classifier may wish to compensate for the saturation that has become, in some way, mode, inappropriate due to luma tone mapping). A good practical exemplary modality works with a general saturation function of the non-information destructive type. This also means that this saturation function is, in some part, very flat, so it can also be reversed. But in some embodiments, the saturation function may only need to be applied in the LDR-2-HDR update, and then it may be more liberal. In Figure 3, a smooth saturation from s_in to s_out was shown, which can be encoded with numerous S1, S2, S3 values in a LUT on the S_im signal. These could be s_out values for equidistant s_in values (a sufficient amount so that the desired curve can be reasonably smoothly recovered in the decoder), but this would also be, for example, function-shaped control points. A desaturation function can, for example, be encoded as a line with a slope of less than 45 degrees (in a graph of s_in versus s_out). In such a case of desaturation, the image signal could only have an integer or swing value for the multiplier in the metadata. It is assumed in the elucidation example that s_out will be the saturation of the HDR image, and it is necessary to reinforce the saturation of the darker, now darkened, colors of the scene to increase the color, but one skilled in the art can understand that there may be different processing variants within the same structural coding philosophy. It is assumed, for the sake of simplicity of elucidation, that the saturation is carried out in the uv space, for example, regardless of the luma, the operation s_out = s_in+MS(s_in)*s_in can be performed. The MS(s_in) is then the retrievable multiplier value of the function, as seen in 2b and encoded in LUT 206, which stretches the saturation vector in a hue direction compared to some white point. It is assumed, for the sake of simplicity, that the present UV space was defined in a cylindrical space with maximum saturation at the periphery (and coded as 1.0). Logically, one skilled in the art will understand that one can encode the present saturation strategy in another colorimetric definition, or given that the definition is, for example, in cylindrical Y'uv space, the designer of the hardware or decoder software can choose to actually perform the same in an equivalent way in another color space, such as the YCrCb space based on RGB, etc. The classifier can also determine and encode, in S_im, luma-dependent saturation strategies, that is, functions that change saturation, whose multiplier varies with the luminance of the processed color. Basically, a more advanced embodiment of S_im will have a saturation coding structure. This could be, for example, a web-based definition that has numerous key hues (e.g. the 6: RGBCYM) a multiplier function defined over luma: MS(Y’). From this, what can be encoded as 6 LUTs of similar values to 206, at the receiving end the decoder can determine a saturation strategy for all colors in the scale through interpolation. A more complex strategy can even introduce saturation variability in the radial direction. This can be easily coded by determining these functions (similar to those seen in Figure 2b, but now variable over the height of the luma on the scale) in a simply parametric way, for example, as displacement, gamma, gain functions. In this case, one has: s_out = s_in+F(s_in, Y') for the key hues, and, for example, in the case of a three-parameter function format control, one can encode in S_im as much as 3x6 LUTs that specify the luma behavior of, for example, the saturation_gamma parameter as variant over Y', or 6 LUTs for the hues, but where not a single multiplier value is encoded at each position, but a trio [sat_offset(Y' _i), sat_gain(Y'_i), sat_gama(Y'_i)]_LUT_of_yellow, consecutively through countless i positions that sample the possible lumas on the scale.

[00112] Now, in some embodiments of an encoder (and corresponding decoder), there is an optional transformation to u'v' for the color characteristics of the pixels, which will now be elucidated (but other embodiments may alternatively or additionally encode, for example, in R'G'B' or YCrCb etc. directly, and do not have, at least, the optional unit 107 on the inside; It is also noted that for some Yu'v' the processing can be mathematically rewritten as equivalent linear RGB processing).

[00113] Having applied dynamic range transformation to create the correct LDR appearance (e.g. in RGB space, or XYZ etc.), it is assumed that the mapping has not yet been done in Y' space uv, the color transformation unit 107 of the exemplary elucidation modality will not convert to the present representation of u'v', and the lumas Y' in this color representation are determined by the present total tone mapping function (i.e. is, the intermediate LDR image lumas LDR_i), and u, v as per the above equations. Colorimetric transformations can also be carried out in unit 107, which condition the colors when an RGB dependent on a different device or multi-primary space is predicted. For example, if the present M_HDR was encoded with a smaller RGB triangle, but the LDR is for a wide-scale screen, the classifier may already predefine a saturation boost strategy, although things are often reversed, in which case case, the unit 107 may implement a chromatic scale mapping.

[00114] Finally, the resulting LDR_uv is encoded with a classical LDR image or video compressor 108, i.e. generally DCT or wavelet transformed etc.

[00115] This compressed LDR_c image is sent to a formatter 116, which adds the metadata into the applied mapping function according to a standardized format so that it is appropriately available on a receiving side. That is, this formatter helps the sensitivity value (RHO or alternatively, SENS) to be tone mapped additionally to adjust the LDR appearance as normally determined by the human classifier (although, in the near future, some encoders may be smart enough to perform the adjustment themselves) with function definition parameters 205 generally as a LUT of values (F1, F2, ...), the saturation encoding 206, for example, also, a set of parameters that define a multilinear function etc.

[00116] Additional tone mapping, for technical reasons, is generally stored separately in the image or video signal S_im, preferably as a set of integer or real values 207, which can be used to store, for example, a LUT 256 points or 1,024 points.

[00117] The encoded LDR_c can be decoded back to LDR_d, and then updated by the color mapping unit 118 so that the classifier can see through the image output 119 what the reconstructed HDR Rec_HDR would look like at a receiving end . If you wish, you could even test the influence of some typical compression settings, for example, even strong compression. The decoder described here could also be used in a recoding strategy, where the classification appearance may have already been prepared previously, but now, for example, a low-quality, highly compressed LDR version is determined again for some specific application. image/video communication. This secondary classifier can even readjust the parameters. Depending on whether it has the original M_HDR available, it may, for example, re-determine the devaluation functions to obtain a new, more appropriately adjusted LDR appearance (e.g., serving mobile phone viewers), and then In fact, it can even do this when it only has the good Rec_HDR available instead of M_HDR. Separating a technical classification part to more appropriately allocate luma codes is very useful for such scenarios. Because the functions mapping LDR_o (and its corresponding nearby reconstruction LDR_ul) determine the actual artistic appearance of LDR, they may have been determined once and for all by the primary classifier at or near the time of the initial content production. But, the encoder can still automatically or semi-automatically, with the involvement of the secondary classifier, determine the technical mapping with the small modifications like 301, and the corresponding LDR_i (or LDR_t), and the encoded metadata Ff1, Ff2, in the set of real values or integers 207 in S_im, which can logically be different for different technological limitations, such as the amount of bits (e.g. only 8 bits for the luma channel).

[00118] The decoder 150 may be an IC in, for example, as in this elucidation, a signal decoder or computer connectable to a screen 160 or television (so when it says decoder invented to cover any small realization thereof as a “ signal decoder on a USB stick” or any embodiment of large apparatus and benefits from the present invention as a signal decoder with hard disk and optical disk reading facilities, and the encoder can be anything from a small device to a large rating system etc.), but logically the television may not be a limited monitor but comprises all this decoding technology in its own IC. Display 160 can be either an LDR display or an HDR display, or basically any display connected via any image communication technology via image output 157, such as wireless streaming to a portable multimedia device or a professional cinema projector.

[00119] The decoder obtains the present formatted S_im via image input 158, and a deformatter 151 will then separate it into an LDR_c image (IMG in Figure 2) for decompression by a JPEG-type or JPEG-type decompressor. Classic MPEG 152, and the metadata P parameters (for example, a sensitivity setting of 1000, and some values that can be used to reconstruct a working tone mapping or saturation mapping format). Optionally, in the decoder is the tone resurfacing unit 159, due to the fact that since this remapping technique is normally not a gross deformation of the LDR LDR_ul appearance intended by the classifier, some decoders may support or ignore the same. Fully HDR-capable decoders must, however, use this unit 159 to apply a technical re-correction strategy as encoded in the Ff values 207, to arrive at the corrected LDR LDR_ul appearance (which is a close approximation of LDR_o). This corrected LDR (LDR_ul) image is directed to an additional screen color adjustment unit 154. This unit 154 can apply the optimization required for a specific wide-scale display, say, 1,300 cd/m2 (1,300 nit) (tuning). Although variants are possible, a typical decoder for the present HDR encoding philosophy has been designed, which has an image processing path to recover the LDR_ul (or if 159 is not present, its approximation LDR_t), but also has a second image processing path to determine Rec_HDR. This is performed in the dynamic range conversion unit 153, which generally applies to inverse mappings applied in the encoder (indeed, in the signal an encoder will generally encode the parameters of this inverse mapping, i.e. an update). The screen color adjustment unit 154 will generally be arranged to combine the information in the two classifications, which could be accomplished based on the use of only one image and the color mapping parameters P, but it is assumed that in this elucidation mode , which takes an image of Rec_HDR and LDR_ul as input and then interpolates those, according to which screen with which brightness peak is connected to be provided with the appropriately classified images.

[00120] With the exception of tone mapping to obtain the correct brightness appearance, a color transformation unit 155 may generally be understood to be arranged to perform chromatic adaptations to optimize for a color scale other than the coding scale (e.g. , Rec. 2020 for DCI-P3 or Rec. 709 etc.).

[00121] What will be produced through image output 157 and, therefore, calculated by unit 154, will logically depend on the connected screen. If it is an LDR screen, unit 154 may send, for example, LDR_ul, after logically correct color remapping (by unit 155) from the Y'uv encoding to an R'G' encoding B' dependent on specific device, for example. If the connected screen 160 is close to a peak brightness screen of 5,000 cd/m2 (5,000 nit) (see also how the decoding apparatus can ask a TV for its capabilities in WO 2013/046096; a controller 161 may perform such communication with the screen and even with the viewer to obtain their preferences, and may be willing to configure how the screen adjustment unit 154 should behave and what type of image appearance it should calculate and produce) the Rec_HDR appearance image can be produced, again after proper formatting according to what the television wishes to receive (i.e. this can still be a Y'uv encoding, e.g. the present S_im format with , now an HDR appearance image stored in 201/IMG, and some functional metadata may also be transmitted so that the television can perform some final colorimetric appearance adjustment based on information about how the rating changes across the spectrum of possibilities rendering image as encoded in this metadata, or may already be an R'G'B' HDR screen trigger image). For displays with intermediate peak brightness, unit 154 can produce a suitable drive image, again in the present Y'uv format or in another format.

[00122] Finally, the content creator can describe in the signal whether he wants the unit offset mapping 159 not to be skipped, for example, due to the fact that the content creator thinks that LDR_t seriously deviates from LDR_ul . This can be done by encoding a Boolean 209 in a field of the IGNORE_TECHNICAL_MAPPING metadata.

[00123] It should be clear to the reader that where only the minimum of a set of parameters has been elucidated, certainly along the same several rational sets of functional color mapping metadata can be encoded in S_im, for example, a set for go from the single IMG image (which is an LDR image) to a reference, for example, a [05000] nit HDR appearance image, and a second set can be added to go to, for example, an MDR appearance of 1500nit. And while performing a specific decomposition of a function format with sensitivity, range, gain, and additional tuning is advantageous, and at least good for technical elucidation, any of the mappings, e.g., the LDR-2-MDR mapping can be encoded in S_im in a condensed form, for example, filling only the LUT tone mapping or value set 205, which encodes the final mapping function (i.e., all together, sensitivity, tuning and technical mapping).

[00124] Figure 4 schematically shows a typical embodiment of the present decoder main unit 400 (in this example, the minimum part of mode ii, without technical relationship or Yu'v' conversion etc.). After a decompressor 401 does run length and arithmetic decoding, and inverse DCT etc., an LDR_t image is obtained, which is assumed to be in gamma 2.2 representation. (i.e. with lumas or R’G’B’ components defined according to Rec. 709) and normalized. There may be a first control unit 420, which may directly send this image to a connected LDR TV 410 (which directly means that there may logically be some legacy formatting involved; in principle, LDR_t could also be, for example, an image linear, in which case there will be a need to regamma-2.2 it before sending it to the LDR screen, but it may be advantageous if not necessary; additional tone mapping functions will generally be different. depending on what type LDR_t is, which can also be indicated with an IND_2 indicator in S_im). Then, a first tone mapping unit 402 performs the inverse mapping of the arbitrary tone mapping, and the defining parameters of this P_CC function format are received in the MET(F) metadata. Then, a second tone mapping unit 403 performs tone mapping by again darkening darker colors relative to brighter ones, for example, applying the rho equation above, with a received RHO value. The unit could also calculate the RHO value from a PB_HDR peak received screen brightness received from the connected HDR screen 411. Next, a third tone mapping unit 404 performs a gamma power function with a received GAM value that is, for example, 2.4, preferably. Then a 405 multiplier can perform a multiplication with GAI, which by default can be 1.0. Optionally, a color saturation processor 406 may perform some saturation processing. Finally, the control unit 421 may send the image to the HDR screen 411, and it may perform some additional processing, for example, to correctly format the image according to a pattern that the screen understands, for example, through of an HDMI connection.

[00125] Figure 6 shows an embodiment of a simple encoder dynamic range conversion unit. It comprises a normalization unit 601 to normalize all color components to 1 (i.e., if, for example, R, G and B are normalized to 1.0, then the maximum luminance will be normalized to 1.0, and vice versa). The Yn_HDR normalized pixel luminances of HDR images (or in equivalent embodiments, e.g., the normalized linear RGB components) go to a first tone mapper 602 that performs a gamma operation, with a gamma as desired by the classifier (or unit automatic classification), but ultimately settles at 1/(2,4). Then, a second tone mapper 603 performs the transformation that appropriately lightens the dark HDR colors, e.g. with a suitable RHO factor proposed by the classification system depending on the dynamic range difference between (the peak brightness of) M_HDR and the LDR typically 100 cd/m2 (100 nit) LDR, and generally accepted ultimately by the classifier, which may or may not change this initially proposed RHO value. Then, using the third tone mapper 604, the classifier begins fine-tuning by searching for various objects in the image, and finally defines a standardized tone mapping curve CC, by changing several of these lumas according to the objects. important image images of the classifier. This produces the lumas Yn_LDR and the LDR_o image, with all the data ready to be encoded.

[00126] The algorithmic components disclosed herein may (in whole or in part) be obtained in practice as hardware (e.g., parts of an application-specific IC), or as software running on a special digital signal processor, or a generic processor, etc. .

[00127] One skilled in the art will understand from our presentation which components may be optional enhancements and may be designed in combination with other components, and how the (optional) steps of the methods correspond to the respective apparatus means, and vice versa . The word “apparatus” in this application is used in its broadest sense, namely, a group of means that allow achieving a specific objective, and can thus be (a small part of) an IC, or a dedicated device (such as a device with a screen), or part of a networked system, among other things. The term “arrangement” is also intended to be used in its broadest sense to include, among other things, a single apparatus, a part of an apparatus, a set of (parts of) apparatus operating together, etc. .

[00128] A computer program product version of the present embodiment as a denotation should be understood as encompassing any physical embodiment of a set of commands that enables a processor for general or specific purposes, after a series of load steps (which may include intermediate conversion steps, such as translating to an intermediate language, and a final processor language) enter the commands into the processor and execute any of the features of the invention. In particular, the computer program product may be conceived as data on a carrier, such as a disk or tape, data present in a memory, data moving over a network connection - wired or wireless - or program code on paper. In addition to program code, characteristic data necessary for the program may also be incorporated as a computer program product. It must be clear that, by computer, we must understand any device capable of performing data computations, that is, it can also be, for example, a mobile phone. Also, device claims may cover computer-implanted versions of the embodiments.

[00129] Some of the steps necessary for the operation of the method may already be present in the processor functionality rather than described in the computer program product, such as the data input and data output steps.

[00130] It should be noted that the modalities mentioned above illustrate, and do not limit, the invention. At points where one skilled in the art can easily map the examples presented to other regions of the claims, for the sake of brevity, not all options have been mentioned in depth. In addition to the combinations of elements of the invention as combined in the claims, other combinations of the elements are possible. Any combination of elements can run on a single dedicated element.

[00131] Any parenthetical reference sign in the claim is not intended to limit the claim. The use of the verb “understand” and its conjugations does not exclude the presence of elements or steps not mentioned in any of the claims or in the present description. The word “one” or “an” before an element does not exclude the presence of a plurality of such elements.

Claims (6)

1. METHOD OF CODING A HIGH DYNAMIC RANGE IMAGE (M_HDR), comprising the steps of: - converting the high dynamic range image into a shorter luminance dynamic range image (LDR_o), the method being characterized by calculating the luminance shortest dynamic range image (LDR_o) by applying: a) normalizing the wide dynamic range image to a luma axis scale that is [0,1] producing a normalized wide dynamic range image with normalized colors that have normalized luminances (Yn_HDR); b) calculation of a gamma function on the normalized luminances that produce gamma-converted luminances (xg); c) a first tone mapping that produces lumas (v) which is defined as v = log(1+(RHO-1)*xg)/log(RHO) where RHO is a constant that has a predetermined value; and d) an arbitrary strictly increasing tone mapping function for the lumas to obtain the luma emission (Yn_LDR) from the shortest dynamic range image (LDR_o); and e) outputting in an image signal (S_im) an encoding of the pixel colors of the shortest luminance dynamic range image (LDR_o), and f) outputting in an image signal (S_im) as metadata values encoding the format of function of the above color conversions b to d, whose metadata allows a receiver to reconstruct a reconstructed high dynamic range image (Rec_HDR) from the shorter luminance dynamic range image (LDR_o), a value that is a function of RHO is emitted in the metadata. 2. METHOD OF CODING A LARGE DYNAMIC RANGE IMAGE (M_HDR), according to claim 1, characterized in that the gamma function calculation uses a gamma value equal to 1/(2,4). 3. METHOD OF CODING A LARGE DYNAMIC RANGE IMAGE (M_HDR), according to claim 1, characterized by determining a gain value (gai) to map the maximum luma of the shortest dynamic range image (LDR_o) to a specific value from the possible values in the reconstructed high dynamic range image (Rec_HDR), and encode this gain value into the image signal (S_im). 4. IMAGE ENCODER (100) ARRANGED TO ENCODE A HIGH DYNAMIC RANGE (M_HDR) IMAGE, comprising: - a dynamic range conversion unit (104) arranged to convert the high dynamic range image into a high dynamic range image. shortest luminance (LDR_o), the image encoder being characterized by the dynamic range conversion unit (104) comprising: a) a normalizer (601) arranged to normalize the large dynamic range image with respect to a luma axis that is in the range of [0,1] and to emit normalized luminances (Yn_HDR); b) a gamma conversion unit (602) arranged to apply a gamma function to the normalized luminances and to output gamma-converted luminances (xg); c) a first tone mapping unit (603) arranged to apply a first tone mapping that yields lumas (v) which is defined as v = log(1+(RHO- 1)*xg)/log(RHO), where RHO is a constant that has a predetermined value; d) an arbitrary tone mapping unit (604) arranged to apply an arbitrary strictly increasing function that maps the lumas (v) to emit lumas (Yn_LDR) of the shortest dynamic range image (LDR_o); wherein the image encoder (100) further comprises: - an image compressor (108) arranged to apply a data reduction transformation to the colors of the shortest dynamic range image (LDR_o), the colors of which are organized into component images , and wherein the reduction transformation involves at least applying a DCT transformation to blocks of adjacent color component values, producing a compressed encoding (LDR_c) of the pixel colors of the shorter luminance dynamic range image; and - a formatter (110) arranged to output, in an image signal (S_im), the compressed coding (LDR_c), and arranged to, additionally, output in the image signal values (S_im), encode the function format of the color conversions as metadata, which metadata allows a receiver to reconstruct a large dynamic range image (Rec_HDR) based on the luminance shorter dynamic range image (LDR_o), the values comprising a value characterizing RHO. 5. IMAGE DECODER (150) ARRANGED TO RECEIVE A HIGH DYNAMIC RANGE IMAGE SIGNAL (S_IM), characterized by comprising: - a deformatter (151) arranged to obtain a compressed pixelated shorter dynamic range image (LDR_c) and data parameter (P) of the image signal (S_im); and - a decompressor (152) arranged to apply at least one inverse DCT transformation to the compressed pixelated shorter dynamic range image (LDR_c) to obtain a pixelated shorter dynamic range image (LDR_t); and - a dynamic range conversion unit (153) arranged to transform the shorter dynamic range image (LDR_t) into a reconstructed high dynamic range image (Rec_HDR), the dynamic range conversion unit (153) comprising : a) an arbitrary tone mapping unit (402) arranged to apply an arbitrary strictly increasing tone mapping, wherein the parameters defining the arbitrary strictly increasing tone mapping (P_CC) are received in the parameter data (P); b) a first tone mapping unit (403) arranged to apply a mapping as defined by a wherein the constant RHO information is received in the parameter data (P); and c) a gamma conversion unit (404) arranged to apply a gamma mapping with a gamma value (GAM). 6. METHOD OF DECODING A HIGH DYNAMIC RANGE IMAGE SIGNAL (S_IM), characterized by comprising: - obtaining a compressed pixelated shorter dynamic range image (LDR_c) and parameter data (P) out of the image signal (S_im ); decompressing the compressed pixelated shorter dynamic range image (LDR_c) by applying at least one inverse DCT transformation to the compressed pixelized shorter dynamic range image (LDR_c) to obtain a pixelized shorter dynamic range image (LDR_t); and transform the shortest dynamic range image (LDR_t) into a reconstructed large dynamic range image (Rec_HDR), by: a) applying an arbitrary strictly increasing tone mapping, with the parameters that define it (P_CC) are received in parameter data (P); b) the application of a mapping as defined by the constant (RHO), where the mapping is defined by c) applying a gamma mapping with a gamma value (GAM) that is equal to 2.4.