WO2024215185A1

WO2024215185A1 - Method for processing video signal and device therefor

Info

Publication number: WO2024215185A1
Application number: PCT/KR2024/095711
Authority: WO
Inventors: 김경용; 김동철; 손주형; 곽진삼
Original assignee: 주식회사 윌러스표준기술연구소
Priority date: 2023-04-13
Filing date: 2024-04-15
Publication date: 2024-10-17

Abstract

This video signal decoding device comprises a processor, wherein the processor can: obtain a plurality of pairs consisting of one reference line and one intra prediction mode, the plurality of pairs including a first pair and a second pair; obtain a list consisting of a plurality of combined pairs including a first combined pair in which the first pair and the second pair have been combined; and restore a current block on the basis of one combined pair in the list.

Description

Video signal processing method and device therefor

The present invention relates to a method and device for processing a video signal, and more particularly, to a video signal processing method and device for encoding or decoding a video signal.

Compression coding refers to a series of signal processing technologies for transmitting digitized information through communication lines or storing it in a form suitable for storage media. The targets of compression coding include objects such as voice, image, and text, and the technology for performing compression coding on images in particular is called video image compression. Compression coding for video signals is performed by removing redundant information by considering spatial correlation, temporal correlation, and probabilistic correlation. However, due to the recent development of various media and data transmission media, more highly efficient video signal processing methods and devices are required.

The purpose of this specification is to provide a video signal processing method and a device therefor to improve the coding efficiency of a video signal.

The present specification provides a video signal processing method and a device therefor.

In the present specification, a video signal decoding device includes a processor, wherein the processor obtains a plurality of pairs each composed of one reference line and one intra prediction mode, the plurality of pairs including a first pair and a second pair, and obtains a list composed of a plurality of integrated pairs including a first integrated pair that integrates the first pair and the second pair, and can reconstruct a current block based on one integrated pair in the list. In addition, the processor obtains a first prediction block based on a first reference line and a first intra prediction mode of the first pair, obtains a second prediction block based on a second reference line and a second intra prediction mode of the second pair, obtains a final prediction block by performing a weighted average of the first prediction block and the second prediction block, and can reconstruct the current block based on the final prediction block.

In this specification, a video signal encoding device includes a processor, and the processor can obtain a bitstream to be decoded by a decoding method.

In the present specification, a computer-readable non-transitory storage medium can store a bitstream decoded by a decoding method.

In the present specification, a decoding method may include the steps of: obtaining a plurality of pairs each composed of one reference line and one intra prediction mode; obtaining a list composed of a plurality of integrated pairs, wherein the plurality of pairs include a first pair and a second pair, and including a first integrated pair that integrates the first pair and the second pair; and reconstructing a current block based on one integrated pair in the list. In addition, the decoding method may further include the steps of: obtaining a first prediction block based on a first reference line and a first intra prediction mode of the first pair; obtaining a second prediction block based on a second reference line and a second intra prediction mode of the second pair; obtaining a final prediction block by weighting and averaging the first prediction block and the second prediction block; and reconstructing the current block based on the final prediction block.

In the present specification, each of the plurality of pairs may be composed of a combination of different reference lines and intra prediction modes.

In this specification, the one integrated pair in the list can be indicated based on a syntax element included in a bitstream.

In the present specification, the plurality of integrated pairs constituting the list can be sorted based on the cost of each of the plurality of integrated pairs.

In this specification, the plurality of integrated pairs can be sorted in order of their corresponding costs.

This specification provides a method for efficiently processing a video signal.

The effects that can be obtained from this specification are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by those skilled in the art to which the present invention belongs from the description below.

FIG. 1 is a schematic block diagram of a video signal encoding device according to one embodiment of the present invention.

FIG. 2 is a schematic block diagram of a video signal decoding device according to one embodiment of the present invention.

Figure 3 illustrates an embodiment in which a coding tree unit within a picture is divided into coding units.

Figure 4 illustrates one embodiment of a method for signaling splitting of a quad tree and a multi-type tree.

FIGS. 5 and 6 illustrate an intra prediction method according to an embodiment of the present invention in more detail.

Figure 7 is a diagram showing the locations of surrounding blocks used to construct a motion candidate list in inter prediction.

FIG. 8 is a diagram illustrating a process of generating a prediction block using DIMD according to one embodiment of the present invention.

FIG. 9 is a diagram showing the locations of surrounding pixels used to derive directional information according to one embodiment of the present invention.

FIG. 10 is a diagram illustrating a method for mapping directional modes according to one embodiment of the present invention.

FIG. 11 is a diagram showing a histogram for deriving an intra prediction directional mode according to one embodiment of the present invention.

FIG. 12 is a diagram illustrating a method for signaling a DIMD mode according to one embodiment of the present invention.

FIG. 13 is a diagram illustrating a method for signaling syntax elements related to intra prediction modes depending on whether DIMD mode is used according to one embodiment of the present invention.

FIG. 14 is a diagram illustrating a method for generating a prediction sample for restoring a current block according to one embodiment of the present invention.

FIG. 15 is a diagram illustrating a method for determining an intra prediction mode according to one embodiment of the present invention.

FIG. 16 illustrates a syntax structure including syntax elements related to DIMD according to one embodiment of the present invention.

FIG. 17 is a diagram showing intra prediction directional mode and weight information for surrounding blocks of a current block according to one embodiment of the present invention.

FIG. 18 is a diagram illustrating a method for determining DIMD combination information according to one embodiment of the present invention.

FIG. 19 is a diagram illustrating a method for generating a prediction sample using intra prediction directional mode information and weights according to one embodiment of the present invention.

FIGS. 20 and 21 are diagrams showing pixel values of surrounding blocks used when deriving an intra prediction directional mode according to one embodiment of the present invention.

FIG. 22 is a diagram illustrating a method for constructing an MPM list including an intra prediction directional mode of a current block according to one embodiment of the present invention.

FIGS. 23 and 24 are diagrams showing templates used to derive an intra prediction mode of a current block according to one embodiment of the present invention.

FIGS. 25 to 28 are diagrams illustrating a method for generating prediction samples (pixels) based on a plurality of reference pixel lines according to one embodiment of the present invention.

FIG. 29 illustrates a method for predicting a sample using multiple reference pixel lines according to one embodiment of the present invention.

FIG. 30 illustrates a method for determining a reference pixel line based on a template according to one embodiment of the present invention.

FIG. 31 illustrates a method for setting a template for testing a reference pixel line adjacent to a current block according to one embodiment of the present invention.

FIG. 32 is a structural diagram illustrating a method for determining an optimal reference pixel line using a plurality of reference pixel lines based on a template according to one embodiment of the present invention.

FIG. 33 illustrates a method for generating prediction samples using planar mode according to one embodiment of the present invention.

Figure 34 shows a transformation set table for LFNST and NSPT transformations according to one embodiment of the present invention.

Figure 35 shows a low frequency region of a current block according to one embodiment of the present invention.

FIG. 36 illustrates a method for deriving a multi-transform set and a LFNST/NSPT set according to one embodiment of the present invention.

FIG. 37 illustrates a method for deriving a multi-transform set and an LFNST set for a vertical planar mode or a horizontal planar mode according to one embodiment of the present invention.

Figure 38 shows a mapping table according to one embodiment of the present invention.

Figure 39 illustrates a conversion type set table according to one embodiment of the present invention.

Figure 40 shows a conversion type combination table according to one embodiment of the present invention.

Figure 41 shows a threshold value table for an IDT conversion type according to one embodiment of the present invention.

Figure 42 shows samples at a block boundary and around the boundary in a deblocking filtering process according to one embodiment of the present invention.

FIG. 43 illustrates a method for a video signal processing device according to one embodiment of the present invention to derive a first or second transform matrix.

Figure 44 illustrates a DC prediction method for vertical or horizontal line units according to one embodiment of the present invention.

Figure 45 illustrates a method for predicting a plane in units of vertical or horizontal lines according to one embodiment of the present invention.

Figure 46 illustrates a sub-block unit prediction method using a planar mode according to one embodiment of the present invention.

Figure 47 illustrates an intra prediction method based on bidirectional prediction according to one embodiment of the present invention.

Figure 48 shows types of transform kernels that can be used for video coding according to one embodiment of the present invention.

Figures 49 and 50 illustrate a part of a sequence parameter set according to one embodiment of the present invention.

FIG. 51 illustrates a portion of the general_constraint_info() syntax structure according to one embodiment of the present invention.

FIG. 52 illustrates a method of constructing an MPM list using a multi-prediction mode according to one embodiment of the present invention.

FIG. 53 shows the locations of reference pixels used when a prediction block is generated using a multi-prediction mode according to one embodiment of the present invention.

FIG. 54 illustrates a method for a video signal processing device according to an embodiment of the present invention to parse planar mode selection information and DC mode selection information from a bitstream.

FIG. 55 illustrates binarization or bin strings for planar mode selection information and DC mode selection information according to one embodiment of the present invention.

FIG. 56 illustrates a method for generating a prediction block for a current block according to a DC prediction mode according to one embodiment of the present invention.

Figure 57 illustrates a method for determining DC mode selection information according to one embodiment of the present invention.

FIG. 58 illustrates a method for generating a prediction block for a current block according to DC mode selection information according to one embodiment of the present invention.

FIG. 59 illustrates a method for generating a prediction block for a current block according to an integrated direction mode according to one embodiment of the present invention.

FIG. 60 illustrates a binarization or empty string for an integrated direction mode according to one embodiment of the present invention.

FIG. 61 illustrates reference pixels for generating a prediction block for a current block according to one embodiment of the present invention.

FIG. 62 illustrates a method for a video signal processing device according to one embodiment of the present invention to configure a reference pixel line list.

Figure 63 illustrates a method for constructing an integrated list according to one embodiment of the present invention.

Figure 64 illustrates a method of combining new candidates according to one embodiment of the present invention.

Figure 65 shows a combination table for reference pixel lines and intra prediction modes according to one embodiment of the present invention.

Figure 66 illustrates a method of using a candidate list based on multiple prediction modes according to one embodiment of the present invention.

FIG. 67 is a diagram showing a block vector related to an IBC encoding method according to one embodiment of the present invention.

Figure 68 is a diagram illustrating a method for predicting the current block using RRIBC in the horizontal direction.

Figure 69 is a diagram illustrating a method for predicting the current block using RRIBC in the vertical direction.

Fig. 70 shows a block vector of a block encoded in Intra TMP mode according to one embodiment of the present invention.

Figure 71 illustrates a case where a current block is divided by the GPM mode according to one embodiment of the present invention and the divided area is encoded in the IBC mode.

Figure 72 shows how a current block is encoded in IBC-CIIP mode according to one embodiment of the present invention.

FIG. 73 shows an example of a reference region and filter shape used to derive CCCM parameters according to one embodiment of the present invention.

Figure 74 illustrates a method for constructing an integrated list according to one embodiment of the present invention.

The terms used in this specification are selected from the most widely used and general terms possible while considering the functions of the present invention, but they may vary depending on the intentions of engineers in the field, customs, or the emergence of new technologies. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in such cases, their meanings will be described in the description of the relevant invention. Therefore, it should be noted that the terms used in this specification should be interpreted based on the actual meaning of the terms and the overall contents of this specification, not simply the names of the terms.

In this specification, 'A and/or B' may be interpreted to mean 'including at least one of A or B'.

In this specification, some terms may be interpreted as follows. Coding may be interpreted as encoding or decoding, as the case may be. In this specification, a device that encodes a video signal to generate a video signal bitstream is referred to as an encoding device or an encoder, and a device that decodes a video signal bitstream to restore a video signal is referred to as a decoding device or a decoder. In addition, in this specification, a video signal processing device is used as a term that includes both an encoder and a decoder. Information is a term that includes values, parameters, coefficients, elements, etc., and since the meaning may be interpreted differently depending on the case, the present invention is not limited thereto. 'Unit' is used to mean a basic unit of image processing or a specific location of a picture, and refers to an image area that includes at least one of a luminance (luma) component and a chroma component. In addition, 'block' refers to an image area including specific components among luminance components and chrominance components (i.e., Cb and Cr). However, depending on the embodiment, terms such as 'unit', 'block', 'partition', 'signal', and 'area' may be used interchangeably. In addition, in this specification, 'current block' means a block that is currently scheduled to be encoded, and 'reference block' means a block that has already been encoded or decoded and is used as a reference in the current block. In addition, in this specification, terms such as 'luma', 'luminance', and 'Y' may be used interchangeably. In addition, in this specification, terms such as 'chroma', 'chroma', 'color difference', and 'Cb or Cr' may be used interchangeably, and since chrominance components are divided into two, Cb and Cr, each chrominance component may be used separately. In addition, in this specification, a unit may be used as a concept including all of a coding unit, a prediction unit, and a transformation unit. A picture refers to a field or a frame, and the terms may be used interchangeably depending on the embodiment. Specifically, if the captured image is an interlace image, one frame is divided into an odd (or odd, top) field and an even (or even, bottom) field, and each field is configured as one picture unit and can be encoded or decoded. If the captured image is a progressive image, one frame is configured as a picture and can be encoded or decoded. In addition, in this specification, the terms 'error signal', 'residual signal', 'residual signal', 'residual signal', and 'differential signal' may be used interchangeably. In addition, in this specification, the terms 'intra prediction mode', 'intra prediction directional mode', 'intra-screen prediction mode', and 'intra-screen prediction directional mode' may be used interchangeably. In addition, in this specification, the terms 'motion', 'movement', and the like may be used interchangeably. In addition, in this specification, 'left', 'upper left', 'upper right', 'right', 'lower right', 'lower', and 'lower left' can be used interchangeably with 'left', 'upper left', 'top', 'upper right', 'right', 'lower right', 'bottom', and 'lower left'. In addition, element and member can be used interchangeably with each other. POC (Picture Order Count) indicates temporal position information of a picture (or frame), can be the playback order displayed on the screen, and can have a unique POC for each picture.

FIG. 1 is a schematic block diagram of a video signal encoding device (100) according to one embodiment of the present invention. Referring to FIG. 1, the encoding device (100) of the present invention includes a transformation unit (110), a quantization unit (115), an inverse quantization unit (120), an inverse transformation unit (125), a filtering unit (130), a prediction unit (150), and an entropy coding unit (160).

The transform unit (110) obtains a transform coefficient value by transforming the residual signal, which is the difference between the input video signal and the prediction signal generated by the prediction unit (150). For example, a discrete cosine transform (DCT), a discrete sine transform (DST), or a wavelet transform may be used. The discrete cosine transform and the discrete sine transform divide the input picture signal into blocks and perform the transform. The coding efficiency may vary depending on the distribution and characteristics of the values in the transform domain during the transform. The transform kernel used for the transform for the residual block may be a transform kernel having the separable characteristics of vertical transform and horizontal transform. In this case, the transform for the residual block may be performed by separating it into vertical transform and horizontal transform. For example, the encoder may perform vertical transform by applying the transform kernel in the vertical direction of the residual block. Additionally, the encoder can perform horizontal transform by applying a transform kernel in the horizontal direction of the residual block. In the present disclosure, the transform kernel may be used as a term referring to a set of parameters used for transforming the residual signal, such as a transform matrix, a transform array, a transform function, and a transform. For example, the transform kernel may be any one of a plurality of available kernels. Additionally, transform kernels based on different transform types may be used for each of the vertical transform and the horizontal transform.

The transformation coefficients are distributed with higher coefficients toward the upper left of the block, and closer to '0' toward the lower right of the block. As the current block size increases, there is a possibility that there will be many '0' coefficients in the lower right area. In order to reduce the transformation complexity of large blocks, only the arbitrary upper left area can be left, and the remaining areas can be reset to '0'.

In addition, error signals may exist only in some regions in a coding block. In this case, the conversion process may be performed only on some arbitrary regions. For example, in a block of size 2Nx2N, an error signal may exist only in the first 2NxN block, and the conversion process may be performed only on the first 2NxN block, but the conversion process may not be performed on the second 2NxN block and may not be encoded or decoded. Here, N may be any positive integer.

The encoder may perform an additional transform before the transform coefficients are quantized. The transform method described above may be referred to as a primary transform, and the additional transform may be referred to as a secondary transform. The secondary transform may be optional for each residual block. According to one embodiment, the encoder may perform the secondary transform for a region where it is difficult to concentrate energy in a low-frequency region with only the primary transform, thereby improving coding efficiency. For example, the secondary transform may be additionally performed for a block where residual values appear largely in a direction other than the horizontal or vertical direction of the residual block. Unlike the primary transform, the secondary transform may not be performed separately into a vertical transform and a horizontal transform. This secondary transform may be referred to as a low-frequency non-separable transform (LFNST).

The quantization unit (115) quantizes the transform coefficient values output from the transform unit (110).

In order to increase coding efficiency, instead of coding the picture signal as it is, a method is used in which a picture is predicted using an already coded area through a prediction unit (150), and a residual value between the original picture and the predicted picture is added to the predicted picture to obtain a restored picture. In order to prevent mismatches from occurring in the encoder and the decoder, when the encoder performs prediction, information that is also available in the decoder must be used. To this end, the encoder performs a process of restoring the encoded current block again. The inverse quantization unit (120) inverse quantizes the transform coefficient value, and the inverse transform unit (125) restores the residual value using the inverse quantized transform coefficient value. Meanwhile, the filtering unit (130) performs a filtering operation to improve the quality of the restored picture and enhance the coding efficiency. For example, a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter may be included. The filtered picture is stored in the Decoded Picture Buffer (DPB, 156) for output or use as a reference picture.

A deblocking filter is a filter for removing distortion within a block generated at the boundary between blocks in a restored picture. The encoder can determine whether to apply a deblocking filter to a boundary based on the distribution of pixels included in several columns or rows based on an arbitrary boundary (edge) within a block. When a deblocking filter is applied to a block, the encoder can apply a long filter, a strong filter, or a weak filter depending on the strength of the deblocking filtering. In addition, horizontal filtering and vertical filtering can be processed in parallel. Sample adaptive offset (SAO) can be used to correct the offset from the original image on a pixel basis for a residual block to which a deblocking filter is applied. In order to correct the offset for a specific picture, the encoder can divide the pixels included in the image into a certain number of areas, determine the area to perform offset correction, and use a method (Band Offset) to apply the offset to the area. Alternatively, the encoder can use a method (Edge Offset) that applies an offset by considering edge information of each pixel. Adaptive Loop Filter (ALF) is a method that divides pixels included in an image into a predetermined group, determines one filter to be applied to the group, and performs filtering differently for each group. Information related to whether to apply ALF can be signaled in units of coding units, and the shape and filter coefficients of the ALF filter to be applied can vary depending on each block. In addition, an ALF filter of the same shape (fixed shape) can be applied regardless of the characteristics of the target block to which it is applied.

The prediction unit (150) includes an intra prediction unit (152) and an inter prediction unit (154). The intra prediction unit (152) performs intra prediction within a current picture, and the inter prediction unit (154) performs inter prediction to predict the current picture using a reference picture stored in a decoded picture buffer (156). The intra prediction unit (152) performs intra prediction from reconstructed areas within the current picture and transfers intra encoding information to the entropy coding unit (160). The intra encoding information may include at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, an MPM index, and information about a reference sample. The inter prediction unit (154) may be configured to include a motion estimation unit (154a) and a motion compensation unit (154b). The motion estimation unit (154a) refers to a specific area of the restored reference picture to find the most similar part to the current area and obtains a motion vector value, which is the distance between the areas. The motion information (reference direction indication information (L0 prediction, L1 prediction, bidirectional prediction), reference picture index, motion vector information, etc.) for the reference area obtained by the motion estimation unit (154a) is transferred to the entropy coding unit (160) so that it can be included in the bitstream. Using the motion information transferred from the motion estimation unit (154a), the motion compensation unit (154b) performs inter motion compensation to generate a prediction block for the current block. The inter prediction unit (154) transfers inter encoding information including motion information for the reference area to the entropy coding unit (160).

According to an additional embodiment, the prediction unit (150) may include an intra block copy (IBC) prediction unit (not shown). The IBC prediction unit performs IBC prediction from reconstructed samples in the current picture and transfers IBC encoding information to the entropy coding unit (160). The IBC prediction unit obtains a block vector value indicating a reference region used for prediction of the current region by referring to a specific region in the current picture. The IBC prediction unit may perform IBC prediction using the obtained block vector value. The IBC prediction unit transfers the IBC encoding information to the entropy coding unit (160). The IBC encoding information may include at least one of size information of the reference region, block vector information (index information for block vector prediction of the current block in the motion candidate list, block vector differential information).

When the picture prediction as above is performed, the transformation unit (110) transforms the residual value between the original picture and the predicted picture to obtain a transformation coefficient value. At this time, the transformation can be performed in units of specific blocks within the picture, and the size of the specific block can be varied within a preset range. The quantization unit (115) quantizes the transformation coefficient value generated by the transformation unit (110) and transfers the quantized transformation coefficient to the entropy coding unit (160).

The above quantized transform coefficients in the form of a two-dimensional array can be rearranged into a one-dimensional array for entropy coding. The method of scanning the quantized transform coefficients can be determined by which scan method is used depending on the size of the transform block and the prediction mode within the screen. As an example of implementation, diagonal, vertical, and horizontal scans can be applied. Such scan information can be signaled on a block-by-block basis and can be derived according to a previously determined rule.

The entropy coding unit (160) entropy-codes information representing quantized transform coefficients, intra-coding information, and inter-coding information to generate a video signal bitstream. The entropy coding unit (160) may use a variable length coding (VLC) method and an arithmetic coding method. The variable length coding (VLC) method converts input symbols into continuous codewords, and the length of the codewords may be variable. For example, frequently occurring symbols are expressed as short codewords, and infrequently occurring symbols are expressed as long codewords. A context-based adaptive variable length coding (CAVLC) method may be used as a variable length coding method. Arithmetic coding converts continuous data symbols into a single prime number by using the probability distribution of each data symbol, and arithmetic coding can obtain the optimal prime number bits required to express each symbol. Context-based Adaptive Binary Arithmetic Code (CABAC) can be used as an arithmetic coding.

CABAC is a method of binary arithmetic coding using several context models generated based on the probability obtained through experiments. The context model can also be called a context model. First, if the symbol is not in binary form, the encoder binarizes each symbol using exp-Golomb, etc. The binarized 0 or 1 can be described as a bin. The CABAC initialization process is divided into context initialization and arithmetic coding initialization. Context initialization is a process of initializing the occurrence probability of each symbol, and is determined according to the type of symbol, quantization parameter (QP), and slice type (I, P, B). The context model with this initialization information can use the probability-based value obtained through experiments. The context model provides the occurrence probability of the Least Probable Symbol (LPS) or the Most Probable Symbol (MPS) for the symbol to be currently coded, and information (valMPS) on which bin value corresponds to the MPS among 0 and 1. One of several context models is selected through the context index (ctxIdx), and the context index can be derived from information about the current block to be encoded or information about the surrounding blocks. Initialization for binary arithmetic coding is performed based on the probability model selected from the context model. Binary arithmetic coding is encoded through a process in which the probability interval corresponding to the bin to be processed is divided into probability intervals through the occurrence probability of 0 and 1, and then the probability interval becomes the entire probability interval for the bin to be processed next. The location information within the probability interval in which the last bin is processed is output. However, since the probability interval cannot be divided infinitely, when it is reduced to a certain size, a renormalization process is performed to expand the probability interval and the corresponding location information is output. In addition, after each bin is processed, a probability update process can be performed in which the probability for the next bin to be processed is newly set through the information of the processed bin.

The above generated bitstream is encapsulated into NAL (Network Abstraction Layer) units as basic units. The NAL unit is divided into a VCL (Video Coding Layer) NAL unit containing video data and a non-VCL NAL unit containing parameter information for decoding the video data, and there are various types of VCL or non-VCL NAL units. The NAL unit consists of NAL header information and RBSP (Raw Byte Sequence Payload) data, and the NAL header information includes summary information about the RBSP. The RBSP of the VCL NAL unit includes an integer number of encoded coding tree units. In order to decode the bitstream in a video decoder, the bitstream must first be divided into NAL unit units, and then each divided NAL unit must be decoded. Meanwhile, information required for decoding a video signal bitstream may be transmitted as included in a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), an Adaptation Parameter Set (APS), etc. An RBSP of a VCL NAL unit may include an integer number of coding tree units. A VPS is a parameter set configured with a common syntax by extracting duplicate parameters from an SPS parameter set signaled for each layer in a bitstream supporting image quality, resolution, and frame rate scalability or a bitstream supporting multi-view. SPS is a parameter set including at least one of a Profile containing information on allowable coding tools (or algorithms) and image formats, a Level containing information on the processing capability of a decoder such as processable image resolution and frame rate and allowable memory size, a Tier containing information on the maximum processable bit rate, and information on image resolution, bit depth, and whether or not to enable a function. PPS is a parameter set including at least one of image resolution, tile division information, whether or not to enable weight prediction, quantization parameters, and filtering-related information. APS is a parameter set including one of ALF filter coefficient information, LMCS-related parameters, and quantization scale parameters depending on the APS type. APS is divided into prefix APS, which is signaled before the VCL NAL unit, and suffix APS, which is signaled after the VCL NAL unit. In the case of ALF APS, it is efficient to apply the ALF filter coefficients derived from the previous picture to the next picture, so it can be signaled as suffix APS.

Meanwhile, the block diagram of FIG. 1 illustrates an encoding device (100) according to one embodiment of the present invention, and the blocks shown separately illustrate elements of the encoding device (100) by logically distinguishing them. Accordingly, the elements of the encoding device (100) described above may be mounted as one chip or as multiple chips depending on the design of the device. According to one embodiment, the operation of each element of the encoding device (100) described above may be performed by a processor (not shown).

FIG. 2 is a schematic block diagram of a video signal decoding device (200) according to one embodiment of the present invention. Referring to FIG. 2, the decoding device (200) of the present invention includes an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (225), a filtering unit (230), and a prediction unit (250).

The entropy decoding unit (210) entropy decodes the video signal bitstream to extract transform coefficient information, intra-coding information, inter-coding information, etc. for each region. For example, the entropy decoding unit (210) can obtain a binarization code for transform coefficient information of a specific region from the video signal bitstream. In addition, the entropy decoding unit (210) inversely binarizes the binarization code to obtain quantized transform coefficients. The inverse quantization unit (220) inversely quantizes the quantized transform coefficients, and the inverse transform unit (225) restores the residual value using the inverse quantized transform coefficients. The video signal processing device (200) restores the original pixel value by adding the residual value obtained by the inverse transform unit (225) with the prediction value obtained by the prediction unit (250).

Meanwhile, the filtering unit (230) performs filtering on the picture to improve the image quality. This may include a deblocking filter for reducing block distortion and/or an adaptive loop filter for removing distortion of the entire picture. The filtered picture is output or stored in the decoded picture buffer (DPB, 256) to be used as a reference picture for the next picture.

The prediction unit (250) includes an intra prediction unit (252) and an inter prediction unit (254). The prediction unit (250) generates a prediction picture by utilizing the encoding type decoded through the entropy decoding unit (210) described above, the transform coefficients for each region, intra/inter encoding information, etc. In order to restore the current block on which decoding is performed, the decoded region of the current picture or other pictures including the current block may be used. A picture (or tile/slice) that uses only the current picture for restoration, that is, performs intra prediction or intra BC prediction, is called an intra picture or I picture (or tile/slice), and a picture (or tile/slice) that can perform all of intra prediction, inter prediction, and intra BC prediction is called an inter picture (or tile/slice). A picture (or tile/slice) that uses at most one motion vector and reference picture index to predict sample values of each block among inter-pictures (or tiles/slices) is called a predictive picture or P-picture (or tile/slice), and a picture (or tile/slice) that uses at most two motion vectors and reference picture indices is called a bi-predictive picture or B-picture (or tile/slice). In other words, a P-picture (or tile/slice) uses at most one motion information set to predict each block, and a B-picture (or tile/slice) uses at most two motion information sets to predict each block. Here, a motion information set includes one or more motion vectors and one reference picture index.

The intra prediction unit (252) generates a prediction block using intra encoding information and reconstructed samples in the current picture. As described above, the intra encoding information may include at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, and an MPM index. The intra prediction unit (252) predicts sample values of the current block using reconstructed samples located on the left and/or above the current block as reference samples. In the present disclosure, the reconstructed samples, the reference samples, and the samples of the current block may represent pixels. In addition, the sample values may represent pixel values.

According to one embodiment, the reference samples may be samples included in a neighboring block of the current block. For example, the reference samples may be samples adjacent to a left boundary of the current block and/or samples adjacent to an upper boundary. In addition, the reference samples may be samples located on a line within a preset distance from a left boundary of the current block and/or samples located on a line within a preset distance from an upper boundary of the current block among samples of neighboring blocks of the current block. In this case, the neighboring blocks of the current block may include at least one of a left (L) block, an upper (A) block, a lower left (BL) block, an above right (AR) block, or an above left (AL) block adjacent to the current block.

The inter prediction unit (254) generates a prediction block using the reference picture and inter encoding information stored in the decoded picture buffer (256). The inter encoding information may include a set of motion information (reference picture index, motion vector information, etc.) of the current block for the reference block. Inter prediction may include L0 prediction, L1 prediction, and bi-prediction. L0 prediction is prediction using one reference picture included in the L0 picture list, and L1 prediction means prediction using one reference picture included in the L1 picture list. For this, one set of motion information (e.g., motion vector and reference picture index) may be required. In the bi-prediction method, up to two reference areas can be used, and these two reference areas may exist in the same reference picture or may exist in different pictures, respectively. That is, in the bi-prediction method, up to two sets of motion information (e.g., motion vectors and reference picture indices) can be used, and the two motion vectors may correspond to the same reference picture index or may correspond to different reference picture indices. At this time, the reference pictures are pictures that are located temporally before or after the current picture and may be completed pictures that have already been restored. According to one embodiment, the two reference areas used in the bi-prediction method may be areas selected from each of the L0 picture list and the L1 picture list. In addition, a prediction method that uses only reference pictures having a POC smaller than that of the current picture or uses only reference pictures having a POC larger than that of the current picture based on the POC (picture order count) indicating the display order of the current picture may be called uni-directional prediction. Also, a prediction method that uses both a reference picture with a POC smaller than that of the current picture and a reference picture with a POC larger than that of the current picture based on the picture order count (POC) that indicates the display order of the current picture can be called bi-directional prediction. A prediction method that uses only one reference picture in uni-directional prediction can be called uni-prediction, and a prediction method that uses two reference pictures in uni-directional prediction can be called bi-prediction or pair prediction.

The inter prediction unit (254) can obtain a reference block of the current block using a motion vector and a reference picture index. The reference block exists in a reference picture corresponding to the reference picture index. In addition, a sample value of a block specified by a motion vector or an interpolated value thereof can be used as a predictor of the current block. For motion prediction with pixel accuracy in sub-pel units, for example, an 8-tap interpolation filter can be used for a luminance signal and a 4-tap interpolation filter can be used for a chrominance signal. However, the interpolation filter for motion prediction in sub-pel units is not limited thereto. In this way, the inter prediction unit (254) performs motion compensation to predict a texture of a current unit from a previously restored picture. At this time, the inter prediction unit can use a motion information set.

According to an additional embodiment, the prediction unit (250) may include an IBC prediction unit (not shown). The IBC prediction unit may reconstruct the current region by referring to a specific region including reconstructed samples in the current picture. The IBC prediction unit may perform IBC prediction using IBC encoding information obtained from the entropy decoding unit (210). The IBC encoding information may include block vector information.

A restored video picture is generated by adding the prediction value output from the intra prediction unit (252) or inter prediction unit (254) and the residual value output from the inverse transformation unit (225). That is, the video signal decoding device (200) restores the current block using the prediction block generated from the prediction unit (250) and the residual obtained from the inverse transformation unit (225).

Meanwhile, the block diagram of FIG. 2 illustrates a decoding device (200) according to one embodiment of the present invention, and the blocks shown separately illustrate elements of the decoding device (200) by logically distinguishing them. Accordingly, the elements of the decoding device (200) described above may be mounted as one chip or as multiple chips depending on the design of the device. According to one embodiment, the operation of each element of the decoding device (200) described above may be performed by a processor (not shown).

Meanwhile, the technology proposed in this specification is applicable to both the method and device of the encoder and the decoder, and the parts described as signaling and parsing may be described for the convenience of explanation. In general, signaling can be described as encoding each syntax from the perspective of the encoder, and parsing can be described as interpreting each syntax from the perspective of the decoder. That is, each syntax can be included in the bitstream from the encoder and signaled, and the decoder can parse the syntax and use it in the restoration process. At this time, the sequence of bits for each syntax listed according to the prescribed hierarchical configuration can be called a bitstream.

A picture can be encoded by being divided into sub-pictures, slices, tiles, etc. A sub-picture can include one or more slices or tiles. When a picture is encoded by being divided into multiple slices or tiles, all slices or tiles in the picture must be decoded before it can be displayed on the screen. On the other hand, when a picture is encoded into multiple sub-pictures, only any sub-picture can be decoded and displayed on the screen. A slice can include multiple tiles or sub-pictures. Or, a tile can include multiple sub-pictures or slices. Since sub-pictures, slices, and tiles can be encoded or decoded independently of each other, it is effective for parallel processing and processing speed improvement. However, since encoded information of adjacent other sub-pictures, other slices, and other tiles cannot be used, there is a disadvantage in that the bit amount increases. A sub-picture, slice, or tile can be encoded by being divided into multiple coding tree units (CTUs).

FIG. 3 illustrates an embodiment in which a Coding Tree Unit (CTU) in a picture is divided into Coding Units (CUs). In the process of coding a video signal, a picture may be divided into a sequence of Coding Tree Units (CTUs). A Coding Tree Unit may be composed of a luminance (luma) Coding Tree Block (CTB), two chroma (chroma) Coding Tree Blocks, and their encoded syntax information. One Coding Tree Unit may be composed of one Coding Unit, or one Coding Tree Unit may be split into multiple Coding Units. One Coding Unit may be composed of a luminance Coding Block (CB), two chroma Coding Blocks, and their encoded syntax information. One Coding Block may be split into multiple sub-Coding Blocks. One Coding Unit may be composed of one Transform Unit (TU), or one Coding Unit may be split into multiple Transform Units. A transform unit may be composed of a luminance transform block (Transform Block, TB), two chrominance transform blocks, and its encoded syntax information. A coding tree unit may be divided into multiple coding units. A coding tree unit may not be divided and may be a leaf node. In this case, the coding tree unit itself may be a coding unit.

A coding unit refers to a basic unit for processing a picture in the processing of a video signal described above, that is, a process such as intra/inter prediction, transformation, quantization, and/or entropy coding. The size and shape of a coding unit within a picture may not be constant. A coding unit may have a square or rectangular shape. A rectangular coding unit (or rectangular block) includes a vertical coding unit (or vertical block) and a horizontal coding unit (or horizontal block). In this specification, a vertical block is a block whose height is greater than its width, and a horizontal block is a block whose width is greater than its height. In addition, a non-square block in this specification may refer to a rectangular block, but the present invention is not limited thereto.

Referring to FIG. 3, the coding tree unit is first divided into a Quad Tree (QT) structure. That is, in the Quad Tree structure, one node having a size of 2NX2N can be divided into four nodes having a size of NXN. In this specification, the Quad Tree may also be referred to as a quaternary tree. The Quad Tree division can be performed recursively, and not all nodes need to be divided to the same depth.

Meanwhile, the leaf node of the aforementioned quad tree can be further split into a Multi-Type Tree (MTT) structure. According to an embodiment of the present invention, in the Multi-Type Tree structure, one node can be split into a binary or ternary tree structure of horizontal or vertical splitting. That is, the Multi-Type Tree structure has four split structures: vertical binary splitting, horizontal binary splitting, vertical ternary splitting, and horizontal ternary splitting. According to an embodiment of the present invention, in each of the above tree structures, the width and height of the node can both have a power of 2 value. For example, in the Binary Tree (BT) structure, a node of 2NX2N size can be split into two NX2N nodes by vertical binary splitting, and can be split into two 2NXN nodes by horizontal binary splitting. Also, in the Ternary Tree (TT) structure, a node of size 2NX2N can be split into nodes of size (N/2)X2N, NX2N, and (N/2)X2N by vertical ternary splitting, and into nodes of size 2NX(N/2), 2NXN, and 2NX(N/2) by horizontal ternary splitting. This multi-type tree splitting can be performed recursively.

A leaf node of a multi-type tree can be a coding unit. If the coding unit is not larger than the maximum transform length, the coding unit can be used as a unit of prediction and/or transformation without further splitting. In one embodiment, if the width or height of the current coding unit is larger than the maximum transform length, the current coding unit can be split into a plurality of transform units without explicit signaling regarding the splitting. Meanwhile, in the quad tree and multi-type tree described above, at least one of the following parameters can be defined in advance or transmitted via an RBSP of a higher level set, such as a PPS, an SPS, a VPS, etc. 1) CTU Size: The size of the root node of the quad tree, 2) Minimum QT Size (MinQtSize): The minimum allowed size of a QT leaf node, 3) Maximum BT Size (MaxBtSize): The maximum allowed size of a BT root node, 4) Maximum TT Size (MaxTtSize): The maximum allowed size of a TT root node, 5) Maximum MTT Depth (MaxMttDepth): The maximum allowed depth of an MTT partition from a leaf node of a QT, 6) Minimum BT Size (MinBtSize): The minimum allowed size of a BT leaf node, 7) Minimum TT Size (MinTtSize): The minimum allowed size of a TT leaf node.

FIG. 4 illustrates an embodiment of a method for signaling splitting of a quad tree and a multi-type tree. Pre-configured flags may be used to signal splitting of the quad tree and the multi-type tree described above. Referring to FIG. 4, at least one of a flag 'split_cu_flag' indicating whether a node is split, a flag 'split_qt_flag' indicating whether a quad tree node is split, a flag 'mtt_split_cu_vertical_flag' indicating a splitting direction of a multi-type tree node, or a flag 'mtt_split_cu_binary_flag' indicating a splitting shape of a multi-type tree node may be used.

According to an embodiment of the present invention, a flag 'split_cu_flag' indicating whether a current node is split may be signaled first. If the value of 'split_cu_flag' is 0, it indicates that the current node is not split, and the current node becomes a coding unit. If the current node is a coding tree unit, the coding tree unit includes one coding unit that is not split. If the current node is a quad tree node 'QT node', the current node is a leaf node 'QT leaf node' of the quad tree and becomes a coding unit. If the current node is a multi-type tree node 'MTT node', the current node is a leaf node 'MTT leaf node' of the multi-type tree and becomes a coding unit.

When the value of 'split_cu_flag' is 1, the current node can be split into nodes of a quad tree or a multi-type tree depending on the value of 'split_qt_flag'. The coding tree unit is a root node of the quad tree and can be first split into a quad tree structure. In the quad tree structure, 'split_qt_flag' is signaled for each node 'QT node'. When the value of 'split_qt_flag' is 1, the node is split into four square nodes, and when the value of 'split_qt_flag' is 0, the node becomes a leaf node 'QT leaf node' of the quad tree and the node is split into multi-type nodes. According to an embodiment of the present invention, the quad tree splitting may be limited depending on the type of the current node. A quad-tree split may be allowed if the current node is a coding tree unit (a root node of a quad-tree) or a quad-tree node, and a quad-tree split may not be allowed if the current node is a multi-type tree node. Each quad-tree leaf node 'QT leaf node' may be further split into a multi-type tree structure. As described above, if 'split_qt_flag' is 0, the current node may be split into multi-type nodes. To indicate the splitting direction and the splitting shape, 'mtt_split_cu_vertical_flag' and 'mtt_split_cu_binary_flag' may be signaled. If the value of 'mtt_split_cu_vertical_flag' is 1, a vertical split of the node 'MTT node' is indicated, and if the value of 'mtt_split_cu_vertical_flag' is 0, a horizontal split of the node 'MTT node' is indicated. Also, if the value of 'mtt_split_cu_binary_flag' is 1, the node 'MTT node' is split into two rectangular nodes, and if the value of 'mtt_split_cu_binary_flag' is 0, the node 'MTT node' is split into three rectangular nodes.

The tree partition structure can be partitioned into luminance blocks and chrominance blocks in the same form. That is, the chrominance block can be partitioned into chrominance blocks by referring to the partition form of the luminance block. If the current chrominance block is smaller than an arbitrary set size, the chrominance block may not be partitioned even if the luminance block is partitioned.

The tree partition structure may have different forms for luminance blocks and chrominance blocks. At this time, partition information for the luminance block and partition information for the chrominance block may be signaled separately. In addition, not only the partition information but also the encoding information for the luminance block and the chrominance block may be different. As an example of implementation, at least one or more intra encoding modes for the luminance block and the chrominance block, encoding information for the motion information, etc. may be different.

The node to be divided into the smallest unit can be processed as one coding block. When the current block is a coding block, the coding block can be divided into several sub-blocks (sub-coding blocks), and the prediction information of each sub-block can be the same or different. For example, when the coding unit is an intra mode, the intra prediction modes of each sub-block can be the same or different. In addition, when the coding unit is an inter mode, the motion information of each sub-block can be the same or different. In addition, each sub-block can be encoded or decoded independently of each other. Each sub-block can be distinguished through a sub-block index (sbIdx). In addition, when the coding unit is divided into sub-blocks, it can be divided horizontally or vertically or diagonally. In the intra mode, the mode that divides the current coding unit into two or four sub-blocks horizontally or vertically is called ISP (Intra Sub Partitions). In the inter mode, the mode that divides the current coding block diagonally is called GPM (Geometric partitioning mode). In GPM mode, the position and direction of the diagonal line are derived using a predefined angle table, and the index information of the angle table is signaled.

Motion information may include one or more of reference direction indication information, reference picture information, motion vector, motion resolution, affine model, CPMV (control point motion vector), block vector, block vector resolution, MHP information, LIC information, filtering information, BCW information, and RRIBC information.

The reference direction indication information consists of L0 prediction, L1 prediction, L0 and L1 prediction, and L0 prediction and L1 prediction are unidirectional prediction and unidirectional prediction, and L0 and L1 prediction are bidirectional prediction. And L0 and L1 prediction can be unidirectional prediction or bidirectional prediction. Here, L0 prediction is predicted using reference pictures in the L0 reference picture list, and L1 prediction can be predicted using reference pictures in the L1 reference picture list. In the L0 reference picture list, a reference picture having a POC smaller than the POC of the current picture can be added to the reference picture list based on the POC of the current picture. In addition, the L0 reference picture list can be composed in order from a reference picture having a POC closer than the POC of the current picture to a reference picture having a POC farther from the POC of the current picture. In the L1 reference picture list, a reference picture having a POC larger than the POC of the current picture can be added to the reference picture list based on the POC of the current picture. In addition, the L1 reference picture list can be organized in the order of reference pictures whose POC is closer to the current picture's POC and reference pictures whose POC is farther away. The L0 and L1 reference picture lists can be different for each slice, subpicture, and picture. In addition, the L0 reference picture list can include reference pictures of the L1 reference picture list. And the L1 reference picture list can include reference pictures of the L0 reference picture list.

Reference picture information may vary from block to block, and may be index information indicating which reference picture in the L0 reference picture list and/or the L1 reference picture list is used to predict the current block. The reference picture information may include at least one of L0 reference picture information and L1 reference picture information.

A motion vector is information that indicates the block in a reference picture that best matches the current block. It is a value that represents the distance to the reference block in horizontal and vertical coordinates based on the upper left position of the current block in the picture.

Motion resolution represents the resolution of the motion vector, and motion resolution can be expressed in units of 4 pixels, 1 pixel (integer pixel), 1/2 pixel, 1/4 pixel, 1/8 pixel, and 1/16 pixel.

A block vector is information that indicates the block that best matches the current block in the area that has already been restored within the current picture. It is a value that represents the distance to the reference block in horizontal and vertical coordinates based on the upper left position of the current block within the picture.

Block vector resolution can be expressed in units of 4 pixels, 1 pixel (integer pixel), 1/2 pixel, 1/4 pixel, 1/8 pixel, and 1/16 pixel.

MHP information may include whether additional motion information is applied and may include additional motion information.

LIC information may include whether LIC is applied to the current block.

Filtering information may include filtering type and coefficient information applied according to the motion resolution of the current block.

CW information may include whether BCW is applied to the current block.

RRIBC information may include information about whether RRIBC is applied and the RRIBC type if the current block is encoded in IBC mode.

Picture prediction (motion compensation) for coding is performed on coding units that cannot be divided any further (i.e., leaf nodes of coding tree units). The basic unit that performs this prediction is referred to as a prediction unit or prediction block hereinafter.

Hereinafter, the term unit used in this specification may be used as a term to replace the prediction unit, which is a basic unit for performing prediction. However, the present invention is not limited thereto, and may be understood as a concept that includes the coding unit in a broader sense.

Figures 5 and 6 illustrate an intra prediction method according to an embodiment of the present invention in more detail. As described above, the intra prediction unit predicts sample values of the current block using reconstructed samples located on the left and/or above the current block as reference samples.

First, FIG. 5 illustrates an embodiment of reference samples used for prediction of a current block in an intra prediction mode. According to an embodiment, the reference samples may be samples adjacent to a left boundary of the current block and/or samples adjacent to an upper boundary. As illustrated in FIG. 5, when the size of the current block is WXH and samples of a single reference line adjacent to the current block are used for intra prediction, reference samples may be set using at most 2W+2H+1 surrounding samples located on the left and/or above the current block.

Meanwhile, pixels of multiple reference lines may be used for intra prediction of the current block. The multiple reference lines may be composed of n lines located within a preset range from the current block. According to one embodiment, when pixels of multiple reference lines are used for intra prediction, separate index information indicating lines to be set as reference pixels may be signaled, and this may be named a reference line index.

In addition, if at least some of the samples to be used as reference samples have not yet been restored, the intra prediction unit may perform a reference sample padding process to obtain a reference sample. In addition, the intra prediction unit may perform a reference sample filtering process to reduce an error of intra prediction. That is, filtering may be performed on surrounding samples and/or reference samples obtained by the reference sample padding process to obtain filtered reference samples. The intra prediction unit predicts samples of the current block using the reference samples obtained in this manner. The intra prediction unit predicts samples of the current block using unfiltered reference samples or filtered reference samples. In the present disclosure, the surrounding samples may include samples on at least one reference line. For example, the surrounding samples may include adjacent samples on a line adjacent to a boundary of the current block.

Next, FIG. 6 illustrates an embodiment of prediction modes used for intra prediction. For intra prediction, intra prediction mode information indicating an intra prediction direction may be signaled. The intra prediction mode information indicates one of a plurality of intra prediction modes constituting an intra prediction mode set. If the current block is an intra prediction block, the decoder receives intra prediction mode information of the current block from the bitstream. The intra prediction unit of the decoder performs intra prediction on the current block based on the extracted intra prediction mode information.

According to an embodiment of the present invention, an intra prediction mode set may include all intra prediction modes used for intra prediction (e.g., a total of 67 intra prediction modes). More specifically, the intra prediction mode set may include a planar mode, a DC mode, and a plurality of (e.g., 65) angular modes (i.e., directional modes). Each intra prediction mode may be indicated by a preset index (i.e., an intra prediction mode index). For example, as illustrated in FIG. 6, an intra prediction mode index 0 indicates a planar mode, and an intra prediction mode index 1 indicates a DC mode. In addition, intra prediction mode indexes 2 to 66 may indicate different angular modes, respectively. The angular modes indicate different angles within a preset angular range, respectively. For example, an angular mode may indicate an angle within an angular range from 45 degrees to -135 degrees in a clockwise direction (i.e., a first angular range). The angular mode may be defined based on the 12 o'clock direction. At this time, intra prediction mode index 2 indicates horizontal diagonal (HDIA) mode, intra prediction mode index 18 indicates horizontal (HOR) mode, intra prediction mode index 34 indicates diagonal (DIA) mode, intra prediction mode index 50 indicates vertical (VER) mode, and intra prediction mode index 66 indicates vertical diagonal (VDIA) mode.

Meanwhile, the preset angle range may be set differently depending on the shape of the current block. For example, if the current block is a rectangular block, a wide-angle mode indicating an angle exceeding 45 degrees or less than -135 degrees in a clockwise direction may be additionally used. If the current block is a horizontal block, the angle mode may indicate an angle within an angle range (i.e., a second angle range) between (45+offset1) degrees and (-135+offset1) degrees in a clockwise direction. At this time, angle modes 67 to 76 that are outside the first angle range may be additionally used. In addition, if the current block is a vertical block, the angle mode may indicate an angle within an angle range (i.e., a third angle range) between (45-offset2) degrees and (-135-offset2) degrees in a clockwise direction. At this time, angle modes -10 to -1 that are outside the first angle range may be additionally used. According to an embodiment of the present invention, the values of offset1 and offset2 may be determined differently depending on a ratio between the width and the height of the rectangular block. Also, offset1 and offset2 can be positive.

According to an additional embodiment of the present invention, the plurality of angular modes constituting the intra prediction mode set may include a basic angular mode and an extended angular mode. In this case, the extended angular mode may be determined based on the basic angular mode.

According to one embodiment, the basic angle mode may be a mode corresponding to an angle used in intra prediction of the existing HEVC (High Efficiency Video Coding) standard, and the extended angle mode may be a mode corresponding to an angle newly added in intra prediction of the next-generation video codec standard. More specifically, the basic angle mode may be an angle mode corresponding to any one of the intra prediction modes {2, 4, 6, …, 66}, and the extended angle mode may be an angle mode corresponding to any one of the intra prediction modes {3, 5, 7, …, 65}. That is, the extended angle mode may be an angle mode between the basic angle modes within the first angle range. Therefore, the angle indicated by the extended angle mode may be determined based on the angle indicated by the basic angle mode.

According to another embodiment, the basic angle mode may be a mode corresponding to an angle within a preset first angle range, and the extended angle mode may be a wide-angle mode outside the first angle range. That is, the basic angle mode may be an angle mode corresponding to any one of the intra prediction modes {2, 3, 4, … , 66}, and the extended angle mode may be an angle mode corresponding to any one of the intra prediction modes {-14, -13, -12, … , -1} and {67, 68, … , 80}. The angle indicated by the extended angle mode may be determined as an opposite angle to the angle indicated by the corresponding basic angle mode. Accordingly, the angle indicated by the extended angle mode may be determined based on the angle indicated by the basic angle mode. Meanwhile, the number of extended angle modes is not limited thereto, and additional extended angles may be defined according to the size and/or shape of the current block. Meanwhile, the total number of intra prediction modes included in the intra prediction mode set may vary depending on the configuration of the basic angle mode and the extended angle mode described above.

In the above embodiment, the interval between the extended angular modes can be set based on the interval between the corresponding basic angular modes. For example, the interval between the extended angular modes {3, 5, 7, … , 65} can be determined based on the interval between the corresponding basic angular modes {2, 4, 6, … , 66}. In addition, the interval between the extended angular modes {-14, -13, … , -1} can be determined based on the interval between the corresponding opposite basic angular modes {53, 53, … , 66}, and the interval between the extended angular modes {67, 68, … , 80} can be determined based on the interval between the corresponding opposite basic angular modes {2, 3, 4, … , 15}. The angular interval between the extended angular modes can be set to be equal to the angular interval between the corresponding basic angular modes. In addition, the number of the extended angular modes in the intra prediction mode set can be set to be less than or equal to the number of the basic angular modes.

According to an embodiment of the present invention, the extended angular mode can be signaled based on the base angular mode. For example, the wide-angle mode (i.e., the extended angular mode) can replace at least one angular mode (i.e., the base angular mode) within the first angular range. The replaced base angular mode can be an angular mode corresponding to the opposite side of the wide-angle mode. That is, the replaced base angular mode is an angular mode corresponding to an angle in the opposite direction to the angle indicated by the wide-angle mode or an angle that is different from the angle in the opposite direction by a preset offset index. According to an embodiment of the present invention, the preset offset index is 1. The intra prediction mode index corresponding to the replaced base angular mode can be remapped to the wide-angle mode to signal the corresponding wide-angle mode. For example, the wide-angle modes {-14, -13, ... , -1} can be signaled by the intra prediction mode indices {52, 53, ... , 66}, respectively, and the wide-angle modes {67, 68, ... , 66} can be signaled by the intra prediction mode indices {52, 53, ... , 66}, respectively. , 80} can be signaled by intra prediction mode indices {2, 3, …, 15}, respectively. By having the intra prediction mode index for the basic angular mode signal the extended angular mode in this way, even if the configurations of the angular modes used for intra prediction of each block are different, the same set of intra prediction mode indices can be used to signal the intra prediction mode. Therefore, the signaling overhead due to the change in the intra prediction mode configuration can be minimized.

Meanwhile, whether to use the extended angle mode may be determined based on at least one of the shape and the size of the current block. According to one embodiment, if the size of the current block is larger than a preset size, the extended angle mode may be used for intra prediction of the current block, otherwise, only the basic angle mode may be used for intra prediction of the current block. According to another embodiment, if the current block is a non-square block, the extended angle mode may be used for intra prediction of the current block, and if the current block is a square block, only the basic angle mode may be used for intra prediction of the current block.

The intra prediction unit determines reference samples and/or interpolated reference samples to be used for intra prediction of the current block based on intra prediction mode information of the current block. If the intra prediction mode index indicates a specific angle mode, the reference sample or interpolated reference sample corresponding to the specific angle from the current sample of the current block is used for prediction of the current pixel. Therefore, different sets of reference samples and/or interpolated reference samples can be used for intra prediction depending on the intra prediction mode. After intra prediction of the current block is performed using the reference samples and intra prediction mode information, the decoder restores sample values of the current block by adding a residual signal of the current block obtained from the inverse transform unit to the intra prediction value of the current block.

Motion information used for inter prediction may include reference direction indication information (inter_pred_idc), reference picture indexes (ref_idx_l0, ref_idx_l1), and motion vectors (mvL0, mvL1). Reference picture list utilization information (predFlagL0, predFlagL1) may be set according to the reference direction indication information. As an example, in case of unidirectional prediction using an L0 reference picture, predFlagL0=1, predFlagL1=0 may be set. In case of unidirectional prediction using an L1 reference picture, predFlagL0=0, predFlagL1=1 may be set. In case of bidirectional prediction using both L0 and L1 reference pictures, predFlagL0=1, predFlagL1=1 may be set.

If the current block is a coding unit, the coding unit may be divided into multiple sub-blocks, and the prediction information of each sub-block may be the same or different. For example, if the coding unit is an intra mode, the intra prediction modes of each sub-block may be the same or different. In addition, if the coding unit is an inter mode, the motion information of each sub-block may be the same or different. In addition, each sub-block may be encoded or decoded independently. Each sub-block may be distinguished through a sub-block index (sbIdx).

The motion vector of the current block is likely to be similar to the motion vectors of the surrounding blocks. Therefore, the motion vectors of the surrounding blocks can be used as motion vector predictors (mvp), and the motion vector of the current block can be derived using the motion vectors of the surrounding blocks. In addition, in order to improve the accuracy of the motion vector, the difference in motion vectors (motion vector difference, mvd) between the optimal motion vector of the current block found from the original image by the encoder and the motion predictor can be signaled.

Motion vectors can have various resolutions, and the resolution of motion vectors can vary on a block-by-block basis. Motion vector resolution can be expressed in integer units, half-pixel units, quarter-pixel units, sixteen-pixel units, and integer-of-four pixel units. Since images such as screen contents are in simple graphical forms such as characters, no interpolation filter needs to be applied, and thus integer units and integer-of-four pixel units can be selectively applied on a block-by-block basis. Blocks encoded in an affine mode capable of expressing rotation and scale have significant shape changes, and thus integer units, quarter-pixel units, and sixteen-pixel units can be selectively applied on a block-by-block basis. Information on whether to selectively apply motion vector resolution on a block-by-block basis is signaled by amvr_flag. If applied, which motion vector resolution to apply to the current block is signaled by amvr_precision_idx.

For blocks to which bidirectional prediction is applied, the weights between the two prediction blocks can be applied equally or differently when applying weighted average, and information about the weights is signaled via bcw_idx.

In order to improve the accuracy of the motion prediction value, the Merge or AMVP (advanced motion vector prediction) method can be selectively used on a block-by-block basis. The Merge method is a method that configures the motion information of the current block to be the same as the motion information of the adjacent blocks to the current block, and has the advantage of increasing the encoding efficiency of the motion information by spatially propagating the motion information without change in the motion region having homogeneity. On the other hand, the AMVP method is a method that predicts motion information in the L0 and L1 prediction directions respectively to express accurate motion information and signals the most optimal motion information. After the decoder derives the motion information for the current block through the AMVP or Merge method, it uses the reference block located in the motion information derived from the reference picture as the prediction block for the current block.

A method of deriving motion information in Merge or AMVP may be a method in which a motion candidate list is constructed using motion prediction values derived from neighboring blocks of a current block, and then index information for an optimal motion candidate is signaled. In the case of AMVP, since motion candidate lists are derived for each of L0 and L1, optimal motion candidate indices (mvp_l0_flag, mvp_l1_flag) for each of L0 and L1 are signaled. In the case of Merge, since one motion candidate list is derived, one merge index (merge_idx) is signaled. The motion candidate lists derived from one coding unit may vary, and a motion candidate index or a merge index may be signaled for each motion candidate list. In this case, a mode in which there is no information on a residual block in a block encoded in Merge mode may be called a MergeSkip mode.

Bidirectional motion information for the current block can be derived by combining AMVP and Merge modes. For example, motion information in the L0 direction can be derived using the AMVP method, and motion information in the L1 direction can be derived using the Merge method. Conversely, L0 can be applied to Merge, and L1 can be applied to AMVP. This encoding mode can be called AMVP-merge mode.

The motion candidate and the motion information candidate in this specification may have the same meaning. In addition, the motion candidate list and the motion information candidate list in this specification may have the same meaning.

SMVD (Symmetric MVD) is a method for reducing the bit amount of transmitted motion information by making the MVD (Motion Vector Difference) values in the L0 and L1 directions symmetrical in the case of bi-directional prediction. The MVD information in the L1 direction that is symmetrical to the L0 direction is not transmitted, and the reference picture information in the L0 and L1 directions is also not transmitted and can be derived during the decoding process.

OBMC (Overlapped Block Motion Compensation) is a method that generates prediction blocks for the current block using the motion information of surrounding blocks when the motion information between blocks is different, and then generates the final prediction block for the current block by weighting and averaging the prediction blocks. This has the effect of reducing the blocking phenomenon that occurs at the block boundaries of a motion compensated image.

In general, the motion accuracy of the merge motion candidate is low. In order to improve the accuracy of such a merge motion candidate, the MMVD (Merge mode with MVD) method can be used. The MMVD method is a method of correcting motion information by using one candidate selected from several motion differential value candidates. Information on the correction value of the motion information obtained through the MMVD method (e.g., an index indicating one candidate selected from the motion differential value candidates, etc.) can be included in the bitstream and transmitted to the decoder. Compared to the conventional method of including the motion information difference value in the bitstream, the number of bits can be saved by including the information on the correction value of the motion information in the bitstream.

The TM (Template Matching) method is a method of compiling a template using surrounding pixels of the current block, finding a matching area with the highest similarity to the template, and correcting motion information. TM (Template Matching) is a method of performing motion prediction in a decoder without including motion information in the bitstream in order to reduce the size of the encoded bitstream. At this time, since the decoder does not have an original image, it can roughly derive motion information for the current block using the already restored surrounding blocks.

The DMVR (Decoder-side Motion Vector Refinement) method is a method to correct motion information through the correlation of already restored reference images in order to find slightly more accurate motion information. It is a method to use the best matching point between reference blocks in the reference pictures within an arbitrary set area of two reference pictures as a new bidirectional motion by using the bidirectional motion information of the current block. When this DMVR is performed, the encoder can correct the motion information by performing DMVR on a block basis, and then divide the block into sub-blocks again and perform DMVR on each sub-block basis to correct the motion information of the sub-block again. This can be called MP-DMVR (Multi-pass DMVR).

The LIC (Local Illumination Compensation) method is a method of compensating for luminance changes between blocks. It derives a linear model using neighboring pixels adjacent to the current block, and then compensates for the luminance information of the current block through the linear model.

Existing video encoding methods perform motion compensation that only considers parallel movements in the up, down, left, and right directions, so encoding efficiency is reduced when encoding videos that contain motions such as enlargement, reduction, and rotation that are commonly encountered in reality. In order to express such motions for enlargement, reduction, and rotation, an Affine model-based motion prediction technology using a 4 (rotation) or 6 (enlargement, reduction, rotation) parameter model can be applied.

BDOF (Bi-Directional Optical Flow) is used to compensate for predicted blocks by estimating the amount of pixel change based on optical flow from a reference block of a block composed of bidirectional motion. The motion information derived from BDOF of VVC can be used to compensate for the motion of the current block.

PROF (Prediction refinement with optical flow) is a technology for improving the accuracy of sub-block unit Affine motion prediction to be similar to the accuracy of pixel unit motion prediction. Similar to BDOF, PROF is a technology that calculates a correction value on a pixel-by-pixel basis for pixel values that have been affine motion compensated on a sub-block-by-subblock basis based on optical flow to obtain a final prediction signal.

The CIIP (Combined Inter-/Intra-picture Prediction) method is a method of generating a final prediction block by weighting and averaging prediction blocks generated by the intra-picture prediction method and prediction blocks generated by the inter-picture prediction method when generating a prediction block for the current block.

The IBC (Intra Block Copy) method is a method of finding the most similar part to the current block in the already restored area of the current picture and using the reference block as a prediction block for the current block. At this time, information related to the block vector, which is the distance between the current block and the reference block, can be included in the bitstream. The decoder can parse the information related to the block vector included in the BeastStream to calculate or set the block vector for the current block.

The BCW (Bi-prediction with CU-level Weights) method is a method that performs a weighted average on two motion-compensated prediction blocks by adaptively applying weights on a block-by-block basis, rather than generating a prediction block by averaging two motion-compensated prediction blocks from different reference pictures.

The Intra TMP (Template Matching Prediction) method is a method in which a video signal processing device constructs a reference template using pixel values of neighboring blocks adjacent to the current block, finds the part most similar to the constructed reference template in an already restored area within the current picture, and then uses the reference block (the part found in the already restored area) as a prediction block for the current block.

The MHP (Multi-hypothesis prediction) method is a method of performing weighted prediction using various prediction signals by transmitting additional motion information to unidirectional and bidirectional motion information during inter-screen prediction.

CCLM(Cross-component linear model) is a method of constructing a linear model by utilizing the high correlation between a luminance signal and a chrominance signal at the same location as the luminance signal, and then predicting a chrominance signal through the linear model. A template is constructed using a block adjacent to the current block whose restoration has been completed, and parameters for the linear model are derived through the template. Next, the current luminance block, which has been restored to the size of the chrominance block, is selectively down-sampled according to the image format. Finally, the chrominance block of the current block is predicted using the down-sampled luminance block and the linear model. At this time, a method of using two or more linear models is called MMLM(Multi-model Linear mode).

GLM (Gradient Linear Model) is a method of predicting a color difference signal by constructing a model by additionally reflecting the gradient between the luminance sample corresponding to the color difference sample and the surrounding luminance samples adjacent to that luminance sample in a linear model such as CCLM.

In independent scalar quantization, the reconstructed coefficient t' _k for an input coefficient t _k depends only on the associated quantization index q _k . That is, the quantization index for any reconstructed coefficient has a different value from the quantization indices for other reconstructed coefficients. At this time, t' _k can be a value that includes quantization error in t _k , and can be different or the same depending on the quantization parameter. Here, t' _k can be named a reconstructed transform coefficient or a dequantized transform coefficient, and the quantization index can also be named a quantized transform coefficient.

In Uniform Reconstruction Quantizers (URQ), the reconstructed coefficients have the characteristic of being spaced at equal intervals. The distance between two adjacent reconstructed values can be called the quantization step size. The reconstructed values can include 0, and the entire set of available reconstructed values can be uniquely defined according to the quantization step size. The quantization step size can vary depending on the quantization parameter.

In the existing method, the set of acceptable restored transform coefficients is reduced due to quantization, and the number of elements in this set can be finite. As a result, there is a limit to minimizing the average error between the original image and the restored image. Vector quantization can be used as a method to minimize this average error.

A simple form of vector quantization used in video coding is sign data hiding. This is a method in which the encoder does not encode a sign for a non-zero coefficient, and the decoder determines the sign for the coefficient based on whether the sum of the absolute values of all coefficients is even or odd. To this end, at least one coefficient may be increased or decreased by '1' in the encoder, and at least one coefficient may be selected and adjusted in value such that it is optimal in terms of cost for rate-distortion. As an example, a coefficient having a value close to the boundary of a quantization interval may be selected.

Another vector quantization method is Trellis-Coded Quantization, which is utilized as an optimal path search technique to obtain an optimized quantization value in dependent quantization in video coding. In units of blocks, quantization candidates for all coefficients in a block are arranged in a trellis graph, and an optimal trellis path between the optimized quantization candidates is searched by considering the cost for rate-distortion. Specifically, dependent quantization applied to video coding can be designed so that a set of allowable reconstructed transform coefficients for a transform coefficient depends on the value of the transform coefficient preceding the current transform coefficient in the restoration order. At this time, by selectively using multiple quantizers according to the transform coefficients, there is an effect of minimizing the average error between the original image and the reconstructed image, thereby increasing the coding efficiency.

Among intra prediction encoding techniques, the MIP (Matrix Intra Prediction) method is a matrix-based intra prediction method. Unlike the prediction method that has directionality from pixels of neighboring blocks adjacent to the current block, it is a method of obtaining a prediction signal by using a predefined matrix and offset values for pixels on the left and top of neighboring blocks.

In order to derive the intra prediction mode of the current block, the intra prediction mode for the template derived through the surrounding pixels of the template, which is an arbitrary region adjacent to the current block and restored, can be used to restore the current block. First, the decoder generates a prediction template for the template using the surrounding pixels (reference) adjacent to the template, and can use the intra prediction mode that generates the prediction template most similar to the already restored template to restore the current block. This method can be called TIMD (Template intra mode derivation).

In general, an encoder can determine a prediction mode for generating a prediction block and generate a bitstream including information about the determined prediction mode. A decoder can set an intra prediction mode by parsing the received bitstream. At this time, the bit amount of information about the prediction mode can be about 10% of the total bitstream size. In order to reduce the bit amount of information about the prediction mode, the encoder may not include information about the intra prediction mode in the bitstream. Accordingly, the decoder can derive (determine) an intra prediction mode for restoring the current block by using the characteristics of the surrounding blocks, and can restore the current block by using the derived intra prediction mode. At this time, the decoder can use a method of applying a Sobel filter in the horizontal and vertical directions to each of the surrounding pixels adjacent to the current block to infer directional information, and then mapping the directional information to the intra prediction mode in order to derive the intra prediction mode. The method by which the decoder derives the intra prediction mode using surrounding blocks can be described as Decoder side intra mode derivation (DIMD).

The neighboring blocks can be blocks of spatial position or blocks of temporal position. The neighboring block spatially adjacent to the current block can be at least one of a left (Left, A1) block, a left below (Left Below, A0) block, an above (Above, B1) block, an above right (Above Right, B0) block, or an above left (Above Left, B2) block. The neighboring block temporally adjacent to the current block can be a block including an upper left pixel position of a bottom right (BR) block of the current block in a corresponding picture (Collocated picture). If the neighboring block temporally adjacent to the current block is encoded in intra mode or the neighboring block temporally adjacent to the current block exists in an unusable position, a block including a horizontal and vertical center (Center, Ctr) pixel position of the current block in the picture corresponding to the current picture (Collocated picture) can be used as a temporal neighboring block. The motion candidate information derived from the corresponding picture can be referred to as TMVP (Temporal Motion Vector Predictor). Only one TMVP can be derived from one block, and after dividing one block into several sub-blocks, each TMVP candidate can be derived for each sub-block. The method of deriving TMVP in sub-block units can be referred to as sbTMVP (sub-block Temporal Motion Vector Predictor).

Whether the methods described in this specification are to be applied can be determined based on at least one of information about the slice type (e.g., whether it is an I slice, a P slice, or a B slice), whether it is a tile, whether it is a subpicture, the size of the current block, the depth of the coding unit, whether the current block is a luminance block or a chrominance block, whether it is a reference frame or a non-reference frame, the temporal hierarchy according to the reference order and hierarchy, etc. The information used to determine whether the methods described in this specification are to be applied can be information agreed upon in advance between the decoder and the encoder. In addition, such information can be determined according to the profile and level. Such information can be expressed as a variable value, and the bitstream can include information about the variable value. That is, the decoder can parse the information about the variable value included in the bitstream to determine whether the above-described methods are to be applied. For example, whether the above-described methods are to be applied can be determined based on the horizontal length or the vertical length of the coding unit. The above-described methods can be applied when the horizontal length or the vertical length is 32 or more (e.g., 32, 64, 128, etc.). Also, the above-described methods can be applied when the horizontal length or the vertical length is less than 32 (e.g., 2, 4, 8, 16). Also, the above-described methods can be applied when the horizontal length or the vertical length is 4 or 8.

Referring to Fig. 8, the decoder can derive a prediction block using surrounding samples (blocks, pixels). At this time, the surrounding samples can be surrounding blocks (pixels) of the current block. Specifically, the decoder can determine intra prediction modes and weight information for restoring the current block through a histogram for directional information (angle information) using the surrounding samples as input.

Fig. 9(a) shows a case where all neighboring blocks of the current block are available to derive directional information, Fig. 9(b) shows a case where the upper boundary of the current block is a sub-picture, slice, tile, or CTU boundary, and Fig. 9(c) shows a case where the left boundary of the current block is a sub-picture, slice, tile, or CTU boundary. Meanwhile, if the neighboring blocks and the current block do not belong to the same sub-picture, slice, tile, or CTU, the neighboring blocks may not be used to derive directional information. The gray dots in Fig. 9 show the positions of pixels used to actually derive directional information, and the dotted lines show the sub-picture, slice, tile, and CTU boundaries. In addition, referring to Figs. 9(d) to (f), pixels located at the boundary may be padded by one pixel outside the boundary to derive directional information. This padding may allow for more precise directional information to be derived.

In order to derive direction information for a pixel at a specific location, a 3x3 Sobel filter of Equation 1 can be applied in the horizontal and vertical directions, respectively. A in Equation 1 can mean pixel information (value) of reconstructed surrounding blocks of the current block of 3x3 size. And the direction information (θ) can be determined using Equation 2. In order to reduce the computational complexity for deriving direction information, the decoder can derive the direction information (θ) only by calculating Gy/Gx of Equation 1 without calculating the atan function of Equation 2.

Referring to FIG. 9, directional information can be calculated for every gray dot shown in FIG. 9, and the directional information can be mapped to an angle of an intra prediction mode. The intra prediction mode set can include a planar mode, a DC mode, and a plurality of (e.g., 65) angular modes (i.e., directional modes). The intra prediction modes can be 67 modes, and the directional information (angle, θ) calculated through mathematical expression 2 can be a value in real units. Therefore, a process of mapping the directional information to a specific intra prediction directional mode is required. The directional mode described in this specification can be the same as the angular mode.

Referring to FIG. 10, the intra prediction directional mode can be divided into four sections based on 0 degrees (index 18), 45 degrees (index 34), 90 degrees (index 50), and 135 degrees (index 66) (see FIG. 6). Referring to FIG. 10, the sections for determining the intra prediction directional mode can be divided into four sections from section 0 to section 3. Section 0 can be from -45 degrees to 0 degrees, section 1 can be from 0 degrees to 45 degrees, section 2 can be from 45 degrees to 90 degrees, and section 3 can be from 90 degrees to 135 degrees. In this case, each section can include 16 intra prediction directional modes. The directional mode can be determined as one of the four sections by comparing the signs and magnitudes of Gx and Gy calculated through mathematical expression 1. For example, if Gx and Gy are positive, and the absolute value of Gx is greater than the absolute value of Gy, interval 1 can be selected. The intra prediction directional mode mapped to each interval can be determined through the directional information (θ) calculated from mathematical expression 2. Specifically, the decoder extends the value by multiplying the directional information (θ) by 2^16. Then, the decoder can compare the extended value with the numbers in the predefined table to find the value closest to the extended value, and determine the intra prediction directional mode based on the closest value. At this time, the number of values in the predefined table can be 17. Specifically, the values of the predefined table can be {0, 2048, 4096, 6144, 8192, 12288, 16384, 20480, 24576, 28672, 32768, 36864, 40960, 47104, 53248, 59392, 65536}. At this time, the difference between the predefined table values can be set differently depending on the difference between the angles of the intra prediction directional mode.

Meanwhile, if the atan calculation is not performed to reduce the computational complexity and only Gy/Gx is used to obtain the directional angle, the difference between the predefined table values may not match the distance between the angles of the intra prediction directional mode. atan has a characteristic that the slope gradually decreases as the input value increases. Therefore, the defined table should also be set in numerical value by considering not only the difference between the angles of the intra prediction directional mode but also the nonlinear characteristic of atan. For example, the difference between the defined table values may be set to gradually decrease. Conversely, the difference between the defined table values may be set to gradually increase.

If the width and height of the current block are different, the available intra prediction directional modes may be different. That is, if the width and height of the current block are different, the interval for deriving the intra prediction directional mode may be different. In other words, the interval for deriving the intra prediction directional mode may be changed based on the width and height of the current block (for example, the ratio of the width and height). For example, if the width of the current block is longer than the height, the intra prediction mode may be remapped to 67 to 80, and the intra prediction modes in the opposite direction may be excluded to 2 to 15. For example, if the width of the current block is n (integer) times longer than the height (for example, 2), the intra prediction modes {3, 4, 5, 6, 7, 8} may be reset (mapped) to {67, 68, 69, 70, 71, 72}, respectively. In addition, if the horizontal length of the current block is longer than the vertical length, the intra prediction mode may be reset to a value that adds '65' to the intra prediction mode. On the other hand, if the horizontal length of the current block is shorter than the vertical length, the intra prediction mode may be reset to a value that subtracts '67' from the intra prediction mode.

A histogram may be used to derive an intra prediction directional mode for restoration of the current block. If there are more blocks without directionality than blocks with directionality as a result of obtaining directional information for surrounding blocks, the prediction mode for the block without directionality may have the highest cumulative value in the histogram. However, since the directional mode must be derived for restoration of the current block, the prediction mode for the block without directionality may be excluded even if the cumulative value in the histogram is the highest. That is, a smooth area with no gradient between surrounding pixels or no directionality may not be used to derive an intra prediction directional mode. For example, the prediction mode for the block without directionality may be a planar mode or a DC mode. If the left surrounding block is a planar mode or a DC mode, the left surrounding block may not be used to derive directional information, and directional information may be derived using only the upper surrounding block. In the case where the surrounding blocks of the current block contain both gentle regions and directional regions, the decoder can generate a histogram using the G value calculated as in Equation 3 to emphasize the directionality. In this case, the histogram may be an accumulated value to which the calculated G value is added for each generated intra prediction directional mode, rather than a frequency-based one in which '1' is added for each generated intra prediction directional mode.

The X-axis of Fig. 11 represents an intra prediction directional mode, and the Y-axis represents an accumulated value of G values. The decoder can select an intra prediction directional mode having the largest accumulated value of G values among the intra prediction directional modes. In other words, the decoder can select an intra prediction directional mode for the current block based on the accumulated value. Referring to Fig. 11, modeA having the largest accumulated value and modeB having the second largest accumulated value can be selected as intra prediction directional modes. The decoder can generate a final prediction block by weighting the prediction block generated by modeA, the prediction block generated by modeB, and finally the prediction block generated by the planar mode to generate a prediction block for the current block. At this time, the weight of each prediction block can be determined using the accumulated value of modeA and modeB. For example, the weight for the prediction block generated by the planar mode can be set to 1/3 of the total weight. The weight for the prediction block generated by modeA can be set to a weight corresponding to the value obtained by dividing the modeA accumulated value by the sum of the accumulated values of modeA and modeB. The weight for the prediction block generated by modeB can be determined as a difference value between the modeA weight and 1/3 of the total weight. In order to make the calculation of the weight more accurate, the decoder can expand the range of the weight by multiplying the weight for the prediction block generated by modeA by an arbitrary value. The weight for the prediction block generated by modeB and the weight for the prediction block generated in the planar mode can also be expanded in the same way.

FIG. 12 is a diagram illustrating a method for signaling DIMD mode according to one embodiment of the present invention.

Specifically, FIG. 12 illustrates a signaling method used to store a syntax element for whether the DIMD mode is applied in a bitstream and transmit it to a decoder. Referring to FIG. 12, a syntax element (cu_dimd_flag) for whether the DIMD mode is used to generate a prediction block for a current block can be parsed when the coding mode of the current block is an intra mode and a syntax element (sps_dimd_enabled_flag) for whether the DIMD mode set in SPS is enabled indicates that the DIMD mode is enabled (e.g., when the value of sps_dimd_enabled_flag is 1), the coding mode of the current block is not a SKIP mode, the current block is a luminance block, and the current block is not an inter coding mode. At this time, if cu_dimd_flag is '1', it can indicate that the current block is decrypted in DIMD mode, and if cu_dimd_flag is '0', it can indicate that the current block is not decrypted in DIMD mode. Meanwhile, if cu_dimd_flag is not parsed, the value of cu_dimd_flag can be set to '0'. sps_dimd_enabled_flag can be controlled by syntax elements included in profile, tier and level syntax. For example, sps_dimd_enabled_flag can be controlled by gci_no_dimd_constraint_flag, which is a syntax element included in general_conatraints_info() syntax. The following behavior can be defined. If the value of gci_no_dimd_constraint_flag is 1, the value of sps_dimd_enabled_flag for all pictures in OlsInScope can be 0. If the value of gci_no_dimd_constraint_flag is 0, there can be no separate constraint. (gci_no_dimd_constraint_flag equal to 1 specifies that sps_dimd_enabled_flag for all pictures in OlsInScope shall be equal to 0. gci_no_dimd_constraint_flag equal to 0 does not impose such a constraint.).

If the current block is decoded in DIMD mode, additional information (syntax elements) related to the encoding mode may not be parsed. Referring to FIG. 13, if the value of cu_dimd_flag is 1, additional information related to the encoding mode of the current block (e.g., intra_mip_flag, intra_subpartitions_mode_flag, intra_luma_mpm_flag, intra_luma_not_planar_flag, intra_luma_mpm_idx, intra_luma_mpm_remainder, etc.) may not be parsed. Afterwards, if a residual signal exists, whether a transform coefficient for the residual signal exists and syntax elements related to the residual signal can be parsed.

Specifically, FIG. 14 is a structural diagram showing a process of generating a prediction sample using the derived intra prediction mode by more effectively deriving an intra prediction mode to improve encoding performance.

Referring to FIG. 14, in the 'prediction mode generator' process, the decoder can derive an intra prediction directional mode for the current block using neighboring samples of neighboring blocks adjacent to the current block. At this time, the decoder can derive at least one intra prediction directional mode and derive a weight for each mode. For example, in order to derive the intra prediction directional mode, the decoder can use a histogram-based method that derives directional information of neighboring block samples through an arbitrary filter and determines frequently occurring directional information as the intra prediction directional mode. In addition, as a method for deriving the intra prediction directional mode, a method can be used that generates an intra prediction pixel for a left pixel adjacent to the current block using only upper pixels adjacent to the current block and determines the intra prediction mode with the least distortion as the intra prediction directional mode for the current block.

Referring to FIG. 14, in the 'intra prediction' process, prediction blocks for the current block can be generated using the intra prediction directional modes and weights derived from the 'prediction mode generator' process. The number of prediction blocks can be determined according to the number of intra prediction directional modes derived from the prediction mode generator process. For example, if the number of derived intra prediction directional modes is 2, there can be 2 prediction blocks for the current block. The prediction blocks generated in the 'intra prediction' process can be subjected to PDPC (Position dependent intra prediction combination) filtering using the method described below.

PDPC (Position dependent intra prediction combination) filtering may be applied to each prediction block generated in the 'intra prediction' process. If PDPC filtering is applied to each prediction block, complexity may increase in terms of the decoder, so if the prediction block is predicted in DIMD mode, PDPC filtering may not be applied to the prediction block. In addition, PDPC filtering may be applied only to either modeA with the largest accumulated value or modeB with the second largest accumulated value. For example, PDPC filtering may be applied only to modeA. In addition, whether PDPC filtering is applied may be determined based on the weight of each directional mode. For example, whether PDPC is applied to all or part of modeA and modeB may be determined based on the difference between the weight for modeA and the weight for modeB. For example, if the difference between the weight for modeA and the weight for modeB is smaller than a specific value, PDPC filtering may be applied to both modeA and modeB. In addition, whether PDPC filtering is applied to modeA and modeB can be determined by comparing the weight for modeA and the weight for modeB respectively with a specific value. If the weight is greater than a specific value, PDPC filtering can be applied to the directional mode of the corresponding weight. For example, if the weight for modeA is greater than a specific value and the weight for modeB is less than a specific value, PDPC filtering can be applied to modeA and PDPC filtering can not be applied to modeB. In addition, regardless of the directional mode, PDPC filtering of a preset form can be applied only to the final prediction block to which the weighted average is applied through the weighted prediction process (see Fig. 14). In addition, PDPC filtering can be applied to the final prediction block to which the weighted average is applied using modeA in the 'weighted prediction' process. PDPC filtering can be applied to the final prediction block to which the weighted average is applied using modeB in the 'weighted prediction' process.

Referring to FIG. 14, in the 'other prediction' process, the decoder can additionally generate a prediction block for the current block. For example, the decoder can generate an intra prediction block using at least one of a planar mode, a DC mode, and a MIP (Matrix Intra Prediction). Whether the 'other prediction' process is performed can be determined using at least one of the intra prediction directional modes induced in the 'prediction mode generator' process and weight information for each prediction direction, quantization parameter information of the current block, the horizontal or vertical length of the current block, information on whether the current block is luminance or chrominance, intra prediction modes around the current block, and information on the presence or absence of transform coefficients around the current block (which may correspond to Additional information A, B, and C of FIG. 14). A method for determining whether the 'other prediction' process is performed will be described below.

In the 'other prediction' process, information about which mode (e.g., planar, DC, MIP mode, etc.) the decoder will use can be agreed upon in advance or signaled via SPS. For example, the decoder can determine the mode based on a syntax element (sps_dimd_default_mode) indicating which mode to use. The decoder can determine which mode to use among the planar mode, the DC mode, and the MIP mode depending on the value of sps_dimd_default_mode. For example, if the value of sps_dimd_default_mode is '0', the decoder can be instructed to use the planar mode, if the value of sps_dimd_default_mode is '1', the decoder can be instructed to use the DC mode, and if the value of sps_dimd_default_mode is a value other than 0 and 1, the decoder can be instructed to use the MIP mode. In addition, if the current block is a luminance block and transform coefficients of neighboring blocks exist, the decoder can generate a prediction block using at least one of the planar mode, the DC mode, and the MIP mode. If the current block is a chrominance block and there is no transform coefficient of a surrounding block, the prediction block can be generated using at least one of the planar mode, the DC mode, and the MIP mode. In addition, if the weights of the intra prediction directional modes derived in the 'prediction mode generator' process are similar to each other (for example, if the difference between the weights of each directional mode is smaller than a specific threshold value), the 'other prediction' process may not be performed. If the weights of the intra prediction directional modes derived in the 'prediction mode generator' process are similar to each other, the decoder can generate the prediction block using at least one of the planar mode, the DC mode, and the MIP mode (i.e., the 'other prediction' process is performed). If the difference between the weights of the intra prediction directional modes derived in the 'prediction mode generator' process is large (for example, if the difference between the weights of each directional mode is larger than a specific threshold value), it means that there is a lot of change between pixels of the surrounding block, and therefore the decoder can generate the prediction block using at least one of the planar mode, the DC mode, and the MIP mode. In addition, if the width and height of the current block are different from each other, the decoder can generate the prediction block using at least one of the planar mode, the DC mode, and the MIP mode. Conversely, if the width and height of the current block are the same, the decoder can generate the prediction block using at least one of the planar mode, the DC mode, and the MIP mode.

In the 'weighted prediction' process, the decoder can generate a single prediction sample by weighting and averaging multiple intra prediction blocks generated in the 'intra prediction' and 'other prediction' processes. The weight for each intra prediction block can be determined based on at least one of the intra prediction directional mode and weight information derived in the 'prediction mode generator' process, quantization parameter information of the current block, the horizontal or vertical length of the current block, information on whether the current block is luminance or chrominance, intra prediction modes around the current block, and information on whether there are transform coefficients around the current block.

Fig. 15 illustrates the 'prediction mode generator' process of Fig. 14 in more detail. Referring to Fig. 15, the 'prediction mode generator' process of Fig. 14 can derive intra prediction directionality through histogram analysis. Specifically, in the 'histogram analysis' process of Fig. 15, the decoder can derive intra prediction directionality by analyzing a histogram obtained using neighboring samples adjacent to the current block. At this time, the decoder can derive the intra prediction directionality mode and weight for the current block by using at least one of the horizontal length and vertical length of the current block, quantization parameter information, possible intra prediction directionality mode information among neighboring blocks of the current block, information on whether a residual signal exists in neighboring blocks of the current block, and information on whether the current block is a luminance block or a chrominance block. A method for deriving the intra prediction directionality mode and weight for the current block will be described below.

The intra prediction directional mode can be set based on the frequency. The decoder can obtain a histogram for the intra prediction directional mode for the surrounding block, analyze the histogram, and select a frequently occurring intra prediction directional mode and a second frequently occurring mode as the prediction directional mode. In addition, the intra prediction directional mode can be set based on an accumulated value (e.g., a G value in FIG. 11). The decoder can analyze a histogram obtained as an accumulated value in which a G value is added to each intra prediction directional mode, and select an intra prediction directional mode having the highest weight and a mode having the second largest weight as the prediction directional mode. In addition, the decoder can select an intra prediction directional mode based on a distance between intra prediction directional modes of the surrounding block and an accumulated value in which a G value is added. The distance between directional modes can mean a difference in the indexes of the directional modes. For example, the difference in the distance between the directional mode of index 66 and the directional mode of index 2 can be 64. Or, since the index of the directional mode is the last index, 66, the distance difference between the directional mode of index 66 and the directional mode of index 2 can be 2. The decoder can obtain a histogram with the accumulated value in which the G value is added to each of the intra prediction directional modes for the surrounding blocks, analyze the histogram, and first select the intra prediction directional mode with the highest accumulated value. Next, the decoder can use the mode with the smallest distance between the mode with the highest accumulated value and the directional mode (the closest mode) among the modes corresponding to the remaining accumulated values except for the highest accumulated value (for example, the mode with the second highest accumulated value, the mode with the third highest accumulated value, the mode with the fourth highest accumulated value, etc.). Meanwhile, the decoder first selects the intra prediction directional mode having the highest accumulation value, and the decoder can use the mode having the highest accumulation value and the directional mode having the largest distance (the farthest distance) among the modes corresponding to the remaining accumulation values except for the highest accumulation value (for example, the mode having the second highest accumulation value, the mode having the third highest accumulation value, the mode having the fourth highest accumulation value, etc.). The accumulation value for each of the intra prediction directional modes can be used when determining weights for the intra prediction directional modes finally determined in the 'Histogram analysis' process.

The decoder of Fig. 15 may have two or more intra prediction directional modes for the current block derived from the 'Histogram analysis' process. When there are two or more intra prediction directional modes derived from the 'Histogram analysis' process, the distances between the intra prediction directional modes may be similar or different. In addition, the accumulated values between the intra prediction directional modes may also be similar or different. Therefore, in order to derive an optimal prediction sample for the current block, the most optimal combination among various mode combinations should be selected. In addition, the decoder may combine not only the intra prediction directional modes derived from the 'prediction mode generator' process of Fig. 14 but also the encoding modes derived from 'other prediction' in order to derive an optimal prediction sample for the current block. Information about such combinations may be included in the bitstream. The mode combination described in this specification may mean using one of mode A, mode B, planar mode, DC mode, and MIP mode, or combining some or all of them.

Next, referring to FIG. 15, the decoder can select an optimal combination for deriving an optimal prediction sample for the current block by using the intra prediction modes determined in the 'Histogram analysis' process and the weight information corresponding to the derived intra prediction modes in the 'Prediction mode analysis' process. Specifically, the decoder can determine information on whether to use a weighted average to generate a prediction sample for the current block, which intra prediction mode to use, and how to set the weight for the intra prediction mode by using the derived intra prediction modes and the weight information corresponding thereto. In addition, the decoder can select an optimal combination for deriving a prediction sample for the current block by using the intra prediction modes determined in the 'Histogram analysis' process and the weight information corresponding to the determined intra prediction modes and at least one of the intra prediction modes of the neighboring blocks in the 'Prediction mode analysis' process. Specifically, the decoder can determine information on whether to use weighted average to generate a prediction sample for the current block, which intra prediction mode to use, and how to set weights for the intra prediction modes. At this time, the decoder can derive optimal combination information (Combination information for prediction modes) for generating a prediction sample for the current block by using at least one or more of the horizontal or vertical length of the current block, quantization parameter information, available intra prediction mode information among the neighboring blocks of the current block, information on whether residual signals exist in the neighboring blocks of the current block, and information on whether the current block is a luminance block or a chrominance block. The combination information can include intra prediction directional mode information and weights for the intra prediction directional modes. For example, if the weight of the mode with the second highest weight among the two derived intra prediction directional modes is '0' or within an arbitrary value, the decoder can generate a prediction block for the current block without applying weighted average, but using only one intra prediction directional mode with the highest weight. At this time, the arbitrary value is an integer greater than or equal to 1, and can be 10. In addition, if at least one of the two derived intra prediction directional modes is the DC mode, the Planar mode, or the MIP mode (i.e., if it is not a directional mode), the decoder can generate the prediction block for the current block by using only the intra prediction directional mode with the highest weight, without applying the weighted average, when generating the prediction block for the current block. In addition, if at least one of the two derived intra prediction modes is the DC mode, the Planar mode, or the MIP mode (i.e., if it is not a directional mode), the decoder can apply the weighted average when generating the prediction block for the current block.

Referring to FIG. 16, if the current block is encoded with DIMD (i.e., if the value of cu_dimd_flag is 1), the decoder can additionally parse a syntax element (cu_dimd_mode) for DIMD combination information (information on modes combined for obtaining prediction samples, mode combination information). At this time, cu_dimd_mode may be parsed differently depending on the number of prediction modes to be combined. For example, if the number of combinations is '2', the decoder can parse only 1 bin. At this time, if the value of cu_dimd_mode is '0', the decoder can generate a prediction sample using modeA and modeB. If the value of cu_dimd_mode is '1', the decoder can generate a prediction sample using modeA, modeB, and the planar mode, or generate a prediction sample using modeA and the planar mode, or generate a prediction sample using modeB and the planar mode. When the number of combinations is '4', the decoder can parse 2 bins. At this time, when the value of cu_dimd_mode is '0', the decoder can generate prediction samples using modeA and modeB. When the value of cu_dimd_mode is '1', the decoder can generate prediction samples using modeA, modeB, and the flat mode. When the value of cu_dimd_mode is '2', the decoder can generate prediction samples using modeA and the flat mode. When the value of cu_dimd_mode is '3', the decoder can generate prediction samples using modeB and the flat mode.

If the syntax element for DIMD combination information is included in the bitstream, there may be a problem that the bit quantity increases. To solve this problem, the syntax element for DIMD combination information is not included in the bitstream, and the decoder can derive combination information through information of the current block and surrounding blocks. As described above, the decoder can derive optimal combination information for generating a prediction sample for the current block.

Specifically, FIG. 17 shows weight information corresponding to the intra prediction directional mode for each block derived in the 'Histogram analysis' step described through FIG. 15. Referring to FIG. 17, the size of the weight (WeightX) can be expressed in alphabetical order. For example, the highest weight can be expressed as 'WeightA', and the second highest weight can be expressed as 'WeightB'. The weight information can exist as many as the number (X) of derived intra prediction directional modes. In addition, the intra prediction mode corresponding to the WeightA weight can be modeA. The characteristics of the weight information corresponding to the intra prediction directional mode can be different depending on the characteristics of the surrounding blocks adjacent to the current block. Referring to FIG. 17, Case 1 shows a case where the intra prediction directional modes of modeA and modeB and the weight information corresponding thereto are similar to each other. Case 2 shows a case where the difference between the intra prediction directional modes of modeA and modeB and the weight information corresponding thereto is greatly different. Case 3 shows a case where the intra prediction directional modes of modeA and modeB are similar, but the differences between the corresponding weight information are significantly different. Case 4 shows a case where the intra prediction directional modes of modeA and modeB are significantly different, but the corresponding weight information is similar to each other.

Specifically, Fig. 18 shows a method for determining optimal DIMD combination information through the difference between the intra prediction directional modes (modeA, modeB) and corresponding weights (WeightA, WeightB) induced in the 'prediction mode generator' process.

Referring to FIG. 18, i) if the difference in absolute value between ModeA and ModeB is less than an arbitrary threshold value (Tmode1, for example, 10) and WeightA is less than an arbitrary threshold value (Tweight1, for example, 0.7), the DIMD optimal combination for the current block may be a combination of modeA and modeB, and the decoder may generate a prediction sample by combining modeA and modeB. ii) if i) is not applicable and WeightA is greater than or equal to an arbitrary threshold value (Tweight2, for example, 0.85), the DIMD optimal combination for the current block may be a combination of only modeA, and the decoder may generate a prediction sample by using only modeA. iii) If ii) does not apply and the difference in absolute values between modeA and modeB is greater than or equal to an arbitrary threshold (Tmode2, for example, 15), the optimal DIMD combination for the current block may be a combination of modeA and some or all of the coding modes derived in the 'other Prediction' process (for example, planar mode, DC mode, MIP mode), and the decoder may generate prediction samples by combining some or all of the coding modes derived in the modeA and 'other Prediction' process (for example, planar mode, DC mode, MIP mode). iv) If iii) is not applicable, the optimal DIMD combination for the current block may be a combination of modeA and modeB and some or all of the coding modes derived in the 'other Prediction' process (e.g., planar mode, DC mode, MIP mode), and the decoder may generate prediction samples by combining some or all of the coding modes derived in the 'other Prediction' process (e.g., planar mode, DC mode, MIP mode).

Below, we describe a method for determining optimal DIMD combination information through the difference between the intra prediction directional modes (modeA, modeB) induced by the decoder and their corresponding weights (WeightA, WeightB).

The decoder can obtain DIMD combination information by comparing the weights of modeA and modeB with the sum of all weights of the histogram (see Fig. 11). Specifically, the DIMD combination information can be obtained by comparing the weights of the directional information for the surrounding blocks of the current block (including the weights of modeA and modeB) (e.g., the sum of the weights) with the weights of modeA and modeB.

For example, if there is one induced intra prediction directional mode (modeA or modeB), and if the ratio of the weight of the corresponding prediction directional mode among the total weights is greater than a specific ratio, the corresponding prediction directional mode may be selected. Meanwhile, if the ratio of the weight of the induced intra prediction directional mode among the total weights is equal to or less than a specific ratio, the DIMD combination information may be a combination of at least one of the induced intra prediction directional mode, the planar mode, the DC mode, and the MIP mode. In this case, the specific ratio may be 1/2, 2/3, 3/4, 3/8, etc.

As another example, if there are two induced intra prediction directional modes (modeA and modeB), and the ratio of the sum of the weights of each of the intra prediction directional modes among the total weights is greater than a specific ratio, the corresponding two intra prediction directional modes can be selected. Meanwhile, if the ratio of the sum of the weights of the two induced intra prediction directional modes among the total weights is equal to or less than a specific ratio, the DIMD combination information can select at least one of the induced intra prediction directional modes, the planar mode, the DC mode, and the MIP mode. For example, modeA, modeB, and the planar mode can be selected. Or modeA and modeB can be selected. In this case, the specific ratio can be 1/2, 2/3, 3/4, 3/8, etc.

Specifically, FIG. 19 shows the 'Intra prediction' and 'weighted prediction' processes of FIG. 14. Referring to FIG. 19, when there are multiple intra prediction directional modes induced by the decoder, the decoder can perform weighted prediction using the weight information of each of the multiple intra prediction directional modes to obtain a prediction sample. The weight information can be reset based on at least one of the horizontal length of the current block, the vertical length, the quantization parameter information, and the information on whether the current block is luminance or chrominance (Additional information).

FIG. 20 and FIG. 21 are diagrams showing pixel values of surrounding blocks used when deriving an intra prediction directional mode according to one embodiment of the present invention.

Figures 20 and 21 show pixel (pixel) values of surrounding blocks used when deriving an intra prediction directional mode in the 'histogram analysis' process of Figure 14. When deriving an intra prediction directional mode, filtering calculation is required for all surrounding pixels located on the left and upper sides of the current block. At this time, the surrounding pixels may be pixels on a line adjacent to or separated from the boundary of the current block. That is, in order to deriv an intra prediction directional mode, filtering calculation must be performed on all surrounding pixels on a line adjacent to or separated from the left and upper boundaries of the current block, so there is a problem that delay due to computational complexity may occur. To solve this problem, the decoder can separate the filtering calculation for the surrounding pixels located on the left and upper sides of the current block and derive the intra prediction directional mode for the surrounding pixels located on the left and upper sides in parallel. In addition, the decoder can also derive directional information by performing filtering calculation only on the surrounding pixels corresponding to an arbitrary set position.

Figures 20(a) and (c) show the surrounding pixels located on the left side of the current block used in the filtering calculation for deriving the intra prediction directional mode. Figures 20(b) and (d) show the surrounding pixels located on the upper side of the current block used in the filtering calculation for deriving the intra prediction directional mode.

The mappable intra directional information may vary depending on the location of the reference pixel that the decoder uses to obtain the histogram described in Fig. 11.

For example, the horizontal and vertical lengths of the current block may be the same. In this case, when deriving directionality information for surrounding pixels located on the left side of the current block in Fig. 20 (a), the mapped intra prediction directional mode can be used only for indices -14 to 34. When deriving directionality information for surrounding pixels located on the upper side of the current block in Fig. 20 (b), the mapped intra prediction directional mode can be used only for indices 34 to 80.

Meanwhile, the horizontal and vertical lengths of the current block may be different, and the positions of the surrounding pixels used when deriving direction information may vary depending on the horizontal and vertical lengths of the current block. For example, if the vertical length of the current block is longer than the horizontal length, the decoder may not use the surrounding pixels located above the current block, but may use only the surrounding pixels located on the left side to derive direction information. This has the effect of reducing computational complexity by using only the surrounding pixels located on the left side, not the surrounding pixels located above. If the horizontal length of the current block is longer than the vertical length, the decoder may apply a greater weight to the surrounding pixels located above the current block than to the surrounding pixels located on the left side to derive direction information. The weights may use specific values agreed upon in advance. For example, if the horizontal length of the current block is longer than the vertical length, a weight of 1 may be used for the surrounding pixels located on the left side of the current block, and a weight of 2 may be used for the surrounding pixels located above the current block. In other words, since the current block is longer horizontally than vertically, it is more effective to derive the intra prediction directional mode by using directional information about the surrounding pixels located above the current block rather than the surrounding pixels located to the left of the current block.

The decoder can perform filtering calculation only on a specific number of pixels among the surrounding pixels located around the current block. In this case, the specific number can be a multiple of N, and N can be 2, 3, 4, etc. Information about N can be included in the picture header information. Referring to Fig. 21, the decoder can perform filtering calculation and derive directionality information only on a position that is a multiple of '2' based on a position moved from the upper left of the current block to (-2, -2) in the x and y axes. In addition, the decoder can derive directionality information if the current block is a luminance block, and not derive directionality information if the current block is a chrominance block. The decoder can apply the directionality information found in the luminance block to the chrominance block. Meanwhile, the directionality information for the luminance block and the directionality information for the chrominance block can be obtained respectively. The chrominance block can utilize information obtained using at least one of the following modes: planar mode, DC mode, horizontal mode, vertical mode, and MIP mode, without using the directional information found in the luminance block.

The intra prediction directional mode of the current block is likely to be similar to the intra prediction directional modes of the neighboring blocks. Therefore, in order to encode the intra prediction directional mode of the current block, an MPM (Most Probable Mode) list is constructed using the intra prediction directional modes of the neighboring blocks, and information about whether the intra prediction directional mode of the current block exists in the MPM list and information about the position at which it exists can be included in the bitstream. That is, information about the intra prediction directional mode of the current block may not be separately included in the bitstream. Therefore, since the intra prediction directional mode of the current block is determined based on the information about whether the intra prediction directional mode of the current block exists in the MPM list and the information about the position at which it exists, information (i.e., the amount of bits) for deriving the intra prediction directional mode of the current block may vary depending on whether the MPM list is constructed effectively.

The method of deriving an intra prediction directional mode using the directional characteristics of surrounding pixels of the current block can also be used in the process of constructing an MPM list. The decoder can use the intra prediction directional mode for the current block, which is derived using the directional characteristics of surrounding pixels of the current block, to encode the intra prediction directional mode for the current block by adding it to the MPM list. This can be used when the surrounding blocks of the current block are not encoded with an intra prediction mode or do not have an intra prediction directional mode, such as the MIP (Matrix intra prediction) mode.

The neighboring blocks adjacent to the current block may include blocks without intra-prediction directional modes and blocks with intra-prediction directional modes. If the neighboring block located on the left side of the current block does not have an intra-prediction directional mode, the decoder can calculate directional characteristics using only the neighboring pixels located on the upper side of the current block and derive the intra-prediction directional mode of the current block. Alternatively, if the neighboring block located on the upper side of the current block has an intra-prediction directional mode and the neighboring block located on the left side of the current block does not have an intra-prediction directional mode, the decoder can include the intra-prediction directional mode of the neighboring block located on the upper side in the MPM list and the intra-prediction directional mode derived from the directional characteristics of the neighboring pixels located on the left side in the MPM list.

Referring to FIG. 22, the DIMD mode may be preferentially included in the MPM list. There may be multiple intra prediction directional modes of the current block derived through the DIMD mode. Therefore, the decoder may obtain prediction samples through multiple prediction. The intra prediction directional mode of a block having an intra prediction directional mode among the surrounding blocks of the current block may be added to the MPM list. If there is a blank space in the MPM list, a mode modified by +1 or -1 in modeA may be added to the list, and a DC mode, a horizontal mode, a vertical mode, and a MIP mode may be added. Instead of the DIMD mode, the TIMD (Template based Intra Mode Derivation) mode may be preferentially included in the MPM list. In addition, both DIMD and TIMD modes may be included in the MPM list. In addition, at least one of the two intra directional prediction modes derived using the DIMD mode and the two intra directional prediction modes derived using the TIMD mode may be included in the MPM list. In addition, if the intra prediction directional mode is used in a first region of one of the GPM blocks divided into two regions, the MPM list may be used to derive the intra prediction directional mode for the first region. The MPM list may include at least one of the two intra prediction directional modes derived using the DIMD mode and the two intra prediction directional modes derived using the TIMD mode.

If the MPM list includes the DIMD mode, information about whether the current block is encoded in the DIMD mode can be derived through the syntax element (mpm_idx). Therefore, additional information related to DIMD may not need to be signaled. At this time, if the current block is encoded in the DIMD mode, the reference line index may be 0 (mrl_ref_idx may be 0). In addition, if the DIMD mode is used, mrl_ref_idx may not be parsed and the value of mrl_ref_idx may be inferred as 0. In addition, the MPM list may include an intra prediction directional mode derived using the DIMD mode. If the intra prediction directional mode derived from DIMD is selected, mrl_ref_idx may be reset. For example, the value of mrl_ref_idx may be reset to any one of 0, 1, 2, ... The decoder can determine whether to add an intra prediction directional mode derived using the DIMD mode in the MPM list based on the value of mrl_ref_idx obtained by parsing mrl_ref_idx, and the priority of the intra prediction directional mode derived using the DIMD mode in the MPM list. For example, if the value of mrl_ref_idx is not 0, the decoder may not include the intra prediction directional mode derived using the DIMD mode in the MPM list. Or, only when the value of mrl_ref_idx is 0, the decoder may include the intra prediction directional mode derived using the DIMD mode in the MPM list. Or, only when the reference pixel line determined from the value of mrl_ref_idx is 0, 1, 2, or 3, the decoder may include the intra prediction directional mode derived using the DIMD mode in the MPM list. Alternatively, if the reference pixel line determined from the value of mrl_ref_idx is greater than 3, the decoder may not include the intra prediction directional mode derived using the DIMD mode in the MPM list. Alternatively, if the value of mrl_ref_idx is not 0, the decoder may include the intra prediction directional mode derived using the DIMD mode in the MPM list.

The intra prediction directional mode derived by the DIMD mode can be used to reorder intra prediction mode candidates in the MPM list. The decoder can derive the intra prediction directional mode using the DIMD mode after constructing an MPM list from neighboring blocks of the current block. The decoder can reorder the intra prediction mode candidates in the MPM list using the derived intra prediction directional mode. At this time, the decoder can reorder the MPM list using one or more of the derived intra prediction directional mode, the horizontal or vertical length of the current block, quantization parameter information, available intra prediction mode information among neighboring blocks of the current block, information on whether a residual signal of neighboring blocks of the current block exists, and information on whether the current block is a luminance block or a chrominance block.

The decoder can reorder the MPM list by using the difference between the intra prediction mode candidates in the MPM list and the derived intra prediction directional mode. For example, the decoder can calculate the difference between the derived intra prediction directional mode and each of the intra prediction mode candidates in the MPM list and sort the MPM list in order of the smallest difference (including 0). The intra prediction mode candidate with the smallest difference in the MPM list can be set to have the smallest index value in the MPM list. In addition, the derived intra prediction directional mode can be set to have the highest priority in the MPM list and can be set to have the smallest index value. Then, the decoder can sort the MPM list after the derived intra prediction directional mode by calculating the difference between the derived intra prediction directional mode and each of the intra prediction mode candidates in the MPM list in order of the smallest difference (including 0). In addition, when two MPM lists are used, the first MPM list can be composed of intra prediction mode candidates similar to the derived intra prediction directional mode in order of the smallest difference. The derived intra prediction directional mode and the intra prediction mode candidates in the MPM list can be organized in order of the smallest difference. The second MPM list can be organized using candidates that are not highly similar to the derived intra prediction directional mode. For example, the second MPM list can be organized in order of the largest difference between the derived intra prediction directional mode and the intra prediction mode candidates in the MPM list. When the size of the MPM list is fixed, there may be an unfilled empty space in the MPM list. At this time, a new prediction candidate derived using one or more of the candidates already included in the MPM list or frequently occurring candidates can be added to the empty space. For example, the new prediction candidate can be a candidate corresponding to a number that is increased or decreased by an arbitrary size in the '+' or '-' direction from the mode number (index) of the candidate already included. At this time, the arbitrary size can be a natural number such as '1', '2', '3', ..., and information about the arbitrary size can be included in the picture header information. In addition, when two MPM lists are used, the first MPM list may be composed of prediction modes obtained by referring to prediction modes of neighboring blocks of the current block, and the second MPM list may be composed of prediction modes derived by DIMD. At this time, if the number of prediction modes included in the MPM list is smaller than the number of prediction modes that can be included in a predefined MPM list, prediction modes derived by applying an offset to the prediction modes included in the MPM list may be added.

The intra prediction directional mode derived by the DIMD mode can be used to reorganize intra prediction mode candidates in the MPM list. The decoder can derive the intra prediction directional mode through the DIMD mode after constructing the MPM list based on the prediction modes of the surrounding blocks of the current block. Then, the decoder can reorganize the intra prediction mode candidates in the MPM list using the derived intra prediction directional mode to reconstruct multiple prediction candidates. At this time, the decoder can reorganize the MPM list using one or more of the derived intra prediction directional mode, the horizontal or vertical length of the current block, quantization parameter information, available intra prediction mode information among the surrounding blocks of the current block, information on whether a residual signal of the surrounding blocks of the current block exists, and information on whether the current block is a luminance block or a chrominance block. A method for reorganizing the MPM list is described below.

The decoder can reassemble the MPM list by using the difference between the derived intra prediction directional mode and the intra prediction mode candidates in the MPM list. For example, the decoder can select candidates whose difference is less than or equal to an arbitrary value, and include multiple prediction candidates composed by combining the derived intra prediction directional mode and the candidates (existing intra prediction modes in the MPM list) in the MPM list. At this time, the decoder can include the candidates in order of the smallest difference to the largest difference in the MPM list. Next, the decoder can insert the candidates whose difference is greater than the arbitrary value into the MPM list in order. At this time, the arbitrary value can be a natural number such as 1, 2, 3, ... For example, it can be assumed that the index of the derived intra prediction mode is '18', the indices of the candidates in the MPM list are '16', '21', '34', '1', '66', and the arbitrary value is 5. At this time, the indices '16' and '21', which have a difference of less than '5' from the induced intra prediction mode, can be changed to multiple prediction candidates, and the candidates in the MPM list can be changed to the prediction modes of indices '16, 18', '21, 18', '34', '1', and '66'. That is, '16, 18' and '21, 18' can be multiple prediction candidates. For example, if the decoder selects '16, 18' which is a multiple prediction candidate in the MPM list, the decoder can generate the final prediction block by weighting and averaging the prediction samples generated with the prediction mode of index 16 and the prediction samples generated with the prediction mode of index 18. At this time, if the number of MPM lists is limited to 5, the MPM list can be '16, 18', '21, 18', '16', '21', '34'. In addition, when two MPM lists are used, in this case, the first MPM list may be composed of candidates that are recombined using candidates that are similar to the derived intra prediction directional mode. The second MPM list may be composed of candidates that are not highly similar to the candidates of the first MPM list and the derived intra prediction directional mode. Accordingly, the first MPM list may be composed of multiple prediction candidates, and the second MPM candidates may be composed of single prediction candidates. Alternatively, the first MPM list may be composed of both single prediction candidates and multiple prediction candidates, and the second MPM candidates may be composed of only single prediction candidates. For example, the derived intra prediction mode may have index '18', the prediction mode candidates in the first MPM list may have indexes '16', '21', '34', '1', '66', and the prediction mode candidates in the second MPM list may have indexes '50', '2', '8', '30', '40', and an arbitrary value may be 5. At this time, the indices '16' and '21', which have a difference of less than 5 from the derived intra prediction mode index 18, can be changed to multiple prediction candidates. At this time, the first MPM list can be composed of the indices '16, 18', '21, 18', '16', '18', '34', and the second MPM list can be composed of the indices '1', '66', '50', '2', '8', '30', '40'.

An intra prediction directional mode can be encoded based on whether it is in the MPM list and, if so, where it is located. If the intra prediction directional mode does not exist in the MPM list, the intra prediction directional mode can be encoded based on the total number of intra prediction directional modes minus the total number of prediction modes in the MPM list. Specifically, there are a total of 67 intra prediction directional modes, and encoding can be performed for 61 of them, excluding the total number of 5 prediction modes in the MPM list and the planar mode. In this case, since the 61 intra prediction directional modes can be encoded using fixed length coding, encoding for a total of 6 bins is required.

Referring to FIG. 23, the decoder can use a template, which is a reconstructed arbitrary region (pixel(s)) adjacent to the current block, to derive an intra prediction mode of the current block. First, the decoder can generate a prediction template for the template using neighboring pixels (reference) adjacent to the template. Then, the decoder can use the intra prediction mode for the prediction template that is most similar to the already reconstructed template to reconstruct the current block. The method of deriving the intra prediction mode of the current block using the above-described template can be described as TIMD (Template intra mode derivation). At this time, the intra prediction mode can be a mode of index 0 to 67, and may only correspond to an intra prediction mode in the MPM list derived from the neighboring blocks of the current block. At this time, the intra prediction mode can be an intra prediction mode in the MPM list derived from the neighboring blocks of the current block and modes that differ from the corresponding intra prediction mode by an arbitrary number. The arbitrary number can be 1, 2, 3, ... Alternatively, the intra prediction mode for the template may only correspond to a directional mode, and may not correspond to a non-directional mode (planar mode, DC mode).

Below, we describe a method for deriving an intra prediction directional mode using the TIMD mode.

i) The decoder can set the size of the template. The horizontal or vertical size (length) of the template can be 4, and if the horizontal or vertical size (length) of the current block is 8 or less, the horizontal or vertical size of the template can be set to 2. ii) The decoder can set the type of the template. The type of the template can be classified into a type that uses only left samples, a type that uses only upper samples, and a type that uses all of the left, upper, and upper-left samples. The decoder can determine the type of the template according to whether the neighboring block is valid or whether the neighboring block can be used to induce the intra prediction directional mode. Meanwhile, if the neighboring block cannot be used to induce the intra prediction directional mode, the TIMD mode can be set to the planar mode, and the weighted averaging may not be performed. iii) The decoder can configure a template for the current block. iv) The decoder can derive intra prediction directional modes for neighboring blocks located on the left, above, upper-left, upper-right, and lower-left of the current block to determine whether the current block has directionality. v) If none of the neighboring blocks of the current block have directionality (for example, non-directional mode (DC mode, planar mode, MIP mode, etc.)), the decoder can select one intra prediction directional mode with the minimum cost and not perform the TIMD mode. In this case, weighted averaging using multiple prediction blocks may not be performed. vi) If at least one block has directionality among the neighboring blocks of the current block, the process described below can be performed. The process described below can be performed based on the intra prediction directional modes existing in the MPM list. This is because complexity can increase if all 67 intra prediction directional modes are checked. a. The decoder can construct an MPM list. b. Next, the decoder can modify the MPM list by adding DC mode, horizontal mode, and vertical mode into the MPM list if they do not exist in the MPM list. c. The decoder can perform an evaluation for all intra prediction directional modes in the modified list to compare costs. The decoder can select a first mode having a smallest cost and a second mode having a second smallest cost. d. The decoder can additionally perform an evaluation for an intra prediction directional mode corresponding to an index that is 1 less than or greater than an intra prediction directional mode index of the first mode and an intra prediction directional mode index of the second mode to increase accuracy. The decoder can perform an additional evaluation and reselect a third mode having a smallest cost and a fourth mode having a second smallest cost. Meanwhile, the first mode and the third mode can be the same, and the second mode and the fourth mode can be the same. e. The decoder can determine whether to perform weighted averaging based on the costs of the third mode and the fourth mode. If the difference between the cost of the third mode and the cost of the fourth mode is less than a specific value, the decoder can perform weighted averaging, and the weights of the third mode and the fourth mode can be determined based on the costs of the third mode and the fourth mode. If the difference between the cost of the third mode and the cost of the fourth mode is greater than a specific value, the decoder can generate the prediction block using only the third mode without performing weighted averaging. In this case, the specific value can be a pre-agreed value.

The size of the template may vary depending on the horizontal length or vertical length of the current block. For example, as in Fig. 23(a), an above template longer than the horizontal length of the current block may be configured. At this time, the vertical length of the above template may be a predetermined length. Similarly, a left template longer than the vertical length of the current block may be configured. At this time, the horizontal length of the left template may be a predetermined length. The predetermined length may be 1, 2, 3, ...

If the current block is located at the CTU boundary (if any of the upper, lower, left, and right boundaries of the current block is included in the boundary of the CTU), the reference pixel for deriving/predicting the template used for the TIMD mode can be changed. Referring to Fig. 24, if the upper boundary of the current block is included in the boundary of the CTU, the number of reference lines located at the upper side of the current block used for configuring the template can be only one. This is to minimize the usage of the line buffer memory. Therefore, the decoder can perform the TIMD mode by configuring only the left template of the current block, without configuring the upper template of the current block. At this time, the reference pixels for predicting the left template can be the upper reference pixel (above reference) and the left reference pixel (left reference) of the current block. At this time, the height of the left template can be the same as the height of the current block, as shown in Fig. 24(a). In addition, as shown in Fig. 24(b), the decoder can check whether the block adjacent to the left of the current block is a block that has already been restored, and if it is a block that has been restored, the height of the left template can be configured to be greater than the height of the current block.

In general, the accuracy of the prediction sample for the current block can be increased as the decoder refers to many adjacent pixels of the current block. On the other hand, referring to many adjacent pixels increases the required memory. In addition, if there is a block among the adjacent blocks of the current block that has not been restored yet, the corresponding area cannot be used as a template. In order to effectively process this increased memory and the area that has not been restored, as shown in Fig. 23(b), the length of the upper template can be set to be the same as the horizontal length of the current block, and the length of the left template can be set to be the same as the vertical length of the current block.

The decoder can use an intra prediction mode derived from a template to obtain a prediction sample for the current block. The decoder can generate a prediction sample using neighboring pixels adjacent to the current block, and can adaptively select which neighboring pixels to use to generate the prediction sample. In addition, the decoder can use multiple reference lines to generate the prediction sample, and in this case, index information of the multiple reference lines can be included in the bitstream.

For entropy coding, a context for the index of multiple reference lines for TIMD mode can be newly defined. The increase in the type of context may be related to memory and context switching complexity. Therefore, the context used for coding and decoding the index of multiple reference lines used in TIMD mode may be a reuse of the context for the index of existing multiple reference lines.

The transformation of the residual signal of the current block can be performed in two steps. The first transform can be a transform such as DCT-II, DST-VII, DCT-VIII, DCT5, DST4, DST1, identity transformation (IDT) that is adaptively applied horizontally and vertically, respectively. The second transform can be additionally applied to the transform coefficients for which the first transform is completed, and the second transform can be computed as a matrix multiplication between the first-transformed transform coefficients and a predefined matrix. The second transform can be described as a low frequency non-separable transform (LFNST). The matrix transform set for the second transform can vary depending on the intra prediction mode of the current block. Coefficient information of the transform matrix used for the second transform can be included in the bitstream.

When a secondary transform is applied to a current block to which the DIMD mode or the TIMD mode is applied, a transform set for the secondary transform can be determined based on an intra prediction mode derived from the DIMD mode or the TIMD mode. Coefficient information of a transform matrix used for the secondary transform can be included in a bitstream. A decoder can set matrix coefficient information of a secondary transform for the DIMD mode or the TIMD mode by parsing the coefficient information included in the bitstream. At this time, one of the two intra prediction modes derived from the TIMD mode can be used to select the primary transform or the secondary transform set. The intra prediction directional mode having the smallest cost by comparing the costs of each of the two intra prediction directional modes can be used to select the primary transform or the secondary transform set. In addition, one of the two intra prediction directional modes derived from the DIMD can be used to select the primary transform or the secondary transform set. By comparing the weights of each of the two intra prediction modes, the intra prediction directional mode with the highest weight can be used to select the first or second transform set.

The TIMD mode has high complexity because it predicts the template of the current block and uses the intra prediction mode derived from the template to generate the prediction block of the current block. Therefore, when the decoder generates the prediction template for the template region, the existing reference sample filtering process may not be performed. In addition, the TIMD mode may not be applied when the ISP mode is applied to the current block or when the CIIP mode is applied to the current block. The ISP mode or the CIIP mode may not be applied to the current block to which the TIMD mode is applied, or the syntax related to the ISP or CIIP may not be parsed. In this case, the value of the syntax related to the ISP or CIIP that is not parsed can be inferred as a predefined value.

Template prediction can be performed by dividing into a left template region and an upper template region adjacent to the current block, and an intra prediction mode can be derived for each template. In addition, two or more intra prediction modes can be derived for each template, and there can be four or more intra prediction modes for the current block. When there are two or more intra prediction modes, a prediction sample for the current block can be generated using all of the derived intra prediction modes, and the decoder can generate a final prediction block for the current block by weighting and averaging the generated prediction samples. At this time, at least three or more of two or more intra prediction modes derived from template prediction, a planar mode, a DC mode, and a MIP mode can be used to generate the prediction sample. For example, when the decoder generates (obtains) a prediction sample for the current block, the decoder can generate a final prediction sample by weighting and averaging the prediction samples generated using the intra prediction modes derived from template prediction and the planar mode.

Even when the CIIP mode is applied, prediction samples can be generated using the methods described above. The CIIP mode is a method that uses both intra prediction and inter prediction when generating prediction samples (blocks) for the current block. The prediction sample for the current block can be generated as a weighted average between the intra prediction samples and the inter prediction samples.

When the CIIP mode is applied to generate intra prediction samples, the DIMD mode or the TIMD mode can be used. In this case, when the DIMD mode is used, the intra prediction sample can be generated based on the DIMD combination information. For example, the decoder can generate the first prediction sample using the intra prediction mode with the highest weight and generate the second prediction sample using the intra prediction mode with the second highest weight. Then, the decoder can generate the final intra prediction block by performing a weighted average of the first prediction sample and the second prediction sample. In this case, the decoder can generate the final intra prediction block by performing a weighted average of three prediction samples, including the sample predicted in the planar mode and the first prediction sample and the second prediction sample among the neighboring blocks of the current block. In this case, the intra prediction sample can be generated based on the TIMD combination information. For example, the decoder can generate two prediction samples using each of the two intra prediction modes. And, the decoder can generate a final intra prediction sample by weighting the two prediction samples. At this time, the decoder can generate a final intra prediction sample by weighting the two prediction samples and the sample predicted in the planar mode.

Intra prediction samples may have different accuracy depending on their location. That is, pixels at locations far from surrounding pixels used for prediction in the prediction sample may include more residual signals than pixels at locations close to the pixels. Therefore, the decoder may divide the prediction sample into vertical, horizontal, and diagonal directions according to the direction of the intra prediction mode, and may set different weights according to the distances from surrounding pixels used for prediction. This may be applied to intra prediction blocks generated using the CIIP mode or intra prediction blocks generated using two or more intra prediction modes, and may set different weights for each pixel in the prediction block according to the distance between the location of the reference pixel and the pixel location in the prediction block. As an example of implementation, if the intra prediction mode of the current block is a mode having a vertical direction or a direction similar to the vertical direction, a higher weight may be set for each pixel location as the pixel location of the prediction block is closer to the upper pixel, and a lower weight may be set for each pixel location as the pixel location is farther from the upper pixel.

If the current block is encoded in the CIIP mode, the decoder can generate a final prediction block by weighting the intra prediction sample and the inter prediction sample. The pixel-wise weight in the inter prediction sample can be set by considering the pixel-wise weight of the intra prediction sample. For example, the pixel-wise weight of the inter prediction sample can be a value obtained by subtracting the pixel-wise weight of the intra prediction sample from the sum of all weights. In this case, the sum of all weights can be a value obtained by adding the weight of the intra prediction sample and the weight of the inter prediction sample at the pixel level.

When two or more intra prediction modes are used to generate prediction samples, the decoder can generate prediction samples based on each intra prediction mode, and perform a weighted average on the generated prediction samples to generate a final prediction sample. When generating prediction samples for each intra prediction mode, a pixel-wise weight according to the intra prediction mode can be applied.

The pixel-wise weights may be set based on at least one of the following: an intra prediction mode, the width and height of the current block, a quantization parameter, information about whether the current block is luminance or chrominance, whether the surrounding block is intra coded, and information about the presence or absence of residual transform coefficients of the surrounding block.

Referring to FIG. 25(a), the video signal processing device can generate a prediction sample (2502) within the current block based on a first reference pixel line (reference line 1) adjacent to the current block (2501) and a second reference pixel line (reference line 2) adjacent above the first reference pixel line. The prediction sample (2502) of FIG. 25(a) is only a sample corresponding to a position according to an embodiment of the present invention, and the position of the pixel is not limited thereto. In this specification, the meaning that the video signal processing device generates may be the same as the meaning that the video signal processing device acquires. FIG. 25(b) is a drawing showing FIG. 25(a) in more detail. For example, the video signal processing device can generate a first prediction pixel (2503) by using a smoothing filter, a cubic filter, or a Gaussian filter according to an intra prediction mode through six reference pixels of the first reference pixel line. And, the video signal processing device can generate a second prediction pixel (2504) by using a smoothing filter, a cubic filter, or a Gaussian filter according to the intra prediction mode through the six reference pixels of the second reference pixel line. The video signal processing device can generate a third prediction pixel (2505) by performing a weighted average through an arbitrarily determined weight on the generated first prediction pixel (2503) and the second prediction pixel (2504). At this time, the six reference pixels of the second reference pixel line may be reference pixels at a position moved by one pixel to the right compared to each pixel of the first reference pixel line, considering the intra prediction mode of the current block, the position of the pixel to be generated, the position of the reference pixel line, etc. The video signal processing device can generate a prediction sample (2502) within the current block based on the third prediction pixel (2505). Alternatively, the video signal processing device may generate the prediction sample (2502) within the current block based on the third prediction pixel (2505) and the distance between the third prediction pixel (2505) and the prediction sample (2502) within the current block. The weight used by the video signal processing device to generate the third prediction pixel (2505) may be an integer greater than or equal to 0. For example, the weight of the first prediction pixel (2503) may be 3, and the weight of the second prediction pixel (2504) may be 1. At this time, the positions of reference pixels used to generate the prediction sample (positions of six reference pixels of the first reference pixel line and six reference pixels of the second reference pixel line in FIG. 25) may vary based on at least one of the following: the intra prediction mode of the current block, the positions of pixels to be generated (for example, the positions of the first prediction pixel (2503), the second prediction pixel (2504), and the third prediction pixel (2505) in FIG. 25), the positions of reference pixel lines (for example, the positions of the first reference pixel line and the second reference pixel line in FIG. 25).

Referring to FIG. 26(a), the video signal processing device can generate a prediction sample (2601) within the current block by using reference pixels at the same positions in the vertical direction of two reference pixel lines. That is, each of the six pixels of the first reference pixel line (Reference line 1) and each of the six pixels of the second reference pixel line (Reference line 2) used to generate the prediction sample within the current block can have the same position in the vertical direction. The positions of the respective pixels can be determined regardless of the positions of the reference pixel lines. Referring to FIG. 26(b), the video signal processing device can generate a first prediction pixel (2602) by using six pixels of the first reference line. The video signal processing device can obtain a second prediction pixel (2603) by using six pixels (2607) of the second reference line. At this time, if a sample of the second reference line, which is the same as the sample of the first reference line in the vertical direction, is used, the right pixel (2606) of the second reference line may not be used. Accordingly, the video signal processing device can configure the right pixel (2606) of the second reference line by copying (padding) the pixel (2605) to the right pixel (2606) of the second reference line, and then use the right pixel (2606) of the second reference line to generate the second prediction pixel (2603). The video signal processing device can obtain the third prediction pixel (2604) by using the first prediction pixel (2602) and the second prediction pixel (2603). The video signal processing device can generate the prediction sample (2601) within the current block by using the third prediction pixel (2604). Only the pixels of the first and second reference pixel lines used to generate the prediction sample within the current block are different from those of FIG. 25 in FIG. 26, and the method for generating the prediction sample within the current block may be the same as that of FIG. 25.

When configuring an MPM list, the video signal processing device can include a DIMD mode that induces an intra prediction directional mode from a reconstructed neighboring block adjacent to a current block in the MPM list. At this time, if there are two intra prediction directional modes induced by the DIMD mode, the video signal processing device can include both intra prediction directional modes in the MPM list.

When a video signal processing device generates a prediction sample for a current block by using a plurality of reference pixel lines (methods of FIGS. 25 and 26), a DIMD mode included in the MPM list is such that pixels adjacent to the current block are not used, and the video signal processing device can derive an intra prediction directional mode by performing the DIMD method at pixel positions indicated by the reference pixel lines. Then, the derived intra prediction directional mode can be included in the MPM list.

In addition, the video signal processing device can derive an intra prediction directional mode using DIMD for each reference pixel line, and then add the derived intra prediction directional mode to the MPM list. The reference pixel line may be a line located at a position spaced apart by 1, 3, 5, 7, or 12 pixels from the upper left position of the current block. Alternatively, the reference pixel line may be a reference pixel line located at an upper adjacent position to the current block. At this time, the intra prediction directional mode derived by the video signal processing device using DIMD for each reference pixel line may be added to the MPM list. If the derived intra prediction directional modes are duplicated, the duplicated intra prediction directional modes may be excluded from the MPM list. In the present specification, a reference pixel line may be indexed. For example, a reference pixel line adjacent to the current block may be indexed and referred to as reference pixel line 0, and reference pixel lines spaced 1 pixel, 2 pixels, ..., n pixels away from the current block may be indexed and referred to as reference pixel line 1, reference pixel line 2, ..., reference pixel line n, respectively.

The reference pixel lines for performing DIMD may vary depending on the reference pixel lines used for the current block. For example, if the reference pixel lines for the current block are 1 and 3, the video signal processing device may perform DIMD using

reference pixel lines

0, 1, and 2 and derive an intra prediction directional mode. In addition, if the reference pixel lines for the current block are 5 and 7, the video signal processing device may perform DIMD using

reference pixel lines

5, 6, and 7 and derive an intra prediction directional mode. In addition, if the reference pixel lines for the current block are 12, the video signal processing device may perform DIMD using

reference pixel lines

11, 12, and 13 and derive an intra prediction directional mode.

If the current block is encoded in the DIMD mode, the encoder can generate and signal a bitstream including information related to reference pixel lines to be used for deriving an intra prediction directional mode. Then, the encoder can generate an intra prediction block using the reference pixel lines on which DIMD is performed and the intra prediction directional mode derived from the DIMD. If the current block is decoded in the DIMD mode, the decoder can parse the information related to the reference pixel lines, and then derive an intra prediction directional mode using the reference pixel lines corresponding to the information related to the reference pixel lines. Then, the decoder can generate an intra prediction block using the reference pixel line information and the intra prediction directional mode derived from the DIMD.

Referring to FIG. 27, a video signal processing device can generate a virtual new reference pixel line using a plurality of reference pixel lines, and generate a prediction sample within a current block based on the new reference pixel line.

Circles in the same horizontal row in Fig. 27 may mean pixels located on the same reference pixel line. Fig. 27(a) shows that the positions of reference pixels used when generating prediction samples using at least one of the intra prediction mode of the current block, the positions of pixels to be generated, and the positions of reference pixel lines vary for each reference pixel line. The video signal processing device may generate samples (2701-a to 2701-d) at the '1' position for each reference pixel line using a smoothing filter, a cubic filter, or a Gaussian filter according to the intra prediction mode, and may generate a virtual new reference pixel line (2702) using multiple samples at the '1' positions. Meanwhile, in order to reduce complexity, samples at the '1' position may not be generated, and a virtual new reference pixel line (2702) may be generated using integer unit reference pixels closest to the '1' position. The video signal processing device may generate prediction samples in the current block using the virtual new reference pixel line. Referring to FIG. 27(b), the video signal processing device can use four reference pixels at the same position in the vertical direction to generate a virtual new reference pixel line (2705). That is, the pixels for generating the virtual new reference pixel line (2705) may be pixels at the same position in the vertical direction, respectively, for each of the six reference pixels of the first reference pixel line and each of the six reference pixels of the second reference pixel line. The video signal processing device can obtain the first prediction pixel (2703) using at least one of the virtual new reference pixel line and position, the intra prediction mode of the current block, and the pixel position to be generated. The video signal processing device can generate the prediction sample (2704) of the current block using the first prediction pixel (2703). The number of reference pixel lines used to generate the virtual new reference pixel line may be a plurality of 2 or more. For example, 2 to 5 reference pixel lines may be used.

Referring to FIG. 28, a video signal processing device may use four reference pixels close to the position of a new reference pixel (2801) to be generated to generate a virtual new reference pixel line. At this time, the position of the new reference pixel may be the same as the reference pixel line in the vertical direction. In addition, the position of the new reference pixel may be a position adjacent to the direction of the intra prediction mode of the current block.

Referring to FIG. 29, a video signal processing device may receive a plurality of reference pixel lines and perform intra prediction to generate a prediction block within a current block. Depending on which reference pixel line is used, a different prediction block may be generated, and the video signal processing device may generate a final prediction block by performing a weighted average according to the weights input for each prediction block. At this time, the weights may be preset values. For example, the weight of a sample predicted by a main reference pixel line may be 3, and the weight of a sample predicted by a sub-reference pixel line may be 1. At this time, the weights may be determined based on at least one or more of the size of the current block, the horizontal or vertical size of the current block, the intra prediction mode of the current block, quantization parameter information, and the distance (or difference) between the main reference pixel line and the sub-reference pixel line. In addition, the reference pixel line may be determined based on at least one or more of the size of the current block, the horizontal or vertical size of the current block, the intra prediction mode of the current block, quantization parameter information, and MRL information. For example, the main reference pixel line may be a reference pixel line adjacent to the current block, and the sub-reference pixel line may be a reference pixel line indicated by the MRL. As another example, the main reference pixel line may be a reference pixel line indicated by the MRL, and the sub-reference pixel line may be a reference pixel line that is located at an arbitrary fixed position away from the reference pixel line indicated by the MRL, and the arbitrary fixed position may be an integer from -N to +N, and N may be an integer greater than 0.

In the method for generating prediction samples within the current block described above, the intra prediction mode used may be the same for each reference pixel line. Or, conversely, in the method for generating prediction samples within the current block described above, the intra prediction mode used may be different for each reference pixel line. That is, a signaled intra prediction mode may be used for the main reference pixel line, and in the sub-reference pixel line, a prediction mode (corresponding to the index) added or subtracted by an arbitrary value from (the index of) the intra prediction mode used in the main reference pixel line may be used. At this time, the arbitrary value may be an integer greater than or equal to 1. In addition, the video signal processing device may determine whether to increase or decrease the arbitrary value according to the value of the intra prediction mode used in the main reference pixel line. For example, the video signal processing device may increase the intra prediction mode by an arbitrary value when the angle is negative, and may decrease the intra prediction mode by an arbitrary value when the angle is positive.

Below we describe a method for determining the optimal reference pixel line for the current block (for restoration of the current block) based on the template.

Referring to FIG. 30, a video signal processing device can configure a reference template using reference pixel lines adjacent to a current block. The video signal processing device can generate a prediction sample for the location of the reference template using

reference pixel lines

1, 2, 3, etc. The video signal processing device can calculate a cost between the generated prediction sample and samples of the reference template. At this time, the cost can be calculated using a method such as SAD (Sum of Absolute Differences) or MRSAD (Mean-Removed SAD). A reference pixel corresponding to the minimum cost can be an optimal reference pixel. In addition, the encoder can rearrange the calculated cost in ascending order, configure a list for reference pixel lines, and then generate and signal a bitstream including information on an index for the optimal reference pixel line. The decoder can configure a list for reference pixel lines using the above-described method, parse the index for the optimal reference pixel line included in the bitstream, and generate a prediction sample using the reference pixel line indicated by the index. As described in this specification, a method for a video signal processing device to determine a reference pixel line based on a template can be described as a Template-based Multiple Reference Line (TMRL) method or a TMRL intra prediction method.

In order to reduce the complexity in determining the optimal reference pixel line, the optimal reference pixel line may not be used for all intra prediction modes, but may be used only for the intra prediction modes included in the MPM list. That is, the encoder may construct a list for the reference pixel lines by combining the intra prediction modes included in the MPM list and a plurality of reference pixel lines, and may calculate the cost between the prediction sample generated using each candidate in the list for the reference pixel line and the reference template. Then, the encoder may rearrange the list in ascending order based on the cost, and may reconstruct the list using only a few combinations with low costs. The encoder may generate and signal a bitstream including information about an index for the optimal combination information among the combination information (any intra prediction mode and any reference pixel lines) in the reconstructed list. The decoder may construct a list for the same reference pixel line using the above-described method, and then parse the index for the optimal combination information included in the bitstream to generate the prediction sample using the optimal combination information indicated by the index.

When prediction samples are generated using multiple reference pixel lines, different weights may be applied to each prediction sample generated through each reference pixel line. At this time, a template-based method may be used to derive an optimal weight. That is, the encoder may construct a list for reference pixel lines by combining intra prediction modes included in the MPM list, multiple reference pixel lines, and weights (for example, one of 3:1 and 2:2) for samples predicted by each reference pixel line, and may calculate a cost between a prediction sample generated using each candidate in the list for the reference pixel lines and a reference template. Then, the video signal processing device may rearrange the MPM list in ascending order based on the cost, and then reconstruct the MPM list by using only a few combinations with low costs. The encoder can generate and signal a bitstream including information about an index for an optimal combination information among combination information within the reconstructed list (any intra prediction mode, any reference pixel lines, and a weight (for example, one of 3:1 or 2:2) for samples predicted by each reference pixel line). After constructing a list for the same reference pixel lines through the above-described method, the decoder can parse the index for the optimal combination information included in the bitstream and generate a prediction sample using the optimal combination information indicated by the index.

In the method for determining a reference pixel line based on a template described through Fig. 30, a reference pixel line adjacent to a current block can be used to construct a template. Accordingly, a reference pixel line determined based on a template can be determined using reference pixel lines that are not adjacent to the current block. Hereinafter, a method in which a reference pixel line adjacent to a current block can also be used to determine a reference pixel line based on a template will be described.

As illustrated in FIG. 31, when a reference template is configured, a reference pixel line adjacent to a current block can also be used to determine a reference pixel line based on the template. Referring to FIG. 31(a), a reference template including only reference pixels adjacent to the left side of the current block can be configured. The encoder can generate a prediction sample for the reference template using at least one of the reference pixel line 0 on the upper side of the current block and/or the reference pixel line 1 on the left side of the current block, and calculate a cost between the reference template and the prediction sample. The encoder can generate a prediction sample for the reference template using

reference pixel lines

1, 2, 3 ..., and calculate a cost between the reference template and the prediction sample. FIG. 31(b) illustrates a case where the reference template includes only reference pixels adjacent to the upper side of the current block. Similarly, the encoder can generate a prediction sample for the reference template using at least one of the reference pixel line 0 on the left side of the current block and/or the reference pixel line 1 on the upper side of the current block, and compute a cost between the reference template and the prediction sample. The encoder can generate a prediction sample for the reference template using

reference pixel lines

1, 2, 3, ..., and compute a cost between the reference template and the prediction sample.

The encoder can construct a list of reference pixel lines by rearranging the computed costs in ascending order, and then generate and signal a bitstream including information about the index of the optimal reference pixel line. The decoder can construct a list of reference pixel lines using the above-described method, and then generate a prediction sample using the optimal reference pixel line determined by parsing the information about the index of the optimal reference pixel line included in the bitstream.

Referring to FIG. 32, an encoder may receive multiple reference pixel lines and perform intra prediction to generate prediction blocks for a template. Depending on which reference pixel line is used, a different prediction block may be generated. The encoder may perform a weighted average according to various weight information input to each of the prediction blocks to finally generate a prediction block for the template. Depending on which reference pixel lines are used and which weights are used, multiple prediction blocks may be generated. After calculating a cost between each prediction block and a reference template, the encoder may rearrange the prediction blocks in ascending order based on the cost corresponding to each prediction block, and may configure a separate list using only a predetermined number of top candidates. At this time, the predetermined number may be an integer greater than or equal to 2, and may be 10. The encoder may generate a prediction block for the current block using the combination information used to generate the prediction block in the separate list. And the encoder can select the optimal candidate from the list in terms of image quality and bit rate, and then generate and signal a bitstream including information about the index of the optimal candidate. The decoder can construct the same separate list through the above-described method, and generate a prediction sample using the optimal combination information indicated by the index of the optimal candidate determined by parsing the information about the index of the optimal candidate included in the bitstream.

When the current block is encoded in the intra prediction mode, the encoded intra prediction mode can be any one of the angular mode, the planar mode, the DC mode, and the MIP mode. The prediction according to the angular mode can be a prediction performed according to 65 angles, and the prediction according to the MIP mode can be a prediction performed based on a predefined matrix. The angular mode can be effective in a block having a characteristic such as an edge in the current block. However, when the current block has a gentle characteristic, a block predicted using the angular mode can generate a discontinuous edge at the boundary between blocks or a visible outline inside the block. This can be a factor that reduces the encoding efficiency. In addition, the DC mode can have a disadvantage of generating a visible edge at the boundary between blocks at a low bit rate. The planar mode can improve the edge problem caused by the angular mode and the DC mode, and can generate a prediction block without discontinuity.

Referring to FIG. 33, according to the planar mode, the video signal processing device can generate a linearly predicted value in the vertical direction and a linearly predicted value in the horizontal direction to generate a prediction sample within the current block. The video signal processing device can generate a prediction sample (value) within the current block by weighting the linearly predicted value in the vertical direction and the linearly predicted value in the horizontal direction.

A linearly predicted value in the vertical direction (predV (x, y)) can be generated based on Equation 4, and a linearly predicted value in the horizontal direction (predH (x, y)) can be generated based on Equation 5. In addition, a new predicted value (pred (x, y)) can be generated based on Equation 6. In Equations 4 to 6, W may be a horizontal size (width) of a current block, and H may be a vertical size (height) of the current block. rec(x, y) may mean a pixel value at the (x, y) coordinate. The predicted value (predV(x, y), predH(x, y), pred(x, y)) may mean a predicted pixel value at the (x, y) coordinate.

A video signal processing device may use only vertical linear prediction when performing prediction related to a current block according to a planar mode. Alternatively, the video signal processing device may use only horizontal linear prediction when performing prediction related to a current block according to a planar mode. Accordingly, the planar mode may be divided into three modes. That is, in addition to a method of weighting and averaging prediction blocks generated using conventional vertical and horizontal linear predictions, it may be divided into a vertical planar mode that uses only vertical linear prediction, and a horizontal planar mode that uses only horizontal linear prediction. An encoder may generate and signal a bitstream including information on which prediction mode among the three planar modes a current block used. A decoder may generate a prediction block for a current block based on a planar mode determined by parsing information included in the bitstream on which prediction mode was used.

An explicit method of signaling the information about which planar mode was used by including it in the bitstream may have a problem of increasing bit volume. To save bit volume, the decoder can implicitly derive the information about which planar mode was used by using at least one of the following: the size of the current block, the width or height of the current block, the width and height size ratio of the current block, the number of pixels in the current block, whether the current block is a luminance signal or a chrominance signal, intra prediction directional mode information of adjacent blocks to the current block, MPM list information for the current block, and intra prediction directional mode information derived from DIMD or TIMD.

The vertical planar mode and the horizontal planar mode can be more effective when the shape of the current block is rectangular rather than square. Therefore, the vertical planar mode and the horizontal planar mode can be applied (activated) only when the horizontal and vertical sizes of the current block are different from each other. In other words, the encoder can generate and signal a bitstream including planar mode selection information on whether the vertical planar mode, the horizontal planar mode, or the conventional planar mode is applied only when the horizontal and vertical sizes of the current block are different after comparing the horizontal and vertical sizes of the current block. The decoder can parse the planar mode selection information only when the horizontal and vertical sizes of the current block are different. On the other hand, the conventional planar mode can be applied when the horizontal and vertical sizes of the current block are the same.

For convenience of explanation in this specification, information about which prediction mode among the three planar modes the current block used may be described as planar mode selection information.

The planar mode selection information can be signaled for each coding unit. However, since the amount of bits may increase if the planar mode selection information is signaled for all coding units, the encoder may generate and signal a bitstream including the planar mode selection information only when a specific condition is satisfied. The decoder may determine an intra prediction directional mode for the current block by parsing the planar mode selection information when the specific condition is satisfied, and may generate a prediction block for the current block using the determined intra prediction directional mode. At this time, the specific condition may be a condition related to the horizontal and vertical sizes of the current block, a ratio of the horizontal and vertical sizes of the current block, whether the intra prediction directional mode of the current block is a specific mode (for example, whether the planar mode, DC mode, vertical mode, or horizontal mode is recognized), whether the coding mode of the current block is DIMD, TIMD, IntraTMP, IBC, ISP, or MIP coding mode, and index information of a reference pixel line used when generating the prediction block for the current block. Whether to encode and decode the plane selection information may be determined based on whether at least one of the specific conditions is satisfied. Specifically, the specific conditions may be 1) when the intra prediction directional mode of the current block is the conventional plane mode (i.e., when the index of the intra prediction directional mode is 0), 2) when the horizontal and vertical sizes of the current block are equal to or smaller than the maximum transform block size, and the product of the horizontal and vertical sizes of the current block is greater than the product of the minimum transform block size and the minimum transform block size, and 3) when the horizontal and vertical sizes of the current block are different from each other. In this case, the size of the minimum transform block may be an integer, such as 4 or 8, and the size of the maximum transform block may be an integer, such as 64, 128, 256, etc. When at least one of the specific conditions 1) to 3) described above is satisfied, the encoder may generate and signal a bitstream including the plane mode selection information, and the decoder may parse the plane mode selection information to determine the intra prediction directional mode for the current block.

If the current block is a chrominance component block, the vertical direction planar mode and the horizontal direction planar mode are not applied, and only the conventional planar mode can be applied. That is, if the current block is a chrominance component block, the video signal processing device can generate a prediction block using the conventional planar mode. Therefore, if the current block is a chrominance component block, the video signal processing device may not signal or parse the planar mode selection information. Meanwhile, even if the current block is a chrominance component block, the vertical direction planar mode or the horizontal direction planar mode can be applied in the same way as the luminance component block. For example, if the vertical direction planar mode is applied to the current block, the decoder can generate a prediction block for the luminance component block and the chrominance component block of the current block using the vertical direction planar mode.

If the horizontal size of the current block is greater than the vertical size, the number of pixels adjacent to the upper side of the current block is greater than the number of pixels adjacent to the left side of the current block, so the decoder can generate a prediction block for the current block using the vertical planar mode. If the vertical size of the current block is greater than the horizontal size, the number of pixels adjacent to the left side of the current block is greater than the number of pixels adjacent to the upper side of the current block, so the decoder can generate a prediction block for the current block using the horizontal planar mode. If the horizontal and vertical sizes of the current block are the same, the decoder can generate a prediction block for the current block using the conventional planar mode. Since the planar mode is implicitly determined according to the horizontal and vertical sizes of the current block, the encoder does not need to generate a bitstream including the planar mode selection information. The decoder can generate a prediction block for the current block according to the planar mode determined according to the horizontal and vertical sizes of the current block.

The vertical planar mode and the horizontal planar mode may not be applied in the ISP mode. The encoder may not signal planar mode selection information (i.e., the bitstream may not include the planar mode selection information) if the current block is encoded in the ISP mode and the current block is encoded in the planar mode. The decoder may not parse the planar mode selection information if the current block is encoded in the ISP mode and the current block is encoded in the planar mode. Conversely, the vertical planar mode and the horizontal planar mode may be applied in the ISP mode. The encoder may generate and signal a bitstream including planar mode selection information if the current block is encoded in the ISP mode and the current block is encoded in the planar mode. The decoder may generate a prediction block for the current block using the planar mode determined by parsing the planar mode selection information if the current block is encoded in the ISP mode and the current block is encoded in the planar mode.

A video signal processing device can replace the horizontal mode (the 18th angle mode in FIG. 6) among the conventional intra prediction directional modes with the horizontal plane mode. In addition, the video signal processing device can replace the vertical mode (the 50th angle mode in FIG. 6) among the conventional intra prediction directional modes with the vertical plane mode. In other words, the video signal processing device can use the horizontal plane mode and the vertical plane mode instead of the horizontal mode (the 18th angle mode in FIG. 6) and the vertical mode (the 50th angle mode in FIG. 6) among the conventional intra prediction directional modes. At this time, the video signal processing device can replace the horizontal mode (the 18th angle mode in FIG. 6) and the vertical mode (the 50th angle mode in FIG. 6) among the conventional intra prediction directional modes with the horizontal plane mode and the vertical plane mode only when the horizontal size and the vertical size of the current block are different.

The vertical planar mode and the horizontal planar mode can be effective for small blocks. Therefore, the video signal processing device can apply the vertical planar mode and the horizontal planar mode only when the size of the current block is smaller than an arbitrary size. At this time, the arbitrary size can be 16 or 32 in width or height. That is, the video signal processing device can apply the vertical planar mode and the horizontal planar mode to the current block when the width or height of the current block is smaller than or equal to 32. Alternatively, the video signal processing device may not apply the vertical planar mode and the horizontal planar mode to the current block when either the width or height of the current block is larger than 32. For example, the encoder may not include planar mode selection information in the bitstream when either the width or height of the current block is larger than 32. The decoder may perform prediction of the current block using the conventional planar mode without parsing the planar mode selection information when the encoding mode of the current block is the planar mode and either the width or height of the current block is larger than 32.

The DIMD and TIMD modes are modes for generating a predicted block by weighting averages of blocks predicted from various intra prediction modes. At this time, a block predicted using the planar mode can be used for the DIMD and TIMD modes. When a video signal processing device generates a predicted block using the planar mode used in the DIMD and TIMD modes, any one of a vertical planar mode, a horizontal planar mode, and a conventional planar mode can be used. Planar mode selection information can be signaled by being included in a bitstream. A decoder can parse the planar mode selection information to determine which of the three planar modes to use. For example, when the planar mode selection information indicates a vertical planar mode, a prediction block used for the DIMD and TIMD modes can be a block predicted using the vertical planar mode.

The MIP mode can be an effective mode for complex areas. In order to improve the encoding performance of the MIP mode, a multi-prediction based MIP mode can be used. That is, the multi-prediction based MIP mode is a method of generating a final prediction block for a current block by weighting and averaging a prediction block generated by the MIP method and a prediction block generated based on an intra-prediction directional mode. An encoder can generate and signal a bitstream including both encoding information for the MIP mode and encoding information for the intra-prediction directional mode. A decoder can parse both encoding information for the MIP mode and encoding information for the intra-prediction directional mode to generate a prediction block to which the MIP mode is applied and a prediction block to which the intra-prediction directional mode is applied, and then generate a final prediction block for the current block by weighting and averaging the two prediction blocks.

The MIP mode can be adaptively performed on a block-by-block basis. The encoder can generate and signal a bitstream including information on whether the multi-prediction-based MIP mode is used. The decoder can parse the information on whether the multi-prediction-based MIP mode is used to determine whether the multi-prediction-based MIP mode or the single-prediction-based MIP mode is used when generating a prediction block for the current block. The single-prediction-based MIP mode is a method of generating a prediction block using only the MIP mode.

In order to reduce complexity and reduce signaled encoding information, the encoder may additionally include and signal in the bitstream information about whether a multi-prediction based MIP mode is used, if the intra prediction mode of the current block is MIP. The encoder may additionally include and signal in the bitstream information about an intra-prediction directional mode, if the multi-prediction based MIP mode is used, if the current block is. The decoder may additionally parse information about whether a multi-prediction based MIP mode is used, if the intra-prediction mode of the current block is MIP, and additionally parse information about an intra-prediction directional mode, if the multi-prediction based MIP mode is used.

If an intra prediction directional mode is additionally signaled, the bit amount may increase and the compression efficiency may decrease. If a multi-prediction-based MIP mode is used for the current block, only an arbitrary determined intra prediction directional mode may be used. At this time, the arbitrary determined intra prediction directional mode may be one of the MPM lists, and the encoder may signal the bitstream by including index information about the intra prediction directional mode to be used in the multi-prediction-based MIP mode among the MPM lists. If a multi-prediction-based MIP mode is used for the current block, the decoder may parse the index information to determine an intra prediction directional mode to be used in the multi-prediction-based MIP mode among the MPM lists.

When the current block uses the multi-prediction based MIP mode, the vertical planar mode, the horizontal planar mode, and the conventional planar mode can be used to generate a prediction block based on the intra prediction directional mode. That is, the encoder can generate and signal a bitstream including the planar mode selection information. When the current block uses the multi-prediction based MIP mode, the decoder can parse the planar mode selection information to determine the intra prediction directional mode to be used in the multi-prediction based MIP mode.

When the multi-prediction based MIP mode is used for the current block, the video signal processing device can determine which of the three planar modes to use by using the intra prediction directional mode of the DIMD derived from the surrounding pixels of the current block. For example, when the multi-prediction based MIP mode is used for the current block, the video signal processing device can apply the horizontal planar mode to the current block to generate a prediction block based on the intra prediction directional mode if the DIMD (or TIMD) mode of the current block is less than 34. Alternatively, when the DIMD (or TIMD) mode of the current block is equal to or greater than 34, the vertical planar mode can be applied to the current block to generate a prediction block based on the intra prediction directional mode.

The video signal processing device may generate prediction blocks for each mode using the vertical planar mode and the horizontal planar mode when the planar mode is used for the current block, and may apply weights to each prediction block to generate a final prediction block. At this time, the weights for each prediction block may be the same or different depending on the intra prediction directional mode of the DIMD derived from the surrounding pixels of the current block. If the intra prediction directional mode of the DIMD derived from the surrounding pixels of the current block is greater than or equal to a certain value, the largest weight may be applied to the block predicted using the vertical planar mode. For example, the weight for the block predicted using the vertical planar mode may be 3, and the weight for the block predicted using the horizontal planar mode may be 1. If the intra prediction directional mode of the DIMD derived from the surrounding pixels of the current block is less than a certain value, the largest weight may be applied to the block predicted using the horizontal planar mode. For example, a weight for a block predicted using a horizontal plane mode may be 3, and a weight for a block predicted using a vertical plane mode may be 1. Any value is an integer, and may be 34 (angle mode 34 in FIG. 6). DIMD information derived from surrounding pixels of a current block may include information about a first intra prediction directional mode, a second intra prediction directional mode, and whether to perform weighted prediction. If weighted prediction is not performed among the DIMD information of surrounding pixels of the current block, the video signal processing device may generate a prediction block by applying the same weight.

When the CIIP mode is used to generate the prediction block of the current block, the prediction block of the current block can be generated by weighting and averaging the prediction block (intra prediction block) using the intra prediction mode and the prediction block (inter prediction block) using the inter prediction mode. At this time, when the intra prediction directional mode used to generate the intra prediction block in the CIIP mode is a planar mode, the encoder can generate and signal a bitstream including planar mode selection information. When the CIIP mode is used and the planar mode is used, the decoder can parse the planar mode selection information to determine the intra prediction mode used to generate the intra prediction block.

When the current block is predicted using the intra prediction mode, the continuity of pixel values at the boundary between the current block and the surrounding blocks may be discontinued. To resolve this discontinuity, PDPC (Position dependent intra prediction combination) filtering may be applied to the generated prediction block. When the vertical planar mode is used to generate the prediction block of the current block, PDPC filtering based on the planar mode may not be performed, but PDPC filtering based on the horizontal angle mode (for example, the 18-degree angle mode in FIG. 6) may be performed. Alternatively, when the vertical planar mode is used to generate the prediction block of the current block, PDPC filtering based on the planar mode may not be performed, but PDPC filtering based on the vertical angle mode (for example, the 50-degree angle mode in FIG. 6) may be performed. In addition, when the horizontal planar mode is used to generate the prediction block of the current block, PDPC filtering based on the planar mode may not be performed, but PDPC filtering based on the vertical angle mode (for example, the 50-degree angle mode in FIG. 6) may be performed. In addition, when the horizontal plane mode is used to generate the prediction block of the current block, PDPC filtering based on the plane mode may not be performed, but PDPC filtering based on the horizontal angle mode (e.g., the 18-degree angle mode of FIG. 6) may be performed. In this specification, the meaning that the current block is predicted may be the same as the meaning that the prediction block of the current block is generated.

Referring to FIG. 33 and mathematical expression 4, when the vertical plane mode is used to generate the prediction block of the current block, the prediction block of the current block can be generated based on the pixel value of rec(-1, H) which is a fixed position and the pixel value of rec(x, -1) which changes dynamically according to the x-axis coordinate. Similarly, referring to FIG. 33 and mathematical expression 5, when the horizontal plane mode is used to generate the prediction block of the current block, the prediction block of the current block can be generated based on the pixel value of rec(W, -1) which is a fixed position and the pixel value of rec(-1, y) which changes dynamically according to the y-axis coordinate. That is, since the prediction block can be generated so that the continuity of pixel values is maintained at the boundary portion between the neighboring blocks, PDPC may not be performed when the prediction block of the current block is generated using the vertical plane mode or the horizontal plane mode.

The PDPC filtering for the above-described planar mode can also be applied when the current block is encoded in CIIP mode.

When a video signal processing device configures an MPM list for a current block, if a neighboring block adjacent to the current block is predicted in a vertical planar mode or a horizontal planar mode, the video signal processing device may set the intra prediction directionality mode of the neighboring block to the planar mode and include it in the MPM list. Alternatively, when the video signal processing device configures an MPM list for a current block, if a neighboring block adjacent to the current block is predicted in a vertical planar mode or a horizontal planar mode, the intra prediction directionality mode of the neighboring block may be set to the DC mode and included in the MPM list. Alternatively, when the video signal processing device configures an MPM list for a current block, if a neighboring block adjacent to the current block is predicted in a horizontal planar mode, the intra prediction directionality mode of the neighboring block may be set to the horizontal angle mode (angle mode 18 in FIG. 6) and included in the MPM list. Alternatively, when the video signal processing device configures the MPM list for the current block, if a neighboring block adjacent to the current block is predicted in the vertical plane mode, the intra prediction directionality mode of the neighboring block may be set to the vertical angle mode (angle mode no. 50 in FIG. 6) and included in the MPM list. Conversely, since the horizontal plane mode may have a vertical characteristic, when the video signal processing device configures the MPM list for the current block, if a neighboring block adjacent to the current block is predicted in the horizontal plane mode, the video signal processing device may set the intra prediction directionality mode of the neighboring block to the vertical angle mode (angle mode no. 50 in FIG. 6) and include it in the MPM list. Alternatively, when the video signal processing device configures the MPM list for the current block, if a neighboring block adjacent to the current block is predicted in the vertical plane mode, the video signal processing device may set the intra prediction directionality mode of the neighboring block to the horizontal angle mode (angle mode no. 18 in FIG. 6) and include it in the MPM list.

The vertical plane mode and the horizontal plane mode proposed in this specification are not signaled as any specific mode, but can be signaled as included in any one of the intra prediction directional modes. Referring to Fig. 6, there are a total of 67 intra prediction directional modes, among which 0 (plane mode) and 1 (DC mode) are non-directional modes, and 2 to 66 are directional modes (angular modes). The intra prediction directional modes can be extended, including the newly defined vertical plane mode and horizontal plane mode. For example, in the existing intra prediction directional mode, mode 2 can be set as the vertical plane mode, and mode 3 can be set as the horizontal plane mode. That is, modes 0 (plane mode), 1 (DC mode), 2 (vertical plane mode), and 3 (horizontal plane mode) can be set as non-directional modes, and modes 4 to 68 can be set to be the same as the existing directional modes 2 to 66.

In the MTS transform, which is the first transform, the transform is calculated by applying the transform kernel to the vertical and horizontal directions of the error block respectively, so it can be said to be a separable transform method. On the other hand, in the LFNST transform, which is the second transform described above, the transform is calculated by applying the transform kernel only once without applying the transform kernel to the vertical and horizontal directions respectively, so it can be said to be a non-separable transform method. In addition, since the second transform described above is additionally applied to the first transformed transform coefficients of the block to which the DCT-2 transform is applied, it can be said to be a two-step transform technique. The second transform described above has high encoding efficiency, but it has the disadvantage of being complicated because the transform kernel is applied a total of three times. In order to reduce this complexity, the NSPT (Non-separable primary transform) method, which is a method of applying the transform using only the second transform method, can be applied. The NSPT transform method is a non-separable transform method, and is calculated by applying the transform kernel only once to the vertical and horizontal directions of the error block without applying the transform kernel separately. In a video signal processing device, the error block of the current block can be transformed or inversely transformed using one of the transform methods among MTS, DCT2 + LFNST, and NSPT transform methods.

There can be 35 transform sets used in LFNST and NSPT transforms, and they can vary depending on the intra prediction mode (see FIG. 6). That is, the video signal processing device can derive a transform set index of LFNST and NSPT transforms corresponding to the intra prediction mode (see FIG. 6) by referring to the transform set table of FIG. 50. In addition, the LFNST and NSPT transform sets can vary depending on the intra prediction mode, information on whether the current block is a luminance block or a chrominance block, the horizontal and vertical sizes of the current block, and whether the intra prediction directional mode of the current block is an extended angle mode. There can be an arbitrary number of transform matrices for each transform set. Here, the arbitrary number can be an integer greater than or equal to 1, and can be 3. The encoder can signal the index information for the optimal transform matrix among multiple transform matrices in the transform set by including it in the bitstream. In the decoder, after parsing the index for the optimal transformation matrix, the inverse transformation can be applied using the transformation matrix corresponding to the index in the transformation set.

The video signal processing device can apply three types of transform kernels, LFNST4, LFNST8, and LFNST16, depending on the size of the transform block. If the horizontal and vertical sizes of the current block (e.g., the transform block) are greater than or equal to 16, the video signal processing device can apply the LFNST16 transform kernel to the current block. If the horizontal and vertical sizes of the current block are greater than or equal to 8 and less than 16, the video signal processing device can apply the LFNST8 transform kernel to the current block. If the horizontal and vertical sizes of the current block are less than 8, the video signal processing device can apply the LFNST4 transform kernel to the current block.

Referring to Fig. 35, the gray area can be a low-frequency area after LFNST conversion, and the gray area can be described as a Region of Interest (ROI). A portion other than a pre-designated ROI can be zeroed out as 0. In the case of LFNST16, since only 96 samples are required, the remaining white sub-blocks except for 6 sub-blocks (gray area) as in Fig. 35(a) can be zeroed out. In the case of LFNST8, since only 64 samples are required, the remaining white sub-blocks except for 4 sub-blocks (gray area) as in Fig. 35(b) can be zeroed out. Finally, in the case of LFNST4, zeroed out can be processed. Here, the size of the sub-blocks in Fig. 35 can be N x N, where N is a positive integer, such as 4.

NSPT transform can be a transform method that replaces the existing DCT2 + LFNST transform method. For a transform block whose size is equal to or smaller than 16x16, one of the following kernels can be applied depending on the size of the transform block: NSPT4x4 (16x16 kernel), NSPT4x8 (32x20 kernel), NSPT8x4 (32x20 kernel), NSPT8x8 (64x32 kernel), NSPT4x16 (64x24 kernel), NSPT16x4 (64x24 kernel), NSPT8x16 (128x40 kernel), NSPT16x8 (128x40 kernel), NSPT4x32 (128x20 kernel), NSPT32x4 (128x20 kernel), NSPT8x32 (256x24 kernel), NSPT32x8 (256x24 kernel). NSPT, similar to LFNST, consists of 35 sets of transform kernels, and each set can consist of three candidates. The encoder can derive a set of transform kernels according to the intra prediction mode, and then generate and signal a bitstream including information about an index of an optimal candidate among the three candidates. The decoder can parse the information about the signaled index, and then use the transform kernel candidate indicated by the index information among the set of transform kernels derived using the intra prediction mode to inversely transform the current transform coefficients and obtain a residual block.

Zero-out may not be performed on a 4x4 block to which NSPT is applied. Also, the number of coefficients to be zeroed out may vary depending on the size of the kernel of NSPT. For example, NSPT of size 32x20 may be applied to a 4x8 block or an 8x4 block. Accordingly, out of 32 transform coefficients, only 20 transform coefficients may be zeroed out, while the remaining 12 transform coefficients may be zeroed out.

Referring to Fig. 36(a), the encoder can select which one of MTS, DCT2 + LFNST, and NSPT to apply to the residual block, and can transform the residual block and obtain transform coefficients based on the selected transform method. At this time, the encoder can signal information on which transform method was applied by including it in the bitstream. If the MTS transform is applied to the residual block, the transform coefficients of the residual block can be obtained by applying the MTS transform, and LFNST and NSPT transforms may not be applied. If the DCT2 + LFNST transform is applied to the residual block, the encoder can apply the DCT2 transform to the residual block to obtain the first transform coefficients, and apply the LFNST transform to the first transform coefficients to obtain the second transform coefficients. At this time, the MTS and NSPT transforms may not be applied. When the NSPT transform is applied to the residual block, the video encoder can output transform coefficients by applying the NSPT transform to the residual block, and at this time, the MTS and DCT2 + LFNST transforms may not be applied to the residual block.

Referring to FIG. 36(b), the decoder can parse information about which transform method is applied from the bitstream, and determine whether one of the inverse transform methods among MTS, DCT2 + LFNST, and NSPT is applied to the current transform coefficient based on the parsed information. Then, the decoder can perform an inverse transform on the transform coefficient based on the determined transform method, and obtain a residual block. If the MTS transform is applied to the current transform coefficient, the decoder can perform an inverse MTS transform on the transform coefficient to obtain a residual block, and at this time, LFNST and NSPT inverse transforms may not be applied. If the DCT2 + LFNST transform is applied to the second transform coefficient, the decoder can perform an inverse LFNST transform on the second transform coefficient to output a first transform coefficient, and perform an inverse DCT2 transform on the first transform coefficient to obtain a residual block, and at this time, MTS and NSPT transforms may not be applied. If the NSPT transform is applied to the current transform coefficients, the decoder can perform the NSPT inverse transform on the current transform coefficients to obtain a residual block, and at this time, the MTS and DCT2 + LFNST transforms may not be applied.

A video signal processing device can derive a transform kernel for each of MTS, LFNST, and NSPT transforms (or inverse transforms) using an intra prediction mode. In addition, the video signal processing device can determine which transform (or inverse transform) is applied among MTS, DCT2 + LFNST, and NSPT. At this time, in order to determine the transform, at least one or more of the horizontal and vertical sizes of a current block, information about whether a component of the current block is a luminance component or a chrominance component, information about whether the current block is a single tree or a dual tree, information about whether the current block is encoded in intra mode or inter mode, information about an encoding mode of the current block (e.g., IBC, Intra TMP, Merge, AMVP, GPM, SGPM, CCLM, CCCM), and information about quantization parameters of the current block may be used.

When the vertical plane mode is used to generate the prediction block of the current block, the characteristics of the error (residual) signal (block) may be similar to the characteristics of the error signal of the vertical angle mode (the 50-degree angle mode in Fig. 6). The multiple transform set (MTS) applied to the error signal may vary based on the intra prediction directional mode of the current block. That is, when the vertical plane mode is used to generate the prediction block of the current block, the encoder may perform the first transform process based on the multiple transform set by using the set of transform matrices of the vertical angle mode (the 50-degree angle mode in Fig. 6) instead of the set of transform matrices of the plane mode for the error signal. Alternatively, the encoder may perform the first transform using a predetermined transform set regardless of the multiple transform set. In addition, the encoder may perform the second transform through LFNST on the first-transformed transform coefficients by using the multiple transform set or the predetermined transform set. The transformation matrix used when performing LFNST may vary depending on the intra prediction directional mode. That is, if the vertical planar mode is used to generate the prediction block of the current block, the encoder may perform the transformation process using the set of secondary LFNST transformation matrices of the vertical angular mode (the 50-angle mode in Fig. 6) instead of the set of secondary LFNST transformation matrices for the planar mode for the error signal. If the horizontal planar mode is used to generate the prediction block of the current block, the encoder may derive the primary or secondary transformation matrix (or the set of matrices, the set of matrices, the set of kernels) using the horizontal angular mode (the 18-angle mode in Fig. 6) instead of the planar mode. When the vertical planar mode is used to generate the prediction block of the current block, the decoder can perform the transformation process by using the set of first LFNST transformation matrices of the vertical angle mode (the 50-degree mode in FIG. 6) instead of the set of first LFNST transformation matrices for the planar mode for the error signal. In addition, when the vertical planar mode is used to generate the prediction block of the current block, the decoder can perform the second transformation process based on the multiple transformation set by using the set of transformation matrices of the vertical angle mode (the 50-degree mode in FIG. 6) instead of the set of transformation matrices of the planar mode for the error signal. In the present specification, the encoder can perform the second transformation after the first transformation in the encoding process, which correspond to the first transformation and the second transformation in the decoding process of the decoder, respectively. That is, the first transformation performed by the encoder corresponds to the second transformation performed by the decoder (the inverse transformation of the first transformation performed by the encoder), and the second transformation performed by the encoder corresponds to the first transformation performed by the decoder (the inverse transformation of the second transformation performed by the encoder).

Referring to FIG. 33 and mathematical expression 4, when the vertical direction planar mode is used to generate the prediction block of the current block, the prediction block can be generated based on the pixel value of rec(-1, H), which is a fixed position, and the pixel value of rec(x, -1) that changes dynamically according to the x-axis coordinate. That is, when the vertical direction planar mode is used, since the pixel value changes in the x-axis, the characteristic of the error signal can be similar to the characteristic of the error signal of the horizontal angle mode (angle mode 50 of FIG. 6). The multiple transformation set applied to the error signal can vary based on the intra prediction directionality mode of the current block. That is, when the vertical direction planar mode is used to generate the prediction block of the current block, the encoder can perform the first transformation process based on the multiple transformation set by using the set of transformation matrices of the horizontal angle mode (angle mode 18 of FIG. 6) instead of the set of transformation matrices of the planar mode for the error signal. Alternatively, the encoder can perform the first transform using a predetermined transform set regardless of the multiple transform set. In addition, the encoder can perform the second transform through LFNST on the first transformed transform coefficients using the multiple transform set or the predetermined transform set. The transform matrix used when performing LFNST can vary depending on the intra prediction directional mode. That is, if the prediction block of the current block is generated using the vertical direction planar mode, the encoder can perform the transform process using the set of second LFNST transform matrices of the horizontal angular mode (the 18th angular mode in FIG. 6) instead of the set of second LFNST transform matrices for the planar mode for the error signal. If the horizontal direction planar mode is used to generate the prediction block of the current block, the encoder can derive the first or second transform matrix (or the set of matrices, the set of matrices, the set of kernels) using the vertical angular mode (for example, the 50th angular mode in FIG. 6) instead of the planar mode. When the vertical planar mode is used to generate the prediction block of the current block, the decoder can perform the transformation process by using the set of first LFNST transformation matrices of the horizontal angular mode (angle mode 18 in FIG. 6) instead of the set of first LFNST transformation matrices for the planar mode for the error signal. In addition, when the vertical planar mode is used to generate the prediction block of the current block, the decoder can perform the second transformation process based on the multi-transform set by using the set of transformation matrices of the horizontal angular mode (angle mode 18 in FIG. 6) instead of the set of transformation matrices of the planar mode for the error signal. When the horizontal planar mode is used to generate the prediction mode of the current block, the encoder can derive the first or second transformation matrix (or the set of matrices, the set of matrices, the set of kernels) by using the vertical angular mode (angle mode 50 in FIG. 6) instead of the planar mode. When the horizontal planar mode is used to generate the prediction block of the current block, the decoder can perform the transformation process by using the set of first LFNST transformation matrices for the vertical angular mode (the 50-degree angular mode in FIG. 6) instead of the set of first LFNST transformation matrices for the planar mode for the error signal. In addition, when the horizontal planar mode is used to generate the prediction block of the current block, the decoder can perform the second transformation process based on the multiple transformation set by using the set of transformation matrices for the vertical angular mode (the 50-degree angular mode in FIG. 6) instead of the set of transformation matrices for the planar mode for the error signal.

A method for selecting a set of multiple transforms available for a current block in a video signal processing device is described when a prediction block of a current block is generated using a vertical planar mode, a horizontal planar mode, or a conventional planar mode.

1) First, the video signal processing device can derive the nSzIdxW and nSzIdxH values based on the size of the current block to map the horizontal and vertical sizes of the current block into a single variable. nSzIdxW can be the minimum value among 3 and the difference between 2 and the logarithm of 2 calculated by the width of the current block, then discarding the decimal places. nSzIdxH can be the minimum value among 3 and the difference between 2 and the logarithm of 2 calculated by the height of the current block, then discarding the decimal places.

2) Next, the video signal processing device can derive the intra directional mode (predMode) of the current block. In the case of the TIMD mode, the intra prediction modes expanded from the existing 67 to 131 can be used, thereby reducing the precision to the existing 67 modes.

3) Next, the video signal processing device can derive the ucMode, nMdIdx, and isTrTransposed values.

A. If the current block is encoded in MIP mode, ucMode can be set to '0', nMdIdx can be set to '35', and isTrTransposed can be set to a value derived from MIP.

B. If the current block is not encoded in the MIP mode, ucMode can be set to the intra directional mode (predMode) of the current block. predMode can mean an index value of the intra directional mode. predMode can be determined through the extended angle mode according to the aspect ratio of the current block. The video signal processing device can clip predMode to a value between 2 and 66. If the prediction block of the current block is generated using the vertical plane mode, the video signal processing device can reset predMode to the horizontal prediction mode (angle mode 18 in FIG. 6). If the prediction block of the current block is generated using the horizontal plane mode, the video signal processing device can reset predMode to the vertical prediction mode (angle mode 50 in FIG. 6). When predMode is greater than the 34th angular mode which is a diagonal mode, the isTrTransposed value can be set to 1, and when predMode is equal to or less than 34, the isTrTransposed value can be set to 0. When predMode is greater than 34, the video signal processing device resets the value of predMode to a value that is a difference of 67 (the maximum value of the intra directional mode index) plus 1. For example, when predMode is 35, the video signal processing device can reset the 35th angular mode to the 33rd angular mode (67+1-35(predMode)). When predMode is 66, the video signal processing device can reset the 66th angular mode to the 2nd angular mode (67+1-66(predMode)). That is, by making it symmetrical with respect to the 34th angular mode which is a diagonal mode, there is an effect of reducing the size of the transformation mapping table of FIG. 35 by about half.

4) The video signal processing device can derive the nSzIdx value through the nSzIdxW, nSzIdxH, and isTrTransposed values. If the isTrTransposed value is '1', the value obtained by multiplying nSzIdxH by 4 and adding nSzIdxW can be set as nSzIdx. If the isTrTransposed value is '0', the value obtained by multiplying nSzIdxW by 4 and adding nSzIdxH can be set as nSzIdx.

5) The video signal processing device can derive nTrSet, which is an index of an available transformation type set, according to the predefined table of FIG. 38 by using nSzIdx, which is the size information of the current block, and nMdIdx, which is the intra-directional mode information of the current block. FIG. 38 defines an index of a transformation type set according to the intra-directional mode (0 to 34 and MIP) of the current block and the size index (0 to 15) of the current block. Referring to FIG. 38, nTrSet can be 80, and if the size of the current block is 4x8 and the intra-directional mode of the current block is 13, nTrSet can be '7'.

6) The video signal processing device can derive a transformation type set corresponding to nTrSet from the table of FIG. 39 by parsing mts_idx included in the bitstream. The transformation types in the vertical and horizontal directions are set differently depending on whether predMode is a value greater than the 34th angle mode, which is a diagonal mode. 0 to 79 of the gray-shaded vertical columns of FIG. 39 may correspond to nTrSet, and 0 to 3 of the gray-shaded horizontal columns may correspond to mts_idx. Referring to FIG. 36, if nTrSet is 7 and the value of mts_idx is 3, 22 may be selected from (2, 17, 18, 22). Then, DST1 and DCT5 corresponding to index 22 of the transformation type combination table of FIG. 40 are selected, and the vertical transformation type of the current block may be set to DST1 and the horizontal transformation type may be set to DCT5. 0 to 24 of the gray-shaded horizontal columns in Fig. 40 are indices selected through Fig. 39, and 0 to 1 of the gray-shaded vertical columns may represent vertical transformation types and horizontal transformation types, respectively. If the predicted directionality mode within the screen of the current block is greater than 34, which is a diagonal mode, the vertical and horizontal transformation types may be interchanged.

If the mts_idx value is '3' and the width and height of the current block are both 16 or less, the vertical or horizontal transformation type can be reset to the IDT transformation type through the process described below.

If the absolute difference between the index of the screen prediction directionality mode of the current block and the index of the horizontal direction mode, 18, is less than any predetermined value, the vertical transformation type may be reset to the IDT transformation type. If the absolute difference between the index of the screen prediction directionality mode of the current block and the index of the horizontal direction mode, 50, is less than any predetermined value, the horizontal transformation type may be reset to the IDT transformation type. At this time, the predetermined value may be an integer and may be determined based on the horizontal or vertical size of the current block. For example, the predetermined value may be determined through the table of Fig. 38. The table of Fig. 41(a) represents a case where the threshold value is set differently whenever the horizontal or vertical size differs by 4, and the table of Fig. 41(b) represents a case where the threshold value is set differently whenever the horizontal or vertical size differs by 2 times. If the size of the current block is 16x16, the vertical transformation type may not be reset to the IDT transformation type, but the existing transformation type may be maintained as is.

In order to reduce the amount of bits for encoding mts_idx, the number of transform type sets can be adaptively changed for each block. The number of transform type sets can be adaptively determined according to the sum of the absolute values of the transform coefficients of the current transform block. For example, the number of transform type sets can be adaptively determined to 1, 4, or 6. If the number of transform type sets is different, the maximum number of bins (bin_) for signaling mts_idx may be different. If the sum of the absolute values of the transform coefficients is less than or equal to 6, the number of transform type sets may be one. Therefore, if the sum of the absolute values of the transform coefficients is less than or equal to 6, the encoder may not include (i.e., may not signal) mts_idx in the bitstream, and the decoder may not parse mts_idx. If the sum of the absolute values of the transform coefficients is greater than 6 and less than or equal to 32, the number of transform type sets may be four, and if the sum of the absolute values of the transform coefficients is greater than 32, the number of transform type sets may be six. Transform Kernel In this case, if the number of transform type sets is plural (e.g., four or six), the encoder may generate and signal a bitstream including mts_idx based on the maximum number of transform type sets, The decoder can parse mts_idx based on a maximum number of transformation type sets, where transformation type sets can be used in the same sense as transformation kernel candidates.

If the current block is encoded in inter mode, the transform kernel set for the current block can be any one of the four combinations of transform kernel sets. In this case, the four combinations can be {(DST7, DST7), (DST7, DCT8), (DCT8, DST7), (DCT8, DCT8)}. The encoder can signal an index indicating the transform kernel set used among the four combinations by including it in the bitstream. The decoder can parse the index to determine the transform kernel set for the current block. For example, if the determined transform kernel set is (DST7, DCT8), DST7 can be applied to the current block in the horizontal direction and DCT8 can be applied in the vertical direction to perform transform/inverse transform. In addition, for complexity optimization, for images larger than 1920x1080 resolution, the maximum CU size to which inter MTS can be applied can be set to 32x32. The maximum CU size to which Inter MTS can be applied can be 16x16 for resolutions other than 1920x1080. Therefore, if the size of the current block is larger than 32x32 or 16x16, Inter MTS may not be applied to the current block, and DCT2 transform may be applied in the horizontal and vertical directions. In addition, DST7 and DCT8, which are transform kernels for the current block whose size is equal to or smaller than 16x16, may be replaced with a separable KLT (Karhunen-Loeve Transform).

Referring to Fig. 42(a), the part indicated by the dotted line between the P block and the Q block may indicate a block boundary. The block boundary may exist at any fixed size, and a block boundary may exist at every size of 4.

Fig. 42(b) shows samples on which filtering is performed based on block boundaries. The video signal processing device can perform a process of alleviating blocking artifacts occurring at block boundaries by performing deblocking filtering on the currently restored block. The deblocking filtering process can include a process of determining a transform block boundary, determining a sub-block boundary, determining a length of a filter to perform filtering, determining a filtering strength (bS), determining a filtering parameter, determining whether to perform filtering, and determining a type of filtering.

Specifically, FIG. 43 illustrates a flowchart of a method for a video signal processing device to derive a first or second transformation matrix (a set of matrices, a set of matrices, a set of kernels) when a current block is predicted using a vertical planar mode, a horizontal planar mode, or a conventional planar mode.

Referring to FIG. 43, a video signal processing device can receive an intra prediction mode and determine whether the prediction mode of a current block is a planar mode. If the prediction mode is not a planar mode, the video signal processing device can derive a first or second transformation matrix (a set of matrices, a set of matrices, a set of kernels) based on the input intra prediction directional mode. If the prediction mode is a planar mode, the video signal processing device can determine whether the prediction mode is a vertical planar mode. If the prediction mode is a vertical planar mode, the video signal processing device can derive a first or second transformation matrix (a set of matrices, a set of matrices, a set of kernels) based on a horizontal angle mode (angle mode 18 of FIG. 6). If the prediction mode is not a vertical planar mode, the video signal processing device can determine whether the prediction mode is a horizontal planar mode. When the prediction mode is a horizontal planar mode, the video signal processing device can derive a first or second transformation matrix (a set of matrices, a set of matrices, a set of kernels) based on a vertical angle mode (a 50-degree angle mode in FIG. 6). When the prediction mode is not a horizontal planar mode, the video signal processing device can derive a first or second transformation matrix (a set of matrices, a set of matrices, a set of kernels) based on a conventional planar mode.

If a prediction block of a current block is generated by using any one of the vertical planar mode, the horizontal planar mode, and the conventional planar mode, the video signal processing device can derive a first or second transformation matrix (a set of matrices, a set of matrices, a set of kernels) using the intra prediction directional mode derived by using the DIMD method. In other words, if a prediction block of a current block is generated by using any one of the vertical planar mode, the horizontal planar mode, and the conventional planar mode, the video signal processing device can perform a first transformation process based on a multi-transform set by using the set of transformation matrices of the intra prediction directional mode derived by using the DIMD method. In addition, if a prediction block of a current block is generated by using any one of the vertical planar mode, the horizontal planar mode, and the conventional planar mode, the video signal processing device can perform a second transformation process based on LFNST by using the set of transformation matrices of the intra prediction directional mode derived by using the DIMD method. The DIMD information derived from the surrounding pixels of the current block may include the first intra prediction directional mode, the second intra prediction directional mode, and information on whether to perform weighted prediction. If weighted prediction is not applied among the DIMD information from the surrounding pixels of the current block in the encoder and decoder, the existing Planar prediction mode can be used to derive the first or second transformation matrix (or a set of matrices, a set of matrices, a set of kernels).

Since the planar mode can be effective in a smooth area, in order to reduce complexity, the encoder can implicitly apply the DCT2 transform method without performing the first transform process based on the multiple transform set. The decoder can apply the DCT2 transform method if the current block is predicted in the planar mode. If the current block is predicted in the conventional planar mode, the video signal processing device can implicitly apply the DCT2 transform method without performing the first transform process based on the multiple transform set. If the current block is predicted in the vertical direction planar mode or the horizontal direction planar mode, the video signal processing device can apply the first transform process based on the multiple transform set. Meanwhile, if the current block is predicted in the vertical direction planar mode or the horizontal direction planar mode, the video signal processing device can implicitly apply the DCT2 transform method without applying the first transform process based on the multiple transform set. If the current block is predicted in the conventional planar mode, the video signal processing device can apply the first transform process based on the multiple transform set. In addition, in the above-described method, if the size of the current block is an arbitrary size, the DST7 transform method may be applied instead of the DCT2 transform method. At this time, the arbitrary size may be a case where the horizontal or vertical size of the current block is greater than or equal to 4 or less than or equal to 16. For example, if the current block is predicted based on the vertical planar mode or the horizontal planar mode, and the horizontal size of the current block is 32 and the vertical size is 16, the video signal processing device may apply DCT2 for horizontal transform and DST7 for vertical transform.

Similar to the planar mode, the DC mode can also be classified into three prediction modes (three DC modes, three-directional DC prediction modes, or three DC prediction modes can be used interchangeably). Specifically, the DC mode can be classified into a mode according to an average between upper pixels of the current block, an average between left pixels of the current block, and an average between upper and left pixels of the current block. If the horizontal size of the current block is greater than the vertical size, the video signal processing device can generate a prediction block of the current block using an average value between upper pixels of the current block. If the vertical size of the current block is greater than the horizontal size, the video signal processing device can generate a prediction block of the current block using an average value between left pixels of the current block. If the horizontal size and the vertical size of the current block are equal, the video signal processing device can generate a prediction block of the current block using an average value between upper and left pixels. The encoder can explicitly signal information indicating which of the three DC prediction modes (i.e., the mode according to the average between the upper pixels of the current block, the average between the left pixels of the current block, and the average between the upper and left pixels of the current block) is used. That is, the encoder can generate and signal a bitstream including information (DC mode selection information) on the basis of which one of the three DC modes the current block is predicted. The method by which the encoder signals the DC mode selection information can be described as an explicit directional DC mode. The decoder can generate a prediction block for the current block based on the mode determined by parsing the DC mode selection information. The encoder can generate and signal a bitstream including information on in which mode the current block was encoded if the prediction mode of the current block is the DC mode. The decoder can determine a final DC mode for the current block by parsing the DC mode selection information if the prediction mode of the current block is the DC mode. For example, if the current block is predicted in DC mode using the average between the left pixels, the encoder can signal that the current block is in DC mode and can signal that the current block is predicted via DC mode using the average between the left pixels using an additional flag bit. If the current block is in DC mode, the decoder can parse the additional flag bit to determine which mode is applied to the current block. The encoder can signal that the coding mode of the current block is DC mode. Alternatively, the decoder can use the MPM list to determine whether the coding mode of the current block is DC mode. Alternatively, the decoder can parse the information (syntax element) indicating whether the coding mode of the current block is DC mode using the MPM list. In this specification, among the three DC modes, the DC mode using the average between the upper pixels of the current block can be described as the Vertical DC mode or the above DC mode, and the DC mode using the average between the left pixels of the current block can be described as all Horizontal DC modes or the left DC mode.

The above three DC prediction modes may not be used in ISP mode. The encoder may not signal a flag bit indicating whether the current block is in DC mode using the average between left pixels or in DC mode using the average between top pixels, if the current block is encoded in ISP mode and the current block is encoded in DC mode. The decoder may not parse a flag bit indicating whether the current block is in DC mode using the average between left pixels or in DC mode using the average between top pixels, if the current block is in ISP mode and the current block is in DC mode.

Meanwhile, the above three DC prediction modes can be used in the ISP mode. The encoder can signal a flag bit indicating which of the three DC prediction modes the prediction mode of the current block is, and include it in the bitstream, if the current block is encoded in the ISP mode and the current block is encoded in the DC mode. The decoder can parse the flag bit to set the final DC mode for the current block, if the current block is encoded in the ISP mode and the current block is encoded in the DC mode.

Whether to apply the above three DC prediction modes can be determined based on the color component of the current block. If the current block is a luminance component block, the above three DC prediction modes can be used. Or, if the current block is a chrominance component block, only the DC prediction mode using the average between the upper and left pixels among the above three DC prediction modes can be used. Therefore, if the current block is a chrominance component block and the DC mode is applied, the encoder may not include a flag bit indicating which of the three DC modes is used in the bitstream. If the current block is a chrominance component block and the DC mode is applied, the decoder may not parse the flag bit indicating which of the three DC modes is used, and may set the DC mode for the current block to the DC prediction mode using the average between the upper and left pixels.

When the ISP mode is applied, the current block can be divided horizontally or vertically. Depending on the horizontal or vertical size of the current block, one block can be divided into two or four blocks. When the three planar modes or the three DC modes are used in a block encoded in the ISP mode, the prediction mode can be determined according to the division shape of the block. When the current block is encoded in the ISP mode and the division shape according to the ISP mode is horizontal division, the horizontal planar mode among the three planar modes can be used. When the current block is encoded in the ISP mode and the division shape according to the ISP mode is horizontal division, the vertical planar mode among the three planar modes can be used. When the current block is encoded in the ISP mode and the division shape according to the ISP mode is horizontal division, a mode using the average between the left pixels among the three DC modes can be used. If the current block is encoded in ISP mode and the division type according to the ISP mode is horizontal division, a mode that uses the average between upper pixels among the three DC modes can be used.

The partitioning shape of the ISP mode can be determined based on the intra prediction directional mode derived from DIMD or the intra prediction directional mode derived from TIMD. If the DIMD (or TIMD) mode of the current block is less than 34, the ISP partitioning shape of the current block can be horizontal partitioning. If the DIMD (or TIMD) mode of the current block is greater than or equal to 34, the ISP partitioning shape of the current block can be vertical partitioning. Conversely, if the DIMD (or TIMD) mode of the current block is less than 34, the ISP partitioning shape of the current block can be vertical partitioning. If the DIMD (or TIMD) mode of the current block is greater than or equal to 34, the ISP partitioning shape of the current block can be horizontal partitioning. If the partition shape of the ISP mode is determined based on the intra prediction directional mode derived from DIMD (or TIMD), the partition shape is determined implicitly, so the encoder may not include information about the ISP partition shape in the bitstream, and the decoder may not parse information about the ISP partition shape.

The video signal processing device can generate prediction blocks for each mode using all three DC modes, and can generate a final prediction block through a weighted average by applying the same or different weights to each prediction block. At this time, if the horizontal size of the current block is greater than the vertical size, the largest weight can be applied to the predicted block using the average between the upper pixels. Also, if the horizontal size of the current block is smaller than the vertical size, the largest weight can be applied to the predicted block using the average between the left pixels. Or, if the intra prediction directional mode of the DIMD derived from the surrounding pixels of the current block is greater than or equal to an arbitrary value, the largest weight can be applied to the predicted block using the average between the upper pixels. If the intra prediction directional mode of the DIMD derived from the surrounding pixels of the current block is smaller than an arbitrary value, the largest weight can be applied to the predicted block using the average between the left pixels. Here, the arbitrary value can be an integer, and can be 34, which is an index of the diagonal mode (angle mode 34 in FIG. 6). The DIMD information derived from the surrounding pixels of the current block may include the first intra prediction directional mode, the second intra prediction directional mode, and information on whether to perform weighted prediction. If weighted prediction is not applied among the DIMD information derived from the surrounding pixels of the current block, the largest weight may be applied to the predicted block using the average of all pixels on the upper and left sides.

If a prediction block is generated based on any one of the three DC prediction modes explicitly signaled by the encoder, the encoder can perform a transform on the error signal (residual signal) using a predetermined transform set, regardless of the multiple transform set. For example, if a prediction block is generated based on any one of the three DC prediction modes explicitly signaled by the encoder, the encoder can perform a transform process based on a Multiple Transform Set (MTS) using a conventional DC mode (or a planar mode) on the error signal to obtain (output) transform coefficients. If a decoder explicitly uses any one of the three DC prediction modes to generate a prediction block of the current block, the decoder can perform an inverse transform on the transform coefficients using a predetermined transform set, regardless of the multiple transform set. For example, if the decoder explicitly uses any one of the three DC prediction modes to generate the prediction block of the current block, the decoder can perform an inverse transform based on MTS using the conventional DC mode (or planar mode) on the transform coefficients and obtain (output) the error signal.

In addition, the encoder can perform a secondary transform on the primary transformed coefficients based on the MTS or a predetermined transform set. The secondary transform can be LFNST. The transform matrix used by the encoder when performing the LFNST can vary depending on the intra prediction directional mode. For example, if the encoder explicitly uses any one of the three DC prediction modes to generate the prediction block of the current block, the primary transformed transform coefficients can be subjected to a secondary transform through LFNST. In this case, the encoder can perform the LFNST transform using the existing DC mode (or planar mode), and obtain (output) the secondary transform coefficients by performing the LFNST transform. Similarly, when the decoder performs the inverse transform using LFNST, the transform matrix used can vary depending on the intra prediction directional mode. For example, if the decoder explicitly uses any one of the three DC prediction modes to generate the prediction block of the current block, the decoder can perform an LFNST inverse transform using the conventional DC mode (or planar mode) on the secondary transform coefficients to obtain (output) the primary transform coefficients. Then, the decoder can perform an inverse transform process based on a multiple transform set (MTS) using the conventional DC mode on the primary transform coefficients to obtain (output) an error signal.

When a video signal processing device explicitly uses one of three DC prediction modes to generate a prediction block of a current block, the video signal processing device may derive a first or second transformation matrix (or a set of matrices, a set of matrices, a set of kernels) using a mode other than the conventional DC prediction mode.

When the encoder generates a prediction block of the current block using the DC prediction mode that uses the average of the left pixel values among the three DC prediction modes, the encoder can perform a first transform process based on MTS by using a set of transform matrices of the horizontal angle mode (e.g., the 18th angle mode in FIG. 6) instead of a set of transform matrices of the existing DC prediction mode for the error signal, and obtain first transform coefficients. In addition, the encoder can perform a transform process by using a set of second transform (e.g., LFNST) matrices of the horizontal angle mode (e.g., the 18th angle mode in FIG. 6) instead of a set of second transform matrices of the existing DC prediction mode for the obtained first transform coefficients, and obtain second transform coefficients. When the decoder generates a prediction block of the current block by using the DC mode that uses the average of the left pixel values among the three DC prediction modes, the decoder can perform an inverse transform process by using a set of second transform matrices of the horizontal angle mode (e.g., the 18th angle mode in FIG. 6) instead of a set of second transform (e.g., LFNST) matrices of the existing DC prediction mode for the second transform coefficients, and obtain a first transform coefficient. In addition, the decoder can perform an inverse transform process based on MTS by using a set of transform matrices of the horizontal angle mode (e.g., the 18th angle mode in FIG. 6) instead of a set of transform matrices of the existing DC prediction mode for the first transform coefficients, and obtain an error signal.

If the prediction block of the current block is generated based on the DC mode that uses the average of the upper pixel values among the three DC prediction modes, the decoder can perform an inverse transformation process using a set of secondary transformation (e.g., LFNST) matrices of the vertical angle mode (e.g., the 50-degree angle mode in FIG. 6) and output a first transformation coefficient. In addition, the decoder can perform an MTS-based inverse transformation process using a set of transformation matrices of the vertical angle mode (e.g., the 50-degree angle mode in FIG. 6) without using a set of transformation matrices of the existing DC prediction mode for the first transformation coefficient, and obtain an error signal.

If the prediction block of the current block is generated using the DC mode that uses the average of the upper pixel values among the three DC prediction modes, the decoder can perform an inverse transformation process using a set of secondary transform (e.g., LFNST) matrices of the horizontal angle mode (e.g., the 18th angle mode in Fig. 6) and output a first transform coefficient. In addition, the decoder can perform an inverse transformation process based on MTS using a set of transform matrices of the horizontal angle mode (e.g., the 18th angle mode in Fig. 6) without using a set of transform matrices of the existing DC prediction mode for the first transform coefficient, and obtain an error signal.

If the prediction block of the current block is generated using the DC mode that uses the average of the upper pixel values among the three DC prediction modes, the video signal processing device can derive the first or second transformation matrix (a set of matrices, a set of matrices, a set of kernels) using the intra prediction directional mode derived using the DIMD method. In other words, if the prediction block of the current block is generated using the DC mode that uses the average of the upper pixel values among the three DC prediction modes, the video signal processing device can perform a transformation process or an inverse transformation process based on MTS using the set of transformation matrices of the intra prediction directional mode derived using the DIMD method. Furthermore, if the prediction block of the current block is generated using the DC mode that uses the average of the upper pixel values among the three DC prediction modes, the video signal processing device can perform a transformation process or an inverse transformation process based on LFNST using the set of transformation matrices of the intra prediction directional mode derived using the DIMD method.

When a prediction block of a current block is generated by explicitly using one of the three DC prediction modes, the video signal processing device may use one of the IDTR and MTS-based transforms for the horizontal or vertical transform.

If the encoder explicitly uses any one of the three DC prediction modes to generate the prediction block of the current block, the encoder can obtain transform coefficients by performing a transform using IDTR in at least one of the horizontal direction and the vertical direction on the error signal, regardless of the multiple transform sets. If the decoder explicitly uses any one of the three DC prediction modes to generate the prediction block of the current block, the decoder can perform an inverse transform using IDTR in at least one of the horizontal direction and the vertical direction on the transform coefficients, regardless of the multiple transform sets, and obtain an error signal.

If the encoder uses the DC mode that uses the average of the upper pixel values to generate the prediction block of the current block, the encoder can apply the IDTR transform for the horizontal direction of the error signal, perform the MTS-based transform using the conventional DC mode for the vertical direction, and obtain the transform coefficients. If the decoder uses the DC mode that uses the average of the upper pixel values to generate the prediction block of the current block, the decoder can apply the IDTR inverse transform for the horizontal direction of the transform coefficients, and perform the MTS-based inverse transform using the conventional DC mode for the vertical direction, and obtain the error signal.

If the encoder generates a prediction block of the current block using the DC mode that uses the average of the upper pixel values, the IDTR transform can be applied to the vertical direction of the error signal, and the MTS-based transform can be performed using the conventional DC mode for the horizontal direction to obtain the transform coefficients. If the decoder generates a prediction block of the current block using the DC mode that uses the average of the upper pixel values, the IDTR inverse transform can be applied to the vertical direction of the transform coefficients, and the MTS-based inverse transform can be performed using the conventional DC mode for the horizontal direction to output the error signal. The IDTR transform can be applied only when the width and/or height of the current block is greater than or equal to a specific value. At this time, the specific value can be 8, 16, or 32.

The video signal processing device can generate the prediction block of the current block through a weighted average between the planar mode and the DC mode. The video signal processing device can generate the prediction block of the current block through a weighted average between the linear planar prediction mode in the vertical direction and the DC mode using the average of the upper pixel values. Alternatively, the video signal processing device can generate the prediction block of the current block through a weighted average between the linear planar prediction mode in the vertical direction and the DC mode using the average of the left pixel values. The video signal processing device can apply the above-described transformation methods to the error signal of the generated prediction block. For example, when the decoder generates the prediction block of the current block through a weighted average between the linear planar prediction mode in the vertical direction and the DC mode using the average of the left pixel values, the decoder can perform an inverse transformation process using a set of secondary transform (e.g., LFNST) matrices of the horizontal angular mode (e.g., the 18th angular mode of FIG. 6) and output primary transform coefficients. In addition, the decoder can perform an inverse transformation process based on MTS by using a set of transformation matrices of a horizontal angle mode (e.g., the 18th angle mode in Fig. 6) instead of a set of transformation matrices of a conventional DC prediction mode for the first transform coefficients, and obtain an error signal. In this specification, the planar mode may be described as a planar prediction mode, and the DC mode may be described as a DC prediction mode.

The DC mode is a prediction method that generates a prediction block for a current block using only one average value, and has the characteristic that all pixels in the prediction block have the same value. In order to increase the prediction efficiency using the DC mode in a block with a gentle change, a video signal processing device can generate a prediction block so that the average value changes according to the vertical or horizontal position in the block, and this is described as a gradient-based DC prediction mode. Referring to FIG. 44, when a prediction block for a current block is generated using the DC mode in the vertical direction, the video signal processing device can calculate an average value of pixels adjacent to the current block upwardly (e.g., hatched pixels in FIG. 444) and then apply it to pixels corresponding to the first horizontal line of the current block (e.g., pixels indicated by 1 in FIG. 44) to generate a prediction sample. Next, the average value can be recalculated according to the gradient between the reference pixel (A) located at the topmost position among the pixels adjacent to the left of the current block and the reference pixel (B) of the next horizontal line. The video signal processing device can generate a prediction sample by applying the recalculated average value to the second horizontal line of the current block (for example, pixels indicated by 2 in FIG. 44). The method of recalculating the average value for the second horizontal line can be calculated through an interpolation method using the distance (L) between the reference pixel (A) and the reference pixel (B), the distance (M) between the reference pixel (A) and the reference pixel (E), and the difference (X) between the pixel values between the reference pixel (A) and the pixel (E) at the position (-1, -1) with respect to the lower left position of the current block. For example, the average value can be recalculated by multiplying the value obtained by dividing L by M by X and adding it to the average value. The same is performed for the next horizontal line, so that a prediction sample for the current block can be generated. That is, the prediction block of the current block can be generated such that the average value for each horizontal (or vertical) line changes according to the gradient of the reference pixels adjacent to the left (or upper) of the current block. The pixel values of the samples in each horizontal (or vertical) direction are configured to be the same, and the prediction block of the current block can be generated such that at least one vertical (or horizontal) pixel value is different. In addition, the line-wise DC prediction mode can be divided into the line-wise DC prediction mode in the vertical direction and the line-wise DC prediction mode in the horizontal direction. If the horizontal size of the current block is larger than the vertical size, since there are many pixels adjacent to the upper side of the current block, the prediction block for the current block can be generated using the line-wise DC prediction mode in the vertical direction. Alternatively, if the horizontal and vertical sizes of the current block are the same, the prediction block for the current block can be generated using the conventional DC prediction mode.

Figure 45 illustrates a plane prediction method in vertical or horizontal line units according to one embodiment of the present invention.

Referring to Figs. 45(a) to (d), when the current block is predicted using the vertical direction planar mode, the position of the reference reference pixel used may vary for each horizontal line. Specifically, the first horizontal line of the current block (pixels indicated as 1 in Fig. 45(a)) can be predicted using pixels adjacent to the current block upwardly (e.g., the hatched pixels in Fig. 45) and the A pixel. The second horizontal line of the current block (pixels indicated as 2 in Fig. 45(a)) can be predicted using pixels adjacent to the current block upwardly (e.g., the hatched pixels in Fig. 45) and the B pixel. In addition, the line-by-line planar prediction mode can be divided into a vertical direction planar mode, a horizontal direction planar mode, and a mode that weights the average of prediction blocks in the vertical and horizontal directions. If the horizontal size of the current block is larger than the vertical size, since the number of pixels adjacent to the current block upwardly is larger, the video signal processing device can generate a prediction block for the current block using the vertical direction planar mode. If the vertical size of the current block is larger than the horizontal size, since there are more pixels adjacent to the left side of the current block, the video signal processing device can generate a prediction block for the current block using the planar mode in the vertical direction. If the horizontal and vertical sizes of the current block are the same, the video signal processing device can generate a prediction block for the current block using the conventional planar mode.

A video signal processing device can divide a current block into sub-blocks of an arbitrary designated size (MxN), and then perform prediction on a sub-block basis using a planar mode. Referring to Figs. 46(a) to (c), positions of reference pixels for prediction (1, 2, 3, 4 in Fig. 46) may vary for each sub-block. Specifically, referring to Fig. 46(a), a current block may be divided into four sub-blocks of MxN size. Positions of reference pixels may be determined based on the upper left position of the current sub-block and/or the horizontal and vertical sizes of the current sub-block. In Fig. 46(a), positions of reference reference pixels of the upper left sub-block may be 1 and 3, positions of reference reference pixels of the upper right sub-block may be 2 and 3, positions of reference reference pixels of the lower left sub-block may be 1 and 4, and positions of reference reference pixels of the lower right sub-block may be 2 and 4. For example, when a prediction block for the upper right sub-block of Fig. 46(a) is generated, if the planar mode is used, the video signal processing device can generate a prediction value for the pixel indicated by X by performing a weighted average of a value generated using the V pixel value and the reference reference pixel 3 and a value generated using the H pixel value and the reference reference pixel 2. Fig. 46(b) shows prediction in the horizontal direction planar mode in units of sub-blocks. For example, when a prediction block for the lower left sub-block of Fig. 46(b) is generated, if the horizontal direction planar mode is used, the video signal processing device can generate a prediction value for the pixel indicated by X by performing a weighted average of a value generated using the H pixel value and the reference reference pixel 1. Fig. 46(c) shows prediction in the vertical direction planar mode in units of sub-blocks. For example, when a prediction block for the upper right sub-block of Fig. 46(c) is generated, if the vertical plane mode is used, the video signal processing device can generate a prediction value for the pixel indicated by X by weighting and averaging the values generated using the V pixel value and the reference reference pixel 3.

A video signal processing device can determine which planar mode among a horizontal planar mode, a vertical planar mode, and a conventional planar mode is applied to each sub-block based on at least one of: a horizontal or vertical size of a current block, a ratio of the horizontal and vertical sizes of the current block, whether the current block is a luminance component block or a chrominance component block, quantization parameter information, intra prediction directional mode information of blocks surrounding the current block, a position of a sub-block, and intra prediction directional mode information derived from pixels surrounding the current block using DIMD.

If the intra prediction directional mode derived from the surrounding pixels of the current block using DIMD is greater than or equal to an arbitrary value, the video signal processing device can generate a prediction block of the sub-block by applying a vertical planar mode. If the intra prediction directional mode derived from the surrounding pixels of the current block using DIMD is less than an arbitrary value, the video signal processing device can generate a prediction block of the sub-block by applying a horizontal planar mode. At this time, the arbitrary value can be an integer and can be 34, which is an index of a diagonal mode (angle mode 34 of FIG. 6). The DIMD information derived from the surrounding pixels of the current block can include a first intra prediction directional mode, a second intra prediction directional mode, and information on whether to perform weighted prediction. If the DIMD information derived from the surrounding pixels of the current block indicates that weighted prediction is not performed, the video signal processing device can generate a prediction block of each sub-block by using a conventional planar mode.

Based on the position of each sub-block, a planar mode used to generate a prediction block of each sub-block can be determined. The video signal processing device can generate a prediction block of an upper-left sub-block using a conventional planar mode, can generate a prediction block of an upper-right sub-block using a vertical planar mode, can generate a prediction block of a lower-left sub-block using a horizontal planar mode, and can generate a prediction block of a lower-right sub-block using a conventional flat mode.

A planar mode prediction method per sub-block can be applied only when a vertical or horizontal planar mode is applied to the current block. In other words, when a conventional planar mode is applied to the current block, a video signal processing device can generate a prediction block of the current block using the conventional planar mode. When a vertical or horizontal planar mode is used for the current block, a video signal processing device can divide the current block into sub-blocks and generate a prediction block of each sub-block using the vertical or horizontal planar mode.

The first horizontal line of the current block (pixels indicated as 1 in Fig. 44(a)) can be predicted using the pixels adjacent to the current block upwardly (e.g., the hatched pixels in Fig. 44) and the A pixel. Similarly, the second horizontal line of the current block (pixels indicated as 2 in Fig. 44(a)) can be predicted using the pixels adjacent to the current block upwardly (e.g., the hatched pixels in Fig. 44) and the B pixel. In addition, the line-by-line planar mode can be divided into a vertical planar mode, a horizontal planar mode, and a weighted-average prediction mode of prediction blocks in the vertical and horizontal directions. If the horizontal size of the current block is larger than the vertical size, since the number of upper pixels adjacent to the current block is larger, the video signal processing device can generate a prediction block for the current block using the vertical planar mode. If the vertical size of the current block is larger than the horizontal size, since there are more pixels adjacent to the left side of the current block, the video signal processing device can generate a prediction block for the current block using the horizontal planar mode. If the horizontal and vertical sizes of the current block are the same, the video signal processing device can generate a prediction block for the current block using the conventional planar mode.

When the prediction block of the current block is generated using the planar mode, the video signal processing device may generate the prediction block using the vertical planar mode, generate the prediction block using the horizontal planar mode, and generate the final prediction block by weighting and averaging the respective prediction blocks. At this time, the weights may be the same or different. If the intra prediction directional mode of the DIMD derived from the surrounding pixels of the current block is greater than or equal to an arbitrary value, the weight of the block predicted using the vertical planar mode may be the largest. The weight of the block predicted using the vertical planar mode may be 3, and the weight of the block predicted using the horizontal planar mode may be 1. If the intra prediction directional mode of the DIMD derived from the surrounding pixels of the current block is less than an arbitrary value, the weight of the block predicted using the horizontal planar mode may be the largest. The weight of the block predicted by the horizontal planar mode may be 3, and the weight of the block predicted by the vertical planar mode may be 1. Here, any value is an integer, and may be 34, which is an index of the diagonal mode (angle mode 34 in FIG. 6). DIMD information derived from surrounding pixels of the current block may include a first intra prediction directional mode, a second intra prediction directional mode, and information on whether to perform weighted prediction. If the DIMD information derived from surrounding pixels of the current block indicates that weighted prediction is not applied, the video signal processing device may apply the same weight to each prediction block. Here, the prediction block generated using the vertical direction planar mode and the prediction block generated using the horizontal direction planar mode may be generated using a sub-block-based planar mode prediction method.

Prediction using the DC mode has a problem of generating discontinuous edges at the boundary between blocks at low bit rates. To improve this problem, a video signal processing device can perform compensation on a prediction block of a current block generated using the DC mode by adding an arbitrary offset value. At this time, information on the arbitrary offset value can be explicitly signaled by being included in a bitstream or can be implicitly derived from neighboring pixel values adjacent to the current block. A decoder can perform compensation on a prediction block of the current block based on an offset value obtained by parsing information on the arbitrary offset value. Alternatively, the decoder can perform compensation on a prediction block of the current block by deriving an offset value from neighboring pixel values adjacent to the current block. Alternatively, a prediction block of the current block can be generated by weighting an average of a block predicted using the planar mode and a block predicted using the DC mode.

The three planar modes or the three DC modes can be derived based on the templates. First, the video signal processing device can construct a reference template including reconstructed neighboring blocks adjacent to the current block (see FIG. 23). Then, the video signal processing device can construct a prediction template for the three planar modes using reference pixels around the reference template. After calculating a cost between the reference template and the prediction template, the video signal processing device can generate a prediction block of the current block using the planar prediction mode indicating the minimum cost.

In addition, the video signal processing device can configure a reference template including reconstructed surrounding blocks adjacent to the current block (see FIG. 23). And the video signal processing device can configure a prediction template for three planar modes using reference pixels around the reference template. The video signal processing device can calculate a cost between the reference template and the prediction template, and then configure a list for planar modes based on the cost. At this time, a planar mode with the smallest cost can be located at the beginning of the list, and the list can be configured in ascending order according to the cost. The encoder can generate and signal a bitstream including index information on an optimal planar mode for generating a prediction block of the current block among the planar modes in the list. The decoder can generate a prediction block of the current block using the planar mode determined by parsing the index information.

The above-described template-based planar mode induction method can be equally applied to the three DC modes.

When a video signal processing device encodes a current block in an intra prediction mode, pixels on the left or upper side of the current block can be effectively predicted because they are adjacent to the surrounding pixels of the restored current block. However, pixels on the right or lower side of the current block have low prediction efficiency because the surrounding pixels have not yet been restored. In other words, the distance from the reference pixel (the surrounding pixel of the restored current block) increases as the pixel is located at the lower right of the current block, so the error signal increases. In order to improve this problem, as shown in FIG. 47, the video signal processing device can restore only one pixel located at the lower right of the current block in advance, and then generate a prediction block of the current block based on the restored lower right pixel and the surrounding pixels adjacent to the current block. At this time, the pixel located at the lower right of the current block can be predicted and encoded from the pixels adjacent to the current block, and information on the difference value, which is the difference between the pixel value located at the lower right of the current block and the predicted value, can be included in the bitstream and signaled. The decoder can reconstruct the pixel located at the lower right of the current block by using the differential value determined by parsing the information about the differential value. Next, the video signal processing device can generate a prediction block of the current block by using the pixels adjacent to the current block and one pixel located at the lower right of the current block. The video signal processing device can predict the right pixels (pixels indicated by 1 in FIG. 47) and lower pixels (pixels indicated by 2 in FIG. 47) of the current block by using the pixels adjacent to the current block and one pixel located at the lower right of the current block, and then generate a prediction block of the current block by using the pixels adjacent to the current block and the lower pixels and right pixels in the current block.

In order to improve coding efficiency, instead of directly coding the aforementioned residual signal, a method of quantizing the transform coefficient values obtained by transforming the residual signal and coding the quantized transform coefficients can be used. As described above, the transform unit can obtain the transform coefficient values by transforming the residual signal. At this time, the residual signal of a specific block can be distributed over the entire area of the current block. Accordingly, the coding efficiency can be improved by concentrating energy in the low-frequency area through frequency domain transformation of the residual signal.

The encoder can obtain at least one residual block containing a residual signal for the current block. The residual block can be either the current block or one of the blocks split from the current block. In this specification, the residual block can be described as a residual array or a residual matrix containing residual samples of the current block. In addition, in this specification, the residual block can represent a block having the same size as the size of a transform unit or a transform block.

The encoder can transform the residual block using a transform kernel. The transform kernel used for transforming the residual block can be a transform kernel having separable characteristics of vertical transform and horizontal transform. In this case, the transform for the residual block can be performed separately as vertical transform and horizontal transform. For example, the encoder can perform vertical transform by applying the transform kernel in the vertical direction of the residual block. In addition, the encoder can perform horizontal transform by applying the transform kernel in the horizontal direction of the residual block. In this specification, the transform kernel may be used as a term referring to a set of parameters used for transforming the residual signal, such as a transform matrix, a transform array, a transform function, and a transform. According to one embodiment, the transform kernel may be any one of a plurality of available kernels. In addition, transform kernels based on different transform types may be used for each of the vertical transform and the horizontal transform. That is, before performing the first transformation, the transformation method for the vertical and horizontal directions can be derived using at least one or more of the intra prediction mode of the current block, the encoding mode, the transformation method parsed from the bitstream, and the size information of the current block. In addition, in order to reduce the computational complexity in the transformation process for a large block, a process of leaving only the low-frequency region and processing the high-frequency region as '0' can be performed. This process is called high-frequency zeroing, and the transformation size at the time of the actual first transformation can be set for this zeroing. In the high-frequency zeroing process, the low-frequency region can be set to an arbitrary fixed size, and for example, the horizontal or vertical sizes can be a combination of 4, 8, 16, 32, etc.

The encoder can quantize the transform block transformed from the residual block by transferring it to the quantization unit. At this time, the transform block can include a plurality of transform coefficients. Specifically, the transform block can be composed of a plurality of transform coefficients arranged in two dimensions. The size of the transform block can be the same as that of the current block or one of the blocks split from the current block, similar to the residual block. The transform coefficients transferred to the quantization unit can be expressed as quantized values.

In addition, the encoder may perform an additional transform before the transform coefficients are quantized. The above-described transform method may be referred to as a primary transform, and the additional transform may be referred to as a secondary transform. The secondary transform may be optional for each residual block. In one embodiment, the encoder may perform the secondary transform for a region where it is difficult to concentrate energy in a low-frequency region with only the primary transform, thereby improving coding efficiency. For example, the secondary transform may be added for a block in which the residual values appear largely in a direction other than the horizontal or vertical direction of the residual block. The residual values of the intra-predicted block may have a higher probability of changing in a direction other than the horizontal or vertical direction compared to the residual values of the inter-predicted block. Accordingly, the encoder may additionally perform the secondary transform on the residual signal of the intra-predicted block. In addition, the encoder may omit the secondary transform on the residual signal of the inter-predicted block. High-frequency zeroing in the first conversion can also be performed in the second conversion process.

As another example, whether to perform a secondary transform may be determined based on the size of the current block or the remaining block. In addition, different sizes of transform kernels may be used based on the size of the current block or the remaining block. For example, an 8X8 secondary transform may be applied to a block whose shorter side among the width or the height is greater than or equal to a first preset length. In addition, a 4X4 secondary transform may be applied to a block whose shorter side among the width or the height is greater than or equal to a second preset length and less than the first preset length. In this case, the first preset length may be a value greater than the second preset length, but the present disclosure is not limited thereto. In addition, unlike the first transform, the secondary transform may not be performed separately into a vertical transform and a horizontal transform. Such a secondary transform may be referred to as a low frequency non-separable transform (LFNST).

In addition, for a video signal of a specific area, even if a frequency transform is performed due to a rapid change in brightness, the high-frequency band energy may not be reduced. Accordingly, the compression performance by quantization may deteriorate. In addition, when a transform is performed on an area where residual values rarely exist, the encoding time and decoding time may unnecessarily increase. Accordingly, the transform for the residual signal of the specific area may be omitted. Whether to perform a transform for the residual signal of the specific area may be determined by a syntax element related to the transform of the specific area. For example, the syntax element may include transform skip information. The transform skip information may be a transform skip flag. If the transform skip information for the residual block indicates a transform skip, the transform for the corresponding residual block is not performed. In this case, the encoder can immediately quantize the residual signal for which the transform of the corresponding area is not performed.

The aforementioned transformation-related syntax elements may be information parsed from a video signal bitstream. A decoder may entropy decode a video signal bitstream to obtain transformation-related syntax elements. In addition, an encoder may entropy code transformation-related syntax elements to generate a video signal bitstream.

The decoder can obtain encoding information necessary for decoding by parsing the transmitted bitstream. At this time, information related to the transformation process includes index information for the first and second transformation types and quantized transformation coefficients. The inverse transformation unit can obtain a residual signal by inversely transforming the inverse quantized transformation coefficients. First, the inverse transformation unit can detect whether inverse transformation is performed for a specific region from a transformation-related syntax element of the specific region. According to one embodiment, if a transformation-related syntax element for a specific transformation block indicates transformation skip, transformation for the corresponding transformation block can be omitted. In this case, both the first inverse transformation and the second inverse transformation can be omitted for the transformation block. In addition, the inverse quantized transformation coefficients can be used as a residual signal. For example, the decoder can restore the current block by using the inverse quantized transformation coefficients as a residual signal. Alternatively, the second inverse transform may be performed and the first inverse transform may be omitted, and the second inverse transformed value may be used as the residual signal. The first inverse transform described above represents an inverse transform for the first transform and may be referred to as an inverse primary transform. The second inverse transform represents an inverse transform for the second transform and may be referred to as an inverse secondary transform or inverse LFNST. In the present invention, the first (inverse) transform may be referred to as a first (inverse) transform, and the second (inverse) transform may be referred to as a second (inverse) transform.

Fig. 48 shows formulas of DCT-II, DCT-V (discrete cosine transform type-V), DCT-VIII (discrete cosine transform type-VIII), DST-I (discrete sine transform type-I), and DST-VII kernels applied to MTS. DCT and DST can be expressed as functions of cosine and sine, respectively, and when the basis function of the transform kernel for the number of samples N is expressed as Ti(j), the index i represents an index in the frequency domain, and the index j represents an index within the basis function. That is, as i becomes smaller, it represents a low-frequency basis function, and as i becomes larger, it represents a high-frequency basis function. When the basis function Ti(j) is expressed as a two-dimensional matrix, it can represent the j-th element of the i-th row, and since the transform kernels illustrated in Fig. 48 all have a separable characteristic, they can perform transformations in the horizontal and vertical directions respectively for the residual signal X. That is, when the residual signal block is X and the transform kernel matrix is T, the transform for the residual signal X can be expressed as TXT'. Here, T' means the transpose of the transform kernel matrix T. Since DCT and DST are in decimal form, not integer form, it is burdensome to implement them as they are in the hardware encoder and decoder. Therefore, the decimal form transform kernel must be approximated to an integer form transform kernel through scaling and rounding. The integer precision of the transform kernel can be determined as 8-bit or 10-bit, but if the precision is low, the encoding efficiency may decrease. Depending on the approximation, the orthonormal property of DCT and DST may not be maintained, but since the encoding efficiency loss due to this is not large, approximating the transform kernel to an integer form is advantageous in terms of implementing the hardware encoder and decoder. IDTR (Identity Transform) is a transform in which the result of the transformation is the same as before the transformation, and is called the identity transform. In general, identity transformation constructs a transformation matrix by setting '1' at positions where rows and columns have the same value. However, here, identity transformation uses an arbitrary fixed value, not the value '1', to increase or decrease the value of the input residual signal equally.

A bitstream consists of one or more coded video sequences (CVSs), and a CVS is encoded independently of other CVSs. Each CVS consists of one or more layers, and each layer can represent a specific quality, a specific resolution, or a general image, a depth map, or a transparency map. In addition, a coded layer video sequence (CLVS) means a layer-wise CVS composed of consecutive (in decoding order) PUs within the same layer. For example, there may be a CLVS for a specific quality layer, and there may be a CLVS for a depth map.

Figures 49 and 50 illustrate a portion of a sequence parameter set according to one embodiment of the present invention.

Figure 49 illustrates how to signal in SPS a syntax element indicating whether vertical plane mode and horizontal plane mode are enabled. Also, Figure 45 illustrates how to signal in SPS whether the method of explicitly signaling three DC prediction modes is enabled.

When the value of sps_directional_planar_enabled_flag in Fig. 49 is 1 (true), it can indicate that the vertical planar mode and the horizontal planar mode are enabled. When the value of sps_directional_planar_enabled_flag is 0 (false), it can indicate that the vertical planar mode and the horizontal planar mode are disabled. That is, the vertical planar mode and the horizontal planar mode are not used for all blocks. When sps_directional_planar_enabled_flag is not parsed (not included in the bitstream), the value of sps_directional_planar_enabled_flag can be inferred as 0.

sps_directional_planar_enabled_flag equal to 1 specifies that the vertical planar and the horizontal planar mode is enabled for the CLVS. sps_directional_planar_enabled_flag equal to 0 specifies that the vertical planar and the horizontal planar mode is disabled for the CLVS. When sps_directional_planar_enabled_flag is not present, it is inferred to be equal to 0.

Similar to the method of signaling in the SPS described above, syntax elements indicating whether the vertical planar mode and the horizontal planar mode are enabled can also be signaled in the PPS. That is, whether the vertical planar mode and the horizontal planar mode are enabled can vary for each picture and/or frame depending on the syntax elements signaled in the PPS (e.g., pps_directional_planar_enabled_flag).

When the value of sps_directional_DC_enabled_flag in FIG. 49 is 1 (true), it can indicate that a method of explicitly signaling the three DC prediction modes is enabled. When the value of sps_directional_DC_enabled_flag is 0 (false), it can indicate that a method of explicitly signaling the three DC prediction modes is not enabled. When sps_directional_DC_enabled_flag is not parsed (not included in the bitstream), the value of sps_directional_DC_enabled_flag can be inferred to be 0.

sps_directional_DC_enabled_flag equal to 1 specifies that the explicit vertical DC and the explicit horizontal DC mode is enabled for the CLVS. sps_directional_DC_enabled_flag equal to 0 specifies that the explicit vertical DC and the explicit horizontal DC mode is disabled for the CLVS. When sps_directional_DC_enabled_flag is not present, it is inferred to be equal to 0.

FIG. 50 illustrates a method of signaling in an SPS a syntax element indicating whether a method of generating a prediction block using multiple reference pixel lines described through FIGS. 25 to 29 is activated. The method of generating a prediction block using multiple reference pixel lines may be described as intra fusion.

If the value of sps_intra_fusion_enabled_flag in FIG. 50 is 1 (true), it can indicate that intra fusion is enabled. If the value of sps_intra_fusion_enabled_flag is 0 (false), it can indicate that intra fusion is disabled. That is, intra fusion is not performed on any blocks. If sps_intra_fusion_enabled_flag is not parsed (not included in the bitstream), the value of sps_intra_fusion_enabled_flag can be inferred as 0.

sps_intra_fusion_enabled_flag equal to 1 specifies that the intra fusion method is enabled for the CLVS. sps_intra_fusion_enabled_flag equal to 0 specifies that intra fusion method is disabled for the CLVS. When sps_intra_fusion_enabled_flag is not present, it is inferred to be equal to 0.

Similar to the signaling method in the SPS described above, a syntax element indicating whether intra fusion is enabled can also be signaled in the PPS. That is, whether intra fusion is enabled can vary for each picture and/or frame depending on the syntax element signaled in the PPS (e.g., pps_intra_fusion_enabled_flag).

FIG. 51 shows constraint flags related to vertical plane mode, horizontal plane mode, intra fusion, and explicit directional DC mode. The general_constraint_info() syntax of FIG. 51 can be called from the profile_tier_level() syntax. The profile_tier_level() syntax can be called from the sequence parameter set RBSP syntax, the video parameter set RBSP syntax, and the decoding capability information RBSP syntax. Individual syntax elements of the general_constraint_info() syntax may have corresponding syntax elements in the sequence parameter set RBSP, and whether the corresponding sequence parameter set RBSP syntax element is activated/deactivated may be constrained by the definition of the corresponding flag.

Constraint flags related to vertical planar mode and horizontal planar mode can be gci_no_directional_planar_constraint_flag.

If the value of gci_no_directional_planar_constraint_flag is equal to 1, the value of sps_directional_planar_enabled_flag is constrained to 0 for all pictures in OlsScope. That is, disabling of vertical planar mode and horizontal planar mode can be constrained (forced). If the value of gci_no_directional_planar_constraint_flag is equal to 0, the value of sps_directional_planar_enabled_flag is not constrained.

gci_no_directional_planar_constraint_flag equal to 1 specifies that sps_directional_planar_enabled_flag for all pictures in OlsInScope shall be equal to 0. gci_no_directional_planar_constraint_flag equal to 0 does not impose such a constraint.

The constraint flag associated with explicit directional DC mode can be gci_no_directional_DC_constraint_flag.

If the value of gci_no_directional_DC_constraint_flag is equal to 1, the value of sps_directional_DC_enabled_flag is constrained to 0 for all pictures in OlsScope. That is, the disabling of explicit directional DC mode can be constrained (forced). If the value of gci_no_directional_DC_constraint_flag is equal to 0, the value of sps_directional_DC_enabled_flag is not constrained.

gci_no_directional_DC_constraint_flag equal to 1 specifies that sps_directional_DC_enabled_flag for all pictures in OlsInScope shall be equal to 0. gci_no_directional_DC_constraint_flag equal to 0 does not impose such a constraint.

The constraint flag related to intra fusion can be gci_no_intra_fusion_constraint_flag.

If the value of gci_no_intra_fusion_constraint_flag is equal to 1, the value of sps_intra_fusion_enabled_flag is constrained to 0 for all pictures in OlsScope. That is, disabling of intra fusion can be constrained (forced). If the value of gci_no_intra_fusion_constraint_flag is equal to 0, the value of sps_intra_fusion_enabled_flag is not constrained to 0.

gci_no_intra_fusion_constraint_flag equal to 1 specifies that sps_intra_fusion_enabled_flag for all pictures in OlsInScope shall be equal to 0. gci_no_intra_fusion_constraint_flag equal to 0 does not impose such a constraint.

When a video signal processing device generates a prediction block for a current block, it can use a multi-prediction mode that weights and averages two or more intra prediction blocks. When the DIMD or TIMD mode is used, the two intra prediction directional modes can be implicitly determined from a neighboring block or an MPM list. In a block that is not in the DIMD or TIMD mode, the two intra prediction directional modes can be signaled from the bitstream, but this may increase the bit amount and lower the compression efficiency.

To mitigate this increase in bit volume, when a video signal processing device configures an MPM list, the intra prediction directional mode to be included in each item of the list may be set to two or more instead of one.

In Fig. 52(a), A, B, C, …, F, G can represent any intra prediction directional mode, one of MIP, Intra TMP, IBC, DIMD, TIMD, and “N/A” indicates that it is not used, and the 4th and 5th in the MPM list can be configured with only one intra prediction directional mode, not a multi-prediction mode. As in Fig. 52(b), a single prediction mode can be added first to the MPM list and then a multi-prediction mode can be added next, and “N/A” indicates that it is not used, and the 1st and 2nd in the MPM list can be configured with only one intra prediction directional mode, not a multi-prediction mode.

An encoder can adaptively signal, on a block-by-block basis, in the bitstream information on whether to use an MPM list consisting of only one intra-prediction directional mode (a single-prediction-based MPM list) or a multi-prediction-based MPM list. A decoder can parse the information on whether to use an MPM list consisting of only one intra-prediction directional mode (a single-prediction-based MPM list) or a multi-prediction-based MPM list to determine which MPM list is used to generate a prediction block for the current block.

The video signal processing device can determine whether to use a multi-prediction-based MPM list by using at least one of the horizontal or vertical size of the current block, the ratio of the horizontal and vertical sizes of the current block, whether the current block is a luminance block or a chrominance block, quantization parameter information, intra prediction directional mode information of blocks surrounding the current block, and encoding mode information of the current block.

Depending on how the multi-prediction-based MPM list is constructed, the encoding efficiency can vary, and various methods can be applied, as follows.

The video signal processing device can first configure a single prediction-based MPM list, and then configure a multi-prediction-based MPM list using the single prediction-based MPM list. For example, the video signal processing device can configure a multi-prediction-based MPM list by combining each intra-prediction directional mode in the single prediction-based MPM list. At this time, the video signal processing device can reorder the single prediction-based MPM list using a template cost-based reordering method used in the TIMD method. The video signal processing device can configure the multi-prediction-based MPM list only by using a combination of a first intra-prediction directional mode and intra-prediction directional modes other than the first of the reordered single prediction-based MPM list.

The video signal processing device can construct a multi-prediction-based MPM list by using combinations of intra-prediction directional modes derived from DIMD and single-prediction-based MPM lists. For example, the video signal processing device can combine a first intra-prediction directional mode derived from DIMD with a first intra-prediction directional mode within a single-prediction-based MPM list and add it to a first of the multi-prediction-based MPM list. Further, the video signal processing device can combine a second intra-prediction directional mode derived from DIMD with the first intra-prediction directional mode within a single-prediction-based MPM list and add it to a second of the multi-prediction-based MPM list. Further, the video signal processing device can combine a first intra-prediction directional mode derived from DIMD with a second intra-prediction directional mode within a single-prediction-based MPM list and add it to a third of the multi-prediction-based MPM list. In this way, the video signal processing device can construct a multi-prediction-based MPM list by using all possible combinations.

The video signal processing device can configure a multi-prediction-based MPM list by using multiple predefined multi-prediction modes. Here, the multiple predefined multi-prediction modes can be (18, 0), (50, 0), (34, 0), (2, 0), (66, 0), (18, 1), (50, 1), (34, 1), (2, 1), (66, 1), etc., and X and Y in "(X, Y)" can be the intra prediction directional mode (indices) of FIG. 6.

A video signal processing device can reorder a multi-prediction-based MPM list using a TIMD-based reordering method. The video signal processing device can reorder a multi-prediction-based MPM list using only a few low-cost items.

When a video signal processing device generates a prediction block using multiple prediction modes, the intra prediction mode used to generate each prediction block can be signaled based on the Primary MPM and the Secondary MPM. The Primary MPM can include six intra prediction directional modes, and the Secondary MPM can include 16 intra prediction directional modes. The Secondary MPM can be configured using several predefined intra prediction modes. For example, if the prediction modes of DC, 50, 18, 46, 54, 14, 22, 42, 58, 10, 26, 38, 62, 6, 30, 34, 66, 2, 48, 52, 16 (see the intra prediction mode of FIG. 6) do not exist in the Primary MPM, the video signal processing device can configure the list of Secondary MPMs by sequentially adding the prediction modes of DC, 50, 18, 46, 54, 14, 22, 42, 58, 10, 26, 38, 62, 6, 30, 34, 66, 2, 48, 52, 16 until the list of Secondary MPMs is completely configured. A video signal processing device can determine an intra prediction directional mode used for generating a prediction block of the current block by deriving one intra prediction directional mode from a Primary MPM and another intra prediction directional mode from a Secondary MPM, when multiple prediction modes are used for the current block. When multiple prediction modes are used for the current block, an encoder can signal which intra prediction directional mode is used in the Primary MPM and the Secondary MPM by including index information in a bitstream. When multiple prediction modes are used for the current block, a decoder can determine the intra prediction directional modes used for generating the prediction blocks for the current block, within a Primary MPM and a Secondary MPM list, after parsing the index information.

When multiple prediction modes are used for the current block, the video signal processing device can determine intra prediction directional modes used to generate prediction blocks of the current block using the intra prediction directional mode derived from the DIMD, the Primary MPM, and the Secondary MPM. When multiple prediction modes are used for the current block, the encoder can signal by including information on whether the intra prediction directional mode derived from the DIMD is used and index information indicating which intra prediction directional mode is used within the Primary MPM and the Secondary MPM in a bitstream. When multiple prediction modes are used for the current block, the decoder can determine intra prediction directional modes used to generate prediction blocks for the current block by parsing information on whether the intra prediction directional mode derived from the DIMD is used and index information indicating which intra prediction directional mode is used within the Primary MPM and the Secondary MPM.

If a multi-prediction mode is used in the current block, the video signal processing device can configure a secondary MPM list as a multi-prediction-based MPM list using the primary MPM list. For example, if the primary MPM modes generated from the surrounding blocks are Planar, 18, 54, 30, 45, 17, modes such as Planar+18, Planard+54, etc. can be added to the secondary MPM list to configure a multi-prediction-based MPM list. If a multi-prediction mode is used in the current block, the encoder can configure a secondary MPM list as a multi-prediction-based MPM list using the primary MPM list, and then determine a multi-prediction mode used to generate a prediction block of the current block from the secondary MPM list, and signal it by including index information for the determined secondary MPM list in the bitstream. The decoder can parse the index information to determine a multi-prediction mode used to generate a prediction block of the current block from the secondary MPM list.

In addition, when a multi-prediction mode is used for the current block, the video signal processing device can configure a secondary MPM list as a multi-prediction-based MPM list by using the intra-prediction directional mode derived from DIMD and the primary MPM list. For example, when the primary MPM modes generated from the surrounding blocks are Planar, 18, 54, 30, 45, 17, the list of secondary MPMs can be configured as a multi-prediction-based MPM list by adding modes such as Planar+18, Planard+54, etc. When a multi-prediction mode is used for the current block, the encoder can configure a secondary MPM list as a multi-prediction-based MPM list by using the intra-prediction directional mode derived from DIMD and the primary MPM list. The encoder can determine a multi-prediction mode used for generating a prediction block of the current block from the list of secondary MPMs, and can signal index information for the determined list of secondary MPMs by including it in a bitstream. The decoder can determine the multiple prediction modes to be used to generate the prediction block of the current block from the list of Secondary MPMs by parsing the index information.

When a multi-prediction mode is used to generate a prediction block, the video signal processing device can determine the positions of reference pixels used to generate each prediction block, respectively. That is, the video signal processing device can designate a left reference pixel line and an upper reference pixel line used to generate each prediction block, respectively, from among various reference pixel lines (see FIG. 31). After the encoder determines an optimal reference pixel line, the encoder can include position information of the left reference pixel line and the upper reference pixel line in a bitstream and signal them. The decoder can parse each position information to determine the positions of the left reference pixel line and the upper reference pixel line used to generate each prediction block, respectively.

Referring to FIG. 53, when a prediction block is generated by using the first intra prediction directional mode among the multiple prediction modes, the video signal processing device can generate the prediction block by using only the left reference pixel of the current block. In addition, when a prediction block is generated by using the second intra prediction directional mode among the multiple prediction modes, the video signal processing device can generate the prediction block by using only the upper reference pixel of the current block. The first intra prediction directional mode can only use modes with numbers less than 34 among the intra prediction directional modes of FIG. 6, and the second intra prediction directional mode can only use modes with numbers greater than or equal to 34 among the intra prediction directional modes of FIG. 6. Accordingly, when a multiple prediction-based MPM list is configured, the first intra prediction directional mode can only use modes with numbers less than 34, and the second intra prediction directional mode can only use modes with numbers greater than or equal to 34.

Referring to FIG. 53, the video signal processing device can designate a left reference pixel line and an upper reference pixel line used for generating each prediction block, respectively. After the encoder determines the optimal reference pixel line, the encoder can include the position information of the left reference pixel line and the position information of the upper reference pixel line in the bitstream and signal them. The decoder can parse each position information to determine the positions of the left reference pixel line and the upper reference pixel line used for generating each prediction block. In addition, the position information of the upper reference pixel line can be derived from the position information of the left reference pixel line, and the position information of the upper reference pixel line and the position information of the left reference pixel line can be the same. When signaling the position information of the upper reference pixel line, the encoder can include only the difference value with the position information of the left reference pixel line in the bitstream and signal it. The decoder can parse the position information of the left reference pixel line, parse the difference value with respect to the position information of the upper reference pixel line, and then determine the position information of the upper reference pixel line through the difference value with respect to the position information of the left reference pixel line and the position information of the upper reference pixel line.

When a secondary transform is applied to a current block with multiple prediction modes, a transform set for the secondary transform can be determined based on two intra prediction directional modes. At this time, one of the two intra prediction directional modes can be used to select the primary transform or the secondary transform set, and the first intra prediction directional mode can be used. The encoder can signal information about which of the two intra prediction directional modes is to be used to derive the primary transform or the secondary transform set by including it in the bitstream. The decoder can parse the information about which of the two intra prediction directional modes is to be used to derive the primary transform or the secondary transform set, and determine the intra prediction directional mode used to derive the primary inverse transform or the secondary inverse transform set for the current error block. Alternatively, when multiple prediction modes are applied to the current block, the video signal processing device may determine the intra prediction directional mode used to derive the first transform (inverse transform) or second transform (inverse transform) set as a predefined mode. Here, the predefined mode may be a Planar mode, a DC mode, etc.

The video signal processing device may not perform PDPC on a prediction block to which the multi-prediction mode is applied. The video signal processing device may perform PDPC on a prediction block to which the multi-prediction mode is applied based on a first intra-prediction directional mode among two intra-prediction directional modes. The video signal processing device may perform a first PDPC on a prediction block to which the multi-prediction mode is applied based on the first intra-prediction directional mode among two intra-prediction directional modes, and then perform a second PDPC on a prediction block to which the multi-prediction mode is applied based on the second intra-prediction directional mode.

A video signal processing device can signal DC selection information indicating which DC prediction mode among three DC prediction modes is used by integrating it with plane selection information. That is, DC prediction modes in three directions can be explicitly signaled. An encoder can signal the plane selection information by including it in a bitstream only under certain conditions, and a decoder can set an intra prediction directional mode for a current block by parsing the plane selection information only under certain conditions, and can generate a prediction block for the current block using the set intra prediction directional mode. Here, the certain conditions can be conditions related to the horizontal and vertical sizes of the current block, the ratio of the horizontal and vertical sizes of the current block, whether the horizontal and vertical sizes of the current block are the same, whether the intra prediction directional mode of the current block is a certain mode (e.g., whether it is a planar mode, DC, HOR, VER, etc.), whether the coding mode of the current block is a DIMD, TIMD, IntraTMP, IBC, ISP, or MIP coding mode, and conditions related to index information of a reference pixel line used when generating a prediction block for the current block. A video signal processing device can determine whether to encode and decode at least one of the plane selection information and the DC selection information in various ways by using at least one of specific conditions as follows. 1) Only when the intra prediction directional mode of the current block is a planar mode (an existing planar prediction mode, in which the index of the intra prediction directional mode is '0'), the encoder can additionally include information about at least one of the plane selection information and the DC selection information in a bitstream and signal it, and the decoder can additionally parse information about at least one of the plane selection information and the DC selection information to set the intra prediction directional mode for the current block. 2) Only if the horizontal and vertical sizes of the current block are equal to or smaller than the maximum transform block size, and the product of the horizontal and vertical sizes of the current block is greater than the product of the minimum transform block size and the minimum transform block size, the encoder may additionally signal at least one of the plane selection information and the DC selection information in the bitstream, and the decoder may additionally parse the information about at least one of the plane selection information and the DC selection information to set the intra prediction directional mode for the current block. Here, the size of the minimum transform block may be an integer, such as 4 or 8. Here, the size of the maximum transform block may be an integer, such as 64, 128, 256, and so on. 3) Only if the horizontal and vertical sizes of the current block are different, the encoder may additionally signal at least one of the plane selection information and the DC selection information in the bitstream, and the decoder may additionally parse the information about at least one of the plane selection information and the DC selection information to set the intra prediction directional mode for the current block.

b in Fig. 54 is a variable that is initially set to 0, and 'decodeBinX()' can be a function that outputs one bin value after entropy decoding the bitstream using CABAC. 'decodeBinX()' can output a value of 0 or 1.

The decoder can perform the parsing process described in Fig. 50 only when the intra prediction directional mode of the current block is the planar mode (the existing planar prediction mode, when the index of the intra prediction directional mode is '0'). plIdx in Fig. 54 is a variable indicating the planar prediction mode, and dcIdx may be a variable indicating the DC prediction mode. If the value of plIdx is 0, it may indicate the existing planar prediction mode. If the value of plIdx is 1, it may indicate a horizontal linear planar prediction mode (Horizontal Planar), and if the value of plIdx is 2, it may indicate a vertical linear planar prediction mode (Vertical Planar). If the value of dcIdx is 0, it may indicate the existing DC mode (a mode selected from among the DC mode using the average of the left pixel values according to the horizontal and vertical sizes of the current block, the DC mode using the average of the upper pixel values, and the DC mode using the average of the left and upper pixel values). If the value of dcIdx is 1, it can indicate the DC mode (Horizontal DC mode or left DC mode) that uses the average of the left pixel values. If the value of dcIdx is 2, it can indicate the DC mode (Vertical DC mode or above DC mode) that uses the average of the upper pixel values. Here, the existing DC mode indicated when the value of dcIdx is 0 can be signaled using the existing method via the MPM list.

Referring to FIG. 54, a parsing method for the added DC modes, Horizontal DC mode and Vertical DC mode, is described. The video signal processing device can repeatedly use CABAC, perform entropy decoding (decodeBin()), add the obtained (output) value to the temporary variable 'b', and then set the intra prediction mode for the current block according to the value of 'b'. If the value of 'b' decoded by decodeBin1() for the first time is 0, the video signal processing device can set the intra prediction mode for the current block to the existing planar prediction mode. If the value of the decoded 'b' is greater than 0, the decoder adds the value decoded by the next decodeBin2() to 'b', and if the value of 'b' is 1, the intra prediction mode for the current block can be set to the horizontal planar linear prediction mode (Horizontal Planar). The video signal processing device can set the intra prediction mode for the current block to a vertical planar linear prediction mode (Vertical Planar) after adding the value decoded by decodeBin3() to 'b' if the value of the decoded 'b' is greater than 1, and if the value of 'b' is 2, the video signal processing device can set the intra prediction mode for the current block to a DC mode (Horizontal DC, dcIdx set to '1') using the average of the left pixel values if the value of 'b' is greater than 2, and if the value of 'b' is added to 'b' if the value of b is '3'. The video signal processing device can set the intra prediction mode for the current block to a DC mode (Vertical DC, dcIdx set to '2') using the average of the upper pixel values if the value of 'b' is greater than 3.

The video signal processing device can set the value of dcIdx for the DC mode (Vertical DC) that uses the average of the upper pixel values to 1, and can set the value of dcIdx for the DC mode (Horizontal DC) that uses the average of the left pixel values to 2. If the dcIdx value is changed, the method of setting the intra prediction mode by the parsing process of Fig. 54 can also be changed as follows. For example, the video signal processing device can set the intra prediction mode for the current block to the DC mode (Vertical DC, dcIdx set to '1') that uses the average of the upper pixel values after adding the value decoded by the next decodeBin4() to 'b' if the value of 'b' is 3. The video signal processing device can set the intra prediction mode for the current block to the DC mode (Horizontal DC, dcIdx set to '2') that uses the average of the left pixel values when the value of 'b' is greater than 3.

The probability models used in the entropy decoding (decodeBin1(), decodeBin2(), decodeBin3(), decodeBin4()) of Fig. 54 may be different from each other. Alternatively, entropy decoding for decodeBin1(), decodeBin2(), decodeBin3(), decodeBin4() may be performed using a single probability model.

Binarization and bin string can be used interchangeably, and can be used to convert each syntax into a binary code in the entropy coding and decoding process of CABAC. The binarization method can be different for each syntax. A variable length method or a fixed length method can be used to generate a binarized bin string. Depending on the distribution of the probability that an arbitrary symbol of the syntax occurs, the variable length method or the fixed length method can be selected and used. If the probability that an arbitrary symbol of the syntax occurs is the same as the probability that another symbol occurs, the fixed length method can be used. If the probability that an arbitrary symbol of the syntax occurs is significantly higher than the probability that another symbol occurs, the variable length method can be used. Fig. 55(a) shows an bin string binarized by a variable length method, and Fig. 55(b) shows an bin string binarized by a fixed length method. The binarized bitstream for the planar mode selection information and DC mode selection information used in the method of Fig. 54 may be that used in Fig. 55(a).

Referring to FIG. 56, if the value of dcIdx, which is a variable (information) indicating a DC prediction mode, is 0, the video signal processing device can perform the existing DC prediction method. If the vertical and horizontal sizes of the current block are the same, the video signal processing device can generate a prediction block for the current block using the average value of all reference pixels on the top and left sides of the current block. If the horizontal size of the current block is larger than the vertical size, the video signal processing device can generate a prediction block for the current block using the average value of the upper reference pixels of the current block. If the vertical size of the current block is larger than the horizontal size, the video signal processing device can generate a prediction block for the current block using the average value of the left reference pixels of the current block.

A method for generating a prediction block for a current block when the value of dcIdx is not 0 is as follows (5610). When the value of dcIdx is 1, the video signal processing device can generate a prediction block for the current block using an average value of left reference pixels of the current block. When the value of dcIdx is 2, the video signal processing device can generate a prediction block for the current block using an average value of upper reference pixels of the current block.

An encoder can generate and signal a bitstream including DC mode selection information only when the width and height of the current block are the same. A decoder can parse the DC mode selection information included in the bitstream only when the width and height of the current block are the same, and set an intra prediction mode for the current block based on the parsing result.

Referring to Fig. 57(a), the decoder can parse the planar mode selection information and the DC mode selection information based on at least one of the 'b' values and the result of comparing the horizontal size and the vertical size of the current block (e.g., whether they are the same). Referring to Fig. 57(b), the decoder can set the intra prediction mode for the current block using the 'b' value obtained through the process of Fig. 57(a). For example, if the 'b' value is 0, the value of plIdx may be set to 0 (i.e., the existing planar prediction mode), if the 'b' value is 1, the value of plIdx may be set to 1 (i.e., the linear planar prediction mode in the horizontal direction), and if the 'b' value is 2, the value of plIdx may be set to 2 (i.e., the linear planar prediction mode in the vertical direction).

Referring to FIG. 58, if the vertical and horizontal sizes of the current block are the same, the video signal processing device can additionally check DC mode selection information (dcIdx) (5810). If the value of dcIdx is 0, the video signal processing device can generate a prediction block for the current block using the average value of all reference pixels on the top and left of the current block. If the value of dcIdx is 1, the video signal processing device can generate a prediction block for the current block using the average value of the reference pixels on the left of the current block. If the value of dcIdx is 2, the video signal processing device can generate a prediction block for the current block using the average value of the reference pixels on the top of the current block.

If the horizontal size of the current block is greater than the vertical size, the video signal processing device can generate a prediction block for the current block using the average value of the reference pixels at the top of the current block. If the vertical size of the current block is greater than the horizontal size, the video signal processing device can generate a prediction block for the current block using the average value of the reference pixels at the left side of the current block.

Since the encoder signals DC mode selection information together with planar mode selection information, and the decoder parses the planar mode selection information and DC mode selection information together, the planar mode selection information and DC mode selection information can be integrated and described as a unified directional mode in this specification.

Referring to FIG. 59, the video signal processing device can generate a prediction block for the current block by using the average value of the upper reference pixels of the current block when the integrated direction mode is the DC mode (Above DC mode) or the Vertical DC mode that uses the average of the reference pixel values at the top of the current block. The video signal processing device can generate a prediction block for the current block by using the average value of the left reference pixels of the current block when the integrated direction mode is the DC mode (Left DC mode) or the Horizontal DC mode that uses the average of the reference pixel values at the left of the current block (5910).

If the unified direction mode is neither the Above DC mode nor the Vertical DC mode nor the Left DC mode nor the Horizontal DC mode, the video signal processing device can generate the prediction block for the current block in the conventional DC mode. At this time, if the vertical and horizontal sizes of the current block are the same, the video signal processing device can generate the prediction block for the current block using the average value of all reference pixels on the top and the left side of the current block. If the horizontal size of the current block is larger than the vertical size, the video signal processing device can generate the prediction block for the current block using the average value of the upper reference pixels of the current block. If the vertical size of the current block is larger than the horizontal size, the video signal processing device can generate the prediction block for the current block using the average value of the reference pixels on the left side of the current block.

Fig. 60(a) is similar to Fig. 55(a) described above, except that the Horizontal DC mode of Fig. 55(a) is changed to Above DC mode, and the Vertical DC mode of Fig. 55(a) is changed to Left DC mode to represent a binarized empty string.

If the width of the current block is greater than its height, the Above DC mode can be implicitly used for generating the prediction block of the current block. In this case, the encoder may not signal which mode the current block uses among the Above DC mode and the Left DC mode. The encoder may only signal information about whether the explicit DC mode is applied when the width of the current block is greater than its height. The decoder parses the information about whether the explicit DC mode is applied when the width of the current block is greater than its height, and if the explicit DC mode is applied, it may use the Left DC mode among the Above DC mode and the Left DC mode.

If the horizontal and vertical lengths of the current block are the same, the encoder can apply both signaling methods for the Above DC and Left DC modes, as shown in Fig. 60(a). If the horizontal and vertical lengths of the current block are the same, the decoder can parse the integrated direction mode, as shown in Fig. 60(a), and set which mode among the Above DC and Left DC modes is applied to the current block.

Specifically, FIG. 61 illustrates a prediction method that is enabled by the Above DC mode and the Left DC mode. Referring to FIG. 61, if the horizontal length and the vertical length of the current block are the same, the video signal processing device can generate a prediction block for the current block using the average value of the reference pixels at the top of the current block (6110). Alternatively, if the horizontal length and the vertical length of the current block are the same, the video signal processing device can generate a prediction block for the current block using the average value of the reference pixels at the left side of the current block (6120). In addition, if the horizontal length of the current block is greater than the vertical length, the video signal processing device can generate a prediction block for the current block using the average value of the pixels at the left side of the current block (6130). In addition, if the horizontal length of the current block is less than the vertical length, the video signal processing device can generate a prediction block for the current block using the average value of the pixels at the top of the current block (6140).

The encoder can signal the new DC mode (Above DC mode, Left DC mode) as a sub-mode of the planar mode. The decoder can also parse the new DC mode as a sub-mode of the planar mode. When the new DC mode is applied to the current block, the prediction mode for the prediction block for the current block can be set to the planar mode. However, when the video signal processing device generates the prediction block for the current block, if the current block is in the new DC mode, the prediction mode for the prediction block can be the new DC mode. In addition, when the video signal processing device derives the transform kernel, if the new DC mode is applied to the current block, the video signal processing device can change the prediction mode to the DC mode when deriving the transform kernel. When the prediction mode is the Above DC mode, the video signal processing device can use the vertical transform kernel derivation method, and when the prediction mode is the Left DC mode, the video signal processing device can use the horizontal transform kernel derivation method. When the prediction mode is the Above DC mode, the video signal processing device can use the horizontal transform kernel derivation method. When the prediction mode is Left DC mode, the video signal processing device can use a vertical direction transform kernel derivation method. The transform kernel can be a first-order transform kernel and a second-order transform kernel.

The video signal processing device may obtain filtered reference samples by performing filtering on surrounding samples of the current block and/or reference samples obtained by the reference sample padding process when a new DC mode is applied to the current block. Then, the video signal processing device may predict samples of the current block using the obtained reference samples. Alternatively, filtering on the reference samples may be performed only when the current block is larger than a predetermined size. Alternatively, filtering on the reference samples may be performed only when the current block is smaller than a predetermined size. The predetermined size may be expressed as the product of the width and height of the current block, and the predetermined size may be, for example, 32. The video signal processing device may predict samples of the current block using reference samples obtained by the surrounding samples of the current block and/or reference samples obtained by the reference sample padding process when a new DC mode is applied to the current block. The surrounding samples may include samples on at least one reference line. For example, the surrounding samples may include adjacent samples on a line adjacent to a boundary of the current block.

When the video signal processing device derives an MPM list for a current block, if a new DC mode is applied to a neighboring block, the intra prediction mode for the neighboring block may be changed from a planar mode to a DC mode and added to the MPM list. When the video signal processing device derives an MPM list for a current block, if a new DC mode is applied to a neighboring block, a DIMD mode derived using the neighboring block may be added to the MPM list.

If the current block is generated as a prediction block by explicitly using any one of the three DC prediction modes, the video signal processing device can perform PDPC filtering on the prediction block to generate a filtered prediction block.

If the current block is generated as a prediction block by explicitly using any one of the three DC prediction modes, the video signal processing device may not perform PDPC filtering on the prediction block.

If the prediction block of the current block is generated by explicitly using one of the three DC prediction modes, the video signal processing device may perform PDPC filtering based on the horizontal angle mode (e.g., the 18-degree angle mode of FIG. 6) or the vertical angle mode (e.g., the 50-degree angle mode of FIG. 6) instead of performing PDPC filtering based on the conventional DC prediction mode. For example, if the prediction block of the current block is generated by using the DC mode that uses the average of the pixel values on the left side of the current block, the video signal processing device may perform PDPC filtering based on the horizontal angle mode (e.g., the 18-degree angle mode of FIG. 6). Alternatively, if the prediction block of the current block is generated by using the DC mode that uses the average of the pixel values on the left side of the current block, the video signal processing device may perform PDPC filtering based on the vertical angle mode (e.g., the 50-degree angle mode of FIG. 6).

To reduce signaling bits due to the new DC mode, the encoder can signal only the information (e.g., a flag) about whether the new DC mode is applied to the current block in the bitstream. The decoder can parse the information about whether the new DC mode is applied to the current block to check and set whether the new DC mode is applied to the current block. If the new DC mode is applied to the current block, the video signal processing unit can implicitly select which of the two DC modes (i.e., Above DC mode and Left DC mode) to apply to the current block. The video signal processing device can determine which of the two DC modes (Above DC mode, Left DC mode) to apply to the current block by using at least one of the intra prediction directional mode derived from the surrounding pixels of the current block using DIMD, whether to blend the DIMD modes derived from the surrounding pixels of the current block, the intra prediction directional mode of the surrounding blocks of the current block, the encoding mode of the surrounding blocks of the current block, whether the horizontal size of the current block is the same as the vertical size of the current block, and a comparison of the horizontal size of the current block and the vertical size of the current block. For example, the video signal processing device can generate the prediction block by applying the Above DC mode to the current block if the intra prediction directional mode derived from the surrounding pixels of the current block using DIMD is greater than or equal to an arbitrary value. The video signal processing device can generate the prediction block by applying the Left DC mode to the current block if the intra prediction directional mode derived from the surrounding pixels of the current block using DIMD is less than the arbitrary value. Any value can be an integer, for example, 34, which is the value of the diagonal mode in Fig. 6. The method by which the video signal processing device implicitly selects which of the two DC modes (i.e., Above DC mode and Left DC mode) to apply to the current block can be applied when the horizontal and vertical sizes of the current block are the same.

If the DIMD mode derived from the surrounding pixels of the current block is a planar mode or a DC mode, the DIMD mode can be a DIMD mode without blending, in which case there can be only one intra prediction directional mode in the DIMD mode derived from the surrounding pixels of the current block. If the DIMD mode derived from the surrounding pixels of the current block is not a planar mode or a DC mode, the DIMD mode is a DIMD mode with blending, in which case there can be two or more DIMD modes derived from the surrounding pixels of the current block.

If a new DC mode is applied to the current block and the horizontal size of the current block is larger than the vertical size, the Above DC mode is the existing DC mode, so the video signal processing device can apply the Left DC mode to generate a prediction block for the current block. If a new DC mode is applied to the current block and the vertical size of the current block is larger than the horizontal size, the video signal processing device can apply the Above DC mode to generate a prediction block for the current block.

In order to reduce signaling bits due to the new DC mode, the video signal processing device may determine which of the three DC modes to apply to the current block by using at least one of: an intra prediction directional mode derived using DIMD from surrounding pixels of the current block, an intra prediction directional mode of a surrounding block of the current block, whether to blend DIMD modes derived from surrounding pixels of the current block, an encoding mode of a surrounding block of the current block, whether the horizontal length and the vertical length of the current block are the same, and a comparison of the horizontal length of the current block and the vertical length of the current block.

For example, a. If the intra prediction directional mode derived from the surrounding pixels of the current block using DIMD is greater than or equal to the first arbitrary value and less than the second arbitrary value (case a), the video signal processing device can apply the Left DC mode to generate a prediction block for the current block. If the intra prediction directional mode derived from the surrounding pixels of the current block using DIMD is greater than or equal to the second arbitrary value and less than the third arbitrary value (case b), the video signal processing device can apply the Above DC mode and the Left DC mode to generate a prediction block for the current block. If the intra prediction directional mode derived from the surrounding pixels of the current block using DIMD is greater than or equal to the third arbitrary value (case c), the video signal processing device can apply the Above DC mode to generate a prediction block for the current block. These methods (cases a, b, and c) can be applied when the horizontal and vertical sizes of the current block are the same. Furthermore, this method can be applied when no blending is applied to the DIMD mode derived from the surrounding pixels of the current block (when the DIMD mode derived from the surrounding pixels of the current block has only one intra prediction directional mode). In this case, the first random value, the second random value, and the third random value are integer values that can vary depending on the horizontal and vertical sizes of the current block, for example, 2, 26, and 42, respectively.

The TMRL method described above with reference to FIG. 30 can be extended and used to determine the optimal intra directional mode, whether to apply the ISP mode, whether to apply the Intra TMP and IBC modes, whether to apply the SGPM mode, and whether to apply the weighted average method of multiple intra directional modes.

The video signal processing device can configure a reference template (e.g., the template of FIG. 30) using a reference pixel line adjacent to the current block. The size of the reference template can be set based on the horizontal and vertical sizes of the current block. At this time, the width of the upper reference template can be the same as the width of the current block. Or, the width of the upper reference template can be larger or smaller than the width of the current block. The height of the upper reference template can be an arbitrary size, M. In addition, the height of the left reference template can be the same as the height of the current block. Or, the height of the left reference template can be larger or smaller than the height of the current block. The width of the left reference template can be an arbitrary size, N. At this time, M and N can be integers greater than or equal to 1, and can be set to the same or different values. The video signal processing device can set the width of the upper reference template (or the height of the left reference template) to be larger than the width of the current block by J (or larger than the height of the current block by K) when the horizontal and vertical sizes of the current block are within the arbitrary size. At this time, any size can be 4x4, 4x8, 8x4, 8x8, etc., and J and K can be integers greater than or equal to 1.

A video signal processing device can configure a reference pixel line list for a current block. At this time, the reference pixel line list can be configured based on the position of the current block and the maximum size of the CTU. When the position of the current block is expressed as (x, y), the video signal processing device can determine an allowable reference pixel line based on how far the y value is from the upper boundary of the CTU, and configure a reference pixel line list within the allowable reference pixel lines. For example, when the y value is 4, since y is 4 away from the upper boundary of the CTU and the number of pixels that can be used as the upper reference pixel based on the position of the current block is 4, the video signal processing device can include a reference pixel line whose index of the reference pixel line is 4 or less in the reference pixel line list. The indices of the reference pixel lines are 1, 3, 5, 7, and 12. Since the indices of the reference pixel lines are 4 or less and the index 0 of the reference pixel line used as the reference template is excluded from the reference pixel list, the reference pixel list can be composed only of reference pixel lines corresponding to

indices

1 and 3. Referring to FIG. 62, the video signal processing device can use three reference pixel lines when the y value is 4, and the reference pixel line list can be composed of reference pixel lines corresponding to

indices

1, 2, 3, and 4. Alternatively, the video signal processing device can configure the reference pixel line list with reference pixel lines corresponding to

indices

1, 2, and 3, excluding the reference pixel line corresponding to index 4 when the y value is 4. When encoding TIMD and the general intra prediction mode, the video signal processing device can use reference pixel lines corresponding to

indices

1, 3, 5, 7, and 12. Even when the TMRL method is used, the video signal processing device can use reference pixel lines corresponding to

indices

1, 3, 5, 7, and 12. Alternatively, when the TMRL method is used, the video signal processing device can configure a reference pixel line list differently from a reference pixel line list used in another encoding mode. For example, when the TMRL method is used, the video signal processing device can configure a reference pixel line list composed of reference pixel lines corresponding to

indices

2, 4, 6, 8, 9, 10, and 11.

The video signal processing device can configure an intra prediction mode list for the current block. At this time, the intra prediction mode list can be configured based on the MPM list. In addition, the video signal processing device can configure the intra prediction mode list including three DC prediction modes. If the DC mode exists in the intra prediction mode list, the video signal processing device can selectively add the three DC prediction modes to the intra prediction mode list according to the horizontal and vertical sizes of the current block. For example, if the DC mode exists in the intra prediction mode list and the horizontal size of the current block is larger than the vertical size, the video signal processing device can add the DC mode using the average of the left pixel values to the intra prediction mode list. If the DC mode does not exist in the intra prediction mode list, the video signal processing device can add the DC mode using the average of the left pixel values and the DC mode using the average of the upper pixel values to the intra prediction mode list. In addition, the video signal processing device can add three directional planar prediction modes to the intra prediction mode list.

A general MPM list can be composed of a primary MPM including 6 intra prediction modes and a secondary MPM including 16 intra prediction modes, and the size of the MPM list used in the intra prediction mode list can be different from the size of the general MPM list. That is, the size of the MPM list used in the intra prediction mode list can be X, and X is an integer greater than or equal to 1, such as 10. The video signal processing device can configure the intra prediction mode list through an MPM list having a size of 10. Alternatively, the video signal processing device can add 10 intra prediction modes to the intra prediction mode list from the general MPM list.

Each candidate of the intra prediction mode list may be composed of only one intra prediction mode. Alternatively, each candidate of the intra prediction mode list may be composed of two or more intra prediction modes as one candidate. For example, the video signal processing device may select N intra prediction modes of neighboring blocks adjacent to the current block, and then combine the N intra prediction modes to generate a new candidate and add the candidate to the intra prediction mode list. Based on the template cost of each candidate of the intra prediction mode list, candidates in the order of low to high template costs may be combined to generate a new candidate, and the new candidate may be added to the intra prediction mode list. In addition, the video signal processing device may add an intra prediction mode derived by the DIMD method to the intra prediction mode list. In this case, if there are two or more intra prediction modes derived by the DIMD method, the video signal processing device may configure the derived intra prediction mode as one candidate of the intra prediction mode list. Additionally, the video signal processing device can add an intra prediction mode derived by the TIMD method to the intra prediction mode list, and when there are two or more intra prediction modes derived by the TIMD method, the derived intra prediction mode can be configured as one candidate of the intra prediction mode list.

A video signal processing device can construct an integrated list using a reference pixel line list and an intra prediction mode list, and candidates of the integrated list can be constructed with reference pixel lines and intra prediction modes. At this time, the size of the integrated list can be an integer value of 20. Since the size of the integrated list is limited, the video signal processing device can construct the integrated list using only the top 20 candidates having high costs based on template costs. At this time, the template cost can be calculated based on the cost between the reference template and the prediction template. The prediction template can be obtained through a prediction sample for a reference template generated using any reference pixel line among the reference pixel line list and any intra prediction mode among the intra prediction mode list. At this time, the cost can be calculated using a method such as SAD or MR-SAD. Based on the template cost, a reference pixel line and an intra prediction mode having a minimum cost can be used to construct a prediction sample for a current block. The encoder can signal information indicating an index of a candidate indicating an optimal reference pixel line and an intra prediction mode by including it in the bitstream after reordering the unified list in ascending order of template cost, and the decoder can generate a prediction sample for the current block based on the optimal reference pixel line and intra prediction mode obtained by parsing the information.

Since the video signal processing device performs template-based reordering when applying the TMRL method, it may not perform the MPM list reordering process based on the template cost performed during the construction of the MPM list.

The method of rearranging the integrated list based on the template cost may have different encoding efficiency depending on how similar the template is to the characteristics of the current block. That is, due to the template having low similarity, the video signal processing device may have to encode a high index indicating an optimal candidate in the rearranged integrated list rather than a low index, and thus, there may be a problem that the amount of encoded bits increases. Therefore, the video signal processing device may rearrange the integrated list using at least one of a comparison between template costs for each candidate, a comparison between intra prediction modes of each candidate, and the width and height of the current block. For example, if the difference between the template cost values of two candidates is within a first value, the video signal processing device may delete one of the two candidates from the integrated list. In another embodiment, if the difference between the intra prediction modes of two candidates is within a second value, the video signal processing device may delete one of the two candidates from the integrated list. At this time, the first value and the second value can be integers greater than or equal to 1, for example, the first value can be 20 and the second value can be 5.

The video signal processing device can configure the integrated list by prioritizing the intra prediction mode. For example, the video signal processing device can rearrange the intra prediction mode list based on the template cost, bring the intra prediction mode candidates one by one in order from the candidate with the lowest template cost to the candidate with the highest template cost, and configure the integrated list by combining them with the reference pixel list. Accordingly, the integrated list can be configured in a form in which candidates using the same intra prediction mode form a group, and candidates within a group having the same intra prediction mode can be configured to have different reference pixel lines.

The video signal processing device can configure the unified list by prioritizing the reference pixel line. For example, the video signal processing device can configure the unified list in a form in which candidates using the same reference pixel line form a group, and the candidates within the group having the same reference pixel line can be configured to have different intra prediction modes. At this time, the video signal processing device can use at least one of the horizontal and vertical sizes of the current block and information (encoding mode, reference pixel line information, intra prediction mode, etc.) of the surrounding blocks to determine which reference pixel line to add to the unified list by prioritizing it.

The video signal processing device can reorder the unified list with the intra prediction mode as the priority. The video signal processing device can further reorder the unified list reordered based on the template cost using at least one of the intra prediction mode, the reference pixel line, and the template cost difference between each candidate. For example, the video signal processing device can reorder the unified list based on the reference pixel line, and can change the order of candidates using a specific reference pixel line to an upper part of the unified list to increase the priority of the corresponding candidate. At this time, the specific reference pixel line can be determined based on the template cost. For example, the video signal processing device can determine the reference pixel line of a candidate having the lowest template cost in the unified list as the specific reference pixel line.

The video signal processing device can configure a second unified list by adding a new candidate to the first unified list (the unified list described above). At this time, the new candidate to be added may be a candidate composed of Intra TMP, IBC, SGPM, TIMD, DIMD, MIP, an arbitrary reference pixel line and an arbitrary intra prediction mode, three DC prediction modes, and three planar prediction modes. At this time, the video signal processing device can determine an arbitrary reference pixel line and an arbitrary intra prediction mode by using surrounding block information and at least one or more of the candidates of the first unified list. The video signal processing device can configure a new candidate by using the intra prediction mode of the candidate having the lowest template cost, but setting the reference pixel line to a reference pixel line corresponding to index 0, and then add the new candidate to the second unified list. The video signal processing device can generate a new candidate by combining an intra prediction mode that does not exist in the unified list with an arbitrary reference pixel line, and then add the new candidate to the second unified list. The encoder can signal in the bitstream information indicating an index for a best candidate in the second unified list, and the decoder can parse the information to generate a prediction block for the current block using the best candidate for the index indicated by the information.

A video signal processing device can generate a new candidate by using a first unified list rearranged based on a template cost and add the new candidate to the unified list. Furthermore, the video signal processing device can perform a reordering based on the template cost for the newly added candidate to generate a reordered second unified list. At this time, the video signal processing device can generate a new candidate by combining two or more candidates from among the candidates of the first unified list. If the new candidate is composed of a combination of two candidates, the new candidate can be composed of a first reference pixel line and a first intra prediction mode, a second reference pixel line and a second intra prediction mode. If the new candidate is composed of a combination of three candidates, the new candidate can be composed of a first reference pixel line and a first intra prediction mode, a second reference pixel line and a second intra prediction mode, and a third reference pixel line and a third intra prediction mode. When a prediction block is generated using a new candidate composed of a combination of two candidates, the video signal processing device can generate a final prediction block through a weighted average between a first prediction block predicted in a first intra prediction mode by referring to a first reference pixel line and a second prediction block predicted in a second intra prediction mode by referring to a second reference pixel line.

Fig. 63(a) may represent the first integrated list described above, and Fig. 63(b) may represent the second integrated list. The first candidate of Fig. 63(b) may be configured as a combination between the first candidate and the second candidate of the first integrated list. The first candidate of Fig. 63(b) may be configured as a combination between the first candidate and the second candidate of the first integrated list. That is, the first candidate of Fig. 63(b) may be configured as L0 for the first pixel line, A for the first intra prediction mode, L1 for the second reference pixel line, and B for the second intra prediction mode.

Index 1 and index 2 corresponding to the rows of Fig. 64(a) and (b) can represent the candidate order of the first unified list, and can be configured so as not to overlap each other. The order of the columns of Fig. 64(a) and (b) represents the combination order, and a new candidate can be configured (generated, obtained) according to the combination order, and the video signal processing device can add the new candidate configured according to the combination order to the second unified list. Fig. 64(a) is a method for giving priority to a combination for the first candidate in the first unified list, and Fig. 64(b) can represent a method for giving priority to a combination having a lower sum of template costs.

The video signal processing device may determine whether to add the new candidate to the second unified list by using at least one of the intra prediction mode and the reference pixel line of the new candidate. For example, if two intra prediction modes of the candidates combined to form the new candidate are the same or the two intra prediction modes differ by a value within a certain range, the video signal processing device may not add the new candidate to the second unified list. At this time, the certain value may be an integer greater than or equal to 1, such as 5.

The reference pixel line and the intra prediction mode can be signaled separately. However, even if the reference pixel line and the intra prediction mode are different, similar prediction blocks can be generated in certain situations. The video signal processing device can configure a combination table for the reference pixel line and the intra prediction mode. The size of the combination table can be set to a maximum size as an integer greater than or equal to 1.

Referring to FIG. 65, a combination table can have 10 candidates, and one candidate can be composed of 3 combination items. One candidate in the combination table can have multiple combinations of reference pixel lines and intra prediction modes that can generate similar prediction blocks. That is, one candidate in the combination table can have multiple combinations. For example, an encoder can signal information on whether a combination table is applied to a current block by including it in a bitstream. In addition, the encoder can signal index information indicating optimal combination candidates based on the combination table by including it in a bitstream. A decoder can parse information on whether a combination table is applied to a current block, and if a combination table is used in the current block, parse index information indicating optimal combination candidates, and generate a prediction block for the current block by using a reference pixel line and intra prediction mode of a combination with the lowest template cost among combination items indicated by the index information. The index information indicating optimal combination candidates can indicate one of the indices (6510) of FIG. 65.

Alternatively, the encoder can signal information on whether the combination table is applied to the current block by including it in the bitstream. Furthermore, the encoder can signal index information indicating optimal combination candidates based on the combination table and index information on optimal combination items within the optimal combination candidates by including it in the bitstream. The decoder parses information on whether the combination table is applied to the current block, and if the combination table is used for the current block, generates a prediction block for the current block by using reference pixel lines and intra prediction modes determined by parsing the index information indicating optimal combination candidates and the index information on optimal combination items within the optimal combination candidates. The index information indicating optimal combination candidates may indicate any one of the indices (6510) of FIG. 65, and the index information on optimal combination items within the optimal combination candidates may indicate any one of the combination items (6520) of FIG. 65.

In a method for deriving an intra prediction mode using the TIMD method, the three DC prediction modes and the three planar prediction modes can also be included as candidates for the TIMD intra prediction mode, and the optimal intra prediction mode can be derived based on the template cost. In addition, the three DC prediction modes and the three planar prediction modes can be included in the MPM list.

Referring to FIG. 66, a video signal processing device can generate a prediction block of a current block using a candidate list based on a multi-prediction mode. The video signal processing device can configure a multi-prediction mode candidate list for the current block. At this time, the video signal processing device can configure the multi-prediction mode candidate list using at least one of surrounding block information, DIMD information of the current block, TIMD information of the current block, MPM list of the current block, and Non-MPM list of the current block. The surrounding block information may be a block adjacent to the current block, a block that is not adjacent to the current block and is separated by a predetermined distance, a surrounding block stored in a separate memory, etc. The surrounding block information may be intra-prediction directional mode information of the surrounding block and a weight value for each mode. The predetermined distance may be determined based on at least one of the upper left sample position of the current block, the horizontal and vertical sizes of the current block, and the predetermined horizontal and vertical sizes. For example, a video signal processing device can derive a surrounding block based on a position shifted horizontally to the left by the width of the current block based on the upper left sample position of the current block and a position shifted vertically upward based on the upper left sample position of the current block.

A method for a video signal processing device to configure a candidate list based on a multi-prediction mode is described. The video signal processing device can generate a multi-prediction mode candidate by using at least one of neighboring block information, DIMD information of a current block, TIMD information of a current block, an MPM list of a current block, a Non-MPM list of a current block, DIMD information derived from a cumulative histogram of several histograms used to derive DIMD from a neighboring block, and top N (a predetermined number) intra-prediction modes with a high occurrence frequency among intra-prediction modes used in the neighboring block. The predetermined number can be an integer of 5. For example, a multi-prediction mode candidate based on occurrence frequency can include five intra-prediction modes. In this case, the multi-prediction mode can be composed of two or more intra-prediction directional modes, and the intra-prediction directional modes of one multi-prediction mode candidate can be different from each other. In addition, the multi-prediction mode can include only an intra-prediction directional mode having directionality. In addition, the multi-prediction mode may not include a planar mode, which is a non-directional mode. If the intra prediction directional mode derived from the surrounding block is a planar mode, the intra prediction directional mode derived from the surrounding block may be excluded from the multi-prediction mode. In addition, the multi-prediction mode may not include the DC mode, which is a non-directional mode. If the intra prediction directional mode derived from the surrounding block is a DC mode, the intra prediction directional mode derived from the surrounding block may be excluded from the multi-prediction mode. The multi-prediction mode candidate generated by the above-described method may be added to the multi-prediction mode candidate list. The video signal processing device may determine whether the same multi-prediction mode candidate exists in the multi-prediction mode candidate list, and may add the generated multi-prediction mode candidate to the multi-prediction mode candidate list only if there is no such multiple prediction mode candidate. The maximum size of the multi-prediction mode candidate list may be a pre-specified number, and the pre-specified number may be an integer, such as 25. If the maximum size of the multi-prediction mode candidate list is a pre-specified number, the video signal processing device may not add any more multi-prediction mode candidates.

The video signal processing device can generate a reordered multi-prediction mode candidate list by performing reordering of a multi-prediction mode candidate list based on a template cost. Meanwhile, the reordering process may not be performed. The reordering based on the template cost may be performed by the method described above through FIG. 30. The video signal processing device can configure a reference template using reconstructed surrounding samples adjacent to a current block. The video signal processing device can generate a prediction template for the reference template by using a multi-prediction mode candidate in the multi-prediction mode candidate list and a pre-designated reference sample line (

Reference line

1, 2, 3, ... of FIG. 30). The pre-designated reference sample line may be Reference line 1 (e.g., a line adjacent to the current block template). The video signal processing device can calculate a cost between the reference template and the prediction template. Here, the cost may be the sum of the absolute values of the differences (SAD) between samples. In addition, the cost may be the sum of the absolute values of the differences (MR-SAD) between the average difference sample values obtained by subtracting the average of each template from each sample value. The intra prediction directional mode of the multiple prediction mode candidate used to generate the prediction template may be converted to a range of the intra prediction mode that is extended beyond the range of the existing intra prediction mode, and then used to generate the prediction template. For example, if the intra prediction directional mode of the multiple prediction mode candidate is N, the extended intra prediction directional mode M may be (N *2) -2. In addition, since the multiple prediction mode candidate is composed of two or more intra prediction directional modes, the prediction templates generated using each intra prediction directional mode may be weighted averaged by applying a pre-specified weight, and the video signal processing device may generate the weighted averaged prediction template. At this time, the weight may be a pre-specified value, and the weights for each intra prediction directional mode may be the same or different from each other. The list of multiple prediction mode candidates may be re-sorted based on the calculated cost, and may be re-sorted in ascending order.

A video signal processing device can reconstruct a prediction mode candidate list using the MPM list and the rearranged multiple prediction mode candidate list. At this time, the video signal processing device can construct the prediction mode candidate list using a part of the MPM list and a part of the rearranged multiple prediction mode candidate list. Candidates from a first candidate of the MPM list to a pre-specified number of candidates can be added to the prediction mode candidate list, and the pre-specified number can be 5. Candidates from among the multiple prediction mode candidates of the rearranged multiple prediction mode candidate list can be added to the prediction mode candidate list in a pre-specified number in order from a low cost to a high cost, and the pre-specified number can be an integer and can be 16.

The encoder can determine an optimal prediction mode candidate for the current block from the derived prediction mode candidate list (or a multiple prediction mode candidate list if the prediction mode candidate list reconstruction process is not performed), and then generate and signal a bitstream including index information for the optimal prediction mode candidate. The decoder can parse the index information for the optimal prediction mode candidate from the bitstream, and obtain a prediction block for the current block using the determined optimal prediction mode candidate. The video signal processing device can obtain a final prediction block of the current block through a weighted average between a prediction block obtained based on a planar mode (or a DC mode) and a prediction block obtained with the optimal prediction mode candidate.

Whether a method for generating a prediction block using a candidate list based on multiple prediction modes is activated may be determined based on at least one of whether the current block is a luminance block or a chrominance block, whether an intra prediction mode is applied to the current block, whether one of the DIMD, TIMD, SGPM, ISP, MIP, Intra TMP, and TMRL modes is applied to the current block, reference pixel line information of the current block, horizontal and vertical sizes of the current block, and whether the current block is located at the upper boundary of a CTU including the current block. For example, when one of the DIMD, TIMD, SGPM, ISP, MIP, Intra TMP, and TMRL modes is applied to the current block, the method for generating a prediction block using a candidate list based on multiple prediction modes may not be activated, and the video signal processing device may not signal or parse syntax related to the method for generating a prediction block using a candidate list based on multiple prediction modes.

The candidate list based on the derived multiple prediction modes can be used when the coding mode of the current block is one of DIMD, TIMD, SGPM, MIP, Intra TMP, and TMRL.

When DIMD is applied to the current block, the video signal processing device can add the prediction mode derived by using DIMD to the candidate list based on the derived multiple prediction mode. At this time, the candidate list based on the derived multiple prediction mode can be re-ordered based on the template cost. The encoder can generate and signal a bitstream including index information for an optimal prediction mode in the candidate list based on the derived multiple prediction mode, and the decoder can parse the index information to determine a prediction mode for the current block in the candidate list based on the derived multiple prediction mode.

When TIMD is applied to the current block, the video signal processing device can add the prediction mode derived using TIMD to the candidate list based on the derived multiple prediction mode, and the candidate list based on the derived multiple prediction mode can be reordered based on the template cost. The encoder can generate and signal a bitstream including index information for an optimal prediction mode in the candidate list based on the derived multiple prediction mode, and the decoder can parse the index information for the optimal prediction mode to determine the optimal prediction mode in the candidate list based on the derived multiple prediction mode.

When SGPM is applied to the current block, the video signal processing device can use the candidate list based on the derived multiple prediction modes to derive intra prediction modes of the two divided regions, the encoder can generate and signal a bitstream including index information on an optimal prediction mode for one or more regions among the two divided regions, and the decoder can parse the index information to determine the prediction mode for one or more regions among the two divided regions from the list.

When DIMD is applied to a current block, a video signal processing device can add block vector candidates derived using Intra TMP to a candidate list based on the derived multiple prediction mode, and the candidate list based on the derived multiple prediction mode can be reordered based on a template cost. An encoder can generate and signal a bitstream including index information for an optimal prediction mode in the candidate list based on the derived multiple prediction mode, and a decoder can parse the index information to determine an optimal prediction mode in the candidate list based on the derived multiple prediction mode.

When TMRL is applied to the current block, the video signal processing device can construct a unified candidate list using the candidate list based on the induced multiple prediction mode and the reference sample line. The unified list can be reordered based on the template cost. The encoder can generate and signal a bitstream including index information for an optimal prediction mode and reference sample line in the candidate list based on the induced multiple prediction mode, and the decoder can parse the information for the corresponding index to determine an optimal prediction mode reference sample line in the candidate list based on the induced multiple prediction mode from the unified list.

The Non-MPM list can be reordered based on template cost using templates, and the Non-MPM list can be used to signal or parse the optimal intra prediction mode.

The MPM list for the current block (e.g., Primary MPM list, Secondary_ MPM list, or Non-MPM list) can be organized or rearranged according to the occurrence frequency of intra prediction modes used in the surrounding blocks. The intra prediction modes used in the surrounding blocks can be at most J intra prediction modes if the surrounding blocks are DIMD, at most K intra prediction modes if the surrounding blocks are TIMD mode, at most L intra prediction modes if the surrounding blocks are SGPM mode, and at most L intra prediction modes if the surrounding blocks are Intra TMP or IBC mode, and can be intra prediction modes stored in a block to which Intra TMP or IBC mode is applied (or intra prediction modes of a block indicated by a block vector of a block to which Intra TMP or IBC mode is applied). Information about the surrounding blocks can include blocks adjacent to the current block, blocks not adjacent to the current block but separated by a predetermined distance, and surrounding blocks stored in a separate memory. In addition, information about the surrounding blocks can include intra prediction directionality of the surrounding blocks. It may include mode information and a weight value for each mode. The video signal processing device may generate a histogram for the intra prediction mode using the intra prediction mode derived from the surrounding block. At this time, J, K, and L may be integers and may be 5, 2, 2, respectively. The video signal processing device may configure an MPM list for the current block using the histogram for the intra prediction mode. The MPM list may be added in order from the highest occurrence frequency to the lowest occurrence frequency, and if the primary MPM list is filled up to a pre-specified maximum number, the MPM list may be added in order of the next occurrence frequency. If the secondary MPM list is filled up to a pre-specified maximum number, the MPM list may be added in order of the next occurrence frequency to the Non-MPM list. The histogram for the intra prediction mode may be updated for each block and used to configure the MPM list or the Non-MPM list of the block.

In addition, the video signal processing device can first configure the MPM list of the current block, and then perform reordering of the MPM list or the Non-MPM list using the histogram for the intra prediction mode. Specifically, the video signal processing device can sequentially set reference intra prediction modes in order from a high occurrence frequency to a low occurrence frequency in the histogram for the intra prediction mode, and then reorder the MPM list or the Non-MPM list based on the reference intra prediction mode. Then, the video signal processing device can change the order of the intra prediction mode that is the same as the reference intra prediction mode or has a difference by a predetermined value in the MPM list or the Non-MPM list to a higher order in the MPM list or the Non-MPM list. The predetermined value can be an integer of 3.

The order in which the video signal processing device adds the intra prediction mode to the Non-MPM list may vary based on the horizontal and vertical sizes of the current block. For example, if the horizontal size of the current block is larger than the vertical size, the intra prediction modes (e.g., the prediction modes of FIG. 6) may be added in the order of a pre-specified first intra prediction mode to a pre-specified second intra prediction mode. The first intra prediction mode may be a prediction mode of index 66, and the second intra may be a prediction mode of index 2.

In addition, if the horizontal size of the current block is larger than the vertical size, the intra prediction modes may be added in the order of a pre-designated first intra prediction mode to a pre-designated second intra prediction mode, and then in the order of a pre-designated third intra prediction mode to a pre-designated fourth intra prediction mode. The first intra prediction mode may be a prediction mode of index 34, the second intra prediction mode may be a prediction mode of index 66, the third intra prediction mode may be a prediction mode of index 33, and the fourth intra prediction mode may be a prediction mode of index 2.

The IBC coding method (IBC mode) is a method of finding the most similar part (reference block) to the current block in the already restored area of the current picture and using the reference block as a prediction block for the current block. At this time, the encoder can generate a bitstream including information related to a block vector, which is a distance between the current block and the reference block. The decoder can parse information related to the block vector included in the BeastStream to calculate or set a block vector for the current block. The IBC coding method can be applied to a chrominance block. In the chrominance block, the block vector of the luminance block corresponding to the chrominance block can be used as the block vector of the chrominance block without finding a new block vector, and this coding method can be called the DBV (Direct Block Vector) mode.

The RRIBC (Reconstruction-Reordered IBC) encoding mode can be used in an IBC block (a block to which the IBC encoding method is applied). RRIBC can be composed of a vertical flip and a horizontal flip. A block to which RRIBC is applied is flipped when reconstructed according to the RRIBC type of the current block. The encoder can flip the original block to be encoded before finding the most similar part to the current block from the reference picture. That is, the most similar part from the reference picture is found using the flipped original block. Therefore, a block that is not flipped is used as the prediction block, and the residual block can also be a block that is not flipped. The decoder can flip the final reconstructed block according to the RRIBC type of the current block.

In FIG. 68 and FIG. 69, (Xn, Yn) represents the center position of the surrounding blocks, and (Xc, Yc) represents the center position of the current block. BVnh and BVnv represent a horizontal block vector and a vertical block vector of the surrounding blocks, respectively, and BVCh and BVCv represent a horizontal block vector and a vertical block vector of the current block, respectively. As shown in FIG. 53, if the RRIBC type of the current block is in the horizontal direction, BVCh can be calculated as 2 * (Xn - Xc) + BVnh, and as shown in FIG. 53, if the RRIBC type of the current block is in the vertical direction, BVCv can be calculated as 2 * (yn - yc) + BVnv. At this time, since BVnh and BVnv use the restored block, the signs of BVnh and BVnv can only be negative.

If the current block is encoded with RRIBC, the video signal processing device can determine an optimal block vector by constructing a block vector candidate list. At this time, the video signal processing device can construct the block vector candidate list according to the RRIBC type of the current block. For example, if the RRIBC type of the current block is horizontal, the video signal processing device can construct the block vector candidate list using only the neighboring blocks of the current block encoded with RRIBC in the horizontal direction. In addition, the video signal processing device can construct the block vector candidate list regardless of the RRIBC type of the current block. For example, if the RRIBC type of the current block is horizontal, the block vector candidate list can be constructed using not only the neighboring blocks of the current block encoded with RRIBC in the horizontal direction, but also the neighboring blocks of the current block encoded with RRIBC in the vertical direction and/or blocks encoded with general motion and/or blocks encoded with block vectors.

When a video signal processing device predicts a current block using general motion, the video signal processing device can construct a motion candidate list depending on whether the neighboring blocks of the current block are encoded in the IBC mode or the RRIBC mode. At this time, when the neighboring blocks of the current block are encoded in the RRIBC mode, the video signal processing device can construct a motion candidate list by additionally considering the RRIBC type. For example, when the video signal processing device constructs a motion candidate list for the current block, when the encoding mode of the neighboring blocks of the current block is the RRIBC mode and the RRIBC type is the vertical direction or the horizontal direction, the block vector of the neighboring blocks may not be included in the motion candidate list. Alternatively, when the video signal processing device predicts the current block using general motion, the video signal processing device can construct a motion candidate list regardless of the encoding mode of the neighboring blocks of the current block. That is, the video signal processing device can construct a motion candidate list regardless of whether the neighboring blocks of the current block are encoded in the IBC mode or the RRIBC mode. For example, when a video signal processing device configures a motion candidate list for a current block, it may include block vectors of surrounding blocks of the current block in the motion candidate list even if the encoding mode of the surrounding blocks is the IBC mode and the RRIBC type is the vertical direction or the horizontal direction.

The RRIBC encoding method is effective for images having symmetrical characteristics. The symmetrical characteristics may mean perfect horizontal (or vertical) symmetry with the same distance apart from one central axis between the current block and the reference block. The vertical (or horizontal) direction (symmetry axis) of the current block may be located on the same line as the vertical (or horizontal) direction (symmetry axis) of the reference block. In this case, the block vector in the vertical (or horizontal) direction may be set to '0', and the block vector in the horizontal direction may be set to any negative value other than '0'. The current block and the reference block may be configured to be symmetrical with different distances apart from one central axis between the current block and the reference block, and the current block and the reference block may be located on different vertical lines. In this case, the block vector in the vertical direction may be set to any negative value other than '0'. That is, the video signal processing device can encode or decode the current block using a symmetric block having a block vector value of an arbitrary negative value other than '0' in both the horizontal and vertical directions.

Referring to FIG. 70, the Intra TMP method (Intra TMP encoding mode) is a method in which a video signal processing device constructs a reference template using pixel values of neighboring blocks adjacent to a current block, and then finds a part most similar to the reference template in an already restored area (reference block) within the current picture, and then uses the reference block (Ref. luma block of FIG. 70) as a prediction block for the current block. At this time, there may be one or more block vectors used to generate a prediction block for the current luminance block, and FIG. 70 shows a case in which there are two block vectors. The video signal processing device can generate a prediction block for the luminance block by weighting and averaging reference blocks predicted from two block vectors of the current luminance block. In the case of a chrominance block, the video signal processing device can derive a block vector from a luminance block corresponding to the chrominance block, and then generate a chrominance prediction block using the block vector. There may be more than one block vector used to generate a prediction block for the current chrominance block, and Fig. 70 shows a case where there are two block vectors for the chrominance block. In this case, the block vector of the chrominance block may be the same as or different from the block vector derived from the luminance block.

The current block can be encoded or decoded in the IBC-GPM mode. The IBC-GPM mode may mean a mode in which the current block is divided using the GPM mode, and the divided region is encoded in the intra prediction mode or the IBC prediction mode. Referring to FIG. 71, the current block can be divided into two regions based on the dotted line. At this time, one of the divided regions can be encoded in the intra prediction mode, and the other can be encoded in the IBC prediction mode. For example, the left region among the divided regions can be encoded in the intra prediction mode, and the right region can be encoded in the IBC prediction mode. At this time, the region encoded in the IBC prediction mode can be encoded in the Intra TMP prediction mode, and this can be described as the IntraTMP-GPM mode.

A video signal processing device can predict a current block using a weighted average between a block predicted in the intra mode and a block predicted in the IBC mode, and this can be called the IBC-CIIP mode. At this time, a block predicted in the Intra TMP mode can be used instead of a block predicted in the IBC mode, and this can be called the IntraTMP-CIIP mode.

A video signal processing device can obtain a reconstructed luminance block by adding a residual signal for a luminance prediction block of a current block and a luminance block, and can configure a CCP model by using a correlation between the reconstructed luminance block and the luminance prediction block. At this time, the CCP model can be one of CCLM, MMLM, GLM, CCCM, MM-CCCM, GL-CCCM, CCCM-ND, and CCCM-MDF. The CCP model derived from the luminance block can be applied to a chrominance prediction block to generate a first chrominance prediction block to which the CCP model is applied. A final chrominance block can be generated by adding an error signal for the first chrominance prediction block and the chrominance block.

CCCM(Convolutional cross-component intra prediction model) is a method of predicting a chrominance signal using a nonlinear model that is constructed by utilizing a high correlation between a luminance signal and a chrominance signal at the same location as the luminance signal. Fig. 73(a) shows the positional relationship between reference samples (vertical hatching, 7320) for applying CCCM to a current prediction block (diagonal hatching, 7310) and side samples (horizontal hatching) required when applying a cross-shaped filter. The current prediction block (MxN) can be composed of reference samples in the upper 6 rows (2Mx6), reference samples in the left 6 rows (6x2N), and 6x6 reference samples in the upper left, where the ratio of the number of chroma to luma samples is 1:1. When a video signal processing device applies a cross-shaped sample (Fig. 73(b)) filter to a chroma sample prediction relationship (Fig. 73(c)) for CCCM, there are cases where the reference sample area is out of the range. At this time, additionally required samples may be side samples. The chroma sample prediction relation of Fig. 73(c) may be applied to each chroma component (i.e., Cb, Cr). The sample at the C (Center) position in Fig. 73(b) may be a luma sample corresponding to the Cb and Cr chroma samples, and N (North), E (East), S (South), and W (West) may be luma samples adjacent to the luma sample at the C position. Side samples may additionally require one sample each for an area other than the reference sample depending on the position of the C sample. If the sample value existing at the position of the side sample is not available, the sample value at the unavailable position may be padded with the C sample value. The P value of Fig. 73(c) may be a nonlinear term. The P value may be calculated as P = ( C*C + midVal ) >> bitDepth , and in case of 10-bit content, P = ( C*C + 512 ) >> 10. In Fig. 73(c), the B value can be an integer offset value as a bias value. The B value can be an intermediate value of bitDepth (bit depth). In case of 10-bit content, the B value can be 512.

MM-CCCM (multi-model CCCM) is a method that derives two CCCM parameters based on the average value of the reference area (or the reconstructed current luminance block).

GL-CCCM (Gradient and location based convolutional cross-component model) is an additional CCCM mode that uses gradient and location information. In the case of the existing CCCM mode, the video signal processing device can derive the chrominance sample for the current block by using the luminance sample at the corresponding position from the predicted chrominance sample location, four samples around the luminance sample, and coefficient information. At this time, in the case of the GL-CCCM mode, the video signal processing device can derive the chrominance sample for the current block by reflecting the vertical and horizontal differences for the luminance sample at the corresponding position from the predicted chrominance sample location and eight samples around the luminance sample, and also by using the position value of the current luminance sample and its coefficient information.

When the CCCM mode is applied, the video signal processing device can apply a downsampling filter to match the resolution difference between the luminance block and the chrominance block. This is to reduce the resolution of the luminance block to that of the chrominance block. The mode for applying the downsampling filter can be described as CCCM-MDF (CCCM with multiple downsampling filters).

When the inter encoding mode is applied to the current block, the video signal processing device can derive linear and nonlinear models between a luminance prediction block (Y') derived using motion information of the current block and a first chrominance prediction block (Cb', Cr') derived using motion information of the current block (Derive filter), generate a reconstructed luminance block of the current block using a luminance residual block of the current block, and apply the derived linear and nonlinear models to the reconstructed luminance block of the current block (Apply filter) to generate a second chrominance prediction block of the current block. Thereafter, the video signal processing device can add the chrominance residual block to the second chrominance prediction block of the current block predicted using the linear and nonlinear models to finally generate the chrominance block (Cb, Cr) of the current block. This method can be described as a cross-component residual model (CCRM).

Referring to FIG. 74, a method of constructing the integrated list described above through FIGS. 1 to 73 is described.

A video signal decoding device (e.g., a decoder) can obtain a plurality of pairs each consisting of one reference line and one intra prediction mode (S7410). The plurality of pairs can include a first pair and a second pair.

A video signal decoding device can obtain a list composed of a plurality of integrated pairs including a first integrated pair that integrates the first pair and the second pair (S7420).

The video signal decoding device can restore the current block based on one integrated pair in the above list (S7430).

Each of the above multiple pairs may be composed of different combinations of reference lines and intra prediction modes.

A video signal decoding device can obtain a first prediction block based on a first reference line and a first intra prediction mode of the first pair, obtain a second prediction block based on a second reference line and a second intra prediction mode of the second pair, and obtain a final prediction block by performing a weighted average on the first prediction block and the second prediction block. And the video signal decoding device can reconstruct the current block based on the final prediction block.

The one integrated pair in the above list can be indicated based on a syntax element included in the bitstream.

The plurality of integrated pairs constituting the above list can be sorted based on the cost of each of the plurality of integrated pairs.

The above multiple integrated pairs can be sorted in order of their corresponding costs.

The methods described herein may be performed by a processor of a decoder or an encoder. In addition, the encoder may generate a bitstream that is decoded by the methods described above. In addition, the bitstream generated by the encoder may be stored in a computer-readable, non-transitory storage medium (recording medium).

This specification is mainly described from the decoder's perspective, but it can be operated in the same way in the encoder. The term parsing in this specification is described with a focus on the process of obtaining information from a bitstream, but in terms of the encoder, it can be interpreted as configuring the corresponding information in the bitstream. Therefore, the term parsing is not limited to the decoder's operation, but can be interpreted as the act of configuring a bitstream in the encoder. In addition, such a bitstream can be stored and configured in a computer-readable recording medium.

The embodiments of the present invention described above can be implemented through various means. For example, the embodiments of the present invention can be implemented by hardware, firmware, software, or a combination thereof.

In the case of hardware implementation, the method according to embodiments of the present invention can be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of implementation by firmware or software, the method according to the embodiments of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above. The software code may be stored in a memory and may be driven by a processor. The memory may be located inside or outside the processor, and may exchange data with the processor by various means already known.

Some embodiments may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules, that are executed by a computer. Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Additionally, computer-readable media can include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically includes computer-readable instructions, data structures, or other data in a modulated data signal, such as program modules, or other transport mechanisms, and includes any information delivery media.

The above description of the present invention is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential characteristics of the present invention. Therefore, it should be understood that the embodiments described above are exemplary and limited in all respects. For example, each component described as a single component may be implemented in a distributed manner, and likewise, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

Claims

In a video signal decoding device,

Includes a processor,

The above processor,

Obtain multiple pairs consisting of one reference line and one intra prediction mode,

The above plurality of pairs includes a first pair and a second pair,

Obtain a list consisting of a plurality of integrated pairs including a first integrated pair that integrates the first pair and the second pair,

A video signal decoding device that restores a current block based on one integrated pair in the above list.
In paragraph 1,

A video signal decoding device, wherein each of the above plurality of pairs is composed of a combination of different reference lines and intra prediction modes.
In paragraph 1,

The above processor,

Obtaining a first prediction block based on the first reference line and the first intra prediction mode of the first pair,

Obtaining a second prediction block based on the second reference line of the second pair and the second intra prediction mode,

The final prediction block is obtained by weighting and averaging the first prediction block and the second prediction block,

A video signal decoding device for restoring the current block based on the final prediction block.
In paragraph 1,

A video signal decoding device, wherein said one integrated pair in said list is indicated based on a syntax element included in a bitstream.
In paragraph 1,

A video signal decoding device, wherein the plurality of integrated pairs constituting the above list are sorted based on the cost of each of the plurality of integrated pairs.
In paragraph 5,

A video signal decoding device, wherein the above plurality of integrated pairs are sorted in order of decreasing corresponding costs.
In a video signal encoding device,

Includes a processor,

The above processor obtains a bitstream decoded by a decoding method,

The above decoding method is,

A step of obtaining multiple pairs consisting of one reference line and one intra prediction mode;

The above plurality of pairs includes a first pair and a second pair,

A step of obtaining a list composed of a plurality of integrated pairs including a first integrated pair that integrates the first pair and the second pair; and

A video signal encoding device, comprising a step of restoring a current block based on one integrated pair in the above list.
In Article 7,

A video signal encoding device, wherein each of the above plurality of pairs is composed of a combination of different reference lines and intra prediction modes.
In Article 7,

The above decoding method is,

A step of obtaining a first prediction block based on the first reference line and the first intra prediction mode of the first pair;

A step of obtaining a second prediction block based on the second reference line of the second pair and the second intra prediction mode;

A step of obtaining a final prediction block by weighting and averaging the first prediction block and the second prediction block; and

A video signal encoding device further comprising a step of restoring the current block based on the final prediction block.
In Article 7,

A video signal encoding device, wherein said one integrated pair in said list is indicated based on a syntax element included in a bitstream.
In Article 7,

A video signal encoding device, wherein the plurality of integrated pairs constituting the above list are sorted based on the cost of each of the plurality of integrated pairs.
In Article 11,

A video signal encoding device, wherein the above plurality of integrated pairs are sorted in order of decreasing corresponding costs.
In a computer-readable non-transitory storage medium storing a bitstream, the bitstream is decoded by a decoding method,

The above decoding method is,

A step of obtaining multiple pairs consisting of one reference line and one intra prediction mode;

The above plurality of pairs includes a first pair and a second pair,

A step of obtaining a list composed of a plurality of integrated pairs including a first integrated pair that integrates the first pair and the second pair; and

A non-transitory storage medium comprising a step of restoring a current block based on one of the integrated pairs in the above list.
In Article 13,

A video signal encoding device, wherein each of the above plurality of pairs is composed of a combination of different reference lines and intra prediction modes.
In Article 13,

The above decoding method is,

A step of obtaining a first prediction block based on the first reference line and the first intra prediction mode of the first pair;

A step of obtaining a second prediction block based on the second reference line of the second pair and the second intra prediction mode;

A step of obtaining a final prediction block by weighting and averaging the first prediction block and the second prediction block; and

A non-transitory storage medium further comprising a step of restoring the current block based on the final predicted block.
In Article 13,

A non-transitory storage medium in which said one integrated pair in said list is indicated based on a syntax element included in the bitstream.
In Article 7,

A non-transitory storage medium, wherein the plurality of integrated pairs constituting the above list are sorted based on the cost of each of the plurality of integrated pairs.
In Article 17,

A non-transitory storage medium in which the above multiple integrated pairs are sorted in order of their corresponding costs.