TWI657443B - Memory device and operation method thereof - Google Patents

Abstract

The invention discloses a memory device comprising a plurality of bit lines, a plurality of word lines and a control circuit. The bit line is used to receive a plurality of pixel data of an image. Each word line is coupled to a plurality of parameter units. The parameter unit of each character line is configured according to a plurality of parameters of a filter. The configuration of each character line is different from each other. When a first region of the image is processed by the filter, the control circuit inputs pixel data located in the first region of the image into the bit line and enables one of the word lines to operate. When processing a second region of the image with the filter, the control circuit keeps the pixel data located in the second region overlapping with the first region and inputs the pixel data to the bit line, and the second region does not overlap with the first region. The pixel data is input to the bit line and one of the word lines is enabled for operation.

Description

Memory device and method of operating same

The present invention relates to a memory device and method of operation thereof.

Object recognition is widely used in various fields, such as self-driving, satellite image analysis, face recognition, and the like. In general, a lot of energy is required in the process of object identification. In an increasingly precious future, how to reduce energy consumption is one of the goals of the industry.

It is an object of the present invention to provide a memory device and method of operating the same that can reduce the energy consumption of image processing.

An embodiment of the invention discloses a memory device comprising a plurality of bit lines, a plurality of word lines, and a control circuit. The bit line is used to receive a plurality of data of an image. Each word line is coupled to a plurality of parameter units, and each word line is coupled to the bit line by a parameter unit, wherein the parameter unit of each word line is configured according to a plurality of parameters of a filter, and each character element The configuration of the parameter units of the line are different from each other. The control circuit is coupled to the word line and the bit line. When a first region of the image is processed by the filter, the control circuit inputs the data located in the first region of the image into the bit line and enables one of the word lines to perform the operation. When When the filter processes a second region of the image, the control circuit keeps the data overlapping the first region in the second region of the image and inputs the data to the bit line, and does not overlap the first region in the second region of the image. The data is entered into the bit line and one of the word lines is enabled for operation.

An embodiment of the present invention provides a method for operating a memory device, including: providing an image, wherein the image includes a plurality of materials; and processing a first region of the image by a filter, by the memory device a control circuit for inputting the data of the first region of the image into a plurality of bit lines of the memory device; wherein the control circuit enables one of a plurality of word lines of the memory device to The character line is operated with the data of the first area of the image, wherein each of the word lines is coupled to a plurality of parameter units, and the parameter units of each of the word lines are based on the plurality of filters The parameters are configured, and the configuration of the parameter units of the word lines are different from each other; when the second area of the image is processed by the filter, the control circuit is kept in the second area of the image and The pieces of data overlapped by the first area are input to the bit lines, and the pieces of data that are not overlapped with the first area in the second area of the image are input to the bit lines; and by the control Circuit enable Wherein the other of wordlines, to the other word line and calculates the plurality of data of the second region of the image.

By the memory device and the operation method thereof, the energy consumption in image processing can be effectively reduced, and the calculation time can be further shortened.

In order to better understand the above and other aspects of the present invention, the following detailed description of the embodiments and the accompanying drawings

1‧‧‧Image Processing System

12‧‧‧ Feature Extraction Module

14‧‧‧Classification module

122‧‧‧Feature matching unit

20‧‧‧ memory device

BL1~BL8‧‧‧ bit line

WL1~WL12‧‧‧ character line

P11~P14, P21~P24‧‧‧ parameter unit

IMG‧‧‧ images

F1, F2‧‧‧ filter

SL1~SL12‧‧‧ source line

S501~S509‧‧‧Steps

1 is a block diagram of an image processing system according to an embodiment of the present invention; 2A and 2B are schematic views showing a process of processing an image by a filter according to an embodiment of the present invention; and FIG. 3 is a view showing a method according to the present invention; FIG. 4 is a block diagram of a memory device according to an embodiment of the invention; FIG. 5 is a flow chart showing a method for operating a memory device according to an embodiment of the invention; 6A is a schematic diagram of processing an image by a filter according to another embodiment of the present invention; and FIG. 6B is a schematic diagram showing the operation of the memory device according to another embodiment of the present invention.

Please refer to FIG. 1 . FIG. 1 is a block diagram of an image processing system according to an embodiment of the invention. The image processing system 1 includes a feature extraction module 12 and a classification module 14. The image processing system 1 can be applied, for example, to techniques such as object recognition or neural networks. The image processing system 1 can be used to process an image IMG.

The feature extraction module 12 can include a plurality of feature matching units 122. Each feature matching unit 122 can process the image IMG, for example, with a filter, wherein the filter can be, for example, a geometric pattern such as a square, a circle, a triangle, or the like. In an embodiment, as shown in FIGS. 2A and 2B, the feature matching unit 122 performs the image IMG with a filter F1 having a size of 2×2. Reason. The filter F1 includes a plurality of parameters a1, a2, a3, a4. The image IMG is a 4x4 image, including multiple pixel data X11~X44. When the image IMG is processed by the filter F1, the image IMG is convolved with the filter F1. As shown in Fig. 2B, there is a relative position between the filter F1 and the image IMG, and the relative position is changeable. The "relative position" refers to the area of the image IMG to be processed by the filter F1. For example, when processing a first region of the image IMG with the filter F1, the filter F1 may be covered on the pixel data X11, X21, X12, X22 of the first region of the image IMG, and the filter F1 is The parameters a1 to a4 are multiplied by the pixel data X11, X21, X12, and X22, respectively, and the output F11 of the feature matching unit 122 is equal to X11*a1+X21*a2+X12*a3+X22*a4. Similarly, when the filter F1 covers the pixel data X12, X22, X13, X23 of a second region of the image IMG, the output F12 of the feature matching unit 122 is equal to X12*a1+X22*a2+X13*a3+X23 *a4, and so on. When the feature matching program is completed, the feature matching results F11~F33 can be obtained (as shown in FIG. 2A). These results are sorted by the classification module 14.

The feature matching unit 122 can be implemented by one or more memory devices, wherein the memory device can be a volatile memory or a non-volatile memory, such as a NAND type or a NOR type memory. Please refer to FIG. 3. FIG. 3 is a block diagram of a memory device according to an embodiment of the invention. The memory device 20 includes a plurality of bit lines BL1 BLBL4, a plurality of source lines SL1 and SL2, a plurality of word lines WL1 and WL2, and a control circuit CTL, wherein the control circuit CTL is coupled to the word lines WL1 and WL2. And bit lines BL1~BL4. The bit lines BL1 to BL4 can be supplied to the pixel data of the image IMG as an input terminal. The word line WL1 includes a plurality of parameter units P11 to P14, and the word line WL1 is coupled to the bit lines BL1 BLBL4 and the source line SL1 through the parameter units P11 to P14, respectively. Word line WL2 includes multiple parameter sheets The elements P21 to P24, and the word line WL2 are respectively coupled to the bit lines BL1 to BL4 and the source line SL2 through the parameter units P21 to P24.

The parameter units P11 to P14 of the word line WL1 are arranged in accordance with the parameters a1 to a4 of the filter F1. Similarly, the parameter units P21 to P24 of the word line WL2 are also configured according to the parameters a1 to a4 of the filter F1. However, the parameter units P11 to P14 of the word line WL1 and the parameter units P21 to P21 of the word line WL2 correspond to different arrangements of the parameters a1 to a4. In the present embodiment, the parameter unit P11 corresponds to the parameter a1, the parameter unit P12 corresponds to the parameter a2, the parameter unit P13 corresponds to the parameter a3, and the parameter unit P14 corresponds to the parameter a4. The parameter unit P21 corresponds to the parameter a3, the parameter unit P22 corresponds to the parameter a4, the parameter unit P23 corresponds to the parameter a1, and the parameter unit P24 corresponds to the parameter a2.

As shown in FIG. 4, in an embodiment, each parameter unit P11~P14, P21~P24 may include a transistor M and one (or more) resistors R. When the word line is enabled by the control circuit CTL, the transistor M of the parameter unit on the word line is turned on, so that the word line (assuming the word line WL1 is enabled) can be associated with the bit line BL1. The signal on ~BL4 is operated, and the result SUM1 is output through the source line SL1. The resistance value of the resistor R can be configured according to the parameter corresponding to the parameter unit. For example, the resistance value of the resistance R of the parameter unit P11 can be configured according to the parameter a1, the resistance value of the resistance R of the parameter unit P12 can be configured according to the parameter a2, and so on.

In an embodiment, the configuration of the parameter units P11~P14, P21~P24 may be fixed and pre-programmed in the memory device 20. In another embodiment, The configuration of the parameter units P11~P14, P21~P24 is changeable, and the parameter units P11~P14, P21~P24 can be performed according to a parameter configuration table stored in the memory device 20 or an external device (not shown). program. The external device is, for example, a volatile memory or a non-volatile memory. The parameter configuration table can record the correspondence between the parameter units P11~P14, P21~P24 of the word line WL1~WL2 and the parameters a1~a4 of the filter F1. That is to say, the parameter configuration table records the arrangement of the parameters corresponding to each character line.

Referring to FIG. 5, FIG. 5 is a flow chart showing a method of operating the memory device. This method of operation can be used to operate the memory device 20. The operation method includes steps S501 to S509.

In step S501, an image IMG is provided, wherein the image IMG includes a plurality of pixel data X11~X44.

In step S503, when the first region of the image IMG is processed by the filter F1, the pixel data X11, X21, X12, and X22 of the first region of the image IMG are input to the bit lines BL1 to BL4 by the control circuit CTL. For example, the control circuit CTL inputs the pixel data X11 into the bit line BL1, the pixel data X21 into the bit line BL2, the pixel data X12 into the bit line BL3, and the pixel data X22 into the bit line BL4.

In step S505, one of the control word lines WL1 of the enable word line is operated by the word line WL1 and the pixel data X11, X21, X12, X22 of the first area of the image IMG. In one embodiment, the pixel data X11, X21, X12, and X22 are input to the bit lines BL1 BLBL4 in the form of voltage signals. When the pixel data X11 passes through the resistance R of the parameter unit P11 (the resistance value is set according to the parameter a1) The flow will flow into the source line SL1, and the generated current can be regarded as the result of X11*a1. Similarly, the current generated by the pixel data X21 through the resistance R of the parameter unit P12 (the resistance value is set according to the parameter a2) can be regarded as the result of X21*a2, and so on. The currents generated by the respective pixel data X11, X21, X12, and X22 through the parameter units P11 to P14 are collected on the source line SL1. The result of the above operation SUM1 (here, F11) is obtained by sensing the total current on the source line SL1 by a sense amplifier (not shown). In other words, the output SUM1 of the source line SL1 is represented by X11*a1+X21*a2+X12*a3+X22*a4, that is, F11.

In step S507, when the second region of the image IMG is processed by the filter F1, the pixel data X12, X22 that are overlapped with the first region in the second region of the image IMG are held by the control circuit CTL. The bit lines BL3 and BL4 input the pixel data X13 and X23 which are not overlapped with the first area in the second area of the image IMG, and are input to the bit lines BL1 and BL2. In other words, when updating the input of the bit lines BL1 BL BL4, the control circuit CTL retains the inputs on the bit lines BL3, BL4 (ie, the pixel data X12, X22), and updates only the inputs on the bit lines BL1, BL2. The pixel data X13 is also input to the bit line BL1, and the pixel data X23 is input to the bit line BL2.

In step S509, the word line WL2 is enabled by the control circuit CTL, and the pixel data WL2 and the pixel data X12, X22, X13, and X23 of the second region of the image IMG are operated. Since the parameter units P21, P22, P23, and P24 are configured to correspond to the parameters a3, a4, a1, and a2, respectively, the sum of the currents (SUM2) collected on the source line SL2 can be regarded as X13*a3+X23*a4+X12. The result of *a1+X22*a2 is F12.

In an implementation, the control circuit CTL can determine the word line to be used for operation according to a relative position of the filter F1 and the image IMG. That is to say, the control circuit CTL can select the word line having the correspondingly configured parameter unit according to the reserved input on the bit lines BL1 BL BL4 and the updated input to obtain a correct calculation result. In another embodiment, the control circuit CTL can determine the word line to be used for the operation according to the parameter configuration table.

In an embodiment, when the filter F1 is laterally moved from a first side boundary of the image IMG to a second side boundary of the image IMG, the filter F1 may return to the first side boundary of the image IMG. And move one pixel unit down. At this time, the control circuit CTL can update the inputs on all of the bit lines BL1 BL BL4. For example, after the pixel data X13, X23, X14, and X24 of the image IMG are processed by the filter F1, the filter F1 can be moved to the pixel data X21, X31, X22, and X32 of the image IMG, and the control circuit CTL can The inputs on the bit lines BL1 to BL4 are updated to the pixel data X21, X31, X22, and X32, respectively, and are operated by the word line WL1.

According to the above memory device and operation method, when the filter is moved from the first region of the image to the second region, the control circuit only updates the pixel data that is not overlapped between the first region and the second region of the image to the bit line. The pixel data overlapped between the first area and the second area of the image input on the bit line is not changed, and the operation is performed by the corresponding word line. Therefore, the number of times of data movement required to be performed in the calculation process can be effectively reduced, thereby reducing energy consumption.

Please refer to another embodiment shown in FIGS. 6A and 6B. The memory device of this embodiment can be used to implement the filter F1 and the filter F2. Image IMG and filters F1, F2 as in 6A As shown in the figure, the configuration of the parameter unit of each character line is as shown in Fig. 6B. For a clearer explanation, the control circuit in Fig. 6B is omitted, and the parameter unit is represented by the parameter of the corresponding filter F1, F2 or 0.

In this embodiment, the control circuit of the memory device only needs to update the input on the bit line twice to calculate the product IMG and the convolution of the filters F1 and F2. As shown in FIG. 6B, the pixel data BL1 to BL8 input to the bit lines BL1 to BL8 at the first time T1 are pixel data X11, X21, X31, X41, X12, X22, X32, and X42, respectively. At the second time T2, the control circuit retains the inputs on the bit lines BL5~BL8, and updates the inputs on the bit lines BL1~BL4 to the pixel data X13, X23, X33, X43, respectively. At the third time T3, the control circuit retains the inputs on the bit lines BL1 BLBL4 and updates the inputs on the bit lines BL5 BLBL8 to the pixel data X14, X24, X34, X44, respectively.

At the first time T1, the control circuit enables the word lines WL1, WL3, WL5, WL7, WL9, WL11 to obtain F11, F21, F31, G11, G21, G31. At the second time T2, the control circuit enables the word lines WL2, WL4, WL6, WL8, WL10, WL12 to obtain F12, F22, F32, G12, G22, G32. At the third time T3, the control circuit enables the word lines WL1, WL3, WL5, WL7, WL9, WL11 to obtain F13, F23, F33, G13, G23, G33.

In the present embodiment, by arranging the parameters of the filters F1 and F2 on different word lines in different arrangement manners, not only the energy consumption can be reduced, but also the time required for the calculation can be shortened.

It should be noted that the foregoing embodiments are for illustrative purposes only. The size of the image IMG is 4×4, and the filters F1 and F2 are 2×2, but the present invention. Can be applied to images and filters of any size. In addition, the number of bit lines, word lines, and parameter units of the memory device can be designed according to different needs.

In conclusion, the present invention has been disclosed in the above embodiments, but it is not intended to limit the present invention. A person skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, the scope of the invention is defined by the scope of the appended claims.

Claims (10)

A memory device includes: a plurality of bit lines for receiving a plurality of data of an image; a plurality of word lines, each of the word lines being coupled to a plurality of parameter units, and each of the word lines passing through the The parameter units are coupled to the bit lines, wherein the parameter units of each of the word lines are configured according to a plurality of parameters of a filter, and the configuration of the parameter units of the word lines are different from each other. And a control circuit coupled to the word lines and the bit lines, wherein when a first region of the image is processed by the filter, the control circuit is located in the first region of the image Inputting the data into the bit lines and enabling one of the word lines to perform an operation; when processing a second region of the image with the filter, the control circuit maintains the image located at the image The data overlapping the first area in the second area is input to the bit lines, and the pieces of data that are not overlapped with the first area in the second area of the image are input to the bit lines And enabling one of the other character lines to proceed Count. The memory device of claim 1, wherein the configuration of the parameter units is fixed and pre-programmed in the memory device; or the configuration of the parameter units is changeable, and The parameter units are programmed according to a parameter configuration table stored in the memory device or an external device. The memory device of claim 1, wherein the control circuit determines the word line to be used for operation according to a relative position of the filter and the image or a parameter configuration table. The memory device of claim 1, wherein each of the parameter units includes one or more resistors, and the resistance values of the one or more resistors are determined according to the parameters of the filter. The memory device of claim 1, wherein the configuration of the parameter units corresponding to the word lines corresponds to different arrangements of the parameters of the filter. A method of operating a memory device, comprising: providing an image, wherein the image includes a plurality of data; when processing a first region of the image with a filter, the image is controlled by a control circuit of the memory device The data of the first area is input to a plurality of bit lines of the memory device; and the control circuit enables one of the plurality of word lines of the memory device to use the word line and the image The data of the first area is calculated, wherein each of the word lines is coupled to a plurality of parameter units, and the parameter units of each of the word lines are configured according to a plurality of parameters of the filter, and the The configuration of the parameter units of the word line are different from each other; when the second area of the image is processed by the filter, the control circuit maintains the overlap with the first area in the second area of the image And inputting the data to the bit lines, and inputting the data that is not overlapped with the first area in the second area of the image into the bit lines; and enabling the characters by the control circuit One of the other Performing operations on said plurality of data of the second region and the word line of the other image. The method of operation of claim 6, wherein the configuration of the parameter units is fixed and pre-programmed in the memory device; or the configuration of the parameter units is changeable, and A parameter configuration table stored in the memory device or an external device programs the parameter units. The operating method of claim 6, wherein the control circuit determines the word line to be used for calculation according to a relative position of the filter and the image or a parameter configuration table. The method of operation of claim 6, wherein each of the parameter units includes one or more resistors, and the resistance values of the one or more resistors are determined according to the parameters of the filter. The method of operation of claim 6, wherein the configuration of the parameter units corresponding to the word lines corresponds to a different arrangement of the parameters of the filter.