US8891878B2 - Method for representing images using quantized embeddings of scale-invariant image features - Google Patents
Method for representing images using quantized embeddings of scale-invariant image features Download PDFInfo
- Publication number
- US8891878B2 US8891878B2 US13/525,222 US201213525222A US8891878B2 US 8891878 B2 US8891878 B2 US 8891878B2 US 201213525222 A US201213525222 A US 201213525222A US 8891878 B2 US8891878 B2 US 8891878B2
- Authority
- US
- United States
- Prior art keywords
- image
- matrix
- features
- quantization
- random
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2133—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on naturality criteria, e.g. with non-negative factorisation or negative correlation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
Definitions
- This invention relates generally to extracting features from images, and more particularly to using the features to query databases.
- Augmented reality is a significant application to leverage recent advances in computing devices, and more particularly mobile devices.
- the mobile devices can include clients such as mobile telephones (cell phone), personal digital assistants (PDA), a tablet computers and the like.
- clients such as mobile telephones (cell phone), personal digital assistants (PDA), a tablet computers and the like.
- PDA personal digital assistants
- Such devices have limited memory, processing, communication, and power resources.
- augmented reality applications present a special chalange in mobile environments.
- the devices can acquire images or videos using either a camera or network.
- the images can be of real world scenes, or synthetic data, such as computer graphic images or animation videos.
- the devices can augment the experience for a user by overlaying useful information on the images or videos.
- the useful information can be in the form of metadata.
- the metadata can be information about a historical landmark, nutrition information about a food item, or a product identified with a (linear or matrix) bar in an image.
- SIFT scale-invariant feature transform
- SURF speeded up robust feature
- GIST G-invariant feature transform
- SIFT and SURF acquire local details in an image, and therefore, have been used to match local features or patches. They can also be used for image matching and retrieval by combining hypotheses from several patches using, for example, the popular “Bag-of-Features” approach.
- GIST acquires global properties of the image and has been used for image matching.
- a GIST vector is an abstract representation of a scene that can activate a memory representations of scene categories, e.g., buildings, landscapes, landmarks, etc.
- SIFT has the best performance in the presence of common image deformations, such as translation, rotation, and a limited amount of scaling.
- the SIFT feature vector for a single salient point in an image is a real-valued, unit-norm 128-dimensional vector. This demands a prohibitively large bit rate required for the client to transmit the SIFT features to a database server for the purpose of image matching, especially if features from several salient points are needed for reliable matching.
- BoostSSC Similarity Sensitive Coding
- RBM Restricted Boltzmann Machines
- PCA Principle Component Analysis
- LDA Linear Discriminant Analysis
- a low-bit rate descriptor uses Compressed Histogram of Gradients (CHoG) specifically for augmented reality applications.
- CHoG Compressed Histogram of Gradients
- gradient distributions are explicitly compressed, resulting in low-rate scale invariant descriptors.
- LSH Locality Sensitive Hashing
- Random projections are determined from scale invariant features followed by one-bit quantization.
- the resulting descriptors are used to establish visual correspondences between images acquired in a wireless camera network.
- the same technique can be applied to content-based image retrieval, and a bound is obtained for the minimum number of bits needed for a specified accuracy of nearest neighbor search.
- those methods do not consider a tradeoff between dimensionality reduction and quantization levels.
- the embodiments of the invention provide a method for representing an image by extracting featrures from the image.
- the image is a query image.
- the representation can be performed in a client, and the extracted feaures can be transmitted to a server to search a databases of similarly represented images for matching. Metadata of matching images can be returned to the client.
- the method extracts scale invariant features from the query image. Then, a small number of quantized random projections of the features are determined and quantized. The quantized projections are used to search a database at the server storing images in a similar form.
- the server performs a nearest neighbor search in a low-dimensional subspace of the quantized random projections, and returns metadata corresponding to the query image.
- the embodiments invention allow a trade-off that balances the number of random projections and the number of bits, i.e., the quantizing levels, used to store the projection.
- the method achieves a retrieval accuracy up to 94%, while requiring a mobile client device to transmit only 2.5 kB to the server for each image. This as a significant improvement over 1-bit quantization schemes known in the art.
- the method is particularly suited for mobile client applications that are resource constrained.
- FIG. 1A is a flow diagram of a method for representing an image by a client according to embodiments of the invention
- FIG. 1B is pseudocode of the method for representing the image according to embodiments of the invention.
- FIG. 2 is pseudocode of the method for representing images at a server according to embodiments of the invention
- FIG. 3 is pseudocode of the method for searching a database according to embodiments of the invention.
- FIG. 4 is a schematic of operations of the embodiments.
- the embodiments of the invention provide a method for extracting features from a query image in a client.
- the features can be transmitted to a server, and used to search a database to retrieve similar images, and image specific metadata that are appropriate in an augmented reality application.
- FIG. 1 shows a method for representing an image 101 .
- Features 102 are extracted 110 from the image. It is understood that the method can be applied to a sequence of images, as in a video.
- the images can be of real world scenes, or synthetic data.
- the images can be acquired directly by an embedded camera, or downloaded via a network.
- the features are scale-invariant.
- the features are multiplied 120 by a matrix 103 of random entries to produce a matrix of random projections 121 .
- the matrix of random projections is quantized 130 to produce a matrix of quantization indices 104 that represent the image.
- the indices matrix can re-arranged into a query vector.
- the query vector is transmitted to the server, which searches 140 the database 151 for similar images, and retrieves the metadata 152 for the client.
- the invention is based, in part, on a low-dimensional embedding of scale-invariant features extracted from images.
- the use of embeddings is justified by the following result, which forms a starting point of our theoretical development.
- This lemma means that a small set of points, e.g., features, in a high-dimensional space can be embedded into a space which has a substantially lower dimension, while still preserving the distances between the points.
- mapping ⁇ :R d ⁇ R k computable in randomized polynomial time, such that for all u,v ⁇ X, e.g., pixels at locations (u,v) in image X 101 .
- the dimensionality k of the points in the range of ⁇ is independent of the dimensionality of points in X and proportional to the logarithm of number of points in X. Since k increases as ln n, the Johnson-Lindenstrauss Lemma establishes a dimensionality reduction result, in which any set of n points (features) in d-dimensional Euclidean space can be embedded into k-dimensional Euclidean space. This is extremely beneficial for querying very large databases, i.e., a large n) with several attributes, i.e., a large d.
- One way to construct the embedding function ⁇ is to project the points (features) from X onto a spherically random hyperplane passing through the origin. In practice, this is accomplished by multiplying the data vector with the matrix 103 of independent and identically distributed (i.i.d.) if each random variables.
- the random matrix with i.i.d. N(0,1) entries provides the distance-preserving properties in Theorem 1 with high probability. The following result makes this notion precise.
- ⁇ (u) is a k-dimensional embedding of a d-dimensional vector.
- Theorem 2 holds for other distributions on a(i,j), besides the normal distribution. In what follows, however, we consider only the normal Gaussian distribution.
- Proposition 1 For real numbers ⁇ >0 and ⁇ (0,1), let there be a positive integer k that satisfies (1).
- a matrix A ⁇ R k ⁇ d whose entries a(i,j) are drawn i.i.d. from a N(0,1) distribution.
- q(w) be an uniform scalar quantizer with step size ⁇ applied independently to each element of w. Then, for all u,v ⁇ X, the mapping
- g ⁇ ( u ) 1 k ⁇ q ⁇ ( Au ) satisfies (1 ⁇ ) ⁇ u ⁇ v ⁇ g ( u ) ⁇ g ( v ) ⁇ (1+ ⁇ ) ⁇ u ⁇ v ⁇ + ⁇ with probability at least as large as 1 ⁇ n ⁇ .
- Tthe accuracy of the quantized embedding depends on the scalar quantization interval ⁇ . This, in turn, depends on the design of the scalar quantizer and the bit-rate B used to encode each coefficient.
- non-uniform quantization There are two additional issues: non-uniform quantization, and saturation.
- a non-uniform scalar quantizer tuned to the distribution of the projections, could improve the performance of embedding.
- the quantization still suffers from the same trade-off between number of bits per measurement and the number of projections.
- adjusting the saturation rate of the uniform quantizer is a way to tune the quantizer to the distribution of the projections. Reducing the range of the quantizer S, reduces the quantization interval ⁇ and the ambiguity due to quantization.
- a user of a mobile client device wants to find out more information about a query image, such as history of a monument, or nutrition information for a food item.
- the user can acquire the query image with a camera in the device, or down-load the image via a network.
- the mobile device having limited resource, uses a low complexity method to generate a representation of the image to be transmitted.
- the bandwidth required to transmit the representation is low.
- the server has sufficient resources to quickly process the query, and transmit the metadata to the client.
- the method initialize the random projection matrix A ⁇ R k ⁇ d with elements a(i,j): N(0,1). Images J 1 , J 2 . . . , J t for s real or synthetic scenes are acquired, where s ⁇ t, and generate the metadata D i , i ⁇ 1, 2, . . . , s ⁇ for each object.
- the scale-invariant feature extraction method is applied to each image J i , i ⁇ 1, 2, . . . , t ⁇ , which extracts several d-dimensional features from each image.
- the number of features extracted from each image need not be equal.
- the procedure is performed by the mobile device using the same random projection matrix A as the server.
- the distribution of the a(i,j) can be approximated by a pseudorandom number generator.
- the seed of the pseudorandom number generator is sent to the mobile device as a one-time update, or included as part of the client software installation. The seed ensures that the mobile device and the server generate the same realization of the matrix A by having identical seeds.
- each q(i,j) is represented by ⁇ log 2 L ⁇ bits.
- the computational complexity at the mobile device is primarily determined by the scale-invariant feature extraction method, and one matrix multiplication.
- the number of bits transmitted by the client to the server is kM ⁇ log 2 L ⁇ bits.
- the client can reduce the number of random projections k, the quantization levels L, or the number of features M extracted from the query image.
- FIG. 3 shows the pseudocode for the approximate nearest neighbor search 140 performed by the server. Briefly, nearest neighbors are found in the space of the quantized embeddings of image descriptors. The nearest neighbors are aggregated to obtain the matching image, and thence the associated metadata 152 .
- the query image 201 acquired.
- the server uses random projection 120 matching with representations of images in the database 151 .
- Image indices 401 are obtained as a function of the number of matching occurances. The index with the highest number of occurances is used to locate the associated metadata 152 .
- the nearest neighbor procedure initializes an s-dimensional histogram vector h to all zeros.
- Receive Q [q 1 , q 2 , . . . , q M ] 104 representing the query image 101 .
- the embodiments of the invention enable randomized embeddings of scale invariant image features for image searching while reducing resource consumption at the client compared with directly using the scale invariant features as in the prior art.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Processing Or Creating Images (AREA)
- Image Analysis (AREA)
Abstract
Description
(1−ε)∥u−v∥ 2≦|∥ƒ(u)−ƒ(v)∥2≦(1+ε)∥u−v∥ 2
satisfies the distance preserving property in
satisfies
(1−ε)∥u−v∥−Δ≦∥g(u)−g(v)∥≦(1+ε)∥u−v∥+Δ
with probability at least as large as 1−n−β.
as in
(1−ε)∥u−v∥≦∥ƒ(u)−ƒ(v)∥≦(1+ε)∥u−v∥.
∥g(u)−g(v)∥≦∥g(u)−ƒ(u)∥+∥ƒ(u)−ƒ(v)∥+∥ƒ(v)−g(v)∥≦Δ/2+(1+ε)∥u−v∥+Δ/2.
The proof for the left half is similar.
Claims (18)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/525,222 US8891878B2 (en) | 2012-06-15 | 2012-06-15 | Method for representing images using quantized embeddings of scale-invariant image features |
US13/733,517 US8768075B2 (en) | 2011-11-08 | 2013-01-03 | Method for coding signals with universal quantized embeddings |
JP2013100965A JP5950864B2 (en) | 2012-06-15 | 2013-05-13 | A method for representing images using quantized embedding of scale-invariant image features |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/525,222 US8891878B2 (en) | 2012-06-15 | 2012-06-15 | Method for representing images using quantized embeddings of scale-invariant image features |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/291,384 Continuation-In-Part US8837727B2 (en) | 2011-11-08 | 2011-11-08 | Method for privacy preserving hashing of signals with binary embeddings |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/733,517 Continuation-In-Part US8768075B2 (en) | 2011-11-08 | 2013-01-03 | Method for coding signals with universal quantized embeddings |
Publications (2)
Publication Number | Publication Date |
---|---|
US20130336588A1 US20130336588A1 (en) | 2013-12-19 |
US8891878B2 true US8891878B2 (en) | 2014-11-18 |
Family
ID=49755988
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/525,222 Active 2032-12-12 US8891878B2 (en) | 2011-11-08 | 2012-06-15 | Method for representing images using quantized embeddings of scale-invariant image features |
Country Status (2)
Country | Link |
---|---|
US (1) | US8891878B2 (en) |
JP (1) | JP5950864B2 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10878291B2 (en) | 2019-03-28 | 2020-12-29 | International Business Machines Corporation | Visually guided query processing |
US10885098B2 (en) | 2015-09-15 | 2021-01-05 | Canon Kabushiki Kaisha | Method, system and apparatus for generating hash codes |
US11341277B2 (en) * | 2018-04-20 | 2022-05-24 | Nec Corporation | Method and system for securing machine learning models |
Families Citing this family (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6041789B2 (en) * | 2013-01-03 | 2016-12-14 | 三菱電機株式会社 | Method for encoding an input signal |
US9158807B2 (en) * | 2013-03-08 | 2015-10-13 | International Business Machines Corporation | Fast distributed database frequency summarization |
US10386435B2 (en) | 2013-11-06 | 2019-08-20 | The Research Foundation For The State University Of New York | Methods and systems for fast auto-calibrated reconstruction with random projection in parallel MRI |
JP6456031B2 (en) * | 2014-03-25 | 2019-01-23 | キヤノン株式会社 | Image recognition apparatus, image recognition method, and program |
US20170039198A1 (en) * | 2014-05-15 | 2017-02-09 | Sentient Technologies (Barbados) Limited | Visual interactive search, scalable bandit-based visual interactive search and ranking for visual interactive search |
US20150331908A1 (en) | 2014-05-15 | 2015-11-19 | Genetic Finance (Barbados) Limited | Visual interactive search |
US9432702B2 (en) * | 2014-07-07 | 2016-08-30 | TCL Research America Inc. | System and method for video program recognition |
US9337815B1 (en) * | 2015-03-10 | 2016-05-10 | Mitsubishi Electric Research Laboratories, Inc. | Method for comparing signals using operator invariant embeddings |
US10394777B2 (en) | 2015-09-24 | 2019-08-27 | Google Llc | Fast orthogonal projection |
CN107636639B (en) * | 2015-09-24 | 2021-01-08 | 谷歌有限责任公司 | Fast orthogonal projection |
KR102221118B1 (en) * | 2016-02-16 | 2021-02-26 | 삼성전자주식회사 | Method for extracting feature of image to recognize object |
US10909459B2 (en) | 2016-06-09 | 2021-02-02 | Cognizant Technology Solutions U.S. Corporation | Content embedding using deep metric learning algorithms |
US10579688B2 (en) * | 2016-10-05 | 2020-03-03 | Facebook, Inc. | Search ranking and recommendations for online social networks based on reconstructed embeddings |
US10984045B2 (en) | 2017-05-24 | 2021-04-20 | International Business Machines Corporation | Neural bit embeddings for graphs |
WO2019040136A1 (en) * | 2017-08-23 | 2019-02-28 | Google Llc | Multiscale quantization for fast similarity search |
US10755144B2 (en) | 2017-09-05 | 2020-08-25 | Cognizant Technology Solutions U.S. Corporation | Automated and unsupervised generation of real-world training data |
US10755142B2 (en) | 2017-09-05 | 2020-08-25 | Cognizant Technology Solutions U.S. Corporation | Automated and unsupervised generation of real-world training data |
US11574201B2 (en) | 2018-02-06 | 2023-02-07 | Cognizant Technology Solutions U.S. Corporation | Enhancing evolutionary optimization in uncertain environments by allocating evaluations via multi-armed bandit algorithms |
CN116384497B (en) * | 2023-05-11 | 2023-08-25 | 深圳量旋科技有限公司 | Reading and writing system, related method, device and equipment for quantum computing experimental result |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130028516A1 (en) * | 2009-09-01 | 2013-01-31 | Children"s Medical Center Corporation | Image registration methods and apparatus using random projections |
US8542869B2 (en) * | 2010-06-02 | 2013-09-24 | Dolby Laboratories Licensing Corporation | Projection based hashing that balances robustness and sensitivity of media fingerprints |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4766197B2 (en) * | 2009-01-29 | 2011-09-07 | 日本電気株式会社 | Feature selection device |
JP5347897B2 (en) * | 2009-10-15 | 2013-11-20 | 株式会社リコー | Annotation apparatus, method and program |
JP2011150541A (en) * | 2010-01-21 | 2011-08-04 | Sony Corp | Learning apparatus, learning method and program |
-
2012
- 2012-06-15 US US13/525,222 patent/US8891878B2/en active Active
-
2013
- 2013-05-13 JP JP2013100965A patent/JP5950864B2/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130028516A1 (en) * | 2009-09-01 | 2013-01-31 | Children"s Medical Center Corporation | Image registration methods and apparatus using random projections |
US8542869B2 (en) * | 2010-06-02 | 2013-09-24 | Dolby Laboratories Licensing Corporation | Projection based hashing that balances robustness and sensitivity of media fingerprints |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10885098B2 (en) | 2015-09-15 | 2021-01-05 | Canon Kabushiki Kaisha | Method, system and apparatus for generating hash codes |
US11341277B2 (en) * | 2018-04-20 | 2022-05-24 | Nec Corporation | Method and system for securing machine learning models |
US10878291B2 (en) | 2019-03-28 | 2020-12-29 | International Business Machines Corporation | Visually guided query processing |
Also Published As
Publication number | Publication date |
---|---|
JP2014002723A (en) | 2014-01-09 |
US20130336588A1 (en) | 2013-12-19 |
JP5950864B2 (en) | 2016-07-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8891878B2 (en) | Method for representing images using quantized embeddings of scale-invariant image features | |
Guo et al. | Content-based image retrieval using features extracted from halftoning-based block truncation coding | |
Chandrasekhar et al. | Compressed histogram of gradients: A low-bitrate descriptor | |
Duan et al. | Overview of the MPEG-CDVS standard | |
Chen et al. | Tree histogram coding for mobile image matching | |
Chen et al. | Residual enhanced visual vectors for on-device image matching | |
Sánchez et al. | High-dimensional signature compression for large-scale image classification | |
Duan et al. | Compact descriptors for visual search | |
US9256617B2 (en) | Apparatus and method for performing visual search | |
US20130039566A1 (en) | Coding of feature location information | |
US8774509B1 (en) | Method and system for creating a two-dimensional representation of an image based upon local representations throughout the image structure | |
US20170026665A1 (en) | Method and device for compressing local feature descriptor, and storage medium | |
US20180341805A1 (en) | Method and Apparatus for Generating Codebooks for Efficient Search | |
Chandrasekhar et al. | Survey of SIFT compression schemes | |
Li et al. | Quantized embeddings of scale-invariant image features for mobile augmented reality | |
Chen et al. | A hybrid mobile visual search system with compact global signatures | |
Chandrasekhar et al. | Feature matching performance of compact descriptors for visual search | |
Rane et al. | Quantized embeddings: An efficient and universal nearest neighbor method for cloud-based image retrieval | |
Li et al. | Online variable coding length product quantization for fast nearest neighbor search in mobile retrieval | |
Chandrasekhar et al. | Quantization schemes for low bitrate compressed histogram of gradients descriptors | |
Wu et al. | Codebook-free compact descriptor for scalable visual search | |
Guo et al. | Parametric and nonparametric residual vector quantization optimizations for ANN search | |
Boufounos et al. | Dimensionality reduction of visual features for efficient retrieval and classification | |
Du et al. | A Low Overhead Progressive Transmission for Visual Descriptor Based on Image Saliency. | |
Khapli et al. | Compressed domain image retrieval using thumbnails of images |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MITSUBISHI ELECTRIC RESEARCH LABORATORIES, INC., M Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:RANE, SHANTANU;BOUFOUNOS, PETROS;LI, MU;SIGNING DATES FROM 20120615 TO 20120627;REEL/FRAME:028449/0482 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551) Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |