Effect of Modified Convolution on Local Descriptor Based Face Recognition

Effect of Modified Convolution on Local Descriptor based Face Recognition G. N. Girish1,∗ , C. L. Shrinivasa Naika2 and Pradip K. Das2 1 2 Department of Computer Science and Engineering, UBDTCE, Davanagere, Karnataka. Department of Computer Science and Engineering, Indian Institute of Technology, Guwahati, Assam. e-mail: 1 [email protected] Abstract. Face recognition is one of the challenging research area of face image analysis. It is well known that Local Binary Pattern (LBP) based face recognition system’s performance depends on number of grids and scale of the operator. The effect of the large scale operators together with the number of grids and the dependency on labeled binary codes are not explored much. This paper aims to study empirically the effect of number of available binary codes, large scale operator and number of grids for face verification. Face verification experiments were conducted in which Principal Component Analysis (PCA) is applied to reduce the dimensionality of the features extracted using Multi-scale Block LBP (MB-LBP). The experiments were evaluated using ORL, JAFFE and INDIAN benchmark face recognition databases. It is observed that number of available binary labeled codes decreases as the scale of the operator and number of grids are increased when MB-LBP is convoluted with traditional convolution techniques. Moreover, it is noticed that increase in operator scale would decreases the number of available codes which in turn affects the performance. In order to negate this effect, this paper proposes to use Modified Convolution (MC) technique to increase the available binary labeled codes. Experiments on benchmark databases suggested that use of modified convolution method would increase the available codes which in turn increases the performance of the face verification system. Keywords: Face recognition, Local binary patterns, Principal component analysis, Convolution, Modified convolution, Face image analysis. 1. Introduction Face recognition is one of the most active research area from past few decades, as it has important applications in surveillance systems, mug shot searching and video monitoring, biometric identification [1]. Face recognition system performance is influenced by several difficulties like uncontrolled environment, pose variations, facial expressions, illumination variations, high dimensionality, age, etc. Nevertheless, various methods have been proposed which are invariant to these challenges. Face Recognition System (FRS) may be configured into two modes viz: face identification system and face verification system. Face identification system performs 1 : N matching to verify a probe image of a subject against known ∗ 390 Corresponding author. ICISP-2014 Editors: K. R. Venugopal, K. B. Raja and L. M. Patnaik pp. 390–397. Effect of Modified Convolution on Local Descriptor based Face Recognition set of gallery images of subjects. Face verification system performs 1 : 1 matching to verify a subject when probe image of that subject along with the id against the set of gallery images of subject with same id. FRS consists of different components like image acquisition, pre-processing (normalization), feature extraction and classification. Feature extraction techniques may be classified into holistic and local feature based methods depending on whether, feature is extracted from the input image. In holistic methods recognition is performed by deriving global face representation. Holistic methods are Principal Component Analysis (PCA) [2], Linear Discriminant Analysis (LDA) [3,4], Independent Component Analysis (ICA) [5], to name a few. Feature based methods captures most prominent local micro structure representation of face images. Bereta et al. [6] have surveyed LBP based operators used to derive face representation in Face Recognition domain. The authors derived a taxonomy of these operators considering with and without Gabor filtered images. The local descriptors based methods have proved to be better than the holistic approaches. Many researchers did not consider to study the impact of larger scales of the operator but considered different number of girds per image for improving recognition at a fixed scale. This may be due to the fact that the recognition rate would decrease for larger scales and also for more number of grids [7]. The number of grids affects the spatial information and the statistical property of the feature histogram [6]. It is observed that the number of grids and the scales of the operator effects the number of available binary codes as shown in Figure 3. The histogram property depends on the frequency of occurrence of bin values and number of bins in the histogram. The traditional convolution technique used to derive LBP face representation ignores the boundary pixels reducing the number of available codes. This would in turn affect the recognition rate of the FRS. This paper uses Modifies Convolution (MC) [8,9] method to increase the number of available codes, study the impact of larger scales and different number of grids for Face Recognition in verification mode. The rest of the paper is organized as follows, Section 2 briefly introduces the MB-LBP operator and MC technique for MB-LBP operator, followed by the experiments and discussions in Section 3 and the last Section concludes the paper. 2. Proposed Method This paper proposes the use of Modified Convolution (MC) method to convolute the grids of the image to derive concatenated histogram using MB-LBP operator. The MC increases the number of available codes by coding every pixels of the image without ignoring boundary pixels. The following subsections illustrates MB-LBP operator and the MC method. 2.1 MB-LBP operator Multi Scale Block Local Binary Pattern (MB-LBP) is an extension to LBP operator with respect to neighborhood of different operator sizes from 3 × 3 to s × s. MB-LBP code for a pixel is calculated by comparing the average gray values of neighbor blocks with the average gray value of central block, resulting neighbors greater than or equal to average gray value of central block are coded as binary bit 1 otherwise bit 0. These binary digits concatenated from top-left neighbor block in clockwise direction then corresponding decimal number obtained for that concatenated binary digits is used as MB-LBP code, as shown in Figure 1. In MB-LBP, s × s denotes the scale of the MB-LBP operator. The scalar values of averages over blocks can be computed very efficiently from the integral image [10]. 391 G. N. Girish, et al. Figure 1. Illustration of coding a pixel using MB-LBP operator with scale 6 × 6. 2.2 Modified Convolution (MC) Technique The traditional convolution method used with LBP based operators are not convolved from boundary pixels of the face image so as to reduce the influence of boundary pixels on the feature histogram. For example, convolution of MB-LBP operator starts at pixel location (x, y) of the face image with scale s × s where s is the scale of the sub regions of operator. As the s increases, the number of MB-LBP codes decreases which may affect the number of available labeled MB-LBP codes. The available labeled MB-LBP codes can be estimated as [(n − (3 ∗ w − 1)) × (m − (3 ∗ h − 1))] × (number of grids the face image is divided) where, n and m are width and height of the image grid, w and h are width and height of the MB-LBP operator. The number of labeled LBP codes per image directly depends on the scale of the operator which in turn depends on grid size. The available number of codes cause the feature histogram to contain bins with less frequency of gray level values, which in turn reduces discriminative ability of the operator. The statistical property of the histogram is dependent on the number of bins and frequency of the occurrence of each bins. The decrease in the number of LBP codes would affect the feature histogram and in turn the performance of the face recognition system. Hence, the MC technique for MB-LBP operator is proposed which considers labeling the boundary pixels of the image to derive the face representation for face recognition. This technique increases the available labeled MB-LBP codes per image and the scale of the operator does not influence the size of the cropped image and the grid size when MC is used with any LBP based operator to extract the feature histograms. Figure 2, shows the proposed MC technique together with MB-LBP. In Figure 2(a), 6 × 6 MB-LBP operator is placed at pixel (0, 0) of the image and coded as decimal 56. In Figure 2(b), 6×6 size MB-LBP operator is placed at pixel (1, 1) of the image and coded as decimal 252. The bold numerals represent average value of the respective regions. Since the MC codes each pixels of the image the available labeled MBLBP codes could be estimated as [(n − w + 1) × (m − h + 1)] × (number of grids the face image is divided) where, n and m are width and height of the image grid, w and h are width and height of the MB-LBP operator. Figure 3, shows the total number of available discriminant codes using MB-LBP with traditional convolution technique [11] and MBLBP with MC technique (MB-LBP with MC) for an image of 4 grids from ORL dataset. 3. Experiments and Results Experiments were conducted on different benchmark databases to assert that increase in the number of available codes affect the performance of the face recognition system configured in verification mode. 392 Effect of Modified Convolution on Local Descriptor based Face Recognition Figure 2. Modified Convolution: (a) 6 × 6 scale MB-LBP thresholding when region of the operator is out of bound with the image boundary; (b) 6 × 6 scale MB-LBP thresholding when region of the operator partially overlaps with the image boundary. Read ‘Avg’ as average. Figure 3. Total number of available codes for an image of 4 grids from ORL dataset using MB-LBP with traditional convolution technique and MC technique. 393 G. N. Girish, et al. Figure 4. Verification rate at 1% FAR using MahCos distance for different number of grids per image on ORL database. Figure 5. Verification rate at 1% FAR using MahCos distance for different number of grids per image on JAFFE database. Olivetti Research Laboratory (ORL), Japanese Female Facial Expression (JAFFE) and INDIAN face datasets were considered. The verification rate is evaluated using different similarity metrics namely Mahalanobis Cosine Distance (MahCos), Euclidian distance (Euc), Cosine distance (Cos) and City Block distance (CTB). 3.1 Dataset Preparation and Feature Extraction There are two phases involved in face recognition process viz: training phase and testing phase. In both the phases dataset preparation and feature extraction tasks are performed. The training and testing datasets are built for each benchmark databases as explained. ORL dataset contains 10 different images 394 Effect of Modified Convolution on Local Descriptor based Face Recognition Figure 6. Verification rate at 1% FAR using MahCos distance for different number of grids per image on INDIAN face database. each of 40 distinct subjects. All images are pre-normalized to size of 112 × 92 pixels. Randomly 3 images were selected as training set and other 7 images as testing set. The JAFFE database contains 213 images of 7 facial expressions (6 basic facial expressions + 1 neutral) posed by 10 Japanese female models. Each image has been rated on 6 emotion adjectives by 60 Japanese subjects. The images are normalized into size of 150 × 110, using FaceTool [12]. The training set is build by considering 3 images with sad, angry and fear facial expressions and the remaining 7 images of 6 basic facial expression and 1 neutral expression are included in test set for each subject. The INDIAN face database contains images of 40 distinct subjects with 11 different poses for each individual. A subset of 30 subjects are chosen from dataset for evaluation of experiments, which are preprocessed using histogram equalization and resized to 270×200 pixels using FaceTool. Further, the training set is built by selecting 4 images of upper frontal, frontal, looking right and looking left. The testing set consists of 7 images containing different poses and facial expression (smile, laughter, sad/disgust). For each training set images from different datasets a concatenated histogram is obtained using MB-LBP and MC method dividing into 4 girds, each of size 56×46 and 16 girds each of size 28×23 in case of the ORL database on each image. In case of the JAFFE database images were divided into 15 grids, each of size 50×22 and 25 grids, each of size 30×22 to extract concatenated histogram. The INDIAN face training set images are divided into 15 girds, each of size 90 × 40 and 25 grids, each of size 54 × 40 to get concatenated histograms. After computing feature vector of all images in each training set using MB-LBP with MC, these feature histograms are transformed into eigenface space using PCA. Similarly for testing set eigenface are derived following same grid settings. These eigenfaces are then used by the similarity matching using different similarity metrics in verification mode. The PhD (Pretty helpful Development functions) for face recognition toolbox [13,14] is used for conducting all the experiments. 395 G. N. Girish, et al. 3.2 Results and Analysis Experiments were conducted with by varying operator scales (3 × 3, 6 × 6, 9 × 9, 12 × 12, 15 × 15, 18 × 18, 21 × 21, 24 × 24, 27 × 27 and 30 × 30) on all the datasets. The obtained results in terms of FAR (False Acceptance Ratio) using ORL dataset with 4 and 16 girds are shown in Figure 4. It is observed in case of 4 grids that the performance increases with increase in scale of the operator but further increase in the scales would reduces the verification rate of the system (observe from 24 × 24 onwards). This may be the fact that the available number of codes decrease as the scale of the operator increases. In case of 16 grids, the performance is increases as the scale of the operator increases but the verification rate is reduced further increase in the scale of the operator. This may be due to fact that the initial gain in performance is due to increase in spatial information embedded in histogram as the scales increases the number of available codes decreases affecting the histogram property [6]. The maximum performance is obtained for 16 grids is 97.14% at operator size 12 × 12 which is higher than (95.91%) [7]. Figure 5 shows the verification rate for JAFFE dataset with 15 and 25 grids per image. Similar observation as in case of the ORL database can be made. The maximum performance of 100% is obtained. Verification rate for INDIAN face dataset is shown in Figure 6, the similar trend as in the case of ORL and JAFFE verification rate are observed. The maximum performance of 85.83% is obtained. 4. Conclusion In this work, MC technique for MB-LBP operator is proposed to increase available labeled binary code. This work mainly addressed the impact of operator scale number of grids per image on labeled binary code for face verification task. MB-LBP with MC gives better recognition rate on benchmarked datasets when compared to the results obtained in [7] with MB-LBP with traditional convolution technique. Use of the MC methods enabled to analyze the effect of operator scale and number of grids. The verification trend is similar across the database as it depends of variation in available labeled binary codes. Acknowledgments Authors would like to thank VGST, Dept. of IT/BT, Govt. of Karnataka for providing computing facility. Authors gracefully thank Vitomir et al. for providing the PhD toolbox. References [1] M. Bereta, P. Karczmarek, W. Pedrycz and M. Reformat, Local Descriptors in Application to the Aging Problem in Face Recognition, Pattern Recogn., vol. 46 (10), pp. 2634–2646, (2013). [2] M. Turk and A. Pentland, Eigenfaces for Recognition, J. Cognitive Neuroscience, vol. 3(1), pp. 71–86, (1991). [3] P. N. Belhumeur, J. P. Hespanha and D. J. Kriegman, Eigenfaces vs. Fisherfaces: Recognition using Class Specific Linear Projection, In Computer Vision ECCV’96, Springer, pp. 43–58, (1996). [4] D. Swets and J. juyang Weng, Using Discriminant Eigenfeatures for Image Retrieval, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, pp. 831–836, (1996). [5] M. S. Bartlett, H. M. Lades and T. J. Sejnowski, Independent Component Representations for Face Recognition, (1998). [6] M. Bereta, W. Pedrycz and M. Reformat, Local Descriptors and Similarity Measures for Frontal Face Recognition: A comparative analysis, Journal of Visual Communication and Image Representation, vol. 24(8), pp. 1213–1231, (2013). [7] G. N. Girish, D. Pradip K. and N. C. L. Shrinivasa, Face Recognition using MB-LBP and PCA: A Comparative Study, In International Conference on Computer Communication and Informatics (ICCCI-2014), IEEE, Coimbatore INDIA, (2014). 396 Effect of Modified Convolution on Local Descriptor based Face Recognition [8] S. Naika C. L., P. K. Das and S. B. Nair, Asymmetric Region Local Binary Pattern Operator for Persondependent Facial Expression Recognition, In International Conference on Computing, Communication and Applications (ICCCA), pp. 1–5, (2012). [9] S. Naika C. L., S. S. Jha, P. K. Das and S. B. Nair, Automatic Facial Expression Recognition using Extended AR-LBP, In K. R. Venugopal, L. Patnaik (Eds.), Wireless Networks and Computational Intelligence, vol. 292 of Communications in Computer and Information Science, Springer Berlin Heidelberg, pp. 244–252, (2012). [10] P. Viola and M. J. Jones, Robust Real-time Face Detection, International Journal Computer Vision, vol. 57(2), pp. 137–154, (2004). [11] X. Z. ShengCai Liao, Z. Lei, L. Zhang, S. Z. Li, Learning Multi-scale Block Local Binary Patterns for Face Recognition, In S.-W. Lee, S. Z. Li (Eds.), Proceedings of International Conference on Advances in Biometrics ICB 2007, vol. 4642 of Lecture Notes in Computer Science, Springer, pp. 828–837, (2007). [12] A computational tool (February 2014). URL https://fei.edu.br/cet/facepreprocessing− tool.zip [13] V. Štruc and N. Pavešić, Gabor-based Kernel Partial-least-squares Discrimination Features for Face Recognition, Informatica (Vilnius), 20 (1), pp. 115–138, (2009). [14] V. Štruc and N. Pavešić, The Complete Gabor-fisher Classifier for Robust Face Recognition, EURASIP Advances in Signal Processing 2010, vol. 26, (2010). 397