Deep Feature Representation and Similarity Matrix based Noise Label Refinement Method for Efficient Face Annotation.

A. Suruliandi; A. Kasthuri; S.P. Raja

doi:10.9781/ijimai.2021.05.001

Authors

A. Suruliandi Manonmaniam Sundaranar University
A. Kasthuri Manonmaniam Sundaranar University
S.P. Raja Vellore Institute of Technology University

DOI:

https://doi.org/10.9781/ijimai.2021.05.001

Keywords:

Convolutional Neural Network (CNN), Deep Learning, Face Annotation, Noise Label Refinement, Similarity

Abstract

Face annotation is a naming procedure that assigns the correct name to a person emerging from an image. Faces that are manually annotated by people in online applications include incorrect labels, giving rise to the issue of label ambiguity. This may lead to mislabelling in face annotation. Consequently, an efficient method is still essential to enhance the reliability of face annotation. Hence, in this work, a novel method named the Similarity Matrix-based Noise Label Refinement (SMNLR) is proposed, which effectively predicts the accurate label from the noisy labelled facial images. To enhance the performance of the proposed method, the deep learning technique named Convolutional Neural Networks (CNN) is used for feature representation. Several experiments are conducted to evaluate the effectiveness of the proposed face annotation method using the LFW, IMFDB and Yahoo datasets. The experimental results clearly illustrate the robustness of the proposed SMNLR method in dealing with noisy labelled faces.

Downloads

Download data is not yet available.

References

X. Su, J. Peng, X. Feng, and J. Wu, “Labeling faces with names based on the name semantic network,” Multimedia Tools and Applications, vol. 75, no. 11, pp. 6445-6462, 2016.

D. Wang, S. C. Hoi, Y. He, J. Zhu, T. Mei, and J. Luo, “Retrieval-based face annotation by weak label regularized local coordinate coding,” IEEE transactions on pattern analysis and machine intelligence, vol. 36, no. 3, pp. 550-563, 2014.

C. H. Chen, V. M. Patel, and R. Chellappa, “Learning from ambiguously labeled face images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 7, pp.1653-1667, 2017.

S. C. Huang, M. K. Jiau, and Y. H. Jian, “Optimisation of automatic face annotation system used within a collaborative framework for online social networks,” IET Computer Vision, vol. 10, no. 5, pp. 351-360, 2016.

D. Wang, S. S. Hoi, and Y. He, “A unified learning framework for auto face annotation by mining web facial images,” Proceedings of the 21st ACM international conference on Information and knowledge management, 2012, pp. 1392-1401.

D. Wang, S. C. Hoi, Y. He, and J. Zhu, “Mining weakly labeled web facial images for search-based face annotation,” IEEE Transactions on Knowledge and Data Engineering, vol. 26, no. 1, pp.166-179, 2012.

G. Gao, M. Xu, J. Shen, H. Ma, and S. Yan, “Cast2face: assigning character names onto faces in movie with actor-character correspondence,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 12, pp. 2299-2312, 2015.

M. Zhu, N. Wang, X. Gao,J. Li J, and Z. Li, “Face Photo-Sketch Synthesis via Knowledge Transfer,” In Proceedings of the Twenty-Eight International Joint Conference on Articial Intelligence (IJCAI), 2019, pp. 1048-1054.

B. Shikha, P. Gitanjali, and D. Pawan Kumar, “An Extreme Learning Machine-Relevance Feedback Framework for Enhancing the Accuracy of a Hybrid Image Retrieval System,” International Journal of Interactive Multimedia & Artificial Intelligence, vol. 6, no. 2, pp. 15- 27, 2020, doi: 10.9781/ijimai.2020.01.002.

H. Ito, and H. Koshimizu, “Face image retrieval and annotation based on two latent semantic spaces in fiars,” In Eighth IEEE International Symposium on Multimedia (ISM’06), 2006, pp. 831-836.

A. Kasthuri, and A. Suruliandi, “A survey on face annotation techniques,” In 4th International Conference on Advanced Computing and Communication Systems (ICACCS), IEEE, 2017, pp. 1-9.

Y. Yang, Y. Liu, and J. Liu, “Automatic face image annotation based on a single template with constrained warping deformation,” IET Computer Vision, vol. 7, no. 1, pp. 20-28, 2013.

J. Zhu, S.C. Hoi, and M. R. Lyu, “Face annotation using transductive kernel fisher discriminant,” IEEE Transactions on Multimedia, vol. 10, no. 1, pp. 86-96, 2007.

H. Zitouni, M. F. Bulut, and P. Duygulu, “Recognizing faces in news photographs on the web,” In 2009 24th International Symposium on Computer and Information Sciences, IEEE, 2009, pp. 50-55.

G. B. De Souza, D. F. da Silva Santos, R.G. Pires, A. N. Marana, and J. P. Papa, “Deep texture features for robust face spoofing detection,” IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 64, no. 12, pp. 1397-1401, 2017.

K. Tang, X. Hou, Z. Shao, and L. Ma, “Deep feature selection and projection for cross-age face retrieval,” In 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), IEEE, 2017, pp. 1-7.

A. Kasthuri, A. Suruliandi, and S. P. Raja, “Gabor-oriented local order feature-based deep learning for face annotation,” International Journal of Wavelets, Multiresolution and Information Processing, vol. 17, no. 05, p. 1950032, 2019, doi.org/10.1142/S0219691319500322.

K. Anburajan, S. Andavar, and P. Elango, “An Empirical Evaluation of Name Semantic Network for Face Annotation,” Recent Advances in Computer Science and Communications (Formerly: Recent Patents on Computer Science), vol. 13, no. 4, pp. 557-571, 2020.

V. Rihani, A. Bhandari, and C. P. Singh, “Face Recognition Using Convolution Filters and Neural Networks,” In IC-AI, 2006, pp. 185-190.

N. Venkatesh, M. Subhransu, and R. Manmatha, “Automatic image annotation using deep learning representations”, In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, ACM, 2015.

R. Wang, Y. Xie, J. Yang, L. Xue, M. Hu, and Q. Zhang, “Large scale automatic image annotation based on convolutional neural network,” Journal of Visual Communication and Image Representation, vol. 49, pp. 213-224, 2017.

S. Wu, Y. C. Chen, X. Li, A. C. Wu, J. J. You, and W. S. Zheng, “An enhanced deep feature representation for person re-identification,” In 2016 IEEE winter conference on applications of computer vision (WACV), 2016, pp. 1-8.

X. Wu, R. He, Z. Sun, and T. Tan, “A light CNN for deep face representation with noisy labels,” IEEE Transactions on Information Forensics and Security, vol. 13, no. 11, pp. 2884-2896, 2018.

M. Khari, A. K. Garg, R. G. Crespo, and E. Verdú, “Gesture Recognition of RGB and RGB-D Static Images Using Convolutional Neural Networks,” International Journal of Interactive Multimedia & Artificial Intelligence, vol. 5, no. 7, p. 22, 2019, doi: 10.9781/ijimai.2019.09.002.

M. S. Maheshan, B. S. Harish, and N. Nagadarshan, “A Convolution Neural Network Engine for Sclera Recognition,” International Journal of Interactive Multimedia & Artificial Intelligence, vol. 6, no. 1, pp. 78-83, 2020, doi: 10.9781/ijimai.2019.03.006.

L. Celona, S. Bianco, and R. Schettini, “Fine-grained face annotation using deep multi-task CNN,” Sensors, vol. 18, no. 8, p. 2666, 2018.

W. Jiang, and W. Wang, “Face detection and recognition for home service robots with end-to-end deep neural networks,” In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp. 2232-2236.

X. Sun, P. Wu, and S.C Hoi, “Face detection using deep learning: An improved faster RCNN approach,” Neurocomputing, vol. 299, pp. 42-50, 2018.

Y. Ma, Y. Liu, Q. Xie, and L. Li, “CNN-feature based automatic image annotation method,” Multimedia Tools and Applications, vol. 78, no. 3, pp. 3767-3780, 2019.

O. C. Kurban, T. Yildirim, and A. Bilġ iç, “A multi-biometric recognition ̇ system based on deep features of face and gesture energy image,” In 2017 IEEE International Conference on INnovations in Intelligent SysTems and Applications (INISTA,) 2017, pp. 361-364.

Z. Zeng, S. Xiao, K. Jia, T. H. Chan, S. Gao, D. Xu, and Y. Ma, “Learning by associating ambiguously labeled images,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 708-715.

T. Cour, B. Sapp, and B. Taskar, “Learning from partial labels”, The Journal of Machine Learning Research, vol. 12, pp. 1501-1536, 2011.

X. Liu, S. Li, M. Kan, S. Shan, and X. Chen, “Self-error-correcting convolutional neural network for learning with noisy labels,” In 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 111-117.

X. Su, J. Peng, X. Feng, J. Wu, J. Fan, and L. Cui, “Cross-modality based celebrity face naming for news image collections,” Multimedia Tools and Applications, vol. 73, no. 3, pp. 1643-1661, 2014.

V. Kumar, A. Namboodiri, and C.V Jawahar, “Semi-supervised annotation of faces in image collection,” Signal, Image and Video Processing, vol. 12, no. 1, pp. 141-149, 2018.

V. Maihami, and F. Yaghmaee, “Automatic image annotation using community detection in neighbor images,” Physica A: Statistical Mechanics and its Applications, 507, 2018, pp. 123-132.

M. Zhu, N. Wang, X. Gao, and J. Li, “Deep graphical feature learning for face sketch synthesis,” In Proceedings of the 26th International Joint Conference on Artificial Intelligence, 2017, pp. 3574-3580.

M. Zhu, J. Li, N. Wang and X. Gao, “A Deep Collaborative Framework for Face Photo–Sketch Synthesis,” In IEEE Transactions on Neural Networks and Learning Systems vol. 30, no. 10, pp. 3096-3108, 2019.

X. Wang, S. Wang, J. Wang, H. Shi, and T. Mei, “Co-mining: Deep face recognition with noisy labels,” In Proceedings of the IEEE international conference on computer vision, 2019, pp. 9358-9367.

S.C. Hoi, D. Wang, I.Y. Cheng, E.W. Lin, J. Zhu, Y. He, and C. Miao, “Fans: face annotation by searching large-scale web facial images,” In Proceedings of the 22nd international conference on World Wide Web, ACM, 2013, pp. 317-320.

S. C. Huang, M. K. Jiau and C. A. Hsu, “A High-Efficiency and High-Accuracy Fully Automatic Collaborative Face Annotation System for Distributed Online Social Networks,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 24, no. 10, pp. 1800-1813, 2014.

J. Han, J. Hu, and W. Deng, “Constrained Spectral Clustering on Face Annotation System,” In: Chinese Conference on Pattern Recognition, Springer, Singapore, 2016, pp. 3-12.

S. Xiao, D. Xu, and J. Wu, “Automatic face naming by learning discriminative affinity matrices from weakly labeled images,” IEEE transactions on neural networks and learning systems, vol. 26, no. 10, pp. 2440-2452, 2015.

D. Wang, and A. K. Jain, “Face retriever: Pre-filtering the gallery via deep neural net,” In International Conference on Biometrics (ICB), IEEE, 2015, pp. 473-480.

Q. Cao, Y. Ying, and P. Li, “Similarity metric learning for face recognition,” In Proceedings of the IEEE International Conference on Computer Vision, 2013, pp. 2408-2415.

H. V. Nguyen, and L. Bai, “Cosine similarity metric learning for face verification,” In: Asian conference on computer vision, Springer, Berlin, Heidelberg, 2010, pp. 709-720.

Y. Ji, T. Lin, and H. Zha, “Mahalanobis distance based non-negative sparse representation for face recognition,” In International Conference on Machine Learning and Applications, IEEE, 2009, pp. 41-46.

E. Mostafa, A. M. Ali, and A. A. Farag, “Learning a non-linear combination of Mahalanobis distances using statistical inference for similarity measure,” IET Computer Vision, vol. 9, no. 4, pp. 541-548 2015.

M. Wang, Z. Wang, and J. Li, “Deep convolutional neural network applies to face recognition in small and medium databases,” In 4th International Conference on Systems and Informatics (ICSAI), IEEE, 2017, pp. 1368-1372.