A Comprehensive Review of Facial Beauty Prediction Using Multi-task Learning and Facial Attributes

Ali H. Ibrahem; Adnan M. Abdulazeez

doi:10.14500/aro.11850

Ali H. Ibrahem 1 Department of IT, Technical College of Informatics - Akre, Akre University for Applied Sciences, Kurdistan Region – F.R. Iraq https://orcid.org/0009-0000-2829-4707
Adnan M. Abdulazeez Technical College of Engineering, Duhok Polytechnic University, Kurdistan Region – F.R. Iraq https://orcid.org/0000-0002-4357-7331

Keywords: Convolutional Neural Network, Facial beauty prediction, Facial attractiveness, Human Rater

Abstract

Beauty multi-task prediction from facial attributes is a multidisciplinary challenge at the intersection of computer vision, machine learning, and psychology. Despite the centrality of beauty in human perception, its subjective nature—shaped by individual, social, and cultural influences—complicates its computational modeling. This review addresses the pressing need to develop robust and fair predictive models for facial beauty assessments by leveraging deep learning techniques. Using facial attributes such as symmetry, skin complexion, and hairstyle, we explore how these features influence perceptions of attractiveness. The study adopts advanced computational methodologies, including convolutional neural networks and multi-task learning frameworks, to capture nuanced facial cues. A comprehensive analysis of publicly available datasets reveals critical gaps in diversity, biases, and ground truth annotation for training effective models. We further examine the methodological challenges in defining and measuring beauty, such as data imbalances and algorithmic fairness. By synthesizing insights from psychology and machine learning, this work highlights the potential of interdisciplinary approaches to enhance the reliability and inclusivity of automated beauty prediction systems.

Downloads

Download data is not yet available.

Author Biographies

Ali H. Ibrahem, 1 Department of IT, Technical College of Informatics - Akre, Akre University for Applied Sciences, Kurdistan Region – F.R. Iraq

Ali H. Ibrahem is an Assistant Lecturer at the Ministry of Education. He earned his B.Sc. degree in Computer Science from Mosul University, Iraq, and his M.Sc. degree in Information Technology from Dr. Babasaheb Ambedkar University, India. Currently, he is a Ph.D. student at Akre University for Applied Sciences. His research interests include machine learning, deep learning, blockchain, and web applications.

Adnan M. Abdulazeez, Technical College of Engineering, Duhok Polytechnic University, Kurdistan Region – F.R. Iraq

Adnan M. Abdulazeez is a Professor at the Technical College of Engineering. He got the B.Sc. degree in Electrical and Electronic Engineering, the M.Sc. degree in Control and Computer Engineering, and the Ph.D. degree in Computer Engineering. His research interests are in machine learning, deep learning, and data mining. Dr. Adnan is a member of the IEEE society.

References

Aarabi, P., Hughes, D., Mohajer, K., and Emami, M., 2001. The Automatic Measurement of Facial Beauty. In: 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat. No. 01CH37236). IEEE.

Ahmadimehr, S., and Moridani, M.K., 2020. Evaluating facial attractiveness through proportions analysis based on geometric features. Journal of Image Processing & Pattern Recognition Progress, 7(2), pp.20-26.

Altwaijry, H., and Belongie, S., 2013. Relative Ranking of Facial Attractiveness. In: 2013 IEEE Workshop on Applications of Computer Vision (WACV). IEEE.

Bougourzi, F., Dornaika, F., Barrena, N., Distante, C., and Taleb-Ahmed, A., 2023. CNN based facial aesthetics analysis through dynamic robust losses and ensemble regression. Applied Intelligence, 53(9), pp.10825-10842.

Cao, K., Choi, K.N., Jung, H., and Duan, L., 2020. Deep learning for facial beauty prediction. Information, 11(8), p.391.

Chen, F., Xu, Y., and Zhang, D., 2014. A new hypothesis on facial beauty perception. ACM Transactions on Applied Perception, 11(2), pp.1-20.

Chen, Y., Zhang, Y., Huang, Z., Luo, Z., and Chen, J., 2021. CelebHair: A New Large-scale Dataset for Hairstyle Recommendation Based on CelebA. In: International Conference on Knowledge Science, Engineering and Management. Springer.

Cowen, A.S., Keltner, D., Schroff, F., Jou, B., Adam, H., and Prasad, G., 2021. Sixteen facial expressions occur in similar contexts worldwide. Nature, 589(7841), pp.251-257.

Eisenthal, Y., Dror, G., and Ruppin, E., 2006. Facial attractiveness: Beauty and the machine. Neural Computation, 18(1), p.119-142.

Fan, D., Kim, H., Kim, J., Liu, Y., and Huang, Q., 2019. Multi-task learning using task dependencies for face attributes prediction. Applied Sciences, 9(12), p.2535.

Favorskaya, M.N., and Pakhirka, A.I., 2023. Age-Group Estimation of Facial Images Using Multi-task Ranking CNN. In: International KES Conference on Intelligent Decision Technologies. Springer.

Gan, J., Jiang, K., Tan, H., and He, G., 2020. Facial beauty prediction based on lighted deep convolution neural network with feature extraction strengthened. Chinese Journal of Electronics, 29(2), pp.312-321.

Gan, J., Luo, H., Xiong, J., Xie, X., Li, H., and Liu, J., 2023. Facial beauty prediction combined with multi-task learning of adaptive sharing policy and attentional feature fusion. Electronics, 13(1), p.179.

Gan, J., Xiang, L., Zhai, Y., Mai, C., He, G., Zeng, J., Bai, Z., Labati, R., Piuri, V., and Scotti, F., 2020. 2M BeautyNet: Facial beauty prediction based on multitask transfer learning. IEEE Access, 8, pp.20245-20256.

Gan, J., Xie, X., Zhai, Y., He, G., Mai, C., and Luo, H., 2023. Facial beauty prediction fusing transfer learning and broad learning system. Soft Computing, 27(18), pp.13391-13404.

Gao, L., Li, W., Huang, Z., Huang, D., and Wang, Y., 2018. Automatic Facial Attractiveness Prediction by Deep Multi-task Learning. In: 2018 24th International Conference on Pattern Recognition (ICPR). IEEE.

Gray, D., Yu, K., Xu, W., and Gong, Y., 2010. Predicting Facial Beauty Without Landmarks. In: Computer Vision-ECCV 2010: 11th European Conference on Computer Vision. Proceedings, Part VI 11. Springer,

Heraklion, Crete, Greece. Grgic, M., Delac, K., and Grgic, S., 2011. SCface-surveillance cameras face database. Multimedia Tools and Applications, 51, pp.863-879.

Gunes, H., and Piccardi, M., 2006. Assessing facial beauty through proportion analysis by image processing and supervised learning. International Journal of Human-computer Studies, 64(12), pp.1184-1199.

Jamoliddin, U., and Yoo, J.H., 2022. Age and gender classification with small scale cnn. The Journal of the Korea Institute of Electronic Communication Sciences, 17(1), pp.99-104.

Kalyta, O., Krakb, I., Barmaka, O., Wojcikd, W., and Radiuk, P., 2022. Method of Facial Geometric Feature Representation for Information Security Systems. In: 3rd International Workshop on Intelligent Information Technologies & Systems of Information Security. Khmelnytskyi, Ukraine.

Lebedeva, I., Guo, Y., and Ying, F., 2021. Transfer learning adaptive facial attractiveness assessment. Journal of Physics: Conference Series, 1922, p.012004.

Lebedeva, I., Guo, Y., and Ying, F., 2022. MEBeauty: Amulti-ethnic facial beauty dataset in-the-wild. Neural Computing and Applications, 34, pp.14169-14183.

Lebedeva, I., Guo, Y., and Ying, F., 2023. Personalized facial beauty assessment: A meta-learning approach. The Visual Computer, 39(3), pp.1095-1107.

Liang, L., Xie, D., Jin, L., Xu, J., Li, M., and Lin, L., 2017. Region-aware Scattering Convolution Networks for Facial Beauty Prediction. In: 2017 IEEE International Conference on Image Processing (ICIP). IEEE.

Lin, L., Shen, Z., Yin, J.L., Liu, Q., Yu, Y., and Chen, W., 2023. MetaFBP: Learning to Learn High-Order Predictor for Personalized Facial Beauty Prediction. In: Proceedings of the 31st ACM International Conference on Multimedia.

Lin, Y., Zhibin, G., Zhang, S., Li, L., and Huang, L., 2021. Multi-Task Network and Optimization for Face Detection and Attribute Analysis. In: 2021 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA). IEEE.

Liu, Z., Luo, P., Wang, X., and Tang, X., 2015. Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision. IEEE.

Mao, L., Yan, Y., Xue, J.H., and Wang, H., 2020. Deep multi-task multi-label CNN for effective facial attribute classification. IEEE Transactions on Affective Computing, 13(2), pp.818-828.

Moridani, M.K., Jamiee, N., and Saghafi, S., 2023. Human-like evaluation by facial attractiveness intelligent machine. International Journal of Cognitive Computing in Engineering, 4, pp.160-169.

Mu, Y., 2013. Computational facial attractiveness prediction by aesthetics-aware features. Neurocomputing, 99, pp.59-64. Nguyen, T.V., Liu, S., Ni, B., Tan, J., Rui, Y., and Yan, S., 2013.

Towards decrypting attractiveness via multi-modality cues. ACM Transactions on Multimedia Computing, Communications, and Applications, 9(4), p.28.

Panić, N., Marjanović, M., and Bezdan, T., 2024. Ethnic representation matters: Investigating bias in facial age prediction models. Mathematics, 12(15), pp.1-30.

Rohani, M., Farsi, H., and Mohamadzadeh, S., 2023. Deep multi-task convolutional neural networks for efficient classification of face attributes. International Journal of Engineering, 36(11), pp.2102-2111.

Saeed, J.N., Abdulazeez, A.M., and Ibrahim, D.A., 2022. FIAC-Net: Facial Image Attractiveness Classification based on Light Deep Convolutional Neural Network. In: 2022 Second International Conference on Computer Science, Engineering and Applications (ICCSEA). IEEE.

Saeed, J.N., and Abdulazeez, A.M., 2021. Facial beauty prediction and analysis based on deep convolutional neural network: Areview. Journal of Soft Computing and Data Mining, 2(1), pp.1-12.

Saeed, J.N., and Abdulazeez, A.M., 2022. 2D Facial Images Attractiveness Assessment Based on Transfer Learning of Deep Convolutional Neural Networks. In: 2022 4th International Conference on Advanced Science and Engineering (ICOASE). IEEE.

Sagonas, C., Antonakosa, E., Tzimiropoulosb, G., Zafeirioua, S., and Pantic, M., 2016. 300 faces in-the-wild challenge: Database and results. Image and Vision Computing, 47, pp.3-18.

Savchenko, A.V., 2021. Facial Expression and Attributes Recognition Based on Multi-task Learning of Lightweight Neural Networks. In: 2021 IEEE 19th International Symposium on Intelligent Systems and Informatics (SISY). IEEE, pp.119-124.

Schmid, K., Marx, D., and Samal, A., 2008. Computation of a face attractiveness index based on neoclassical canons, symmetry, and golden ratios. Pattern Recognition, 41(8), pp.2710-2717.

Sekhar, J.C., Joel Josephson, P., Chinnasamy, A., Maheswari, V., Sankar, V., and Kalangi, R.R., 2024. Automated face recognition using deep learning technique and center symmetric multivariant local binary pattern. Neural Computing and Applications, pp.1-19.

Štěpánek, L., Kasal, P., and Mestak, J., 2018. Evaluation of Facial Attractiveness for Purposes of Plastic Surgery Using Machine-learning Methods and Image Analysis. In: 2018 IEEE 20th International Conference on e-Health Networking, Applications and Services. IEEE.

Taherkhani, F., Dabouei, A., Soleymani, S., Dawson, J., and Nasrabadi, N.M., 2021. Tasks structure regularization in multi-task learning for improving facial attribute prediction. arXiv preprint arXiv:2108.04353.

Vahdati, E., and Suen, C.Y., 2021. Facial beauty prediction from facial parts using multi-task and multi-stream convolutional neural networks. International Journal of Pattern Recognition and Artificial Intelligence, 35(12), p.2160002.

Wang, F., Han, H., Shan, S., and Chen, X., 2017. Deep Multi-task Learning for Joint Prediction of Heterogeneous Face Attributes. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). IEEE.

Weng, N., Wang, J., Li, A., and Wang, Y., 2021. Two-stream Temporal Convolutional Network for Dynamic Facial Attractiveness Prediction. In: 2020 25th International Conference on Pattern Recognition (ICPR). IEEE.

Xiao, Q., Wu, Y., Wang, D., Yang, Y.L., and Jin, X., 2021. Beauty3DFaceNet: Deep geometry and texture fusion for 3D facial attractiveness prediction. Computers & Graphics, 98, pp.11-18.

Xie, D., Liang, L., Jin, L., Xu, J., and Li, M., 2015. Scut-fbp: A Benchmark Dataset for Facial Beauty Perception. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics. IEEE.

Xie, Y., Wang, K., Meng, J., Yue, J., Meng, L., Yi, W., Jung, T.P., Xu, M., and Ming, D., 2023. Cross-dataset transfer learning for motor imagery signal classification via multi-task learning and pre-training. Journal of Neural Engineering, 20(5), p.056037.

Xu, L., Xiang, J., and Yuan, X., 2018. CRnet: Classification and Regression Neural Network for Facial Beauty Prediction. In: Pacific Rim Conference on Multimedia. Springer, Cham.

Xu, M., Chen, F., Li, L., Shen, C., Lv, P., Zhou, B., and Ji, R., 2018. Bio-inspired deep attribute learning towards facial aesthetic prediction. IEEE Transactions on Affective Computing, 12(1), pp.227-238.

Yin, X., Tai, Y., Huang, Y., and Liu, X., 2020. Fan: Feature Adaptation Network for Surveillance Face Recognition and Normalization. In: Proceedings of the Asian Conference on Computer Vision.

Yuan, H., He, Y., Du, P., and Song, L., 2024. Multi-Task Learning Using Uncertainty to Weigh Losses for Heterogeneous Face Attribute Estimation. [Preprint].

Zhai, Y., Huang, Y., Xu, Y., Gan, J., Cao, H., Deng, W., Labati, R.D., Piuri, V., and Scotti, F., 2020. Asian female facial beauty prediction using deep neural networks via transfer learning and multi-channel feature fusion. IEEE Access, 8, pp.56892-56907.

Zhai, Y., Huang, Y., Xu, Y., Zeng, J., Yu, F., and Gan, J., 2016. Benchmark of a Large Scale Database For Facial Beauty Prediction. In: Proceedings of the 1st International Conference on Intelligent Information Processing.

Zhang, B., Xiao, X., and Lu, G., 2018. Facial beauty analysis based on features prediction and beautification models. Pattern Analysis and Applications, 21, pp.529-542.

Zheng, K., Tian, L., Li, Z., Li, H., and Zhang, J., 2024. Incorporating eyebrow and eye state information for facial expression recognition in mask-obscured scenes. Electronic Research Archive, 32(4), pp.2745-2771.