A Comprehensive Review of Facial Beauty Prediction Using Multi-task Learning and Facial Attributes
DOI:
https://doi.org/10.14500/aro.11850Keywords:
Convolutional Neural Network, Facial beauty prediction, Facial attractiveness, Human RaterAbstract
Beauty multi-task prediction from facial attributes is a multidisciplinary challenge at the intersection of computer vision, machine learning, and psychology. Despite the centrality of beauty in human perception, its subjective nature—shaped by individual, social, and cultural influences—complicates its computational modeling. This review addresses the pressing need to develop robust and fair predictive models for facial beauty assessments by leveraging deep learning techniques. Using facial attributes such as symmetry, skin complexion, and hairstyle, we explore how these features influence perceptions of attractiveness. The study adopts advanced computational methodologies, including convolutional neural networks and multi-task learning frameworks, to capture nuanced facial cues. A comprehensive analysis of publicly available datasets reveals critical gaps in diversity, biases, and ground truth annotation for training effective models. We further examine the methodological challenges in defining and measuring beauty, such as data imbalances and algorithmic fairness. By synthesizing insights from psychology and machine learning, this work highlights the potential of interdisciplinary approaches to enhance the reliability and inclusivity of automated beauty prediction systems.
Downloads
References
Aarabi, P., Hughes, D., Mohajer, K., and Emami, M., 2001. The Automatic Measurement of Facial Beauty. In: 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat. No. 01CH37236). IEEE.
Ahmadimehr, S., and Moridani, M.K., 2020. Evaluating facial attractiveness through proportions analysis based on geometric features. Journal of Image Processing & Pattern Recognition Progress, 7(2), pp.20-26.
Altwaijry, H., and Belongie, S., 2013. Relative Ranking of Facial Attractiveness. In: 2013 IEEE Workshop on Applications of Computer Vision (WACV). IEEE. DOI: https://doi.org/10.1109/WACV.2013.6475008
Bougourzi, F., Dornaika, F., Barrena, N., Distante, C., and Taleb-Ahmed, A., 2023. CNN based facial aesthetics analysis through dynamic robust losses and ensemble regression. Applied Intelligence, 53(9), pp.10825-10842. DOI: https://doi.org/10.1007/s10489-022-03943-0
Cao, K., Choi, K.N., Jung, H., and Duan, L., 2020. Deep learning for facial beauty prediction. Information, 11(8), p.391. DOI: https://doi.org/10.3390/info11080391
Chen, F., Xu, Y., and Zhang, D., 2014. A new hypothesis on facial beauty perception. ACM Transactions on Applied Perception, 11(2), pp.1-20. DOI: https://doi.org/10.1145/2622655
Chen, Y., Zhang, Y., Huang, Z., Luo, Z., and Chen, J., 2021. CelebHair: A New Large-scale Dataset for Hairstyle Recommendation Based on CelebA. In: International Conference on Knowledge Science, Engineering and Management. Springer. DOI: https://doi.org/10.1007/978-3-030-82153-1_27
Cowen, A.S., Keltner, D., Schroff, F., Jou, B., Adam, H., and Prasad, G., 2021. Sixteen facial expressions occur in similar contexts worldwide. Nature, 589(7841), pp.251-257. DOI: https://doi.org/10.1038/s41586-020-3037-7
Eisenthal, Y., Dror, G., and Ruppin, E., 2006. Facial attractiveness: Beauty and the machine. Neural Computation, 18(1), p.119-142. DOI: https://doi.org/10.1162/089976606774841602
Fan, D., Kim, H., Kim, J., Liu, Y., and Huang, Q., 2019. Multi-task learning using task dependencies for face attributes prediction. Applied Sciences, 9(12), p.2535. DOI: https://doi.org/10.3390/app9122535
Favorskaya, M.N., and Pakhirka, A.I., 2023. Age-Group Estimation of Facial Images Using Multi-task Ranking CNN. In: International KES Conference on Intelligent Decision Technologies. Springer. DOI: https://doi.org/10.1007/978-981-99-2969-6_13
Gan, J., Jiang, K., Tan, H., and He, G., 2020. Facial beauty prediction based on lighted deep convolution neural network with feature extraction strengthened. Chinese Journal of Electronics, 29(2), pp.312-321. DOI: https://doi.org/10.1049/cje.2020.01.009
Gan, J., Luo, H., Xiong, J., Xie, X., Li, H., and Liu, J., 2023. Facial beauty prediction combined with multi-task learning of adaptive sharing policy and attentional feature fusion. Electronics, 13(1), p.179. DOI: https://doi.org/10.3390/electronics13010179
Gan, J., Xiang, L., Zhai, Y., Mai, C., He, G., Zeng, J., Bai, Z., Labati, R., Piuri, V., and Scotti, F., 2020. 2M BeautyNet: Facial beauty prediction based on multitask transfer learning. IEEE Access, 8, pp.20245-20256. DOI: https://doi.org/10.1109/ACCESS.2020.2968837
Gan, J., Xie, X., Zhai, Y., He, G., Mai, C., and Luo, H., 2023. Facial beauty prediction fusing transfer learning and broad learning system. Soft Computing, 27(18), pp.13391-13404. DOI: https://doi.org/10.1007/s00500-022-07563-1
Gao, L., Li, W., Huang, Z., Huang, D., and Wang, Y., 2018. Automatic Facial Attractiveness Prediction by Deep Multi-task Learning. In: 2018 24th International Conference on Pattern Recognition (ICPR). IEEE. DOI: https://doi.org/10.1109/ICPR.2018.8545033
Gray, D., Yu, K., Xu, W., and Gong, Y., 2010. Predicting Facial Beauty Without Landmarks. In: Computer Vision-ECCV 2010: 11th European Conference on Computer Vision. Proceedings, Part VI 11. Springer, DOI: https://doi.org/10.1007/978-3-642-15567-3_32
Heraklion, Crete, Greece. Grgic, M., Delac, K., and Grgic, S., 2011. SCface-surveillance cameras face database. Multimedia Tools and Applications, 51, pp.863-879. DOI: https://doi.org/10.1007/s11042-009-0417-2
Gunes, H., and Piccardi, M., 2006. Assessing facial beauty through proportion analysis by image processing and supervised learning. International Journal of Human-computer Studies, 64(12), pp.1184-1199. DOI: https://doi.org/10.1016/j.ijhcs.2006.07.004
Jamoliddin, U., and Yoo, J.H., 2022. Age and gender classification with small scale cnn. The Journal of the Korea Institute of Electronic Communication Sciences, 17(1), pp.99-104.
Kalyta, O., Krakb, I., Barmaka, O., Wojcikd, W., and Radiuk, P., 2022. Method of Facial Geometric Feature Representation for Information Security Systems. In: 3rd International Workshop on Intelligent Information Technologies & Systems of Information Security. Khmelnytskyi, Ukraine.
Lebedeva, I., Guo, Y., and Ying, F., 2021. Transfer learning adaptive facial attractiveness assessment. Journal of Physics: Conference Series, 1922, p.012004. DOI: https://doi.org/10.1088/1742-6596/1922/1/012004
Lebedeva, I., Guo, Y., and Ying, F., 2022. MEBeauty: Amulti-ethnic facial beauty dataset in-the-wild. Neural Computing and Applications, 34, pp.14169-14183. DOI: https://doi.org/10.1007/s00521-021-06535-0
Lebedeva, I., Guo, Y., and Ying, F., 2023. Personalized facial beauty assessment: A meta-learning approach. The Visual Computer, 39(3), pp.1095-1107. DOI: https://doi.org/10.1007/s00371-021-02387-w
Liang, L., Xie, D., Jin, L., Xu, J., Li, M., and Lin, L., 2017. Region-aware Scattering Convolution Networks for Facial Beauty Prediction. In: 2017 IEEE International Conference on Image Processing (ICIP). IEEE. DOI: https://doi.org/10.1109/ICIP.2017.8296805
Lin, L., Shen, Z., Yin, J.L., Liu, Q., Yu, Y., and Chen, W., 2023. MetaFBP: Learning to Learn High-Order Predictor for Personalized Facial Beauty Prediction. In: Proceedings of the 31st ACM International Conference on Multimedia. DOI: https://doi.org/10.1145/3581783.3612319
Lin, Y., Zhibin, G., Zhang, S., Li, L., and Huang, L., 2021. Multi-Task Network and Optimization for Face Detection and Attribute Analysis. In: 2021 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA). IEEE. DOI: https://doi.org/10.1109/ICAICA52286.2021.9497937
Liu, Z., Luo, P., Wang, X., and Tang, X., 2015. Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision. IEEE. DOI: https://doi.org/10.1109/ICCV.2015.425
Mao, L., Yan, Y., Xue, J.H., and Wang, H., 2020. Deep multi-task multi-label CNN for effective facial attribute classification. IEEE Transactions on Affective Computing, 13(2), pp.818-828. DOI: https://doi.org/10.1109/TAFFC.2020.2969189
Moridani, M.K., Jamiee, N., and Saghafi, S., 2023. Human-like evaluation by facial attractiveness intelligent machine. International Journal of Cognitive Computing in Engineering, 4, pp.160-169. DOI: https://doi.org/10.1016/j.ijcce.2023.04.001
Mu, Y., 2013. Computational facial attractiveness prediction by aesthetics-aware features. Neurocomputing, 99, pp.59-64. Nguyen, T.V., Liu, S., Ni, B., Tan, J., Rui, Y., and Yan, S., 2013. DOI: https://doi.org/10.1016/j.neucom.2012.06.020
Towards decrypting attractiveness via multi-modality cues. ACM Transactions on Multimedia Computing, Communications, and Applications, 9(4), p.28.
Panić, N., Marjanović, M., and Bezdan, T., 2024. Ethnic representation matters: Investigating bias in facial age prediction models. Mathematics, 12(15), pp.1-30. DOI: https://doi.org/10.3390/math12152358
Rohani, M., Farsi, H., and Mohamadzadeh, S., 2023. Deep multi-task convolutional neural networks for efficient classification of face attributes. International Journal of Engineering, 36(11), pp.2102-2111. DOI: https://doi.org/10.5829/IJE.2023.36.11B.14
Saeed, J.N., Abdulazeez, A.M., and Ibrahim, D.A., 2022. FIAC-Net: Facial Image Attractiveness Classification based on Light Deep Convolutional Neural Network. In: 2022 Second International Conference on Computer Science, Engineering and Applications (ICCSEA). IEEE. DOI: https://doi.org/10.1109/ICCSEA54677.2022.9936582
Saeed, J.N., and Abdulazeez, A.M., 2021. Facial beauty prediction and analysis based on deep convolutional neural network: Areview. Journal of Soft Computing and Data Mining, 2(1), pp.1-12.
Saeed, J.N., and Abdulazeez, A.M., 2022. 2D Facial Images Attractiveness Assessment Based on Transfer Learning of Deep Convolutional Neural Networks. In: 2022 4th International Conference on Advanced Science and Engineering (ICOASE). IEEE. DOI: https://doi.org/10.1109/ICOASE56293.2022.10075585
Sagonas, C., Antonakosa, E., Tzimiropoulosb, G., Zafeirioua, S., and Pantic, M., 2016. 300 faces in-the-wild challenge: Database and results. Image and Vision Computing, 47, pp.3-18. DOI: https://doi.org/10.1016/j.imavis.2016.01.002
Savchenko, A.V., 2021. Facial Expression and Attributes Recognition Based on Multi-task Learning of Lightweight Neural Networks. In: 2021 IEEE 19th International Symposium on Intelligent Systems and Informatics (SISY). IEEE, pp.119-124. DOI: https://doi.org/10.1109/SISY52375.2021.9582508
Schmid, K., Marx, D., and Samal, A., 2008. Computation of a face attractiveness index based on neoclassical canons, symmetry, and golden ratios. Pattern Recognition, 41(8), pp.2710-2717. DOI: https://doi.org/10.1016/j.patcog.2007.11.022
Sekhar, J.C., Joel Josephson, P., Chinnasamy, A., Maheswari, V., Sankar, V., and Kalangi, R.R., 2024. Automated face recognition using deep learning technique and center symmetric multivariant local binary pattern. Neural Computing and Applications, pp.1-19. DOI: https://doi.org/10.1007/s00521-024-10447-0
Štěpánek, L., Kasal, P., and Mestak, J., 2018. Evaluation of Facial Attractiveness for Purposes of Plastic Surgery Using Machine-learning Methods and Image Analysis. In: 2018 IEEE 20th International Conference on e-Health Networking, Applications and Services. IEEE. DOI: https://doi.org/10.1109/HealthCom.2018.8531195
Taherkhani, F., Dabouei, A., Soleymani, S., Dawson, J., and Nasrabadi, N.M., 2021. Tasks structure regularization in multi-task learning for improving facial attribute prediction. arXiv preprint arXiv:2108.04353.
Vahdati, E., and Suen, C.Y., 2021. Facial beauty prediction from facial parts using multi-task and multi-stream convolutional neural networks. International Journal of Pattern Recognition and Artificial Intelligence, 35(12), p.2160002. DOI: https://doi.org/10.1142/S0218001421600028
Wang, F., Han, H., Shan, S., and Chen, X., 2017. Deep Multi-task Learning for Joint Prediction of Heterogeneous Face Attributes. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). IEEE. DOI: https://doi.org/10.1109/FG.2017.30
Weng, N., Wang, J., Li, A., and Wang, Y., 2021. Two-stream Temporal Convolutional Network for Dynamic Facial Attractiveness Prediction. In: 2020 25th International Conference on Pattern Recognition (ICPR). IEEE. DOI: https://doi.org/10.1109/ICPR48806.2021.9413333
Xiao, Q., Wu, Y., Wang, D., Yang, Y.L., and Jin, X., 2021. Beauty3DFaceNet: Deep geometry and texture fusion for 3D facial attractiveness prediction. Computers & Graphics, 98, pp.11-18. DOI: https://doi.org/10.1016/j.cag.2021.04.023
Xie, D., Liang, L., Jin, L., Xu, J., and Li, M., 2015. Scut-fbp: A Benchmark Dataset for Facial Beauty Perception. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics. IEEE. DOI: https://doi.org/10.1109/SMC.2015.319
Xie, Y., Wang, K., Meng, J., Yue, J., Meng, L., Yi, W., Jung, T.P., Xu, M., and Ming, D., 2023. Cross-dataset transfer learning for motor imagery signal classification via multi-task learning and pre-training. Journal of Neural Engineering, 20(5), p.056037. DOI: https://doi.org/10.1088/1741-2552/acfe9c
Xu, L., Xiang, J., and Yuan, X., 2018. CRnet: Classification and Regression Neural Network for Facial Beauty Prediction. In: Pacific Rim Conference on Multimedia. Springer, Cham. DOI: https://doi.org/10.1007/978-3-030-00764-5_61
Xu, M., Chen, F., Li, L., Shen, C., Lv, P., Zhou, B., and Ji, R., 2018. Bio-inspired deep attribute learning towards facial aesthetic prediction. IEEE Transactions on Affective Computing, 12(1), pp.227-238. DOI: https://doi.org/10.1109/TAFFC.2018.2868651
Yin, X., Tai, Y., Huang, Y., and Liu, X., 2020. Fan: Feature Adaptation Network for Surveillance Face Recognition and Normalization. In: Proceedings of the Asian Conference on Computer Vision. DOI: https://doi.org/10.1007/978-3-030-69532-3_19
Yuan, H., He, Y., Du, P., and Song, L., 2024. Multi-Task Learning Using Uncertainty to Weigh Losses for Heterogeneous Face Attribute Estimation. [Preprint].
Zhai, Y., Huang, Y., Xu, Y., Gan, J., Cao, H., Deng, W., Labati, R.D., Piuri, V., and Scotti, F., 2020. Asian female facial beauty prediction using deep neural networks via transfer learning and multi-channel feature fusion. IEEE Access, 8, pp.56892-56907. DOI: https://doi.org/10.1109/ACCESS.2020.2980248
Zhai, Y., Huang, Y., Xu, Y., Zeng, J., Yu, F., and Gan, J., 2016. Benchmark of a Large Scale Database For Facial Beauty Prediction. In: Proceedings of the 1st International Conference on Intelligent Information Processing. DOI: https://doi.org/10.1145/3028842.3028863
Zhang, B., Xiao, X., and Lu, G., 2018. Facial beauty analysis based on features prediction and beautification models. Pattern Analysis and Applications, 21, pp.529-542. DOI: https://doi.org/10.1007/s10044-017-0647-2
Zheng, K., Tian, L., Li, Z., Li, H., and Zhang, J., 2024. Incorporating eyebrow and eye state information for facial expression recognition in mask-obscured scenes. Electronic Research Archive, 32(4), pp.2745-2771. DOI: https://doi.org/10.3934/era.2024124
Downloads
Published
How to Cite
Issue
Section
License
Authors who choose to publish their work with Aro agree to the following terms:
-
Authors retain the copyright to their work and grant the journal the right of first publication. The work is simultaneously licensed under a Creative Commons Attribution License [CC BY-NC-SA 4.0]. This license allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
-
Authors have the freedom to enter into separate agreements for the non-exclusive distribution of the journal's published version of the work. This includes options such as posting it to an institutional repository or publishing it in a book, as long as proper acknowledgement is given to its initial publication in this journal.
-
Authors are encouraged to share and post their work online, including in institutional repositories or on their personal websites, both prior to and during the submission process. This practice can lead to productive exchanges and increase the visibility and citation of the published work.
By agreeing to these terms, authors acknowledge the importance of open access and the benefits it brings to the scholarly community.
ARO Journal is a scientific, peer-reviewed, periodical, and diamond OAJ that has no APC or ASC.