A METHOD FOR HEAD POSE ESTIMATION BASED ON 3D POINTS OF FACIAL LANDMARKS AND APPLICATION TO MONITOR ONLINE EXAMINATION
DOI:
https://doi.org/10.59266/houjs.2023.270Keywords:
Monitor online examination, computer vision, convolutional neural network, random forest regressionAbstract
Head pose estimating (HPE) is a complex problem requiring image processing, computer vision, and machine learning techniques. Current methods rely on convolutional neural networks (CNNs) to establish the mapping between 2D image space and the 3D face model, enabling the determination of head pose angles. HPE is applicated in various practical and highly significant areas such as security surveillance, driver attention monitoring, online learning and testing supervision, and more. This study utilizes a modern CNN model to detect facial landmarks. It proposes a method for estimating facial pose angles using a random forest algorithm based on 3D facial landmarks extracted from 2D images. This approach enables the determination of head pose angles from the given images. Experimental results on four popular datasets demonstrate the effectiveness of the proposed method, achieving low estimation errors, particularly outperforming other methods on two of the four datasets. We also present an integrated design of the proposed method with an online learning management system to facilitate monitoring and assessment of learners’ engagement and performance during learning and testing activities.
References
[1]. T. Liu, J. Wang, B. Yang và X. Wang, “NGDNet: Nonuniform Gaussian- label distribution learning for infrared head pose estimation and on-task behavior understanding in the classroom,” Neurocomputing, tập 436, p. 210–220, 2021.
[2]. Z. Hu, Y. Xing, C. Lv, P. Hang và J. Liu, “Deep convolutional neural network- based Bernoulli heatmap for head pose estimation,” Neurocomputing, tập 436, p. 198–209, 2021.
[3]. Y. Xu, C. Jung và Y. Chang, “Head pose estimation using deep neural networks and 3D point clouds,” Pattern Recognition, tập 121, số 108210, pp. 1-10, 2022.
[4]. Z. Cao, Z. Chu, D. Liu và Y. Chen, “A Vector-based Representation to Enhance Head Pose Estimation,” Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1188-1197, 2021.
[5]. D. T. Long, “A Facial Expressions Recognition Method Using Residual Network Architecture for Online Learning Evaluation,” Journal of Advanced Computational Intelligence and Intelligent Informatics, tập 25, số 6, pp. 1-10, 2021.
[6]. D. T. Long, “A Lightweight Face Recognition Model Using Convolutional Neural Network for Monitoring Students in E-Learning,” International Journal of Modern Education and Computer Science, tập 6, pp. 16-28, 2020.
[7]. D. T. Long, T. T. Tung và T. T. Dung, “A Facial Expression Recognition Model using Lightweight Dense- Connectivity Neural Networks for Monitoring Online Learning Activities,” International Journal of Modern Education and Computer Science, tập 6, pp. 53-64, 2022.
[8]. B. N. Anh, N. T. Son, P. T. Lam, L.
P. Chi, N. H. Tuan, N. C. Dat, N. H. Trung, M. U. Aftab và T. V. Dinh, “A Computer-Vision Based Application for Student Behavior Monitoring in Classroom,” Applied sciences , tập 9, số 4729, pp. 1-17, 2019.
[9]. Y. Kartynnik, A. Ablavatski, I. Grishchenko và M. Grundmann, “Real- time Facial Surface Geometry from Monocular Video on Mobile GPUs,” CVPR Workshop on Computer Vision for Augmented and Virtual Reality, số https://arxiv.org/abs/1907.06724, pp. 1-4, 2019.
[10]. A. F. Abate, C. Bisogni, A. Castiglione và M. Nappi, “Head pose estimation: An extensive survey on recent techniques and applications,” Pattern Recognition, tập 127, số 108591, pp. 1-14, 2022.
[11]. M. H. Lipu, A. Ayob, M. H. M. Saad và M. Faisal, “State of Charge Estimation for Lithium-ion Battery Based on Random Forests Technique with Gravitational Search Algorithm,” IEEE PES Asia-Pacific Power and Energy Engineering Conference (APPEEC), pp. 45-50, 2018.
[12]. A. Asperti và D. Filippini, “Deep Learning for Head Pose Estimation: A Survey,” SN Computer Science, tập 4, số 349, pp. 1-41, 2023.
[13]. S. Malek và S. Rossi, “Head pose estimation using facial-landmarks classification for children rehabilitation games,” Pattern Recognition Letters, tập 152, p. 406–412, 2021.
[14]. M. Shao, Z. Sun, M. Ozay và T. Okatani, “Improving Head Pose Estimation with a Combined Loss and Bounding Box Margin Adjustment,” IEEE International Conference on Automatic Face & Gesture Recognition, số doi: 10.1109/ FG.2019.8756605, pp. 1-5, 2019.
[15]. H. Wang, Z. Chen và Y. Zhou, “Hybrid coarse-fine classification for head pose estimation,” ArXiv, tập abs/1901.06778, pp. 1-5, 2019.
[16]. M. Ariz, A. Villanueva và R. Cabeza, “Robustandaccurate 2D-tracking-based 3D positioning method: Application to head pose estimation,” Computer Vision and Image Understanding, tập 180, pp. 13-22, 2019.
[17]. K. Khan, J. Ali, K. Ahmad, A. Gul,
G. Sarwar, S. Khan, Q. T. H. Ta, T.-S. Chung và M. Attique, “3D Head Pose Estimation through Facial Features and Deep Convolutional Neural Networks,” Computers, Materials & Continua, tập 66, số 2, pp. 1757-1770, 2021.
[18]. H. Kawai, J. Chen, P. Ishwar và J. Konrad, “VAE/WGAN-Based Image Representation Learning For Pose-Preserving Seamless Identity Replacement In Facial Images,” IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP), số https://doi.org/10.1109%2Fmlsp.2019.8918926, 2019.
[19]. K. Khan, R. U. Khan, R. Leonardi, P. Migliorati và S. Benini, “Head pose estimation: A survey of the last ten years,” Signal Processing: Image Communication, tập 99, số 116479, pp. 1-16, 2021.
