J Korean Assoc Oral Maxillofac Surg 2022; 48(4): 201~206
Changes in a facial recognition algorithm following different types of orthognathic surgery: a comparative study
Won-Yong Kim, Se Jin Han
Department of Oral and Maxillofacial Surgery, Dankook University Dental Hospital, Cheonan, Korea
Se Jin Han
Department of Oral and Maxillofacial Surgery, Dankook University
Dental Hospital, 119 Dandae-ro, Dongnam-gu, Cheonan 31116, Korea
TEL: +82-41-550-0271
E-mail: hanimplant@dankook.ac.kr
ORCID: https://orcid.org/0000-0003-4949-4462
Received May 23, 2022; Revised June 22, 2022; Accepted July 5, 2022.; Published online August 31, 2022.
© Korean Association of Oral and Maxillofacial Surgeons. All rights reserved.

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
 Abstract
Objectives: Contemporary biometric technologies have been gaining traction in both public and private security sectors. Facial recognition is the most commonly used biometric technology for this purpose. We aimed to evaluate the ability of a publicly available facial recognition application program interface to calculate similarity scores of presurgical and postsurgical photographs of patients who had orthognathic surgery.
Materials and Methods: Presurgical and postsurgical photographs of 75 patients who had orthognathic surgery between January 2018 and November 2020 in our department were used. Frontal photographs of patients in relaxed and smiling states were taken. The patients were classified into three groups: Group 2 had one-jaw surgery, Group 3 had two-jaw surgery to correct mandibular prognathism, and Group 4 had two-jaw surgery to correct facial asymmetry. For comparison, photographs of 10 participants were used as controls (Group 1). Two facial recognition application programs (Face X and Azure) were used to assess similarity scores.
Results: The similarity scores in the two programs showed significant results. The similarity score of the control group, which did not undergo orthognathic surgery, was the highest. The results for Group 2, Group 3, and Group 4 were higher in the order of Group 2, Group 3, and Group 4.
Conclusion: In this study, all orthodontic patients were recognized as the same person using the face recognition program before and after surgery. A significant difference in similarity results was obtained between the groups with both Face X and Azure and in both relaxed and smiling states.
Keywords: Facial recognition, Algorithm, Orthognathic surgery, Photographs
I. Introduction

A biometric system is an automated technology that assesses an individual based on physiological or biological characteristics to confirm their identity1,2. Examples of physical characteristics used in biometric technologies include fingerprints, iris patterns, retina shape, hand shape, facial morphology, and vein patterns3,4. The face is the most common and familiar biometric element in daily life. Facial recognition can be used for various purposes in the real world such as surveillance, access control, and identity verification5. However, the human face is not ideal compared with other biometric factors. In general, facial data are less accurate because they are heavily influenced factors such as cosmetics, disguise, and lighting6. However, biometric recognition using faces can be easily implemented without active engagement or cooperation by the individuals.

Facial recognition technology compares new images with images in a searchable database, and identifies matches and potential matches7. Facial recognition is primarily represented by two processes: authentication and identification. The authentication process can be defined as a one-to-one match that associates a facial image with a bank of available face image data with a matching personality. Facial identification is a one-to-many problem of matching the query image of a face with the available images in the face database8.

Orthognathic surgery improves facial proportions and harmony by altering the positions of the upper and lower jaws9 by changing the hard tissue and soft tissue10. Orthognathic surgery is classified as single jaw surgery, which deforms only the mandible, and double jaw surgery, which deforms both the upper and lower jaws. There are various types of double jaw surgery, such as canting, yawing, advancement, and posterior impaction, depending on the surgical method for the maxilla. We aimed to evaluate a publicly available facial recognition application program interface for calculating similarity scores for presurgical and postsurgical photographs of patients undergoing orthognathic surgeries.

II. Materials and Methods

1. Subjects

This study included patients who had orthognathic surgery at the Department of Oral and Maxillofacial Surgery, Dankook University Dental Hospital between January 2018 and November 2020. Patients were excluded if they had orthognathic surgery after November 2020, did not have a follow-up picture 1 year after surgery, had plastic surgery in addition to orthognathic surgery, or if their facial pictures were of poor quality. A total of 75 patients were divided into four groups: one control group and three groups corresponding to the experimental groups. The experimental groups comprised patients that had one-jaw surgery, patients that had two-jaw surgery for mandibular protrusion, and patients that had two-jaw surgery for facial asymmetry.(Fig. 1)

This study was approved by the Institutional Review Board of Dankook University Dental Hospital (IRB No. DKUDH IRB 2022-3-008). Written informed consent was obtained from all patients.

2. Methods

The surgery was performed by an oral and maxillofacial surgeon, and sagittal split ramus osteotomy was performed on all mandibles. In the case of the maxilla, maxillary canting was not performed in patients of Group 3; however, maxillary canting correction was performed in Group 4. Following the surgery, the patients collectively underwent intermaxillary fixation. Following their discharge, a periodic follow-up was performed, and clinical and radiographic images were acquired.

3. Study methods

The study was conducted using photos of expressionless and smiling expressions of 75 patients taken prior to orthognathic surgery and 1 year following surgery. For this study, two facial recognition programs were used, Face X and Azure, and the face comparison function of each program was employed. Facial similarity was obtained by applying the facial photographs of all patients taken before surgery and 1 year after surgery to the program, and the results were analyzed.

4. Analysis

Statistical analyses were conducted by a single researcher using IBM SPSS Statistics (ver. 28; IBM, Armonk, NY, USA), and a one-way ANOVA was performed. Facial similarity values were compared between each group; P<0.05 were considered to indicate statistically significant differences.

III. Results

1. Study model characteristics

Group 1, the control group, comprised eight male and two female participants, with a mean age of 28.90 years. Group 2, i.e., the group that had one-jaw surgery, comprised 21 individuals (14 male and 7 female), with a mean age of 22.09 years. Group 3, the group that had two-jaw surgery for mandibular protrusion, comprised 28 individuals (13 male and 15 female), with a mean age of 22.48 years. Group 4, the group that had two-jaw surgery for facial asymmetry, comprised 26 individuals (10 male and 16 female), with a mean age of 22.46 years.(Table 1)

2. Facial similarity results (Face X)

Following comparison of the relaxed facial photographs, the similarity values before and after surgery were 0.86±0.15 in Group 1, 0.79±0.07 in Group 2, 0.76±0.10 in Group 3, and 0.74±0.10 in Group 4. The mean value was 0.78±0.09. The P-value was 0.009.(Tables 2, 3)

Following comparison of the smiling state photographs, the preoperative and postoperative similarity values were 0.85±0.16 in Group 1, 0.78±0.08 in Group 2, 0.76±0.09 in Group 3, and 0.75±0.07 in Group 4. The mean value was 0.78±0.08. The P-value was 0.005.(Tables 2, 4)

3. Facial similarity results (Azure)

Following comparison of the relaxed state photographs, the similarity values before and after surgery were 0.98±0.01 in Group 1, 0.96±0.01 in Group 2, 0.94±0.02 in Group 3, and 0.92±0.01 in Group 4. The mean value was 0.95±0.02. The P-value was less than 0.001.(Tables 2, 5)

Following comparison of the smiling state photographs, the similarity values before and after surgery were 0.98±0.01 in Group 1, 0.96±0.01 in Group 2, 0.94±0.02 in Group 3, and 0.93±0.01 in Group 4. The mean value was 0.95±0.02. The P-value was less than 0.001.(Tables 2, 6)

IV. Discussion

In this study, facial comparison was performed by extracting features from the probe images and comparing the geometrical relationship between the constructed points with the existing database to identify the closest match and record the similarity as a percentage11. This study was performed assuming that the soft tissue changes after surgery would affect the facial similarity. Frontal photos were used, and the maxillary canting correction was considered to induce a greater change in the face; as a result, the groups were classified accordingly.

Two facial recognition programs (Face X and Azure) were used to analyze facial similarity. There was a slight difference in the overall similarity values between the two programs. When the same photograph was analyzed, the facial similarity value obtained using Face X was less than that obtained using Azure. To identify the reason for this difference, we assessed the analysis information from the two programs. Face X conducted similarity analysis using a total of 68 landmarks (Fig. 2), and Azure conducted an analysis using 27 landmarks.(Fig. 3) This difference in the number of contributes to the overall similarity values. Specifically, the similarity value is low because more changes were recognized when analyzing more landmarks. When assessing the similarity of the numerical results, the results of both the Face X and Azure programs yielded high results in the following order: Group 1, Group 2, Group 3, and Group 4.

There were differences in the analysis methods and results for each program; however, there were no differences in pre- and postoperative photos of the same person when comparing facial expressions. Therefore, according to this study, orthognathic surgery did not affect a person being recognized. This may be attributed to the change in soft tissue morphologies in relation to the position of the maxilla and mandible after orthognathic surgery; however, because the measurement points analyzed by the face recognition program are distributed across the entire face, there were few special changes in the measurement points.

According to a study by Dragon et al.7 on the relationship between facial recognition technology and orthognathic surgery, the mean similarity value was 97 (in quadrant 96-99) for patients who had two-jaw surgery and 99 (in quadrant 99-99) for patients who had one-jaw surgery. This study confirmed that the similarity level was lower in the case of two-jaw surgery than in the case of one-jaw surgery.

This study had some limitations. First, a person’s face is three-dimensional; however, in this study, facial comparison was performed using the patient’s clinical photographs; therefore, the three-dimensional element of the actual face was not considered. Second, photographs of the patient’s face before and 1 year after surgery were not taken by the same photographer. Third, the location and environment where the facial photo was taken were the same; however, small differences in the brightness and shooting angle of the photo were impossible to eliminate. Finally, the number of patients in each group was different. When conducting additional studies, a larger sample size is necessary. Furthermore, more meaningful results will be obtained if a study is conducted to recognize and analyze the three-dimensional faces of an actual patient in real time.

V. Conclusion

In this study, facial similarity was analyzed by applying before and after facial photographs of patients who had orthognathic surgery to facial recognition programs. The Face X and Azure programs were able to identify people who have undergone orthognathic surgery. When considering the similarity results, a significant difference was observed between the groups using both Face X and Azure and in both the relaxed and smiling state photographs. Strong similarity results were observed in the group that had two-jaw surgery for mandibular protrusion and the group that had two-jaw surgery for facial asymmetry.

Authors’ Contributions

W.Y.K. participated in data collection and wrote the manuscript. S.J.H. participated in sutdy design, coordination and helped to draft the manuscript. All authors read and approved the final manuscript.

Ethics Approval and Consent to Participate

This study was approved by the Institutional Review Board of Dankook University Dental Hospital (IRB No. DKUDH IRB 2022-3-008). Written informed consent was obtained from all patients.

Consent for Publishing Photographs

Written informed consent was obtained from the patients for publication of this article and accompanying images.

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

Figures
Fig. 1. Example facial photographs of each group: (A) Group 1 patient, (B) Group 2 patient, (C) Group 3 patient, and (D) Group 4 patient. (Group 1: the control group, Group 2: the group that had one-jaw surgery, Group 3: the group that had two-jaw surgery for mandibular protrusion, Group 4: the group that had two-jaw surgery for facial asymmetry)
Fig. 2. Landmarks analyzed by the Face X program.
Fig. 3. Landmarks analyzed by the Azure program.
Tables

Sample characteristics

Group 1 Group 2 Group 3 Group 4
Sex
Male 8 14 13 10
Female 2 7 15 16
Mean age (yr) 28.90 22.09 22.48 22.46

Group 1: the control group, Group 2: the group that had one-jaw surgery, Group 3: the group that had two-jaw surgery for mandibular protrusion, Group 4: the group that had two-jaw surgery for facial asymmetry.


P-value obtained from analysis on each program

P-value
Face X (relaxed state) 0.009
Face X (smiling state) 0.005
Azure (relaxed state) <0.001
Azure (smiling state) <0.001

In both programs, the P-value was lower than 0.001, indicating a significant result.


Similarity score (Face X – relaxed state)

Mean±SD 95% CI
Face X (relaxed state)
Group 1 0.86±0.15 0.84-0.88
Group 2 0.79±0.07 0.72-0.80
Group 3 0.76±0.10 0.73-0.80
Group 4 0.74±0.10 0.70-0.78
Mean 0.78±0.09 0.75-0.79

(SD: standard deviation, CI: confidence interval) Group 1: the control group, Group 2: the group that had one-jaw surgery, Group 3: the group that had two-jaw surgery for mandibular protrusion, Group 4: the group that had two-jaw surgery for facial asymmetry.

Mean similarity score was highest in the order of Group 1, Group 2, Group 3, and Group 4.


Similarity score (Face X – smiling state)

Mean±SD 95% CI
Face X (smiling state)
Group 1 0.85±0.16 0.84-0.87
Group 2 0.78±0.08 0.75-0.82
Group 3 0.76±0.09 0.73-0.80
Group 4 0.75±0.07 0.72-0.78
Mean 0.78±0.08 0.76-0.79

(SD: standard deviation, CI: confidence interval)

Group 1: the control group, Group 2: the group that had one-jaw surgery, Group 3: the group that had two-jaw surgery for mandibular protrusion, Group 4: the group that had two-jaw surgery for facial asymmetry.

Mean similarity score was highest in the order of Group 1, Group 2, Group 3, and Group 4.


Similarity score (Azure – relaxed state)

Mean±SD 95% CI
Azure (relaxed state)
Group 1 0.98±0.01 0.97-0.99
Group 2 0.96±0.01 0.95-0.96
Group 3 0.94±0.02 0.94-0.95
Group 4 0.92±0.01 0.92-0.93
Mean 0.95±0.02 0.94-0.95

(SD: standard deviation, CI: confidence interval)

Group 1: the control group, Group 2: the group that had one-jaw surgery, Group 3: the group that had two-jaw surgery for mandibular protrusion, Group 4: the group that had two-jaw surgery for facial asymmetry.

Mean similarity score was highest in the order of Group 1, Group 2, Group 3, and Group 4.


Similarity score (Azure – smiling state)

Mean±SD 95% CI
Azure (smiling state)
Group 1 0.98±0.01 0.98-0.99
Group 2 0.96±0.01 0.95-0.96
Group 3 0.94±0.02 0.94-0.95
Group 4 0.93±0.01 0.93-0.94
Mean 0.95±0.02 0.94-0.95

(SD: standard deviation, CI: confidence interval)

Group 1: the control group, Group 2: the group that had one-jaw surgery, Group 3: the group that had two-jaw surgery for mandibular protrusion, Group 4: the group that had two-jaw surgery for facial asymmetry.

Mean similarity score was highest in the order of Group 1, Group 2, Group 3, and Group 4.


References
  1. Esan OA, Zuva T, Ngwira SM, Zuva K. Performance improvement of authentication of fingerprints using enhancement and matching algorithms. Int J Emerg Technol Adv Eng 2013;3:472-82.
  2. Dhameliya MD, Chaudhari JP. A multimodal biometric recognition system based on fusion of palmprint and fingerprint. IJETT 2013;4:1908-11.
  3. Birgale L, Kokare M. Iris recognition without iris normalization. J Comput Sci 2010;6:1042-7. https://doi.org/10.3844/JCSSP.2010.1042.1047.
    CrossRef
  4. Moghaddam HA, Ghayoumi M. Facial image feature extraction using support vector machines. In VISAPP:480-5.
    CrossRef
  5. Fontaine X, Achanta R, Süsstrunk S. Face recognition in real-world images. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP):1482-6. https://doi.org/10.1109/ICCCSP.2018.8452853.
    KoreaMed CrossRef
  6. Adjabi I, Ouahabi A, Benzaoui A, Taleb-Ahmed A. Past, present, and future of face recognition: a review. Electronics 2020;9:1188. https://doi.org/10.3390/electronics9081188.
    CrossRef
  7. Dragon CB, Shroff B, Carrico C, Stilianoudakis S, Strauss R, Lindauer SJ. The effect of orthognathic surgery on facial recognition algorithm analysis. Am J Orthod Dentofacial Orthop 2020;158:84-91. https://doi.org/10.1016/j.ajodo.2019.11.013.
    Pubmed CrossRef
  8. Singh S, Prasad SVAV. Techniques and challenges of face recognition: a critical review. Procedia Comput Sci 2018;143:536-43. https://doi.org/10.1016/J.PROCS.2018.10.427.
    CrossRef
  9. Naran S, Steinbacher DM, Taylor JA. Current concepts in orthognathic surgery. Plast Reconstr Surg 2018;141:925e-936e. https://doi.org/10.1097/PRS.0000000000004438.
    Pubmed CrossRef
  10. Jung J, Lee CH, Lee JW, Choi BJ. Three dimensional evaluation of soft tissue after orthognathic surgery. Head Face Med 2018;14:21. https://doi.org/10.1186/s13005-018-0179-z.
    Pubmed KoreaMed CrossRef
  11. Agrawal AK, Singh YN. Evaluation of face recognition methods in unconstrained environments. Procedia Comput Sci 2015;48:644-51. https://doi.org/10.1016/j.procs.2015.04.147.
    CrossRef


Current Issue

31 August 2022
Vol. 48
No. 4 pp. 189~245

Indexed in