- Email: [email protected]
The reliability of flexible nasolaryngoscopy in the identification of vocal fold movement impairment in young infants
International Journal of Pediatric Otorhinolaryngology 100 (2017) 157e159
Contents lists available at ScienceDirect
International Journal of Pediatric Otorhinolaryngology journal homepage: http://www.ijporlonline.com/
The reliability of flexible nasolaryngoscopy in the identification of vocal fold movement impairment in young infants Yi-Chun Carol Liu a, Tyler McElwee b, Mary Musso a, Tara L. Rosenberg a, Julina Ongkasuwan a, * a b
Department of Otolaryngology Head and Neck Surgery, Baylor College of Medicine/Texas Children's Hospital, Houston, TX, USA Baylor College of Medicine, Houston, TX, USA
a r t i c l e i n f o
a b s t r a c t
Article history: Received 1 June 2017 Received in revised form 5 July 2017 Accepted 7 July 2017 Available online 8 July 2017
Objective: Flexible nasolaryngoscopy (FNL) is considered the gold standard for evaluation of vocal fold mobility but there has been no data on the reliability of interpretation in the infant population. Visualization may be limited by excessive movement, secretions, or floppy supraglottic structures that prevent accurate diagnosis of vocal fold movement impairment (VFMI). We sought to evaluate the inter- and intra-rater reliability of FNL for the evaluation of VFMI in young infants. Study type: Case-control. Methods: Twenty infants were identified: 10 with VFMI and 10 normal as seen on FNL. Three pediatric otolaryngologists reviewed the video without sound and rated the presence and/or degree of vocal fold mobility. Twelve videos were repeated to assess intra-rater reliability. Results: There was substantial agreement between the reviewers regarding the identification normal vs. any type of VFMI (kappa ¼ 0.67) but only moderate agreement regarding the degree of vocal fold movement (kappa ¼ 0.49). Intra-rater reliability ranges from moderate to perfect agreement (kappa ¼ 0.48e1). Conclusion: FNL in infants is an extremely challenging procedure. Clinically, physicians frequently use the quality of the cry and the past medical and surgical history to help make a judgment of vocal fold movement when the view is suboptimal. These other factors, however, may bias the interpretation of the FNL. Without sound, there is only moderate inter-rater and variable intra-rater reliability for the identification of degree of movement on FNL. Otolaryngologists must be cognizant of the limitations of FNL when using it as a clinical tool or as a “gold standard” against which other modalities are measured. © 2017 Published by Elsevier Ireland Ltd.
Keywords: Flexible nasolaryngoscopy Larynx Vocal fold paralysis Vocal fold motion impairment Inter-rater reliability Intra-rater reliability Level of evidence: 3b case control study
1. Introduction
2. Methods
Flexible nasolaryngoscopy (FNL) is considered the gold standard for evaluation of vocal fold (VF) mobility; however, no data on the reliability of interpretation in the infant population exists [1e3]. Friedman reported difficulty with adequate visualization of the endolarynx of children under 3 years old using a single bedside FNL in up to 20% of the time [4]. Visualization may be limited by excessive movement, secretions, or floppy supraglottic structures thus prevent accurate diagnosis of vocal fold movement impairment (VFMI). In this study, we sought to evaluate the inter- and intra-rater reliability of FNL for the evaluation of VFMI in young infants.
With institutional review board approval, the recorded laryngoscopies of twenty consecutive infants were identified from the pediatric otolaryngology flexible nasolaryngoscopy database using CPT code of 31575 and 92511 and ICD-10 code of J38: 10 with VFMI and 10 with normal vocal fold mobility as diagnosed on FNL by the treating fellowship trained pediatric otolaryngology attending. Three fellowship trained pediatric otolaryngologists, who were not involved in the clinical care the patients, reviewed the videos, in random order, without sound or clinical history. Twelve videos were randomly selected to be repeated to assess intra-rater reliability but not all twenty videos were included to prevent bias on examination. The reviewers rated the degree of vocal fold mobility (normal, partial, paralysis) on each side. Inter- and intra-rater agreement between reviewers were
* Corresponding author. 6701 Fannin, Suite 640, Houston, TX 77030, USA. E-mail address: [email protected] (J. Ongkasuwan). http://dx.doi.org/10.1016/j.ijporl.2017.07.005 0165-5876/© 2017 Published by Elsevier Ireland Ltd.
158
Y.-C.C. Liu et al. / International Journal of Pediatric Otorhinolaryngology 100 (2017) 157e159
calculated using Cohen and Fleiss' kappa coefficients respectively, with weighted kappa coefficients calculated for agreement of movement and types of movement (normal, paresis or paralysis). Using the Cohen's kappa statistical metric, value < 0 indicates no agreement, 0e0.20 as slight, 0.21e0.40 as fair, 0.41e0.60 as moderate, 0.61e0.80 as substantial, and 0.81e1 as almost perfect agreement. Using the Fleiss' kappa statistical metric, value < 0.40 indicates poor agreement, 0.41e0.75 as fair to good, and 0.76e1 as excellent [5]. Raw data from the reviewers are included in Table 1. 3. Results The infants ranged in age from 12 to 330 days (median 31.5 days) at the time of FNL. There was substantial agreement between the reviewers regarding the identification of normal vs. any type of VFMI (paresis or paralysis) (kappa ¼ 0.67, 95% CI: 0.47e0.87). Agreement regarding the degree of VF movement (normal vs. paresis vs. paralysis) was moderate (kappa ¼ 0.49, 95% CI: 0.35e0.64). Reviewer 1 had moderate intra-rater reliability for the presence of VFMI (kappa ¼ 0.53, 95% CI: 0.06e0.99) and for the degree of movement (kappa ¼ 0.48, 95% CI: 0.06e0.90). Reviewer 2 had substantial intra-rater agreement for the identification of VFMI (kappa ¼ 0.82, 95% CI: 0.49e1) and the degree of movement (kappa ¼ 0.87, 95% CI: 0.62e1). Reviewer 3 had perfect intra-rater reliability for the both the presence of VFMI (kappa ¼ 1, 95% CI: 1-1) and the degree of movement (kappa ¼ 1, 95% CI:1-1). 4. Discussion Vocal fold movement impairment (VFMI) can be a result of
mechanical restriction or neuronal cause. VFMI is not a simple allor-none diagnosis as there can be varying degrees of vocal fold movement. Flexible nasolaryngoscopy (FNL) is the most frequently used diagnostic tool and is considered the gold standard for evaluation of vocal fold mobility but there is no data on the reliability of interpretation in the infant population. Madden et al. reported that, in adults, inter-rater and intra-rater reliability of FNL is consistent amongst general otolaryngologists and in fellowship trained laryngologists with an inter-rater reliability of 95% [2]. However, FNL in neonates and infants can be much more challenging than in adults. Factors such as excessive movement of the patient, secretions, or floppy supraglottic structures obscure the view of the larynx. Other diagnostic methods have been studied, including laryngeal ultrasound which has been proposed to be an effective supplemental diagnostic tool in the neonatal population with congenital cardiac issues [4,6]. Laryngeal electromyography (LEMG) has also been studied with similar results to FNL but is still susceptible to individual variation in interpretation [7]. In addition, the placement of LEMG needles into the intrinsic laryngeal musculature is not generally feasible in infants and children without anesthesia. Lastly, functional 3D computer tomography has been proposed as a diagnostic tool for VFMI. However, the limitation of the use of computer tomography is the radiation exposure and the standard errors inherent in reconstructing images making it not an ideal diagnostic tool in pediatric patient population [8]. Secondary to the difficulty of FNL visualization of the larynx in the neonatal and infant population, otolaryngologists frequently use additional information such as clinical history, surgical history and cry volume [9]. For our study, we purposely chose to eliminate the audio from the videos to decrease the bias of the raters by limiting them to visual evaluation only.
Table 1 FNL video key and reviewer results. Study ID
Video
Key
Reviewer 1
Review 2
Review 3
1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 10
L VF Paresis B/l Paresis L VF Paresis B/l Paresis Normal L VF Paresis Normal Normal L VF Paralysis L VF Paresis
Normal Normal L VF Paralysis L VF Paralysis L VF Paralysis L VF Paresis Normal Normal L VF Paralysis L VF Paralysis
R VF Paresis B/l Paralysis L VF Paresis Normal Normal L VF Paralysis Normal Normal L VF Paralysis L VF Paresis
11 12 13 14 15 16 17 18 19 20 16 5 7 8 9 3 15 6 19 17 4 18
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
L VF Paresis B/l Paresis L VF Paresis Normal Normal L VF Paralysis Normal Normal L VF Paralysis R VF Paresis L VF ParalysisR VF Paresis L VF ParalysisR VF Paresis L VF Paralysis Normal Normal L VF Paralysis Normal L VF Paresis L VF Paralysis Normal Normal L VF Paralysis Normal L VF Paralysis Normal Normal Normal L VF Paralysis L VF Paresis L VF Paresis L VF Paralysis L VF Paralysis Normal Normal Normal
Normal Normal B/l Paralysis Normal L VF Paralysis L VF Paralysis Normal Normal L VF Paresis Normal L VF Paralysis Normal Normal Normal L VF Paralysis Normal L VF Paralysis Normal Normal Normal B/l VF Paralysis Normal
Normal Normal B/l VF Paralysis Normal L VF Paralysis L VF Paralysis Normal Normal L VF Paresis Normal L VF Paralysis Normal Normal Normal L VF Paralysis L VF Paralysis L VF Paralysis L VF Paresis L VF Paresis Normal L VF Paralysis Normal
Normal Normal R VF Paralysis Normal L VF Paresis L VF Paralysis Normal Normal L VF Paresis Normal L VF Paralysis Normal Normal Normal L VF Paralysis L VF Paresis L VF Paresis L VF Paralysis L VF Paresis Normal Normal Normal
Y.-C.C. Liu et al. / International Journal of Pediatric Otorhinolaryngology 100 (2017) 157e159
In the adult literature, most raters have high intra-rater reliability but the inter-rater reliability varies, especially in identifying the types of VFMI (paralysis vs. paresis) [2]. Our data showed substantial inter-rater agreement when differentiating normal vs. any movement impairment; however, the agreement fell to moderate when trying to differentiate paralysis from paresis. The interrater reliability in our study was lower than the previous published adult literature. The difference is likely a reflection of the aforementioned challenges when performing infant FNL. In order to improve on inter-rater reliability of identifying VFMI, there is continuous discussion of establishing a classification system for VFMI. However, the difficulty with a classification system remains that there is no easily obtainable objective measurement that can be used. Rosow et al. suggested rating categories with explicit definitions and examples, but recognized individual variation in laryngeal anatomy and innervation resulted in a heterogenous appearance of VFMI [1]. In addition, the neonatal and infant population pose additional challenges in obtaining an optimal view of the larynx to facilitate accurate diagnosis. Further research is needed to develop a universally acceptable classification system for VFMI. 5. Conclusion FNL in neonates and infants can be challenging. Clinically, physicians frequently use the quality of the cry and the past medical and surgical history to help make a judgment of vocal fold movement when the view is suboptimal. These other factors, however, may bias the interpretation of the FNL. Without sound, we found that there is only moderate inter-rater reliability for the identification of the degree of movement on FNL. Otolaryngologists must
159
be cognizant of the limitations of FNL when using it as a clinical tool or as a “gold standard” against which other modalities are measured. Conflict of interest None. Financial disclosures None. References [1] D.E. Rosow, L. Sulica, Laryngoscopy of vocal fold paralysis: evaluation of consistency of clinical findings, Laryngoscope 120 (7) (2010 Jul) 1376e1382. [2] L.L. Madden, C.A. Rosen, Evaluation of vocal fold motion abnormalities: are we all seeing the same thing? J. Voice 31 (1) (2017 Jan) 72e77. [3] L.1 Sulica, A. Blitzer, Vocal fold paresis: evidence and controversies, Pin Otolaryngol. Head Neck Surg. 15 (3) (2007 Jun) 159e162. [4] E.M. Friedman, Role of ultrasound in the assessment of vocal cord function in infants and children, Ann. Otol. Rhinol. Laryngol. 106 (1997) 199e209. [5] J.L. Fleiss, Statistical Methods for Rates and Proportions, second ed., Wiley, New York, NY, 1981. [6] J.1 Ongkasuwan, E.2 Ocampo, B.3 Tran, Laryngeal ultrasound and vocal fold movement in the pediatric cardiovascular intensive care unit, Laryngoscope 127 (1) (2017 Jan) 167e172. [7] P. Woo, T.F. Isseroff, A. Parasher, A. Richards, M. Sivak, Laryngeal Electromyographic findings in patients with vocal fold motion asymmetry, Laryngoscope 126 (8) (2016 Aug) E273eE277. [8] B.C. Jun, H.T. Kim, H.S. Kim, S.H. Cho, Clinical feasibility of the new technique of functional 3D laryngeal CT, Acta Otolarngol. 125 (7) (2015 Jul) 774e778. [9] Y.C. Liu, I. Varier, J. Ongkasuwan, The use of audiometric measurement for assessment of vocal fold function in post-extubation infants, JAMA Otolaryngol. Head Neck Surg. (2017 Jun 29), http://dx.doi.org/10.1001/jamaoto.2017.0848 [Epub ahead of print].
Contents lists available at ScienceDirect
International Journal of Pediatric Otorhinolaryngology journal homepage: http://www.ijporlonline.com/
The reliability of flexible nasolaryngoscopy in the identification of vocal fold movement impairment in young infants Yi-Chun Carol Liu a, Tyler McElwee b, Mary Musso a, Tara L. Rosenberg a, Julina Ongkasuwan a, * a b
Department of Otolaryngology Head and Neck Surgery, Baylor College of Medicine/Texas Children's Hospital, Houston, TX, USA Baylor College of Medicine, Houston, TX, USA
a r t i c l e i n f o
a b s t r a c t
Article history: Received 1 June 2017 Received in revised form 5 July 2017 Accepted 7 July 2017 Available online 8 July 2017
Objective: Flexible nasolaryngoscopy (FNL) is considered the gold standard for evaluation of vocal fold mobility but there has been no data on the reliability of interpretation in the infant population. Visualization may be limited by excessive movement, secretions, or floppy supraglottic structures that prevent accurate diagnosis of vocal fold movement impairment (VFMI). We sought to evaluate the inter- and intra-rater reliability of FNL for the evaluation of VFMI in young infants. Study type: Case-control. Methods: Twenty infants were identified: 10 with VFMI and 10 normal as seen on FNL. Three pediatric otolaryngologists reviewed the video without sound and rated the presence and/or degree of vocal fold mobility. Twelve videos were repeated to assess intra-rater reliability. Results: There was substantial agreement between the reviewers regarding the identification normal vs. any type of VFMI (kappa ¼ 0.67) but only moderate agreement regarding the degree of vocal fold movement (kappa ¼ 0.49). Intra-rater reliability ranges from moderate to perfect agreement (kappa ¼ 0.48e1). Conclusion: FNL in infants is an extremely challenging procedure. Clinically, physicians frequently use the quality of the cry and the past medical and surgical history to help make a judgment of vocal fold movement when the view is suboptimal. These other factors, however, may bias the interpretation of the FNL. Without sound, there is only moderate inter-rater and variable intra-rater reliability for the identification of degree of movement on FNL. Otolaryngologists must be cognizant of the limitations of FNL when using it as a clinical tool or as a “gold standard” against which other modalities are measured. © 2017 Published by Elsevier Ireland Ltd.
Keywords: Flexible nasolaryngoscopy Larynx Vocal fold paralysis Vocal fold motion impairment Inter-rater reliability Intra-rater reliability Level of evidence: 3b case control study
1. Introduction
2. Methods
Flexible nasolaryngoscopy (FNL) is considered the gold standard for evaluation of vocal fold (VF) mobility; however, no data on the reliability of interpretation in the infant population exists [1e3]. Friedman reported difficulty with adequate visualization of the endolarynx of children under 3 years old using a single bedside FNL in up to 20% of the time [4]. Visualization may be limited by excessive movement, secretions, or floppy supraglottic structures thus prevent accurate diagnosis of vocal fold movement impairment (VFMI). In this study, we sought to evaluate the inter- and intra-rater reliability of FNL for the evaluation of VFMI in young infants.
With institutional review board approval, the recorded laryngoscopies of twenty consecutive infants were identified from the pediatric otolaryngology flexible nasolaryngoscopy database using CPT code of 31575 and 92511 and ICD-10 code of J38: 10 with VFMI and 10 with normal vocal fold mobility as diagnosed on FNL by the treating fellowship trained pediatric otolaryngology attending. Three fellowship trained pediatric otolaryngologists, who were not involved in the clinical care the patients, reviewed the videos, in random order, without sound or clinical history. Twelve videos were randomly selected to be repeated to assess intra-rater reliability but not all twenty videos were included to prevent bias on examination. The reviewers rated the degree of vocal fold mobility (normal, partial, paralysis) on each side. Inter- and intra-rater agreement between reviewers were
* Corresponding author. 6701 Fannin, Suite 640, Houston, TX 77030, USA. E-mail address: [email protected] (J. Ongkasuwan). http://dx.doi.org/10.1016/j.ijporl.2017.07.005 0165-5876/© 2017 Published by Elsevier Ireland Ltd.
158
Y.-C.C. Liu et al. / International Journal of Pediatric Otorhinolaryngology 100 (2017) 157e159
calculated using Cohen and Fleiss' kappa coefficients respectively, with weighted kappa coefficients calculated for agreement of movement and types of movement (normal, paresis or paralysis). Using the Cohen's kappa statistical metric, value < 0 indicates no agreement, 0e0.20 as slight, 0.21e0.40 as fair, 0.41e0.60 as moderate, 0.61e0.80 as substantial, and 0.81e1 as almost perfect agreement. Using the Fleiss' kappa statistical metric, value < 0.40 indicates poor agreement, 0.41e0.75 as fair to good, and 0.76e1 as excellent [5]. Raw data from the reviewers are included in Table 1. 3. Results The infants ranged in age from 12 to 330 days (median 31.5 days) at the time of FNL. There was substantial agreement between the reviewers regarding the identification of normal vs. any type of VFMI (paresis or paralysis) (kappa ¼ 0.67, 95% CI: 0.47e0.87). Agreement regarding the degree of VF movement (normal vs. paresis vs. paralysis) was moderate (kappa ¼ 0.49, 95% CI: 0.35e0.64). Reviewer 1 had moderate intra-rater reliability for the presence of VFMI (kappa ¼ 0.53, 95% CI: 0.06e0.99) and for the degree of movement (kappa ¼ 0.48, 95% CI: 0.06e0.90). Reviewer 2 had substantial intra-rater agreement for the identification of VFMI (kappa ¼ 0.82, 95% CI: 0.49e1) and the degree of movement (kappa ¼ 0.87, 95% CI: 0.62e1). Reviewer 3 had perfect intra-rater reliability for the both the presence of VFMI (kappa ¼ 1, 95% CI: 1-1) and the degree of movement (kappa ¼ 1, 95% CI:1-1). 4. Discussion Vocal fold movement impairment (VFMI) can be a result of
mechanical restriction or neuronal cause. VFMI is not a simple allor-none diagnosis as there can be varying degrees of vocal fold movement. Flexible nasolaryngoscopy (FNL) is the most frequently used diagnostic tool and is considered the gold standard for evaluation of vocal fold mobility but there is no data on the reliability of interpretation in the infant population. Madden et al. reported that, in adults, inter-rater and intra-rater reliability of FNL is consistent amongst general otolaryngologists and in fellowship trained laryngologists with an inter-rater reliability of 95% [2]. However, FNL in neonates and infants can be much more challenging than in adults. Factors such as excessive movement of the patient, secretions, or floppy supraglottic structures obscure the view of the larynx. Other diagnostic methods have been studied, including laryngeal ultrasound which has been proposed to be an effective supplemental diagnostic tool in the neonatal population with congenital cardiac issues [4,6]. Laryngeal electromyography (LEMG) has also been studied with similar results to FNL but is still susceptible to individual variation in interpretation [7]. In addition, the placement of LEMG needles into the intrinsic laryngeal musculature is not generally feasible in infants and children without anesthesia. Lastly, functional 3D computer tomography has been proposed as a diagnostic tool for VFMI. However, the limitation of the use of computer tomography is the radiation exposure and the standard errors inherent in reconstructing images making it not an ideal diagnostic tool in pediatric patient population [8]. Secondary to the difficulty of FNL visualization of the larynx in the neonatal and infant population, otolaryngologists frequently use additional information such as clinical history, surgical history and cry volume [9]. For our study, we purposely chose to eliminate the audio from the videos to decrease the bias of the raters by limiting them to visual evaluation only.
Table 1 FNL video key and reviewer results. Study ID
Video
Key
Reviewer 1
Review 2
Review 3
1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 10
L VF Paresis B/l Paresis L VF Paresis B/l Paresis Normal L VF Paresis Normal Normal L VF Paralysis L VF Paresis
Normal Normal L VF Paralysis L VF Paralysis L VF Paralysis L VF Paresis Normal Normal L VF Paralysis L VF Paralysis
R VF Paresis B/l Paralysis L VF Paresis Normal Normal L VF Paralysis Normal Normal L VF Paralysis L VF Paresis
11 12 13 14 15 16 17 18 19 20 16 5 7 8 9 3 15 6 19 17 4 18
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
L VF Paresis B/l Paresis L VF Paresis Normal Normal L VF Paralysis Normal Normal L VF Paralysis R VF Paresis L VF ParalysisR VF Paresis L VF ParalysisR VF Paresis L VF Paralysis Normal Normal L VF Paralysis Normal L VF Paresis L VF Paralysis Normal Normal L VF Paralysis Normal L VF Paralysis Normal Normal Normal L VF Paralysis L VF Paresis L VF Paresis L VF Paralysis L VF Paralysis Normal Normal Normal
Normal Normal B/l Paralysis Normal L VF Paralysis L VF Paralysis Normal Normal L VF Paresis Normal L VF Paralysis Normal Normal Normal L VF Paralysis Normal L VF Paralysis Normal Normal Normal B/l VF Paralysis Normal
Normal Normal B/l VF Paralysis Normal L VF Paralysis L VF Paralysis Normal Normal L VF Paresis Normal L VF Paralysis Normal Normal Normal L VF Paralysis L VF Paralysis L VF Paralysis L VF Paresis L VF Paresis Normal L VF Paralysis Normal
Normal Normal R VF Paralysis Normal L VF Paresis L VF Paralysis Normal Normal L VF Paresis Normal L VF Paralysis Normal Normal Normal L VF Paralysis L VF Paresis L VF Paresis L VF Paralysis L VF Paresis Normal Normal Normal
Y.-C.C. Liu et al. / International Journal of Pediatric Otorhinolaryngology 100 (2017) 157e159
In the adult literature, most raters have high intra-rater reliability but the inter-rater reliability varies, especially in identifying the types of VFMI (paralysis vs. paresis) [2]. Our data showed substantial inter-rater agreement when differentiating normal vs. any movement impairment; however, the agreement fell to moderate when trying to differentiate paralysis from paresis. The interrater reliability in our study was lower than the previous published adult literature. The difference is likely a reflection of the aforementioned challenges when performing infant FNL. In order to improve on inter-rater reliability of identifying VFMI, there is continuous discussion of establishing a classification system for VFMI. However, the difficulty with a classification system remains that there is no easily obtainable objective measurement that can be used. Rosow et al. suggested rating categories with explicit definitions and examples, but recognized individual variation in laryngeal anatomy and innervation resulted in a heterogenous appearance of VFMI [1]. In addition, the neonatal and infant population pose additional challenges in obtaining an optimal view of the larynx to facilitate accurate diagnosis. Further research is needed to develop a universally acceptable classification system for VFMI. 5. Conclusion FNL in neonates and infants can be challenging. Clinically, physicians frequently use the quality of the cry and the past medical and surgical history to help make a judgment of vocal fold movement when the view is suboptimal. These other factors, however, may bias the interpretation of the FNL. Without sound, we found that there is only moderate inter-rater reliability for the identification of the degree of movement on FNL. Otolaryngologists must
159
be cognizant of the limitations of FNL when using it as a clinical tool or as a “gold standard” against which other modalities are measured. Conflict of interest None. Financial disclosures None. References [1] D.E. Rosow, L. Sulica, Laryngoscopy of vocal fold paralysis: evaluation of consistency of clinical findings, Laryngoscope 120 (7) (2010 Jul) 1376e1382. [2] L.L. Madden, C.A. Rosen, Evaluation of vocal fold motion abnormalities: are we all seeing the same thing? J. Voice 31 (1) (2017 Jan) 72e77. [3] L.1 Sulica, A. Blitzer, Vocal fold paresis: evidence and controversies, Pin Otolaryngol. Head Neck Surg. 15 (3) (2007 Jun) 159e162. [4] E.M. Friedman, Role of ultrasound in the assessment of vocal cord function in infants and children, Ann. Otol. Rhinol. Laryngol. 106 (1997) 199e209. [5] J.L. Fleiss, Statistical Methods for Rates and Proportions, second ed., Wiley, New York, NY, 1981. [6] J.1 Ongkasuwan, E.2 Ocampo, B.3 Tran, Laryngeal ultrasound and vocal fold movement in the pediatric cardiovascular intensive care unit, Laryngoscope 127 (1) (2017 Jan) 167e172. [7] P. Woo, T.F. Isseroff, A. Parasher, A. Richards, M. Sivak, Laryngeal Electromyographic findings in patients with vocal fold motion asymmetry, Laryngoscope 126 (8) (2016 Aug) E273eE277. [8] B.C. Jun, H.T. Kim, H.S. Kim, S.H. Cho, Clinical feasibility of the new technique of functional 3D laryngeal CT, Acta Otolarngol. 125 (7) (2015 Jul) 774e778. [9] Y.C. Liu, I. Varier, J. Ongkasuwan, The use of audiometric measurement for assessment of vocal fold function in post-extubation infants, JAMA Otolaryngol. Head Neck Surg. (2017 Jun 29), http://dx.doi.org/10.1001/jamaoto.2017.0848 [Epub ahead of print].