- Email: [email protected]
Readability of online patient education materials for velopharyngeal insufficiency
Accepted Manuscript Readability of online patient education materials for velopharyngeal insufficiency Deborah X. Xie, Ray Y. Wang, Siva Chinnadurai PII:
S0165-5876(17)30449-4
DOI:
10.1016/j.ijporl.2017.09.016
Reference:
PEDOT 8703
To appear in:
International Journal of Pediatric Otorhinolaryngology
Received Date: 7 July 2017 Revised Date:
15 September 2017
Accepted Date: 17 September 2017
Please cite this article as: D.X. Xie, R.Y. Wang, S. Chinnadurai, Readability of online patient education materials for velopharyngeal insufficiency, International Journal of Pediatric Otorhinolaryngology (2017), doi: 10.1016/j.ijporl.2017.09.016. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Title: Readability of Online Patient Education Materials for Velopharyngeal Insufficiency Authors: Deborah X. Xie1; Ray Y. Wang1; Siva Chinnadurai, MD, MPH1
RI PT
Affiliations: 1Vanderbilt University Medical Center, Department of Otolaryngology-Head & Neck Surgery, Nashville, TN USA
Conflict of interest: none
AC C
EP
TE D
M AN U
Corresponding author: Siva Chinnadurai, MD, MPH [email protected] 2200 Children’s Way Doctor’s Office Tower, 7th floor Nashville TN, 37232
SC
Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
ACCEPTED MANUSCRIPT
ABSTRACT Objective: Evaluate the readability of online and mobile application health information about velopharyngeal insufficiency (VPI).
RI PT
Methods: Top website and mobile application results for search terms “velopharyngeal insufficiency”, “velopharyngeal dysfunction”, “VPI”, and “VPD” were analyzed. Readability was determined using 10 algorithms with Readability Studio Professional Edition
SC
(Oleander Software Ltd; Vandalia, OH). Subgroup analysis was performed based on search
journal, or professional organization.
M AN U
term and article source – academic hospital, general online resource, peer-reviewed
Results: 18 unique articles were identified. Overall mean reading grade level was a 12.89±2.9. The highest reading level among these articles was 15.47 – approximately the level of a college senior. Articles from “velopharyngeal dysfunction” had the highest mean
TE D
reading level (13.73±2.11), above “velopharyngeal insufficiency” (12.30±1.56) and “VPI” (11.66±1.70). Articles from peer-reviewed journals had the highest mean reading level (15.35±2.79), while articles from academic hospitals had the lowest (12.81±1.66). There
EP
were statistically significant differences in reading levels between the different search
AC C
terms (P<0.01) and article source types (P<0.05). Only one mobile application was identified with VPI information, with a readability of 10.68. Conclusion: Despite published reading level guidelines, online patient education materials for VPI are disseminated with language too complex for most readers. There is also a lack of VPI-related mobile application data available for patients. Patients will benefit if future updates to websites and disseminated patient information are undertaken with health literacy in mind. Future studies will investigate patient comprehension of these materials.
ACCEPTED MANUSCRIPT
KEY WORDS: velopharyngeal insufficiency, readability, patient education, online health
AC C
EP
TE D
M AN U
SC
RI PT
information
ACCEPTED MANUSCRIPT
1. Introduction Health literacy has been suggested to predict healthcare outcomes and costs [1, 2]. A study on primary care visits estimated that 50% of patients leave the office without a full
RI PT
understanding of what the physician has told them [3]. This highlights a need to produce patient resources that are accessible and readable by patients with limited health literacy. Healthcare organizations, including the American Medical Association (AMA) and National
SC
Institutes of Health (NIH), have recommended that patient education materials be written no higher than a sixth- to eighth-grade reading level [2, 4]. Prior assessment of patient
M AN U
education material in otolaryngology and other medical specialties have found that many websites and articles are written as high as a 17th-grade level [5-17]. Wong and Levi describe that the readability of materials published by the American Academy of Otolaryngology-Head and Neck Surgery Foundation is trending towards the positive, with
TE D
material becoming more readable [9]. However, with the increasing number of adults relying on technology for health information there is still significant room for improvement [18].
EP
Development of patient education materials on velopharyngeal insufficiency (VPI)
AC C
can present a challenging problem given its heterogeneous causes and multidisciplinary management strategies. The complexity of this disease necessitates use of language that is understandable by patients and families. One prior study evaluated the online cleft lip and palate-related materials to be at a mean reading level between 8th and 9th grade [19]. However, to our knowledge, there has been no published data regarding VPI education materials available to patients. We aim to assess the readability of online and mobile application VPI information that patients are likely to use for supplementary information.
ACCEPTED MANUSCRIPT
2. Materials and Methods We conducted a descriptive study evaluating and reporting on the readability of
RI PT
online and mobile application patient resources for velopharyngeal insufficiency. A Google search for velopharyngeal insufficiency was performed using Google Chrome. Mobile applications were identified using the Apple App Store (Apple, Inc., Cupertino, CA),
SC
accessed in May 2017. We evaluated four distinct search terms representing a range and variation of terms patients might use to identify information regarding their diagnosis.
M AN U
These terms include “velopharyngeal insufficiency”, “VPI”, “velopharyngeal dysfunction”, and “VPD”. The first ten results for “velopharyngeal insufficiency” and “velopharyngeal dysfunction” were selected for our analysis. Given that “VPI” and “VPD” are acronyms for multiple organizations and entities unrelated to velopharyngeal insufficiency, we limited
TE D
our analysis to the applicable results from the first two pages of the search (20 results) for each. Given the fewer number of mobile applications relative to web pages, we also included the search term “otolaryngology” to identify applications that may have
EP
information on velopharyngeal insufficiency contained in the material.
AC C
For the online portion, the web browser’s history, cookies, and preferences were cleared prior to performing each of the searches; location information and user preferences were also disabled to eliminate biased results based on previous searches. Non-text and non-English media were excluded. Pictures, captions, hyperlinks, and advertisements were deleted. Headings and sentence fragments were also removed. Inclusion criteria for mobile applications were those that had a text section on velopharyngeal insufficiency. NonEnglish applications were included.
ACCEPTED MANUSCRIPT
Readability analysis was performed using Readability Studio Professional Edition (Oleander Software Ltd; Vandalia, OH). Reading level was assessed using the following scales: Bormuth Grade Placement, Coleman-Liau, New Dale-Chall, Fry readability formula,
RI PT
Gunning-Fog, Harris-Jacobson, LIX, Raygor Readability Estimate, RIX, and SMOG (Table 1). Output from the program represents the reading grade level at which the text is written. For scales that provided a grade level range, the median of the range was used. If the upper
SC
limit of the readability scale was exceeded by the text, the value of the upper limit was used. Batch analyses were performed for the following website groups: All documents from all search terms
•
Documents by search term (“velopharyngeal insufficiency”, “VPI”, “velopharyngeal dysfunction”, and “VPD”)
•
M AN U
•
Websites by institution (information from academic hospitals, general online
TE D
resources, peer-reviewed journals, professional organizations) One-way ANOVA was used to establish statistical differences between groups, with
AC C
3. Results
EP
significance set a priori at P<0.05.
3.1 Website results
A total of 22 articles were identified – 10 articles each for “velopharyngeal
insufficiency” and “velopharyngeal dysfunction”, two results for “VPI”, and no results for “VPD.” Of these articles, there were 18 unique results. The mean reading level for each website across the 10 tests is summarized in Table 2. The mean reading grade level of all articles was 12.9±2.9. The lowest mean reading level was 8.23 for
ACCEPTED MANUSCRIPT
www.seattlechildrens.org. The highest mean reading level was 15.5 for www.ncbi.nlm.nih.gov - approximately the level of a college senior. Figure 1 shows the composite distribution of reading levels of all articles for each test.
RI PT
3.2 Mobile application results
No mobile applications were identified with the symptom-specific search terms. A total of 39 English mobile applications were identified with the search term
SC
“otolaryngology”. Over half of the applications (61.5%) were targeted towards ENT
providers, including mobile applications of conferences and surgical simulators. Only 5
M AN U
applications were targeted towards a patient audience. One of these five applications contained information on velopharyngeal insufficiency. This was from a mobile application of an otolaryngology clinic, with the page on velopharyngeal insufficiency accessible via an in-application search. The same information on VPI was also listed on their webpage, which
readability level of 10.68.
TE D
had not been identified in our Google search. The text in this mobile application page had a
3.3 Comparison by search terms
EP
The reading level of website articles by search term is shown in Figure 2 and Figure
AC C
3. The mean reading level by search term was 12.30±1.56 (velopharyngeal insufficiency), 11.66±1.70 (VPI), and 13.73±2.11 (velopharyngeal dysfunction). One-factor ANOVA showed a statistically significant difference between means (p<0.05). 3.4 Comparison by website institution The websites were categorized by the type of institution that published the patient education materials, as shown in Figure 4. The lowest mean reading level was a 12.81±1.66 from 10 articles published by academic hospitals. Two articles were published in peer-
ACCEPTED MANUSCRIPT
reviewed journal articles with a mean reading level of 15.35±2.79 for the highest mean. Additionally, there were 3 articles from professional organizations with a mean reading level of 12.95±1.63, and 3 articles from general online resources with a mean reading level
RI PT
of 14.42±2.26. One factor ANOVA showed statistically significant differences between each type of institution (p<0.01). 4. Discussion
SC
The internet provides patients access to a large volume of information. Websites such as dedicated patient education websites, blogs, and healthcare institutions are easily
M AN U
accessible. Approximately 60% of adults have used the internet to search for health-related information.8 The utility of these resources is dependent upon a patient’s ability to understand them. Previous studies in other medical and surgical fields have shown that the readability of most patient education materials is far above the average American reading
TE D
level.5-17 In this study, we analyze the readability of online and mobile application patient health information regarding velopharyngeal insufficiency. Velopharyngeal insufficiency is a common problem treated by otolaryngologists.
EP
There are a variety of congenital and acquired etiologies of varying severity with a range of
AC C
treatment options, which can be difficult for patients to understand. Despite guidelines by the AMA and NIH, the material analyzed in this study is written with language too complex for most readers. The simplest article of our analysis was written above an 8th grade reading level. A college education was necessary to understand over 60% of articles. These findings support previously published data that online patient education material is not appropriately readable by many patients and families.
ACCEPTED MANUSCRIPT
A complete lack of VPI-specific mobile applications highlights an area for improvement. Even within the otolaryngology-specific applications identified, only 12.8% of applications appeared to be designed for patient consumption, and only one of these
RI PT
contained information on velopharyngeal insufficiency. A mobile application may be an efficient medium through which interdisciplinary care information may be disseminated for these complex patients.
SC
There was significant variance between the readability of the 18 articles (p<0.01). This finding is unsurprising considering the diverse nature of websites analyzed in our
M AN U
search. The general public was the intended target audience for most search results, but we also encountered two articles from peer-reviewed journals. These journal articles were written at a mean reading level of 15.35±2.79. Since the target audience of peer-reviewed journal articles is healthcare professionals, it would be inappropriate to suggest this
TE D
material should be simplified to a 6th-8th grade reading level. However, patients could easily find these articles by searching the internet, as both articles were within the top 10 search results for “velopharyngeal dysfunction”. Their presence in our search above articles
EP
written for patients may reflect a flaw in the search engine’s algorithm. As technological
AC C
advances continue, we suggest that these algorithms be modified to provide results for their appropriate target audience. However, it was important that the results from peerreviewed journals be included in our study to accurately represent the breadth of information available to patients, and the higher literacy level needed to comprehend these resources. Moreover, even when excluding these sources intended for medical professionals, the mean reading level is still elevated at 12.75±2.17.
ACCEPTED MANUSCRIPT
Academic hospitals provided articles that had the lowest grade level readability score with a mean reading level of 12.18±1.66. Despite being above the literacy level of a high school graduate, academic hospitals are publishing material that is closer to the
RI PT
recommended reading level compared to general online resources, professional
organizations, and peer-reviewed journals. Comparing results by search term, we again see significant variation in the readability (p<0.05). Results for “VPI” had the lowest mean
SC
scores with a mean level of 11.66±1.70. Unfortunately, these results may not be very accessible to patients through a Google search. The use of acronyms yielded many
M AN U
extraneous results unrelated to medicine or otolaryngology. There were only two applicable links of the first 20 results with search term “VPI” and no applicable links for “VPD”.
Interestingly, a few notable websites focusing on the clinical care of patients with
TE D
VPI were not identified with our Google Search, including the American Cleft PalateCraniofacial Association (ACPA) and the Cleft Palate Foundation (CPF) [20]. While families will search the web independently, providers may direct patients towards these
EP
professional organizations for information. In analyzing ten randomly selected patient
AC C
information pages from the ACPA website, the mean readability was 10.73±2.87, ranging from 7.37 to 15.22. This sample was just significantly improved from our Google Search results (p=0.05), though this may be skewed by the low sample sizes. The ACPA provides detailed definitions and answers to frequently asked questions without excessive medical jargon, which other sources may aim to emulate. There are some limitations to this study. The program uses calculated readability as a proxy for patient comprehension of material. However, there are inherent flaws of
ACCEPTED MANUSCRIPT
readability calculators that make this substitution imperfect. Calculated grade level is based on word length and sentence length, without assessing the accuracy or clarity of text. Simplifying text to a lower readability score does not ensure increased comprehension [9].
RI PT
Additionally, scores may be skewed due to the length of words found in medical jargon that are often used in these articles. The CDC emphasizes the use of visuals to convey
information, such as images or pictographs [10]. However, these visual aids were not
SC
assessed in our study. Finally, many of the scales used to calculate readability have a
maximum score, but this upper limit was not consistent across scales. Fifteen of the 18
M AN U
distinct articles had readability levels exceeding the upper limit of at least one readability scale used in our analysis. Not every formula may be appropriate for our study, as some do not have ability to detect differences between texts written at a reading level above the 11th
5. Conclusion
TE D
grade.
Online and mobile application materials on velopharyngeal insufficiency that are
EP
accessed by patients and families are written at levels above the current recommendations
AC C
by the AMA and NIH. Despite a wide variety of search results and website types, all 18 articles identified in our study were written above the recommended AMA and NIH guidelines. Future website updates should be geared towards improving the readability of patient education materials as part of a systematic approach to increase patient comprehension and provide patient-centered care.
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
References [1] Health literacy: report of the Council on Scientific Affairs. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association. JAMA. 281 (1999) 552-557. [2] B.D. Weiss. Health literacy and patient safety: Help patients understand. http://www.med.fsu.edu/userFiles/file/ahec_health_clinicians_manual.pdf, (accessed November 3 2016). [3] T. Bodenheimer. Transforming Practice. New Engl J Med. 359 (2008) 2086-2089. [4] How to Write Easy-to-Read Health Materials. MedlinePlus. http://www.nlm.nih.gov/medlineplus/etr.html, (accessed November 3 2016). [5] J.A. Eloy, S. Li, K. Kasabwala, N. Agarwal, D.R. Hansberry, S. Baredes, M. Setzen. Readability assessment of patient education materials on major otolaryngology association websites. Otolaryngol Head Neck Surg. 147 (2012) 848-854. [6] P.F. Svider, N. Agarwal, O.J. Choudhry, A.F. Hajart, S. Baredes, J.K. Liu, J.A. Eloy. Readability assessment of online patient education materials from academic otolaryngology-head and neck surgery departments. Am J Otolaryngol. 34 (2013) 31-35. [7] D.R. Hansberry, N. Agarwal, R. Shah, P.J. Schmitt, S. Baredes, M. Setzen, P.W. Carmel, C.J. Prestigiacomo, J.K. Liu, J.A. Eloy. Analysis of the readability of patient education materials from surgical subspecialties. Laryngoscope. 124 (2014) 405-412. [8] H. Rayess, G.F. Zuliani, A. Gupta, P.F. Svider, A.J. Folbe, J.A. Eloy, M.A. Carron. Critical Analysis of the Quality, Readability, and Technical Aspects of Online Information Provided for Neck-Lifts. JAMA Facial Plast Surg. 19 (2017) 115-120. [9] K. Wong, J.R. Levi. Readability Trends of Online Information by the American Academy of Otolaryngology-Head and Neck Surgery Foundation. Otolaryngol Head Neck Surg. 156 (2017) 96-102. [10] K. Wong, J.R. Levi. Partial Tonsillectomy. Ann Otol Rhinol Laryngol. 126 (2017) 192198. [11] K. Wong, J.R. Levi. Readability of pediatric otolaryngology information by children's hospitals and academic institutions. Laryngoscope. 127 (2017) E138-e144. [12] V. Balakrishnan, Z. Chandy, A. Hseih, T.L. Bui, S.P. Verma. Readability and Understandability of Online Vocal Cord Paralysis Materials. Otolaryngol Head Neck Surg. 154 (2016) 460-464. [13] D.V. Cherla, S. Sanghvi, N. Agarwal, J.A. Eloy, W.T. Couldwell, J.K. Liu. Analysis of internet-based patient education materials related to pituitary tumors. Endocrine Pract. 20 (2014) 1044-1050. [14] M.W. Chu, J.A. Cook, S.S. Tholpady, C.E. Schmalbach, A. Momeni. Facial Plastic Surgery Patient Resources Exceed National Institute Recommendations. J Craniofac Surg. 28 (2017) 759-763. [15] G. Huang, C.H. Fang, N. Agarwal, N. Bhagat, J.A. Eloy, P.D. Langer. Assessment of online patient education materials from major ophthalmologic associations. JAMA Ophthalmol. 133 (2015) 449-454. [16] A.P. O'Connell Ferster, A. Hu. Evaluating the quality and readability of Internet information sources regarding the treatment of swallowing disorders. Ear Nose Throat J. 96 (2017) 128-138.
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
[17] L. Wozney, J. Chorney, A. Huguet, J.S. Song, E.F. Boss, P. Hong. Online Tonsillectomy Resources: Are Parents Getting Consistent and Readable Recommendations? Otolaryngol Head Neck Surg. 156 (2017) 844-852. [18] M. Bobian, A. Kandinov, N. El-Kashlan, P.F. Svider, A.J. Folbe, R. Mayerhoff, J.A. Eloy, S.N. Raza. Mobile applications and patient education: Are currently available GERD mobile apps sufficient? Laryngoscope. (2016). Epub. [19] Antonarakis GS, Kiliaridis S. Internet-derived information on cleft lip and palate for families with affected children. Cleft Palate Craniofac J. 46 (2009) 75-80. [20] Cleft & Craniofacial Educational Materials. ACPA. http://www.cleftline.org/who-weare/what-we-do/publications, (accessed September 14 2017).
ACCEPTED MANUSCRIPT
FIGURE LEGENDS: Preference for color photographs: online only Figure 1: Box and whisker plot for reading levels of search results of all articles. Dots
RI PT
represent outliers.
Velopharyngeal dysfunction. Figure 3: Mean reading levels by search term.
AC C
EP
TE D
M AN U
Figure 4: Mean reading levels by website institution.
SC
Figure 2: Reading levels by search term (A) Velopharyngeal insufficiency, (B) VPI, and (C)
ACCEPTED MANUSCRIPT
Table 1: List of readability formulas provided by Readability Studio Professional Edition Formula
L = total letters; W = total words; D = number of familiar Dale-Chall words; S = total sentences Coleman-Liau Index
ܮ ܦଷ 3.761864 + 1.053153 ൬ ൰ − 2.138595 ൬ ൰ ܹ ܹ ܹ + 0.152832 ൬ ൰ ܵ ܹ ଶ − 0.002077 ൬ ൰ ܵ 0.0588 × ܮ− 0.296 × ܵ − 15.8
RI PT
Readability Tool Bormuth Grade placement
L = average number of letters per 100 words; S = average number of sentences per 100 words New Dale-Chall
Gunning-Fog Index
SC
ܷܹ ܹ ൰ + 0.16 ൬ ൰ + 0.642 ܹ ܵ This raw core is rounded to one decimal place and converted to readability based on the Harris Jacobson Table ܹ ܹܮ + × 100 ܵ ܹ This value is converted to a grade level based on the LIX index table Select three 100-word passages from the beginning, middle, and end of a text. Calculate average number of sentences and average word length (with ≥6 letters) in each passage. Plot on Raygor estimate graph to determine reading level. ܮ ܵ This value is converted to a grade level based on the RIX Index table
TE D
W = total words; S = total sentences; CW = complex words Harris-Jacobson Wide Range Readability Formula
Select three 100-word passages from the beginning, middle, and end of a text. Calculate the average number of sentences and syllables in each passage. Plot on Fry Graph to determine reading level. ܹ ܹܥ 0.4 ൬ ൰ + 100 ൬ ൰൨ ܵ ܹ
M AN U
CW = complex words (not on Dale-Chall word list); W = total words; S = total sentences Fry readability formula
ܹܥ ܹ 15.79 ൬ ൰ + 0.0496 ൬ ൰ + 3.6365 ܹ ܵ
EP
UW = uncommon words; W = total words; S = total sentences Läsbarhetsindex (LIX)
AC C
W = total words; S = total sentences; LW = total number of long words (>6 letters) Raygor Readability Estimate
Rate Index (RIX)
LW = long words; S = total sentences Simple Measure of Gobbledygook (SMOG) W = number of words with >3 syllables; S = total number of sentences
24.5 ൬
30 1.043 × ඨܹ × ൬ ൰ + 3.129 ܵ
ACCEPTED MANUSCRIPT
Table 2: Mean reading level by article, listed in order of appearance on Google Search * indicates duplicate website identified through multiple search terms. Website
Institution
Title
Reading level
Velopharyngeal insufficiency
*http://www.seattlechildrens.or g/medicalconditions/chromosomalgenetic-conditions/vpi *http://emedicine.medscape.co m/article/873018-overview *https://en.wikipedia.org/wiki/ Velopharyngeal_insufficiency *http://www.cincinnatichildrens .org/health/v/velopharyngeal http://www.chsc.org/Main/Veip haryngeal-Insufficiency-VPI.aspx http://www.merckmanuals.com /professional/ear,-nose,-andthroat-disorders/oral-andpharyngealdisorders/velopharyngealinsufficiency http://www.childrenshospital.va nderbilt.org/services.php?mid=7 279 http://www.apraxiakids.org/library/velopharyngealinsufficiency-vpi-and-apraxia http://www.craniofacial.vcu.edu /conditions/vpi.html https://medicine.yale.edu/surge ry/plastics/care/cranio/velopha ryngeal.aspx https://www.cincinnatichildrens .org/service/s/speech/specialtyclinics/vpiclinic/?utm_source=VPI%2BClini c&utm_medium=shortcut&utm_c ampaign=vpi http://www.seattlechildrens.org /medical-conditions/eye-earnose-throat-conditions/vpd *http://www.cincinnatichildrens .org/health/v/velopharyngeal *http://www.seattlechildrens.or g/medicalconditions/chromosomalgenetic-conditions/vpi http://emedicine.medscape.com /article/994975-overview *http://emedicine.medscape.co m/article/873018-overview *https://en.wikipedia.org/wiki/ Velopharyngeal_insufficiency
Academic hospital
Velopharyngeal Dysfunction
10.67
General online resource General online resource Academic hospital Professional organization Professional organization
Velopharyngeal Insufficiency Velopharyngeal Insufficiency Velopharyngeal Dysfunction Velopharyngeal Insufficiency Velopharyngeal Insufficiency
TE D
M AN U
SC
RI PT
Search term
AC C
EP
VPI
Velopharyngeal dysfunction
Academic hospital
14.8
13.69 9.84
11.45 15.47
12.65
Academic hospital Academic hospital
What is Velopharyngeal Insufficiency Velopharyngeal Insufficiency (VPI) and apraxia Velopharyngeal Insufficiency – VPI Velopharyngeal Insufficiency (VPI)
Academic hospital
About the VPI clinic
15.09
Academic hospital
Velopharyngeal Dysfunction
8.23
Academic hospital Academic hospital
Velopharyngeal Dysfunction Velopharyngeal Dysfunction
9.84
General online resource General online resource General online resource
Velopharyngeal Dysfunction Velopharyngeal Insufficiency Velopharyngeal Insufficiency
14.78
Professional organization
11.94
10.89 11.6
10.67
14.8 13.69
ACCEPTED MANUSCRIPT
Peer-reviewed journal Academic hospital
Velopharyngeal Dysfunction Velopharyngeal Dysfunction
15.22
Academic hospital
Velopharyngeal Dysfunction Program Velopharyngeal Dysfunction
14.83
Velopharyngeal nomenclature: incompetence, inadequacy, insufficiency, and dysfunction
15.47
Peer-reviewed journal
M AN U TE D EP AC C
14.49
RI PT
Academic hospital
SC
http://www.ncbi.nlm.nih.gov/p mc/articles/PMC3706038 http://www.chop.edu/centersprograms/velopharyngealdysfunction-program http://www.nationwidechildren s.org/velopharyngealdysfunction-program https://www.rileychildrens.org/ health-info/velopharyngealdysfunction http://digital.library.pitt.edu/c/c leftpalate/pdf/e20986v25n4.13. pdf
13.52
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
S0165-5876(17)30449-4
DOI:
10.1016/j.ijporl.2017.09.016
Reference:
PEDOT 8703
To appear in:
International Journal of Pediatric Otorhinolaryngology
Received Date: 7 July 2017 Revised Date:
15 September 2017
Accepted Date: 17 September 2017
Please cite this article as: D.X. Xie, R.Y. Wang, S. Chinnadurai, Readability of online patient education materials for velopharyngeal insufficiency, International Journal of Pediatric Otorhinolaryngology (2017), doi: 10.1016/j.ijporl.2017.09.016. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Title: Readability of Online Patient Education Materials for Velopharyngeal Insufficiency Authors: Deborah X. Xie1; Ray Y. Wang1; Siva Chinnadurai, MD, MPH1
RI PT
Affiliations: 1Vanderbilt University Medical Center, Department of Otolaryngology-Head & Neck Surgery, Nashville, TN USA
Conflict of interest: none
AC C
EP
TE D
M AN U
Corresponding author: Siva Chinnadurai, MD, MPH [email protected] 2200 Children’s Way Doctor’s Office Tower, 7th floor Nashville TN, 37232
SC
Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
ACCEPTED MANUSCRIPT
ABSTRACT Objective: Evaluate the readability of online and mobile application health information about velopharyngeal insufficiency (VPI).
RI PT
Methods: Top website and mobile application results for search terms “velopharyngeal insufficiency”, “velopharyngeal dysfunction”, “VPI”, and “VPD” were analyzed. Readability was determined using 10 algorithms with Readability Studio Professional Edition
SC
(Oleander Software Ltd; Vandalia, OH). Subgroup analysis was performed based on search
journal, or professional organization.
M AN U
term and article source – academic hospital, general online resource, peer-reviewed
Results: 18 unique articles were identified. Overall mean reading grade level was a 12.89±2.9. The highest reading level among these articles was 15.47 – approximately the level of a college senior. Articles from “velopharyngeal dysfunction” had the highest mean
TE D
reading level (13.73±2.11), above “velopharyngeal insufficiency” (12.30±1.56) and “VPI” (11.66±1.70). Articles from peer-reviewed journals had the highest mean reading level (15.35±2.79), while articles from academic hospitals had the lowest (12.81±1.66). There
EP
were statistically significant differences in reading levels between the different search
AC C
terms (P<0.01) and article source types (P<0.05). Only one mobile application was identified with VPI information, with a readability of 10.68. Conclusion: Despite published reading level guidelines, online patient education materials for VPI are disseminated with language too complex for most readers. There is also a lack of VPI-related mobile application data available for patients. Patients will benefit if future updates to websites and disseminated patient information are undertaken with health literacy in mind. Future studies will investigate patient comprehension of these materials.
ACCEPTED MANUSCRIPT
KEY WORDS: velopharyngeal insufficiency, readability, patient education, online health
AC C
EP
TE D
M AN U
SC
RI PT
information
ACCEPTED MANUSCRIPT
1. Introduction Health literacy has been suggested to predict healthcare outcomes and costs [1, 2]. A study on primary care visits estimated that 50% of patients leave the office without a full
RI PT
understanding of what the physician has told them [3]. This highlights a need to produce patient resources that are accessible and readable by patients with limited health literacy. Healthcare organizations, including the American Medical Association (AMA) and National
SC
Institutes of Health (NIH), have recommended that patient education materials be written no higher than a sixth- to eighth-grade reading level [2, 4]. Prior assessment of patient
M AN U
education material in otolaryngology and other medical specialties have found that many websites and articles are written as high as a 17th-grade level [5-17]. Wong and Levi describe that the readability of materials published by the American Academy of Otolaryngology-Head and Neck Surgery Foundation is trending towards the positive, with
TE D
material becoming more readable [9]. However, with the increasing number of adults relying on technology for health information there is still significant room for improvement [18].
EP
Development of patient education materials on velopharyngeal insufficiency (VPI)
AC C
can present a challenging problem given its heterogeneous causes and multidisciplinary management strategies. The complexity of this disease necessitates use of language that is understandable by patients and families. One prior study evaluated the online cleft lip and palate-related materials to be at a mean reading level between 8th and 9th grade [19]. However, to our knowledge, there has been no published data regarding VPI education materials available to patients. We aim to assess the readability of online and mobile application VPI information that patients are likely to use for supplementary information.
ACCEPTED MANUSCRIPT
2. Materials and Methods We conducted a descriptive study evaluating and reporting on the readability of
RI PT
online and mobile application patient resources for velopharyngeal insufficiency. A Google search for velopharyngeal insufficiency was performed using Google Chrome. Mobile applications were identified using the Apple App Store (Apple, Inc., Cupertino, CA),
SC
accessed in May 2017. We evaluated four distinct search terms representing a range and variation of terms patients might use to identify information regarding their diagnosis.
M AN U
These terms include “velopharyngeal insufficiency”, “VPI”, “velopharyngeal dysfunction”, and “VPD”. The first ten results for “velopharyngeal insufficiency” and “velopharyngeal dysfunction” were selected for our analysis. Given that “VPI” and “VPD” are acronyms for multiple organizations and entities unrelated to velopharyngeal insufficiency, we limited
TE D
our analysis to the applicable results from the first two pages of the search (20 results) for each. Given the fewer number of mobile applications relative to web pages, we also included the search term “otolaryngology” to identify applications that may have
EP
information on velopharyngeal insufficiency contained in the material.
AC C
For the online portion, the web browser’s history, cookies, and preferences were cleared prior to performing each of the searches; location information and user preferences were also disabled to eliminate biased results based on previous searches. Non-text and non-English media were excluded. Pictures, captions, hyperlinks, and advertisements were deleted. Headings and sentence fragments were also removed. Inclusion criteria for mobile applications were those that had a text section on velopharyngeal insufficiency. NonEnglish applications were included.
ACCEPTED MANUSCRIPT
Readability analysis was performed using Readability Studio Professional Edition (Oleander Software Ltd; Vandalia, OH). Reading level was assessed using the following scales: Bormuth Grade Placement, Coleman-Liau, New Dale-Chall, Fry readability formula,
RI PT
Gunning-Fog, Harris-Jacobson, LIX, Raygor Readability Estimate, RIX, and SMOG (Table 1). Output from the program represents the reading grade level at which the text is written. For scales that provided a grade level range, the median of the range was used. If the upper
SC
limit of the readability scale was exceeded by the text, the value of the upper limit was used. Batch analyses were performed for the following website groups: All documents from all search terms
•
Documents by search term (“velopharyngeal insufficiency”, “VPI”, “velopharyngeal dysfunction”, and “VPD”)
•
M AN U
•
Websites by institution (information from academic hospitals, general online
TE D
resources, peer-reviewed journals, professional organizations) One-way ANOVA was used to establish statistical differences between groups, with
AC C
3. Results
EP
significance set a priori at P<0.05.
3.1 Website results
A total of 22 articles were identified – 10 articles each for “velopharyngeal
insufficiency” and “velopharyngeal dysfunction”, two results for “VPI”, and no results for “VPD.” Of these articles, there were 18 unique results. The mean reading level for each website across the 10 tests is summarized in Table 2. The mean reading grade level of all articles was 12.9±2.9. The lowest mean reading level was 8.23 for
ACCEPTED MANUSCRIPT
www.seattlechildrens.org. The highest mean reading level was 15.5 for www.ncbi.nlm.nih.gov - approximately the level of a college senior. Figure 1 shows the composite distribution of reading levels of all articles for each test.
RI PT
3.2 Mobile application results
No mobile applications were identified with the symptom-specific search terms. A total of 39 English mobile applications were identified with the search term
SC
“otolaryngology”. Over half of the applications (61.5%) were targeted towards ENT
providers, including mobile applications of conferences and surgical simulators. Only 5
M AN U
applications were targeted towards a patient audience. One of these five applications contained information on velopharyngeal insufficiency. This was from a mobile application of an otolaryngology clinic, with the page on velopharyngeal insufficiency accessible via an in-application search. The same information on VPI was also listed on their webpage, which
readability level of 10.68.
TE D
had not been identified in our Google search. The text in this mobile application page had a
3.3 Comparison by search terms
EP
The reading level of website articles by search term is shown in Figure 2 and Figure
AC C
3. The mean reading level by search term was 12.30±1.56 (velopharyngeal insufficiency), 11.66±1.70 (VPI), and 13.73±2.11 (velopharyngeal dysfunction). One-factor ANOVA showed a statistically significant difference between means (p<0.05). 3.4 Comparison by website institution The websites were categorized by the type of institution that published the patient education materials, as shown in Figure 4. The lowest mean reading level was a 12.81±1.66 from 10 articles published by academic hospitals. Two articles were published in peer-
ACCEPTED MANUSCRIPT
reviewed journal articles with a mean reading level of 15.35±2.79 for the highest mean. Additionally, there were 3 articles from professional organizations with a mean reading level of 12.95±1.63, and 3 articles from general online resources with a mean reading level
RI PT
of 14.42±2.26. One factor ANOVA showed statistically significant differences between each type of institution (p<0.01). 4. Discussion
SC
The internet provides patients access to a large volume of information. Websites such as dedicated patient education websites, blogs, and healthcare institutions are easily
M AN U
accessible. Approximately 60% of adults have used the internet to search for health-related information.8 The utility of these resources is dependent upon a patient’s ability to understand them. Previous studies in other medical and surgical fields have shown that the readability of most patient education materials is far above the average American reading
TE D
level.5-17 In this study, we analyze the readability of online and mobile application patient health information regarding velopharyngeal insufficiency. Velopharyngeal insufficiency is a common problem treated by otolaryngologists.
EP
There are a variety of congenital and acquired etiologies of varying severity with a range of
AC C
treatment options, which can be difficult for patients to understand. Despite guidelines by the AMA and NIH, the material analyzed in this study is written with language too complex for most readers. The simplest article of our analysis was written above an 8th grade reading level. A college education was necessary to understand over 60% of articles. These findings support previously published data that online patient education material is not appropriately readable by many patients and families.
ACCEPTED MANUSCRIPT
A complete lack of VPI-specific mobile applications highlights an area for improvement. Even within the otolaryngology-specific applications identified, only 12.8% of applications appeared to be designed for patient consumption, and only one of these
RI PT
contained information on velopharyngeal insufficiency. A mobile application may be an efficient medium through which interdisciplinary care information may be disseminated for these complex patients.
SC
There was significant variance between the readability of the 18 articles (p<0.01). This finding is unsurprising considering the diverse nature of websites analyzed in our
M AN U
search. The general public was the intended target audience for most search results, but we also encountered two articles from peer-reviewed journals. These journal articles were written at a mean reading level of 15.35±2.79. Since the target audience of peer-reviewed journal articles is healthcare professionals, it would be inappropriate to suggest this
TE D
material should be simplified to a 6th-8th grade reading level. However, patients could easily find these articles by searching the internet, as both articles were within the top 10 search results for “velopharyngeal dysfunction”. Their presence in our search above articles
EP
written for patients may reflect a flaw in the search engine’s algorithm. As technological
AC C
advances continue, we suggest that these algorithms be modified to provide results for their appropriate target audience. However, it was important that the results from peerreviewed journals be included in our study to accurately represent the breadth of information available to patients, and the higher literacy level needed to comprehend these resources. Moreover, even when excluding these sources intended for medical professionals, the mean reading level is still elevated at 12.75±2.17.
ACCEPTED MANUSCRIPT
Academic hospitals provided articles that had the lowest grade level readability score with a mean reading level of 12.18±1.66. Despite being above the literacy level of a high school graduate, academic hospitals are publishing material that is closer to the
RI PT
recommended reading level compared to general online resources, professional
organizations, and peer-reviewed journals. Comparing results by search term, we again see significant variation in the readability (p<0.05). Results for “VPI” had the lowest mean
SC
scores with a mean level of 11.66±1.70. Unfortunately, these results may not be very accessible to patients through a Google search. The use of acronyms yielded many
M AN U
extraneous results unrelated to medicine or otolaryngology. There were only two applicable links of the first 20 results with search term “VPI” and no applicable links for “VPD”.
Interestingly, a few notable websites focusing on the clinical care of patients with
TE D
VPI were not identified with our Google Search, including the American Cleft PalateCraniofacial Association (ACPA) and the Cleft Palate Foundation (CPF) [20]. While families will search the web independently, providers may direct patients towards these
EP
professional organizations for information. In analyzing ten randomly selected patient
AC C
information pages from the ACPA website, the mean readability was 10.73±2.87, ranging from 7.37 to 15.22. This sample was just significantly improved from our Google Search results (p=0.05), though this may be skewed by the low sample sizes. The ACPA provides detailed definitions and answers to frequently asked questions without excessive medical jargon, which other sources may aim to emulate. There are some limitations to this study. The program uses calculated readability as a proxy for patient comprehension of material. However, there are inherent flaws of
ACCEPTED MANUSCRIPT
readability calculators that make this substitution imperfect. Calculated grade level is based on word length and sentence length, without assessing the accuracy or clarity of text. Simplifying text to a lower readability score does not ensure increased comprehension [9].
RI PT
Additionally, scores may be skewed due to the length of words found in medical jargon that are often used in these articles. The CDC emphasizes the use of visuals to convey
information, such as images or pictographs [10]. However, these visual aids were not
SC
assessed in our study. Finally, many of the scales used to calculate readability have a
maximum score, but this upper limit was not consistent across scales. Fifteen of the 18
M AN U
distinct articles had readability levels exceeding the upper limit of at least one readability scale used in our analysis. Not every formula may be appropriate for our study, as some do not have ability to detect differences between texts written at a reading level above the 11th
5. Conclusion
TE D
grade.
Online and mobile application materials on velopharyngeal insufficiency that are
EP
accessed by patients and families are written at levels above the current recommendations
AC C
by the AMA and NIH. Despite a wide variety of search results and website types, all 18 articles identified in our study were written above the recommended AMA and NIH guidelines. Future website updates should be geared towards improving the readability of patient education materials as part of a systematic approach to increase patient comprehension and provide patient-centered care.
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
References [1] Health literacy: report of the Council on Scientific Affairs. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association. JAMA. 281 (1999) 552-557. [2] B.D. Weiss. Health literacy and patient safety: Help patients understand. http://www.med.fsu.edu/userFiles/file/ahec_health_clinicians_manual.pdf, (accessed November 3 2016). [3] T. Bodenheimer. Transforming Practice. New Engl J Med. 359 (2008) 2086-2089. [4] How to Write Easy-to-Read Health Materials. MedlinePlus. http://www.nlm.nih.gov/medlineplus/etr.html, (accessed November 3 2016). [5] J.A. Eloy, S. Li, K. Kasabwala, N. Agarwal, D.R. Hansberry, S. Baredes, M. Setzen. Readability assessment of patient education materials on major otolaryngology association websites. Otolaryngol Head Neck Surg. 147 (2012) 848-854. [6] P.F. Svider, N. Agarwal, O.J. Choudhry, A.F. Hajart, S. Baredes, J.K. Liu, J.A. Eloy. Readability assessment of online patient education materials from academic otolaryngology-head and neck surgery departments. Am J Otolaryngol. 34 (2013) 31-35. [7] D.R. Hansberry, N. Agarwal, R. Shah, P.J. Schmitt, S. Baredes, M. Setzen, P.W. Carmel, C.J. Prestigiacomo, J.K. Liu, J.A. Eloy. Analysis of the readability of patient education materials from surgical subspecialties. Laryngoscope. 124 (2014) 405-412. [8] H. Rayess, G.F. Zuliani, A. Gupta, P.F. Svider, A.J. Folbe, J.A. Eloy, M.A. Carron. Critical Analysis of the Quality, Readability, and Technical Aspects of Online Information Provided for Neck-Lifts. JAMA Facial Plast Surg. 19 (2017) 115-120. [9] K. Wong, J.R. Levi. Readability Trends of Online Information by the American Academy of Otolaryngology-Head and Neck Surgery Foundation. Otolaryngol Head Neck Surg. 156 (2017) 96-102. [10] K. Wong, J.R. Levi. Partial Tonsillectomy. Ann Otol Rhinol Laryngol. 126 (2017) 192198. [11] K. Wong, J.R. Levi. Readability of pediatric otolaryngology information by children's hospitals and academic institutions. Laryngoscope. 127 (2017) E138-e144. [12] V. Balakrishnan, Z. Chandy, A. Hseih, T.L. Bui, S.P. Verma. Readability and Understandability of Online Vocal Cord Paralysis Materials. Otolaryngol Head Neck Surg. 154 (2016) 460-464. [13] D.V. Cherla, S. Sanghvi, N. Agarwal, J.A. Eloy, W.T. Couldwell, J.K. Liu. Analysis of internet-based patient education materials related to pituitary tumors. Endocrine Pract. 20 (2014) 1044-1050. [14] M.W. Chu, J.A. Cook, S.S. Tholpady, C.E. Schmalbach, A. Momeni. Facial Plastic Surgery Patient Resources Exceed National Institute Recommendations. J Craniofac Surg. 28 (2017) 759-763. [15] G. Huang, C.H. Fang, N. Agarwal, N. Bhagat, J.A. Eloy, P.D. Langer. Assessment of online patient education materials from major ophthalmologic associations. JAMA Ophthalmol. 133 (2015) 449-454. [16] A.P. O'Connell Ferster, A. Hu. Evaluating the quality and readability of Internet information sources regarding the treatment of swallowing disorders. Ear Nose Throat J. 96 (2017) 128-138.
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
[17] L. Wozney, J. Chorney, A. Huguet, J.S. Song, E.F. Boss, P. Hong. Online Tonsillectomy Resources: Are Parents Getting Consistent and Readable Recommendations? Otolaryngol Head Neck Surg. 156 (2017) 844-852. [18] M. Bobian, A. Kandinov, N. El-Kashlan, P.F. Svider, A.J. Folbe, R. Mayerhoff, J.A. Eloy, S.N. Raza. Mobile applications and patient education: Are currently available GERD mobile apps sufficient? Laryngoscope. (2016). Epub. [19] Antonarakis GS, Kiliaridis S. Internet-derived information on cleft lip and palate for families with affected children. Cleft Palate Craniofac J. 46 (2009) 75-80. [20] Cleft & Craniofacial Educational Materials. ACPA. http://www.cleftline.org/who-weare/what-we-do/publications, (accessed September 14 2017).
ACCEPTED MANUSCRIPT
FIGURE LEGENDS: Preference for color photographs: online only Figure 1: Box and whisker plot for reading levels of search results of all articles. Dots
RI PT
represent outliers.
Velopharyngeal dysfunction. Figure 3: Mean reading levels by search term.
AC C
EP
TE D
M AN U
Figure 4: Mean reading levels by website institution.
SC
Figure 2: Reading levels by search term (A) Velopharyngeal insufficiency, (B) VPI, and (C)
ACCEPTED MANUSCRIPT
Table 1: List of readability formulas provided by Readability Studio Professional Edition Formula
L = total letters; W = total words; D = number of familiar Dale-Chall words; S = total sentences Coleman-Liau Index
ܮ ܦଷ 3.761864 + 1.053153 ൬ ൰ − 2.138595 ൬ ൰ ܹ ܹ ܹ + 0.152832 ൬ ൰ ܵ ܹ ଶ − 0.002077 ൬ ൰ ܵ 0.0588 × ܮ− 0.296 × ܵ − 15.8
RI PT
Readability Tool Bormuth Grade placement
L = average number of letters per 100 words; S = average number of sentences per 100 words New Dale-Chall
Gunning-Fog Index
SC
ܷܹ ܹ ൰ + 0.16 ൬ ൰ + 0.642 ܹ ܵ This raw core is rounded to one decimal place and converted to readability based on the Harris Jacobson Table ܹ ܹܮ + × 100 ܵ ܹ This value is converted to a grade level based on the LIX index table Select three 100-word passages from the beginning, middle, and end of a text. Calculate average number of sentences and average word length (with ≥6 letters) in each passage. Plot on Raygor estimate graph to determine reading level. ܮ ܵ This value is converted to a grade level based on the RIX Index table
TE D
W = total words; S = total sentences; CW = complex words Harris-Jacobson Wide Range Readability Formula
Select three 100-word passages from the beginning, middle, and end of a text. Calculate the average number of sentences and syllables in each passage. Plot on Fry Graph to determine reading level. ܹ ܹܥ 0.4 ൬ ൰ + 100 ൬ ൰൨ ܵ ܹ
M AN U
CW = complex words (not on Dale-Chall word list); W = total words; S = total sentences Fry readability formula
ܹܥ ܹ 15.79 ൬ ൰ + 0.0496 ൬ ൰ + 3.6365 ܹ ܵ
EP
UW = uncommon words; W = total words; S = total sentences Läsbarhetsindex (LIX)
AC C
W = total words; S = total sentences; LW = total number of long words (>6 letters) Raygor Readability Estimate
Rate Index (RIX)
LW = long words; S = total sentences Simple Measure of Gobbledygook (SMOG) W = number of words with >3 syllables; S = total number of sentences
24.5 ൬
30 1.043 × ඨܹ × ൬ ൰ + 3.129 ܵ
ACCEPTED MANUSCRIPT
Table 2: Mean reading level by article, listed in order of appearance on Google Search * indicates duplicate website identified through multiple search terms. Website
Institution
Title
Reading level
Velopharyngeal insufficiency
*http://www.seattlechildrens.or g/medicalconditions/chromosomalgenetic-conditions/vpi *http://emedicine.medscape.co m/article/873018-overview *https://en.wikipedia.org/wiki/ Velopharyngeal_insufficiency *http://www.cincinnatichildrens .org/health/v/velopharyngeal http://www.chsc.org/Main/Veip haryngeal-Insufficiency-VPI.aspx http://www.merckmanuals.com /professional/ear,-nose,-andthroat-disorders/oral-andpharyngealdisorders/velopharyngealinsufficiency http://www.childrenshospital.va nderbilt.org/services.php?mid=7 279 http://www.apraxiakids.org/library/velopharyngealinsufficiency-vpi-and-apraxia http://www.craniofacial.vcu.edu /conditions/vpi.html https://medicine.yale.edu/surge ry/plastics/care/cranio/velopha ryngeal.aspx https://www.cincinnatichildrens .org/service/s/speech/specialtyclinics/vpiclinic/?utm_source=VPI%2BClini c&utm_medium=shortcut&utm_c ampaign=vpi http://www.seattlechildrens.org /medical-conditions/eye-earnose-throat-conditions/vpd *http://www.cincinnatichildrens .org/health/v/velopharyngeal *http://www.seattlechildrens.or g/medicalconditions/chromosomalgenetic-conditions/vpi http://emedicine.medscape.com /article/994975-overview *http://emedicine.medscape.co m/article/873018-overview *https://en.wikipedia.org/wiki/ Velopharyngeal_insufficiency
Academic hospital
Velopharyngeal Dysfunction
10.67
General online resource General online resource Academic hospital Professional organization Professional organization
Velopharyngeal Insufficiency Velopharyngeal Insufficiency Velopharyngeal Dysfunction Velopharyngeal Insufficiency Velopharyngeal Insufficiency
TE D
M AN U
SC
RI PT
Search term
AC C
EP
VPI
Velopharyngeal dysfunction
Academic hospital
14.8
13.69 9.84
11.45 15.47
12.65
Academic hospital Academic hospital
What is Velopharyngeal Insufficiency Velopharyngeal Insufficiency (VPI) and apraxia Velopharyngeal Insufficiency – VPI Velopharyngeal Insufficiency (VPI)
Academic hospital
About the VPI clinic
15.09
Academic hospital
Velopharyngeal Dysfunction
8.23
Academic hospital Academic hospital
Velopharyngeal Dysfunction Velopharyngeal Dysfunction
9.84
General online resource General online resource General online resource
Velopharyngeal Dysfunction Velopharyngeal Insufficiency Velopharyngeal Insufficiency
14.78
Professional organization
11.94
10.89 11.6
10.67
14.8 13.69
ACCEPTED MANUSCRIPT
Peer-reviewed journal Academic hospital
Velopharyngeal Dysfunction Velopharyngeal Dysfunction
15.22
Academic hospital
Velopharyngeal Dysfunction Program Velopharyngeal Dysfunction
14.83
Velopharyngeal nomenclature: incompetence, inadequacy, insufficiency, and dysfunction
15.47
Peer-reviewed journal
M AN U TE D EP AC C
14.49
RI PT
Academic hospital
SC
http://www.ncbi.nlm.nih.gov/p mc/articles/PMC3706038 http://www.chop.edu/centersprograms/velopharyngealdysfunction-program http://www.nationwidechildren s.org/velopharyngealdysfunction-program https://www.rileychildrens.org/ health-info/velopharyngealdysfunction http://digital.library.pitt.edu/c/c leftpalate/pdf/e20986v25n4.13. pdf
13.52
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT