- Email: [email protected]
An audit of voice recognition errors in neuroradiology reports
Abstract / Clinical Radiology 69 (2014) S11eS22
Methodology: A prospective audit was undertaken over a 6-week period across the department. Operators were unaware and were assumed to be cognisant of the 3 patient point ID check procedure from local training. Results of 1st audit round: A total of 154 procedures were audited from various imaging modalities and procedures were performed by different operators. Only 47/154 procedures had a full 3-point identification performed prior to commencement of examination. 1st action plan: Regular mandatory training of all staff and a mandatory 10 second time out pre procedure led by the operator was proposed. Results of 2nd round: A total of 178 procedures from various modalities and different level of operators were audited. 158/178 procedures had a full 3-point identification performed prior to commencement of examination. 2nd action plan: Although a positive improvement has been identified, further re-iteration is needed of the importance of the check prior to commencement of any radiological procedure. Audit of the proportion of CTs informally double reported at West Suffolk Hospital
S19
Grade given Correct Descriptor used DEXA recommendation Methodology: Reporting of vertebral fractures in our institution was evaluated for the following examinations performed on females over the age of 45 years, against the set standard. 406 thoracic and lumbar plain films 163 thorax/abdomen/pelvis CTs Results of 1st audit round: The vertebral fracture identification rate was 94.2% on plain film but only 13.5% on CT. However when fractures were identified, there was generally consistent use of the word ’fracture’ rather than ’wedging’ or ’compression’ in both modalities. 1st action plan: The enormous discrepancy in CT implies that radiologists are not routinely using sagittal reconstructions to evaluate the spine. The findings were presented in a local audit meeting to encourage the use of 3D reconstruction capability of CT to actively look for spinal fractures even on general CT scans. An audit of voice recognition errors in neuroradiology reports
Authors: Zosia Rodak*, E.R. Darrah Background: There is increasing recognition that double reporting, particularly of cross-sectional imaging is important for governance and the RCR has recognised its value for revalidation evidence. The literature and current guidance relates to “formal, blinded double reporting”, but a significant amount of more informal double reporting occurs in all radiology departments, as second opinions, MDT review etc. This audit is to quantify this and assess its potential value. Standard, indicator and target: Independent providers of radiology services double report 5-10% of scans as part of standard governance and the RCR includes double reporting in its toolkit for revalidation. Methodology: From 19/03/12 - 25/03/12 and 22/04/13 - 28/04/13, all radiologists at West Suffolk Hospital recorded when they double reported a CT, the reason for doing so, and if any change was noted. This was compared to the total number of CTs. Results of 1st audit round: 82 of 247 of the scans were double reported. 86% were reviewed in MDTs. 3.6% of scans were reported differently. 1st action plan: As a large proportion of scans are already being doublereported informally day-to-day, this should continue. The plan is to formalise this into a report for our governance meeting and to make it an annual process that can feed into ISAS outcomes. Results of 2nd round: 74 of 287 (25.7%) of the scans were double reported. 85% were reviewed in MDTs. 2.7% of scans were reported differently. 2nd action plan: To continue with this as an annual process to contribute to ISAS outcomes.
Authors: Mark Igra*, Stuart Currie, Jeremy Macmullen-Price, Daniel J. Warren, Ian Craven Background: Voice recognition (VR) use has increased since the 1980s. This increase has lead to more errors in reports.1,2 The Royal College of Radiologists states that wording of reports should be clear.3 Errors can have significant clinical consequences if not corrected promptly.4 Errors are more likely in noisy departments, those with high workloads or if English is not the reporter’s first language. Standard, indicator and target: Our aims were to determine the frequency and type of errors and whether errors affected report quality. 100% of reports should be free of errors affecting report interpretation. Image interpretation was not assessed. Methodology: 300 reports from 6 consultants (50 consecutive each) were analysed. Errors were categorized as wrong word substitutions, nonsense phrases, deletions, insertions or punctuation errors. Whether the error affected report interpretation was determined. Results of 1st audit round: Average words per report varied from 62 to 190, and average words per error was 929. Total number of individual errors per consultant varied from 2 to 37. 88% of reports were completely error free. 99.4% of reports were free of significant errors. 1st action plan: Data was presented to the department with advice on reducing grammatical errors. Re-audit will take place annually to ensure we are maintaining or improving standards. VR is an effective reporting tool but checking reports prior to verifying remains essential. Reminders on reporting software to check may help further reduce errors.
Standards of spinal fractures reports in plain radiograph and CT Audit of voice recognition system report accuracy Authors: Ayano Tachibana*, H’ssein Al-Chalabi, Clare Groves Authors: Ai-Lee Chang*, Dorothy Keane, Lance Cope Background: Osteoporotic vertebral fractures are associated with significant risk of further spinal and hip fractures, with subsequent increase in morbidity and mortality. Secondary prevention strategies are dependent on accurate reporting of spinal fractures. The International Osteoporosis Foundation recommends the use of clear, unambiguous terminology using the word ‘fracture’, and grading the fracture, based on the semi-quantitative system outlined by Genant and Wu (1993). Standard, indicator and target: Standard of 100% was set for criteria including:Fractures identified Term “Fracture” used if identified
* * * *
Guarantor Guarantor Guarantor Guarantor
and and and and
correspondent: correspondent: correspondent: correspondent:
Zosia Rodak Ayano Tachibana Mark Igra Ai-Lee Chang
Background: Audit to assess errors in voice recognition system (VRS) generated reports due to failure of software to correctly identify words and failure of user to spot error in text. Standard, indicator and target: VRS generated reports should be accurate and clear. Overall error rate <5%VRS reports. There should be no major errors where the report makes no sense, target 0%. Indicator:% of reports containing an error. Methodology: 50CR and 20CT consecutive reports for each VRS user Error classified as minor, moderate, major Clinical details and body of report reviewed VRS users: CT6 CR8.
Methodology: A prospective audit was undertaken over a 6-week period across the department. Operators were unaware and were assumed to be cognisant of the 3 patient point ID check procedure from local training. Results of 1st audit round: A total of 154 procedures were audited from various imaging modalities and procedures were performed by different operators. Only 47/154 procedures had a full 3-point identification performed prior to commencement of examination. 1st action plan: Regular mandatory training of all staff and a mandatory 10 second time out pre procedure led by the operator was proposed. Results of 2nd round: A total of 178 procedures from various modalities and different level of operators were audited. 158/178 procedures had a full 3-point identification performed prior to commencement of examination. 2nd action plan: Although a positive improvement has been identified, further re-iteration is needed of the importance of the check prior to commencement of any radiological procedure. Audit of the proportion of CTs informally double reported at West Suffolk Hospital
S19
Grade given Correct Descriptor used DEXA recommendation Methodology: Reporting of vertebral fractures in our institution was evaluated for the following examinations performed on females over the age of 45 years, against the set standard. 406 thoracic and lumbar plain films 163 thorax/abdomen/pelvis CTs Results of 1st audit round: The vertebral fracture identification rate was 94.2% on plain film but only 13.5% on CT. However when fractures were identified, there was generally consistent use of the word ’fracture’ rather than ’wedging’ or ’compression’ in both modalities. 1st action plan: The enormous discrepancy in CT implies that radiologists are not routinely using sagittal reconstructions to evaluate the spine. The findings were presented in a local audit meeting to encourage the use of 3D reconstruction capability of CT to actively look for spinal fractures even on general CT scans. An audit of voice recognition errors in neuroradiology reports
Authors: Zosia Rodak*, E.R. Darrah Background: There is increasing recognition that double reporting, particularly of cross-sectional imaging is important for governance and the RCR has recognised its value for revalidation evidence. The literature and current guidance relates to “formal, blinded double reporting”, but a significant amount of more informal double reporting occurs in all radiology departments, as second opinions, MDT review etc. This audit is to quantify this and assess its potential value. Standard, indicator and target: Independent providers of radiology services double report 5-10% of scans as part of standard governance and the RCR includes double reporting in its toolkit for revalidation. Methodology: From 19/03/12 - 25/03/12 and 22/04/13 - 28/04/13, all radiologists at West Suffolk Hospital recorded when they double reported a CT, the reason for doing so, and if any change was noted. This was compared to the total number of CTs. Results of 1st audit round: 82 of 247 of the scans were double reported. 86% were reviewed in MDTs. 3.6% of scans were reported differently. 1st action plan: As a large proportion of scans are already being doublereported informally day-to-day, this should continue. The plan is to formalise this into a report for our governance meeting and to make it an annual process that can feed into ISAS outcomes. Results of 2nd round: 74 of 287 (25.7%) of the scans were double reported. 85% were reviewed in MDTs. 2.7% of scans were reported differently. 2nd action plan: To continue with this as an annual process to contribute to ISAS outcomes.
Authors: Mark Igra*, Stuart Currie, Jeremy Macmullen-Price, Daniel J. Warren, Ian Craven Background: Voice recognition (VR) use has increased since the 1980s. This increase has lead to more errors in reports.1,2 The Royal College of Radiologists states that wording of reports should be clear.3 Errors can have significant clinical consequences if not corrected promptly.4 Errors are more likely in noisy departments, those with high workloads or if English is not the reporter’s first language. Standard, indicator and target: Our aims were to determine the frequency and type of errors and whether errors affected report quality. 100% of reports should be free of errors affecting report interpretation. Image interpretation was not assessed. Methodology: 300 reports from 6 consultants (50 consecutive each) were analysed. Errors were categorized as wrong word substitutions, nonsense phrases, deletions, insertions or punctuation errors. Whether the error affected report interpretation was determined. Results of 1st audit round: Average words per report varied from 62 to 190, and average words per error was 929. Total number of individual errors per consultant varied from 2 to 37. 88% of reports were completely error free. 99.4% of reports were free of significant errors. 1st action plan: Data was presented to the department with advice on reducing grammatical errors. Re-audit will take place annually to ensure we are maintaining or improving standards. VR is an effective reporting tool but checking reports prior to verifying remains essential. Reminders on reporting software to check may help further reduce errors.
Standards of spinal fractures reports in plain radiograph and CT Audit of voice recognition system report accuracy Authors: Ayano Tachibana*, H’ssein Al-Chalabi, Clare Groves Authors: Ai-Lee Chang*, Dorothy Keane, Lance Cope Background: Osteoporotic vertebral fractures are associated with significant risk of further spinal and hip fractures, with subsequent increase in morbidity and mortality. Secondary prevention strategies are dependent on accurate reporting of spinal fractures. The International Osteoporosis Foundation recommends the use of clear, unambiguous terminology using the word ‘fracture’, and grading the fracture, based on the semi-quantitative system outlined by Genant and Wu (1993). Standard, indicator and target: Standard of 100% was set for criteria including:Fractures identified Term “Fracture” used if identified
* * * *
Guarantor Guarantor Guarantor Guarantor
and and and and
correspondent: correspondent: correspondent: correspondent:
Zosia Rodak Ayano Tachibana Mark Igra Ai-Lee Chang
Background: Audit to assess errors in voice recognition system (VRS) generated reports due to failure of software to correctly identify words and failure of user to spot error in text. Standard, indicator and target: VRS generated reports should be accurate and clear. Overall error rate <5%VRS reports. There should be no major errors where the report makes no sense, target 0%. Indicator:% of reports containing an error. Methodology: 50CR and 20CT consecutive reports for each VRS user Error classified as minor, moderate, major Clinical details and body of report reviewed VRS users: CT6 CR8.