Chest Roentgenograms in Diagnosis of Traumatic Rupture of the Aorta

Chest Roentgenograms in Diagnosis of Traumatic Rupture of the Aorta* Observer Richard

Variation

E. Burney, R.

and

Shu-C

hen

A significant

Ph.D.

p value

is not

value

server

agreement

with we

genographic

signs

rupture

of the

clinical

practice.

always

respect

rule

to

observers,

as well

he

out

plays

an

rupture

Agreement

of chest chest

important

role

in the

aorta

(TRA),

diagnosis

of pa-

that accounts for up to 15 percent of deaths vehicle accidents.’ Although in retrospective many

abnormalities For

on chest

editorial

film

comment

have see

been

with TB.A, association

trauma patients mediastinum,

is examined; obscuration

cation

aortopulmonary

window,

of the

tube,

displacement paraspinus bronchus.2 In

stripe,

injury motor reviews

thought

page

be associated cant statistical

of the

from

to

only seven have any signifiwhen a large population of

nasogastric and

depression

previous

studies,

of the opacifi-

of the

left

two

terpretation of the chest servers has been observed.

variation

or

chest and

mainstem the

in-

film between groups of obIn one, clinicians caring for

films for signs of trauma, obscuration of the aortic

to be significantly

all

observers,

associated

while

with

associations

university

the

Departments

of Surgery,

Radiology,

with

between

and

comparisons

agreement.

Of

TRA,

the

seven

identification

of the aortic knob agreement and are

in clinical

findings

practice.

but

TRA

and

not

were

for others.

of observers

about

agreement

interpretation

undertook agreement

of signs

the with

of trauma that this

which

present respect

significantly information

of these

significant

Unfortunately,

information

At the

do

not allow or provide

with

of trauma.

We

respect therefore

study to explore to the roentgenographic associated will help

signs

University

ordered include

may

with answer

be

most

observer signs

TRA, believing the question of useful in clinical

trauma

a rapid

or signs

of Michigan,

randomly

sorted,

numbers.

These

patients

who

films

identifying

were

age,

the

Of the

have

presence been

mediastinum, apical

fractures,

depression

history,

clinical the

contusion,

projection,

during

two radiologists,

which

of the

had

cap,

tube,

opacification obliteration

period

or absence

code arch

to 1980

studied,

for whom

16 had

in conjunction of the

with

first

aortopulmonary aortopulmonary

CHEST

TRA. as to were

TRA:

pneumothorax,

of the aortic

of the

assigned

of 15 roentgenographic

of the left mainstem of the

were study,

undergone

and 1970

149 cases

chest

for the

had

of of

admission aortography

covered, who

a history

regardless

Department

hemothorax,

obscuration

The

been

Additionally,

with

and four surgeons, “blinded” and results of aortography,

reported

fracture

TRA.

undergone

Radiology

has

criteria which evidence of

trauma

findings.

all patients

available.

name,

truncal

marks

represented

to identify

num,

of the

for trauma

a nasogastnc of

clinical

by a member

with

all patients

or severe

or other

of 149 trauma

selected

asked

at the 31st

aortography

associated on almost

injury

roentgenographic

ing

METHODS Hospitals,

previously obtained

deceleration

patient

Biostatistics,

of Michigan

has been

aortography

films

AND

as a matter of policy on the basis of very broad the presence of clinical or roentgenographic

Six physicians,

other

Ann Arbor. Annual Meeting, Association of University Radiologists, Mobile, AL, March 24, 1983. Manuscript received August 15; revision accepted January 9. Reprint requests: DE Burney, C4020 Outpatient, University Michigan Hospital, Ann Arbor 48109

Presented

other

association of association

chest

TRA for most

and

but

and obscuration interobserver

to be useful

found

widening

most measures of statistical meaningful ranking of level

findings *From

random associated

observers

aortography

widened knob were

between

than

MATERIAL

the patient differed from radiologists in their interpretations of important roentgenographic findings.3 In a subsequent investigation,4 in which observers blindly read mediastinum

was

mediastinal

signs

kappa

Statis-

practice.

of the

in

knob,

of Cohen’s

of agreement.

.0000)

=

aortic

widening consistent

likely

(p to

the

better

extent

regard

roentgenographic

hemothorax, widening

most

chest

our

found

the

to the

585

these are widening of the aortic knob,

no

for some

trauma

lethal

showed

by calculation

the

cx-

was

of traumatic

a potentially

of

of mediastinal show the most

all

between

abdominal

and

with

roentgenographic

the initial undergone

roentgenograms

blunt

in were

and

estimates agreement

all observers obscuration

useful

groups,

by chi-square

statistic, which tically significant

traumatic

to be

M.D.;

amined

roent-

radiologists

specialty

suffering of the

seven with

two

the

of

interob-

in blinded fashion, patients who had THA.

Whitehouse,

roentgenogram of

likely

and

as between

interpretation tients

chest

are most

surgeons

to interpret individually, films of 149 trauma

aortography

measure

associated

(TRA)

M.

examining

which

statistically Four

good

By to

M.D.;

Walter

a

test.

determined

aorta

R. Gundry,

FC.C.P.;

of a diagnostic

interpretation,

Interpretation

Steven

M.D.,

Wu,

clinical

asked chest

M.D.;

Mackenzie,

James

in

or second

ribs,

bronchus,

rib

other

displacement

knob, clear

widen-

pulmonary of

pneumomediastispace window

I 85 I 5 I

MAY,

on

anterior on

1984

lateral

605

projection,

widening

clavicle.

Specific

were

not

were

strength

accumulated

of MIDAS

through

the

PHI

and

rates

the

is designed

group

of the

these

findings

if the t v sets Kappa

agreements

and

radiologists.

the

Cohen’s

chance

and

than

two variables

between

using

either

Kappa

that

compared the

were

taken

significant

by

a more

ment

both

of

RESULTS

Seven statistical

chest film association

num, obscuration placement of the aortopulmonary

window,

and

sensitivity

of these from a high

tions)

left

0.272

of the

bronchus

findings

in

of 0.89

depression.

the

(83 of 93 possible

widening

to 0.23

which

to 0.162,

(The

is only

maximum

fair to moderate

1-Interpretation

of Seven

Findings

knob

for these

of association

statistic is 1.00.) There was no difference radiologists as a group and surgeons

Table

geons widened

tube from by noted as a

Sign ficantly Reading

No. Widened

with Films

No.(+)Findings/

Films

Read

No.

TRA

as an

group

found

other

did

but

association

radiologists

while

surgeons

found

a signifi-

of the clear artery on an-

did

level

of

findings

the

a

not.

of displace-

1’RA,

surgeons

as a group,

moderate,

and

not

(Table

1).

of agreement sought as well

0.4-0.59;

none,

for as for

fair,

0.3-

0-0.14.

There

was

radiologists

and

sur-

regard to only and obscuration

2). This calculation

was

two findings, of the aortic

done

by taking

only

there was complete agreement Fair agreement was found for

cases for which each group. and

by

in the

the

between

groups with mediastinum

hemothorax

Associated 149 Chest

No.(+)Findings/

(Table

those within

excep-

often

Kappa varies between 0 (no and 1.00 (complete agreefive levels of Kappa agreement:

agreement as

the

more

estimating each

whatsoever)

moderate

(9 of 39 possible

strength

for

ment). We defined good, Kappa = 0.6-0.8; 0.39; pooor, 0.15-0.29;

The

found

with

and surgeons observers.

agreement

of TB.A observa-

detection

while

calculated

individual

TRA with

was

hand,

statistic

radiologists

paraspinous

for displacement of the nasogastric strength of association, PHI, ranged

parameters. the PHI between

widening

with

in

param-

one

tube

other

projection,

was

disof the

or any other

with TRA of opacification the aorta and pulmonary

Kappa

finding

TRA

a significant

the

(ie, the

the

for which

found On

The

found to have strong TRA: widened mediasti-

mainstem

for mediastinal

observations) (Table 1). The

terior were

of the aortic knob, hemothorax, nasogastric tube, opacification

stripe, varied

findings with

not.

have

was no difference, however, of TRA using hemothorax

nasogastric

cant association space between

alone.

which

with

of the

did

complete

associated

association

Radiologists

rould

indepen-

distribution

is therefore

one

is obtained

to

(Table 1). were two findings

There

statistic,

of interpretation

mediastinum

detection

indicator

rate

thought

There

resulting

relative

Kappa

agreements

account

disagreements,

of agreement

available

being

into

of hemothorax,

Cramer’s

for this purpose.5

takes

significantly

Center.

variables.

with

eter

packages

for comparing

of observations

thus

with

to the

films

of widened

statistical

measured

agreements

PHI

regard of

question)

of significance

and Cramer’s

of association

of many was

for level

with

proportion

Computing

is convenient

specifically

actual

measure

fracture

tion

special

strength

agreement

comparing

analyzed

of Michigan

of association

Observer

dent.

and

to interpret

chi-square

BMDP,

of 0 to 1 and

strengths

occur

and using

University

statistic a scale

which

stripe,

as to how

of associations

the help

on

paraspinous

given.

All data and

of the instructions

opacification

by Two Radiologists

of the and

aortopulmonary Four

Surgeons

“P”

TIIA

Valuet

Sensitivity

PHI

mediastinum

Surgeons

280/594

(47%)

0.89

.284

160/296

(54%)

55/62 28/31

0.0000

Radiologists

0.0000

0.90

.248

Surgeons

283/594

(48%)

52162

0.0000

0.84

.247

Radiologists Clear space

1421296

(48%)

24/31

0.0005

0.77

.201

192/410

(47%)

35/41*

0.0000

0.85

.257

57/131

(44%)

7/14

0.60

0.50

.045

2021593

(34%)

39/63

0.0000

0.63

.207

88/296

(30%)

19/31

0.0000

0.61

.236

Surgeons

87/593

(15%)*

21/62

0.0000

0.34

.185

Radiologists

71/296

(24%)

12119

0.04

0.39

.117

Surgeons

75/593

(13%)

15/60

0.004

0.24

.118

Radiologists

42/296

(14%)

8/31

0.05

0.26

.113

5/26

0.17

0.19

4/13

0.01

0.30

.106 .277

Obscuration

of knob

opacification

Surgeons Radiologists Paraspinous

stripe

widening

Surgeons Radiologists

Hemothorax

Left bronchus

Displacement

depression

NC

tube

Surgeons

19/167(11%)

9/84

Radiologists *p less

than

tSignificance 606

0.01

between

of association

surgeons

(11%)

and radiologists,

of roentgenographic

other finding

comparisons

with TRA

in these

columns

not significant.

by chi-square.

Diagnosis at Traumatic Ruptu,e at the Aorta (BumW

et sQ

Table 2-Agreement Between Radiologists and Surgeons as Groups with Regard to Chest Film Findings Associated with TRA in Trauma Patients5 of

Value Finding Widened

mediastinum

Obscuration

of knob

Hemothorax Opacification

of AP window

Widening

of paraspinous

Left

bronchus

main

Displacement

of NC

tube

were

compared

within

each

group.

window,

and

spinous

stripe,

which

there

six

observers,

can be made. Kappa value

because

meaningful

these

on

.33

Fair

Hemothorax

.06

None

Displced

.01

None

Opacification

The

TRA are given best agreement

tification

of fracture

complete

of the bronchus,

were

are

findings

and

obscuration

Kappa

values

of the

values tube

are seen and

not

in-

clavicle

not associ-

and regard

and

for

are listed

are

but

rib

multiple

with TRA. mediasti-

the

Table

Table Kappa

delineation 4, which

of the range

range

gives

the

of Kappa

number

of

3-Range of Agreement Level Statistic for 15 Interobserver Six Observers Interpreting

and Average Comparisons Chest Films Range

Finding Findings

having

significant

association

mediastinum

Obscuration

of knob

Displacement

of NC

tube

Hemothorax Opacification

of AP window

Obscuration

paraspinous

Left main

bronchus

with

associated Fractured

Multiple Fracture

of

Kappa

Widened

depression

rib fractures 1st, 2nd Rib contusion

tube

Level*

Value of Among

Average Kappa

Value

11

1

7

6

7

1

1

4

6

4

3

8

4

.6-8,

kappa=

aortic

fair, poor,

the

number

.25 .20

.01-39

.16

.68-1.00

.36-81

.82 .68

.49-71

.58

.37-57

.50

.23-49

.35

for

TRA in 15

frequency of good or to obscuration of the than

greater

of times

other

than

than

that

parame-

other

specialty between

highest

with

and

respect

was compared

pairing

param-

0.05).

the

occurred

lowto the

(Table

of highest and differ significantly

5). The

lowest from

level of the pre-

given the number of potential pairno significant variability occurred

frequency and thus, of speciality.

In his sophisticated

(ROC)

examination

of decision-making

management, Lusted6 has called attention calls the Receiver Operating Characteristic of a test,

characteristic

which

of most

positive

number

.05-35

was seen with

whether agreement within to be better than agreement

frequency did not

in patient what he

This

.08-41

6 .3-39;

DIscussIoN

.48 .40

The respect

(p less

of agreement

because

.50

.20-.62

of agreement associated

greater

hemothorax

of specialty

.39-64

.46

lack

is significantly

except

observed agreements

6 fair,

widened mediastinum (p less than 0.05); of this level of agreement for widened

mediastinum

dicted ings,

and

is significantly

levels

3 .4-59;

0-14)

comparisons. agreement with

knob

groups,

agreement,

significantly

ters except the frequency eters

moderate

no agreement,

parameter

.36-67 0-.81

5

bronchus

interobserver moderate

est

2

of

.15-29;

each

None

2

depression

poor,

Poor

paraspinous

*(Good;

true

with TRA

Fair

14 2

good,

TRA

clavicle

Pulmonary Apical cap

stripe

is shown

of times

NC

main

kind

is wide.

A better

2

To determine groups tended

comparto iden-

aortic knob. Comparable for displacement of the

hemothorax,

Moderate

of knob

moderate,

for which

TRA

with which

for completeness was seen with

of the

ent

stripe

paraor

too infrequent

seven

Signs

TRA

APwindow

comparisons

view

mediastinum

Widened

agreement

neither of which was associated agreement was seen for widened

nasogastric

Not

None

.

lateral

findings

ated with ison. The

mean

Obscuration

was

are significant associations and the remaining findings

num

Widened

Fair

15 interobserver

evaluation.

fractures, Moderate

Moderate

.34

tube.

window

with

Good

The range of the Kappa values and mean for these comparisons is given in Table 3. pneumomediastinum, and loss of the

Pneumothorax, aortopulmonary

of Agreement

Roentgenographic

Agreem

.53

for widening of the left

depression

for

of

Left

of a nasogastric

With

cluded

for

of Levels

Observers

Agreement Moderate

.

4-Frequency Associated

.48

.

no agreement

displacement

there first,

stripe

depression

5Cases

in

Level

Kappa

Table Between

tests

is a measure tests

Table

problem

of utility.

as the

(ie, sensitivity)

of false positives same

that

increases, described

Highest

of

so will

the

relationship. in more

of Highest and Lowest by Specialty Pairing

5-Frequency Agreement

It is

proportion

in a reciprocal

has been

to

Level

Lowest

Level

philoof

Level

Radiologist-radiologist

2 (1)*

1 (1)

Radiologist-surgeon

7 (6)

5 (6)

Surgeon-surgeon

3 (5)

6 (5)

*Contains Differences

expected between

frequency expected

given

and

number observed

of potential

are not

pairings. statistically

significant. CHESTI85I5IMAY,1984

607

sophical acceptance

terms as the increase of a false hypothesis)

to minimize esis).7 In terms

type

1 error

of the

attempts

have

genographic percent,

the

We

have

roentgenographic and with

the

incidence

signs

might

associated

to the

The

result

significant interpretation

of observer

associated with of unnecessary

from

agreement measurements rate of positive findings

(ie, positive

ments/total

observations)

may

are large agree on,

exist

to the

with

regard

Yerushalmy agreements

concludes that and agreements

findings

mends

are

calculation

best method perception, limitation account Table

with

Widened Opacification

of knob

He specificity

because decision

recomas the

of observer same kind of

they do not take into analysis at one time. Rate, and Kappa Value Findings interpreted by

and Surgeons

Depression

our

To illustrate

values

Displacement tube

of sensitivity

agreelarge great

by

findings

when

the

the

is to find

most

those

tests

agreement.

Table

the sensi-

6 compares

rate, and associated

Kappa values for with TRA. While and is the

differences

the other failure of in observer

by Kappa. everyone is equally film interpretation.

To

qualified reduce

or the

inherent error in identification and interpretation roentgenographic findings, one text recommends practice of joint viewing of roentgenograms by

of the both

radiologist

is an

practice, of clinical

the patient Nevertheless, to have to read

been detected, all the films. has

been

history

when

and

that

one clinical

this

considers contact

the with

provided,

shown

to be increased

although

of the

In this

an aortogram

film.4” of TEA

three observers would have had The rate of detection of positive

suggestive

provided.2’1

cases,

We think

especially history

on the interpretation of the chest in the present study, for all cases

findings been

clinician.’0

and

excellent influence

it

disease

study,

no clinical

was known

had

when

been

a clin-

in question that

done

has

history in each

was of the

TEA.

to detect

Analogous to the question of the effect of multiple observers is the question of the effect of the presence of multiple positive findings, all associated with TEA, each

of which may be detected with more or less Even if the findings associated with TEA independent, and that seems an unlikely pos-

variability. were

sibility, this study suggests that variability terpretation may well negate any incremental an additional finding. This is consistent with observations, in which we mental addition of positive in detection

in the would

Agreement

.90/89

0.766

0.48

were

known

for the

.77/84

0.807

0.53

seek.

Since,

however,

.50/85

0.806

0.33

respect terpret,

.61/63 .39/34

0.875 0.889

0.06 0.34

.26/24

0.958

0.01

.30/19

0.935

ing

have found findings

of TEA,

but

in invalue of previous

that the increhas not led to

rather

to increases

number of negative aortograms ordered.3’ be desirable if, as Lusted6 suggests, the

(Rad/Surg)

identification the

to the various and determining task,

findings

it

would

about which

of the

ROC

It ROC

diseases

is not

we

known

with

findings most clinicians this would be an overwhelm-

appear most

useful persons

to

identify

can

agree.

in-

those

CONCLuSIONS

We have .

allows a

it

is therefore

to illustrate

detected

increases

of

is that

disparity between agreement is evident, also evident

of left

bronchus

best

agreement findings

Positive Kappa

test

which

is the

this point,

tivity, positive roentgenographic marked parameters

of positive This

purpose, there

Sensitivity

of paraspinous

Hemothorax

608

measures. and

which

ical

of clear

stripe

NC

agree-

misleadingly

overall to posi-

Agreement

space Widening

+ negative

appear

calculation of both with respect only

to Roentgenographic

mediastinum

Obscuration

in which the rate of positive

of assessing the variability but these suffer from the

Radiologists

“a

films.

of sensitivity

6-Sensitivity,

Respect

that

of positive

inadequate

as agreements all parts of the

is a com-

attention to the into observer

number

for

about

Obviously not capable of chest

numbers of negative films while major disagreements

small

useful

statistic

with regard to observer the biases introduced findings or apparently

is low.

criteria. With utility of these es-

Kappa

detection

agreement

in chest roentgen with currently avail-

agreements

in the

of positives

indiscriminate

al8 state

by situations will be low. The

high when there which observers

tive

et

Yerushalmy9 has called that can be introduced

agreement

with

of various

agreement

Herman

number of disagreements cannot be resolved

able criteria.” kinds of bias

to 100 based

of Cohen’s

of ranking tests while avoiding of negative

variation rate

clinician.

Indeed,

matter.

as

TEA is of importance,

with

practicing

measurement

plex

of roent-

value

application of existing roentgenographic this in mind, examination of the clinical pecially

that

yield

reported

and

signs thought to be enormous number

which

various

hypoth-

TRA ordered

value

means ment, numbers

suspect

by variations

relative

TEA

aortograms

the

in detecting aortograms

concerned

to the

we

to push

true

The

(ie,

in attempts

relevant roentgenographic signs has with relatively little marginal yield.

been

respect

2 error

of the

study,

made

interpretation number of

upon marginally increased greatly

type

(rejection

present

been

in

inherent

to the

examined

interpretation

observer of 15 chest

Diagnosis

of Traumatic

agreement film

findings

with

respect

associated

Rupture of the Aorta (Barney

et a!)

with

traumatic

four

surgeons

rupture of the aorta, two radiologists.

using

and

a panel

The

Williams S, Burney RE, Mackenzie JR, Cho KJ. after blunt chest trauma: a reassessment of the radiographic findings associated with traumatic rupture of the aorta. Invest Rad 1983; 18:230-37 4 Gundry SR. Burney RE, Mackenzie JR. Wilton GP, Whitehouse WM, Wu S. Assessment of mediastinal widening and other radiographic abnormalities associated with traumatic rupture of the aorta. J Trauma 1983; 23:293-99

of

findings

3 Gundry

widened mediastinum and of obscuration of the aortic knob ranked the highest in terms of observer agreement

both

while

between

also

groups

showing

detection

of TEA.

pretation

of hemothorax

low.

random

Only

respect

between

highest

Agreement

regard

was fair, while was

findings;

their

individuals,

sensitivity

with

agreement

to other

for individuals among the

and

the

seen

for to the

with

and

obscuration

to be useful

in

day-to-day

chest

MM, trauma:

Brown

2 Gundry Indications ment.

Sloan

H.

general

& Co.

TEA, aortic

chest

principles

of management.

practice.

9 Yerushalmy

of the

aorta.

10

In: Blunt

Boston:

multivariate

in

of error,

patient

management.

and their

consequences

1963; 8:97-107

Sci

and description

of roentgenographic

RG,

Pare

of diseases

of the

chest.

Philadelphia:

WB

in in

pulmo-

Bad Clin N Am 1969; 7:381-92 JAB Perception in chest roentgenology.

shadows.

Fraser

Discrete

1975:395-97

Gerson DE, Hessel SJ, et a!. Disagreements interpretation. Chest 1975; 68:278-82 J. The statistical assessment of the variability

perception

Diagnosis

Little,

In:

Saunders,

1970:99-102

1977

S. Burney

for aortography

J Trauma

PC, roentgen

PW.

studies

Behav

diagnosis.

8 Herman

nary rupture

Holland

MIT Press,

Decision-making

in medical

observer

Traumatic

SR, Williams

L.

SE, MA:

N Engl J Med 1971; 284:416-24 7 Scheff’IJ. Decision rules, types

REFERENCES 1 Kirsh

Fienberg

Cambridge,

6 Lusted

TEA that

of the

YMM,

analysis.

with

was also inconsistent. We conclude chest film findings associated with

widened mediastinum knob are most likely

5 Bishop

was

frequently

asssociation

the inter-

sensitivity

SR.

Radiography

of

RE,

in blunt

Cho

thoracic

KJ, Mackenzie trauma:

11

JR.

Doubilet clinical

a reassess-

P, Herman history.

PG.

Am

J

Interpretation

Radiol

1981;

of radiographs:

effect

of

137:1055-58

1982; 22:664-71

Intensive This fifth annual Hopkins University, Corn,

Sciences,

Director,

955-2609).

The

Training in Spirometry

Testing

course, presented by the School of Hygiene and Public Health, The Johns will be held in Baltimore June 2-22. For information, contact Dr. Jacqueline Continuing

Johns

Hopkins

Education

University,

Program, 615

Department

North

Wolfe

of

Street,

Environmental

Baltimore

Health

21205 (301:

CHEST/85/5/MAY,1984

609