Subsequent health-care utilization associated with early physical therapy for new episodes of low back pain in older adults

Accepted Manuscript Title: Subsequent healthcare utilization associated with early physical therapy for new episodes of low back pain in older adults Author: Deven A. Karvelas, Sean D. Rundell, Janna L. Friedly, Alfred C. Gellhorn, Laura S. Gold, Bryan A. Comstock, Patrick J. Heagerty, Brian W. Bresnahan, David R. Nerenz, Jeffrey G. Jarvik PII: DOI: Reference:

S1529-9430(16)31013-0 http://dx.doi.org/doi: 10.1016/j.spinee.2016.10.007 SPINEE 57178

To appear in:

The Spine Journal

Received date: Revised date: Accepted date:

12-9-2015 29-8-2016 12-10-2016

Please cite this article as: Deven A. Karvelas, Sean D. Rundell, Janna L. Friedly, Alfred C. Gellhorn, Laura S. Gold, Bryan A. Comstock, Patrick J. Heagerty, Brian W. Bresnahan, David R. Nerenz, Jeffrey G. Jarvik, Subsequent healthcare utilization associated with early physical therapy for new episodes of low back pain in older adults, The Spine Journal (2016), http://dx.doi.org/doi: 10.1016/j.spinee.2016.10.007. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1 1 2 3

Subsequent Healthcare Utilization Associated With Early Physical Therapy for New Episodes of Low Back Pain in Older Adults

4

Deven A. Karvelas, MD1

5

Sean D. Rundell, PT, DPT, PhD2,3

6

Janna L. Friedly, MD2,3

7

Alfred C. Gellhorn, MD4

8

Laura S. Gold, PhD3,7,9

9

Bryan A. Comstock, MS5,6

10

Patrick J. Heagerty, PhD5,6

11

Brian W. Bresnahan, PhD3,7,9

12

David R. Nerenz, PhD8

13

Jeffrey G. Jarvik, MD, MPH3,7,9,10

14 15 16

Author Affiliations:

17

1

Rebound Orthopedics and Neurosurgery. Vancouver, WA

18

2.

Department of Rehabilitation Medicine. University of Washington. Seattle, WA

19

3.

Comparative Effectiveness, Cost, and Outcomes Research Center. University of

20 21

Washington. Seattle, WA 4.

22

Department of Rehabilitation and Regenerative Medicine. Weill Cornell Medical Center. New York, NY

23

5.

Center for Biomedical Statistics. University of Washington. Seattle, WA

24

6.

Department of Biostatistics. University of Washington. Seattle, WA

Page 1 of 34

2 1

7.

Department of Pharmacy. University of Washington. Seattle, WA

2

8.

Neuroscience Institute. Henry Ford Hospital. Detroit, MI

3

9.

Department of Radiology. University of Washington. Seattle, WA

4

10.

Departments of Neurological Surgery, Health Services, Orthopedics and Sports

5

Medicine. University of Washington. Seattle, WA

6 7

Corresponding Author:

8

Deven Karvelas, MD

9

Rebound Orthopedics and Neurosurgery

10

200 NE Mother Joseph Place, Suite 210

11

Vancouver, WA 98664

12

Phone: 360-254-6161

13

[email protected]

14 15

Word Count: 4,351

16

Number of Tables: 5

17

Number of Figures: 1

18 19 20 21

Key Words: Low Back Pain, Acute Low Back Pain, Older Adults, Physical Therapy, Early Physical Therapy, Healthcare Utilization, Cost

22

Abstract

23

Background: The association between early physical therapy (PT) and subsequent

24

healthcare utilization following a new visit for low back pain is not clear, particularly in

25

the setting of acute low back pain.

Page 2 of 34

3 1

Purpose: To estimate the association between initiating early physical therapy following

2

a new visit for an episode of low back pain and subsequent back-pain-specific health care

3

utilization in older adults.

4

Study Design/Setting: Prospective cohort study. Data were collected at 3 integrated

5

health care systems in the United States through the Back Pain Outcomes using

6

Longitudinal Data (BOLD) registry.

7

Patient Sample: 4,723 adults, aged 65 and older, presenting to a primary care setting

8

with a new episode of low back pain.

9

Outcome Measures: Primary outcome was total back-pain-specific relative value units

10

(RVUs), from days 29-365. Secondary outcomes included overall RVUs for all

11

healthcare and use of specific health care services including: imaging (x-ray and MRI or

12

CT), Emergency Department visits, physician visits, physical therapy, spinal injections,

13

spinal surgeries and opioid use.

14

Methods: We compared patients that had early PT (initiated within 28 days of the index

15

visit) to those not initiating early PT using appropriate, generalized linear models to

16

adjust for potential confounding variables. BOLD was funded by an Agency for

17

Healthcare Research and Quality grant, R01 HS019222-01. The authors report no

18

potential conflict of interest related to this study.

19

Results: Adjusted analysis found no statistically significant difference in total spine

20

RVUs between the two groups (ratio of means 1.19, 95% CI of 0.72 to 1.96, p=0.49). For

21

secondary outcomes, only the difference between total spine imaging RVUs and total PT

22

RVUs was statistically significant. The early PT group had greater PT RVUs, the ratio of

Page 3 of 34

4 1

means was 2.56 (95% CI of 2.17-3.03, p<0.001.) The early PT group had greater imaging

2

RVUs, the ratio of means was 1.37 (95% CI of 1.09-1.71, p=0.01.)

3

Conclusions: We found that, in a group of older adults presenting for a new episode of

4

low back pain, the use of early PT is not associated with any statistically significant

5

difference in subsequent back-pain-specific healthcare utilization compared to patients

6

not receiving early PT.

7 8 9

Introduction

10 11

Low back pain carries a lifetime incidence between 60% and 85% [1, 2], is the second

12

most common reason for a physician visit, and accounts for a substantial portion of our

13

annual healthcare expenditures [1, 3]. Estimates of the direct annual costs for treatment of

14

low back pain range from $12.2 to $90.6 billion with even higher estimates of indirect

15

costs [4]. Outcomes have not improved, even with increased resources allocated to the

16

treatment of low back pain [5]. Physical therapy is a commonly-prescribed treatment for

17

low back pain[6]. However, while there is evidence that physical therapy interventions

18

are beneficial in the treatment of chronic low back pain, evidence of their effectiveness

19

for acute episodes of low back pain is mixed [7-18]. Current guidelines do not

20

recommend the use of physical therapy for acute low back pain [19-21]; rather they

21

recommend patients be educated, advised to remain active, provided with self-care

22

recommendations and managed with acetaminophen or non-steroidal anti-inflammatory

Page 4 of 34

5 1

drugs [19, 21]. Formal physical therapy for low back pain is not recommended until the

2

subacute or chronic setting [18, 19].

3 4

One argument against the use of physical therapy services in the acute setting is that the

5

majority of acute low back pain is self-limiting, typically resolving within 4-12 weeks

6

[22-25]. Therefore, the use of physical therapy for acute back pain may increase

7

healthcare expenditures unnecessarily. Instead, a “stepped approach” of starting with low

8

intensity, low-cost interventions and stepping up care in patients who do not respond has

9

been suggested as a way to manage the cost of care without worsening outcomes [26].

10

However, a systematic review by Hestbaek et al. found that 62% of patients with acute

11

low back pain still had back pain at 12 months and that acute low back pain was

12

associated with a high rate of recurrence [27]. This calls into question the assumption that

13

the majority of acute low back pain is self-limiting and suggests that early identification

14

of patients who are more likely to have continued or recurrent pain and disability may be

15

warranted if effective interventions could be identified. Early physical therapy for acute

16

low back pain may be one such strategy. Rundell et al. found that older adults receiving

17

physical therapy within 28 days after a new physician visit for low back pain showed a

18

modest improvement in function at 12 months compared to those who did not [28].

19

Furthermore, there is recent evidence that use of physical therapy for acute low back pain

20

may lead to a reduction in subsequent healthcare utilization and therefore may not

21

increase overall healthcare costs [14, 17, 29].

22

Page 5 of 34

6 1

For example, Gellhorn et al. found that use of physical therapy for acute episodes of low

2

back pain in the Medicare population was associated with lower subsequent healthcare

3

utilization compared to later use of physical therapy [17]. Given that this was an analysis

4

of claims data, the inability to determine disease severity was an important limitation.

5

Another important limitation of this study was that an episode of low back pain was

6

defined as acute if the patient had not had any physician visits, surgeries or injections for

7

low back pain in the previous year. However, many patients may have had low back pain

8

for months, or even years, before presenting for these services.

9 10

The purpose of this study is to estimate the association between initiating early physical

11

therapy following a new visit for an episode of low back pain and subsequent back-pain-

12

specific health care utilization in older adults after adjusting for disease severity,

13

symptom duration and a number of sociodemographic factors. Given the clinical

14

uncertainty of the utility of physical therapy care in the acute period, we also conducted a

15

sub-analysis of only those patients with self-reported acute low back pain.

16 17 18

Materials and Methods

19 20

Data Source

21 22

We used the Back Pain Outcomes using Longitudinal Data (BOLD) registry, a

23

prospective cohort study of adults, aged 65 and older, with new primary care visits for

Page 6 of 34

7 1

low back pain [30]. The BOLD registry enrolled 5,239 patients presenting to a primary

2

care physician with a new episode of low back pain from three major integrated health

3

care systems (Kaiser-Permanente Northern California, Henry Ford Health System in

4

Detroit, and Harvard Vanguard Medical Associates/Harvard Pilgrim Health Care in

5

Boston.) A new episode of back pain was defined as no prior visits to a health care

6

provider for back pain in the previous 6 months. Major exclusion criteria were previous

7

lumbar spine surgery, developmental spine deformities, inflammatory

8

spondyloarthropathy, spinal malignancy or infection, or a history of cancer or HIV in the

9

last 5 years. Participants were also excluded if full electronic medical record (EMR) data

10

were not available or if they withdrew or passed away prior to the end of the 12 months.

11

A small number of participants (n=134) were also excluded because they had a PT code

12

identified in the first 28 days but no PT evaluation code. These participants were

13

excluded as they may have been continuing PT that was initiated prior to their index back

14

pain visit. (Figure1)

15 16

We collected billing diagnosis (International Classification of Disease, 9th Revision

17

(ICD9) codes) and procedure-related data (Current Procedural Terminology (CPT) codes)

18

from the EMR for 12 months before and after the index visit. At baseline, we collected

19

demographic information as well as robust clinical information using validated patient-

20

reported outcomes. The following measures were administered through written

21

questionnaires or phone interviews at baseline and again at 3, 6, and 12 months: Roland-

22

Morris Disability Questionnaire (RMDQ), numerical rating scales of average back and

23

leg pain in the past 7 days, Brief Pain Inventory activity interference scale, Patient Health

Page 7 of 34

8 1

Questionnaire-4 Depression and Anxiety screen, the EQ-5D, and the Behavioral Risk

2

Factor Surveillance System Survey falls questionnaire. Further details regarding

3

enrollment in the BOLD registry and data collection methods are described elsewhere

4

[30].

5 6 7

Treatment Groups

8 9

Early physical therapy use was defined as initiation of physical therapy initiated during

10

the first 28 days following an index visit for low back pain. We identified those initiating

11

physical therapy using the Current Procedural Terminology (CPT) code for a physical

12

therapy evaluation (97001). One site (Henry Ford Health System in Detroit) used site-

13

specific indicators for physical therapy visits rather than CPT codes, so we defined

14

physical therapy initiation as the first physical therapy visit after the index visit for low

15

back pain for these participants. The comparison group was comprised of participants not

16

undergoing physical therapy within the first 28 days. This group included participants

17

with no physical therapy during the study period and those beginning physical therapy

18

after day 28.

19 20 21

Outcomes

22

Page 8 of 34

9 1

To quantify utilization, we used relative value units (RVUs), a system for reimbursement

2

based on the relative amount of work involved with each procedure in the CPT code set

3

[31, 32]. RVUs are based on three components: physician work (52% of total RVU; time

4

and skill required to provide the service), practice expense (44% of total RVU; expenses

5

for space, supplies, and office staff), and malpractice expense (4% of total RVU) [33].

6

The RVU system is used by the Centers for Medicare and Medicaid Services, and about

7

three-quarters of public and private payers use components of the RVU system to

8

reimburse physicians [34]. Because RVUs provide consistent measures of resource

9

consumption in different medical contexts, they are frequently used as surrogates for

10

outcomes such as expenditures [35-37], productivity [38-43], extent of healthcare

11

services provided [44-47], and physician time [48].

12 13

The primary outcome was total back-pain-specific relative value units (RVUs), also

14

referred to as total spine RVUs, from days 29-365. We chose this as our primary outcome

15

as this provides a single numeric value that is representative of total back-pain-specific

16

healthcare utilization. Secondary outcomes included overall RVUs for all healthcare and

17

use of specific health care services including: imaging (x-ray and MRI or CT),

18

Emergency Department visits, physician visits, physical therapy, spinal injections, spinal

19

surgeries and opioid use. As our intervention (physical therapy) was also part of our

20

primary outcome, we analyzed total spine RVUs for days 1-365 as an exploratory

21

analysis in addition to the primary analysis of days 29-365.

22 23

Page 9 of 34

10 1

Calculation of overall and back-pain-specific RVUs

2 3

We analyzed EMR data from 365 days prior to the date of the index visit as well as the

4

365 days following the index visit to identify hospitalizations, procedures and provider

5

visits, including CPT codes [49] for each procedure or visit. We then mapped each CPT

6

code to its year-specific RVU value [50-52]. To ensure uniformity among the study sites,

7

we did not include geographic modifiers in the RVU calculations. For each individual,

8

we summed all RVUs accumulated from day 29-365 and from day 1-365. We separately

9

summed all RVUs accumulated 365 days prior to the index visit. Similarly, we summed

10

RVUs that were specific to treatment of low back pain between day 29-365 and day 1-

11

365.

12 13

When possible, we used an algorithm that combined CPT and ICD-9-CM codes to

14

determine whether RVUs were back-pain-specific. However, most of the data from

15

Kaiser did not include ICD-9-CM codes. Since some CPT codes are generic (e.g.

16

evaluation and management visits), we only counted procedures at Kaiser as back-pain-

17

related if they took place on the same day as other back-pain-related CPT codes (e.g. x-

18

ray of lumbar spine) or if they occurred on the day of the index visit. One exception to

19

this was physical therapy visits, all of which were assumed to be back-pain-specific as no

20

method was available to determine with certainty whether this was the case. Additionally,

21

some data in the EMRs included patient encounters for procedures such as vaccinations,

22

assessment of blood pressure, and surgical aftercare that did not include CPT codes.

Page 10 of 34

11 1

These were assigned the year-appropriate RVUs associated with CPT code 99211, a 5-

2

minute evaluation and management visit that did not involve physician interaction.

3 4 5

Analysis

6 7

We characterized each treatment group at baseline using descriptive statistics. We then

8

calculated unadjusted means and standard deviations for total spine RVUs, total RVUs,

9

and RVUs for each health service category from days 29-365 for each group. We also

10

described the proportion of patients using a health service category by treatment group,

11

and among those using a category, we calculated means and standard deviations for the

12

number of tests, visits, units, procedures, or morphine equivalent dose per day depending

13

on the use category.

14 15

Since spine RVUs are similar to a count outcome, we estimated the association between

16

early physical therapy and total spine RVUs using generalized linear models with a log

17

link function, Poisson distribution, and robust variance estimates [53]. We adjusted all

18

models for age, sex, race, ethnicity, education, marital status, smoking status, duration of

19

back pain, back-related disability, back pain NRS, leg pain NRS, expectations for

20

recovery, anxiety symptoms, depression symptoms, EQ-5D, study site, and total RVUs

21

for the 365 days prior to the index visit.

22

Page 11 of 34

12 1

In a secondary analysis, we estimated the association between early physical therapy and

2

total RVUs, RVUs by health service category, days with an opioid prescribed, and any

3

use of a health service category. We used similar, adjusted generalized linear models to

4

examine the RVU outcomes and days with an opioid prescribed. However, when

5

examining days with an opioid prescribed, we adjusted for opioid prescription during the

6

365 days prior to index rather than prior RVUs. We estimated associations between early

7

physical therapy and any use of a health service category using logistic regression. We

8

adjusted the logistic regression models for the same confounders mentioned above.

9 10 11

Sub-analysis Treatment Groups

12 13

In order to get a more accurate picture of the association between early physical therapy

14

for acute low back pain and subsequent back-pain-specific health care utilization, we

15

performed a sub-analysis of participants with acute low back pain using self-reported

16

symptom duration. Definitions of acute low back pain vary in duration from 4 weeks to

17

12 weeks [17, 18]. A standard definition for chronic low back pain is somewhat better

18

accepted, with a recent NIH research task force specifically defining this as “a back pain

19

problem that has persisted at least 3 months and has resulted in pain on at least half of the

20

days in the past 6 months [24].” For the purposes of this sub-analysis, we selected

21

participants reporting symptom duration (with respect to the current episode of low back

22

pain for which they were presenting) of less than one month. Given the lack of

23

standardized definitions, we decided that this was most likely to exclude those with

Page 12 of 34

13 1

subacute or chronic pain by any definition. We examined the association between early

2

physical therapy and total spine RVUs using the same generalized linear model in this

3

subgroup. Another exploratory analysis involved stratifying the results from the original

4

analysis by study site, and repeating the primary analysis using total spine RVUs from

5

day 1-365.

6 7 8 9

Missing data

10 11

All included patients had complete EMR data for the 12 months before and after their

12

index visit. There were 133 patients with at least one missing covariate. These patients

13

were included in the descriptive results, but they were automatically dropped from the

14

adjusted models. All analyses were performed with STATA IC 12.1 (College Station,

15

TX), and we set the a priori significance level at p < 0.05.

16 17 18

Results

19 20

Initial unadjusted groups included 4,095 participants who did not undergo early physical

21

therapy and 628 participants who did undergo early physical therapy (Figure 1). The two

22

treatment groups were similar at baseline with a few important exceptions. The early PT

23

group had more participants with 0-3 months of symptom duration than the no early PT

Page 13 of 34

14 1

group (66.3% vs. 51%). The early PT group had fewer participants with greater than 1

2

year of symptom duration than the no early PT group (18.8% vs. 36.7%). The early PT

3

group had more participants with leg pain present (73.3%) compared to the no early PT

4

group (62.3%). The early PT group was more educated and included fewer Black or

5

African American participants. Considerable site variability in use of early PT existed,

6

with participants from Kaiser Permanente making up 82.8% of the early PT group versus

7

only 59.6 percent of the no early PT group (Table 1).

8 9

The early PT group had greater unadjusted RVUs (total RVUs, total spine RVUs and all

10

secondary outcomes, except for total surgical RVUs) from days 29-365 compared to the

11

no early PT group (Table 2). Unadjusted analysis of total units of healthcare use and

12

percentage of participants using specific healthcare services over the same time period

13

showed a similar pattern. The early PT group used more of all healthcare services except

14

for Emergency Department visits and spine surgeries (Table 3).

15 16

Adjusted regression models showed no statistically significant difference between the

17

two groups for our primary outcome of total spine RVUs (ratio of means 1.19, 95% CI of

18

0.72 to 1.96, p=0.49). After adjustment, the early PT group had slightly higher RVUs for

19

all secondary RVU outcomes, but these differences were attenuated, and only total PT

20

RVUs (ratio of means 2.56, 95% CI 2.17 to 3.03, p<0.001) and total spine imaging RVUs

21

(ratio of means 1.37, 95% CI of 1.09 to 1.71, p=0.01) remained statistically significant

22

(Table 4).

23

Page 14 of 34

15 1

After adjustment, the early PT group had slightly higher odds of using physical therapy,

2

physician visits, injections and any spine imaging. These differences were smaller than

3

before adjustment, but did remain statistically significant (Table 5). Additionally, before

4

adjustment the early PT group showed higher odds of opioid use, but after adjustment

5

this difference was no longer statistically significant. There was also no statistically

6

significant difference in total days with an opioid prescribed (ratio of means 0.89, 95% CI

7

of 0.63 to 1.27, p=0.52). (Tables 3 and 5)

8 9

For the subgroup analysis, the early PT group had 246 (39.2%) participants with acute

10

back pain and the no early PT group had 1348 (32.9%). This analysis showed a stronger

11

association between total spine RVUs and early PT. However, similar to our primary

12

analysis, no statistically significant difference in adjusted total spine RVUs was noted

13

between groups (ratio of means 2.01, 95% CI of 0.98 to 4.15, p=0.06).

14 15

As an exploratory analysis, we also stratified results for total spine RVUs for days 29-365

16

by site. At Kaiser-Permanente Northern California and Henry Ford Health System in

17

Detroit the results were similar to our primary analysis, with no statistically significant

18

difference in adjusted total spine RVUs between groups (ratio of means 1.18, 95% CI of

19

0.68 to 2.02, p=0.56 and ratio of means 1.21, 95% CI of 0.68 to 2.17, p=0.52,

20

respectively). However, at Harvard Vanguard Medical Associates/Harvard Pilgrim

21

Health Care in Boston a small statistically significant increase in adjusted total spine

22

RVUs was noted in the early PT group (ratio of means 1.71, 95% CI of 1.09 to 2.68,

Page 15 of 34

16 1

p=0.02). This between-site difference in the association between early PT and total spine

2

RVUs was not statistically significant, p=0.50.

3 4

As another exploratory analysis, we also compared adjusted total spine RVUs from day

5

1-365. Results were similar to the primary analysis of days 29-365. This demonstrated

6

that even with the addition of the RVUs from the physical therapy received during days

7

1-28, there was no statistically significant difference in adjusted total spine RVUs at the

8

end of one year (ratio of means 1.13, 95% CI of 0.76 to 1.70, p=0.54).

9 10 11

Discussion

12 13

In this large prospective cohort study of older adults with low back pain, we found that

14

early PT was not associated with a statistically significant increase or decrease in total

15

back-pain-specific RVUs over the course of the following year. This was true even when

16

the RVUs for the initial physical therapy were included. The early PT group was

17

associated with a small but statistically significant increase in total physical therapy use

18

as well as spine imaging, even after adjusting for available confounding variables.

19

However, the lack of statistically significant difference seen in total back-pain-specific

20

RVUs suggests that these differences were offset by less healthcare utilization in other

21

areas. Our analysis of the subgroup of participants with acute low back pain did show a

22

stronger association between total spine RVUs and early PT. This may suggest a higher

23

proportion of self-limited low back pain within the acute low back pain subgroup.

Page 16 of 34

17 1

However, the noted difference in adjusted total spine RVUs was not statistically

2

significant.

3 4

The differences we found in adjusted RVUs were small. For context, as of January 1,

5

2015 the Centers for Medicare & Medicaid Services used a conversion factor of 1 RVU=

6

$35.75. An X-ray exam of the lumbar spine (2 or 3 views) is 1.07 RVUs, a physical

7

therapy evaluation is 2.18 RVUs, and an epidural injection is 3.37 RVUs. Thus, the

8

difference seen in physical therapy RVUs corresponds closely to a single physical

9

therapy evaluation, and the difference seen in spine imaging is less than a single lumbar

10

spine X-ray [54].

11 12

As noted above, the early PT group did have greater unadjusted RVUs from days 29-365

13

compared to the no early PT group. However, higher healthcare utilization in the early PT

14

group could certainly be explained by a variety of confounding factors. For example, the

15

early PT group had more participants with acute symptoms and more participants with

16

leg pain. These differences in clinical characteristics alone could have contributed to

17

greater healthcare utilization.

18 19

In contrast to Gellhorn et al, our study did not show any association between early PT

20

and decreased subsequent healthcare utilization [17]. The results of this study are,

21

however, consistent with those of Fritz et al. who found that initial management with

22

physical therapy in the first 14 days following a new visit for low back pain was not

23

associated with increased or decreased health care costs or utilization in the following

Page 17 of 34

18 1

year [29]. One possible explanation why the results of Gellhorn et al. differ from those of

2

this study and Fritz et al. is that Gellhorn et al. compared participants receiving early

3

physical therapy to those receiving late physical therapy (defined as greater than 3

4

months after initial visit), which is different from our approach (and that of Fritz et al.)

5

which compared an early PT group to those not receiving PT during the same early time

6

period. Another possible explanation is that, similar to our study, Fritz et al. controlled

7

for more patient variables than Gellhorn et al. In addition to age, gender, comorbidity and

8

initial diagnosis, Fritz et al also controlled for initial management variables such as

9

opioid prescription, corticosteroid prescription, imaging etc, which may have served

10

indirectly as indicators of clinical severity. Our study was also able to use patient

11

reported outcome data to directly account for clinical variability, such as symptom

12

severity and duration. One advantage of our study is that it includes data from three major

13

integrated health care systems from separate geographical regions, while Fritz et al. was

14

only able to look at a single delivery system in one geographic region.

15 16

This study does have important limitations. It was noted that one of our sites, Harvard

17

Vanguard Medical Associates/Harvard Pilgrim Health Care in Boston showed a small

18

statistically significant increase in adjusted total spine RVUs in the early PT group. The

19

reason for this difference between sites is not entirely clear based on the data available to

20

us. However, it may point to differences in the types of Physical therapy treatments

21

provided, and highlights additional areas for study in the future.

22

Page 18 of 34

19 1

Physical therapy itself is a complex intervention with substantial heterogeneity and this

2

study did not account for the variability in types of treatments provided by the physical

3

therapist or the quality of treatments. Therefore, we were only able to comment on the

4

effect of starting any physical therapy treatment without ability to discern the possible

5

impacts of the type, quantity or quality of physical therapy treatments received. However,

6

it isn’t clear that the specific treatments provided in physical therapy are associated with

7

different outcomes. Some studies do show that adherence to an active approach to

8

physical therapy for patients with acute low back pain results in better outcomes and

9

lower subsequent rates of healthcare utilization, but no studies have found one specific

10

set or type of exercises to be superior to another [12, 13]. Similarly, for chronic low back

11

pain, there is good evidence to support the use of physical therapy, but no evidence to

12

support any specific type of physical therapy [7].

13 14

Another important limitation of this study is the omission of other commonly used

15

ancillary therapies such as chiropractic care, acupuncture or massage. While these

16

therapies do contribute to healthcare utilization, use of these services is likely limited in

17

an integrated health care setting with limited coverage for these treatments, and we were

18

unable to capture use of these treatments outside of the health system.

19 20

Also, it should be noted that this study only included adults aged 65 and older and

21

therefore the results of this study may not be generalizable to younger patient populations.

22

Page 19 of 34

20 1

Despite these limitations, this study does provide important insight into a commonly seen

2

group of patients receiving treatment consistent with routine, real world clinical practice.

3

This study is unique in controlling for a number of potential confounding clinical

4

variables, which allowed for more accurate insight into the outcomes associated with

5

early PT use.

6 7

As mentioned previously, Rundell et al has previously published patient reported

8

outcomes (also obtained from the BOLD data set) for older adults receiving physical

9

therapy within 28 days after a new physician visit for low back pain and described a

10

modest improvement in function (RMDQ) at 12 months compared to those who did not

11

[28]. The results of this study, in combination with data that demonstrates modest

12

improvement in function with the use of early PT for some older adults, suggest that

13

prescribing early PT for older adults may lead to modestly improved outcomes without

14

an increase in overall healthcare utilization [28].

15 16 17

Conclusion

18 19

We found that, in a group of older adults presenting for a new episode of low back pain,

20

the use of early PT is not associated with any statistically significant difference in

21

subsequent back-pain-specific healthcare utilization compared to patients not receiving

22

early PT.

23

Page 20 of 34

21 1 2 3 4

References

5

1.

6 7

363-70. 2.

8 9

Andersson, G.B., Epidemiological features of chronic low-back pain. Lancet, 1999. 354(9178): p. 581-5.

3.

10 11

Deyo, R.A. and J.N. Weinstein, Low back pain. N Engl J Med, 2001. 344(5): p.

Druss, B.G., et al., The most expensive medical conditions in America. Health Aff (Millwood), 2002. 21(4): p. 105-11.

4.

Dagenais, S., J. Caro, and S. Haldeman, A systematic review of low back pain cost

12

of illness studies in the United States and internationally. Spine J, 2008. 8(1): p.

13

8-20.

14

5.

15 16

Martin, B.I., et al., Expenditures and health status among adults with back and neck problems. Jama, 2008. 299(6): p. 656-64.

6.

Hart, L.G., R.A. Deyo, and D.C. Cherkin, Physician office visits for low back pain.

17

Frequency, clinical evaluation, and treatment patterns from a U.S. national

18

survey. Spine (Phila Pa 1976), 1995. 20(1): p. 11-9.

19

7.

Standaert, C.J., S.M. Weinstein, and J. Rumpeltes, Evidence-informed

20

management of chronic low back pain with lumbar stabilization exercises. Spine J,

21

2008. 8(1): p. 114-20.

22 23

8.

Malmivaara, A., et al., The treatment of acute low back pain--bed rest, exercises, or ordinary activity? N Engl J Med, 1995. 332(6): p. 351-5.

Page 21 of 34

22 1

9.

Skargren, E.I., P.G. Carlsson, and B.E. Oberg, One-year follow-up comparison of

2

the cost and effectiveness of chiropractic and physiotherapy as primary

3

management for back pain. Subgroup analysis, recurrence, and additional health

4

care utilization. Spine (Phila Pa 1976), 1998. 23(17): p. 1875-83; discussion 1884.

5

10.

Wand, B.M., et al., Early intervention for the management of acute low back

6

pain: a single-blind randomized controlled trial of biopsychosocial education,

7

manual therapy, and exercise. Spine (Phila Pa 1976), 2004. 29(21): p. 2350-6.

8

11.

9

acute low-back pain patients: a one-year prospective study. J Occup Rehabil,

10 11

Gatchel, R.J., et al., Treatment- and cost-effectiveness of early intervention for

2003. 13(1): p. 1-9. 12.

Fritz, J.M., J.A. Cleland, and G.P. Brennan, Does adherence to the guideline

12

recommendation for active treatments improve the quality of care for patients

13

with acute low back pain delivered by physical therapists? Med Care, 2007.

14

45(10): p. 973-80.

15

13.

16 17

Fritz, J.M., et al., Physical therapy for acute low back pain: associations with subsequent healthcare costs. Spine (Phila Pa 1976), 2008. 33(16): p. 1800-5.

14.

Fritz, J.M., et al., Primary care referral of patients with low back pain to physical

18

therapy: impact on future health care utilization and costs. Spine (Phila Pa 1976),

19

2012. 37(25): p. 2114-21.

20

15.

Pinnington, M.A., J. Miller, and I. Stanley, An evaluation of prompt access to

21

physiotherapy in the management of low back pain in primary care. Fam Pract,

22

2004. 21(4): p. 372-80.

Page 22 of 34

23 1

16.

Fritz, J.M., A. Delitto, and R.E. Erhard, Comparison of classification-based

2

physical therapy with therapy based on clinical practice guidelines for patients

3

with acute low back pain: a randomized clinical trial. Spine (Phila Pa 1976),

4

2003. 28(13): p. 1363-71; discussion 1372.

5

17.

6 7

Gellhorn, A.C., et al., Management patterns in acute low back pain: the role of physical therapy. Spine (Phila Pa 1976), 2012. 37(9): p. 775-82.

18.

Hayden, J.A., M.W. van Tulder, and G. Tomlinson, Systematic review: strategies

8

for using exercise therapy to improve outcomes in chronic low back pain. Ann

9

Intern Med, 2005. 142(9): p. 776-85.

10

19.

Chou, R., et al., Diagnosis and treatment of low back pain: a joint clinical

11

practice guideline from the American College of Physicians and the American

12

Pain Society. Ann Intern Med, 2007. 147(7): p. 478-91.

13

20.

Koes, B.W., et al., An updated overview of clinical guidelines for the management

14

of non-specific low back pain in primary care. Eur Spine J, 2010. 19(12): p. 2075-

15

94.

16

21.

Van Tulder, M., et al., Chapter 3. European guidelines for the management of

17

acute nonspecific low back pain in primary care. Eur Spine J, 2006. 15 Suppl 2: p.

18

S169-91.

19

22.

20 21

Pengel, L.H., et al., Acute low back pain: systematic review of its prognosis. Bmj, 2003. 327(7410): p. 323.

23.

Bergquist-Ullman, M. and U. Larsson, Acute low back pain in industry. A

22

controlled prospective study with special reference to therapy and confounding

23

factors. Acta Orthop Scand, 1977(170): p. 1-117.

Page 23 of 34

24 1

24.

2 3

Coste, J., et al., Clinical course and prognostic factors in acute low back pain: an inception cohort study in primary care practice. Bmj, 1994. 308(6928): p. 577-80.

25.

Horal, J., The clinical appearance of low back disorders in the city of Gothenburg,

4

Sweden. Comparisons of incapacitated probands with matched controls. Acta

5

Orthop Scand Suppl, 1969. 118: p. 1-109.

6

26.

7 8

approaches for primary care. Ann Intern Med, 2001. 134(9 Pt 2): p. 911-7. 27.

9

2003. 12(2): p. 149-65. 28.

12 13

Hestbaek, L., C. Leboeuf-Yde, and C. Manniche, Low back pain: what is the long-term course? A review of studies of general patient populations. Eur Spine J,

10 11

Von Korff, M. and J.C. Moore, Stepped care for back pain: activating

Rundell, S.D., et al., Clinical outcomes of early and later physical therapist services for older adults with back pain. Spine J, 2015.

29.

Fritz, J.M., et al., Initial management decisions after a new consultation for low

14

back pain: implications of the usage of physical therapy for subsequent health

15

care costs and utilization. Arch Phys Med Rehabil, 2013. 94(5): p. 808-16.

16

30.

17 18

Data (BOLD) registry. BMC Musculoskelet Disord, 2012. 13: p. 64. 31.

19 20

23

Hsiao, W.C., et al., Results and policy implications of the resource-based relativevalue study. N Engl J Med, 1988. 319(13): p. 881-8.

32.

21 22

Jarvik, J.G., et al., Study protocol: the Back Pain Outcomes using Longitudinal

Hsiao, W.C., et al., Estimating physicians' work for a resource-based relativevalue scale. N Engl J Med, 1988. 319(13): p. 835-41.

33.

Satiani, B., Use, misuse, and underuse of work relative value units in a vascular surgery practice. J Vasc Surg, 2012. 56(1): p. 267-72.

Page 24 of 34

25 1

34.

American Academy of Pediatrics. 2013 RBRVS: What Is It And How Does It

2

Affect Pediatrics? 2013 [ Cited Jan 22, 2013]; Available from:

3

http://www.aap.org/en-us/professional-resources/practice-support/coding-

4

resources/Documents/rbrvsbrochure.pdf.

5

35.

Lind, B.K., et al., Effect of Risk Adjustment Method on Comparisons of Health

6

Care Utilization Between Complementary and Alternative Medicine Users and

7

Nonusers. J Altern Complement Med, 2012.

8

36.

Miller, C.D., et al., Stress CMR imaging observation unit in the emergency

9

department reduces 1-year medical care costs in patients with acute chest pain: a

10

randomized study for comparison with inpatient care. JACC Cardiovasc Imaging,

11

2011. 4(8): p. 862-70.

12

37.

13 14

Raja, A.S., et al., Abdominal imaging utilization in the emergency department: trends over two decades. Int J Emerg Med, 2011. 4: p. 19.

38.

Cheriff, A.D., et al., Physician productivity and the ambulatory EHR in a large

15

academic multi-specialty physician group. Int J Med Inform, 2010. 79(7): p. 492-

16

500.

17

39.

Moote, M., et al., Productivity assessment of physician assistants and nurse

18

practitioners in oncology in an academic medical center. J Oncol Pract, 2012.

19

8(3): p. 167-72.

20

40.

Parent, M.B., K. McArthur, and J. Sava, Are emergency general surgery patients

21

more work than trauma patients? Characterizing surgeon work in an acute care

22

surgery practice. J Trauma Acute Care Surg, 2013. 74(1): p. 289-93.

Page 25 of 34

26 1

41.

2 3

Khandelwal, C.M., et al., Relative value unit impact of complex skin closures to academic surgical melanoma practices. Am J Surg, 2012. 204(3): p. 327-31.

42.

Silich, B.A. and J.J. Yang, Measuring emergency physicians' work: factoring in

4

clinical hours, patients seen, and relative value units into 1 metric. West J Emerg

5

Med, 2012. 13(2): p. 176-80.

6

43.

Davis, P.L., Assessing the potential versus the actual earnings of academic

7

radiologists: effects of unequal duty service assignments. Acad Radiol, 2001.

8

8(8): p. 782-91.

9

44.

10 11

less imaging in emergency departments? J Am Coll Radiol, 2012. 9(1): p. 50-7. 45.

12 13

Moser, J.W. and K.E. Applegate, Imaging and insurance: do the uninsured get

Moser, J.W. and K.E. Applegate, Use of inpatient imaging services by persons without health insurance. J Am Coll Radiol, 2012. 9(1): p. 42-9.

46.

Mortele, K.J., et al., Acute pancreatitis: imaging utilization practices in an urban

14

teaching hospital--analysis of trends with assessment of independent predictors in

15

correlation with patient outcomes. Radiology, 2011. 258(1): p. 174-81.

16

47.

17 18

urban level 1 trauma center. AJR Am J Roentgenol, 2001. 176(4): p. 851-4. 48.

19 20

Henley, M.B., et al., Trends in case-mix-adjusted use of radiology resources at an

Martin, J.D., et al., A real world analysis of payment per unit time in a Maryland Vascular Practice. J Vasc Surg, 2010. 52(4): p. 1094-8; discussion 1098-9.

49.

American Medical Association (AMA). CPT - Current Procedural Terminology.

21

2013 [cited May 8, 2013]; Available from: http://www.ama-

22

assn.org/ama/pub/physician-resources/solutions-managing-your-practice/coding-

23

billing-insurance/cpt.page.

Page 26 of 34

27 1

50.

Medicare program; payment policies under the physician fee schedule, five-year

2

review of work relative value units, clinical laboratory fee schedule: signature on

3

requisition, and other revisions to part B for CY 2012. Final rule with comment

4

period. Fed Regist, 2011. 76(228): p. 73026-474.

5

51.

Medicare program; payment policies under the physician fee schedule and other

6

revisions to Part B for CY 2011. Final rule with comment period. Fed Regist,

7

2010. 75(228): p. 73169-860.

8

52.

9

revisions to Part B for CY 2010. Final rule with comment period. Fed Regist,

10 11

2009. 74(226): p. 61737-2188. 53.

12 13

Medicare program; payment policies under the physician fee schedule and other

Jarvik, J.G., et al., Association of early imaging for back pain with clinical outcomes in older adults. JAMA, 2015. 313(11): p. 1143-53.

54.

Center for Medicare and Medicaid Services (CMS), CMS.gov Physician Fee

14

Schedule. 2015 [cited July 20, 2015]; Available from:

15

http://www.cms.gov/apps/physician-fee-schedule/search/search-criteria.aspx.

16 17 18

Figure Legend

19

Figure 1. Treatment groups and reasons for exclusion

20

Page 27 of 34

28 1

Table 1. Baseline Characteristics of patients presenting to primary care for a new back

2

pain visit by early physical therapy use No Early PT (n=4095)

1

2

Variable

Early PT (n=628)

mean (SD )

mean (SD)

n (column %)

n (column %)

Total (N=4723)

Demographics Age (mean years, SD)

73.8 (6.9)

73.0 (6.4)

73.1 (6.8)

2669 (65.2)

398 (63.4)

3067 (64.9)

240 (5.9)

44 (7.0)

284 (6.0)

18 (0.4)

3 (0.5)

21 (0.4)

Black

668 (16.3)

67 (10.7)

735 (15.6)

White

2971 (72.6)

471 (75.0)

3442 (72.9)

Other

417 (10.2)

82 (13.1)

499 (10.6)

39 (1.0)

8 (1.3)

47 (1.0)

High School Graduate or Less

1262 (30.8)

147 (23.4)

1409 (29.8)

Some College or Trade School

1168 (28.5)

192 (30.6)

1360 (28.8)

At Least College Graduate

1653 (40.4)

289 (46.0)

1942 (41.1)

12 (0.3)

0 (0)

12 (0.3)

Married or Partner

2454 (59.9)

394 (62.7)

2848 (60.3)

Separated, Divorced, or Never Married

680 (16.6)

109 (17.4)

789 (16.7)

Widowed

949 (23.2)

124 (19.8)

1073 (22.7)

12 (0.3)

1 (0.2)

13 (0.3)

< 1 month

1348 (32.9)

246 (39.2)

1594 (33.8)

1 - 3 months

740 (18.1)

170 (27.1)

910 (19.3)

3 - 6 months

265 (6.5)

52 (8.3)

317 (6.7)

Sex (n % Female) Ethnicity (n % Hispanic ) NA Race (n %)

NA Education (n %)

NA Marital Status (n %)

NA Back Pain History and Current Episode Symptom Duration (n %)

6 - 12 months

235 (5.7)

42 (6.7)

277 (5.9)

1 - 5 years

636 (15.5)

65 (10.4)

701 (14.8)

> 5 years

868 (21.2)

53 (8.4)

921 (19.5)

3 (0.1)

0 (0)

3 (0.1)

5.0 (2.8)

5.2 (2.7)

5.1 (2.8)

5.4 (2.5)

5.7 (2.6)

5.5 (2.5)

2551 (62.3)

460 (73.3)

3011 (63.8)

Functional Status (mean RMDQ , SD) 6 Pain Interference with Activity (mean BPI , SD)

9.6 (6.5)

10.1 (6.0)

9.6 (6.4)

3.3 (2.5)

3.7 (2.5)

3.3 (2.5)

Expectation for Recovery (mean NRS, SD)

5.3 (3.7)

6.5 (3.3)

5.5 (3.7)

NA 3

Back pain intensity (mean NRS , SD) Leg pain intensity (mean NRS, SD)

4

Leg pain present (n %) 5

3

Page 28 of 34

29 1

Table 1 Continued. Baseline Characteristics of patients presenting to primary care for a new back

2

pain visit by timing of physical therapy use. General Health Status EQ-5D (mean, SD)

0.76 (0.18)

0.73 (0.17)

0.75 (0.18)

Never

2232 (54.5)

359 (57.2)

2591 (54.9)

Quit > 1 year ago

1585 (38.7)

243 (38.7)

1828 (38.7)

Smoker or quit < 1 year ago

265 (6.5)

25 (4.0)

290 (6.1)

NA

13 (0.3)

1 (0.2)

14 (0.3)

Positive Depression Screen (n %)

329 (8.0)

56 (8.9)

385 (8.2)

Positive Anxiety Screen (n %)

501 (12.2)

89 (14.2)

590 (590 (12.5)

Detroit

803 (19.6)

63 (10.0)

866 (18.3)

Northern California

2440 (59.6)

520 (82.8)

2960 (62.7)

Boston

Other Smoking (n %)

Study Site (n %)

852 (20.8)

45 (7.2)

897 (19.0)

7

35.9 (78.6)

40.6 (120.8)

36.5 (85.4)

Prior Year, Any Opioid Use (n %) 1 Physical Therapy 2 Standard Deviation 3 Numerical Rating Scale 4 Patients with leg pain present only 5 Roland Morris Disability Questionnaire 6 Brief Pain Inventory 7 Relative Value Units

648 (15.8)

101 (16.31)

749 (15.9)

Prior Year’s RVUs (mean, SD)

3 4 5 6 7 8 9 10

Page 29 of 34

30

1

1

Table 2. Unadjusted mean RVUs and unadjusted ratio of mean RVUs for days 29-365 for early PT

Early PT2 (n=628) 3

Mean SD Median 64.7 141.3 25.8 17.0 93.2 0

2

users versus no early PT users

No Early PT (n=4095) Interquartile Range 12.6 53.0 0 5.9

Mean 50.8 9.5

SD 125.3 72.0

Median 20.6 0

Total RVU Total Spine RVU Total Physical 3.7 5.6 1.8 0.0 5.3 1.3 3.6 0 Therapy RVU Total Injections RVU 1.0 4.7 0 0 0 0.6 3.8 0 Total Spine Image RVU 2.9 7.0 0 0 1.1 1.8 5.4 0 Total Surgical RVU 10.1 81.1 0 0 0 5.1 59.0 0 2 1 relative value units, 2 Physical Therapy, 3 Standard Deviation, 4 Confidence Interval

Interquartile Range 8.4 46.4 0 0.5

Ratio of means 1.28 1.79

95% CI4 1.06 1.54 1.10 2.91

p-value 0.01 0.02

0

0

2.86

2.47

3.31

<0.001

0

0

1.79

1.20

2.68

0.01

0 0

0 0

1.64

1.33

2.03

<0.001

1.99

0.97

4.11

0.06

3 4

Page 30 of 34

31

1

Table 3. Unadjusted health service use and unadjusted odds ratio for using a health service

2

for days 29-365.

N All Imaging

142

Xray

76

MRI5 & CT6 Visits

144 7

PT (units )

397

Physician (Visits) Emergency (Visits) Procedures Injections Surgeries

551

Opioids Total Days with Opioids Prescribed MED8/day

140

154 58 10

Early PT1 (n=628) Coun Mean 2 % t 22.6 % 207 1.5 12.1 % 99 1.3 22.9 % 163 1.1 63.2 % 87.7 % 24.5 % 9.2% 1.6% 22.3 %

3

SD

N

1.0

627

0.8

389

0.4

670

2296

5.8

5.9

796

2855

5.2

4.4

2911

326

2.1

1.9

989

110 15

1.9 1.5

1.1 0.8

220 49 679

for early PT users versus no early PT users

No Early PT (n=4095) Coun Mea % t n 15.3 % 890 1.4 9.5% 16.4 % 19.4 % 71.1 % 24.2 % 5.4% 1.2% 16.6 %

4

95% CI

pvalue

SD

OR

1.0

1.62

1.32

1.98

<0.001

489

1.3

0.8

1.31

1.01

1.70

<0.001

730

1.1

0.3

1.52

1.24

1.86

<0.001

4573 1429 3

5.7

5.2

7.12

5.95

8.53

<0.001

4.9

3.8

2.91

2.27

3.73

<0.001

2025

2.0

1.7

1.02

0.84

1.24

0.84

455 75

2.1 1.5

1.5 0.7

1.79 1.34

1.32 0.67

2.43 2.65

<0.001 0.41

1.44

1.18

1.77

<0.001

38.2

66.6

52.3

77.9

4.2

9.7

7.2

18.6

*0.05 *0.06

Page 31 of 34

32

1

*t-test

2 3 4 5

1 Physical Therapy, 2 Mean and standard deviation of those using the corresponding health service 3 Standard Deviation, 4 Odds Ratio, 5 Magnetic Resonance Imaging, 6 Computed Tomography, 7 One unit = a single PT procedure code, 8 Morphine Equivalent Doses

6

Page 32 of 34

33

1

1

2

Table 4. Adjusted mean RVUs and adjusted ratio of mean RVUs for days 29-365 for early PT

Early PT

Mean

2 3 4 5 6 7

SD4

3

users versus no early PT users

No Early PT

Mean

SD

Ratio of mean RVUs

95% CI5

p-value

Total RVUs 56.1 5.0 52.3 2.1 1.07 0.88 1.31 0.49 Total Spine RVUs 12.2 2.6 10.2 1.2 1.19 0.72 1.96 0.49 Total Physical Therapy RVUs 3.4 0.2 1.3 0.1 2.56 2.17 3.03 <0.001 Total Injections RVUs 0.8 0.1 0.6 0.1 1.33 0.89 2.00 0.17 Total Spine Image RVUs 2.5 0.3 1.8 0.1 1.37 1.09 1.71 0.01 Total Surgical RVUs 5.9 1.9 5.7 1.0 1.03 0.50 2.12 0.94 1 Generalized linear models adjusted for age, sex, race, ethnicity, education, marital status, smoking status, duration of back pain, back-related disability, back pain NRS, leg pain NRS, expectations for recovery, anxiety symptoms, depression symptoms, EQ-5D, study site, and prior total RVUs. 2 Relative Value Units, 3 Physical Therapy, 4 Standard Deviation, 5 Confidence Interval

Page 33 of 34

34

1

Table 5. Adjusted1 odds ratios for using a healthcare service from day 29-365 for early PT2 users versus no early PT users

Healthcare Service Imaging XRay MR5 & CT6 Physical Therapy ED6 Visits Physician Visits Injections Surgeries Opioids 2 3 4 5 6 7 8 9

OR3 1.30 1.03 1.18 5.96 0.97 1.47 1.42 0.75 1.13

95% CI4 1.04 1.62 0.78 1.35 0.94 1.47 4.90 7.26 0.78 1.19 1.01 2.13 1.02 1.97 0.37 1.54 0.90 1.43

p-value 0.02 0.85 0.16 <0.001 0.75 0.04 0.04 0.43 0.29

1 Logistic regression models adjusted for age, sex, race, ethnicity, education, marital status, smoking status, duration of back pain, back-related disability, back pain NRS, leg pain NRS, expectations for recovery, anxiety symptoms, depression symptoms, EQ-5D, study site, and prior total RVUs/opioid days 2 Physical Therapy, 3 Odds Ratio, 4 Confidence Interval, 5 Magnetic Resonance Imaging, 6 Computed Tomography, 7 Emergency Department

Page 34 of 34