- Email: [email protected]
‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities
Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
Contents lists available at ScienceDirect
Journal of Transport & Health journal homepage: www.elsevier.com/locate/jth
‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities Patricia Cullen a,b,n, Anna Chevalier a,b, Kate Hunter a,b,c, Tom Gadsden b, Rebecca Ivers a,b a b c
The George Institute for Global Health, PO Box M201, Missenden Road, NSW 2050, Australia School of Public Health, Sydney Medical School, The University of Sydney, Australia The Poche Centre for Indigenous Health, the University of Sydney, Australia
art ic l e i nf o
a b s t r a c t
Article history: Received 29 March 2016 Received in revised form 6 July 2016 Accepted 10 July 2016
Introduction: The low level of licenced drivers in remote Aboriginal communities in the Northern Territory (NT) Australia constitutes a risk factor for transport injury, high incarceration rates, and reduced ability to access employment and health services. The DriveSafe NT Remote program was implemented by the NT Government to increase driver licensing in remote communities. This evaluation reviews the program delivery, acceptability, implementation challenges and licensing outcomes. Methods: A mixed-methods approach was used, incorporating program observation and key informant perspectives (n ¼30). Program data (collected April 2012 to June 2014) and de-identified licensing data from the NT Motor Vehicle Registry were analysed for trends in service delivery and licensing rates pre and post-program. Results: Interviewees reported strong support for the program, and regarded the program as highly engaging and acceptable. There was a greater increase in new licences at intervention sites (Learner 24% and Open licence 18%) compared with other remote areas (Learner licence 13% and Open licence 8%). There appeared to be a dose response relationship with greater licence outcomes at communities that received higher levels of program delivery. Discussion: DriveSafe NT Remote is a Government delivered program providing licensing services to Aboriginal clients in remote Northern Territory communities, and is increasing driver licensing rates. The flexible delivery and culturally responsive approach should lead to further positive licensing outcomes. Addressing barriers to licence participation faced by Aboriginal people is an innovative approach to reducing transport disadvantage and positively impacting health and well-being in remote communities. This evaluation demonstrates a pragmatic approach to assessing program implementation of a multi-site community-based program in a vulnerable and underserviced population; it is anticipated this approach could be applied to other settings to ensure programs responsively address road safety in Aboriginal populations. & 2016 Elsevier Ltd. All rights reserved.
Keywords: Aboriginal Driver licensing Community Evaluation Implementation Culturally responsive
1. Introduction Transport is essential to maintain employment, attend education, socialise and access healthcare. Transport disadvantage results from persistent difficulties accessing public transport, inability to maintain private transport and difficulties meeting transport costs, with far reaching impacts on individual and community health and well-being (Currie and Senbergs, 2007; Rosier and McDonald, 2011). Reduced licence participation in Aboriginal and Torres Strait Islander communities contributes to transport disadvantage, higher rates of transportrelated morbidity and mortality and is a barrier to economic participation, social inclusion, health and well-being (Elliot and Shananhan Research, 2008; Helps et al., 2008; Ivers et al., 2016, 2011). The Northern Territory (NT) has the highest rate of road transport-related injury of any jurisdiction in Australia, and the Aboriginal population of the NT has a fatality rate due to road transportation more than double the rest of the NT population (Australian Institute of Health and Welfare, 2012, p. 24). This has been attributed to known risk factors for transport-related injury in Aboriginal communities: remoteness, non-use of seatbelts, alcohol use and vehicle overcrowding (Clapham n
Corresponding author. E-mail address: [email protected] (P. Cullen).
http://dx.doi.org/10.1016/j.jth.2016.07.004 2214-1405/& 2016 Elsevier Ltd. All rights reserved.
Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
2
et al., 2008; Henley and Harrison, 2013). Unlicenced driving is also common, due to the vastness of the region and poor access to licensing services. As a result of not having a licence while living in a remote area with little public transport, people are more inclined to make unsafe driving choices or engage in illegal driving practices (Job and Bin-Sallik, 2013). Low rates of licensing in communities also contributes to an over-represented Aboriginal prison population due to a high number of repeat licence offences and difficulty accessing employment and developmental opportunities (Job and Bin-Sallik, 2013; Somssich, 2009). Across Australia, the NT recorded the highest imprisonment rate at 885 prisoners per 100,000 adult population in 2015 (Australian Bureau of Statistics, 2015). Aboriginal and Torres Strait Islander prisoners made up 84% of the prison population, despite comprising only 32% of the NT population (Australian Bureau of Statistics, 2015). The rate of incarceration for vehicle regulatory and driving offences is 25% of NT prisoners, of which approximately 97% are Aboriginal (Anthony and Blagg, 2012). Several reports have highlighted the complex range of barriers that contribute to the high rate of unlicenced drivers in remote Aboriginal communities: a lack of appropriate identification documentation (birth certificates), limited access to licensing and testing facilities, low levels of literacy, lack of self-esteem, high cost of driving lessons, poor access to eligible supervisory drivers for learners, language barriers, reduced availability of licensing services, a lack of access to appropriate vehicles, a general culture of being unlicenced and unsafe driving practices (Elliot and Shananhan Research, 2008; Helps et al., 2008; Job and Bin-Sallik, 2013). The DriveSafe NT Remote program was implemented by the NT Government to address barriers to licensing faced by remote communities. Similar programs have been developed and implemented previously, yet many have proved unsustainable due to inadequate funding and an inability to demonstrate outcomes. Hence, licensing support programs are under-evaluated and there is limited evidence to establish best practice for program development and implementation. This process evaluation aims to review program implementation and explore whether DriveSafe is being implemented as intended, addressing the needs of the target communities and leading to positive licensing outcomes.
2. Methods 2.1. Design A mixed-methods approach was selected to explore informant perspectives, program delivery and licensing outcomes using deidentified licensing data for remote communities in the NT from the Motor Vehicle Registry, Northern Territory Department of Transport.
2.2. The program The DriveSafe NT Remote program commenced in April 2012, and was sequentially rolled out across 23 communities. Prior to the delivery of the program, the DriveSafe team visited a community three times on average to conduct community consultations, promote the program and identify a host organisation. Then the program team and host organisation ensured community members have proof of identification to apply for a Learner licence. Once community members have identity documentation, DriveSafe field staff deliver the Learner theory component of the program and administer the Learner's licence theory test. The licensing process in the NT is outlined in Fig. 1. Field staff return at regular intervals to deliver professional driving lessons. In between professional lessons, participants engaged in supervised driving practice where available with local community members who were licenced drivers. In the final stage of the program, field staff deliver road safety education, and administer the Provisional driver licence practical test. A graduation ceremony is held to present the licences.
2.3. Quantitative data sources and collection 2.3.1. Program data Program data was routinely collected throughout delivery from April 2012 to May 2014, and was supplied to evaluators in a deidentified form. Data consisted of licensing outcomes, birth certificates obtained and professional lessons delivered. Data was reported for 15 locations: seven primary locations were excluded on the basis of incomplete data and/or where program delivery commenced in May 2014, and data for Jabiru and Gunbalanya was combined. The 15 locations are shown on a map of the NT along with Darwin and Alice Springs (Fig. 2). Licence Class
Minimum Age
Eligibility for licence
Period of licensure
Learner licence
16 years
Pass a computer based theory test
Provisional licence
16 years and 6 months
Pass a practical on road driving test
Hold the Learner licence for 6 months Hold the Provisional licence for 24 months if under 25 years of age Hold the Provisional licence for 12 months if over 25 years of age
Full licence
18 years and 6 months
No testing or requirements, licence is upgraded upon renewal once provisional licence period has passed
Fig. 1. Licensing requirements, eligibility and time periods in the Northern Territory, Australia.
Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
3
Fig. 2. Map of Darwin, Alice Springs and DriveSafe Remote NT communities.
Table 1 Program delivery and outcomes at DriveSafe NT Remote regions from April 2012 to May 2014. Region
Groote Eylandt Tiwi Galiwinku Maningrida Wadeye Gapuwiyak Jabiru, Gunbalanyab Roper Gulf/Ngukurr Nhulunbuy Barkley/Tennant Creek Katherine Mandorah Batchelor Milingimbi/Ramingining Vicdaly/Lajamanu Total
a b
Program delivery
Program outcomes
Birth certificatesa
Driving lessons
Learner licences
Provisional licences
Total licences
53 43 98 63 17 60 47 28 26 9 2 0 0 21 1 468
203 255 523 54 60 528 52 33 127 51 0 80 4 0 0 1970
191 170 198 176 148 121 110 111 117 81 22 25 10 5 0 1586
97 82 48 43 41 57 33 30 16 8 10 0 3 0 0 486
288 252 246 219 189 178 143 141 133 89 32 25 13 5 0 1953
Not all program participants required birth certificates as they had attained proof of identification documents prior to the program delivery. Data for Jabiru and Gunbalanya are combined.
2.3.2. Licensing data Licensing data for remote communities in the NT were obtained from the Motor Vehicle Registry, NT Department of Transport. The data included licences held within remote regions at a single point in time each year from 1 December 2010 to 1 May 2014, by licence type, postcode and suburb. There were two licence types: Learner and Open licences. A person with a Learner licence held no other class or type of licence. A person with an Open licence may hold a Provisional or full-unrestricted licence. 2.4. Qualitative data sources and collection This aspect of the evaluation involved reviewing program materials, observing delivery and conducting in-depth semi-structured interviews with program staff, program participants and community stakeholders. Purposive sampling identified key informants to Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
4
participate in interviews (Patton, 1990). Snowball sampling was employed, with all interviewees asked to recommend other interviewees (Liamputtong and Ezzy, 2005). The number of interviews was determined by data saturation.
2.4.1. Interviews with program staff To enable feedback on both program structure and delivery, interviews were conducted with DriveSafe and host organisation program staff (n ¼8) in November 2012 and April 2014. 2.4.2. Interviews with clients Interviews were conducted with clients (n ¼10) in November 2012 in Wurrumiyanga community (Tiwi islands). Of the 10 interviewees, half were female and 60% aged 18–34 years old. Participants were asked about their driving and licensing experiences before and after the program, experiences with the program, as well as course content and teaching style.
2.4.3. Interviews with stakeholders Face-to-face or telephone interviews about delivery and impact of the program were conducted with community stakeholders (n ¼12) between November 2012 and April 2014. This included representatives of organisations delivering services to the community.
2.5. Analysis 2.5.1. Quantitative data Program data was descriptively analysed by community to determine number of delivery days, professional driving lessons, birth certificates and licences obtained. Due to the post-program evaluation design and small sample size, significance testing was not undertaken. Trends were examined of licences held in remote areas pre and post commencement of the program by licence type. Comparisons were made between locations that received the program and similar locations that did not receive the program.
2.5.2. Qualitative data Interviews were audio recorded and professionally transcribed; the data was managed using Nvivo 10 software (QSR International Pty Ltd, 2012). A general inductive approach to the analysis was employed, with themes emerging from initial categorisation of the data (Thomas, 2006). The research team sought feedback from a panel of expert advisors that included Aboriginal and non-Aboriginal researchers with expertise in community health, road safety, driver licensing and injury prevention.
2.6. Ethics The evaluation was approved by Human Research Ethics Committees from the NT Department of Health and Menzies School of Health Research. Program data was provided to the research team in de-identified form. All participants provided written consent. 600
500
400 Delivery days
Lessons
300
200
100
0
Community Fig. 3. DriveSafe NT Remote program delivery days and lessons at each region from April 2012 to May 2014.
Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
5
350
Licence outcomes
300 250 200 150 100 50 0
0
20
40
60
80
100
120
140
160
Program delivery days Fig. 4. Positive association between program delivery days in communities and total licence (Learner and Provisional) outcomes achieved. Table 2 Change in licences held for DriveSafe NT Remote compared to other remote locations, by licence type from 1st Dec 2010 to 1st May 2014. Location
Pre
% Changea
Post
2010
2011
Average
2012
2013
2014
Average
Learner
DriveSafe Other Remote Sub-total
97 108 205
100 113 213
99 111 209
115 118 233
130 123 253
122 134 256
122 125 247
24% 13% 18%
Open
DriveSafe Other Remote Sub-total
312 548 860
317 579 896
315 564 878
343 582 925
378 623 1001
390 623 1013
370 609 980
18% 8% 12%
a
% Change pre to post ¼ difference pre- to post-average/pre-average as %.
3. Results 3.1. Program delivery Between April 2012 and May 2014, 832 program delivery days and 1970 driving lessons were provided by the program. The number of lessons delivered in each primary location ranged from a high of 528 at Gapuwiyak to a low of four at Batchelor, and none at three other communities (Table 1). Licensing outcomes also varied widely between locations, with a tendency toward increased outcomes at locations with higher levels of program delivery (Table 1). Comparison between communities found Galiwinku, Gapuwiyak, Tiwi and Groote Eylandt were the four regions with greatest levels of combined program delivery days and lessons (Fig. 3). This corresponded with the four locations that attained the highest number of Provisional licences (Groote Eylandt, Tiwi, Gapuwiyak and Galiwinku) (Table 1). A positive association was found between higher levels of program delivery days (dose) and total licence outcomes (response) (Fig. 4). 3.2. NT licensing data On average, during the 3.5 years from December 2011 to May 2014, 80% of people in remote regions holding a driver licence held an Open licence and 20% held a Learner licence. The average number of Learner licences held was similar in areas where the program was delivered (49%), compared to other remote areas (51%). However, trends over time found Learner licences increased more on average where the program was delivered (24%) compared to other remote areas (13%) (Table 2). On average, over the 3.5 year period, the number of Open licences held was lower where the program had been delivered (37%) compared to other remote areas (63%). Despite this, trends over time also found Open licences increased more where the program was delivered (18%) compared to other remote areas (8%) (Table 2). 3.3. Informant perspectives Using a general inductive approach, the initial categories were refined. Five main themes emerged aligning with the evaluation aims: 1) Meeting the need; 2) Collaboration and respect; 3) Breaking down the barriers; 4) ‘I think it was the people involved’; 5) Sustainability and overcoming challenges. 3.3.1. Meeting the need: ‘It’s exactly what we needed’ In exploring the need for the program, it was evident there was a substantial need not just for licensing access but for service delivery appropriate for culturally-diverse and under-resourced remote communities. Multiple communities approached the team to be included in the program, and at each site there were multiple requests for further programs. This was consistent with community stakeholder and program participant interviews that overwhelmingly identified a need for licensing service delivery in remote NT communities. Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
6
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
“I just hope it goes on and on and on, because there is just such a definite need” Interviewees emphasised licensing support will help address road safety issues in remote communities. Unlicenced driving was consistently highlighted as the main issue in communities, particularly involving young and under-age drivers. Additionally, interviewees frequently referred to the importance of a driver licence to securing employment and accessing opportunities. “Because we don’t have an MVR office here, there’s a lot of people, particularly young people, who don’t have a licence… and it’s causing problems, they will still drive cars. and it makes it hard for younger and older people to get work as well” “It was just perfect…. Getting that program, helping those young people get their licence – it’s immeasurable” Interviewees emphasised the value of the program in reducing offending and incarceration. “Now those kids can drive a car without the worry of maybe driving without a licence and ending up coming to the police and in gaol over such a simple thing as not having a licence” “It might mean that a lady who might have kids, a young lady might not go to gaol because her husband makes her drive but she hasn’t got a licence or something, so then those kids get to keep their Mum, she doesn’t go to gaol” 3.3.2. Collaboration and respect It emerged that by prioritising collaboration and respect, the DriveSafe program was highly responsive to cultural norms and unanticipated challenges. Interviewees frequently cited the positive attitude of DriveSafe staff and their ability to overcome challenges and problem-solve with communities. When the program team arrived in a community, word-of-mouth quickly spread such that the demand to participate was often greater than anticipated. In communities where demand for the program exceeded expectations, program staff allocated extra time to ensure they could service as many participants as possible. Interviewees reported DriveSafe was very successful in collaboratively addressing the specific needs of each community. For example, one community requested assistance for 15 teachers to get their bus licence; this benefited the whole community as students could be taken to educational and sporting events. “Well, it’s an opportunity for the kids so they don’t miss out. It just means more opportunity, more bus licences, you can get kids to and from things and the teachers feel they’re important” The program considered and respected the cultural norms of each community by spending time engaging with the host organisation, assessing the needs and adjusting the program delivery accordingly. It was common in the remote regions for community members to have had little exposure to formal schooling; the programs' responses to this included: a) tailoring the program to participants' literacy needs; b) using oral testing for the Learner theory; c) presenting program materials in an engaging and interactive format; d) providing regular breaks to enhance participant concentration. In some communities it was preferable to hold Learner theory classes with male and female community members separately. Similarly, the program aimed to gender match driving instructors with Learner drivers; where this was not feasible Learner drivers were able to bring a support person to the lesson. The impact of this responsiveness was acknowledged by interviewees. “There would be kids there who have never – and especially female, who I reckon never would have got a licence in their life without this program” 3.3.3. Breaking down the barriers Attaining a licence was seen to be a significant achievement, and DriveSafe was perceived as reducing the barriers to licensing in remote communities. Several elements of the program delivery model emerged as essential to reducing the barriers to licensing. Commonly cited was the program was free and based in the community. These were deemed to be essential components of the program, as participation in a program further away would have been inaccessible for remote community members who do not have access to transport and/or have a reduced income. Further, being able to participate in a familiar environment and close proximity to family and social networks is inherently important to Aboriginal people, who have strong kinship and cultural connections to their community. “Kids see it as something important….that was something those kids genuinely wanted to do…. and just didn’t have the ability or opportunity to make it happen” The DriveSafe team's ability to organise birth certificates was also viewed as fundamental to accessing a licence. Proof of identity can be a considerable barrier for remote community members, as without private transport it can be difficult to access relevant Government departments. Additionally, there is an increased likelihood in remote communities a person's birth may not have been registered and there are often issues with identity documents with different names or misspellings. “So we do the ID. We get a birth certificate and we actually identify who they are. We tidy up the database and go, ‘Right, you used to be known by six names; you are now known by one. We’ve taken a photo; we’ve merged everything. Our database now is perfect and you’re ready to go’” 3.3.4. ‘I think it was the people involved’ DriveSafe relies heavily on the commitment and expertise of the field staff that travel between communities; accordingly the skills and performance of field staff were frequently discussed by interviewees. Program management interviewees indicated field staff are critical to the success of the program and cited the importance of having staff skilled in delivering road safety messages and licensing support, as well as engaging community with cultural respect. Field staff were able to work autonomously and were skilled at adapting the program to the local context, working within the capacity and resources of each community. Program management acknowledged that while the majority of staff were not Aboriginal, they had extensive experience working with remote Aboriginal communities. Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
7
“It’s a great program, it’s a great idea but if you have the wrong people running it” Field staff are handpicked from experienced motor vehicle registry licenced testing officers, and some were also licenced driving instructors. Further, community confidence and engagement was enhanced by having consistent trainers and instructors over the program duration. “As long as you keep the right people offering the program, that’s the biggest drama up here is the turnover of people” Program participants reported the driving instructors were skilled, appropriate, patient and funny. Participants reported learning about technical skills associated with driving a car, as well as road safety issues, such as speeding, drinking and restraint use. Driving practice or ‘jumping in the car’ was commonly highlighted as the best part of the program, made better because of the rapport with instructors. This was reinforced by stakeholder interviewees, who rated the quality of the program presentation highly, and particularly the rapport DriveSafe staff developed with participants, which created a supportive environment. Stakeholders also emphasised the program’s delivery of important road safety information (speeding, drink driving, distraction, mobile phone use, seatbelts) was enhanced by delivery that was engaging and meaningful to remote community members. 3.3.5. Sustainability and overcoming challenges There is considerable variation across communities in terms of capacity, resources and engagement with the program, which have presented ongoing implementation challenges; however staff also thought the ability to adapt to these differences was a key strength of the program. For instance, initially, the program relied heavily on professional driving instructors to deliver driving lessons in each community, costing up to $10,000 per week. This model was not deemed to be cost-effective nor sustainable in the long term. To improve cost-effectiveness, the program has focused on developing local capacity to provide supervised driving lessons, which is proving critical to program sustainability. Limited availability of resources has also been a challenge, particularly in the case of acquiring a vehicle to be kept in communities for supervised driving practice. Staff reported a number of communities did not have the capacity to attain or maintain a vehicle, which impacted their ability to deliver driving lessons and achieve licensing outcomes. To address these challenges, DriveSafe adapted the program model to gain access to funding from the Remote Jobs and Communities Program (RJCP) providers (Department of Employment, 2014). RJCP is an Australian Government funded program which supports training and other activities for job readiness and building community capacity. Management staff cited the development of this relationship between DriveSafe and RJCP's in participating communities as integral to building community capacity to deliver supervised driving practice and improving program sustainability. Project staff highlighted the need to revise Government regulations and policy to enable Government vehicles could be utilised for Learner drivers. This was seen as a substantial barrier to delivering supervised driving practice and was required to be dealt with on a case by case basis. Furthermore, use of these vehicles would facilitate better employment pathways as Learners could potentially be employed and gain a driver licence as part of a work plan.
4. Discussion The DriveSafe program is achieving licensing outcomes in underserviced remote NT regions. The program has demonstrated capacity for a flexible approach that is responsive to remote communities. There was variation in program delivery across regions, which predominately reflects flexible service provision adapted to meet community needs. Licensing trends show Learner and Open licences have increased more on average where the program has been delivered compared to other remote areas. There was a positive association between communities that received a high dose of the program and increased licensing outcomes. This relationship was reinforced by staff reports that program delivery is strengthened in communities that actively engage with the program. It is possible this relationship is underscored by need for the program, whereby communities with a high need receive an increased dose and are more highly engaged, which leads to greater licensing outcomes. There is a recognised need to address high levels of road transport injury in vulnerable populations (World Health Organisation, 2013). In Australia, remote communities with a high proportion of Aboriginal residents represent an underserviced and vulnerable population with low rates of licence participation, high rates of incarceration and high levels of road transport injury (Edmonston et al., 2003; Elliot and Shananhan Research, 2008; Helps et al., 2008; Ivers et al., 2011; Naylor, 2010; NSW Auditor General, 2013; Somssich, 2009). The DriveSafe program is addressing areas of greatest need, as the average number of Open licences held was higher in other remote areas compared to where the program has been delivered. This finding is consistent with interviewee reports that the program is targeting communities and clients with a high level of need. The implementation of DriveSafe has demonstrated the program's capacity for flexible delivery and community engagement. While DriveSafe is regarded as highly acceptable to remote communities and exhibits many indicators of long-term viability, increased Aboriginal involvement and leadership is recommended to promote inclusive partnerships with Aboriginal communities and build local capacity (Martiniuk et al., 2010; NSW Auditor General, 2013). The NSW Auditor General (2013) has recommended that programs prioritise the involvement of Aboriginal people in development, implementation and delivery; this enhances the cultural appropriateness of materials, teaching methods, aids in capacity building and ultimately promotes program longevity. Thereby, the program's sustainability and acceptability could be enhanced with an increased focus on employing Aboriginal people at a community and program management level. While the process for attaining a driver's licence varies considerably between jurisdictions, there is a recognised need to facilitate fair and equitable access to the licensing system (Elliot and Shananhan Research, 2008; Helps et al., 2008; Job and Bin-Sallik, 2013; Naylor, 2010; NSW Auditor General, 2013; Somssich, 2009; Williamson et al., 2011). Several Australian states have made progress toward improving licensing access for underserviced populations. The Victorian Government has funded the implementation of the L2P Learner Driver Mentor Program to assist young people (aged 16–20 years) who are experiencing disadvantage accessing a licence. As of December 2013, the program is running in 62 out of Victoria's 78 local government areas, and has capacity to assist 2000 young people at a time. The Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
8
L2P program is free for participants, however it does not assist with licence or test fees and participants must already hold their Learner licence to be eligible for enrolment (Freethy, 2012). In New South Wales, the Driving Change program began in February 2013 and has been implemented in twelve communities; it is hosted by a community organisation at each site and employs a local Aboriginal coordinator to provide intensive case-management for participants through the licensing system (Ivers et al., 2012). Driving Change provides assistance with all aspects of licensing including proof of identity, financial assistance for test/licence fees, Learner Driver Knowledge Test preparation, supervised driving hours and assistance to manage state debt and/or fine (Cullen et al., 2016). Similar to DriveSafe Remote, Driving Change is strengthened by community engagement and relies on the recruitment of community volunteers to deliver supervised driving practice. Reduced licence participation is Aboriginal communities has been cited as both a cause and consequence socioeconomic disadvantage and a contributor to health disparity (Ivers et al., 2016). Ivers et al. (2016) reported that having a driver licence was significantly associated with higher odds of full-time employment and post-secondary education; those who held a driver licence (provisional or full) were four times more likely to be in full-time employment and two to four times more likely to have higher levels of formal education. The authors concluded that reduced driver licensing participation has far-reaching impacts on the health and wellbeing of Aboriginal people. Transport disadvantage due to reduced licence participation is particularly problematic in remote communities as there are limited options for transport without access to a private car and licenced driver. Further, transport disadvantage contributes to unsafe transport choices including over-crowding of vehicles and unlicenced driving, which is a risk-factor for transport injury and incarceration (Job and BinSallik, 2013; Somssich, 2009; Transport for NSW, 2014; Watson, 2004; Watson et al., 2011). This evaluation has demonstrated that DriveSafe is positively impacting licensing in remote communities, which beyond improving the mobility of community members will reduce the risk of transport injury, incarceration and promote community health through improved employability, access to services and opportunities. The impact of licence participation and access in other Indigenous populations globally (e.g. Native American) has not been reported. While there is evidence that Indigenous populations in the United States are over-represented in transport fatality, little is known about the role that driver training and licensing may play as a protective factor (Pollack et al., 2012). Further, evidence suggests that Maori populations in New Zealand experience significant transport disadvantage, which has been described as ethnically mediated transport disadvantage (Raerino et al., 2013). Recommendations for addressing this issue in Indigenous populations include recognising the importance of Indigenous systems of wellbeing in the design of transport policy and determining the enablers of safe transport in communities (Raerino et al., 2013). To this end, the current evaluation has provided insight into the enablers of safe and legal driving in remote Aboriginal communities in Australia; it is anticipated that program evaluation employing this pragmatic approach could determine barriers and facilitators to safe transport in other Indigenous contexts. This process evaluation has several strengths and weaknesses. Importantly, the evaluation sought to obtain evidence of program impact and acceptability from multiple data sources. This provided rich evidence of the multiple methods the program is using to impact licensing, road safety and mobility in remote communities. Further, emphasis was placed upon ensuring community members were included in the evaluation, in particular to explore the acceptability and impact of the program. A limitation of the current study was the small sample of communities included in the qualitative interviews; the present results could be extended to a greater sample of communities who had accessed the program in particular to explore the generalisability of the program to other contexts. Due to the postprogram design, there was no control group and participants were not randomised. The program and licensing data was aggregated at the community level and individual data was not available, hence it was not possible to track the progress of individual participants. Nevertheless, process evaluations of multi-site community programs provide a valuable and pragmatic approach to ensuring the intervention is being implemented as intended to the target population. Overall, this evaluation indicates that a community-based approach is sufficiently flexible and highlights the importance of strong community support to facilitate equitable access to licensing. It is likely that other vulnerable populations, which are not limited to Aboriginal or remote communities, but also include culturally and linguistically diverse populations (e.g. refugees) could benefit from an inclusive and pragmatic approach to overcome barriers to licensing.
5. Conclusion Remote communities in the NT represent an underserviced population for driver licensing services, which contributes to low rates of licenced drivers, high incarceration rates and impedes access to essential health services and employment. DriveSafe NT Remote is addressing areas of high need (underserviced remote communities) and is regarded highly. The program is positively impacting licensing rates in remote communities, which reduces the risk of transport injury, and contributes to improved mobility, employability and access to essential services. Despite early challenges to implementation and resourcing, the program’s resilience is evidenced by the delivery of licensing outcomes in remote NT communities. Further, a dose response relationship was evident, with licence outcomes greatest at communities receiving a high dose of program delivery. In terms of delivery, the program’s greatest strength is the consistently high level of cultural responsiveness and adaptability to changing conditions, which should facilitate ongoing success. While driver licensing legislation varies across jurisdictions, it seems likely that this model could be successfully applied elsewhere.
Funding Contributing authors were contracted by the Northern Territory (NT) Government to undertake an independent evaluation of the DriveSafe NT Remote program. The DriveSafe NT Remote program was initiated by the NT Government, funded by the former National Road Safety Council (with funding administered by the Department of Infrastructure and Transport, Australian Government), Territory Insurance Office (TIO), the Australian Government Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA) and the NT Government’s Department of Transport. Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
9
Acknowledgements The authors acknowledge the contribution of DriveSafe NT Remote staff and communities.
References Anthony, T., Blagg, H., 2012. Addressing the “crime problem’’ of the Northern Territory Intervention: Alternate paths to regulating minor driving offences in remote Indigenous communities (CRG 38/09-10). Criminology Research Advisory Council. Australian Bureau of Statistics, 2015. 4517.0 - Prisoners in Australia. (4517.0). Canberra. Retrieved from: 〈http://www.abs.gov.au/AUSSTATS/[email protected]/Lookup/4517.0Main þFeatures12015?OpenDocument〉. Australian Institute of Health and Welfare, 2012. Trends in serious injury due to land transport accidents, Australia 2000-01 to 2008-09. Injury research and statistics series no. 66 Cat. no. INJCAT 142. Canberra. Cullen, P., Clapham, K., Byrne, J., Hunter, K., Rogers, K., Senserrick, T., Ivers, R., 2016. Implementation of a driver licensing support program in three Aboriginal communities: a brief report from a pilot program. Health Promot. J. Aust. (Accepted) Currie, G., Senbergs, Z., 2007. Indigenous communities: transport disadvantage and Aboriginal communities. In: Currie, G., Stanley, J., Stanely, J. (Eds.), No Way To Go: Transport and Social Disadvantage in Australian Communities. Monash University ePress, Clayton, Victoria. Department of Employment, 2014. Remote Jobs and Communities Program. From: 〈https://employment.gov.au/remote-jobs-and-communities-program〉. Edmonston, C.J., Rumble, N., Powell, J., Butler, S., Nona, H., Watson, B.C., Schonfeld, C.C., 2003. Working with Indigenous communities to improve driver licensing protocols and offender management. Paper presented at the 2003 Road Safety Research: Policing and Education Conference, Sydney, NSW. Elliot and Shananhan Research, 2008. An Investigation of Aboriginal Driver Licensing Issues – Aboriginal Licensing Final Report. NSW Roads and Traffic Authority, Sydney. Freethy, C., 2012. L2P – learner driver mentor program: extending driver licensing reach in disadvantaged communities. Paper presented at the Australasian College of Road Safety Conference, Sydney, NSW. Helps, Y.L.M., Moller, J., Kowanko, I., Harrison, J.E., O’Donnell, K., de Crespigny, C., 2008. Aboriginal People Travelling Well: Issues of safety, transport and health. Department of Infrastructure, Transport, Regional Development and Local Government, Canberra, ACT. Ivers, R., Byrne, J., Hunter, K., Clapham, K., 2012. Development of a community based Aboriginal driver licensing service: the AstraZeneca Young Health Programme. Paper presented at the Australasian College of Road Safety Conference, Sydney, NSW. Ivers, R., Hunter, K., Clapham, K., Helps, Y., Senserrick, T., Byrne, J., Harrison, J.E., 2016. Driver licensing: descriptive epidemiology of a social determinant of Aboriginal and Torres Strait Islander health. Aust. NZ J. Public Health . http://dx.doi.org/10.1111/1753-6405.12535. Ivers, R., Lyford, M., Johnson, J., Boufous, S., Senserrick, T., Clapham, K., 2011. Road safety and driver licensing in Aboriginal people in remote NSW. Paper presented at the Coalition for Research to Improve Aboriginal Health, Sydney, NSW. Job, R.F.S., Bin-Sallik, M.A., 2013. Indigenous road safety in Australia and the “Drivesafe NT Remote” project. J. Australas. Coll. Road Saf. 24 (2), 21–27. Liamputtong, P., Ezzy, D., 2005. Qualitative Research Methods. Oxford University Press, Mebourne. Martiniuk, A., Ivers, R., Senserrick, T., Boufous, S., Clapham, K., 2010. Effective and inclusive intervention research with Aboriginal populations: an Evidence Check rapid review brokered by the Sax Institute for NSW Health. Naylor, B., 2010. L-plates, logbooks and losing-out: regulating for safety – or creating new criminals? Altern. Law J. 35 (2), 94–98. NSW Auditor General, 2013. New South Wales Auditor-General's Report to Parliament: Improving Legal and Safe Driving Among Aboriginal People. Audit Office of New South Wales, Sydney, NSW. Patton, M., 1990. Qualitative Evaluation and Research Methods, 2nd ed. Sage, Newbury Park. Pollack, K.M., Frattaroli, S., Young, J.L., Dana-Sacco, G., Gielen, A.C., 2012. Motor vehicle deaths among American Indian and Alaska native populations. Epidemiol. Rev. 34 (1), 73–88. http://dx.doi.org/10.1093/epirev/mxr019. QSR International Pty Ltd., 2012. NVivo qualitative data analysis software (Version 10). Raerino, K., Macmillan, A.K., Jones, R.G., 2013. Indigenous Māori perspectives on urban transport patterns linked to health and wellbeing. Health Place 23, 54–62. http://dx. doi.org/10.1016/j.healthplace.2013.04.007. Rosier, K., McDonald, M., 2011. The relationship between transport and disadvantage in Australia. Australian Institute of Family Studies, Melbourne. Somssich, E., 2009. Driver training and licensing issues for indigenous people. J. Australas. Coll. Road. Saf. 20 (1), 31–36. Thomas, D.R., 2006. A general inductive approach for analyzing qualitative evaluation data. Am. J. Eval. 27 (2), 237–246. http://dx.doi.org/10.1177/1098214005283748. Transport for NSW, 2014. NSW Aboriginal Road Safety Action Plan 2014-2017. (978-1-922030-43-6). Sydney: Transport for NSW Retrieved from: 〈http://roadsafety.transport. nsw.gov.au/downloads/aboriginal-road-safety-plan.pdf〉 (Verified 3 March 2016). Watson, B.C., 2004. The crash risk of disqualified/suspended and other unlicensed drivers. Paper presented at the 17th International Conference on Alcohol, Drugs and Traffic Safety, Glasgow, UK. Watson, B.C., Armstrong, K.A., Watson, A., Wilson, A., Livingstone, K., 2011. A roadside survey of unlicensed driving in Queensland. Paper presented at the Australasian Road Safety Research, Policing and Education Conference, Perth, WA. Williamson, G., Thompson, K., Tedmanson, D., 2011. Supporting Aboriginal people to obtain and retain driver licences: an informed review of the literature and relevant initiatives. Centre for Sleep Research and Human Factors Group, School of Psychology, Social Work and Social Policy, University of South Australia, Adelaide. World Health Organisation, 2013. Global Status Report on Road Safety: Supporting a Decade of Action. WHO, Geneva.
Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
Contents lists available at ScienceDirect
Journal of Transport & Health journal homepage: www.elsevier.com/locate/jth
‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities Patricia Cullen a,b,n, Anna Chevalier a,b, Kate Hunter a,b,c, Tom Gadsden b, Rebecca Ivers a,b a b c
The George Institute for Global Health, PO Box M201, Missenden Road, NSW 2050, Australia School of Public Health, Sydney Medical School, The University of Sydney, Australia The Poche Centre for Indigenous Health, the University of Sydney, Australia
art ic l e i nf o
a b s t r a c t
Article history: Received 29 March 2016 Received in revised form 6 July 2016 Accepted 10 July 2016
Introduction: The low level of licenced drivers in remote Aboriginal communities in the Northern Territory (NT) Australia constitutes a risk factor for transport injury, high incarceration rates, and reduced ability to access employment and health services. The DriveSafe NT Remote program was implemented by the NT Government to increase driver licensing in remote communities. This evaluation reviews the program delivery, acceptability, implementation challenges and licensing outcomes. Methods: A mixed-methods approach was used, incorporating program observation and key informant perspectives (n ¼30). Program data (collected April 2012 to June 2014) and de-identified licensing data from the NT Motor Vehicle Registry were analysed for trends in service delivery and licensing rates pre and post-program. Results: Interviewees reported strong support for the program, and regarded the program as highly engaging and acceptable. There was a greater increase in new licences at intervention sites (Learner 24% and Open licence 18%) compared with other remote areas (Learner licence 13% and Open licence 8%). There appeared to be a dose response relationship with greater licence outcomes at communities that received higher levels of program delivery. Discussion: DriveSafe NT Remote is a Government delivered program providing licensing services to Aboriginal clients in remote Northern Territory communities, and is increasing driver licensing rates. The flexible delivery and culturally responsive approach should lead to further positive licensing outcomes. Addressing barriers to licence participation faced by Aboriginal people is an innovative approach to reducing transport disadvantage and positively impacting health and well-being in remote communities. This evaluation demonstrates a pragmatic approach to assessing program implementation of a multi-site community-based program in a vulnerable and underserviced population; it is anticipated this approach could be applied to other settings to ensure programs responsively address road safety in Aboriginal populations. & 2016 Elsevier Ltd. All rights reserved.
Keywords: Aboriginal Driver licensing Community Evaluation Implementation Culturally responsive
1. Introduction Transport is essential to maintain employment, attend education, socialise and access healthcare. Transport disadvantage results from persistent difficulties accessing public transport, inability to maintain private transport and difficulties meeting transport costs, with far reaching impacts on individual and community health and well-being (Currie and Senbergs, 2007; Rosier and McDonald, 2011). Reduced licence participation in Aboriginal and Torres Strait Islander communities contributes to transport disadvantage, higher rates of transportrelated morbidity and mortality and is a barrier to economic participation, social inclusion, health and well-being (Elliot and Shananhan Research, 2008; Helps et al., 2008; Ivers et al., 2016, 2011). The Northern Territory (NT) has the highest rate of road transport-related injury of any jurisdiction in Australia, and the Aboriginal population of the NT has a fatality rate due to road transportation more than double the rest of the NT population (Australian Institute of Health and Welfare, 2012, p. 24). This has been attributed to known risk factors for transport-related injury in Aboriginal communities: remoteness, non-use of seatbelts, alcohol use and vehicle overcrowding (Clapham n
Corresponding author. E-mail address: [email protected] (P. Cullen).
http://dx.doi.org/10.1016/j.jth.2016.07.004 2214-1405/& 2016 Elsevier Ltd. All rights reserved.
Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
2
et al., 2008; Henley and Harrison, 2013). Unlicenced driving is also common, due to the vastness of the region and poor access to licensing services. As a result of not having a licence while living in a remote area with little public transport, people are more inclined to make unsafe driving choices or engage in illegal driving practices (Job and Bin-Sallik, 2013). Low rates of licensing in communities also contributes to an over-represented Aboriginal prison population due to a high number of repeat licence offences and difficulty accessing employment and developmental opportunities (Job and Bin-Sallik, 2013; Somssich, 2009). Across Australia, the NT recorded the highest imprisonment rate at 885 prisoners per 100,000 adult population in 2015 (Australian Bureau of Statistics, 2015). Aboriginal and Torres Strait Islander prisoners made up 84% of the prison population, despite comprising only 32% of the NT population (Australian Bureau of Statistics, 2015). The rate of incarceration for vehicle regulatory and driving offences is 25% of NT prisoners, of which approximately 97% are Aboriginal (Anthony and Blagg, 2012). Several reports have highlighted the complex range of barriers that contribute to the high rate of unlicenced drivers in remote Aboriginal communities: a lack of appropriate identification documentation (birth certificates), limited access to licensing and testing facilities, low levels of literacy, lack of self-esteem, high cost of driving lessons, poor access to eligible supervisory drivers for learners, language barriers, reduced availability of licensing services, a lack of access to appropriate vehicles, a general culture of being unlicenced and unsafe driving practices (Elliot and Shananhan Research, 2008; Helps et al., 2008; Job and Bin-Sallik, 2013). The DriveSafe NT Remote program was implemented by the NT Government to address barriers to licensing faced by remote communities. Similar programs have been developed and implemented previously, yet many have proved unsustainable due to inadequate funding and an inability to demonstrate outcomes. Hence, licensing support programs are under-evaluated and there is limited evidence to establish best practice for program development and implementation. This process evaluation aims to review program implementation and explore whether DriveSafe is being implemented as intended, addressing the needs of the target communities and leading to positive licensing outcomes.
2. Methods 2.1. Design A mixed-methods approach was selected to explore informant perspectives, program delivery and licensing outcomes using deidentified licensing data for remote communities in the NT from the Motor Vehicle Registry, Northern Territory Department of Transport.
2.2. The program The DriveSafe NT Remote program commenced in April 2012, and was sequentially rolled out across 23 communities. Prior to the delivery of the program, the DriveSafe team visited a community three times on average to conduct community consultations, promote the program and identify a host organisation. Then the program team and host organisation ensured community members have proof of identification to apply for a Learner licence. Once community members have identity documentation, DriveSafe field staff deliver the Learner theory component of the program and administer the Learner's licence theory test. The licensing process in the NT is outlined in Fig. 1. Field staff return at regular intervals to deliver professional driving lessons. In between professional lessons, participants engaged in supervised driving practice where available with local community members who were licenced drivers. In the final stage of the program, field staff deliver road safety education, and administer the Provisional driver licence practical test. A graduation ceremony is held to present the licences.
2.3. Quantitative data sources and collection 2.3.1. Program data Program data was routinely collected throughout delivery from April 2012 to May 2014, and was supplied to evaluators in a deidentified form. Data consisted of licensing outcomes, birth certificates obtained and professional lessons delivered. Data was reported for 15 locations: seven primary locations were excluded on the basis of incomplete data and/or where program delivery commenced in May 2014, and data for Jabiru and Gunbalanya was combined. The 15 locations are shown on a map of the NT along with Darwin and Alice Springs (Fig. 2). Licence Class
Minimum Age
Eligibility for licence
Period of licensure
Learner licence
16 years
Pass a computer based theory test
Provisional licence
16 years and 6 months
Pass a practical on road driving test
Hold the Learner licence for 6 months Hold the Provisional licence for 24 months if under 25 years of age Hold the Provisional licence for 12 months if over 25 years of age
Full licence
18 years and 6 months
No testing or requirements, licence is upgraded upon renewal once provisional licence period has passed
Fig. 1. Licensing requirements, eligibility and time periods in the Northern Territory, Australia.
Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
3
Fig. 2. Map of Darwin, Alice Springs and DriveSafe Remote NT communities.
Table 1 Program delivery and outcomes at DriveSafe NT Remote regions from April 2012 to May 2014. Region
Groote Eylandt Tiwi Galiwinku Maningrida Wadeye Gapuwiyak Jabiru, Gunbalanyab Roper Gulf/Ngukurr Nhulunbuy Barkley/Tennant Creek Katherine Mandorah Batchelor Milingimbi/Ramingining Vicdaly/Lajamanu Total
a b
Program delivery
Program outcomes
Birth certificatesa
Driving lessons
Learner licences
Provisional licences
Total licences
53 43 98 63 17 60 47 28 26 9 2 0 0 21 1 468
203 255 523 54 60 528 52 33 127 51 0 80 4 0 0 1970
191 170 198 176 148 121 110 111 117 81 22 25 10 5 0 1586
97 82 48 43 41 57 33 30 16 8 10 0 3 0 0 486
288 252 246 219 189 178 143 141 133 89 32 25 13 5 0 1953
Not all program participants required birth certificates as they had attained proof of identification documents prior to the program delivery. Data for Jabiru and Gunbalanya are combined.
2.3.2. Licensing data Licensing data for remote communities in the NT were obtained from the Motor Vehicle Registry, NT Department of Transport. The data included licences held within remote regions at a single point in time each year from 1 December 2010 to 1 May 2014, by licence type, postcode and suburb. There were two licence types: Learner and Open licences. A person with a Learner licence held no other class or type of licence. A person with an Open licence may hold a Provisional or full-unrestricted licence. 2.4. Qualitative data sources and collection This aspect of the evaluation involved reviewing program materials, observing delivery and conducting in-depth semi-structured interviews with program staff, program participants and community stakeholders. Purposive sampling identified key informants to Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
4
participate in interviews (Patton, 1990). Snowball sampling was employed, with all interviewees asked to recommend other interviewees (Liamputtong and Ezzy, 2005). The number of interviews was determined by data saturation.
2.4.1. Interviews with program staff To enable feedback on both program structure and delivery, interviews were conducted with DriveSafe and host organisation program staff (n ¼8) in November 2012 and April 2014. 2.4.2. Interviews with clients Interviews were conducted with clients (n ¼10) in November 2012 in Wurrumiyanga community (Tiwi islands). Of the 10 interviewees, half were female and 60% aged 18–34 years old. Participants were asked about their driving and licensing experiences before and after the program, experiences with the program, as well as course content and teaching style.
2.4.3. Interviews with stakeholders Face-to-face or telephone interviews about delivery and impact of the program were conducted with community stakeholders (n ¼12) between November 2012 and April 2014. This included representatives of organisations delivering services to the community.
2.5. Analysis 2.5.1. Quantitative data Program data was descriptively analysed by community to determine number of delivery days, professional driving lessons, birth certificates and licences obtained. Due to the post-program evaluation design and small sample size, significance testing was not undertaken. Trends were examined of licences held in remote areas pre and post commencement of the program by licence type. Comparisons were made between locations that received the program and similar locations that did not receive the program.
2.5.2. Qualitative data Interviews were audio recorded and professionally transcribed; the data was managed using Nvivo 10 software (QSR International Pty Ltd, 2012). A general inductive approach to the analysis was employed, with themes emerging from initial categorisation of the data (Thomas, 2006). The research team sought feedback from a panel of expert advisors that included Aboriginal and non-Aboriginal researchers with expertise in community health, road safety, driver licensing and injury prevention.
2.6. Ethics The evaluation was approved by Human Research Ethics Committees from the NT Department of Health and Menzies School of Health Research. Program data was provided to the research team in de-identified form. All participants provided written consent. 600
500
400 Delivery days
Lessons
300
200
100
0
Community Fig. 3. DriveSafe NT Remote program delivery days and lessons at each region from April 2012 to May 2014.
Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
5
350
Licence outcomes
300 250 200 150 100 50 0
0
20
40
60
80
100
120
140
160
Program delivery days Fig. 4. Positive association between program delivery days in communities and total licence (Learner and Provisional) outcomes achieved. Table 2 Change in licences held for DriveSafe NT Remote compared to other remote locations, by licence type from 1st Dec 2010 to 1st May 2014. Location
Pre
% Changea
Post
2010
2011
Average
2012
2013
2014
Average
Learner
DriveSafe Other Remote Sub-total
97 108 205
100 113 213
99 111 209
115 118 233
130 123 253
122 134 256
122 125 247
24% 13% 18%
Open
DriveSafe Other Remote Sub-total
312 548 860
317 579 896
315 564 878
343 582 925
378 623 1001
390 623 1013
370 609 980
18% 8% 12%
a
% Change pre to post ¼ difference pre- to post-average/pre-average as %.
3. Results 3.1. Program delivery Between April 2012 and May 2014, 832 program delivery days and 1970 driving lessons were provided by the program. The number of lessons delivered in each primary location ranged from a high of 528 at Gapuwiyak to a low of four at Batchelor, and none at three other communities (Table 1). Licensing outcomes also varied widely between locations, with a tendency toward increased outcomes at locations with higher levels of program delivery (Table 1). Comparison between communities found Galiwinku, Gapuwiyak, Tiwi and Groote Eylandt were the four regions with greatest levels of combined program delivery days and lessons (Fig. 3). This corresponded with the four locations that attained the highest number of Provisional licences (Groote Eylandt, Tiwi, Gapuwiyak and Galiwinku) (Table 1). A positive association was found between higher levels of program delivery days (dose) and total licence outcomes (response) (Fig. 4). 3.2. NT licensing data On average, during the 3.5 years from December 2011 to May 2014, 80% of people in remote regions holding a driver licence held an Open licence and 20% held a Learner licence. The average number of Learner licences held was similar in areas where the program was delivered (49%), compared to other remote areas (51%). However, trends over time found Learner licences increased more on average where the program was delivered (24%) compared to other remote areas (13%) (Table 2). On average, over the 3.5 year period, the number of Open licences held was lower where the program had been delivered (37%) compared to other remote areas (63%). Despite this, trends over time also found Open licences increased more where the program was delivered (18%) compared to other remote areas (8%) (Table 2). 3.3. Informant perspectives Using a general inductive approach, the initial categories were refined. Five main themes emerged aligning with the evaluation aims: 1) Meeting the need; 2) Collaboration and respect; 3) Breaking down the barriers; 4) ‘I think it was the people involved’; 5) Sustainability and overcoming challenges. 3.3.1. Meeting the need: ‘It’s exactly what we needed’ In exploring the need for the program, it was evident there was a substantial need not just for licensing access but for service delivery appropriate for culturally-diverse and under-resourced remote communities. Multiple communities approached the team to be included in the program, and at each site there were multiple requests for further programs. This was consistent with community stakeholder and program participant interviews that overwhelmingly identified a need for licensing service delivery in remote NT communities. Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
6
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
“I just hope it goes on and on and on, because there is just such a definite need” Interviewees emphasised licensing support will help address road safety issues in remote communities. Unlicenced driving was consistently highlighted as the main issue in communities, particularly involving young and under-age drivers. Additionally, interviewees frequently referred to the importance of a driver licence to securing employment and accessing opportunities. “Because we don’t have an MVR office here, there’s a lot of people, particularly young people, who don’t have a licence… and it’s causing problems, they will still drive cars. and it makes it hard for younger and older people to get work as well” “It was just perfect…. Getting that program, helping those young people get their licence – it’s immeasurable” Interviewees emphasised the value of the program in reducing offending and incarceration. “Now those kids can drive a car without the worry of maybe driving without a licence and ending up coming to the police and in gaol over such a simple thing as not having a licence” “It might mean that a lady who might have kids, a young lady might not go to gaol because her husband makes her drive but she hasn’t got a licence or something, so then those kids get to keep their Mum, she doesn’t go to gaol” 3.3.2. Collaboration and respect It emerged that by prioritising collaboration and respect, the DriveSafe program was highly responsive to cultural norms and unanticipated challenges. Interviewees frequently cited the positive attitude of DriveSafe staff and their ability to overcome challenges and problem-solve with communities. When the program team arrived in a community, word-of-mouth quickly spread such that the demand to participate was often greater than anticipated. In communities where demand for the program exceeded expectations, program staff allocated extra time to ensure they could service as many participants as possible. Interviewees reported DriveSafe was very successful in collaboratively addressing the specific needs of each community. For example, one community requested assistance for 15 teachers to get their bus licence; this benefited the whole community as students could be taken to educational and sporting events. “Well, it’s an opportunity for the kids so they don’t miss out. It just means more opportunity, more bus licences, you can get kids to and from things and the teachers feel they’re important” The program considered and respected the cultural norms of each community by spending time engaging with the host organisation, assessing the needs and adjusting the program delivery accordingly. It was common in the remote regions for community members to have had little exposure to formal schooling; the programs' responses to this included: a) tailoring the program to participants' literacy needs; b) using oral testing for the Learner theory; c) presenting program materials in an engaging and interactive format; d) providing regular breaks to enhance participant concentration. In some communities it was preferable to hold Learner theory classes with male and female community members separately. Similarly, the program aimed to gender match driving instructors with Learner drivers; where this was not feasible Learner drivers were able to bring a support person to the lesson. The impact of this responsiveness was acknowledged by interviewees. “There would be kids there who have never – and especially female, who I reckon never would have got a licence in their life without this program” 3.3.3. Breaking down the barriers Attaining a licence was seen to be a significant achievement, and DriveSafe was perceived as reducing the barriers to licensing in remote communities. Several elements of the program delivery model emerged as essential to reducing the barriers to licensing. Commonly cited was the program was free and based in the community. These were deemed to be essential components of the program, as participation in a program further away would have been inaccessible for remote community members who do not have access to transport and/or have a reduced income. Further, being able to participate in a familiar environment and close proximity to family and social networks is inherently important to Aboriginal people, who have strong kinship and cultural connections to their community. “Kids see it as something important….that was something those kids genuinely wanted to do…. and just didn’t have the ability or opportunity to make it happen” The DriveSafe team's ability to organise birth certificates was also viewed as fundamental to accessing a licence. Proof of identity can be a considerable barrier for remote community members, as without private transport it can be difficult to access relevant Government departments. Additionally, there is an increased likelihood in remote communities a person's birth may not have been registered and there are often issues with identity documents with different names or misspellings. “So we do the ID. We get a birth certificate and we actually identify who they are. We tidy up the database and go, ‘Right, you used to be known by six names; you are now known by one. We’ve taken a photo; we’ve merged everything. Our database now is perfect and you’re ready to go’” 3.3.4. ‘I think it was the people involved’ DriveSafe relies heavily on the commitment and expertise of the field staff that travel between communities; accordingly the skills and performance of field staff were frequently discussed by interviewees. Program management interviewees indicated field staff are critical to the success of the program and cited the importance of having staff skilled in delivering road safety messages and licensing support, as well as engaging community with cultural respect. Field staff were able to work autonomously and were skilled at adapting the program to the local context, working within the capacity and resources of each community. Program management acknowledged that while the majority of staff were not Aboriginal, they had extensive experience working with remote Aboriginal communities. Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
7
“It’s a great program, it’s a great idea but if you have the wrong people running it” Field staff are handpicked from experienced motor vehicle registry licenced testing officers, and some were also licenced driving instructors. Further, community confidence and engagement was enhanced by having consistent trainers and instructors over the program duration. “As long as you keep the right people offering the program, that’s the biggest drama up here is the turnover of people” Program participants reported the driving instructors were skilled, appropriate, patient and funny. Participants reported learning about technical skills associated with driving a car, as well as road safety issues, such as speeding, drinking and restraint use. Driving practice or ‘jumping in the car’ was commonly highlighted as the best part of the program, made better because of the rapport with instructors. This was reinforced by stakeholder interviewees, who rated the quality of the program presentation highly, and particularly the rapport DriveSafe staff developed with participants, which created a supportive environment. Stakeholders also emphasised the program’s delivery of important road safety information (speeding, drink driving, distraction, mobile phone use, seatbelts) was enhanced by delivery that was engaging and meaningful to remote community members. 3.3.5. Sustainability and overcoming challenges There is considerable variation across communities in terms of capacity, resources and engagement with the program, which have presented ongoing implementation challenges; however staff also thought the ability to adapt to these differences was a key strength of the program. For instance, initially, the program relied heavily on professional driving instructors to deliver driving lessons in each community, costing up to $10,000 per week. This model was not deemed to be cost-effective nor sustainable in the long term. To improve cost-effectiveness, the program has focused on developing local capacity to provide supervised driving lessons, which is proving critical to program sustainability. Limited availability of resources has also been a challenge, particularly in the case of acquiring a vehicle to be kept in communities for supervised driving practice. Staff reported a number of communities did not have the capacity to attain or maintain a vehicle, which impacted their ability to deliver driving lessons and achieve licensing outcomes. To address these challenges, DriveSafe adapted the program model to gain access to funding from the Remote Jobs and Communities Program (RJCP) providers (Department of Employment, 2014). RJCP is an Australian Government funded program which supports training and other activities for job readiness and building community capacity. Management staff cited the development of this relationship between DriveSafe and RJCP's in participating communities as integral to building community capacity to deliver supervised driving practice and improving program sustainability. Project staff highlighted the need to revise Government regulations and policy to enable Government vehicles could be utilised for Learner drivers. This was seen as a substantial barrier to delivering supervised driving practice and was required to be dealt with on a case by case basis. Furthermore, use of these vehicles would facilitate better employment pathways as Learners could potentially be employed and gain a driver licence as part of a work plan.
4. Discussion The DriveSafe program is achieving licensing outcomes in underserviced remote NT regions. The program has demonstrated capacity for a flexible approach that is responsive to remote communities. There was variation in program delivery across regions, which predominately reflects flexible service provision adapted to meet community needs. Licensing trends show Learner and Open licences have increased more on average where the program has been delivered compared to other remote areas. There was a positive association between communities that received a high dose of the program and increased licensing outcomes. This relationship was reinforced by staff reports that program delivery is strengthened in communities that actively engage with the program. It is possible this relationship is underscored by need for the program, whereby communities with a high need receive an increased dose and are more highly engaged, which leads to greater licensing outcomes. There is a recognised need to address high levels of road transport injury in vulnerable populations (World Health Organisation, 2013). In Australia, remote communities with a high proportion of Aboriginal residents represent an underserviced and vulnerable population with low rates of licence participation, high rates of incarceration and high levels of road transport injury (Edmonston et al., 2003; Elliot and Shananhan Research, 2008; Helps et al., 2008; Ivers et al., 2011; Naylor, 2010; NSW Auditor General, 2013; Somssich, 2009). The DriveSafe program is addressing areas of greatest need, as the average number of Open licences held was higher in other remote areas compared to where the program has been delivered. This finding is consistent with interviewee reports that the program is targeting communities and clients with a high level of need. The implementation of DriveSafe has demonstrated the program's capacity for flexible delivery and community engagement. While DriveSafe is regarded as highly acceptable to remote communities and exhibits many indicators of long-term viability, increased Aboriginal involvement and leadership is recommended to promote inclusive partnerships with Aboriginal communities and build local capacity (Martiniuk et al., 2010; NSW Auditor General, 2013). The NSW Auditor General (2013) has recommended that programs prioritise the involvement of Aboriginal people in development, implementation and delivery; this enhances the cultural appropriateness of materials, teaching methods, aids in capacity building and ultimately promotes program longevity. Thereby, the program's sustainability and acceptability could be enhanced with an increased focus on employing Aboriginal people at a community and program management level. While the process for attaining a driver's licence varies considerably between jurisdictions, there is a recognised need to facilitate fair and equitable access to the licensing system (Elliot and Shananhan Research, 2008; Helps et al., 2008; Job and Bin-Sallik, 2013; Naylor, 2010; NSW Auditor General, 2013; Somssich, 2009; Williamson et al., 2011). Several Australian states have made progress toward improving licensing access for underserviced populations. The Victorian Government has funded the implementation of the L2P Learner Driver Mentor Program to assist young people (aged 16–20 years) who are experiencing disadvantage accessing a licence. As of December 2013, the program is running in 62 out of Victoria's 78 local government areas, and has capacity to assist 2000 young people at a time. The Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
8
L2P program is free for participants, however it does not assist with licence or test fees and participants must already hold their Learner licence to be eligible for enrolment (Freethy, 2012). In New South Wales, the Driving Change program began in February 2013 and has been implemented in twelve communities; it is hosted by a community organisation at each site and employs a local Aboriginal coordinator to provide intensive case-management for participants through the licensing system (Ivers et al., 2012). Driving Change provides assistance with all aspects of licensing including proof of identity, financial assistance for test/licence fees, Learner Driver Knowledge Test preparation, supervised driving hours and assistance to manage state debt and/or fine (Cullen et al., 2016). Similar to DriveSafe Remote, Driving Change is strengthened by community engagement and relies on the recruitment of community volunteers to deliver supervised driving practice. Reduced licence participation is Aboriginal communities has been cited as both a cause and consequence socioeconomic disadvantage and a contributor to health disparity (Ivers et al., 2016). Ivers et al. (2016) reported that having a driver licence was significantly associated with higher odds of full-time employment and post-secondary education; those who held a driver licence (provisional or full) were four times more likely to be in full-time employment and two to four times more likely to have higher levels of formal education. The authors concluded that reduced driver licensing participation has far-reaching impacts on the health and wellbeing of Aboriginal people. Transport disadvantage due to reduced licence participation is particularly problematic in remote communities as there are limited options for transport without access to a private car and licenced driver. Further, transport disadvantage contributes to unsafe transport choices including over-crowding of vehicles and unlicenced driving, which is a risk-factor for transport injury and incarceration (Job and BinSallik, 2013; Somssich, 2009; Transport for NSW, 2014; Watson, 2004; Watson et al., 2011). This evaluation has demonstrated that DriveSafe is positively impacting licensing in remote communities, which beyond improving the mobility of community members will reduce the risk of transport injury, incarceration and promote community health through improved employability, access to services and opportunities. The impact of licence participation and access in other Indigenous populations globally (e.g. Native American) has not been reported. While there is evidence that Indigenous populations in the United States are over-represented in transport fatality, little is known about the role that driver training and licensing may play as a protective factor (Pollack et al., 2012). Further, evidence suggests that Maori populations in New Zealand experience significant transport disadvantage, which has been described as ethnically mediated transport disadvantage (Raerino et al., 2013). Recommendations for addressing this issue in Indigenous populations include recognising the importance of Indigenous systems of wellbeing in the design of transport policy and determining the enablers of safe transport in communities (Raerino et al., 2013). To this end, the current evaluation has provided insight into the enablers of safe and legal driving in remote Aboriginal communities in Australia; it is anticipated that program evaluation employing this pragmatic approach could determine barriers and facilitators to safe transport in other Indigenous contexts. This process evaluation has several strengths and weaknesses. Importantly, the evaluation sought to obtain evidence of program impact and acceptability from multiple data sources. This provided rich evidence of the multiple methods the program is using to impact licensing, road safety and mobility in remote communities. Further, emphasis was placed upon ensuring community members were included in the evaluation, in particular to explore the acceptability and impact of the program. A limitation of the current study was the small sample of communities included in the qualitative interviews; the present results could be extended to a greater sample of communities who had accessed the program in particular to explore the generalisability of the program to other contexts. Due to the postprogram design, there was no control group and participants were not randomised. The program and licensing data was aggregated at the community level and individual data was not available, hence it was not possible to track the progress of individual participants. Nevertheless, process evaluations of multi-site community programs provide a valuable and pragmatic approach to ensuring the intervention is being implemented as intended to the target population. Overall, this evaluation indicates that a community-based approach is sufficiently flexible and highlights the importance of strong community support to facilitate equitable access to licensing. It is likely that other vulnerable populations, which are not limited to Aboriginal or remote communities, but also include culturally and linguistically diverse populations (e.g. refugees) could benefit from an inclusive and pragmatic approach to overcome barriers to licensing.
5. Conclusion Remote communities in the NT represent an underserviced population for driver licensing services, which contributes to low rates of licenced drivers, high incarceration rates and impedes access to essential health services and employment. DriveSafe NT Remote is addressing areas of high need (underserviced remote communities) and is regarded highly. The program is positively impacting licensing rates in remote communities, which reduces the risk of transport injury, and contributes to improved mobility, employability and access to essential services. Despite early challenges to implementation and resourcing, the program’s resilience is evidenced by the delivery of licensing outcomes in remote NT communities. Further, a dose response relationship was evident, with licence outcomes greatest at communities receiving a high dose of program delivery. In terms of delivery, the program’s greatest strength is the consistently high level of cultural responsiveness and adaptability to changing conditions, which should facilitate ongoing success. While driver licensing legislation varies across jurisdictions, it seems likely that this model could be successfully applied elsewhere.
Funding Contributing authors were contracted by the Northern Territory (NT) Government to undertake an independent evaluation of the DriveSafe NT Remote program. The DriveSafe NT Remote program was initiated by the NT Government, funded by the former National Road Safety Council (with funding administered by the Department of Infrastructure and Transport, Australian Government), Territory Insurance Office (TIO), the Australian Government Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA) and the NT Government’s Department of Transport. Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i
P. Cullen et al. / Journal of Transport & Health ∎ (∎∎∎∎) ∎∎∎–∎∎∎
9
Acknowledgements The authors acknowledge the contribution of DriveSafe NT Remote staff and communities.
References Anthony, T., Blagg, H., 2012. Addressing the “crime problem’’ of the Northern Territory Intervention: Alternate paths to regulating minor driving offences in remote Indigenous communities (CRG 38/09-10). Criminology Research Advisory Council. Australian Bureau of Statistics, 2015. 4517.0 - Prisoners in Australia. (4517.0). Canberra. Retrieved from: 〈http://www.abs.gov.au/AUSSTATS/[email protected]/Lookup/4517.0Main þFeatures12015?OpenDocument〉. Australian Institute of Health and Welfare, 2012. Trends in serious injury due to land transport accidents, Australia 2000-01 to 2008-09. Injury research and statistics series no. 66 Cat. no. INJCAT 142. Canberra. Cullen, P., Clapham, K., Byrne, J., Hunter, K., Rogers, K., Senserrick, T., Ivers, R., 2016. Implementation of a driver licensing support program in three Aboriginal communities: a brief report from a pilot program. Health Promot. J. Aust. (Accepted) Currie, G., Senbergs, Z., 2007. Indigenous communities: transport disadvantage and Aboriginal communities. In: Currie, G., Stanley, J., Stanely, J. (Eds.), No Way To Go: Transport and Social Disadvantage in Australian Communities. Monash University ePress, Clayton, Victoria. Department of Employment, 2014. Remote Jobs and Communities Program. From: 〈https://employment.gov.au/remote-jobs-and-communities-program〉. Edmonston, C.J., Rumble, N., Powell, J., Butler, S., Nona, H., Watson, B.C., Schonfeld, C.C., 2003. Working with Indigenous communities to improve driver licensing protocols and offender management. Paper presented at the 2003 Road Safety Research: Policing and Education Conference, Sydney, NSW. Elliot and Shananhan Research, 2008. An Investigation of Aboriginal Driver Licensing Issues – Aboriginal Licensing Final Report. NSW Roads and Traffic Authority, Sydney. Freethy, C., 2012. L2P – learner driver mentor program: extending driver licensing reach in disadvantaged communities. Paper presented at the Australasian College of Road Safety Conference, Sydney, NSW. Helps, Y.L.M., Moller, J., Kowanko, I., Harrison, J.E., O’Donnell, K., de Crespigny, C., 2008. Aboriginal People Travelling Well: Issues of safety, transport and health. Department of Infrastructure, Transport, Regional Development and Local Government, Canberra, ACT. Ivers, R., Byrne, J., Hunter, K., Clapham, K., 2012. Development of a community based Aboriginal driver licensing service: the AstraZeneca Young Health Programme. Paper presented at the Australasian College of Road Safety Conference, Sydney, NSW. Ivers, R., Hunter, K., Clapham, K., Helps, Y., Senserrick, T., Byrne, J., Harrison, J.E., 2016. Driver licensing: descriptive epidemiology of a social determinant of Aboriginal and Torres Strait Islander health. Aust. NZ J. Public Health . http://dx.doi.org/10.1111/1753-6405.12535. Ivers, R., Lyford, M., Johnson, J., Boufous, S., Senserrick, T., Clapham, K., 2011. Road safety and driver licensing in Aboriginal people in remote NSW. Paper presented at the Coalition for Research to Improve Aboriginal Health, Sydney, NSW. Job, R.F.S., Bin-Sallik, M.A., 2013. Indigenous road safety in Australia and the “Drivesafe NT Remote” project. J. Australas. Coll. Road Saf. 24 (2), 21–27. Liamputtong, P., Ezzy, D., 2005. Qualitative Research Methods. Oxford University Press, Mebourne. Martiniuk, A., Ivers, R., Senserrick, T., Boufous, S., Clapham, K., 2010. Effective and inclusive intervention research with Aboriginal populations: an Evidence Check rapid review brokered by the Sax Institute for NSW Health. Naylor, B., 2010. L-plates, logbooks and losing-out: regulating for safety – or creating new criminals? Altern. Law J. 35 (2), 94–98. NSW Auditor General, 2013. New South Wales Auditor-General's Report to Parliament: Improving Legal and Safe Driving Among Aboriginal People. Audit Office of New South Wales, Sydney, NSW. Patton, M., 1990. Qualitative Evaluation and Research Methods, 2nd ed. Sage, Newbury Park. Pollack, K.M., Frattaroli, S., Young, J.L., Dana-Sacco, G., Gielen, A.C., 2012. Motor vehicle deaths among American Indian and Alaska native populations. Epidemiol. Rev. 34 (1), 73–88. http://dx.doi.org/10.1093/epirev/mxr019. QSR International Pty Ltd., 2012. NVivo qualitative data analysis software (Version 10). Raerino, K., Macmillan, A.K., Jones, R.G., 2013. Indigenous Māori perspectives on urban transport patterns linked to health and wellbeing. Health Place 23, 54–62. http://dx. doi.org/10.1016/j.healthplace.2013.04.007. Rosier, K., McDonald, M., 2011. The relationship between transport and disadvantage in Australia. Australian Institute of Family Studies, Melbourne. Somssich, E., 2009. Driver training and licensing issues for indigenous people. J. Australas. Coll. Road. Saf. 20 (1), 31–36. Thomas, D.R., 2006. A general inductive approach for analyzing qualitative evaluation data. Am. J. Eval. 27 (2), 237–246. http://dx.doi.org/10.1177/1098214005283748. Transport for NSW, 2014. NSW Aboriginal Road Safety Action Plan 2014-2017. (978-1-922030-43-6). Sydney: Transport for NSW Retrieved from: 〈http://roadsafety.transport. nsw.gov.au/downloads/aboriginal-road-safety-plan.pdf〉 (Verified 3 March 2016). Watson, B.C., 2004. The crash risk of disqualified/suspended and other unlicensed drivers. Paper presented at the 17th International Conference on Alcohol, Drugs and Traffic Safety, Glasgow, UK. Watson, B.C., Armstrong, K.A., Watson, A., Wilson, A., Livingstone, K., 2011. A roadside survey of unlicensed driving in Queensland. Paper presented at the Australasian Road Safety Research, Policing and Education Conference, Perth, WA. Williamson, G., Thompson, K., Tedmanson, D., 2011. Supporting Aboriginal people to obtain and retain driver licences: an informed review of the literature and relevant initiatives. Centre for Sleep Research and Human Factors Group, School of Psychology, Social Work and Social Policy, University of South Australia, Adelaide. World Health Organisation, 2013. Global Status Report on Road Safety: Supporting a Decade of Action. WHO, Geneva.
Please cite this article as: Cullen, P., et al., ‘The program was the solution to the problem’: Process evaluation of a multi-site driver licensing program in remote communities. Journal of Transport & Health (2016), http://dx.doi.org/10.1016/j.jth.2016.07.004i