Effect of railway safety education on the safety knowledge and behaviour intention of schoolchildren

Evaluation and Program Planning 55 (2016) 9–16

Contents lists available at ScienceDirect

Evaluation and Program Planning journal homepage: www.elsevier.com/locate/evalprogplan

Effect of railway safety education on the safety knowledge and behaviour intention of schoolchildren Anne Silla *, Veli-Pekka Kallberg 1 VTT Technical Research Centre of Finland Ltd, PO Box 1000, Vuorimiehentie 3, Espoo 02044 VTT, Finland

A R T I C L E I N F O

A B S T R A C T

Article history: Received 28 May 2015 Received in revised form 13 October 2015 Accepted 25 November 2015 Available online 27 November 2015

This study was designed to evaluate whether railway safety lessons are effective in increasing schoolchildren’s safety knowledge and behaviour intention. The railway safety education in schools included a 45-min lesson on safe behaviour in a railway environment directed at 8–11 year old schoolchildren. The lessons were held in four schools located near railway lines in Finland. The effectiveness of this measure was evaluated based on a short survey directed at pupils before the lesson (base level) and around 2–3 months later (post-lesson) based on three variables which are considered as strong determinants of actual behaviour: behaviour intention, estimated dangerousness of the behaviour, and level of knowledge on the legality of the behaviour. The results show that the change in the share of correct answers was positive regarding all questions except for one question in which the difference was not significant. Based on this we can reasonably assume that railway safety education in schools can have a positive effect for all the measured variables, although the effect is not necessarily large. The results indicate that these positive changes can have a positive effect on the frequency of trespassing (i.e. fewer unsafe crossings in the future). We can further assume that reduction in the frequency of trespassing would reduce the frequency of trespassing accidents. ß 2015 Elsevier Ltd. All rights reserved.

Keywords: School education Railway trespassing Surveys Behaviour intention Danger estimation Illegal behaviour

1. Introduction Railway suicides and trespassing accidents resulted in more than 3,600 fatalities in 2011 on the European railway network, representing 88% of all fatalities occurring within the railway system (European Railway Agency, 2014). In order to tackle this problem a European RESTRAIL project was tasked with reducing the occurrence of suicides and trespassing (illegal presence in the railway area) on railway property and the costly service disruption these events cause (RESTRAIL, 2014a). As part of the project 11 pilot tests were carried out in different European countries aiming to provide better insight into the effectiveness of the measures targeted (i) in preventing suicides and suicide attempts, (ii) preventing trespassing accidents, and (iii) mitigating the consequences by speeding up the system recovery from such incidents. This paper presents the Finnish pilot test on education in schools for 8–11 year old children. The pilot aimed to prevent

* Corresponding author. Tel.: +358 407219014; fax: +358 207227000. E-mail addresses: anne.silla@vtt.fi (A. Silla), veepeek@saunalahti.fi (V.-P. Kallberg). 1 Tel.: +358 405317631; fax: +358 207227000. http://dx.doi.org/10.1016/j.evalprogplan.2015.11.006 0149-7189/ß 2015 Elsevier Ltd. All rights reserved.

trespassing. The education campaign was selected as a measure to be tested in Finland, since based on an earlier Finnish study a substantial number of interviewed trespassers and people living close to a railway line considered trespassing to be safe (Silla & Luoma, 2009, 2012). Furthermore, according to the same studies, many trespassers and people living close to a railway line assumed trespassing to be legal. People living close to a railway line also considered that education in schools concerning the dangers of walking on or across the railway tracks is an important measure to prevent railway trespassing (Silla & Luoma, 2012). Similarly to several other European countries, the fatalities related to railway trespassing represent a significant share of all fatalities occurring on Finnish railways. During 2006–2011 approximately eight persons were killed yearly due to railway trespassing (Finnish Transport Agency, 2010–2012; Finnish Rail Administration, 2007– 2009). The schoolchildren were selected as a target group for this pilot test, since their ability to estimate the risks related to trespassing is limited. Therefore, it is important to increase schoolchildren’s awareness of the rules related to railway crossing and to increase their understanding of the risks related to railway trespassing. Examples of topics addressed included e.g. that compared to road vehicles, trains are heavy, can not stop quickly, and frequently move fast. Long braking distances and high speeds mean that even

10

A. Silla, V.-P. Kallberg / Evaluation and Program Planning 55 (2016) 9–16

if an engine driver sees a pedestrian on the track, the distance is seldom long enough to stop the train in time. Even though the killed trespassers are typically adults (Silla & Luoma, 2012) the schoolchildren and young people are also trespassing (Silla & Luoma, 2012; Silla & Luoma, 2011) and some of them are committing vandalism in the railway areas. In the railway environment, pedestrians are always legally responsible for the safe crossing of the tracks. They are obliged to use railway-crossing sites (i.e. level crossings), which are specially marked for that purpose. Therefore, trespassing accidents are always interpreted to result from violations, risky behaviour or errors on the part of the pedestrian. However, although trespassing is illegal, clear and regularly used footpaths across railway lines are found in many places. The results of a survey directed to engine drivers in Finland reported 100 specific locations for railway trespassing in Finnish railway network (Silla & Luoma, 2009). Another Finnish study investigated a 4 km stretch of track and 12 locations with frequent trespassing were identified (Silla & Luoma, 2011). Based on these findings it is safe to assume that railway trespassing is frequent in Finland. There is a high variation in the motives for the trespassing behaviour. A number of studies have proposed that the main reason for trespassing is taking a short cut from A to B because the authorised route is assessed to be too far away (e.g. Lobb, Harre´, & Suddendorf, 2001; Rail Safety and Standards Boards, 2005; Silla & Luoma, 2009). Other reasons for trespassing are, for example, related to recreational purposes (taking a walk along the tracks), hanging around (playing, drinking alcohol, smoking, applying graffiti) or even to committing vandalism. Due to these numerous motives it is clear that the prevention of schoolchildren’s trespassing is more complex issue than to inform them about the dangerousness and illegality of this behaviour. However, the assumption is that after the lesson the schoolchildren will have a better understanding on safe behaviour in railway environment and the consequences of trespassing and therefore they have better capabilities to behave correctly and to resist the temptations resulted e.g. from peer or time pressure. The main aim of this study was to evaluate whether the railway safety lessons were effective in increasing schoolchildren’s safety knowledge and behaviour intention. The presumption is that schoolchildren who participate in the lesson become more aware of the dangers related to railway trespassing and railway lines in general, and are more likely in future to avoid playing, loitering and engaging in vandalism in the railway area and/or taking a shortcut across the tracks.

effectiveness of education campaigns in preventing railway trespassing. Savage (2006) investigated the effect of Operation Lifesaver activities on the number of collisions and fatalities at level crossings. He found that increasing the amount of educational activity will reduce the number of collisions, but the effect on the number of deaths cannot be concluded with statistical certainty. The analysis included only level crossing accidents and thus the effect of education on trespassing fatalities remained unknown. Two studies from New Zealand (Lobb et al., 2001; Lobb, Harre´, & Terry, 2003) have evaluated the effect of education programmes on preventing railway trespassing. The first study (Lobb et al., 2001) evaluated the effect of public education combined with access prevention by fences to reduce trespass at a suburban station in Auckland targeting people of all ages. The results showed that 3 months after the interventions the share of people trespassing (instead of using a nearby over bridge) was 36%, compared to 59% prior to the intervention. Moreover, the reduction was higher for adults than for children. In another study among secondary and high school students Lobb et al. (2003) concluded that punishment may be more effective than education in reducing unsafe behaviour (i.e. unsafe crossings) in the vicinity of railway stations, and substantially more effective than communication in raising awareness. In both of the studies the education programmes where combined with other measures and thus the effect of the pure educational campaign is not clear. Safety education programmes are also widely used to improve road safety. A literature review conducted by Dragutinovic and Twisk (2006) on the effectiveness of road safety education shows that the number of road safety education programmes that are followed by thorough evaluations is also rather limited. Moreover, they found that most evaluation studies use intermediate variables such as knowledge, attitudes and (self-reported) safe behaviour as evaluation criteria instead of crashes. One may argue that the reduction in the number of accidents or near accidents would be the best variable to describe the effects of an education campaign. However, it is rarely possible to conduct evaluation studies on education programmes based on accident data, because (i) crashes and injuries remain rare events in the population of road users, and (ii) in order to have sufficient statistical power to demonstrate an effect on crash-related outcomes, an education programme would require a large number of participants and monitoring of crash and injury records of the participants over a long period of time, which is not practical (Twisk, Vlakveld, Commandeur, Shope, & Kok, 2014).

2. Previous studies

3.1. Railway safety programme

Regardless of the large number of proposed countermeasures to prevent railway trespassing, there is little published research evaluating the effectiveness of any of these interventions (Lobb, 2006). Several railway safety education programmes exist worldwide to provide material to teachers and volunteers to spread information about the dangers related to railway trespassing and loitering in railway areas, and also to provide information on safe behaviour in railway environment directly to children. Relevant websites have been developed in the US (Operation Lifesaver, 2014), UK (Network Rail, 2014), New Zealand (TrackSAFE NZ, 2014) and Australia (TrackSAFE, 2014). According to the website of Operation Lifesaver (Operation Lifesaver, 2014) and those of national railway organisations (e.g. Prorail in the Netherlands, Network Rail in the UK, Kiwirail in New Zealand), several campaigns have been conducted to increase public knowledge, especially among young people, on the dangers and regulations related to railway trespassing and loitering in railway areas. However, there are few if any studies investigating the

Education in schools included a 45-min lesson on safe behaviour in a railway environment directed at 8–11 year old schoolchildren. The length of the lesson was 45 min since this is the conventional length of one lesson in Finnish schools. The main message of the lesson was that railway lines are only meant for trains. After the lesson the children should have understood (i) the main characteristics of railway traffic (railway lines are only meant for railway vehicles, trains cannot yield, trains cannot stop fast, trains always have priority etc.), (ii) that trespassing, playing and loitering in the railway areas are forbidden, and (iii) that they have the responsibility to behave safely in a railway environment. The objective of this measure was to increase the knowledge of schoolchildren on safe behaviour in a railway environment and thus to reduce vandalism, risky situations and possible accidents resulting from railway trespassing (playing, loitering, taking a short cut across the tracks etc.). The material used during the lessons was based on the lesson plans provided by the Finnish Transport Safety Agency on their

3. Method

A. Silla, V.-P. Kallberg / Evaluation and Program Planning 55 (2016) 9–16

11

website. The specific website for railway safety education includes several lesson plans for primary and secondary schoolteachers to use during the school year (Finnish Transport Safety Agency, 2014). It is not known how frequently the material is actually used. The website material is closely related to the material prepared by Operation Lifesaver. 3.2. Design The effect of the school education campaign was evaluated based on a short survey directed at pupils before the lesson (base level) and around 2–3 months later (post-lesson). The survey measured three variables: (i) level of knowledge related to railway trespassing, (ii) behaviour intention, and (iii) pupils’ assessment of safety related to crossing railway lines. The questions were linked to three locations (Fig. 1): unofficial path across the tracks (Location A), unofficial path across the tracks with a hole in the fence (Location B), and level crossing (Location C). The children were allowed to respond anonymously and the answers (base line and post-lesson) were matched at class level. The same students were assumed to have participated in both surveys unless they were sick. Only the answers of classes that had participated in both surveys were included in the analysis. No information about the gender was collected since the authors wanted to keep the data collection process as simple as possible to increase the participation of schools to the study. The study was conducted as a before-after study with no control data. The inclusion of a control group was discussed but eventually dismissed, primarily because the short survey included only a few questions and would probably raise discussions among the pupils and their parents, thus informing the control group and creating bias in their answers to the survey. 3.3. Procedure and participants The lessons were held in four schools located near railway lines in the city of Tampere in Finland in September–November 2013. The schools were selected by experts at the Finnish Transport Agency on the basis of proximity to railway lines but also because the Tampere area has been identified as a problem location for railway vandalism. The study sample was formed by four schools (A, B, C and D) who agreed to participate in the study. The lessons were held by the teachers of each class according to the instructions written by the researchers. No training was provided to the teachers to standardise the practices. In total, 321 schoolchildren in 20 classes participated in the lesson and filled in the base level survey. For unknown reasons the post-lesson survey was not completed by all schoolchildren who took part in the lesson. After removal of classes that did not fill in the post-lesson survey, the matched dataset included answers from 248 pupils in 15 classes both in base level and post-lesson surveys. 4. Results 4.1. Answers regarding behaviour intention The specific questions were Question 1: Would you cross the railway lines at location A (yes/no)? Question 2: Would you cross the railway lines at location B (yes/ no)? Question 3: Would you cross the railway lines at location C (yes/ no)? When looking at all grades together the share of ‘‘correct’’ answers (indicating crossing intention only at places where it is allowed) for questions 1–3 was fairly high already in the before

Fig. 1. Figures of locations in which the questions were linked. (A) Unofficial path across the track. (B) Unofficial path across the tracks with a hole in the fence. (C) Level crossing.

phase (72.2–94.8%), and rose by no more than 2.5 percentage units in the after phase (Table 1). There were no statistically significantly differences between grades 2–4 or schools A–D. 4.2. Answers regarding safety The specific questions were Question 4: How safe do you think crossing is (completely safe/fairly safe/slightly dangerous/very Question 5: How safe do you think crossing is (completely safe/fairly safe/slightly dangerous/very Question 6: How safe do you think crossing is (completely safe/fairly safe/slightly dangerous/very

at location A dangerous)? at location B dangerous)? at location C dangerous)?

A. Silla, V.-P. Kallberg / Evaluation and Program Planning 55 (2016) 9–16

12

Table 1 Number and share of answers (before and after) by question and grade. The ‘‘correct’’ answers are highlighted in grey. 2nd Grade

Question 1

Before After

Question 2

Before After

Question 3

Before After

3rd Grade

4th Grade

Total

Yes

No

Total

Yes

No

Total

Yes

No

Total

No

Total

No. of answers % Share No. of answers % Share

20 23.0 15 16.9

67 77.0 74 83.1

87 100.0 89 100.0

23 32.9 29 40.8

47 67.1 42 59.2

70 100.0 71 100.0

26 28.6 21 24.1

65 71.4 66 75.9

91 100.0 87 100.0

69 27.8 65 26.3

179 72.2 182 73.7

248 100.0 247 100.0

No. of answers % Share No. of answers % Share

5 5.7 1 1.1

82 94.3 88 98.9

87 100.0 89 100.0

4 5.7 4 5.6

68 94.3 67 94.4

70 100.0 71 100.0

4 4.4 2 2.3

87 95.6 86 97.7

91 100.0 88 100.0

13 5.2 7 2.8

235 94.8 241 97.2

248 100.0 248 100.0

No. of answers % Share No. of answers % Share

60 69.0 63 70.8

27 31.0 26 29.2

87 100.0 89 100.0

60 85.7 60 84.5

10 14.3 11 15.5

70 100.0 71 100.0

77 84.6 80 90.9

14 15.4 8 9.1

91 100.0 88 100.0

197 79.4 203 81.9

51 20.6 45 18.1

248 100.0 248 100.0

Yes

lines at location B (question 5, hole in the fence) was considered dangerous more often after the lesson than before (x2(1) = 6.15, p < 0.05). Specifically, the change in proportion of correct answers improved from 92.9% to 98.7%.

When looking at all grades together, the proportion of correct answers for questions 4–6 (slightly dangerous and very dangerous for question 4 and question 5; completely safe and fairly safe for question 6) in the base line survey varied between 75.4% and 93.9% (Table 2). In the after phase this rose by between 2.4 and 6.9 percentage units, the highest change relating to the location with a level crossing (location C). For all respondents together, statistically significant differences in the share of correct answers between base level and post-lesson surveys were obtained for question 5: Crossing the railway lines at location B (hole in the fence) was considered dangerous more often after the lesson than before (x2(1) = 6.62 p < 0.05) and question 6: Crossing the railway lines at location C (level crossing) was considered safe more often after the lesson than before (x2(1) = 3.96 p < 0.05). For the results by grade, the effect was statistically significant in one case: 2nd Grade schoolchildren considered crossing the railway lines at location B (Question 5, hole in the fence) to be more often dangerous after the lesson than before (x2(1) = 5.93, p < 0.05). It should be noted, however, that the proportion of correct answers in the base-level survey was lower (88.4%) for 2nd grade pupils than for 3rd and 4th grade pupils (97.7% and 97.2%). The proportion of correct answers in the post-level survey was roughly the same for all grades (97.7–98.9%). For the results by school the effect was statistically significant in one case: In school C, out of schools A–D, crossing the railway

4.3. Answers regarding legality The specific questions were Question 7: Is crossing the railway lines allowed at location A (yes/no)? Question 8: Is crossing the railway lines allowed at location B (yes/no)? Question 9: Is crossing the railway lines allowed at location C (yes/no)? When looking at all grades together the share of correct answers for questions 7–9 in the base line survey varied between 64.2% and 98.4%, the highest share concerning the crossing of railway lines at the location with a hole in the fence in location B (Table 3). The rise in the share of correct answers varied between 1.1 and 6.0 percentage units, with the highest change concerning the location with a level crossing (location C). For all respondents together, the only statistically significant differences in the share of correct answers between base level and post-lesson surveys were those obtained for question 9: Crossing the railway lines at location C (level crossing) was considered legal more often after the lesson than before (x2(1) = 4.25, p < 0.05).

Table 2 Number and share of answers (before and after) by question and grade. The correct answers are highlighted in grey. CS = Completely safe, FS = Fairly safe, SD = Slightly dangerous, VD = Very dangerous. 2nd Grade SD

VD

FS

SD

VD

FS

SD

VD

34 39.1 42 47.7

33 87 37.9 100.0 33 88 37.5 100.0

3 4.3 5 7.1

17 24.3 17 24.3

34 48.6 26 37.1

16 70 22.9 100.0 22 70 31.4 100.0

2 2.2 0 0.0

19 20.9 19 22.1

45 49.5 47 54.7

25 91 27.5 100.0 20 86 23.3 100.0

8 3.2 5 2.1

53 113 21.4 45.6 49 115 20.1 47.1

1 9 24 1.2 10.5 27.9 0 2 21 0.0 2.3 23.9

52 86 60.5 100.0 65 88 73.9 100.0

1 1.4 0 0.0

2 2.9 1 1.4

15 21.4 13 18.3

52 70 74.3 100.0 57 71 80.3 100.0

0 0.0 0 0.0

2 2.2 1 1.1

28 30.8 14 15.9

61 91 67.0 100.0 73 88 83.0 100.0

2 0.8 0 0.0

13 5.3 4 1.6

21 9 86 24.4 10.5 100.0 13 7 88 14.8 8.0 100.0

18 25.7 20 28.2

43 61.4 42 59.2

3 4.3 6 8.5

Question 5 Before No. of answers % Share After No. of answers % Share

18 20.9 20 22.7

38 44.2 48 54.5

CS

Total

CS

Total

17 19.5 13 14.8

3 3.4 0 0.0

Total

4th Grade

FS

CS Question 4 Before No. of answers % Share After No. of answers % Share

Question 6 Before No. of answers % Share After No. of answers % Share

3rd Grade

6 70 24 8.6 100.0 26.4 3 71 18 4.2 100.0 20.5

53 10 58.2 11.0 63 6 71.6 6.8

Total

CS

FS

4 91 60 134 4.4 100.0 24.3 54.2 1 88 58 153 1.1 100.0 23.5 61.9

SD

VD

Total

74 29.8 75 30.7

248 100.0 244 100.0

67 165 27.1 66.8 48 195 19.4 79.0

247 100.0 247 100.0

34 13.8 25 10.1

247 100.0 247 100.0

19 7.7 11 4.5

A. Silla, V.-P. Kallberg / Evaluation and Program Planning 55 (2016) 9–16

13

Table 3 Number and share of answers (before and after) by question and grade. The correct answers are highlighted in grey. 2nd Grade

Question 7

Before After

Question 8

Before After

Question 9

Before After

3rd Grade

4th Grade

Total

Yes

No

Total

Yes

No

Total

Yes

No

Total

No. of answers % share No. of answers % share

27 31.0 17 20.0

60 69.0 68 80.0

87 100.0 85 100.0

24 34.8 27 38.6

45 65.2 43 61.4

69 100.0 70 100.0

37 41.1 37 43.0

53 58.9 49 57.0

90 100.0 86 100.0

88 35.8 81 33.6

158 64.2 160 66.4

246 100.0 241 100.0

No. of answers % share No. of answers % share

3 3.4 3 4.5

82 96.5 64 95.5

85 100.0 67 100.0

1 1.4 1 1.4

68 98.6 69 98.6

69 100.0 70 100.0

0 0.0 2 2.3

91 100.0 86 97.7

91 100.0 88 100.0

4 1.6 6 2.7

241 98.4 219 97.3

245 100.0 225 100.0

No. of answers % share No. of answers % share

68 81.9 60 89.6

15 18.1 7 10.4

83 100.0 67 100.0

59 85.5 64 91.4

10 14.5 6 8.6

69 100.0 70 100.0

82 90.1 83 94.3

9 9.9 5 5.7

91 100.0 88 100.0

209 86.0 207 92.0

34 14.0 18 8.0

243 100.0 225 100.0

There were no statistically significant differences between grades. For the results by school the effect was statistically significant in one case: In school B, out of schools A–D, crossing the railway lines at location C (question 9, level crossing) was considered more often legal after the lesson than before (x2(1) = 4.16, p < 0.05). Specifically, the change in the proportion of correct answers in school B improved from 81.6% to 96.9%. We are unable to provide a clear explanation for the variation between schools. However, the teachers constructed the content of their lesson independently based on the lesson plan provided, and they could have weighed the issues differently. In particular, in school B they may have spent more time on the legality of crossing railway lines than in other schools, and in school C they may have paid more attention to the dangers of different crossing points. 5. Costs and benefits A simple cost-benefit analysis of this railway safety education programme was carried out. In order to quantify the benefits, we derived a rough estimate of the quantitative effect of school education on fatal trespassing fatalities in Finland over a period of 10 years. In brief, the method consists of the following assumptions and steps: During the 10-year observation period all 10-year-old schoolchildren are given a 45-min lesson on railway safety. It is assumed that the number of annual trespassing accidents each year of the 10-year observation period is reduced by a factor of z(x) = r  p(x)  N, where x is the year (1,. . .,10), r is the expected percentage reduction in the annual number of trespassing fatalities of children who have attended the safety lesson (estimated on the basis of the effect on their knowledge and behaviour intention), p(x) is the proportion of the population who have participated in the education by year x, and N is the average annual number of trespassing fatalities if no education is given. The population of Finland is approximately 5400,000 and the number of 10-year-olds is about 60,000. Therefore the proportion of the population that attend the rail safety lesson will increase every year of the 10-year observation period by 1.11% (=60,000/ 5400,000) Then the proportion of the population who have attended rail safety lessons in year x of the 10-year period would be x  0.0111. The number of prevented trespassing fatalities in year x of the 10-year period would then be x  0.0111  r  N, and the total number of prevented fatalities over the 10-year period would be (1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10)  0.0111  r  N = 55  0.0111  r  N = 0.6111  r  N.

No

Total

Yes

Based on the statistics of recent years, N is approximately 8. If we assume, on the basis of Tables 1–3, that a 5% increase in knowledge and 3% decrease in behaviour intention would result in a 4% decrease in fatal trespassing accident risk of children who attend railway safety lessons, it is possible to estimate the reduction in the number of trespassing fatalities. Then, if all schoolchildren attend a 45-min railway safety lesson during a 10year period, the estimated reduction in trespassing fatalities in that period would be 0.6111  0.04  8 = 0.195. If we assume further that the effect would last for the rest of the child’s life, or 50 years beyond the 10-year period, the total reduction in the number of trespassing fatalities would be about 6  0.195 = 1.17. The current value of preventing a fatality in Finland is 1955,064s (European Railway Agency, 2014), and thus the reduction of 1.17 fatalities would bring benefits worth s2.29 million. If we assume that the average number of children in a school class is 20, then the needed number of classes to provide lessons for 60,000 children every year would be 3000, and the number of classes needed for the 10-year programme would be 30,000. If we further assume that the cost of one lesson is s100, the total cost of the 10-year programme would be about s3 million. Comparing the monetary savings and costs of this programme, we see that the benefits far outweigh the costs if the programme manages to save more than one life. However, it should be noted that in addition to the benefits related to avoided fatalities, other benefits exist that are not included in the above calculations and they should be considered when pondering the possible implementation of this measure. These other benefits are not only related to avoided injuries and reduced vandalism, but also to the number of delays caused by related events and unauthorised people in the track areas. The calculation above is by no means intended to be definitive, but it provides a rough estimate of the quantitative effect of the measure. Furthermore, the calculation process is simple and transparent, making it easy to see how changing the assumptions would affect the end results. A quantitative effect of the measure on the number of trespassing fatalities is also useful when these kinds of educational measures are compared to alternative measures for preventing trespassing fatalities in terms of cost effectiveness. 6. Discussion and conclusions The main aim of this study was to evaluate whether railway safety lessons are effective in increasing schoolchildren’s safety knowledge and behaviour intention; thus the results provide valuable input to the discussion on the effectiveness of railway

14

A. Silla, V.-P. Kallberg / Evaluation and Program Planning 55 (2016) 9–16

safety education campaigns. The effectiveness of this measure was estimated based on three variables: behaviour intention, estimated dangerousness of the behaviour, and level of knowledge on the legality of the behaviour. All these variables are considered as strong determinants of actual behaviour. Behaviour intention: Behaviour intention is assumed to have a direct link to actual behaviour. Therefore the reduction in behaviour intention is assumed to lead to a reduction in the frequency of railway trespassing. Estimated dangerousness of the behaviour: The assumption is that the higher the children evaluate the risk to be, the smaller the probability of an unsafe crossing of railway lines. This is supported by the findings of Silla (2012), which show that perceived risk has proven to be predictive of trespassing behaviour. Specifically, trespassing was considered dangerous by (i) 98.0% of the respondents in the survey, who indicated that they had not trespassed, followed by (ii) 76.8% of the respondents who indicated that they had trespassed and (iii) 50.0% of the interviewed trespassers. Knowledge of legality of the behaviour: The assumption is that the higher the knowledge of the illegality of the crossing, the smaller the probability of an unsafe crossing of railway lines. This is supported by the findings of Silla and Luoma (2012), which indicate that the effect of awareness of legality on the respondents’ own reported trespassing was significant, with a more substantial proportion trespassing among respondents who indicated trespassing to be legal compared to those who considered it illegal. The results show that railway safety education in schools has a positive effect for all the measured variables. Specifically, the change in the share of correct answers was positive except for question 8 (Is crossing the railway lines legal at location B?). However, upon closer examination the number of yes answers was four at base level and six post-lesson, which is not a significant difference. Based on this we can reasonably assume that railway safety education in schools will also have a positive effect on the frequency of trespassing in that area. The size of the effect depends on the children’s base level understanding of the dangers related to railway lines. According to the results of this study, a fairly large majority of the children had a reasonably adequate perception of the dangers related to railways, and their behaviour intention reflected their perception even before the lesson. It may well be that the base level knowledge of schoolchildren is better in schools located near railway lines than in schools located farther away. Nonetheless it is useful to review the topic every now and then so that the children maintain their awareness of these dangers. If looking at the results in more detail we can see two interesting findings. First, even after the lesson a fairly large proportion of schoolchildren (29.2% of 2nd graders, 15.5% of 3rd graders and 9.1% of 4th graders) indicate that that they would not cross the railway lines at location C (level crossing). In addition, some schoolchildren (10.4% of 2nd graders, 8.6% of 3rd graders and 5.7% of 4th graders) answered that it is not allowed to cross the railway lines at location C (level crossing). One explanation for this might be that the parents have said to their children that they are not allowed to go near the railway lines in any place. In both cases the share of ‘‘correct’’ answers is higher the older the schoolchildren are. Therefore, one might assume that older schoolchildren adopted the message delivered by the teachers better than the younger. The second finding proposes that fencing is an effective way to inform the schoolchildren about the dangerousness and illegality of crossing the railway tracks outside level crossings, even when having a hole in the fence. Overall, prevention of trespassing is a challenge. In Finland, for example, we have nearly 6000 km of railway lines in use and unlike in some other countries, railways are not usually isolated from the

surrounding areas by fences. Moreover, trespassing behaviour does not always result in trespassing accidents, and the trespassing practice is much wider than it emerges from the accident statistics. Therefore, there is no information on the amount of trespassing occurring on railways in Finland or in Tampere region where our pilots were conducted. Due to the lack of data on the frequency of railway trespassing this study was solely based on surveys. As far the authors are aware the evaluation related to our pilot is the first attempt to evaluate the effects of education programme related to railway trespassing. Lobb et al. (2001, 2003) have evaluated the education campaigns combined with other measures but based on their studies the effect of the pure educational campaign is not clear. Our results show positive changes in behaviour intention, estimation of danger and understanding of legality of railway trespassing. Even though the obtained positive changes are rather small the results indicate that these changes can have a positive effect on the frequency of trespassing (i.e. fewer unsafe crossings in the future). Based on this we can assume that reduction in the frequency of trespassing could have an effect (i.e. reduction) on the frequency of trespassing accidents. The results of our study are not surprising since Lobb et al. (2001) also reported that several studies have suggested that educational or awareness-inducing interventions have limited effectiveness in reducing accidents or increasing safe behaviour or attitudes. We also made a simple calculation to transform the effects found out in the survey into reductions in fatalities resulting from railway trespassing. This resulted in an estimate that a 10-year nationwide education programme directed at 10-year-old schoolchildren would cost about 3 million euro and reduce the number of trespassing fatalities approximately by one, corresponding to approximately 1% of all trespassing fatalities over a 10-year period. The calculation does not directly match the situation in this pilot test but does provide a rough estimate of the possible effect of this kind of measure on the number of trespassing fatalities. It is also concluded that this kind of rail safety lesson is not necessary cost effective, since our rough calculation estimated the cost of a 10year education programme to be about 3 million euro, which exceeds the estimated benefits resulting from one less fatality (valued at about 2 million). However, it is important to be noted that the benefits of a railway safety education programme are not only limited to avoided fatalities, as considered in the calculations, but also concern the benefits of avoided injuries, reduced vandalism and fewer delays to trains due to related events and unauthorised people in the track areas. According to our experiences the evaluations of educational measures rarely, if ever, try to convert their findings into reductions in accidents or their consequences, which should be their main goal. Our approach shows that the estimate can be drawn in a rational and transparent way, providing added value to this study even if we take into account the numerous assumptions and approximations in the calculation. Education programmes aiming to increase the knowledge on safe behaviour in railway environment are important in raising the awareness of the dangers related to railway trespassing since based on our earlier studies a substantial number of interviewed trespassers and people living close to a railway line considered trespassing to be safe, and so many trespassers and people living close to a railway line assumed trespassing legal. A similar railway safety education programme could be implemented in other countries, given that the main safety message is valid everywhere. However, the material should be adjusted to comply with local circumstances (e.g. typical environments where trespassing occurs). In addition, it should be noted that this measure is expected to be more effective in raising the level of knowledge when implemented in schools located in cities and/or countries in

A. Silla, V.-P. Kallberg / Evaluation and Program Planning 55 (2016) 9–16

which the children’s level of knowledge is not as high in the before phase as in our pilot schools. This study had some limitations that should be kept in mind when generalising the results. First, the researchers did not have the power to randomly assign children to different classes and thus it is possible that the classes differed for some unknown variable that influenced their levels of change over time. Moreover, the different groups of schoolchildren were taught by different teachers, therefore there were likely variations in how the lesson was presented. Second, the study periods for pilot tests in RESTRAIL project were limited and did not allow the collection of data on long term effects of the education programme. It is likely that the effect will diminish over the time unless the railway safety lessons are repeated. Therefore, it is emphasised that the cost-benefit analysis of the education programme is by no means intended to be definitive, but rather provides a rough estimate of the quantitative effect of the measure. The authors would like to note that the collection of long term effects (e.g. one year after the lesson) of this kind of education programme can be challenging since rather often, at least in Finland, the school classes have a different teacher every year which complicates to the collection of evaluation data. Third, due to the lack of data on the frequency of railway trespassing this study was solely based on surveys. In optimal design each school should be linked to one or several trespassing sites and the number of trespasses of schoolchildren would have been monitored in these sites during the study. However, due to the open nature of the Finnish railway environment the identification of unofficial paths across the railway lines in Tampere region and the supervision of those paths would have required extensive amount of resources and thus it was not practical. Lastly, the study can be criticised for the lack of appropriate control group. The challenge of finding a proper control group applies to all evaluation studies related to railway trespassing and not only education programmes. In optimal situation the railway environment (i.e. the extent of fencing, number of unauthorised paths and number of legal crossing possibilities) near the control group should be similar as near the treatment group. One option to solve this challenge is to take both the control and treatment group from the same school. However, in practice this is not so simple since (a) we need to have a school located near railway lines, (b) the school should have more than one class per grade, (c) we should ascertain that the children from treatment group do not discuss about the correct answers of the survey with the children from control group, and (d) the teachers should be committed to the education programme. However, even if we could have found and used a proper control group, it would not have changed the fact that the effect of our railway safety lesson in the form it was realised in our study is fairly small. This is the case because lack of control group in a before-after study typically leads to overestimation of the positive effect of a measure.

7. Lesson learned and future research The evaluation of future education programmes regarding railway safety is strongly recommended since at the moment there is hardly any information on their effectiveness. The implementation of the education programme should be carefully planned and allow the collection of proper evaluation data both before and after the intervention, including the long term effects of the programme (e.g. one year after the lesson). Furthermore, the evaluation on the effects by gender and possible inclusion of a control group to

15

eliminate potential extraneous factors are proposed for future research. Based on the studies conducted so far, even though education and campaigns can reduce railway trespassing and related accidents to certain degree, their effects can be rather limited. Therefore, education and campaigns are not necessarily always the most effective and cost-efficient measures. When possible, a wider range of measures and their combinations as described in the RESTRAIL project (RESTRAIL, 2014b) should be considered. Acknowledgements The authors wish to thank Juha Luoma from VTT for his valuable comments on earlier drafts of this paper. This study was supported by the European Commission under the 7th Framework Programme (RESTRAIL project; Grant Agreement No. 285153). References Dragutinovic, N., & Twisk, D. (2006). The effectiveness of road safety education—A literature review. The Netherlands: R-2006-6- SWOV Institute for Road Safety Research Available in Internet: hhttp://www.swov.nl/rapport/r-2006-06.pdfi (accessed March 6, 2014). European Railway Agency (2014). ERADIS—European railway agency database of interoperability and safety. hhttps://eradis.era.europa.eu/i (accessed July 1, 2014). Finnish Rail Administration (2007–2009). Finnish railway statistics. Separate publications for each year. Helsinki: Finnish Rail Administration hhttp:// portal.liikennevirasto.fi/sivu/www/f/aineistopalvelut/julkaisut/kirjasto/ trafiikki#.U7KdzVXyU-Ui (accessed July 1, 2014). Finnish Transport Agency (2010–2012). Finnish railway statistics. Separate publications for each year. Helsinki: Finnish Transport Agency hhttp:// portal.liikennevirasto.fi/sivu/www/f/aineistopalvelut/julkaisut/kirjasto/ trafiikki#.U7KdzVXyU-Ui (accessed July 1, 2014). Finnish Transport Safety Agency (2014). Rautatieturvallisuustietoa. hhttp:// www.rautatieturvallisuus.fi/rautatieturvallisuusi (accessed July 1, 2014). Lobb, B., Harre´, N., & Suddendorf, T. (2001). An evaluation of a suburban railway pedestrian crossing safety programme. Accident Analysis and Prevention, 33, 157–165. Lobb, B., Harre´, N., & Terry, N. (2003). An evaluation of four types of railway pedestrian crossing safety intervention. Accident Analysis and Prevention, 35, 487–494. Lobb, B. (2006). Trespassing on the tracks: A review of railway pedestrian safety research. Journal of Safety Research, 37, 359–365. Network Rail (2014). Safety education. hhttp://www.networkrail.co.uk/safetyeducation/i (accessed August 15, 2014). Operation Lifesaver (2014). Rail safety education. hhttp://oli.org/i (accessed March 6, 2014). Railway Safety and Standards Board (2005). Trespass and access via the platform end. Final report. Halcrow Group Limited in partnership with Human Engineering hhttp://rssb.co.uki (accessed June 12, 2007). RESTRAIL (2014a). Reduction of suicides and trespasses on railway property. hhttp:// www.restrail.eu/i (accessed August 15, 2014). RESTRAIL (2014b). Railway suicides and trespassing accidents: How to prevent the incidents and mitigate the consequences?. hhttp://restrail.eu/toolbox/i (accessed August, 15th, 2014). Savage, I. (2006). Does public education improve rail-highway crossing safety? Accident Analysis and Prevention, 38, 310–316. Silla, A. (2012). Improving safety on Finnish railways by prevention of trespassing. VTT Science, 27. 49 p.+app. 43 p. Espoo 2012. Silla, A., & Luoma, J. (2009). Trespassing on Finnish railways: Identification of problem sites and characteristics of trespassing behaviour. European Transport Research Review, 1, 47–53. Silla, A., & Luoma, J. (2011). Effect of three countermeasures against the illegal crossing of railway tracks. Accident Analysis and Prevention, 43, 1089–1094. Silla, A., & Luoma, J. (2012). Opinions on railway trespassing of people living close to a railway line. Safety Science, 50, 62–67. TrackSAFE (2014). Teachers. hhttp://www.tracksafeeducation.com.au/Teachers.aspxi (accessed August 15, 2014). TrackSAFE NZ (2014). Education. hhttp://www.railsafety.co.nz/education.htmli (accessed August 15, 2014). Twisk, D. A. M., Vlakveld, W. P., Commandeur, J. J. F., Shope, J. T., & Kok, G. (2014). Five road safety education programmes for young adolescent pedestrians and cyclists: A multi-programme evaluation in a filed setting. Accident Analysis and Prevention, 66, 55–61. Dr. Anne Silla (Female) is a Senior Scientist and has worked at the VTT Technical Research Centre of Finland since 2005. She received her M.Sc. (Technology) from the Tampere University of Technology in 2005 and her Ph.D. in Transportation Engineering from Aalto University in 2013. Silla has also participated in a two-year TRANSPORTNET

16

A. Silla, V.-P. Kallberg / Evaluation and Program Planning 55 (2016) 9–16

Marie Curie Programme which enabled her to spent two years in Germany (2007– 2009) and participate in several international projects and courses meant for doctoral students in the field of transport. Her research activities have mainly focused on improving the safety of road and railway traffic. Examples of her research topics include assessment of safety effects of ITS directed to vulnerable road users (VRUITS), prevention of railway trespassing and railway suicides (RESTRAIL + national projects), improving the safety of level crossings (national projects) and safety management (national projects).

Mr. Veli-Pekka Kallberg, M.Sc.Tech. (Transport Technology), Principal Research Scientist (retired 31.5.2015), graduated from the Helsinki University of Technology (HUT) in 1976 and has since then worked at VTT Technical Research Centre of Finland. For most of his professional career he has worked in various kinds of road safety research projects. Since the late 1990’s he has focused on rail safety and lead several research projects concerning the safety of Finnish railways and especially the safety of level crossings. He has been involved in several EU projects, including the project RESTRAIL aiming to reduce railway suicides and trespassing accidents.