High prevalence of burnout syndrome in orthopedic nurses in Italy

High prevalence of burnout syndrome in orthopedic nurses in Italy

International Journal of Orthopaedic and Trauma Nursing xxx (xxxx) xxxx Contents lists available at ScienceDirect International Journal of Orthopaed...

213KB Sizes 1 Downloads 47 Views

International Journal of Orthopaedic and Trauma Nursing xxx (xxxx) xxxx

Contents lists available at ScienceDirect

International Journal of Orthopaedic and Trauma Nursing journal homepage: www.elsevier.com/locate/ijotn

High prevalence of burnout syndrome in orthopedic nurses in Italy Nicolò Martinellia,∗, Margherita Frattolillob, Valerio Sansonea,c a b c

IRCCS Istituto Ortopedico Galeazzi, Via Riccardo Galeazzi, 4, 20161, Milan, Italy Department of Orthopedic, ICCS Istituto Clinico Città Studi, 20131, Milan, Italy Department of Orthopaedics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122, Milan, Italy

A B S T R A C T

Background: Recent studies reported particularly high levels of burnout in nurses working in Orthopedic and Trauma departments. Objective: The objectives of the present study were to evaluate the estimated prevalence of burnout and psychiatric morbidity in a population of nurses that work in Italian orthopedic and traumatology departments and to reveal eventual correlations between socio-demographic factors, stress factors, and satisfaction at work. Methods: Nurses, employed in orthopedics and traumatology departments, were asked to complete an anonymous questionnaire including: a section on demographic data, the Maslach Burnout Inventory, the Short-Form 12, a section on personal and social life, work conditions, and feelings about the work. Results: A high emotional exhaustion (EE) value was present in 35 nurses (66%), a high depersonalization (DP) value in 44 nurses (83%), and a low personal accomplishment (PA) value in 47 nurses (88%). High levels of burnout (EE and DP) were strongly correlated to nurses' perception of health-related quality of life (in the Mental Component Summary-12). Conclusion: The average subscales of burnout were remarkably high among Italian orthopedic nurses, suggesting the need of further studies for obtaining a more exact prevalence of this phenomenon.

Introduction Historically, the health workers employed in departments such as Oncology, AIDS treatment, Hematology, and the intensive care units are subject to higher levels of stress during their work activity. In these work environments the employees are forced to confront all aspects of death, meanwhile, they must maintain an empathetic relationship with the patients that are suffering or dying. Bressi et al. assessed burnout prevalence in Italian nurses employed in haemato-oncology departments and found high level of emotional exhaustion in 31.9%, a high level of depersonalization in 23.6% and a low level of personal accomplishment in 15.3% (Bressi et al., 2008). In a 2010 study, particularly high levels of burnout were found in nurses working in Orthopedic and Trauma departments in Greece (Kiekkas et al., 2010). The study of Kiekkas et al. included a convenience sample of 60 orthopedic nurses recruited from the orthopedic wards of three hospitals. To the best of our knowledge, this study is the only study specifically assessing burnout syndrome in orthopedic nurses. The unique treatments needs of the orthopedic patient may help to explain this finding. Patients hospitalized because of fragility fractures present different complex orthogeriatric care needs that include pain management, nutrition, hydration, early mobilization and pressure ulcer prevention



(Brent et al., 2018). Furthermore, patient's age, physical disability, cognitive impairment and comorbidities may result in constant assistance. The nursing staff is not only relied upon in the wards but also in the operating theatres (OTs). Several studies have attempted to evaluate the phenomenon of burnout in the latter work environment, one of the most stressful within a hospital. Surprisingly, in one study (Findik, 2015), OT nurses reported a level of burnout similar to the other departments. The study of Findik was conducted in Turkey and included 64 OT nurses employed at three university hospitals and two public hospitals. Nurses worked in different surgical departments, but only five worked for the orthopedic department. This study showed that the lack of precautions to prevent the transmission of infectious agents, lack of protective measures in areas where radioactive materials are present, and lack of support from management increases the incidence of burnout. Furthermore, in the study by Kiekkas et al. the conflict with the orthopedic medical personnel was found to be associated with an increased risk of burnout in nurses (Kiekkas et al., 2010). In a previous study, the phenomenological analysis of the interviews of the nursing staff employed in Orthopedics revealed that nurses' actions were frequently obstructed by the orthopedic surgeons and the doctor-nurse collaboration was frequently problematic (Skei, 2008). The nurses

Corresponding author. IRCCS Istituto ortopedico Galeazzi, Via R. Galeazzi 4, 20100, Milan, Italy. E-mail address: [email protected] (N. Martinelli).

https://doi.org/10.1016/j.ijotn.2019.100747 Received 22 May 2019; Received in revised form 21 October 2019; Accepted 25 November 2019 1878-1241/ © 2019 Elsevier Ltd. All rights reserved.

Please cite this article as: Nicolò Martinelli, Margherita Frattolillo and Valerio Sansone, International Journal of Orthopaedic and Trauma Nursing, https://doi.org/10.1016/j.ijotn.2019.100747

International Journal of Orthopaedic and Trauma Nursing xxx (xxxx) xxxx

N. Martinelli, et al.

underestimated; Personal Accomplishment (PA) – referring to the feelings of competence, efficiency, productivity and completeness of one's work. The subject was asked to answer each question on the basis of a Likert scale that measures the frequency of the event over time from 0 (never) to 6 (every day). The subscales are considered “high”, “moderate,” or “low” based on predetermined cut-off scores derived from normality data. Scores are considered “high” if they are in the upper third of the normative distribution, “moderate” if they are in the middle, and “low” if they are in the lower third. The Italian validation of the questionnaire established the following intervals: high EE ≥ 24, DP ≥ 9, PA < 29; moderate EE = 15–23, DP = 4–8, PA = 30–36; low EE ≤ 14, DP ≤ 3, PA < 37. The Italian reference sample of 748 Italians working in the health professions had an average ± standard deviation of EE = 20.18 ± 11.29; DP = 7.03 ± 5.9; PA = 32.52 ± 8.66.48. 3. The third section is the Italian version of the Short-Form 12 (SF-12). This scale is derived from the SF-36 and consists of 12 questions. The scale measures two indexes: the Physical Component Summary (PCS) and the Mental Component Summary (MCS), which respectively measure the quality of life in relation to the physical and psychological level. This scale has been widely used internationally and its reliability and validity have been also tested in Italian studies (Apolone, 2001). 4. The fourth section consists of questions designed specifically for the study. The questions are derived from and a pilot study involving 5 nurses that was carried out to assess the relevance of the questions with regards to the objectives of the study. Empirical factors linked to high risk of burnout among acute nursing care personnel are evaluated in 13 questions on personal and social life, work conditions, and feelings about the work of the participants. Participants were also asked to assess their workload, the number of shifts, job security, job satisfaction, and quality of patient care provided (assessed as high, medium or low). Furthermore, each participant was asked to evaluate their relationship with doctors and other colleagues in superior positions.

reported difficulty in being assertive in their work place and in working to their full potential as a consequence of the feeling inferior to the surgeons. They also referred to an emotional burden, caused by disrespectful behavior of the doctors, that they never verbalized and therefore always internalized. The study of Skei was conducted in Norway and country's culture may influence how nursing and medical staff interact. In 2013 study by the Ministry of Health, the most frequently performed elective surgery in Italy was the prosthetic replacement of hip and knee joints (Seghieri, 2013). In 2011, a total of 106,803 hip or knee prostheses were carried out. This number reflects the tremendous diffusion of orthopedic surgery in Italy, claiming first place among all surgical disciplines in terms of number of interventions. Piscitelli et al. reported the number of fragility fractures in Italy for the year 2002 subdivided by fracture site: 91.494 hip fractures, 61.009 clinical vertebral fractures, 57.401 humeral fragility fractures, and 94.045 forearm/wrist fragility fractures. Interestingly, the number of hospitalizations for hip fracture was comparable to that of acute myocardial infarction (Piscitelli et al., 2007). This high number of interventions would not be possible without a substantial contribution of nursing staff. For this reason, it is useful to evaluate the prevalence of burnout syndrome among nurses employed in the Italian orthopedics and traumatology departments. The objectives of the present study were first, to evaluate the estimated prevalence of burnout and psychiatric morbidity in a population of nurses that work in Italian orthopedic and traumatology departments and second, to reveal eventual correlations between socio-demographic factors, stress factors, and satisfaction at work. Material and methods In this descriptive study, written consent was obtained, and detailed information was provided by each of the personnel involved. The study was conducted in accordance with the declaration of Helsinki and approved by hospital administrations of two institutions. Participation in the study was voluntary and only the principal investigator (NM) was able to access the collected data. All nurses’ data were anonymized and data were stored on a password-protected computer. This study was conducted with a convenience sample of nurses employed in two different hospitals ( IRCCS Istituto Ortopedico Galeazzi and Istituto Clinico Città Studi). Both hospitals are located in Milan, Italy.. Nurses were asked to complete an anonymous questionnaire. The questionnaires were delivered via the SurveyMonkey software (Palo Alto, CA, USA) to the participants' e-mails. To obtain a higher response rate, an automatic reminder was automatically sent by the software 2 weeks after the first email. The questionnaire included 4 sections corresponding to 4 assessment measures.

Statistical analysis The analysis of the collected data was carried out using the SPPS 20.0 software (Statistical Package for Social Sciences, Inc., Chicago, IL). Cronbach's coefficient was used to evaluate the internal consistency of each MBI subscale, while the Pearson correlation coefficient was used to evaluate the magnitude and direction of the correlations between the three MBI subscales. The scores of the sub scales were studied both on the basis of demographic data and on the basis of the nurses' answers to the fourth section of the questionnaire. For all statistical data, a p < 0.05 was necessary to be considered statistically significant. A one-way analysis of variance was used to assess differences by age, marital status, total time in the profession, and each of the 13 questions in the fourth section of the questionnaire, while a post-hoc test following Bonferroni was used for a “coupled” comparison. Correlations between the scores of the MBI subscales and the two subscales of the SF12 were estimated using the Spearman correlation coefficient.

Evaluation measures/tools 1. The first section contains seven questions related to demographic data (age, gender, level of education, marital status and total time in the profession) and the type of employment. 2. The second section is the Italian version of the Maslach Burnout Inventory (MBI). The MBI is a self-administered questionnaire with 22 items and the following 3 sub-scales: Emotional Exhaustion (EE) – defined as a lack of energy needed to face the workday, accompanied by the feeling of being emptied and having exhausted all one's emotional resources, with feelings common of apathy and detachment in relation to work; depersonalization (DP) – defined as professional behavior characterized by detachment, coldness, boredom, irritation, and harshness, even hostility, leading the subject to seek distance from his/her work and from the relationships with the people they are helping, whose needs tend to be

Results The questionnaire was sent to 95 nurses, fifty-three (55.78%) of whom responded. The majority (75.5%) of the participants were women, with an average age of 38 ± 8 years, and they were employed prevalently in the ward (71.7%), while 15 nurses were employed in OT (28.3%) (Table 1). Most of the participants (54.7%) were over 40 years of age, and 47.2% of them had 16 years of experience working as a nurse. The average EE score was 28.8 ± 8.4; the average DP score was 14.0 ± 5.4; and the average PA score was 21.7 ± 5.1. A high EE value was present in 35 nurses (66%), a high DP value in 44 nurses 2

International Journal of Orthopaedic and Trauma Nursing xxx (xxxx) xxxx

N. Martinelli, et al.

Table 1 Socio-demographic characteristics of the nursing staff enrolled in this study. Variable Sex Male Female Age ≤ 30 years 31–40 years > 40 years Marital status Married Single Divorced/Widow Education level Undergraduate Master's degree Master's degree (level II) Doctor of philosophy Position Nurse Charge Nurse Nursing staff coordinator Work site Operating room Ward Total experience in the profession 0–5 years 6–10 years 11–15 years ≥ 16 years Number of nights per month 0 1-2 ≥3

Table 2 Differences in the MBI subscales, considering the work, personal, and social conditions of the participants.

N (%)

Mean (SD) N (%)

13 (24.5) 40 (75.5)

Salary Unsatisfactory 38.7 (71.7) Almost satisfactory 13 (24.5) Satisfactory 2 (3.7) p Workload High 36 (67.9) Medium 17 (32.1) Low 0 p Number of turns High 28 (52.8) Medium 24 (45.3) Low 1 (1.9) p Rapport with doctorsa Unsatisfactory 21 (39.6) Almost satisfactory 20 (37.7) Satisfactory 12 (22.6) p Quality of care providedb High 5 (9.4) Medium 37 (69.8) Low 11 (20.7) p Fear of the future A lot 14 (26.4) Some 26 (49.0) None 13 (24.6) p

14 (26.4) 14 (26.4) 26 (47.1) 23 (43.4) 27 (50.9) 3 (5.7) 43 (81.1) 5 (9.4) 4 (7.5) 1 (1.9) 39 (73.6) 8 (15.1) 6 (11.3) 15 (28.3) 38 (71.7) 15 (28.3) 8 (15.1) 5 (9.4) 25 (47.2) 18 (34.0) 4 (7.5) 31 (58.5)

EE

DP

PA

29.5 (8.9) 28.0 (7.0) 20.0 (1.4) 0.28

14.9 (5.0) 12.4 (5.4) 6.5 (7.7) 0.04*

22.2 (5.1) 21.0 (4.6) 17.0 (9.8) 0.32

30.2 (8.8) 25.2 (6.8)

15.3 (4.9) 11.1 (5.4)

22.5 (4.9) 20.0 (5.3)

0.07

< 0.01*

0.09

31.6 (8.8) 25.8 (6.9) 20.0 0.02*

16.2 (5.6) 11.6 (4.0) 8.0 < 0.01*

23.1 (5.6) 20.0 (4.2) 26.0 0.07

27.2 (8.1) 28.9 (7.5) 31.2 (10.4) 0.4

11.2 (4.7) 15.1 (5.3) 17.0 (5.0) < 0.01*

20.4 (5.1) 22.4 (4.6) 22.9 (5.9) 0.3

26.6 (8.6) 27.4 (7.6) 34.4 (9.5) 0.04*

14.8 (3.2) 12.3 (5.0) 19.3 (4.2) < 0.01*

25.8 (3.0) 20.3 (4.6) 24.7 (5.6) < 0.01*

31.0 (8.6) 29.4 (7.5) 25.1 (9.4) 0.17

16.3 (5.8) 14.5 (5.1) 10.6 (4.2) 0.01*

23.7 (5.5) 20.4 (5.0) 22.3 (4.5) 0.13

Table legend: EE = Emotional Exhaustion; DP = Depersonalization; PA = Personal Accomplishment. a For DP, there was a statistically significant difference (p < 0.05) between the satisfied and unsatisfied. b For EE, there was a statistically significant difference (p < 0.05) between medium and low levels of quality of care; for DPand PA, there was a statistically significant difference (p < 0.05) between the medium and low levels of quality of care.

(83%) and a low PA value in 47 nurses (88%). The internal consistency of Cronbach's α was satisfactory for all 3 subscales, with an α coefficient of 0.76 for EE, 0.66 for DP e 0.78 for PA. Age, sex, work site, total time in the profession, role, and marital status were not associated with increased levels of burnout. The values of the MBI scales, considering the work, personal, and social conditions of the participants are shown in Table 2 (statistically significant pairwise comparisons are indicated in the table legend). The average EE values were higher for nursing staff with more shifts and in departments providing lower quality care (p < 0.05). The average DP values were higher for nurses with high numbers of shifts, poor quality of care, unsatisfactory rapport with doctors, high workloads, with an unsatisfactory salary, and in nurses with a fear of the future regarding their career (p < 0.05 for all categories in association with DP). Mean PA values were lower (p < 0.05) in nurses who perceived the quality of care provided to patients of average and low quality. Despite a statistical association, the relationship with doctors and burnout needs further investigation, since in this study only one question was directed to such issue. The average PCS-12 and MCS-12 scores were 43.6 ± 5.3 e 43.9 ± 6.3. Statistically significant negative correlations were found between the MCS-12 scale, the EE score (r = - 0.31; p < 0.05) and the DP score (r = −0.44; p < 0.01).

received. In the Greek study (Kiekkas et al., 2010), high EE scores were found in 23 nurses (38.3%), high DP scores in 21 nurses (35.0%), and low PA scores in 32 (53.3%) nurses. The burnout levels shown in the present study are markedly worse: a high EE value was present in 35 nurses (66%), a high DP value in 44 nurses (83%), and a low PA value in 47 nurses (88%). This striking difference could be in part explained by the diversity of the sample and by the place where the study was conducted. The sample consisted of nursing staff employed in both the ward and in the OT, and OT nurses are historically exposed to high stress, very high workloads, and on-call time to name a few. Unfortunately, the sample size did not allow for a more advanced statistical analysis (i.e. regression analysis) to assess the weight of this factor on burnout levels. The treatment needs of the orthopedic patient could explain the high incidence of burnout in orthopedic nurses. In a 2006 study, a 45% incidence of depression was found in a diverse population of orthopedic trauma patients, with 16.7% of them showing moderate to severe or severe depression (Crichlow et al., 2006). The psychological morbidity of these patients was attributed to difficulty in performing normal daily activities because of their temporary physical disability, post-operative pain, sleep disorders, and uncertain functional results. Elderly orthopedic patients also constitute a difficult population to treat, not only for physical disability and the need for intensive care but also because of the high prevalence of comorbidities, especially neurological diseases

Discussion This is the first study in Italy to evaluate levels of burnout in nurses employed in Orthopedic and Traumatology departments. Comparing these results with the averages of the Italian reference sample, it is possible to state that the burnout levels are notably greater, especially as represented by the average values of DP and PA (Sirigatti and Stefanile, 1993). The statistical analysis showed a statistically significant association between the DP scores and the workload and salary 3

International Journal of Orthopaedic and Trauma Nursing xxx (xxxx) xxxx

N. Martinelli, et al.

values for the Italian population. The results of this study indicate high levels of burnout in the orthopedic and trauma nursing staff, suggesting the need of further studies for obtaining a more exact prevalence of this phenomenon.

(eg. Alzheimer's, senile dementias, etc.). However, there is clearly a need for further studies to support this claim. Conflict with medical personnel was also associated with an increased risk of burnout among nurses in orthopedics departments. In the Greek study, high levels of burnout were associated with unsatisfactory relationships with medical staff (Kiekkas et al., 2010). Similarly, in Italian nurses a high DP score was associated with unsatisfactory relationships with orthopedic doctors. This statement must be evaluated considering the work site. The stress of OT nurses is often transferred to them by the orthopedic doctors. Among work conditions, salary was negatively associated levels of burnout (p < 0.05). This result is in contrast with the results of the Kiekkas et al. study, in which salary was not associated burnout, despite the fact that more than 70% of the sample considered the salary unsatisfactory. This discrepancy can be attributed to the region in which this study was conducted as the city of Milan appears to be one of the cities with the highest cost of life in Italy. This factor could have serious repercussions on the quality of life of the nursing staff, 71% (38 nurses) of which reported having an unsatisfactory salary. In this study, the number of shifts and workload were positively associated with levels of burnout (p < 0.05). These results are similar to other previous studies Demir et al. found that EE levels of nurses employed in state hospitals were negatively affected by a lack of staff (Demir et al., 2003). In another study, Aiken et al. found that an adequate number of nursing staff was a key factor in reducing job dissatisfaction and burnout and improving the problem of drop-outs of nurses from hospitals (Aiken et al., 2002). In the present study, the quality of care provided to the patients by nurses was associated with burnout for all three scales. These results are similar to those published another study, in which high levels of burnout were significantly associated with nurses' assessments of the quality of care they provided, regardless of nurse characteristics, working conditions, and other variables (Poghosyan et al., 2010). Finally, according to the results of our study, high levels of burnout (in EE and DP) are strongly correlated to nurses' perception of health-related quality of life (in the MCS-12), which is congruent other reports in the literature. For example, in a recent Chinese study (Wu et al., 2011), stress at work, burnout (with high EE and low PA scores), and the duration of work shifts were the main risk factors for nurses' quality of life. Emotionally, occupational stress has been correlated with anxiety, dysthymia, low self-esteem, depression, and feelings of inadequacy, and is increasingly recognized as an important risk factor for mild psychiatric morbidity (Stathopoulou et al., 2011; Collins and Long, 2003). The most important limitation of this study is the sample size. This could lead to a Type II error, meaning that more burnout-related factors may have been revealed if more nurses had been enrolled. The sample size did not allow for a multivariate analysis necessary to investigate the associations between burnout scales and their correlates. The crosssectional nature of the study may limit the potential to show a causal relationship between the suggested determinants and burnout levels in the orthopedic and trauma nursing staff. Finally, the results may not be generalized to all the Italian hospitals.

Declaration of competing interest No funding were received to conduct this study. The authors declare no conflict of interest related to this publication. Appendix A. Supplementary data Supplementary data to this article can be found online at https:// doi.org/10.1016/j.ijotn.2019.100747. Ethical statement All individuals involved in this study provided written consent to participate. The study was conducted in accordance with the declaration of Helsinki. All the collected data were anonymized. Funding No funding were received to conduct this study. References Aiken, L.H., Clarke, S.P., Sloane, D.M., Sochalski, J., Silber, J.H., 2002. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. J. Am. Med. Assoc. 288 (16), 1987–1993. Apolone, G., 2001. Self-perceived health status assessment in epidemiological studies: evaluation of the SF-12 Health Survey. Results from the MiOS Project. J. Epidemiol. Biostat. 6 (3), 305–316. Brent, L., Hommel, A., Maher, A.B., Hertz, K., Meehan, A.J., Santy-Tomlinson, J., 2018. Nursing care of fragility fracture patients. Injury 49 (8), 1409–1412. Bressi, C., Manenti, S., Porcellana, M., et al., 2008. Haemato-oncology and burnout: an Italian survey. Br. J. Canc. 98 (6), 1046–1052. Crichlow, R.J., Andres, P.L., Morrison, S.M., Haley, S.M., Vrahas, M.S., 2006. Depression in orthopaedic trauma patients. Prevalence and severity. Journal of Bone & Joint Surgery American 88 (9), 1927–1933. Collins, S., Long, A., 2003. Working with the psychological effects of trauma: consequences for mental health-care workers-a literature review. J. Psychiatr. Ment. Health Nurs. 10 (4), 417–424. Demir, A., Ulusoy, M., Ulusoy, M.F., 2003. Investigation of factors influencing burnout levels in the professional and private lives of nurses. Int. J. Nurs. Stud. 40 (8), 807–827. Findik, U.Y., 2015. Operating room nurses' burnout and safety applications. Int. J. Caring Sci. 8 (3), 610–617. Kiekkas, P., Spyratos, F., Lampa, E., Aretha, D., Sakellaropoulos, G.C., 2010. Level and correlates of burnout among orthopaedic nurses in Greece. Orthop. Nurs. 29 (3), 203–209. Piscitelli, P., Iolascon, G., Gimigliano, F., et al., 2007. Incidence and costs of hip fractures compared to acute myocardial infarction in the Italian population: a 4 year survey. Osteoporos. Int. 18 (2), 211–219. Poghosyan, L., Clarke, S.P., Finlayson, M., Aiken, L.H., 2010. Nurse burnout and quality of care: cross- national investigation in six countries. Res. Nurs. Health 33 (4), 288–298. Seghieri, C., 2013. Appropriatezza e variabilità negli interventi di chirurgia elettiva. Il Laboratorio di Management e Sanità, Istituto di Management, Scuola Superiore Sant'Anna, Pisa. www.salute.gov.it/imgs/C_17_minpag_969_documenti_documento_ 20_fileAllegatoDoc.ppt. Sirigatti, S., Stefanile, C., 1993. Manuale e taratura per l'Italia. In: Maslach, C., Jackson, S. (Eds.), MBI Maslach Burnout Inventory. Organizzazioni Speciali, Firenze, pp. p33–p42. Skei, K., 2008. Collaboration at risk: registered nurses' experiences on orthopaedic wards. J. Clin. Nurs. 17 (14), 1907–1914. Stathopoulou, H., Karanikola, M.N., Panagiotopoulou, F., Papathanassoglou, E.D., 2011. Anxiety levels and related symptoms in emergency nursing personnel in Greece. J. Emerg. Nurs. 37 (4), 314–320. Wu, S.Y., Li, H.Y., Tian, J., Zhu, W., Li, J., Wang, X.R., 2011. Health-related quality of life and its main related factors among nurses in China. Ind. Health 49 (2), 158–165.

Conclusions The average subscales of burnout were remarkably high among Italian orthopedic nurses. Among the factors studied, 6 were found to be associated with burnout: salary, workload, number of shifts, rapport with doctors, quality of care provided, and fear of the future. The quality of life of the nursing staff employed in the Orthopedics and Traumatology divisions was considerably lower than the reference

4