The scale of Work-Related Affective Feelings (WORAF)

Applied Ergonomics 82 (2020) 102945

Contents lists available at ScienceDirect

Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo

The scale of Work-Related Affective Feelings (WORAF) a,∗

b

Magdalena Anna Jaworek , Tadeusz Marek , Waldemar Karwowski a b c

T

c

Institute of Economics, Finance, and Management, Jagiellonian University, Krakow, Poland Department of Psychology, University of Social Sciences and Humanities, Warsaw, Poland Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL, 32816, USA

A R T I C LE I N FO

A B S T R A C T

Keywords: Work-related affective feelings Discrete-emotions approach Scale development Validation

Interest in the field of emotions in the workplace continues to grow. However, a narrow range of measurements exists for the examination of affective states in organizational settings, all of which were developed based on the discrete-emotions approach. The main objective of the current study was to develop and validate a new instrument to assess four work-related (WR) affective feelings (WORAF): WR feelings of happiness, WR feelings of anxiety, WR feelings of anger, and WR feelings of dejection. Three independent samples of Polish employees in different occupations were included in the study (n = 297, n = 3019, n = 284). The results provide significant evidence for the validity of the proposed WORAF scale, although further investigations are needed.

1. Introduction Both the work environment and the specific nature of an employee's tasks can be a source of negative and positive work-related experiences. Each employee associates these experiences with different emotions, leading to various moods, which, in turn, are related to work attitudes, behaviors, and activities (see: affective events theory by Weiss and Cropanzano, 1996). The role of emotions and mood in the work environment remains highly understudied today (Ashforth and Humphrey, 1995; Fitness, 2000; Muchinsky, 2000), not only in the field of industrial/organizational psychology but also in the field of human factors and ergonomics (for example, see: Eccles et al., 2011). Therefore, our current knowledge regarding this subject area remains limited. Even so, in recent decades, scholarly interest in examining work-related affective states has increased rapidly (Ashkanasy and Humphrey, 2011; Barsade and Gibson, 2007). For example, Barsade et al. (2003) mention an “affective revolution” in the field of organizational behavior, which according to the authors, began at the end of the last century. Unfortunately, at present, researchers only have access to a narrow range of measurement instruments that can be used to examine affective states in organizational settings (see: Fisher, 1997). Compounding this dilemma, some scholars have identified the shortcomings of the xisting literature as being restricted to investigations of only positive and negative affects, creating a real need to pay more attention to discrete emotions, each of which can be related to different action tendencies (e.g., Ashkanasy and Dorris, 2017). Therefore, the main goal of the current study was to develop and conduct an initial validation of a new

∗

instrument to assess four work-related affective feelings: happiness, anxiety, anger, and dejection. Our proposition meets the expectations of scholars who have stated that to measure affective states “new instruments with different (than dimensional approach) theoretical models are needed” (Crispim et al., 2015, p. 104). 2. The meaning of feeling One significant problem encountered in the investigation of affective states is confusion in the terminology used to indicate emotional states, which hinders the organizational aspects of research in this field. Affect, emotion, mood, and feeling belong to the same semantic category and are often used interchangeably by scholars. However, there is a general agreement that moods are typically less intense states than emotional states and that emotions are more short-lived and more targeted on another individual or in response to an incident than a mood (see: Frijda, 1994; Ekman, 1994; Fisher, 2000; Warr et al., 2014). Affect, according to Russell's concept (2003), “… is the neurophysiological state consciously accessible as the simplest raw (non-reflective) feelings evident in moods and emotions” (p. 148), described by two dimensions: pleasantness and activation (Feldman Barrett and Russell, 1999; Russell, 2003). The two-dimensional structure of an individual's affect is also applied in the models of Larsen and Diener, 1992 and Thayer, 2012. Unlike emotion, mood, or affect, the word feeling has not been described as widely in the literature, and thus, when it comes to defining this term, the same problem seems to exist as with the other terms

Corresponding author. E-mail addresses: [email protected] (M.A. Jaworek), [email protected] (W. Karwowski).

https://doi.org/10.1016/j.apergo.2019.102945 Received 19 September 2018; Received in revised form 13 August 2019; Accepted 22 August 2019 0003-6870/ © 2019 Elsevier Ltd. All rights reserved.

Applied Ergonomics 82 (2020) 102945

M.A. Jaworek, et al.

3.2. The concept of anger

related to affective states. On the one hand, according to Berkovitz, 2000, a feeling encompasses “emotional experiences, moods, and even physical sensations” (p. 2). On the other hand, Plutchik (2001) claims that “emotion (…) includes feelings …” (p.345). Russell (2003) identifies the term feeling with a core affect and describes one's feelings as “an assessment of one's current condition” (p. 148). In contrast, LeDoux, 2015 finds that emotions and conscious feelings are “so intricately entwined” that he uses these terms interchangeably as “the mental states that people experience when they face situations in which survival is challenged or enhanced” (p. 98). Confusion in the terminology of affective states creates difficulties for researchers (see also: Russell and Carroll, 1999). However, it is not our aim to determine the precise definition of feelings, because this subject should be the focus of a separate paper. Our intention in this work is to draw attention to the issues related to understanding and using notions related to emotional reactions. In our study, work-related feelings are used as a broad term related to all affective states, i.e., emotions, moods, and affects referring to the workplace and work-related situations. Feelings are regarded here as subjective experiences and include cognitive elements (see: Schwarz, 2012.

The anger emotion is explicitly associated with fear and anxiety (Lazarus and Cohen-Charash, 2001; see also: discussion); however, in the discrete emotions approach, anger is regarded distinctly as an affective state. Russell (1980) assigns anger to the arousal category, which is opposite to sleepiness, and thus resides between tension characterized by low pleasantness and alertness linked with high pleasantness (both affective states are high activation). In general, anger is regarded as a negative emotion, meaning that anger can trigger destructive behaviors (Neumann and Baron, 1998; Gibson and Callister, 2009; Fitness, 2000). Some scholars regard anger as a powerful emotion (Fischer et al., 2004; Ragins and Winkel, 2011), which is particularly relevant in organizational settings. The environment in a workplace seems to contribute significantly to the elicitation of anger, likely due to its hierarchical structure, due to the inequity of power and the conflicting interests of superiors, subordinates, and clients.

3.3. The concept of sadness Of the emotions studied here, sadness seems to be the least related to occupational settings (see: Lazarus and Cohen-Charash, 2001). However, sadness and related feelings, such as dejection, hopelessness, and unhappiness, are associated with depression, which is the leading cause of disability worldwide (Friedrich, 2017). Some studies have indicated that one of the sources of depressive disorders/symptoms can be the work environment (Theorell et al., 2015) or prolonged job stress (Levi, 2005; Rugulies et al., 2006; Bonde, 2008). In our study, we consider the feeling of dejection, which is related to, but stronger than, sadness. The members of the emotion family of sadness are characterized by low activation and unpleasantness (Feldman Barrett and Russell, 1999; Warr et al., 2014). Feelings in this emotion family may be the result of “having experienced an irrevocable loss” (Lazarus, 2000, p. 234) or separation. In the workplace, sadness and dejection may come from frequent failures at work or poor relationships with peers, both of which may adversely affect job performance because of poor concentration or lowering of the individual's mood, self-esteem, motivation, etc.

3. Work-related feelings In this study, we concentrate on work-related feelings associated with four emotions: happiness (joy), fear, anger, and sadness. Of all emotions that people experience, these are the most common and, as a result, are regarded by scholars as primary (core) emotions (e.g., Kemper, 1987, 2006; see also: Scherer et al., 1986) among the basic emotions (see: Plutchik, 2001; Lazarus, 1991; Oatley and JohnsonLaird, 2014). In addition, the emotions selected herein are components of more complex psychological constructs related to the professional life, which is important with respect to the measurement instruments developed for the study of individuals’ emotions in the work environment. For example, burnout syndrome is related to depressive symptoms and disorders (e.g., Ahola et al., 2005, 2014), which are commonly associated with sadness and dejection. Some studies indicate that workers who exhibit burnout syndrome demonstrate anger and hostile and aggressive behaviors (e.g., Ersoy-Kart, 2009; Fiksenbaum et al., 2006; Muscatello et al., 2006; Queirós et al., 2013). Job stress is related to anxiety and tension. According to a 2014 report by EU-OSHA, because of costs at the societal, organizational, sectorial, and psychosocial levels, work-related stress is one of the most serious problems encountered by contemporary employees. Happiness is part of work-related well-being desired in organizational settings, and includes employee-reported job satisfaction, work engagement, and affective organizational commitment (e.g., Fisher, 2010).

3.4. The concept of happiness Happiness is related to emotions such as joy, excitement, enthusiasm, and contentment. These emotions are components of job satisfaction and work engagement (Bakker and Oerlemans, 2011; Schaufeli and Salanova, 2013) and are probably the most frequently investigated and analyzed positive states related to the work environment. Work-related happiness can be regarded as job satisfaction, experiencing positive emotions at work (the hedonic approach), or as selfactualization, which includes a sense of autonomy and experiencing the flow (the eudaimonic approach) (see: Fisher, 2010). In this study, workrelated happiness was defined as general contentment with one's job and having a positive attitude toward co-workers, supervisors, and professional tasks. An employee who feels a high level of happiness associated with his or her job has a feeling of job satisfaction and accomplishment. Such an understanding of the concept of happiness in relation to the organizational setting is closer to the hedonic approach than the eudaimonic approach. The emotional reactions of the individual depend, to a large extent, on that person's personality. This fact was taken into consideration with regard to verifying one of the hypotheses concerning the congruent validity of the WORAF (the Work-Related Affective Feelings Scale). However, it must be stressed that work-related feelings are mainly regarded as a result of external factors associated with the work environment and job-related tasks and duties.

3.1. The concept of fear Fear belongs to the same group of affective states as anxiety, tension, unease, and being afraid, which characterize high activation and unpleasantness (Feldman Barrett and Russell, 1999; Warr et al., 2014). Because some scholars regard anxiety as “the major emotional consequences of organizational life” (Lazarus and Cohen-Charash, 2001, p.66), in this study, we have decided to concentrate on anxiety rather than fear, which reflects the name of the scale. Feelings of anxiety can be a response to uncertain, existential threats (Lazarus, 2000), anticipated undesirable events (Ortony et al., 1988; in Kiefer, 2002), or danger. In the workplace, an existential threat is usually sensed in the face of organizational changes (Kiefer, 2002), risk of job loss or not receiving a bonus, demotion, or deterioration of the working conditions (see: Lazarus and Cohen-Charash, 2001), all of which can result in job stress.

2

Applied Ergonomics 82 (2020) 102945

M.A. Jaworek, et al.

4. Work-related affective feelings: measurement

was to establish a pool of items that corresponded to particular emotions. Items on the scale were created by subject matter experts and have been widely used in other studies on the development of questionnaire measurement instruments (e.g., Hardesty and Bearden, 2004; Ross et al., 2006). The team of experts consisted of seven people: one with the title of Professor, four with Ph.D. degrees in psychology or management, and two Ph.D. students with M.A. degrees in management. All evaluators were practitioners with extensive job experience in diverse organizational environments: small and big companies, corporations, and the private and public sectors. In five cases, the evaluators were also academics. Before work on item development began, each of the experts was provided with the definitions of the four emotions of interest as derived from two dictionaries of psychology and one Polish language dictionary (VandenBos, 2007; Reber, 2000; Dubisz, 2006). It was necessary for all statements (claims) for use on the scale to pertain to one of the four affective states (happiness, dejection, anger, and fear) related to job situations. Otherwise, no restrictions were applied to the form of the items. The questionnaire statements were developed by the authors based on dictionary definitions, academic knowledge, and work experience of the expert's team. Each statement was thoroughly examined by every member of the team. To ensure that all experts understood the items in the same way, each statement was discussed, and, if necessary, modified. Some groups of items were very similar, so it was decided to reduce them to a single statement that was considered the most adequate in the meaning and form by most of the members of the team. Finally, 28 of 72 items were chosen for the first version of the measurement instrument. Answers to question: Taking into account the last month, how often do you feel at work in a way described below? were scored on a 4-point rating scale, ranging from 1 (almost never) to 4 (almost always). In Study 1, which was the pilot study, the version containing 28 items (WORAF-28) with questions about demographic data was used. Our intention was to collect data from subjects representing various professions employed in big and small companies, in the public and private sectors. Therefore, the only constraint while collecting data was the professional activity of the subjects. Data were gathered by 3 students pursuing M.A. degrees in either psychology or management. Each of the students participated in a research methodology course and was additionally instructed by a member of the research team on the basic rules and experimental protocols. Information regarding the confidentiality of the collected data, as well as protecting the subjects’ identity and assuring their voluntary participation in the study, was also provided. All data were collected using the paper-and-pencil method.

The investigation of affective reactions requires adequate measurement tools. The most common methods for assessing affective states entail paper-and-pencil instruments, of which the most popular among researchers are probably the PANAS scale and the POMS (respectively: Positive and Negative Affect Schedule and Profile of Mood States; see: Ekkekakis, 2012). Some of these measurement instruments, such as the Job-Related Affective Well-Being Scale (JAWS; Van Katwyk et al., 2000) or Job Affective Scale (JAS; Brief et al., 1988), were developed specifically to evaluate emotional states at work. The JAS was constructed on the basis of the Watson & Tallegen concept of affect (1985), which is defined in relation to two dimensions of valence (i.e., positive and negative affect). In contrast, the theoretical background of the JAWS was Russell's circumplex model of affect (1980), described by valence and activation. These scales are useful for examining general affective states, when energy and/or pleasure-displeasure are key issues, but not discrete emotions. However, using the dimensional approach to study affective states can sometimes prove insufficient. For example, according to Russell's model, different emotions such as fear and anger are assigned to the same type of affect, described by high activation and unpleasantness (see: Russell, 1980). Different emotions are indeed associated with different behaviors and reactions (Raghunathan and Pham, 1999; Raghunathan et al., 2006; Izard and Ackerman, 2000) and may lead to distinct psychological and/or organizational consequences (see also: Ashkanasy and Dorris, 2017), which in some cases may be crucial for researchers. For this reason, we decided to develop a new instrument to evaluate affective states at work, which we have named the Work-Related Affective Feelings Scale (WORAF). 5. Current research The current study was carried out in accordance with the recommendations of the Bioethics Committee at the Jagiellonian University in Krakow, Poland. The protocol was approved by the Bioethics Committee. All subjects provided written informed consent in accordance with the Declaration of Helsinki. First, an initial version of the scale with 28-items (WORAF-28) was developed. The theoretical structure was tested in Study 1, while Studies 2 and 3 tested factorial validity. Replication of the factor structure of the WORAF was considered as additional evidence for the validity of the measurement instrument (scale stability). The convergent and discriminant validity of the scale was verified in Study 3. In addition, in Study 2, the measurement invariance was tested. In all three studies, the consistency of the subscales were also tested. All statistical analyses were performed with IBM SPSS Statistics 24 and AMOS 24.

6.1.2. Participants The sample for Study 1 included 297 employees of whom 59.6% were women and 36.7% were men (5.2% did not report their gender). Subjects were employed in different organizations and institutions. The major occupational groups included in the sample were managers (13.1%), sales representatives (11.4%), and office workers in the private sector (10.8%). At least 39% of the sample subjects were whitecollar workers. Most participants of the Study 1 were employed in the current workplace between 3 and 5 years (26.6%), and of these, 27.9% in their current position from 3 to 5 years. The average work experience was 11.09 years (SD = 8.8). The participants worked on average 42.65 h per week (SD = 8.5).

6. Study 1 The goal of the first study was to develop the initial version of the WORAF and then to test this initial version in a pilot study. The factorial structure of the WORAF was verified, and the internal consistencies of the subscales were calculated. Consistent with the discreteemotions approach (e.g., Plutchik, 2001; Ekman, 1992, 1999; Lazarus, 1991; Izard and Ackerman, 2000), we assumed that the studied workrelated feelings (i.e., anxiety, dejection, anger, happiness) constitute separate constructs (Hypothesis 1). In this case, because the studied work-related feelings are regarded as distinct, albeit still related to each other, the verification of Hypothesis 1 is identical with testing the discriminant validity of the scale.

6.2. Results and conclusions While performing the exploratory factor analyses, the principal axes method was used. Because the method predicted that the studied workrelated feelings (as factors) might be inter-correlated, the oblimin (oblique) rotation was used (DeVellis, 2017). The results of the KaiserMeyer-Okin measure of sampling adequacy (KMO = 0.933), significant Bartlett's test of sphericity (χ2(378) = 4875.82; p < 0.0001), and anti-

6.1. Methods 6.1.1. Procedure and measurement Our first step to develop a scale for measuring work-related feelings 3

Applied Ergonomics 82 (2020) 102945

M.A. Jaworek, et al.

Table 1 Items, factor loading of the WORAF scale in study 1 (with EFA), and in study 2 and 3 (with CFA). Item wording

Study 1

Study 2

Study 3

EFA

CFA

CFA

0.72 0.72 0.77 0.71 0.66 0.73 0.59 0.67

0.67 0.58 0.76 0.53 0.61 0.65 0.54 0.65

0.81 0.80 0.77 0.72 0.67 0.59 0.63

0.80 0.86 0.83 0.79 0.69 0.53 0.72

0.78 0.78 0.82 0.53 0.45

0.73 0.83 0.82 0.74 0.30

0.77 0.74 0.68 0.64

0.75 0.74 0.74 0.60

1 WR FEELINGS OF ANXIETY I feel fear at work I feel that matters related to work are getting out of control, which makes me panic. What is happening at work, fills me with anxiety and makes me feel threatened. I'm thinking that on Monday I need to go to work I feel anxious. I have symptoms of anxiety and nervousness at work, and I'm not able to calm down. Actions taken by my co-workers and/or supervisors make me feel uncertain. I am concern that I won't be able to meet the work requirements. I feel uncertain at work. WR FEELINGS OF HAPPINESS I find my work enjoyable. My job brings me satisfaction. My job gives me a sense of fulfilment. I find contentment in my work. Overall I feel relaxed and free. I am happy with my relations with my superiors. I have a positive attitude toward the task and problems which I am facing at work. WR FEELINGS OF DEJECTION At work I feel like I reached the bottom. When it goes to my job, it cannot be worse. Most work related activities make me feel sad and useless. I don't see any career path in front of me I have a sense of being suspended from what is happening at work. WR FEELINGS OF ANGER Recently everything related to my work makes me angry. I find everything at work annoying. The tasks I am getting from my supervisor make me furious. There are moments when I feel very irritated.

2

3

4

0.71 0.64 0.60 0.59 0.52 0.48 0.47 0.43 0.89 0.82 0.70 0.69 0.63 0.54 0.44 −0.92 −0.81 −0.60 −0.34 −0.32 0.87 0.61 0.54 0.36

In Study 1, only factor loadings > 0.30 are shown.

In the face of good results of the EFA and satisfactory reliability values of the subscales (ranging between 0.82 and 0.90; see: Table 2), we chose to use the WORAF-24 version in the following study. Correlational analyses between WR feelings of happiness and the remaining subscales ranged between (r = −0.43) and (r = −0.53) and between the rest of the subscales from (r = 0.62) to (r = 0.72). Details are shown in Table 2.

image correlation matrix (all diagonal values were greater than 0.88) confirmed the high factorability of the sample and supported the use of a factor analysis in the development of this new scale. The results of the EFA of the WORAF-28 version revealed a fivefactor structure solution. Thus, we decided to exclude three items from the original version of the scale because of inconsistency with the main factor (all primarily assigned to WR feelings on the dejection scale), and low factor loading of the one item (0.32) belonged to WR feelings on the happiness scale. Next, the EFA was performed again, this time revealing a four-factor solution overlapping with the theoretical structure. Details are shown in Table 1. Four factors of the final 24-item scale (WORAF-24) explained 54.46% of the variance. The first factor, with 8 items, explained 39.88% of the variance and was named WR feelings of anxiety (e.g., ‘I am concerned that I won't be able to meet the requirements of my job’). The second factor, explaining 8.91% of the variance, was formed by 7 items and was named WR feelings of happiness (e.g., ‘I find my work enjoyable’). The next factor, with 5 items and named WR feelings of dejection, explained 3.17% of the variance (5 items, e.g., ‘Most work-related activities make me feel sad and useless’). The fourth, and final, factor explaining 2.49% of the variance with 4 items was named WR feelings of anger (4 items, e.g., ‘Recently everything related to my work makes me angry’).

7. Study 2 The purpose of Study 2 was to verify the results obtained in previous research with a revised version of the WORAF, as is recommended by some scholars (Wright et al., 2017). Thus, the factorial structure was tested with CFA, and the internal consistencies of the subscales were calculated again. In the CFA, the model parameters were estimated using the maximum likelihood function. The goodness of fit of the models was evaluated using the following criteria for fit indices: chisquare/df ratio (χ2/df) ≤ 5.0 (Wheaton et al., 1977; in: Hooper et al., 2008), CFI (comparative fit index), TLI (Tucker-Lewis index), IFI (incremental fit index) ≥ 0.90, RMSEA (root mean squared error of approximation) ≤ 0.08, and SRMR (standardized root mean squared residual) ≤ 0.08 (Byrne, 2010). In addition, the robustness of the WORAF was assessed with an invariance test.

Table 2 Means, standard deviations, Cronbach's alphas, and intercorrelations among subscales in Studies 1, 2, and 3.

1. 2. 3. 4.

WR WR WR WR

feelings feelings feelings feelings

of of of of

happiness anxiety dejection anger

M (SD)

Cronbach's alpha

1

2

3

2.59 1.88 1.72 2.01

0.88/0.88/0.90 0.90/0.88/0.83 0.83/0.80/0.78 0.82/0.80/0.80

– −0.51**/-0.44**/-0.54** −0.43**/-0.49**/-0.60** −0.53**/-0.48**/-0.67**

– 0.72**/0.68**/0.52** 0.72**/0.69**/0.56**

– 0.62**/0.62**/0.58**

(0.68)/2.67 (0.66)/2.77 (0.70) (0.70)/1.75 (0.59)/1.60 (0.48) (0.69)/1.60(0.58)/1.42 (0.51) (0.71)/1.87 (0.60)/1.80 (0.58)

**p < 0.01. 4

Applied Ergonomics 82 (2020) 102945

M.A. Jaworek, et al.

7.1. Methods

were replicated, to a large extent, by Study 2.

7.1.1. Procedures and measurements The procedure for data collection and measurement in Study 2 was almost the same as in Study 1. The data were collected by 34 students pursuing M.A. degrees in psychology or management, who were instructed in terms the research methodology and protocol just as the students in Study 1. However, this time, the data were gathered not only using the paper-and-pencil method (63%) but also via the Internet. In the latter case, an e-mail communication included a link to an online questionnaire. Detailed statistical analyses showed that these two groups were comparable in terms of gender and age, but only after excluding the youngest group (20–30). Nevertheless, the follow-up analyses were carried out using the entire population sample.

7.2.2. Invariance tests Invariance tests can provide information regarding whether a set of measured constructs is comparable across the given groups. In this work, the invariance test of the WORAF was examined using multigroup confirmatory factor analysis across two basic demographic factors: gender and age. To do so, the configural model in which all parameters were estimated without any equality constraints was run. Then, the metric invariance was tested by constraining the factor loadings between the observed items and the corresponding latent variable to be equal across the compared groups. In addition, the scalar invariance was tested by constraining the factor loadings, with the indicator intercepts required to be equal across the gender groups, as follows. The invariance of the WORAF across men and women was tested first. The model for the configural invariance of the measure was found to fit the data either well or very well. A comparison of the unconstrained model (model 1) and the model with the constrained factor loadings (model 2) showed an non-significant difference (Δχ2 = 19.82, df = 20, p = 0.469), which implied that the factor loadings were invariant across the gender groups. However, because the results did not support scalar invariance (model 3a: Δχ2 = 15.42, df = 4, p = 0.004), additional analyses were performed. Exclusion of WR feelings of the anxiety scale from the tested model (3b) produced a non-significant chisquare difference test value (Δχ2 = 0.83, df = 3, p = 0.81), which supported scalar invariance for three of the four scales of the WORAF. Detailed analyses showed that two items of the WR feelings scale exhibited invariance (model 3c: Δχ2 = 17.44, df = 10, p = 0.065). Invariance tests of the measure for age groups were performed second. The model for configural invariance of the WORAF fit the data well or very well, but a comparison of the unconstrained model (model 4) and the model with the constrained factor loadings (model 5) produced a significant chi-square difference test value (Δχ2 = 33.75, df = 20, p = 0.028). The next set of analyses revealed that after excluding one age group (20–30 years), the comparison between the two models (models 6 and 7) appeared to be non-significant (Δχ2 = 27.5, df = 20, p = 0.122). These results indicated that the factor loadings varied slightly between the youngest group and the rest of the age groups. In conclusion, these results confirmed both the same measurement pattern and psychological meanings for the latent constructs across gender and all age groups, excluding the youngest (Vandenberg and Lance, 2000). Scalar invariance analyses conducted in the gender groups showed that the three studied constructs (WR feelings of happiness, anger, and dejection) were measured in the same way in each group. In the case of WR feelings of the anxiety scale, only partial scalar invariance was supported. Details are provided in Table 4 below.

7.1.2. Participants The sample for Study 2 included 3019 Polish employees, of whom 63.6% were women and 31.2% were men (5.2% did not identify their gender). The subjects were employed in a variety of organizations and institutions. The major occupational groups included in the sample were teachers (19.3%), nurses (11.6%), and managers (10.6%). At least 36% of the sample was white collar workers. Approximately 29% were aged under 30, almost 34% between 31 and 40, 22% between 41 and 50, approximately 12% between 51 and 60, and 2% over 60 years. Most participants of the study 2 were employed with the current place of work for between 3 and 5 years (20.4%), and of these, 23.9% in their current position from 3 to 5 years. The average amount of work experience was 14.8 years (SD = 10.38). The participants worked on average 41.11 h per week (SD = 11.59). 7.2. Results and conclusions 7.2.1. Factorial validity and reliability analysis To confirm the factor structure of the measurement, confirmatory factor analyses were carried out. Analyses were conducted with a revised version of the WORAF-24. In addition, an analysis of the onefactor model (24-items) was conducted. In the case of the four-factor model, the CFI, TLI, and IFI indices ranged between 0.93 and 0.94, and the RMSEA and SRMR were small (respectively, 0.05 and 0.04), which indicates a good fit of the model to the data. Only one indicator (cmin/ df) was greater than the recommended value. All items were significantly loaded on latent variables, with coefficients ranging from 0.45 to 0.82. In the one-factor model, the fit indices were less than, or greater than the acceptable thresholds, depending on the indicator. All subscales yielded good reliability using Cronbach's alpha, with values ranging from 0.80 for WR feelings of anger and dejection to 0.88 for WR feelings of anxiety and happiness. The last scale was negatively related to the rest of the subscales with a range from (r = −0.43) to (r = −0.53). Moreover, WR feelings of anger, anxiety, and dejection were correlated with values between (r = 0.51) and (r = 0.63). Tables 3 and 4 present the most important statistical properties of the WORAF. The results of the confirmatory analysis, as well as the tests for internal consistency of the four subscales, confirmed the factorial validity and reliability of the WORAF-24 scale. Therefore, the results of Study 1

8. Study 3 The main goal of Study 3 was to verify the convergent validity of the scale. Due to the lack of instruments available based on the discrete emotions approach, we decided to use the measurement instrument based on the dimensional model of emotions. It was assumed that these two approaches to examining emotions, which are not in opposition to one another but complement each other, and their respective results should be convergent (see: hypotheses 2a,b and 3a-c). Because personality traits are associated with affective states, such traits were also included within the analyses (see: hypotheses 4a-e). For the purpose of convergent validity, partial correlation analyses with gender and age control variables were conducted between the WORAF scales, where the four groups of emotions are considered in the dimensional approach, as well as personality traits. In addition, the factorial structure with CFA and internal consistencies of the subscales were tested once more.

Table 3 Goodness-of-fit indexes for the WORAF scale in Studies 2 and 3. Study

Model

χ2

Df

χ2/df

CFI

TLI

IFI

RMSEA

SRMR

2

1-factor 24-items 24-items

11740.78 2226.28 573.39

434 246 246

27.05 9.05 2.33

0.75 0.94 0.90

0.75 0.93 0.89

0.75 0.94 0.90

0.09 0.05 0.07

0.08 0.04 0.07

3

χ /df – normed chi-square; CFI – comparative fit index; TLI – Tucker-Lewis Index; IFI – Incremental Fit Index; RMSEA – root-mean-square error of approximation; SRMR - standardized root mean squared residual. 2

5

Applied Ergonomics 82 (2020) 102945

M.A. Jaworek, et al.

Table 4 Invariance tests of the WORAF scale (Study 2). Model

χ2

df

Gender: men (n = 936) and women (n = 1931) Model 1 (unconstrained) 2697.76 492 Model 2 (factor loadings constrained) 2717.58 512 Model 3a (factor loading and the indicator intercepts constrained) 2797.44 532 1326.76 228 Model 3b (factor loading and the indicator intercepts constrained)a Model 3c (factor loading and the indicator intercepts constrained)b 2795.42 526 Age: 20–30 (n = 897), 31–40 (n = 1016), 41–50 (n = 658), 51–60 (n = 371), above 60 (n = 62) Model 4 (unconstrained) 4653.78 1440 Model 5 (factor loadings constrained) 4687.53 1460 Age: all age groups, except for 20- to 30-year-olds Model 6 (unconstrained) 3285.96 1116 Model 7 (factor loadings constrained) 3313.44 1136 a b

CFI

TLI

IFI

RMSEA

SRMR

0.94 0.94 0.93 0.95 0.93

0.93 0.93 0.93 0.95 0.93

0.94 0.94 0.93 0.95 0.93

0.04 0.04 0.04 0.04 0.04

0.05 0.05 0.05 0.05 0.05

0.91 0.91

0.91 0.91

0.91 0.91

0.03 0.03

0.06 0.06

0.91 0.91

0.91 0.92

0.91 0.91

0.03 0.03

0.05 0.05

Model without the WR feelings of anxiety scale. Model with WR feelings of happiness, anger, and dejection scales and two items from WR feelings of anxiety scale.

To examine the convergent validity of the WORAF, first, correlational analyses were performed with the four categories of emotions that were distinguished in the dimensional model of emotion (Russell, 1980, 2003; Van Katwyk et al., 2000). It was predicted that the WR feelings of happiness would be positively related with HPHA and HPLA (High pleasurable-High and Low arousal, respectively) and negatively, with LPLA and LPHA (Low pleasurable-High and Low arousal, respectively) (Hypothesis 2a). In contrast, positive correlations were predicted for WR feelings of anxiety, anger, and dejection with LPLA and LPHA, and negative, with HPHA and HPLA (Hypothesis 2b). However, because the subscales of the JAWS include items that are less congruent with the WR feelings studied here, it was also decided to abstract the items that are the most consistent with the WR feelings of anger, anxiety, and dejection. Thus, it was hypothesized that in comparison to the rest of the WR feelings, the decreasing order of correlation strength would be: anger relates to Anger_LPHA (Hypothesis 3a), anxiety to Anxiety_LPHA (Hypothesis 3b), and dejection to Dejection_LPLA (Hypothesis 3c). Verification of the convergent and discriminant validity of the WORAF was also performed by examining correlations between the work-related feelings and personality traits. According to the APA Dictionary of Psychology, personality is described as the configuration of characteristics and behaviors that comprises (among others) emotional patterns (VandenBos, 2007, p. 689). Thus, an individual's emotionality is part of his or her personality, which reveals itself regardless of that individual's profession or place of work. Although the WORAF measures affective states, which are not stable and may change over time, nonetheless such affective states are related, to an extent, to personality traits. Among other traits, neuroticism is the most associated with anxiety, angry hostility, and depression (John and Srivastava, 1999). Therefore, it was predicted that the WR feelings of anxiety, anger, and dejection would be negatively related, whereas the WR feelings of happiness would be positively related to emotional stability (Hypothesis 4a). Because low agreeableness refers to aggressive emotions and attitudes and physical aggression (e.g., Barlett and Anderson, 2012; Zajenkowski, 2017), we expected a negative relationship between agreeableness and WR feelings of anger (Hypothesis 4b). Extraversion is associated with positive emotions and activity (John and Srivastava, 1999). Thus, we also predicted extraversion to be positively related to WR feelings of happiness (Hypothesis 4c). Emotions of anxiety and fear are associated with withdrawal rather than seeking new experiences. Therefore, we expected a negative relationship between WR feelings of anxiety and openness to experience (Hypothesis 4d). Because conscientiousness seems to refer the least to emotionality, we predicted no significant correlations with any WR feelings (Hypothesis 4e), which was regarded as evidence for discriminant validity. Because work-related feelings are considered to be a consequence of one's work settings, we anticipated that predicted correlations with personality traits would not be strong.

8.1. Methods 8.1.1. Procedure and measurements Data were collected by two researchers using the paper-and-pencil method. The respondents were students of external studies, and the data were gathered before the start of classes. Every participant in the examination was informed about the confidentiality of the collected data and measures that would be taken to protect their identity and was assured of their voluntary participation. In the study, the additional measurement instruments used, other than the WORAF, included The Job-Related Affective Well-Being Scale (JAWS) (see Van Katwyk et al., 2000; Basińska et al., 2014) and the Ten Item Personality Inventory (TIPI) (see Gosling et al., 2003; Łaguna et al., 2014), both in Polish language adaptations. 8.1.1.1. The Job-Related Affective Well-Being Scale (JAWS). To measure the affective states related to a participant's job across the most recent 30 days, the Polish version of the 20-item JAWS was used. The structure of the JAWS is based on two dimensions: valence and arousal. This tool assesses 4 groups of emotions (each includes 5 items): High pleasurableHigh arousal (HPHA), High pleasurable-Low arousal (HPLA), Low pleasurable-High arousal (LPHA), and Low pleasurable-Low arousal (LPLA). However, because the subscales of the JAWS include items that are less congruent with the studied WR feelings, we decided to abstract the items that are the most consistent with the WR feelings of anger, anxiety, and dejection. The new subscales of the JAWS consist of the following items: Anxiety_LPHA (anxious, frightened), Anger_LPHA (angry, furious), and Dejection_LPLA (depressed, gloomy). Every item is scored on a 5-point scale ranging from 1 (never) to 5 (extremely often). 8.1.1.2. Ten Item Personality Inventory (TIPI). The Polish version of the TIPI scale was used to measure 5 personality domains: extraversion, agreeableness, conscientiousness, emotional stability, and openness to experience. The instrument consists of 10 items, and each item consists of two descriptors. Every item is rated on a 7-point scale ranging from 1 (disagree strongly) to 7 (agree strongly). 8.1.2. Participants The sample for Study 3 was somewhat different in comparison to the samples used in Study 1 and Study 2. All 284 respondents were recruited from external study programs (M.A. and B.A. programs) through the psychology and management departments from universities located in three different cities in Poland. The participants of the study were employed in different organizations and occupations, mostly as office workers in the private sector (20.8%), specialists (15.8%), and accountants/financial analysts (10.2%). At least 40% of the sample was white-collar workers, and all participants had 6

Applied Ergonomics 82 (2020) 102945

M.A. Jaworek, et al.

completed at least secondary education. The sample consisted of 70.1% women and 25.4% men (4.6% did not identify their gender). Almost 80% were aged less than 30, approximately 15% between 31 and 40, 4.6% between 41 and 50, and only one person between 51 and 60. Most participants of the Study 3 were employed with their current work place between 3 and 5 years (25.4%), and of these, 28.2% in their current position from 0 to 6 months. The average work experience was 5.9 years (SD = 5.55). The participants worked on average 41.12 h per week (SD = 9.9).

Table 5 Correlations among subscales of the WORAF scale, JAWS, and TIPI (controlled for age and gender; Study 3).

JAWS 1. HPHA 2. HPLA 3. LPHA 4. LPLA JAWS (modified version) 5. Anxiety_LPHA 6. Dejection_LPLA 7. Anger_LPHA TIPI 8. Extraversion 9. Agreeableness 10. Conscientiousness 11. Emotional stability 12. Openness to experience

8.2. Results and conclusions 8.2.1. Confirmatory factor analysis In the CFA, the model parameters were estimated using the maximum likelihood function. The results of the confirmatory factor analysis revealed a satisfactory model fit to the data (cmin/df = 2.33; CLI and IFI = 0.90; RMSEA and SRMR = 0.07). Only one indicator (TLI) achieved a value that was slightly less than the acceptable threshold (0.89). All items were significantly loaded on latent variables, with coefficients ranging from 0.30 to 0.86. In general, these results were close to the outcomes from Study 2, which supports scale stability.

WR feelings of happiness

WR feelings of anxiety

WR feelings of dejection

WR feelings of anger

0.72*** 0.76*** −0.55*** −0.65***

−0.41*** −0.56*** 0.65*** 0.55***

−0.46*** −0.52*** 0.47*** 0.59***

−0.57*** −0.66*** 0.72*** 0.69***

−0.44*** −0.58*** −0.54***

0.71*** 0.55*** 0.51***

0.33** 0.50** 0.42**

0.48*** 0.65*** 0.75***

0.23*** ns ns 0.34*** 0.19**

−0.21*** −0.14* ns −0.45*** −0.24***

ns ns −0.13* −0.23*** −0.14*

ns −0.18** ns −0.33*** −0.13*

*p < 0.05; **p < 0.01; ***p < 0.001. HPHA = High pleasurable-High arousal, HPLA = High pleasurable-Low arousal, LPHA = Low pleasurable-High arousal, LPLA = Low pleasurable-Low arousal.

8.2.2. Reliability analysis Tests of internal consistency using Cronbach's alpha for each of the factors (subscales) showed good reliability values ranging from 0.78 for WR feelings of dejection to 0.83 for WR feelings of anxiety. The WR feelings of happiness were negatively related to the rest of the subscales between (r = −0.54) and (r = −0.67), whereas correlations between WR feelings of anxiety, anger, and dejection ranged from (r = 0.52) to (r = 0.58).

and agreeableness (r = −0.14), and WR feelings of dejection with conscientiousness (r = −0.13). However, these correlations are weak and very weak and, in comparison to the rest of the results, are not very meaningful with regard to supporting or rejecting hypotheses with respect to the convergent validity of the scale.

8.2.3. Convergent validity Convergent validity determines whether a set of examined constructs is related to theoretically relevant constructs. Consistent with our expectations, we found that work-related feelings of happiness were significantly positively correlated with HPLA and HPHA (respectively, r = 0.76 and r = 0.72) and negatively, with LPHA and LPLA (respectively, r = −0.55 and r = −0.65). Opposite associations were observed for the relationships between work-related feelings of anxiety, anger, and dejection and the categories of emotions of the JAWS, and workrelated feelings of happiness. Thus, hypotheses 2a and b were fully confirmed. The direction and strength of the correlations between the WR feelings and congruent modified subscales were close to the comparison with relations developed using the JAWS full subscales (all 5 items). However, the strength of the correlations with the rest of the variables was weakened (see: Table 5). The strongest relationships were observed between the WR feelings of anger and Anger_LPHA (r = 0.75) and WR feelings of anxiety and Anxiety_LPHA (r = 0.71), which confirmed hypotheses 3a and b. In regard to WR feelings of dejection, among all of the new JAWS subscales, the strongest correlation was observed with Dejection_LPLA. However, if we compare correlations of Dejection_LPLA with all WR feelings, this relation appears to be the weakest (r = 0.50 in comparison to r = 0.55 for WR feelings of anxiety and r = 0.65 for WR feelings for anger). Thus, hypothesis 3c was not supported. In regard to personality domains and work-related feelings, the stated hypotheses (4a-e) were entirely confirmed. The strongest relationships were observed in the case of emotional stability: negative with WR feelings of anxiety (r = −0.45) and anger (r = −0.33), and positive with WR feelings of happiness (r = 0.34). The remaining predicted correlations were weaker, although significant, and consistent with expectations. No associations between conscientiousness and WR feelings of happiness, dejection, and anger were found, which was also in accordance with our hypothesis. Some unexpected relationships were observed, e.g., WR feelings of anxiety with extraversion (r = −0.21)

9. Discussion The main objective of the present study was to develop and perform an initial validation of a new instrument for measuring feelings related to work settings. The scale consisted of four independent subscales, which were used to assess: WR feelings of anxiety, anger, dejection, and happiness. These subscales could be used separately, depending on the goals of a given study. Overall, the series of tests conducted in this work provided significant evidence for the validity of the Work-Related Feelings Scale (WORAF), although further investigations are still needed (see: limitations of the study). Hypothesis 1, namely, that predicted work-related feelings constitute separate constructs, was proved to a large extent. Factor analyses conducted for items of the WORAF-24 in Studies 1, 2, and 3 fully confirmed the theoretical structure of the measurement instrument. Only in Study 3 were some inconsistencies, although tiny, observed. Confirmatory factor analyses supported the factorial structure of the WORAF-24. The reliability of the subscales achieved good and very good values across all three studies. The results of the factorial structure of the WORAF-24 version and consistency of the subscales were replicated in three independent studies, which may be regarded as further evidence for the validity of the measurement instrument (scale stability). Although we assumed that the studied WR feelings were distinct constructs, we did not exclude that at the same time, these feelings can be mutually related, especially regarding anxiety, anger, and dejection. These assumptions were consistent with common knowledge, the discrete-emotions approach, and other empirical studies. The results of this study support both assumptions. On the one hand, these are distinct constructs (with overlapping theoretical and empirical structures in the EFA), while on the other hand, these constructs are mutually related to each other (moderate correlations between constructs). The close relationship between anger, anxiety, and sadness has been confirmed by several scholars. For example, according to Lazarus and Cohen-Charash 7

Applied Ergonomics 82 (2020) 102945

M.A. Jaworek, et al.

(2001), anger and anxiety are “strikingly interdependent” (p. 65). Ekman (2003) wrote: “Anger is rarely felt alone for long. Fear often precedes and follows anger …” (p. 113). Some authors indicate that sadness and anger can also be experienced together (Izard and Ackerman, 2000; Barr-Zisowitz, 2000). The energy of anger counteracts the slowing effect of sadness, and in turn, sadness moderates the intensity of the anger (Blumberg & Izard, 1986; in Izard and Ackerman, 2000). The theoretical assumptions and results of our research are supported by empirical studies conducted through the dimensional approach, indicating that three of the work-related feelings studied are closely related (see: Russell, 1980; Van Katwyk et al., 2000; Basińska et al., 2014). Irritation at work, as a manifestation of anger, can be associated with feelings of uncertainty related to anxiety and/or emotions of dejection resulting from a sense of helplessness. In conclusion, moderate correlations between these three subscales do not stem from the weakness of the WORAF as a measurement instrument but rather from the inherent nature of the affective states examined. The WORAF, as a multiple-item measurement instrument, developed through the discrete emotions approach and, at the same time, related to the work setting, is a new instrument that meets the expectations of other scholars (e.g., Crispim et al., 2015; Ashkanasy and Dorris, 2017). Different affective states can lead to distinct behaviors and organizational consequences, such that the discrete emotions perspective can be more useful than the dimensional approach, when behavioral reactions as a result of the emotional states experienced are taken into consideration (see also the theories of Weiss and Cropanzano, 1996; Rafaeli and Sutton, 1989). Thus, this new scale can be used by researchers interested in expanding our knowledge of the psychological functioning of people in work settings and by those interested in studying all four or any one of the work-related feelings measured by the WORAF (the subscales are independent). The developed scale is applicable to professionally active people employed in the private sector, public institutions, and small businesses or corporations. However, because the measurement instrument was not tested among representatives of all professions, its application is currently limited to white-collar workers (also managers), teachers, health and welfare professionals, specialists (IT, HR, marketing), sales representatives, telesales operators, and similar professions. Our current study has several limitations. First, the WR feelings of dejection subscale received worse indicators than the rest of the scales. The theoretical structure was not fully replicated in all three studies, and convergent validity was only partially confirmed. Therefore, this subscale should be considered with particular attention in further validation studies. Second, test-retest reliability is needed to complete an assessment of the psychometric properties of the scale. Third, further research should be conducted to determine the predictive validity of the instrument. This would take into account work-related phenomena strictly associated with affective states like burnout syndrome, work, engagement, job stress, work satisfaction, etc. (see: Bakker and Oerlemans, 2011; Schaufeli and Salanova, 2013). Fourth, the WORAF includes only the selected affective states. Further investigations concerning the development of new scales to measure work-related feelings should also include guilt, shame, jealousy, hope, pride, compassion, or even love, as suggested by other theorists (see: Lazarus and CohenCharash, 2001). Fifth, none of the three samples, although each was slightly different from the others, contains representatives of all professions (e.g., the complete sample body lacks representatives employed in heavy industry, construction, mining, military, etc.), which must be taken into consideration, before using the WORAF in an applied context or replication of the present study's results. Finally, it should also be acknowledged that all participants were Polish, so the results cannot be generalized to include other nationalities, especially because the constructs studied can be subject to the influence of culture and social context (Fischer et al., 2004). Thus, further investigations with the WORAF scale, using working populations other than Polish employees,

are highly recommended. 10. Conclusions The broad majority of the stated hypotheses concerning verification of the validity of the WORAF were confirmed, and tests of the reliability of the subscales and invariance tests obtained satisfactory results. Overall, the current findings are promising and allow for further research to be conducted with the measurement instrument studied. The present scale may also be treated as a starting point to develop similar instruments to assess other affective states related to the specific workplace conditions. References Ahola, K., Honkonen, T., Isometsä, E., Kalimo, R., Nykyri, E., Aromaa, A., et al., 2005. The relationship between job-related burnout and depressive disorders – results from the Finnish Health 2000 Study. J. Affect. Disord. 88, 55–62. https://doi.org/10.1016/j. jad.2005.06.004. Ahola, K., Hakanaen, J., Perhoniemi, R., Mutanen, P., 2014. Relationship between burnout and depressive symptoms: a study using the person-centred approach. Burnout Res. 1 (1), 29–37. https://doi.org/10.1016/j.burn.2014.03.003. Ashforth, B.E., Humphrey, R.H., 1995. Emotion in the workplace: a reappraisal. Hum. Relat. 48 (2), 97–124. Ashkanasy, M.N., Dorris, A.D., 2017. Emotions in the workplace. Annu. Rev. Organ. Psychol. Organ. Behav. 4, 67–90. https://doi.org/10.1146/annurev-orgpsych032516-113231. Ashkanasy, N.M., Humphrey, R.H., 2011. Current emotion research in organizational behavior. Emotion Review 3, 214–230. https://doi.org/10.1037/1076-8998.5.2.219. Bakker, A.B., Oerlemans, W.G.M., 2011. Subjective well-being in the organization. In: Cameron, K., Spreitzer, G. (Eds.), The Oxford Handbook of Positive Organizational Scholarship. Oxford University Press, pp. 178–189. https://doi.org/10.1093/ oxfordhb/9780199734610.013.0014. Barlett, P.C., Anderson, C.A., 2012. Direct and indirect relations between the Big 5 personality traits and aggressive and violent behavior. Personal. Individ. Differ. 52, 870–875. https://doi.org/10.1016/j.paid.2012.01.029. Barr-Zisowitz, C., 2000. „Sadness” – is there such a thing? In: Lewis, M., Haviland-Jones, J.M. (Eds.), Handbook of Emotions. Guilford Press, New York, pp. 607–622. Barsade, S.G., Gibson, D., 2007. Why Does Affect Matter in Organizations? Academy of Management Perspectives, February, pp. 36–59. https://doi.org/10.5465/AMP.2007. 24286163. Barsade, S., Brief, A., Spataro, S., 2003. The affective revolution in organizational behavior: the emergence of a paradigm. In: Greenberg, J. (Ed.), Organizational Behavior: the State of the Science. Lawrence Erlbaum Associates, Publishers, London, pp. 3–52. Basińska, B.A., Gruszczyńska, E., Schaufeli, W.B., 2014. Psychometric properties of the Polish version of the job-related affective well-being scale. Int. J. Occup. Med. Environ. Health 27 (6), 993–1004. https://doi.org/10.2478/s13382-014-0329-x. Berkovitz, L., 2000. Causes and Consequences of Feelings. Cambridge University Press. Bonde, J.P., 2008. Psychosocial factors at work and risk of depression; a systematic review of the epidemiological evidence. Occup. Environ. Med. 65 (7), 438–445. https://doi.org/10.1136/oem.2007.038430. Brief, A.P., Burke, M.J., George, J.M., Robinson, B.S., Webster, J., 1988. Should negative affectivity remain an unmeasured variable in the study of job stress? J. Appl. Psychol. 73, 193–198. Byrne, B.M., 2010. Structural Equation Modeling with AMOS. Basic Concepts, Applications, and Programming, second ed. Routledge Taylor & Francis Group, New York, London. Crispim, A.C., Moraes Cruz, R., Baasch, D., Amorim, L., Trevisan, R.L., Da Silva, M.A., 2015. Measurement of affect: from theoretical and instrumental perspectives. Psychol. Res. 5 (2), 96–107. https://doi.org/10.17265/2159-5542/2015.02.002. DeVellis, R.F., 2017. Scale Development. Theory and Applications. Sage, Los Angeles. Dubisz, S. (Ed.), 2006. Uniwersalny Słownik Języka Polskiego (The Universal Dictionary of Polish). Wydawnictwo Naukowe PWN, Warsaw. Eccles, D.W., Ward, P., Woodman, T., Janelle, C.M., Le Scanff, C., Ehrlinger, J., Costanier, C., Coombes, S.A., 2011. Where's the emotion? How sport psychology can inform research on emotion in human factors. Hum. Factors: J. Hum. Factors Ergon. Soc. 53, 180–202. https://doi.org/10.1177/0018720811403731. Ekkekakis, P., 2012. Affect, mood, and emotion. In: Tanenbaum, G., Eklund, R.C., Kamata, A. (Eds.), Measurement in Sport and Exercise Psychology. Human Kinetics, pp. 321–332. Ekman, P., 1992. Are the e basic motions? Psychol. Rev. 99 (3), 550–553. Ekman, P., 1994. Moods, Emotions, and Traits. In: Ekman, P., Davidson, R.J. (Eds.), The Nature of Emotion. Fundamental Questions. Oxford University Press, pp. 56–58. Ekman, P., 1999. Basic emotions. In: Dalgleish, T., Power, M. (Eds.), Handbook of Cognition and Emotion. John Wiley and Sons Ltd., pp. 45–60. Ekman, P., 2003. Emotions Revealed. Recognizing Faces and Feelings to Improve Communication and Emotional Life. Times Books, Henry Holt and Company, New York. Ersoy-Kart, M., 2009. Relations among social support, burnout, and experience of anger: an investigation among emergency nurses. Nurs. Forum 44 (3), 165–174. https://doi.

8

Applied Ergonomics 82 (2020) 102945

M.A. Jaworek, et al.

21:7<801::AID-JOB999>3.0.CO;2-A. Muscatello, M.R.A., Bruno, A., Carioccio, C., Cedro, C., La Torre, D., Rosa, A.E., Zoccali, R., et al., 2006. Association between burnout and anger in oncology versus ophthalmology health care professionals. Psychol. Rep. 99, 641–650. https://doi.org/10. 2466/pr0.99.2.641-650. Neumann, J.H., Baron, R.A., 1998. Workplace Violence and Workplace Aggression: Evidence Concerning Specific Forms, Potential Causes, and Preferred Targets. Journal of Management 24 (3), 391–419. Oatley, K., Johnson-Laird, P.N., 2014. Cognitive approaches to emotions. Trends Cogn. Sci. 18 (3), 134–140. https://doi.org/10.1016/j.tics.2013.12.004. Plutchik, R., 2001. The nature of emotions. Am. Sci. 89, 344–350. Queirós, C., Kaiseler, M., da Silva, A.L., 2013. Burnout as predictor of aggressivity among police officers. Eur. J. Polic. Stud. 1 (2), 110–135. Rafaeli, A., Sutton, R.I., 1989. The expression of emotion in organizational life. Res. Organ. Behav. 11, 1–42. Raghunathan, R., Pham, M.T., 1999. All negative moods are not equal: motivational influences of anxiety and sadness on decision making. Organ. Behav. Hum. Decis. Process. 79 (1), 56–77. Raghunathan, R., Pham, M.T., Corfman, K.P., 2006. Informational properties of anxiety and sadness, and displaced coping. J. Consum. Res. 32, 596–601. Ragins, B.R., Winkel, D.,E., 2011. Gender, emotion and power in work relationships. Hum. Resour. Manag. Rev. 21, 377–393. https://doi.org/10.1016/j.hrmr.2011.05. 001. Reber, A.S., 2000. Słownik Psychologii (Dictionary of Psychology). Scholar, Warsaw. Ross, S.D., James, J.D., Vargas, P., 2006. Development of a scale to measure team brand associations in professional sport. J. Sport Manag. 20, 260–279. https://doi.org/10. 1123/jsm.20.2.260. Rugulies, R., Bultmann, U., Burr, H., 2006. Psychosocial work environment and incidence of severe depressive symptoms: prospective findings from a 5-year follow-up of the Danish work environment cohort study. Am. J. Epidemiol. 163 (10), 877–887. https://doi.org/10.1093/aje/kwj119. Russell, J.A., 1980. A circumplex model of affect. J. Personal. Soc. Psychol. 39 (6), 1161–1178. Russell, J.A., 2003. Core affect and the psychological construction of emotion. Psychol. Rev. 110 (1), 145–172. https://doi.org/10.1037/0033-295X.110.1.145. Russell, J.A., Carroll, J.M., 1999. On the bipolarity of positive and negative affect. Psychological Bulletin 125 (1), 3–30. https://doi.org/10.1037/0033-2909.125.1.3. Schaufeli, W.B., Salanova, M., 2013. Burnout, boredom and engagement in the workplace. In: Peeters, M.C.W., de Jonge, J., Taris, T.W. (Eds.), An Introduction to Contemporary Work Psychology. John Wiley and Sons, New York, pp. 293–320. Scherer, K., Wallbott, H., Summerfield, A., 1986. Experiencing Emotion: A Cross-Cultural Study. Cambgidge University Press, Cambridge. Schwarz, N., 2012. Feelings-as-information theory. In: Van Lange, P.A.M., Kruglanski, A.W., Higgins, E.T. (Eds.), Handbook of theories of social psychology. Thousand Oaks, CA: Sage Publications Ltd, pp. 289–308. Thayer, R.E., 2012. Moods of energy and tension that motivate. In: Ryan, R.M. (Ed.), The Oxford handbook of human motivation. Oxford University Press, pp. 408–419. Theorell, T., Hammarström, A., Aronsson, G., Träskman Bendz, L., Grape, T., Hogstedt, C., Marteinsdottir, I., Skoog, I., Hall, C., 2015. A systematic review including metaanalysis of work environment and depressive symptoms. BMC Public Health 15, 738. https://doi.org/10.1186/s12889-015-1954-4. Van Katwyk, P.T., Fox, S., Spector, P.E., Kelloway, E.K., 2000. Using the job-related affective well-being scale (JAWS) to investigate affective responses to work stressors. J. Occup. Health Psychol. 5 (2), 219–230. https://doi.org/10.1037//1076-8998.5.2. 219. Vandenberg, R.J., Lance, C.E., 2000. A review and synthesis of the measurement invariance literature: suggestions, Practices, and recommendations for organizational research. Organ. Res. Methods 3 (1), 4–70. https://doi.org/10.1177/ 109442810031002. VandenBos, G.R. (Ed.), 2007. APA Dictionary of Psychology. American Psychological Association, Washington, DC. Warr, P., Bindl, U.K., Parker, S.K., Inceoglu, I., 2014. Four-quadrant investigation of jobrelated affects and behaviours. Eur. J. Work Organ. Psychol. 23 (3), 342–363. https://doi.org/10.1080/1359432X.2012.744449. Watson, D., Tellegen, A., 1985. Toward a consensual structure of mood. Psychol. Bull. 98, 219–235. Weiss, H.M., Cropanzano, R., 1996. Affective events theory: a theoretical discussion of the structure, causes and consequences of affective experiences at work. Res. Organ. Behav. 18, 1–74. Wright, T.A., Quick, J.C., Hannah, S.,T., Hargrove, M.B., 2017. Best practice recommendations for scale construction in organizational research: the Development and initial validation of the Character Strength Inventory (CSI). J. Organ. Behav. 38, 615–628. https://doi.org/10.1002/job.2180. Zajenkowski, M., 2017. Hostile and energetic: anger is predicted by low agreeableness and high energetic arousal. PLoS One 20 (9), e0184919. https://doi.org/10.1371/ joural.pone.0184919. Łaguna, M., Bąk, W., Purc, E., Mielniczuk, E., Oleś, P.K., 2014. Krótki inwentarz osobowości TIPI-P w badaniach polskich. Ann. Psychol. XVII (2), 403–419.

org/10.1111/j.1744-6198.2009.00139.x. European Agency for Safety and Health at Work, 2014. Calculating the Costs of WorkRelated Stress and Psychosocial Risks. European Risk Observatory Literature Review.. Report retrieved from EU-OSHA Website. https://osha.europa.eu/en/tools-andpublications/publications/literature_reviews/calculating-the-cost-of-work-relatedstress-and-psychosocial-risks, Accessed date: 4 September 2018 10.2802/20493. Feldman Barrett, L., Russell, J., 1999. The structure of current affect: controversies and emerging consensus. Curr. Dir. Psychol. Sci. 8 (1), 10–14. Fiksenbaum, L., Marjanovic, Z., Greenglass, E.R., Coffey, S., 2006. Emotional exhaustion and state anger in nurses who worked during the sars outbreak: the role of perceived threat and organizational support. Can. J. Community Ment. Health 25 (2), 89–1003. https://doi.org/10.7870/cjcmh-2006-0015. Fischer, A., Rodriguez Mosquera, P.M., van Vianen, A.E.M., Manstead, A.S.R., 2004. Gender and culture differences in emotion. Emotion 4 (1), 87–94. https://doi.org/10. 1037/1528-3542.4.1.87. Fisher, C., 1997, June. Emotions at work: what do people feel and how should we measure it? In: Paper Presented at the Second Biennial Australian Industrial and Organisational Psychology Conference, Melbourne, . https://epublications.bond. edu.au/cgi/viewcontent.cgi?referer=https://www.google.com/&httpsredir=1& article=1000&context=business_pubs, Accessed date: 4 September 2018. Fisher, C.D., 2000. Mood and emotions while working: missing pieces of job satisfaction? J. Organ. Behav. 21 (2), 185–202. https://doi.org/10.1002/(SICI)10991379(200003)21:2<185::AID-JOB34>3.0.CO;2-M. Fisher, D.C., 2010. Happiness at work. Int. J. Manag. Rev. 12, 384–412. https://doi.org/ 10.1111/j.1468-2370.2009.00270.x. Fitness, J., 2000. Anger in the workplace: an emotion script approach to anger episodes between workers and their superiors, co-workers and subordinates. J. Organ. Behav. 21, 147–162. https://doi.org/10.1002/(SICI)1099-1379(200003)21:2<147::AIDJOB35>3.0.CO;2-T. Friedrich, M.J., 2017. Depression is the leading cause of disability around the world. The Journal of the American Medical Association 317 (15). https://doi.org/10.1001/ jama.2017.3826. Frijda, N., 1994. Varieties of Affect: Emotions and Episodes, Moods, and Sentiments. The Nature of Emotion. Fundamental Questions. Oxford University Press, pp. 59–67. Gibson, D.E., Callister, R., 2009. Anger in organizations: review and future directions. In: Paper Presented at 22nd Annual International Association of Conflict Management Conference, Japan, Kyoto. Electronic Copy Retrieved from Social Science Research Network, . http://ssrn.com/abstract=1484915, Accessed date: 4 September 2018. Gosling, S.D., Rentfrow, P.J., Swann, W.B., 2003. A very brief measure of the Big-Five personality domains. J. Res. Personal. 37, 504–528. Hardesty, D.M., Bearden, W.O., 2004. The use of expert judges in scale development. Implications for improving face validity of measures of unobservable constructs. J. Bus. Res. 57, 98–107. https://doi.org/10.1016/S0148-2963(01)00295-8. Hooper, D., Coughlan, J., Mullen, M., 2008. Structural equation modelling: guidelines for determining model fit. Electron. J. Bus. Res. Methods 6 (1), 53–60. www.ejbrm.com. Izard, C.E., Ackerman, B.P., 2000. Motivational, organizational, and regulatory functions of discrete emotions. In: Lewis, M., Haviland-Jones, J.M. (Eds.), Handbook of Emotions. The Guilford Press, New York, London, pp. 253–264. John, O.P., Srivastava, S., 1999. The big-five trait taxonomy: history, measurement, and theoretical perspectives. In: Pervin, L., John, O.P. (Eds.), Handbook of Personality: Theory and Research. Guilford Press, New York, pp. 102–138. Kemper, T., 1987. How many emotions are there? Wedding the social and the automatic components. Am. J. Sociol. 93 (2), 263–289. Kemper, T.D., 2006. Power and status and the power-status theory of emotions. In: Stets, J.E., Turner, J.H. (Eds.), Handbook of the Sociology of Emotions. Handbooks of Sociology and Social Research. Springer, Boston, MA, pp. 87–113. https://doi.org/ 10.1007/978-0-387-30715-2_5. Kiefer, T., 2002. Analyzing emotions for a better understanding of organizational change: fear, joy, and anger during a merger. In: Ashkanasy, N.M., Zerbe, W.J., Härtel, ChE.J. (Eds.), Managing Emotions in the Workplace. Routledge Taylor and Francis Group, London and New York, pp. 45–69. Larsen, R.J., Diener, E., 1992. Promises and problems with circumplex model of emotion. Review of Personality and Social Psychology 13, 25–59. Lazarus, R., 1991. Progress on a cognitive-motivational-relational theory of emotion. American Psychologist 46 (8), 819–834. https://doi.org/10.1037/0003-066X.46.8. 819. Lazarus, R., 2000. How Emotions Influence Performance in Competitive Sports. The Sport Psychologist 14 (3), 229–252. https://doi.org/10.1123/tsp.14.3.229. Lazarus, R., Cohen-Charash, Y., 2001. Descrete emotions in organizational life. In: Payne, R.L., Cooper, C.L. (Eds.), Emotions at Work. Theory, Research and Applications for Management. John Wiley and Sons Ltd., pp. 45–81. LeDoux, J.E., 2015. Feelings: What Are They and How Does the Brain Make Them? Journal of the American Academy of Arts & Sciences 144 (1), 96–111. https://doi. org/10.1162/DAED_a_00319. Levi, L., 2005. Working life and mental health – a challange to psychiatry? World Psychiatry 4 (1), 53–57. Muchinsky, P.M., 2000. Emotions in the workplace: the neglect of organizational behavior. J. Organ. Behav. 21, 801–805. https://doi.org/10.1002/1099-1379(200011)

9