Dealing with clients’ diversity in test situations: Client categorisations in psychologists’ accounts of their practices

International Journal of Educational Research 63 (2014) 15–25

Contents lists available at SciVerse ScienceDirect

International Journal of Educational Research journal homepage: www.elsevier.com/locate/ijedures

Dealing with clients’ diversity in test situations: Client categorisations in psychologists’ accounts of their practices Miche`le Grossen *, Douchka Florez, Ste´phanie Lauvergeon 1 Department of Psychology, University of Lausanne, Switzerland

A R T I C L E I N F O

A B S T R A C T

Article history: Received 16 June 2012 Accepted 25 September 2012 Available online 22 November 2012

Reducing a test administration to standardised procedures reflects the test designers’ standpoint. However, from the practitioners’ standpoint, each client is unique. How do psychologists deal with both standardised test administration and clients’ diversity? To answer this question, we interviewed 17 psychologists working in three public services for children and adolescents about their assessment practices. We analysed the ‘‘client categorisations’’ they produced in their accounts. We found that they had shared perceptions about their clients’ diversity, and reported various non-standard practices that complemented standardised test administration, but also differed from them or were even forbidden. They seem to experience a dilemma between: (a) prescribed and situated practices; (b) scientific and situated reliability; (c) commutative and distributive justice. For practitioners, dealing with clients’ diversity is a practical problem, halfway between a problem-solving task and a moral dilemma. ß 2012 Elsevier Ltd. All rights reserved.

Keywords: Test use Categories Situated action Socio-cultural psychology Reliability Trust

1. Introduction The use of psychological tests to assess cognitive or emotional abilities is a widespread institutional practice in many segments of our society. As an assessment instrument, a test is constructed through a standardisation procedure, enabling comparison to be made between an individual and a reference population. Various prescribed procedures regulate the use of a test (administration of the test, instructions to testees, scoring, etc.), which guarantee that the results are not dependent on the tester’s personal practice and, more generally, on the situation in which a test has been done. Thus, the tester has to be neutral, as if he or she were interchangeable. Both the standardisation of the test design and the prescription of fixed procedures aim at ensuring the reliability of a test, that is ‘‘the agreement between two efforts to measure the same trait through maximally similar methods’’ (Campbell & Fiske, 1959, p. 83), and, hence, its validity. Now, one could adopt a different stance, and approach test practices from the standpoint of work psychology and ergonomics. In fact, based on observations of men and women in their workplace, these scientific disciplines have long ago shown that work consists of adapting prescribed procedures to concrete working conditions (Clot, 1999). In reality, no work prescription is able to take into account or foresee all the particularities of a concrete work situation. Consequently, work as it should be carried out (prescribed work) never totally coincides with work as it is actually carried out (observed work). This

* Corresponding author at: Department of Psychology, University of Lausanne, Anthropole, CH-1015 Lausanne, Switzerland. Tel.: +41 21 692 32 60; fax: +41 21 692 32 65. E-mail address: [email protected] (M. Grossen). 1 Now at: Institut Universitaire de Me´decine Sociale et Pre´ventive, CHUV, Lausanne, Switzerland. 0883-0355/$ – see front matter ß 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ijer.2012.09.003

16

M. Grossen et al. / International Journal of Educational Research 63 (2014) 15–25

gap, far from being an abnormal deviation or a professional mistake, testifies to workers’ abilities to contextualise their actions. Put differently, it is a sign indicating the situatedness of cognition (Resnick, Sa¨ljo¨, Pontecorvo, & Burge, 1997). Unfortunately, assessment practices in test situations have not given rise to much research, with some exceptions, however, in sociology and psychology. With a lens close to that of work psychology, Roth (1974) and later Mehan, Hertweck, and Meihls (1986) have focused on assessment practices in a well-known intelligence test, the Wechsler Intelligence Scale for Children (WISC). Through fine-grained analyses of the psychologist-child interactions, they found that the testers adjusted their actions according to the child’s behaviour and answers, and were not as neutral as required. Their results showed that tester–testee interactions are not simply a variable that should be controlled, but constitute the very process through which a child provides a response. In developmental psychology, similar observations have been made in socio-cultural studies to development and cognition. Many scholars, inspired by Ragnar Rommetveit’s criticisms (1978) of the Piagetian theory, have shown that psychological assessment is a complex communicative situation in which the tester and the testee interpret each other’s intentions, discourse and actions (e.g., Aronsson & Hundeide, 2002; Donaldson, 1978; Elbers & Kelderman, 1994; Sa¨ljo¨, 1991). A tester, far from being totally neutral, provides many clues that orient the child’s actions and enable him/her to interpret the situation and task. Hence, answering a test requires intensive interactional work and cannot be disentangled from the social conditions in which the answer is provided (Grossen & Perret-Clermont, 1994). All these studies converge towards the same conclusion: reducing the administration of a test to a set of prescribed procedures reflects the standpoint of test designers who seek to construct reliable and valid tests. However, from the standpoint of practitioners who deal with ‘‘flesh and blood’’ clients, there are never two strictly identical situations and each client is unique. Hence their problem is, precisely, how to deal with the clients’ diversity. We may then ask: How do psychologists deal with both the prescription of neutrality and with clients’ diversity? At a methodological level, we tried to answer this question by using research interviews in which we questioned psychologists on their assessment practices. Of course, discourse on practice is not practice itself, but our interviews did inform us about what practices are relevant, or not, from the practitioners’ point of view. As Bruner (1990) put it: ‘‘A culturally sensitive psychology [. . .] is and must be based not only on what people actually do but what they say they do and what they say caused them to do what they did’’ (p. 16). According to him, what individuals say about their acts informs us about the meaning they give to their acts, and this meaning is related to the cultural and social contexts in which the act takes place. In analysing these interviews, we chose to focus on what we called ‘‘client categorisations’’, that is, formulations that referred to a client, such as ‘‘hyper-anxious child’’. In fact, a first approach to the data showed that these client categorisations were very frequent, and that psychologists seemed to evoke them in order to qualify their reports of generic practices. An analysis of client categorisations therefore seemed to be a relevant approach in investigating the way in which psychologists dealt with their clients’ diversity. 2. Categorisations in professional practice Categorisation is a research object that pertains to various disciplines and theoretical strands. In sociology, it is linked to labelling theory, which may for example be applied in organisational behaviour (e.g., Ashforth & Humphrey, 1997). In this field, Rosenthal and Peccei (2006) analysed how service agents in the British welfare administration categorised their clients when they referred to them in a research interview. They found that these categorisations of clients were related to the advisers’ job characteristics and roles, as well as to the tasks they were expected to carry out. They also found that these categorisations were not free from moral values and tended to valorise clients who facilitated the advisers’ job and enabled them to meet the requirements of the work organisation of their service. In a totally different theoretical tradition, categorisation also pertains to the field of Conversation Analysis, in which the notion of Membership Categorisation and its corresponding method of analysis (Membership Categorisation Analysis) have been developed in parallel to sequential analysis of conversation (Sacks, 1995; see also Hester & Eglin, 1997). In this vein, Choo, Renshaw, and Austin (2005) analysed how teachers and parents of children with moderate to severe disabilities coconstructed the category ‘‘disability’’ in their daily interactions. Through analysis of the day-to-day entries in a Communication Book, used for communication between teachers and parents, they found that even though the child’s disability was not always in the foreground, the child was accomplished as passive, lacking competence and agency, so that the category ‘‘disability’’ was always implicitly present. In psychology, categorisation is a classic object of investigation, notably in experimental social psychology, where it is related to the issue of stereotypes and prejudices (Tajfel, 1981). It is also investigated in other fields of psychology, in particular in discursive psychology (e.g., Billig, 1985; Edwards, 1991). In contrast to a strictly cognitive view, Billig (1985) proposed that categories are not ready-made resources that are stored in our mind and can be retrieved on request. They are embedded in social activities, goal-orientated, addressed to somebody (a person, a group or an institution) and, hence, anticipate possible reactions. As Edwards (1991) put it, they are ‘‘shaped for their functions in talk, for the business of doing situated social actions’’ (p. 517). From this standpoint, categorisation is a social process (categories are socially constructed), a discursive process (categories are produced through discourse), and a pragmatic process (categories are goal-oriented).

M. Grossen et al. / International Journal of Educational Research 63 (2014) 15–25

17

Closer to our own theoretical framework (Grossen, 2010), categorisation also belongs to socio-cultural studies, where it relates to the study of institutional discourse and social practices (e.g., Hjo¨rne & Sa¨ljo¨, 2004; Linell & Fredin, 1995). In this field, Ma¨kitalo and Sa¨ljo¨ (2002) claimed that categories find their relevance within a certain context and that we should recognise both their ‘‘historicity’’ (p. 76) and their interactional and situated achievement. In her analysis of institutional categories in encounters between job applicants and vocational guidance officers in a public employment agency in Sweden, Ma¨kitalo (2003) focused on the job applicants’ and officers’ accounting practices, and showed, among other things, that the historicity of categories was particularly evident when there were communicative dilemmas or gaps in the interactions between a job applicant and an officer. The interactional achievement of categories could then be viewed as a pragmatic and situated means of solving a dilemma that had a meaning beyond the present situation. From this research field, we may retain the idea that, as central features of institutional discourse, categories are resources that orient the professionals’ actions and decisions. As Hjo¨rne and Sa¨ljo¨ (2004) put it: ‘‘Categories are not just names for things and relationships, they do a real job by signifying and informing us about how to classify and act in specific contexts’’ (p. 2). Moreover, categories not only have relevance within the interactional situation but in addition echo broader social situations. Let us now examine how, in our study, we approach the issue of categorisation. 3. Interviewing psychologists about their assessment practices 3.1. Data collection The data were collected from three services in a town in the French-speaking part of Switzerland. These services were all dedicated to children and/or adolescents but had different missions: (1) The School Psychology Service worked with Primary School pupils and offered help to children with school or psychological difficulties. It carried out mostly brief psychological examinations; (2) the School & Career Service worked with Secondary School students. Its mission was mainly to assess the students’ level of school achievement and to orient them towards adequate school careers or professional training. In this service, psychologists used tests in face-to-face settings but also in group situations, such as classrooms or small groups; (3) the Psychological Counselling Centre was a child and adolescent psychiatry service in which psychologists conducted psychological examinations as well as psychotherapy. In these services, the tests that were more frequently used were, as in many other places in the world, the WISC (French version of the WISC-III, Third Edition), the Raven Progressive Matrices and projective tests, mainly the Rorschach test and the Thematic Aperception Test (TAT). There were also various aptitude, personality, and career interest tests. The latter were specific to the School & Career Service, in which, on the other hand, psychologists rarely used projective tests. Seventeen psychologists volunteered to participate in the study: six (Ann, Carol, Flora; Boris, Dan, and Edo) in the School Psychology Service; five (Georgia, Hilda, Ida, Jane, Kevin) in the School & Career Service and six (Lea, Marta, Nina, Olga, Paula, Rina) in the Psychological Counselling Centre. They all had more than one year of professional experience. They were all graduates in psychology with a similar initial training, but different further specialised or professional training. Whereas in the School Psychology Service, and even more so in the Psychological Counselling Centre, many psychologists worked in a psychoanalytical framework, this was not the case in the School & Career Service. The participants in the study were interviewed for about one hour in their workplace. The interviews were semi-directed and specifically concerned their practices in the administration of a test and, more broadly, in psychological examination, with questions such as: How do you proceed in carrying out a psychological assessment? What tests do you use and for which purpose? How do you choose which tests to use in a psychological examination? Do you have a preference in the order of administration of tests? What did you learn from experience? The interviews were conducted in French and transcribed according to some transcription conventions (cf. Appendix A). 3.2. Research questions and method of analysis The aim of our study was, firstly, to grasp which assessment practices were important from the practitioners’ point of view; secondly, to collect reported practices that would show how the psychologists reported dealing with the clients’ diversity, and indicate whether their reported practices differed from the standardised procedures prescribed in Users’ Manuals. More specifically, our research questions were: what categorisations did the psychologists mobilise when they referred to their clients? What practices appeared to be important in the practitioners’ reports and how did they differ from prescribed practices? How did these categorisations orient the practitioners’ actions? At a methodological level, analysis of the psychologists’ interviews was carried out in two steps: (1) We looked for all the client categorisations in the corpus. These categorisations could be made through: (a) noun complements (e.g., ‘‘hyperanxious child’’, ‘‘students in the 8th grade’’, ‘‘students who have difficulties’’); (b) predicative adjectives (e.g., ‘‘he is anxious’’); (c) a client’s action expressed by a verb (e.g., ‘‘could he do the reasoning?’’, ‘‘he’s fed up with it’’). 379 categorisations were found and submitted to a content analysis. (2) We examined the context in which these categorisations were produced in order to understand with which actions they were associated and what resources for action they might be.

18

M. Grossen et al. / International Journal of Educational Research 63 (2014) 15–25

4. The psychologists’ client categorisations The content analysis carried out on the 379 categorisations found in our corpus resulted in nine mutually exclusive content categories. The labels of these content categories are indicated in capitals, whereas the psychologists’ discourse is reported within quotation marks. PSYCHOLOGICAL STATES (n = 57, 15 psychologists) referred to various negative psychological states which were presented either as stable (‘‘depressed’’, ‘‘not well at all’’, ‘‘who had clinical problems’’, ‘‘hyper-anxious’’, ‘‘aggressive’’, ‘‘inhibited’’, ‘‘blocked’’, ‘‘lacking in concentration’’, ‘‘slow’’, ‘‘hyperactive’’) or, more rarely, as temporary (‘‘crisis situation’’). COMPETENCE (n = 52, 13 psychologists) concerned the client’s level of competence, either in general (‘‘intelligent’’, ‘‘highly gifted’’, ‘‘who has a good intellectual mode of functioning’’), at school (‘‘who have school difficulties’’, ‘‘who does well at school’’) or in the test to which he or she was submitted (‘‘do not achieve’’ vs. ‘‘achieve’’). INITIAL BEHAVIOUR (n = 45, 10 psychologists) referred to the clients’ attitudes or behaviour at the beginning of a test session. They concerned either a lack of motivation (‘‘who lies on the table’’, ‘‘who begins to stir up trouble’’, ‘‘less collaborative’’, ‘‘did not see any interest’’) or the client’s motivation (‘‘begin to recount their life story’’, ‘‘who were willing to come and do tests’’). They also referred to the clients’ acquaintance with a test or a test situation. REACTIONS (n = 45, 14 psychologists) grouped various psychological reactions that a client may have when faced with a test situation or a specific test. These reactions were expressed in negative terms (‘‘growing anxious’’, ‘‘who has a psychological reaction to failure’’, ‘‘who gets nervous’’, ‘‘feels persecuted’’), as well as in positive terms (‘‘feels well’’, ‘‘quiet’’, ‘‘finds an opportunity of feeling valorised’’). Within these various categorisations, ‘‘tiredness’’ was frequently evoked as an outcome of being tested. MODE OF ANSWERING A TEST (n = 42, 12 psychologists) referred to the process through which the clients produced a response and coped with difficulties (‘‘how the child answers’’, ‘‘how he functions over a length of time’’). Some categorisations referred to tests in general and others to specific tests, in particular the Rorschach test or the WISC. SCHOOL STATUS (n = 38, 8 psychologists) referred to the clients’ school grade (‘‘fourth or fifth grade children’’), school track (‘‘students from a ((disadvantaged)) track’’), or, exceptionally, professional status. AGE (n = 16, 6 psychologists) referred to the client’s age expressed in number of years (‘‘a four-year old child’’) or in broader terms such as ‘‘young children’’. TEST INSTRUCTIONS (n = 11; 4 psychologists) referred to the clients’ understanding of the instructions (‘‘he has understood’’). DESCRIPTION OF ACTIONS (n = 42, 12 psychologists) referred to the clients’ actions during test administration, typically utterances such as ‘‘the answers he gives’’ or ‘‘they carry out the test’’. 31 categorisations (given by 10 psychologists) did not fit into these categories either because they were unique or because they were scarcely informative (‘‘children we generally see’’). An analysis of the distribution of these client categorisations in the three services showed that in the School & Career Service, all the psychologists made client categorisations pertaining to the category SCHOOL STATUS, which was more than in the two other services (1 out of 6 in the School Psychology Service; 2 out of 6 in the Psychological Counselling Centre). A similar result was observed for the category TEST INSTRUCTIONS (4 in the School & Career Service, none in the two other services). One reason for these differences is probably that many of the psychological assessments of the School & Career Service were carried out within schools and had to be adapted according to school tracks. Moreover, these psychological assessments were collective and therefore required unambiguous test instructions. We also observed that in the Psychological Counselling Centre and in the School & Career Service, all the psychologists made client categorisations belonging to the category REACTIONS, and made more of them than the psychologists of the School Psychology Service (3 out of 6). This result may be explained by elements that are not the same in each service: in the Psychological Counselling Centre, the psychoanalytical framework to which most of the psychologists referred led them to pay attention to transference, hence to mention their clients’ reactions. In the School & Career Service, this difference was due to their frequent use of concentration tests, which led them to be particularly attentive to the clients’ reactions when doing these tests. This analysis was a first approach to understanding how, in these three services, the psychologists dealt with clients’ diversity. It sheds light on the elements to which they were sensitive, but also, indirectly, less or not sensitive. However, so far, this first approach did not inform us about the practical orientation of these client categorisations.

5. Client categorisations as resources for action In this second analysis, we examined the context in which the client categorisations were produced. We were able to identify six sets of actions, that, taken together, described the sequentiality of a test session: (a) determining the conditions under which a test can be used; (b) organising a whole session of psychological examination; (c) choosing a test for a client; (d) choosing the order of the tests to be done; (e) administering a test; (f) evaluating the client in the course of the administration of a test. Let us now present the categorisations that were evoked in each of these sets of actions.

M. Grossen et al. / International Journal of Educational Research 63 (2014) 15–25

19

5.1. Determining the conditions under which a test can be used Under this label, we grouped actions that described the beginning of a test session and the conditions in which a client should be able to complete a test. Seven psychologists reported checking first whether their client was, or was not, in a condition to give a test. Their report of these actions was mostly associated with client categorisations belonging to the content categories INITIAL BEHAVIOUR and PSYCHOLOGICAL STATE. Three psychologists mentioned several categorisations related to the content category INITIAL BEHAVIOUR. According to Jane, a lack of motivation might be a reason for postponing the administration of a test or for abandoning it, whereas Ann and Carol both mentioned that when a child seems highly motivated to have an open discussion, they do not use a test. For example, Carol interpreted the fact that a child draws and speaks spontaneously as sign of her motivation. Consequently, she reported postponing, at least temporarily, the administration of a test (client categorisations are in italics): [1]

BUT IT’S VERY AMAZING, sometimes children arrive err here with their own question (. . .) they draw, they start with their story, then I won’t do a test I let them go (. . .) (Carol, 3C, 313)

Three psychologists also mentioned categorisations related to a client’s PSYCHOLOGICAL STATE as justifications for postponing a test administration. For Ann, an ‘‘inhibited’’ child, or for Olga, a child ‘‘invaded by persecution anxieties’’, are not in the right state of mind to be tested. Observations of this sort might also lead to practices different from those prescribed, for example doing only parts of a test: [2]

For example I am thinking of a little child who had- who was very very pre-school I’d say. I did clinical observations with PARTS of tests, and then I saw at which moment he was testable (Ann, 2A, 150b,)

The notion of ‘‘testability’’ (‘‘testable’’) mentioned by Ann in this excerpt shows the practical orientation of these categorisations. In these cases, the psychologists’ categorisations seemed to be driven by an underlying question ‘‘Is the client testable or not?’’, a question, which was itself guided by the psychologists’ concern with obtaining results that best reflected their clients’ capacities or psychological functioning. 5.2. Organising a whole session of psychological examination The second set of actions referred to the step-by-step organisation of a whole psychological examination (that is, usually, three to five sessions) and was mentioned by twelve psychologists. These actions concerned the setting (face-to-face, collective administration, self-administered tests at home), the adaptations of standard procedures (test-retest procedure, administration of complementary tests), the sequential order of activities (for example, beginning with a collective test and following with a face-to-face interview), and, in many cases, the administration of the WISC in two sessions or more. While mentioning these actions, the psychologists made mostly client categorisations belonging to the content categories PSYCHOLOGICAL STATE, REACTIONS and INITIAL BEHAVIOUR. Seven psychologists mentioned client categorisations related to the content category PSYCHOLOGICAL STATES. All these categorisations, except one, concerned administration of the WISC. For example, categories such as ‘‘being very anxious’’ or ‘‘having difficulties in concentrating’’ or, as in Excerpt [3], ‘‘agitation’’ were justifications for splitting the WISC into two or more sessions: [3]

(. . .) certain children who have who have an enormous psychomotor agitation, well I do short sessions (. . .) (Lea, 7L, 745)

Six psychologists mentioned categorisations that pertained to REACTIONS. For example, Edo reported some reactions caused by the length of the WISC: [4]

(. . .) a WISC is long, so it’s important to split up the administration so that the child is not persecuted o:r shows signs of lassitude and then that he throws the material away because he’s fed up with it and from this moment you see that the answers he gives are all wrong because he doesn’t reflect anymore (Edo, 2E, 241–245)

In the psychologists’ report, such reactions should be avoided because they cast doubt on the reliability of the results. Four psychologists also mentioned categorisations of the content category INITIAL BEHAVIOUR. In this case, as in the preceding set of actions (Section 5.1), these categorisations stressed the importance of paying attention to the client’s motivation in the organisation of a psychological examination. 5.3. Choosing a test A third set of actions referred to the choice of tests to be done with a client. It was mentioned many times by all psychologists, except one. This choice concerned mainly the use of the WISC and of a projective test (Rorschach test or TAT).

20

M. Grossen et al. / International Journal of Educational Research 63 (2014) 15–25

The categorisations relating to these actions belonged above all to the content categories COMPETENCE, PSYCHOLOGICAL STATES, SCHOOL and REACTIONS. Seven psychologists mentioned categorisations related to COMPETENCE. In one half of these categorisations, a lack of competence (‘‘mastering French insufficiently’’, ‘‘being weak at school’’, etc.) was presented as a reason for using a test in order to assess the client’s level of competence. In the other half, however, the psychologists emphasised that appearances might be misleading. For example, Carol reported using a cognitive test to check whether her impression of a ‘‘good’’ cognitive level was confirmed or not:

STATUS, AGE

[5] THERE ARE SOMETIMES big differences between children who can verbalize extremely well and there is a very mature discussion with a psychologist in an office, then they arrive in the classroom, they don’t know- they cannot respond to an instruction. so I try not to miss this aspect, because this is something that one- that is less- that is maybe less apparent, it’s also a reason why I say that one cannot do a clinical evaluation of the IQ (Carol, 8C, 517-519) Boris, Olga and Paula also reported using projective tests to check their impression that a client did not have any psychological troubles. Olga expressed this concern with regard to ‘‘gifted’’ children: [6]

so I for gifted children’ in general (laughter) I won’t- I’ll do a Rorschach, TAT, because I have observed there was nevertheless troubles at a psychological level’ quite important, / and I’ll try to talk about this, instead of continuing to stick to ‘‘he is of an extraordinary intelligence’’ (Olga, 8O, 839–841)

In all these cases, tests were used as a challenge to the psychologist’s clinical impressions or ‘‘feelings’’, as a guarantee against the risk of subjectivity. Hence, their reliability was not absolute but relative to the psychologist’s awareness that he or she might be deceived by his or her own perceptions. Six psychologists made categorisations belonging to the content category PSYCHOLOGICAL STATES, for example: ‘‘slow’’, ‘‘inhibited’’, ‘‘reserved’’, ‘‘mute’’, ‘‘at risk of having a psychotic breakdown’’, ‘‘in need of mastering’’, ‘‘having difficulties in concentrating’’. All these categorisations, except four, justified the psychologist’s decision to use or not to use a projective test. For example, Flora reported using a projective test only when she had certain suspicions: if I see immediately that he is an adolescent who is not at risk of having a psychotic breakdown or whatever, it’s VERY rare that I propose a test battery. On the contrary if I have doubts, yes (Flora, 1F, 56)

[7]

Categorisations of the content categories SCHOOL STATUS and AGE were given by respectively five and three psychologists. In all these cases, except one, they simply mentioned that a test was designed for a certain age or school level. Jane, however, reported not administering career interest tests to students from disadvantaged school tracks for a different reason: [8]

Jane

so, doing interest tests so that it:/it broadens too much the possibilities, it’s not very good because afterwards one is always obliged (laughter) [to narrow everything]

I

[yeah it’s]

Jane

quite delicate. it’s the reason why I rarely do interest tests with students of ((disadvantaged)) sections (Jane, 16J, 638-640)

Jane’s justification has nothing to do with the reliability of the test itself. Rather, it invokes moral reasons that are linked both to the client’s position and to her own. In fact, if the client shows a wide range of interests but does not have the concrete possibilities of pursuing them, why look for these interests? This would only lead to disappointing the client who might question the relevance of the psychologist’s choice. In contrast to this example, it is worth stressing that, by analogy, nobody reported not administering a WISC because a weak IQ might have moral consequences for a client. Categorisations of the content category REACTIONS were made by two psychologists. For example, Marta reported that she always used both the WISC and the Rorschach test, and justified it by the clients’ reactions to the Rorschach test: [9]

((speaking of the WISC)) but I have a feeling that there is always a subtest which they can hang on to / they can improve their self image, they feel well, / because I have a feeling that the Rorschach’ if they don’t feel comfortable, from card one to card ten’ it won’t go better, / (Marta, 8M, 332–335)

In Marta’s report, the systematic association between these two tests seems not only to aim at assessing various aspects of the person, but at balancing the possible negative reactions that a client might experience during the Rorschach test. Lea evokes a similar ethical concern: expecting a client to feel ‘‘comfortable’’ in a given test is a good reason for choosing it: [10]

I’ll choose Patte Noire ((a projective test)) if I think the child will be more comfortable in Patte Noire, (Lea, CL2, 13L, 169)

M. Grossen et al. / International Journal of Educational Research 63 (2014) 15–25

21

For these psychologists, the choice of a test was not only based on their psychometric qualities, but included also ethical concerns that went beyond the prescribed use of these tests and took the diversity of the clients’ reactions into consideration. 5.4. Choosing the order of the tests to be done A fourth set of actions concerned the order in which a series of tests should be done, and was mentioned by six psychologists. Four psychologists mentioned the order in which the WISC and the Rorschach test should be administered. They all reported beginning a test session with the WISC and continuing with the Rorschach test. On this occasion, they produced mostly categorisations of the content categories INITIAL REACTIONS. Olga, for example, mentioned two categorisations, the first from the category OTHER (‘‘know at school’’) and the second from the category REACTIONS (‘‘relatively reassured’’): [11]

so me I ideally’ I like to begin by a- a cognitive evaluation because it’s- I think the children even if it’s stressful’ at the same time it’s quite close to what they know at school, so they are relatively reassured, and then I do psycho-affective tests afterwards (Olga, 5O, 444–448)

Olga justifies her practice by her anticipation of the clients’ reactions. In so doing, she indirectly provides a theory about the relationship between a test situation and other situations, a theory that could be formulated as: ‘‘The closer a test is to a familiar situation, the more reassured the testee feels’’. In this case, the WISC was not only used to assess the child’s level of intelligence but as an opening sequence to a less familiar test. As was also the case in Excerpt [10], putting the client at ease appeared to be a prerequisite to ensuring the most reliable results possible. The order of the tests also concerned concentration tests. Jane and Hilda reported presenting them at the end of a session and justified it by a categorisation based on the client’s reaction after a long test session (Hilda: ‘‘to see if he can still focus his thoughts’’). However, Hilda reported feeling puzzled when a client obtained good results in a concentration test presented at the very end of a session, while he or she had concentration problems in other tests given earlier: [12]

sometimes I’ve been in trouble because I REALLY saw that the person well was unable to stand err these for two hours, all of a sudden he did a good ((name of the concentration test)) so I was in trouble (laughter) (Hilda, 5H, 371)

Her puzzlement was so important that, as she reported with a sense of humour, after these experiences, she avoided administrating concentration tests. In this case, it is as if the client’s reactions were so different from her expectations and from the test prescriptions that she could not consider the test as reliable anymore. 5.5. Administering a test This set of actions, mentioned by eleven psychologists, referred to the concrete procedure involved in the administration of a test. The respondents focused mainly upon the collective tests, the Rorschach test (five psychologists) and the WISC (three psychologists). We shall focus on the two latter tests, which were the most frequently used. Concerning the Rorschach test, four psychologists reported proceeding to an inquiry phase after test completion. In this phase, which is part of the test prescriptions (e.g., Rausch de Traubenberg, 1970), the examiner has to take up each response and to discuss it with the client in order to interpret the response more accurately. Most of the categorisations were produced while describing the inquiry phase and belonged to the content category MODE OF ANSWERING. All the psychologists insisted on the need to follow the prescribed test procedure accurately. Only Rina reported making an exception: [13]

when I see: that an adolescent becomes anxious then I immediately carry out the inquiry after the card (Rina, 4Q, 399)

In this example, the categorisation ‘‘anxious’’ (content category REACTION) orients Rina’s actions and accounts for her deviation from prescribed procedures. Four psychologists talked about their practices in the WISC and on this occasion made various categorisations belonging to the categories COMPETENCE, INITIAL BEHAVIOUR, MODE OF ANSWER and PSYCHOLOGICAL STATES. These categorisations testified to their close attention to their clients’ diversity and accounted for various non-standard practices: [14]

(. . .) for example it’s true that my relationship with the chronometer is relatively loose, /first because I find it very difficult to give a test to children who have difficulties and to be ruled by the chronometer / the chronometer is a huge fact- factor of stress and anxiety for children, so it’s true that I’m rather discreet with the chronometer / it’s not on the table big like that (laughs)(. . .) so it’s true that I do a test administration that could be described as more clinical than purely psychometric, // and that’s it (Olga, 3O, 400-409)

Olga alludes to what she and other psychologists have called a ‘‘clinical administration of the test’’, that is, a test administration aimed at understanding the client’s reasoning. Olga’s example shows how her client categorisation (‘‘who

22

M. Grossen et al. / International Journal of Educational Research 63 (2014) 15–25

have difficulties’’, category COMPETENCE) prompted her to do a clinical administration before the end of the test and, hence, to deviate from the standard procedure. The client’s difficulties in the course of a test administration justified many other non-standard practices, for example allowing more time to end a subtest, continuing the administration beyond the limit of incorrect answers, encouraging the client to give an answer, or attributing a score after a client’s self-correction. From a test designer’s standpoint, all these nonstandard practices ruin the reliability of the test. However, from a socio-cultural approach, we may ask why they are so widespread. The following piece of dialogue between Lea and the Interviewer helps us understand this point:

[15] 265

Lea

(. . .) / but I think that we’re not: fortunately / we’re not rigid, and rigidified within things, / I think that:

266

I

[in particular on test administration]

267

Lea

[but it’s where the tranferential part] and counter-tranferential come in, / if we feel the child as hyper-anxious’

268

I

mmh

269

Lea

hyper-bad’ etcetera, it will- well, fortunately we are human,

270

I

yeah so it’s- what is this humanity in a- in a- in front of a tool that is supposed to be: [completely]

271

Lea

[]

271

I

objective as a cognitive test, /

273

Lea

but: one has to differentiate rigour and then rigidity, well, / one needs a certain rigour’ with respect to test administration, / with respect to the analysis with respect to coding, (. . .)

289

Lea

but well I think that there is a certain rigour in the administration, and then there is a certain flexibility that means that we are not rigid, / but rigour I am very attached to it, / in the way of doing things // (Lea, 21L, 22L, 664-681)

Lea evokes the psychoanalytical notion of transference and counter-transference to refer to her own feeling in front of a child. She defines test administration as a situation implying a relationship between the tester and the testee, a definition that leads her to state that ‘‘fortunately we are human’’. She (Lea 273) then introduces a distinction between ‘‘rigour’’ and ‘‘rigidity’’. Whereas ‘‘rigour’’ refers to the prescribed procedures, the word ‘‘rigidity’’ is a clearly negative judgement, that is, a value, which is not supposed to be found in a scientific world. Her distinction leads to a sort of compromise that she calls ‘‘flexibility’’ and calls attention to the client’s specificity. Her discourse, which was very representative of the other psychologists’ discourse, can be interpreted as the expression of a tension between two possible versions of ‘‘reliability’’: a scientific version that corresponds to that of the test designers and a situated version based on the diversity of the psychologists’ clients. In the latter version, the notion of reliability includes a moral dimension that Nina, like many of our psychologists, expressed:

[16]

I try to see everything that he can give at the MAXimum of his competences, / and then: if he’s a hyper-agitated child and who comes for this problem I think it’s not fair:’ but this is- (laughter) gee Wechsler saw it differently, but me no (laughter) I think it’s not very fair that a child who has attention disorders’ to put him one hour and a half in front of a task, well, / because then’ it’s putting him in difficulties and it’s maybe also pushing him to a point where he refuses the subtests after a while, / whereas by doing it in many sittings he accepts it (Nina, 7N, 687–690)

Nina makes a clear distinction between Wechsler’s point of view and her own. Her reasoning draws upon various client categorisations (‘‘hyper-agitated child’’, ‘‘who comes for this problem’’, ‘‘who has attention disorders’’) and leads to the idea that a standard test administration would be ‘‘not fair’’. Now, the word ‘‘fair’’ does not belong to the scientific lexicon, although it has a long history in the field of psychometrics (‘‘culture-fair’’ tests, e.g., Richardson, 2002). It has a moral orientation and shows that, by considering the clients’ diversity, the psychologists are able to avoid being what Lea called ‘‘rigid’’. In this context, a ‘‘clinical test administration’’ as well as other non-standard procedures can be seen as various compromises between a scientific version of reliability, in which standardised practice is requested, and a situated version of reliability, which is concerned with the uniqueness of a client and the psychologist’s need to adapt his or her practice to the clients’ diversity.

M. Grossen et al. / International Journal of Educational Research 63 (2014) 15–25

23

5.6. Evaluating the client in the course of a test administration This set of actions concerned the assessment that takes place during the administration of a test and was mentioned by nine psychologists. According to them, raw scores obtained in a test (i.e., the WISC) are insufficient to assess a client’s competence or psychological functioning, and should be completed by clinical observations made during the administration of a test. Two main practices were reported: (1) observing the client during the administration in order to understand how he or she answers, copes with the task, or faces difficulties (six psychologists); (2) doing what the respondents called a ‘‘clinical rehearsal’’, that is, administering a whole test again (or part of it) in order to see whether the client can improve his or her performance, and to understand his or her reasoning or mental functioning (six psychologists). When they reported these practices, six psychologists made categorisations related to the content category PSYCHOLOGICAL STATES: [17] the point where reliability is the most delicate, is- WHEN I administered the WISC or the test, was the child at the top of his possibilities’ or was it a day when he was lacking concentration, did not sleep the night before and therefore his results are weaker, obviously this is difficult to know (Dan, 8D, 390–391) In Dan’s report, ‘‘lacking concentration’’, not being ‘‘at the top of his possibilities’’, ‘‘did not sleep the night before’’, are all elements that have to be considered in order to evaluate the reliability of the results. As with Lea and Nina in excerpts [15] and [16], his conception of reliability is context-bound and based on the assumption that temporary or unexpected circumstances are always liable to influence the testee’s performance. Four psychologists made categorisations pertaining to the content category MODE OF ANSWERING. According to them, the process through which a client gives a response belongs to the assessment process itself, and can be examined through clinical rehearsal, as Dan reported: [18]

I take it up by looking how far I can bring the child to reas- to succeed, why- by trying a bit to understand- when there are things that I didn’t understand, what happens, what makes that he was blocked for example (Dan, 4D, 240) (. . .) and another thing I also do is putting much more emphasis on the process, on the HOW the child answers, HOW he copes with problem solving strategies (Dan, 5D, 374)

Globally, the analysis on the psychologists’ discourse showed that according to these practitioners, a psychological assessment is achieved both by the scores obtained in a test and by observations that are strictly linked to the specificity of the situation and of the client. 6. Discussion The starting point of the study presented in this article was the observation that, beyond the numerous prescriptions that are given in test manuals or in textbooks, little is known about the psychologists’ assessment practices in test situations. Through semi-structured interviews with seventeen psychologists working in three different public services for children and adolescents, we observed that the psychologists’ accounts of their practices were scattered with what we called ‘‘client categorisations’’, that is, noun complements, predicative adjectives, or verbs that referred to a client. On an empirical level, we took these client categorisations as a method for grasping how the practitioners dealt with the diversity of their clients. More specifically, our aim was to capture the practitioners’ perceptions of their own practices, to examine whether and how their reported practices differed from prescribed practices, and to which practical purpose they were oriented. In a first analysis of the data, we found that these categorisations could be grouped into content categories that referred mostly to the clients’ initial behaviour, psychological states, competence, reactions to the test situation, and mode of answering a test. This analysis revealed that our respondents had some shared perceptions about their clients and were sensitive to their diversity. We also found that, when reporting their practices, the psychologists focused upon the psychological dimensions of their clients’ diversity: their motivation, sensitivity and reactions to the situation, competence, mode of reasoning or formulating a response, etc. On the other hand, they reported very few socio-cultural elements, such as bilingualism, migration, socio-economic status, etc., a result that shows that their client categorisations reflect the psychologists’ role within a professional arena made up of various professions and their allegiance to bodies of knowledge pertaining to psychology. Moreover, a comparison between the psychologists’ categorisations in the three services gave us some reason to believe that their categorisations were not simply based on their personal experience and representations, but partly reflected the mission, the work organisation, and the theoretical orientation of their service. In a second analysis of the data, we found that these categorisations were associated with six sets of actions that, taken together, described the sequentiality of a whole psychological examination: determining the conditions under which a test can be used, organising a psychological examination, choosing a test, choosing the order of the tests to be done, administering a test and assessing the client during the administration of a test. The analysis of the client

24

M. Grossen et al. / International Journal of Educational Research 63 (2014) 15–25

categorisations provided by the psychologists in each of these sets of actions showed that the psychologists’ accounts did not simply echo the test designers’ voices. Drawing upon both scientific knowledge and professional experience, they reported various non-standard practices that complemented the procedures prescribed in manuals, but also differed from them or were even forbidden. For these psychologists, administering a test was not the mere application of procedures, but a situated activity that relied heavily on the psychologist’s capacity to adapt his or her actions according to a diversity of situations, clients and contexts. In other words, these situated practices had their own rationality within a certain activity. 7. Conclusion Our analysis of the data allows us to infer that the interviewed psychologists seemed to experience a sort of dilemma between what they were supposed to do and what they thought they should do in a concrete situation. This dilemma is composed of three types of tensions that are partially related: (1) A tension between, on the one hand, prescribed practices, that aim at creating comparable test conditions, at ensuring reliability and are, by definition, generic and a-contextualised; on the other hand, situated practices, that stem from the specificity of a situation, are addressed to a particular client in a given situation, and are therefore partly improvised. The ‘‘clinical administration of a test’’ and other non-standard practices may be viewed as a manifestation of this tension and a compromise between the constraints imposed by standardised (and a-contextualised) tests and the constraints imposed by concrete and unique situations. Hence, they cannot be regarded as mere deviations from standard practices, but constitute practical solutions to cope with concrete situations that are so variable that no set of prescriptions would ever be able to anticipate them all. (2) A tension between two versions of the notion of reliability: scientific reliability as defined by the test designers, and situated reliability, as applied by practitioners in concrete situations. Whereas in the first version, reliability requires the examiner to take an objective stance and to reduce as much as possible his or her own involvement within the situation, in the second version, reliability is guided by the singularity of each client and situation, and by moral concerns. As a way of coping with the clients’ diversity, situated reliability requires the examiner’s ability to identify the specificity of each situation and client, and therefore to undertake a task that is far too complex to be described in standardised procedures. Therefore it is based on the examiner’s agency, subjectivity and personal involvement. As a morally loaded notion, situated reliability has much in common with trust, a notion that involves both knowledge and ignorance, both belief and morality (Simmel, 1950; see also Grossen & Salazar Orvig, in press; Markova´, Linell, & Gillespie, 2008). Situated reliability may then be defined as a form of trust based on the expectation that the results of a test give a reasonably trustworthy picture of a client’s competence or psychological functioning. Hence, the psychologists’ concern was not to have a ‘‘true’’ picture of the clients’ abilities; rather, it was, firstly, to give the clients a chance to show the best of their abilities (or its corollary: to avoid getting results that did not reflect the client’s capacities because of the conditions in which the test was administered), secondly, to get results that were sufficiently credible (or reliable) to help them orient their actions. From the psychologists’ standpoint, the problem was not to have a test that was scientifically reliable, but an instrument they could use as a sort of buffer or protection against a not completely trustworthy subjectivity. (3) A tension between two competing notions of justice (Aristotle, 1980): rectificatory (or commutative) justice, which is based on an arithmetic distribution of resources, and distributive justice, in which the distribution of resources is based on a former state of goods distribution. As we showed, the psychologists’ practical problem seemed that of being ‘‘fair’’ in their assessment practices, that is, of doing justice to the particularity of each client. This search for ‘‘fairness’’ (illustrated by the notion of ‘‘testability’’ that emerged from our data) showed that these practitioners were conscious of the consequences of labelling their clients (Gill & Maynard, 1995) and tried, somehow, to stave off this risk. From this angle, standardised, and thus prescribed, practices pertain to commutative justice (everybody gets the same ‘‘treatment’’), whereas situated practices are torn between commutative and distributive justice. It was as if, in order to take their clients’ diversity into account, the psychologists had to counterbalance the potential injustice of a standardised situation and to develop practices that strayed from standardisation.

The interweaving of these three types of tensions shows that dealing with diversity in a test situation (that is, dealing with many individuals who, taken in the unpredictability of a concrete situation, are all unique) is not regarded as a problem from the test designers’ standpoint, since it is supposed to be solved by the prescription of standardised procedures. However, from the practitioners’ standpoint, it constitutes a practical problem, which lies halfway between a problem-solving task and a moral dilemma. Acknowledgments The authors are grateful to the three services, which allowed them to conduct this study and wish to thank the psychologists who agree to give them some of their precious time.

M. Grossen et al. / International Journal of Educational Research 63 (2014) 15–25

25

Appendix A. Norms of transcription [] (laughter) CAPITAL LETTER : (or::) ‘ , . / // <. . ..> (. . .) ((. . .))

Overlaps Non-verbal behaviour, such as laughter, whispers, etc. Stress of a word or a syllable Lengthening of a syllable Rising intonation (not necessarily a question) Falling intonation Short pause Pause (1 s) Pause (2 s), etc. Self-interruption or false start Uncertain transcription Cutting in the transcription Transcriptor’s comment or our comment

References Aristotle. (1980). The Nicomachean ethics. Oxford, UK: Oxford University Press. Aronsson, K., & Hundeide, K. (2002). Relational rationality and children’s interview responses. Human Development, 45(3), 174–186 http://dx.doi.org/10.1159/ 000057073. Ashforth, B., & Humphrey, R. (1997). The ubiquity and potency of labelling in organizations. Organization Science, 8(1), 43–57. Billig, M. (1985). Prejudice, categorization and particularization: From a perceptual to a rhetorical approach. European Journal of Social Psychology, 15, 79–103. Bruner, J. (1990). Acts of meaning. London: Harvard University Press. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105. Choo, J., Renshaw, P., & Austin, A. (2005, August). The accomplishment of the child-with-a-disability in parent–professional’s everyday interaction. Paper presented at the Biennial Conference of the European Association for Research on Learning and Instruction (EARLI), Nicosia, Cyprus, August 23–27. Clot, Y. (1999). La fonction psychologique du travail. [The psychological function of work]. Paris: Presses Universitaires de France. Donaldson, M. (1978). Children’s mind. Glasgow: Fontana. Edwards, D. (1991). Categories are for talking. On the cognitive and discursive bases of categorisation. Theory and Psychology, 1(4), 515–542. Elbers, E., & Kelderman, A. (1994). Grounds rules for testing: Expectations and misunderstandings in test situation. European Journal of Psychology of Education, 9(1), 110–120. Gill, V. T., & Maynard, D. W. (1995). On labelling in actual interaction: Delivering and receiving diagnoses of developmental disabilities. Social Problems, 42(1), 11–37. Grossen, M. (2010). Interaction analysis and psychology: A dialogical perspective. Integrative Psychological & Behavioral Science, 44, 1–22. Grossen, M., & Perret-Clermont, A.-N. (1994). Psycho-social perspective on cognitive development: Construction of adult-child intersubjectivity in logic tasks. In R. Maier & W. de Graaf (Eds.), Sociogenesis reexamined (pp. 243–260). New York: Springer. Grossen, M., & Salazar Orvig, A. Forms of trust/distrust and dialogicality in focusgroup discussions about medical confidentiality. In I. Markova´, & P. Linell (Eds.). Dialogical perspectives on trust. Charlotte (NC): Information Age, in press. Hester, S., & Eglin, P. (1997). Membership categorization analysis: An introduction. In S. Hester & P. Eglin (Eds.), Culture in action: Studies in membership categorization analysis (studies in ethnomethodology and conversation analysis) (pp. 1–23). Lanham, MD: International Institute for Ethnomethodology and University Press of America. Hjo¨rne, E., & Sa¨ljo¨, R. (2004). There is something about Julia: Symptoms, categories, and the process of invoking ADHD in the Swedish school: A case study. Journal of Language, Identity and Education, 3(1), 1–24. Linell, P., & Fredin, E. (1995). Negotiating terms in social welfare office talk. In A. Firth (Ed.), The discourse of negotiation. Studies of language in the workplace (pp. 299–318). Oxford: Pergamon. Ma¨kitalo, A. (2003). Accounting practices as situated knowing: Dilemmas and dynamics in institutional categorization. Discourse Studies, 5(4), 495–516. Ma¨kitalo, A., & Sa¨ljo¨, R. (2002). Talk in institutional context and institutional context in talk: Categories as situated practices. Text, 22(1), 57–82. Markova´, I., Linell, P., & Gillespie, A. (2008). Trust and distrust in society. In I. Markova´ & A. Gillespie (Eds.), Trust and distrust: Sociocultural perspectives (pp. 3–27). Charlotte, NC: Information Age. Mehan, H., Hertweck, A., & Meihls, J. L. (1986). Handicapping the handicapped. Decision making in student’s educational careers. Stanford: Stanford University Press. Rausch de Traubenberg, N. (1970). La pratique du Rorschach. [Using the Rorschach]. Paris: Presses Universitaires de France. Resnick, L. B., Sa¨ljo¨, R., Pontecorvo, C., & Burge, B. (Eds.). (1997). Discourse, tools and reasoning: Essays on situated cognition. Berlin: Springer. Richardson, K. (2002). What IQ tests test. Theory and Psychology, 12(3), 283–314. Rommetveit, R. (1978). On Piagetian cognitive operations, semantic competence and message structure in adult–child interaction. In I. Markova´ (Ed.), The social context of language (pp. 113–150). Chichester: Wiley. Rosenthal, P., & Peccei, R. (2006). The social construction of clients by service agents in reformed welfare administration. Human Relations, 59(12), 1633–1658. Roth, D. (1974). Intelligence testing as a social activity. In A. Cicourel, S. H. M. Jennings, K. H. Jennings, K. C. W. Leiter, R. MacKay, & H. Mehan (Eds.), Language use and school performance. New York, NY: Academic Press. Sacks, H. (1995). Lectures on conversation. Oxford, UK: Blackwell. Sa¨ljo¨, R. (1991). Piagetian controversies, cognitive competence, and assumptions about human communication. Educational Psychology Review, 3(2), 117–126. Simmel, G. (1950). The sociology of Georg Simmel (K.H. Wolff, Trans.). New York, NY: Free Press (Original work published 1908). Tajfel, H. (1981). Human groups and social categories. Cambridge, UK: Cambridge University Press.