- Email: [email protected]
You want to ask them what?!?!?!?!
You Want To Ask Them What ?!?!?! MICHAEL MORRIS The declining rate of births to teenagers across the U.S. in recent years has managed to bypass Persnickety Falls, a rural community in the southwestern corner of your state. Indeed, Persnickety’s moderately high teen birthrate has remained largely unchanged over the past decade, as have its rates of sexually transmitted diseases among teens. After several months of extensive, often acrimonious, and occasionally even humorous debate (“Our name shouldn’t be ‘Persnickety Falls,’ it should be ‘Knickers Fall’!”), the Persnickety Falls Board of Education has approved, in principle, the introduction of a life-skills program into the curriculum of the community’s middle schools. A major component of the program would focus on issues relevant to sexual behavior. You are a university-based evaluator from another part of the state, and today you are attending a planning meeting in which representatives of the following stakeholder groups are present: • • • • •
The board of education School administrators Teachers Parents A local human services organization that specializes in youth-oriented interventions. This organization is the prime candidate to design and offer the life-skills program. • A small, prestigious, private philanthropic foundation that is considering funding the program for its first three years. The foundation has called the meeting, which is being facilitated by one of the foundation’s program officers. The foundation has made it clear that its willingness to fund the life-skills program is contingent on comprehensive needs assessment, implementation, and outcome data being gathered as part of a systematic evaluation of the program. Your previous work with both the foundation and the local human services organization has resulted in your being recommended by both of these stakeholders to conduct the evaluation. A school administrator has just asked you about the types of information that are routinely gathered in such evaluations. In the course of your answer, you describe the varieties of behavior (sexual and otherwise) that standardized needs assessment surveys in this domain often focus upon. As you conclude your description, you look around the conference table and notice that the color has drained from the faces of most of the school board representatives, administrators, teachers, and parents. A flurry of loudly voiced objections immediately cascade into the conversation, of which the following are but a sample: • “There’s no way parents are going to let you ask their kids about THAT!!!” • “Why can’t we simply see if the birth rate goes down after the program is introduced? Why do we need all of this other stuff?” 83
84
AMERICAN JOURNAL OF EVALUATION, 24(1), 2003
• “If I let a survey ask these questions in my school on a Monday afternoon, I’ll be out of my job as principal by Tuesday morning!” • “What happens when some nosy newspaper reporter gets hold of the results? Can you imagine the headlines?” The representatives of the human services organization are conspicuously silent throughout this eruption, while the foundation folks continue to assert that their financial support requires a comprehensive evaluation to be performed, without commenting specifically on the objections that have been raised. They seem to be looking to you for sage counsel and group facilitation that will resolve the matter. The implicit message appears to be, “You are the evaluator, after all. Help everybody see what needs to be done here.” As the gaze of just about every individual in the room returns to you, you’re thinking, “What really does need to be done here? Is a ‘comprehensive evaluation’ involving the collection of highly sensitive data feasible in this situation? Would not collecting such data simply reinforce a community climate that is contributing to the problems the program is supposed to address? To the extent that there are conflicting views around the table concerning what an adequate evaluation might entail, what should my role be in the discussion? An advocate for a certain perspective, such as my own? A facilitator who tries to identify the ‘common ground’ shared by all participants? Do I need to be particularly responsive to the foundation’s views, given that the foundation would be the one paying me if the evaluation took place? What does the foundation actually mean when it says it wants a ‘comprehensive evaluation?’ To what degree do any or all of these issues have ethical implications?” You realize that you better say something soon.
The board of education School administrators Teachers Parents A local human services organization that specializes in youth-oriented interventions. This organization is the prime candidate to design and offer the life-skills program. • A small, prestigious, private philanthropic foundation that is considering funding the program for its first three years. The foundation has called the meeting, which is being facilitated by one of the foundation’s program officers. The foundation has made it clear that its willingness to fund the life-skills program is contingent on comprehensive needs assessment, implementation, and outcome data being gathered as part of a systematic evaluation of the program. Your previous work with both the foundation and the local human services organization has resulted in your being recommended by both of these stakeholders to conduct the evaluation. A school administrator has just asked you about the types of information that are routinely gathered in such evaluations. In the course of your answer, you describe the varieties of behavior (sexual and otherwise) that standardized needs assessment surveys in this domain often focus upon. As you conclude your description, you look around the conference table and notice that the color has drained from the faces of most of the school board representatives, administrators, teachers, and parents. A flurry of loudly voiced objections immediately cascade into the conversation, of which the following are but a sample: • “There’s no way parents are going to let you ask their kids about THAT!!!” • “Why can’t we simply see if the birth rate goes down after the program is introduced? Why do we need all of this other stuff?” 83
84
AMERICAN JOURNAL OF EVALUATION, 24(1), 2003
• “If I let a survey ask these questions in my school on a Monday afternoon, I’ll be out of my job as principal by Tuesday morning!” • “What happens when some nosy newspaper reporter gets hold of the results? Can you imagine the headlines?” The representatives of the human services organization are conspicuously silent throughout this eruption, while the foundation folks continue to assert that their financial support requires a comprehensive evaluation to be performed, without commenting specifically on the objections that have been raised. They seem to be looking to you for sage counsel and group facilitation that will resolve the matter. The implicit message appears to be, “You are the evaluator, after all. Help everybody see what needs to be done here.” As the gaze of just about every individual in the room returns to you, you’re thinking, “What really does need to be done here? Is a ‘comprehensive evaluation’ involving the collection of highly sensitive data feasible in this situation? Would not collecting such data simply reinforce a community climate that is contributing to the problems the program is supposed to address? To the extent that there are conflicting views around the table concerning what an adequate evaluation might entail, what should my role be in the discussion? An advocate for a certain perspective, such as my own? A facilitator who tries to identify the ‘common ground’ shared by all participants? Do I need to be particularly responsive to the foundation’s views, given that the foundation would be the one paying me if the evaluation took place? What does the foundation actually mean when it says it wants a ‘comprehensive evaluation?’ To what degree do any or all of these issues have ethical implications?” You realize that you better say something soon.