Using experts’ experiences through stories in teaching new product development11

The Journal of Product Innovation Management 19 (2002) 54 – 68

Using Experts’ Experiences Through Stories In Teaching New Product Development Julia´n Herna´ndez-Serranoa, Spiro E. Stefanoub,*, Lamartine F. Hoodb, Barry L. Zoumasb a

b

University of Puerto Rico, Humacao, Puerto Rico Pennsylvania State University, University Park, PA, USA Accepted 17 August 2001

“An expert is a person who has made all the mistakes that can be made in a very narrow field.” Niels Bohr (Nobel Prize, Physics, 1922)

Abstract The situations that new product development students typically have to face in the workplace are filled with unstructured problems. A systematic collection and organization of a number of experts’ experiences, encoded as cases into a case library, and presented in the form of stories to learners while interacting with a task environment is considered by many scholars as a way to increase unstructured problem-solving skill in these learners. Despite the acknowledged potential of case libraries in other fields, no case libraries have been built to support the teaching of the NPD process. Prior to developing one, we conducted a study to shed some light into the effects of experts’ stories and unstructured problem solving. Forty-four undergraduates were subjected to the following: a pretest, random assignment to one of three groups (experimental or case library with stories, comparable or fact sheet with material comparable to the stories but presented as facts, and control or text randomly selected from a textbook unrelated to the material), and post test. The tests attempt to measure whether the experimental group incorporated the lessons to be learned from the stories in the case library. The results indicate that indeed the case library supported unstructured problem-solving skills when compared to fact sheets or random text. General implications for new product development educators are provided. © 2002 Elsevier Science Inc. All rights reserved.

1. Introduction College graduates, the emerging professionals in business and industry, will be expected to possess skills in problem solving, critical thinking, teamwork, and knowledge integration [32]. They will rely on solid undergraduate educational experiences to prepare them for the projected challenges in the work force. In fact, educators are being urged to regard the development of problem-solving skills in learners as one of the most important goals of education [9]. However, undergraduates usually are not trained to solve industry problems in their coursework. With their solutions to problems being too simplistic and, often, not applicable to real world situations, undergraduates are not exposed to the complex dynamic situations that profession-

* Corresponding author. Tel.: ⫹1-814-863-8635; fax: ⫹1-814-8653746. E-mail address: [email protected] (S.E. Stefanou).

als face in industry which are considered necessary for developing expertise [11,31]. When novice product developers enter the workforce, they usually lack the necessary skills and are ill equipped to solve problems.1 New product development (NPD) problems do not have unique solutions, are open-ended, are composed of many subproblems, frequently have many possible solution paths, and possess no clear beginning or end [11,16,30]. Such problems are necessarily unstructured. On the other hand, undergraduates are usually familiar with the problems typically found at the end of textbook chapters. These problems by nature are wellstructured, characterized as usually having a correct and convergent answer or solution, involve well-constrained parameters involving a limited number of rules and concepts, and a knowable process for arriving at a preferred solution [11]. Thus, the status quo does not prepare students to contribute effectively to the product development workforce upon graduation. Teaching unstructured problem-solving skills at the undergraduate level is crit-

0737-6782/02/$ – see front matter © 2002 Elsevier Science Inc. All rights reserved. PII: S 0 7 3 7 - 6 7 8 2 ( 0 1 ) 0 0 1 1 8 - 7

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

ical for preparing students to become effective product development professionals. This article articulates an approach to promoting the development of problem-solving ability when faced with unstructured problems in NPD. The central theme is that novice product developers will benefit greatly by being exposed to the experiences of product development professionals2 through NPD stories. By collecting hundreds— even thousands— of these stories covering various facets of the NPD process, and making these stories easily accessible in a case library to product development novices, it is expected that these novices will become better problemsolvers. Novices will avail themselves of the expertise contained in the case library when facing similar problem situations. In addition, these stories are expected to become part of the repertoire of exemplars that they can apply during future problem-solving needs. It should be recognized that that a NPD case library is necessarily context bound; that is, a case library should be developed for each specific industry. This article describes the creation of a case library for product development in the food industry and presents the results from a study applying a case library to a group of undergraduate students in support of a case presenting the development of a new pasta product.3

2. Stories, learning and problem-solving Schank and his colleagues [27,28] are early proponents for using stories to promote problem-solving skills and find that relating and listening to stories is an important element in learning. Brunner [2], Polkinghorne [23], p. 51, and Randall [24], p. 13, and also argue that since the sharing of stories through our lives is so important, we must possess some kind of “narrative intelligence” allowing one to formulate or follow a story. They all recognize stories as being part of our cognitive system for thinking, explaining, understanding, remembering and memorizing. Others go as far as hypothesizing that acquiring an experience indirectly through a story is tantamount to going through the experience oneself. The memory structures used for understanding the story are the same as those used to carry out the task. Therefore, presenting the experiences of expert problem solvers in a field to novice problem solvers in that field can prove to be beneficial to the novices [8].4 It follows that making a case library composed of experts’ experiences available to novices while they problem solve can have the effect of making them better problem solvers. Additional support for the idea that novices can become better problem solvers through stories also finds a foundation in the case-based reasoning (CBR) literature. CBR is founded primarily on the assumption that cases and examples are useful for aiding problem solving through analogical reasoning. This is accomplished by focusing the novice’s attention on what is important, making ideas available on how to move forward, and providing grounds for preas-

55

sessing the consequences of their decisions or actions [15]. Given the absence of previous experiences by novices, experiences available through a case library are expected to augment their repertoire of experiences by connecting with those of others (i.e. experts). Since CBR focuses on prior experiences (own and others’), these prior experiences serve as a basis for interpreting a future situation, forewarning one of potential problems, realizing what to avoid, and foreseeing the consequences of decisions or actions. Jonassen [11] proposes that instructional materials geared toward supporting unstructured problem-solving skills should incorporate cases representing real-world, authentic problems. The obvious source of these cases is by interviewing practitioners. Schank et al. [27], p. 48, propose that learning environments promoting problem solving should have cases where the student is presented with the nature, applicability, and consequences of skills. In addition, learning environments (specifically those following cognitive apprenticeship prescriptions) should have ways to incorporate demonstrations on how experts solve problems [4]. Students only see worked-out solutions in textbooks that do not show the false starts or dead ends characterizing real-world problem solving. By having an expert reflect on how she solves problems, learners can benefit greatly by observing this otherwise covert problem-solving process at work. Experts more frequently than not are found to rely on a repertoire of examples— case histories and explanations articulated in the form of stories—when wrestling with problematic issues. The stories emerge from cases. For example, the case of developing a new food product can present many different stories that can be of learning value in different parts of the NPD process. When a practitioner tries to explain a complex situation to a co-worker, the problem may be framed as a story and it is this story that provides the listener with the close encounter to the firsthand experience [29]. The solutions to problems learned this way quickly become part of the repertoire for future problem solving. Stories promote understanding, enrich explanations and help the learner arrive at diagnoses to problems. Stories also help in the teaching and learning of new methods by alleviating uncertainty, helping adopt new perspectives on problems, forewarning about failures, providing solutions, expanding the problem space, helping identify causes to problems, and, helping anticipate future problems [18,21].5 In summary, the major reasons these and many other scholars suggest that exposing novices to stories will promote problem-solving skills are: Y stories embody the language of diagnosis and its processes [21], p. 104; Y stories are one of the principal human gifts for presenting, dramatizing, and explicating conflicts [2], p. 95; Y stories have the advantage of credibility because they are based on personal experience [21], p. 127;

56

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

Fig. 1. Case library concept.

Y stories are the most natural form for sharing work experience [21], p. 140; Y stories are packages of situated knowledge [26], p. 108; Y the narrative form and storytelling allow us to construct imaginative “what if” scenarios [23], p. 14; Y stories are embodied in a structure most natural for committing experiences to longer-term memory [3], p. 189; Y stories draw together the laws, personal dispositions, questions of character, actions, and processes used into a whole interpretative “package” when deliberating prior to reaching decisions [23], p. 19, and by showing “the interconnectedness and significance of seemingly random activities” (p. 36); and, Y with stories, problem situations do not appear as detached as impersonal, textbook presentations [22], p. 94). Given the support found in the literature for the application of stories, learning and problem solving, a collection of stories in a case library can contribute positively to novices’ problem-solving skills in a learning environment by providing them support when they are engaged in: Y Determining the source of the problem Y Providing solutions Y Providing explanations (for successes, failures, and unexpected outcomes) Y Drawing conclusions from experts’ experiences Y Predicting the result of actions in given situations

Y Y Y Y Y Y

Making inferences Generating implications Identifying alternative actions Articulating an argument in favor of a decision Stating hypotheses Given a design problem, proposing a design that partially or wholly addresses that problem.

3. The case library A case library is a systematic collection and organization of a number of experts’ experiences presented in the form of stories to the learner as the learner interacts with a task environment [7]. The learning environment can be driven by the case-method, an expert system, a goal-based scenario [25], tutorial or any other computer-based learning or support system. Fig. 1 provides a conceptual view of a case library and its relationship with various learning environments. As the learner is interacting with the learning environment, for example, he will most naturally encounter a series of unstructured problems; for example, should the company launch the product nationwide or regionally? Would it be better to contract a marketing firm for the launch or should it be done in-house? At those points in the learning environment where some of these highly unstructured problems need to be faced, a scheme for providing access to the stories in the case library is provided. For example, the learner can be stimulated to think deeply about these issues

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

57

Fig. 2. Sample story.

by providing a template with a partially formed argument in support of launching the product regionally and internationally. Alternatively, the learner can be asked to complete a partially completed electronic worksheet for requesting funds for contracting a marketing firm for the launch, or the material presented in the learning environment has specific built-in links enticing the learner to learn more about a thorny issue. Upon either submitting a template, completing a worksheet or clicking on a link, a process of story presentation is triggered by issuing a request to a database through context-specific indexes (e.g. launch ⫹ outsourcing ⫹ marketing ⫹ success), which in turn will search for (and, hopefully, find) a model story from other companies that also faced this situation. Stories contained in a case library must be brief and compact collections of wisdom on very specific contexts that do not distract the problem solver from the task at hand. The novice product developer needs to experience the impact of the story as a quick burst of wisdom. If the story is presented in the form of a narrative, it should be structured in a few brief paragraphs that include the presentation of the problem situation and context, the dilemma being faced by the main character (company or expert), and the resolution and lesson. Fig. 2 presents an example of a story. A story should mimic the quick interaction with an expert following a question, a puzzled look, or a request for advice. Research shows that the interaction between people at work revolves around sharing stories in this form [21,23,29].

3.1. Value-added contribution of a case library Once the learner who listens to (or reads) a story and later is forced to deal with a problem with similar contextual elements, he is expected to remember the outcome of the story (or stories) and will be better prepared to face the new challenge [26]. This allows the learner to modify his plans for successfully achieving a goal, be attentive to particular obstacles, articulate problem-solving methods, recognize the merits of different methods, avoid failures, and, in general, engage in problem-solving behavior that is guided and prompted by the stories he has come in contact with in the past [4]. By presenting the “lesson” to be learned from the story as a fact sheet (i.e. a story without characters, argument, or timeline), Schank [26], p. 90, argues the lesson will be difficult to evaluate and remember, rendering the experience virtually useless. 4. Constructing A case library for NPD Stories and case libraries have been used in the past for enhancing work performance or promoting problem-solving skills.6 Despite the number of case libraries that have been created for research or performance, no case libraries have been created targeting the NPD process specifically. Drawing from the experience of these prior efforts, we illustrate how such a library can be created to promote unstructured problem-solving skills to learners working in NPD. NPD

58

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

Fig. 3. Subskill analysis on identifying the market opportunity.

problems involve a great number of professionals and talents, such as technicians, specialists, brand managers, salespeople, engineers, finance managers and even senior management. With so many perspectives and specific priorities, novices are easily overwhelmed with the complexity of the NPD process, often missing “the big picture.” Following Kolodner’s [14] suggestion regarding the construction of a case library, we embarked on a comprehensive assessment of the tasks in which a typical team of food product developers engages in a day-to-day basis as they bring products to market. A comprehensive assessment of the tasks begins with defining the task domain clearly. Once defined, the task domain is broken into subtasks, with the subtasks broken into their components skills. This process stops when the required tasks are general skills typically possessed by junior or senior undergraduates, such as evaluating, judging and making recommendations. The results of this skills analysis process are partially shown in Fig. 3. Such a formalization of the product development is consistent with the Stage-Gate method [6], which provides a map for guiding product development projects from idea to market. The decision points are the gates assessing the results from previous stages. This analysis shows that developing a product development plan requires: 1) identifying the market opportunity; 2) generating a product idea/concept; 3) evaluating preliminary plans for the purpose of deciding to develop the product; 4) performing research and development on the concept; 5) designing a production plan; 6) seeking management commitment to launch the product; 7) developing a market plan and test; 8) planning the launch; and, 9) tracking the launch. Further explanation of the subtask Identify Market Opportunity in

Fig. 4 reveals that performing this task requires one to: 1) review market opportunity in the context of the firm’s strategic direction, 2) articulate the firm’s competencies, 3) evaluate line expansion, and 4) evaluate line extension. Taking any of these subsubtasks to an even lower level progressively demonstrates the further complexity of the NPD process (noted on the figure as reference to another page where further subtasks are articulated). This skills hierarchy of the NPD process presents a “roadmap” to guide the task of building the NPD case library. Every skill (or “box” identified in the “road map”) identified by the experts will require the elicitation of short stories regarding their experiences during past problemsolving situations. For example, a retired Hershey Foods Corporation executive recounted the sample story contained in Fig. 2 after being presented with the subskills Specify product positioning. This subskill feeds into the skill Design preliminary marketing plan, which in turn feeds into the skill Evaluate preliminary plans for the purpose of deciding to develop the product (shown as number 3 in Fig. 3), which is one of the key skills for achieving the final goal of developing a product development plan. Although identifying skills is a prerequisite effort, the actual learning value is not realized until appropriate learning issues are identified. This involves converting these “boxes” into learning issues. For example, the subskill Specify product positioning specifically responds to the following learning issue: Make/justify product positioning recommendations based on simulated/actual data regarding: Y product attributes

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

59

Fig. 4. Hierarchical skills analysis of the NPD.

Y location Y launch strategy. The process of eliciting stories from a dozen or more experts is more effective and efficient when the interviewer possesses this “roadmap.”7 However, the result from an interview rarely looks like the actual story. The unstructured story captured on an audiotape must be edited to incorporate a minimum number of characteristics embodying a viable learning experience. Edelson [7] suggests the following criteria for constructing stories. One: The stories must have characters. Although the characters can be fictitious, greater credibility is realized when using real characters, such as easily recognizable companies or chief executive officers. Two: The stories must have an identifiable incident or a real-life drama such as the struggle associated with a product failure or the benefits accruing from a success. Three: Each story must deal with specifics and not be structured abstractly as an ensemble of factual information without characters, the tension associated with the product development challenge or a timeline. An effective story needs to make the context clear by couching the storyline in an appropriate timeline by referring specifically to the incidents as they happened, and by describing the main character(s) briefly. Four: Each story must have a point and provide a resolution contained in a lesson to the listener (or reader) regarding the problematic situation experienced by the storyteller (expert). Table 1 presents the recommendations for the story elicitation efforts. It concentrates on at least five story types that are most effective in transferring expert knowledge to novices and specific cognitive outcomes in novices that each story type is expected to promote. For example, the Golden Almond Bar story in Fig. 2 is forewarning novices, among other things, of the complexities involved in linking a particular product positioning and a brand name. It is expected that novices will consider this issue and anticipate a similar problem on a particular product they may be developing, thus preempting the problem before it happens. The greater

the number of stories dealing with product positioning themes to which novices are exposed, the more lessons they will absorb; that is, experts’ reasoning is transferred to novices. There are a number of ways that a case library can be incorporated into a learning environment for promoting unstructured problem-solving skills (Fig. 1). One way of Table 1 Generic story types for creating a case library Generic story typea

Desired cognitive outcome in novicesb

Suggest sample solutions to problems (as raised by particular tasks)

● Planning activities to avoid problems

Suggest warnings against certain solutions (sample failures)

Suggest methods for achieving specific tasks Reveal obstacles to be encountered in the tasks

Illustrate the relative merit of different approaches (methods) or solutions or perspectives

● Repairing problems ● Providing solutions with a high chance for success ● Justifying solutions ● Predicting successful results (based on solution) ● Anticipating problems before they happen ● Identifying unacceptable solutions ● Predicting unsuccessful results (based on warning) ● Planning activities to avoid problems ● Justifying solutions ● Anticipating problems before they happen ● Identifying what’s important in problem situation ● Recognizing what good and bad solutions look like

● Evaluating multiple solutions ● Evaluating multiple perspectives a b

Kolodner (1993). Jonassen (1997).

60

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

Fig. 5. Sample story scenario.

incorporating a case library is to create a learning environment that simply engages the novice with a problem, exploits resulting expectation failures and presents the story [7,26].8 This requires designing the learning environment to be capable of capturing, understanding and judging potential solutions for the learning environment to start the story presentation process; that is, the problem has to be highly structured. Another approach is to use the story presentation process for unstructured problem solving. Schank [26] proposes an outcomes-based approach to learning which exploits the natural curiosity that humans have on discovering “the rest of the story.” Once the outcome of the story is known, the novice learns how to react to a future situation with contextual factors similar to those surrounding that story by generalizing from this example and figuring out why the situation turned out the way it did. In this way, a novice revises a conclusion that promotes future understanding by clarifying the significance of events that have occurred and led to a particular outcome [23,26].

5. Empirical evidence for the case library approach to teaching NPD We conducted an experimental study employing an outcomes-based approach to learning by creating an interactive case library associated to a new food product development Web-based case study. In general, students were asked to interact individually with the case, which contained links to stories in the case library placed in strategic points designed to reinforce, expand and elaborate on emerging learning issues as they appeared in the case narrative. Fig. 5 shows three of these various links represented by icons leading novices into learning about important issues

being covered in the case. While the learning environment at the level of the first icon deals with identifying a brand name for a future product that will be accepted in all regions of the U.S. (in this case, a refrigerated pasta), the pasta icon lures the novice—when she slides the mouse over the icon—to read a story showing how the Hershey Foods Corporation failed in a similar effort. Although the story is about the dry pasta business, the lesson to be learned is still applicable. It is expected that novices reading this story think through the issue of selecting national brand names for new products more conscientiously. This, in turn, is expected to support their problem-solving skills as they adopt this story and apply its lesson as required in the immediate learning environment or in a similar future situation. Our study attempts to assess the cognitive effects attained by undergraduates while using a food product development case library in an unstructured problem-solving environment [10]. Access to the case library was provided following an outcomes-based learning approach [26]. The learning environment from which access to the case library was obtained consisted of a section of a new food product development case dealing with the introduction of a new refrigerated pasta product, “Nestle´ Refrigerated Foods: Contadina Pasta & Pizza (A)” [1], specifically the section entitled “Market Potential,” which was placed on the Web, eight Web pages in length. Twenty-four stories were used during the study. These stories offered additional material elaborating on the issues being raised by the case through links placed at strategic points on the Web page displayed (as illustrated in Fig. 5). Students were asked to consult the stories by clicking on the links when they encountered the issues being presented in the case. The stories elaborated on issues such as evaluating the actual demand, evaluating the demand on related prod-

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

61

Fig. 6. Sample fact sheet.

ucts, performing a simulation of consumer purchasing habits, evaluating the consumer profile within new market, gathering qualitative data on retail product concepts, specifying target market, specifying product positioning, and identifying the promotional strategies. Forty-four senior food marketing/retail students in a private university in the northeastern U.S. participated in the study. In this course, students must deal with a number of unstructured problems typical of the food retail/marketing discipline. This course deals with topics such as the forces driving the NPD process, role and importance of NPD to the firm, classification of NPD, strategic planning for NPD, new product organization, new product failures and successes, the NPD process, internal analysis and category assessment, ideation and concept development, sales volume estimates, research and development, and production and logistics, among other issues. Students were randomly assigned to three groups: a) an experimental group with access to the twenty-four food product development stories, b) a comparable group with access to twenty-four fact sheets, and c) a control group with access to twenty-four randomly selected segments of text unrelated to the topics being presented in the stories or fact sheets. The stories were collected from four practitioners ranging in experience from 15 to over 25 years in the areas of sales, marketing, research and development, and food retail. All stories were elicited through interviews and edited to meet the specific criteria identified by Edelson [7]. The four practitioners currently or formerly employed by four different major U.S. food companies. One practitioner was until recently employed as a regional sales manager responsible for all retail clients operating within the major metropolitan centers in the northeastern U.S. (15⫹ years experience as food sales representative). The third practitioner is currently

employed as a vice president of marketing in a pet foods company (15⫹ years marketing experience). The fourth practitioner recently retired from the senior vice presidency of a major food product company (28⫹ years developing new food products). And the fourth practitioner had recently retired as CEO of a large U.S. food retail chain (28⫹ years food retail experience). All story collection sessions took place on the university premises and lasted about 2–3 hr for each expert. The story collection process proceeded as follows. First, we presented the practitioner/expert with the “Market Potential” section of the Contadina case one paragraph at a time. A discussion with the practitioner ensued regarding the learning issues apparent in each paragraph. Based on each learning issue identified by the expert, we proceeded to probe their memory for cases that reminded them of the issues being presented in each paragraph. Thus the storytelling would proceed. More than 40 stories were audio taped. Immediately after, we transcribed and reshaped the unstructured narrative into story form. Finally we edited the stories for clarity and content. A senior instructional design faculty member of another university and an instructional design doctoral graduate student closely associated with this research were asked to review the stories to determine if they supported the learning issues previously identified. This review generated suggestions that were incorporated into the final version of the stories. Once the twenty-four stories were selected and finalized, we divested each story of the characters, the incident, the timeline and plot, leaving only the generalities stemming from the corresponding story and an abstracted lesson; that is, we recasted the story into a fact sheet. Fig. 6 presents an example of a fact sheet corresponding to the story presented in Fig. 2. Likewise, we asked the same senior instructional design faculty member

62

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

Fig. 7. Sample random text.

and the instructional design senior graduate student who reviewed the stories to review the fact sheets and pass judgment on whether the fact sheets corresponded to the stories. Their suggestions were incorporated into the final version of the fact sheets. Finally, the random text was randomly selected from a product development textbook making sure that the material did not touch upon the issued being discussed in the Contadina case or the stories and fact sheets. Fig. 7 presents an example of the random text. Three different websites were created corresponding to each treatment situation. As shown in Fig. 5, the links on the experimental website led the students to the stories. Similarly, links on the comparable website led the students to see the fact sheets, and likewise with the control group, which saw only random text. Prior to initiating the study, all students were trained during one class period on how to interpret and connect the stories and fact sheets to the main issues being raised in the case using a short piece of the Contadina case named “Branding.” It is important to note that at this moment the students had not been assigned to any particular treatment group and the researchers and faculty alike did not know which group they were assigned. Therefore, all students received identical training. Prior to releasing the material, all students were asked to take a pretest to establish a baseline. The pretest consisted of two parts. For the first part, students were presented with a short paragraph relating a short story and were asked to make a selection (in a multiple choice format) explaining a failure, recognize a problem, recognize and adapt a solution, predict an outcome, explain a success, judge alternate strategies/actions, or identify the information needed by choosing from among five choices. The post test, which was administered after three weeks for determining cognitive gain, was identical in format but with different situations.

For both tests, one of the choices embodied a lesson from one or more stories. Meanwhile, during these three weeks between retest and post test, the two faculty teaching the course were discussing other portions of the Contadina case during class making sure that the “Market Potential” section was not discussed openly and always referring students to the study material. After these three weeks, the post test, which was identical in format to the pretest but presenting different situations, was administered for determining cognitive gain. In both the pretest and post test, one of the choices embodied a lesson from one or more stories. 6. Method Table 2 presents an example of a multiple-choice item as used in the post test. It is important to note that the 11 questions as articulated in the multiple-choice tests (with the exception of item 1) were not intended to elicit declarative knowledge or simply recalling of a story. After the problem situation was presented in the root of the multiple-choice test, the questions asked in the post test, and the corresponding outcomes, were as follows (the questions asked in the pretest were virtually identical): Item 1:

Item 2:

Item 3:

This situation is an example of the following: Outcome: Reminding (identify example/outcome) Why do you think this company has failed to make this product a success? Outcome: Identify/explain failure Select the most appropriate solution to the problem Outcome: Identify/recognize problem

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68 Table 2 Multiple-choice test item as used in post test 5.* A major U.S. food company that markets a successful line of salty snack products created a series of snack products to tap the aging baby-boomer population who are starting to experience higher cholesterol levels. Medical research demonstrates clearly that higher levels of cholesterol can contribute to an individual’s incidence of heart disease and stroke. As everyone else, this company regards this population segment as the best hope to fuel growth for the next decade. The products are one of a kind using a new cholesterollowering ingredient, Cholestra, produced by a major pharmaceutical firm. The decision was made to sell the new snack as a line extension of one of its traditional and highly successful products. Against all expectations, the product is not selling. The company is not able to figure out the problem. After spending millions of dollars in R&D, the company watches in horror as its products are being returned even before the shelf life cycle ends. What do you think is the problem the company is facing? a. They do not possess the know-how to market this unique snack product to baby-boomers. b. There is probably no market for a snack health product in the U.S. or anywhere else. c. They are not marketing the product to retailers appropriately. d. They perceive its market strictly in terms of snack food consumption. e. They are operating in a market that is over saturated with snack products. * The “preferred” answer is “d.”

Item 4:

Why do you think the company ended up with this problem? Outcome: Select/recognize solution, adaptation [13] Items 5 and 6: What do you think is the problem the owners are facing? Outcome: Identify/recognize problem Identify/explain failure Item 7: What do you think will most likely happen next and why? Outcome: Identify/predict outcome Item 8: Why do you think Seven Seas’ actions led to the success of the product? Outcome: Identify/explain success Item 9: Instead of this promotion campaign, what other approach you think should have been pursued? Outcome: Identify/judge alternate strategies/actions Item 10: Why do you think the company is not able to sell this product? Outcome: Identify/recognize problem Item 11: What piece of information would you like to have most? Outcome: Identify information needed (on obstacle) The multiple-choice tests (items 1–11) are designed with five possible choices laid out in the following manner: one

63

choice, the “distractor” [20], p. 138, which was unrelated to the issues being discussed; three choices that were entirely plausible; and only one choice that alluded to the lesson to be learned from a story. The preferred choice was randomly associated with a letter from A to E throughout the 11 test items. A score of 1 (one) was given for choosing the preferred answer and 0 (zero) for any other choice. The 44 student scores corresponding to the multiple-choice pretest and post tests were tabulated, identified with each student’s unique access code and classified by group (control, comparable or experimental).

7. Analysis The sample had a gender ratio of 20 males/24 females, all but 1 between the ages of 20 –26 years old, all senior food marketing majors, forty-three Caucasians and one Asian. Most students reported that the primary reason they were taking the course was because it was a graduation requirement. Table 3 presents the ANOVA results for the pretest showing that there were no significant differences between the groups (p ⫽ 0.603, ␣ ⫽ 0.05) suggesting that all groups possessed about the same unstructured problem-solving skill level at the outset. However, we find a significant difference between the groups in the post test (p ⫽ 0.001, ␣ ⫽ 0.05). Table 4 presents the Tukey follow-up results showing the impact on score gain for all three groups revealing that the case library group is the cause of the significant difference in gain when compared to the fact sheets or random text groups (p ⫽ 0.002 and 0.003 respectively, ␣ ⫽ 0.05). The results indicate that the multiple-choice tests offer support to the hypothesis that students who are exposed to a NPD case library will obtain higher scores compared to a group exposed to comparable fact sheets (p ⫽ 0.001) and compared to a group that received random text (p ⫽ 0.008).9 The results also show that within the context of this study, it did not make a difference whether students were exposed to fact sheets or simply random text (p ⫽ 0.754). In addition, the coefficients of variation of student scores as percentages of change from pretests to post tests reveal that only the case library group experiences a sharp drop in variation (⫺39.5%) from pretest to post test (see Table 3). A drop in variation indicates better convergence of treatment scores, which is attributed to the effect the case library had on the students in the experimental group. In summary, the results of the multiple-choice tests support the hypothesis that students who are exposed to a case library will obtain higher scores compared to a group exposed to comparable fact sheets (p ⫽ 0.001, ␣ ⫽ 0.05) and compared to a group receiving random text (p ⫽ 0.008, ␣ ⫽ 0.05). These results offer the strongest evidence yet that a case library can influence students’ decisions given that the

64

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

Table 3 Multiple-choice test results Descriptive Statistics

Gain score

Post test

Pre test

ANOVA N

Mean

Std. dev.

CV (%)

1. Random text 2. Fact sheets 3. Stories

14 16 14

0.000 0.125 2.071

1.359 1.204 1.859

1. Random text 2. Fact sheets 3. Stories

14 16 14

3.643 3.250 5.429

1.447 1.342 1.697

39.7 41.3 31.3

1. Random text 2. Fact sheets 3. Stories

14 16 14

3.643 3.125 3.357

1.336 1.088 1.737

36.7 34.8 51.7

⌬CV (%) 8.2 18.7 ⫺39.5

Sum of squares

df

Mean square

F

P

Between groups

38.481

2 19.240

8.699

0.001

Within groups Total Between groups

90.679 129.159 39.153

41 2.212 43 2 19.576

8.758

0.001

Within groups Total Between groups

91.643 130.795 2.003

41 43 2

0.512

0.603

80.179 82.182

41 43

Within groups Total

2.235 1.002 1.956

Notes: CV ⫽ coefficient of variation. ⌬CV is the percentage change in CV between pretest and post test. There were a total of 11 questions. Mean scores correspond to the choices made that referred to a lesson elaborated on a story.

choices they have before them in the multiple-choice tests that are unstructured. Given the test design where there was not one “correct” answer but rather a preferred answer (i.e. the choice that alluded to one or more lessons contained in the stories in the case library), the students who read the stories generally seem to have been persuaded to decide in accordance with a story’s lesson. Considering that there was only one recall item in the test and ten items eliciting a number of highorder skills, such as explaining, predicting and judging, this result lends support to the hypothesis that a case library will help students obtain higher scores in a multiple-choice test that assesses higher-order cognitive skills.

8. Limitations and further considerations It is important to stress that this study is an attempt to explore the phenomenon of learning through stories. Considering the sparse research combining NPD and unstructured problem solving, the results of the study will stimulate further research building on the findings reported here. However, great care must be taken when interpreting the results. First, cognitive gain was assessed through multiplechoice tests. Given the unstructured nature of problem solving in NPD, less rigid forms of assessment should be considered in future studies, such as open-ended argumentation,

Table 4 Multiple-choice tests Tukey follow-up results (items 1 through 11) Dependent variable

Gain Score

(I) group

Random text Fact sheets Stories

Post Test

Random text Fact sheets Stories Random text

Pretest

Fact sheets Stories

(J) group

Fact sheets Stories Random text Stories Random text Fact sheets Fact sheets Stories Random text Stories Random text Fact sheets Fact sheets Stories Random text Stories Random text Fact sheets

Mean difference (I–J)

Std. error

⫺.1250 ⫺2.0714 .1250 ⫺1.9464 2.0714 1.9464 .3929 ⫺1.7857 ⫺.3929 ⫺2.1786 1.7857 2.1786 .5179 .2857 ⫺.5179 ⫺.2321 ⫺.2857 .2321

.5442 .5621 .5442 .5442 .5621 .5442 .5471 .5651 .5471 .5471 .5651 .5471 .5118 .5286 .5118 .5118 .5286 .5118

p

.971 .002 .971 .003 .002 .003 .754 .008 .754 .001 .008 .001 .574 .852 .574 .893 .852 .893

95% confidence Interval Lower bound

Upper bound

⫺1.4484 ⫺3.4383 ⫺1.1984 ⫺3.2699 .7046 .6230 ⫺.9376 ⫺3.1598 ⫺1.7233 ⫺3.5090 .4116 .8481 ⫺.7266 ⫺.9995 ⫺1.7623 ⫺1.4766 ⫺1.5170 ⫺1.0123

1.1984 ⫺.7046 1.4484 ⫺.6230 3.4383 3.2699 1.7233 ⫺.4116 .9376 ⫺.8481 3.1598 3.5090 1.7623 1.5710 .7266 1.0123 .9995 1.4766

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

open discussion, and so forth Second, the validation of the dependent measures can be problematic. Faced with the extraordinary complexities of the human mind, conducting research on cognitive matters is inherently complex. Further, conducting research on unstructured problem solving raises the level of complexity to an even higher level due to the lack of consensus, even among seasoned NPD experts, regarding the nature of the problem, solution paths, and so forth Therefore, we relied on the professional judgment of six food systems experts from major U.S. corporations for providing and helping “packaging” the stories and on academic experts for evaluating learning environments, assessment instruments, research methods, and so forth, prior to conducting the study. We conducted a pilot study a year prior to this study and built on that experience, particularly in the areas of assessment and learning environment design. Armed with this expertise and prior experience, we designed the assessment instruments with the goal of measuring cognitive gain. Third, the story presentation process, through links leading to text and one picture, may understandably appear simplistic. Our intention is to establish a baseline from which to apply more sophisticated research methods in the future, such as employing multimedia, encouraging smallgroup discussion in and out of class, and so forth Encouraged by the findings presented here, we feel that a range of additional computational resources should also be implemented and researched. Finally, we chose to implement the study as an individual endeavor with the sole intention of managing complexity. Future studies should be designed that investigate the phenomenon of NPD stories and unstructured problem solving in various class formats, such as case-based, problem-based and lecture-based learning formats, and for longer periods of time. Having established that the results are promising at the individual level, now we can continue to improve on the implementation and delivery methods at many levels. While conducting the interviews with experts when collecting the stories, we found out that some experts at first have some difficulty in articulating solutions to problems while using prior experiences related in the form of stories for supporting their arguments. After some coaching, usually all started to feel comfortable with this form of thinking, problem solving and relating stories. The same phenomenon may be taking place with the students; that is, some of them may not feel comfortable of using their “narrative intelligence.” This may be due to the individual differences we all embody. We conjecture that an NPD course supported with a case library will likely encounter a problem with students not be comfortable using their “narrative intelligence.” This problem is exacerbated by the students’ insufficient exposure to storytelling during problem solving given their general lack of field experience. We propose that rigorous training can provide these students the necessary skills to problem solve in narrative form.

65

9. Concluding comments Case libraries have been regarded as particularly useful for supporting problem-solving processes in settings where there are no clear-cut answers [15]; that is, in settings where problems are unstructured. These libraries have been proposed for learning environments that exploit expectation failures [7], as case-based aids [17] and for many other uses [25]. A case library has the potential to promote student learning in teaching the NPD process and is likely to find its most valued use as a learning resource in varied educational formats. The use of the case library is not a substitute for a rigorously organized course. However, case-, problem- and lecture-based course formats can all be complemented via the case library as students follow, understand, recount, question and explain stories. The stories can serve to motivate class discussion about what is going on in industry, or what actually occurred and why it occurred. In the context of case- and problem-based formats, the stories also help provide additional breath of experience that the specific cases or the research project cannot offer alone. As such the stories provide an opportunity to engage in a preliminary evaluation of the consequences of decisions or actions one takes in the case- or problem-based course. The case library is an important resource to consult when the learner faces a problem. With the proper indexing, the case library can be searched for stories addressing issues related to the theme at hand. The case library is also a medium for sharing experiences beyond the casual relating of stories on a one-on-one basis that one may find in a setting when anecdotes are being related. Finally, linking stories to strategic points, as it was done in our study, can be expected to effectively complement computer-based tutorials, manuals and other forms of online materials that do not incorporate cases or stories traditionally. Based on our experience, developing supporting material in the form of a case library does require great commitment. To date, we have engaged a dozen experts. Expertise does not come easy. Experts are usually busy doing what they do best: solving problems. Once entry is obtained in a company, then at best, as stated earlier, one expert leads to another. However, company policy can prohibit discussion of company-related problems. Polkinghorne [9] states that stories in a work setting tend to be primarily driven by a social or joint activity creating culture. This culture provides the basis for continued problem solving through storytelling. Although our research is primarily concerned with the use of stories in an undergraduate setting, we agree with others that learning through stories occurs naturally in a field setting. The more students are exposed to the field, the more they will naturally relate to this form of learning for they have been sufficiently exposed to the storytelling culture. For this reason we feel that graduate students who have been generally exposed to unstructured problems in the field, the lab or in graduate

66

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

courses can benefit more from this approach than undergraduates. Likewise, executive-level courses can likely present the fertile ground for employing this form of learning given their continued exposure to problem solving and storytelling in their respective work areas. We foresee that a case library can stimulate fruitful discussion with the addition of these executive stories acquired after many years of experience. We also foresee a case library as been a great resource for NPD practitioners already engaged in problem solving at many levels (trainees, young practitioners and experts alike) as well as in many areas (a general case libraries dealing with food product development linked to other more specific case libraries dealing with marketing, production, sales, etc.).10 The stories contained within these case libraries will most likely promote practitioners’ “narrative intelligence” as experienced within the corporation. The lessons learned can be expected to be most powerful considering that the characters, incidents, issues, and so forth, are easily identifiable within the context of their daily work or past company culture. Calls for employing storytelling— or exemplars or short cases or even examples—in teaching are increasing [3,19, 25,29] and most likely will result in the future design and implementation of innovative forms of teaching and learning incorporating the storytelling model. Our efforts have been specifically targeted at the NPD process as it relates to promoting problem-solving skills to novices prior to facing the complexities of industry.

4.

5.

6.

Notes 1. For this article, novices are defined as follows: A novice practitioner is a person who has graduated from a university between two to no more than four years prior to this study, and who has been working between at least one but no more than three years full time in the development of new products or some major associated aspect (laboratory analysis, packaging, ingredients, etc.). Although this person may be experienced in some jobs or tasks previously performed, he would be considered a novice in the position he currently holds. A novice learner is an undergraduate student who has less than two years fulltime work experience; this experience is primarily acquired through short-term internships or through summer or part-time work within a product development concern. 2. The same benefits may apply to more experienced practitioners. In this article we focus on novices as undergraduate students. 3. By any measure, the U.S. food processing industry is a dynamic contributor to the U.S. economy. Based on three conventional measures of size (value of shipments, number of employees, value added), it is the

7.

8.

largest or second largest of the 20 industry groups comprising the manufacturing sector. Furthermore, on a global basis, the industry is substantially larger than in any other country, accounting for 22% of the world’s processed food production in dollar terms. The successful development and marketing of new and improved products will be key to maintaining the competitiveness of U.S. food processing companies [5]. For this article, an expert is a person who falls into at least one of the following two categories: 1) who has 25⫹ years of industry experience working in a number of positions reflecting increasing levels of responsibility (ideally starting in a staff-level position, to managing important product development projects, to having obtained a position with executive-level decision-making power such as vice president, CEO, etc.); or 2) who has 15⫹ years of experience specifically working in some specialized aspect of the product development process (specialists in packaging, formulation, regulatory issues, etc.). In the work setting, Herna´ ndez-Serrano [10] finds that stories, and storytelling were important for maintaining meaningful problem-solving discussions, for finding solutions to problems, preserving the collective memory (or history), and acquiring, and learning problem-solving skills. For example, ARCHIE for supporting architectural design plans, JULIA for nutritional purposes, CYCLOPS for landscape design, KRITIK for the design of small mechanical assemblies, CLAVIER for assisting in the monitoring of the manufacture of composite materials, PERSUADER for assisting in solving labor management disputes, YELLO for teaching novice advertising professionals how to sell yellow pages advertising, GuSS for guiding students through the process of social skills development, BROADCAST NEWS for teaching a variety of social studies topics, TAXOPS for teaching about tax opportunities, CREANIMATE for teaching biology topics, and many others [7,12,14,25]. Our experience has been that once the interviewer focuses the expert into a clearly specified skill, the articulation of past experiences happens almost naturally. It is not uncommon to gather a dozen stories in a 11⁄2-2 hour period. Although some experts occasionally require some coaching others are more natural storytellers. The most difficult aspect of starting to collect stories is identifying an initial expertise pool. Once three or four of these experts are engaged, identifying additional experts becomes easier as experts themselves become important contacts for identifying further expertise within the same company or outside of it. An expectation failure emerges when the learner is

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

expecting her solution to be successful in solving the problem but what she finds is that the solution fails to solve the problem. 0 That is, upon offering a solution to the problem, the novice may propose a solution that leads to an unexpected failure of the problem. 9. The gain score determines if there are significant differences between the three groups (control, comparable, and experimental) based on the differences obtained from pretest to post test scores. This score was chosen in this study to determine difference for the following reason. Random selection ensures a homogeneous sample. Theoretically, all pretest scores should be equal. However, minor differences are the norm. For this study, the control group’s pretest average score happened to be the highest score albeit with a statistically insignificant difference when compared to comparable and experimental groups (0.5179 and 0.2857 respectively, Table 4). The random text group did not gain in score from pretest to post test. Therefore, we feel the gain score reflects the cognitive gain more clearly. 10. Preliminary results from in-depth interviews with novices involved in technical tasks of the NPD process at a major US food corporation reveal that these novices, in general, prefer to receive explanations in the form of stories when eliciting expert advice. In addition, these novice practitioners recognize that stories are a legitimate form of learning and that the spontaneous emergence of storytelling in their respective work areas is dependent on the individual preferences of the experts with whom they interact (e.g. chemists, microbiologists, ingredient specialists). Some of the experts turn to storytelling for explanation and elaboration while others turn to charts, tables, technical reports and factual forms of explanation. However, when pressed to explain a particularly complex problem confronted lately, practitioners will articulate it in the form of a story adding support to what other researchers have already found [21,29]. In addition, the attraction of stories as a teaching instrument starts before novices enter the workplace. Novices relate that the most memorable learning experiences while in college were those that involved substantial storytelling on the part of the instructor.

Acknowledgments Funding support from the Higher Educational Challenge Grant and the Agricultural Telecommunications Grant Programs, United States Department of Agriculture is gratefully acknowledged. We also express our appreciation to Professors John Lord and Ned Dunn at St. Joseph’s University for providing access to the pool of students in product development used in the pilot study. The comments of

67

Candice Stefanou, the anonymous referees and the editor are greatly appreciated.

References [1] Bell M, Rangan VK. Nestle´ Refrigerated foods: Contadina Pasta, pizza (A). Boston: Harvard Business School Publishing, 1996. [2] Bruner J. Acts of meaning. Cambridge: Harvard University Press, 1990. [3] Coles R. The call of stories: teaching, and the moral imagination. Boston: Houghton Mifflin Company, 1989. [4] Collins A. Cognitive apprenticeship and instructional technology. In: Idol L, Jones BF, editors. educational values and cognitive instruction: implications for reform. Hillsdale, NJ: Lawrence Erlbaum Associates, 1991. p. 121–38. [5] Connor JM, Schiek W. Food processing: an industrial powerhouse in transition. 2nd ed. New York: Wiley-Interscience, 1997. [6] Cooper RG. Winning at new products: accelerating the process from idea to launch. Cambridge, MA: Perseus Publishing, 2001. [7] Edelson DC. Learning from stories: indexing, and reminding in a socratic case-based teaching system for elementary school biology. Doctoral dissertation, Northwestern University, UMI’s Dissertation Abstracts, 1993. [8] Ferguson W, Bareiss R, Birnbaum L, Osgood R. ASK Systems: an approach to the realization of story-based teachers. The Journal of the Learning Sciences 1991;2(1):95–134. [9] Gagne´ RM. the conditions of learning. 3rd ed. New York: Holt, Rinehart, Winston, 1980. [10] Herna´ ndez-Serrano J. Effects, meaning and learning processes regarding experts’ stories and novice problem-solving in an ill-structured problem-solving environment. Doctoral dissertation, Pennsylvania State University, 2001. [11] Jonassen DH. Instructional design models for well-structured and unstructured problem-solving learning outcomes. Educational Technology: Research and Development 1997;45:65–94. [12] Kass A, Burke R, Blevis E, Williamson M. Constructing learning environments for complex social skills. The Journal of the Learning Sciences 1993/1994;3(4):387– 427. [13] Kolodner JL. An introduction to case-based reasoning. Artificial Intelligence Review 1992;6:3–34. [14] Kolodner JL. Case-based reasoning. San Mateo, CA: Morgan Kaufmann Publishers, Inc., 1993. [15] Kolodner JL. Educational implications of analogy - A view from case-based reasoning. Am Psychol 1997;52(1):57– 66. [16] Kolodner JL, Hmelo CE, Narayanan NH. Problem-based learning meets case-based reasoning. In: Edelson DC, Domeshek EA, editors. Proceedings of the International Conference on the Learning Sciences. Evanston, IL: Northwestern University, 1996, p. 188 –95. [17] Kolodner JL, Guzdial M. Theory and practice of case-based learning aids. In: Jonassen DH, Land SM, editors. Theoretical foundations of learning environments. Mahwah, NJ: Lawrence Erlbaum Associates, 2000. [18] Lave J, Wenger E. Situated learning: legitimate peripheral participation (learning in doing: social, cognitive, and computation perspectives. New York: Cambridge University Press, 1991. [19] McEwan H, Egan K. Narrative in teaching, learning, and research. NY: Teachers College Press, 1995. [20] Nitko AJ. Educational assessment of students. Englewood Cliffs, NJ: Prentice-Hall, Inc., 1996. [21] Orr JE. Talking about machines: an ethnography of a modern job. Ithaca, NY: Cornell University Press, 1996. [22] Paley VG. Looking for magpie: another voice in the classroom. In: McEwan H, Egan K, editors. Narrative in teaching, learning, and research. NY: Teachers College Press, 1995. [23] Polkinghorne D. Narrative knowing, and the human sciences. Albany, NY: State University of New York Press, 1988.

68

J. Herna´ ndez-Serrano et al. / The Journal of Product Innovation Management 19 (2002) 54 – 68

[24] Randall WL. Narrative intelligence and the novelty of our lives. Journal of Aging Studies 1999;13(1):11–28. [25] Schank RC. Tell me a story: a new look at real, and artificial memory. New York: Scribner’s Sons, 1990. [26] Schank RC. Dynamic memory revisited. New York: Cambridge University Press, 1999. [27] Schank RC, Fano A, Jona M, Bell B. The design of goal-based scenarios. The Journal of the Learning Sciences 1993;3(4):305– 46. [28] Schank RC, Cleary C. Engines for education. Hillsdale, NJ: Erlbaum, 1995. [29] Scho¨ n DA. The reflective practitioner: how professionals think in action. New York: Basic Books, 1993.

[30] Sinnott JD. A model for solution of unstructured problems: implications for everyday, and abstract problem solving. In: Sinnott JD, editor. Everyday problem solving: theory, and applications. New York: Praeger, 1989. p. 72–99. [31] Spiro RJ, Feltovich PJ, Jacobson MJ, Coulson RL. Cognitive flexibility, constructivism, and hypertext: random access instruction for advanced knowledge acquisition in unstructured domains. In: Duffy TM, Jonassen DH, editors. Constructivism and the technology of instruction: a conversation. NJ: Lawrence Erlbaum, 1992. p. 57–75. [32] Woolverton MW, Downey WD. A look at agribusiness education since the national agribusiness report: the Lincoln report revisited. American Journal of Agricultural Economics 1999;81(5):1050 –5.