- Email: [email protected]
How mathematicians assign points to student proofs
G Model
ARTICLE IN PRESS
MATBEH-575; No. of Pages 11
Journal of Mathematical Behavior xxx (2017) xxx–xxx
Contents lists available at ScienceDirect
The Journal of Mathematical Behavior journal homepage: www.elsevier.com/locate/jmathb
How mathematicians assign points to student proofs David Miller a,∗ , Nicole Infante a , Keith Weber b a b
West Virginia University, 320 Armstrong Hall, PO Box 6310, Morgantown, WV 26506, United States Rutgers University, 10 Seminary Place, Room 233, New Brunswick, NJ 08901, United States
a r t i c l e
i n f o
Article history: Received 21 November 2016 Received in revised form 13 February 2017 Available online xxx Keywords: Grading Proof Proof evaluation Transition-to-proof course
a b s t r a c t In this paper, we present an exploratory study on the important but under-researched area in undergraduate mathematics education: How do mathematics professors assign points to the proofs that their students submit? We interviewed nine mathematicians while they assigned points to three student-generated proofs from a transition-to-proof course. We observed that (i) One proof that contained a generic sub-proof was evaluated as correct by all nine participants and was given full credit by six participants, (ii) there were ten instances in which a mathematician did not assign full credit to a proof that she evaluated as correct, (iii) there was substantial variation in the points assigned to one proof, and (iv) mathematicians assigned points based not primarily on the correctness of the written artifact that they were given, but rather based on their models of students’ understanding. We discuss the importance of these observations and how they can inform future research. © 2017 Elsevier Inc. All rights reserved.
1. Introduction In the United States, many mathematics majors are required to take a transition-to-proof course prior to taking prooforiented courses such as real analysis and abstract algebra. In a typical transition-to-proof course, the expectation is that students will spend considerable time developing and mastering the mechanics of proof writing. The purpose of this paper is to focus on one aspect of the teaching of a transition-to-proof course: how mathematicians assess the proofs that their students submit for credit. There is a growing body of research on how proof is introduced to university students (e.g., Alcock, 2010; Hemmi, 2006; Nardi, 2008; Moore, 1994; Weber, 2004, 2012) and how students understand proof and related notions after completing a transition-to-proof course (e.g., Hawthorne & Rasmussen, 2015; Moore, 1994; Weber, 2010). There is little research on how proofs are assessed in these courses (Moore, 2016). Grading is an important part of pedagogical practice (e.g., Iannone & Simpson, 2011; Mejia-Ramos et al., 2012; Resnick & Resnick, 1992; Van de Watering, Gijbels, Dochy, & Van der Rijt, 2008); the grades that students receive on the proofs that they submit shape students’ beliefs about what their professors value, what type of product is acceptable as a mathematical contribution, and how they should engage in proof writing. Consequently, understanding how proofs are graded is useful for understanding both the pedagogy of mathematicians and students’ beliefs about proof. In an exploratory study, Moore (2016) found that the four mathematics professors that he interviewed deemed their grading, including both the marks they assigned and the commentary they provided, an essential part of their teaching. In
∗ Corresponding author. E-mail address: [email protected] (D. Miller). http://dx.doi.org/10.1016/j.jmathb.2017.03.002 0732-3123/© 2017 Elsevier Inc. All rights reserved.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model
ARTICLE IN PRESS
MATBEH-575; No. of Pages 11
D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
2
this paper, we present another exploratory study, delving into more depth on several of the issues that Moore looked at while focusing on mathematicians’ perceptions of omissions in student-generated proofs. 2. Related literature 2.1. Proof in university mathematics Proof is widely considered to be a cornerstone of mathematical practice. Consequently, an important aim of enculturating university mathematics students into the discipline of mathematics is engaging them in the activities of reading and writing proofs. A comprehensive review of the proving literature and a detailed discussion on the epistemology of proof is beyond the scope of this paper. However, we briefly review three key findings: First, most mathematics students at the university level struggle to write proofs (e.g., Iannone & Inglis, 2010; Weber, 2001; Weber & Alcock, 2004), even though proof writing is a skill emphasized in many advanced mathematics courses. Second, university mathematics students typically perform poorly when they are asked to distinguish valid proofs from invalid arguments (e.g., Inglis & Alcock, 2012; Selden & Selden, 2003; Weber, 2010), suggesting that students may submit invalid justifications because they cannot distinguish invalid justifications from invalid proofs. Third, some researches have suggested that many students are utterly perplexed about the enterprise of proof (e.g., Mamona-Downs & Downs, 2005). A recent comprehensive review of this literature is given in (in press) Stylianides, Stylianides, & Weber (2017). In mathematics education, there has been an extended debate on what is, or should, constitute a mathematical proof (e.g., Balacheff, 2008; Cirillo et al., 2015). In this paper, we take an agnostic stance on this question and focus instead on what mathematicians consider to be a proof, at least in the context of a transition-to-proof course. Hence, by “proof”, we simply mean a written artifact that a mathematician would evaluate to be a proof. At least in North America, a pivotal experience in students’ enculturation of proof occurs in the context of a transition-toproof course. Mathematics majors typically take such a course in their sophomore or junior year, prior to taking theoretical proof-oriented courses such as abstract algebra and real analysis. In a transition-to-proof course, students are expected to learn both how to write proofs and about the nature of proof itself through a variety of activities, including having students practice applying a variety of proof techniques (e.g., proof by induction, proof by contradiction) across different mathematical contexts (e.g., Alcock, 2010; Moore, 1994). 2.2. Summative assessment in advanced mathematics Summative assessments play an important role in the teaching of advanced mathematics. The assessments given in mathematics courses provide students with a clear indication of what mathematics professors value and exert an influence on the mathematics that students learn (Iannone & Simpson, 2011; Mejia-Ramos et al., 2012; Resnick & Resnick, 1992; Van de Watering et al., 2008). Iannone and Simpson (2011) claimed that research on assessment in collegiate mathematics courses is sparse. The existing research focuses primarily on assessment in calculus (e.g., Boesen, Lithner, & Palm, 2010; Bergqvist, 2007; Lithner, 2006) and the types of questions that students are asked to complete, rather than how students are assessed and how marks are assigned in upper-level proof-oriented courses. In proof-oriented courses, some researchers have claimed that most assessments are largely comprised of proving tasks. For instance, Weber (2001) argued that students’ ability to construct proofs is “typically the only means of assessing students’ performance” (p. 101). Raman (2004) analyzed the exercises related to continuity in a real analysis textbook and found that most consisted of requiring the student to establish that a function was continuous or deduce some conclusion from the hypothesis that a function was continuous. Annie Selden (personal communication) analyzed the exercises in a typical real analysis textbook and found that over 80% were proving tasks. 2.3. Mathematicians’ grading While research on the types of assessment items that students are given in their advanced mathematics courses is emerging, there is little research on how mathematicians assign points to students’ proofs in these courses. Here we summarize the main findings from Moore’s (2016) and Lew and Mejia-Ramos’ (2015) exploratory studies on this topic. Moore (2016) asked four mathematics professors to assign points to seven authentic student proofs with the aim of investigating the consistency, or lack thereof, in the marks that professors assigned. The main findings from Moore’s study were that there was substantial variation in the scores they assigned to some proofs. Further, this variation that Moore observed was usually not due to one mathematician overlooking a flaw in the proof that another mathematician identified. Rather, the mathematicians simply disagreed how many points (if any) should be deducted from the same purported flaw that they both noticed. Moore proposed the following account for these disparities of grading: The mathematicians were using the written artifacts that students submitted to estimate how well students understood the proofs of the statements. The professors all remarked that grading was an important part of their pedagogical practice. Moore (2016) called for qualitative research to better understand the various criteria that professors use to grade proofs. The current study answers this call. Lew and Mejia-Ramos (2015) presented eight mathematicians with proof fragments that were written in awkward unconventional language and they were told these proof fragments were from whole proofs that students submitted for Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
3
credit. The mathematicians were asked if they thought the language was unconventional and if they would take off points for it if they were grading it. Lew and Mejia-Ramos reported that their participants usually found students’ language and notation to be unconventional, but most said they would not deduct points for the awkward phrasings (and sometimes not even remark on it in their feedback to students). The participants said they did not deduct points because the goal of a transition-to-proof course was to teach students how to develop arguments and that they were less concerned with students mastering mathematical language and notation.
2.4. Omissions in proof Proof is often characterized as a connected series of statements where each statement is justified as being a legitimate start of a proof (e.g., an axiom, a definition, a previously established result) or a consequence of previous assertions (e.g., Griffiths, 2000). In practice, proofs are rarely this detailed and often contain omissions. The mathematician Thomas Hales characterizes the difference between idealized complete proofs and the actual proofs that mathematicians publish as follows: “Traditional mathematical proofs are written in a way to make them easily understood by mathematicians. Routine logical steps are omitted. An enormous amount of context is assumed on the part of the reader. Proofs, especially in topology and geometry, rely on intuitive arguments in situations where a trained mathematician would be capable of translating those intuitive arguments into a more rigorous argument. In a formal proof, all the intermediate logical steps are supplied. No appeal is made to intuition, even if the translation from intuition to logic is routine. Thus, a formal proof is less intuitive, and yet less susceptible to logical errors” (as cited in Devlin, 2003). As a working assumption, we used the notions that (i) proofs are logical chains of reasoning beginning from appropriate assumptions to deduce the claim to be proven and (ii) routine logical details in a proof are often omitted when designing our study. To see how this plays out in undergraduate mathematics, consider the following proof from a popular transitionto-proof textbook. Let a and b be natural numbers and let aZ and bZ be the sets of all integer multiples of a and b, respectively. Prove that if a divides b, then bZ ⊆ aZ. Proof. Suppose that a divides b. Then there exists an integer c such that b = ac. Let x ∈ bZ. Then x is a multiple of b, so there exists an integer d such that x = bd. But then x = bd = (ac)d = a(cd). Therefore x is a multiple of a, so x ∈ aZ. (Smith, Eggen, & St. Andre, 2014, p. 73). We note three omissions in this proof. First, the proof does not specify the scope of the variables a and b. Presumably, these variables should be thought of as natural numbers as this was the assumption in the given statement, but this is not explicitly stated in the proof. Second, there is no concluding statement in the proof. That is, the proof does not conclude with the proven statement “if a|b, (bZ ⊆ aZ)” or even the consequent of this implication “(bZ ⊆ aZ)”. Third, there are minor logical gaps in the proof. For instance, the proof does not specify that x is a multiple of a because x is the product of a and an integer. Likewise, the proof does not justify why cd is an integer (i.e., the product of an integer and an integer is an integer). These minor omissions are left for the reader to infer. To avoid misinterpretation, we do not believe that the proof above is deficient or that the proof would be improved by adding text to address these omissions. As Hales noted, these types of omissions are common and have the virtue that they increase the comprehensibility of the proof (see also Davis & Hersh, 1981; Rav, 1999). Further, mathematicians find omissions to be both common and desirable in the pedagogical proofs that they present to students, both for the sake of time and because mathematics professors believe that students can benefit from filling in the gaps of proofs themselves (e.g., Lai & Weber, 2014; Lai, Weber, & Mejia-Ramos, 2012). There are several studies on how mathematicians treat gaps in the proofs that they read in their professional practice, including how gaps affect their judgment of the validity of a proof (e.g., Inglis & Alcock, 2012; Inglis et al., 2013; Weber, 2008). However, there is no research in how they treat omissions in proofs that students submit. The purpose of this study is to explore how omissions in student proofs influence mathematics professor’s judgment on the validity of the proof and the points that they assign to the proof. In this paper, we consider mathematicians’ treatment of three types of omissions in a proof: (i) a gap in which some deductive justifications are left for the reader to infer, (ii) an intuitive argument from an example that is not accompanied with a more formal argument that is based on precise definitions, and (iii) the introduction of variables without specifying their scope. With all this in mind, the goal of this paper is to build on Moore’s study by exploring the following research questions:
• How do mathematics professors treat omissions on proofs when assigning points to student proofs in a transition-to-proof course? Which omissions do they deduct points for and which do they allow? • What factors do mathematicians consider when assigning points to student proofs in a transition-to-proof course?
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
4
3. Methods 3.1. Participants Nine mathematicians in a mathematics department at a large state university in the United States agreed to participate in this study. These mathematicians represented a variety of mathematical subfields, including combinatorics, graph theory, number theory, partial differential equations, topology, and approximation theory. All were tenure-track or tenured at the time of the study with three professors representing each level of Assistant, Associate, and Full Professor. Six of the nine participants had taught a course that introduced proof to the students for the first time. The other three participants (M1, M6, and M7) were assistant professors who had not yet taught an introductory proof, but each had taught upper-level proof courses. These participants were included in our sample to get the perspective of assistant professors. We anonymized the data by referring to the first mathematician that we interviewed as M1, the second as M2, and so on, and assigned a feminine pronoun for each mathematician. 3.2. Materials In this study, we examined proofs of three propositions from number theory that might be proven in a transition-toproof course. For each proposition, we generated two proofs. The Incomplete Proof is a proof that we designed to employ a logically correct line of reasoning but to omit some of the steps of the proof. The Complete Proof (CP) is a modification of the Incomplete Proof in which some of the omitted details were included. We note that the CPs are not fully complete; more logical details could still have been added. Instead, by Complete Proof, we mean that in our judgment, most mathematicians would consider these proofs to be acceptable products that would not be improved by adding more details. We present each proposition and its CP below, placing boxes around the text that we omitted from the Incomplete Proof. If n2 + 1 is a prime number greater than 5, then the digit in the 1’s place of n is 0, 4, or 6.
Proposition 1.
Proof 1. Suppose that n2 + 1 is a prime number greater than 5. Thus n2 + 1 is odd . Then n is of the form 10k + 0, 10k + 2, 10k + 4, 10k + 6, or 10k + 8. Then n2 + 1 is of one of the following forms:
2
(10k + 0) + 1 = 100k2 + 1 = 10 10k
2
+ 1;
2
(10k + 2) + 1 = 100k2 + 40k + 4 + 1 = 10 10k2 + 4k + 5;
2
(10k + 4) + 1 = 100k2 + 80k + 16 + 1 = 10 10k2 + 8k + 1 + 7; 2
2
(10k + 6) + 1 = 100k2 + 120k + 36 + 1 = 10 10k2 + 12k + 3 + 7; (10k + 8) + 1 = 100k2 + 160k + 64 + 1 = 10 10k2 + 16k + 6 + 5. Of these, n = 10k + 2 and n = 10k + 8 yield expressions for n2 + 1 that are clearly divisible by 5 and thus not prime (since n2 + 1 > 5). The only choices for n, then, are those numbers with 1’s digits of 0, 4, or 6. Proposition 2.
There exists a sequence of 100 consecutive integers, none of which is prime.
Proof 2. We will show that the 100 consecutive integers 101! + 2, 101! + 3, 101! + 4, . . ., 101! +101 contains no primes. Since both terms of 101! + 2 contain a factor of 2, 2 is a common factor of 101! + 2 and thus 101! + 2 is not prime. Since both terms a factor of 3, 3 is a common factor of 101! + 3 and thus 101! + 3 is not prime. Similarly, for any of 101! + 3 contain j ∈ 2, 3, 4, . . ., 101 , j is a factor of 101! + j and thus 101! + j is not prime. Thus, the sequence of 100 consecutive integers given above contains no primes. Proposition 3. Given two angles x and y, we define that x ∼ y iff sin2 x + cos2 y = 1. Show that ∼ is an equivalence relation on R. Proof 3.
(Reflexive): Let x ∈ R. Since sin2 x + cos2 x = 1 by Pythagorean Theorem, x ∼ x.
(Symmetric): Let x, y ∈ R and x ∼ y. So sin2 x + cos2 y = 1. Using Pythagorean Theorem on both sin2 x and cos2 y, we have (1 − cos2 x) + (1 − sin2 y) = 1 which implies sin2 y + cos2 x = 1 and hence y ∼ x. (Transitive): Let x, y, and z ∈ R, x ∼ y and y ∼ z. So sin2 x + cos2 y = 1 and sin2 y + cos2 z = 1. Adding up these two equations we have
sin2 x + cos2 y + sin2 y + cos2 z = 2.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
5
Rewriting we have sin2 x + (cos2 y + sin2 y) + cos2 z = 2. Thus sin2 x + 1 + cos2 z = 2 or sin2 x + cos2 z = 1 and hence x ∼ z. Therefore ∼ is an equivalence relation. Incomplete Proof 1 was designed to be a proof that contained logical gaps. Intermediate steps for why n was congruent modulo 0, 2, 4, 6, and 8– specifically that n was even– were not included. Likewise, no justification was given for why n = 10k+2 and n = 10k + 8 were not plausible candidates for n2 + 1 being prime. (That is, we do not say n2 in these cases would be a multiple of 5). The implicit justification included in CP1 was the previous work that showed n2 would be congruent to 5 modulo 10 and hence would be divisible by 5 and thus would be composite. In Incomplete Proof 2, the claim “101! + n will be composite for any integer n between 2 and 101” is only justified for n = 2 but is presented in such a way that this would work for any integer. In the research literature, this type of proof by generic example is most commonly referred to as a generic proof (e.g., Leron & Zaslavsky, 2013; Rowland, 2001). We will use this term in the remainder of the paper. We view this justification as an intuitive argument that could be formalized into a formal justification, such as the one presented in CP2. We note that CP2 contains a redundancy—since the main claim was justified with the arbitrary variable j, it became superfluous to justify the claim in the particular cases where j = 2 or j = 3. However, we left these justifications in for consistency with the design of our study. Incomplete Proof 3 does not mention the scope of the variables (i.e., that x, y, and z are real numbers which is relevant as ∼ as a relation on the real numbers) or a concluding sentence. We are not claiming the Incomplete Proofs are incorrect or even deficient, but rather want to understand mathematicians’ evaluations of these proofs qua student contributions.
3.3. Procedure Each participant met individually with the first two authors and was videotaped during a task-based interview. The interviewers made sure that each interviewee understood that the given proofs were from a transition-to-proof class. Each interview contained three phases. In the Lecture Proof Evaluation phase, the participant was told that a professor presented Incomplete Proof 1 in lecture, asked if they thought the proof was correct and then to comment on the pedagogical quality and appropriateness of the proof. This process was repeated for Incomplete Proof 2 and Incomplete Proof 3. In the Student Proof Evaluation phase, the participant was presented with Incomplete Proof 1 and told that a student submitted that proof for credit. They were asked to evaluate whether Incomplete Proof 1 was correct, assign a score on a ten-point scale to that proof, and explain why they assigned that number of points. They were then shown CP1 and asked to do the same thing. This process was repeated for the two proofs of Proposition 2 and Proposition 3, with the exception that the Incomplete Proof and CP were given to the participants simultaneously. This modification was made because once participants became aware that there would be multiple proofs, they insisted on seeing both proofs before assigning a score. In the Open-Ended Interview phase, each participant was asked general questions about their pedagogical practice with respect to proof, with an emphasis on the grading of proof and the relationship between the proofs students write and the proofs that they view in lecture. These questions were:
• Do you expect the proofs that students hand in to have the same level of rigor as the proofs that the professors present in their lectures? • What are the most important reasons that we expect students to hand in proofs for their homework? • What are you looking for when you read these proofs?
As needed, the interviewers asked clarifying questions about the responses given to these questions.
3.4. Analysis All interviews were transcribed. The authors engaged in thematic analysis (Braun & Clarke, 2006) as follows. First, each author individually familiarized themselves with the data by reading each transcript while flagging and commenting on passages that might be of theoretical interest. We then (i) identified reasons that participants did or did not deduct points for some aspect of a proof, and (ii) what factors the participants identified as important when assigning points to the proof. We met to discuss and compare our findings. We then formulated a list of phenomena of interest that warranted further investigation. Next, each author individually read the transcripts again, searching for occasions in which a participant made a comment related to one of our previously formulated phenomena of interest; we then put this excerpt into a file of comments related to that phenomenon. Through this process, we developed criteria and descriptions of these themes. Finally, we each individually went through the data set and identified passages that would satisfy the criteria for each of these themes.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model
ARTICLE IN PRESS
MATBEH-575; No. of Pages 11
D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
6
Table 1 Mathematicians’ assessment of proofs with gaps authored by students. Proof
M1
M2
M3
M4
M5
M6
M7
M8
M9
1 – Incomplete 1 – CP 2 – Incomplete 2 – CP 3 – Incomplete 3 – CP
6 9 10 10 10 10
9 10 10 10 9 10
8 10 8 10 10 10
9 10 10 10 10 10
9 10 10 10 10 10
7 9 8 10 10 10
10 10 10 10 10 10
6 9 8 10 10 10
8 9 10 10 8 10
4. Results 4.1. Summary of evaluations Although some participants were critical of the pedagogical quality of some of the Incomplete Proofs for lecture purposes, they usually evaluated them to be correct. In the Lecture Proof Evaluation phase of the interview, in all but one instance, the participants judged the Incomplete Proof that they read to be correct. The one exception was when M8 could not decide if Incomplete Proof 1 was correct. In the Student Proof Evaluation phase of the interview, none of the participants changed their evaluations of the correctness of the proof. Hence, there was only one instance (M8 evaluating Incomplete Proof 1) in which a participant evaluated the student proof as incorrect. The points that the participants assigned to the proofs that they evaluated in the Student Evaluation phase are presented in Table 1. There was not agreement among the participants on how many points should be awarded for Incomplete Proof 1, with scores ranging from 6 to 10. For Incomplete Proofs 2 and 3, participants were fairly consistent in assigning points and only a few participants assigned less than 10 points. 4.1.1. Incomplete Proof 1 As noted above, M8 could not decide on the validity of Incomplete Proof 1 and she suspected that it was incorrect. Here we restrict our analysis to the other eight participants. As can be seen from Table 1, there was substantial variation in the points that the participants assigned to Proof 1, with M1 taking off four points from the proof and M7 giving the proof full credit. Hence the scores on this proof provide further support for Moore’s (2016) tentative finding that there may be substantial variation in the ways that professors score proofs. Further, seven participants judged the proof to be “correct” but did not give the proof full credit, which suggests that correct proofs would not necessarily receive full credit. The seven participants who took off points for this proof (recall we excluded M8) almost all did so because of the gaps in the proof that we highlighted in the methods section. The one exception was that four participants also wanted to see motivation for the cases (i.e. reasoning using the Division Algorithm). Despite this, four acknowledged that they would nonetheless be pleased if a student handed this in. For instance, M5 said when grading Proof 1, “I would have a little celebration if I got this for homework”. M1 acknowledged, as she graded Proof 1, that she was not giving full credit for a valid proof, but said: M1: I may reserve a small part for coming up with an elegant solution, but something that uses complete sentences [. . .] But in terms of something which is a correct proof, presented well, has sentences and the logic flows, that is not as elegant as it could be or as streamlined as it could be, I may only take off maybe like 10 percent.1 This provides a potential rationale for why correct proofs might not receive full credit. It also suggests that mathematicians are not expecting perfection from their students and would be pleased with students’ submissions, even if they have some flaws. 4.1.2. Incomplete Proof 2 All participants evaluated Incomplete Proof 2 as correct and six of nine said they would give this proof full credit in a transition-to-proof course. This illustrates how mathematicians sometimes find generic proofs to be acceptable both as proofs in lectures and even as student submissions, at least in the context of a transition-to-proof course. The three participants who took points off did so because the general case was not proven, but two did so only with ambivalence. M6 claimed that it would depend on the student, saying, “if I know the student was good and he understands what he is doing, I would accept it” and “I think if I see they understand it, especially these with the j, I think I wouldn’t take off points. I would accept it like this”. M8 said she would only deduct points in a transition-to-proof course because the point of the course was to learn how to write proofs appropriately; M8 said she would not deduct points if the proof were handed in for a subsequent course. The six participants who did not take points off claimed there was no significant difference in the understanding of the student who wrote Incomplete Proof 2 and CP2 and hence both proofs should receive full credit. These participants seemed
1 The participants’ transcripts were lightly edited to improve their readability. Stutters, repeated words and incomplete phrases were removed. At no point did we add any text and we do not believe we altered the meaning that the participants were intending to convey.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
7
Table 2 Models of Understanding when Assessing Proofs. Theme
Participants
Proofs exhibiting clear understanding get full credit, even if important steps are omitted Good proofs from weaker students are viewed suspiciously Good students get the benefit of the doubt if a step is omitted Certain aspects in a proof denote a lack of understanding and encourage a more critical reading of the proof
M4, M5, M6, M8, and M9 M4, M6, M7, M8, and M9 M3, M6, and M7 M1, M3, M6, and M8
to view the differences as amounting to an issue of notation or formatting. M1’s representative comments are presented below: M1: I really don’t find that to be so horrible here [referring to Incomplete Proof 2]. If I were grading this, this would be a 10. I don’t think that I would take off points for something like that, given that some students are really doing something that is great. [. . .] [CP2] is certainly more polished, but I am convinced that the student that hands in this paper [Incomplete Proof 2] understands what is going on, why those 100 consecutive integers are composite integers. I think this student [CP2] has a more mature handle on the notation and that kind of thing, but having some facility with mathematical notation and abstraction is a skill that will naturally follow, if you can get people to a point where they can distinguish between correct logic and incorrect logic. The other participants expressed similar sentiments, namely that they were pleased with the reasoning in Incomplete Proof 2; to these participants, the differences between the two proofs were not conceptual but due only to notation, and this was a secondary concern in their grading. This finding is surprising in that even some proponents of generic proofs in the mathematics education literature acknowledge that these proofs have limitations (e.g., Leron & Zaslavsky, 2013; Malek and Movshovitz-Hadar, 2011), a point to which we will return in the concluding section. 4.1.3. Incomplete Proof 3 M2 and M9 took points off Incomplete Proof 3 because it did not explicitly give a conclusion. The other seven participants gave Incomplete Proof 2 full credit. All pointed out that they did not think that specifying the scope of x, y, and z added anything to the proof. M3 said, “those extra words don’t give me anything extra – I don’t think that the student gets extra points for being pedantic. The reasoning is the same”. Five participants remarked that this omission was inconsequential as the fact that x, y, and z were real numbers was not used anywhere in the proof. A representative response is given below: M8: They said let x be an element of R, but who cares? It doesn’t get used anywhere. If you are going to go to that level then you need to say something like “since the Pythagorean Theorem applies for all real numbers and not just elements between 0 and 2”. Again in my mind, the point of the course like this is to construct arguments. And that deters from trying to build logical arguments when you start nit picking at that level. 4.2. The emphasis of students’ understanding when assigning points One theme that manifested itself in a variety of ways was that participants claimed that the points that they assigned was dependent upon their perceptions of participants’ understanding of the proof that they submitted. Two participants, M9 and M8, discussed this in general terms. For instance, M9 said while grading Incomplete Proof 1: M9: I think the way that I grade things is you’re trying to see if the student understands and you believe he understands. Not so much that they have every period or word that you are looking for, but did they understand the concept . . . and if you could question them, then they could fill in the gaps, but they may have left them out. (italics were our emphasis). While discussing the third bulleted open-ended interview question (see page 11), M8 lamented that the desire to measure a student’s understanding, rather than the written product of the proof itself, posed an inherent difficulty in assigning points to proofs: “The hard part is trying to make sure a student understands what the symbols and statements [that are written in the proof] actually refer to. How do you know if a student knows what n refers to? How do you know if a student knows what 2 refers to?” In brief, the words that students write down might not accurately convey what they understood. To us, this is an important point. These participants do not decide whether to fill in a gap if the missing steps are “routine” to a mathematician or even to a typical student, but whether they thought the particular student who wrote the proof could fill in the gap. Table 2 shows subthemes for students’ understanding that emerged from the data. In the remainder of the sub-section, we discuss three particular ways that mathematicians would use their estimates of students’ understanding to assign proofs. 4.2.1. The sophistication of the proof Five participants indicated that they would not deduct points for omissions from a proof if it was clear to them that the student understood the proof (i.e. row 1 of Table 2). This occurred when the participants believed that the proof was Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
8
sufficiently sophisticated that it could not be produced without understanding. We give two such excerpts below. While M5 was assessing Incomplete Proof 1, she said: M5: [On why she would not deduct for a particular gap in Incomplete Proof 1] I guess I would say that if somebody was going to write a textbook and this was going to be a sample in the textbook, I would want them to say a little bit more. But anyone who says what is on the paper here wouldn’t say it without understanding it. So jumping from the fact that n squared is even to n is even, that does not bother me at all, M4 explained what she was thinking when assigning points to Incomplete Proof 2 by stating I: What is your reasoning between giving them [Incomplete Proof 2 and CP2] tens? M4: They [the authors of Incomplete Proof 2 and CP2] both see why it [Proposition 2] is true. . .this [referring to CP2] is getting close to being overwritten. So I don’t really want to praise it all that much. This student [the author of the CP2] gets the idea. In this case it is fairly clear, that is enough. So you might want to say something, if you are trying to steer them in the right direction, but I am not sure what. I am so afraid to say anything to sort of lead this person [the author of Incomplete Proof 2] to a proof like that [CP2] [. . .] Occasionally I have seen students with overwritten proofs that you sort of want to tell them to ease up a little and of course the students that don’t provide all the arguments. 4.2.2. The perceived ability of the student Six of the nine participants remarked that their perceptions of the ability of their students influenced their grading (i.e. row 2 and 3 of Table 2). Three participants claimed that they would give a strong student the benefit of the doubt if they saw an omission in the proof, as we illustrate with M7 as she graded Proof 2. M7: It is okay to leave some steps out because I might get to a point where I respect their mathematical minds enough so that I give them the benefit of the doubt that they understood what was going on without writing it down. So that is actually nice, a lot of people do that – in our own research we do that. (italics are our emphasis). A second way in which participants discussed the importance of students’ abilities was by noting that they were suspicious when students of low perceived ability handed in a proof that was particularly elegant or compact on the grounds that the student may have copied the proof from somewhere. We illustrate this with M4’s comments below as she assessed CP 1. I: Would you make any comments on the students’ papers? M4: I would say why? Or explain why? And then I would think, did the student copy this somewhere? [The interviewers and M4 both laugh] Because it is sort of written in a mature style, leaving things out which are yes indeed. As I said before, compact proof, nicely done. Maybe too nicely! In summary, a sizeable minority of the participants would give the benefit of the doubt to a student who they perceived to be strong while the majority of participants would be suspicious of a high quality proof written by a student of low perceived ability. 4.2.3. Particular mistakes in a proof denote a lack of understanding Four participants claimed that there were certain mistakes or extraneous facts in a proof that signaled to them that the student who wrote the proof did not understand the mathematics that they were attempting to convey (i.e. Row 4 of Table 2). Consequently, these participants would examine the proofs more closely. We illustrate this with the transcript from M3 below as she answers the first bulleted open-ended interview question (see page 11). M3: They make category mistakes. They talk about subsets as being elements of something in an inappropriate context . . . That makes me really look at other things with say more skepticism. If they make that big of mistake, it makes me suspect that they are trying to get through on rote and not with any genuine understanding. . . I tend to look at everything that they write with a little more skepticism and I am more likely to find a lapse that I might have missed, if someone had a polished presentation that is more consistent. In the excerpt above, M3 argues that category errors are a signal to her that the student is trying to write a proof without understanding, which leads her to search more carefully for mistakes elsewhere in the proof than she normally would. M8 stressed that mathematicians are pretty good about gauging when a student is trying to feign understanding rather than write a coherent argument. This ability plays a role in breaking her scan of the proof to search more carefully for errors. 5. Discussion 5.1. Summary of findings 5.1.1. Novel findings In this article, we have made two observations that we believe are new to the mathematics education literature. First, there were ten instances in which a participant simultaneously found an Incomplete Proof correct but did not award the proof full credit. There are at least two interpretations of these findings. The first is that correctness is not the only criterion Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
9
that is used by mathematicians for grading proofs. Other criteria, such as elegance and clarity, are also included. This is expressed in M1’s comment that she reserved a small amount of credit for an elegant proof or a proof that uses complete sentences. A second possibility is that whether a proof is correct or not is not being viewed as a binary judgment. Rather, the validity of a proof can be thought of as a continuum where a proof that exceeds a certain threshold is judged as correct. This interpretation allows the possibility that some proofs might be “more correct” than others. In this interpretation, when the participants said Incomplete Proof 1 and Incomplete Proof 2 were “correct”, they were not saying the proofs were flawless, but only that the flaws in the proof were not significant enough to render the proof as incorrect. The second novel finding is that all nine participants regarded Incomplete Proof 2 (which relied on a generic proof) as correct and six of the nine participants assigned full credit for a transition-to-proof student who produced such a proof. (More generally, most deductions that we observed were because the mathematicians felt that more justification was needed to bridge a perceived logical gap). Some mathematics educators have emphasized that generic proofs are not, in fact, proofs. For instance, although Leron and Zaslavsky (2013) highlighted the value of generic proofs, they claimed that: “The main weakness of generic proof is obviously, that it does not really prove the theorem. The ‘fussiness’ of the full, formal, deductive proof is necessary to ensure that the theorem’s conclusion infallibly follows from its premises” (Leron & Zaslavsky, 2013; p. 27). Malek and Movshovitz-Hadar (2011) even objected to labeling such an argument as a “generic proof” since these researchers believed such an argument is clearly not a proof; Malek and Moshovits-Hadar argued that it would be better to call such arguments “transparent pseudo-proofs” instead. The majority of the participants in our study thought the generic proof in Incomplete Proof 2 was both valid in a lecture context and worthy of full credit as a student submission. There are at least two ways to interpret this data. The first is that to some mathematicians, some generic proofs actually are bona fide proofs in some contexts. That is, Leron and Zaslavsky’s (2013) assertion that a “generic proof. . . obviously. . . does not really prove the theorem” (p. 27) is not universally true. This would be consistent with Harel and Sowder’s (2007) claim that generic proofs are instances of transformative proof schemes, which is a sub-category of the deductive proof scheme (i.e., generic proofs are deductive and can perhaps bestow certainty). Alternatively, it might be the case that mathematicians do not intend to give perfect proofs in lectures, but instead aim to enhance understanding. Likewise, their evaluations of students’ work in a transition-to-proof course are based on the quality of students’ reasoning and how well they understood why the assertions are true, rather than on the language that they used to express this. This is consistent with the transcripts in this paper and Lew’s (2016) studies on mathematicians’ beliefs about language in a transition-to-proof course. Investigating these issues further would be a promising avenue of future research. 5.1.2. Corroborative findings There were several other findings in this study that have corroborated and built upon the claims made by others in the literature based on small-scale qualitative studies. First, we found wide variation in how many points were assigned to Incomplete Proof 1, with one mathematician awarding this proof full credit and two mathematicians only assigning this proof a score of 6 out of 10. This replicated Moore’s (2016) result that there is not homogeneity in how mathematicians assign scores to student-generated proofs in a transition-to-proof course. It extends Moore’s findings by showing that this phenomenon can occur even with a proof that mathematicians evaluated as correct. Second, our data corroborated Moore’s (2016) suggestion that when mathematicians grade a student-generated proof, they are not strictly grading the written artifact itself. Instead, they are using the written artifact to form a model of students’ understanding. Further, this extends Moore’s hypothesis by proposing mechanisms for the ways in which models of students’ understanding can influence grading. Better students are sometimes given the benefit of the doubt when omissions occur because the mathematicians believe they can likely fill in the gap. A serious error in a proof can suggest a lack of understanding and lead a mathematician to scrutinize a proof more closely; hence mathematicians would be more likely to identify an error in that proof. Some mathematicians even suggested that if a good proof comes from a weak student, they would be concerned that the proof was copied. Finally, none of the participants were bothered by Incomplete Proof 3, in which the scope of the variables used in the proof was not specified, a finding that Lew and Mejia-Ramos (2015) also reported in their qualitative study with eight mathematicians. 5.2. Limitations of our study There were several limitations to this study. The obvious limitation to this exploratory study is sample size. Only nine participants took part in this study but, as importantly, they only graded the proofs of three propositions. Consequently, care must be taken in generalizing beyond the specific tasks that were used in this study. For instance, we observed that most participants in this study evaluated the generic proof in Incomplete Proof 2 as worthy of full credit. Even if this result generalizes to the broader population of mathematicians (i.e., most mathematicians would give full credit to a student who submitted Incomplete Proof 2 as a proof in a transition-to-proof course), we should be wary of inferring that most mathematicians (or even these participants) would give full credit to any generic proof. To warrant such a conclusion, it would be necessary to conduct this study with more proofs to be graded. Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
10
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
A second limitation concerned our task selection. We did not anticipate that participants would mostly give full credit to Incomplete Proof 2 and Incomplete Proof 3. We consider the observation that participants assigned full credit and their reasons for doing so to be research contributions. Nonetheless, we did not get to observe a wide variety of omissions for which an instructor deducted points. Future research with more problematic student proofs would help address this issue. Finally, we asked participants to evaluate the Incomplete Proofs as lecture proofs before assigning points to them as student contributions. Our hypothesis was that participants would use different standards in evaluating the quality of a lecture proof and a student-generated proof, although this hypothesis wasn’t borne out in our study. Thus, a possible confound of our study was that the participants may have felt internal pressure to label the Incomplete proofs as “correct” to remain consistent with their judgments with the Incomplete proof as lecture proofs. Even though the participants provided coherent explanations for why they had done so, these explanations may have been post hoc rationalizations. We would need to conduct the study without the Lecture Proof Evaluation phase of the study to be certain that this did not make a difference. 5.3. Directions for future research As with any exploratory study, the aim is not to produce conclusive generalizable findings. Rather it is to make observations that can inform future research. We suggest six directions for future research. The first direction concerns the variation that Moore (2016) and our research team found in the points or grades that professors assigned to students’ proofs in a transition-to-proof course. Previous research has found that mathematicians are aware that they sometimes evaluate mathematician-generated proofs differently (e.g., Inglis & Alcock, 2012; Inglis et al., 2013; Weber, 2008). Is this the case with the evaluation of student-generated proofs? That is, do mathematicians think the same student-generated proof in a transition-to-proof course would receive roughly the same marks, independent of grader? If mathematicians were informed that this was not necessarily the case, would they see this as problematic? We also saw that mathematicians’ were not bothered by either the scope of variables in Incomplete Proof 3 or the use of a generic sub-proof in Incomplete Proof 2. The second direction for future research would involve seeing the extent that these results generalized to other proofs. For instance, when would mathematicians deduct points if a student omitted the scope of the variables used within a proof? When would a generic proof or sub-proof be permissible? The third direction is based on our observations that most mathematicians did not take off points for the students writing generic proofs or not listing the scope of variables. Is this because mathematicians thought the proofs were correct? Or were mathematicians assigning full credit for another reason, such as evaluating the proofs against some other criteria (e.g., the quality of students’ reasoning) or viewing the proofs as imperfect but “good enough” for someone struggling to learn a new language? Whether mathematicians desire correctness or something else would be a good topic to consider in future studies. The fourth direction is to compare a professor’s proof presentation in lecture with her evaluation of student proofs. Do professors assign points on proofs based on the emphasis they place on certain aspects of proofs in lectures? For example, do they deduct points for the scope of variables or a general formula in a generic proof, if they have placed importance on these during the proofs they present in lecture. The fifth direction is how students react to the marks that they receive. How do students react when they have points deducted on their assignments? How, and to what extent, do they learn from these experiences? How does this shape what students believe is valued in a proof? Finally, what are students’ perceptions on how their proofs are graded? Do students expect there to be consistency or variation in how their proofs are scored? Are students aware of the various factors that mathematicians take into account when assigning a grade? Despite grading proofs being a crucial part of instruction in advanced mathematics (Moore, 2016), these questions remain largely unexplored. References Alcock, L. (2010). Mathematicians’ perspectives on the teaching and learning of proof. pp. 63–91. Research in collegiate mathematics education (VII). Balacheff, N. (2008). The role of the researcher’s epistemology in mathematics education: An essay on the case of proof. ZDM, 40(3), 501–512. Bergqvist, E. (2007). Types of reasoning required in university exams in mathematics. The Journal of Mathematical Behavior, 26(4), 348–370. Boesen, J., Lithner, J., & Palm, T. (2010). The relation between types of assessment tasks and the mathematical reasoning students use. Educational Studies in Mathematics, 75(1), 89–105. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. Davis, P., & Hersh, R. (1981). The mathematical experience. New York: Viking Penguin Inc. Devlin, K. (2003). The shame of it all. In From Devlin’s Angle, an on-line publication of the Mathematical Association of America. [Downloaded from: http://www.maa.org/external archive/devlin/devlin 06 03.html. Last downloaded: February 7, 2017] Griffiths, P. A. (2000). Mathematics at the turn of the millennium. The American Mathematical Monthly, 107(1), 1–14. Harel, G., & Sowder, L. (2007). Toward comprehensive perspectives on the learning and teaching of proof. pp. 805–842. Second handbook of research on mathematics teaching and learning (2). Hawthorne, C., & Rasmussen, C. (2015). A framework for characterizing students’ thinking about logical statements and truth tables. International Journal of Mathematical Education in Science and Technology, 46(3), 337–353. Hemmi, K. (2006). Approaching proof in a community of mathematical practice. Doctoral dissertation. Stockholm University. Iannone, P., & Inglis, M. (2010). Undergraduate students’ use of deductive arguments to solve ‘prove that. . .’ tasks. Proceedings of the 7th Congress of the European Society for Research in Mathematics Education. Iannone, P., & Simpson, A. (2011). The summative assessment diet: How we assess in mathematics degrees. Teaching Mathematics and Its Applications, 30(4), 186–196.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
11
Inglis, M., & Alcock, L. (2012). Expert and novice approaches to reading mathematical proofs. Journal for Research in Mathematics Education, 43(4), 358–390. Inglis, M., Mejia-Ramos, J. P., Weber, K., & Alcock, L. (2013). On mathematicians’different standards when evaluating elementary proofs. Topics in Cognitive Science, 5, 270–282. Lai, Y., & Weber, K. (2014). Factors mathematicians profess to consider when presentingpedagogical proofs. Educational Studies in Mathematics, 85, 93–108. Lai, Y., Weber, K., & Mejía-Ramos, J. P. (2012). Mathematiciansí perspectives onfeatures of a good pedagogical proof. Cognition and Instruction, 30(2), 146–169. Leron, U., & Zaslavsky, O. (2013). Generic proving: Reflections on scope and method. For the Learning of Mathematics, 33(3), 24–30. Lew, K., & Mejia-Ramos, J. P. (2015). Linguistic norms of undergraduate mathematics proof writing as discussed by mathematicians and understood by students. In K. Bieda (Ed.), Proceedings of the 37th Conference of the North American Chapter of the Psychology of Mathematics Education. Lew, K. (2016). Conventions of the language of mathematical proof writing at the undergraduate level. Unpublished doctoral dissertation: Rutgers University. Lithner, J. (2006). A framework for analysing creative and imitative mathematical reasoning. Department of Mathematics and Mathematical Statistics, Umeå universitet. Malek, A., & Movshovitz-Hadar, N. (2011). The effect of using transparent pseudo-proofs in linear algebra. Research in Mathematics Education, 13(1), 33–58. Mamona-Downs, J., & Downs, M. (2005). The identity of problem solving. The Journal of Mathematical Behavior, 24(3), 385–401. Mejia-Ramos, J. P., Fuller, E., Weber, K., Rhoads, K., & Samkoff, A. (2012). An assessmentmodel for proof comprehension in undergraduate mathematics. Educational Studies in Mathematics, 79(1), 3–18. Moore, R. C. (1994). Making the transition to formal proof. Educational Studies in Mathematics, 27(3), 249–266. Moore, R. C. (2016). Mathematics professors’ evaluations of students’ proofs: A complex teaching process. International Journal of Research in Undergraduate Mathematics Education, 2(2), 246–278. Nardi, E. (2008). Amongst mathematicians: Teaching and learning mathematics at university level. USA: Springer. Raman, M. (2004). Epistemological messages conveyed by three high-school and college mathematics textbooks. The Journal of Mathematical Behavior, 23(4), 389–404. Rav, Y. (1999). Why do we prove theorems? Philosophia Mathematica, 7(1), 5–41. Resnick, L. B., & Resnick, D. P. (1992). Assessing the thinking curriculum: New tools for educational reform. In Changing assessments. Netherlands: Springer [(pp. 37–75). Rowland, T. (2001). Generic proofs in number theory. In S. Campbell, & R. Zazkis (Eds.), Learning and teaching number theory: Research in cognition and instructions (pp. 157–184). Westport, CT: Ablex. Selden, A., & Selden, J. (2003). Validations of proofs considered as texts: Can undergraduates tell whether an argument proves a theorem? Journal for Research in Mathematics Education, 4–36. Smith, D., Eggen, M., & Andre, R. S. (2014). A transition to advanced mathematics. Nelson Education. Stylianides, G., Stylianides, A., & Weber, K. (2017). Research on the teaching and learning of proof: Taking stock and moving forward. In J. Cai (Ed.), Compendium for research in mathematics education (pp. 237–266). Reston, VA: National Council of Teachers of Mathematics. Van de Watering, G., Gijbels, D., Dochy, F., & Van der Rijt, J. (2008). Students’ assessment preferences, perceptions of assessment and their relationships to study results. Higher Education, 56(6), 645–658. Weber, K. (2001). Student difficulty in constructing proofs: The need for strategic knowledge. Educational Studies in Mathematics, 48(1), 101–119. Weber, K. (2004). Traditional instruction in advanced mathematics courses: A case study ofone professor’s lectures and proofs in an introductory real analysis course. Journal of Mathematical Behavior, 23(2), 115–133. Weber, K. (2008). How mathematicians determine if an argument is a valid proof. Journal for Research in Mathematics Education, 39, 431–459. Weber, K. (2010). Mathematics majors’ perceptions of conviction, validity, and proof. Mathematical Thinking and Learning, 12(4), 306–336. Weber, K. (2012). Mathematicians’ perspectives on their pedagogical practice with respect to proof. International Journal of Mathematics Education in Science and Technology, 43, 463–482. Weber, K., & Alcock, L. (2004). Semantic and syntactic proof productions. Educational Studies in Mathematics, 56(3), 209–234.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
ARTICLE IN PRESS
MATBEH-575; No. of Pages 11
Journal of Mathematical Behavior xxx (2017) xxx–xxx
Contents lists available at ScienceDirect
The Journal of Mathematical Behavior journal homepage: www.elsevier.com/locate/jmathb
How mathematicians assign points to student proofs David Miller a,∗ , Nicole Infante a , Keith Weber b a b
West Virginia University, 320 Armstrong Hall, PO Box 6310, Morgantown, WV 26506, United States Rutgers University, 10 Seminary Place, Room 233, New Brunswick, NJ 08901, United States
a r t i c l e
i n f o
Article history: Received 21 November 2016 Received in revised form 13 February 2017 Available online xxx Keywords: Grading Proof Proof evaluation Transition-to-proof course
a b s t r a c t In this paper, we present an exploratory study on the important but under-researched area in undergraduate mathematics education: How do mathematics professors assign points to the proofs that their students submit? We interviewed nine mathematicians while they assigned points to three student-generated proofs from a transition-to-proof course. We observed that (i) One proof that contained a generic sub-proof was evaluated as correct by all nine participants and was given full credit by six participants, (ii) there were ten instances in which a mathematician did not assign full credit to a proof that she evaluated as correct, (iii) there was substantial variation in the points assigned to one proof, and (iv) mathematicians assigned points based not primarily on the correctness of the written artifact that they were given, but rather based on their models of students’ understanding. We discuss the importance of these observations and how they can inform future research. © 2017 Elsevier Inc. All rights reserved.
1. Introduction In the United States, many mathematics majors are required to take a transition-to-proof course prior to taking prooforiented courses such as real analysis and abstract algebra. In a typical transition-to-proof course, the expectation is that students will spend considerable time developing and mastering the mechanics of proof writing. The purpose of this paper is to focus on one aspect of the teaching of a transition-to-proof course: how mathematicians assess the proofs that their students submit for credit. There is a growing body of research on how proof is introduced to university students (e.g., Alcock, 2010; Hemmi, 2006; Nardi, 2008; Moore, 1994; Weber, 2004, 2012) and how students understand proof and related notions after completing a transition-to-proof course (e.g., Hawthorne & Rasmussen, 2015; Moore, 1994; Weber, 2010). There is little research on how proofs are assessed in these courses (Moore, 2016). Grading is an important part of pedagogical practice (e.g., Iannone & Simpson, 2011; Mejia-Ramos et al., 2012; Resnick & Resnick, 1992; Van de Watering, Gijbels, Dochy, & Van der Rijt, 2008); the grades that students receive on the proofs that they submit shape students’ beliefs about what their professors value, what type of product is acceptable as a mathematical contribution, and how they should engage in proof writing. Consequently, understanding how proofs are graded is useful for understanding both the pedagogy of mathematicians and students’ beliefs about proof. In an exploratory study, Moore (2016) found that the four mathematics professors that he interviewed deemed their grading, including both the marks they assigned and the commentary they provided, an essential part of their teaching. In
∗ Corresponding author. E-mail address: [email protected] (D. Miller). http://dx.doi.org/10.1016/j.jmathb.2017.03.002 0732-3123/© 2017 Elsevier Inc. All rights reserved.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model
ARTICLE IN PRESS
MATBEH-575; No. of Pages 11
D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
2
this paper, we present another exploratory study, delving into more depth on several of the issues that Moore looked at while focusing on mathematicians’ perceptions of omissions in student-generated proofs. 2. Related literature 2.1. Proof in university mathematics Proof is widely considered to be a cornerstone of mathematical practice. Consequently, an important aim of enculturating university mathematics students into the discipline of mathematics is engaging them in the activities of reading and writing proofs. A comprehensive review of the proving literature and a detailed discussion on the epistemology of proof is beyond the scope of this paper. However, we briefly review three key findings: First, most mathematics students at the university level struggle to write proofs (e.g., Iannone & Inglis, 2010; Weber, 2001; Weber & Alcock, 2004), even though proof writing is a skill emphasized in many advanced mathematics courses. Second, university mathematics students typically perform poorly when they are asked to distinguish valid proofs from invalid arguments (e.g., Inglis & Alcock, 2012; Selden & Selden, 2003; Weber, 2010), suggesting that students may submit invalid justifications because they cannot distinguish invalid justifications from invalid proofs. Third, some researches have suggested that many students are utterly perplexed about the enterprise of proof (e.g., Mamona-Downs & Downs, 2005). A recent comprehensive review of this literature is given in (in press) Stylianides, Stylianides, & Weber (2017). In mathematics education, there has been an extended debate on what is, or should, constitute a mathematical proof (e.g., Balacheff, 2008; Cirillo et al., 2015). In this paper, we take an agnostic stance on this question and focus instead on what mathematicians consider to be a proof, at least in the context of a transition-to-proof course. Hence, by “proof”, we simply mean a written artifact that a mathematician would evaluate to be a proof. At least in North America, a pivotal experience in students’ enculturation of proof occurs in the context of a transition-toproof course. Mathematics majors typically take such a course in their sophomore or junior year, prior to taking theoretical proof-oriented courses such as abstract algebra and real analysis. In a transition-to-proof course, students are expected to learn both how to write proofs and about the nature of proof itself through a variety of activities, including having students practice applying a variety of proof techniques (e.g., proof by induction, proof by contradiction) across different mathematical contexts (e.g., Alcock, 2010; Moore, 1994). 2.2. Summative assessment in advanced mathematics Summative assessments play an important role in the teaching of advanced mathematics. The assessments given in mathematics courses provide students with a clear indication of what mathematics professors value and exert an influence on the mathematics that students learn (Iannone & Simpson, 2011; Mejia-Ramos et al., 2012; Resnick & Resnick, 1992; Van de Watering et al., 2008). Iannone and Simpson (2011) claimed that research on assessment in collegiate mathematics courses is sparse. The existing research focuses primarily on assessment in calculus (e.g., Boesen, Lithner, & Palm, 2010; Bergqvist, 2007; Lithner, 2006) and the types of questions that students are asked to complete, rather than how students are assessed and how marks are assigned in upper-level proof-oriented courses. In proof-oriented courses, some researchers have claimed that most assessments are largely comprised of proving tasks. For instance, Weber (2001) argued that students’ ability to construct proofs is “typically the only means of assessing students’ performance” (p. 101). Raman (2004) analyzed the exercises related to continuity in a real analysis textbook and found that most consisted of requiring the student to establish that a function was continuous or deduce some conclusion from the hypothesis that a function was continuous. Annie Selden (personal communication) analyzed the exercises in a typical real analysis textbook and found that over 80% were proving tasks. 2.3. Mathematicians’ grading While research on the types of assessment items that students are given in their advanced mathematics courses is emerging, there is little research on how mathematicians assign points to students’ proofs in these courses. Here we summarize the main findings from Moore’s (2016) and Lew and Mejia-Ramos’ (2015) exploratory studies on this topic. Moore (2016) asked four mathematics professors to assign points to seven authentic student proofs with the aim of investigating the consistency, or lack thereof, in the marks that professors assigned. The main findings from Moore’s study were that there was substantial variation in the scores they assigned to some proofs. Further, this variation that Moore observed was usually not due to one mathematician overlooking a flaw in the proof that another mathematician identified. Rather, the mathematicians simply disagreed how many points (if any) should be deducted from the same purported flaw that they both noticed. Moore proposed the following account for these disparities of grading: The mathematicians were using the written artifacts that students submitted to estimate how well students understood the proofs of the statements. The professors all remarked that grading was an important part of their pedagogical practice. Moore (2016) called for qualitative research to better understand the various criteria that professors use to grade proofs. The current study answers this call. Lew and Mejia-Ramos (2015) presented eight mathematicians with proof fragments that were written in awkward unconventional language and they were told these proof fragments were from whole proofs that students submitted for Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
3
credit. The mathematicians were asked if they thought the language was unconventional and if they would take off points for it if they were grading it. Lew and Mejia-Ramos reported that their participants usually found students’ language and notation to be unconventional, but most said they would not deduct points for the awkward phrasings (and sometimes not even remark on it in their feedback to students). The participants said they did not deduct points because the goal of a transition-to-proof course was to teach students how to develop arguments and that they were less concerned with students mastering mathematical language and notation.
2.4. Omissions in proof Proof is often characterized as a connected series of statements where each statement is justified as being a legitimate start of a proof (e.g., an axiom, a definition, a previously established result) or a consequence of previous assertions (e.g., Griffiths, 2000). In practice, proofs are rarely this detailed and often contain omissions. The mathematician Thomas Hales characterizes the difference between idealized complete proofs and the actual proofs that mathematicians publish as follows: “Traditional mathematical proofs are written in a way to make them easily understood by mathematicians. Routine logical steps are omitted. An enormous amount of context is assumed on the part of the reader. Proofs, especially in topology and geometry, rely on intuitive arguments in situations where a trained mathematician would be capable of translating those intuitive arguments into a more rigorous argument. In a formal proof, all the intermediate logical steps are supplied. No appeal is made to intuition, even if the translation from intuition to logic is routine. Thus, a formal proof is less intuitive, and yet less susceptible to logical errors” (as cited in Devlin, 2003). As a working assumption, we used the notions that (i) proofs are logical chains of reasoning beginning from appropriate assumptions to deduce the claim to be proven and (ii) routine logical details in a proof are often omitted when designing our study. To see how this plays out in undergraduate mathematics, consider the following proof from a popular transitionto-proof textbook. Let a and b be natural numbers and let aZ and bZ be the sets of all integer multiples of a and b, respectively. Prove that if a divides b, then bZ ⊆ aZ. Proof. Suppose that a divides b. Then there exists an integer c such that b = ac. Let x ∈ bZ. Then x is a multiple of b, so there exists an integer d such that x = bd. But then x = bd = (ac)d = a(cd). Therefore x is a multiple of a, so x ∈ aZ. (Smith, Eggen, & St. Andre, 2014, p. 73). We note three omissions in this proof. First, the proof does not specify the scope of the variables a and b. Presumably, these variables should be thought of as natural numbers as this was the assumption in the given statement, but this is not explicitly stated in the proof. Second, there is no concluding statement in the proof. That is, the proof does not conclude with the proven statement “if a|b, (bZ ⊆ aZ)” or even the consequent of this implication “(bZ ⊆ aZ)”. Third, there are minor logical gaps in the proof. For instance, the proof does not specify that x is a multiple of a because x is the product of a and an integer. Likewise, the proof does not justify why cd is an integer (i.e., the product of an integer and an integer is an integer). These minor omissions are left for the reader to infer. To avoid misinterpretation, we do not believe that the proof above is deficient or that the proof would be improved by adding text to address these omissions. As Hales noted, these types of omissions are common and have the virtue that they increase the comprehensibility of the proof (see also Davis & Hersh, 1981; Rav, 1999). Further, mathematicians find omissions to be both common and desirable in the pedagogical proofs that they present to students, both for the sake of time and because mathematics professors believe that students can benefit from filling in the gaps of proofs themselves (e.g., Lai & Weber, 2014; Lai, Weber, & Mejia-Ramos, 2012). There are several studies on how mathematicians treat gaps in the proofs that they read in their professional practice, including how gaps affect their judgment of the validity of a proof (e.g., Inglis & Alcock, 2012; Inglis et al., 2013; Weber, 2008). However, there is no research in how they treat omissions in proofs that students submit. The purpose of this study is to explore how omissions in student proofs influence mathematics professor’s judgment on the validity of the proof and the points that they assign to the proof. In this paper, we consider mathematicians’ treatment of three types of omissions in a proof: (i) a gap in which some deductive justifications are left for the reader to infer, (ii) an intuitive argument from an example that is not accompanied with a more formal argument that is based on precise definitions, and (iii) the introduction of variables without specifying their scope. With all this in mind, the goal of this paper is to build on Moore’s study by exploring the following research questions:
• How do mathematics professors treat omissions on proofs when assigning points to student proofs in a transition-to-proof course? Which omissions do they deduct points for and which do they allow? • What factors do mathematicians consider when assigning points to student proofs in a transition-to-proof course?
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
4
3. Methods 3.1. Participants Nine mathematicians in a mathematics department at a large state university in the United States agreed to participate in this study. These mathematicians represented a variety of mathematical subfields, including combinatorics, graph theory, number theory, partial differential equations, topology, and approximation theory. All were tenure-track or tenured at the time of the study with three professors representing each level of Assistant, Associate, and Full Professor. Six of the nine participants had taught a course that introduced proof to the students for the first time. The other three participants (M1, M6, and M7) were assistant professors who had not yet taught an introductory proof, but each had taught upper-level proof courses. These participants were included in our sample to get the perspective of assistant professors. We anonymized the data by referring to the first mathematician that we interviewed as M1, the second as M2, and so on, and assigned a feminine pronoun for each mathematician. 3.2. Materials In this study, we examined proofs of three propositions from number theory that might be proven in a transition-toproof course. For each proposition, we generated two proofs. The Incomplete Proof is a proof that we designed to employ a logically correct line of reasoning but to omit some of the steps of the proof. The Complete Proof (CP) is a modification of the Incomplete Proof in which some of the omitted details were included. We note that the CPs are not fully complete; more logical details could still have been added. Instead, by Complete Proof, we mean that in our judgment, most mathematicians would consider these proofs to be acceptable products that would not be improved by adding more details. We present each proposition and its CP below, placing boxes around the text that we omitted from the Incomplete Proof. If n2 + 1 is a prime number greater than 5, then the digit in the 1’s place of n is 0, 4, or 6.
Proposition 1.
Proof 1. Suppose that n2 + 1 is a prime number greater than 5. Thus n2 + 1 is odd . Then n is of the form 10k + 0, 10k + 2, 10k + 4, 10k + 6, or 10k + 8. Then n2 + 1 is of one of the following forms:
2
(10k + 0) + 1 = 100k2 + 1 = 10 10k
2
+ 1;
2
(10k + 2) + 1 = 100k2 + 40k + 4 + 1 = 10 10k2 + 4k + 5;
2
(10k + 4) + 1 = 100k2 + 80k + 16 + 1 = 10 10k2 + 8k + 1 + 7; 2
2
(10k + 6) + 1 = 100k2 + 120k + 36 + 1 = 10 10k2 + 12k + 3 + 7; (10k + 8) + 1 = 100k2 + 160k + 64 + 1 = 10 10k2 + 16k + 6 + 5. Of these, n = 10k + 2 and n = 10k + 8 yield expressions for n2 + 1 that are clearly divisible by 5 and thus not prime (since n2 + 1 > 5). The only choices for n, then, are those numbers with 1’s digits of 0, 4, or 6. Proposition 2.
There exists a sequence of 100 consecutive integers, none of which is prime.
Proof 2. We will show that the 100 consecutive integers 101! + 2, 101! + 3, 101! + 4, . . ., 101! +101 contains no primes. Since both terms of 101! + 2 contain a factor of 2, 2 is a common factor of 101! + 2 and thus 101! + 2 is not prime. Since both terms a factor of 3, 3 is a common factor of 101! + 3 and thus 101! + 3 is not prime. Similarly, for any of 101! + 3 contain j ∈ 2, 3, 4, . . ., 101 , j is a factor of 101! + j and thus 101! + j is not prime. Thus, the sequence of 100 consecutive integers given above contains no primes. Proposition 3. Given two angles x and y, we define that x ∼ y iff sin2 x + cos2 y = 1. Show that ∼ is an equivalence relation on R. Proof 3.
(Reflexive): Let x ∈ R. Since sin2 x + cos2 x = 1 by Pythagorean Theorem, x ∼ x.
(Symmetric): Let x, y ∈ R and x ∼ y. So sin2 x + cos2 y = 1. Using Pythagorean Theorem on both sin2 x and cos2 y, we have (1 − cos2 x) + (1 − sin2 y) = 1 which implies sin2 y + cos2 x = 1 and hence y ∼ x. (Transitive): Let x, y, and z ∈ R, x ∼ y and y ∼ z. So sin2 x + cos2 y = 1 and sin2 y + cos2 z = 1. Adding up these two equations we have
sin2 x + cos2 y + sin2 y + cos2 z = 2.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
5
Rewriting we have sin2 x + (cos2 y + sin2 y) + cos2 z = 2. Thus sin2 x + 1 + cos2 z = 2 or sin2 x + cos2 z = 1 and hence x ∼ z. Therefore ∼ is an equivalence relation. Incomplete Proof 1 was designed to be a proof that contained logical gaps. Intermediate steps for why n was congruent modulo 0, 2, 4, 6, and 8– specifically that n was even– were not included. Likewise, no justification was given for why n = 10k+2 and n = 10k + 8 were not plausible candidates for n2 + 1 being prime. (That is, we do not say n2 in these cases would be a multiple of 5). The implicit justification included in CP1 was the previous work that showed n2 would be congruent to 5 modulo 10 and hence would be divisible by 5 and thus would be composite. In Incomplete Proof 2, the claim “101! + n will be composite for any integer n between 2 and 101” is only justified for n = 2 but is presented in such a way that this would work for any integer. In the research literature, this type of proof by generic example is most commonly referred to as a generic proof (e.g., Leron & Zaslavsky, 2013; Rowland, 2001). We will use this term in the remainder of the paper. We view this justification as an intuitive argument that could be formalized into a formal justification, such as the one presented in CP2. We note that CP2 contains a redundancy—since the main claim was justified with the arbitrary variable j, it became superfluous to justify the claim in the particular cases where j = 2 or j = 3. However, we left these justifications in for consistency with the design of our study. Incomplete Proof 3 does not mention the scope of the variables (i.e., that x, y, and z are real numbers which is relevant as ∼ as a relation on the real numbers) or a concluding sentence. We are not claiming the Incomplete Proofs are incorrect or even deficient, but rather want to understand mathematicians’ evaluations of these proofs qua student contributions.
3.3. Procedure Each participant met individually with the first two authors and was videotaped during a task-based interview. The interviewers made sure that each interviewee understood that the given proofs were from a transition-to-proof class. Each interview contained three phases. In the Lecture Proof Evaluation phase, the participant was told that a professor presented Incomplete Proof 1 in lecture, asked if they thought the proof was correct and then to comment on the pedagogical quality and appropriateness of the proof. This process was repeated for Incomplete Proof 2 and Incomplete Proof 3. In the Student Proof Evaluation phase, the participant was presented with Incomplete Proof 1 and told that a student submitted that proof for credit. They were asked to evaluate whether Incomplete Proof 1 was correct, assign a score on a ten-point scale to that proof, and explain why they assigned that number of points. They were then shown CP1 and asked to do the same thing. This process was repeated for the two proofs of Proposition 2 and Proposition 3, with the exception that the Incomplete Proof and CP were given to the participants simultaneously. This modification was made because once participants became aware that there would be multiple proofs, they insisted on seeing both proofs before assigning a score. In the Open-Ended Interview phase, each participant was asked general questions about their pedagogical practice with respect to proof, with an emphasis on the grading of proof and the relationship between the proofs students write and the proofs that they view in lecture. These questions were:
• Do you expect the proofs that students hand in to have the same level of rigor as the proofs that the professors present in their lectures? • What are the most important reasons that we expect students to hand in proofs for their homework? • What are you looking for when you read these proofs?
As needed, the interviewers asked clarifying questions about the responses given to these questions.
3.4. Analysis All interviews were transcribed. The authors engaged in thematic analysis (Braun & Clarke, 2006) as follows. First, each author individually familiarized themselves with the data by reading each transcript while flagging and commenting on passages that might be of theoretical interest. We then (i) identified reasons that participants did or did not deduct points for some aspect of a proof, and (ii) what factors the participants identified as important when assigning points to the proof. We met to discuss and compare our findings. We then formulated a list of phenomena of interest that warranted further investigation. Next, each author individually read the transcripts again, searching for occasions in which a participant made a comment related to one of our previously formulated phenomena of interest; we then put this excerpt into a file of comments related to that phenomenon. Through this process, we developed criteria and descriptions of these themes. Finally, we each individually went through the data set and identified passages that would satisfy the criteria for each of these themes.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model
ARTICLE IN PRESS
MATBEH-575; No. of Pages 11
D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
6
Table 1 Mathematicians’ assessment of proofs with gaps authored by students. Proof
M1
M2
M3
M4
M5
M6
M7
M8
M9
1 – Incomplete 1 – CP 2 – Incomplete 2 – CP 3 – Incomplete 3 – CP
6 9 10 10 10 10
9 10 10 10 9 10
8 10 8 10 10 10
9 10 10 10 10 10
9 10 10 10 10 10
7 9 8 10 10 10
10 10 10 10 10 10
6 9 8 10 10 10
8 9 10 10 8 10
4. Results 4.1. Summary of evaluations Although some participants were critical of the pedagogical quality of some of the Incomplete Proofs for lecture purposes, they usually evaluated them to be correct. In the Lecture Proof Evaluation phase of the interview, in all but one instance, the participants judged the Incomplete Proof that they read to be correct. The one exception was when M8 could not decide if Incomplete Proof 1 was correct. In the Student Proof Evaluation phase of the interview, none of the participants changed their evaluations of the correctness of the proof. Hence, there was only one instance (M8 evaluating Incomplete Proof 1) in which a participant evaluated the student proof as incorrect. The points that the participants assigned to the proofs that they evaluated in the Student Evaluation phase are presented in Table 1. There was not agreement among the participants on how many points should be awarded for Incomplete Proof 1, with scores ranging from 6 to 10. For Incomplete Proofs 2 and 3, participants were fairly consistent in assigning points and only a few participants assigned less than 10 points. 4.1.1. Incomplete Proof 1 As noted above, M8 could not decide on the validity of Incomplete Proof 1 and she suspected that it was incorrect. Here we restrict our analysis to the other eight participants. As can be seen from Table 1, there was substantial variation in the points that the participants assigned to Proof 1, with M1 taking off four points from the proof and M7 giving the proof full credit. Hence the scores on this proof provide further support for Moore’s (2016) tentative finding that there may be substantial variation in the ways that professors score proofs. Further, seven participants judged the proof to be “correct” but did not give the proof full credit, which suggests that correct proofs would not necessarily receive full credit. The seven participants who took off points for this proof (recall we excluded M8) almost all did so because of the gaps in the proof that we highlighted in the methods section. The one exception was that four participants also wanted to see motivation for the cases (i.e. reasoning using the Division Algorithm). Despite this, four acknowledged that they would nonetheless be pleased if a student handed this in. For instance, M5 said when grading Proof 1, “I would have a little celebration if I got this for homework”. M1 acknowledged, as she graded Proof 1, that she was not giving full credit for a valid proof, but said: M1: I may reserve a small part for coming up with an elegant solution, but something that uses complete sentences [. . .] But in terms of something which is a correct proof, presented well, has sentences and the logic flows, that is not as elegant as it could be or as streamlined as it could be, I may only take off maybe like 10 percent.1 This provides a potential rationale for why correct proofs might not receive full credit. It also suggests that mathematicians are not expecting perfection from their students and would be pleased with students’ submissions, even if they have some flaws. 4.1.2. Incomplete Proof 2 All participants evaluated Incomplete Proof 2 as correct and six of nine said they would give this proof full credit in a transition-to-proof course. This illustrates how mathematicians sometimes find generic proofs to be acceptable both as proofs in lectures and even as student submissions, at least in the context of a transition-to-proof course. The three participants who took points off did so because the general case was not proven, but two did so only with ambivalence. M6 claimed that it would depend on the student, saying, “if I know the student was good and he understands what he is doing, I would accept it” and “I think if I see they understand it, especially these with the j, I think I wouldn’t take off points. I would accept it like this”. M8 said she would only deduct points in a transition-to-proof course because the point of the course was to learn how to write proofs appropriately; M8 said she would not deduct points if the proof were handed in for a subsequent course. The six participants who did not take points off claimed there was no significant difference in the understanding of the student who wrote Incomplete Proof 2 and CP2 and hence both proofs should receive full credit. These participants seemed
1 The participants’ transcripts were lightly edited to improve their readability. Stutters, repeated words and incomplete phrases were removed. At no point did we add any text and we do not believe we altered the meaning that the participants were intending to convey.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
7
Table 2 Models of Understanding when Assessing Proofs. Theme
Participants
Proofs exhibiting clear understanding get full credit, even if important steps are omitted Good proofs from weaker students are viewed suspiciously Good students get the benefit of the doubt if a step is omitted Certain aspects in a proof denote a lack of understanding and encourage a more critical reading of the proof
M4, M5, M6, M8, and M9 M4, M6, M7, M8, and M9 M3, M6, and M7 M1, M3, M6, and M8
to view the differences as amounting to an issue of notation or formatting. M1’s representative comments are presented below: M1: I really don’t find that to be so horrible here [referring to Incomplete Proof 2]. If I were grading this, this would be a 10. I don’t think that I would take off points for something like that, given that some students are really doing something that is great. [. . .] [CP2] is certainly more polished, but I am convinced that the student that hands in this paper [Incomplete Proof 2] understands what is going on, why those 100 consecutive integers are composite integers. I think this student [CP2] has a more mature handle on the notation and that kind of thing, but having some facility with mathematical notation and abstraction is a skill that will naturally follow, if you can get people to a point where they can distinguish between correct logic and incorrect logic. The other participants expressed similar sentiments, namely that they were pleased with the reasoning in Incomplete Proof 2; to these participants, the differences between the two proofs were not conceptual but due only to notation, and this was a secondary concern in their grading. This finding is surprising in that even some proponents of generic proofs in the mathematics education literature acknowledge that these proofs have limitations (e.g., Leron & Zaslavsky, 2013; Malek and Movshovitz-Hadar, 2011), a point to which we will return in the concluding section. 4.1.3. Incomplete Proof 3 M2 and M9 took points off Incomplete Proof 3 because it did not explicitly give a conclusion. The other seven participants gave Incomplete Proof 2 full credit. All pointed out that they did not think that specifying the scope of x, y, and z added anything to the proof. M3 said, “those extra words don’t give me anything extra – I don’t think that the student gets extra points for being pedantic. The reasoning is the same”. Five participants remarked that this omission was inconsequential as the fact that x, y, and z were real numbers was not used anywhere in the proof. A representative response is given below: M8: They said let x be an element of R, but who cares? It doesn’t get used anywhere. If you are going to go to that level then you need to say something like “since the Pythagorean Theorem applies for all real numbers and not just elements between 0 and 2”. Again in my mind, the point of the course like this is to construct arguments. And that deters from trying to build logical arguments when you start nit picking at that level. 4.2. The emphasis of students’ understanding when assigning points One theme that manifested itself in a variety of ways was that participants claimed that the points that they assigned was dependent upon their perceptions of participants’ understanding of the proof that they submitted. Two participants, M9 and M8, discussed this in general terms. For instance, M9 said while grading Incomplete Proof 1: M9: I think the way that I grade things is you’re trying to see if the student understands and you believe he understands. Not so much that they have every period or word that you are looking for, but did they understand the concept . . . and if you could question them, then they could fill in the gaps, but they may have left them out. (italics were our emphasis). While discussing the third bulleted open-ended interview question (see page 11), M8 lamented that the desire to measure a student’s understanding, rather than the written product of the proof itself, posed an inherent difficulty in assigning points to proofs: “The hard part is trying to make sure a student understands what the symbols and statements [that are written in the proof] actually refer to. How do you know if a student knows what n refers to? How do you know if a student knows what 2 refers to?” In brief, the words that students write down might not accurately convey what they understood. To us, this is an important point. These participants do not decide whether to fill in a gap if the missing steps are “routine” to a mathematician or even to a typical student, but whether they thought the particular student who wrote the proof could fill in the gap. Table 2 shows subthemes for students’ understanding that emerged from the data. In the remainder of the sub-section, we discuss three particular ways that mathematicians would use their estimates of students’ understanding to assign proofs. 4.2.1. The sophistication of the proof Five participants indicated that they would not deduct points for omissions from a proof if it was clear to them that the student understood the proof (i.e. row 1 of Table 2). This occurred when the participants believed that the proof was Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
8
sufficiently sophisticated that it could not be produced without understanding. We give two such excerpts below. While M5 was assessing Incomplete Proof 1, she said: M5: [On why she would not deduct for a particular gap in Incomplete Proof 1] I guess I would say that if somebody was going to write a textbook and this was going to be a sample in the textbook, I would want them to say a little bit more. But anyone who says what is on the paper here wouldn’t say it without understanding it. So jumping from the fact that n squared is even to n is even, that does not bother me at all, M4 explained what she was thinking when assigning points to Incomplete Proof 2 by stating I: What is your reasoning between giving them [Incomplete Proof 2 and CP2] tens? M4: They [the authors of Incomplete Proof 2 and CP2] both see why it [Proposition 2] is true. . .this [referring to CP2] is getting close to being overwritten. So I don’t really want to praise it all that much. This student [the author of the CP2] gets the idea. In this case it is fairly clear, that is enough. So you might want to say something, if you are trying to steer them in the right direction, but I am not sure what. I am so afraid to say anything to sort of lead this person [the author of Incomplete Proof 2] to a proof like that [CP2] [. . .] Occasionally I have seen students with overwritten proofs that you sort of want to tell them to ease up a little and of course the students that don’t provide all the arguments. 4.2.2. The perceived ability of the student Six of the nine participants remarked that their perceptions of the ability of their students influenced their grading (i.e. row 2 and 3 of Table 2). Three participants claimed that they would give a strong student the benefit of the doubt if they saw an omission in the proof, as we illustrate with M7 as she graded Proof 2. M7: It is okay to leave some steps out because I might get to a point where I respect their mathematical minds enough so that I give them the benefit of the doubt that they understood what was going on without writing it down. So that is actually nice, a lot of people do that – in our own research we do that. (italics are our emphasis). A second way in which participants discussed the importance of students’ abilities was by noting that they were suspicious when students of low perceived ability handed in a proof that was particularly elegant or compact on the grounds that the student may have copied the proof from somewhere. We illustrate this with M4’s comments below as she assessed CP 1. I: Would you make any comments on the students’ papers? M4: I would say why? Or explain why? And then I would think, did the student copy this somewhere? [The interviewers and M4 both laugh] Because it is sort of written in a mature style, leaving things out which are yes indeed. As I said before, compact proof, nicely done. Maybe too nicely! In summary, a sizeable minority of the participants would give the benefit of the doubt to a student who they perceived to be strong while the majority of participants would be suspicious of a high quality proof written by a student of low perceived ability. 4.2.3. Particular mistakes in a proof denote a lack of understanding Four participants claimed that there were certain mistakes or extraneous facts in a proof that signaled to them that the student who wrote the proof did not understand the mathematics that they were attempting to convey (i.e. Row 4 of Table 2). Consequently, these participants would examine the proofs more closely. We illustrate this with the transcript from M3 below as she answers the first bulleted open-ended interview question (see page 11). M3: They make category mistakes. They talk about subsets as being elements of something in an inappropriate context . . . That makes me really look at other things with say more skepticism. If they make that big of mistake, it makes me suspect that they are trying to get through on rote and not with any genuine understanding. . . I tend to look at everything that they write with a little more skepticism and I am more likely to find a lapse that I might have missed, if someone had a polished presentation that is more consistent. In the excerpt above, M3 argues that category errors are a signal to her that the student is trying to write a proof without understanding, which leads her to search more carefully for mistakes elsewhere in the proof than she normally would. M8 stressed that mathematicians are pretty good about gauging when a student is trying to feign understanding rather than write a coherent argument. This ability plays a role in breaking her scan of the proof to search more carefully for errors. 5. Discussion 5.1. Summary of findings 5.1.1. Novel findings In this article, we have made two observations that we believe are new to the mathematics education literature. First, there were ten instances in which a participant simultaneously found an Incomplete Proof correct but did not award the proof full credit. There are at least two interpretations of these findings. The first is that correctness is not the only criterion Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
9
that is used by mathematicians for grading proofs. Other criteria, such as elegance and clarity, are also included. This is expressed in M1’s comment that she reserved a small amount of credit for an elegant proof or a proof that uses complete sentences. A second possibility is that whether a proof is correct or not is not being viewed as a binary judgment. Rather, the validity of a proof can be thought of as a continuum where a proof that exceeds a certain threshold is judged as correct. This interpretation allows the possibility that some proofs might be “more correct” than others. In this interpretation, when the participants said Incomplete Proof 1 and Incomplete Proof 2 were “correct”, they were not saying the proofs were flawless, but only that the flaws in the proof were not significant enough to render the proof as incorrect. The second novel finding is that all nine participants regarded Incomplete Proof 2 (which relied on a generic proof) as correct and six of the nine participants assigned full credit for a transition-to-proof student who produced such a proof. (More generally, most deductions that we observed were because the mathematicians felt that more justification was needed to bridge a perceived logical gap). Some mathematics educators have emphasized that generic proofs are not, in fact, proofs. For instance, although Leron and Zaslavsky (2013) highlighted the value of generic proofs, they claimed that: “The main weakness of generic proof is obviously, that it does not really prove the theorem. The ‘fussiness’ of the full, formal, deductive proof is necessary to ensure that the theorem’s conclusion infallibly follows from its premises” (Leron & Zaslavsky, 2013; p. 27). Malek and Movshovitz-Hadar (2011) even objected to labeling such an argument as a “generic proof” since these researchers believed such an argument is clearly not a proof; Malek and Moshovits-Hadar argued that it would be better to call such arguments “transparent pseudo-proofs” instead. The majority of the participants in our study thought the generic proof in Incomplete Proof 2 was both valid in a lecture context and worthy of full credit as a student submission. There are at least two ways to interpret this data. The first is that to some mathematicians, some generic proofs actually are bona fide proofs in some contexts. That is, Leron and Zaslavsky’s (2013) assertion that a “generic proof. . . obviously. . . does not really prove the theorem” (p. 27) is not universally true. This would be consistent with Harel and Sowder’s (2007) claim that generic proofs are instances of transformative proof schemes, which is a sub-category of the deductive proof scheme (i.e., generic proofs are deductive and can perhaps bestow certainty). Alternatively, it might be the case that mathematicians do not intend to give perfect proofs in lectures, but instead aim to enhance understanding. Likewise, their evaluations of students’ work in a transition-to-proof course are based on the quality of students’ reasoning and how well they understood why the assertions are true, rather than on the language that they used to express this. This is consistent with the transcripts in this paper and Lew’s (2016) studies on mathematicians’ beliefs about language in a transition-to-proof course. Investigating these issues further would be a promising avenue of future research. 5.1.2. Corroborative findings There were several other findings in this study that have corroborated and built upon the claims made by others in the literature based on small-scale qualitative studies. First, we found wide variation in how many points were assigned to Incomplete Proof 1, with one mathematician awarding this proof full credit and two mathematicians only assigning this proof a score of 6 out of 10. This replicated Moore’s (2016) result that there is not homogeneity in how mathematicians assign scores to student-generated proofs in a transition-to-proof course. It extends Moore’s findings by showing that this phenomenon can occur even with a proof that mathematicians evaluated as correct. Second, our data corroborated Moore’s (2016) suggestion that when mathematicians grade a student-generated proof, they are not strictly grading the written artifact itself. Instead, they are using the written artifact to form a model of students’ understanding. Further, this extends Moore’s hypothesis by proposing mechanisms for the ways in which models of students’ understanding can influence grading. Better students are sometimes given the benefit of the doubt when omissions occur because the mathematicians believe they can likely fill in the gap. A serious error in a proof can suggest a lack of understanding and lead a mathematician to scrutinize a proof more closely; hence mathematicians would be more likely to identify an error in that proof. Some mathematicians even suggested that if a good proof comes from a weak student, they would be concerned that the proof was copied. Finally, none of the participants were bothered by Incomplete Proof 3, in which the scope of the variables used in the proof was not specified, a finding that Lew and Mejia-Ramos (2015) also reported in their qualitative study with eight mathematicians. 5.2. Limitations of our study There were several limitations to this study. The obvious limitation to this exploratory study is sample size. Only nine participants took part in this study but, as importantly, they only graded the proofs of three propositions. Consequently, care must be taken in generalizing beyond the specific tasks that were used in this study. For instance, we observed that most participants in this study evaluated the generic proof in Incomplete Proof 2 as worthy of full credit. Even if this result generalizes to the broader population of mathematicians (i.e., most mathematicians would give full credit to a student who submitted Incomplete Proof 2 as a proof in a transition-to-proof course), we should be wary of inferring that most mathematicians (or even these participants) would give full credit to any generic proof. To warrant such a conclusion, it would be necessary to conduct this study with more proofs to be graded. Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
10
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
A second limitation concerned our task selection. We did not anticipate that participants would mostly give full credit to Incomplete Proof 2 and Incomplete Proof 3. We consider the observation that participants assigned full credit and their reasons for doing so to be research contributions. Nonetheless, we did not get to observe a wide variety of omissions for which an instructor deducted points. Future research with more problematic student proofs would help address this issue. Finally, we asked participants to evaluate the Incomplete Proofs as lecture proofs before assigning points to them as student contributions. Our hypothesis was that participants would use different standards in evaluating the quality of a lecture proof and a student-generated proof, although this hypothesis wasn’t borne out in our study. Thus, a possible confound of our study was that the participants may have felt internal pressure to label the Incomplete proofs as “correct” to remain consistent with their judgments with the Incomplete proof as lecture proofs. Even though the participants provided coherent explanations for why they had done so, these explanations may have been post hoc rationalizations. We would need to conduct the study without the Lecture Proof Evaluation phase of the study to be certain that this did not make a difference. 5.3. Directions for future research As with any exploratory study, the aim is not to produce conclusive generalizable findings. Rather it is to make observations that can inform future research. We suggest six directions for future research. The first direction concerns the variation that Moore (2016) and our research team found in the points or grades that professors assigned to students’ proofs in a transition-to-proof course. Previous research has found that mathematicians are aware that they sometimes evaluate mathematician-generated proofs differently (e.g., Inglis & Alcock, 2012; Inglis et al., 2013; Weber, 2008). Is this the case with the evaluation of student-generated proofs? That is, do mathematicians think the same student-generated proof in a transition-to-proof course would receive roughly the same marks, independent of grader? If mathematicians were informed that this was not necessarily the case, would they see this as problematic? We also saw that mathematicians’ were not bothered by either the scope of variables in Incomplete Proof 3 or the use of a generic sub-proof in Incomplete Proof 2. The second direction for future research would involve seeing the extent that these results generalized to other proofs. For instance, when would mathematicians deduct points if a student omitted the scope of the variables used within a proof? When would a generic proof or sub-proof be permissible? The third direction is based on our observations that most mathematicians did not take off points for the students writing generic proofs or not listing the scope of variables. Is this because mathematicians thought the proofs were correct? Or were mathematicians assigning full credit for another reason, such as evaluating the proofs against some other criteria (e.g., the quality of students’ reasoning) or viewing the proofs as imperfect but “good enough” for someone struggling to learn a new language? Whether mathematicians desire correctness or something else would be a good topic to consider in future studies. The fourth direction is to compare a professor’s proof presentation in lecture with her evaluation of student proofs. Do professors assign points on proofs based on the emphasis they place on certain aspects of proofs in lectures? For example, do they deduct points for the scope of variables or a general formula in a generic proof, if they have placed importance on these during the proofs they present in lecture. The fifth direction is how students react to the marks that they receive. How do students react when they have points deducted on their assignments? How, and to what extent, do they learn from these experiences? How does this shape what students believe is valued in a proof? Finally, what are students’ perceptions on how their proofs are graded? Do students expect there to be consistency or variation in how their proofs are scored? Are students aware of the various factors that mathematicians take into account when assigning a grade? Despite grading proofs being a crucial part of instruction in advanced mathematics (Moore, 2016), these questions remain largely unexplored. References Alcock, L. (2010). Mathematicians’ perspectives on the teaching and learning of proof. pp. 63–91. Research in collegiate mathematics education (VII). Balacheff, N. (2008). The role of the researcher’s epistemology in mathematics education: An essay on the case of proof. ZDM, 40(3), 501–512. Bergqvist, E. (2007). Types of reasoning required in university exams in mathematics. The Journal of Mathematical Behavior, 26(4), 348–370. Boesen, J., Lithner, J., & Palm, T. (2010). The relation between types of assessment tasks and the mathematical reasoning students use. Educational Studies in Mathematics, 75(1), 89–105. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. Davis, P., & Hersh, R. (1981). The mathematical experience. New York: Viking Penguin Inc. Devlin, K. (2003). The shame of it all. In From Devlin’s Angle, an on-line publication of the Mathematical Association of America. [Downloaded from: http://www.maa.org/external archive/devlin/devlin 06 03.html. Last downloaded: February 7, 2017] Griffiths, P. A. (2000). Mathematics at the turn of the millennium. The American Mathematical Monthly, 107(1), 1–14. Harel, G., & Sowder, L. (2007). Toward comprehensive perspectives on the learning and teaching of proof. pp. 805–842. Second handbook of research on mathematics teaching and learning (2). Hawthorne, C., & Rasmussen, C. (2015). A framework for characterizing students’ thinking about logical statements and truth tables. International Journal of Mathematical Education in Science and Technology, 46(3), 337–353. Hemmi, K. (2006). Approaching proof in a community of mathematical practice. Doctoral dissertation. Stockholm University. Iannone, P., & Inglis, M. (2010). Undergraduate students’ use of deductive arguments to solve ‘prove that. . .’ tasks. Proceedings of the 7th Congress of the European Society for Research in Mathematics Education. Iannone, P., & Simpson, A. (2011). The summative assessment diet: How we assess in mathematics degrees. Teaching Mathematics and Its Applications, 30(4), 186–196.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002
G Model MATBEH-575; No. of Pages 11
ARTICLE IN PRESS D. Miller et al. / Journal of Mathematical Behavior xxx (2017) xxx–xxx
11
Inglis, M., & Alcock, L. (2012). Expert and novice approaches to reading mathematical proofs. Journal for Research in Mathematics Education, 43(4), 358–390. Inglis, M., Mejia-Ramos, J. P., Weber, K., & Alcock, L. (2013). On mathematicians’different standards when evaluating elementary proofs. Topics in Cognitive Science, 5, 270–282. Lai, Y., & Weber, K. (2014). Factors mathematicians profess to consider when presentingpedagogical proofs. Educational Studies in Mathematics, 85, 93–108. Lai, Y., Weber, K., & Mejía-Ramos, J. P. (2012). Mathematiciansí perspectives onfeatures of a good pedagogical proof. Cognition and Instruction, 30(2), 146–169. Leron, U., & Zaslavsky, O. (2013). Generic proving: Reflections on scope and method. For the Learning of Mathematics, 33(3), 24–30. Lew, K., & Mejia-Ramos, J. P. (2015). Linguistic norms of undergraduate mathematics proof writing as discussed by mathematicians and understood by students. In K. Bieda (Ed.), Proceedings of the 37th Conference of the North American Chapter of the Psychology of Mathematics Education. Lew, K. (2016). Conventions of the language of mathematical proof writing at the undergraduate level. Unpublished doctoral dissertation: Rutgers University. Lithner, J. (2006). A framework for analysing creative and imitative mathematical reasoning. Department of Mathematics and Mathematical Statistics, Umeå universitet. Malek, A., & Movshovitz-Hadar, N. (2011). The effect of using transparent pseudo-proofs in linear algebra. Research in Mathematics Education, 13(1), 33–58. Mamona-Downs, J., & Downs, M. (2005). The identity of problem solving. The Journal of Mathematical Behavior, 24(3), 385–401. Mejia-Ramos, J. P., Fuller, E., Weber, K., Rhoads, K., & Samkoff, A. (2012). An assessmentmodel for proof comprehension in undergraduate mathematics. Educational Studies in Mathematics, 79(1), 3–18. Moore, R. C. (1994). Making the transition to formal proof. Educational Studies in Mathematics, 27(3), 249–266. Moore, R. C. (2016). Mathematics professors’ evaluations of students’ proofs: A complex teaching process. International Journal of Research in Undergraduate Mathematics Education, 2(2), 246–278. Nardi, E. (2008). Amongst mathematicians: Teaching and learning mathematics at university level. USA: Springer. Raman, M. (2004). Epistemological messages conveyed by three high-school and college mathematics textbooks. The Journal of Mathematical Behavior, 23(4), 389–404. Rav, Y. (1999). Why do we prove theorems? Philosophia Mathematica, 7(1), 5–41. Resnick, L. B., & Resnick, D. P. (1992). Assessing the thinking curriculum: New tools for educational reform. In Changing assessments. Netherlands: Springer [(pp. 37–75). Rowland, T. (2001). Generic proofs in number theory. In S. Campbell, & R. Zazkis (Eds.), Learning and teaching number theory: Research in cognition and instructions (pp. 157–184). Westport, CT: Ablex. Selden, A., & Selden, J. (2003). Validations of proofs considered as texts: Can undergraduates tell whether an argument proves a theorem? Journal for Research in Mathematics Education, 4–36. Smith, D., Eggen, M., & Andre, R. S. (2014). A transition to advanced mathematics. Nelson Education. Stylianides, G., Stylianides, A., & Weber, K. (2017). Research on the teaching and learning of proof: Taking stock and moving forward. In J. Cai (Ed.), Compendium for research in mathematics education (pp. 237–266). Reston, VA: National Council of Teachers of Mathematics. Van de Watering, G., Gijbels, D., Dochy, F., & Van der Rijt, J. (2008). Students’ assessment preferences, perceptions of assessment and their relationships to study results. Higher Education, 56(6), 645–658. Weber, K. (2001). Student difficulty in constructing proofs: The need for strategic knowledge. Educational Studies in Mathematics, 48(1), 101–119. Weber, K. (2004). Traditional instruction in advanced mathematics courses: A case study ofone professor’s lectures and proofs in an introductory real analysis course. Journal of Mathematical Behavior, 23(2), 115–133. Weber, K. (2008). How mathematicians determine if an argument is a valid proof. Journal for Research in Mathematics Education, 39, 431–459. Weber, K. (2010). Mathematics majors’ perceptions of conviction, validity, and proof. Mathematical Thinking and Learning, 12(4), 306–336. Weber, K. (2012). Mathematicians’ perspectives on their pedagogical practice with respect to proof. International Journal of Mathematics Education in Science and Technology, 43, 463–482. Weber, K., & Alcock, L. (2004). Semantic and syntactic proof productions. Educational Studies in Mathematics, 56(3), 209–234.
Please cite this article in press as: Miller, D., et al. How mathematicians assign points to student proofs. Journal of Mathematical Behavior (2017), http://dx.doi.org/10.1016/j.jmathb.2017.03.002