Standard-based science education and critical thinking

Accepted Manuscript Title: Standard-based science education and critical thinking Author: Sufian A. Forawi PII: DOI: Reference:

S1871-1871(16)30008-6 http://dx.doi.org/doi:10.1016/j.tsc.2016.02.005 TSC 344

To appear in:

Thinking Skills and Creativity

Received date: Revised date: Accepted date:

5-8-2015 21-1-2016 16-2-2016

Please cite this article as: & Forawi, Sufian A., Standard-based science education and critical thinking.Thinking Skills and Creativity http://dx.doi.org/10.1016/j.tsc.2016.02.005 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Standard-based science education and critical thinking Sufian A. Forawi College of Education, the British University in Dubai, 345015, United Arab Emirates. +971 50 1270746; fax: +971 4 3664698. Email: [email protected] Highlights    

Pre-service teachers are aware of some thinking attributes of science standards The scientific inquiry standard had the highest critical thinking skills Least standard to include thinking is the structure and function of living systems Main attribute for thinking critically is to establish science relationships

ABSTRACT This study investigated pre-service teachers’ perceptions and utilization of critical thinking in standardbased science education. A convenience sample of 120 pre-service teachers participated in the study by examining the United States’ National K-12 Science Education Standards, using the Critical Thinking Attribute Survey (CTAS) originally developed and validated by the authors to measure critical thinking attributes. The main results of the study identified the science standards that exhibit critical thinking from the pre-service teachers’ perspectives. The process-oriented standards, i.e. the inquiry, nature of science, technology, personal and societal perspectives had higher means than the content standards, of life, physical and Earth sciences. Several specific standard objectives are presented from top and bottom attribution to critical thinking. For example, standard benchmarks that are rated the highest included: think critically and logically to establish relationships between evidence and explanations; design and conduct scientific experiments; and acquire the abilities necessary to do inquiry investigations. Examples of the least standard benchmarks included: structure and function in living systems, transfer of energy, and properties and changes of matter. Discussion is provided to connect results with the current literature review and models of critical thinking, along with recommendations and implications to teacher education and K-12 science education practice and research.

Key Words:

Critical thinking Pre-service teachers Standard-based education Science education

1. Introduction

In a world that is growing ever more complex and changing at an ever-increasing rate, students should be equipped with life skills that include critical thinking (CT). The importance of developing and acquiring CT for the populace, for economic, social, political and daily life uses, is apparent (Pattanapichet & Wichadee, 2015). In addition, terrorism and ill thinking, in many parts of the world, need to be combated by educating generations to value life by healthy reasoning and higher-order thinking skills. While the 1980s and ’90s brought much rhetoric about the need for improving student achievement and accountability, events at the beginning of the 21st century have helped us to realize that knowledge alone will not be sufficient to improve the quality of life in a global society. The development of CT has been one of the most essential objectives of education for many years, in areas such as economics (Heijltjes, van Gog, Leppink & Paas, 2014), literacy (Boyd, 2012), geography (Korkmaz & Karakus, 2009), mathematics (O’Keeffe & O’Donoghue, 2015), and higher education (Choy & Cheah, 2009). As reviewed in this study, science education research has widely studied CT and reasoning (Authors, 2000; Cameron & Richmond, 2002; Dolan & Grady, 2010; McCollister & Sayler, 2010; Yuan, Liao, &Wang, 2014). Yet, some researchers (Author, 2012; Osborne, Erduran, & Simon, 2004; Scott, 2008) have voiced concerns about students’ inability to think critically. Attention to CT goes back to the early days of education. Vieira, Tenreiro-Vieira, and Martin (2011) point to Plato and Aristotle as the founders of the critical thinking. Al-Mubaid (2014) views critical thinking as a mental process that involves a high quality and high level of thinking for problem-solving and decision-making. Terms such as higher level thinking and reflective thinking have often been used interchangeably with the term critical thinking throughout the literature (Crenshaw, Hale, & Harper, 2011; Geertsen, 2003; Ness, 2015; Wallace, Berry, & Cave, 2009). However, while there are many definitions of the term critical thinking, for the purpose of this study, Paul’s and Elder’s(2007) critical thinking definition is used – ‘the art of thinking about thinking in an intellectually disciplined manner’. According to Paul, this type of thinking involves three essential components: (1) analyzing, (2) assessing, and (3) improving. As one embarks on the process of analyzing and assessing, thinking is taken to more critical levels or thinking is made better. Developing science education standards is a major task that requires time, effort, and money. The United States’ National Science Education Standards (NSES), investigated in this study, and the newly developed Next Generation Science Standards (NGSS), all focus on improving science education regarding what students should know, achieve and be able to do (NGSS, 2013; NRC, 2012, 2000). This is one of several examples worldwide for the standard-

based education that attempts to increase K-12 students’ understanding of scientific content and practices. The NGSS (2013) framework hence emphasizes that: “…learning about science and engineering involves integration of the knowledge of scientific explanations (i.e., content knowledge) and the practices needed to engage in scientific inquiry and engineering design. Thus the framework seeks to illustrate how knowledge and practice must be intertwined in designing learning experiences in K–12 science education.” The standards are intended to present both knowledge and engagement skills which subsume cognitive and physical skills. The NGSS mainly focus on science in an integrated mode, including mathematics, technology and engineering, and they highly value science performance. While this is a new direction, yet, it limits focusing only on the science subject. Therefore, the rationale of using the NSES in this study is mainly because they are based solely on science, and the focus of this study aims to investigate pre-service teacher’s perceptions of the linkage between science and CT. Quitadamo, Brahler, and Crouch (2009) found that students on one of the very effective Science, Technology, Engineering, and Mathematics (STEM) undergraduate programs who were involved in peer-lead small-group dynamics showed small but significantly greater critical thinking gains. This finding encourages more research to be conducted on this topic, particularly to further investigate critical thinking skills related to science and mathematics curricula. In one of our earlier studies (Authors, 2000), we found that process skills and integrated inquiry instruction were common science buzzwords linked to new science curricula and the development of critical thinking. Taylor, Jones, Broadwell, and Oppewal (2008) describe how scientists view working with science teachers in a manner that recognizes and develops critical thinking. Therefore, the main purpose of this study is to investigate pre-service teachers’ perceptions and utilization of critical thinking of the US national science education standards. The question is whether pre-service teachers recognize that thinking is an inherent part of the science standards and curriculum. Many of our present education majors have come through systems where the curriculum was more fact-driven, that is, taught using traditional teacherdirected methods. Indeed much of what they continue to get in higher education often focuses on learning the content in lecture-driven classrooms where there is little time for students to question and process the information. Ultimately, how pre-service teachers interpret the standards will vary because each brings a different lens through which they examine them, hence providing a rationale for this study to investigate pre-service teacher’s perceptions and utilization of critical thinking of the science education standards. It is important that pre-service teachers have

knowledge of CT and be able to practice assessing it, especially in the standard-based education systems.

2. Theoretical background 2.1. Critical thinking history and definitions Traditionally, critical thinking definition involves evaluating thinking through classification. Bissell and Lemons (2006) consider Bloom's taxonomy the best way to categorize critical thinking in the classroom. This classification can be used to evaluate critical thinking using the six levels of cognitive thinking. Students can progress through the levels of the taxonomy from lowest to highest. Although critical thinking exists at every level, Paul (2002) found that the higher-order thinking skills are often experienced at the synthesis, evaluation, and design stages. There are two theoretical perspectives that often describe the philosophical orientation of critical thinking: the philosophical tradition of CT, and the logical aspects of thinking. In this study, the authors adopt the philosophical tradition that focuses on the identification of thinking abilities, which is appropriate to the Paulian model and the perceptual scope of the present study. Such CT abilities are the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action. In this common form of CT, specific universal intellectual values are identified that transcend subject matter divisions: clarity, accuracy, precision, consistency, relevance, sound evidence, reasons, depth, breadth, and fairness (Paul & Elder, 2007). Critical thinking also involves evaluating reasoning and the factors considered in making decisions. Despite the lack of consensus, the definitions of two philosophers, Robert Ennis and Richard Paul, have served as the theoretical reference and the historical context for the present study. In 1962, Ennis defined critical thinking as logical thinking characterized by complex cognitive skills. Then, at the end of the 1980s, he adjusted his definition to include the influence of creative thinking and predispositions. For him, critical thinking is a ‘reflected thinking focused on what is to be believed or accomplished’ (Ennis, 1993, p. 180). The Paulian approach to critical thinking is an internationally acclaimed and widely referenced model of critical thinking. It considers a practical, accessible, and well-researched mode. In particular, Paul (1996, in Paul and Elder, 2007) defines critical thinking as an art of analyzing and evaluating thinking with a view to improving it through the standards, elements and habits of the mind. In short, CT is a self-

directed, self-disciplined, self-monitored, and self-corrective activity. Both definitions consider CT as a cognitive activity reflected in beliefs and actions, yet for Paul thinking can be improved by education standards and by self-refinement. Such understanding makes CT an activity that can be taught and learned, which seems appropriate for this study. Historically, the 1980s witnessed a growing interest in and a renewed concern for the promotion of CT. Boisvert (1999) in Vieira, Tenreiro-Vieira, and Martin (2011) considers this decade to be a milestone in three phases that represent the evolution of CT in education. In the first phase, before the 1980s, education focused on CT abilities which were chosen as objectives and were seen as an end in themselves. In the second phase, during the 1980s, the focus was on the processes of critical and creative thinking required for problem-solving, decision-making and research, considering that cooperative learning and graphical organizers were two pedagogical innovations which further characterized this period. The third phase, which started in the 1990s, is characterized by the attention and importance ascribed to the use of CT processes and abilities in diverse situations within the school and students’ personal lives. During this latter period, there is a certain insistence on the creative use and transfer of these thinking abilities as a means for metacognitive reflection, in which students are expected to become more aware of their own thinking processes and better informed about the thinking strategies of others. Particularly, the science curriculum plays a major role in providing opportunities for students to use and acquire higher-order thinking skills. However, neither students nor teachers seem to have made the appropriate development of critical and creative thinking skills to date (Authors, 2012).

2.2. Teaching of critical thinking

The teaching of critical thinking is important for all students in all subjects. Different disciplines are characterized by particular approaches to critical thinking, and a large part of studying those disciplines means learning to think like an expert of that discipline. All disciplines require you to ask questions, relate theory to practice, find and use appropriate evidence, evaluate, find links, and categorize. However, some instructional techniques are found to assist in the development of critical thinking, such as debates, investigations, and problem-solving. Also, Afamasaga-Fuata (2010) stated that traditional pedagogical techniques, such as lectures and examinations, center on knowledge acquisition, while debates in the technology classroom can effectively facilitate critical thinking. Science is often concerned with interpreting within a framework, describing, explaining, predicting, and identifying cause and effect. In particular, science seems prone to developing such

skills. Integrating critical thinking with other competencies, like guided inquiry, experimenting, seeking evidence with support, teamwork and communication, are a few of the science-related skills which can easily be enhanced through the development of critical thinking skills. Science learning can be improved if it is taken to higher levels by aiming at critical thinking through scientific experimentation, variable manipulation, data collection, deeper analysis, and integration of knowledge and processes. As we live in a world with an increased attention to the sciences, this explosion of scientific knowledge and its implications intrinsically affect everyone’s lives. In truth, never before has there been a pressing need to prepare students and populace to take a stand in science issues, detect fallacies in arguments, and provide rational and logical decisions on societal issues, which can all be achieved by possessing critical thinking skills (Lunnetta, Hofstein, & Clough, 2007).Scientific knowledge was traditionally taught as an isolated body that is often memorized by students. It also has to be learned for understanding, application, analysis, evaluation and creation. Therefore, CT can contribute toward a better understanding of science, to preparestudents to act in the context of problem-solving and decision-making with regard to the way science and technology are used to change society, and vice versa. Critical thinking is described to be an ability that students can acquire in their short-term memory. The individuals usually behave more or less not only by the ability but also by tendencies, and this aspect is called critical thinking disposition (Bereiter, 1995; Cacioppo & Petty, 1982; Ennis, 1986; Yuan, Liao, & Wang, 2014). Critical thinking disposition takes a long time to be acquired and it is essential for learners to have practice in thinking critically about an issue (Norris, 1985). The authors (2012) believe that critical thinking dispositions can be developed in students by applying intellectual standards to the elements of reasoning. Along with the ability to perform critical thinking skills, “a critical thinker must also have a strong intention to recognize the importance of good thinking and have the initiative to seek better judgment.” (Ku, 2009, p. 71) This willingness to self-regulate can be described in terms of executive function, key dispositions toward thinking, the motivation to think and learn, and the perceived need to use specific cognitive processes when solving problems. Executive function refers to metacognitive processes, used to self-regulate thought (Dwyer, Hogan, & Stewart, 2014, as indicated in Figure 1 below). Fig. 1. Interdependencies among CT skills, self-regulatory functions and reflective judgment (Dwyer, Hogan, & Stewart, 2014)

Critical thinking is a worthwhile activity for all students, particularly for gifted students. The optimal school learning environment for gifted students is one where scholastic rigor is the

standard. This rigor is needed both o stimulate the students intellectually and enhance their academic growth. Whether enrolled in preschool, elementary, middle, or high school, the integration of critical thinking skills into the daily content and lessons is essential for achieving this rigor. This infusion, along with taking into account the students’ interest, readiness, and learning styles, provides the foundation and walls for raising the ceiling of students’ scholastic growth and intellectual stimulation. However, there are many arguments that teachers do not know how to teach critical thinking skills or the way to integrate critical thinking skills in their teaching strategies. Brodbear (2003) contends that teachers have deeper information about content knowledge, but do not have the pedagogical content knowledge. This provides another rationale for this study to examine if pre-service teachers are able to identify and assess the critical thinking embedded in the science standards. Teachers who tether curriculum and standards to the language, contexts and thinking that students bring to class effectively enhance student thinking and learning and validate educational purpose (Boyd, 2012). In particular, Boyd (2012) attested that patterns of teacher talk shape student language, reasoning, and critical thinking, and that teacher planning and in-the-moment decision-making impact on what and how content is delivered. Good teaching builds on what students know, and good teachers seize teachable moments to negotiate and personalize content and student learning in a meaningful, relevant fashion. All these are connected to creating an environment where critical thinking can in fact be embedded in.

2.3. Measuring of critical thinking

While the critical thinking skill is easily recognized as an important learning objective, measuring it is relatively difficult. If the majority of the class time is spent discussing and solving ill-defined problems, for example, will students become actively and meaningfully involved in owning learning and will there be any gains in CT skills? In some countries, e.g. USA, UK, and Australia, efforts have been made to integrate critical thinking into science curricula, recognizing the importance of such curriculum for diverse society, citizenship competence, and science literacy. However, this objective has not been appropriately implemented in classrooms due to many challenges. One of the obstacles is the fact that teachers do not have a clear idea about critical thinking because the meaning ascribed to critical thinking in different contexts is rarely explicit (Genç, 2008; Vieira, Tenreiro-Vieira, & Martin, 2011). Another obstacle to measuring CT relates to its multifaceted nature. Ness (2015) equates innovation with creative and critical thinking and further considers the engine of scientific progress. He also provides examples of

tools of innovation such as enhancing observation, using analogies, changing points of view, juggling opposites, broadening perspective, reversal, reorganization and combination, and getting the most from groups. Several instruments are used to measure critical thinking, such as Watson and Glaser Critical Thinking Appraisal (WGCTA) (Watson, Glaser, &Rust, 2002), the California Critical Thinking Disposition (CCTDI) (Banning, 2006), and the Critical Thinking Disposition Survey (CTDS). They mainly measure general aspects of CT, such as inference, recognition of assumptions, deduction, interpretation, open-mindedness, analyticity, self-confidence, inquisitiveness, and maturity. The Critical Thinking Attribute Survey (CTAS), used in this study, incorporates some of these aspects, yet measures discipline-specific CT skills related to education and teacher development to appropriately be used with teacher education students. Measuring cognitive attributes such as being nonjudgmental, willingness to question, mindfulness, reexamination, and plasticity, are part of the CT attributes worth studying. Both in-service and preservice teachers, therefore, should be trained on recognizing and incorporating CT skills in their instruction. One of this study’s objectives is to provide pre-service teachers with the opportunity to understand CT skills and identify them as they relate to the science education standards which are part of their practicum and school experience. This task is major, along with some other obstacles in the sciences, such as a lack of adequate and sustainable professional development, emphasis of rote memory instruction, reliance on standard-based curricula with less emphasis on higher-order thinking objectives, and inadequate science teacher education programs.

3. Purpose and questions of the study The main purpose of this study was to investigate pre-service teachers’ perceptions of the linkage of critical thinking attributes and the National Science Education Standards. In particular, the study reported those of the USA National Science Education standards that relate to critical thinking. Therefore, the study aimed to answer the following questions:

1. How do science teachers perceive the way science education standards are linked to CT? 2. What CT attributes are associated with science education standards’ objectives and benchmarks? 4. Methods 4.1. Participants

One hundred and twenty pre-service teachers who were in their senior year in a Midwestern university in the United States participated as a convenient sample to ensure orientation about CT and instruments used in the study. The students were part of a National Council for Accreditation of Teacher Education (NCATE) program. The pre-service teachers participated during their science methods courses in three different programs: elementary, middle and secondary in one academic year, fall, spring, and summer semesters, to ensure coverage of broader Science Education Standards at K-12 level. They were asked for their consent to take part in the study and were oriented about the critical thinking instrument. Seventy-four were female (62%) and 46 were male (38%) which is representative of teacher education students in this region.

4.2. Science Education Standards and activities

This study aimed to identify theK-12 science content standards and their objectives that require critical thinking from pre-service teachers’ perspective. The science content standards, referred to here, are part of the National Science Education Standards (NSES) developed by the United States’ National Research Council (2000) for the kindergarten to grade 12’s science curriculum. The newly developed US science standards, the Next Generation Science Standards (NGSS, 2013) are not incorporated in this study as the study didn’t investigate integration of mathematics, technology, and engineering, the main focus of the NGSS; rather it only investigated the science standards, as indicated in the NSES, and their connection to CT. The standards outline what students should know, understand, and be able to do in natural science at K-12 levels. The content standards - Science as Inquiry, Physical Science, Life Science, Earth and Space Science, Science and Technology, Science in Personal and Social Perspectives, History and Nature of Science - are a complete set of outcomes for students to attain. According to the NSES (NRC, 2000), the scientific inquiry as a learning process engages the students mentally in a constructivist environment and the scientific inquiry in its multiple stages such as writing, communicating and reflecting within a range of specific activities involves the students in peers work and cooperative learning. The science standards are developed to impact changes in the educational programs, instruction, and assessments (Bybee, 2014). The present study focused only on the possible changes in the science teacher education programs based on pre-service teachers’ perceptions, awareness and knowledge of CT.

The standard for unifying concepts and processes is presented for grade K-12, because this standard is associated with major conceptual and procedural schemes and transcends disciplinary boundaries, and therefore needs to be developed over the entire educational curricula. The standard identifies the conceptual and procedural schemes unifying science disciplines and provides students with powerful ideas to help them understand the natural world. The next seven categories are clustered for grades K-4, 5-8, and 9-12. These clusters were selected based on a combination of factors including cognitive development theory, the classroom experience of teachers, organization of schools, and the frameworks of other disciplinary-based standards. The sequence of the seven grade-level content standards is not arbitrary; each standard subsumes the knowledge and skills of the other standards. Students' understandings and abilities are grounded in the experience of inquiry, and inquiry is the foundation for the development of the understandings and abilities of the other content standards. The personal and social aspects of science are emphasized increasingly in the progression from ‘science as inquiry’ standards to the ‘history and nature of science’ standards. Students need solid knowledge and understanding in physical, life, and Earth and space science if they are to apply science. Multidisciplinary perspectives also increase from the subject-matter standards to the standard on the history and nature of science, providing many opportunities for integrated approaches to science teaching.

4.3. Critical Thinking Attribute Survey (CTAS)

A set of critical thinking attributes was used, that had been developed and validated by Wright & Forawi (2000) and from the relevant literature. The major premise of this set, called here the Critical Thinking Attribute Survey (CTAS), rests on the understanding of critical thinking as a process of evaluating ideas, investigating topics, solving problems, and making decisions. More specifically, the CTAS measures the accuracy of statements and the soundness of reasoning that lead to conclusions and the interpretation of claims and results. The CTAS encompasses ten major attributes: think independently and develop courage; explore how egocentricity and sociocentricity affect feeling, thought and behavior; suspend judgment or prior conceptions; utilize various processes to resolve, re-address and re-analyze complex situations to gain new insight; develop and use valid criteria for evaluation; raise and pursue significant questions; analyze arguments, interpretations, beliefs or theories, and their implications; generate and assess solutions; make interdisciplinary connections to everyday life; and think precisely about thinking and use critical thinking vocabulary. Participating pre-service teachers were oriented on how to use the instrument by checking each of its items with all the objectives of the

standards from kindergarten to grade 12. Cronbach's (1971) alpha internal consistency coefficient for the CTAS was 0.73, set at p ˂ .05, which is considered a highly moderate one. According Johnson and Christensen (2014) the minimum acceptable level of reliability is .70 for nonclinical tests. This reliability indicates that the CTAS scores reliably discriminate among participants’ responses in this study. This reliability coefficient was similar to reliabilities in previous research with a similar sample size of pre-service teachers and college students. Also, the content validity of the CTAS items was judged by a panel of experts during its development (Authors, 2012; 2000).

4.4. Design and procedures

The study followed a quantitative method approach as explained by Creswell (2003), collecting data gathered from participants’ perceptions of critical thinking of the science national education standards by use of the CTAS instrument. The quantitative method approach is valuable, as in this study, in identifying and further explaining participants’ perceptions of CT and the standards. First, the participating pre-service teachers were given instruction and activities on critical thinking and the Science Education Standards. Second, they were instructed on the CTAS and how it measures critical thinking. Third, pre-service teachers, individually, identified the critical thinking attributes of the objectives of the National Science Education Standards. Finally, the quantitative data were gathered by adding the total numbers of participants’ checks of the critical thinking attributes with their appropriate National Science Education Standards.

5. Data analysis and findings Pre-service teachers’ responses were coded and analyzed to identify those science standards that exhibit critical thinking attributes. The tables below present the statistical results of analysis of participants’ CTAS responses to the National Science Education Standards, which were followed by discussion. To answer the study’s first question, the main result identified the National Science Education Standards that are perceived by pre-service teachers to have critical thinking attributes. The total checks made by participants for each standard was collated and recorded. The science standards that marked as high, (more than 50% of total responses,) on critical thinking attributes as presented in the CTAS included four science standards: Inquiry; History and the Nature of Science; Science in Personal and Social Perspectives; and Science and Technology. These four science content standards were prescribed with their activities to mainly

provide science processes and skills via delivering content. The objectives of these process standards tend to be more investigative and open-ended. The table below shows summary results of a number of checks and benchmarks of the science standards that are connected to critical thinking as perceived by pre-service teachers. In particular, checks of how often standard objectives are checked, the standard mean and standard deviation of checks, and number of benchmarks of each standard are displayed.

Then, the CFA was performed for all the seven standards as shown in Table 3. A couple of items were extracted as their loading was less than 0.3 as shown in the table. These results are in line with the results of the science education standards shown in table 1.

Participating pre-service teachers indicated the objectives of the science standards that exhibit critical thinking skills. They perceived the objectives of the ‘inquiry standard’ to be most closely related to critical thinking as identified by the CTAS, with the highest mean (M=13.820, SD .544). This was followed by the other process standards which are perceived to be related to critical thinking with high means - the ‘history and nature of science’ (M=11.194, SD=0.644), the ‘personal and social perspective’ (M=10.166, SD=0.742), and the ‘science & technology (M=8.944, SD=0.940). The traditional content standards had the lowest means - Earth (M=4.458, 1.204), life (M=3.388, SD=1.282), and physical sciences (M=3.125, SD=1.462) – exhibited the fewest objectives with critical thinking attributes according to the CTAS instrument. Q To further identify how each of the standard objectives or benchmarks were viewed by the participating pre-service teachers on their attribute to critical thinking, all the total 130 benchmarks were ranked in descending order. Table 4 shows the ten most-checked benchmarks with their number of CTAS items (1 to 10), minimum and maximum CTAS items checked, and mean of how often each is checked using the CTAS. These top benchmarks, their mean ranging between 5.50, and 4.57, are seen related to critical thinking as they directly indicate descriptors such as thinking logically and critically, using of inquiry investigations, making sound decisions, etc. The bottom five least-checked benchmarks, Table 5, show content-related objectives at low order thinking, indicated by low means ranging between 0 and 1.80, such as knowing the structure of living systems, properties of matter, Earth systems, and the transfer of energy. To further answer the first question, Table 6 presents the means, standard deviations and number of checks of each of the critical thinking attributes as presented in the CTAS. As shown, almost all items were checked more than 50% of times (0.5), with the highest three items were

CTA7 – Analyze arguments, interpretations, beliefs, or theories and their implications, CTA 4 – Utilize various processes to resolve, re-address, and re-analyze complex situations to gain new insight, and CTA 8 – Generate and assess solutions. (4.26, 4.21, & 4.0 means, respectively). These critical thinking attributes were considered the most relevant to being presented and developed from pre-service teachers’ view of the science standards. 6. 7. Discussions In discussing the findings of this study, participants identified the science standard benchmarks that relate and do not relate to developing students’ critical thinking skills. In particular, the crux of the results indicated several process-oriented and interdisciplinary science standards, such as inquiry, nature of science, technology, and social perspectives, which were found to be highly attributive to developing critical thinking skills. However, the content standards, life, physical and Earth science, had the fewest attributes toward developing these higher-order skills. The results showed that students’ critical thinking skills are perceived to likely be present in the process, societal-oriented standards where learning is open-ended and guided (Authors, 2012). This enables students to increase their metacognitive awareness so that they think and talk about their own thinking process (Halpern, 2007). This direction is also supported in the new NGSS (2013) document in realizing that standards alone as a practical framework do not work, therefore it is mandatory to implement effective new teaching strategies to achieve the desired outcomes. The science inquiry teaching approach, as found in the study to exhibit CT, has been one of the strategies recommended for the effective implementation of the new NGSS (2013) framework as a vehicle for cutting across disciplines, not only in the science classroom, but also at the school, home and community levels to improve students’ scientific literacy and learning. Regarding the findings of the most-checked CTAS instruments, three main broad high order skills were found to be the highest: the ability to analyze arguments, theories, and implications to gain new insights; utility of various processes; and generation of solutions. Previous research indicated that the broad skills outlined in the science standards can be enacted in many ways and can therefore accommodate what students may bring to the classroom allowing for more dialogic instruction (Boyd, 2012; Quinn, Burbach, Matkin, & Flores, 2009) and that may be extended to critical thinking levels (Authors, 2012). Teachers, including pre-service teachers, can enable the development of critical thinking, only if they can recognize it through the

planning of the standards and activities and enact it in instruction. This is similar to the newly developed NGSS for science and engineering, eight practices that include: 1) asking questions and defining the main problems or the scientific hypothesis, 2) developing and using models, 3) carrying out investigations after planning them, 4) interpreting and analyzing data, with a cross – disciplinary approach, 5) computer technology to construct explanations and design solutions, 6) engaging students in argumentation based on evidence, 7) obtaining the results, and 8) evaluating and communicating information. Such practices are directly related to the NSES process and nature standards that are found to closely develop students’ critical thinking. Abrami, Bernard, Borokhovski, Wade, Surkes, Tamim, and Zhang (2008) further emphasize that critical thinking enables students to become independent lifelong learners who most likely can develop and progress to become better citizens. Recent debates argue that even the supremacy of nations is becoming more dependent on their critical thinking potentials rather than economic matters (Vieira, Tenreiro-Vieira, & Martins, 2011). Furthermore, Scearce (2007) suggests that critical thinking with solid scientific literacy could be crucial to sustain a country’s advances in economy and technology. Similarly to the present study’s results on science standards, Heijltjes, van Gog, Leppink, and Paas (2014) found, in a business experimental study, that some cases were practiced while others were not, which influenced market open-mindedness. Therefore, promoting critical thinking in education and more particularly in the science classroom could contribute significantly to individuals’ and nations’ development. For discussion of the qualitative themes of the standard activity analysis, four main features are derived to include peer interaction, independency and guidance, decision-making, and science reasoning. The findings of this study clearly indicated that students’ critical thinking performance and understanding can be differentiated according to the active learning experiences that students are engaged in. These features are relevant to the development of critical thinking through science active learning. Similar features were also identified by Geier, et al, (2008) who consider them to be key strategies for teaching critical thinking. Pre-service teachers’ analysis of the science standards’ activities revealed a consensus on the importance of planning activities that permit students to work independently with some teacher guidance. In their view, the active participation of students coupled with appropriate guidance can be an effective source of the development of critical thinking skills by learners. Peer interaction in science activities is recognized as an essential feature that assists in developing critical thinking. Guided inquiry instruction is seen to encourage such group

interaction of ideas. Thinking is a spontaneous mental process that needs to be structured to avoid misleading conclusions, and must have a bias because most people tend to lack educated rational reasoning (Scriven & Paul, 2004). Therefore, it is necessary to find methods which could help in developing critical thinking skills among students. One of the effective methods presented by Snyder and Snyder (2008) is active learning. This pedagogical approach allows students to be engaged in inquiry-based learning experiences. Inquiry-based teaching practices have been repeatedly cited in relevant literature as an effective way to foster critical thinking in the classroom (Quitadamo, Brahler, & Crouch, 2009). The activity examples given were the pendulum motion, chemical bonding, the cell and building habitats in which students should work in small groups to think of ideas, identify problems, test hypotheses, find solutions and make conclusions. The unguided teaching strategy, which is referred to as discovery learning or full inquiry, indicated that exploration without guidance from the teacher does not work (Authors, 2011). Therefore, the guidance of scientific activities and teaching with the features found in this study are proven to be beneficial and complement student interactions. These results support research studies by Mayer (2004), and Geissler, Edison, and Wayland (2012) that conclude that teaching with guidance is more effective than discovery learning. The other two features recognized by participants were decision-making and science reasoning which are directly related to critical thinking elements. They include the construction of valid arguments and contra-arguments based on accurate evidence, clear purpose, other views and logical application, promoting interaction among students as they learn, asking open-ended questions that do not have a ‘right’ answer, and transferring experiences to other fields and everyday life. In support of these features, Paul and Elder (2004) argue that critical thinking is a structured cognitive process that requires active and skillful engagement in problem-solving, formulating concepts, application, reasoning, analysis, synthesis, observation, data collection and evaluation, reflection, and communication. Most of the critical thinking definitions reviewed in this study share a common theme that reflect the higher-order cognitive processes associated with critical thinking. Therefore, critical thinking is a profound mature mental process that relies heavily on rational reasoning, and thus it should not be seen merely as an outcome.

8. Conclusions

Developing national standards is an important and complex undertaking. Yet, once these standards are developed, they do not immediately impact on policy and practice (Cameron & Richmond, 2002). National curricula are also significant and of such great magnitude that they should be carefully prescribed. Actions by many individuals and institutions are needed if meaningful and lasting changes are to occur in a system. A review of the literature reveals that although the teaching of critical thinking skills is a significant aim of K-12 pedagogy, much ambiguity exists regarding the topic. In fact, due to the lack of teacher and instructor familiarity with the concept, compounded by student resistance to put forth the intellectual labor to take charge of their own thinking, students are mainly exposed to didactic instruction that does not prepare them with real-world problem-solving skills (Crenshaw, Hale, & Harper, 2011). This study addressed these problems in the following way. The study aimed to identify those science content objectives that require critical thinking from the pre-service teachers’ perspective. Participants of the present study developed a discourse and familiarized themselves with an instrument that measures the critical thinking aspects of national science standards and their curricular activities. The findings of this study may advance the research into pre-service teachers’ perceptions and utility of critical thinking. The process standards, such as inquiry, history and nature of science, technology and personal and social perspective and their activities, are perceived to have more attributes of critical thinking. Pre-service and in-service science teachers often focus on the teaching of content knowledge as the traditional science curriculum governs. Duckworth (2006) has criticized traditional content-driven science curricula that focus on maintaining an orderly classroom and reduce student confusion by delivering a curriculum that consists of a carefully prescribed sequence of activities, rather than promoting the growth and development of individual students. Therefore, if students were to think critically, classroom experiences should include more process instruction, such as inquiry, the nature of science, and personal and societal connections. While it is not the purpose of this study to identify why certain science standards indicate more opportunity for developing critical thinking, it is reasonable to suggest that process standards, as they allow for interactive learning experiences, can provide opportunities for using critical and higher thinking skills. In particular, Dalon and Grady’s (2010) study supports this result, that students increased their critical thinking attributes through actual science inquiry activities at undergraduate level. One of the limitations of this study is that the ability of these pre-service teachers to think critically was not assessed. It would be naïve to assume that all pre-service teachers are necessarily critical thinkers. Yet, they can be made aware of how to recognize CT and be trained to include it in their instruction, the results of which would be a suitable subject for a future

extended study. Also, critical thinking skills are difficult to transfer to students especially if teachers are not aware of the nature of critical thinking and the way it should be incorporated into their classroom practices (Lauer 2005). Furthermore, McCollister and Sayler (2010) assert that teachers will most likely avoid schooling matters that they are not able to elucidate although many seem to prefer and enjoy communicating with students using higher levels of thinking (Stedman & Adams 2012). In addition, it is essential that teacher preparation programs take on the challenge of ensuring that student teachers are well prepared, not only in utilizing critical thinking skills (Paul, 1992) but also in adequately transferring them to their students. Finally, a limitation may be seen in the difficulty in interpreting the CTAS by pre-service teachers and the rarity of planning CT instruction with prospective teachers. The implications of this study are deemed important in identifying whether science education standards and prescribed activities and texts foster critical thinking, especially when pre-service teachers seemed to rely on what is dictated in the standards. Pre-service science teachers are encouraged to identify critical thinking attributes when planning for science instruction and providing learning opportunities. Pre-service teachers will impact future science teaching based on the way science standards and curricula are prepared. This idea also is supported by a crucial need for inclusion of critical thinking attributes into science instruction for better learning experiences in schools and beyond. Finally, the inclusion of critical thinking attributes is recognized as an integral part of science instruction, and further is recommended to be transferred to out-of-school situations. Therefore, future research studies should aim towards examining similar standards, textbooks and curricular activities, with different samples and in other countries. Also, a study into how pre-service and in-service science teachers incorporate reasoning and critical thinking skills into teaching and advancing students’ learning needs to take place. Teachers’ perceptions of critical thinking should not be confined to its epistemology, and higher-order cognitive levels, but their understanding should include all of its components (Lunnetta, Hofstein, & Clough, 2007). Therefore, there should be more emphasis on the associated thinking skills and habits of mind required to be demonstrated and practiced in the classroom rather than the shallow definition of critical thinking (Choy & Cheah, 2009).Moreover, further research is recommended to investigate the effect of inquiry-based laboratory instruction on the development of critical thinking skills. This study demonstrates that students learn the most when they experience learning activities themselves, and where they apply critical thinking skills with some assistance. Finally, this study showed the connections of the science process and

content standards, the intellectual standards, and elements of reasoning that can be used to promote the students’ ability to think critically.

References Abrami, P., Bernard, R., Borokhovski, E., Wade, A., Surkes, M., Tamim, R. and Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: Critical Thinking and Science Education. Science & Education, 11(4), 361-375. Afamasaga-Fuata’i, K. (2008). Students’ conceptual understanding and critical thinking: A case for concept maps and vee diagrams in mathematics problem-solving. Al-Mubaid, H. (2014). A new method for promoting critical thinking on online education. International Journal of Advanced Corporate Learning, 7, 4, 34-37. Authors (2012). Perceptions on critical thinking attributes. Authors (2011). Inquiry instruction and the nature of science. Authors (2000). Social challenges require critical and creative thinkers. Banning, M. (2006). Nursing research: Perspectives on critical thinking. British Journal of Nursing, 15, 458-461. Bensley, A. D., & Spero, R. A. (2014). Improving critical thinking skills and metacognitive monitoring through direct infusion. Thinking Skills and Creativity, 12, 55-68. Bereiter, C. (1995). A dispositional view of transfer. In A. McKeough, J. Lupart, & A. Marini (Eds.), Teaching for mastery: Fostering generalization in learning (pp. 21-34). Mahwah, NJ: Lawrence Erlbaum. Bissell, A. N. and Lemons, P. P. (2006). A new method for assessing critical thinking in the classroom. BioScience, 56(1). 66-72. Boyd, M. (2012). Morning meeting in a third grade classroom: Literacy and learning. Journal of Classroom Interaction, 47(2), 4-12. Broadbear, J. T. (2003). Essential elements of lessons designed to promote critical thinking. The Journal of Scholarship of Teaching and Learning, 3(3), pp. 1-8. Bybee, R. W. (2014). NGSS and the next generation of science teachers. Journal of Science Teacher Education, vol. 25(2), pp. 211-221. Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116-131. Cameron, P. and Richmond, G. (2002). Critical thinking across the curriculum. Community College Enterprise, 8(2), 59-70.

Champagne, A. B. and Klopfer, L. E. (1977). Problem solving as outcome and method in science teaching. School Science and Mathematics, 80, 3-8. Choy, S.E. and Cheah. P. K. (2009). Teacher perceptions of critical thinking among students and its influence on higher education. International Journal of Teaching and Learning in Higher Education, 20(2), 198-206. Crenshaw, P., Hale, E., and Harper, S. (2011). Producing intellectual labor in the classroom: The utilization of a critical thinking model to help students take command of their thinking. Journal of College Teaching & Learning, 8(7), 13-22. Creswell, J. W. (2003). Research design—Qualitative, quantitative and mixed methods approaches.2nded. London: Sage Publications. Cronbach, L. J. (1971). “Test Validation” In Educational Measurement. 2nd ed. Ed. R. L. Thorndike. Washington, DC: American Council on Education, 443–507. Dolan, E. and Grady, J. (2010). Recognizing students’ scientific reasoning: A tool for categorizing complexity of reasoning during teaching by inquiry. Journal of Science Teacher Education, 21, 31-55. Dewey, J. (1916). Democracy and education: An introduction to education. USA: McMillan. Duckworth, E. (2006). Critical exploration in the classroom. In E. Duckworth (Ed.), The having of wonderful ideas and other essays on teaching and learning (3rd ed., pp.157–172). New York, NY: Teachers College Press. Dwyer, C. P., Hogan, M, J.,& Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12, 43-52. Ennis, R. (1996) Critical thinking. Upper Saddle River, NJ: Prentice Hall. Geertsen, H. R. (2003). Rethinking thinking about higher-level thinking. Teaching Sociology, 31, 1-19. Geier, R., Blumenfeld, P.C., Marx, R.W., Krajcik, J.S., Fishman, B., Soloway, E., and ClayChambers, J. (2008). Standardized test outcomes for students engaged in inquiry-based science curricula in the context of urban reform. Journal of Research in Science Teaching, 45(8), 922-939. Geissler, G. L., Edison, S. W., and Wayland, J. P. (2012). Improving students' critical thinking, creativity, and communication skills. Journal of Instructional Pedagogies, 8, 1-11. Retrieved from http://search.proquest.com/docview/1020694208?accountid=35812 Genç, S. Z. (2008). Critical thinking tendencies among teacher candidates. Educational Sciences: Theory & Practice, 8(1), 107-116.

Gough, N. (2002). Thinking/acting locally/globally: Western science and environmental education. International Journal of Science Education, 24(11), 1217-37. Halpern, D. F. (2007). The nature and nurture of critical thinking. In R. J. Sternberg, H. L. Roediger III, and D. F. Halpern (Eds.), Critical Thinking in Psychology. Cambridge, NY: Cambridge University Press. Heijltjes, A., van Gog, T., Leppink, C., Paas, F. (2014). Improving critical thinking: Effects of dispositions and instructions on economics students’ reasoning skills. Learning & Instruction, 29, 31-42. Johnson, R. & Christensen, L. (2014). Educational Research: Quantitative, Qualitative, and Mixed Approaches.5thedition. Sage: London, UK. Jungwirth, E. and Dreyfus, A. (1990). Diagnosing the attainment of basic enquiry skills: The 100-year old quest for critical thinking. Journal of Biological Education, 24(1), 42-49. Lauer, T. (2005). Teaching critical-thinking skills using course content material. Journal of College Science Teaching, 34(6), 34-44. Lunnetta, V.N., Hofstein, A., and Clough, M. P. (2007). Learning and teaching in the school science laboratory: an analysis of research, theory and practice, In N. Lederman, & S. Abel (Eds.), Handbook of Research on Science Education, 393-441, Mahwah, NJ: Lawrence Erlbaum. Korkmaz, O. & Karakus, U. (2009). The impact of blended learning model on student attitudes towards geography course and their critical thinking dispositions and levels. Turkish Online Journal of Educational Technology, 8(4), 51-63. Mayer, R. E. (2004). Should there be a three-strike rule against pure discovery learning? The case for guided methods of instruction. American Psychologist, 59(1), 14-19. McCollister, K. & Sayler, M. (2010). Lift the ceiling: Increase rigor with critical thinking skills. Gifted Child Today, 33(1). 41- 47. National Research Council (2012). A framework for K-12 science education: practices, crosscutting concepts, and core ideas. Washington, DC: The National Academic Press. National Research Council (2000). National Science Education Standards. Washington, D. C.: National Academic Press. Ness, R. (2015). Promoting innovative thinking. American Journal of Public Health, 105, 114118. Next Generation of Science Standards(2013), available online: http://www.cesa7.org/science/documents/DesigningProfessionalDevelopmentfortheNGS S. 2012-13\Science Advisory Council\Feb. Retrieved May 2013.

Norris, S. P. (1985). Synthesis of research on critical thinking. Educational Leadership, 42, pp. 40-45. O’Keeffe, L. & O’Donoghue, J. (2015). A role for language analysis in mathematics textbook analysis. International Journal of Science and Mathematics Education, 13(3), 605-630. Paul, R., & Elder, L. (2008). The miniature guide to critical thinking concepts and tools (5th ed.). Dillon Beach, CA: The Foundation for Critical Thinking. Paul, R., and Elder, L. (2007). Critical thinking competency standards. Dillon Beach, CA: Foundation for Critical Thinking. Paul, R. (1992). Critical thinking: What every person needs to survive in a rapidly changing world. Center for Critical Thinking: Santa Rosa, CA. Pattanapichet, F. and Wichadee, S. (2015). Using space in social media to promote undergraduate students’ critical thinking skills. Turkish Online Journal of Distance Education, 16(4). Quinn, C., Burbach, M., Matkin, G., and Flores, K. (2009). Critical thinking for natural, agricultural, and environmental ethics education. Journal of Natural Resources and Life Sciences Education, 38, 221-227. Quitadamo, I.J., Brahler, C.J., and Crouch, G.J. (2009). Peer-led team learning: A prospective method for increasing critical thinking in undergraduate science course. Science Educator, 18(1), 29-39. Rannikmäe, M., Teppo, M., and Holbrook, J. (2010). Popularity and relevance of science education literacy: using a context-based approach. Science Education International, 21, 116-125. Scearce, C. (2007). Scientific literacy. ProQuest Discovery Guides. Retrieved on November 28, 2012 fromhttp://165.215.193.121/discoveryguides/scilit/review.pdf. Schwartz, R. S. (2000). Achieving the reforms vision: The effectiveness of a specialists-led elementary science program. School, Science and Mathematics, 100(4), 81-94. Scott, S. (2008). Perceptions of students' learning critical thinking through debate in a technology classroom: A case study. The Journal of Technology Studies, 7(1), 39-44. Scriven, M., and Paul, R. (2004). The Critical Thinking Community. Retrieved November 28, 2012 from http://www.criticalthinking.org/aboutCT/definingCT.shtml. Snyder, L.G. and Snyder M.J. (2008). “Teaching critical thinking and problem solving tools.” Dillon Beach, CA: The Foundation for Critical Thinking. Reece, G. (2002). Critical thinking and transferability: A review of the literature. Retrieved from http://www.library.american.edu/Help/research/lit_review/critical_thinking.pdf

Stedman, N. L. P., and Adams, B. L. (2012). Identifying faculty's knowledge of critical thinking concepts and perceptions of critical thinking instruction in higher education. NACTA Journal, 56(2), 9-14. Retrieved from http://search.proquest.com/docview/1040721673?accountid=35812. Taylor, A., Jones, M., Broadwell, B. and Oppewal, T. (2008). Creativity, inquiry, or accountability? Scientists' and Teachers' Perceptions of Science Education, Science Education, 92, 1058-1075. Vieira, R., Tenreiro-Vieira, C., and Martins, I. (2011). Critical thinking: Conceptual clarification and its importance in science education. Science Education International, 22(1), 43-54. Wallace, B., Berry, A., and Cave, D. (2009). Teaching problem solving and thinking skills through science. Abington, Oxon: Routledge. Watson, G., Glaser, E. M., & Rust, J. (2002). Manual of the Watson-Glaser Critical Thinking Appraisal (UK Edition). San Antonio, TX: Psychological Corporation. Yuan, S., Liao, H., Wang, Y., (2014). Developing of a scale to measure the critical thinking disposition of medical care professional. Social Behavior and Personality, 42(2), 303-312.

Fig. 1. Interdependencies among CT skills, self-regulatory functions and reflective judgment (Dwyer, Hogan, & Stewart, 2014)

Table 1 Summary of means, checks, benchmarks, and standard deviations based on pre-service teachers’ responses of the CTAS (N=120) Standard Science as Inquiry History and Nature of Science

Personal Prespective Technology Earth Science Life Science

Physical Science Total

N of Checks 995 804 635 644 321 388 225 4109

# Benchmarks

Mean

SD

6 7 5 7 9 13 11 58

13.820 11.194 9.092 8.944 4.458 3.388 3.125

0.544 0.644 0.742 0.940 1.204 1.282 1.462

Table 2 KMO and Bartlett's Test Kaiser-Meyer-Olkin Measure of sampling Adequacy

.687

Bartlett’s Test of Sphericity Approx. Chi-Square

68.694

df

28

Sig.

.000

Table 3 The Confirmatory Factorial Analysis Results Inquiry

Physics

Life

Earth

Tech

Personal

History

.724

.696

.578

.634

.328

.789

.663

.449

.818

.920

.802

.635

.780

.787

.825

.932

.870

.716

.623

.852

.838

.549

.497

.787

.812

.773

.902

.685

.858

.488

.876

.377

.866

.874

.345

.842

.903

.793

.713

.960

.730

.783

.652

.549

.702

.823

.911

.616

.825

.442

.820

.772

.549

Table 4 Ten most often checked benchmarks in descending order of their means N of Checked

Benchmark Think critically and logically to make the relationships between evidence and explanations Design and conduct a scientific investigation Develop descriptions, explanations, predictions, and models using evidence Have abilities necessary to do scientific inquiry Use appropriate tools and techniques to gather, analyze, and interpret data Recognize scientific use of empirical standards, logical arguments, and skepticism Make appropriate choices when designing and participating in scientific investigations Understanding about scientific inquiry Use mathematics in all aspects of scientific inquiry

Min

Max

6.00

2.00

10.00

5.50

0.130

7.00

1.00

10.00

5.14

0.125

1.00

5.00

5.00

5.00

0.144

1.00

5.00

5.00

5.00

0.162

13.00

1.00

10.00

5.00

0.167

7.00

1.00

10.00

5.00

0.159

11.00 5.00 6.00

1.00 3.00 1.00

10.00 10.00 10.00

4.82 4.80 4.67

0.181 0.181 0.187

CTAS Items

X

SD

Understand basic concepts and principles of science and technology

7.00

1.00

10.00

4.57

0.198

Table 5 Five least-checked benchmarks Benchmark Structure and function in living systems Transfer of energy Energy in the Earth system Properties and changes of properties in matter Structure of the Earth system

N of CTAS

Min

Max

X

SD

3 3

1

3

1.8

1 1 1 0

3 1 1 0

1.8 1 1 0

0.242 0.234

1 1 0

0.282 0.321 0

Table 6 Checked items, means, and standard deviations based on CTAS items Critical Thinking Attributes CTA1 – Think independently and develop intellectual courage CTA2 – Explore how egocentricity and sociocentricity affect feeling, thought and behavior CTA3 – Suspend judgment or prior conceptions CTA4 – Utilize various processes to resolve, re-address, and reanalyze complex situations to gain new insight CTA5 – Develop and use valid criteria for evaluation CTA6 – Raise and pursue significant questions CTA7 – Analyze arguments, interpretations, beliefs, or theories, and their implications CTA8 – Generate and assess solutions CTA9 – Make arguments, interpretations, beliefs, or theories, and their implications CTA10 – Think precisely about thinking, using critical thinking vocabulary Valid N (listwise)

# Checked Item 82

Mean

SD

0.669

0.018

54

0.392

0.022

72

0.615

0.013

95

0.830

0.002

67 77

0.515 0.653

0.005 0.013

98

0.884

0.004

93

0.800

0.007

81

0.661

0.011

60

0.430

0.013