- Email: [email protected]
Assessing creativity in a ‘New Generation’ architecture degree
Accepted Manuscript Title: Assessing creativity in a ‘New Generation’ Architecture degree Author: Andrys Onsman PII: DOI: Reference:
S1871-1871(15)30018-3 http://dx.doi.org/doi:10.1016/j.tsc.2015.07.001 TSC 311
To appear in:
Thinking Skills and Creativity
Received date: Revised date: Accepted date:
2-7-2014 15-7-2015 20-7-2015
Please cite this article as: Onsman, A.,Assessing creativity in a ‘New Generation’ Architecture degree, Thinking Skills and Creativity (2015), http://dx.doi.org/10.1016/j.tsc.2015.07.001 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
*Title page with author details
Assessing creativity in a ‘New Generation’ Architecture degree. Andrys Onsman Melbourne Centre for the Study of Higher Education, University of Melbourne, Melbourne, Australia
Ac ce pt e
d
M
an
us
cr
ip t
[email protected]
Page 1 of 19
*Manuscript without author identifiers Click here to view linked References
Assessing creativity in a ‘New Generation’ Architecture degree.
an
us
cr
ip t
Abstract: The creative aspect of architecture remains a contentious issue, especially in terms of teaching and assessing at the tertiary level. Whereas creativity is generally acknowledged as an essential component of the design process, where and how it fits into the overall structure remains open to contestation. This paper develops an argument that identifying creativity as an alternate to expertise, allows teaching, learning and assessment to focus on process and product, especially in the Design Studio context. To normalize assessment a scoring rubric based on observed leaning outcomes that accommodate the development of expertise and creativity as aspects of both product and process generated.
M
Keywords: Creativity; Architecture; Assessment; Design Studio
Ac ce pt e
d
1. Introduction The notion of how creativity becomes manifest in architecture has long vexed academic discourse because the definition of creativity as a professional attribute of architects remains elusive. Nearly a century ago Otto Rank (Rank, 1932) suggested a classificatory scale of creativity as a personal trait with three recognizable features: adapting to the social norms, rebelling against the social norms and moving beyond the social norms to satisfy an innate drive to create. Rank’s theory retains some traction today: instruments such as the Revised NEO Personality Inventory [Costa & McCrae, 1992) are used to suggest that creative people are substantially more neurotic and somewhat more extroverted than non-creative people (Gelade, 1997). Interestingly, the study also indicated that creative people are less conscientious. More recently it has been argued (Lehrer, 2012) that rather than neuroses being the driver of creative thought, it is the ability to form unexpected neural links to connect hitherto unconnected stored data. Mackinnon (Mackinnon, 1965) compared three “samples” of architects. Sample I consisted of highly creative architects; Sample II of architects with at “least two years of work experience and association with” highly creative architects; and Sample III of architects who had never worked with highly creative architects. Sample I architects actualized their creative potential, whereas Sample III architects resorted to the more conventional standards of society and of their profession”. Sample II architects displayed an overlap of both groups. In Rankian terms Sample III architects are at the stage of having adapted to social/professional norms, while Sample I architects have progressed to
1
Page 2 of 19
ip t
operating beyond social/professional norms. MacKinnon sums up his study by stating that Rank described the adapted man as one who most fully incorporates within himself the norms and dictates of society; 98% of our least creative Architects III check the adjective conscientious. He described the creative man, the artist, as one who in large measure creates his own reality; 98% of creative Architects I say they are imaginative. In describing the conflicted neurotic type, Rank observed, as many others also have, the relation of neurosis to civilization; the adjective checked most often, by 98% of our intermediate Architects II, is the adjective civilized. (Mackinnon, 1965; p281)
M
an
us
cr
Rather than leave Sample II architects as neurotic, Mackinnon argues that they are in a transitional stage. To suggest nowadays that amongst architects’ conscientiousness is the opposite of imagination and that civility is a socioprofessional interregnum between the two, would require more substantive support to say the least. Rank was strongly influenced by the emerging psychoanalysis of Freud and despite little of that theory retaining academic credibility (Crews, 1966), its influence is particularly evident in studies such as MacKinnon’s where the focus of creativity being on the person rather than on the process or the product. One of the problems with studies that focus on the person rather than the process or product is that inevitably they are analyses of self-reporting. Both the self-reporting and the analysis of the gathered data are interpretative and contestable.
Ac ce pt e
d
2. Defining creativity Creativity is widely considered to be a cornerstone of architecture, and most if not all university pro-grams refer to it as a desirable graduate attribute or intended learning outcome (Williams & Askland, 2012) However, it remains a contentious and ill-defined concept, with a long history of propositions and contestations. As Jones, Rodgers and Nicholl (2013) assert, Despite the enormous amounts of research to understand better and support creativity in design … it is still difficult to locate any common agreement among researchers on operational definitions of what it means for a designed product, space, experience, service, or system to be ‘creative’ (Jones, Rodgers & Nicholl, 2013:1). There is broad consensus that creativity involves a human process that leads to a product: Creativity is the interaction among aptitude, process, and environment by which an individual or group produces a perceptible product that is both novel and useful as defined within a social context. (Plucker at al, 2004: 90) Further, the notion of a person being creative has a deal of traction in current literature. Finally, there is growing support for an environment having agency in creativity (Williams, Ostwald & Askland, 2010; Suwala, 2012; Sternberg, 2006). However, although person, place, process and product are generally considered key aspects of creativity, it is also noted that a creative person (Landry, 2012)
2
Page 3 of 19
working in a creative environment (either virtual (Bhagwatwar, Massey & Dennis, 2013) or corporeal (Junaidy & Nagai, 2013)), engaging a creative process (Paley, 2010) may, as Harold Osbourne points out, still not end up with a creative product (Osbourne, 2012), which hints at a suggestion that the process and product may be as much accidental or incidental as it is intentional (Austin, Devin & Sullivan, 2012) and that the tag creative is a socially constructed response to, rather than inherent in, the product.
us
cr
ip t
Much of the contention surrounding the definition of creativity as a process results from the central place given to the occurrence of a significant event - the so-called ‘creative leap’, a sudden insight that is almost immediately recognised as a solution (Dorst & Cross, 2001). However, that recognition is necessarily retrospective and consequently not entirely or unquestionably reliable: even the creator who is aware that a creative act has occurred may not be able to articulate how (or why) it occurred. Moreover, they argue, studying creativity in design can be problematic because there can be no guarantee that a creative leap actually will occur during a design process.
Ac ce pt e
d
M
an
The notion of a leap suggests the crossing of a gap, with an implied corollary of it being a leap into the unknown. In more prosaic terms, a creative leap refers to an unexpected association that crosses knowledge domains (Rouse & Morris, 1986). Although knowledge domains remain a vaguely defined abstraction, in general the term refers to neural networks associated by linked triggers. By way of illustration, experts possess and can readily access extensive and highly integrated bodies of knowledge within a domain where the layman will have one larger but less sophisticated domain (Alexander, 2003). Experts are adept at seeing the underlying structure of a problem, and are able to easily select appropriate procedures for solving it. Expertise differs from creativity in that the former is acquired through repetitive practice that establishes neural pathways and/or muscle memories, whereas the latter involves the access of cross-domain knowledge and hitherto untried procedural steps. Here the scale of creativity becomes apparent: ranging from the creative leap made by an expert that is so small that only other experts will recognize it as such, to watershed epiphanies that change the world forever. Nonetheless, each is a leap and each is creative in that it deviates from a standardized procedure. The proposed understanding of who can do such creative leaping has developed over time from the individual artistic genius of the past to the collaborated output of the collective of the present. Glăveanu (2010) sums up the development as a continuum from the He-paradigm, which sees creativity as inspired genius, via the I-paradigm, which argues that everyone is creative, to the We-paradigm, which sees creativity as cumulative, collaborative and contextual. Rather than seeing each of these paradigms as replacing the previous one, it may be more profitable to consider them as complementary and not necessarily exclusive. To exploit a multi-faceted conception of creativity in the pedagogical context, the focus is returned to learning and teaching. There seems to be little doubt that in the first instance, the creative process is both cognitive and affective (Oettingen,
3
Page 4 of 19
Marquardt & Gollwitzer, 2012; Demirkan & Afacan, 2012; Wang & Chen, 2012). Initially the ideation process involves propositions and responses (‘if-then’ statements) that draw on extant knowledge domains (Rutter et al, 2012; Stouffs et al, 2012). The creation of a product that is both innovative and satisfies the demands of the brief involves not only creative leaps but also semantic cognition and relational reasoning, a procedural capability that Vandervert et al. call ‘conceptual flexibility’ (Vadervert, Schimpf & Liu, 2007).
Ac ce pt e
d
M
an
us
cr
ip t
There has been a long established acceptance of the theory that creative thinking involves both an affective or generative aspect and a cognitive or evaluative aspect, also called divergent and convergent thinking (Hunsaker & Callahan, 1995; Jaarsveld, Lachmann & van Leeuwen, 2012). The former involves the association of seemingly random, disparate ideas as potential solutions, the latter to the likely success of that association. Generally the affective component is considered intuitive and artistic (de la Fuente, 2013) while the cognitive aspect aligns closely with critical thinking, the “purposeful, self-regulatory judgment that drives problem-solving and decision-making” [Quitadamo, et al, 2008; p328]. However, thinking, as an inclusive neurological function, does not itself distinguish between cognitive and affective aspects (Burton, 2008). Because creativity is fundamentally product-centric, the creator constantly makes evaluative and selective judgments, and at times com-promises. That process depends on both raising a possibility and evaluating its likely efficacy, and ends with (or without) a product that is offered for public critique. In sum then, the areas where both formative and summative assessments of creativity are likely to be most effective pedagogically are in terms of the process and product, with the understanding that place and person may also have some degree of agency in the creative process. There is however a fundamental pedagogical difficulty in evaluating creativity when it is seen as the manufacture of an innovative, even unique solution. How can a product that solves a posed problem or satisfies a brief be judged to be ‘creative’ if, as Csikszentmihalyi (2009) asserts, creativity is simply a post hoc consensual agreement among those whose judgments are accepted as expert opinion rather than an individual aspiration? Thomas Frank posits that innovation and creativity exist “only when the correctly credentialed hive-mind agrees that it does” (Thomas, 2013). As such the creativity allocated to a product is far from being fixed. Certainly, history is littered with products which creative worth has risen and fallen apropos changing opinion and zeitgeist. A student who creates something entirely new is at risk of failing if the assessors do not appreciate or value the product in the same way that Vincent van Gogh failed to win over the critics of his time: a judgment reversed subsequently. As part of an accredited course of study a student generally needs to show mastery of the creative process during an assessment task. However, most tasks consist of collaborative projects, an approach to assessment that reflects the Weparadigm rather than the I-paradigm of creativity. The assessment rubrics however, especially the criteria that seeks to evaluate creativity, generally refer to a manufactured product rather than to process. The latter is often in the form
4
Page 5 of 19
of a metacognition or supra-analysis made manifest in a sustained self-reportage of some kind, like a diary, workbook or reflective journal.
an
us
cr
ip t
3. Architecture at the University of Melbourne. The problem that ensues when creativity is listed as an intended learning outcome or graduate attribute is brought into sharper focus when it is compared to expertise. As argued above, expertise is high-level mastery of a skill, independent of creativity. For example, given some basic parameters, it is possible to gain high-level mastery of playing a musical instrument, to develop specific expertise. However, such expertise is generally not valued as highly as expertise coupled with creativity because perceived artistry demands some measure of both. Architecture at its most valued also demands both expertise and creativity. Assessing the mastery of a skill is usually taken to be less contestable than assessing the capacity to subvert a normalized process in order to conceive a product that satisfies a stipulated set of design criteria. However, not valuing such subversion is tantamount to stifling creativity in the educational environment. Therefore, it is crucial that ways to incorporate it into the structure of the curriculum be found.
d
M
The academic structure at the University of Melbourne (the so-called Melbourne Model) consists of 9 ‘New Generation’ undergraduate degrees: Bachelor of Science, Bachelor of Music, Bachelor of Agriculture, Bachelor of Environments, Bachelor of Commerce, Bachelor of Biomedicine, Bachelor of Fine Arts, Bachelor of Oral Health and Bachelor of Arts. Although collectively these degrees have 80 majors, professional specialization generally occurs at the postgraduate level.
Ac ce pt e
The Bachelor of Environments (hereafter B.Envs) is the generalist degree that acts as the springboard for post-graduate degrees in Architecture, Civil Engineering, Construction Management, Environment, Environmental Engineering, Geomatics, Forest Ecosystem Science, Landscape Architecture, Property, Spatial Information Science, Urban Horticulture, Urban Planning and Urban Design. Currently the degree has an annual subscription of about 700 students. Although the governance of the B.Envs is housed in the Faculty of Architecture, Building and Planning courses are delivered by teaching staff from four faculties: Architecture, Engineering, Science and Land Management. In terms of creativity as a desirable graduate attribute, the B.Envs conforms to the broad international norm. Among the degree’s purported graduate attributes is the ability to “demonstrate innovation and creativity”. They will be “creative thinkers” who are capable of solving problems, among other ways, “creatively” (University of Melbourne, 2013). 4. Articulating creativity as an intended learning outcome Articulating creativity as an intended learning outcome in the B.Envs is complicated by the structure of the New Generation degree. Although it is a generalist, preparatory degree, there are eleven Majors, each of which guide the student towards a particular employment sector or an articulated postgraduate.
5
Page 6 of 19
In effect, the B.Envs has a set of graduate attributes that are actualized within the collected intended learning objectives of its Majors.
cr
ip t
Interestingly, whereas the list of the University’s generic graduate attributes includes an observed learning out-come that concerns creativity – the demonstration of “critical, creative thinking skills and strong reasoning skills” – and despite the Faculty’s emphases on its pre-eminence, there is no direct mention of creativity amongst the B.Envs intended learning outcomes. Nor do any of the Majors’ intended learning outcomes specify creativity as a desirable graduate attribute. Architecture lists “to select from a range of theories (architectural, artistic, philosophical, scientific, etc.) and make them relevant to the architectural project at hand” (University of Melbourne, 2013) as learning outcomes but there is little in the way of direct reference to an expectation that B.Envs graduates with a Major in Architecture have shown creativity.
an
us
At the postgraduate level, the main focus of the Master of Architecture is architectural design and practice. To that end it employs research-led teaching, problem-based collaborative learning and professional engagement. Among its listed intended learning outcomes is a detailed articulation of what it conceives creativity to be and what it expects its graduates to have demonstrated:
M
The cognitive and creative skills to develop and evaluate a design concept that demonstrates the exercise of theoretical reflection, critical choice, imagination and professional responsibility, through the exploration, testing and refinement of different technical and aesthetic alternatives.
Ac ce pt e
d
The technical and creative skills to produce a design that demonstrates an appreciation of economic factors, environmental issues, social and cultural issues, building systems and materials. The technical and communication skills to generate design and contractual documentation that clearly conveys information to both specialist and nonspecialist audiences and that enables a design project to be realized (University of Melbourne, 2013).
In terms of learning, creativity appears as a key element in both the conception and the realization of a design project. To that end, the Master of Architecture focuses on studio-based experiential learning that emphases both analytical and creative thinking, through the production of ideas and products. To achieve idea realization, specialist technical subjects are included in the curriculum. The aim is to have graduates who possess “cutting-edge practical skills and knowledge that are grounded in architectural history, theory and technology and who are trained and motivated to create innovative and sustainable architecture, relevant to time and place, people and culture”. To that intent, the studios are both project focused and research-led with an emphasis on rigorous analysis and the creative production of ideas: You will learn in a dynamic studio culture where emphasis is placed on the potential and importance of design. You will interact with fellow students
6
Page 7 of 19
and faculty to problem solve and propose design solutions to both real life and imagined situations (University of Melbourne, 2013).
an
us
cr
ip t
The studio is designed to be seen “as the design laboratory for testing ideas and producing unexpected results with creativity at the heart of what we do” (University of Melbourne, 2013) and assumes discipline specific modes of knowing: The interrogative, iterative and critical search that defines architecture and architectural education, has its own unique (or nearly unique) mode of knowledge creation. An architectural design program is not a simulation of office practice, nor is it an isolated, self-referential exercise in individual expression. A design studio is a ‘production machine’ where ideas exist through material form – word, text, image, model, action. The knowledge is not so much ‘discovered’ (as if it was already pre-existent) as it is ‘created’ and ‘substantiated’. It attains plausibility by the work that is done on it/to it and by the critique that either makes it evaporate or gives evidence of its supporting logic and viability. This is part of the work to ‘make it make sense’ (University of Melbourne, 2013).
Ac ce pt e
d
M
5. Assessing creativity as an observed learning outcome While creativity is clearly articulated as an intended learning outcome, how it is to be assessed is mostly less clear. In Architecture education the affective domain in assessment, though highly valued, is often overlooked – probably because it is the most difficult domain in which to achieve consensus and equity (Savic & Kasheff, 2013). Although there are some lingering concerns, the use of scoring rubrics that are constructively aligned to clearly articulated in-tended learning outcomes or objectives has been shown to be generally beneficial to student learning (Orrel, Cooper & Bowdern, 2010). On the other hand, developing a rubric generally implies the articulation of a standardized procedure: a checklist of observed compliance (Sale, 2014). Creativity, however, has been argued to be the deliberate subversion of a set of procedural steps in order to arrive at an idiosyncratic strategy that potentially leads to the manufacture of an innovative product. Therefore, any rubric that purports to accommodate creativity as a valued and evaluated aspect of studio-based objective-oriented learning ought to recognise the creative leap, regardless of its magnitude, within the learning context that has been articulated clearly in the intended learning objectives. In the first instance then, both teacher and learner have a clear understanding of the intended learning outcomes. Delving further and using one studio-based subject as an example, Design Workshop 7, Studio In{tr}ex: Emergent Systems has a bewildering array of intended learning objectives, stated in various publications. First, and most readily visible are the intended learning objectives described in the studio description generally accessible on the web: This studio will look at interactive systems as a unified problem-solving approach to design solutions: Systems
Feature
Process
Purpose
Generative
Create infinite solutions
Rule-based instructions
Elimination and optimal
7
Page 8 of 19
Agent-based
Self-organization
Emergent
Optimal design generation
Behaviour modeling + agent autonomy Distributed instructions
selection Self-resolution to given parameters Solving complex problems
cr
ip t
Research and design will focus on the interaction within and between computational systems. These systems each have their own unique features and purposes. Yet, they share a common theme – observing and analysing complexity as a chain of causes and effects. Students will look into various aspects of architectural design (structure, circulation, composition etc.) and choose one to develop extensively using systems they created in the studio. It will compel them to readdress how architects approach profound limitations found in architectural design by using a series of clear computational instructions to speculate and analyse their final design outcomes.
an
us
Although the prose and table are under the heading “Learning Objectives”, in terms of assessment, there is nothing within the above passage that can be used by the assessor without reference to a personal and idiosyncratic interpretation of what the student is actually meant to demonstrate in order to be assessed as, for example, truly exceptional. The justification for such a lack of guidance is based on the assumption that without constrictive parameters, students will be encouraged to seek creative approaches and solutions.
d
M
The studio guide provided for students who have enrolled contains much clearer intended learning objectives for each section of the studio. For example, at the end of the Generative Systems project students will be able to:
Ac ce pt e
Understand what generative systems are; Design their own generative scripts in Rhino + Grasshopper Implement rule-based systems to generate design; and Use Rhino as a fabrication interface to laser/card cut and/or 3D print projects.
Similar objectives are articulated for the Agent-Base Systems and Emergent Systems sections of the studio. It is noticeable that each of the objectives is a practical skill that does not stipulate a creative component as desirable or valuable. However, as this studio is one of the most speculative and experimental on offer there is an over-arching encouragement for students to be exploratory in how they approach each assignment. Under a heading of “Learning Outcomes” a set of aims includes verbs such as explore, discover and assess designs, evaluate outcomes, showcase, test and analyse their design, present research outcome in a coherent narrative story and document a carefully curated journal. Overall, these intended learning outcomes can be grouped into three broad categories: planning, producing and evaluating. While the three collectively constitute development in pedagogic terms, each individually is also a coherent sub-process that requires cognitive, affective and psychomotor input. To produce an innovative resolution of the assigned brief will require appropriate creative input as much as skill-based expertise and project management.
8
Page 9 of 19
us
cr
ip t
To moderate the studio’s assessment, a scoring guide that subdivided each criterion into component parts was completed by a panel of assessors and provided to the students as feedback to the project task (Figure 1). Four assessment criteria were identified: (Research informed) Design (50%); Cultural Context (20%); Communications (15%); and Technology and Environment (15%). Each of the criteria was further subdivided into component part. For example, the Design criterion was subdivided into five further aspects: Research, Intent, System Design, Architecture Program and Architectural Design, each of which was scored out of 10. Clarity of criteria was a necessity because a panel of three tutors assessed each student presentation. To that end criterion came with a set of guidance notes and each grade was exemplified by a “primary verbal descriptor for attainment of Intended Learning Outcomes”. To ensure fairness, each task was assessed by a panel of three academics, as well as by the coordinating course tutor, who collated the panel’s responses into a summative assessment pro-forma.
Ac ce pt e
d
M
an
Figure 1. MSD Design Studio Summative Assessment Pro-forma – assessors rubric.
Because anecdotal feedback indicated discontent with the rubric, more formal commentary was sought in interviews with senior tutors (who are in effect the course managers and chief assessors), with a focus on whether the extant rubric was effective in firstly addressing the issue of creativity in an appropriate and satisfactory manner, and secondly whether it addressed the issue of the fragmentation caused by the itemized nature of the assessment criteria by referring to internal consistency. In order to elicit forthright opinion, the questions were open-ended and anonymity was assured. Responses indicate that there was concern amongst the assessors about the descriptors for each level of achievement provided to the assessors. For example, 9
Page 10 of 19
a response considered excellent (H1: 80 -100%) was described as Truly Exceptional/Outstanding, whereas a response considered Very Good (H2A: 75 79%) was described as Exceptional/Outstanding. How an assessor is meant to distinguish what ‘truly’ is intended to signify in this context is not made any easier by the descriptor details:
H2A
Very Good
Truly Exceptional/Outstanding demonstration of learning outcomes. Exemplary range and depth of attainment of intended learning outcomes, secured by discriminating command of a comprehensive range of relevant materials & analyses, and by deployment of considered judgment relating to key issues, concepts and procedures Exceptional/Outstanding demonstration of learning outcomes. Conclusive attainment of virtually all intended learning outcomes, clearly grounded on a close familiarity with a wide range of supporting evidence, constructively utilized to reveal appreciable depth of understanding.
ip t
Excellent
cr
H1
an
us
Further feedback suggests that the rubric was inappropriate to the ambitions of the assessment process because it concentrated the evaluation on the inherent qualities of the product rather than on the product as a demonstration of the acquisition of the intended learning outcomes. One tutor encapsulated the perceived shortcomings of the rubric by stating it encouraged the assessors to focus on what the students “had made rather than on what they had learnt”.
Ac ce pt e
d
M
In practice, the assessors indicated that they tended to mostly ignore the descriptors, the guidance notes and primary grade categories, and instead relied on grade scores that were based principally on their personal past experience. Further, the aggregation of allotted scores was subject to moderation so that after a comparative ranking process, a student may end up with a higher primary grade than the assessment indicates is warranted (as was the case in Figure 1, where the student despite not having reached H1 standard on any criterion, was nonetheless award an H1 overall). In terms of assessing creativity, feedback suggested tutors generally felt that because the emphasis of the subject was on acquiring the skill set and expertise to make innovative ideas manifest in a variety of ways, students tended to concentrate on acquiring skills rather than exploring ideas. A noticeably disconnect became apparent when comparing two subsequent data sets. First, grade score averages indicate that students scored higher in this subject than most other subjects in the course. Second, qualitative feedback from student evaluations indicated that overall, they enjoyed this subject less than most others, citing a lack of creative engagement as the primary cause, regardless of the fact that, as pointed out by Chang and Lin (2013), the perception of creativity inherent in the product by assessors positively affects the grade given. In summary, there appears to be a disconnect between how the intended learning outcomes are understood by students and the scoring criteria articulated by the assessment rubric. As the intended learning outcomes were clearly articulated in terms of process, product and explication, attention must be turned to developing an assessment rubric that facilitates their evaluation in an educatively transparent manner. The paper now proposes an alternative mechanism for a more constructively aligned assessment process that is focused
10
Page 11 of 19
on development rather than monitoring deficiencies. Kaufman et al (2012) point out: There currently is not a creativity checklist that exists that has the criterionrelated concurrent and predictive validity one would like of tests being used for decision-making purposes (Kaufman et al, 2012: 67).
M
an
us
cr
ip t
6. Refining the assessment rubric. In response to the apparent disconnect between the course’ intended learning outcomes and those of the assessment rubric, a structure for assessment based on the theoretical framework outlined above was generated in consultation with senior tutors and course coordinators. Rather than focus on specific skills or knowledge, three areas of emphasis within the process were identified: planning, manufacture and evaluation. Based on the notion that an analytic rubric works best in tandem with a holistic rubric (i.e. one that considers the response holistically), this approach has component parts but each also has an overall score based on the structure of the observed learning outcome (SOLO), the taxonomy devised by Biggs and Collis (Biggs, 1999; Biggs & Collis, 1982) that allows a uniform description of student achievement based on how the level of complexity displayed aligns with the intended learning outcomes. Levels of understanding range from pre-structural, via uni-structural, multi-structural and relational to an extended abstract understanding. As such, the taxonomy is readily adaptable to the object-oriented learning that is the core of workshop and studio-based courses because it applies readily to both process and product.
Ac ce pt e
d
The main benefit of SOLO-based rubric is that levels of achievement reflect increasingly more complex approaches to realising an assessment, in terms of both process and product. Creativity is possibly an aspect of every level, but is more likely to become a deliberate strategy at the relational and extended abstract levels of achievement. Indeed, it could be argued that achieving an extended abstract response necessarily incorporates an element of creativity. As an example an extended abstract response for the analysis of requirements would display a student’s capacity and willingness to make connections beyond the immediate parameters of the brief, which is a key aspect of creativity. At the same time it allows the subject to retain its less specific but important learning objective of getting students to see interactive systems as a unified problemsolving approach to design solutions. A pre-structural response has nothing or little in the way of ideation, or responses to the brief. A uni-structural response has one or more unrelated ideas, whereas a multi-structural response displays numerous ideas that have a basic connectivity. A relational response demonstrates a capacity to elaborate on the combination of relevant ideas, to show cause and effect, to analyse or analogize. An extended abstract response shows a capacity to generalize, to predict, to evaluate, to propose a new (to them) combination of elements into a innovative and purposeful resolution. Most relevant to this paper is that without specific articulation to that end, the simplified assessment rubric requires a consideration of expertise and creativity
11
Page 12 of 19
ip t
as achievements in higher order competence. For that purpose, assessors were alerted to the Creativity/Expertise conceptual framework outlined above before they were asked to apply the revised scoring rubric in a practical situation. The holistic scoring rubric was described as complementary to, and a means of reintegrating the disparate elements of, the analytical scoring rubric. It was not intended to influence the grading of the response to the assessment task but rather to provide an indication of its overall quality. To that end, assessors were asked to indicate into which level they would place the product, the result of the assessment task as a whole. Figure 2: Holistic Assessment pro-forma - the structure of the observed learning outcome.
Level
Descriptors
The product* meets all requirements of the assignment at a very high standard. It displays expertise and purposeful creativity in conception, development and finalization. The product’s internal elements and their relations are coherent and cohesive and used to generalize, to evaluate and critique, to innovate and to predict. Relational The product is realised to a high standard; it satisfies most task specifications, with only minor errors. It achieves a good level of functionality** and aesthetics. It displays a high level of competence. The product’s internal elements are sequenced, classified, compared and used to show cause. MultiThe product is realised to an acceptable standard, with generally adequate levels structural of functionality but a number of problems evident. It displays developing competence. The product’s internal elements are described and defined, but not linked in a coherent way or to any purpose. UniThe product does not meet acceptable standards. Requirements of the assignment structural are unrealized and its functionality is problematic. There are numerous and significant errors. It demonstrates an awareness of competence not yet achieved. The product’s internal elements are described without definition. PreThe product is incomplete or not attempted. There is no evidence of competence. structural There are no appropriate internal elements. * Product in this context refers to what is formally submitted for assessment ** Functionality in this context refers to fitness for purpose as indicated by the assessment task
Ac ce pt e
d
M
an
us
cr
Extended Abstract
Indications from anecdotal feedback are that the addition of this comparatively small criterion to the assessment architecture caused a disproportionately large reconceptualization of both purpose and process. Assessors commented that using the criterion in the assessment process provided clarity and justification during the moderation process. A number of the assessors participating in the trial indicated that post hoc they were keen to use the criterion as and in feedback to the students because it allowed them to articulate “a qualitative response” to the product, without “splintering it into bits” because the scale made it “clear what each criteria is about. Also, each description in the second column links to and explains well each concept in the first column”. The second theme identified in the evaluative commentary was the suggestion that the holistic scale provided greater clarity to the assessors about what constitutes purposeful creativity, the notion of coming up with workable solutions to particular problems in an unexpected, innovative or unusual manner. In particular, the revised rubric provided assessors with a framework not only for the justification of an assessment but also for an explanation of that assessment that allowed a clear indication of how students might improve future assessment. In essence it allowed assessors to add a formative component to a summative assessment.
12
Page 13 of 19
cr
ip t
7. Conclusion It is probably inevitable that any attempt to coral creativity into a demystified, learnable and assessable quality will be met with resistance from those who cling to the notion that creative people are born so blessed. Conservatoria of Music, for example, continue to demand evidence of artistic capacity as an entry requirement, which implies a belief that creativity can only be developed if it already exists, that talent can only be nurtured and be allowed to flourish. The counter-argument is that there are many more orchestral players than soloists. Whereas the latter are venerated for their artistry, the former are rewarded for their expertise.
an
us
This paper has argued that the essence of creativity - the so-called creative leap is a capacity to over-ride an entrenched cognitive process in order to devise new ideas, solutions and products. Whether that leap is miniscule or immense is of lesser importance than that it is made and in learning terms that it is attempted. What is of greatest importance in the learning environment is to develop an experiential recognition of what is most effective and efficient in terms of realizing the brief, and that, in terms of innovation, both creativity and expertise are necessary components.
Ac ce pt e
d
M
Further, this paper has argued that such experiential recognition develops when the creative process is seen as constituent part of the critical thinking and analytical skills, the manufacturing skills and the evaluation skills that all production processes require to varying degrees. Whereas the artist is free to devise his or her own brief, a plan that can legitimately change as the product develops, architects are more often required to adhere to an extant brief, and creativity is generally constrained by those parameters. Essentially however, the process is the same. It is at least theoretically possible that not every brief presented to an architect will require innovation for its realization but most will rely on some degree of creativity, even if that consists of nothing more than a recombination of known elements (Guilford, 1967). There is a great deal more conceptual and practical research to be done in this area. Professions such as architecture generally rely on their practitioners having some degree of creative capacity, and there is an expectation that that capacity is nurtured and developed during training. During the institutional phase of learning, meaningful pedagogy requires that students are made aware of the component elements of the conception/realization process that will become the cornerstone of their professional practice. They need to be guided in their application through simulations and learning environments, and having internalized the skills and knowledge, be assessed by way of appropriate tasks that allow them to demonstrate their learning in an optimal and equitable manner. In proposing that the mechanism for assessment progresses from a deficit model to a developmental model, coupled with the deliberate and continuous contextualization of teaching activities within the intended learning objectives,
13
Page 14 of 19
this paper seeks to intertwine creativity with expertise as prerequisite skills for the Architecture graduate. However, it also acknowledges that there remain unanswered questions: Creativity assessment is a work in progress—we know far less about creativity and its measurement than we would like to know (Kaufman et al, 2012: 69).
ip t
7. References
cr
Alexander P. (2003). Can We Get There From Here? Educational Researcher 32: 3
us
Amabile, T. M., Conti, R., Coon, H., Lazenby, J., & Herron, M. (1996). Assessing the work environment for creativity. Academy of management journal, 39(5), 11541184.
an
Austin, R. D., Devin, L., and Sullivan, E. E. (2012). Accidental innovation: Supporting valuable unpredictability in the creative process. Organization Science, 23(5), 1505-1522.
M
Bhagwatwar, A., Massey, A., and Dennis, A. R. (2013). Creative Virtual Environments: Effect of Supra-liminal Priming on Team Brainstorming. In System Sciences (HICSS), 2013 46th Hawaii International Confer-ence on (pp. 215-224). IEEE.
Ac ce pt e
d
Biggs, J (1999) Teaching for Quality Learning at University, Buckingham: SRHE and Open University Press Biggs, J. and Collis, K. F. (1982) Evaluating the Quality of Learning - the SOLO taxonomy, New York: Academic Press Burton R. (2008). On Being Certain: Believing You Are Right Even When You're Not. New York: St. Mar-tin's Press. Callahan, C. M., Hunsaker, S. L., Adams, C. M., Moore, S. D., & Bland, L. C. (1995). Instruments used in the identification of gifted and talented students (Report No. RM-95130). Charlottesville, VA: National Research Center on the Gifted and Talented. Chang, H. T., & Lin, T. I. (2013). Discovering Taiwanese design college students’ learning performance and imaginative capacity. Thinking Skills and Creativity, 10, 23-39. Costa, P. and McCrae, R. (1992). NEO PI-R professional manual. Odessa, FL: Psychological Assessment Resources, Inc. Crews, F. (1966). The Verdict on Freud, Psychological Science 7(2) 63-68.
14
Page 15 of 19
Csikszentmihalyi, M. (2009). Creativity: Flow and the Psychology of Discovery and Invention. HarperCol-lins. Cummings, L. (1965). Organizational climates for creativity. Academy of Management Journal, 8(3), 220-227.
ip t
Davies, D., Jindal-Snape, D., Collier, C., Digby, R., Hay, P., & Howe, A. (2012). Creative Learning Environments in Education–a systematic literature review. Thinking Skills and Creativity, 8: 80–91.
cr
de la Fuente, E. D. U. A. R. D. O. (2013). Why Aesthetic Patterns Matter: Art and a “Qualitative” Social Theory. Journal for the Theory of Social Behaviour. Article first published online: 27/8/2013, DOI: 10.1111/jtsb.12036.
us
Demirkan, H., and Afacan, Y. (2012). Assessing creativity in design education: Analysis of creativity factors in the first-year design studio. Design Studies, 33(3), 262-278.
an
Dorst, K & Cross, N. (2001). Creativity in the design process: co-evolution of problem–solution, Design Studies, 22(5): 425–437.
M
Gelade, G. A. (1997). Creativity in conflict: The personality of the commercial creative. The Journal of genetic psychology, 158(1), 67-78.
d
Glăveanu, V. (2010) Paradigms in the study of creativity: introducing the perspective of cultural psychology. New ideas in psychology, 28 (1), 79-93.
Ac ce pt e
Guilford, J. P. (1967). Creativity: Yesterday, today and tomorrow. The Journal of Creative Behavior, 1(1), 3-14. Hollander, F. (2011). Power and Creativity: Creative vs. Non-creative Environment. Unpublished Masters thesis, Sociale Wetenschappen, Universiteit Utrecht. http://dspace.library.uu.nl/handle/1874/206012. Hudson, K. (2013). What Does an Ideal Creative Environment Look Like? http://drkenhudson.com/what-does-an-ideal-creative-environment-look-like/ Jaarsveld, S., Lachmann, T., and van Leeuwen, C. (2012). Creative reasoning across developmental levels: Convergence and divergence in problem creation. Intelligence, 40(2), 172-188. Jindal-Snape, D., Davies, D., Collier, C., Howe, A., Digby, R., & Hay, P. (2013). The impact of creative learning environments on learners: A systematic literature review. Improving Schools, 16(1), 21-31. Jones, P., Rodgers, P. A., and Nicholl, B. (2013). A study of university design tutors' perceptions of creativity. International Journal of Design Creativity and Innovation, (ahead-of-print), 1-12.
15
Page 16 of 19
Junaidy, D. W., and Nagai, Y. (2013). Co-creation model for traditional artisans in the current creative environment. In Proceedings of the 9th ACM Conference on Creativity & Cognition (pp. 324-327). ACM. Kaufman, J. C., Plucker, J. A., & Russell, C. M. (2012). Identifying and assessing creativity as a component of giftedness. Journal of Psychoeducational Assessment, 30(1), 60-73.
cr
ip t
Landry, H. J. (2012). Creativity and Personality Integration. Canadian Journal of Counselling and Psycho-therapy/Revue canadienne de counseling et de psychothérapie, 3(3).
Lehrer, J. (2012). Imagine: the science of creativity, Melbourne: Text Publishing
us
Mackinnon, D. W. (1965). Personality and the realization of creative potential. American Psychologist, 20(4), 273.
an
Oettingen, G., Marquardt, M. K., and Gollwitzer, P. M. (2012). Mental contrasting turns positive feedback on creative potential into successful performance. Journal of Experimental Social Psychology, 48(5), 990-996.
M
Orrel, J, Cooper, L, Bowden, M. (2010). Work in integrated learning: a guide to effective practice. London: Routledge.
Ac ce pt e
d
Osborne, H. (2012). Creativity in Art and in Life. Dialectics and Humanism, 7(3), 99-103. Paley, S. J. (2010). The Art of Invention: The Creative Process of Discovery and Design. New York: Prometheus Books. Plucker, J. A., Beghetto, R. A., & Dow, G. (2004). Why isn’t creativity more important to educational psychol-ogists? Potential, pitfalls, and future directions in creativity research. Educational Psychologist, 39, 83-96. Quitadamo I. J., Faiola C. L., Johnson J. E., and Kurtz M. J. (2008). Communitybased inquiry improves critical thinking in general education biology. CBE Life Science Education, 7, pp 27-337. Rank, O. (1932). Art and Artist: creative urge and personality development, London: Alfred A. Knopf. Rouse, W. B., and Morris, N. M. (1986). On looking into the black box: Prospects and limits in the search for mental models. Psychological bulletin, 100(3), 349. Rutter B, Kröger S, Stark R, Schweckendiek J, Windmann S, Hermann C, and Abraham A, (2012). Can clouds dance? Neural correlates of passive conceptual expansion using a metaphor processing task: Implications for creative cognition, Brain and Cognition, 78(2), 114–122
16
Page 17 of 19
Sale D. (2014). The Challenge of Reframing Engineering Education, Singapore: Springer (see especially pp 59-80 – ‘Assessing Learning’) Savic, M. & M. Kashef, (2013). Learning outcomes in affective domain within contemporary architectural curricula, International Journal of Technology and Design Education, 23(4), 987-1004.
ip t
Sternberg, R. (2006). ‘The nature of creativity’, Creativity Research Journal, 18, 132.
cr
Stouffs, R., Janssen, P., Roudavski, S., and Tunçer, B. (2013). The Creative Value Of Bad Ideas. In Conference on Computer-Aided Architectural Design Research in Asia (CAADRIA 2013) (Vol. 853, p. 862).
us
Suwala, L. (2012). The Geography of Creativity, Regional Studies, 46(8), 11031105.
an
Thomas F. (2013). Ted Talks Are Lying to You. The Salon, www.salon.com/2013/10/13/ted_talks_are_lying_to_you/
M
University of Melbourne, (2013). Bachelor of Environments course guide, https://handbook.unimelb.edu.au/view/2012/B-ENVS
Ac ce pt e
d
Vandervert, L. R., Schimpf, P. H., and Liu, H. (2007). How working memory and the cerebellum collaborate to produce creativity and innovation. Creativity Research Journal, 19(1), 1-18. Wang, L. C., and Chen, M. P. (2012). The Effects of Cognitive-Affective Interaction Strategy on Novices' Creative Performance in Game Design Project. In Advanced Learning Technologies (ICALT), 2012 IEEE 12th International Conference on (pp. 549-553), IEEE. Williams A. and Askland H. (2012). Assessing creativity: Strategies and tools to support teaching and learning in architecture and design, Sydney: Australian Government Office for Learning and Teaching, final report. Williams, A., Ostwald M. and Askland, H. (2010). Creativity, Design and Education: Theories, Positions and Challenges, Sydney: Australian Learning & Teaching Council.
17
Page 18 of 19
*Highlights (for review)
Assessing creativity in a ‘New Generation’ Architecture degree. Main changes made in response to reviewers’ feedback:
ip t
1There is broad consensus that creativity involves a human process that leads to a product: Creativity is the interaction among aptitude, process, and environment by which an individual or group produces a perceptible product that is both novel and useful as defined within a social context. (Plucker at al, 2004: 90)
an
us
cr
2Further feedback suggests that the rubric was inappropriate to the ambitions of the assessment process because it concentrated the evaluation on the inherent qualities of the product rather than on the product as a demonstration of the acquisition of the intended learning outcomes. One tutor encapsulated the perceived shortcomings of the rubric by stating it encouraged the assessors to focus on what the students “had made rather than on what they had learnt”.
Ac ce pt e
d
M
3In terms of assessing creativity, feedback suggested tutors generally felt that because the emphasis of the subject was on acquiring the skill set and expertise to make innovative ideas manifest in a variety of ways, students tended to concentrate on acquiring skills rather than exploring ideas. A noticeably disconnect became apparent when comparing two subsequent data sets. First, grade score averages indicate that students scored higher in this subject than most other subjects in the course. Second, qualitative feedback from student evaluations indicated that overall, they enjoyed this subject less than most others, citing a lack of creative engagement as the primary cause, regardless of the fact that, as pointed out by Chang and Lin (2013), the perception of creativity inherent in the product by assessors positively affects the grade given. 4In summary, there appears to be a disconnect between how the intended learning outcomes are understood by students and the scoring criteria articulated by the assessment rubric. As the intended learning outcomes were clearly articulated in terms of process, product and explication, attention must be turned to developing an assessment rubric that facilitates their evaluation in an educatively transparent manner. The paper now proposes an alternative mechanism for a more constructively aligned assessment process that is focused on development rather than monitoring deficiencies. Kaufman et al (2012) point out: There currently is not a creativity checklist that exists that has the criterionrelated concurrent and predictive validity one would like of tests being used for decision-making purposes (Kaufman et al, 2012: 67).
Page 19 of 19
S1871-1871(15)30018-3 http://dx.doi.org/doi:10.1016/j.tsc.2015.07.001 TSC 311
To appear in:
Thinking Skills and Creativity
Received date: Revised date: Accepted date:
2-7-2014 15-7-2015 20-7-2015
Please cite this article as: Onsman, A.,Assessing creativity in a ‘New Generation’ Architecture degree, Thinking Skills and Creativity (2015), http://dx.doi.org/10.1016/j.tsc.2015.07.001 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
*Title page with author details
Assessing creativity in a ‘New Generation’ Architecture degree. Andrys Onsman Melbourne Centre for the Study of Higher Education, University of Melbourne, Melbourne, Australia
Ac ce pt e
d
M
an
us
cr
ip t
[email protected]
Page 1 of 19
*Manuscript without author identifiers Click here to view linked References
Assessing creativity in a ‘New Generation’ Architecture degree.
an
us
cr
ip t
Abstract: The creative aspect of architecture remains a contentious issue, especially in terms of teaching and assessing at the tertiary level. Whereas creativity is generally acknowledged as an essential component of the design process, where and how it fits into the overall structure remains open to contestation. This paper develops an argument that identifying creativity as an alternate to expertise, allows teaching, learning and assessment to focus on process and product, especially in the Design Studio context. To normalize assessment a scoring rubric based on observed leaning outcomes that accommodate the development of expertise and creativity as aspects of both product and process generated.
M
Keywords: Creativity; Architecture; Assessment; Design Studio
Ac ce pt e
d
1. Introduction The notion of how creativity becomes manifest in architecture has long vexed academic discourse because the definition of creativity as a professional attribute of architects remains elusive. Nearly a century ago Otto Rank (Rank, 1932) suggested a classificatory scale of creativity as a personal trait with three recognizable features: adapting to the social norms, rebelling against the social norms and moving beyond the social norms to satisfy an innate drive to create. Rank’s theory retains some traction today: instruments such as the Revised NEO Personality Inventory [Costa & McCrae, 1992) are used to suggest that creative people are substantially more neurotic and somewhat more extroverted than non-creative people (Gelade, 1997). Interestingly, the study also indicated that creative people are less conscientious. More recently it has been argued (Lehrer, 2012) that rather than neuroses being the driver of creative thought, it is the ability to form unexpected neural links to connect hitherto unconnected stored data. Mackinnon (Mackinnon, 1965) compared three “samples” of architects. Sample I consisted of highly creative architects; Sample II of architects with at “least two years of work experience and association with” highly creative architects; and Sample III of architects who had never worked with highly creative architects. Sample I architects actualized their creative potential, whereas Sample III architects resorted to the more conventional standards of society and of their profession”. Sample II architects displayed an overlap of both groups. In Rankian terms Sample III architects are at the stage of having adapted to social/professional norms, while Sample I architects have progressed to
1
Page 2 of 19
ip t
operating beyond social/professional norms. MacKinnon sums up his study by stating that Rank described the adapted man as one who most fully incorporates within himself the norms and dictates of society; 98% of our least creative Architects III check the adjective conscientious. He described the creative man, the artist, as one who in large measure creates his own reality; 98% of creative Architects I say they are imaginative. In describing the conflicted neurotic type, Rank observed, as many others also have, the relation of neurosis to civilization; the adjective checked most often, by 98% of our intermediate Architects II, is the adjective civilized. (Mackinnon, 1965; p281)
M
an
us
cr
Rather than leave Sample II architects as neurotic, Mackinnon argues that they are in a transitional stage. To suggest nowadays that amongst architects’ conscientiousness is the opposite of imagination and that civility is a socioprofessional interregnum between the two, would require more substantive support to say the least. Rank was strongly influenced by the emerging psychoanalysis of Freud and despite little of that theory retaining academic credibility (Crews, 1966), its influence is particularly evident in studies such as MacKinnon’s where the focus of creativity being on the person rather than on the process or the product. One of the problems with studies that focus on the person rather than the process or product is that inevitably they are analyses of self-reporting. Both the self-reporting and the analysis of the gathered data are interpretative and contestable.
Ac ce pt e
d
2. Defining creativity Creativity is widely considered to be a cornerstone of architecture, and most if not all university pro-grams refer to it as a desirable graduate attribute or intended learning outcome (Williams & Askland, 2012) However, it remains a contentious and ill-defined concept, with a long history of propositions and contestations. As Jones, Rodgers and Nicholl (2013) assert, Despite the enormous amounts of research to understand better and support creativity in design … it is still difficult to locate any common agreement among researchers on operational definitions of what it means for a designed product, space, experience, service, or system to be ‘creative’ (Jones, Rodgers & Nicholl, 2013:1). There is broad consensus that creativity involves a human process that leads to a product: Creativity is the interaction among aptitude, process, and environment by which an individual or group produces a perceptible product that is both novel and useful as defined within a social context. (Plucker at al, 2004: 90) Further, the notion of a person being creative has a deal of traction in current literature. Finally, there is growing support for an environment having agency in creativity (Williams, Ostwald & Askland, 2010; Suwala, 2012; Sternberg, 2006). However, although person, place, process and product are generally considered key aspects of creativity, it is also noted that a creative person (Landry, 2012)
2
Page 3 of 19
working in a creative environment (either virtual (Bhagwatwar, Massey & Dennis, 2013) or corporeal (Junaidy & Nagai, 2013)), engaging a creative process (Paley, 2010) may, as Harold Osbourne points out, still not end up with a creative product (Osbourne, 2012), which hints at a suggestion that the process and product may be as much accidental or incidental as it is intentional (Austin, Devin & Sullivan, 2012) and that the tag creative is a socially constructed response to, rather than inherent in, the product.
us
cr
ip t
Much of the contention surrounding the definition of creativity as a process results from the central place given to the occurrence of a significant event - the so-called ‘creative leap’, a sudden insight that is almost immediately recognised as a solution (Dorst & Cross, 2001). However, that recognition is necessarily retrospective and consequently not entirely or unquestionably reliable: even the creator who is aware that a creative act has occurred may not be able to articulate how (or why) it occurred. Moreover, they argue, studying creativity in design can be problematic because there can be no guarantee that a creative leap actually will occur during a design process.
Ac ce pt e
d
M
an
The notion of a leap suggests the crossing of a gap, with an implied corollary of it being a leap into the unknown. In more prosaic terms, a creative leap refers to an unexpected association that crosses knowledge domains (Rouse & Morris, 1986). Although knowledge domains remain a vaguely defined abstraction, in general the term refers to neural networks associated by linked triggers. By way of illustration, experts possess and can readily access extensive and highly integrated bodies of knowledge within a domain where the layman will have one larger but less sophisticated domain (Alexander, 2003). Experts are adept at seeing the underlying structure of a problem, and are able to easily select appropriate procedures for solving it. Expertise differs from creativity in that the former is acquired through repetitive practice that establishes neural pathways and/or muscle memories, whereas the latter involves the access of cross-domain knowledge and hitherto untried procedural steps. Here the scale of creativity becomes apparent: ranging from the creative leap made by an expert that is so small that only other experts will recognize it as such, to watershed epiphanies that change the world forever. Nonetheless, each is a leap and each is creative in that it deviates from a standardized procedure. The proposed understanding of who can do such creative leaping has developed over time from the individual artistic genius of the past to the collaborated output of the collective of the present. Glăveanu (2010) sums up the development as a continuum from the He-paradigm, which sees creativity as inspired genius, via the I-paradigm, which argues that everyone is creative, to the We-paradigm, which sees creativity as cumulative, collaborative and contextual. Rather than seeing each of these paradigms as replacing the previous one, it may be more profitable to consider them as complementary and not necessarily exclusive. To exploit a multi-faceted conception of creativity in the pedagogical context, the focus is returned to learning and teaching. There seems to be little doubt that in the first instance, the creative process is both cognitive and affective (Oettingen,
3
Page 4 of 19
Marquardt & Gollwitzer, 2012; Demirkan & Afacan, 2012; Wang & Chen, 2012). Initially the ideation process involves propositions and responses (‘if-then’ statements) that draw on extant knowledge domains (Rutter et al, 2012; Stouffs et al, 2012). The creation of a product that is both innovative and satisfies the demands of the brief involves not only creative leaps but also semantic cognition and relational reasoning, a procedural capability that Vandervert et al. call ‘conceptual flexibility’ (Vadervert, Schimpf & Liu, 2007).
Ac ce pt e
d
M
an
us
cr
ip t
There has been a long established acceptance of the theory that creative thinking involves both an affective or generative aspect and a cognitive or evaluative aspect, also called divergent and convergent thinking (Hunsaker & Callahan, 1995; Jaarsveld, Lachmann & van Leeuwen, 2012). The former involves the association of seemingly random, disparate ideas as potential solutions, the latter to the likely success of that association. Generally the affective component is considered intuitive and artistic (de la Fuente, 2013) while the cognitive aspect aligns closely with critical thinking, the “purposeful, self-regulatory judgment that drives problem-solving and decision-making” [Quitadamo, et al, 2008; p328]. However, thinking, as an inclusive neurological function, does not itself distinguish between cognitive and affective aspects (Burton, 2008). Because creativity is fundamentally product-centric, the creator constantly makes evaluative and selective judgments, and at times com-promises. That process depends on both raising a possibility and evaluating its likely efficacy, and ends with (or without) a product that is offered for public critique. In sum then, the areas where both formative and summative assessments of creativity are likely to be most effective pedagogically are in terms of the process and product, with the understanding that place and person may also have some degree of agency in the creative process. There is however a fundamental pedagogical difficulty in evaluating creativity when it is seen as the manufacture of an innovative, even unique solution. How can a product that solves a posed problem or satisfies a brief be judged to be ‘creative’ if, as Csikszentmihalyi (2009) asserts, creativity is simply a post hoc consensual agreement among those whose judgments are accepted as expert opinion rather than an individual aspiration? Thomas Frank posits that innovation and creativity exist “only when the correctly credentialed hive-mind agrees that it does” (Thomas, 2013). As such the creativity allocated to a product is far from being fixed. Certainly, history is littered with products which creative worth has risen and fallen apropos changing opinion and zeitgeist. A student who creates something entirely new is at risk of failing if the assessors do not appreciate or value the product in the same way that Vincent van Gogh failed to win over the critics of his time: a judgment reversed subsequently. As part of an accredited course of study a student generally needs to show mastery of the creative process during an assessment task. However, most tasks consist of collaborative projects, an approach to assessment that reflects the Weparadigm rather than the I-paradigm of creativity. The assessment rubrics however, especially the criteria that seeks to evaluate creativity, generally refer to a manufactured product rather than to process. The latter is often in the form
4
Page 5 of 19
of a metacognition or supra-analysis made manifest in a sustained self-reportage of some kind, like a diary, workbook or reflective journal.
an
us
cr
ip t
3. Architecture at the University of Melbourne. The problem that ensues when creativity is listed as an intended learning outcome or graduate attribute is brought into sharper focus when it is compared to expertise. As argued above, expertise is high-level mastery of a skill, independent of creativity. For example, given some basic parameters, it is possible to gain high-level mastery of playing a musical instrument, to develop specific expertise. However, such expertise is generally not valued as highly as expertise coupled with creativity because perceived artistry demands some measure of both. Architecture at its most valued also demands both expertise and creativity. Assessing the mastery of a skill is usually taken to be less contestable than assessing the capacity to subvert a normalized process in order to conceive a product that satisfies a stipulated set of design criteria. However, not valuing such subversion is tantamount to stifling creativity in the educational environment. Therefore, it is crucial that ways to incorporate it into the structure of the curriculum be found.
d
M
The academic structure at the University of Melbourne (the so-called Melbourne Model) consists of 9 ‘New Generation’ undergraduate degrees: Bachelor of Science, Bachelor of Music, Bachelor of Agriculture, Bachelor of Environments, Bachelor of Commerce, Bachelor of Biomedicine, Bachelor of Fine Arts, Bachelor of Oral Health and Bachelor of Arts. Although collectively these degrees have 80 majors, professional specialization generally occurs at the postgraduate level.
Ac ce pt e
The Bachelor of Environments (hereafter B.Envs) is the generalist degree that acts as the springboard for post-graduate degrees in Architecture, Civil Engineering, Construction Management, Environment, Environmental Engineering, Geomatics, Forest Ecosystem Science, Landscape Architecture, Property, Spatial Information Science, Urban Horticulture, Urban Planning and Urban Design. Currently the degree has an annual subscription of about 700 students. Although the governance of the B.Envs is housed in the Faculty of Architecture, Building and Planning courses are delivered by teaching staff from four faculties: Architecture, Engineering, Science and Land Management. In terms of creativity as a desirable graduate attribute, the B.Envs conforms to the broad international norm. Among the degree’s purported graduate attributes is the ability to “demonstrate innovation and creativity”. They will be “creative thinkers” who are capable of solving problems, among other ways, “creatively” (University of Melbourne, 2013). 4. Articulating creativity as an intended learning outcome Articulating creativity as an intended learning outcome in the B.Envs is complicated by the structure of the New Generation degree. Although it is a generalist, preparatory degree, there are eleven Majors, each of which guide the student towards a particular employment sector or an articulated postgraduate.
5
Page 6 of 19
In effect, the B.Envs has a set of graduate attributes that are actualized within the collected intended learning objectives of its Majors.
cr
ip t
Interestingly, whereas the list of the University’s generic graduate attributes includes an observed learning out-come that concerns creativity – the demonstration of “critical, creative thinking skills and strong reasoning skills” – and despite the Faculty’s emphases on its pre-eminence, there is no direct mention of creativity amongst the B.Envs intended learning outcomes. Nor do any of the Majors’ intended learning outcomes specify creativity as a desirable graduate attribute. Architecture lists “to select from a range of theories (architectural, artistic, philosophical, scientific, etc.) and make them relevant to the architectural project at hand” (University of Melbourne, 2013) as learning outcomes but there is little in the way of direct reference to an expectation that B.Envs graduates with a Major in Architecture have shown creativity.
an
us
At the postgraduate level, the main focus of the Master of Architecture is architectural design and practice. To that end it employs research-led teaching, problem-based collaborative learning and professional engagement. Among its listed intended learning outcomes is a detailed articulation of what it conceives creativity to be and what it expects its graduates to have demonstrated:
M
The cognitive and creative skills to develop and evaluate a design concept that demonstrates the exercise of theoretical reflection, critical choice, imagination and professional responsibility, through the exploration, testing and refinement of different technical and aesthetic alternatives.
Ac ce pt e
d
The technical and creative skills to produce a design that demonstrates an appreciation of economic factors, environmental issues, social and cultural issues, building systems and materials. The technical and communication skills to generate design and contractual documentation that clearly conveys information to both specialist and nonspecialist audiences and that enables a design project to be realized (University of Melbourne, 2013).
In terms of learning, creativity appears as a key element in both the conception and the realization of a design project. To that end, the Master of Architecture focuses on studio-based experiential learning that emphases both analytical and creative thinking, through the production of ideas and products. To achieve idea realization, specialist technical subjects are included in the curriculum. The aim is to have graduates who possess “cutting-edge practical skills and knowledge that are grounded in architectural history, theory and technology and who are trained and motivated to create innovative and sustainable architecture, relevant to time and place, people and culture”. To that intent, the studios are both project focused and research-led with an emphasis on rigorous analysis and the creative production of ideas: You will learn in a dynamic studio culture where emphasis is placed on the potential and importance of design. You will interact with fellow students
6
Page 7 of 19
and faculty to problem solve and propose design solutions to both real life and imagined situations (University of Melbourne, 2013).
an
us
cr
ip t
The studio is designed to be seen “as the design laboratory for testing ideas and producing unexpected results with creativity at the heart of what we do” (University of Melbourne, 2013) and assumes discipline specific modes of knowing: The interrogative, iterative and critical search that defines architecture and architectural education, has its own unique (or nearly unique) mode of knowledge creation. An architectural design program is not a simulation of office practice, nor is it an isolated, self-referential exercise in individual expression. A design studio is a ‘production machine’ where ideas exist through material form – word, text, image, model, action. The knowledge is not so much ‘discovered’ (as if it was already pre-existent) as it is ‘created’ and ‘substantiated’. It attains plausibility by the work that is done on it/to it and by the critique that either makes it evaporate or gives evidence of its supporting logic and viability. This is part of the work to ‘make it make sense’ (University of Melbourne, 2013).
Ac ce pt e
d
M
5. Assessing creativity as an observed learning outcome While creativity is clearly articulated as an intended learning outcome, how it is to be assessed is mostly less clear. In Architecture education the affective domain in assessment, though highly valued, is often overlooked – probably because it is the most difficult domain in which to achieve consensus and equity (Savic & Kasheff, 2013). Although there are some lingering concerns, the use of scoring rubrics that are constructively aligned to clearly articulated in-tended learning outcomes or objectives has been shown to be generally beneficial to student learning (Orrel, Cooper & Bowdern, 2010). On the other hand, developing a rubric generally implies the articulation of a standardized procedure: a checklist of observed compliance (Sale, 2014). Creativity, however, has been argued to be the deliberate subversion of a set of procedural steps in order to arrive at an idiosyncratic strategy that potentially leads to the manufacture of an innovative product. Therefore, any rubric that purports to accommodate creativity as a valued and evaluated aspect of studio-based objective-oriented learning ought to recognise the creative leap, regardless of its magnitude, within the learning context that has been articulated clearly in the intended learning objectives. In the first instance then, both teacher and learner have a clear understanding of the intended learning outcomes. Delving further and using one studio-based subject as an example, Design Workshop 7, Studio In{tr}ex: Emergent Systems has a bewildering array of intended learning objectives, stated in various publications. First, and most readily visible are the intended learning objectives described in the studio description generally accessible on the web: This studio will look at interactive systems as a unified problem-solving approach to design solutions: Systems
Feature
Process
Purpose
Generative
Create infinite solutions
Rule-based instructions
Elimination and optimal
7
Page 8 of 19
Agent-based
Self-organization
Emergent
Optimal design generation
Behaviour modeling + agent autonomy Distributed instructions
selection Self-resolution to given parameters Solving complex problems
cr
ip t
Research and design will focus on the interaction within and between computational systems. These systems each have their own unique features and purposes. Yet, they share a common theme – observing and analysing complexity as a chain of causes and effects. Students will look into various aspects of architectural design (structure, circulation, composition etc.) and choose one to develop extensively using systems they created in the studio. It will compel them to readdress how architects approach profound limitations found in architectural design by using a series of clear computational instructions to speculate and analyse their final design outcomes.
an
us
Although the prose and table are under the heading “Learning Objectives”, in terms of assessment, there is nothing within the above passage that can be used by the assessor without reference to a personal and idiosyncratic interpretation of what the student is actually meant to demonstrate in order to be assessed as, for example, truly exceptional. The justification for such a lack of guidance is based on the assumption that without constrictive parameters, students will be encouraged to seek creative approaches and solutions.
d
M
The studio guide provided for students who have enrolled contains much clearer intended learning objectives for each section of the studio. For example, at the end of the Generative Systems project students will be able to:
Ac ce pt e
Understand what generative systems are; Design their own generative scripts in Rhino + Grasshopper Implement rule-based systems to generate design; and Use Rhino as a fabrication interface to laser/card cut and/or 3D print projects.
Similar objectives are articulated for the Agent-Base Systems and Emergent Systems sections of the studio. It is noticeable that each of the objectives is a practical skill that does not stipulate a creative component as desirable or valuable. However, as this studio is one of the most speculative and experimental on offer there is an over-arching encouragement for students to be exploratory in how they approach each assignment. Under a heading of “Learning Outcomes” a set of aims includes verbs such as explore, discover and assess designs, evaluate outcomes, showcase, test and analyse their design, present research outcome in a coherent narrative story and document a carefully curated journal. Overall, these intended learning outcomes can be grouped into three broad categories: planning, producing and evaluating. While the three collectively constitute development in pedagogic terms, each individually is also a coherent sub-process that requires cognitive, affective and psychomotor input. To produce an innovative resolution of the assigned brief will require appropriate creative input as much as skill-based expertise and project management.
8
Page 9 of 19
us
cr
ip t
To moderate the studio’s assessment, a scoring guide that subdivided each criterion into component parts was completed by a panel of assessors and provided to the students as feedback to the project task (Figure 1). Four assessment criteria were identified: (Research informed) Design (50%); Cultural Context (20%); Communications (15%); and Technology and Environment (15%). Each of the criteria was further subdivided into component part. For example, the Design criterion was subdivided into five further aspects: Research, Intent, System Design, Architecture Program and Architectural Design, each of which was scored out of 10. Clarity of criteria was a necessity because a panel of three tutors assessed each student presentation. To that end criterion came with a set of guidance notes and each grade was exemplified by a “primary verbal descriptor for attainment of Intended Learning Outcomes”. To ensure fairness, each task was assessed by a panel of three academics, as well as by the coordinating course tutor, who collated the panel’s responses into a summative assessment pro-forma.
Ac ce pt e
d
M
an
Figure 1. MSD Design Studio Summative Assessment Pro-forma – assessors rubric.
Because anecdotal feedback indicated discontent with the rubric, more formal commentary was sought in interviews with senior tutors (who are in effect the course managers and chief assessors), with a focus on whether the extant rubric was effective in firstly addressing the issue of creativity in an appropriate and satisfactory manner, and secondly whether it addressed the issue of the fragmentation caused by the itemized nature of the assessment criteria by referring to internal consistency. In order to elicit forthright opinion, the questions were open-ended and anonymity was assured. Responses indicate that there was concern amongst the assessors about the descriptors for each level of achievement provided to the assessors. For example, 9
Page 10 of 19
a response considered excellent (H1: 80 -100%) was described as Truly Exceptional/Outstanding, whereas a response considered Very Good (H2A: 75 79%) was described as Exceptional/Outstanding. How an assessor is meant to distinguish what ‘truly’ is intended to signify in this context is not made any easier by the descriptor details:
H2A
Very Good
Truly Exceptional/Outstanding demonstration of learning outcomes. Exemplary range and depth of attainment of intended learning outcomes, secured by discriminating command of a comprehensive range of relevant materials & analyses, and by deployment of considered judgment relating to key issues, concepts and procedures Exceptional/Outstanding demonstration of learning outcomes. Conclusive attainment of virtually all intended learning outcomes, clearly grounded on a close familiarity with a wide range of supporting evidence, constructively utilized to reveal appreciable depth of understanding.
ip t
Excellent
cr
H1
an
us
Further feedback suggests that the rubric was inappropriate to the ambitions of the assessment process because it concentrated the evaluation on the inherent qualities of the product rather than on the product as a demonstration of the acquisition of the intended learning outcomes. One tutor encapsulated the perceived shortcomings of the rubric by stating it encouraged the assessors to focus on what the students “had made rather than on what they had learnt”.
Ac ce pt e
d
M
In practice, the assessors indicated that they tended to mostly ignore the descriptors, the guidance notes and primary grade categories, and instead relied on grade scores that were based principally on their personal past experience. Further, the aggregation of allotted scores was subject to moderation so that after a comparative ranking process, a student may end up with a higher primary grade than the assessment indicates is warranted (as was the case in Figure 1, where the student despite not having reached H1 standard on any criterion, was nonetheless award an H1 overall). In terms of assessing creativity, feedback suggested tutors generally felt that because the emphasis of the subject was on acquiring the skill set and expertise to make innovative ideas manifest in a variety of ways, students tended to concentrate on acquiring skills rather than exploring ideas. A noticeably disconnect became apparent when comparing two subsequent data sets. First, grade score averages indicate that students scored higher in this subject than most other subjects in the course. Second, qualitative feedback from student evaluations indicated that overall, they enjoyed this subject less than most others, citing a lack of creative engagement as the primary cause, regardless of the fact that, as pointed out by Chang and Lin (2013), the perception of creativity inherent in the product by assessors positively affects the grade given. In summary, there appears to be a disconnect between how the intended learning outcomes are understood by students and the scoring criteria articulated by the assessment rubric. As the intended learning outcomes were clearly articulated in terms of process, product and explication, attention must be turned to developing an assessment rubric that facilitates their evaluation in an educatively transparent manner. The paper now proposes an alternative mechanism for a more constructively aligned assessment process that is focused
10
Page 11 of 19
on development rather than monitoring deficiencies. Kaufman et al (2012) point out: There currently is not a creativity checklist that exists that has the criterionrelated concurrent and predictive validity one would like of tests being used for decision-making purposes (Kaufman et al, 2012: 67).
M
an
us
cr
ip t
6. Refining the assessment rubric. In response to the apparent disconnect between the course’ intended learning outcomes and those of the assessment rubric, a structure for assessment based on the theoretical framework outlined above was generated in consultation with senior tutors and course coordinators. Rather than focus on specific skills or knowledge, three areas of emphasis within the process were identified: planning, manufacture and evaluation. Based on the notion that an analytic rubric works best in tandem with a holistic rubric (i.e. one that considers the response holistically), this approach has component parts but each also has an overall score based on the structure of the observed learning outcome (SOLO), the taxonomy devised by Biggs and Collis (Biggs, 1999; Biggs & Collis, 1982) that allows a uniform description of student achievement based on how the level of complexity displayed aligns with the intended learning outcomes. Levels of understanding range from pre-structural, via uni-structural, multi-structural and relational to an extended abstract understanding. As such, the taxonomy is readily adaptable to the object-oriented learning that is the core of workshop and studio-based courses because it applies readily to both process and product.
Ac ce pt e
d
The main benefit of SOLO-based rubric is that levels of achievement reflect increasingly more complex approaches to realising an assessment, in terms of both process and product. Creativity is possibly an aspect of every level, but is more likely to become a deliberate strategy at the relational and extended abstract levels of achievement. Indeed, it could be argued that achieving an extended abstract response necessarily incorporates an element of creativity. As an example an extended abstract response for the analysis of requirements would display a student’s capacity and willingness to make connections beyond the immediate parameters of the brief, which is a key aspect of creativity. At the same time it allows the subject to retain its less specific but important learning objective of getting students to see interactive systems as a unified problemsolving approach to design solutions. A pre-structural response has nothing or little in the way of ideation, or responses to the brief. A uni-structural response has one or more unrelated ideas, whereas a multi-structural response displays numerous ideas that have a basic connectivity. A relational response demonstrates a capacity to elaborate on the combination of relevant ideas, to show cause and effect, to analyse or analogize. An extended abstract response shows a capacity to generalize, to predict, to evaluate, to propose a new (to them) combination of elements into a innovative and purposeful resolution. Most relevant to this paper is that without specific articulation to that end, the simplified assessment rubric requires a consideration of expertise and creativity
11
Page 12 of 19
ip t
as achievements in higher order competence. For that purpose, assessors were alerted to the Creativity/Expertise conceptual framework outlined above before they were asked to apply the revised scoring rubric in a practical situation. The holistic scoring rubric was described as complementary to, and a means of reintegrating the disparate elements of, the analytical scoring rubric. It was not intended to influence the grading of the response to the assessment task but rather to provide an indication of its overall quality. To that end, assessors were asked to indicate into which level they would place the product, the result of the assessment task as a whole. Figure 2: Holistic Assessment pro-forma - the structure of the observed learning outcome.
Level
Descriptors
The product* meets all requirements of the assignment at a very high standard. It displays expertise and purposeful creativity in conception, development and finalization. The product’s internal elements and their relations are coherent and cohesive and used to generalize, to evaluate and critique, to innovate and to predict. Relational The product is realised to a high standard; it satisfies most task specifications, with only minor errors. It achieves a good level of functionality** and aesthetics. It displays a high level of competence. The product’s internal elements are sequenced, classified, compared and used to show cause. MultiThe product is realised to an acceptable standard, with generally adequate levels structural of functionality but a number of problems evident. It displays developing competence. The product’s internal elements are described and defined, but not linked in a coherent way or to any purpose. UniThe product does not meet acceptable standards. Requirements of the assignment structural are unrealized and its functionality is problematic. There are numerous and significant errors. It demonstrates an awareness of competence not yet achieved. The product’s internal elements are described without definition. PreThe product is incomplete or not attempted. There is no evidence of competence. structural There are no appropriate internal elements. * Product in this context refers to what is formally submitted for assessment ** Functionality in this context refers to fitness for purpose as indicated by the assessment task
Ac ce pt e
d
M
an
us
cr
Extended Abstract
Indications from anecdotal feedback are that the addition of this comparatively small criterion to the assessment architecture caused a disproportionately large reconceptualization of both purpose and process. Assessors commented that using the criterion in the assessment process provided clarity and justification during the moderation process. A number of the assessors participating in the trial indicated that post hoc they were keen to use the criterion as and in feedback to the students because it allowed them to articulate “a qualitative response” to the product, without “splintering it into bits” because the scale made it “clear what each criteria is about. Also, each description in the second column links to and explains well each concept in the first column”. The second theme identified in the evaluative commentary was the suggestion that the holistic scale provided greater clarity to the assessors about what constitutes purposeful creativity, the notion of coming up with workable solutions to particular problems in an unexpected, innovative or unusual manner. In particular, the revised rubric provided assessors with a framework not only for the justification of an assessment but also for an explanation of that assessment that allowed a clear indication of how students might improve future assessment. In essence it allowed assessors to add a formative component to a summative assessment.
12
Page 13 of 19
cr
ip t
7. Conclusion It is probably inevitable that any attempt to coral creativity into a demystified, learnable and assessable quality will be met with resistance from those who cling to the notion that creative people are born so blessed. Conservatoria of Music, for example, continue to demand evidence of artistic capacity as an entry requirement, which implies a belief that creativity can only be developed if it already exists, that talent can only be nurtured and be allowed to flourish. The counter-argument is that there are many more orchestral players than soloists. Whereas the latter are venerated for their artistry, the former are rewarded for their expertise.
an
us
This paper has argued that the essence of creativity - the so-called creative leap is a capacity to over-ride an entrenched cognitive process in order to devise new ideas, solutions and products. Whether that leap is miniscule or immense is of lesser importance than that it is made and in learning terms that it is attempted. What is of greatest importance in the learning environment is to develop an experiential recognition of what is most effective and efficient in terms of realizing the brief, and that, in terms of innovation, both creativity and expertise are necessary components.
Ac ce pt e
d
M
Further, this paper has argued that such experiential recognition develops when the creative process is seen as constituent part of the critical thinking and analytical skills, the manufacturing skills and the evaluation skills that all production processes require to varying degrees. Whereas the artist is free to devise his or her own brief, a plan that can legitimately change as the product develops, architects are more often required to adhere to an extant brief, and creativity is generally constrained by those parameters. Essentially however, the process is the same. It is at least theoretically possible that not every brief presented to an architect will require innovation for its realization but most will rely on some degree of creativity, even if that consists of nothing more than a recombination of known elements (Guilford, 1967). There is a great deal more conceptual and practical research to be done in this area. Professions such as architecture generally rely on their practitioners having some degree of creative capacity, and there is an expectation that that capacity is nurtured and developed during training. During the institutional phase of learning, meaningful pedagogy requires that students are made aware of the component elements of the conception/realization process that will become the cornerstone of their professional practice. They need to be guided in their application through simulations and learning environments, and having internalized the skills and knowledge, be assessed by way of appropriate tasks that allow them to demonstrate their learning in an optimal and equitable manner. In proposing that the mechanism for assessment progresses from a deficit model to a developmental model, coupled with the deliberate and continuous contextualization of teaching activities within the intended learning objectives,
13
Page 14 of 19
this paper seeks to intertwine creativity with expertise as prerequisite skills for the Architecture graduate. However, it also acknowledges that there remain unanswered questions: Creativity assessment is a work in progress—we know far less about creativity and its measurement than we would like to know (Kaufman et al, 2012: 69).
ip t
7. References
cr
Alexander P. (2003). Can We Get There From Here? Educational Researcher 32: 3
us
Amabile, T. M., Conti, R., Coon, H., Lazenby, J., & Herron, M. (1996). Assessing the work environment for creativity. Academy of management journal, 39(5), 11541184.
an
Austin, R. D., Devin, L., and Sullivan, E. E. (2012). Accidental innovation: Supporting valuable unpredictability in the creative process. Organization Science, 23(5), 1505-1522.
M
Bhagwatwar, A., Massey, A., and Dennis, A. R. (2013). Creative Virtual Environments: Effect of Supra-liminal Priming on Team Brainstorming. In System Sciences (HICSS), 2013 46th Hawaii International Confer-ence on (pp. 215-224). IEEE.
Ac ce pt e
d
Biggs, J (1999) Teaching for Quality Learning at University, Buckingham: SRHE and Open University Press Biggs, J. and Collis, K. F. (1982) Evaluating the Quality of Learning - the SOLO taxonomy, New York: Academic Press Burton R. (2008). On Being Certain: Believing You Are Right Even When You're Not. New York: St. Mar-tin's Press. Callahan, C. M., Hunsaker, S. L., Adams, C. M., Moore, S. D., & Bland, L. C. (1995). Instruments used in the identification of gifted and talented students (Report No. RM-95130). Charlottesville, VA: National Research Center on the Gifted and Talented. Chang, H. T., & Lin, T. I. (2013). Discovering Taiwanese design college students’ learning performance and imaginative capacity. Thinking Skills and Creativity, 10, 23-39. Costa, P. and McCrae, R. (1992). NEO PI-R professional manual. Odessa, FL: Psychological Assessment Resources, Inc. Crews, F. (1966). The Verdict on Freud, Psychological Science 7(2) 63-68.
14
Page 15 of 19
Csikszentmihalyi, M. (2009). Creativity: Flow and the Psychology of Discovery and Invention. HarperCol-lins. Cummings, L. (1965). Organizational climates for creativity. Academy of Management Journal, 8(3), 220-227.
ip t
Davies, D., Jindal-Snape, D., Collier, C., Digby, R., Hay, P., & Howe, A. (2012). Creative Learning Environments in Education–a systematic literature review. Thinking Skills and Creativity, 8: 80–91.
cr
de la Fuente, E. D. U. A. R. D. O. (2013). Why Aesthetic Patterns Matter: Art and a “Qualitative” Social Theory. Journal for the Theory of Social Behaviour. Article first published online: 27/8/2013, DOI: 10.1111/jtsb.12036.
us
Demirkan, H., and Afacan, Y. (2012). Assessing creativity in design education: Analysis of creativity factors in the first-year design studio. Design Studies, 33(3), 262-278.
an
Dorst, K & Cross, N. (2001). Creativity in the design process: co-evolution of problem–solution, Design Studies, 22(5): 425–437.
M
Gelade, G. A. (1997). Creativity in conflict: The personality of the commercial creative. The Journal of genetic psychology, 158(1), 67-78.
d
Glăveanu, V. (2010) Paradigms in the study of creativity: introducing the perspective of cultural psychology. New ideas in psychology, 28 (1), 79-93.
Ac ce pt e
Guilford, J. P. (1967). Creativity: Yesterday, today and tomorrow. The Journal of Creative Behavior, 1(1), 3-14. Hollander, F. (2011). Power and Creativity: Creative vs. Non-creative Environment. Unpublished Masters thesis, Sociale Wetenschappen, Universiteit Utrecht. http://dspace.library.uu.nl/handle/1874/206012. Hudson, K. (2013). What Does an Ideal Creative Environment Look Like? http://drkenhudson.com/what-does-an-ideal-creative-environment-look-like/ Jaarsveld, S., Lachmann, T., and van Leeuwen, C. (2012). Creative reasoning across developmental levels: Convergence and divergence in problem creation. Intelligence, 40(2), 172-188. Jindal-Snape, D., Davies, D., Collier, C., Howe, A., Digby, R., & Hay, P. (2013). The impact of creative learning environments on learners: A systematic literature review. Improving Schools, 16(1), 21-31. Jones, P., Rodgers, P. A., and Nicholl, B. (2013). A study of university design tutors' perceptions of creativity. International Journal of Design Creativity and Innovation, (ahead-of-print), 1-12.
15
Page 16 of 19
Junaidy, D. W., and Nagai, Y. (2013). Co-creation model for traditional artisans in the current creative environment. In Proceedings of the 9th ACM Conference on Creativity & Cognition (pp. 324-327). ACM. Kaufman, J. C., Plucker, J. A., & Russell, C. M. (2012). Identifying and assessing creativity as a component of giftedness. Journal of Psychoeducational Assessment, 30(1), 60-73.
cr
ip t
Landry, H. J. (2012). Creativity and Personality Integration. Canadian Journal of Counselling and Psycho-therapy/Revue canadienne de counseling et de psychothérapie, 3(3).
Lehrer, J. (2012). Imagine: the science of creativity, Melbourne: Text Publishing
us
Mackinnon, D. W. (1965). Personality and the realization of creative potential. American Psychologist, 20(4), 273.
an
Oettingen, G., Marquardt, M. K., and Gollwitzer, P. M. (2012). Mental contrasting turns positive feedback on creative potential into successful performance. Journal of Experimental Social Psychology, 48(5), 990-996.
M
Orrel, J, Cooper, L, Bowden, M. (2010). Work in integrated learning: a guide to effective practice. London: Routledge.
Ac ce pt e
d
Osborne, H. (2012). Creativity in Art and in Life. Dialectics and Humanism, 7(3), 99-103. Paley, S. J. (2010). The Art of Invention: The Creative Process of Discovery and Design. New York: Prometheus Books. Plucker, J. A., Beghetto, R. A., & Dow, G. (2004). Why isn’t creativity more important to educational psychol-ogists? Potential, pitfalls, and future directions in creativity research. Educational Psychologist, 39, 83-96. Quitadamo I. J., Faiola C. L., Johnson J. E., and Kurtz M. J. (2008). Communitybased inquiry improves critical thinking in general education biology. CBE Life Science Education, 7, pp 27-337. Rank, O. (1932). Art and Artist: creative urge and personality development, London: Alfred A. Knopf. Rouse, W. B., and Morris, N. M. (1986). On looking into the black box: Prospects and limits in the search for mental models. Psychological bulletin, 100(3), 349. Rutter B, Kröger S, Stark R, Schweckendiek J, Windmann S, Hermann C, and Abraham A, (2012). Can clouds dance? Neural correlates of passive conceptual expansion using a metaphor processing task: Implications for creative cognition, Brain and Cognition, 78(2), 114–122
16
Page 17 of 19
Sale D. (2014). The Challenge of Reframing Engineering Education, Singapore: Springer (see especially pp 59-80 – ‘Assessing Learning’) Savic, M. & M. Kashef, (2013). Learning outcomes in affective domain within contemporary architectural curricula, International Journal of Technology and Design Education, 23(4), 987-1004.
ip t
Sternberg, R. (2006). ‘The nature of creativity’, Creativity Research Journal, 18, 132.
cr
Stouffs, R., Janssen, P., Roudavski, S., and Tunçer, B. (2013). The Creative Value Of Bad Ideas. In Conference on Computer-Aided Architectural Design Research in Asia (CAADRIA 2013) (Vol. 853, p. 862).
us
Suwala, L. (2012). The Geography of Creativity, Regional Studies, 46(8), 11031105.
an
Thomas F. (2013). Ted Talks Are Lying to You. The Salon, www.salon.com/2013/10/13/ted_talks_are_lying_to_you/
M
University of Melbourne, (2013). Bachelor of Environments course guide, https://handbook.unimelb.edu.au/view/2012/B-ENVS
Ac ce pt e
d
Vandervert, L. R., Schimpf, P. H., and Liu, H. (2007). How working memory and the cerebellum collaborate to produce creativity and innovation. Creativity Research Journal, 19(1), 1-18. Wang, L. C., and Chen, M. P. (2012). The Effects of Cognitive-Affective Interaction Strategy on Novices' Creative Performance in Game Design Project. In Advanced Learning Technologies (ICALT), 2012 IEEE 12th International Conference on (pp. 549-553), IEEE. Williams A. and Askland H. (2012). Assessing creativity: Strategies and tools to support teaching and learning in architecture and design, Sydney: Australian Government Office for Learning and Teaching, final report. Williams, A., Ostwald M. and Askland, H. (2010). Creativity, Design and Education: Theories, Positions and Challenges, Sydney: Australian Learning & Teaching Council.
17
Page 18 of 19
*Highlights (for review)
Assessing creativity in a ‘New Generation’ Architecture degree. Main changes made in response to reviewers’ feedback:
ip t
1There is broad consensus that creativity involves a human process that leads to a product: Creativity is the interaction among aptitude, process, and environment by which an individual or group produces a perceptible product that is both novel and useful as defined within a social context. (Plucker at al, 2004: 90)
an
us
cr
2Further feedback suggests that the rubric was inappropriate to the ambitions of the assessment process because it concentrated the evaluation on the inherent qualities of the product rather than on the product as a demonstration of the acquisition of the intended learning outcomes. One tutor encapsulated the perceived shortcomings of the rubric by stating it encouraged the assessors to focus on what the students “had made rather than on what they had learnt”.
Ac ce pt e
d
M
3In terms of assessing creativity, feedback suggested tutors generally felt that because the emphasis of the subject was on acquiring the skill set and expertise to make innovative ideas manifest in a variety of ways, students tended to concentrate on acquiring skills rather than exploring ideas. A noticeably disconnect became apparent when comparing two subsequent data sets. First, grade score averages indicate that students scored higher in this subject than most other subjects in the course. Second, qualitative feedback from student evaluations indicated that overall, they enjoyed this subject less than most others, citing a lack of creative engagement as the primary cause, regardless of the fact that, as pointed out by Chang and Lin (2013), the perception of creativity inherent in the product by assessors positively affects the grade given. 4In summary, there appears to be a disconnect between how the intended learning outcomes are understood by students and the scoring criteria articulated by the assessment rubric. As the intended learning outcomes were clearly articulated in terms of process, product and explication, attention must be turned to developing an assessment rubric that facilitates their evaluation in an educatively transparent manner. The paper now proposes an alternative mechanism for a more constructively aligned assessment process that is focused on development rather than monitoring deficiencies. Kaufman et al (2012) point out: There currently is not a creativity checklist that exists that has the criterionrelated concurrent and predictive validity one would like of tests being used for decision-making purposes (Kaufman et al, 2012: 67).
Page 19 of 19