Towards a framework for evaluation of computer visual simulations in environmental design

Towards a framework for evaluation of computer visual simulations in environmental design Lamine Mahdjoubi, School of Engineering and the Built Environment, University of Wolverhampton, Wulfruna Street, Wolverhampton, WV1 1SB, UK John Wiltshire, Department of Architecture, Planning and Landscape, University of Newcastle, Newcastle upon Tyne, NE1 7RU, UK Research has been conducted into the assessment of visual simulation as a credible method for predicting future environments, especially in building and landscape evaluation. Despite this work a conceptual framework to guide research and practice in this field has not been developed. The recent and increasingly widespread use of computer simulation in design has created an urgent need to develop a conceptual framework which can test prevailing assumptions and provide the basis for the development of an accepted theory in the field. This paper proposes such a framework, and identifies and relates the significant variables, which impact upon the application of visual simulation in an environmental design, task-orientated context.  2001 Elsevier Science Ltd. All rights reserved Keywords: environmental design, simulation, evaluation, computersupported design, framework

D

1 Groa¨k, S ‘Representation in in building’ RSA Journal Vol 4 No 4 (1998) pp 51–59 2 Sheppard, S R J ‘Landscape portrayals: their use, accuracy and validity in simulating proposed landscape changes’. Unpublished PhD dissertation, University of California, Berkeley (1982)

uring an architectural project life cycle, various types of visual simulation tools are used for communication, collaboration and decision-making. Simulations in this field have and, still rely to a great extent on drawings and diagrams, sketches, photographs and physical models. Recent advances in computer technology and simulation software have fostered the use of computer generated imagery to emulate real life scenes or objects, or to speculate about future events or projects. According to Groak (p 51)1 this new technology offers ‘new possibilities, extending the scope of good design and, in particular, of increasing the real involvement of clients and users in the design process.’ However, this field of study is not much closer to an established conceptual framework today than it was 20 years ago. As early as 1982, Sheppard2 www.elsevier.com/locate/destud 0142-694X/01 $ - see front matter Design Studies 22 (2001) 193–209 PII: S0142-694X(00)00027-2  2001 Elsevier Science Ltd All rights reserved Printed in Great Britain

193

drew attention to the absence of agreed standards for the production and evaluation of visual simulations. Five years later, he reiterated that there was still a lack of hard evidence to test existing theory and practice3. On their work into accuracy of computer visual simulations, Watzec and Ellsworth4 (p 23) observed that ‘Given the state of our knowledge on accuracy relationships and the emerging roles of ’computer imaging,‘ systematic research is needed to assess the defensibility of these visual simulations.’ Recently, Keul and Martens5 stressed that since 1973, simulation studies have had no real impact on what designers and planners do. This gap in our knowledge has persisted and consequently, it would appear that intuition, prior practice and trial-and-error approaches have guided the production of visual simulation material. There has been no systematic effort to bring together the results of research in this field.

3 Sheppard, S R J Visual simulations—a user’s guide for architects, engineers and planners Van Nostrand Reinhold, New York (1989) 4 Watzec, KA and Ellsworth, JC ‘Perceived accuracy of computer visual simulations’ Landscape Journal Vol 13 No 1 (1994) pp 21–36 5 Keul, A G and Martens, B ‘Simulation—how does it shape the message?’ Proceedings 2 [nd] EAEA-Conference held at Vienna (1995) pp 47–53 6 Jenkins, A M MIS design variables and decision-making performance: a simulation experiment Umi Research Press, Michigan (1983) p 1 7 McKechnie, G E ‘Simulation techniques in environmental psychology’ in D Stokols (ed) Psychological perspectives in environment and behavior Plenum Publishing Corp, New York (1976) 8 Appleyard, D ‘Understanding professional media: issues, theory, and a research agenda’ in I Altman and J F Wohlwill (eds) Human behavior and environment Vol 2 Plenum Press, New York (1977) pp 43– 88

194

The need for a theoretical foundation in any field is well understood. Theory is a critical and necessary component for the development of a paradigm to guide research within a field. Jenkins6 (p 1) pointed out that ‘Theory building and testing are the fundamental foci for generating research questions and testable hypotheses.’ Moreover, a framework allows research findings to support general trends or identify discrepancies that need to be resolved, to place the questions asked by researchers in context and provide a basis for improved practice. The aim of this paper is, therefore, to provide a basis for a conceptual framework to guide research towards the development of a theoretical foundation in the study of the evaluation of computer-generated visual simulations.

1

Review of simulation evaluation paradigms

McKechnie7 suggested that simulation techniques are crucial for environmental design decision-making process. Environmental simulation is ‘an attempt to represent environmental reality’ (Appleyard8, p 60). Therefore, the thrust of research in this field has attempted to clarify the issue of realism in environmental simulation. The key issue involved investigating the ability of simulations to predict real-life situations. This often involved presenting simulations to observers and comparing their responses with those obtained by viewing the real-life environment or a photorealistic representation (photograph, slide, film etc). Despite the proliferation of this research, only a limited number of studies attempted to develop a framework for the analysis and evaluation of simulation. Appleyard8, in a critique of current practice, advocated the following criteria for assessing simulation quality and ensuring response equivalence:

Design Studies Vol 22 No 2 March 2001

쐌 ‘Realistic’ and ‘accurate’ to reflect how the project will be experienced; 쐌 ‘Comprehensible’ and ‘evaluatable’ to enable people of all educational levels to be able to understand the content of the simulation and evaluate it for their own purposes; 쐌 ‘Engaging’ and ‘interesting’ to keep the viewers focused on the message conveyed by the simulation. However, these criteria were and have remained largely prescriptive and untested8. Later, Sheppard3 elaborated on this work, developing and refining these criteria. He proposed five criteria for improved comprehension, credibility and bias-free visual simulation, namely: 쐌 ‘Representativeness’ the degree to which simulations represents typical views of the project; 쐌 ‘Accuracy’ the degree of similarity in appearance between the ‘simulated’ and the ‘real’ scene; 쐌 ‘Visual clarity’ the degree to which detail, parts, and overall content of the simulation can be easily understood and recognized; 쐌 ‘Interest’ the degree to which simulations hold the interest of the audience; 쐌 ‘Legitimaty’ the extent to which the correctness of the simulation can be demonstrated and justified.

1.1

Comparison between frameworks

Both authors agreed that visual simulations should be interesting, comprehensible and accurate (Table 1). In addition to these criteria, Sheppard3 stressed that good visual simulations should be demonstrable and avoid misleading the public by presenting biased views. Table 1 Comparison between Appleyard’s and Sheppard’s criteria for evaluation of simulation quality

Interesting/engaging Comprehensible/visually clear Accurate Realistic Representative Evaluatable Legitimate

Appleyard’s criteria

Sheppard’s criteria

√ √ √ √

√ √ √

√

√ √

Towards a framework for evaluation of computer visual simulations in environmental design

195

Despite the similarities between the two frameworks, there are also fundamental differences. For instance, Sheppard did not consider realism as a central issue in visual simulation research, while Appleyard stressed that simulations should be subject to evaluation. In addition to these distinctions, there are also differences in terminology and guidance for future research. Consequently, it is important to examine each criterion in detail, to account for its research implications on simulation evaluation.

1.1.1

Interest and engagement in simulation evaluation

Research in the affective-motivational role of pictures in education has shown that illustrations have the ability to arouse interest and emotion, encourage learning and curiosity and maintain higher concentration9. However, little is known about the role of visual simulation, as a stimulator or inhibitor of interest in environmental design evaluation.

9 Peeck, J ‘The role of illustrations in processing and remembering illustrated text’ in D M Willows and H A Houghton (eds) The psychology of illustration Springer-Verlag, London (1987) pp 115–151 10 Sawczuk, B ‘The management of the design process’ in M P Nicholson (ed) Architectural management E & FN Spon, London (1992) pp 84–89 11 Wrona, S K ‘The profits of CAAD can be increased by an integrated participatory design approach’ in A Pipes (ed) Computer-aided architectural futures Butterworths, London (1986) pp 53–57 12 Novitski, BJ ‘Integrated software and new electronic models enhance communication among design and construction disciplines’ Architecture—The AIA Journal Vol 82 No 4 (1993) pp 97–99 13 Al-Kodmany, K ‘Using visualisation techniques for enhancing public participation in planning and design: process, evaluation and implementation’ Landscape and Urban Planning Vol 45 No 1 (1999) pp 37–45 14 Tsuyoshi, T S ‘Open design environment and collaborative design’ in T Maver and J Petric (eds) The virtual studio eCAADe, 12 [th] European Conference on Education in Computer Aided Design Glasgow (1994)

196

In contrast, a great deal of work was done to test the ability of visual simulation in encouraging participation in planning and design process. According to Sawczuk10 (p 88) ‘the design process revolves around client’s needs and therefore the client should be part of the team and obviously requires managing.’ It was reported that visual simulation tools have changed the way designers operate and above all, provided them with powerful means to communicate with each other and with their clients11. These new tools have improved the communication process between design and construction disciplines12. Others reported that visualisation has favoured public participation in the planning and design13. In recent years, a substantial effort has been dedicated to the development of software tools for design collaboration and shared decision-making14. This work clearly suggests that simulation tools have the potential to improve communication, and above all, favour participation in the design process. However, more research is needed to clarify the potential of simulation tools to arouse interest.

1.1.2

Comprehension in simulation evaluation

Comprehension, as an aid of participation in the decision-making process, is closely linked to the issue of realism8. This approach stressed that while professionals can grasp simple presentations, such as abstract diagrams and plans, the public finds it easier to understand highly realistic media, such as television, films and photographs. It also suggested that comprehensibility is related to familiarity with the medium. An unfamiliar or abstract medium ‘demands more effort than people are willing to give. They also see it as unrealistic’ (Appleyard8, p 61). Consequently, the need for multiple media, represented at different levels of detail to reflect the ‘multilev-

Design Studies Vol 22 No 2 March 2001

el’ comprehension, was advocated. Sheppard2 believed that comprehension of simulation is a prerequisite for lay-people’s participation in the decisionmaking process. He stressed that abstraction in visual simulation can affect experiential reality. These views were examined by recent empirical studies. For instance, Harrilchak15 investigated the effect of rendering techniques on the evaluation of computer-generated design simulations, by evaluating nine different projects rendered under four computer-assisted media styles. The findings revealed that while skilled participants appreciated traditional media, such as drawings, unskilled participants preferred photorealistic presentations. Similarly, it was reported that when lay-people were exposed to architects’ drawings ‘plans had little meaning as the people could not understand what was represented.’ However, when presented with a 3D form of representation, the ‘model made the plan understandable’ (Hardie16, p 60). It was also claimed that Computer Aided Design software has also improved the communication process between professionals17. However, these studies have not revealed how much comprehension was achieved thanks to visual simulation media, while carrying out specific design tasks.

15 Harrilchak, M A ‘The affect of rendering techniques on the evaluation of computer generated design simulations’. Unpublished MLA dissertation, Colorado State University of New York Colorado, New York (1993) 16 Hardie, G J ‘Community participation based on threedimensional simulation models’ Design Studies Vol 9 No 1 (1988) pp 56–61 17 Ward, S ‘Technology puts time on your side’ Building Design July 28 (1995) p 43 18 Kalay, Y E ‘Computational environment to support design collaboration’ Automation in Construction Vol 8 (1998) pp 37–42 19 Valkenburg, R C ‘Shared understanding as a condition for team design’ Automation in Construction Vol 7 (1998) pp 111– 121 20 Mahdjoubi, L and Wiltshire, T J ‘How much detail is needed in computer generated architectural displays’ in S Emmitt (ed) Detail design in architecture BRC, Northampton (1996) pp 88–98

Kalay18 (p 40) pointed out that while computer programmes ‘made communication easier and more efficient, they have not, in and of themselves, improved shared understanding, which is fundamental for making joint decisions and for negotiating tradeoffs among competitive worldviews.’ Research findings indicated that software has failed to improve collaboration, mainly because of their neglect of social and professional characteristics of the decision-makers19. Consequently, the simulation evaluators’ characteristics in terms of level of expertise and familiarity with visual simulation media are important considerations in this line of research. Moreover, the specific attributes of the message also affect how the information is perceived and understood.

1.1.3

Representativeness

Representativeness is the degree to which a simulation should depict important and typical views and conditions of the projects, which match the ones prevailing in reality. The representativeness of simulation has first been the subject of scrutiny by research who found that most visual simulation used in landscape evaluation practice were criticised for their apparent bias and poor quality3. Likewise, research in the UK20 examined a sample of artists’ impressions of buildings, commonly used in architectural design, and concluded that simulations were produced to portray buildings in the best possible way. A great deal of emphasis in simulation represen-

Towards a framework for evaluation of computer visual simulations in environmental design

197

tation has been on marketing or selling the idea, which often resulted in an idealised environment. ‘The one factor that has kept professionals in communication with the public has been the necessity of “selling” their proposals. The present world of experiential simulation is an idealized one where the sun always shines, vegetation grows in luxuriant profusion, the water is pure, the streets clean, the people well-dressed and happy; an all is new, nature remains undamaged, and only pleasant vistas offer themselves to view’ (Appleyard8, pp 45–46). The current stress on ‘selling’ rather than ‘faithfully portraying’ significant views of the project has affected the truthfulness of visual simulation material. Bok21 suggested that ‘lying’ has become a common characteristic of the modern world. She distinguishes between ‘clearly intended’ lies and ‘more marginal forms of duplicity’, which include evasion, euphemism and exaggeration. The latter are also other forms of lies aimed at disguising the truth or misinformation. Indeed, media such as photography, which were seen before as truthful, have been the subject of increasing suspicion in recent years. Indeed, ‘computers can be used to make grass look greener, water bluer and skies brighter; to add and remove flowers, trees, telephone lines, passing cars and pedestrians; and to make selected parts of an image larger or smaller. All of these and more can be done with greater precision and more convincingly than was typically the case with noncomputerized techniques of altering photographs.’ (Messaris22, p 153). Consequently, the credibility of visual simulation media has been diminishing23. This research suggests that prevailing practice in the production of visual simulation material, as a means of building or landscape representation, often resulted in an unfaithful representation of the environment they depict. It indicated that the emphasis on realism and using simulation for selling purposes have affected their degree of credibility.

1.1.4 21 Bok, S Lying: moral choice in public and private life Harvester Press, Hassocks (1978) 22 Messaris, P Visual persuasion: the role of images in advertising Sage Publications, London (1997) 23 Weeler, T and Gleason, T ‘Photography or photofiction: an ethical protocol for the digital age’ Visual Communication Quarterly Vol 2 No 2 (1995) pp 8–12

198

Realism and accuracy in environmental simulation

The issues of realism and accuracy have raised considerable interest in visual simulation research. However, in the literature in this field, there is often confusion between these concepts. To clarify this issue, Appleyard8 identified two types of responses to an environmental evaluation; apparent and actual realism. Apparent realism is related to the ability of visual simulations to emulate the significant features of real-world experience to convince the viewer that the simulation is realistic. However, actual realism is more concerned with whether simulations were perceived as equivalent to real-life situations. Therefore, apparent realism ‘does not necessarily

Design Studies Vol 22 No 2 March 2001

entail a precise match between the appearance of a picture and the appearance of reality’ (Messaris, p 3)22. A realistic simulation should convince a client that a project is well planned. Consequently, it should be perceived as realistic without being necessarily accurate8. Actual realism is more concerned with accuracy between real and simulated environments. Sheppard3 defined accuracy as when a simulation and its real-life referent are not significantly different in appearance. Nevertheless, he admitted that this concept has not always easy to describe or measure. Despite these attempts to distinguish between the two concepts, it remained difficult to assess whether the goal of studies in this field was to measure responses to apparent or actual realism. Overwhelmingly, research has been interested in assessing the ability of visual simulations to elicit responses equivalent to those obtained with site photographs, films or viewing the real-environment. The experimental simulations were often manipulated according to the level of detail/texture, scale or sometimes by distorting the colour-contrast to measure their effects on response equivalence.

24 Howard, B R et al. ‘A comparative analysis of affective responses to real and represented environments’ in W J Mitchell (ed) Environmental design: research and practice University of California Press, California (1972) 25 Watt, A 3D computer graphics Addison-Wesley, Wokingham (1993) 26 Kaplan, R and Kaplan, S Cognition and the environment New York, Praeger (1983) 27 Craik, K H et al. ‘Impression of a place: effects of media and familiarity among environmental professionals’. Research technical report, Berkeley, Institute of Assessment personality assessment and research, University of California (1980) 28 Appleyard, D and Fishman, L ‘High rise buildings versus San Francisco: measuring visual and symbolic impact’ in J D Conway (ed) Human response to tall buildings Dowden, Hutchinson and Ross, Stroudburg PA (1977) pp 81– 100

Early work sought to increase the realism of the representations until they obtain the same responses as the real environment24. This approach was recently reiterated in the context of computer graphics research. Watt (p ix)25 stated that ‘the current thrust of research in this area is towards greater and greater realism. Reflection models are continually being refined. The oftstated goal of such efforts is to produce images that are indistinguishable from images obtained from, say a camera television’. The fundamental assumption of the realism approach is that effective and useful application of simulation requires reproducing the environment it depicts. This approach has often led to an emphasis on detail, on fidelity and exactness in the simulation26. Work in this field was often applied to assess users’ responses to the simulated environment in comparison to the real one. For instance, Craik et al.27 showed a representation at a high level of detail to a group of subjects and claimed, as a result, that 10% of the subjects found it difficult to believe that they were not watching a film of the real world. Appleyard and Fishman28 research claimed that the more accurate the simulation in depicting the environment it portrays, the observers’ responses would be similar to the ones obtained by viewing the visual reality. Therefore, higher realism in terms of texture, detail, colour are expected to correlate with higher response equivalence. A more exhaustive study conducted by Sheppard2 revealed that environment professionals rated most highly models

Towards a framework for evaluation of computer visual simulations in environmental design

199

that contained either a high level of detail or were photograph-based simulations. This study also suggested that the abstraction, omission of colour and important views, detail and context resulted in low credibility ratings. More recently, Watzek and Ellsworth4 designed a study to measure the scale of accuracy for computer simulations of environmental projects. A series of degraded computer simulation was compared to a known accurate base image for each project type. It revealed that the reduction of accuracy of the visual simulation material resulted in lower perception of likeness between the simulation and the base image. Yet other experimental research has produced mixed results. For instance, using black and white material portraying a real-life scene has produced good response equivalence for Wood29 while Sims30 subjects’ rated it negatively. Similarly, Sims30 demonstrated that by showing his subjects a detailed pencil drawing, the response equivalence could be high, while Sheppard2 found that his subjects rated low these type of drawings. Sheppard2 (p 208) concluded as a result of his research that ‘It appears that the relationship between accuracy and response equivalence is a complex one and other factors than accuracy alone (as appraised) are involved.’ It seems that the mixed results of this line of research may be related to how the respondents’ perceived the purpose of the simulation. They may have rated some abstract simulations highly, as they believed they contained enough visual properties to achieve response equivalence, in the context of the objectives of the simulation evaluation.

29

Wood, W ‘An analysis of simulation media’. Unpublished MS. thesis, Vancouver, School of Architecture, University of British Columbia (1972) 30 Simms, W R ‘Iconic simulations: an evaluation of their effectiveness as techniques for simulation environmental experience along cognitive, affective and behavioral dimensions’. Unpublished PhD dissertation MIT, Cambridge (1974) 31 Gibson, J J The perception of the visual world Houghton Mufflin Co, Boston (1950) 32 Gibson, J J ‘The ecological approach to the visual perception of pictures’ Leonardo Vol 111 (1978) pp 227–235 33 Marr, A Vision—a computational investigation into the human representation and processing of visual information W H Freeman and Company, San Francisco (1980) p 31

200

1.1.5

Visual realism and level of detail

An emerging body of research has questioned the fundamental concepts of the visual realism approach. For instance Gibson31 who for several decades advocated that whatever information comes to the eye from a real life scene will also come from a properly displayed picture on the screen, later entirely changed his views about the issue. He reported that the approach consisting of fooling people that they are assessing the real environment should be avoided. He reached the conclusion that ‘no painter and no photographer should ever strive to give the viewer the feeling that he is looking at a real place, objects, person, or event. There is no need to do so. In any case the effort is bound to be failure.’ (Gibson32, p 228). He advocated that only the properties that are relevant or significant to the purpose of the simulation are worth considering. Similarly, others indicated that realistic pictures might contain too many irrelevant details, which are not necessarily desirable or useful33. This suggests that a simulation must contain the appropriate information to support the viewers in the tasks in which they are engaged.

Design Studies Vol 22 No 2 March 2001

Modern researchers stressed that ‘the purpose of a simulation determines the degree of approximation to reality. In principle, a simulation is not a replica of a real-life situation but a representation of it’ (Lawrence34, p 302). Others pointed out that despite the major effort and resources devoted to validating simulations in order to increase their credibility, this has not succeeded, mainly because they failed to live up to expectations and the task in hand35. These criticisms suggest that simulations do not need to be a replica of real-life referent to achieve their purposes. Additionally, this research argued that a simulation should be determined by the tasks in hand.

34 Lawrence, R J ‘Architectural design tools: simulation, communication and negotiation’ Design Studies Vol 14 No 3 (1993) pp 299–313 35 Bartholemew, D ‘PiT project on validation’ Building Environmental Performance Analysis Club (BEPAC) Vol 14 No 1 (1996) pp 20 36 Richens, P and Schofield, S ‘Interactive computer rendering’ Architectural Review Quarterly Autumn (1995) 1(1) 37 Mahdjoubi, L and Wiltshire, T J ‘Computer mediated architect–client collaboration’ Proceedings INCIT 96, The Institute of Engineers, Sydney (1996) pp 13–18 38 Strothotte, T ‘How to render frames and influence people’ Eurographics Vol 13 No 3 (1994) pp C455–C486 39 Shumann, J et al. ‘Assessing the effect of non-photorealistic images’ Proceedings CAD Chi 96, April 13–18, (1996) pp 35–41

The strongest criticism of the visual realism approach is related to their lack of flexibility and poor understanding of how designers work. Photorealism was criticised for being too complete and non-negotiable36. Others blamed the realism approach for being insensitive to the need for flexibility and adaptability at the early stages of the design process37. They advocated that ‘Architects often prefer to show their clients sketches of their designs, rather than photorealistic images....showing the client a sketch and thereby transputing to him or her the message “this is the first draft” is more powerful and convincing than showing the client a photorealistic image and adding an appropriate verbal comment.’ (Strothotte et al.38, p C-466). As early as 1996, Schumann et al.39 reported that one of the main reasons given by more than 75% of architects interviewed for not using a CAD system for visualisation and decision making is the quality of the presentations that commercially available software is able to produce. They feared that the form of output could deter their clients. As a result of the growing discontent with photorealism, a new trend has emerged in recent years labeled the ‘Non-photorealistic’ approach. This style of representation has been created to ‘allow for a more relevant, and economical, alternative to photorealism, sharing instead some of the qualities of painting, drawing and print-making’(Rickens and Schofield36, p 1). The effect of non-photorealistic images on users was tested by an empirical study39 carried out with 54 architects, who were asked to compare the output of a non-photorealistic image with a 3D wireframe CAD drawing. This research found that the majority of designers preferred to use nonphotorealistic images to show their design output to their clients at the early stages of the design process. This study has confirmed that a simple style of representation, notably images with a low level of detail, can be a valid architectural design tool. However, it provided no evidence to show whether such images can be used for communication and collaboration purposes between architects and their clients.

Towards a framework for evaluation of computer visual simulations in environmental design

201

Most simulation research has used professionals to evaluate the quality and response equivalence. Non-photorealistic tools are primarily aimed at architects, but little is yet known of how lay-people will react to this technique of rendering. The exception is an early study26, which found that even non-trained viewers were capable of making appropriate judgments of architectural models, even when they lacked detail. This study revealed that there were severe discrepancies between viewers according to their level of training (non design-trained and design-trained), when they viewed high level of detail models. It seems that design-trained viewers were more sensitive to the architectural details contained in the model26. This work indicated that the task (design objective) has a significant impact on how a simulation is produced and displayed. It also showed that abstract simulations are valid representations of real-life for appropriate design objectives.

2

Conceptual framework for simulation evaluation

The aim is to propose a theoretical framework, which makes sense of past research and gives shape and structure to future developments in this field. However, the possible avenues of research are extensive and open-ended. Thus, the research objectives are a major determinant in the selection of the significant variables to consider. The explicit clarification of the research objectives is paramount to constructing a useful framework that can effectively guide and stimulate research and application. The main objective is to examine the underlying factors that affect the assessment of computer generated visual simulations during the architectural design process. Consequently, it is essential to identify the key variables that need to be taken into account in the design of this framework.

40 Lucey, T ‘Information, data and communication’ in Management information systems 7th edition, D P Publications, London (1995) pp 12–13

202

To ensure that information has value in a decision-making context requires considering the decision-maker and the decision being dealt with40. This involves full consideration of the characteristics that make information relevant, the way information is communicated and how decision-makers perceive and understand information. The paradigms reviewed in this paper stressed the exploration of two key elements; the visual simulation and the viewers. This includes the content and style of representation of the visual simulation, including the way it is presented to the viewers. The appraiser’s characteristics include the level of expertise, familiarity and demographic variables. However, the contention of this paper that a fundamental dimension often neglected in this field of work, pertains to the purpose of the simulation.

Design Studies Vol 22 No 2 March 2001

Indeed, it is suggested that the specific characteristics of the tasks carried out during the design process determine how simulations are used and in turn determine how they are produced and presented. The proposition is that the task determines the level and style representation required to support an effective communication process. Consequently, three core elements are included in the framework, as illustrated in Figure 1. They consist of the following: 쐌 A decision-maker, whose role is to evaluate the content of the visual simulation to reach an appropriate design decision; 쐌 Visual representation or visual simulation, which is a computer-generated environmental design representation, displaying views of the project; 쐌 A task, which refers to a specific design activity, carried out during the design process. The designed framework provides a conceptual setting for conducting research in this field and illustrates the specificity of the relationships between these components. It is suggested that the interaction between the three components; i.e. visual representation, design tasks, and decisionmaker account for the quality of the resulting design decision. Several variables are associated with each component. Research in this field indicated that a visual simulation could be studied in terms of its content, style of representation, and presentation methods. Likewise, a decision-maker’s characteristics were examined in terms of level of expertise and demographic variables. Each of these variables and some of their interactions have been the subject of extensive investigations. However,

Figure 1 Framework for the study of simulation evaluation

in

environmental

design

Towards a framework for evaluation of computer visual simulations in environmental design

203

the main contribution of this paper lies in the study of the roles and effects of the design task on the appraisal of visual simulation quality. This leads to a detailed examination of the potential relationships between these core elements and their associated variables. Consequently, a conceptual model was constructed to provide a setting for the investigation of the interactions between design tasks, visual simulation and decision-makers’ characteristics.

41 Smith, G F and Browne, G J ‘Conceptual foundations of design problem solving’ IEEE Transactions on Systems Man and Cybernetics Vol 23 No 5 (1996) pp 1209–1219 42 Vries, M D and Wagter, H. ‘A CAAD model for use in early design phases’ in M McMullough,, W J Mitchell and P Purcell (eds) The electronic design studio: Architectural knowledge and media in the computer era The MIT Press, Cambridge Massachusetts (1990) pp 215–228

2.1 Conceptual model for analysis of computersimulation and design decision-making The conceptual framework illustrated in Figure 2 reveals the variables that are considered relevant. The following section describes the key elements and their associated variables, including some of the possible interactions.

2.1.1

Task oriented design

The design tasks carried out during the design process vary according to the stage of development of the project. Traditionally these tasks were analysed in terms of problem solving, processes and methods41. More recently research42 suggested that environmental design could be examined

Figure 2 Conceptual model for the study of simulation evaluation in environmental design

204

Design Studies Vol 22 No 2 March 2001

instead in the context of decision-making. This approach seems to be more relevant to the context of evaluation of environmental design simulations. Design decisions, according to Prasad43 (p 313) ‘are a complex array of diverse and often contradictory cognitive activities’. Throughout an architectural project cycle, numerous decisions are made in relation to cost, quality of design, time and the like. These decisions evolve as the project progresses. It has long been accepted that decisions made at the early stages of the design process are of paramount importance and can strongly affect later stages. Research indicates that the decision-making process is closely related to the level of detail. Information is more effective if the right data is presented at the right level of detail44. Visual simulation evaluation changes over time as new information become available. Vries and Wagter42 argued that decisions vary according to the stage in the design process, which in turn affects the level of information required. In the early design stages, the decisions are crude, as there is little concern for detail. However, as the project progresses, the decisions become more refined as the focus is on very detailed aspects of the design. Clearly, the design, production and presentation of visual simulation material should reflect these incremental changes. Consequently, low level of detail simulations may be adequate at the early stages of the design process, as there is relatively little architectural information. The stress at these early stages is on flexibility and adaptability, thus allowing interaction between those involved in the process.

43 Prasad, B Concurrent engineering fundamentals Prentice Hall, Upper Saddle River NJ (1997) 44 Feary, H ‘Fast fact take away’ The Grocer January 30 (1999) pp 42–43 45 Leslie, H D and McKay, D J ‘Managing information to support project decision-making in the building and construction industry’ CSIRO Division of Building, Construction and Engineering and the National Committee on Rationalised Building, Sydney (1995) 46 Nicholson, R ‘The need for environmental modelling at the early design stage’ Building Environmental Performance Analysis Club (BEPAC) Vol 13 No 1 (1995) pp 21

As the design progresses, more detail may be needed and this is reflected in the amount of information needed to be included in the simulation. Information about size, space organisation, building materials, costs, colours, and the like, becomes available. Therefore, the simulation techniques need to be relatively less abstract and convincing to both the designer and the client teams. This leads to a higher demand for the level of detail in the visual representations. Some authors45,46 called for computer decisionsupport to be an integral with the decision-making process, which involves applying these supporting tools incrementally over the whole design process. Moreover, some studies indicated that there is a relationship between the content of the simulation and the degree of involvement in the design process. Pleading for the virtues of low-level representations, Kaplan and Kaplan26 (p 202) pointed out that ‘A simplified model of the environment

Towards a framework for evaluation of computer visual simulations in environmental design

205

is more likely to parallel people’s cognitive structure. Hence the very simplicity of the model may encourage its use. A simplified model also encourages generality; details make things particular, thus narrowing their range of appropriateness. Finally, simplification reduces the total load to one’s processing’. Stea’s47 observation supports this claim. He noticed that high level of detail models discouraged participants from suggesting changes to the proposal. He postulated that the ‘neat finish’ appearance of models in fact discourage people from proposing modifications to the project. However, none of these studies provided empirical evidence to these claims. To examine design tasks in the context of environmental design, the psychology of the human response to illustrations provided a framework, which involves analysing the roles and effects of visuals along several basic dimensions. Peeck9 classified the role of illustrations into two broad clusters; an affective-motivational one, which is related to enjoyment, attention and motivation, and a cognitive one, which deals with comprehension and retention. This approach was selected to examine the role and effects of visual simulation evaluation. Three core dimensions of simulation evaluation are suggested as being particularly relevant to simulation evaluation and decision-making; incentive-motivational, affective and cognitive functions. The affective function measures the aesthetic judgements and effects of the style of representation on decision-makers’ attitudes and preferences. While, the cognitive function evaluates the appraisal of adequacy of computer visual simulation along cognitive dimensions such as spatial relationship, size, location, scale, and the like. Finally, the motivational function assesses the role of computer visualisation as a facilitator or an inhibitor of participation in the design process.

47 Stea, D ‘Participatory planning and design in intercultural and international practice’ in D Canter (ed) New directions in environmental participation Avebury, Aldershot (1988) pp 50–67

206

Each function is associated with a specific design activity, carried out during the environmental design process. Affective functions are evaluated in terms of rating the aesthetic aspects of a simulation representing interior and exterior views of buildings. Cognitive functions are assessed along cognitive tasks such as critical space dimensions, space relationships, location of integration of the building in its setting and the like. Motivational functions measure the effect of style of representation and simulation content on the degree of involvement in the design process.

2.1.2

Decision-maker’s characteristics

The decision-maker’s demographic characteristics and professional bias are identified to be significant in many studies. The demographic variable

Design Studies Vol 22 No 2 March 2001

includes age and gender. As indicated by research findings, these variables are the most influential on the decision-making process. The other major variable is related to the skill level and occupation in dealing with the message conveyed by the visual simulation. As highlighted by the literature review, there are significant differences between professionals and the public. In this context, it will be expected that those who have an expert knowledge of computer simulation and lay-people will judge the message differently. Therefore, the key variables identified include occupation and the level of expertise in the use of visual simulation. Several studies have reported a difference in the evaluation of decisiontasks between experts and lay-people. ‘Differences between lay and expert appraisals can be anticipated because of variations in the decision context, experience, perceived role, and ability to cope with a highly complex and uncertain situation.’ (Barker48, p 15). Kaplan and Kaplan49 argued that some of the causes of variation in preference between people are based on experiences, which include professional training in design related fields.

48 Barker, M L ‘Planning for environmental indices: Observers appraisals of air quality’ in K H Craig and H E Zube (eds) Perceived environmental quality—research and applications Plenum Press, New York (1974) pp 47–58 49 Kaplan, R and Kaplan, S The experience of nature Cambridge University Press, Cambridge (1988) 50 Vischer, J and Marcus C ‘Design awards: who cares?’ in P Bart, A Chen and G Francescato (eds) Knowledge design, Proceedings of the thirteenth environmental design research association conference EDRA, Maryland (1982) pp 210–223 51 Scott, B ‘Aesthetic evaluation: a comparison of architects and non-architects’ BA architectural report, School of Architecture, Victoria University (1987) 52 Whitfield, A and Wiltshire, J ‘Design training and aesthetic evaluation: an intergroup comparison’ Journal of Environmental Psychology Vol 2 (1982) pp 109–117 53 Nasar, J L The evaluative image of the city Sage Publications, Thousand Oaks CA (1998)

It is has been established that designers and the public have different norms, values, attitudes and preferences26. As a result, they often assess the quality of buildings through different criteria. For instance, Vischer and Marcus50 carried out a study designed to compare, on the one hand, the values of designers and jurors with those of the building occupants. This research found that designers and jurors agreed amongst themselves in terms of important criteria and their ranking. Similarly the building occupants generated, amongst themselves, a list of important factors and their ranking, which however, was in total conflict with the ones decided by the designers-jurors group. While the most important factor for the designersjurors group was the exterior appearance of buildings, the occupants put this factor at the bottom of their list, as their prime concern was maintainability. Scott51 compared the aesthetic evaluation of architects and non-architects by selecting his subjects from architectural students as they progressed through their course. This study found a mismatch in terms of preferences between first year students, whose values compare to non-architects with those in their last years of study, as their preferences tend to be closely related to the values of practicing architects. Whitfied and Wiltshire52 reported similar findings, by demonstrating that design training had a significant effect on aesthetic evaluation. Nasar53 (p 6) concluded that architects and the public have different building preferences. In short ‘What architects like, the public dislikes, and what the public likes, architects dislike. They see different meanings in the same building’.

Towards a framework for evaluation of computer visual simulations in environmental design

207

These variations in attitudes and values exist even between professionals. For instance, Zube54 highlighted that the setting of air quality standards in the United States has raised considerable disagreement amongst professionals. Research has indicated that other demographic variables also have a significant impact on decision-making. The relationship between age and decision-making is also an important consideration. Results from research have found that age55 and gender56 affect the outcome of a decision-making process. This indicates that, an individual’s characteristics, such as demographic (age, gender), personal characteristics (level of education and experience) are important considerations in visual simulation research. These conclusions show that the viewers’ characteristics may affect the evaluation of the simulation material. It might be expected that differences in the level of expertise, professional education, gender and age might alter how a simulation content is perceived. In addition to the characteristics of the evaluators of the visual simulation, the impact of the tasks/activities on the design decision-making is fundamental to this research.

2.1.3

Characterisitcs of the computer visual simulation

This has received a great deal of attention. Three principle aspects of visual simulation have been studied. The first one pertains to the content of the simulation, which included the level of detail and accuracy. The second is related to the style of representation, where the main issues included sketch versus photorealistic visual and black and white versus colour simulations. The style of representation is related to qualitative aspects of the visual simulation in terms of artistic representation (sketch or cartoon) versus highly accurate display (‘photorealistic’ image). Finally, the methods of presentation correspond to the degree to which simulations represents typical views of the project, as experienced for normal usages.

54 Zube, E H Environmental evaluation; perception and public policy Cambridge UP, Cambridge (1985) 55 Tremblay, D ‘Influence of agism on decision-making—the perception of upper management’ Canadian Journal on Aging Vol 14 No 3 (1995) pp 464–479 56 Ganzel, AC ‘Adolescent decision making: the influence of mood, age, and gender on the consideration of information’ Journal of Adolescent Research Vol 14 No 3 (1999) pp 289–318

208

As argued earlier, the production and presentation of visual simulation material is dependent upon the design tasks and the viewers’ characteristics. The level of realism and style of representation should reflect the design task at hand37. These relationships may have a strong impact on the viewers’ perception of visual presentation in terms of confidence, accuracy, comprehension and holding their interest. Consequently, the level of detail and style of presentation are prime considerations in the appraisal of visual simulation material.

2.1.4

Quality of design decisions

The quality of a design decision has been tested in several ways. Time and cost are common measures for decision-making quality6. However, in

Design Studies Vol 22 No 2 March 2001

the context of environmental design evaluation, a key issue that dominated research in the field, pertains to the degree of credibility or confidence in simulation material. A great deal of work was carried out to measure the validity of the effectiveness of visuals to replicate the real environment. These studies reported that the perceived accuracy of visual simulations, the amount and quality of information and representativeness of views have important implications on the perception of credibility. However, Appleyard7 was critical of this approach, which attempts to fool people that in fact they are looking at the real environment. Kaplan and Kaplan (p 206)26 pointed out that ‘The issue is whether people respond the same way to the model as they would to reality with respect to the purpose. The model is intended as an aid to thought, not full fledged substitute to reality.’ In the context of environmental design there is a need to assess the degree of confidence or credibility in relation to the purpose of the visual simulation. This entails appraising people’s confidence in the presentation to the design tasks in hand. The simulation must be appropriate to its purpose and should not distort or bias the design decision purely as a function of the properties of the simulation itself. The important issue is whether there is sufficient and accurate information to support the decision-making process.

3

Concluding remarks

Despite the importance of the subject no recent research agenda has been devised to propose a conceptual framework to guide research in this field. In 1987 Zube57 (p 75) reiterated that ‘The question of how much detail is required in representation is an important and answered question’. At present our knowledge about this fundamental question is still very limited. Yet, the recent proliferation of computer visualisation tools in environmental design field and the increasing reliance of planners and designers on them as design, communication and evaluation tools, combined with conflicting approaches to software development in this field, have created more urgency for further work in this field. 57 Zube, E H, Simcox, E and Law, C S ‘Perceptual landscape simulations: history and prospects’ Landscape Journal Vol 6 No 1 (1987) pp 62–80

This paper has demonstrated that more knowledge is needed, before researchers can provide useful guidance on the creation and use of visual simulation in the context of environmental design.

Towards a framework for evaluation of computer visual simulations in environmental design

209