- Email: [email protected]
Combined effects of sound and illuminance on indoor environmental perception
Applied Acoustics 141 (2018) 136–143
Contents lists available at ScienceDirect
Applied Acoustics journal homepage: www.elsevier.com/locate/apacoust
Combined effects of sound and illuminance on indoor environmental perception
T
⁎
Wonyoung Yang , Hyeun Jun Moon Department of Architectural Engineering, Dankook University, Yongin 16890, South Korea
A R T I C LE I N FO
A B S T R A C T
Keywords: Acoustic perception Multisensory interaction Visual perception Gender difference
Cross-modal effects of sound and illuminance were investigated in an indoor environmental chamber with 60 university students (30 men and 30 women) aged 18–26 years. A within-subject factorial design was employed with four independent variables: noise level (45, 55, 65, and 75 dBA), noise type (Music, Water, Babble, and Fan), illuminance level (150, 500, and 1000 lx), and sex (female or male). The test conditions represented daily indoor environments, which was moderately noisy and did not present severe lighting conditions. Acoustic (loudness, annoyance, pleasantness, and naturalness) and visual (brightness and relaxation) semantic attributes were assessed using an 11-point numeric scale. The illuminance level of the ambient lighting system did not affect acoustic perception. Brightness was not altered by sound, but relaxation was affected by sound. Crossmodal interactions were asymmetric between sound and illuminance in indoor environmental settings. Women were more sensitive to the perception of both acoustic and illuminance stimuli than men, at high level of stimulation.
1. Introduction Cross-modal interaction in hearing and vision has been investigated as a part of multisensory interaction. Multisensory interaction research has exploded over the last few decades in cognitive neuroscience and neurophysiology [1]. Spatial and temporal factors influence multisensory integration [2], and the effects of both semantic and synesthetic congruency on multisensory information processing have been studied [3,4]. Effects of sound on visual perception have been studied with relatively simple audiovisual stimuli in early studies. Broadbent [5] reported that performance on light-watching became relatively less efficient with continuous exposure to noise using two visual vigilance tasks. Marks [6] found that most subjects matched pure tones to visual brightness of gray surfaces. Shams et al. [7] showed that visual perception (number of flashes) could be qualitatively altered by sound (number of beeps). The perceived duration [8], intensity [9], and contrast detection [10,11] of a visual stimulus has been shown to be influenced by accompanying sound signals. Scheier et al. [12] have shown that visual temporal resolution can either be improved or degraded by sound, depending on the temporal relationship of these variables. In contrast, Odgaard et al. [13] and Marks et al. [14] reported that brightness did not seem to be enhanced by sound. Effects of vision on acoustic perception have been also studied with ⁎
relatively simple audiovisual stimuli. It is well known that the spatial location of a visual stimulus can modify the apparent location of a simultaneously presented sound [15] (for example, in visual capture or ventriloquism). Saldana and Rosenblum [16] suggested that only discontinuous visual stimuli have a strong effect on the perception of the sound. Odgaard et al. [17] reported that white noise presented with light tended to be rated as louder than noise presented alone. Marks et al. [14] observed that the effect of visual stimulation on auditory pitch and loudness discrimination in an unspeeded discrimination test. If the time constraint leads to a reduction of the number of items attempted by all participants to < 90%, the test is considered as speeded [18]. Of note, Odgaard et al. [13] and Marks et al. [14] reported that visual perception was unaffected by sound. The asymmetry of crossmodal interactions that occurs in unspeeded discrimination contrasts with the bidirectional interactions observed in speeded identification. Shared sensory mechanisms underlie the cross-modal interactions, but these mechanisms change as a function of the task. A caveat of these experiments in cognitive neuroscience is that they do not involve commonly occurring, realistic audiovisual situations. For instance, the auditory stimuli that were used in these studies were white noise or pure tones, and the visual stimuli were flashes [19]. Currently, research on the cross-modal interactions between hearing and vision is very popular in the context of multimedia applications [20], and has also been extended to environmental approaches. More
Corresponding author. E-mail address: [email protected] (W. Yang).
https://doi.org/10.1016/j.apacoust.2018.07.008 Received 2 April 2018; Received in revised form 2 July 2018; Accepted 6 July 2018 0003-682X/ © 2018 Elsevier Ltd. All rights reserved.
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
Fig. 1. Layout of the lighting fixtures and test laboratory.
Ma and Nie [28] reported that the influence of brightness on noise annoyance caused by road traffic noise was obvious in the indoor environment. The effect of color on the evaluation of noise annoyance was not significant, although color (red, yellow, cyan, and green) and brightness showed an interaction effect on the evaluation of noise annoyance. Low brightness significantly increased subjective measures of noise annoyance. However, they did not quantify brightness in their experimental setting. Liebl et al. [29] suggested that the effects on cognitive performance and well-being must be considered separately since these effects are rarely consistent. The interaction effects of background speech (24 and 94% word intelligibility) and lighting conditions (static and dynamic lighting at 55–60 lx) on cognitive performance have not been reported. Nevertheless, these variables were found to have an interaction effect on perceived performance during task processing. The participants perceived that performance was better if background speech of low intelligibility was combined with static lighting. The objective of this study is to explore the cross-modal effects of indoor sounds and illumination levels on the sensation and perception of each sensory modality in indoor settings. In short, the present study investigates how acoustic factors influence visual judgments and how visual factors influence auditory judgments which thus far have not been examined in more realistic indoor settings. To date, no study has assessed the combined effects of sound and illuminance on human perception of both sensory modalities simultaneously. This study is based on the hypothesis that cross-modal interaction is possible between sound and light. We used a 4 × 4 × 3 × 2 (sound levels × sound types × illuminance levels × gender) within-subjects design. Gender was included because gender differences exist in individual sound and visual perception [30,31].
realistic audiovisual stimuli have been involved in these more recent experiments, including scenes and environmental sounds [21–23]. Visual stimuli have been subdivided to analyze cross-modal interactions in environmental approaches. In this study, audiovisual interactions in indoor environmental approaches are reviewed. Studies on audiovisual interactions in indoor environmental settings have been developed with subdivided visual stimuli and noise using various levels of sensation, perception, cognitive performance, and productivity. However, the results seem to be inconsistent. Knez and Hygge [24] found no interactions between noise and color temperature of light, but separate main effects were found for irrelevant speech (38 and 66 dBA) and light (3000 and 4000 K at 500 lx) on cognitive performance. The participants’ long-term memory recall was better in silence than in the irrelevant speech condition, and in the warm-white lighting (3000 K) than in the cool-white lighting (4000 K). Veitch [25] attempted to integrate light and noise to examine their interactional effects on reading comprehension. The illumination levels were 200, 400, and 600 lx with standard cool-white fluorescent lights and the office noise levels were approximately 50 and 70 dBA. No main effects of noise, illuminance, gender, or any effects of interactions of these variables were observed on cognitive performance. One possible explanation for this was that the condition of allowing the participants to leave at any time they chose may have neutralized the effects of the uncontrollable, unpredictable noise on cognitive performance [25]. Hygge and Knez [26] found interactions between broadband low-frequency noise (38 and 58 dBA) and light (300 and 1500 lx with a color temperature of 3000 K) on cognitive performance as well as perception. The main findings for the effects of noise and light were better free recall scores and higher activation scores in the low noise condition at 1500 lx than 300 lx. Akbari et al. [27] examined the relationship between lighting (lx at the height of 30 in. from the surface of the work station) and noise level (Leq 8 h of daily and 40 h of weekly exposure) on human productivity in the automotive assembly industry. Noise levels in the workplace have negative effects on workers’ productivity which leads to a decrease in the organization’s productivity and a corresponding decrease in the quality and quantity of services and products. Lighting did not have an effect on human productivity and changes in lighting were not related to changes in human productivity.
2. Methods 2.1. Experimental conditions The experiment was run in a test laboratory (4.0 m × 5.0 m × 2.4 m), furnished as a small classroom. The test laboratory in this study was built for indoor environmental research. The 137
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
were carried out with an audiometer (GSI 18) in a soundproof booth and all participants had normal hearing (0–25 dB HL). All participants had self-reported normal or corrected-to-normal vision and none of them had color deficiency. They provided informed consent and received a description of the experiment with brief instructions prior to the test regarding the operation of the test device.
2.3. Experimental design and procedure A within-subject factorial design was employed with four independent variables: noise level (45, 55, 65, and 75 dBA), noise type (Music, Water, Babble, and Fan), illuminance level (150, 500, and 1000 lx), and sex (female or male). In some of the analyses, ambient noise was supplemented as a reference case. Three illuminance levels were tested in three experimental sessions. Fig. 3 shows experimental procedure for each session. In each illuminance session, participants listened to and evaluated a total of 17 sound stimuli (four types of sound sources and four levels of each) including a reference case at the ambient noise level of 35 dBA in the laboratory in a random order. Each sound stimulus was presented for 25 s and a 50-s response time was allowed. A 30-min adaptation period was implemented at the beginning of each session for thermal as well as light and dark adaptation. Each session took about 55 min. The subjective acoustic and visual characteristics were assessed using an 11-point numeric scale recommended by ISO 15666 [32], as shown in Fig. 4. A total of six semantic attributes of each sound stimulus were evaluated at the fixed illuminance level. For acoustic attributes, loudness, annoyance, pleasantness, and naturalness were examined. For visual attributes, brightness and relaxation were assessed. Loudness and annoyance have frequently been used in noise research [33,34]. Pleasantness and naturalness have been widely used in positive sound evaluation [35,36]. Brightness and relaxation have been used for assessing subjective impression in lighting perception studies [37,38]. Loudness for sound level, and brightness for illuminance were sensation-level semantic attributes in each modality. Annoyance, pleasantness, naturalness of sounds, and relaxation of vision were more likely perceptive attributes. In each session, a group of four participants (the maximum size of each group was four) was seated in the test laboratory. The participants were provided with definitions of the semantic attributes but were not informed of the actual experimental conditions in order to prevent the development of experimental bias. A quick survey regarding the music stimulus was performed. Prior to the test, the participants were asked for their likes and dislikes of the music stimulus, Vivaldi’s The Four Seasons or the 1st movement “Spring” in order to understand the participants’ reaction to the music stimulus. The subjective assessment was conducted using a tablet pad and the data were automatically saved on a server. A factorial analysis of variance (ANOVA) was used to test six subjective environmental attributes in the experiment to fit four independent variables, i.e., noise level, noise type, illuminance, and gender. ANOVA is a powerful statistical test and was used in this case, although normality cannot be guaranteed for subjective ratings [39]. All the statistical analyses were carried out using the Minitab® 17 with P = 0.05 as the level of significance.
Fig. 2. Frequency spectra of the sound sources.
room temperature (25 °C) and humidity (45%) were controlled by variable refrigerant flow systems, humidifiers, and ventilation systems located inside and outside the test laboratory. The ventilation system was run throughout the experiment. Six ceiling-mounted fluorescent lighting fixtures with two tubes (Osram FHF32SSEX-D) were installed with a dimming control system. The color temperature of the lamp was 6500 K according to the specification sheets provided by the manufacturer. A loudspeaker system (Turbosound Milan M10) for sound presentation was located in the center at the front of the room as shown in Fig. 1. The reverberation time of the test laboratory was measured as 0.33 s at 500 Hz for one-third octave bands. The ambient noise level in the laboratory was 35 dBA (Solo dB01) when the thermal systems were running (see Fig. 2). The illuminance levels on the participants’ desk surface were 150, 500, and 1000 lx. Four different sound sources (Music, Water, Babble, and Fan) were played through the loudspeaker at four levels (45, 55, 65, and 75 dBA). Vivaldi’s The Four Seasons or the 1st movement “Spring” performed by Amsterdam Sinfonietta in Concertgebouw in 2014 was used as a music sound source. The water sounds from an indoor water fountain were recorded in the test laboratory. The babble noise was recorded in a noisy cafeteria. The fan noise was also recorded in the test laboratory with ventilation fans running. The levels of the sound sources were adjusted by an audio controller. The illuminance levels along the desk surface during the experiment were 158.4, 516.5, and 997.4 lx (Konica Minolta T-10A), on average. The illuminance level differences across the participants’ positions were measured at ± 20.2 lx. The sound levels were measured at each of the four participants positions, and their differences were at ± 0.3 dBA. 2.2. Participants A total of 60 university students (30 men and 30 women) aged 18–26 years participated in the three experimental sessions and received financial support for their participation. Hearing screening tests
Fig. 3. Experimental procedure for each session.
138
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
Fig. 4. Semantic judgment test interface.
3. Results
The second order interaction between sound level and type also influenced all the acoustic attributes. Fig. 5(a) and (b) show the acoustic attributes according to sound level. Loudness and annoyance increased as sound level increased. At lower sound levels, loudness and annoyance were significantly lower in women than men, but at higher sound levels, loudness and annoyance were significantly higher in women than men. Pleasantness showed an inverted U-shape according to sound level. The pleasantness peak appeared at a noise level of 55 dBA. Naturalness decreased in a statistically significant manner as sound level increased. Pleasantness and naturalness tended to be higher in women than men up to 65 dBA, but lower in women than men at a noise level of 75 dBA. However, these
The overall data for the subjective assessment with four sound types were analyzed to detect unimodal and cross-modal effects on environmental sensation and perception. Table 1 lists the results of the factorial ANOVA at P < 0.05. None of the third- and forth-order interaction effects were statistically significant. 3.1. Unimodal effects of sound level and type The level and type of sound significantly influenced all the acoustic attributes assessed: loudness, annoyance, pleasantness, and naturalness.
Table 1 Main and interaction effects of sound level, sound type, illuminance, and gender (P < 0.05). Independent variable
Main effect
Interaction effect
DF
Sound level
3
Sound type
3
Illuminance
2
Gender
1
Sound level × Sound type
9
Sound level × Illuminance
6
Sound level × Gender
3
Sound type × Illuminance
6
Sound type × Gender
3
Illuminance × Gender
2
Error Total
2784 2879
Acoustic Environment
Light Environment
Loudness
Annoyance
Pleasantness
Naturalness
F P F P F P F P
5139.17 < 0.0005 6.04 < 0.0005
1315.4 < 0.0005 179.38 < 0.0005
55.59 < 0.0005 394 < 0.0005
199.33 < 0.0005 216.29 < 0.0005
F P F P F P F P F P F P
8.22 0.004
4.17 0.041
9.15 < 0.0005
11.95 < 0.0005
2.16 0.022
2.5 0.008
11.36 < 0.0005
13.9 < 0.0005
4.51 0.004
5.55 0.001
Brightness
Relaxation
1227.16 < 0.0005 5.75 0.017
11.06 < 0.0005 3.7 0.011 6.55 0.001 38.62 < 0.0005
4.88 0.002 11.89 < 0.0005
139
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
movement “Spring”, and 22 neither liked nor disliked it. Three participants who disliked it were excluded from the sound type analysis. Therefore, the music stimulus was defined as a musical piece that was not disliked by participants. The fan was the most annoying sound among the four sound types. The mean annoyance values of the music and water sounds at 65 and 75 dBA were even lower than those of the babble and fan at 55 and 65 dBA. The mean pleasantness values according to the type of sound were significantly different throughout the sound levels. The pleasantness curves for Music, Water, and Babble sounds were similar to that of the combined data, which had a peak between 55 and 65 dBA; however, pleasantness of the fan noise had a peak at 45 dBA. The most pleasant sound was Music, followed by Water, Babble, and Fan. The water sounds were rated as the most natural sounds and the fan noise was evaluated as the least natural sound. Gender differences appeared to be statistically significant with regard to annoyance and pleasantness. However, the second order interaction between sound level and gender appeared to be significant with regard to the four acoustic attributes. Women tended to vote higher scores in both annoyance and pleasantness as shown in Fig. 7. 3.2. Unimodal effects of illuminance The level of illuminance significantly influenced both brightness and relaxation. Fig. 5(c) shows the lighting attributes according to illuminance levels. Brightness significantly increased with increasing illuminance levels. Brightness was significantly higher in women than men only at 1000 lx. Relaxation curves according to illuminance were reminiscent of the pleasantness according to sound level. They also had peaks at 500 lx. Relaxation was always higher in women than men throughout all of the illuminance levels tested. Gender differences were observed in both brightness and relaxation. Women always voted higher than men for both lighting attributes as shown in Fig. 6. The second order interaction between illuminance and gender was only found for brightness. 3.3. Cross-modal effects of sound and illuminance The combined effects of sound and illuminance on subjective lighting attributes were assessed, but no statistically significant effects of illuminance on subjective acoustic attributes were found as shown in Fig. 8 and Table 1, both for the overall data and for the separate data set by sound level. Loudness, annoyance, pleasantness, and naturalness were not affected by illuminance levels. Brightness tended to increase with increased sound levels, but this effect was not statistically significant. Relaxation decreased with increased sound levels in a statistically significant manner. The mean brightness value was always higher in men than women; however, the mean relaxation was always higher in women than men. We did not observe any effects of the types of sound on brightness but the types of sound did affect relaxation. Table 2 lists the results of the ANOVA and Bonferroni post hoc test for relaxation according to sound level and type. Sound levels below 55 dBA or music stimulus did not decrease visual relaxation. 4. Discussion Fig. 5. Unimodal effects (means and 95% confidence intervals) (a) Loudness and annoyance according to sound level (b) Pleasantness and naturalness according to sound level (c) Brightness and relaxation according to illuminance level.
Sensory enhancement of acoustic perception by illuminance was not observed in this study, which is consistent with the findings of Veitch [25] and Knez and Hygge [24]. These two studies used office-like audio-visual stimuli. The illuminance levels using fluorescent lights [24,25] were implemented as visual stimuli. Office noise [25] and segments of a meaningful, irrelevant, and conversational speech between teenagers with a chatter background [24] were used for auditory stimuli. However, the interactions between noise and light reported by these studies were focused on cognitive performance. In contrast, the
trends were statistically significant only for pleasantness. The second order interactions between sound level and sound type were found as shown in Fig. 6 and Table 1. Thirty-five out of 60 participants responded that they liked Vivaldi’s The Four Seasons, the 1st 140
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
Fig. 6. Acoustic attributes according to sound type (means and 95% confidence intervals) (a) Loudness (b) Annoyance (c) Pleasantness (d) Naturalness.
semantic parameter of brightness did not measure for the space where the participants were located; rather, it measured for the scenes which they watched, and therefore cannot be directly compared with the results in this study. The effects of sound on illuminance perception were observed in visual relaxation but not in brightness, which is consistent with the cognitive performance research conducted by Veitch [25] and Knez and Hygge [24], with regards to indoor noise and spatial illuminance. Hygge and Knez [26] found interactions between noise and indoor lighting on cognitive performance and self-reported affect; however, they varied the color of light instead of illuminance. Among studies which have dealt with self-luminous visual stimuli, Odgaard et al. [13] and Marks et al. [14] report an absence of enhancement of brightness by sound. However, they only considered the intensity of the light in perceptive attributes of visual stimuli. To date, no studies have examined the effects of sound on more detailed illuminance perception other than brightness in indoor environments. Brightness and relaxation can be classified differently with regards to sensation and perception. Brightness refers to visual sensations caused by a certain physical illuminance level, but relaxation refers to visual perceptions caused by visual sensations such as brightness due to illuminance levels. Thus, this study observed that sound affected visual perception but not visual sensation. It is reminiscent that cross-modal effects of thermal conditions and noise were not found in thermal sensation but in thermal comfort [40]. However, there is a need for more research on cross-modal interactions to draw a general conclusion. Gender differences in auditory perception should be considered with regards to sound levels. On the whole, women rated the sound sources as louder, more annoying, more pleasant, and more natural than men did. However, loudness and annoyance were greater in men than women at soft sound levels such as 35 and 45 dBA, whereas loudness and annoyance were greater in women than men at loud sound levels such as 65 and 75 dBA. For pleasantness and naturalness, women rated the sounds as more pleasant and natural than men did up to 65 dBA, but less pleasant and natural at 75 dBA. Thus, women seem to be more sensitive to auditory perception than men at high level of sound. Gender differences in brightness should also be considered with regards to illuminance levels. Up to 500 lx, no gender differences were found in brightness; but at 1000 lx, brightness was significantly greater in women than men with. Women were more sensitive to brightness perception than men at high level of illuminance. However, no effects of illuminance level on gender difference were found in relaxation. The present experiment was conducted in a test laboratory with limited test configurations. Our focus in this study was the evaluation of
Fig. 7. Gender differences in subjective assessment (means and 95% confidence intervals).
focus of this study is sensation and perception evaluated by semantic parameters. In contrast to the findings of Marks et al. [14], Odgaard et al. [17], and Ma and Nie [28] with simplified audio-visual stimuli, other studies have reported a lack of effects of lighting on acoustic perception. In the studies by Marks et al. [14] and Odgaard et al. [17], 250-ms sinusoid [14] and 40-ms white noise [17] were used as auditory stimuli. A 250ms flash using rectangular patches of white light (1.5 cm wide × 1.2 cm high) at the center of a Super VGA monitor [14] and a 40-ms flash using LED in front of the participant’s chinrest [17] were simultaneously implemented as visual stimuli. In these two studies, the visual stimulus was akin to a self-luminous spot rather than an overall illuminated space. The illuminant system in the test laboratory consisted of ceilingmounted lighting and two fluorescent tubes, which are widely used in regular offices and classrooms. Furthermore, in this study, the duration of sound was 25 s, but the illuminance level was fixed during the session based on the adaptation period. Therefore, the auditory stimuli could be tailored to each individual participant. This condition resembles real-world situations where lighting is turned on during working hours with time-varying sound sources. Ma and Nie [28] used a 47-inch plasma TV for a graphical 3D model of the indoor scenes and recorded traffic noise through loudspeakers in a semi-anechoic chamber. Their 141
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
Table 2 Results of the ANOVA and Bonferroni post hoc test for relaxation according to sound level and type. Sound level (dBA)
N
Mean Relaxation
Sound type
N
Mean Relaxation
35 45 55 65 75
180 711 711 711 711
6.572 6.29028 6.19028 6.02361 5.78611
Ambient Music Water Babble Fan
180 684 720 720 720
6.572 6.20694 6.15417 5.98333 5.94583
F P
11.70 < 0.0005
F P
6.10 < 0.0005
A A AB BC C
A AB BC BC C
attributes tested in this study. A critical step in this regard would be to develop research projects aimed at multisensory integration in indoor environments. It will be important to extend the experimental sample to other populations to further elucidate the impact of factors such as age, cultural origin, and industrial origin. 5. Conclusions This paper investigated the combined effect of sound and illuminance level of daily indoor environments that are moderately noisy and do not present severe lighting conditions in a sample of 60 college-aged individuals. The summary of findings is as follows: 1. The acoustic conditions affected visual relaxation. 2. The illuminance conditions did not affect acoustic perception. 3. Cross-modal interactions were asymmetric between sound and illuminance in indoor environmental settings. 4. Women were more sensitive to the perception of both acoustic and illuminance stimuli than men, at high level of stimulation. Acknowledgement This research was supported by grants (NRF – 2015R1D1A1A01057041, NRF – 2018R1D1A1B07048157) of the Basic Science Research Program of the National Research Foundation (NRF) funded by the Ministry of Education, Republic of Korea. References [1] Alais D, Newell FN, Mamassian P. Multisensory processing in review: from physiology to behaviour. Seeing Perceiv 2010;23:3–38. [2] Spence C. Crossmodal correspondences: a tutorial review. Atten Percep Psychophys 2011;73:971–95. [3] Chen Y-C, Spence C. When hearing the bark helps to identify the dog: semanticallycongruent sounds modulate the identification of masked pictures. Cognition 2010;114:389–404. [4] Evans KK, Treisman A. Natural cross-modal mappings between visual and auditory features. J vis 2009;10(6). [5] Broadbent DE. Some effects of noise on visual performance. Quart J Experim Psychol 1954;6:1–5. [6] Marks LE. On associations of light and sound: the mediation of brightness, pitch, and loudness. Am J Psychol 1974;87:173–88. [7] Shams L, Kamitani Y, Shimojo S. Illusions: what you see is what you hear. Nature 2000;408:788. [8] Walker JT, Scott KJ. Auditory–visual conflicts in the perceived duration of lights, tones, and gaps. J Exp Psychol Hum Percept Perform 1981;7:1327. [9] Stein BE, London N, Wilkinson LK, Price DD. Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J Cognit Neurosci 1996;8:497–506. [10] Sasaki H, Todorokihara M, Ishida T, Miyachi J, Kitamura T, Aoki R. Effect of noise on the contrast detection threshold in visual perception. Neurosci Lett 2006;408:94–7. [11] Lippert M, Logothetis NK, Kayser C. Improvement of visual contrast detection by a simultaneous sound. Brain Res 2007;1173:102–9. [12] Scheier C, Nijhawan R, Shimojo S. Sound alters visual temporal resolution. Investigative Ophthalmology & Visual Science 9650 Rockville Pike, Bethesda, MD 20814-3998 USA: Assoc Research Vision Ophthalmology Inc; 1999. p. S792-S. [13] Odgaard EC, Arieh Y, Marks LE. Cross-modal enhancement of perceived brightness: sensory interaction versus response bias. Percep Psychophys 2003;65:123–32.
Fig. 8. Cross-modal effects (means and 95% confidence intervals) (a) loudness and annoyance according to illuminance level (b) pleasantness and naturalness according to illuminance level (c) brightness and relaxation according to sound level.
limited conditions of indoor environments for young adults. This raises the question as to whether other types of lighting, light color, room temperature, room size etc. may also affect the semantic environmental 142
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
road traffic noise in indoor environment. 2014. [29] Liebl A, Haller J, Jödicke B, Baumgartner H, Schlittmeier S, Hellbrück J. Combined effects of acoustic and visual distraction on cognitive performance and well-being. Appl Ergon 2012;43:424–34. [30] D’Alessandro LM, Norwich KH. Loudness adaptation measured by the simultaneous dichotic loudness balance technique differs between genders. Hear Res 2009;247:122–7. [31] Shaqiri A, Brand A, Roinishvili M, et al. Gender differences in visual perception. Perception 2016;45:280. 39th European Conference on Visual Perception (ECVP). [32] ISO. ISO/TS 15666: 2003 Acoustics-Assessment of noise annoyance by means of social and socio-acoustic surveys. International Organisation for Standardization; 2003. [33] Berglund B, Berglund U, Lindvall T. Scaling loudness, noisiness, and annoyance of aircraft noise. J Acoust Soc Am 1975;57:930–4. [34] Berglund B, Berglund U, Lindvall T. Scaling loudness, noisiness, and annoyance of community noises. J Acoust Soc Am 1976;60:1119–25. [35] Hao Y, Kang J, Wörtche H. Assessment of the masking effects of birdsong on the road traffic noise environment. J Acoust Soc Am 2016;140:978–87. [36] Yang W, Moon HJ, Kim M-J. Perceptual assessment of indoor water sounds over environmental noise through windows. Appl Acoust 2018;135:60–9. [37] Ishida K, Inoue Y, Utiyama H, Kurata J. The effect of season on impression of lighting: part 1: within- and between-subject variation. J Illum Eng Inst Jpn 2011;95:439–45. [38] Manav B. An experimental study on the appraisal of the visual environment at offices in relation to colour temperature and illuminance. Build Environ 2007;42:979–83. [39] Budescu DV, Appelbaum MI. Variance stabilizing transformations and the power of the F test. J Educ Statist 1981;6:55–74. [40] Yang W, Moon HJ. Cross-modal effects of noise and thermal conditions on indoor environmental perception and speech recognition. Appl Acoust 2018;141:1–8.
[14] Marks LE, Ben-Artzi E, Lakatos S. Cross-modal interactions in auditory and visual discrimination. Int J Psychophysiol 2003;50:125–45. [15] Choe CS, Welch RB, Gilford RM, Juola JF. The “ventriloquist effect”: visual dominance or response bias? Percep Psychophys 1975;18:55–60. [16] Saldaña HM, Rosenblum LD. Visual influences on auditory pluck and bow judgments. Percep Psychophys 1993;54:406–16. [17] Odgaard EC, Arieh Y, Marks LE. Brighter noise: sensory enhancement of perceived loudness by concurrent visual stimulation. Cogn Affect Behav Neurosci 2004;4:127–32. [18] Nunnally JC, Bernstein IH. Psychometric theory. New York: McGraw-Hill; 1994. [19] Viollon S, Lavandier C, Drake C. Influence of visual setting on sound ratings in an urban environment. Appl Acoust 2002;63:493–511. [20] Fastl H. Audio-visual interactions in loudness evaluation. Proc of Proc Int Congress on Acoustics ICA 2004, 18 Intern Congress on Acoustics, Kyoto, Japan; 2004. [21] Southworth MF. The sonic environment of cities. Massachusetts Institute of Technology; 1967. [22] Anderson LM, Mulligan BE, Goodman LS, Regen H. Effects of sounds on preferences for outdoor settings. Environ Behav 1983;15:539–66. [23] Carles J, Bernáldez F, Lucio Jd. Audio-visual interactions and soundscape preferences. Landscape Res 1992;17:52–6. [24] Knez I, Hygge S. Irrelevant speech and indoor lighting: effects on cognitive performance and self-reported affect. Appl Cogn Psychol 2002;16:709–18. [25] Veitch JA. Office noise and illumination effects on reading comprehension. J Environ Psychol 1990;10:209–17. [26] Hygge S, Knez I. Effects of noise, heat and indoor lighting on cognitive performance and self-reported affect. J Environ Psychol 2001;21:291–9. [27] Akbari J, Dehghan H, Azmoon H, Forouharmajd F. Relationship between lighting and noise levels and productivity of the occupants in automotive assembly industry. J Environ Public Health 2013;2013. [28] Ma H, Nie W. Influence of visual factors on noise annoyance evaluation caused by
143
Contents lists available at ScienceDirect
Applied Acoustics journal homepage: www.elsevier.com/locate/apacoust
Combined effects of sound and illuminance on indoor environmental perception
T
⁎
Wonyoung Yang , Hyeun Jun Moon Department of Architectural Engineering, Dankook University, Yongin 16890, South Korea
A R T I C LE I N FO
A B S T R A C T
Keywords: Acoustic perception Multisensory interaction Visual perception Gender difference
Cross-modal effects of sound and illuminance were investigated in an indoor environmental chamber with 60 university students (30 men and 30 women) aged 18–26 years. A within-subject factorial design was employed with four independent variables: noise level (45, 55, 65, and 75 dBA), noise type (Music, Water, Babble, and Fan), illuminance level (150, 500, and 1000 lx), and sex (female or male). The test conditions represented daily indoor environments, which was moderately noisy and did not present severe lighting conditions. Acoustic (loudness, annoyance, pleasantness, and naturalness) and visual (brightness and relaxation) semantic attributes were assessed using an 11-point numeric scale. The illuminance level of the ambient lighting system did not affect acoustic perception. Brightness was not altered by sound, but relaxation was affected by sound. Crossmodal interactions were asymmetric between sound and illuminance in indoor environmental settings. Women were more sensitive to the perception of both acoustic and illuminance stimuli than men, at high level of stimulation.
1. Introduction Cross-modal interaction in hearing and vision has been investigated as a part of multisensory interaction. Multisensory interaction research has exploded over the last few decades in cognitive neuroscience and neurophysiology [1]. Spatial and temporal factors influence multisensory integration [2], and the effects of both semantic and synesthetic congruency on multisensory information processing have been studied [3,4]. Effects of sound on visual perception have been studied with relatively simple audiovisual stimuli in early studies. Broadbent [5] reported that performance on light-watching became relatively less efficient with continuous exposure to noise using two visual vigilance tasks. Marks [6] found that most subjects matched pure tones to visual brightness of gray surfaces. Shams et al. [7] showed that visual perception (number of flashes) could be qualitatively altered by sound (number of beeps). The perceived duration [8], intensity [9], and contrast detection [10,11] of a visual stimulus has been shown to be influenced by accompanying sound signals. Scheier et al. [12] have shown that visual temporal resolution can either be improved or degraded by sound, depending on the temporal relationship of these variables. In contrast, Odgaard et al. [13] and Marks et al. [14] reported that brightness did not seem to be enhanced by sound. Effects of vision on acoustic perception have been also studied with ⁎
relatively simple audiovisual stimuli. It is well known that the spatial location of a visual stimulus can modify the apparent location of a simultaneously presented sound [15] (for example, in visual capture or ventriloquism). Saldana and Rosenblum [16] suggested that only discontinuous visual stimuli have a strong effect on the perception of the sound. Odgaard et al. [17] reported that white noise presented with light tended to be rated as louder than noise presented alone. Marks et al. [14] observed that the effect of visual stimulation on auditory pitch and loudness discrimination in an unspeeded discrimination test. If the time constraint leads to a reduction of the number of items attempted by all participants to < 90%, the test is considered as speeded [18]. Of note, Odgaard et al. [13] and Marks et al. [14] reported that visual perception was unaffected by sound. The asymmetry of crossmodal interactions that occurs in unspeeded discrimination contrasts with the bidirectional interactions observed in speeded identification. Shared sensory mechanisms underlie the cross-modal interactions, but these mechanisms change as a function of the task. A caveat of these experiments in cognitive neuroscience is that they do not involve commonly occurring, realistic audiovisual situations. For instance, the auditory stimuli that were used in these studies were white noise or pure tones, and the visual stimuli were flashes [19]. Currently, research on the cross-modal interactions between hearing and vision is very popular in the context of multimedia applications [20], and has also been extended to environmental approaches. More
Corresponding author. E-mail address: [email protected] (W. Yang).
https://doi.org/10.1016/j.apacoust.2018.07.008 Received 2 April 2018; Received in revised form 2 July 2018; Accepted 6 July 2018 0003-682X/ © 2018 Elsevier Ltd. All rights reserved.
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
Fig. 1. Layout of the lighting fixtures and test laboratory.
Ma and Nie [28] reported that the influence of brightness on noise annoyance caused by road traffic noise was obvious in the indoor environment. The effect of color on the evaluation of noise annoyance was not significant, although color (red, yellow, cyan, and green) and brightness showed an interaction effect on the evaluation of noise annoyance. Low brightness significantly increased subjective measures of noise annoyance. However, they did not quantify brightness in their experimental setting. Liebl et al. [29] suggested that the effects on cognitive performance and well-being must be considered separately since these effects are rarely consistent. The interaction effects of background speech (24 and 94% word intelligibility) and lighting conditions (static and dynamic lighting at 55–60 lx) on cognitive performance have not been reported. Nevertheless, these variables were found to have an interaction effect on perceived performance during task processing. The participants perceived that performance was better if background speech of low intelligibility was combined with static lighting. The objective of this study is to explore the cross-modal effects of indoor sounds and illumination levels on the sensation and perception of each sensory modality in indoor settings. In short, the present study investigates how acoustic factors influence visual judgments and how visual factors influence auditory judgments which thus far have not been examined in more realistic indoor settings. To date, no study has assessed the combined effects of sound and illuminance on human perception of both sensory modalities simultaneously. This study is based on the hypothesis that cross-modal interaction is possible between sound and light. We used a 4 × 4 × 3 × 2 (sound levels × sound types × illuminance levels × gender) within-subjects design. Gender was included because gender differences exist in individual sound and visual perception [30,31].
realistic audiovisual stimuli have been involved in these more recent experiments, including scenes and environmental sounds [21–23]. Visual stimuli have been subdivided to analyze cross-modal interactions in environmental approaches. In this study, audiovisual interactions in indoor environmental approaches are reviewed. Studies on audiovisual interactions in indoor environmental settings have been developed with subdivided visual stimuli and noise using various levels of sensation, perception, cognitive performance, and productivity. However, the results seem to be inconsistent. Knez and Hygge [24] found no interactions between noise and color temperature of light, but separate main effects were found for irrelevant speech (38 and 66 dBA) and light (3000 and 4000 K at 500 lx) on cognitive performance. The participants’ long-term memory recall was better in silence than in the irrelevant speech condition, and in the warm-white lighting (3000 K) than in the cool-white lighting (4000 K). Veitch [25] attempted to integrate light and noise to examine their interactional effects on reading comprehension. The illumination levels were 200, 400, and 600 lx with standard cool-white fluorescent lights and the office noise levels were approximately 50 and 70 dBA. No main effects of noise, illuminance, gender, or any effects of interactions of these variables were observed on cognitive performance. One possible explanation for this was that the condition of allowing the participants to leave at any time they chose may have neutralized the effects of the uncontrollable, unpredictable noise on cognitive performance [25]. Hygge and Knez [26] found interactions between broadband low-frequency noise (38 and 58 dBA) and light (300 and 1500 lx with a color temperature of 3000 K) on cognitive performance as well as perception. The main findings for the effects of noise and light were better free recall scores and higher activation scores in the low noise condition at 1500 lx than 300 lx. Akbari et al. [27] examined the relationship between lighting (lx at the height of 30 in. from the surface of the work station) and noise level (Leq 8 h of daily and 40 h of weekly exposure) on human productivity in the automotive assembly industry. Noise levels in the workplace have negative effects on workers’ productivity which leads to a decrease in the organization’s productivity and a corresponding decrease in the quality and quantity of services and products. Lighting did not have an effect on human productivity and changes in lighting were not related to changes in human productivity.
2. Methods 2.1. Experimental conditions The experiment was run in a test laboratory (4.0 m × 5.0 m × 2.4 m), furnished as a small classroom. The test laboratory in this study was built for indoor environmental research. The 137
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
were carried out with an audiometer (GSI 18) in a soundproof booth and all participants had normal hearing (0–25 dB HL). All participants had self-reported normal or corrected-to-normal vision and none of them had color deficiency. They provided informed consent and received a description of the experiment with brief instructions prior to the test regarding the operation of the test device.
2.3. Experimental design and procedure A within-subject factorial design was employed with four independent variables: noise level (45, 55, 65, and 75 dBA), noise type (Music, Water, Babble, and Fan), illuminance level (150, 500, and 1000 lx), and sex (female or male). In some of the analyses, ambient noise was supplemented as a reference case. Three illuminance levels were tested in three experimental sessions. Fig. 3 shows experimental procedure for each session. In each illuminance session, participants listened to and evaluated a total of 17 sound stimuli (four types of sound sources and four levels of each) including a reference case at the ambient noise level of 35 dBA in the laboratory in a random order. Each sound stimulus was presented for 25 s and a 50-s response time was allowed. A 30-min adaptation period was implemented at the beginning of each session for thermal as well as light and dark adaptation. Each session took about 55 min. The subjective acoustic and visual characteristics were assessed using an 11-point numeric scale recommended by ISO 15666 [32], as shown in Fig. 4. A total of six semantic attributes of each sound stimulus were evaluated at the fixed illuminance level. For acoustic attributes, loudness, annoyance, pleasantness, and naturalness were examined. For visual attributes, brightness and relaxation were assessed. Loudness and annoyance have frequently been used in noise research [33,34]. Pleasantness and naturalness have been widely used in positive sound evaluation [35,36]. Brightness and relaxation have been used for assessing subjective impression in lighting perception studies [37,38]. Loudness for sound level, and brightness for illuminance were sensation-level semantic attributes in each modality. Annoyance, pleasantness, naturalness of sounds, and relaxation of vision were more likely perceptive attributes. In each session, a group of four participants (the maximum size of each group was four) was seated in the test laboratory. The participants were provided with definitions of the semantic attributes but were not informed of the actual experimental conditions in order to prevent the development of experimental bias. A quick survey regarding the music stimulus was performed. Prior to the test, the participants were asked for their likes and dislikes of the music stimulus, Vivaldi’s The Four Seasons or the 1st movement “Spring” in order to understand the participants’ reaction to the music stimulus. The subjective assessment was conducted using a tablet pad and the data were automatically saved on a server. A factorial analysis of variance (ANOVA) was used to test six subjective environmental attributes in the experiment to fit four independent variables, i.e., noise level, noise type, illuminance, and gender. ANOVA is a powerful statistical test and was used in this case, although normality cannot be guaranteed for subjective ratings [39]. All the statistical analyses were carried out using the Minitab® 17 with P = 0.05 as the level of significance.
Fig. 2. Frequency spectra of the sound sources.
room temperature (25 °C) and humidity (45%) were controlled by variable refrigerant flow systems, humidifiers, and ventilation systems located inside and outside the test laboratory. The ventilation system was run throughout the experiment. Six ceiling-mounted fluorescent lighting fixtures with two tubes (Osram FHF32SSEX-D) were installed with a dimming control system. The color temperature of the lamp was 6500 K according to the specification sheets provided by the manufacturer. A loudspeaker system (Turbosound Milan M10) for sound presentation was located in the center at the front of the room as shown in Fig. 1. The reverberation time of the test laboratory was measured as 0.33 s at 500 Hz for one-third octave bands. The ambient noise level in the laboratory was 35 dBA (Solo dB01) when the thermal systems were running (see Fig. 2). The illuminance levels on the participants’ desk surface were 150, 500, and 1000 lx. Four different sound sources (Music, Water, Babble, and Fan) were played through the loudspeaker at four levels (45, 55, 65, and 75 dBA). Vivaldi’s The Four Seasons or the 1st movement “Spring” performed by Amsterdam Sinfonietta in Concertgebouw in 2014 was used as a music sound source. The water sounds from an indoor water fountain were recorded in the test laboratory. The babble noise was recorded in a noisy cafeteria. The fan noise was also recorded in the test laboratory with ventilation fans running. The levels of the sound sources were adjusted by an audio controller. The illuminance levels along the desk surface during the experiment were 158.4, 516.5, and 997.4 lx (Konica Minolta T-10A), on average. The illuminance level differences across the participants’ positions were measured at ± 20.2 lx. The sound levels were measured at each of the four participants positions, and their differences were at ± 0.3 dBA. 2.2. Participants A total of 60 university students (30 men and 30 women) aged 18–26 years participated in the three experimental sessions and received financial support for their participation. Hearing screening tests
Fig. 3. Experimental procedure for each session.
138
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
Fig. 4. Semantic judgment test interface.
3. Results
The second order interaction between sound level and type also influenced all the acoustic attributes. Fig. 5(a) and (b) show the acoustic attributes according to sound level. Loudness and annoyance increased as sound level increased. At lower sound levels, loudness and annoyance were significantly lower in women than men, but at higher sound levels, loudness and annoyance were significantly higher in women than men. Pleasantness showed an inverted U-shape according to sound level. The pleasantness peak appeared at a noise level of 55 dBA. Naturalness decreased in a statistically significant manner as sound level increased. Pleasantness and naturalness tended to be higher in women than men up to 65 dBA, but lower in women than men at a noise level of 75 dBA. However, these
The overall data for the subjective assessment with four sound types were analyzed to detect unimodal and cross-modal effects on environmental sensation and perception. Table 1 lists the results of the factorial ANOVA at P < 0.05. None of the third- and forth-order interaction effects were statistically significant. 3.1. Unimodal effects of sound level and type The level and type of sound significantly influenced all the acoustic attributes assessed: loudness, annoyance, pleasantness, and naturalness.
Table 1 Main and interaction effects of sound level, sound type, illuminance, and gender (P < 0.05). Independent variable
Main effect
Interaction effect
DF
Sound level
3
Sound type
3
Illuminance
2
Gender
1
Sound level × Sound type
9
Sound level × Illuminance
6
Sound level × Gender
3
Sound type × Illuminance
6
Sound type × Gender
3
Illuminance × Gender
2
Error Total
2784 2879
Acoustic Environment
Light Environment
Loudness
Annoyance
Pleasantness
Naturalness
F P F P F P F P
5139.17 < 0.0005 6.04 < 0.0005
1315.4 < 0.0005 179.38 < 0.0005
55.59 < 0.0005 394 < 0.0005
199.33 < 0.0005 216.29 < 0.0005
F P F P F P F P F P F P
8.22 0.004
4.17 0.041
9.15 < 0.0005
11.95 < 0.0005
2.16 0.022
2.5 0.008
11.36 < 0.0005
13.9 < 0.0005
4.51 0.004
5.55 0.001
Brightness
Relaxation
1227.16 < 0.0005 5.75 0.017
11.06 < 0.0005 3.7 0.011 6.55 0.001 38.62 < 0.0005
4.88 0.002 11.89 < 0.0005
139
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
movement “Spring”, and 22 neither liked nor disliked it. Three participants who disliked it were excluded from the sound type analysis. Therefore, the music stimulus was defined as a musical piece that was not disliked by participants. The fan was the most annoying sound among the four sound types. The mean annoyance values of the music and water sounds at 65 and 75 dBA were even lower than those of the babble and fan at 55 and 65 dBA. The mean pleasantness values according to the type of sound were significantly different throughout the sound levels. The pleasantness curves for Music, Water, and Babble sounds were similar to that of the combined data, which had a peak between 55 and 65 dBA; however, pleasantness of the fan noise had a peak at 45 dBA. The most pleasant sound was Music, followed by Water, Babble, and Fan. The water sounds were rated as the most natural sounds and the fan noise was evaluated as the least natural sound. Gender differences appeared to be statistically significant with regard to annoyance and pleasantness. However, the second order interaction between sound level and gender appeared to be significant with regard to the four acoustic attributes. Women tended to vote higher scores in both annoyance and pleasantness as shown in Fig. 7. 3.2. Unimodal effects of illuminance The level of illuminance significantly influenced both brightness and relaxation. Fig. 5(c) shows the lighting attributes according to illuminance levels. Brightness significantly increased with increasing illuminance levels. Brightness was significantly higher in women than men only at 1000 lx. Relaxation curves according to illuminance were reminiscent of the pleasantness according to sound level. They also had peaks at 500 lx. Relaxation was always higher in women than men throughout all of the illuminance levels tested. Gender differences were observed in both brightness and relaxation. Women always voted higher than men for both lighting attributes as shown in Fig. 6. The second order interaction between illuminance and gender was only found for brightness. 3.3. Cross-modal effects of sound and illuminance The combined effects of sound and illuminance on subjective lighting attributes were assessed, but no statistically significant effects of illuminance on subjective acoustic attributes were found as shown in Fig. 8 and Table 1, both for the overall data and for the separate data set by sound level. Loudness, annoyance, pleasantness, and naturalness were not affected by illuminance levels. Brightness tended to increase with increased sound levels, but this effect was not statistically significant. Relaxation decreased with increased sound levels in a statistically significant manner. The mean brightness value was always higher in men than women; however, the mean relaxation was always higher in women than men. We did not observe any effects of the types of sound on brightness but the types of sound did affect relaxation. Table 2 lists the results of the ANOVA and Bonferroni post hoc test for relaxation according to sound level and type. Sound levels below 55 dBA or music stimulus did not decrease visual relaxation. 4. Discussion Fig. 5. Unimodal effects (means and 95% confidence intervals) (a) Loudness and annoyance according to sound level (b) Pleasantness and naturalness according to sound level (c) Brightness and relaxation according to illuminance level.
Sensory enhancement of acoustic perception by illuminance was not observed in this study, which is consistent with the findings of Veitch [25] and Knez and Hygge [24]. These two studies used office-like audio-visual stimuli. The illuminance levels using fluorescent lights [24,25] were implemented as visual stimuli. Office noise [25] and segments of a meaningful, irrelevant, and conversational speech between teenagers with a chatter background [24] were used for auditory stimuli. However, the interactions between noise and light reported by these studies were focused on cognitive performance. In contrast, the
trends were statistically significant only for pleasantness. The second order interactions between sound level and sound type were found as shown in Fig. 6 and Table 1. Thirty-five out of 60 participants responded that they liked Vivaldi’s The Four Seasons, the 1st 140
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
Fig. 6. Acoustic attributes according to sound type (means and 95% confidence intervals) (a) Loudness (b) Annoyance (c) Pleasantness (d) Naturalness.
semantic parameter of brightness did not measure for the space where the participants were located; rather, it measured for the scenes which they watched, and therefore cannot be directly compared with the results in this study. The effects of sound on illuminance perception were observed in visual relaxation but not in brightness, which is consistent with the cognitive performance research conducted by Veitch [25] and Knez and Hygge [24], with regards to indoor noise and spatial illuminance. Hygge and Knez [26] found interactions between noise and indoor lighting on cognitive performance and self-reported affect; however, they varied the color of light instead of illuminance. Among studies which have dealt with self-luminous visual stimuli, Odgaard et al. [13] and Marks et al. [14] report an absence of enhancement of brightness by sound. However, they only considered the intensity of the light in perceptive attributes of visual stimuli. To date, no studies have examined the effects of sound on more detailed illuminance perception other than brightness in indoor environments. Brightness and relaxation can be classified differently with regards to sensation and perception. Brightness refers to visual sensations caused by a certain physical illuminance level, but relaxation refers to visual perceptions caused by visual sensations such as brightness due to illuminance levels. Thus, this study observed that sound affected visual perception but not visual sensation. It is reminiscent that cross-modal effects of thermal conditions and noise were not found in thermal sensation but in thermal comfort [40]. However, there is a need for more research on cross-modal interactions to draw a general conclusion. Gender differences in auditory perception should be considered with regards to sound levels. On the whole, women rated the sound sources as louder, more annoying, more pleasant, and more natural than men did. However, loudness and annoyance were greater in men than women at soft sound levels such as 35 and 45 dBA, whereas loudness and annoyance were greater in women than men at loud sound levels such as 65 and 75 dBA. For pleasantness and naturalness, women rated the sounds as more pleasant and natural than men did up to 65 dBA, but less pleasant and natural at 75 dBA. Thus, women seem to be more sensitive to auditory perception than men at high level of sound. Gender differences in brightness should also be considered with regards to illuminance levels. Up to 500 lx, no gender differences were found in brightness; but at 1000 lx, brightness was significantly greater in women than men with. Women were more sensitive to brightness perception than men at high level of illuminance. However, no effects of illuminance level on gender difference were found in relaxation. The present experiment was conducted in a test laboratory with limited test configurations. Our focus in this study was the evaluation of
Fig. 7. Gender differences in subjective assessment (means and 95% confidence intervals).
focus of this study is sensation and perception evaluated by semantic parameters. In contrast to the findings of Marks et al. [14], Odgaard et al. [17], and Ma and Nie [28] with simplified audio-visual stimuli, other studies have reported a lack of effects of lighting on acoustic perception. In the studies by Marks et al. [14] and Odgaard et al. [17], 250-ms sinusoid [14] and 40-ms white noise [17] were used as auditory stimuli. A 250ms flash using rectangular patches of white light (1.5 cm wide × 1.2 cm high) at the center of a Super VGA monitor [14] and a 40-ms flash using LED in front of the participant’s chinrest [17] were simultaneously implemented as visual stimuli. In these two studies, the visual stimulus was akin to a self-luminous spot rather than an overall illuminated space. The illuminant system in the test laboratory consisted of ceilingmounted lighting and two fluorescent tubes, which are widely used in regular offices and classrooms. Furthermore, in this study, the duration of sound was 25 s, but the illuminance level was fixed during the session based on the adaptation period. Therefore, the auditory stimuli could be tailored to each individual participant. This condition resembles real-world situations where lighting is turned on during working hours with time-varying sound sources. Ma and Nie [28] used a 47-inch plasma TV for a graphical 3D model of the indoor scenes and recorded traffic noise through loudspeakers in a semi-anechoic chamber. Their 141
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
Table 2 Results of the ANOVA and Bonferroni post hoc test for relaxation according to sound level and type. Sound level (dBA)
N
Mean Relaxation
Sound type
N
Mean Relaxation
35 45 55 65 75
180 711 711 711 711
6.572 6.29028 6.19028 6.02361 5.78611
Ambient Music Water Babble Fan
180 684 720 720 720
6.572 6.20694 6.15417 5.98333 5.94583
F P
11.70 < 0.0005
F P
6.10 < 0.0005
A A AB BC C
A AB BC BC C
attributes tested in this study. A critical step in this regard would be to develop research projects aimed at multisensory integration in indoor environments. It will be important to extend the experimental sample to other populations to further elucidate the impact of factors such as age, cultural origin, and industrial origin. 5. Conclusions This paper investigated the combined effect of sound and illuminance level of daily indoor environments that are moderately noisy and do not present severe lighting conditions in a sample of 60 college-aged individuals. The summary of findings is as follows: 1. The acoustic conditions affected visual relaxation. 2. The illuminance conditions did not affect acoustic perception. 3. Cross-modal interactions were asymmetric between sound and illuminance in indoor environmental settings. 4. Women were more sensitive to the perception of both acoustic and illuminance stimuli than men, at high level of stimulation. Acknowledgement This research was supported by grants (NRF – 2015R1D1A1A01057041, NRF – 2018R1D1A1B07048157) of the Basic Science Research Program of the National Research Foundation (NRF) funded by the Ministry of Education, Republic of Korea. References [1] Alais D, Newell FN, Mamassian P. Multisensory processing in review: from physiology to behaviour. Seeing Perceiv 2010;23:3–38. [2] Spence C. Crossmodal correspondences: a tutorial review. Atten Percep Psychophys 2011;73:971–95. [3] Chen Y-C, Spence C. When hearing the bark helps to identify the dog: semanticallycongruent sounds modulate the identification of masked pictures. Cognition 2010;114:389–404. [4] Evans KK, Treisman A. Natural cross-modal mappings between visual and auditory features. J vis 2009;10(6). [5] Broadbent DE. Some effects of noise on visual performance. Quart J Experim Psychol 1954;6:1–5. [6] Marks LE. On associations of light and sound: the mediation of brightness, pitch, and loudness. Am J Psychol 1974;87:173–88. [7] Shams L, Kamitani Y, Shimojo S. Illusions: what you see is what you hear. Nature 2000;408:788. [8] Walker JT, Scott KJ. Auditory–visual conflicts in the perceived duration of lights, tones, and gaps. J Exp Psychol Hum Percept Perform 1981;7:1327. [9] Stein BE, London N, Wilkinson LK, Price DD. Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J Cognit Neurosci 1996;8:497–506. [10] Sasaki H, Todorokihara M, Ishida T, Miyachi J, Kitamura T, Aoki R. Effect of noise on the contrast detection threshold in visual perception. Neurosci Lett 2006;408:94–7. [11] Lippert M, Logothetis NK, Kayser C. Improvement of visual contrast detection by a simultaneous sound. Brain Res 2007;1173:102–9. [12] Scheier C, Nijhawan R, Shimojo S. Sound alters visual temporal resolution. Investigative Ophthalmology & Visual Science 9650 Rockville Pike, Bethesda, MD 20814-3998 USA: Assoc Research Vision Ophthalmology Inc; 1999. p. S792-S. [13] Odgaard EC, Arieh Y, Marks LE. Cross-modal enhancement of perceived brightness: sensory interaction versus response bias. Percep Psychophys 2003;65:123–32.
Fig. 8. Cross-modal effects (means and 95% confidence intervals) (a) loudness and annoyance according to illuminance level (b) pleasantness and naturalness according to illuminance level (c) brightness and relaxation according to sound level.
limited conditions of indoor environments for young adults. This raises the question as to whether other types of lighting, light color, room temperature, room size etc. may also affect the semantic environmental 142
Applied Acoustics 141 (2018) 136–143
W. Yang, H.J. Moon
road traffic noise in indoor environment. 2014. [29] Liebl A, Haller J, Jödicke B, Baumgartner H, Schlittmeier S, Hellbrück J. Combined effects of acoustic and visual distraction on cognitive performance and well-being. Appl Ergon 2012;43:424–34. [30] D’Alessandro LM, Norwich KH. Loudness adaptation measured by the simultaneous dichotic loudness balance technique differs between genders. Hear Res 2009;247:122–7. [31] Shaqiri A, Brand A, Roinishvili M, et al. Gender differences in visual perception. Perception 2016;45:280. 39th European Conference on Visual Perception (ECVP). [32] ISO. ISO/TS 15666: 2003 Acoustics-Assessment of noise annoyance by means of social and socio-acoustic surveys. International Organisation for Standardization; 2003. [33] Berglund B, Berglund U, Lindvall T. Scaling loudness, noisiness, and annoyance of aircraft noise. J Acoust Soc Am 1975;57:930–4. [34] Berglund B, Berglund U, Lindvall T. Scaling loudness, noisiness, and annoyance of community noises. J Acoust Soc Am 1976;60:1119–25. [35] Hao Y, Kang J, Wörtche H. Assessment of the masking effects of birdsong on the road traffic noise environment. J Acoust Soc Am 2016;140:978–87. [36] Yang W, Moon HJ, Kim M-J. Perceptual assessment of indoor water sounds over environmental noise through windows. Appl Acoust 2018;135:60–9. [37] Ishida K, Inoue Y, Utiyama H, Kurata J. The effect of season on impression of lighting: part 1: within- and between-subject variation. J Illum Eng Inst Jpn 2011;95:439–45. [38] Manav B. An experimental study on the appraisal of the visual environment at offices in relation to colour temperature and illuminance. Build Environ 2007;42:979–83. [39] Budescu DV, Appelbaum MI. Variance stabilizing transformations and the power of the F test. J Educ Statist 1981;6:55–74. [40] Yang W, Moon HJ. Cross-modal effects of noise and thermal conditions on indoor environmental perception and speech recognition. Appl Acoust 2018;141:1–8.
[14] Marks LE, Ben-Artzi E, Lakatos S. Cross-modal interactions in auditory and visual discrimination. Int J Psychophysiol 2003;50:125–45. [15] Choe CS, Welch RB, Gilford RM, Juola JF. The “ventriloquist effect”: visual dominance or response bias? Percep Psychophys 1975;18:55–60. [16] Saldaña HM, Rosenblum LD. Visual influences on auditory pluck and bow judgments. Percep Psychophys 1993;54:406–16. [17] Odgaard EC, Arieh Y, Marks LE. Brighter noise: sensory enhancement of perceived loudness by concurrent visual stimulation. Cogn Affect Behav Neurosci 2004;4:127–32. [18] Nunnally JC, Bernstein IH. Psychometric theory. New York: McGraw-Hill; 1994. [19] Viollon S, Lavandier C, Drake C. Influence of visual setting on sound ratings in an urban environment. Appl Acoust 2002;63:493–511. [20] Fastl H. Audio-visual interactions in loudness evaluation. Proc of Proc Int Congress on Acoustics ICA 2004, 18 Intern Congress on Acoustics, Kyoto, Japan; 2004. [21] Southworth MF. The sonic environment of cities. Massachusetts Institute of Technology; 1967. [22] Anderson LM, Mulligan BE, Goodman LS, Regen H. Effects of sounds on preferences for outdoor settings. Environ Behav 1983;15:539–66. [23] Carles J, Bernáldez F, Lucio Jd. Audio-visual interactions and soundscape preferences. Landscape Res 1992;17:52–6. [24] Knez I, Hygge S. Irrelevant speech and indoor lighting: effects on cognitive performance and self-reported affect. Appl Cogn Psychol 2002;16:709–18. [25] Veitch JA. Office noise and illumination effects on reading comprehension. J Environ Psychol 1990;10:209–17. [26] Hygge S, Knez I. Effects of noise, heat and indoor lighting on cognitive performance and self-reported affect. J Environ Psychol 2001;21:291–9. [27] Akbari J, Dehghan H, Azmoon H, Forouharmajd F. Relationship between lighting and noise levels and productivity of the occupants in automotive assembly industry. J Environ Public Health 2013;2013. [28] Ma H, Nie W. Influence of visual factors on noise annoyance evaluation caused by
143