On the variability of room acoustical parameters: Reproducibility and statistical validity

Applied Acoustics 37 (1992) 175-198

On the Variability of Room Acoustical Parameters: Reproducibility and Statistical Validity Xavier Pelorson, Jean-Paul Vian & Jean-Dominique Polack Centre Scientifique et Technique du Bfitiment, 24 rue Joseph Fourier, 38400 Saint-Martin d'H6res, France (Received 1 August 1991; revised version received 31 January 1992; accepted 5 February 1992) A BSTRA CT For many years acoustic quality has been evaluated using a set of objective parameters. Although there is now a growing concensus about the kind and number of parameters to be used, problems still occur when trying to characterise the acoustics of a hall using these parameters. The first part of this work focusses on the specific problems which occur while measuring objective parameters in halls. The influence of each element of the measuring system is considered; it is shown for instance that many parameters vary strongly with the sound source. The spatial variability of parameters is then discussed. The effects of small spatial variations on each parameter are evaluated and compared to reproducibility and within hall variations. It is found that some parameters, including the well known reverberation time, are unable to describe spatial variations within a hall. Lastly, the critical problem of hall to hall comparisons is considered. It is shown that in some cases a comparison between two different halls is impossible because the parameter values overlap strongly. Statistical comparison between halls usually assumes that parameter values within halls reject this assumption for many parameters. Stability of the parameters distribution is then evaluated.

INTRODUCTION In the acoustical assessment o f concert halls, reverberation time has long been k n o w n to be an essential parameter. Yet there is no d o u b t that other 175 Applied Acoustics 0003-682X/92/$05-00 © 1992 Elsevier Scicnce Publishers Ltd, England. Printed in Great Britain

Xavier Pelorson, Jean-Paul Vian, Jean-Dominique Polack

176

objective parameters are necessary to describe adequately the acoustical quality of a hall. Several parameters have been proposed in the recent past,* but it has become increasingly obvious 2 that most of them are highly correlated. Consequently, four or five parameters are usually measured in addition to the reverberation time. The current work was started after extensive use of recent measurement techniques a had raised specific issues linked with parameter measurements. Besides the above-mentioned correlations between parameters, some doubts still exist about their ability to describe fully the acoustical characteristics of an auditorium. 4 Even though the acoustical relevance of parameters has been established through correlation analysis between a selection of objective parameters and subjective preference of listeners, 5'6 contradictions still remain. 7 A new approach to the acoustical relevance of parameters is proposed in the following. We decided not to go through the classical preference analysis but to focus on the variations of parameters within halls, and between halls. Very little is known about the differences thus observed, except that variations can be greater within the same hall at different positions than between different halls at similar positions. 6 Our aim is to analyse extensively how objective parameters vary, regardless of their subjective meaning. First of all, we recall the definitions of the most frequently used parameters, and we describe our measurement system. Correlations between these parameters are checked. We then look at the behaviour of these parameters in terms of reproducibility, stability and influence of the measurement technique. The accuracy to be expected from parameter measurements and their ability to describe spatial variations within a hall are assessed. In the last section, we carry out a statistical analysis of the distribution of each parameter within one hall, to find out whether this distribution remains the same between halls, as is necessary if we are to discriminate between halls by means of a group of parameters.

ROOM A C O U S T I C A L P A R A M E T E R S : D E F I N I T I O N A N D MEASUREMENTS

Definitions Among all the parameters proposed in the literature we will refer to only nine, selected because of their wide use in room acoustics measurements. We let h(t) be the impulse response of the hall.

The variability of room acoustical parameters

177

(1)

Reverberation time (RT) is probably the most widely used parameter. We measure this quantity using the Schroeder integrated impulse response technique, 3 and linear regression between - 5 and - 3 5 dB (or - 2 5 dB when the dynamic range is insufficient). (2) Early decay time (EDT), proposed by Jordan, 8 is measured by the same method as RT, between 0 and - 1 0 d B . (3) Clarity index (C80), proposed by Reichardt et aL, 9 is a measure of the ratio, expressed in dB, of early energy (before 80 ms) to late energy (after 80 ms): FJ~°mSh2(t)dt -] •

c80= 10.,ogL

j

(4) Definition (D50), introduced by Thiele, 1° is the ratio of the early energy with a time limit of 50 ms to the total energy, and is commonly expressed as a percentage: D 5 0 = ~g°msh2(t)dt ~°h2(t)dt (5) Lochner and Burger's signal to noise ratio (S/q'q),11 expressed in dB, is another early to late energy ratio: •

FS95msa(t)h2(t)dt]

S/N: lO.,ogL.

]

with a weighting factor a(t) following the law f l , 01~ a(t) =

--

t _< 35ms

(t -- 95)

35 ms < t _< 95 ms

1,0,

t _> 95 ms

(6) Centre time (Tc) (proposed by Cremer; ~ see also Ref. 12), expressed in seconds, has the advantage of not assuming any arbitrary limit between early and late reflections as in C80 or D50: Tc = j'~"~ t. h2(t) dt ~ o o h2(t)dt In other words, Tc is the centre of gravity of the impulse response energy. (7) Strength index (G) 6'1a is a measure of the loudness. It is expressed as the difference, in dB, between the sound pressure level and the sound power level of the sound source: G=Lp--Lw

Xavier Pelorson, Jean-Paul Vian, Jean-Dominique Polack

178

Lw can be measured or calculated using the sound pressure level at 1 m from the source and the directivity factor Q of the sound source:

Lw = Lp(1 m) + 10. log (4n) - 10. log(Q)

(8) Speech transmission index (STI) 14 is being used increasingly as a parameter for speech intelligibility, as it makes sense in drama theatres or opera houses, where intelligibility of actors and singers is an important factor. (9) Lateral efficiency (LE) ~5 is a more recent parameter measured with bi-directional microphones. Jordan defined LE as the ratio of the energy arriving between 25 ms (some workers choose a 5-ms time limit) and 80ms measured by a 'figure-of-eight' microphone to the early energy (arriving before 80 ms), measured by an omnidirectional microphone: LE = j'~Om~h2(l)dt 2 ~o80ms ho(t) dt

Like D50, LE is commonly expressed as a percentage.

Measurement technique All the above room acoustics parameters are calculated from the room impulse response. The methods of obtaining impulse responses are all based on the same principle: ---excitation of the hall with a sound source; --recording of the halls response; ---calculation of the impulse response. The main difference between the measurement methods lies in the choice of the sound source. Sometimes impulsive sources (such as revolvers, balloons or electric sparks) are used, despite many problems of reproducibility, stability and spectrum. Our measurement system is based on the emission of binary pseudorandom noise sequences generated by the maximal length shift register technique (MLS). The response of the hall is recorded by an omnidirectional microphone (or a figure-of-eight microphone for LE measurements). The impulse response of the hall is then estimated by cross-correlation between the hall's response and the excitation signal, computed with the Hadamard transform. 16 Using octave band filtering (125 Hz-4 kHz), all criteria (except STI) are calculated (Fig. 1). Compared with other measuring methods, this technique has several advantages:

The variability o f room acoustical parameters

Noile

179

I

"i-~-

generationJ 4 ~ Auditorium

¢rosl correlationJ

h,. l ^

.R..mmp°..r..p...,

I rvrl, I

t

""-"

. . . . . . . .

-

t I o ~-" "~' ''-'~ t

125Hz

-

250Hz

"

500Hz

-

I

t-

1kill

t Objective Parameters Frequency

RT 20 EDT C80 D50 Tc El000 RT Man LE

125 Hz

250 Hz

500 Hz

I kHz

2 kHz

4 kHz

1"51 1"27 2"4 42"2 101"3 - 3"0 1-60 9"3

1"77 1"46 2"0 44.0 114-3 - 1"3 1"71 20"6

1"76 1"75 - 1"1 35"3 141"3 -0-5 1"69 23"9

1"75 1-70 1"4 43"6 127-0 0"0 1"66 26-4

1"55 1"31 2"9 54"0 99"1 4-6 1-53 30"5

1"33 1-07 4"0 62"8 80"3 1"7 1-46 22-4

The STI value is 0-55. Fig. 1. Block diagram of impulse response measurements and parameter calculations in auditoria. ADC is a 16 bits analog to digital converter, DAC a 16 bits digital-to-analog converter.

180

Xavier Pelorson, Jean-Paul Vian, Jean-Dominique Polack

(1)

It requires only an IBM-compatible portable computer, a digital board, a microphone and an amplified loudsPeaker box. (2) The use of the Hadamard transform makes the calculation of the impulse response very fast (a few seconds only for the calculation of 32 000 signal points). (3) The use of averaged sequences increases the already large dynamic range of the measurements. (4) The MLS has a fairly flat spectrum (at least for the six octave bands considered) and thus no deconvolution is required. The main drawback of this measurement technique lies in the choice of the sound source (a loudspeaker box); first, because it is not an omnidirectional one and, second, because its frequency response does not allow measurements to be made at the lowest frequencies. Correlations between parameters

Measurement results from 14 concert halls have been analysed. The volume of the halls varied from 3000 to 21 000 m3. The measurement technique used (presented in the previous section) was exactly the same for all the auditoria. Correlation coefficients were calculated over 3000 values (including the six octave-band values) for each of the parameters listed above, except for the strength index (900 results) and the lateral efficiency (700 results). To be quantitatively valid, a correlation analysis should be derived from random measurements, and the measured values are also supposed to follow a normal distribution. Because these assumptions are not, generally, fulfilled by the measurements taken, one must consider the following results as a qualitative indicator of mutual dependence of parameters. Table 1 shows the

TABLE 1 Correlation Matrix between Parameters (Bold Numbers indicate Correlations Higher than 0.9

RT EDT C80 D50 S/N Tc G LE

RT

EDT

C80

1 0"56 -0"3 -0-34 -0-34

I -0-88 -0"83 -0-84

1 0"93 0"98

0"55

0"94 -0"95

0"6 0-03

0"5 -0-03 0-23 -0-25

1)50

SIN

Tc

G

1 -0"94 0"05 -0-25

1 0.1 0-22

1 0"53

I

0"97 -0"94 -0.15 -0"27

The variability o f room acoustical parameters I

'

'

'

'

"

...................................

-"

'i,,.~"

I

'

•

I

'

'

'

,,.

'

'

i

'

'

'

I

- .....

i

"~ • . . . .

I

,

,

,

,

I

II

,

,

,

,

I$

i

I

. . . .

I

10

C~e,

115

~B

Correlation b e t w e e n S / N a n d C 8 0 (correlation factor R = 0"98).

'

-%a

--+

'

'

I

.........................

'

'

'

:'

'

'

"

i .........................

J

i

'

'

'

'

4 .........................

i

"

'

° -.e,,~L..~ °

!

. .........................

~..-

"-.........................

i""

j

°

i

-.÷ ......................... .:'.------.;-...'-:.-....'i~-o~.-:....;.---.-~ .........................

i -n

I

~. .........................

i -11

o

i

o ]

-.".................... :.....=......-~. ~ ~_~...;~/:.:;~~...w....... .............. :i........................ i... i

i"

i

". !

.'.

"oi.~

-I ° .

....

•

".....-

i .4."

i "

.

~

"

!

-,,

..~...................... ,..~.....-....,..~,.'~.~}~,~,.,.~ ............... ",..~..." ....................... i" • .B ~,' " i ~.'o. . . . . . .,, ,,.. ~.~ .~. ~ . • .i . • .

-~H~

-':"*

...........

"'-.--~"

...........

~1 ................

•

.

•

."

-a~

-3m

-.,

,.~ ......................... n . . . -15

i- ......................... i . , . -J,

~.~.

_

~' i

,

.~.~.,.....

:.

-

i i

"

:

.:.

...............

:...

i

•

i ......................... i . . . .

.~ ......................... i , . .

~

'1'

C~I)~

Fig. 3.

'

...... '................................ ~............................... i................................ t-

-6

Fig. 2.

'

i ................................ r ............................................................ i i ,,

i

-14

'

18 !

i i

.~ ......................... i , , ," JJL

|

~.. i JLIS

dB

Correlation b e t w e e n G a n d C 8 0 (correlation factor R ffi - 0 - 0 3 ) .

Xavier Pelorson, Jean-Paul Vian, Jean-Dominique Polack

182

results obtained for eight parameters. Figures 2 and 3 show typical examples of correlations between S/N and C80 and between G and C80 respectively. We do not present results for the STI because it is a wide frequency band parameter whereas all the others are octave-bands parameters. However, we have found a high correlation between the C80 and the modulation transfer index, taken by octave bands, from which the STI is derived. The above results are very similar to those of other workers. 2'5 One important difference is the correlation between E D T and RT: some workers found it high, whereas we obtained a value of only 0.56. A more detailed study shows that the relationship between E D T and RT is dependent on the hall considered. Depending on the concert-hall, we have found correlation factors varying from 0-3 to 0.92. Because many parameters present high mutual dependences, it is not necessary to consider all of them. In the following, we will refer to only five slightly correlated parameters: RT, EDT, C80, LE and G.

REPRODUCIBILITY AND EXTERNAL INFLUENCES To evaluate the accuracy of the measurement and the influence of each element of the measuring system, impulse responses of different halls have been analysed. Sources of error that have been more particularly studied are: --accuracy --influence --influence ---effects of

of calculation of the parameter of the microphone of the loudspeaker system small spatial displacements of source or microphone.

Reproducibility To be used as reference values in the following, we have selected results derived from five or six repeated measurements at one location. Between each measurement the complete measuring system (loudspeaker, microphone, etc.) is removed and reinstalled a few minutes or hours later. The same experiment had been performed in four halls (only two for G and LE) and for different locations in each hall. Thus, referring to International Standard ISO 5725,17 these tests are very close to reproducibility, although we do not consider here the influence of the operator, which should have only a very small effect in a completely numerical measurement method. As we found very close results for all the situations considered, Table 2 presents only mean results for halls and positions for the six octave-bands. As is customary, we have used the standard deviation about the mean as a measure of reproducibility.

183

The variability o f room acoustical parameters

TABLE 2 Standard Deviations with Repeated Measurements Octave band 125 Hz

250 Hz

500 Hz

I kHz

2 kHz

4 kHz

0.05 0.03 0.3 5-0 0-6

0-02 0-03 0-1 4-3 0"4

0.04 0-08 0"5 3"6 0.4

0-02 0-04 0.3 2"2 0.3

0"02 0.06 0"3 2"8 0"3

0-01 0"06 0.4 3"6 0"5

RT (s) EDT (s) C80 (dB) LE (%) G (dB)

Accuracy of parameter calculation Comparisons of several measurement techniques raised the question of the importance of the method of the numerical calculation for most parameters. Measuring C80, for instance, requires the calculation of the early (80-ms) energy. Usually, because of the resolution of the measurement technique used, this energy cannot be exactly calculated. As an example, we studied the effects of an 8-ms time limit imprecision on C80. Figure 4 shows the results obtained for the same location with the early energy calculated before 72 ms, before 80ms and before 88ms. In most cases, we found that a good approximation can be made using an interpolation technique. Finally, measuring reverberation time, although it is a common measure, can also lead to fluctuations. In fact, measuring RT values by means oflinear regression only makes sense for diffuse or nearly diffuse sound fields. In many halls, the energy decay curve over 20 or 30 dB is far from a straight line and thus RT measurements are highly fluctuating, especially at low frequencies. 15

C88,

dB

18

.................................. ~.............~

~...............

"- 88

.................................... O

i

I

t

1 2 5 2 5 0 5 0 0 lJ( Frequencyj Fig. 4 .

Time

Limit:

ms

"*" 7 2 ma

o

21(

41(

flz

Effects of an 8-ms imprecision o n t h e calculation of C80.

Xavier Pelorson, Jean-Paul Vian, Jean-Dominique Polack

184

EI)T,8

l

flicrophone

Used:

0,5

Heunsnn

-~- Bit

8

i

i

/

125258588

1X

Frequency,

i ZI( Hz

41(

Fig. 5. Comparison of measurements taken with two microphones.

Influence of microphones We have compared measurements taken with two types of microphone; one is a Bruel & Kjaer 1/2 inch (model 4155) and the other is a Neumann 'studio microphone' (TLM 170i). Figure 5 shows an example of results for EDT. For all parameters, very small differences were found to be comparable with reproducibility. As the general characteristics of both microphones are very similar, at least for the six octave bands studied, this result is not surprising.

Influence of loudspeaker More than any other item, the choice of the loudspeaker system for exciting the auditorium can lead to considerable variations. For this study, we have used three units: two 'classical' loudspeaker boxes and one dodecahedron 18'

C88,

dB

5~

.................................... i

8 .......~

Loudspeaker Used:

~

~Eltpson

...................

-'-ElectroUoice

-5

.................................... i i 1 2 5 2 5 8 5 8 8 ;X ;X 4X Frequency, Hz

~ Sphere

Fig. 6. Comparison of C80 measurement made with three sound sources.

The variability of room acoustical parameters

185

built in our laboratory. Figure 6 shows a typical example of results for three measurements at one location with the three loudspeaker units. The same experiment was carried out in different halls and for many locations in each hall. The general results were found to be similar; parameters such as RT or EDT seem to be less affected than those based on energy ratios such as C80 or LE. Although it is difficult to find a relationship between the physical characteristics of a loudspeaker system and the parameters measured, it seems likely that these variations can be explained by the different directivities of the loudspeakers. Consequently, to be comparable, measurements made by different groups should have, at least, been taken with the same sound source. Moreover, the variations between two locations within one auditorium are created not only by the effective acoustical differences between the two seats but also by directivity effects of the sound source.

Small variations of the source-microphone position This part of the study was intended to determine the effects of small variations of the source or the microphone position on parameters, to find how accurately one can characterise a location by means of objective parameters. Figures 7 and 8 present two examples of results obtained with the microphone moved in steps of 10cm from the reference position. The same kind of study had been carried out in four halls. In all cases, a 30-cm displacement leads to measurable variations for all parameters. As stated above, mean results do not seem to be dependent on the hall in which they were carried out. Only displacement larger than 50 cm seemed to lead to different behaviour of the parameters depending on the auditorium. Table 3 presents the standard deviation of five acoustic parameters averaged over four halls for a 30-cm displacement of the microphone in every direction. 51 C88,

•o .

lererence

4.

10

d2 c..

8

1 2 5 2 5 0 5 0 0 I]( Frequency.

2](

4"20

cm

"e-3O

am

-~50

cm

4](

Hz

Fig. 7. Variation of C80 when the microphone is moved.

186

Xavier Pelorson, Jean-Paul Vian, Jean-Dominique Polack

.u- R e f e r e n c e

.L 1 0 cm

1 2 5 2 5 0 5 8 6 11( Frequency, Fig. 8.

21( Hz

4"ZO

cm

-°-30

cm

"~50

c~

4](

Variation of early decay time when the microphone is moved.

Similar tests have been performed by Bradley and HalliwelP (in the Opera of the National Arts Center in Ottawa). Although they used a totally different measurement technique (a revolver as sound source), it is interesting to notice how close their results are to ours. TABLE 3

Standard Deviations of Parameters for a 30-cm Displacement Octave band

RT (s) EDT (s) C80 (dB) LE (%) G (dB)

125Hz

250Hz

500Hz

l kHz

2 kHz

4 kHz

0-08 0"8 0"8 8.6 0.7

0.05 0.09 0"9 8.6 0.6

0.08 0"15 0-6 4.1 0-6

0-11 0.13 0.5 4,6 0-7

0-08 0.14 0.4 4.2 0.6

0-06 0.05 1.3 6.6 0.8

Results are averaged from 12 locations in four halls (except for LE and G, where five locations in two halls were used).

Variations within a hall

Last, but not least, Table 4 presents some statistics on measurements made in 11 halls (one with four different conditions). LE was measured in only five halls, G in 10 halls. As a measure of dispersion we consider two quantities: o is the standard deviation of the measured values and 6 is the mean range value (difference between the lowest and the highest value averaged over all octave bands). As variations with frequency remained weak (standard deviations were found to be slightly larger for the lowest octave bands), these results are averaged over six octave bands. In general, the overall variations

The variability of room acoustical parameters

187

TABLE 4

Mean Range Value 6 (Difference between the Lowest and Highest Value Averaged over all Octave Bands) and Standard Deviation 0 of Four Parameters Measured in 11 Halls (212 Locations)

(m 3)

Number of seats

EDT (s)

RT (s)

C80 (dB)

G (dB)

BSTI

21000

2700

BST2

21000

2700

BST3

21000

2700

BST4

21000

2700

TMP

18500

2300

TDA

9800

1500

TCE

7000

1400

OL

6500

1200

DAU

5200

976

SOV

3600

225

3 300

~ 100

LYR

3000

340

CMR

2400

421

OLM

2200

320

o=0"1 6=ff15 o=0"1 6=0-2 o=0-15 6=0"2 o=0"1 6=0"2 o=0-1 6=0"4 o=0"15 6=0"3 o=0"1 6=0-4 a=0"15 6=0.5 o=0"1 6=0"15 o=0-1 6=0.4 0 = 0"5 6=2-1 o=0.1 6=0.4 0=0"2 6=0"8 0=0-2 6=0"4

0=2 6=8 0=2 6=9 0=3 6=8 0=2 6=10 0=2-5 6=9 0=3 6=10 0=2"5 6=9 0=2 6=10 0=2 6=10 o=1"7 6=7 0 = 2"5 6=14 o=1-5 6=6 0=3 6=11 0=2 6=8

0=2 6=4"5 0=2 6=5 o=1"5 6=4 0=2 6=5 o=1.9 6=9 0=2"5 6=6 0=2"5 6=11 o=1.5 6=3-5 0=3 6=6 o=1.8 6=7

POV

0=0-3 6=1"0 0=0"3 6=0"8 0=0"3 6=1-1 0=0"3 6=1-0 0=0"25 6=0.9 0=0"3 6=0"9 0=0"2 6=0"8 0=0.2 6=0.9 0=0"2 6=0"8 0=0-2 6=0-7 o = 0.5 6=2-0 o=0"15 6=0-5 0=0.3 6=1-I 0=0"3 6=0"7

Volume

0=0.9 6=5

LE (%)

0=9 6=48

o=12 6=55

0=10 6=40

o=10 6=37

0=2 6=3 0=2 6=6

o=12 6=36

on parameters depend on neither the volume nor the shape of the hall, yet the hall with the largest variations (POV) is the one which has probably the worst acoustical quality. In this extreme case, the variability of parameters seems to be an indicator of a lack of quality. RT and LE overall variations are small compared with the effects of small displacements, as shown in Figs 9 and 10. Thus, in many cases, these parameters will not be suitable for describing positional differences within a hall. On the other hand, the considerable range of C80 values raises the question of the relevance of that parameter. For instance, it seems impvssible, unless a high number of

Xavier Pelorson, Jean-Paul Vian, Jean-Dominique Polack

188

S4La'ndard D e v i a t i o n duo %o: 0.5

x

0.4

I Rop~oduclbillty

8.3

l ~ l V a r l a t l o n o£ Nlc~ophone Poeltion ~30 c~

0,2

~Average Deviation

w i t h i n H a l l s (15) 0

12S ~

~ ~ 21( Fs'equency, H=

4X

Mean overall standard deviation of RT compared with reproducibility and -t- 30 cm displacement of the microphone.

Fig. 9.

Standar.d D e v i a t i o n duo %o: 15 Sigma

f

I Roproducibllity [] U a ~ i a t l o n of HicPophono P o e i t ion 2~

c**

Avorago l k s v i a t ion w i t h i n H a l l 8 (15) 12~ 258 5 0 0 1]( 21( Fvoquencg. Hz Fig. 10.

4K

Mean overall standard deviation of LE compared with reproducibility and +_30 cm displacement of the microphone.

measurement locations is used, to have a sufficiently accurate C80 mean value to compare halls. In addition, can such fluctuations be relevant in comparison with the moderately low variation of the subjective judgements of clarity? STATISTICAL VALIDITY Purpose From our previous work, we have now a good idea of the accuracy one can expect from objective auditorium acoustical parameters. According to this,

The variability of room acoustical parameters

189

we can anticipate how, even with a directional sound source, to differentiate one auditorium from another using combinations of parameters measured at given locations. This is often done in a very simple way by considering mean values and dispersions of parameters in each hall. To quantify these differences several parametric tests can be performedJ a. However, considering the extensive results obtained in several halls, we found situations where such an analysis failed. As an example, we will consider the results for two halls. The first hall (BST1) is a large opera house (21 000m a, 2700 seats) and the second (OLM) is a small auditorium (2200 m 3, 320 seats). Although both of them have a fairly good acoustical reputation, there is no doubt that the subjective impressions in those two halls are different, yet we found it impossible, despite many measurements (up to 75), to differentiate these halls using only mean parameter values and dispersion of results. Figures 11-13 are three examples of results. In such a situation it is not surprising that all statistical tests performed could not differentiate one hall from another. Both halls have very close mean RT values. As the fluctuations of RT within a hall are very small, RT and even EDT measurements cannot differentiate these two halls. On the other hand, because of the high dispersion of energy parameter values in both halls there is a strong overlap between measurements, which makes differentiation impossible. In fact, very little is known about the distribution of the criteria within halls. Important work has been done on the relationship between objective parameters and some architectural design variables or the objective

3-

EDT •

2.5" 2

[]

BST

-*- OLM

1.5 1 8,5'

e

I

125

I

250

I

500

Frequencyj

Fig. 11.

I

lk

I

2k

I

4k

fix

Mean EDT values and dispersion ~ r the halls BST1 and OLM.

190

Xavier Pelorson, Jean-Paul Vian, Jean-Dominique Polack

C88, dB

5.

-'~

BST

--- OLM

8-

-5-

I

125

I

258

I

I

588

Frequency, Fig. 12.

I

lk

I

2k

4k

H:

Mean C80 values and dispersion for the halls BSTI and OLM.

parameters' mutual connections. Implicitly, as m a n y workers use statistical tests such as correlations or factorial analysis, they are assuming the distribution o f parameters to be normal. Unfortunately, no study has been done on the justification for such an assumption. Therefore, in the following section we will try to evaluate the goodness o f fit o f the measured parameter distribution with a theoretical distribution. The first model to be tested is, of course, the normality: we will then try to evaluate the stability o f the parameter distribution. -28 G, dB

-25 ¸

-~- BST -~- OLM

-38 t

m

-35

o

I

-48 125

I

258

I

588

I

lk

I

2k

I

4k

Frequencym Hz Fig. 13.

Mean G values and dispersion for the halls BSTI and OLM.

The variability of room acoustical parameters

191

Statistical tests

Several tests can be performed to evaluate the goodness of fit of a sample.19 The classical test is the X2, test, which has many advantages but also some drawbacks, such as the need to distribute sample values into cells. One can easily show that g 2 test results are highly influenced by the extreme cells of the distribution, which here contain the smallest number of values. We have therefore chosen more recent tests based on the empirical distribution function (EDF statistics)fl ° The procedure is based on comparison of the EDF and a theoretical distribution function. The null hypothesis can be expressed as: (Ho): The distribution of parameters measured in a hall at a sample of locations does not differ from the theoretical distribution (i.e. differences between the experimental distribution and the theoretical distribution are purely random). Given n random standard samples (measurements): "YI -~X2 -~ "'" -~"Yn

we let F(x~) be the theoretical distribution function. The statistics are calculated from: (1) Kolmogorov-Smirnov statistic:

D + = max f ( i ~ l~.i<..Lkn/

F(x,)]

D - - = max [F(xi)~'<"L,

(/--1)In

D = max(D+, D-) (2) Cramer-Von Mises statistic: n

Z( w2 =

(2i-1)2~ 1 ~n j 4- 12---n

F(x,)

i=l

(3) Kuiper statistic:

V=D+

+D-

(4) Watson statistic:

U 2 = W 2 -- n F where n

F(x,) n

Xavier Pelorson, Jean-Paul Vian, Jean-Dominique Polack

192

(5) Anderson-Darling statistic: n

A2_ i=

~1 (2i - 1)[In (F(xi)) + In (1 -- F(x, +1 -i)] --n

Using appropriate tables one can reject or not reject the hypothesis H0. One must also choose a level of significance a. This value indicates the probability of making a Type 1 error (rejecting Ho when it is in fact true). We have here chosen a level of ~ = 0-05.

Main results Halls under analysis Five halls were chosen for this analysis: - - ' O L M ' is a small parallelepipedic concert hall, with a volume of about 2200 m a and 320 seats; --'SOV' is a conic-shaped organ auditorium, with a volume of about 3600 m a and 225 seats; - - ' L Y R ' is another small concert hall, of roughly paraUelepipedic shape, with a volume of about 3000m 3 and 340 seats; - - ' T C E ' and 'TMP' are two lyrical concert halls, both of typical Italianstyle horseshoe shape; the volume of T C E is about 7000 m a with 1400 seats whereas T M P has a volume of 18 500m 3 with 2300 seats. The choice of these auditoria was made because of their size and geometry. As we have shown, to measure such fluctuating parameters as C80 or G requires many measurements.

Measurement conditions All parameters were evaluated per octave bands (125 Hz--4 kHz) as described in the section on measurement technique. The measuring system (microphone, loudspeaker box, etc.) was exactly the same for all measuring sessions, and in all the cases the halls were empty. Measurement positions In OLM, measurement positions were chosen throughout the audience area, whereas in the other four halls, because oftheir symmetrical shape, only onehalf of the audience area was considered. As it is an important assumption for the validity of E D F tests, measurement positions were randomly chosen by a Monte Carlo method (40-75 positions). Depending on the size of the hall, one or two sound source positions on the stage were chosen, each on the hall axis.

193

The variability of room acoustical parameters T e s t s for n o r m a l i t y

The theoretical distribution function is _

1

F(x,) - ~ . j

I _x`oo exp ( ---~-) dt

using rational approximation, one can express F(x~) as F ( x i ) = 1 - 1 ( 1 + a l x + a2 x2 + a a x 3 + a4x*) - 4 + O(x) L

with a I =0-196854 a 2 =0"115 194

a3 = 0.000 344 a4 = 0-019 527

and 10(x)l < 2.5 x 10 -4, which is sufficiently accurate for our study. Five E D F statistics were c o m p u t e d for each of the six octave bands and for five parameters. Table 5 presents one result for TMP. In this example, the critical value for the K o l m o g o r o v - S m i r n o v test is 0.895. Thus one must reject the hypothesis Ho of normality for LE at all frequencies, for R T at 2 k H z and for C80 at 4 kHz. Table 6 summarizes the results for all situations and all statistics. As examples, Figs 14 and 15 show two typical situations. It should be noted that all tests do not give the same conclusions. This is not surprising, because these tests do not have the same power; this depends on the size or the type o f sample under analysis. One m u s t also keep in m i n d that, as for all statistical tests, not rejecting Ho does not m e a n that H o is true. O f course, slight changes would occur if a different level of significance had been chosen, but the general results would remain the same: - - t h e r e is no reason to reject the hypothesis of normality of EDT; - - t h e LE distribution at all frequencies seems far from a normal one; - - C 8 0 , G and even R T at some frequencies c a n n o t be considered as normally distributed. TABLE 5 Results from Kolmogorov-Smirnov Statistics D in TMP (Bold Numbers are Values of D Higher than 0-895) Octave band

RT

EDT C80 LE G

125 Hz

250 Hz

500 Hz

l kHz

2 kHz

4 kHz

0-85 0-49 0-54 1"41 0"80

0-55 0-63 0"53 1-39 0"49

0-44 0-58 0-63 1"38 0-72

0-56

0"96

1"32

0-55 0"73 1-34 0-73

0-55 0-54 1"39 0-70

0-71 0"95 1-37 0-44

Xavier Pelorson, Jean.Paul Vian, JeanDominique Polack

194

TABLE 6 General Results from EDF Statistics: Percentage of Tests that Led to Rejection of H o with a Level of 0-05

Octave band

RT EDT C80 LE G

125 Hz

250 Hz

500 Hz

l kHz

0 0 4 75 15

24 0 16 75 0

28 0 0 50 0

8 0 0 85 70

2 kHz

4 kHz

8 0 12.5 85 20

20 4 44 60 25

These results show the limitations of many past studies, and confirm how difficult it is to compare two halls using parametric tests based on the comparison of mean values and standard deviations of parameters, as all these tests are based on the same assumption of normality. Furthermore, our results show the limitations of many statistical studies based on objective parameters (factorial analysis, correlations, etc.), as once again normality is one of the basic assumptions. Tests for a constant distribution The aim of the following tests is slightly different from that in the preceding section: we wish to find whether, regardless of the distribution of parameters within a hall, this distribution is the same for other halls. The purpose of this part of the study is not to compare the distribution of measured parameters 1.0 ¢:

0"8

.

/

f

0.6

0-4 _

E O-2

xperlmen~l results

~

~

_ Tlleoretical

U

model

0 1.0

rl

1.2

1:3

1:4

l:s

1.6

EDT, s Fig. 14. Result of Kolmogorov-Smirnov test for EDT at 1 kHz in OLM (no significant difference).

The variability of room acoustical parameters 1.o

Y

/

0-8

Ig a" 0

0.6

.> ...y

E u 0.2

195

- Experimental results --Theoretical model

C80. dB Fig. 15.

Result of Kolmogorov-Smirnov test for C80 at 4 kHz in OLM ((Ho) rejected).

with any analytical theoretical distribution function F(x~) but to compare the distribution functions of measured paameters between several halls. For instance, five cumulative distribution functions of C80 at 4 kHz are shown in Fig. 16. E D F statistics were processed to quantify the differences between the various distributions of parameters. One example result, comparing TMP and TCE, is presented in Table 7. Results for other halls and statistics show the same conclusion: - - T h e fit between measured distributions is much better than with a normal distribution. For instance, no significant differences were found

"'' f

F

i

U

I

8.8

i ve

8.7 0.6[

F

8,5[

re

8'4 t

e n c Y

B,2

_ ~ r .dr

~

"+" LYR

*SOU "~ TCE

~ /

8.11 i~

i

-3

-i

-Z

i

.....

I

I

I

-1 9 1 2 3 S t a n d a r d i z e d CO8 (4 kHz) Fig. 16. Comparison of five empirical distribution functions of C80 (standardized) at 4 kHz.

196

Xavier Pelorson, Jean-Paul Vian, Jean-Dominique Polack TABLE 7 Results of Kolmogorov-Smirnov Statistics for the Comparison of the Distribution of Parameters in TMP and SOV (Bold Numbers are Values Higher than the Critical Value (1-22)) Octave band

RT EDT C80 LE G

125 Hz

250 Hz

500 Hz

l kHz

2 kHz

4 kHz

1"91 0"70 0"48 1-05 0"81

4"44 0"59 0"98 I'06 0"81

1-07 0"35 0"72 1"72 1"08

0-75 0"64 0'83 1"88 1"05

0'74 0"45 1"03 1"82 0"78

1"51 0"06 0'75 1"57 0"97

between C80 distributions, and in only one case (comparison between L Y R and T M P at 500 Hz) was behaviour of G found to differ from one hall to another. --Distributions of E D T cannot be compared with those of energy ratio parameters. Indeed, the E D T distribution within a hall seems close to a normal one, but we have shown that the C80 distribution is certainly not normal. - - T w o parameters, RT and LE, still show irregular distributions at some frequencies. Conclusions A m o n g the five parameters considered, only E D T was found to have a distribution close to normality. A more detailed study showed that, in fact, two parameters seemed to follow other kinds of distribution laws. Thus, comparisons between halls could be possible using these three parameters. However, problems still occur with two parameters, RT and LE. This confirms the conclusion reached in the section on small variations of the source-microphone position: these two parameters are not suitable for describing spatial variation comparisons. Of course, these results need to be confirmed by further analysis in various halls.

CONCLUSION In this paper, we have shown the difficulties that arise when attempting to assess the acoustical quality of an auditorium by means of objective measurements and parameters. In the first part, we tried to evaluate the accuracy one can expect from parameter measurements. The influence of

The variability of room acoustical parameters

197

each element of the measuring system was evaluated. We found that energy ratio parameters, such as clarity index and lateral efficiency, depend strongly on the sound source, as the results vary considerably depending on the loudspeaker system used. A statistical study, based on many measurements in five halls with a directional loudspeaker, showed that, except for the early decay time, EDT, the distribution of parameters within a hall cannot be considered as normal. However, in addition to EDT, the clarity index, C80, and strength index, G, although they are difficult to measure accurately, seem to have a sufficiently regular distribution to be used for comparing one hall to another. Finally, no specific distribution within a hall can be assessed for the lateral efficiency, LE, and reverberation time, RT. This is not surprising, as we showed that these parameters generally cannot describe spatial variations within a hall. Therefore, conclusions drawn from this work are likely to be pessimistic; only three parameters seemed to be statistically relevant for a room acoustical quality analysis including seat to seat variations. Of course, this does not mean that RT and LE are totally useless; they were found to reflect only the hall itself but not the spatial variations within the hall. As preference studies have shown this number of parameters to be insufficient, there is an urgent need for better parameters, or at least better definitions of existing parameters.

REFERENCES 1. Jordan, V. L., A group of objective acoustical criteria for concert halls. Applied Acoustics, 14 (1981) 253-66. 2. Jullien, J. P., Correlation among objective criteria of room acoustic quality. Proc. 12th ICA, Toronto. Beauregard Press Ltd, Canada, 1986, E4-9. 3. Schroeder, M. R., New method of measuring reverberation time. J. Acoust. Soc. Am., 37 (1965) 409-12. 4. Bradley, J. S. & Halliwell, R. E., Accuracy and reproducibility of auditorium acoustics measures. PROC. IOA 10, Part 2. Institute of Acoustics, Edinburgh, 1988, pp. 399-406. 5. Lehmann, P. & Wilkens, H., Zusammenhang subjektiver Beurteilungen von Konzersaelen mit raumakustichen Kriterien. Acustica, 45 (1980) 256-68. 6. Cremer, L. & Muller, H. A., Principles and Applications of Room Acoustics, transl. T. J. Schultz. Applied Science Publishers, London, 1982. 7. Barron, M., Subjective study of British symphony concert halls. Acustica, 66 (1988) 1-14. 8. Jordan, V. L., Room acoustics and architectural development in recent years. Applied Acoustics, 2 (1969) 59-81. 9. Reichardt, W., Alim, D. A. & Schmidt, W., Definition und Messgrundlungen eines objektiven Masses zur Ermittlung der Grenze zwischen brauchbarer und

198

10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20.

Xavier Pelorson, Jean-Paul Vian, Jean-Dominique Polack unbrauchbarer Durchsichtigkeit bei Musikdarbeitungen. Acustica, 32 (1975) 126-37. Thiele, R., Richtungsverteilung und Zeitfolge der Schallriickwiirfe in R~iumen. Acustica, 3 (1953) 100-12. Lochner, J. P. A. & Burger, J. F., The intelligibility of speech under reverberant conditions. Acustica, 11 (1961) 195-200. Kiirer, R., Zur Gewinnung von Einzahlkriterien bei Impulsmessung in der Raumakustik. Acustica, 21 (1969) 370-2. Barron, M. & Lee, L. J., Energy relations in concert auditoriums I. J. Acoust. Soc. Am., 84(2) (1988) 618-28. Steeneken, H. J. M. & Houtgast, T., A physical method for measuring speechtransmission quality. J. Acoust. Soc. Am., 67(1) (1980) 318-26. Jordan, V., Acoustical Design of Concert Halls and Theatres. Applied Science Publishers, London, 1980. Borish, J. & Angel, J. B., An efficient algorithm for measuring the impulse response using pseudorandom noise. J. Audio. Eng. Soc., 31(7) (1983) 478-87. ISO 5725-1986, Precision of test methods--Determination of repeatability and reproducibility for a standard test method by inter-laboratory tests. International Organization for standardization, Geneva. Kendall, M. G. & Stuart, A., The Advanced Theory of Statistics. Charles W. Griffin, London, 1961. Shapiro, S. S., Wiik, H. B. &Chen, H. J., A comparative study of various tests for normality. J. Am. Stat. Ass., 63 (1968) 1343-72. Stephens, M. A., EDF statistics for the goodness of fit and some comparisons. J. Am. Stat. Ass., 69 (1974) 730-7.