- Email: [email protected]
Numerical comparison of sampling strategies for BRDF data manifolds
Accepted Manuscript Numerical comparison of sampling strategies for BRDF data manifolds Mikhail Langovoy, Franko Schmähling, Gerd Wübbeler PII: DOI: Reference:
S0263-2241(16)30471-7 http://dx.doi.org/10.1016/j.measurement.2016.08.010 MEASUR 4288
To appear in:
Measurement
Received Date: Revised Date: Accepted Date:
6 April 2016 4 August 2016 11 August 2016
Please cite this article as: M. Langovoy, F. Schmähling, G. Wübbeler, Numerical comparison of sampling strategies for BRDF data manifolds, Measurement (2016), doi: http://dx.doi.org/10.1016/j.measurement.2016.08.010
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Numerical comparison of sampling strategies for BRDF data manifolds Mikhail Langovoy∗ Physikalisch-Technische Bundesanstalt, Abbestrasse 2-12, D-10587, Berlin, Germany. e-mail: [email protected]
and Franko Schm¨ ahling Physikalisch-Technische Bundesanstalt, Abbestrasse 2-12, D-10587, Berlin, Germany. e-mail: [email protected]
and Gerd W¨ ubbeler Physikalisch-Technische Bundesanstalt, Abbestrasse 2-12, D-10587, Berlin, Germany. e-mail: [email protected] Abstract: The bidirectional reflectance distribution function (BRDF) is one of the fundamental concepts in such diverse fields as multidimensional reflectometry, computer graphics and computer vision. BRDF manifolds form an infinite-dimensional space, but typically the available measurements are very scarce. Therefore, an efficient learning strategy is crucial when performing the measurements. In this paper, we perform simulation studies within a mathematical framework that allows to establish more efficient BRDF sampling and measurement strategies in the sense of statistical design of experiments and generalized proactive learning. Our simulation studies suggest that the default BRDF measurement strategy is suboptimal for a wide class of loss functions. Keywords and phrases: Design of experiments, active learning, BRDF, diffuse reflection, specular reflection, computer graphics, computer vision, metrology, data analysis, statistics of manifolds, distance on the space of BRDFs.
∗ Corresponding
author.
1 imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
2
1. Introduction For many applications, the appearance of real-world surfaces is sufficiently well characterized by the bidirectional reflectance distribution function (BRDF). In the case of a fixed wavelength, BRDF describes reflected light as a fourdimensional function of incoming and outgoing light directions. In computer graphics and computer vision, usually either physically inspired analytic reflectance models, like [3], or parametric reflectance models chosen via qualitative criteria, like [11] or [10], to name a few, are used to model BRDFs. In multidimensional reflectometry, an alternative approach is usually taken. One directly measures values of the BRDF for different combinations of the incoming and outgoing angles and then fits the measured data to a selected analytic model using optimization techniques. In this paper, following the path started in [15], [16], we treat BRDF measurements as samples of points from a high-dimensional and highly non-linear non-convex manifold. 2. Active learning and design of experiments In general, BRDF is a 5-dimensional manifold, having 4 angular and 1 wavelength dimension. Note that even a set of 1-dimensional manifolds is infinitedimensional (and k-dimensional manifolds are not to be confused with parametric k-dimensional families of functions). At the same time, a typical measuring device only takes between 50 and 1000 points for all the BRDF layers together. In view of this, the available measurement points are indeed very scarce for a complicated problem such as BRDF estimation. Therefore, an efficient sampling strategy is required when performing the measurements. The angular positions of the light source and the sensor utilized within the measurement process have to be chosen to maximize the information contained in the output. Since sets of BRDF measurements are, in fact, observed random manifolds, we are dealing here with manifold-valued data. Standard statistical and machine learning methods can not be safely directly applied to BRDF data. Statistical design of experiments (see [6], [4]) is a well-developed area of quantitative data analysis. Generally speaking, our task fits the framework of the statistical design of experiments. However, previous research in this field was often more concerned with (important) topics such as manipulation checks, interactions between factors, delayed effects, repeatability, among many others. This shifted the focus away from considering design of statistical experiments on structured, constrained, or infinite-dimensional data. In contrast to much of the previous work on the design of experiments, BRDF measurements are carried out in strictly defined settings (run by qualified experts and in a way that is precisely known, reproducible, and determined by the physical and technical characteristics of the gonioreflectometer). Therefore, there is less room for human or random errors and influences. On the other hand, BRDF measurements are collections of points representing manifolds, so defining even the simplest statistical quantities in this case turns out to be a nontrivial and conceptual task. Overall, our methodology developed in [17], [18] represents a far-reaching generalization of the active machine learning framework, also generalizing the
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
3
proactive learning setup of [5]. To be more specific, active learning, as a special case of semi-supervised machine learning, oftentimes deals with finite sets of labels and aims at solving classification or clustering problems with a finite number of classes. While there have been a number of promising practical applications, most of the existing theory deals with analysis of performance of specific algorithms. In this paper, we give an illustrative example for results of [18], where new techniques for the design of experiments and active learning were developed, in order to establish more efficient sampling strategies. To achieve this, we extend and apply the virtual gonioreflectometer simulation software developed in [21] to assess different sets of measurement geometries. 3. BRDF definition In the most basic case, the bidirectional reflectance distribution function (BRDF), fr (ωi , ωr )) is a four-dimensional function that defines how light is reflected at an opaque surface. The function takes a negative incoming light direction, ωi , and outgoing direction, ωr , both defined with respect to the surface normal n, and returns the ratio of reflected radiance exiting along ωr to the irradiance incident on the surface from direction ωi . Each direction ω is itself parameterized by azimuth angle φ and zenith angle θ, therefore the BRDF as a whole is 4-dimensional. The BRDF has units sr−1 , with steradians (sr) being a unit of solid angle. The BRDF was first defined by Nicodemus in [25]. The defining equation is:
fr (ωi , ωr ) =
d Lr (ωr ) d Lr (ωr ) = . d Ei (ωi ) Li (ωi ) cos θi d ωi
(1)
where L is radiance, or power per unit solid-angle-in-the-direction-of-a-ray per unit projected-area-perpendicular-to-the-ray, E is irradiance, or power per unit surface area, and θi is the angle between ωi and the surface normal, n. The index i indicates incident light, whereas the index r indicates reflected light. 4. Statistical analysis of BRDF models A useful data analysis procedure for any BRDF model has to exhibit certain robustness against outliers and dependent or mixed errors. Our study of statistical estimation for BRDF models in [16], [15] clarified certain pitfalls in the analysis of BRDF data, and helped to develop more refined estimates for more realistic BRDF models. Suppose we have measurements of a BRDF available for the set of incoming angles (p) Pinc (p) (p) Pinc Ωinc = ωi p=1 = θi , ϕi . p=1
(2)
Here Pinc ≥ 1 is the total number of incoming angles where the measurements (p) were taken. Say that for an incoming angle ωi we have measurements available for angles from the set of reflection angles
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
Ωref l =
P[ inc
4
Ωref l (p) ,
(3)
p=1
where Ωref l (p) =
ωr(q)
Pref l (p)
=
q=1
θr(q) , ϕ(q) r
Pref l (p) , q=1
Pinc where Pref l (p) p=1 are (possibly different) numbers of measurements taken for corresponding incoming angles. Overall, we have the set of random observations
(p) (p) (p) (p) θ ,ϕ f θi , ϕi , θr(q) , ϕ(q) ∈ Ωinc , θr(q) , ϕ(q) ∈ Ωref l (p) r r i i
. (4)
Our aim is to infer properties of the BRDF funtion (1) from the set of observations (4). In general, the connection between the true BRDF and its measurements is described via a stochastic transformation T , i.e. f (ωi , ωr ) = T fr (ωi , ωr ) ,
(5)
where T : M × P × F 4 → F4 ,
(6)
with M = (M, A, µ) is an (unknown) measurable space, P = (Π, P, P) is an unknown probability space, F4 is the space of all Helmholtz-invariant energy preserving 4-dimensional BRDFs, and F4 is the set of all functions of 4 arguments on the 3-dimensional unit sphere S 3 in R4 . Equations (5) and (6) mean that there could be errors of both stochastic or non-stochastic origin. In this setting, the problem of inferring the BRDF can be seen as a statistical inverse problem (see also [14]). We refer to [16] for a more detailed discussion of this connection. 5. Foundations of sensitivity analysis for BRDF models There are many measures of distance and quasi-distance used in mathematics and statistics: Lp , 1 ≤ p < +∞, L∞ , Sobolev distances [28], Kullback-Leibler information divergence [9], chi-squared distance used in correspondence analysis [13], [12]. It is known from both mathematics and statistics that not all distances are equally appropriate in all applications. The standard L2 -norm for BRDFs f1 = f1 (ωi , ωr ), f2 = f2 (ωi , ωr ) is defined via L22 (f1 , f2 ) =
Z Z Z Z
(f1 − f2 )(θi , φi , θr , φr ) 2 dθi dφi dθr dφr ,
(7)
Ω
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
5
where Ω is the set containing all possible values of the four parameters. In [22] and [23], we developed a perception-based quasi-metric for BRDFs. Our main idea can be formulated as follows:
a ”distance” d between BRDFs is physically reasonable if and only if d(f1 , f2 ) is small if and only if two balls illuminated according to f1 and f2 would look similar to a human observer .
To be more precise: in the above statement, we consider two balls of the same size that are made of materials with BRDFs f1 and f2 respectively. Both balls are illuminated with a point source of light. 6. Measure of distance between BRDFs The following decomposition is typically used in BRDF studies. The main idea of this decomposition is to (roughly) separate diffuse and specular parts of each BRDF, in order to treat each part with a more appropriate technique. BRDF (θi , ϕi , θr , ϕr ) = fdif f (θi , ϕi , θr , ϕr ) + fspec (θi , ϕi , θr , ϕr ) .
(8)
Therefore, for each BRDF f ∈ F4 we have a decomposition of the parameter space Ω Ω = Ωdif f (f ) ∪ Ωspec (f ) ,
µΩ Ωdif f (f ) ∩ Ωspec (f ) = 0 .
(9)
As before, we denote ωi = (θi , ϕi ), ωr = (θr , ϕr ), ω = (ωi , ωr ). Put µdif f (f ) = µBRDF (f )
, Ωdif f (f )
µspec (f ) = µBRDF (f )
.
(10)
Ωspec (f )
An important point is that µdif f and µspec could and often should be different. We refer to [22] and [23] for details on the psychophysical interpretation of these decompositions. In this paper we use the following rule to define a ”specular” and a ”diffuse” part of the BRDF. Definition 1. Ωspec (f ) = {ω | f (ω) ≥ ∆1 }
(11)
Ωdif f (f ) = Ω \ Ωspec (f )
(12)
For a data-driven automated choice of ∆1 , it is also possible to use spatial scan estimators introduced in [19] and [20]. We use in this paper a useful simplification of the quasi-distance from [22]. imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
6
Definition 2. Let f1 , f2 be BRDFs. Define the element of distance between f1 and f2 in F4 at a point ω as
ρBRDF (f1 , f2 )(ω) =
ρdif f (f1 , f2 )(ω), ω ∈ Ωdif f (f1 ) ∩ Ωdif f (f2 ); ρspec (f1 , f2 )(ω), ω ∈ Ωspec (f1 ) ∩ Ωspec (f2 )∩ {ω | ρspec (f1 , f2 )(ω) < ∆1 }; ∆2 , ∆2 ,
ω ∈ Ωspec (f1 ) ∩ Ωspec (f2 )∩ {ω | ρspec (f1 , f2 )(ω) ≥ ∆1 };
(13)
ω ∈ (Ωspec (f1 ) ∩ Ωdif f (f2 )) ∪(Ωdif f (f1 ) ∩ Ωspec (f2 )) .
Here ∆1 and ∆2 are pre-selected constants, with ∆2 penalizing missing or added peaks and smoothing out the influence of large peaks and of gross errors in measurements of peaks. ∆1 determines the level where we switch the regime in our analysis. We noticed that, for actual BRDF measurements, it is often reasonable to pick ∆1 ≈ 1. The choice of ∆2 depends on how strongly one aims to penalize for missing peaks or added peaks. In practice, it makes sense to set ∆2 to be no greater than the upper value that can be reliably measured by the particular gonioreflectometer. Often, 1 ≤ ∆2 ≤ 3 were reasonable choices. A value of ∆2 close to 0 would have an effect on the metric similar to attributing very low value to the peaks. For a data-driven automated choice of ∆1 and ∆2 , it is possible to use spatial scan estimators introduced in [19] and [20]. Definition 3. We define the metric (distance) µBRDF on F4 as follows. For any BRDFs f1 , f2 ∈ F4 , set Z µBRDF (f1 , f2 ) =
ρBRDF (f1 , f2 )(ω) dµΩ (ω) .
(14)
Ω
An important example is given by Lp -norms. If µdif f = Lpp , then ρdif f (x, y) = |x − y|p . As µdif f we can reasonably use any Lp -based measure. It is possible to use an Lp1 -based measure µspec , possibly for some p1 6= p. A number of truncated measures has been considered in robust statistics. A proper choice of parameters ∆1 , ∆2 is closely related to the model selection problem, see [14]. The above expression for µBRDF can be computed explicitly in special cases, for example, when both BRDFs are known explicitly beforehand, either analytically or in a parametric form. To account for applications to real data, we introduced in [22] the following approximate empirical quasi-distance between BRDFs. Definition 4. Suppose that BRDFs f1 and f2 were measured on the following collection of points: Ωmeas =
(p) (p) (p) (p) (q) (q) θi , ϕi , θr(q) , ϕ(q) θ , ϕ ∈ Ω , θ , ϕ ∈ Ω (p) . inc ref l r r r i i
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
7
(15) We define the empirical metric (distance) µ bBRDF on F4 as follows. For any BRDFs f1 , f2 ∈ F4 , that are both measured on Ωmeas , set µΩ (Ω) µ bBRDF (f1 , f2 ) = Ωmeas
X
ρBRDF (f1 , f2 )(ω) .
(16)
ω∈Ωmeas
7. Active manifold learning strategies In BRDF sampling, the equispaced-angular grid pictured in Figure 1 is the standard (see, e.g., [8]). However, as we show in this paper, this choice of measurement points leads to inefficient sampling.
Fig 1: Standard grid, inefficient sampling Uniformly distributed points on a sphere look very different, see Figure 2. Since it was already understood in the community (see, for example, [8]) that the standard grid is suboptimal, there were multiple heuristic attempts to propose trickier grids that better reflect the typical structure of BRDF models. A good example is shown in Figure 3. Any result showing that the new strategy is better than the default strategy, at least for one specific loss function, for one specific BRDF, and one specific estimating procedure, is already instrumental in understanding the general picture of learning BRDF manifolds from scarce expensive data. This basic case is straightforwardly formulated in the language of mathematical optimization, so we are able to obtain theoretical guarantees on learning accuracy, at least for some special cases. Let us recall the mathematical framework for active learning of BRDFs from [18]. Consider BRDF f ∈ F4 , where F4 is defined in Section 4. Suppose that f is measured on the finite set Ωmeas (n) (compare with (15)) imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
8
Fig 2: Uniformly distributed points on a sphere
Ωmeas (n) =
(p) (p) (p) (p) (q) (q) θi , ϕi , θr(q) , ϕ(q) θ , ϕ ∈ Ω , θ , ϕ ∈ Ω (p) , inc ref l r r r i i (17)
where P inc X Ωref l (p) . n = Ωmeas (n) =
(18)
p=1
Definition 5. Cost function Cost of a measurement configuration Ωmeas is a Ωmeas Lebesgue measurable function Cost : R → R+ . Let Dist be a function (measurable for a suitably chosen σ-algebra) such that Dist : F4 × F4 → R+ .
(19)
For our purposes, we typically like Dist to be inducing either a quasi-distance or a pseudo-distance on F40 , where F40 ⊆ F4 is a sufficiently reach subset. As an example, µBRDF from (16), Section 6, is often used in our practical experiments. Standard Lp -distances are easier for theoretical comparisons of efficiencies. Definition 6. Sampling strategy Ω is a sequence Ω = {Ωn0 }∞ n0 =1 where for each n0 there exists an integer n ≥ n0 such that Ωn0 = Ωmeas (n) for some measurement configuration Ωmeas (n) defined according to (17), and for any integers n1 ≤ n2 it holds that Ωn1 ≤ Ωn2 . Consider arbitrary fixed statistical estimator of BRDFs En : Rn → F4 .
(20)
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
9
Fig 3: Tricky grid, heuristic choice
Consider two sampling strategies: Ω1 , Ω2 . Suppose that both strategies have uniformly admissible costs (see [18]). The problem of generalized active learning for BRDF sampling can be stated in the following way. Find a sampling strategy ∞ Ωmeas = Ωmeas (n) n=nmin such that for all n ≥ nmin Ωmeas (n) = arg
min Ω : Cost(Ω)
Dist En (f ; Ω), f
(21)
Definition 7. Suppose f ∈ F4 is a particular (possibly unknown) BRDF. Let Ω1 , Ω2 sampling strategies be sampling strategies with Cmax -uniformly admissible costs. We say that strategy Ω1 is asymptotically more efficient for learning f than the strategy Ω2 , and write Ω1
lim sup n→∞
Dist En (f ; Ω1 (n)), f < 1. Dist En (f ; Ω2 (n)), f
(22)
Notice that, for the task of evaluating sampling quality, expected errors (over classes of BRDFs) are more interesting than maximal errors (over the same classes). A novel framework for treating this more general problem was developed in [17], [18]. 8. Simulation environment In analogy to the real experiment the computer model consists of three parts, the mechanical parts, the IxMD or array-spectroradiometer for spectrally resolved imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
10
measurements, the light source, which ensure a stable diffuse illumination of the sample, and of course the sample which reflectance behaviour we want to be characterized via the virtual experiment. The software consists of a set of interacting components, every part represents an entity with own methods and properties and in that way we covered the principle of modular software design [2], [24]. This approach allows us to construct measuring systems for various applications in a very flexible way [26]. Based on the generic library of the virtual near-field goniophotometer [27], we are able to model every part of the gonioreflectometer as a single object and finally integrated all objects together to build a computer model of the whole measurement system. Every object is linked to another object and the core procedure of every component is the calculation of the transformation matrix, which enables linked transformations between the objects [7], [1], [29]. To build the software counterpart for the real (world) gonioreflectometer we were able to use large parts of the generic library of the near-field goniophotometer in a straightforward manner or after minor changes. Especially for the mechanical parts, i.e. the parts of robot arm, the motion unit for the light source and the detector. In addition, the source code for the light source and the sample is deployed to fit to our application. In regard to the sample surface, at first, we are able to simulate the so-called goniochromatic behaviour, i.e. the situation where the colour is changing with the change of reflectance direction under the condition that the direction of incoming light is the same for all reflectance directions. The second option is to simulate gloss appearance via the Cook-Torrance BRDF model on the sample surface. The computer model should be as close as possible to the real experiment in order to be able to draw reliable conclusions from the virtual experiments. For computer graphics’ reflection models such as the Cook-Torrance model [3], there is a correspondence between the parameters of these models and the appearance measurement scale for gloss. The Cook-Torrance model is a further advancement of the specular–microfacet Torrance-Sparrow model, accounting for wavelength and thus color shifting. 9. Results of virtual experiments Virtual experiments, besides being cheap, offer a number of advantages in research. In particular, one can emulate and check specific measurement devices, even without building them physically. Cameleon 3.0 is the software that emulates the real-life robot-based gonioreflectometer developed and constructed in PTB [21]. To illustrate the point of this paper, we have chosen a standard CookTorrance model with the single specular peak at ϕ = 180◦ and θ = 45◦ . Apart from the random sampling, no measurement errors were introduced to this basic model, in order to focus on assessment of the relative efficiency of the sampling strategies. The four sampling strategies were chosen due to the following reasons. The standard equi-angular sampling, which is the default sampling method used in most of the labs that measure BRDFs with gonio-reflectometers. Uniform (random) sampling: applicable when no prior knowledge about the BRDF is available. Deterministic grid-sampling near the specular peak: this approach
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
11
appears to be reasonable when prior knowledge about the characteristics of the BRDF is available (e.g., for a glossy surface). Gaussian (random) sampling: similar to sampling near the specular peak, applicable if prior knowledge about the BRDF is available. BRDF sampling modeled with the equispaced-angular grid is pictured in Figure 4. As we show below, this choice of measurement points leads to inefficient sampling. BRDF sampling with uniformly distributed points on a sphere can be seen on Figure 5. Two of the more elaborated grids that better reflect the typical structure of BRDF models are shown in Figures 6 and 7.
Fig 4: Standard grid, inefficient sampling
Fig 5: Uniform random sampling
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
12
Fig 6: Sampling more in the region of the peak
Fig 7: Gaussian sampling concentrated at the peak The following table illustrates, on the basis of 100 repetitions, what are typical approximation errors made while following the above four different sampling strategies. The standard equi-angular sampling is clear inferior for both types of loss functions, especially for the perception-based µBRDF . We used the thresholds ∆1 = 1 and ∆2 = 0.1. The second value is a fairly arbitrary choice, but can be used when putting little weight on misplaced peaks is appropriate. This is the case in the present example, as we know in advance that the actual locations of peaks are in perfect alignment, and the only misalignment of peaks that we get is coming from interpolation errors (compare (11) and (13)). In this special case, a big ∆2 will lead to multiplying interpolation errors rather than to penalizing wrong locations of peaks. As we are not focusing on interpolation errors in this paper, we picked a small ∆2 to downplay their effect.
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
standard equi-angular sampling uniform sampling sampling near peak gaussian sampling
µBRDF 3.4093 3.3600 2.3526 2.2628
13
L2 0.1199 0.1102 0.0545 0.0583
Similar results occur for a variety of other simulated BRDFs. Based on our experience, the standard equi-angular sampling is inferior for several types of loss functions (even for those cases when penalties for misplaced peaks are small, like in the above example). For more accurate sampling, it is better to increase the number of measurements in specular regions, even though that reduces the number of measurements in the diffuse part. The explanation for this effect is that the diffuse part of the BRDF varies much slower and can be reconstructed on the basis of a substantially smaller amount of observations. 10. Conclusions BRDF manifolds form an infinite-dimensional space, but typically the available measurements are very scarce for complicated problems such as BRDF estimation. Therefore, an efficient sampling strategy is crucial when performing the measurements. In this paper, we extend and apply the virtual gonioreflectometer simulation software to assess different sets of measurement geometries. We show that the default BRDF measurement strategy is suboptimal for a variety of loss functions. Acknowledgments. The authors would like to thank Anna Langovaya and the two referees for valuable suggestions that helped to greatly improve the presentation of the paper. This work has been carried out within EMRP project IND 52 ’Multidimensional reflectometry for industry’. The EMRP is jointly funded by the EMRP participating countries within EURAMET and the European Union.
References [1] Arfken, G. B. and Weber, H. J. (2011). Mathematical methods for physicists: a comprehensive guide. Academic press. [2] Booch, G. (2006). Object-oriented analysis and design with applications. Pearson Education India. [3] Cook, R. L. and Torrance, K. E. (1981). A reflectance model for computer graphics. In ACM Siggraph Computer Graphics 15 307–316. ACM. [4] Cox, D. R. and Reid, N. (2000). The theory of the design of experiments. CRC Press. [5] Donmez, P. and Carbonell, J. G. (2008). Proactive learning: costsensitive active learning with multiple imperfect oracles. In Proceedings of the 17th ACM conference on Information and knowledge management 619–628. ACM. [6] Fisher, S. R. A., Genetiker, S., Fisher, R. A., Genetician, S., ´ ne ´ticien, S. (1960). The design of experiments Fisher, R. A. and Ge 12. Oliver and Boyd Edinburgh. imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
14
[7] Goldstein, H., Poole, C. P. and Safko, J. L. (2014). Classical Mechanics: Pearson New International Edition. Pearson Higher Ed. ¨ pe, A. and Hauer, K.-O. (2010). Three-dimensional appearance char[8] Ho acterization of diffuse standard reflection materials. Metrologia 47 295. [9] Kullback, S. and Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics 79–86. [10] Lafortune, E. P., Foo, S.-C., Torrance, K. E. and Greenberg, D. P. (1997). Non-linear approximation of reflectance functions. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques 117–126. ACM Press/Addison-Wesley Publishing Co. [11] Lambert, J. H. and Anding, E. (1892). Lamberts Photometrie: (Photometria, sive De mensura et gradibus luminis, colorum et umbrae) (1760). Ostwalds Klassiker der exakten Wissenschaften v. 1-2. W. Engelmann. [12] Langovaya, A. and Kuhnt, S. (2011). Correspondence analysis and moderate outliers. Agrocampus Ouest, Rennes FRANCE 19. [13] Langovaya, A., Kuhnt, S. and Chouikha, H. (2013). Correspondence Analysis in the Case of Outliers. In Classification and Data Mining 63–70. Springer. [14] Langovoy, M. (2007). Data-driven goodness-of-fit tests. University of G¨ ottingen, G¨ ottingen. Ph.D. thesis. [15] Langovoy, M. (2015). Machine Learning and Statistical Analysis for BRDF Data from Computer Graphics and Multidimensional Reflectometry. IAENG International Journal of Computer Science 42 23–30. [16] Langovoy, M. (2015). A unified approach to data analysis and modeling of the appearance of materials for computer graphics and multidimensional reflectometry. In IAENG Transactions on Engineering Technologies 15–29. Springer. [17] Langovoy, M. (2015). Active learning framework for manifold-valued data in computer graphics and reflectometry. EURAMET JRP-i21 Deliverable 4.3.1. EURAMET, Braunsweig. [18] Langovoy, M. (2016). Generalized active learning and design of statistical experiments for learning of high-dimensional manifolds. EURAMET JRPi21 Deliverable 4.3.2. EURAMET, Braunsweig. [19] Langovoy, M., Habeck, M. and Schoelkopf, B. (2011). Adaptive nonparametric detection in cryo-electron microscopy. In Proceedings of the 58-th World Statistical Congress. Session: High Dimensional Data 4456 – 4461. [20] Langovoy, M., Habeck, M. and Schoelkopf, B. (2011). Spatial statistics, image analysis and percolation theory. In The Joint Statistical Meetings Proceedings. Time Series and Network Section 5571 – 5581. [21] Langovoy, M., Schmaehling, F. and Wuebbeler, G. (2015). Chameleon: a parallel computing software for Monte Carlo modeling, visualization and sensitivity analysis of BRDF measurements of materials with complex appearance, based on a real-life robotic gonioreflectometer of the PTB. EURAMET JRP-i21 Deliverable 4.2.3. EURAMET, Braunsweig. ¨ bbeler, G. and Elster, C. (2014). Novel metric for [22] Langovoy, M., Wu analysis, interpretation and visualization of BRDF data. Submitted. [23] Langovoy, M. and Wuebbeler, G. (2014). Sensitivity analysis for BRDF measurements of standard reference materials. EURAMET JRPi21 Deliverable 4.1.3. EURAMET, Braunsweig. imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
15
[24] Meyer, B. (1988). Object-oriented software construction. [25] Nicodemus, F. E. (1965). Directional reflectance and emissivity of an opaque surface. Applied Optics 4 767 – 775. ¨ hling, F. and et. al. (2015). Virtual experiments for photometric [26] Schma and radiometric measurements. Proc. of. CIE 2015 28th session. ¨ hling, F., Wu ¨ bbeler, G., Lopez, M., Gassmann, F., [27] Schma ¨ Kruger, U., Schmidt, F., Sperling, A. and Elster, C. (2014). Virtual experiment for near-field goniophotometric measurements. Applied optics 53 1481–1487. [28] Sobolev, S. On a theorem of functional analysis. [29] Vince, J. (2013). Mathematics for computer graphics. Springer Science & Business Media.
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
Highlights.
• Introduces active learning to metrology and visual appearance data analysis
• Describes a framework to search for efficient BRDF sampling and measurement strategies
• Shows that standard equi-angular sampling is inferior for many types of loss functions
• For more accurate sampling, measure more in specular regions, less in diffuse ones.
S0263-2241(16)30471-7 http://dx.doi.org/10.1016/j.measurement.2016.08.010 MEASUR 4288
To appear in:
Measurement
Received Date: Revised Date: Accepted Date:
6 April 2016 4 August 2016 11 August 2016
Please cite this article as: M. Langovoy, F. Schmähling, G. Wübbeler, Numerical comparison of sampling strategies for BRDF data manifolds, Measurement (2016), doi: http://dx.doi.org/10.1016/j.measurement.2016.08.010
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Numerical comparison of sampling strategies for BRDF data manifolds Mikhail Langovoy∗ Physikalisch-Technische Bundesanstalt, Abbestrasse 2-12, D-10587, Berlin, Germany. e-mail: [email protected]
and Franko Schm¨ ahling Physikalisch-Technische Bundesanstalt, Abbestrasse 2-12, D-10587, Berlin, Germany. e-mail: [email protected]
and Gerd W¨ ubbeler Physikalisch-Technische Bundesanstalt, Abbestrasse 2-12, D-10587, Berlin, Germany. e-mail: [email protected] Abstract: The bidirectional reflectance distribution function (BRDF) is one of the fundamental concepts in such diverse fields as multidimensional reflectometry, computer graphics and computer vision. BRDF manifolds form an infinite-dimensional space, but typically the available measurements are very scarce. Therefore, an efficient learning strategy is crucial when performing the measurements. In this paper, we perform simulation studies within a mathematical framework that allows to establish more efficient BRDF sampling and measurement strategies in the sense of statistical design of experiments and generalized proactive learning. Our simulation studies suggest that the default BRDF measurement strategy is suboptimal for a wide class of loss functions. Keywords and phrases: Design of experiments, active learning, BRDF, diffuse reflection, specular reflection, computer graphics, computer vision, metrology, data analysis, statistics of manifolds, distance on the space of BRDFs.
∗ Corresponding
author.
1 imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
2
1. Introduction For many applications, the appearance of real-world surfaces is sufficiently well characterized by the bidirectional reflectance distribution function (BRDF). In the case of a fixed wavelength, BRDF describes reflected light as a fourdimensional function of incoming and outgoing light directions. In computer graphics and computer vision, usually either physically inspired analytic reflectance models, like [3], or parametric reflectance models chosen via qualitative criteria, like [11] or [10], to name a few, are used to model BRDFs. In multidimensional reflectometry, an alternative approach is usually taken. One directly measures values of the BRDF for different combinations of the incoming and outgoing angles and then fits the measured data to a selected analytic model using optimization techniques. In this paper, following the path started in [15], [16], we treat BRDF measurements as samples of points from a high-dimensional and highly non-linear non-convex manifold. 2. Active learning and design of experiments In general, BRDF is a 5-dimensional manifold, having 4 angular and 1 wavelength dimension. Note that even a set of 1-dimensional manifolds is infinitedimensional (and k-dimensional manifolds are not to be confused with parametric k-dimensional families of functions). At the same time, a typical measuring device only takes between 50 and 1000 points for all the BRDF layers together. In view of this, the available measurement points are indeed very scarce for a complicated problem such as BRDF estimation. Therefore, an efficient sampling strategy is required when performing the measurements. The angular positions of the light source and the sensor utilized within the measurement process have to be chosen to maximize the information contained in the output. Since sets of BRDF measurements are, in fact, observed random manifolds, we are dealing here with manifold-valued data. Standard statistical and machine learning methods can not be safely directly applied to BRDF data. Statistical design of experiments (see [6], [4]) is a well-developed area of quantitative data analysis. Generally speaking, our task fits the framework of the statistical design of experiments. However, previous research in this field was often more concerned with (important) topics such as manipulation checks, interactions between factors, delayed effects, repeatability, among many others. This shifted the focus away from considering design of statistical experiments on structured, constrained, or infinite-dimensional data. In contrast to much of the previous work on the design of experiments, BRDF measurements are carried out in strictly defined settings (run by qualified experts and in a way that is precisely known, reproducible, and determined by the physical and technical characteristics of the gonioreflectometer). Therefore, there is less room for human or random errors and influences. On the other hand, BRDF measurements are collections of points representing manifolds, so defining even the simplest statistical quantities in this case turns out to be a nontrivial and conceptual task. Overall, our methodology developed in [17], [18] represents a far-reaching generalization of the active machine learning framework, also generalizing the
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
3
proactive learning setup of [5]. To be more specific, active learning, as a special case of semi-supervised machine learning, oftentimes deals with finite sets of labels and aims at solving classification or clustering problems with a finite number of classes. While there have been a number of promising practical applications, most of the existing theory deals with analysis of performance of specific algorithms. In this paper, we give an illustrative example for results of [18], where new techniques for the design of experiments and active learning were developed, in order to establish more efficient sampling strategies. To achieve this, we extend and apply the virtual gonioreflectometer simulation software developed in [21] to assess different sets of measurement geometries. 3. BRDF definition In the most basic case, the bidirectional reflectance distribution function (BRDF), fr (ωi , ωr )) is a four-dimensional function that defines how light is reflected at an opaque surface. The function takes a negative incoming light direction, ωi , and outgoing direction, ωr , both defined with respect to the surface normal n, and returns the ratio of reflected radiance exiting along ωr to the irradiance incident on the surface from direction ωi . Each direction ω is itself parameterized by azimuth angle φ and zenith angle θ, therefore the BRDF as a whole is 4-dimensional. The BRDF has units sr−1 , with steradians (sr) being a unit of solid angle. The BRDF was first defined by Nicodemus in [25]. The defining equation is:
fr (ωi , ωr ) =
d Lr (ωr ) d Lr (ωr ) = . d Ei (ωi ) Li (ωi ) cos θi d ωi
(1)
where L is radiance, or power per unit solid-angle-in-the-direction-of-a-ray per unit projected-area-perpendicular-to-the-ray, E is irradiance, or power per unit surface area, and θi is the angle between ωi and the surface normal, n. The index i indicates incident light, whereas the index r indicates reflected light. 4. Statistical analysis of BRDF models A useful data analysis procedure for any BRDF model has to exhibit certain robustness against outliers and dependent or mixed errors. Our study of statistical estimation for BRDF models in [16], [15] clarified certain pitfalls in the analysis of BRDF data, and helped to develop more refined estimates for more realistic BRDF models. Suppose we have measurements of a BRDF available for the set of incoming angles (p) Pinc (p) (p) Pinc Ωinc = ωi p=1 = θi , ϕi . p=1
(2)
Here Pinc ≥ 1 is the total number of incoming angles where the measurements (p) were taken. Say that for an incoming angle ωi we have measurements available for angles from the set of reflection angles
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
Ωref l =
P[ inc
4
Ωref l (p) ,
(3)
p=1
where Ωref l (p) =
ωr(q)
Pref l (p)
=
q=1
θr(q) , ϕ(q) r
Pref l (p) , q=1
Pinc where Pref l (p) p=1 are (possibly different) numbers of measurements taken for corresponding incoming angles. Overall, we have the set of random observations
(p) (p) (p) (p) θ ,ϕ f θi , ϕi , θr(q) , ϕ(q) ∈ Ωinc , θr(q) , ϕ(q) ∈ Ωref l (p) r r i i
. (4)
Our aim is to infer properties of the BRDF funtion (1) from the set of observations (4). In general, the connection between the true BRDF and its measurements is described via a stochastic transformation T , i.e. f (ωi , ωr ) = T fr (ωi , ωr ) ,
(5)
where T : M × P × F 4 → F4 ,
(6)
with M = (M, A, µ) is an (unknown) measurable space, P = (Π, P, P) is an unknown probability space, F4 is the space of all Helmholtz-invariant energy preserving 4-dimensional BRDFs, and F4 is the set of all functions of 4 arguments on the 3-dimensional unit sphere S 3 in R4 . Equations (5) and (6) mean that there could be errors of both stochastic or non-stochastic origin. In this setting, the problem of inferring the BRDF can be seen as a statistical inverse problem (see also [14]). We refer to [16] for a more detailed discussion of this connection. 5. Foundations of sensitivity analysis for BRDF models There are many measures of distance and quasi-distance used in mathematics and statistics: Lp , 1 ≤ p < +∞, L∞ , Sobolev distances [28], Kullback-Leibler information divergence [9], chi-squared distance used in correspondence analysis [13], [12]. It is known from both mathematics and statistics that not all distances are equally appropriate in all applications. The standard L2 -norm for BRDFs f1 = f1 (ωi , ωr ), f2 = f2 (ωi , ωr ) is defined via L22 (f1 , f2 ) =
Z Z Z Z
(f1 − f2 )(θi , φi , θr , φr ) 2 dθi dφi dθr dφr ,
(7)
Ω
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
5
where Ω is the set containing all possible values of the four parameters. In [22] and [23], we developed a perception-based quasi-metric for BRDFs. Our main idea can be formulated as follows:
a ”distance” d between BRDFs is physically reasonable if and only if d(f1 , f2 ) is small if and only if two balls illuminated according to f1 and f2 would look similar to a human observer .
To be more precise: in the above statement, we consider two balls of the same size that are made of materials with BRDFs f1 and f2 respectively. Both balls are illuminated with a point source of light. 6. Measure of distance between BRDFs The following decomposition is typically used in BRDF studies. The main idea of this decomposition is to (roughly) separate diffuse and specular parts of each BRDF, in order to treat each part with a more appropriate technique. BRDF (θi , ϕi , θr , ϕr ) = fdif f (θi , ϕi , θr , ϕr ) + fspec (θi , ϕi , θr , ϕr ) .
(8)
Therefore, for each BRDF f ∈ F4 we have a decomposition of the parameter space Ω Ω = Ωdif f (f ) ∪ Ωspec (f ) ,
µΩ Ωdif f (f ) ∩ Ωspec (f ) = 0 .
(9)
As before, we denote ωi = (θi , ϕi ), ωr = (θr , ϕr ), ω = (ωi , ωr ). Put µdif f (f ) = µBRDF (f )
, Ωdif f (f )
µspec (f ) = µBRDF (f )
.
(10)
Ωspec (f )
An important point is that µdif f and µspec could and often should be different. We refer to [22] and [23] for details on the psychophysical interpretation of these decompositions. In this paper we use the following rule to define a ”specular” and a ”diffuse” part of the BRDF. Definition 1. Ωspec (f ) = {ω | f (ω) ≥ ∆1 }
(11)
Ωdif f (f ) = Ω \ Ωspec (f )
(12)
For a data-driven automated choice of ∆1 , it is also possible to use spatial scan estimators introduced in [19] and [20]. We use in this paper a useful simplification of the quasi-distance from [22]. imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
6
Definition 2. Let f1 , f2 be BRDFs. Define the element of distance between f1 and f2 in F4 at a point ω as
ρBRDF (f1 , f2 )(ω) =
ρdif f (f1 , f2 )(ω), ω ∈ Ωdif f (f1 ) ∩ Ωdif f (f2 ); ρspec (f1 , f2 )(ω), ω ∈ Ωspec (f1 ) ∩ Ωspec (f2 )∩ {ω | ρspec (f1 , f2 )(ω) < ∆1 }; ∆2 , ∆2 ,
ω ∈ Ωspec (f1 ) ∩ Ωspec (f2 )∩ {ω | ρspec (f1 , f2 )(ω) ≥ ∆1 };
(13)
ω ∈ (Ωspec (f1 ) ∩ Ωdif f (f2 )) ∪(Ωdif f (f1 ) ∩ Ωspec (f2 )) .
Here ∆1 and ∆2 are pre-selected constants, with ∆2 penalizing missing or added peaks and smoothing out the influence of large peaks and of gross errors in measurements of peaks. ∆1 determines the level where we switch the regime in our analysis. We noticed that, for actual BRDF measurements, it is often reasonable to pick ∆1 ≈ 1. The choice of ∆2 depends on how strongly one aims to penalize for missing peaks or added peaks. In practice, it makes sense to set ∆2 to be no greater than the upper value that can be reliably measured by the particular gonioreflectometer. Often, 1 ≤ ∆2 ≤ 3 were reasonable choices. A value of ∆2 close to 0 would have an effect on the metric similar to attributing very low value to the peaks. For a data-driven automated choice of ∆1 and ∆2 , it is possible to use spatial scan estimators introduced in [19] and [20]. Definition 3. We define the metric (distance) µBRDF on F4 as follows. For any BRDFs f1 , f2 ∈ F4 , set Z µBRDF (f1 , f2 ) =
ρBRDF (f1 , f2 )(ω) dµΩ (ω) .
(14)
Ω
An important example is given by Lp -norms. If µdif f = Lpp , then ρdif f (x, y) = |x − y|p . As µdif f we can reasonably use any Lp -based measure. It is possible to use an Lp1 -based measure µspec , possibly for some p1 6= p. A number of truncated measures has been considered in robust statistics. A proper choice of parameters ∆1 , ∆2 is closely related to the model selection problem, see [14]. The above expression for µBRDF can be computed explicitly in special cases, for example, when both BRDFs are known explicitly beforehand, either analytically or in a parametric form. To account for applications to real data, we introduced in [22] the following approximate empirical quasi-distance between BRDFs. Definition 4. Suppose that BRDFs f1 and f2 were measured on the following collection of points: Ωmeas =
(p) (p) (p) (p) (q) (q) θi , ϕi , θr(q) , ϕ(q) θ , ϕ ∈ Ω , θ , ϕ ∈ Ω (p) . inc ref l r r r i i
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
7
(15) We define the empirical metric (distance) µ bBRDF on F4 as follows. For any BRDFs f1 , f2 ∈ F4 , that are both measured on Ωmeas , set µΩ (Ω) µ bBRDF (f1 , f2 ) = Ωmeas
X
ρBRDF (f1 , f2 )(ω) .
(16)
ω∈Ωmeas
7. Active manifold learning strategies In BRDF sampling, the equispaced-angular grid pictured in Figure 1 is the standard (see, e.g., [8]). However, as we show in this paper, this choice of measurement points leads to inefficient sampling.
Fig 1: Standard grid, inefficient sampling Uniformly distributed points on a sphere look very different, see Figure 2. Since it was already understood in the community (see, for example, [8]) that the standard grid is suboptimal, there were multiple heuristic attempts to propose trickier grids that better reflect the typical structure of BRDF models. A good example is shown in Figure 3. Any result showing that the new strategy is better than the default strategy, at least for one specific loss function, for one specific BRDF, and one specific estimating procedure, is already instrumental in understanding the general picture of learning BRDF manifolds from scarce expensive data. This basic case is straightforwardly formulated in the language of mathematical optimization, so we are able to obtain theoretical guarantees on learning accuracy, at least for some special cases. Let us recall the mathematical framework for active learning of BRDFs from [18]. Consider BRDF f ∈ F4 , where F4 is defined in Section 4. Suppose that f is measured on the finite set Ωmeas (n) (compare with (15)) imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
8
Fig 2: Uniformly distributed points on a sphere
Ωmeas (n) =
(p) (p) (p) (p) (q) (q) θi , ϕi , θr(q) , ϕ(q) θ , ϕ ∈ Ω , θ , ϕ ∈ Ω (p) , inc ref l r r r i i (17)
where P inc X Ωref l (p) . n = Ωmeas (n) =
(18)
p=1
Definition 5. Cost function Cost of a measurement configuration Ωmeas is a Ωmeas Lebesgue measurable function Cost : R → R+ . Let Dist be a function (measurable for a suitably chosen σ-algebra) such that Dist : F4 × F4 → R+ .
(19)
For our purposes, we typically like Dist to be inducing either a quasi-distance or a pseudo-distance on F40 , where F40 ⊆ F4 is a sufficiently reach subset. As an example, µBRDF from (16), Section 6, is often used in our practical experiments. Standard Lp -distances are easier for theoretical comparisons of efficiencies. Definition 6. Sampling strategy Ω is a sequence Ω = {Ωn0 }∞ n0 =1 where for each n0 there exists an integer n ≥ n0 such that Ωn0 = Ωmeas (n) for some measurement configuration Ωmeas (n) defined according to (17), and for any integers n1 ≤ n2 it holds that Ωn1 ≤ Ωn2 . Consider arbitrary fixed statistical estimator of BRDFs En : Rn → F4 .
(20)
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
9
Fig 3: Tricky grid, heuristic choice
Consider two sampling strategies: Ω1 , Ω2 . Suppose that both strategies have uniformly admissible costs (see [18]). The problem of generalized active learning for BRDF sampling can be stated in the following way. Find a sampling strategy ∞ Ωmeas = Ωmeas (n) n=nmin such that for all n ≥ nmin Ωmeas (n) = arg
min Ω : Cost(Ω)
Dist En (f ; Ω), f
(21)
Definition 7. Suppose f ∈ F4 is a particular (possibly unknown) BRDF. Let Ω1 , Ω2 sampling strategies be sampling strategies with Cmax -uniformly admissible costs. We say that strategy Ω1 is asymptotically more efficient for learning f than the strategy Ω2 , and write Ω1
lim sup n→∞
Dist En (f ; Ω1 (n)), f < 1. Dist En (f ; Ω2 (n)), f
(22)
Notice that, for the task of evaluating sampling quality, expected errors (over classes of BRDFs) are more interesting than maximal errors (over the same classes). A novel framework for treating this more general problem was developed in [17], [18]. 8. Simulation environment In analogy to the real experiment the computer model consists of three parts, the mechanical parts, the IxMD or array-spectroradiometer for spectrally resolved imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
10
measurements, the light source, which ensure a stable diffuse illumination of the sample, and of course the sample which reflectance behaviour we want to be characterized via the virtual experiment. The software consists of a set of interacting components, every part represents an entity with own methods and properties and in that way we covered the principle of modular software design [2], [24]. This approach allows us to construct measuring systems for various applications in a very flexible way [26]. Based on the generic library of the virtual near-field goniophotometer [27], we are able to model every part of the gonioreflectometer as a single object and finally integrated all objects together to build a computer model of the whole measurement system. Every object is linked to another object and the core procedure of every component is the calculation of the transformation matrix, which enables linked transformations between the objects [7], [1], [29]. To build the software counterpart for the real (world) gonioreflectometer we were able to use large parts of the generic library of the near-field goniophotometer in a straightforward manner or after minor changes. Especially for the mechanical parts, i.e. the parts of robot arm, the motion unit for the light source and the detector. In addition, the source code for the light source and the sample is deployed to fit to our application. In regard to the sample surface, at first, we are able to simulate the so-called goniochromatic behaviour, i.e. the situation where the colour is changing with the change of reflectance direction under the condition that the direction of incoming light is the same for all reflectance directions. The second option is to simulate gloss appearance via the Cook-Torrance BRDF model on the sample surface. The computer model should be as close as possible to the real experiment in order to be able to draw reliable conclusions from the virtual experiments. For computer graphics’ reflection models such as the Cook-Torrance model [3], there is a correspondence between the parameters of these models and the appearance measurement scale for gloss. The Cook-Torrance model is a further advancement of the specular–microfacet Torrance-Sparrow model, accounting for wavelength and thus color shifting. 9. Results of virtual experiments Virtual experiments, besides being cheap, offer a number of advantages in research. In particular, one can emulate and check specific measurement devices, even without building them physically. Cameleon 3.0 is the software that emulates the real-life robot-based gonioreflectometer developed and constructed in PTB [21]. To illustrate the point of this paper, we have chosen a standard CookTorrance model with the single specular peak at ϕ = 180◦ and θ = 45◦ . Apart from the random sampling, no measurement errors were introduced to this basic model, in order to focus on assessment of the relative efficiency of the sampling strategies. The four sampling strategies were chosen due to the following reasons. The standard equi-angular sampling, which is the default sampling method used in most of the labs that measure BRDFs with gonio-reflectometers. Uniform (random) sampling: applicable when no prior knowledge about the BRDF is available. Deterministic grid-sampling near the specular peak: this approach
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
11
appears to be reasonable when prior knowledge about the characteristics of the BRDF is available (e.g., for a glossy surface). Gaussian (random) sampling: similar to sampling near the specular peak, applicable if prior knowledge about the BRDF is available. BRDF sampling modeled with the equispaced-angular grid is pictured in Figure 4. As we show below, this choice of measurement points leads to inefficient sampling. BRDF sampling with uniformly distributed points on a sphere can be seen on Figure 5. Two of the more elaborated grids that better reflect the typical structure of BRDF models are shown in Figures 6 and 7.
Fig 4: Standard grid, inefficient sampling
Fig 5: Uniform random sampling
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
12
Fig 6: Sampling more in the region of the peak
Fig 7: Gaussian sampling concentrated at the peak The following table illustrates, on the basis of 100 repetitions, what are typical approximation errors made while following the above four different sampling strategies. The standard equi-angular sampling is clear inferior for both types of loss functions, especially for the perception-based µBRDF . We used the thresholds ∆1 = 1 and ∆2 = 0.1. The second value is a fairly arbitrary choice, but can be used when putting little weight on misplaced peaks is appropriate. This is the case in the present example, as we know in advance that the actual locations of peaks are in perfect alignment, and the only misalignment of peaks that we get is coming from interpolation errors (compare (11) and (13)). In this special case, a big ∆2 will lead to multiplying interpolation errors rather than to penalizing wrong locations of peaks. As we are not focusing on interpolation errors in this paper, we picked a small ∆2 to downplay their effect.
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
standard equi-angular sampling uniform sampling sampling near peak gaussian sampling
µBRDF 3.4093 3.3600 2.3526 2.2628
13
L2 0.1199 0.1102 0.0545 0.0583
Similar results occur for a variety of other simulated BRDFs. Based on our experience, the standard equi-angular sampling is inferior for several types of loss functions (even for those cases when penalties for misplaced peaks are small, like in the above example). For more accurate sampling, it is better to increase the number of measurements in specular regions, even though that reduces the number of measurements in the diffuse part. The explanation for this effect is that the diffuse part of the BRDF varies much slower and can be reconstructed on the basis of a substantially smaller amount of observations. 10. Conclusions BRDF manifolds form an infinite-dimensional space, but typically the available measurements are very scarce for complicated problems such as BRDF estimation. Therefore, an efficient sampling strategy is crucial when performing the measurements. In this paper, we extend and apply the virtual gonioreflectometer simulation software to assess different sets of measurement geometries. We show that the default BRDF measurement strategy is suboptimal for a variety of loss functions. Acknowledgments. The authors would like to thank Anna Langovaya and the two referees for valuable suggestions that helped to greatly improve the presentation of the paper. This work has been carried out within EMRP project IND 52 ’Multidimensional reflectometry for industry’. The EMRP is jointly funded by the EMRP participating countries within EURAMET and the European Union.
References [1] Arfken, G. B. and Weber, H. J. (2011). Mathematical methods for physicists: a comprehensive guide. Academic press. [2] Booch, G. (2006). Object-oriented analysis and design with applications. Pearson Education India. [3] Cook, R. L. and Torrance, K. E. (1981). A reflectance model for computer graphics. In ACM Siggraph Computer Graphics 15 307–316. ACM. [4] Cox, D. R. and Reid, N. (2000). The theory of the design of experiments. CRC Press. [5] Donmez, P. and Carbonell, J. G. (2008). Proactive learning: costsensitive active learning with multiple imperfect oracles. In Proceedings of the 17th ACM conference on Information and knowledge management 619–628. ACM. [6] Fisher, S. R. A., Genetiker, S., Fisher, R. A., Genetician, S., ´ ne ´ticien, S. (1960). The design of experiments Fisher, R. A. and Ge 12. Oliver and Boyd Edinburgh. imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
14
[7] Goldstein, H., Poole, C. P. and Safko, J. L. (2014). Classical Mechanics: Pearson New International Edition. Pearson Higher Ed. ¨ pe, A. and Hauer, K.-O. (2010). Three-dimensional appearance char[8] Ho acterization of diffuse standard reflection materials. Metrologia 47 295. [9] Kullback, S. and Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics 79–86. [10] Lafortune, E. P., Foo, S.-C., Torrance, K. E. and Greenberg, D. P. (1997). Non-linear approximation of reflectance functions. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques 117–126. ACM Press/Addison-Wesley Publishing Co. [11] Lambert, J. H. and Anding, E. (1892). Lamberts Photometrie: (Photometria, sive De mensura et gradibus luminis, colorum et umbrae) (1760). Ostwalds Klassiker der exakten Wissenschaften v. 1-2. W. Engelmann. [12] Langovaya, A. and Kuhnt, S. (2011). Correspondence analysis and moderate outliers. Agrocampus Ouest, Rennes FRANCE 19. [13] Langovaya, A., Kuhnt, S. and Chouikha, H. (2013). Correspondence Analysis in the Case of Outliers. In Classification and Data Mining 63–70. Springer. [14] Langovoy, M. (2007). Data-driven goodness-of-fit tests. University of G¨ ottingen, G¨ ottingen. Ph.D. thesis. [15] Langovoy, M. (2015). Machine Learning and Statistical Analysis for BRDF Data from Computer Graphics and Multidimensional Reflectometry. IAENG International Journal of Computer Science 42 23–30. [16] Langovoy, M. (2015). A unified approach to data analysis and modeling of the appearance of materials for computer graphics and multidimensional reflectometry. In IAENG Transactions on Engineering Technologies 15–29. Springer. [17] Langovoy, M. (2015). Active learning framework for manifold-valued data in computer graphics and reflectometry. EURAMET JRP-i21 Deliverable 4.3.1. EURAMET, Braunsweig. [18] Langovoy, M. (2016). Generalized active learning and design of statistical experiments for learning of high-dimensional manifolds. EURAMET JRPi21 Deliverable 4.3.2. EURAMET, Braunsweig. [19] Langovoy, M., Habeck, M. and Schoelkopf, B. (2011). Adaptive nonparametric detection in cryo-electron microscopy. In Proceedings of the 58-th World Statistical Congress. Session: High Dimensional Data 4456 – 4461. [20] Langovoy, M., Habeck, M. and Schoelkopf, B. (2011). Spatial statistics, image analysis and percolation theory. In The Joint Statistical Meetings Proceedings. Time Series and Network Section 5571 – 5581. [21] Langovoy, M., Schmaehling, F. and Wuebbeler, G. (2015). Chameleon: a parallel computing software for Monte Carlo modeling, visualization and sensitivity analysis of BRDF measurements of materials with complex appearance, based on a real-life robotic gonioreflectometer of the PTB. EURAMET JRP-i21 Deliverable 4.2.3. EURAMET, Braunsweig. ¨ bbeler, G. and Elster, C. (2014). Novel metric for [22] Langovoy, M., Wu analysis, interpretation and visualization of BRDF data. Submitted. [23] Langovoy, M. and Wuebbeler, G. (2014). Sensitivity analysis for BRDF measurements of standard reference materials. EURAMET JRPi21 Deliverable 4.1.3. EURAMET, Braunsweig. imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
M. Langovoy and F. Schm¨ ahling and G. W¨ ubbeler/BRDF sampling
15
[24] Meyer, B. (1988). Object-oriented software construction. [25] Nicodemus, F. E. (1965). Directional reflectance and emissivity of an opaque surface. Applied Optics 4 767 – 775. ¨ hling, F. and et. al. (2015). Virtual experiments for photometric [26] Schma and radiometric measurements. Proc. of. CIE 2015 28th session. ¨ hling, F., Wu ¨ bbeler, G., Lopez, M., Gassmann, F., [27] Schma ¨ Kruger, U., Schmidt, F., Sperling, A. and Elster, C. (2014). Virtual experiment for near-field goniophotometric measurements. Applied optics 53 1481–1487. [28] Sobolev, S. On a theorem of functional analysis. [29] Vince, J. (2013). Mathematics for computer graphics. Springer Science & Business Media.
imsart-generic ver. 2014/07/30 file: BRDF_Sampling_Strategies_Version_5.tex date: August 11, 2016
Highlights.
• Introduces active learning to metrology and visual appearance data analysis
• Describes a framework to search for efficient BRDF sampling and measurement strategies
• Shows that standard equi-angular sampling is inferior for many types of loss functions
• For more accurate sampling, measure more in specular regions, less in diffuse ones.