Uncertainty propagation in computationally expensive models: A survey of sampling methods and application to scatterometry

Measurement 97 (2017) 79–87

Contents lists available at ScienceDirect

Measurement journal homepage: www.elsevier.com/locate/measurement

Uncertainty propagation in computationally expensive models: A survey of sampling methods and application to scatterometry Sebastian Heidenreich a,⇑, Hermann Gross a, Markus Bär a, Louise Wright b a b

Physikalisch-Technische Bundesanstalt, Abbestr. 2-12, 10587 Berlin, Germany Mathematics and Modelling Group, National Physical Laboratory, Hampton Road, Teddington TW11 0LW, UK

a r t i c l e

i n f o

Article history: Received 26 May 2015 Received in revised form 2 June 2016 Accepted 7 June 2016 Available online 8 June 2016 Keywords: Smart sampling Statistical inverse problem Scatterometry

a b s t r a c t Partial differential equations with uncertain input parameters are used in many applications in metrology, physics and engineering. The effect of input uncertainties on the solutions can be determined by the law of propagation of uncertainties. According to the guide to the expression of uncertainties in measurements (GUM) and its supplements, Monte Carlo sampling is recommended for nonlinear problems. In practice, large sampling sizes have to be chosen to ensure accuracy and precision. For computationally expensive problems only small sampling sizes are accessible. In this article we study and compare the propagation of uncertainties using three different sampling methods. The sampling methods chosen are Monte Carlo sampling, Latin hypercube sampling and a Sobol sequence based quasi Monte Carlo sampling. The methods are applied to the inverse problem of scatterometry with several simplifying assumptions in the measurement model. The solution of the inverse problem of scatterometry involves finite element solutions of a two dimensional Helmholtz equation. We found that among methods chosen Latin hypercube provides the most accurate and reliable results with respect to estimates of the geometry parameters, uncertainties and to repeatability. Ó 2016 Elsevier Ltd. All rights reserved.

1. Introduction From engineering, risk analysis and medical applications to metrology there is an increasing interest for uncertainty quantification [1–3]. In many of these applications partial differential equations (PDEs) are used for modeling complex phenomena. The solutions of the PDEs depend in general on parameters, boundary and initial conditions often which are not exactly known. These uncertain input parameters yield to variations of the PDEs solution (propagation of uncertainties). A representative example is the uncertainty propagation in computational fluid dynamics [4–6]. Many methods for uncertainty quantification were developed and applied to manifold applications over the last decades. A recently developed technique considers PDEs with random parameters as stochastic PDEs and expand its solution into polynomials. This approach is called polynomial chaos and generalized polynomial chaos, respectively. Polynomial chaos was successfully employed to many applications to investigate the propagation of input uncertainties [7–14]. Polynomial chaos is complex in use,

but very successful if the number of uncertain input parameters is not too large. A simpler approach for the propagation of uncertainties which is independent of the number of inputs is random sampling. In particular, it samples from the random input variables and for every sample the output quantity is determined which is given for example by the solution of a PDE. In our application, optical scatterometry, we have to solve a PDE and in addition an inverse problem which makes it computationally expensive. The problem with sampling is typically that the sampling size (number of samples) has to be large for accurate results. In this paper we apply different sampling methods for the propagation of uncertainty in the inverse problem of scatterometry to investigate which method yields to accurate results for small sampling sizes. In the first part we introduce the sampling methods chosen and study repeatability on a ‘‘toy model” (nonlinear function). In the second part we introduce the application scatterometry and we finally discuss the propagation of uncertainty for the sampling methods used.

2. Sampling methods ⇑ Corresponding author. E-mail address: [email protected] (S. Heidenreich). http://dx.doi.org/10.1016/j.measurement.2016.06.009 0263-2241/Ó 2016 Elsevier Ltd. All rights reserved.

In the present paper we have chosen three different sampling methods for the investigation of uncertainty propagation. Widely

80

S. Heidenreich et al. / Measurement 97 (2017) 79–87

Fig. 1. Means and standard deviations of ten sample means of the same sample size plotted against sample size for three sample generation methods (color online). The reference sample mean is 0.67. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

distributed and easy to use is the simple Monte Carlo (MC) sampling. Here, random numbers are pseudo-randomly drawn from the desired distribution. In the main study reported here we consider solely Gaussian distributed random variables as inputs for the propagation of uncertainties. As pseudo-random number generator we used the function ‘‘randn” in the computer software Matlab. For nonlinear models the use of the MC method is recommended by the Guide to the expression of uncertainty in measurement, supplement 1 [24] and can be considered as a standard method in metrology for nonlinear models. However, accurate results need large sampling sizes not available for computationally expensive models. In that case, it is suggested to use only 50–100 samples. As a second sampling method we selected the quasi-Monte Carlo (qMC) method. QMC belongs to the smart sampling methods where a series of numbers is used instead of pseudo-random numbers. Here, we used a low-discrepancy series developed by Sobol

[29] which represent samples of a high dimensional uniform distribution. The predefined point selection yields often to better convergence properties of qMC [30]. We used a Sobol sequence generator based on the papers of Bratley and Joe [31,32] which is included in Matlab. To obtain normal distributed samples we used the inverse cumulative distribution function (CDF) technique. This technique uses the fact that a continuous CDF is a mapping of the domain of the CDF to the interval ð0; 1Þ. Thus, the variable X ¼ F 1 ðUÞ has the distribution F, when U is uniform on ð0; 1Þ. We used the Sobol sequence to sample the uniform distribution Uð0; 1Þ and the inverse CDF technique to obtain normal distributed numbers. The third chosen method is the Latin hypercube sampling (LHS). LHS was developed by McKay and coworkers as an alternative to simple Monte Carlo sampling [26]. LHS is a smart sampling method that ensures that every subregion of every random variable is sampled. Thus it fills the space equally and provides unbiased estima-

S. Heidenreich et al. / Measurement 97 (2017) 79–87

81

Fig. 2. Means and standard deviations of ten sample standard deviations of the same sample size plotted against sample size for three sample generation methods (color online). The reference sample standard deviation is 0.57. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

tor [26]. In our investigations, we assume the independence of the random input variables. LHS divided each variable into N regions of equal probability and select one sample from each interval. The samples for each input are shuffled so that there is no correlation between inputs. In many applications the convergence of LHS is faster than that of MC [28,27].

This approach is clearly not feasible for computationally expensive models such as the main model used here, so a smaller ‘‘toy” problem has been used to investigate the repeatability of the methods described above. The toy problem has a single output Y and two inputs X 1 and X 2 . X 1 is uniformly distributed on the interval ½0; 1 and X 2 has a triangular distribution on the interval ½0; 1, with a mode of 0.25. and the output is defined by

3. Repeatability

Y ¼ X 1 X 2 þ 0:5X 3 þ 0:5X 1 þ sinð2pX 1 Þ:

Repeatability of results is an important property for sampling methods. If a method is to give good results it should produce a low sample-to-sample variance of the quantities of interest. The best way to test repeatability is to carry out repeated evaluations using the same sample size and to calculate the sample-tosample variability directly.

Large-scale random sampling shows that this function has a mean of 0.67 and a standard deviation of 0.57. Ten sample sizes (between 10 and 100 in steps of 10) have been used. Each of the sampling methods have been used to generate ten samples of each sample size, and the toy model has been used to generate results for each sample. For each sample, a mean and

ð1Þ

82

S. Heidenreich et al. / Measurement 97 (2017) 79–87

Fig. 3. Cross section of a EUV photo mask of one period.

standard deviation has been calculated. The mean and standard deviation of the ten sample means have been calculated for each combination of sample size and method, along with the mean and standard deviation of the sample standard deviations. Figs. 1 and 2 show the results of these calculations. Fig. 1 shows the mean (sub-figure (a)) and sample-to-sample standard deviation (sub-figure (b)) of the sample means for each sample size. The standard deviations of the Latin hypercube samples are smaller than those of the other two methods throughout, suggesting that the Latin hypercube method has better repeatability than random sampling and Sobol sequences for estimation of the mean. The upper figure suggests that the mean of the means calculated from Sobol sequences and Latin hypercube samples gives a better estimate of the reference mean of 0.67 than the value calculated by random sampling. Fig. 2 shows the mean (sub-figure (a)) and sample-to-sample standard deviation (sub-figure (b)) of the sample standard deviations for each sample size. As with the sample mean, the Latin hypercube sample shows the lowest sample-to-sample standard deviation and hence the best repeatability (lower plot), and the

Fig. 4. MC reference (Color online): Mean values of the reconstructed parameters with increasing samples. The red lines denote the ‘‘true” values. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

S. Heidenreich et al. / Measurement 97 (2017) 79–87

83

Fig. 5. MC reference (Color online): Standard deviations of estimated parameters with increasing samples. The red lines denote the ‘‘true” values. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

mean values are close to the reference value of 0.57. The Sobol sequences show poor repeatability for the standard deviation and are generally less accurate than the other two methods.

4. The inverse problem of scatterometry For metrology the quantification of uncertainties in the inverse problem of optical scatterometry is of great interest [25] and may be a significant step to the traceability of this technique. In scatterometry, periodic nano-structured surfaces are illuminated by a monochromatic light source and diffracted intensities are measured. Relative maxima of diffracted intensities (efficiencies) are used to reconstruct the unknown geometry. In particular, we used a model of an EUV photo mask geometry that consists of a multilayer, two capping layers, and periodic straight absorber lines made of three different materials. The pitch is 280 nm, the geometry parameters are the bottom-CD = 91:62 nm (width of bottom), the height of the second absorber layer = 81:119 nm and the sidewall angle (SWA) = 89:25 . A cross section of the EUV mask is shown in Fig. 3. More details about EUV scatterometric set up can be found in Perlich and Scholze [21,22].

4.1. Forward model The map from the geometry design to diffracted efficiencies is given by the forward model which is based on the Maxwell’s equations describing the light propagation. The specific line structure chosen implies invariance of optical properties in one spatial direction, here labeled as the z-direction. Here, Maxwell’s equations can be reduced and the wave propagation is described by the two dimensional Helmholtz equation in the x—y plane, 2

Duðx; yÞ þ k ðx; yÞuðx; yÞ ¼ 0;

ð2Þ

where u is the transversal field component that oscillates in the zdirection and kðx; yÞ is the wave number

kðx; yÞ ¼ nðx; yÞ

x c

:

ð3Þ

The wave number is given by the refraction index nðx; yÞ and the ratio of frequency and speed of light. nðx; yÞ is assumed to be constant for areas filled with the same material. The boundary conditions that are imposed on the PDE are periodic on the lateral boundaries due to the periodic structure and usual outgoing wave

S. Heidenreich et al. / Measurement 97 (2017) 79–87

81.13

81.13

81.125

81.125

81.125

81.12 81.115 81.11 0

height/nm

81.13

height/nm

height/nm

84

81.12 81.115

81.115 81.11

81.11

50

0

sampling steps

0

50

92.5

92.2

92

91.8

81.13 81.125

height/nm

width/nm

91.5

50

sampling steps

sampling steps

92

width/nm

81.12

91.6 91.4

81.12 81.115

91.2

91 0

20

40

60

91

80

81.11 0

40

60

0

80

sampling steps

sampling steps

90

89.5

89.5

89.5

88.5 0

20

40

60

sampling steps

80

SWA / °

90

89

89

88.5 0

20

40

60

50

sampling steps

90

SWA / °

SWA / °

20

89

88.5

80

0

20

40

60

80

sampling steps

sampling steps

Fig. 6. Reconstructed geometry parameters (color online). Blue circles are means of the parameters and the error bar indicates the standard uncertainty by the two times standard deviation. The red lines show reference geometry values. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

conditions in the infinite regions. For rigorous calculations we used the finite element (FEM) solver DIPOG [16]. We fixed the wave length of the incident light k ¼ 13:4 nm, chose an incident angle of 6 and used values of the optical parameters given in previous publications [17]. For every set of geometry parameters p ¼ ðp1 ; . . . ; pN ÞT , there is a set of efficiencies f 1 ðpÞ; . . . ; f M ðpÞ which defines a nonlinear map (forward model)

p # f j ðpÞ:

ð4Þ

Efficiencies obtained from experiments include additional noise. For scatterometry, two main sources of noise are known [17]. Variations of the power of the incident light beam and noise from background fluctuations. To include these effects, we extend our model by adding a Gaussian distributed noise with zero mean j :

yj ¼ f j ðpÞ þ j ;

ð5Þ

The statistical inverse problem in scatterometry is defined to be the reconstruction of the geometry and error parameters from noisy diffracted intensities. In recent investigations it was shown for unknown error parameters a; b the maximum likelihood estimation provides an accurate and precise method for the determination of geometry parameters [17]. According to the error model, we have chosen the likelihood function

Lða; b; pÞ ¼

m Y 2 2 1=2 ð2pðða  f j ðpÞÞ þ b ÞÞ j¼1

"

 exp 

The first term models power fluctuations and the second term represents some background noise. The effect of line roughness is not considered here for convenience, but it can be easily included by multiplying the forward model with a damping factor [18,20]

2

2

2ðða  f j ðpÞÞ þ b Þ

;

ð7Þ

where m denotes the number of diffraction orders included in the measurement data set. In this approach the corresponding maximum likelihood estimator (MLE) reads [15,17]

a;b;p

ð6Þ

#

2

ðf j ðpÞ  yj Þ

^ p ^h ¼ ða ^ Þ ¼ arg maxLða; b; pÞ; ^; b;

where the standard deviation is given by

r2j ¼ ða  f j Þ2 þ b2 :

4.2. The inverse problem

ð8Þ

where for technical reasons the negative logarithm of the likelihood function is minimized. In the framework of MLE, uncertainties are typically obtained from the Fisher information matrix. However, for problems with multiple maxima in the likelihood function, uncertainties are systematically underestimated [33]. Here, we use the principle of ‘‘propagation of uncertainty” that gives more

85

S. Heidenreich et al. / Measurement 97 (2017) 79–87

10

25

0.02

0.02

-3

20

0.01

a

0.01

a

a

15 10 5

0

0

0 0

20

40

60

80

-5

0

20

sampling steps

40

60

0

80

20

10 -3

10 -3

10

25 2

2

1

1

40

60

80

sampling steps

sampling steps

-4

20

b

b

b

15 10 5

0

0 0

20

40

60

80

0 0

sampling steps

20

40

60

sampling steps

80

-5 0

20

40

60

80

sampling steps

Fig. 7. Reconstructed error parameters (color online). Blue circles are means of the parameters and the error bar indicates the standard uncertainty by the two times the standard deviation. The red lines show reference geometry values. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

reliable estimates of uncertainties and is in agreement with GUM [24]. 5. Reconstruction of geometry parameters and uncertainty quantification

dard deviation calculated with 104 samples to obtain an estimate for the 95% coverage interval.

For the comparison of the uncertainty propagation using different sampling methods we have to define reference values. 5.1. Reference method As reference method we used Monte Carlo sampling with a large sampling size of 104 , i.e., we drew random vectors ðf 1 ; . . . ; f M Þi with i ¼ 1; . . . ; 104 and M ¼ 20, from a normal distribution with zero mean and standard deviation

rj ¼

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 2 ða  f j Þ þ b ;

estimator, e.g. according to Firth [23], are outside the scope of the present article. Fig. 5 shows the standard deviation r for every parameter with increasing number of samples. As reference value for the uncertainties of parameters we used 1:96 times the stan-

ð9Þ

with error parameters a ¼ 1% and b ¼ 0:001. In previous studies the error model Eq. (5) was successfully applied to evaluate scatterometry experiments of an extreme ultraviolet photo mask. The values of the error parameters found for the experimental data are slightly larger, but of the same magnitude than those used here for the simulations, i.e., a  3% and b ¼ 103 (see [19] for details). Each random vector is added to the efficiencies f j ðpÞ to model measurement noise. To this end, we obtained 104 simulated scatterograms consisting of 20 light diffracted efficiencies for one chosen geometry. For each scatterogram we estimated the width, height, side wall angle (SWA), and the noise parameters by MLE. Fig. 4 shows the convergence of estimated parameters with increasing number of samples. All curves converge rapidly to the input geometry values, except for the noise parameter b that still shows a deviation of about 20%. We found the bias for different initial conditions. Improvements to decrease the bias of the MLE

5.2. Results In our analysis we have chosen sampling sizes of 10 and 80 samples. At every sampling step we selected a random number according to the sampling method and generate corresponding efficiencies. From these efficiencies we solved the inverse problem by the MLE method. Fig. 6 shows reconstruction results for the geometry parameters. The blue circles are the mean of the estimates and the green error bars cover the range of 1:96r where r is the standard deviation. For all methods estimates rapidly relax to the reference geometry parameters and uncertainties are consistent. Fig. 7 displays results for the error parameters a and b. The parameter a is accurately approximated by all methods, but b differs by about 20%. The underestimation of b was also present in the reference MC method and comes from the MLE. In Tables 1 and 2 the mean of the reconstruction results are given for a sampling size of 80 and 10, respectively. Even for only 10 samples the values are very close to the reference values. This is not surprising because the MLE estimation gives precise reconstructions at every sample step. Standard uncertainties are considered as the 95% coverage interval of a Gaussian distributed output distribution which is 1.96 times the standard deviation. The obtained uncertainties after 80 samples and 10 samples are summarized in Tables 3 and 4 respectively. Deviation of the calculated uncertainties from the reference values can be up to fifty percent. Table 5 shows the relative errors for the very small sampling size of 10. Both MC and Sobol samplings reach errors of 30–50% for speci-

86

S. Heidenreich et al. / Measurement 97 (2017) 79–87

Table 1 Mean of reconstructed parameters using a sample size of 80. Parameter

Reference

MC ref

MC GUM

LHS

Sobol

Width/nm Height/nm SWA/° a b/%

91.6200 81.1190 89.2500 0.01 0.001

91.6148 81.1191 89.2521 0.00984 0.00081

91.6814 81.1195 89.2517 0.00969 0.00083

91.6226 81.1190 89.2504 0.00993 0.00078

91.6207 81.1190 89.2430 0.00982 0.00080

Table 2 Mean of reconstructed parameters using a sampling size of 10. Parameter

Reference

MC ref

MC GUM

LHS

Sobol

Width/nm Height/nm SWA/° a b/%

91.6200 81.1190 89.2500 0.01 0.001

91.6148 81.1191 89.2521 0.00984 0.00081

91.7199 81.1210 89.2592 0.01062 0.00083

91.6136 81.1193 89.2229 0.01167 0.00072

91.5895 81.1185 89.2406 0.00953 0.00082

Table 3 Standard uncertainty (two times the standard deviation) of reconstructed parameters using a sample size of 80. Parameter

Reference

MC (GUM)

LHS

Sobol

Width/nm Height/nm SWA/° a b/%

0.3896 0.0063 0.2605 0.0071 0.00065

0.4224 0.0071 0.2841 0.0091 0.00064

0.4185 0.0064 0.2572 0.0070 0.00060

0.3748 0.0057 0.2419 0.0064 0.00067

by the propagation law of uncertainties. In particular, input uncertainties are given by the measurement error and output uncertainties are determined by the propagation of input distributions which makes it necessary to solve an inverse problem. The inverse problem was solved by the Maximum likelihood method. Solving the inverse problem in scatterometry is computationally expensive and a reduction of function evaluations and samples respectively is desirable. We investigated three different sampling methods: Monte Carlo sampling, Latin hypercube and Quasi Monte Carlo based on a Sobol sequence. In a study with a toy model we test the repeatability of the sampling methods first. When the sampling method is repeated the mean of parameters and the standard deviation varies. The MC and the qMC methods show a large variability where the variability of LHS was very small. In the next step we used the sampling methods to determine uncertainties from reconstructed geometry parameters. All methods are useful and give respectable results, but for small and very small sampling sizes the LHS method provides more accurate uncertainties. Close related to the question of the sampling size is the speedup of the different methods. In genreral, the speedup depends on modelspecific calculations when the inverse problem is solved. However, we can estimate the speed up for our present example. To reach the same value for the standard uncertainty using MC as for LHS with a sampling size of 10, a sampling size of about 50 is needed. In this case LHS is five times faster than MC (100 min versus 20 min). To conclude, uncertainties in scatterometry measurements can be quantified efficiently by LHS. Acknowledgments

Table 4 Standard uncertainty (two times the standard deviation) of reconstructed parameters using a sampling size of 10. Parameter

Reference

MC (GUM)

LHS

Sobol

Width/nm Height/nm SWA/° a b/%

0.3896 0.0063 0.2605 0.0071 0.00065

0.3570 0.0065 0.1701 0.0069 0.00055

0.3655 0.0064 0.2361 0.0078 0.00068

0.3783 0.0029 0.2374 0.0057 0.00061

Table 5 Relative deviation of the standard uncertainty from the reference values using 10 samples. Parameter

MC (GUM) (%)

LHS (%)

Sobol (%)

Width Height SWA a b

8.4 3.2 34.7 2.8 15.4

6.2 1.6 9.4 9.8 4.6

2.9 54.0 8.9 19.7 6.1

fic parameters. Only values provided by LHS deviate less than 10% which is remarkably good for the considered sampling size. To this end, sampling methods are useful to estimate standard uncertainties in scatterometry. All methods are simple to implement and easy to use. Among the three methods considered LHS provides the most accurate parameter means, standard uncertainties and repeatability. Even for only 10 samples uncertainties differ from the reference values by less than 10%.

6. Conclusion In this article we addressed the determination of uncertainties in scatterometry by sampling methods. We have chosen an error model for the measurand and determined standard uncertainties

This work was founded through the European Metrology Research Program (EMRP) Project NEW04 Uncertainty. The EMRP is jointly funded by EMRP partnering countires within EURAMET and the European Union. References [1] M.G. Morgan, M. Henrion, Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis, Cambridge University Press, NYC, 1990. [2] Y. Rabin, A general model for the propagation of uncertainties in measurements into heat transfer simulations and its application to cryosurgery, Cryobiology 46 (2003) 109–120. [3] T.E.J. Behrens, M.W. Woolrich, M. Jenkinson, H. Johansen-Berg, R.G. Nunes, S. Clare, P.M. Matthews, J.M. Brady, S.M. Smith, Characterization and propagation of uncertainty in diffusion-weighted MR imaging, Mag. Res. Med. 50 (2003) 1077–1088. [4] P.J. Roache, Quantification of uncertainty in computational fluid dynamics, Annu. Rev. Fluid. Mech. 29 (1997) 123–160. [5] M. Christie, V. Demyanov, Demet Erbas, Uncertainty quantification for porous media flows, J. Comp. Phys. 217 (2006) 143–158. [6] M.M. Putke, A.C. Taylor, P.A. Newman, L.L. Green, Approach for input uncertainty propagation and robust design in CFD using sensitivity derivatives, Trans. ASME 124 (2002) 60–69. [7] B. Sudret, Global sensitivity analysis using polynomial chaos expansions, Reliab. Eng. Syst. Safe. 93 (7) (2008) 964. [8] Y. Marzouk, D. Xiu, A stochastic collocation approach to Bayesian inference in inverse problems, Commun. Comput. Phys. 6 (4) (2009) 826. [9] X. Wan, G.E. Karniadakis, The Wiener–Askey polynomial chaos for stochastic differential equations, J. Comput. Phys. 209 (2005) 617. [10] B.J. Debusschere, H.N. Najm, P.P. Pebay, O.M. Knio, R.G. Ghanem, O.P. Le Matire, SIAM J. Sci. Comput. 26 (2) (2004) 698. [11] O.P. Le Maitre, O.M. Knio, H.N. Najm, R.G. Ghanem, A stochastic projection method for fluid flow: I. Basic formulation, J. Comput. Phys. 173 (2) (2001) 481. [12] H.N. Najm, Uncertainty quantification and polynomial chaos techniques in computational fluid dynamics, Ann. Rev. Fluid Mech. 41 (2009) 35. [13] S. Heidenreich, H. Gross, M.A. Henn, C. Elster, M. Bär, A surrogate model enables a Bayesian approach to the inverse problem of scatterometry, J. Phys.: Conf. Ser. 490 (1) (2014) 012007. [14] S. Heidenreich, H. Gross, M. Bär, Bayesian approach to the statistical inverse problem of scatterometry: comparison of three surrogate models, Int. J. Uncertain. Quant. 5 (2015) 511. [15] R. Millar, Maximum Likelihood Estimation and Inference, Wiley, 2011.

S. Heidenreich et al. / Measurement 97 (2017) 79–87 [16] The finite element solver was developed at WIAS. . [17] M.-A. Henn, H. Gross, F. Scholze, M. Wurm, C. Elster, M. Bär, A maximum likelihood approach to the inverse problem of scatterometry, Opt. Expr. 20 (2012) 12771–12786. [18] M.-A. Henn, S. Heidenreich, H. Gross, A. Rathsfeld, F. Scholze, M. Bär, Improved reconstruction by determination of line roughness in extreme ultraviolet scatterometry, Opt. Lett. 37 (24) (2012) 5229. [19] M.-A. Henn, H. Gross, S. Heidenreich, F. Scholze, C. Elster, M. Bär, Improved reconstruction of critical dimensions in extreme ultraviolet scatterometry by modeling systematic errors, Meas. Sci. Technol. 25 (4) (2014) 044003. [20] H. Gross, M.-A. Henn, S. Heidenreich, A. Rathsfeld, M. Bär, Modeling of line roughness and its impact on the diffraction intensities and the reconstructed critical dimensions in scatterometry, Appl. Opt. 51 (2012) 7384. [21] J. Perlich, F. Kamm, J. Rau, F. Scholze, G. Ulm, Characterization of extreme ultraviolet masks by extreme ultraviolet scatterometry, J. Vac. Sci. Technol. B15 (1997) 361–368. [22] F. Scholze, C. Laubis, Use of EUC scatterometry for the characterization of line profiles and line roughness on photomasks, in: Proc. SPIE 6792, 6792OU, 2008. [23] D. Firth, Bias reduction of maximum likelihood estimates, Biometrika 80 (1993) 27–38.

87

[24] BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP and OIML, Joint Committee for Guides in Metrology, Evaluation of measurement data – Supplement 1 – Propagation of distributions using a Monte Carlo method, JCGM 100:2008, 2008. [25] T.A. Germer, H.J. Patrick, R.M. Silver, B. Bunday, Developing an uncertainty analysis for optical scatteometry, in: Proc. SPIE 7272, 2009. [26] M.D. Mckay, R.J. Beckman, W.J. Conover, A Comparison of three methods of selecting values of input variables in the analysis of output from a computer code, Technometrics 42 (1) (1979) 55–61. [27] W.-L. Loh, On latin hypercube sampling, Ann. Stat. 24 (1996) 2058–2080. [28] J.C. Helton, F.J. Davis, Latin hypercube sampling and the propagation of uncertainty in analysis of complex systems, Reliab. Eng. Syst. Safe. 81 (2003) 23–69. [29] I.M. Sobol, Distribution of points in a cube and approximate evaluation of integrals, U.S.S.R. Comp. Maths. Math. 7 (1967) 86. [30] I.M. Sobol, On quasi-Monte Carlo integration, Math. Comp. Simul. 47 (1998) 103. [31] P. Bratley, B.L. Fox, ACM Trans. Math. Softw. 14 (1) (1988) 88. [32] S. Joe, F.Y. Kuo, ACM Trans. Math. Softw. 29 (1) (2003) 49. [33] S. Heidenreich, M-.A. Henn, H. Gross, B. Bodermann, M. Bär, Alternative methods for uncertainty evaluation in EUV scatterometry, in: SPIE 87890T-8, 2013.