## Abstract

This chapter addresses the stochastic modeling of functional response, which is a major concern in engineering implementation. We first introduce a general framework and several conventional models for functional data, including the functional linear model, penalized regression splines, and the spatial temporal model. However, in engineering practice, a naive mathematical modeling of functional response may fail due to the lack of expressing the underlying physical mechanism. We propose a series of quasiphysical models to handle the functional response. A motivating example of metamaterial design is thoroughly discussed to demonstrate the idea of quasiphysical models. In real applications, various uncertainties have to be taken into account, such as that of the permittivity or permeability of the substrate of the metamaterial. For the propagation of uncertainty, simulation‐based methods are discussed. A Bayesian framework is presented to deal with the model calibration in the case of functional response. Experimental results illustrate the efficiency of the proposed method.

### Keywords

- functional response
- meta model
- Bayesian uncertainty quantification
- model calibration
- metamaterial design

## 1. Introduction

In recent years, computer experiments have become widely adopted in both engineering applications and scientific research to replace or support their physical counterparts. Functional response is the mathematical representation of system behaviors, where the data are collected over an interval of some input indices. With the advance of modern simulation and experiment technology, accessing functional data becomes easier. Functional response can be in the form of one‐dimensional data such as a curve or higher dimensional data such as an image, which can provide better physical insights. However, even with the advancement of computer technology, full simulation based on a finite element method or a finite difference method still takes an extensive amount of time. To reduce the amount of simulation time, historical simulated data are usually used to build a cheaper metamodel [1], in which the functional response of unobserved input can be predicted by either regression or interpolation. The simplest representation of functional data can be considered basis expansion, where polynomials are used to formulate the input‐output relation [2]. For frequency response analysis, Fourier series are usually applied to replace the polynomials [3]. Both methods are categorized as linear regression, which requires parameter estimations. Nonparametric approaches were also used to analyze functional data in many scientific and engineering fields [4]. The purpose of building these models is to provide the “best” estimate regarding the given data, while providing a statistical scheme for prediction at unobserved inputs.

In this chapter, we provide a more sophisticated approach to naturally analyze functional responses, which may suggest more insightful conclusions that may not be apparent otherwise. We introduce one motivating example of functional response in computer experiments. In the design of metamaterial, the goal is to establish a relationship between the physical dimensions of a unit cell and its electromagnetic (EM) frequency response [5]. In practice, designers usually evaluate the EM properties of a metamaterial microstructure via full‐wave simulation data, such that corresponding adjustments are constantly made to the design (dielectric architecture, microstructure topologies, etc.) until a desired performance is achieved. Figure 1 depicts an example of unit cell design whose response phases differ on a frequency span along with the varying geometric parameter. Naïve regression‐based metamodels fail in dealing with such a problem because they require building regressions for each output, which could be very expensive and leaves the correlation between different frequencies unutilized. Moreover, when resonance is involved, the functional data cannot be well described by polynomials or splines. However, this can be overcome by some quasiphysical models, which explore the essential physical mechanism. In addition, a more general two‐stage modeling scheme can be applied, where in Stage I, we approximate the response with rational functions. This allows us to decompose the continuous response into a few discrete parameters. Stage II consists of a nonparametric metamodel to capture the input dependence.

## 2. General models for functional response

Various statistical models, including the spatial temporal model, functional linear model, and penalized regression splines, have been widely discussed in the past. Most models share a unified expression that sums up a mean function

where

### 2.1. Functional linear model

To model the functional response, the primary task is to estimate the mean function

with basis functions

Given a certain index ** x**, whereas

*. However, smoothing over*f

*componentwise, using penalized splines, can enhance the efficiency of estimates [6].*f

Penalized regression splines implement estimation of smoothing basis functions in functional linear models by minimizing the penalized least squares. They are widely adopted in modeling functional responses due to their easy implementation and low computational cost [6].

Noted that the primary purpose of applying penalized regression splines is to estimate the basis function

with

where

where

The smoothing method with penalized splines estimates also requires selection of the number of knots and the order * p*, which may vary from case to case. Fortunately, the estimates are not sensitive to these choices; and cubic splines are suggested in most cases [6], which ensure continuous second‐order and piecewise continuous third‐order derivatives at the knots. Meanwhile, knots are usually selected from the interval over which

*is evenly distributed, or*f

*.*f

### 2.2. Spatial temporal model

The spatial temporal model is defined by the sum of a mean function,

Both of the preceding models aim to represent the functional data in terms of their mean functions. In contrast, the spatial temporal model utilizes the property of the normal distribution of the residuals; thus, the output can be seen as a realization of a Gaussian random field. We assume a mean function

where

where

where * i,j*) element

*‐dependent hyperparameter that controls the properties of the covariance.*f

Suppose we have obtained observation

** Step 1:** For

** Step 2:** According to the data

** Step 3**: For new index

y

### 2.3. Quasiphysical model

Metamaterial frequency response, for example, modeling the resonance response is often quite challenging and cannot be achieved with the models introduced above. This is due to that the above models are based upon linear regression and simply encode the index dependence within the linear index‐dependent smooth basis functions. However, when distinct resonance peaks exist, a common scenario in radio frequency engineering, fitting to these smooth basis functions often, leads to poor accuracy [10]. To deal with these problems, we tend to utilize some underling physical mechanism and establish a quasiphysical modeling method. For example, the mean function

Then, we have

where

The aforementioned Drude‐Lorentz model allows high accuracy only when the metamaterial system works in a static or quasistatic regime, such that the metamaterial architecture can be seen as a single piece of effective medium. However, for complex metamaterial systems, the working regime is beyond static; thus, approximation accuracy by such a model is severely deteriorated. We noted that EM waves propagate through each layer of metamaterial like a current on transmission lines. Such a perspective transfers the EM field problems to circuit problems. Hence, function response to a continuous spectrum is reduced to discrete LRC (short form of inductor, resistor and capacitor) networks. We propose a two‐stage modeling scheme, where in the first stage, a vector fitting (VF) technique is adopted to provide accurate rational approximation to frequency responses with distinct resonances. Its results are easily interpreted as an equivalent circuit. The approximation accuracy to a frequency response and its corresponding equivalent circuits are shown in part (a) and (b) of Figure 3, respectively. And in Stage II, the empirical circuit elements are then taken as the target response in statistical models to establish the mapping input‐output relation by performing regression, which also allows predictions at unobserved input sites. Part (c) of Figure 3 presents the GP surface built of circuit parameters over two input variables. A graphical display of this two‐stage approach is illustrated in Figure 4. To predict functional response at unobserved input, it is implemented by first predict the presenting circuit elements and then recover the response.

## 3. Uncertainty quantification

In the engineering modeling and design, uncertainty is ubiquitous, due to the inability to specify a “true” input or model parameter. Quantifying the uncertainty of the model, e.g., in the form of predictive confidence intervals, is of great importance for decision making and advanced design [1]. In general, uncertainty quantification can be divided into two major types of problem: forward uncertainty propagation and inverse assessment of model and parameter uncertainties [12].

The full relationship between experimental output

where

where

In forward problems, with an uncertain input ** x** and given model parameters

**, the model output**θ

**such that it makes the model’s output fit the experimental data as accurate as possible (or satisfy some precision requirements).**θ

### 3.1. Metamodel‐based uncertainty propagation

The main problem in analyzing uncertainty propagation is obtaining an analytical representation of the metamodel for any arbitrary (uncertain) input values. Given its probability density, the Bayesian framework can provide a probability measure of random inputs on the output field. The purpose of such an operation is to evaluate the influence of an uncertain input on the model response.

Assume that the Gaussian process regression model is trained on a dataset with the input

At a deterministic test input

and variance

where * i*th element of column vector

*th element is given by*ij

The final goal is to propagate uncertainty through the link function

and the variance

The uncertainty propagation is induced by the variability of the input variable. For example, in metamaterial engineering, the dimension of a design parameter, say the thickness of the metallic microstructure layer, could differ from what has been instructed during the manufacturing processes. From measurements, the value of such a variable would rather follow a distribution than be pre‐specified as an exact value. Therefore, the analysis of uncertainty propagation is needed to be in the metamaterial design process.

### 3.2. Bayesian calibration

Compared to uncertainty forward propagation, the inverse problem is more difficult yet of great importance in enhancing the fidelity of metamodels. Two major aspects concerning the inverse problem are measuring model discrepancy and model calibration. In this chapter, we use the formulation to address both issues within an updating process, similar to that proposed in Ref. [12].

#### 3.2.1. The model

In this section, we introduce the details of performing Bayesian calibration with regard to Eq. (13). The calibration parameters, denoted by

Before offering the detailed statistics for uncertainty quantification, we must note that the purpose of parameter calibration is to provide an accurate prediction with the metamodel with a small amount of data. An even smaller amount of experimental data is acquired to calibrate and validate the main model. To select the “best” experiment samples, uniform experimental design techniques are usually applied [6]. A Latin hypercube sampling, for example, is widely used for such cases, mainly due to its good coverage property [13].

The data corresponding to the metamodel

where

can provide smooth samples to infer the latent function variable. In Eq. (21), the value of hyperparameters can be inferred via Markov Chain Monte Carlo (MCMC) techniques.

#### 3.2.2. Data and prior distribution

Let us denote the matrix of basis functions * Stage I* given the simulation and observation of functional response. Meanwhile, they are normally distributed given the full set of parameters

The goal of calibration is to obtain

where

To specify the variance matrix of * i*,

*) element is*i'

*,*i

*) element*j

where

To derive the posterior distribution under the Bayesian framework, the prior distributions of parameters,

where

#### 3.2.3. Posterior distribution

Conditional on full data, the independence of parameters leads to the full joint posterior distribution

To obtain

where

#### 3.2.4. Calibration and prediction

Since the posterior distribution specified in Eq. (27) is a highly intractable function of

However, the purpose of calibration of parameters is to predict the real process rather than achieve their values. Therefore, in practice, we are rather more interested in expressing the posterior distribution of

where

and covariance

Inference about

So far, we have accomplished calibrating a metamodel in the Bayesian framework using the experimental data, which accounts for parameter uncertainty and corrects the model discrepancy and experimental uncertainty.

## 4. Simulation study

This section demonstrate the results obtained using the Bayesian uncertainty quantification framework for the metamaterial design problem with the models described in Sections 2 and 3, with examples. Of both propagation and inverse assessment, the overall model is formulated in Eq. (13), where geometric variable

To demonstrate parameter calibration within the metamaterial modeling and design, we consider an example where the real part of the permittivity of a dielectric material,

## 5. Conclusion

In this chapter, we review several conventional model for functional response and present the quasiphysical model for functional response. Compared with the conventional models, this model can reveal the physical insight more clearly and make better use of historical experience. The two‐stage method was presented to model the frequency response of metamaterial and facilitate the design process. Using this approach, we decomposed the complex modeling problem into a vector fitting‐based equivalent circuit modeling process and a GP regression process, which can easily generate the mapping function from the structure’s geometric design. The predictive property of this model enables the massive reduction of time‐consuming simulations.

Another important topic with this chapter was the development and application of a Bayesian uncertainty quantification approach in dealing with functional response. Both forward uncertainty propagation and inverse assessment of the model were discussed, and a Bayesian framework was presented with simulation experimental results to deal with the model calibration for functional response. We envision that our two‐stage approach can be generalized to model any functional responses of a rational form. With the Bayesian framework for the functional data of computer experiments, we were able to incorporate our prior knowledge into the model and obtain a probabilistic measure of the uncertainty associated with metamaterial system design. This general methodology enables researchers and designers to achieve high efficiency and accuracy in modeling functional response with a considerably small amount of data. With a Bayesian calibration framework, we are able to constantly increase the precision of predictions of the functional response at unobserved sites, thus replacing expensive physical experiments.

## Acknowledgments

This work was supported by the Shenzhen Key Laboratory of Data Science and Modeling.

## References

- 1.
Kennedy M, O’Hagan A. Predicting the output from a complex computer code when fast approximations are available. Biometrika. 1998; 87 :1–13. DOI: 10.1093/biomet/87.1.1 - 2.
Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning. New York:Springer; 2001. p. 745. DOI: 10.1007/978‐0‐387‐84858‐7 - 3.
Ramsay J, Silverman B. Applied functional data analysis: Methods and case studies. New York: Springer; 2002. DOI: 10.1007/b98886 - 4.
Cressie A. Statistics for spatial data. Terra Nova. 1992; 4 (5):613–617. DOI: 10.1002/9781119115151 - 5.
Liu R, Ji C, Mock J, Chin J, Cui T, Smith D. Broadband ground‐plane cloak. Science. 2009; 323 (5912):366–369. DOI: 10.1126/science.1166949 - 6.
Fang KT, Li R, Sudjianto A. Design and modeling for computer experiments. Boca Raton, FL, USA: Chapman and Hall/CRC Press. 2005. DOI: 10.1201/9781420034899 - 7.
Ruppert D, Carroll R. Theory & methods: Spatially-adaptive penalties for spline fitting. Australian & New Zealand Journal of Statistics. 2000; 42 (2):205–223. DOI: 10.1111/1467‐842X.00119 - 8.
Graven P, Wahba G. Smoothing noisy data with spline functions. Numerische Mathematik. 1979; 31 :377–403. DOI:10.1007/BF01404567 - 9.
Carroll R, Chen R, George E, Li T, Newton H, Schmiediche H, et al. Ozone exposure and population density in Harris County, Texas. Journal of the American Statistical Association. 1997; 92 (438):392–404. DOI: 10.2307/2965684 - 10.
Liu B, Ji C. Bayesian nonparametric modeling for rapid design of metamaterial microstructures. International Journal of Antennas & Propagation. 2014; 2014 :187–187 - 11.
Cui T J, Smith D, Liu R. Metamaterials: Theory, Design, and Applications. New York: Springer Publishing Company, Incorporated; 2009. DOI: 10.1007/978‐1‐4419‐0573‐4. - 12.
Kennedy M, O’Hagan A. Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 2001; 63 (3):425–464. DOI:10.1111/1467‐9868.00294 - 13.
McKay M, Beckman R, Conover W. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics. 1979; 21 (2):239–245. DOI:10.2307/1271432 - 14.
Kennedy M, O’Hagan A. Supplementary details on Bayesian calibration of computer. Technical Report., University of Nottingham. Statistics Section; 2001 - 15.
Neal RM, Probabilistic inference using Markov Chain Monte Carlo methods. 1997