In this chapter, we review nonlinear models for vector time series data and develop new nonparametric estimation and inference for them. Vector time series data exist widely in practice. In financial markets, multiple time series are usually correlated. When analyzing several interdependent time series, in general one should consider them as a single vector time series fitted by multivariate models, which provides a useful tool for modeling interdependencies among multiple time series and for simultaneously analyzing feedback and Granger causality effects. Since nonlinear features are widely observed in time series, we consider nonlinear methodology for modeling nonlinear vector time series data, which allows flexibility in the model structure and avoids the curse of dimensionality.
- multivariate threshold autoregressive model
- nonparametric smoothing
- generalized likelihood ratio
Multiple time series are of considerable interest in an array of domains, such as finance, economics, engineering and so on. The data are collected in time order and consist of several related variables of interest, for instance, the data of stock price indexes and the status data of important instruments such as shuttles. It is of much practical significance to model this kind of data well. Moreover, a lot of commonly seen multiple time series are correlated, which makes it reasonable to regard them as a single vector and to fit them using multivariate models. Multivariate models perform well in exploring the interdependencies among multiple time series and capturing the dynamic structure.
Plenty of contributions have been made in the field of parametric models for multivariate time series. For instance, Sims proposed vector autoregressive (VAR) models in 1980 , Engle and Kroner considered multivariate generalized autoregressive conditional heteroscedastic (GARCH) models in 1995 , and Tsay developed the multivariate threshold models in 1998 . Compared to parametric models, nonparametric models require less assumption about the model structure and are more flexible. Combined with the fact that nonlinearity widely exists in time series, it is ideal to model the multiple time series using nonparametric models. However, not much of achievements have been made about this. This is partly due to the complexity of nonparametric smoothing as well as the curse of dimensionality. With these objectives in mind, Jiang proposed the multivariate functional-coefficient model in 2014 , which provides a useful tool for modeling vector time series data.
In this chapter, we first review some vector time series models, next extend them to include an error-correction term by incorporating cointegration among integrated variables, then develop a single index model for choosing the smoothing variable and a variable selection method for the multivariate functional-coefficient models, and finally study multivariate time-varying coefficient models and related hypothesis testing problems.
The remainder of this chapter is organized as follows. In Section 2 we review vector autoregressive (VAR) models. In Section 3, we consider multivariate functional-coefficient regression models and their extensions, where a model selection rule is also proposed. In Section 4 we introduce multivariate time-varying coefficient models and propose a generalized likelihood ratio test. In Section 5 we make a conclusion and discuss some interesting research topics to be completed.
2. Review of VAR models
The vector autoregressive model is a generalization of the univariate autoregressive model for forecasting a vector of time series. This model was pioneered by Sims in Ref.  and it has acquired a prominent role in analyzing macroeconomic time series. Prior to 1980, large-scale statistical dynamic simultaneous equations model (DSEMs) was widely used in empirical macroeconomics, which often contained dozens or even hundreds of equations. As the economic environment has grown more complicated, the traditional simultaneous models have grown. Sims believed that since these models do not dichotomize variables into “endogenous” and “exogenous,” the exclusion restrictions used to identify the simultaneous equations models make little sense. Thus, he advocated the vector autoregressive model (VAR) to model the interrelationships among a set of macroeconomic variables. In the structure of VAR models, each variable is a linear function of past lags of itself and past lags of the other variables. Sims demonstrated that VARs provide a flexible and tractable framework for analyzing economic time series. While hardly relying on economic theorems, VAR models have proven efficient in capturing the dynamics of multivariate systems as well as forecasting . Specifically, a vector autoregressive model of order p [VAR(p)] has the following general form:
or the equation set
Using lag-operator L, Eq. (1) can be written as the following form:
In other words, the determinant of the matrix polynomial has no roots in and on the complex unit circle. If the stability conditions are satisfied and the process can be extended to the infinite past, then the VAR process is stationary.
For model (1), since the right-hand side consists of only predetermined variables and the error terms are assumed to be independent white noise with time-invariant covariance, each equation can be estimated by ordinary least squares (OLS). Zellner proved that the OLS estimator coincides with the generalized LS (GLS) estimator .
The celebrated model (1) is easy to fit, and its autoregressive structure allows one to study the feedback effects and the Granger causality. However, model (1) employs only the lagged values of yt for forecast and ignores other potentially important variables’ effect. In addition, as time evolved, the coefficients remain constant, which may contrast the real situations where the dynamic structure of the relationship among different time series involves with time.
3. Multivariate functional-coefficient regression models and extensions
We briefly reviewed VAR models in the previous section. This parametric method has been significantly developed and widely applied to econometric dynamics as well as other domains. An alternative to modeling vector time series is the nonparametric method, which requires much fewer assumptions on the model structure and may shed light on the later parametric fitting. To illustrate the basic idea of this approach, let us begin with the multivariate threshold autoregressive model .
3.1. Multivariate threshold autoregressive model
The multivariate threshold autoregressive model is a generalization of the univariate threshold autoregressive model . The idea is to partition one-dimensional variable into s regimes and impose an AR model with exogenous variables in each regime. Consider a
where p and q are nonnegative integers and
being a positive-definite matrix and
a sequence of serially uncorrelated random vectors with mean zero and covariance matrix
Model (4) is piecewise linear in the threshold space of
3.2. Multivariate functional-coefficient models
The multivariate functional-coefficient model has the following form:
For model (5), a weighted local least squares estimation method was provided in . Let
where ∥ ⋅ ∥ denotes the Euclidean norm,
Under certain conditions, the weighted estimators are asymptotically normal (see ).
Recall that, in model (5),
3.3. Extension of multivariate functional-coefficient models
Due to the fact that many economic factors are not stationary, classic regression analysis requiring the stationarity condition suffers from a great limitation. Cointegration analysis has become a formidable toolkit in analyzing non-stationary economic time series. The concept of cointegration goes back to Granger  and initiated a literal research boom. Engle & Granger proposed the well-known Engel-Granger test to examine whether there is a cointegrating relationship among a set of first-order integrated variables .
Motivated by Granger and Engel & Granger, Jiang proposed an error-correction version of model (5) by incorporating the cointegrating relationship of first-order integrated variables . This allows us to cope with the nonstationarity of vector time series and to improve the accuracy of forecasting.
Due to the widespread presence of cointegrating variables in finance and economics, model (7) should improve the practicability of model (5). However, model (7) requires specification of variable zt. This can be relaxed by using the idea of single index models. Recall that model (5) can be represented in succinct form (6). The similar operations can be applied to model (7). Now set
It can be shown that
shares the same asymptotic normality as the Oracle weighted estimator in the sense that it knows the true value of
3.4. Variable selection of multivariate functional-coefficient models
In this section, we consider variable selection of model (6). Increasing the lags p and q will necessarily reduce the sum of squared errors. However, doing so will increase the burden of coefficient estimation and may also lead to overfitting. Hence, for the multivariate functional-coefficient model, order selection is of much importance.
Two widely used model selection criteria are Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC). However, these stepwise methods yield heavy burden on computation and furthermore bring difficulty in establishing asymptotics for the estimation of selected models. The problems become more severe for high-dimensional data. Various regularization methods have been proposed to deal with these problems. Among them, a popular approach, called LASSO, proposed by Tibshirani, performs variable selection and parameter estimation simultaneously. See Ref. . For univariate varying-coefficient regression models with i.i.d. data, Wang and Xia  developed a shrinkage estimation method by combining the idea of group LASSO  and kernel smoothing. In the following we develop a shrinkage estimation method for multivariate functional-coefficient model (6):
where the functional-coefficient matrix
4. Multivariate time-varying coefficient models
Running the local linear smoother for model (12), we minimize
over P and Q, where
Using similar arguments to , we can show that this estimator is asymptotically normal with mean zero and variance
4.1. Generalized likelihood ratio tests
The multivariate time-varying coefficient regression model is flexible and powerful to estimate the dynamic changes of coefficients. After fitting a given dataset, some important questions arise, for example, whether the coefficient functions are actually constant or of some particular forms? This leads to statistical hypothesis testing. To answer these questions, we develop generalized likelihood ratio statistics to test corresponding hypothesis testing problems about the coefficient functions .
For the multivariate time-varying coefficient model (12), assume
Consider the following hypothesis testing problem
In the following example, we consider the case when
The significance level is set to be 5%, and the critical values in simulations are calculated similarly by using the conditional bootstrap method in Ref.  for each given
Figure 1 displays the power curves in difference scenarios. We can tell from Figure 1 that the patterns of power curves look like half of an inverted normal density. All the curves rise monotonically from a height equal to the significance level of 5% until eventually it reaches its maximum height of around 90%. It is evident from Figure 1 that the test is powerful for all three different distributions of error terms. Moreover, the test becomes more powerful as sample size increases. These indicate that the proposed test keeps the size and is powerful for distinguishing the difference between the null and the alternative.
In this chapter, we have reviewed some parametric and nonparametric methods for modeling nonlinear vector time series data, which include the VAR model, the multivariate threshold autoregressive model, and the multivariate functional-coefficient regression model. These models have great significance in econometrical and statistical theory and application. Based on the weighted local least square estimation, we have proposed a variable selection method for the functional-coefficient model. This model selection procedure is applicable to the proposed multivariate single index models and multivariate time-varying coefficient models. We have also extended the generalized likelihood ratio test to the time-varying coefficient model and demonstrated its performance through simulation. The proposed methodology is very useful for modeling nonlinear dynamic structures inherited in financial data. However, there are many problems remain unsolved for our procedure, such as the limiting theory about the proposed methodology. Future work includes, but not limited to, extending our models to nonstationary settings and exploring their performance in different applications.
This research was supported in part by NSFC grant 71361010 and by funds provided by the University of North Carolina at Charlotte.
Sims CA. Macroeconomics and reality. Econometrica. 1980; 48(1):1-48. DOI: 10.2307/1912017
Engle RF, Kroner KF. Multivariate simultaneous generalized ARCH. Econometric Theory. 1995; 11(1):122-150. DOI: 10.1017/S02664666000009063
Tsay RS. Testing and modeling multivariate threshold models. Journal of the American Statistical Association. 1998; 93(443):1188-1202. DOI: 10.1080/01621459
Jiang J. Multivariate functional-coefficient regression models for nonlinear vector time series data. Biometrika. 2014; 101(3):689-702. DOI: 10.1093/biomet/asu011
Zellner A. An efficient method of estimating seemingly unrelated regressions and tests for aggregation bias. Journal of the American Statistical Association. 1962; 57(298):348-368. DOI: 10.2307/2281644
Tong H. On a threshold model. In: Chen C, editor. Pattern Recognition and Signal Processing. Netherlands: Sijthoff & Noordhoff; 1978. p. 575-586. DOI: 10.1007/978-94-009-9941-1_24
Chen R, Tsay RS. Functional-coefficient autoregressive models. Journal of the American Statistical Association. 1993; 88(421):298-308. DOI: 10.2307/2290725
Hastie T, Tibshirani R. Varying-coefficient models. Journal of the Royal Statistical Society. Series B (Methodological). 1993; 55(4):757-796
Cai Z, Fan J, Yao Q. Functional-coefficient regression models for nonlinear time series. Journal of the American Statistical Association. 2000; 95(451):941-956. DOI: 10.2307/2669476
Huang JZ, Shen H. Functional coefficient regression models for non-linear time series: A polynomial spline approach. Scandinavian Journal of Statistics. 2004; 31(4):515-534. DOI: 10.1111/j.1467-9469.2004.00404.x
Campbell JY, Yogo M. Efficient tests of stock return predictability. Journal of Financial Economics. 2006; 81:27-60
Paye BS, Timmermann A. Instability of return prediction models. Journal of Empirical Finance. 2006; 13:274-315
Cai Z, Wang Y. Testing predictive regression models with nonstationary regressors. Journal of Econometrics. 2014; 178:4-14
Cai Z, Li Q, Park JY. Functional-coefficient models for nonstationary time series data. Journal of Econometrics. 2009; 148:101-113
Sims CA. Money, income, and causality. American Economic Review. 1972; 62:540-552
Psaradakis Z, Ravn MO, Sola M. Markov switching causality and the money-output relationship. Journal of Applied Econometrics. 2005; 20:665-683
Berger H, Österholm P. Does money still matter for U.S. output? Economics Letters. 2009; 102:143-146
Subrahmanyam A, Titman S. Feedback from stock prices to cash flows. Journal of Finance. 2001; 56:2389-2413
Åström KJ, Murray RM. Feedback Systems: An Introduction for Scientists and Engineers. Princeton University Press: Princeton, NJ; 2010
Granger CWJ. Some properties of time series data and their use in econometric model specification. Journal of Econometrics. 1981; 16(1):121-130. DOI: 10.1016/0304-4076(81)90079-8
Engle RF, Granger CWJ. Co-integration and error correction: Representation, estimation, and testing. Econometrica. 1987; 55(2):251-276. DOI: 10.2307/1913236
Tibshirani, R. Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society. Series B (Methodological). 1996; 58(1):267-288. DOI: 10.1111/j.1467-9868.2011.00771.x
Wang H, Xia Y. Shrinkage estimation of the varying coefficient model. Journal of the American Statistical Association. 2009; 104(486):747-757. DOI: 10.1198/jasa.2009.0138
Yuan M., Lin Y. Model Selection and Estimation in Regression With Grouped Variables. Journal of the Royal Statistical Society Series B (Statistical Methodology). 2006; 68(1):49-67. DOI: 10.1111/j.1467-9868.2005.00532.x
Liu Y. Generalized quasi-likelihood ratio statistics for multivariate time-varying coefficient regression models [dissertation]. Charlotte: University of North Carolina at Charlotte; 2013. 47 p
Fan J, Zhang C, Zhang J. Generalized likelihood ratio statistics and Wilks phenomenon. The Annals of Statistics. 2001; 29(1):153-193. DOI: 10.1214/aos/996986505
Fan J, Jiang J. Nonparametric inference for additive models. Journal of the American Statistical Association. 2005; 100:890-907