Open access peer-reviewed chapter

The Periodic Restricted EXPAR(1) Model

Written By

Mouna Merzougui

Submitted: 01 July 2020 Reviewed: 17 September 2020 Published: 24 November 2020

DOI: 10.5772/intechopen.94078

From the Edited Volume

Recent Advances in Numerical Simulations

Edited by Francisco Bulnes and Jan Peter Hessling

Chapter metrics overview

386 Chapter Downloads

View Full Metrics

Abstract

In this chapter, we discuss the nonlinear periodic restricted EXPAR(1) model. The parameters are estimated by the quasi maximum likelihood (QML) method and we give their asymptotic properties which lead to the construction of confidence intervals of the parameters. Then we consider the problem of testing the nullity of coefficients by using the standard Likelihood Ratio (LR) test, simulation studies are given to assess the performance of this QML and LR test.

Keywords

  • nonlinear time series
  • periodic restricted exponential autoregressive model
  • quasi maximum likelihood estimation
  • confidence interval
  • LR test

1. Introduction

Since the 1920s, linear models with Gaussian noise have occupied a prominent place, they have played an important role in the specification, prevision and general analysis of time series and many specific problems were solved by them. Nevertheless, many physical and natural processes exhibit nonlinear characteristics that are not taken into account with linear representation and are better explicated and fitted by nonlinear models. For example, ecological and environmental fields present phenomena close to the theory of nonlinear oscillations, such as limit cycle behavior remarked in the famous lynx or sunspot series, leading to the consideration of more complex models from the 1980s onwards. A first nonlinear model possible is the Volterra series which plays the same role as the Wold representation, for linear series. The interest of this representation is rather theoretical than practical, for this reason, specific parametric nonlinear models were presented as the ARCH and Bilinear models suitable for financial and economic data, threshold AutoRegressif TAR and exponential AREXPAR models suitable for ecological and meteorological data. These nonlinear models have been applied with great success in many important real-life problems. Basics of nonlinear time series analysis can be found in [1, 2, 3] and references therein.

Amplitude dependent frequency, jump phenomena and limit cycle behavior are familiar features of nonlinear vibration theory and to reproduce them [4, 5] introduced the exponential autoregressive EXPAR models. The start was by taking an autoregressive AR model Yt, say, and then make the coefficients dependent in an exponential way of Yt12.

Several papers treated the probabilistic and statistic aspects of EXPAR models. A direct method of estimation is proposed by [5], it consists to fix the nonlinear coefficient in the exponential term at one of a grid of values and then estimate the other parameters by linear least squares and use the AIC criterion to select the final parameters, necessary and sufficient conditions of stationarity and geometric ergodicity for the EXPAR1 model are given by [6], the problem of estimation of nonlinear time series in a general framework by conditional least squares CLS and maximum likelihood ML methods is treated by [7] with application in EXPAR models, a forecasting method is proposed by [8], the LAN property was shown in [9] and asymptotically efficient estimates was constructed there for the restricted EXPAR1, a genetic algorithm for estimation is used in [10], Bayesian analysis of these models is introduced in [11], a parametric and nonparamtric test for the detection of exponential component in AR1 is constructed by [12], sup-tests are constructed by [13] with the trilogy Likelihood Ratio (LR), Wald and Lagrange Multiplier (LM) for linearity in a general nonlinear AR1 model with EXPAR1 as special cases, the extended Kalman filter EKF is used in [14]. Given that nonlinear estimation is time consuming [15] proposed to estimate heuristically the nonlinear parameter from the data and this is a very interesting remark because when the nonlinear parameter is known we get the Restricted EXPAR model. The applications of the EXPAR model are multiple: ecology, hydrology, speech signal, macroeconomic and others see, for example, [16, 17, 18, 19, 20, 21].

On the other hand, fitted seasonal time series exhibiting nonlinear behavior such cited before and having a periodic autocovariance structure by SARIMA models will be inadequate. These models are linear and the seasonally adjusted data may still show seasonal variations because the structure of the correlations depends on the season. The solution is the use of a periodic version of EXPAR models. The notion of periodicity, introduced by [22], was used to fit hydrological and financial series and allowed the emergence of new classes of time series models such as Periodic GARCH, Periodic Bilinear, MPAR model. Motivated by all this, we introduced recently the Periodic restricted EXPAR1 model see [23], which consists of having different restricted EXPAR1 for each cycle and we established a most stringent test of periodicity since a periodic model is more complicated than a nonperiodic one and its consideration must be justified. We studied the problem of estimation by the least squares (LS) method in [24] and the test of Student was used for testing the nullity of the coefficients in the application. Traditionally, the step of estimation must be followed by tests of nullity of coefficients and the major tests used are Wald, LR and LM tests. We used a Wald test for testing the nullity of one coefficient and consequently testing linearity in [25].

In this chapter, we will present the quasi maximum likelihood (QML) estimation of the parameters, which are the LS estimators in [24] under the assumption that the density is Gaussian, these estimators are asymptotically normal under quite general conditions. This will play a role in the construction of the confidence interval for the parameters and then we treat the problem of testing the nullity of parameters which lead us to a linearity test using the standard and well known LR test. This test is based on the comparison between the maximum of the constrained and unconstrained quasi log likelihood, see for example [26] or [27], the null hypothesis is accepted, if the difference is small enough or equivalently H0 ought to be rejected for large values of the difference. The problem is standard because the periodic model is restricted, i.e. the nonlinear parameter is known and for the other parameters 0 is an interior point of the parameter space, then the LR statistic asymptotically follows the χ2 distribution under H0 just like the Wald test, but we chose the former because it does not require estimation of the information matrix. It is known that the two tests are asymptotically equivalent and may be identical see [26] for more details.

The chapter is organized as follows. In Section 2, we introduce the Restricted PEXPAR model and we present the asymptotic normality of the QML estimators and we construct confidence intervals of the parameters. Section 3 provides the LR test for nullity of one coefficient and a test for linearity, a small simulation shows the efficiency of these tests.

Advertisement

2. The Periodic Restricted EXPAR1 model and QML estimation

2.1 Restricted PEXPAR1 model

Let Ytt1 be a seasonal stochastic process with period SS2.

Definition 1

The process Ytt1 is a Periodic Restricted EXPonential AutoRegressive model (restricted PEXPAR1) of order 1 if it is a solution of the nonlinear difference equation given by

Yt=φt,1+φt,2expγYt12Yt1+εt,tN,E1

where εtt1 is iid0σt2, φt,1 and φt,2 are the autoregressive parameters and γ>0 is the known nonlinear parameter. A heuristic determination of γ from data is

γ̂=logεmax1tnYt2,E2

where ε is a small number and n is the number of observations. (cf. [15]).

The autoregressive parameters and the innovation variance are periodic of period S, that is,

φt+kS,1=φt,1,φt+kS,2=φt,2andσt+kS2=σt2,k,tN.E3

To point out the periodicity, let t=i+,i=1,,S and τN, then Eq. (1) becomes

Yi+=φi,1+φi,2expγYi+12Yi+1+εi+,i=1,,S,τNE4

In Eq. (4), Yi+ is the value of Yt during the i-th season of the cycle τ and φi,1,φi,2 are the model parameters at the season i. It is clear that the parameters depend on Yi+1 in the sense that for large Yi+1 we have φi,1+φi,2expγYi+12φi,1 while for small Yi+1: φi,1+φi,2expγYi+12φi,1+φi,2 of course the change is done smoothly between these regimes. In application, the restricted PEXPAR1 model is fitted to seasonal time series displaying nonlinearity features like amplitude dependent frequency.

These forms of models are new in the literature of the time series it is interesting to make several simulations to see their characteristics. An important fact is their property of non normality as is shown by histogram in Figure 1 and confirmed by the test of Shapiro Wilk where the pvalue=0.008226 is less than 0.05. The realization of the process (A) is given in Figure 1 from it and from the correlogram we can see that the process is stationary in each season due to the fast decay to 0 as h increases. Another interesting fact, that these types of models can exhibit, is the limit cycle behavior which is a well known feature in nonlinear vibrations and is one of possible mode of oscillations. Such phenomena is shown in Figure 2 from model (B).

Figure 1.

Realization of (A) with corresponding histogram and correlogram.

Figure 2.

Limit cycle from PEXPAR21 model.

Model (A):Y1+2τ=0.3+2expY2τ2Y2τ+ε1+2τY2+2τ=0.8+expY1+2τ2Y1+2τ+ε2+2τ.E5
Model (B):Y1+2τ=0.21.5expY2τ2Y2τ+ε1+2τY2+2τ=0.8+0.3expY1+2τ2Y1+2τ+ε2+2τ.E6

2.2 QML Estimation

Let φ¯=φ¯1φ¯SR2S the parameter vector where φ¯i=φi,1φi,2, i=1,,S. We want to estimate the true parameter φ¯0 from observations Y1,,Yn where n=mS which means that we have m full period of data. The problem is resolved by the QML method and under the conditions:

A1: The Periodical restricted exponential autoregressive parameters φ¯ satisfy the stationary periodically condition of (1). A sufficient condition is given by φi,1<1,φi,2R,i=1,,S.

A2: The periodically ergodic process YttN is such that EYt4<, for any tN.

Periodic stationarity has not been treated for this model so stationarity is required for each season hence A1. We can replace the assumption A2 by Eεt4<, for any tN, since Eεt4<EYt4<. Under this condition significant outliers are improbable and the existence of the information matrix is guaranteed.

Given initial value Y0, the conditional log likelihood of the observations evaluated at φ¯ depends on f . The QML estimator is obtained by replacing f by the N0σt2:

Lnφ¯Y1Yn=mS2log2πm2i=1Slogσi2i=1Sτ=0m1Yi+φi,1+φi,2expγYi+12Yi+12σi22,E7

(assuming) σi0.

Let φ¯̂ the QML estimator, one can see that maximizing Ln is equivalent to minimization of the quantity:

Qnφ¯=1nt=1nYtφt,1+φt,2expγYt12Yt12.E8

The initial value is unknown but its choice is not important for the asymptotic behavior of the QML estimator so we put Y0=0, which defines the operational criterion

Q˜nφ¯=1Si=1SQ˜i,mφ¯iE9

and

Q˜i,mφ¯i=1mτ=0m1Yi+φi,1+φi,2expγYi+12Yi+12.E10

The first order condition of the QML minimization problem is a system of 2S linear equations with 2S unknowns. The solution is

φ̂i,1φ̂i,2=τ=0m1Y+i12τ=0m1Y+i12expγY+i12τ=0m1Y+i12expγY+i12τ=0m1Y+i12exp2γY+i121×τ=0m1Y+i1Y+iτ=0m1Y+i1Y+iexpγY+i12σ̂i2=1mτ=0m1Y+iφ̂i,1+φ̂i,2expγY+i12Y+i12.E11

We remark that the QML estimator is the LS estimator and we can proof the next theorem in the same way.

Theorem

The QML estimator is strongly consistent and we have for i=1,,S

mφ̂i,1φi,1φ̂i,2φi,2mLN0¯2σi2EYi12EXi12expγYi12EYi12expγYi12EYi12exp2γYi121.E12

Furthermore, φ¯̂i,m and φ¯̂j,m are asymptotically independant, ij,i,j=1,,S.

Proof

The proof is very standard in the literature of time series. The consistency is based on an ergodicity argument and for the normality a central limit version for martingale differences is used. The detail is similar to the LSE (see [24]) hence it is omitted. The independence of the εi+ implies that all the terms for ij are zero, this implies that mφ¯̂i,mφ¯i and mφ¯̂j,mφ¯j,ij, are asymptotically uncorrelated.

The QML estimators (Eq. (4)) yields a point estimator, a confidence interval (CI) gives a region where the parameters fall in with a given probability (usually 95% or 90%). Based on the asymptotic normality of the QML estimators, with asymptotic probability 1α,φi,j is in the interval

φ̂i,j±Φ1α/2σ̂miΓijj,j=1,2,i=1,,S,E13

where

Γi=EYi12EYi12expγYi12EYi12expγYi12EYi12exp2γYi121,E14

and Φ1α/2 is the 1α/2 quantile of the standard normal distribution. That is, the CI contains the true parameters in 1001α% of all repeated samples.

To examine the performance of the QML estimators, we construct CI of the parameters from the simulation of restricted PEXPAR21 model with parameters: φ¯1=0.8,1.2' and φ¯2=0.40.9' with sizes n=200,500 and 1000 and for the significance levels: α=10% and 5% and 1000 replications. From the Tables 13 we deduce that the parameters are well estimated and when n increases the length of CI decreases showing that the estimates are consistent. Obviously, a higher confidence level produces wider CI.

n=200CIφ1,1CIφ1,2CIφ2,1CIφ2,2
α=10%0.82060.77961.1136,1.25760.3801,0.41190.95590.8156
α=5%0.81260.76611.0893,1.26180.3808,0.41940.99670.8260

Table 1.

CI of parameters for n = 200.

n=500CIφ1,1CIφ1,2CIφ2,1CIφ2,2
α=10%0.80380.78791.1551,1.21130.3874,0.40010.90150.8470
α=5%0.80970.79121.1783,1.24530.3873,0.40200.91040.8448

Table 2.

CI of parameters for n = 500.

n=1000CIφ1,1CIφ1,2CIφ2,1CIφ2,2
α=10%0.80270.79521.1909,1.21910.3978,0.40400.90720.8793
α=5%0.80300.79381.1797,1.21390.3958,0.40340.91190.8791

Table 3.

CI of parameters for n = 1000.

Advertisement

3. Likelihood Ratio tests

3.1 Test for the Nullity of One Coefficient

The asymptotic normality of the QML in Eq. (12) can be exploited to perform tests on the parameters. This problem is very standard, especially when 0 is an interior point of the parameter space and can be done with the trilogy: Wald, LR and LM tests. We treated the former in [25] and in this chapter, we will use the LR test which is based upon the difference between the maximum of the likelihood under the null and under the alternative hypotheses and has the advantage of not estimating information matrix. In this section, we are interested in testing assumptions of the form

H0:φi,2=0orH0:φi,1=0vsH1:φi,20orH1:φi,10,E15

for some given i. Under H1, we have the QML estimator φ̂¯i given by Eq. (11) and mean square error Q˜i,mφ̂¯i given by Eq. (10) and φ˜¯i=φ˜i,10, is the QML estimator given under H0 where

φ˜i,1=τ=0m1Y+i1Y+iτ=0m1Y+i12E16

and the corresponding mean square error under the null

Q˜i,mφ˜¯i=1mτ=0m1Yi+φ˜i,1Yi+12.E17

The usual LR statistic is

λi,m=Lφ˜¯iσ˜i2Lφ̂¯iσ̂i2=Q˜i,mφ̂¯iQ˜i,mφ˜¯im2E18

then the test rejects H0 at the asymptotic level α when

LRi,m=2logλi,m=mlogQ˜i,mφ˜¯iQ˜i,mφ̂¯i>χ121α,E19

where χ121α is the 1αquantile of the χ2 distribution with 1 degree of freedom.

In the same manner we can test the nullity of φi,1 by taken φ˜¯i=0φ˜i,2 and

Q˜i,mφ˜¯i=1mτ=0m1Yi+φ˜i,2expγYi+12Yi+12.E20

Example 1

In the simulation we focused on testing the nullity of φi,2 only. We simulated 1000 independent samples of length n=200 and 500 of 3 models.

Model I: Periodic autoregressive (PAR21) with the parameters φ¯=0.7,0.4'.

Model II: Restricted PEXPAR21 with the parameters φ¯=0.7,0,0.42' and γ=1

Model III: Restricted PEXPAR21 with the parameters φ¯=0.7,1.5,0.42' and γ=1.

The model I is chosen to calculate the level, the model III is chosen to calculate the power, the choice of model II is to show that the test is efficient since in the first cycle we have an AR1 and in the second cycle a restricted EXPAR1. On each realisation we fitted a restricted PEXPAR21 model by QML and carried out tests of H0:φi,2=0 against H1:φi,20. The rejection frequencies at significance level 5% and 10% are reported in Tables 4 and 5. Figure 3 shows the asymptotic distribution of LRi,m under the null hypothesis. From the tables we see that the levels of the LR test are pretty well controlled since for n=500, we note a relative rejection frequency of 5.5% for φ1,2 and 5.1% for φ2,2, which are not meaningfully different from the nominal 5%, the same remark is made for α=10% where the relative rejection frequency is of 9.5% and 10.3%. From model III, the rejection frequencies which represent the empirical power increase with the length n indicating the good performance and the consistency of the test. To illustrate that the asymptotic distribution of LRi,m under the null hypothesis is the standard χ12 we have the histograms in Figure 3 where we see that the distribution of LRi,m has the well known shape of χ12.

Modelαφ1,2φ2,2
I5%10%0.0520.1050.0660.103
II5%10%0.0540.1170.9981
III5%10%0.96710.9300.997

Table 4.

The rejection frequency computed on 1000 replications of simulations of length n=200.

Modelαφ1,2φ2,2
I5%10%0.0550.0950.0510.103
II5%10%0.0530.09811
III5%10%0.991111

Table 5.

The rejection frequency computed on 1000 replications of simulations of length n=500.

Figure 3.

Asymptotic distribution of LR.

3.2 Test for linearity in Restricted PEXPAR(1) model

The most important case to test is when φi,2=0,i, which correspond to the linear periodic autoregressive model PARS1 of period S. The null hypothesis is then

H0:φi,2=0,ivsH1:i/φi,20.E21

H1 correspond to the restricted PEXPARS1 model, that is, the linear PARS1 model is nested within the nonlinear restricted model and it can be obtained by limiting the parameters φi,2 to be zero i, hence we have a problem of testing the linearity hypothesis.

The standard LR test statistic is

λm=i=1SQ˜i,mφ̂¯iQ˜i,mφ˜¯im2.E22

The test rejects H0 at the asymptotic level α when

LRm=2logλm=mi=1SlogQ˜i,mφ˜¯iQ˜i,mφ̂¯i>χS21α,E23

where χS21α is the 1αquantile of the χ2 distribution with S degrees of freedom which is simply the number of supplementary parameters in H1.

Example 2

Table 6 shows the rejection frequency computed on 10000 replications of simulations of length n=200 and 500 from the 2 models.

ModelαLR test n=200LR test n=500
I5%10%0.06150.12250.05810.1075
II5%10%0.99990.999911

Table 6.

The rejection frequency computed on 10000 replications.

Model I: PAR41 with the parameters φ¯=0.8,0.5,0.90.4'.

Model II: Restricted PEXPAR41 with φ¯=0.8,2,0.51.5,0.9,1.10.4,0.6' and γ=1., Figure 4 shows the asymptotic distribution of LRm under the null hypothesis. The results show that the empirical levels are acceptable, for n=500, we have a relative rejection frequency of 5.81% (resp. 10.75%) which is very close to 5% (resp. 10%), the empirical power increases with the size n which means that the test is consistent. The rejection region is LRm>χ421α, where χ421α is the 1α quantile of the χ2 distribution with 4 degrees of freedom. From Figure 4, we see that the asymptotic distribution of LRm (in full line) is close to the χ42 (in dashed lines), this confirm the above theoretical result.

Figure 4.

Asymptotic distribution of LRm.

Advertisement

4. Conclusion

The periodic restricted EXPAR model is added to the family of nonlinear and periodic models. Interest is focused on estimation and testing problems. The periodic stationarity allows to calculate the QML estimators and derived tests of coefficients, cycle by cycle, and therefore use standard techniques. From this point of view, we can extend several results concerning the classical EXPAR to the periodic case.

References

  1. 1. Priestley MB. Non-linear and Non-stationary Time Series Analysis. New York: Academic Press; 1988
  2. 2. Tong H. Nonlinear Time Series: a Dynamical System Approach. Oxford: Oxford University Press; 1990
  3. 3. Douc R, Moulines E, Stoffer D. Nonlinear Time Series: Theory, Methods, and Applications with R Examples. Texts in Statistical Science. CRC Press; 2013
  4. 4. Ozaki T. Non-linear time series models for non-linear random vibrations. Journal of Applied Probability. 1980;17:84-93
  5. 5. Haggan, V. and Ozaki,T. (1981), Modeling nonlinear random vibrations using an amplitude-dependent autoregressive time series model. Biometrik Vol.68, No.1, PP(96–189).
  6. 6. Chan KS, Tong H. On the use of the deterministic Lyapunov function for the ergodicity of stochastic difference equations. Advances in Applied Probability. 1985;17(3):666-678
  7. 7. Tjøstheim D. Estimation in Nonlinear Time Series Models. Elsevier Science Publishers B.V. North-Holland. Stochastic Processes and their Applications. 1986;21:251-273
  8. 8. Al-Kassam MS, Lane JA. Forecasting Exponential Autoregressive Models Of Order 1. Journal Of Time Series Analysis. 1989;10(2):95-113
  9. 9. Koul HL, Schick A. Efficient estimation in nonlinear autoregressive time series models. Bernoulli. 1997;3:247-277
  10. 10. Shi Z, Aoyama H. Estimation of exponential autoregressive time series model by using genetic algorithm. Journal of Sound and Vibration. 1997;205(3):309-321
  11. 11. Ismail, M. A. (2001). Bayesian Analysis of Exponential AR Models. The Far East Journal of Statistics, Vol 5, pp 1–15. (India)
  12. 12. Allal, J. and El Melhaoui, S.(2006).Optimal Detection Of Exponential Component. Journal Of Time Series Analysis Vol. 27, No. 6, PP(793–810).
  13. 13. Francq, C., Horvath, L. and Zakoian, J.M. (2008). Sup-tests for linearity in a general nonlinear AR(1) model when the supremum is taken over the full parameter space. MPRA Paper No. 16669.
  14. 14. Ghosh H, Gurung B, Gupta P. Fitting EXPAR Models Through the Extended Kalman Filter. Sankhyā : The Indian Journal of Statistics. 2015;77(1):27-44
  15. 15. Shi Z, Tamura Y, Ozaki T. Monitoring the stability of BWR oscillation by nonlinear time series modeling. Annals of Nuclear Energy. 2001;28:953-966
  16. 16. Terui, N. and Van Dijk, H. K. (1999). Combined Forecasts from Linear and Nonlinear Time Series Models. Econometric Institute Report EI-9949/A.
  17. 17. Ishizuka K, Kato H, Nakatani T. Speech signal analysis with exponential autoregressive model, Proc. the 30th International Conference of Acoustics, Speech. Signal Processing. 2005;1:225-228
  18. 18. Amiri, E., (2012). Forecasting GDP Growth rate with Nonlinear Models. 1stInternational Conference on Econometrics Methods and Applications. ICEKU2012 (25–27).
  19. 19. Gurung, B. (2013) An Application of Exponential Autoregressive (EXPAR) Nonlinear Time-series Model. International Journal of Information and Computation Technology. ISSN 0974–2239, 3(4), 261–266.
  20. 20. Katsiampa P. A new approach to modelling nonlinear time series: Introducing the ExpAR-ARCH and ExpAR-GARCH models and applications. 4th Student Conference on Operational Research. In: 34–51. 2014
  21. 21. Azouagh N, El Melhaoui S. An Exponential Autoregressive model for the forecasting of annual sunspots number. Electronic Journal of Mathematical Analysis and Applications. 2019;7(3):17-23
  22. 22. Gladyshev EG. Periodically correlated random sequences. Soviet. Math. 1961;2:385-388
  23. 23. Merzougui M, Dridi H, Chadli A. Test for periodicity in restrictive EXPAR models. Communication in Statistics-Theory and Methods. 2016;45(9):2770-2783
  24. 24. Merzougui, M. (2017) Estimation in periodic restricted EXPAR(1) models. Communications in Statistics - Simulation and Computation, D.O.I.10.1080/03610918.2017.1361975.
  25. 25. Merzougui, M. (2020) Wald Tests in the restricted Periodic EXPAR1 model. Biostatistics and Biometrics Open Access Journal. V 10(1). DOI: 10.19080/BBOAJ.2020.10.555780.
  26. 26. Engle RF. Handbook of Econometrics. Vol. II: Elsevier Science Publishers BV; 1984
  27. 27. Bierens HJ. Estimation, testing, and specification of cross-section and time series models. Cambridge University Press; 1994

Written By

Mouna Merzougui

Submitted: 01 July 2020 Reviewed: 17 September 2020 Published: 24 November 2020