Forecasting accuracy (MSE) for different forecasting methods.
This paper is devoted to the investigation and application of fuzzy inductive modeling method group method of data handling (GMDH) in problems of forecasting in the financial sphere. GMDH method belongs to self-organizing methods and allows to discover internal hidden laws in the appropriate object area. The advantage of GMDH algorithms is the possibility of constructing optimal models. In the generalization of GMDH in case of uncertainty, new method fuzzy GMDH is described which enables to construct fuzzy models almost automatically. The algorithms of fuzzy GMDH for different membership functions are considered. The extensions of fuzzy GMDH for different partial descriptions—orthogonal polynomials of Chebyshev and trigonometric polynomials of Fourier—are considered. The problem of adaptation of fuzzy models obtained by FGMDH is considered, and the corresponding adaptation algorithm is described. The experimental investigations of the suggested FGMDH in the problem of forecasting macroeconomic indicators of Ukraine are carried out, and comparison with classic GMDH and neural network BP is performed.
- fuzzy GMDH
- orthogonal partial descriptions
- model adaptation
One of the most important problems in the sphere of economy and finance is the problem of forecasting economic and financial processes. The distinguishing properties of these processes are the following:
The form of functional dependence is unknown, and only model class is determined.
Short data samples.
Time series in general case is nonstationary.
In this case the application of traditional methods of statistical analysis (e.g., regression analysis) is impossible, and it’s necessary to apply methods based on computational intelligence (CI). To this class belongs group method of data handling (GMDH) developed by Ivakhnenko [1, 2] and extended by his colleagues. GMDH method belongs to self-organizing methods and allows to discover hidden laws in the appropriate object area. The advantage of GMDH algorithms is the capability of constructing optimal models.
But classic GMDH has the following shortcomings:
GMDH utilizes least squared method (LSM) for finding the model coefficients, but matrix of linear equations may be close to degenerate, and the corresponding solution may appear non-stable. Therefore, the special methods for its regularization should be used.
GMDH doesn’t work in the case of qualitative or fuzzy input data.
Fuzzy GMDH is also based on the same principles as classical GMDH but construct fuzzy models.
The main goals of this paper are to investigate different modifications of FGMDH, analyze their properties, and investigate its efficiency as compared with classical GMDH in forecasting problems.
2. Problem formulation
A set of initial data is given inclusive of input variables and output variables , where is -n-tuple vector, N is a number of observations, and input data may be incomplete or fuzzy, in particular given in interval form. The task is to construct an adequate fuzzy forecasting model , and besides, the obtained model should have the minimal complexity.
2.1 Principal ideas of GMDH: fuzzy model construction
GMDH utilizes LSM for finding the model coefficients, but matrix of linear equations may be close to degenerate, and the corresponding solution may appear non-stable and very volatile. Therefore, the special regularization methods should be applied.
After application of GMDH point-wise estimations is obtained, but in many cases, it’s desirable to find interval value for coefficient estimates.
GMDH doesn’t work in the case of incomplete, qualitative, or fuzzy input data.
Therefore, in the last 10 years, the new variant of GMDH—fuzzy GMDH—was developed and improved which may work with fuzzy and qualitative input data and is free of classical GMDH drawbacks .
The principle of multiplicity of models
The principle of external complement which means that the whole sample should be divided into two parts—training subsample and test subsample
The principle of self-organization
The principle of freedom of choice
Fuzzy GMDH is also based on these principles but construct fuzzy models. Let’s consider its main ideas.
where is a fuzzy number of triangular form described by a pair of parameters , where is interval center, is its width, and , Zi is the input variables.
Then Y is a fuzzy number, parameters of which are determined as follows:
The interval center
The interval width
For example, for the partial description (PD) of the kind
it’s necessary to substitute in the general model (1)
Let the training sample be , . Then for the model (1) to be adequate, it’s necessary to find such parameters , which satisfy the following inequalities:
Let’s formulate the basic requirements for the linear interval model of a kind (4).
It’s necessary to find such values of the parameters of fuzzy coefficients for which:
Real values of the observed outputs should drop in the estimated interval for .
The total width of the estimated interval for all sample points should be minimal.
where k is a number of a point.
As one can easily see, the task (6)–(8) is a LP problem. However, the inconvenience of the model (6)–(8) for the application of standard LP methods is that there are no constraints of non-negativity for variables . Therefore for its solution, it’s reasonable to pass to the dual LP problem by introducing dual variables and , . Using simplex method after finding the optimal solution for the dual problem, the optimal solutions of the initial direct problem will be also found.
3. Description of fuzzy GMDH algorithm
Choose the general model type by which the sought dependence will be described.
Choose the external criterion of optimality (criterion of regularity or non-biasedness).
Choose the type of partial descriptions (e.g., linear or quadratic one).
Divide the sample into training and test subsamples.
Put zero values to the counter of model number k and to the counter of iteration number r.
Calculate the value of external criterion (or ) at the test sample.
. If for r = 1 or for r >1, then , , and go to step 9; otherwise go to step 6.
Calculate the best value of the criterion for models of rth iteration or . If , then go to step 6; otherwise go to step 10.
If or ≥ , then go to step 11; otherwise select F best models, assign and , go to step 6, and execute (r + 1)th iteration.
Select the best model of the previous iteration using external criterion and moving back by its connections and successively passing all the previous rows find analytical form the constructed model.
4. Analysis of different membership functions
In the first papers devoted to fuzzy GMDH , the triangular membership functions (MFs) were considered. But as fuzzy numbers may also have the other kinds of MF, it’s important to consider the other classes of MF in the problems of modeling using FGMDH. In paper  fuzzy models with Gaussian and bell-shaped MFs were investigated.
Consider a fuzzy set with Gaussian MF:
Let the linear interval model for partial description of FGMDH take the form (4). Then the problem is formulated as follows:
Find such fuzzy numbers , with parameters , that:
The observation would belong to a given estimate interval for the set with degree not less than , .
The width of estimated interval of the degree would be minimal.
To solve this problem like the case with triangular MF, it’s reasonable to pass to the dual LP problem of the form
with constraints of equalities and inequalities
Analyzing dual LP program (12)–(15), it’s easy to notice that this problem is always solvable as there trivial solution always exists. Therefore the initial problem (10) and (11) also always has solutions with any data.
Thus, fuzzy GMDH allows to construct fuzzy models and has the following advantages:
The problem of optimal model determination is transformed to the problem of linear programming, which is always solvable.
As the result of method work, the interval regression model is being built.
5. Fuzzy GMDH with different partial descriptions: orthogonal polynomials
As it is well known from the general GMDH theory, model pretenders are generated on the base of so-called partial description—elementary models of two variables. Usually as partial descriptions, linear or quadratic polynomials are used. The alternative to this class of models is application of orthogonal polynomials. The choice of orthogonal polynomials as partial descriptions is determined by the following advantages:
Due to orthogonal property, the determination of polynomial coefficients goes faster than for non-orthogonal polynomials.
The coefficients of polynomial approximating equation don’t depend on the real degree of initial polynomial model, so if a priori the real polynomial degree is not known, one may calculate the polynomials of various degrees, and by this property the coefficients obtained for polynomials of lower degrees remain the same after transfer to higher polynomial degrees. This property is the most important during investigation of real degree of approximating polynomial.
5.1 Chebyshev’s orthogonal polynomials
Chebyshev’s orthogonal polynomials in general case have the following form :
These polynomials have the following orthogonality property:
where is a weighting coefficient in the Eq. (17).
The approximating Chebyshev’s orthogonal polynomial for is obtained on the base of function S minimization:
where from (18) we obtain the following expression for coefficients:
Hence, the approximating equation takes the form
As it may be readily seen from the presented expressions, coefficient in Eq. (19) doesn’t depend on the choice of degree m. Thus, the variable m doesn’t demand recalculation of , while such recalculation is necessary for non-orthogonal approximation.
The best degree of approximating may be obtained on the base of hypothesis that investigation results have independent Gaussian distribution in the bounds of some polynomial function of definite degree, for example, , where
and a dispersion of distribution don’t depend on μ.
It’s clear that for very small (=0,1,2,…), decreases as m grows.
As in accordance with previously formulated hypothesis, dispersion doesn’t depend on μ; therefore, the best degree is a minimal m, for which .
For determining it’s necessary to calculate the approximating polynomials of various degrees. As coefficients in Eq. (20) don’t depend on , the determination of the best degree of polynomial is accelerated.
Let us have the forecasted variable and input variables . Let’s search the relation between them in the following form:
where is a fuzzy number of triangular type given as ,
The degree of function is determined using hypothesis defined in the preceding section. So if we denote , we’ll get linear interval model in its classical form.
5.2 Investigation of trigonometric polynomials as partial descriptions
Let function be periodic with period defined at the interval , and its derivative is also defined at the interval . Then the following equality holds
Coefficients and are calculated by Euler formulas:
A trigonometric polynomial of the degree M is called the following polynomial:
The following theorem is true stating that exists such , which minimizes the following criterion:
Hence the coefficients of corresponding trigonometric polynomial are determined by formulas
Let it be the forecasted variable and input variables . Let’s search the dependence among them in the form
where is a fuzzy number of triangular type given as , functions are determined in such a way:
The degree of function is determined by the theorem described in the preceding section. Therefore, if we assign , the linear interval model will be obtained in its classical form.
6. Adaptation of fuzzy GMDH models
While forecasting by self-organizing methods (fuzzy GMDH, in particular), the problem of adaptation arises in the case of the training sample size increase when it’s needed to correct the obtained model in accordance with new available data. Taking into account new information obtained while forecasting adaptation may be done by two approaches. The first one is to correct parameters of a forecasting model with new data assuming that model structure didn’t change. The second approach consists in adaptation of not only model parameters but its optimal structure as well. This way demands the repetitive use of full GMDH algorithm and is connected with huge volume of calculations.
The second approach is used if adaptation of parameters doesn’t provide good forecast and the new real output values don’t drop in the calculated interval for its estimate.
In our work the first approach is used based on adaptation of FGMDH model parameters with new available data. Here the recursive identification methods are preferably used, especially the recursive LSM. In this method the parameter estimates at the next step are determined on the base of estimates at the previous step, model error, and some information matrix which is modified during all estimation process and therefore contains data which may be used at the next steps of adaptation process .
Hence, model coefficient adaptation will be simplified substantially. If we store information matrix obtained while identification of optimal model using fuzzy GMDH, then for model parameters adaptation, it will be enough to fulfill only one iteration by recursive LSM method.
6.1 The application of recurrent LSM for model coefficients adaptation
Consider the following model:
where is a dependent (output) variable, is a measurement vector, are random disturbances, and is a parameter vector to be estimated.
where is a coefficient vector which is determined by formula
where is so-called an information matrix, determined by formula
As one can easily see in (35), the information matrix may be obtained independent on parameter estimation process and parallel to it. The adaptation of two parameter vectors is performed in such a way using the formulas (35)
7. Experimental investigations of FGMDH in forecasting
The goal of experiments was the forecasting of macroeconomic indicators of Ukraine and estimating of efficiency of suggested FGMDH. In experiments, the database was utilized which contains monthly values of 24 macroeconomic indicators of Ukrainian economy, since July 1995 till 2013. As forecasting variables consumer price index (CPI) and gross national product (GNP) were chosen.
While constructing forecasting models, the technique of sliding window was utilized, whose size was determined automatically by regression analysis. For determination of input variables significant for forecasting, the methods of regression analysis were also used.
The following experiments were performed:
Forecasting model construction with application of different membership functions: triangular, Gaussian, and bell-wise one
For macroeconomic indicators (CPI and GNP) forecasting the construction of forecasting models using different partial descriptions—classic Chebyshev’s polynomials and trigonometric polynomials.
For adaptation of models, the algorithm of stochastic approximation and recurrent least squared method (RLSM) were applied.
Comparative analysis of the suggested algorithms with classic GMDH and neural networks (NN), in particular neural network backpropagation, was performed.
7.1 Comparison of different membership functions
The experimental investigations of fuzzy forecasting models were carried out with following MF: triangular, Gaussian, and bell-wise. As accuracy criteria RMSE was chosen. RMSE values while forecasting CPI are presented in Figure 1.
As one can see, the most efficient for constructing linear interval models is application of bell-wise membership functions for fuzzy coefficients, on the second place are Gaussian MFs, and the worst forecasting accuracy was achieved with triangular MF. In the next experiment, the task was to forecast GPD values.
In Figure 2 the obtained RMSE values for forecasting GNP are presented.
As one can see, the results are practically the same as in the previous experiment. The best accuracy was attained with bell-wise MF.
7.2 Comparison of different partial descriptions
In the next series of experiments, the investigations of FGMDH models with the following partial descriptions were carried out: quadratic polynomials, Chebyshev’s polynomials, trigonometric polynomials, and ARIMA models. In Figure 3 accuracy of forecasting PCI is presented with different PD.
As we can see, the best results are obtained with models which use trigonometric polynomials as PD. Somewhat worse are results with classic quadratic polynomials. And the worst turned out to be ARIMA models as PD. It may be explained by the fact that ARIMA models are functions of one variable. That is a serious drawback of such models.
7.3 Comparison of crisp and fuzzy GMDH
For more comprehensive efficiency comparison of crisp and fuzzy GMDH, existing implementation of GMDH was extended by inclusion of new types of PD orthogonal polynomials: Chebyshev’s and trigonometric and ARIMA models as PD. As adaptation algorithm stochastic approximation and recurrent LSM were implemented. In Figures 4 and 5, the mean RMSE values for crisp and fuzzy GMDH in the whole range of data variation are presented for different types of PD without adaptation and with adaptation algorithms.
As one can easily see from presented results, the fuzzy algorithm GMDH shows better forecasting accuracy than classic GMDH for all adaptation algorithms.
So the results of experiments have confirmed indisputable advantages of fuzzy GMDH over classic GMDH for problem of forecasting macroeconomic indicators. In the next experiments, the comparison of fuzzy GMDH with results of neural network (NN) backpropagation was performed. The final results—MSE values on five forecasting points while forecasting CPI and GNP—are presented in Table 1.
|Adaptation algorithm||Without adaptation||Stochastic approximation||RLSM|
|Triangle MF + quadratic polynomial||0.308||530.3||0.184||330.0||0.173||311.9|
|Gaussian MF + quadratic polynomial||0.294||531.3||—||—||—||—|
|Bell-wise MF + quadratic polynomial||0.268||497.9||—||—||—||—|
|Triangular MF + Chebyshev’s polynomial||0.403||621.4||0.341||458.1||0.337||377. 2|
|Triangular MF + Laguerre polynomial||0.372||589.5||0.264||442.9||0.293||378.5|
|Triangular MF + trigonometric polynomial||0.261||537.7||0.185||347.9||0.165||331.9|
|Triangular MF + ARIMA model||0.862||704.3||0.683||513.5||0.597||472.6|
|GMDH + quadratic polynomial||0.343||596.7||0.204||428.2||0.192||369.2|
|GMDH + Chebyshev’s polynomial||0.425||641.4||0.351||473.2||0.347||398.4|
|GMDH + Laguerre polynomial||0.396||598.5||0.292||459.0||0.274||376.4|
|GMDH + trigonometric polynomial||0.291||574.8||0.182||349.5||0.177||332.2|
|GMDH + ARIMA model||0.902||728.4||0.749||518.7||0.714||498.3|
Summing the experimental results, the following conclusions were made:
Forecasting accuracy of fuzzy GMDH algorithms are, in a whole, better than of non-fuzzy GMDH.
Forecasting accuracy of non-fuzzy and fuzzy GMDH algorithms are better than that of NN backpropagation. Modification of membership functions doesn’t lead to significant changes of forecasting quality, but the best results were obtained with bell-wise and Gaussian MF.
The best forecasting accuracy for considered problems was obtained with models of fuzzy GMDH using quadratic and trigonometric partial descriptions.
The best adaptation algorithm for fuzzy GMDH models is recurrent RLSM.
Consider the case of symmetrical membership function for parameters Ai, so they can be described by the pair of parameters (, ), where
, , is the interval width, ≥ 0,
and is input variable which is also a fuzzy number of triangular shape, defined by three parameters , where is a lower border, is a center, and is an upper border of fuzzy number.
In this paper fuzzy inductive modeling method FGMDH is considered.
The algorithms of FGMDH with different membership functions and different partial descriptions, including orthogonal polynomials, were presented and analyzed.
The experimental investigations of GMDH and fuzzy GMDH in problems of macroeconomic index forecast in Ukrainian economy were carried out.
The comparative investigations of FGMDH with ARIMA and neural network backpropagation were performed.
Experimental result analysis has confirmed the high accuracy of fuzzy GMDH in problems of forecasting in macroeconomy.