Banks Financial State Analysis and Bankruptcy Risk Forecasting with Application of Fuzzy Neural Networks

The problem of banks bankruptcy risk forecasting under uncertainty is considered. For its solution, the application of computational intelligence methods fuzzy neural networks ANFIS and TSK and inductive modeling method FGMDH was suggested and explored. Experimental investigations were carried out and estimation of the efficiency of the suggested methods was performed at the problems of bankruptcy risk forecasting for Ukrainian and leading European banks. The efficiency comparison with classic statistical methods such as ARMA, logit, and probit models was fulfilled. The comparative experiments with rating system CAMELS and matrix method were carried out. In general, the comparative analysis had shown that fuzzy forecasting methods and techniques give better results than conventional crisp methods for forecasting bankruptcy risk. On the whole, the conclusions of experiments with European banks completely confirmed the conclusions of experiments with Ukrainian banks. But at the same time, the crisp methods are more simple in implementation and demand less time for their adjustment. The set of informative bank financial factors for bankruptcy risk forecasting was determined and estimated.


Introduction
The problems of banks financial state analysis and bankruptcy risk forecasting are of great importance. The opportune discovery of coming bankruptcy allows top bank managers to make urgent decisions for preventing the bankruptcy. Nowadays, there are a lot of methods and techniques of banks state analysis and determination of bank rating-WEB Money, CAMEL [1], Moody's S&P, etc. But their common drawback is that all of them work with complete and reliable data and cannot give correct results in case of incomplete and unreliable input data. This is especially actual for the Ukrainian banking system where bank managers often provide the incorrect reports about bank financial state to obtain new credits and loans.
Therefore, it is very important to create new methods for banks bankruptcy risk forecasting under uncertainty. The main goal of present investigation is to consider and estimate novel methods of bank financial state analysis and bankruptcy risk forecasting under uncertainty and compare with classical methods. The implementation and assessment of the efficiency of the suggested methods are performed at the problems of bankruptcy risk forecasting for Ukrainian and European banks.

Bankruptcy risk forecasting of Ukrainian banks 2.1 Problem statement
As it is well known, the year 2008 was the crucial year for the bank system of Ukraine. If the first three quarters were periods of fast growth and expansion, the last quarter became the period of collapse in the financial sphere. A lot of Ukrainian banks faced the danger of coming default.
For this research, the quarterly accountancy bank reports used were obtained from National bank of Ukraine site. For analysis, the financial indices of 170 Ukrainian banks were taken up to the date January 01, 2008 and July 01, 2009, that is, about two years before crises and just before the start of crises [2].
The important problem that occurred before the start of the investigations is which financial indices are to be used for better forecasting of possible bankruptcy. Thus, another goal of this exploration was to detect the most relevant financial indicators for obtaining maximal accuracy of forecasting.
For analysis, the following indicators of banks accountancy were considered: assets, capital, financial means, and their equivalents; and physical person's entities, juridical person's entities, liabilities, and net incomes (losses).
The collected indicators were used for analysis by fuzzy neural networks as well as classic statistical methods. As output data of models for Ukrainian banks were two values: 1, if the significant worsening of bank financial state is not expected in the nearest future À1, if the bank bankruptcy is expected in the nearest future.

FNN TSK model and hybrid training algorithm
For forecasting of banks bankruptcy risk, the application of fuzzy neural networks (FNN) ANFIS and TSK was suggested [3]. The application of FNN is determined by following reasons: the capability to work with incomplete and unreliable information under uncertainty; and the capability to use expert information in the form of fuzzy inference rules.
Let us consider the mathematical model and training algorithm of a fuzzy neural network TSK (Takagi, Sugeno, Kang'a), which is generalization of the neural network ANFIS. The rule base of FNN TSK with M rules and N variables can be written as follows [3]: i is the value of linguistic variable x i for the rule R k with membership function (MF) of the form 4.The fourth layer consists of two summing neurons, one of which calculates the weighted sum of the signals y k x ð Þ, and the second one calculates the sum of the weights ∑ M k¼1 w k .
5. The fifth layer is composed of a single output neuron. In it, weight normalizing is performed and the output signal determined in accordance with the expression: This is also nonparametric layer. From this description follows that TSK fuzzy network contains only two parametric layers: first and third, the parameters of which are determined in the training process. Parameters of the first layer c , we call nonlinear, and the parameters of the third layer p kj n o -linear weights. The general expression for the functional dependence (4) for the network TSK is defined as follows: If we assume that at any given time moment, the nonlinear parameters are fixed, then the function y x ð Þ would be linear with respect to the variable x j :  ð Þlinear and nonlinear parameters that must be determined in the learning process. This is a very large value. In order to reduce the number of parameters for adaptation, we operate with fewer number of MF. In particular, it can be assumed that some of the parameters of one function MF μ

Hybrid learning algorithm for fuzzy neural networks
Considering a hybrid learning algorithm which is used for FNN TSK, all parameters can be divided into two groups. The first group includes linear parameters p kj of the third layer, and the second group includes nonlinear parameters (MF) of the first layer. Adaptation occurs in two stages.
In the first stage after fixing the individual parameters of the membership function by solving a system of linear equations, linear parameters of polynomial p kj are calculated. With the known values of MF dependence, input-output can be represented as a linear form with respect to the parameters p kj : where With the dimension L of training sample x l ð Þ ; d l ð Þ , l ¼ 1; 2; …; L ð Þand replacement of the network output by expected value d l ð Þ , we get a system of L linear equations of the form Ap ¼ d: Matrix A dimension is equal to L N þ 1 ð ÞM: By thus, a number of rows L usually is much greater than a number of columns N þ 1 ð ÞM: The solution of this equations system may be obtained by conventional methods as well as using pseudoinverse matrix A at one step: where A þ is a pseudoinverse matrix for matrix A. In the second stage, after fixing the values of linear parameters p kj , the actual output signals y ℓ ð Þ , ℓ ¼ 1, 2, …, L are determined using a linear equations system: Then, the error vector ε ¼ y À d ð Þand the criterion E are calculated: The error signals are sent through the network backward according to the method of back propagation until the first layer, at which gradient vector components of the objective function with respect to parameters c After calculating the gradient vector, a step of gradient descent method is made. The corresponding formulas (for the simplest method of the steepest descent) are the following: where n is a number of iterations. After verifying the nonlinear parameters, the process of adaptation of linear parameters TSK (first phase) restarts and nonlinear parameters are further adapted (second stage). This cycle continues until all the parameters will be stabilized.
Formulas (11)-(13) require the calculation of the gradient of the objective function with respect to the parameters of the MF. The final form of these formulas depends on the type of MF. For example, if using the generalized bell-wise functions: the corresponding formulas for gradient of the objective function for one pair of data x; d ð Þ take the form [3]: In the practice of the hybrid learning method implementation, the dominant factor in adaptation is considered to be the first stage in which weights p kj are determined using pseudoinverse in one step. To balance its impact, the second stage should be repeated many times in each cycle.
It is worth to note that the described hybrid algorithm is one of the most effective ways of training fuzzy neural networks. Its principal feature is the division of the process into two stages separated in time. Since the computational complexity of each nonlinear optimization algorithm depends nonlinearly on the number of parameters subject to optimization, the reduction in the dimensions of optimization significantly reduces the total amount of calculations and increases the speed of convergence of the algorithm. Due to this, hybrid algorithm is one of the most efficient in comparison with conventional gradient-based methods.

The application of fuzzy neural networks for financial state forecasting
A special software kit was developed for FNN ANFIS and TSK application in bankruptcy risk forecasting problems. As input data, the financial indicators of Ukrainian banks in financial accountant reports were used in the period of 2008-2009 [2] . As the output values were used +1, for bank nonbankrupt and À1, for bank bankrupt. In the investigations, various financial indicators were analyzed, and different number of rules for FNN and the analysis of data collection period influence on forecasting accuracy were performed.
The results of experimental investigations of FNN application for bankruptcy risk forecasting are presented below.
In the first series of experiments, input data at the period of January 2008 were used (that is for two years before possible bankruptcy) and possible banks bankruptcy was forecasted at the beginning of 2010.
The results of application of FNN TSK are presented in Table 1.
The similar experiments were carried out with FNN ANFIS.

Experiment No. 2:
The goal of the next experiment was to find out the dependence of rule number on predicting accuracy. Input data-the same financial indices as in experiment 1.
The results of application of FNN TSK are presented in Table 2.
The similar experiments were carried out with FNN ANFIS.

Experiment No. 3:
The comparative analysis of forecasting results versus the number of rules is presented in Table 3 [4].
Comparing the results in Table 3, one may conclude FNN TSK has better accuracy than FNN ANFIS.
The goal of the next experiments was to explore the influence of training and test samples size on accuracy of forecasting.

Total amount of errors 5 % of errors 10
First type of errors 0 Second type of errors 5 Table 1.
Results of FNN TSK forecasting.

Results
Total amount of errors 6 % of errors 12 First type of errors 1 Second type of errors 5  Table 3.

Comparative analysis of FNN ANFIS and TSK in dependence on rules number.
Input data-financial indicators: assets, entity, cash (liquid assets), household deposits, and liabilities.
The results for FNN TSK are presented in Table 4.
The similar experiments were carried out with FNN ANFIS. After analysis of the experimental results the following conclusions were made: FNN TSK ensures the higher accuracy of risk forecasting than FNN ANFIS; the variation of the number of rules in the training and test samples makes slight influence on the accuracy of forecasting; and the goal of the next series of experiments was to determine the optimal input data (financial indicators) for bankruptcy risk forecasting. The period of input data was January 2008.

Experiment No. 5:
Number of banks and rules were the same as in previous experiment 4.
Input data-financial indicators (taken from banks financial accountant reports): profit of current year, net percentage income, net commission income; and net expense on reserves and net bank profit/losses.
The results of FNN TSK application are presented in Table 5.

Experiment No. 6:
Number of banks and rules were the same as in the previous experiment 5.

Results
Total number of errors 7 % of errors 10 First type of errors 1 Second type of errors 6 Table 4.
Results of FNN TSK forecasting.

Total number of errors 13 % of errors 19
First type of errors 6 Second type of errors 7 Table 5.
Results of FNN TSK forecasting.
Input data-the financial indicators (taken from banks financial accountant reports): general reliability factor (own capital/assets); instant liquidity factor (liquid assets/liabilities); cross coefficient (total liabilities/working assets); general liquidity coefficient (liquid assets + defended capital + capitals in reserve fund/total liabilities); and coefficient of profit fund capitalization (own capital/charter fund).
The results for FNN TSK are presented in Table 6.
It is worth to note that these financial indicators are also used as input data in Kromonov's method of banks bankruptcy [5][6][7], whose results are presented below.
Number of rules = 5. The results of application of FNN TSK for forecasting with these input indicators are presented in Table 7.
It should be noted that these indicators are used as input in the method of Euro Money [1].

Total number of errors 7 % of errors 10
First type of errors 1 Second type of errors 6 Table 6.
Results of FNN TSK forecasting.

Results:
Total number of errors 12

% of errors 17
First type of errors 5 Second type of errors 7 Table 7.
Results of FNN TSK forecasting.
Number of rules = 5.
Input data-financial indicators (banks financial accountant reports): general reliability factor (own capital/assets); instant liquidity factor (liquid assets/liabilities); cross coefficient (total liabilities/working assets); general liquidity coefficient (liquid assets + defended capital + capitals in reserve fund/total liabilities); coefficient of profit fund capitalization (own capital/charter fund); and coefficient entity security (secured entity/own entity).
The results of FNN TSK application with these financial indicators are presented in Table 8.
The comparative analysis of forecasting results using different sets of financial indicators are presented in Table 9.
Next experiment was aimed on finding the influence of data collection period on the forecasting results. It was suggested to consider two periods: January of 2008 (about 1.5 year before the crisis) and July of 2009 (just before the start of crisis).
Number of rules = 10.

Total amount of errors 8 % of errors 13
First type of errors 1 Second type of errors 7  Table 9.
The dependence of forecasting accuracy on sets of input financial indices.
Input data-financial indices, the same as in experiment 8.
In Table 10, the comparative results of forecasting versus period of input data are presented.

The application of fuzzy GMDH for financial state forecasting
In the process of investigations, fuzzy group method of data handling (FGMDH) was also suggested for financial state of Ukrainian banks forecasting [3]. GMDH is the inductive modeling method that enables to construct a model automatically by experimental data [3]. As input data, the same indices were used as in the experiments with FNN TSK.
In Table 11, the forecasting accuracy of FGMDH is presented in dependence on input data collection period.
If we compare the results of FGMDH with the results of FNN TSK, one can see that FNN TSK gives better results for short-term risk forecasting (one year before possible bankruptcy) while FGMDH has better accuracy using older input data and so it has advantages in long-term forecasting (2 or more years).

The generalized analysis of crisp and fuzzy forecasting methods
In the concluding experiments, the comparative analysis of application of all the considered methods was carried out. The following methods were considered [4] Table 11.
Comparative results of forecasting using method FGMDH in dependence on period of input data collection.
As input data, the financial indices of Ukrainian banks on July 2007 year were used. The results of application of all methods for bankruptcy risk analysis are presented in Table 12. 3.3 Application of rating system CAMEL for assessment of financial state of Ukrainian banks The most widely used approach of banks financial state analysis and bankruptcy risk forecasting is based on the application of rating systems. The determination of bank rating is one of the methods that enables to obtain complex financial assessment of bank financial state and compare them. There are various private and official banks rating systems. The most known of them are systems developed by world leaders in this sphere-rating companies Fitch, Standard & Poor's, Moody's, etc. Officially recognized banks rating system that is widely used in the world is system CAMELS. It's American rating system was developed and implemented by Federal reserve System (FRS) and Federal Deposit Insurance Corporation (FDIC) in 1978 [1].
Supervision over banks activity based on risk estimation by system CAMELS lies in determination of general bank state using the common criteria that defines all aspects and spheres of bank activity. This system is also widely used in Ukraine by National Bank of Ukraine (NBU) according to developed "Statement of order of rating estimates determination by rating system 'CAMELS'." Rating system CAMELS allows NBU to estimate general financial state and stability of banking system of Ukraine. Such assessment enables to obtain information for priority determination in banking supervision activity and necessary materials and financial resources for performing adequate control over banking system. At the same time, system CAMELS envisages the detail supervision and analysis of bank state. Such analysis may be performed only while complex inspecting checking of bank activity, which enables to determine how the top managers analyze and control bank risks.
The base of rating system, CAMELS, is risk assessment and determination of rating estimates by each component of the system: capital adequacy, assets quality, management, liquidity, and sensitivity. Due to rating system, each bank obtain digital rating by all six components, and integral (complex) rating estimate is determined on the base of rating estimates of all components. Components of rating system are estimated by 5 balls scale in which estimate 1 is the highest, and estimate 5 is the lowest one. Integral estimate is also determined by 5 balls scale. Banks that obtained integral rating estimate 1 or 2 are considered reliable by all the factors capable to overcome economic depression and its management believed to be qualified.
Banks that got integral estimate 3 have substantial drawbacks, which may lead to serious problems with liquidity and solvency if these drawbacks won't be corrected in proper time. In this case, bank's supervision system should give recommendations to managers how to overcome existing problems. Banks that got rating estimate 4 or 5 have serious problems, which demand strict supervision and special urgent actions to prevent possible bankruptcy (see Table 13).

Comparative experimental investigations of efficiency of bankruptcy risk forecasting by systems CAMEL and FNN TSK
For bankruptcy risk forecasting in banking sphere of Ukraine, a special data set was collected consisting of 160 Ukrainian banks in the period 2012-2014 . It was divided into training and test subsamples in ratio 70/30 for FNN TSK, i.e., training samples consisted of 110 banks and test samples of 50 banks. The experiments were carried out, and the following results were obtained for FNN TSK (in average, 20 experiments were performed for each year and rules number), which are presented in Table 2. The data were collected in the year indicated in the first column, and the forecasting was made for next year, e.g., 2012-5-means bankruptcy risk forecasting in 2013 with use of 5 rules in FNN TSK by data of 2012. Two types of experiments were carried out with fixed parameters of membership functions (MF) and with training MF parameters. In Table 14, forecasting results for FNN TSK with adaptation of parameters are presented and in Table 15 with fixed parameters values.
In Table 16, forecasting results for FNN TSK with triangular MF are presented, while in Table 17 with trapezoidal MF.

Bank financial state
Bank is stable, reliable, and has skilled management Bank has substantial drawbacks, which may lead to serious problems in future.
Bank faces very serious problems, which may lead to bankruptcy.

Control from banks supervision service
Bank supervision service should give clear instructions to overcome existing problems.
Banks need urgent actions to prevent possible bankruptcy.

Application of special actions
Proper influence actions are performed over bank due to demands of existing regulation laws of NBU.   The application of well-known matrix method by Nedosekin [11,12] with level (threshold) of cut 0.7 gave the following results presented in Table 18.
Results obtained by rating system CAMEL are presented in   In Figure 2, the probability of error for different forecasting methods and various MF is presented.
In Figure 3, dependence of error probability versus number of rules in FNN TSK is presented.
Analyzing the performed experiments, the following conclusions may be made. 3. With the increase of rules, number error probability first decreases, then attains minimum and then begins to raise.

General conclusions of investigations for Ukrainian banks
Various methods for Ukrainian banks financial state forecasting were considered and analyzed. The following methods were considered [3,4]: fuzzy neural network  Table 19.
Forecasting results of rating system CAMEL.  ANFIS, fuzzy neural network TSK, Kromonov's method, Byelorussian bank association method, rating system CAMELS, and matrix method (Nedosekin). As the input data, the financial indices of Ukrainian banks were considered. While experiments with the adequate financial indicators were detected using which the best forecasting results for Ukrainian banks were obtained: general reliability factor (own capital/assets); instant liquidity factor (liquid assets/liabilities); cross coefficient (total liabilities/working assets); general liquidity coefficient (liquid assets + defended capital + capitals in reserve fund/total liabilities); and coefficient of profit fund capitalization.
1. It was established that FNN TSK gives much more accurate results than FNN ANFIS. With increase of rules, number error probability first decreases, then attains minimum and then begins to raise.
2. The fuzzy GMDH gives better results using older data that is, more preferable for long-term forecasting (two or more years).
3. The comparison of FNN TSK with standard rating system CAMELS has shown that TSK enables to obtain more accurate bankruptcy risk forecasting. 4.In general, the comparative analysis had shown that fuzzy forecasting methods and techniques give better results than the conventional crisp and rating methods for forecasting bankruptcy risk. But at the same time, the crisp methods are more simple in implementation and demand less time for their adjustment.

Comparative analysis of bankruptcy risk forecasting methods for European banks under uncertainty 4.1 Introduction
The results of successful application of fuzzy methods for bankruptcy risk forecasting of Ukrainian banks under uncertainty stimulated the further investigations of these methods application for financial state analysis of European leading banks.
The main goal of this exploration was to investigate novel methods of European banks bankruptcy risk forecasting, which may work under uncertainty with incomplete and unreliable data.
Besides, the other goal of this investigation was to determine which factors (indicators) are to be used in forecasting models to obtain results close to real data. Therefore, we used a set of financial indicators (factors) of European banks according to the International accountant standard IFRS. The annual financial indicators of about 300 European banks were collected in 2004-2008, preceding the start of crisis of bank system in Europe in 2009. The data source is the information system Bloomberg [8]. The resulting sample included the reports only from the largest European banks as system Bloomberg contains the financial reports only from such banks. For correct utilization, input data were normalized in interval [0,1].

Application of fuzzy neural networks for European banks bankruptcy risk forecasting
The period for which the data were collected was 2004-2008. The possible bankruptcy was analyzed in 2009. The indicators of 165 banks were considered among which more than 20 banks displayed the worsening of the financial state in that year. Fuzzy neural networks and Fuzzy Group Method of Data Handling (FGMDH) were used for bank financial state forecasting.
In accordance with the above stated goal, the investigations were carried out for detecting the most informative indicators (factors) for financial state analysis and bankruptcy forecasting. Taking into account incompleteness and unreliability of input data, FNN ANFIS and TSK were suggested for bankruptcy risk forecasting.
After performing a number of experiments, the data set of financial indicators was found using which FNN made the best forecast. These indicators are the following: debt/assets = (short-term debt + long-term debt)/total assets; loans to deposits ratio; net interest margin (NIM) = net interest income/earning assets; return on equity (ROE) = net income/stockholder equity; return on assets (ROA) = net income/assets equity; cost/income = operating expenses/operating income; and equity/assets = total equity/total assets. The total results of application FNN TSK for different rules number and data collection period are presented in Table 20.
Furthermore, the similar experiments were performed with FNN ANFIS, while the period of data collection varied since 2004-2007. The corresponding results for FNN ANFIS are presented in Table 21 showing the influence of data collection period on forecasting accuracy.
After analysis of these results, the following conclusions were made: 1. FNN TSK has better forecasting accuracy than FNN ANFIS; 2. the best input variables (indicators) for European banks bankruptcy risk forecasting are the following: debt/assets = (short-term debt + long-term debt)/total assets; loans to deposits; net interest margin (NIM) = net interest income/earning assets; return on equity (ROE) = net income/stockholder equity; return on assets (ROA) = net income/assets equity; cost/income = operating expenses/operating income; and equity/assets = total equity/total assets.
Input data collection period (forecasting interval) makes influence on forecasting results.

The application of fuzzy GMDH for bank financial state forecasting
In next experiments, Fuzzy Group Method of Data Handling (FGMDH) was applied for European banks financial state forecasting. Fuzzy GMDH enables to construct forecasting models using experimental data automatically without expert [3]. The additional advantage of FGMDH is possibility to work with the fuzzy information.
As the input data in these experiments, the same indicators as in experiments with FNN TSK were used. In Table 22 If to compare the results of FGMDH with the results of FNN TSK, one can see that neural network has better accuracy at the short forecasting interval (1 year), while fuzzy GMDH has better accuracy at the greater intervals (2 or more years). This conclusion coincides with similar conclusion for Ukrainian banks.
In Table 23, the comparative results of application of different methods for bankruptcy risk forecasting are presented

Regression models
For estimation of fuzzy methods' efficiency at the problem of bankruptcy risk forecasting the comparison with crisp method, the regression analysis of linear models was performed. As input data, the same indicators were used, which were found optimal for FNN. Additionally, the index net financial result was also included in the input set. This index makes great impact on forecasting results.  The input data were normalized before the application. The experiments were carried out with full regression ARMA model, which used eight variables and shortened models with six and four variables.
Each obtained model was checked on testing sample consisting of 50 banks. The comparative forecasting results for all ARMA models are presented in Table 24.
As one may see in Table 24, the application of all types of linear regression models gives the same error of 18%, which is much worse than application of fuzzy neural networks.

Logit models
Furthermore, the experiments were performed using logit models for bankruptcy forecasting [9,10]. The training sample consisted of 165 banks and the testing sample of 50 banks.
The first one was constructed, linear logit model, using all the input variables. It has the following form (estimating and forecasting equations): þC 4 ð Þ * X 3 þ C 5 ð Þ * X 4 þ C 6 ð Þ * X 5 þ C 7 ð Þ * X 6 þ C 8 ð Þ * X 7 þ C 9 ð Þ * X 8 ÞÞ The next constructed model was a linear probabilistic logit model with six independent variables. The final table including the forecasting results of all the logit models is presented below (

Probit models
The next experiments were carried out with probit models [9,10]. The first constructed model was the linear probit model based on 206 banks using all the input variables. It has the following form: As the experiments had shown that the inputs net interest margin (X 3 ) and net financial result (X 8 ) very weakly influence on the forecasting accuracy, they were excluded in the next experiments. The next probit model included six variables.
Furthermore, in this model insignificant variables debt/assets (X 1 ) and loans/ deposits (X 2 ) were excluded, and as a result, linear probit model with four variables was obtained.
Each of the constructed probit models was checked on the test sample of 50 banks. The results of application of all probit models are presented in Table 26.
As one may see from Table 26, the application of all the probit models gives relative error 14-18%, which is much worse than results obtained by fuzzy neural networks. It is worth to mention the decrease of model forecasting quality after exclusion of insignificant variables.

Concluding experiments
In the final series of experiments, investigations and detailed analysis of various methods for forecasting bankruptcy risk were performed. The following methods were investigated: FNN ANFIS, FNN TSK, FGMDH, regression models, logit models, and probit models.
Period of input data was 2007 (1 year before possible bankruptcy). Comparative analysis of all the forecasting methods is presented in Table 27.
As one may see from this table, fuzzy methods and models show much better results than crisp methods: ARMA, logit models, and probit models. When forecasting by one year prior to current date, fuzzy neural network TSK shows better results than FGMDH. But when forecasting for longer intervals (several years), FGMDH is the best method.
In a whole, the conclusions of experiments with European banks completely confirmed the conclusions of experiments with Ukrainian banks.  Table 26.
Forecasting results of probit models.