Introductory Chapter: ANFIS for Modeling Multivariable Nonlinear Functions

1. Introduction: general aspects and applications

Treating information using fuzzy logic and neural networks has developed throughout the last decades, these mathematical theories of artificial intelligence being interesting tools for researchers to solve complex scientific and technical problems. The combination of two concepts, artificial neural networks and Takagi-Sugeno fuzzy inference, led to the development of the adaptive neuro-fuzzy inference system (ANFIS). It has learning and adaptive capabilities, to approximate nonlinear functions, and it is considered to be a universal estimator with the potential for application in nonlinear and complex systems. The purpose of the chapter is to make an introduction for this book, highlighting the diversity of possible applications of ANFIS and presenting an example of a basic application in modeling and estimation [1].

In the architecture of ANFIS, there are two parts: the premise and the consequence. The neural network architecture has five layers. The first layer is the fuzzification layer, where the membership degrees are computed. The second layer is the rule layer, that is, generating the firing strengths for the rules. The third layer normalizes the computed firing strengths. The fourth layer offers the defuzzificated values. The last layer gives the final output. Over the years, researchers have contributed to the development both on the side of artificial neural networks, with new training methods and neuron activation functions, as well as on the fuzzy logic side, with fuzzy inference, membership functions, rule bases, and tuning of parameters, resulting in complex and hybrid structures. In ANFIS approaches, various optimization methods are used such as genetic algorithms, differential evolution, particle swarm optimization, shuffled frog leaping algorithm, satin bowerbird optimization algorithms, metaheuristic techniques, or nature-inspired algorithms [2].

ANFIS, as a universal estimator, has found applications in various sectors of human activity, such as industry (mechanics, electrical engineering, power systems, electronics, and chemistry), economy, business, finance, medicine, biology, and in many scientific fields such as modeling, prediction, machine learning, big-data technologies, control systems, expert systems, and others. This concept has been used for several decades in various modeling and prediction applications. Researchers in ANFIS develop new concepts and tools which enhance human understanding and improve the specialist’s ability to design and implement high-performance systems. The researches cover aspects of ANFIS architectures, optimization techniques, applications of ANFIS as a universal estimator in modeling, and prediction in different domains of activity [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]. Thus, several applications can be listed: in power plants and power distribution networks, in metallurgy in the design of aluminum alloys, in the chemical industry for specific wear rate modeling of poly-tetra-flouro-ethylene composites, in electrical engineering, for inductance profile estimation for switched reluctance motors, for the definition of focal length for zoom lens for digital cameras, an adaptive filtering application as channel equalizer for mobile cellular channels, in medicine, as an application in health monitoring based on multisensor, data fusion and 2D wavelet transform, in civil engineering for prediction of compressive strength of manufactured sand concrete, in tracking disasters in the environment for spatial prediction of landslide susceptibility, in computer science, for data classification, application to identifying the online bearing fault, by filtering impulse noise, or for improving estimation algorithms for distributed parameter systems using sensor networks.

2. Example

2.1 Method

ANFIS has a certain capability of modeling a nonlinear function. Because it is a neural network with five layers, it can be said that the approximation function results as a function composed of five other functions. It is possible to try to use ANFIS for modeling, approximating, or estimating a nonlinear function of several variables, but the obtained result will have a certain error greater or less depending on multiple conditions of the process of obtaining ANFIS, such as the amount of training data used, depending on the structure used for the display, depending on the training method, depending on the available computing power, and others.

ANFIS can be seen as a function of n variables, as a mathematical application on the set of real numbers with values in the set of real numbers:

$$f:R^n\to R$$E1

ANFIS is recommended among recent computing techniques because of their tools to deal with nonlinear modeling. Such an example is presented recently in [18].

We are choosing a similar example, a nonlinear function with two variables, the sinc function:

$$z=\frac{10\sin \left(r\right)}{r},r=\sqrt{x^2+y^2},x\in \left[-10\text{,}10\right],y\in \left[-10\text{,}10\right]$$E2

The graphic of this function is presented in Figure 1.

Figure 1.

Graph of the function.

The chosen example of modeling a nonlinear multivariable function is suitable to use neuro-adaptive learning.

To make calculation for this example, the neuro-fuzzy designer and fuzzy-logic designer apps from MATLAB 2018 and 2023 were used. These apps let the user to design, train, and test adaptive neuro-fuzzy inference systems, using input/output training data, as shown below.

The fuzzy inference system is a mathematical concept that has the following information flow: crisp characteristics from the inputs—membership functions—fuzzy rules for the inputs—fuzzy rules for output characteristics—output membership functions—a single-valued output, or a decision associated with the output. So, membership functions and a rule structure are used, the processes being developed based on the user’s experience of working with the process. The system applies fuzzy inference techniques. In the ANFIS concept, a neuro-adaptive learning method is used, for learning information about an input-output data set of training. The membership function parameters resulted after training. The membership functions of the fuzzy inference system are trained using a back propagation algorithm alone or in combination with a least squares method. A gradient vector is used to compute these parameters. We may say the fuzzy system can learn from the data it is modeling. The parameters of the membership functions may be changed through the learning process. The resulted neural network can interpret the input/output map. The gradient vector provides a criterion of modeling quality, in reducing the training errors. The error criterion is the sum of the squared difference between actual and desired outputs. The data are taken from Eq. (2). The modeling approach resembles system identification techniques. In the first stage, the equations are used as a parameterized model structure. This model is a relation from inputs to membership functions to rules to output to membership function and so on. In the second stage, input/output data are collected to train the ANFIS. The training data must be fully representative for the system characteristic to be modeled. A high number of samples are recommended. The second stage is the model validation, in which a set of input/output data is presented to the trained ANFIS, to see how well the ANFIS predicts the corresponding data set output values. The data set for model validation must be also representative for the system is intended to emulate, but in the same time, it must be distinct from the training data set. This is to avoid a trivial validation procedure. Again, a large amount of data is necessary. In this stage, the generalization capability of the ANFIS is checked. Also, the procedure verifies whether the model is overfitting the training data set.

The following phases are completed within the application: calculation of membership function parameters, generating an initial inference system based on the training data, establishing the structure of the system before training, avoiding overfitting using additional checking data, testing the generalization capability of the trained system using testing data, and exporting the resulted fuzzy system to the computing workspace. In this case, the fuzzy inference system is trained for one output and two inputs. A weighted average defuzzification is used. The output membership function is linear. The number of rules matches the number of output membership function. Every rule has a different consequence. It does not use custom membership functions. Neuro-fuzzy designer is an app from control system design and analysis. The neuro-fuzzy designer MATLAB application may be used for designing, training, and testing adaptive neuro-fuzzy inference systems (ANFIS), using input/output training data.

The architecture of the PC used for training consists of a processor Intel i9, 12th generation, with 3 GHz clock frequency and 32GB RAM memory at 3.4 GHz.

2.2 Results

As presented in the method, a fuzzy inference system is chosen for training, based on Sugeno-type inference, with two inputs and one output, the neural network having three hidden layers. Tests were performed with various input membership functions. The present result was obtained with generalized bell-type membership functions. Tests were performed with three and five input membership functions. The tests with five membership functions required extremely long calculation times and, contrary to expectations, gave larger mean squared errors. 16.008.001 input-output test pairs were used for training. The number of tunable parameters was 45. The optimization method was least squares estimation with backpropagation. The fuzzy inference structure was trained once from error 1.64, during 100 epochs, until error 1.032. The structure obtained after the first training was trained a second time from error 1.03 to error 0.827, during another 100 epochs. The layers of the ANFIS structure used in this example are presented in Figure 2. The graphs of the error variation during the two training periods are shown in Figure 3a and b. The variations of wind values, for the two workouts, are presented in Figure 4a and b. The final, minimal training RMSE, of the ANFIS as an example, was 0.827. The structure of the tuning ANFIS is presented in Figure 5. The membership functions of the first input and second input are presented in Figure 6a and b. The membership functions of the output are presented in Figure 7. The rule base is presented in Figure 8. An example of ANFIS rule inference is presented in Figure 9. The surface of the tuned ANFIS is presented in Figure 10. Figure 11 shows the ANFIS output after the first and second training on the same graph. The error surface obtained for the output of these two trained structures is presented in Figure 12. A validation set with 40.401 input-output pairs was used for validation. Figure 13 compares the outputs of the trained ANFIS and the outputs of the training set on the same graph.

Figure 2.

ANFIS internal structure.

Figure 3.

Training convergence graph.

Figure 4.

Numerical variation of the training error.

Figure 5.

ANFIS structure after training.

Figure 6.

(a) Membership functions of input 1. (b) Membership functions of input 2.

Figure 7.

The membership functions of the output.

Figure 8.

The rules of inference.

Figure 9.

Example of an inference graph.

Figure 10.

Graph of the ANFIS function.

Figure 11.

Output graphs after the first and second training.

Figure 12.

Spatial plot of errors.

Figure 13.

Output graphs: blue – for training data, red – validation output.

3. Conclusion

The chapter presents an example of the development of an ANFIS structure for modeling the function sinc, a nonlinear function of two variables. The stages through which training and validation were carried out are presented. Training results are presented. The ANFIS structure is a standard one, with two inputs, three membership functions for each input, an inference layer with nine nodes and one output with linear activation function. This structure was trained using a huge amount of training pair data. Many training sessions were made, with different parameters. The training was made in two stages, both with 100 epochs, to reduce the training error. The error decreased aperiodic, asymptotically. The descending step of backpropagation was modified automatically at several epochs. The training had to adapt 45 parameters, including the membership functions of the input and output. The membership functions had the shape of the generalized bell. The membership functions of the two inputs are similar. The membership functions of the output, in the number of nine, are linear. The rule base has nine rules. The example shows that a large number of trainings are necessary, using various ANFIS structures and various training sets, and also as a result of the experiments carried out, it turned out that huge computing resources are needed, with millions of training sets and high computing power. The duration of the calculations for training amounted to several hours. After the tests, a structure may be chosen that resulted with a minimum mean squared error. The approximation errors vary a lot on the surface of the nonlinear function, being on certain portions small, acceptable, but on other portions quite large, having to be reduced through other attempts. It is likely that, by doing other trainings, a structure can be obtained that ensures, maybe, smaller approximation errors, but there is no such sure certainty.