Comparison of results.

## 1. Introduction

Many communities obtain their drinking water from underground sources called aquifers. Official water suppliers or public incorporations drill wells into soil and rock aquifers looking for groundwater contained there in order to supply the population with drinking water. An aquifer can be defined as a geologic formation that will supply water to a well in enough quantities to make possible the production of water from this formation. The conventional estimation of the exploration flow involves many efforts to understand the relationship between the structural and physical parameters. These parameters depend on several factors, such as soil properties and hydrologic and geologic aspects [1].

The transportation of water to the reservoirs is usually done through submerse electrical motor pumps, being the electric power one of the main sources to the water production. Considering the increasing difficulty to obtain new electrical power sources, there is then the need to reduce both operational costs and global energy consumption. Thus, it is important to adopt appropriate operational actions to manage efficiently the use of electrical power in these groundwater hydrology problems. For this purpose, it is essential to determine a parameter that expresses the energetic behavior of whole water extraction set, which is here defined as *Global Energetic Efficiency Indicator* (*GEEI*). A methodology using artificial neural networks is here developed in order to take into account several experimental tests related to energy consumption in submerse motor pumps.

The *GEEI* of a depth is given in Wh/m^{3}.m. From a dimensional analysis, we can observe that the smaller numeric value of *GEEI* indicates the better energetic efficiency to the water extraction system from aquifers.

For such scope, this chapter is organized as follows. In Section 2, a brief summary about water exploration processes are presented. In Section 3, some aspects related to mathematical models applied to water exploration process are described. In Section 4 is formulated the expressions for defining the *GEEI*. The neural approach used to determine the *GEEI* is introduced in Section 5, while the procedures for estimation of aquifer dynamic behavior using neural networks are presented in Section 6. Finally, in Section 7, the key issues raised in the chapter are summarized and conclusions are drawn.

## 2. Water Exploration Process

An aquifer is a saturated geologic unit with enough permeability to transmit economical quantities of water to wells [10]. The aquifers are usually shaped by unconsolidated sands and crushed rocks. The sedimentary rocks, such as arenite and limestone, and those volcanic and fractured crystalline rocks can also be classified as aquifers.

After the drilling process of groundwater wells, the test known as *Step Drawdown Test* is carried out. This test consists of measuring the aquifer depth in relation to continue withdrawal of water and with crescent flow on the time. This depth relationship is defined as *Dynamic Level* of the aquifer and the aquifer level at the initial instant, i.e., that instant when the pump is turned on, is defined as *Static Level*. This test gives the maximum water flow that can be pumped from the aquifer taking into account its respective dynamic level. Another characteristic given by this test is the determination of *Drawdown Discharge Curves*, which represent the dynamic level in relation to exploration flow [2]. These curves are usually expressed by a mathematical function and their results have presented low precision.

Since aquifer behavior changes in relation to operation time, the *Drawdown Discharge Curves* can represent the aquifer dynamics only in that particular moment. These changes occur by many factors, such as the following: i) aquifer recharge capability; ii) interference of neighboring wells or changes in its exploration conditions; iii) modification of the static level when the pump is turned on; iv) operation cycle of pump; and v) rest time available to the well. Thus, the mapping of these groundwater hydrology problems by conventional identification techniques has become very difficult when all above considerations are taken into account. Besides the aquifer behavior, other components of the exploration system interfere on the global energetic efficiency of the system.

On the other hand, the motor-pump set mounted inside the well, submersed on the water that comes from the aquifer, receives the whole electric power supplied to the system. From an eduction piping, which also supports physically the motor pump, the water is transported to the ground surface and from there, through an adduction piping, it is transported to the reservoir, which is normally located at an upper position in relation to the well. To transport water in this hydraulic system, it is necessary several accessories (valves, pipes, curves, etc.) for its implementation. Figure 1 shows the typical components involved with a water extraction system by means of deep wells.

The resistance to the water flow, due to the state of the pipe walls, is continuous along all the tubing, and will be taken as uniform in every place where the diameter of the pipe to be constant.

This resistance makes the motor pump to supply an additional pressure (or a load) in order to water can reach the reservoir. Thus, the effect created by this resistance is also called “load loss along the pipe”. Similar to the tubing, other elements of the system cause a resistance to the fluid flow, and therefore, load losses. These losses can be considered local, located, accidental or singular, due to the fact that they come from particular points or parts of the tubing.

Regarding the hydraulic circuit, it is observed that the load loss (distributed and located) is an important parameter, and that it varies with the type and the state of the material.

Therefore, old tubing, with aggregated incrustation along the operational time, shows a load loss different of that present in new tubing. A valve turned off twice introduces a bigger load loss than that when it is totally open. A variation on the extraction flow also creates changes on the load loss. These are some observations, among several other points, that could be done.

Another important factor concerning the global energetic efficiency of the system is the geometrical difference of level. However, this parameter does not show any variation after the total implantation of the system. Concerning this, two statements can be done: i) when mathematical models were used to study the lowering of the piezometric surface, these models should frequently be evaluated in certain periods of time; ii) the exploration flow of the aquifer assumes a fundamental role in the study of the hydraulic circuit and it should be carefully analyzed.

In order to overcome these problems, this work considers the use of parameters, which are easily obtained in practice, to represent the capitation system, and the use of artificial neural networks to determine the exploration flow. From these parameters, it is possible to determine the *GEEI* of the system.

## 3. Mathematical Models Applied to Water Exploration Process

One of the most used mathematical models to simulate aquifer dynamic behavior is the Theis’ model [1,9]. This model is very simple and it is used to transitory flow. In this model, the following hypotheses are considered: i) the aquifer is confined by impermeable formations, ii) the aquifer structure is homogeneous and isotropic in relation to its hydro-geological parameters, iii) the aquifer thickness is considered constant with infinite horizontal extent, and iv) the wells penetrate the entire aquifer and their pumping rates are also considered constant in relation to time.

The model proposed by Theis can be represented by the following equations:

where:

*s* is the aquifer drawdown;

*Q* is the exploration flow;

*T* is the transmissivity coefficient;

*r* is the horizontal distance between the well and the observation place.

Applying the Laplace’s transform on these equations, we have:

where:

*w* is the Laplace’s parameter;

*S* is the storage coefficient.

Thus, the aquifer drawdown in the Laplace’s space is given by:

This equation in the real space is as follows:

The Theis’ solution is then defined by:

where:

Finally, from Equation (10), we have:

where:

*L*^{-1} is the Laplace’s inverse operator.

*K*_{0} is the hydraulic conductivity.

From analysis of the Theis’ model, it is observed that to model a particular aquifer is indispensable a high technical knowledge on this aquifer, which is mapped under some hypotheses, such as confined aquifer, homogeneous, isotropic, constant thickness, etc. Moreover, other aquifer parameters (transmissivity coefficient, storage coefficient and hydraulic conductivity) to be explored must be also defined. Thus, the mathematical models require expert knowledge of concepts and tools of hydrogeology.

It is also indispensable to consider that the aquifer of a specific region shows continuous changes in its exploration conditions. The changes are normally motivated by the companies that operate the exploration systems, by drilling of new wells or changes of the exploration conditions, or still, motivated by drilling of illegal wells. These changes have certainly required immediate adjustment on the Theis’ model. Another fact is that the aquifer dynamic level modifies in relation to exploration flow, operation time, static level, and obviously with those intrinsic characteristics of the aquifer under exploration. In addition, neighboring wells will also be able to cause interference on the aquifer.

Therefore, although to be possible the estimation of aquifer behavior using mathematical models, such as those presented in [11]-[16], they present low precision because it is more difficult to consider all parameters related to the aquifer dynamics. For these situations, intelligent approaches [17]-[20] have also been used to obtain a good performance.

## 4. Defining the Global Energetic Efficiency Indicator

As presented in [3], “Energetic Efficiency” is a generalized concept that refers to set of actions to be done, or then, the description of reached results, which become possible the reduction of demand by electrical energy. The energetic efficiency indicators are established through relationships and variables that can be used in order to monitor the variations and deviations on the energetic efficiency of the systems. The descriptive indicators are those that characterize the energetic situation without looking for a justification for its variations or deviations.

The theoretical concept for the proposed Global Energetic Efficiency Indicator will be presented using classical equations that show the relationship between the absorbed power from the electric system and the other parameters involved with the process.

As presented in [3], the power of a motor-pump set is given by:

where:

*P*_{mp} is the power of the motor-pump set (CV);

^{3});

*Q* is the water flow (m^{3}/s);

*H*_{T} is the total manometric height (m);

Substituting the following values {1 CV ^{3}/s = 1/3600 m^{3}/h; ^{3} } in equation (13), we have:

The total manometric height (*H*_{T}) in elevator sets to water extraction from underground aquifers is given by:

where:

*H*_{T} is the total manometric height (m);*H*_{a} is the dynamic level of the aquifer in the well (m);*H*_{g} is the geometric difference in level between the well surface and the reservoir (m);

Δ*hf*_{t} is total load loss in the hydraulic circuit (m).

From analyses on the variables in (15), it is observed that only the variable corresponding to the geometric difference in level (*H*_{g}) can be considered constant, while other two will change along the operation time of the well.

The dynamic level (*H*_{a}) will change (to lower) since the beginning of the pumping until the moment of stabilization. This observation is verified in short period of time, as for instance, a month. Besides this variation, which can present a cyclic behavior, it is possible that other types of variation, due to interferences from other neighboring wells, can take place as well as alterations in the aquifer characteristics.

The total load loss will also vary during the pumping, and it is dependent on hydraulic circuit characteristics (diameter, piping length, hydraulic accessories, curves, valves, etc.).

These characteristics can be considered constant, since they usually do not change after installed. However, the total load loss is also dependent on other characteristic of the hydraulic circuit, which frequently changes along the useful life of the well. These variable characteristics are given by: i) roughness of the piping system, ii) water flow, and iii) operational problems, such as semi-closed valves, leakage, etc.

Observing again Figure 1, it is verified that the necessary energy to transport the water from the aquifer to the reservoir, overcoming all the inherent load losses, it is supplied by the electric system to the motor-pump set. Thus, using these considerations and substituting (15) in (14), we have:

where:

*P*_{el} is the electric power absorbed from electric system (W);

*Q* is the water flow (m^{3}/h);

*H*_{a} is the dynamic level of the aquifer in the well (m);

*H*_{g} is the geometric difference of level between the well surface and the reservoir (m);

Δ*hf*_{t} is the total load loss in the hydraulic circuit (m);

From (16) and considering that an energetic efficiency indicator should be a generic descriptive indicator, the *Global Energetic Efficiency Indicator* (*GEEI*) is here proposed by the following equation:

Observing equation (17), it is verified that the *GEEI* will depend on electric power, water flow, dynamic level, geometric difference of level, and total load loss of the hydraulic circuit.

The efficiency of the motor-pump set does not take part in (17) because its behavior will be reflected inversely by the *GEEI*. Thus, when the efficiency of the motor-pump set is high, the *GEEI* will be low. Therefore, the best *GEEI* will be those presenting the smallest numeric values.

Another reason to exclude the efficiency of the motor-pump set in (17) is the difficulty to obtain this value in practice. Since it is a fictitious value, it is impossible to make a direct measurement and its value is obtained through relationships between other quantities. After the beginning of the pumping, it is occurred the lowering of water level inside the well. Then, the manometric height changes and as result the water flow also changes. The efficiency of a motor-pump set will also change along its useful life due to the equipment wearing, piping incrustations, leakages in the hydraulic system, obstructions of filters inside the well, closed or semi-closed valves, etc.

Therefore, converting all variables in (17) to meters, the most generic form of the *GEEI* is given by:

The *GEEI* defined in (18) can be used to analyze the well behavior along the time.

## 5. Neural Approach Used to Determine the Global Energetic Efficiency Indicator

Among all necessary parameters to determine the proposed *GEEI*, the determination of the exploration flow is the most difficult to obtain in practice. The use of flow meters, as the electromagnetic ones, is very expensive. The use of rudimentary tests has provided imprecise results.

To overcome this practical problem, it is proposed here the use of artificial neural networks to determine the exploration flow from other parameters that have been measured before determining the *GEEI*.

Artificial Neural Networks (ANN) are dynamic systems that explore parallel and adaptive processing architectures. They consist of several simple processor elements with high degree of connectivity between them [4]. Each one of these elements is associated with a set of parameters, known as network weights, that allows the mapping of a set of known values (network inputs) to a set of associated values (network outputs).

The process of weight adjustment to suitable values (network training) is carried out through successive presentation of a set of training data. The objective of the training is the minimization between the output (response) generated by the network and the respective desired output. After training process, the network will be able to estimate values for the input set, which were not included in the training data.

In this work, an ANN will be used as a functional approximator, since the exploration flow of the well is a dependent variable of those ones that will be used as input variables. The functional approximation consists of mapping the relationship between the several variables that describe the behavior of a real system [5].

The ability of neural artificial networks to mapping complex nonlinear functions makes them an attractive tool to identify and to estimate models representing the dynamic behavior of engineering processes. This feature is particularly important when the relationship between several variables involved with the process is nonlinear and/or not very well defined, making its modeling difficult by conventional techniques.

A multilayer perceptron (MLP), as that shown in Figure 2, trained by the backpropagation algorithm, was used as a practical tool to determine the water flow from the measured parameters.

The input variables applied to the proposed neural network were the following:

• Level of water in meters (*H*_{a}) inside the well at the instant *t*.

• Manometric height in meters of water column (*H*_{m}) at the instant *t*.

• Electric power in Watts (*P*_{el}) absorbed from the electric system at the instant *t*.

The unique output variable was the exploration flow of the aquifer (*Q*), which is expressed in cubic meters per hour. It is important to observe that for each set of input values at a certain instant *t*, the neural network will return a result for the flow at that same instant *t*.

The determination of *GEEI* will be done by using in equation (18) the flow values obtained from the neural network and other parameters that come from experimental measurements.

To training of the neural network, all these variables (inputs and output) were measured and provided to the network. After training, the network was able to estimate the respective output variable. The values of the input variables and the respective output for a certain pumping period, which were used in the network training, are given by a set composed by 40 training patterns (or training vectors).

These patterns were applied to a neural network of MLP type (Multilayer Perceptron) with two hidden layers, and its training was done using the *backpropagation* algorithm based on the Levenberg-Marquardt’s method [6]. A description of the main steps of this algorithm is presented in the Appendix.

The network topology that was used is similar to that presented in Figure 2. The number of hidden layers and the number of neurons in each layer were determined from results obtained in [7,8]. The network is here composed by two hidden layers and the following parameters were used in the training process:

• Number of neurons of the 1^{st} hidden layer: 15 neurons.

• Number of neurons of the 2^{nd} hidden layer: 10 neurons.

• Training algorithm: Levenberg-Maquart.

• Number of training epochs: 5000 epochs.

At the end training process, the mean squared error obtained was 7.9x10^{-5}, which is a value considered acceptable for this application [7].

After training process, values of input variables were applied to the network and the respective values of flow were obtained in its output. These values were then compared with the measured ones in order to evaluate the obtained precision.

Table I shows some values of flow that were given by the artificial neural network (*Q*_{ANN}) and those measured by experimental tests (*Q*_{ET}).

Ha (m) | Hm (m) | Pel (W) | QANN (m3/h) | QET (m3/h) |

25.10 | 8.25 | 26,256 | 74.99 | 75.00 |

31.69 | 40.50 | 26,155 | 53.00 | 62.00 |

31.92 | 48.00 | 25,987 | 56.00 | 56.00 |

31.12 | 48.00 | 25,953 | 55.00 | 55.00 |

32.50 | 48.00 | 25,970 | 54.08 | 54.00 |

32.74 | 48.00 | 25,970 | 54.77 | 54.50 |

33.05 | 48.00 | 25,937 | 54.15 | 54.00 |

33.26 | 48.00 | 25,954 | 58.54 | 54.00 |

33.59 | 48.00 | 25,869 | 53.01 | 53.00 |

33.83 | 48.00 | 25,886 | 53.49 | 53.50 |

34.15 | 48.00 | 25,887 | 53.50 | 53.00 |

34.41 | 48.00 | 25,886 | 53.48 | 53.50 |

34.71 | 48.00 | 25,785 | 53.25 | 53.30 |

34.95 | 48.00 | 25,870 | 53.14 | 53.00 |

35.00 | 48.00 | 25,801 | 53.14 | 53.00 |

In this table, the values in bold were not presented to the neural network during the training.

When the patterns used in the training are presented again, it is noticed that the difference between the results is very small, reaching the maximum value of 0.35% of the measured value. When new patterns are used, the highest error reaches the value of 14.5%. It is also observed that the error value to new patterns decreases when they represent an operational stability situation of the motor-pump set, i.e., they are far away from the transitory period of pumping.

At this point, we should observe that it would be desirable a greater number of training patterns for the neural network, especially if it could be obtained from a wider variation of the range of values.

The proposed *GEEI* was determined by equation (18) and the measured values used were the electric power, the dynamic level, the geometric difference of level, the pressure of output in the well, and the water flow obtained from the neural network.

Figure 3 shows the behavior of *GEEI* during the analyzed pumping period.

The numeric values that have generated the graphic in Figure 3 are presented in Table 2.

Operation Time (min) | GEEI(t) (Wh/m3.m) | Operation Time (min) | GEEI(t) (Wh/m3.m) |

0 | 7.420* | 40 | 5.054 |

1 | 4.456* | 45 | 5.139 |

2 | 5.738* | 50 | 5.134 |

3 | 5.245* | 55 | 5.115 |

4 | 4.896* | 60 | 5.073 |

5 | 4.951* | 75 | 5.066 |

6 | 4.689* | 90 | 5.060 |

7 | 5.078* | 105 | 5.042 |

8 | 4.840* | 120 | 5.037 |

9 | 5.027* | 135 | 5.042 |

10 | 5.090* | 155 | 5.026 |

11 | 5.100* | 185 | 5.032 |

12 | 5.092* | 215 | 5.030 |

14 | 5.066* | 245 | 5.040 |

16 | 5.044* | 275 | 5.034 |

18 | 5.015* | 305 | 5.027 |

20 | 5.006* | 335 | 5.017 |

22 | 5.017 | 365 | 5.025 |

24 | 5.022 | 395 | 5.030 |

26 | 5.032 | 425 | 5.031 |

28 | 5.049 | 455 | 5.020 |

30 | 5.062 | 485 | 5.015 |

35 | 4.663 |

## 6. Estimation of Aquifer Dynamic Behavior Using Neural Networks

In this section, artificial neural networks are now used to map the relationship between the variables associated with the identification process of aquifer dynamic behavior.

The general architecture of the neural system used in this application is shown in Figure 4, where two neural networks of type MLP, MLP-1 and MLP-2, constituted respectively by one and two hidden layers, compose this architecture.

The first network (ANN-1) has 10 neurons in the hidden layer and it is responsible by the computation of the aquifer operation level. The training data for ANN-1 were directly obtained from experimental measurements. It is important to note that this network has taken into account the present level and rest time of the aquifer.

The second network (ANN-2) is responsible by the computation of the aquifer dynamic level and it is composed by 2 hidden layers with both having 10 neurons. For this network, the training data were also obtained from experimental measurements. As observed in Figure 4, the ANN-1 output is provided as an input parameter to the ANN-2. Therefore, the computation of the aquifer dynamic level takes into account the aquifer operation level, the exploration flow and operation time.

After training process of the neural networks, they were used for estimation of the aquifer dynamic level. The simulation results obtained by the networks are presented in Table 3 and Table 4.

Present Level(meters) | Rest Time(hours) | Operation Level(ANN-1) | Operation Level(Exact) | Relative Error(%) |

115.55 | 4 | 103.59 | 104.03 | 0.43 % |

125.86 | 9 | 104.08 | 104.03 | 0.05 % |

141.26 | 9 | 105.69 | 104.03 | 1.58 % |

137.41 | 8 | 102.95 | 104.03 | 1.05 % |

Table 3 presents the simulation results obtained by the ANN-1 for a particular well. The operation levels computed by the network taking into account the present level and rest time of the aquifer were compared with those results obtained by measurements. In this table, the ‘Relative Error’ column provides the relative error between the values estimated by the network and those obtained by measurements.

Operation Flow(m3/h) | Operation Time(hours) | DynamicLevel(ANN-2) | DynamicLevel(Exact) | RelativeError(%) |

145 | 14 | 115.50 | 115.55 | 0.04 % |

160 | 2 | 116.10 | 116.14 | 0.03 % |

170 | 6 | 118.20 | 117.59 | 0.52 % |

220 | 21 | 141.30 | 141.26 | 0.03 % |

The simulation results obtained by the ANN-2 are provided in Table 4. The dynamic level of the aquifer is estimated by the network in relation to operation level (computed by the ANN-1), exploration flow and operation time. These results are also compared with those obtained by measurements. In Table 4, the ‘Relative Error’ column gives the relative error between the values computed by the network and those from measurements.

These results show the efficiency of the neural approach used for estimation of aquifer dynamic behavior. The values estimated by the network are accurate to within 1.5% of the exact values for ANN-1 and 0.5 for ANN-2. From analysis of the results presented in Table 3 and 4, it is verified that the relative error between values provided by the network and those obtained by experimental measurements is very small. For ANN-1, the greatest relative error is 1.58 % (Table 3) and for ANN-2 is 0.52% (Table 4).

## 7. Conclusion

The management of systems that explore underground aquifers includes the analysis of two basic components: the water, which comes from the aquifer; and the electric energy, which is necessary to the transportation of the water to the consumption point or reservoir. Thus, the development of an efficiency indicator that shows the energetic behavior of a certain capitation system is of great importance to efficient management of the energy consumption, or still, to convert the obtained results in actions that become possible a reduction of energy consumption.

The obtained *GEEI* will indicate the global energetic behavior of the water capitation system from aquifers and will be an indicator of occurrences of abnormalities, such as tubing breaks or obstructions.

The application of the proposed methodology uses parameters that have easily been obtained in the water exploration system. The *GEEI* calculus can also be done by operators or to be implemented by means of computational system.

In addition, a novel methodology for estimation of aquifer dynamic behavior using artificial neural networks was also presented in this chapter. The estimation process is carried out by two feedforward neural networks. Simulation results confirm that proposed approach can be efficiently used in these types of problem. From results, it is possible to simulate several situations in order to define appropriate management plans and policies to the aquifer.

The main advantages in using this neural network approach are the following: i) velocity: the estimation of dynamic levels are instantly computed and it is appropriated for application in real time, ii) economy and simplicity: reduction of operational costs and measurement devices, and iii) precision: the values estimated by the proposed approach are as good as those obtained by physical measurements.

## 8. Appendix

The mathematic model that describes the behavior of the artificial neuron is expressed by the following equation:

where *n* is the number of inputs of the neuron; *x*_{i} is the *i*-th input of the neuron; *w*_{i} is the weight associated with the *i*-th input; *b* is the threshold associated with the neuron; *u* is the activation potential; *g*( ) is the activation function of the neuron; *y* is the output of the neuron.

Basically, an artificial neuron works as follows:

(a) Signals are presented to the inputs.

(b) Each signal is multiplied by a weight that represents its influence in that unit.

(c) A weighted sum of the signals is made, resulting in a level of activity.

(d) If this level of activity exceeds a certain threshold, the unit produces an output.

To approximate any continuous nonlinear function a neural network with only a hidden layer can be used. However, to approximate non-continuous functions in its domain it is necessary to increase the amount of hidden layers. Therefore, the networks are of great importance in mapping nonlinear processes and in identifying the relationship between the variables of these systems, which are generally difficult to obtain by conventional techniques.

The network weights (*w*_{j}) associated with the *j*-th output neuron are adjusted by computing the error signal linked to the *k*-th iteration or *k*-th input vector (training example). This error signal is provided by:

where *d*_{j}(*k*) is the desired response to the *j*-th output neuron.

Adding all squared errors produced by the output neurons of the network with respect to *k*-th iteration, we have:

where *p* is the number of output neurons.

For an optimum weight configuration, *E*(*k*) is minimized with respect to the synaptic weight *w*_{ji}. The weights associated with the output layer of the network are therefore updated using the following relationship:

where *w*_{ji} is the weight connecting the *j*-th neuron of the output layer to the *i*-th neuron of the previous layer, and *η* is a constant that determines the learning rate of the backpropagation algorithm.

The adjustment of weights belonging to the hidden layers of the network is carried out in an analogous way. The necessary basic steps for adjusting the weights associated with the hidden neurons can be found in [4].

Since the backpropagation learning algorithm was first popularized, there has been considerable research into methods to accelerate the convergence of the algorithm.

While backpropagation is a steepest descent algorithm, the Marquardt-Levenberg algorithm is similar to the quasi-Newton method, which was designed to approach second-order training speed without having to compute the Hessian matrix.

When the performance function has the form of a sum of squared errors like that presented in (22), then the Hessian matrix can be approximated as

and the gradient can be computed as

where **e **is a vector of network errors, and *J *is the Jacobean matrix that contains first derivatives of the network errors with respect to the weights and biases.

The Levenberg-Marquardt algorithm uses this approximation to the Hessian matrix in the following Newton-like update:

When the scalar *µ* is zero, this is Newton's method, using the approximate Hessian matrix. When *µ* is large, this produces a gradient descent with a small step size. Newton’s method is faster and more accurate near to an error minimum, so the aim is to shift toward Newton’s method as quickly as possible.

Thus, *µ* is decreased after each successful step (reduction in performance function) and is increased only when a tentative step would increase the performance function. In this way, the performance function is always reduced at each iteration of the algorithm [6].

This algorithm appears to be the fastest method for training moderate-sized feedforward neural networks (up to several hundred weights).