Open access peer-reviewed chapter

# On Non-Linearity and Convergence in Non-Linear Least Squares

By Orhan Kurt

Submitted: October 11th 2017Reviewed: March 8th 2018Published: November 5th 2018

DOI: 10.5772/intechopen.76313

## Abstract

To interpret and explain the mechanism of an engineering problem, the redundant observations are carried out by scientists and engineers. The functional relationships between the observations and parameters defining the model are generally nonlinear. Those relationships are constituted by a nonlinear equation system. The equations of the system are not solved without using linearization of them on the computer. If the linearized equations are consistent, the solution of the system is ensured for a probably global minimum quickly by any approximated values of the parameters in the least squares (LS). Otherwise, namely an inconsistent case, the convergence of the solution needs to be well-determined approximate values for the global minimum solution even if in LS. A numerical example for 3D space fixes coordinates of an artificial global navigation satellite system (GNSS) satellite modeled by a simple combination of first-degree polynomial and first-order trigonometric functions will be given. It will be shown by the real example that the convergence of the solution depends on the approximated values of the model parameters.

### Keywords

• nonlinear equation system
• objective function
• least squares
• convergence
• consistency

## 1. Introduction

There are two main computing classes, these are hard and soft computing. Scientists and engineers generally prefer the first-class computing because they can easily establish an explicit mathematical relationship between the model parameters and their data (observations), not in the second class. The relationships between the parameters and data can be linear or nonlinear. If the relations are nonlinear, they should be linearized via Taylor expansion [1, 2, 3, 4, 5, 6, 7]. Therefore, the linear models can be solved by linear algebra [8, 9, 10, 11, 12, 13, 14, 15].

To overcome complicated real-life problems whose mathematical models are not known, the soft computing techniques have been developed in the last decades. We can count well-known techniques, some as artificial neural network (ANN), artificial intelligence (AI), machine learning (ML), deep learning (DP), fuzzy logic (FL) and genetic algorithms (GA) [16, 17, 18]. The techniques inspired by the human intelligence and learning processes can be very time-consuming according to the data given in run due to their processing based on the trial-and-error method. If these techniques are roughly defined, data (experimental outcomes and observations) are separated into two parts in them, learning (or training) data and test data. Mathematical (functional and/or stochastic) relations between data and model parameters are learned from the learning data. The handled model is tested by means of the test data. After that, the trained and developed model, if meets expectations, is used to estimate for producing unobserved data for the scientific (or engineering) problems [16, 17, 18].

In the soft computing techniques, the linear algebra is also a very effective tool to solve the problem as in the hard computing ones. For this reason, we should take a short overview on linear algebra used in science and engineering [16, 17, 18].

## 2. Linear algebra and objective functions

Linear algebra has two basic problems. A solution of linear equations system is one of them; the other is the eigendecomposition. In this chapter, we will use both of them upon a linear equation system as a combined form (Eqs. (8)(11)) in which we will solve the linear equations system by means of the singular value decomposition related with the eigendecomposition (or the matrix diagonalization) [8, 9, 13, 14].

Suppose an estimated unknown vector x̂u=x+δ̂(in interested model) and an experimental data (or observations which are stochastic variables) vector yn=ŷε̂[in which an estimated data and error (residual) vectors are in order of ŷand ε̂] by an objective function and their covariance matrices Σx̂=Σx=σ̂02Qx(for the unknowns) and Σy=σ02P1(for the data), respectively, with a priori variance σ02and a posteriori variance σ̂02. Note that x̂is a non-stochastic vector before estimation, where an approximated values vector is xfor x̂(hat-sign “^” shows an estimated value for interested parameter according to an objective function). In addition, n, mand uare the observation number, the equation number and the unknown number, respectively.

Start with a linear or nonlinear functions vector fmŷx̂=0, we can have a linear mathematical model with a weight matrix (P=σ02Σy1) of the observations for m=n:

εn=An,uδuln,Pn,n,E1
An,u=fŷx̂x̂ŷ,x̂=y,xandln=fyx.

Mathematical model between data and unknowns can be established by Taylor expansion for any model. However, if mpieces function vector fmŷx̂=0is not transformed into ŷnfnx̂=0(for m=n), the error in variable solution as in total least squares (TLS) method can be preferred. Therefore, fmŷx̂=0(for mn) should be differenced as following:

Bm,nεnAm,uδu+lm=0Pn,nE2

where

Bm,n=fŷx̂ŷŷ,x̂=y,x0.

Most of science and engineering problems can be modeled as ŷfx̂=0(m=n). Therefore, the functional model named as indirect adjustment method in the adjustment literature [3, 4, 5, 6, 7] in geomatics engineering has been preferred in the chapter. The weight matrix (Pn,n) of observations (stochastic variables) would be accepted as a unit matrix Pn,n=In,nin here for simplicity.

### 2.1. Objective functions

A generalization for objective functions is LpNorm(p=1,2,3,4,) [9, 10]. The first-degree objective function is L1-norm estimationwhich is accepted as a robust estimation method in just linear models [9, 10, 11].

iεTminL1normestimationLeast absolute residuals,E3
εTεminL2normestimationLeast squares,E4
εmaxminLnormestimationMinmax absolute residuals,E5
i=111T.

The second-degree objective function is L2-norm estimationwhich is known as least squares (LS) method and widely used in hard and soft computations.

The last-degree objective function is L-norm estimationwhich is known as minmax method. In fact, the soft computing techniques use this objective while it applies the trial-and-error method in their learning stages. Eq. (1) under L1-normand L-norm isalso solved by means of linear programming methods, for this reason; the methods may give several solutions (as being in trial-and-error method) to any interested problem [10, 11].

### 2.2. Rank deficiencies in linear models

While a rank is a number that indicates a linear independent column, the number of the coefficient matrix of unknowns in a linear equation system, a rank deficiency represents a linear dependent column number (if it is smaller than the row number) of the coefficient matrix. Inconsistency in the solution stage of a linear equation system results from the (rank) deficiencies. Defining the rank of An,uby rankA=r, a condition ruminmnis always satisfied. In general, n=min well-known (or the indirect) LS used in many scientific problems.

Denoting the rank defect dletter, we can define two type defects .

ds=nr,SurjectivityontomappingE6a
di=ur.InjectivityonetoonemappingE6b

Objective functions are used to remove the surjectivity defect dsoccurred by the redundant observations. The injectivity defect dican consist of three reasons in the estimation problem .

Datum defects (d-defects)are closely related to the origin of the spatial system. The defect arises if the data do not carry any information to cover the absolute spatial position of the problem given.

Configuration (Design) defects (c-defects)occur from weak geometric relation among data and unknowns. To avoid the defect, we can be careful and planned when picking data (whose interval or/and place) and choosing the consistent mathematical model (can use auxiliary variables instead of original ones).

Ill-conditioning defects (i-defects)arise from the large intervals among the elements of the coefficient matrix of unknowns. Norming the matrix can reduce ill-conditioning defects but cannot remove it fully. I-defectsand c-defectscannot be separated from each other easily .

The defects lead to the failure of any given problem to be solved properly. Since the unknown coefficient matrix cannot be inverted by regular (ordinary) inversemethods, we should use pseudo inverseto overcome the effects of the defects [8, 9, 13, 14, 15]. Eigenvalue and singular value decompositions can be used effectively for the pseudoinverse. Denoting a positive definite symmetric matrix N(that is always satisfied for N=ATAor N=AAT), its pseudoinverse is:

Qu,u=N+=SΛ+ST=VΣ+UT,Pseudoinverse ofNE7a
Nu,u=SΛST=UΣVT,Forapositive definite symmetric matrixE7b
Λ+=Σ+=Λr10r,d0r,d0d,d.

Since Nu,uis a positive definite symmetric matrix in the LS, S=U=V. If there is no defect in a matrix N, N1=N+. Therefore, we can use pseudoinverse safely in any given problem [8, 9, 13, 14, 15].

### 2.3. Hard computing

Linearizing from nonlinear functions to their linear form by means of Taylor expansion, a linear equation system is to be handled as Eq. (1). To avoid complicated proofs in the solution of an equation system, the simplified mathematical model can be written in the following (statically rotation invariant ) numerical computation form.

An,uδu=ln,P=I.E8

Meet two states to solve Eq. (8), nuand n>u. The solution for the former state nuis achieved by means of auxiliary variables vector λnwhich can be defined as δu=Au,nTλn. In fact, the auxiliary vector λnis named as a Lagrange multipliers vector or an eigenvalues vector in a homogenous equations system in which l=0for Eq. (8) . Putting back δ=ATλinto Eq. (8), we compute λfirst:

λ̂n=Qn,nln,Qn,n=AAT+.E9

And then δand its variance–covariance matrix if we know the statistical uncertainty of observations (Σl=Σy)are calculated by Eq. (10) and the low error propagation, respectively. We can only calculate the variance–covariance matrix of estimations as in Eq. (10) due to σ̂02=0and taking Σy=Iin the chapter.

δ̂=ATλ̂=ATQl,Σδ̂=σ02ATQQA,E10a
x̂=x+δ̂,Σx̂=Σδ̂,E10b
x̂Tx̂min.E10c

In the state (nu), Aδ̂l=ε̂=0should be provided. If not, continue solution until maxδ̂<=thres=5e12(or maxε̂<=thres=5e12) by taking x=x̂in every iteration step. x̂Tx̂will be the smallest at end of the solution.

Solution to the second state n>uis a situation encountered in many scientific and engineering problems. Multiplying both sides of Eq. (8) by Au,nTthe normal equation system is established and solved with Eq. (11):

δ̂u=QATl,Qu,u=ATA+,E11a
x̂=x+δ̂,Σx̂=σ̂02Q,E11b
σ̂02=ε̂Tε̂nr,Aposteriori variancer=rankAE11c
ε̂=Aδ̂l,E11d
ε̂Tε̂min.L2norm estimationLeast SquareE11e

End the solution if the condition ensured is maxδ̂<=thres=5e12; otherwise, continue the iteration with x=x̂.

Relationships between nonlinearity and LS in a multidimensional surface have been shown by Teunissen et al. [1, 2]. The authors argued the relation on some simple examples and gave some analytical solutions for them. But, they highlighted that those types of analytical solutions have not been given for every problem and emphasized that suitable Taylor expansions have been useful to the solution not being transformed into the analytical ones.

## 3. Geometry of a combination of polynomial and trigonometric functions

These type functions can be used in defining the orbits of artificial satellites (and celestial bodies). Also, the numerical example part of this chapter, to estimate those type functions, will be inspected and applied on a real example. To foresee a model for any problem we should interpret the model parameter and comprehend the geometry of the model (Figure 1). Figure 1.The geometry of a first-degree and first-order combination of polynomial and trigonometric (CPT) function.

With respect to independent variable time t, a combination function of p=1degree polynomial and order q=1trigonometric function(s) [a combination of polynomial degree and trigonometric order (CPT)] to be estimated in the chapter is:

ϕj=aϕ+bϕtj+cϕsindϕ+eϕtj,E12
ϕjXjYjZjSj,j12n.

where tjϕjare data given. In Eq. (12), translation aϕand slope bϕare elements of a line equation which is a first-order polynomial of CPT function. The other model parameters in the trigonometric part of Eq. (12) are defined as an amplitude cϕ, and an initial phase dϕand a frequency (or angular velocity) eϕ=2π/Tϕ(a period Tϕ) of a wave (Figure 1).

In this chapter, the functions ϕjare the coordinate components XjYjZjincoming from a precise orbit file and the geometric distances Sj=Xj2+Zj2+Zj2as a function of the components. However, nonperiodic earth-fixed coordinates (GR) in the SP3 file should be transformed to the periodic space-fixed coordinates (Υ); why is Eq. (12) is suitable for the space-fixed coordinates, not earth-fixed ones (as seen from Figure 3 in the numerical example part) (Figure 2)? Figure 2.Earth (GR) and space-fixed (Υ) coordinates for an artificial satellite.

For this propose, an easy transformation into any epoch (e.g., it can be taken as the first epoch t0of the data) is carried out by:

Xγ,j=R3θjxGR,j,θj=wEtj,E13a
xGR,j=R3θjXγ,j,R3θj=R3Tθj=R31θj,E13b
R3θj=cosθjsinθj0sinθjcosθj0001,Xγ,j=XjYjZjγ,xGR,j=xjyjzjGR.

where wEand R3are in order of the angular velocity of earth and well-known orthogonal rotation matrix around the third axis (Figure 2).

A solution of nonlinear Eq. (12) is realized in the following order. Linearizing Eq. (12) by Taylor expansion and omitting the terms greater than or equal to quadratic ones, the linear equation system as given by Eq. (8) is obtained. The explicit form of the Eq. (8) with respect to the approximate values of unknowns for a CPT is:

Aj=1tjsind0+f0tjc0cosd0+e0tjtjc0cosd0+e0tj,E14a
lj=ϕja0+b0tj+c0sind0+e0tj,j12n.E14b

We can use a recursive solution for Eq. (14) instead of the batch solution as Eq. (11) because of its solution velocity.

δ̂=Qj=1nAjTlj,Q=j=1nAjTAj1=j=1nAjTAj+.E15

Continuation of the solution of Eq. (15) can be performed according to Eq. (11). The model given by Eq. (12) is a simple model to determine the satellite orbit motions. For more complicated models, the readers can utilize [19, 20, 21, 22, 23, 24, 25] resources.

## 4. Numerical example

For a nonlinear estimation of CPT functions, some numerical examples are chosen from GNSS {Global navigation satellite systems = GPS (USA) + GLONASS (RU), GALILEO (EU), COMPASS (CHN)} artificial satellite orbits whose coordinates are downloaded from the internet address ftp://ftp.glonass-iac.ru/MCC/PRODUCTS/17091/final/Sta19426.sp3 .

For this purpose, two estimation software have been developed in 64Bit Python (in accordance with the 2.7 and 3.6 version) and 32Bit C++ (Code::Blocks) environments to see the convergence rate of the mathematical model given in Eq. (12) [27, 28]. The computed elements of CPT functions for the selected four satellites R01 (GLONASS), G03 (GPS), E01 (GALILEO) and C06 (COMPASS) are summarized in Table 1 in which they are ordered from the nearest satellite to the farthest one.

The motions of the CPT functions estimated satellites (in Table 1) with respect to earth- (left column of Figure 3) and space-fixed (right column of Figure 3) coordinate systems are demonstrated in Figure 3. Moreover, coherence between the estimated CPT function (black solid line) of the C06 satellite and its data points given (colorful circles) is represented in detail in Figure 4. Figure 3.Earth (left) and space (right) fixed orbital traces (see the appendix) with time tags of R01, G03, E02, C06 satellites and the motion of X-coordinate axis shown as GR (XGR(t0) position on the intersection Greenwich meridian and equator) symbol att0(=2017 April 01 00:00:00). Figure 4.Temporal changing of space fixed coordinates of C06. The circles and solid lines represent the data points and estimated functions under LS respectively for X (red), Y (green), Z (blue), S (cyan).

We know that accuracy of precise SP3 file coordinates is about σ0=±5cm. If we compare the value with its estimations given in Table 1, we can say that our predicted model is not meet our demands. We should expand the model by raising the degree of polynomial part or/and order of trigonometric part of CPT functions. In fact, we can readily see that the projected model with Eq. (12) will never cover the data. The model is only chosen for this chapter. The more suitable model established on Keplerian orbital elements can be found in the orbit determination literature and in [18, 19, 20].

Comparing the solution velocities (from the iteration numbers with respect to 5e12threshold in Table 1) in different platforms, we can say that the solution velocities in 64Bit Python are generally better then 32Bit ones.

If we chose the threshold as 51e-13, we can see the distinctions of solution convergences between 64Bit and 32Bit running on the estimations of C06 satellite from 1000 (X), 8 (Y), 7 (Z), 1000 (S) in 32Bit C++ and 10 (X), 8 (Y), 6 (Z), 28 (S) in 64Bit Python in Windows. In here, 1000 is the maximum iteration number. If the mathematical model would be more complicated and its data number would be bigger than the number used in the example part, we would see the state more prominently.

Approximated values and the loop element for the unknowns are computed following the order in all solutions of the satellites when running in 32Bit C++ and 64Bit Python platforms for the estimations in Table 1.

a0=0.0,b0=0.0,c0=maxϕj,j12n=96
d0=arcsinϕ1/c0,e0=arcsinϕ2/c0arcsinϕ1/c0/t2t1
x=a0b0c0d0e0T

Maximum iteration number and threshold loop elements are iterMAX=1000and thres=5e12to break the iteration loop.

Taking the approximated values as x=00thres00T0Tand the same loop elements given above, the iteration numbers are handled as 138 (X), 178 (Y), 78 (Z), 16 (S) in 64Bit Python. For x=00c000Tthey are 33 (X), 235 (Y), 96 (Z), 11 (S) in 64Bit Python. Different approximated value selections cause different iteration numbers (namely convergence rate).

### 4.1. An expanded model example by an auxiliary cosine wave

Since the estimated standard deviations σ̂0= {±56.552, ±38.018, ±51.901, ±0.491} (for X, Y, Z, S in Table 1) of the CPT functions for the coordinates of the C06 satellite are not statically equal to their expected values (σ0=±5cm), the CPT model should be expanded. As an example, three more unknowns are added to the model given in Eq. (12)

ϕj=aϕ+bϕtj+cϕsindϕ+eϕtj+fϕcosgϕ+hϕtj

The added terms represent an amplitude fϕ, an initial phase gϕand a frequency hϕof a new wave carried by first (sine) wave. After the first estimation with respect to Eq. (12),

we can choose the approximate values of the new parameters as

f0=maxabsε̂
g0=arcsinε̂1/f0
h0=arcsinε̂2/f0arcsinε̂1/f0/t2t1

from ε̂j=ϕjâ+b̂tj+ĉsind̂+êtj, j12n=96. The approximate value vector of the expanded model by a new wave is:

x=a0b0c0d0e0f0g0h0T=âb̂ĉd̂êf0g0h0T

The approximate values are substituted in the following linearized model as initial values for the loop in LS estimation.

lj=ϕja0+b0tj+c0sind0+f0tj+e0cosh0+g0tj
AjT=1tjsind0+e0tjc0cosd0+e0tjtjc0cosd0+e0tjcosg0+h0tjf0sing0+h0tjtjf0sing0+h0tj

After the evaluation, the improved solution for the C06 satellite is represented in Table 2. We can readily see the improvements upon the downs of the standard deviations from σ̂0={±56.552, ±38.018, ±51.901, ±0.491} (Table 1) into σ̂0={±0.178, ±0.137, ±0.191, ±0.003} (Table 2) in kilometers for the C06 satellite. We can develop the last model more by means of the same manner if we want.

Sat.φXYZUnitS
iter
iter++
5
5
4
4
4
5

10
9
R01
(RU)
aφ
bφ
cφ
dφ
eφ
Tφ
−11.860
0.028
25,474.503
−60°54′04.37″
31°57′53.92″
11h05’44.37”
2.420
0.003
11,178.453
−23°03′56.93″
−31°57′44.92″
11h15’47.54”
2.771
0.052
22,965.361
−30°31′00.32″
−31°57′56.31″
11h15’43.53”
km
km/h
km
deg
deg/h
h
25,508.091
0.001
8.127
42°41′35.87″
−31°58′13.55″
11h15’37.46”
±σ̂02.9021.2662.653km0.211
iter
iter++
6
6
4
4
4
4

30
34
G03
(USA)
aφ
bφ
cφ
dφ
eφ
Tφ
−17.545
−0.132
24,800.602
66°34′39.73”
30°04′56.03”
11h58’01.91”
7.332
−0.045
17,959.166
−48°06′43.68”
30°04′52.50”
11h58’3.31”
−1.824
−0.085
21,757.547
−7°51′47.16”
30°05′00.02”
11h58’00.32”
km
km/h
km
deg.
deg./h
h
26,561.324
−0.003
14.211
−72°24′19.52”
−30°08′11.17”
11h56’44.42”
±σ̂04.8983.6483.961km0.183
iter
iter++
4
5
5
5
4
4

32
93
E01
(EU)
aφ
bφ
cφ
dφ
eφ
Tφ
1.866
0.052
21,349.011
−35°22′15.46″
−25°34′13.06”
14h04’43.81”
1.358
−0.048
26,031.918
85°57′22.67”
−25°34′14.87”
14h04’42.81”
−4.673
0.004
24,878.432
−16°23′29.47”
25°34′18.57”
14h04’40.78”
km
km/h
km
deg.
deg./h
h
29,600.332
−0.000
3.720
8°29′58.23”
−25°42′10.68”
14h00’22.19”
±σ̂01.7961.4281.058km0.236
iter
iter++
7
10
6
7
6
7

28
29
C06
(CHN)
aφ
bφ
cφ
dφ
eφ
Tφ
−407.495
61.945
41,716.806
−24°21′36.29”
−15°07′02.75”
23h48’48.86”
228.037
−6.376
24,832.649
59°24′50.73″
−15°02′16.41”
23h56’22.30”
266.178
−7.892
34,235.695
67°36′9.42″
−15°02′36.74”
23h55’49.94”
km
km/h
km
deg.
deg./h
h
42,175.353
−0.358
227.599
18°32′27.85”
−14°57′17.11”
24h04’21.41”
±σ̂056.55238.01851.901km0.491

### Table 1.

Computed elements of the CPT functions for G03, R01, E01, C06 satellites by IterMAX = 1000 and thres = 5e-12 in loops {iteration numbers of 64Bit Python and 32Bit C++ software in windows are denoted as iterand as iter++respectively}.

Sat.φXYZUnitS
iter271585319
C06
(CHN)
aφ
bφ
cφ
dφ
eφ
fφ
gφ
hφ
223.514
1.515
42,064.675
−25°07′54.78″
−15°02′24.75”
110.710
5°08′57.82”
30°10′58.43”
149.296
0.224
66.183
11°15′24.23″
30°04′30.44″
24,808.034
30°28′55.61″
15°02′17.24”
182.012
0.060
91.527
3°15′58.76″
30°04′02.09″
34,217.994
−22°21′14.60″
−15°02′19.27”
km
km/h
km
deg
deg/h
km
deg.
deg./h
42,169.956
0.055
225.161
19°47′37.54″
−15°02′43.26″
0.838
8°56′45.50″
−29°40′58.52”
±σ̂00.1780.1370.191km0.003

### Table 2.

The results by the expanded model for the C06 satellite in Python 3.6 in windows.

As another example, the expanded model has been trained on the coordinate components of R01 GLONASS satellite by means of Python 3.6 software on Windows. We can also see the improvements upon the downs of the standard deviations (its iteration numbers) from σ̂0={±2.902 (5), ±1.266 (4), ±2.653 (4), ±0.211 (10)} (Table 1) to σ̂0={±0.226 (76), ±0.448 (32), ±0.201 (43), ±0.037 (57)} in kilometers for the R01 satellite.

Condition numbers computed from a rate of maximum and minimum eigenvalues or a rate of singular values under LS are very effective tools for determining the consistency as well. Therefore, a larger condition number can cause larger iteration number (related to convergence rate), We can see those states from the CPT estimations of the C06 satellite with the iteration (iter) and condition numbers (cond). These are given for X, Y, Z, S as iter = {7, 6, 6,28}and cond = {3.4e + 13, 2.0e + 12, 2.8e + 12,1.3e + 09}(Table 1), and as iter = {27, 158, 53,19}and cond = {9.5e + 14, 2.3e + 13, 3.0e + 13,2.6e + 10}(Table 2).

## 5. Conclusions

In this chapter, the least squares (LS) estimations of the artificial satellite orbital movements by a combination of polynomial and trigonometric (CPT) functions have been given after a general overview has been made on the hard and soft computations. In practice, the orbital motions are modeled on Keplerian orbital elements. In contrary to this, the coordinate components have been selected for this chapter due to the nonlinear relations of the components and the unknowns which are the elements of CPT functions. The relations cause inconsistencies in the LS solutions. The inconsistencies result from the two injectivity defects, c-defectsand i-defects. We can readily see the defects from the differences of the convergence rates (in other words the iteration numbers) in different computer platforms and architectures as shown in the chapter. The defects are not fully removed as long as not change the mathematic models. However, we can surpass the effects of those defects in part by means of the pseudoinverse based on the eigendecomposition or the singular value decomposition (SVD) as in here. The surjectivity defect (ds) of the CPT functions not including the datum defects (d-defects) was eliminated by the LS objective function.

For the sake of simplicity for readers, a simple CPT function has been chosen at first. After the initial estimation of the function, the estimated errors vector has been found. We have seen that the errors have had a periodic characteristic in time. So, a new wave defining the error characteristic and been able to carry by the first wave has been planned for expanding the CPTs. It is shown that we can expand a CPT function until ensuring statically equivalency between a priory and a posteriori variances. For instance, one may secure the equivalency of the variances if one would expand more by a new wave in the last estimated model in the same manner.

The convergence rates (upon the iteration numbers) of the LS estimation have been inspected according to the threshold (thres = 5e-12) which is a good value for the estimation of the nonlinear CPT function. An algorithm compiled by different compilers and run in different architectures (with 32 Bit or 64 Bit) changes the convergence rate of the estimations in such as the inconsistent scientific problems. It is also observed that the iteration numbers change when the 64-bit Python software is run on Linux platform which has a different framework than Windows. But, the numbers have not been given in the example part of the chapter. Contrary to inconsistency model, namely in a consistent one, the iteration numbers can take equivalence values in all circumstances. Another way to determine the inconsistency of a model is to obtain its condition number which is computed from a rate of maximum and minimum eigenvalues or of singular values under LS. If the condition number is close to one, the projected model is accepted as a consistent model.

We can use the Soft Computing Methods (SCM) if not an exact mathematical relationship between the data and unknowns. The mathematical model is established by the trial-and-error method in training part of SCM by means of arbitrary weights and activation functions depending on SCM expert forecasts. For the solution of the SCM model during the training, we can use least absolute residuals (LAR) and minmax absolute residuals (MAR) objective functions by the linear programming or the LS estimation as in hard computing method (HCM). In the state, the inconsistency problem can erase whatever the solution method (LAR, MAR or LS) is. The inconstancy can be removed by means of experiences gained from HCMs.

Prior information is very important to select a suitable mathematical model for a scientific problem. For example, comparing a priori variance with a posteriori variance at the end of the estimation is a useful warning to the user to determine the correct mathematical model as seen from the expanded model in the example section of the chapter.

In numerical computation, there are two main phenomena which are the mathematical model (as a combination of functional and stochastic models) and objective function. The solution strategy is of no importance if the same mathematical model and objective function are preferred in the same problem of hard computing. All solution strategies always give same results, only their solution time spans can be distinct from each other (Table 3).

## Acknowledgments

I wish to thank TÜRKSAT A.S(https://www.turksat.com.tr/en) supporting this study.

## Appendix

Epoch(j)tj−t0 [h]Xj [km]Yj [km]Zj [km]tj [μsec]
00.00−17531.30750621541.79205431834.209680691.175390
10.25−19992.14790020678.91348530924.464057691.487182
20.50−22367.33339519727.43736529882.214843691.798804
30.75−24646.58975418691.35330028711.797065692.110527
41.00−26820.03209817575.02117027418.102912692.422088
51.25−28878.20847516383.15400726006.563346692.733923
61.50−30812.14216615120.79931124483.127129693.045689
71.75−32613.37231313793.31878322854.237453693.357416
82.00−34273.99267012406.36650621126.806228693.668853
92.25−35786.68826010965.86567619308.186097693.980453
102.50−37144.7697539477.98394717406.140275694.291853
112.75−38342.2053447949.10747115428.810302694.603337
123.00−39373.6499646385.81375513384.681845694.914789
133.25−40234.4716244794.84342511282.548651695.226157
143.50−40920.7747213183.0710229131.474825695.537828
153.75−41429.4201601557.4749716940.755568695.849062
164.00−41758.042128−74.8931704719.876568696.160359
174.25−41905.061383−1706.9400062478.472204696.471627
184.50−41869.694990−3331.562017226.282782696.782836
194.75−41651.962330−4941.677516−2026.889008697.094239
205.00−41252.687387−6530.258698−4271.222183697.405373
215.25−40673.497209−8090.363611−6496.921900697.716659
225.50−39916.816531−9615.167884−8694.263981698.028206
235.75−38985.858544−11097.996030−10853.639319698.339619
246.00−37884.611845−12532.352178−12965.597898698.651055
256.25−36617.823589−13911.950035−15020.892223698.962566
266.50−35190.978917−15230.741942−17010.519896699.273893
276.75−33610.276761−16482.946839−18925.765115699.585363
287.00−31882.602129−17663.077000−20758.238873699.896674
297.25−30015.495009−18765.963379−22499.917620700.208082
307.50−28017.116067−19786.779434−24143.180195700.519378
317.75−25896.209299−20721.063292−25680.842806700.830889
328.00−23662.061860−21564.738140−27106.191889701.142117
338.25−21324.461276−22314.130731−28413.014646701.453707
348.50−18893.650283−22965.987908−29595.627135701.764972
358.75−16380.279535−23517.491071−30648.899728702.076478
369.00−13795.358449−23966.268499−31568.279851702.387737
379.25−11150.204459−24310.405495−32349.811865702.699185
389.50−8456.390949−24548.452300−32990.154024703.010540
399.75−5725.694161−24679.429736−33486.592421703.321863
4010.00−2970.039352−24702.832610−33837.051887703.633117
4110.25−201.446497−24618.630817−34040.103800703.944417
4210.502568.024186−24427.268222−34094.970801704.255873
4310.755326.326597−24129.659289−34001.528424704.567076
4411.008061.482740−23727.183555−33760.303648704.878433
4511.2510761.636054−23221.677952−33372.470443705.189684
4611.5013415.103835−22615.427078−32839.842340705.501070
4711.7516010.428511−21911.151470−32164.862120705.812547
4812.0018536.427516−21111.993965−31350.588697706.124190
4912.2520982.241560−20221.504257−30400.681303706.435990
5012.5023337.381077−19243.621724−29319.381091706.747258
5112.7525591.770667−18182.656653−28111.490271707.058650
5213.0027735.791345−17043.269964−26782.348927707.370038
5313.2529760.320445−15830.451549−25337.809640707.681516
5413.5031656.769034−14549.497344−23784.210083707.992915
5513.7533417.116707−13205.985271−22128.343727708.304435
5614.0035033.943650−11805.750146−20377.428827708.615955
5714.2536500.459878−10354.857705−18539.075847708.927258
5814.5037810.531573−8859.577863−16621.253481709.238773
5914.7538958.704447−7326.357314−14632.253449709.549980
6015.0039940.224086−5761.791632−12580.654219709.861492
6115.2540751.053248−4172.596961−10475.283826710.172943
6215.5041387.886081−2565.581420−8325.181952710.484439
6315.7541848.159196−947.616367−6139.561422710.796071
6416.0042130.059795674.392412−3927.769279711.107495
6516.2542232.5306762293.533436−1699.247596711.419275
6616.5042155.2721323902.918094536.505839711.730566
6716.7541898.7408945495.7089472769.976807712.042086
6817.0041464.1461417065.1476544991.673762712.353663
6917.2540853.4424328604.5824247192.166646712.665021
7017.5040069.31993810107.4949279362.125250712.976412
7117.7539115.19184011567.52654811492.356997713.287987
7218.0037995.17900612978.50391513573.844046713.599417
7318.2536714.09201614334.46362115597.779572713.910828
7418.5035277.41060815629.67605817555.603122714.221952
7518.7533691.26066516858.66829919439.034930714.533584
7619.0031962.38879618016.24594221240.109087714.845281
7719.2530098.13463119097.51387522951.205467715.156764
7819.5028106.40090720097.89587424565.080300715.468390
7919.7525995.62147421013.15299326074.895308715.779824
8020.0023774.72729921839.40067127474.245306716.091334
8120.2521453.11059122573.12452128757.184187716.402951
8220.5019040.58716623211.19473229918.249189716.714457
8320.7516547.35715023750.87904630952.483389717.026029
8421.0013983.96416024189.85425331855.456321717.337474
8521.2511361.25307524526.21617532623.282652717.649172
8621.508690.32654424758.48807433252.638847717.960633
8721.755982.50033424885.62747533740.777748718.272113
8822.003249.25769824907.03134234085.541002718.583548
8922.25502.20287124822.53958734285.369282718.895203
9022.50−2246.98612624632.43688434339.310236719.206528
9122.75−4986.60523924337.45275834247.024111719.518326
9223.00−7704.97252223938.75992134008.787012719.830016
9323.25−10390.47649123437.97086633625.491749720.141620
9423.50−13031.62453922837.13266533098.646226720.452998
9523.75−15617.09119422138.71999832430.369356720.764247

### Table 3.

Space Fixed Coordinates of C06 inclined geostationary earth orbit in COMPASS (which is Chinese Global Positioning Satellite System) are transformed with respect to t0from earth fixed coordinates downloaded from ftp://ftp.glonass-iac.ru/MCC/PRODUCTS/17091/final/Sta19426.sp3  {t0 = 2017.04.01–00:00:00 (Civil Calendar) = 1942–518,400 (GPS week—week seconds)}.

chapter PDF
Citations in RIS format
Citations in bibtex format

## More

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

## How to cite and reference

### Cite this chapter Copy to clipboard

Orhan Kurt (November 5th 2018). On Non-Linearity and Convergence in Non-Linear Least Squares, Optimization Algorithms - Examples, Jan Valdman, IntechOpen, DOI: 10.5772/intechopen.76313. Available from:

### Related Content

#### Optimization Algorithms

Edited by Jan Valdman

Next chapter

#### A Gradient Multiobjective Particle Swarm Optimization

By Hong-Gui Han, Lu Zhang and Jun-Fei Qiao

#### Applications from Engineering with MATLAB Concepts

Edited by Jan Valdman

First chapter

#### Digital Image Processing with MATLAB

By Mahmut Sinecen

We are IntechOpen, the world's leading publisher of Open Access books. Built by scientists, for scientists. Our readership spans scientists, professors, researchers, librarians, and students, as well as business professionals. We share our knowledge and peer-reveiwed research papers with libraries, scientific and engineering societies, and also work with corporate R&D departments and government entities.