Introductory Chapter: Optimization Problems in Engineering

1. Introduction

Engineers are devoted to solving real-life problems. The solutions to these problems require the minimization (or maximization) of a function (usually called an objective function or cost function). The problem might have some constraints, turning the problem solving to a challenging task. The current set of chapters deals with different applications: supersonic flutter, motion estimation, chemical and environmental processes, complex nonlinear systems, cutting and packing, topology optimization, curve interpolation, etc. Despite the different domains for each application, they have some essential features in common. They have some parameters representing the solution, the parameters are the inputs to the objective function. The objective function must be minimized through the optimization process to eventually maximize the reliability and efficiency of the resulting solution [1].

Two major categories of methods can be used to determine the set of parameters which supplies the minimal (or maximal) value for an objective. The first category relies on deterministic methods [2]. They require a seed solution (also called initial solution) and iteratively determine a solution. Once the initial solution is given, the deterministic method will end up always the same final solution. However, it might be not the global optimum. The second category relies on probabilistic methods (also called metaheuristics) [3]. They do not rely on the initial solution and each execution can reach a different solution.

In the sequence, it explains the deterministic and probabilistic methods. It will be shown that the different methods have different properties. Surely, each problem has the most appropriate method. However, this association might not be an easy task.

2. Deterministic methods

Among the deterministic methods, the Gauss-Newton method is one of the most commonly used methods and it can be formulated as

$$\underset{x}{min}\parallel f\left(x\right){\parallel }_2^2,$$E1

where some initial solution $x_0$ must be provided. The next solution is iteratively determined by

$$x_{k+1}=x_k-\frac{\nabla f\left(x_k\right)}{{\nabla }^{2}f\left(x_k\right)}\cdot {\alpha }_k,$$E2

where ${\alpha }_k$ is the step size, $\nabla f\left(x_k\right)$ denotes the gradient and ${\nabla }^{2}f\left(x_k\right)$ is the Hessian, both at $x_k$. This iteration can happen only if the Hessian is invertible at $x_k$.

The definition of the step size is not trivial, some approaches, like the Armijo rule, using nonmonotone line search were proposed [4]. A very well-known approach uses the Levenberg-Marquardt modification. It can be used even if ${\nabla }^{2}f\left(x_k\right)$ is not a descent direction and if ${\nabla }^{2}f\left(x_k\right)$ is singular. The Levenberg-Marquardt was applied in different applications [5, 6]. These two methods have continuous parameters. In case the parameters are integers, or just a subset of the parameters are integers, another class of methods should be used: integer or mixed-integer programming [7]. The determination of the next solution is a key step in deterministic methods. The current solution is summed by a vector defined by the sensitivity of the method, guiding the algorithm towards the convergence.

This set of chapters has several deterministic methods, in the following they are briefly presented. The chapter “Verification and validation of supersonic flutter of the rudder model for experiment” uses a deterministic approach combined with a gradient evaluation to determine the next solution. The parameters are exclusively continuous. The problem has some constraints related to the first and second frequencies of the rudder.

The chapter “An optimization procedure of the model’s base construction in multimodel representation of complex nonlinear systems” uses a deterministic method. Artificial Neural Networks must be trained before usage. During the training phase, their parameters are determined using some cost function and gradient information. After trained, the artificial neural network will provide the same output for a given input.

The chapter “Approximation algorithm for scheduling a chain of tasks for motion estimation on heterogeneous systems MPSoC” uses a deterministic approach. This research is focused on processing speed. As explained by the authors, motion estimation is a key feature in a video codec. Temporal redundancy can be estimated and used in interframe video compression. The authors explored a parallel approach suitable for embedded platforms.

The chapter “Analytical solutions of some strong oscillators” presents an analytical solution for cubic and quintic Diffing’s oscillators. The authors consider arbitrary initial solutions. The authors point out that the Duffing equation has a large area of application: mechanical engineering, electrical engineering, plasma physics, etc.

The chapter “Optimal heat distribution using asymptotic analysis techniques” uses a deterministic method. The next solution is determined using the concept of topological derivative. The topological derivative defines a vector that iteratively defines the path to the final solution (optimal heat distribution).

3. Metaheuristics

Metaheuristics are a wide range of methods, usually inspired from nature. Every metaheuristic has a random component and this is why they are also known as probabilistic methods. The chapter “Optimization multicriteria scheduling criteria through analytical hierarchy process and lexicography goal programming modeling” is the first example that uses a probabilistic method. Multicriteria decision analysis is a research area where multiple possibilities are considered to determine the best choices. It creates an aleatory matrix of judgments which will determine the coherence among the criteria. The authors present an example about electric cable cut. The following criteria were considered: qualification of workers, safety knowledge, equipment performance, scheduling equipment, production technology, and cost of upgrading waste.

Some metaheuristics used in this set of chapters are briefly introduced: simulated annealing, tabu search, and genetic algorithms.

3.1 Simulated annealing

Simulated annealing is based on metal annealing [8]. It was originally proposed to solve combinatorial problems. However, several proposals to incorporate continuous parameters soon appeared [9, 10, 11]. Simulated annealing has clearly two different phases: exploratory and refinement [12]. Initially, at higher temperature, the domain is explored. At lower temperatures, the solution is refined and convergence happens. The simulated annealing has only one current solution.

The simulated annealing with crystallization factor described in the chapter “Versatility of simulated annealing with crystallization heuristic: its application to a great assortment of problems” is an example of how continuous parameters can be approached. The crystallization factor supplies a sensitivity to the algorithm. It was successfully applied to irregular packing problems with fixed containers, in this situation, the cost function is discrete and some parameters are continuous [13, 14, 15]. This chapter shows different applications, the curve interpolation application has integer and continuous parameters [16, 17, 18, 19, 20].

The objective function of some problems might be costly computationally. For example, when it is required to solve a finite element problem. The simulated annealing can be modified to incorporate a partial evaluation of the objective function. The objective function returns an interval which surely contains the function’s precise value. This approach was successfully applied to the electrical impedance tomography problem where 32 finite element problems were solved every time to evaluate the objective function [21, 22, 23, 24, 25, 26, 27, 28].

Several books have been published about simulated annealing [29, 30]. Simulated annealing has also solved multi-objective problems [31]. A well known simulated annealing for multi-objective is AMOSA [32, 33]. The simulated annealing with crystallization factor has also solved multi-objective problems [34, 35, 36].

The simulated annealing with crystallization heuristic has shown to be very versatile, regarding the type of parameters (integer, continuous, combinatorial) and the type of objective function (continuous, discrete, intervalar, and multiple).

3.2 Tabu search

Tabu search was originally proposed to solve combinatorial problems (like simulated annealing). Tabu search can be viewed as a special case of simulated annealing. However, not with all possibilities available to the simulated annealing. Tabu search uses a local search and it has a structure storing visited solutions. One already visited solution is avoided. The word Tabu means that something cannot be touched, in the sense that already visited solutions must not be visited again. Three types of memories are commonly used to store the visited solutions: short-term, intermediate-term, and long-term [37]. The tabu search, like the simulated annealing, has only one current solution.

The chapter “A methaheuristic tabu search optimization algorithm: applications to chemical and environmental process” shows that tabu search can be used in a wide spectrum of engineering problems.

3.3 Genetic algorithms

Genetic algorithms are based on natural selection and they create a population of solutions. Based on this population of solutions, a new population is created using some operators: cross-over and mutation [38]. Genetic algorithm has a strong exploratory feature. Currently, it can be used with integer and continuous parameters. However, it does not have any convergence criteria. After a certain number of iterations, the best so far solution is defined as the converged solution. This weakness motivates one chapter in this set of chapters.

The chapter “Mixed-discrete nonlinear programming engineering problems” combines genetic algorithms with sequential quadratic programming. The sequential quadratic programming is a well-known gradient-based search algorithm.

4. Conclusions

As described, there are many types of engineering problems and this is the reason for the great number of methods developed to solve them. In this set of chapters, we intend to enumerate a few of them, showing their pros and cons and discuss the types of their parameters, type of the objective functions, computational cost, implementation facility, etc.

Acknowledgments

M. S. G. Tsuzuki is partially supported by CNPq (process 311.195/2019-9). A. K. Sato is supported by FUSP/Petrobras.