Summary of neural network manufacturing applications.
Neural network (NN) model is an efficient and accurate tool for simulating manufacturing processes. Various authors adopted artificial neural networks (ANNs) to optimize multiresponse parameters in manufacturing processes. In most cases the adoption of ANN allows to predict the mechanical proprieties of processed products on the basis of given technological parameters. Therefore the implementation of ANN is hugely beneficial in industrial applications in order to save cost and material resources. In this chapter, following an introduction on the application of the ANN to the manufacturing process, it will be described an important study that has been published on international journals and that has investigated the use of the ANNs for the monitoring, controlling and optimization of the process. Experimental observations were collected in order to train the network and establish numerical relationships between process-related factors and mechanical features of the welded joints. Finally, an evaluation of time-costs parameters of the process, using the control of the ANN model, is conducted in order to identify the costs and the benefits of the prediction model adopted.
- control and monitoring of manufacturing processes
- simulation technologies
The use of artificial intelligence and specifically artificial neural networks (ANNs) has allowed yielding revolutionary advances in manufacturing. However, most of the applications of artificial intelligence in the production field concerned expert systems and fewer attentions were paid to neural networks (NNs). Most important characteristics of the ANNs are:
the self-adaptive behaviour that allows to adapt the forecast to changing of the environment, in this way improve the networks’ ability to learn and to predict;
the parallel computing architecture, that has a great impact in multiple disciplines and applications, from speech and natural language processing, to image processing or problems in bioinformatics and biomedical engineering.
Therefore, they could be of great help for today’s computer integrated manufacturing and in smart factories, according to Industry 4.0 paradigm. Currently, the nature of the manufacturing process is changing with great speed, becoming more sophisticated and continuous variations are occurring due to changes in customer demand and reduced product life cycle. This requires manufacturing technologies that can easily adapt to such changes. In this context, artificial neural networks are a powerful technology to solve this problem. The use of ANNs is also widely used for process monitoring and control applications. The quality of a process can only be provided by in process monitoring through proper measurements. To ensure a high quality of a process, you must follow the following technological steps :
Identify the characteristic changes of a process;
To estimate the changes of the product quality;
Correct any process operations as a result of any anomalies detected from the comparison between the obtained and the desired quality.
These steps should be followed with minimal supervision and assistance from operators, if possible in unmanned manner. In addition, all processes should be implemented with special features such as: storing information, decision making, learning and integration. It should be noted that most manufacturing processes are regulated by many variable parameters and for this reason, such systems have a random, complex and uncertain nature.
This may be attributable to the fact that they are exposed to external disturbance and noise and often subjected to parameter variations. Furthermore, there is often a great interaction between variables and therefore it is not possible to properly define the final quality of the product and the variables that influence it. Due to these characteristics the quality often varies from product to product, impairing its uniformity and decreasing the yield of the product. So, all changes that may occur in manufacturing environments cannot be easily observed by an operator, so in recent years the use of neural networks applied to process monitoring and control has been of great interest. Indeed, it has been shown that the use of artificial intelligence can overcome the above-mentioned problems . The research efforts in this direction will be accelerated with greater interest in the future and will lead to the development of truly intelligent manufacturing systems that are capable of producing products without the supervision or assistance of human operators . Figure 1 classifies the functionalities needed to imbed the artificial neural networks on manufacturing processes and summarizes the current developments in manufacturing application areas.
The real world applications here in manufacturing include the modelling, monitoring and control, identification, planning and scheduling associated with the processes. The purpose of this chapter is to present some applications of artificial neural networks in manufacturing process monitoring and control, among which particular attention will be paid to the study that has been published in international journals and that has investigated the use of the ANNs for the monitoring, control and optimization of a welding processes. Experimental observations were collected in order to train the network and establish numerical relationships between process-related factors and mechanical properties of the welded joints. Finally, an evaluation of the time-cost parameters of the process, using the control of the ANN model, is conducted in order to identify the costs and benefits of the prediction model adopted.
2. Manufacturing applications
Due to many external disturbances and many variations in process parameters, many production processes are complex and time-consuming. For these reasons, it is not always possible to identify the relationship between the product quality and the input variables of the process. Thus, there is interest to integrate the artificial intelligence into the production processes for storing, learning, reasoning and decision making. Such systems are able to adapt to changes in its environment and can truly realize unmanned operations of processes. The adoption of the neural network can be devoted to monitoring and to prediction of different parameters in many industrial areas, in order to solve issue relating to the manufacturing system design, process planning, as well as operational decision making. A summary of main NN applications field is shown in Table 1 .
|Application topic||• Manufacturing system design||Author Ref. num.||Moon and Chi |
Kaparthi and Suresh 
Lee et al. 
|• Manufacturing process control||Cook and Shannon |
|• Robot scheduling||Yih et al. |
|• Manufacturing operational decision||Chryssolouris et al. |
Manufacturing information, such as the sequence of operations, lot size; multiple process plans were given special consideration in their approach to solve the generalized part family formation problem. Many authors also point out that the method of artificial neural networks is flexible and can be efficiently integrated with other manufacturing functions. Below, from the literature, some important artificial intelligence applications to particular production processes are described.
2.1. Injection moulding processes
Injection moulding processes are characterized by dynamic characteristics since process input variables are the melting temperatures, the velocity of the cylinder, the holding, the pressure that produces the polymer flow into the model cavity and they vary in a complex manner. The phenomena occurring in the process are very complex, time-varying, nonlinear and uncertain. This complexity makes it difficult to relate the input operating variables to the product quality such as geometry accuracy and geometry surface smoothness. These processes have been implemented and optimized with the use of artificial intelligence with the use of
2.2. Gas metal arc welding processes
In gas metal arc (GMA) welding processes [10, 11, 12], the flow of an electric current is generated by an electric arc that is maintained between the consumable wire electrode and the welding metal as shown in Figure 2 .
Both the filling metal and the consumable electrode are automatically fed by a wire feeding device. A good quality of the welds is determined by the relatively high depth to width ratio of the molten welding pool. So, the monitoring and control of weld geometry and the surface temperatures that are strongly related with the formation of the weld pool, are very important for the penetration depth or the back bead width. For this purpose, the temperatures by noncontact were measured. Infrared temperature sensing system and recent studies conducted the ANN multilayer perceptron to detect and control with great success all the surface temperature information.
2.3. Arc welding processes
The complexity of the relationship between the process variable and weld quality is the common factor in all manufacturing processes and in particular arc welding processes. For these reasons, the literature documents with some interesting researches, the use of ANNs for quality monitoring and control of the process. In this type of welding process, the ANN input data are generally the surface temperature, the welding voltage and the current and torch speed. The majority type of ANN employed for this case was, again, the multilayer perceptron [10, 13, 14, 15, 16, 17, 18, 19, 20, 21]. Their use was found to be very satisfactory to predict the weld defects, the geometry such as bead width, head height and penetration.
2.4. Machining processes
To perform a correct quality control in all machining systems, it is very important to check particular parameters such as the cutting tool state, the vibrations, the forces and the temperature obtained during real-time machining operation. To optimize this particular type of supervision of the cutting tool state, some authors document in literature the use of the ANNs which used the above-mentioned process data to classify the status of tool wear, prediction tool life and detect tool failure in an on-line manner. Examples of typical sensors are tool dynamometers, acoustic emission sensor accelerometers and thermocouples. In this process, the networks in frequent use are the multilayer perceptron and Kohonen [22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46].
2.5. Semiconductor manufacturing processes
The complexity of plasma etching processes in integrated circuits fabrication promoted the use of ANNs for monitoring and control. In this field, the use of artificial intelligence brings advantages that could not be achieved with traditional open loop controls. Where used the multilayer perceptron networks that are the most popular for this process. The ANNs use the fundamental parameters that affecting process dynamics, such as power, gas flow rate, dc bias voltage and throttle positioning. Proper use of networks in this field allows real-time monitoring and estimation of quality variables such as the etching thickness and the etching time [47, 48, 49, 50, 51, 52].
3. Development and implementation of ANN
The origin of artificial neural networks is based on learning technique that mimics the biological learning process occurring in the brain. Neural networks present a robust way to predict an actual value after a learning activity from a supplied sample set . The ANNs are based on a concept that combines a set of computational procedures with a theoretical basis in order to predict the unknown output parameter in various processes. Generally, neural networks are adopted to subordinate knowledge to observations or when data or activity is so complex that is not allows to identify an optimal solution in a reasonable time. It is difficult in each field of application and even for each task, to compare the use of neural networks versus other prediction techniques (e.g., statistical methods or a support vector machine) because, on the contrary to the conventional computational techniques, they are able to solve nonlinear and ill-defined problems. Many factors underlie this trend, most of them are related to reliability of the predictions, to the robustness and adaptability of the results as well as the learning ability of the neural process. In many cases the forecasts generated by ANNs, if correctly designed, significantly improve with increasing of dataset used as training subset. Consistently, in last years, the adoption of the ANNs in many business areas is increased exponentially and the number of publications, in high-level journals, was grown  ( Figure 3 ).
Under engineering perspective, a ‘good’ ANN is based on models able to imitating the proprieties of natural systems, such as cognitive capabilities, flexibility, robustness, ability to learn and fault tolerance. At this scope the structure and the behaviour of the ANN required a study characterized by different hierarchical levels of organization as neurons, layers, synapses and cognition-behaviour functions. Different areas of application are interested by the ANNs, some of them are astronomy, mathematics, physics chemistry, earth and space sciences, life and medical science and engineering. In recent years USA and EU countries, have approved different initiatives for the study of the human brain in these cases, the ANN, in various forms and at different levels, has been included in thesis research projects. The inter-disciplinary given by system adopted for dataset analysis and by the complexity computational required by the elaboration of the data, allows to design and simulate systems capable to satisfying the needs and the challenges of the real world. Japan in 2014 has been developed a project based named as Brain Mapping by Integrated Neurotechnologies for Disease Studies (Brain/MINDS) , that will be integrated with new biomedical technologies and neural network systems. In Australia, a specific programme has also been set up with preliminary funds of around $250 million over 10 years with the goal of developing the world’s first bionic brain (AusBrain)  based on multilayer perceptron (MLP) system. There is also another ambitious initiative in China (Brainnetome) , the goals of this are to simulate the brain networks for perception, memory, emotion and their disorders as well as to develop advanced technologies to achieve these goals.
3.1. Designing the ANN
An ANN is a computational model that establishes a relationship between process factors and output variables. Artificial neurons are combined through weights, which work as adjustable coefficients. There are many programs and frameworks, either of general purpose or that simulates functions or neural structures (e.g., IQR, NeuroSpaces, NNET, etc.) but there is not a specific simulator that is currently being used by the whole community since some different approaches are more suitable than others, on the basis of the research task being addressed. Moreover, most simulators can take full advantage of their computational capabilities on the basis of the features of the computer hardware to which it is installed .
This correlation depends by the fundamental features of the network, which define the way input and output are connected to each other . The network includes input layer, output layer and a certain number of hidden layers. The fundamental features of the network are:
Dataset splitting, which identifies the subset data to be adopted for the training, the testing and the validation of the ANN development;
Architecture, which determines the connections between layers and neurons;
Learning algorithm, which determines the weights of the links between neurons.
In the following sections the data splitting strategy, the architecture design approach and the learning algorithm identification, is described.
3.1.1. Dataset splitting
The appropriate data splitting can be handled as a statistical sampling problem. Therefore, various classical sampling techniques can be adopted in order to split the data in three subset for training, validation and testing of ANN, most commons are: Simple random sampling (SRS), Trial-and-error methods, Systematic sampling and Convenience sampling. The splitting strategy tries to overcome the high variance of the SRS by repeating the random sampling several times in order to minimize the mean square error (MSE) of the ANN. This technique is high time-consuming and requires significant computational costs. A subset (generally as big as 60% of the available experimental data composed by inputs/output pairs) is used for the ANN training. In this phase, the synaptic weights, which are the links between neurons, have a synaptic weight attached. They are updated repeatedly in order to reduce the error between the experimental outputs and the associated forecasts. A subset (generally as big as 20% of the available experimental data) is adopted for the ANN validation. In particular the validation sets allows to identifying the underlying trend of the training data subset. A subset (generally as big as 20% of the available experimental data) is adopted for testing the forecast reliability of the ANN in the learning phase. In order to deal with the overfitting problem that occurs when the network has memorized the training examples, but it has not learned to generalize to new situations, different approaches are suggested: reduce the number of hidden layers, improve the ‘quality’ of the training-subset adopted, introduce some noisy data into training set, etc. In Ref.  an efficient method is proposed for model establishment by means the identification of a low-dimension ANN learning matrix through the principal component analysis (PCA).
3.1.2. Network architectures
An ANN is a computational model that establishes a relationship between process factors and output variables. Artificial neurons are combined through weights, which work as adjustable coefficients. This correlation depends by the fundamental features of the network, which define the way input and output are connected to each other . The network includes input layer, output layer and a certain number of hidden layers ( Figure 4 ). The fundamental steps for the development of a network are:
Dataset splitting, which identifies the subset data to be adopted for the training, the testing and the validation steps of the ANN development;
Architecture, which determines the connections between layers and neurons;
The learning algorithm, which determines the weights of the links between neurons.
Based on the connection pattern (architecture), ANNs can be grouped into two categories:
Feed-forward networks (e.g., single-layer perceptron, multilayer perceptron, radial basis function nets) in which there are not network connection as loops (as shown in Figure 4 );
Recurrent (or feedback) networks, in which different loops occur in network connections (e.g., competitive networks, Kohonen’s SOM, Hopfield network, ART models).
The first one network are considered “static”, in fact they produce only one set of output values rather than a sequence of values from a given input, and they worked in memory-less condition, this means that their response to an input is independent of the previous network state. The ‘recurrent networks’, on the other hand, are dynamic systems, in which the input pattern leads the network to enter in a new state, when a new input is introduced. Most popular network architecture in use today is the multilayer perceptron neural network (feed-forward network) where the output of a previous layer is the input to the next layer. In this case a biased sum of the weights assigned to different inputs, allows identifying the activation level that, through a transfer function, produces the corresponding output. The network thus has a simple interpretation as a form of input-output model, with the weights and thresholds (biases) the free parameters of the model . The design of the ANN architecture consists of identifying the kind of the structure (between feed-forward and recurrent architectures) and identifies the number of hidden layers and the number of neurons for each layer. On one hand, many neurons can lead to memorize the training sets with lost of the ANN’s capability to generalize. On the other hand, a lack of neurons can inhibit the appropriate pattern classification. Many software allows to identify the best number of hidden layer and neurons (for each layer) through a ‘trial-and-error’ approach. In this case different architectures are iterative tested by software and for each of them, the software provide a “fitness bar” based on the inverse of the mean absolute error (MAE) computed on the testing set. In most cases the higher “fitness bar” identifies the best architecture.
3.1.3. Learning algorithm
The purpose of the learning algorithm is to train the network to predict the output parameter(s) given one or more input parameter(s). There are many types of neural network learning rules. There are three kind of learning algorithm, the first is known as supervised learning, in this case the algorithm allows to predict the output parameter on the basis of a set of known input-output pairs [69, 70]. Second algorithm is unsupervised learning, in this case the output is not given, the aim consisting of inferring a function in order to describe a hidden structure (e.g., clustering, anomaly detection, etc.). Therefore the output parameters are considered ‘unlabelled’ (the observations are not classified) and is not provided any evaluation about the prediction reliability ensured by the ANN . Third algorithm is named reinforcement learning, in this case a continue interaction between the learning system and the environment allows to identify the input-output mapping minimizing the performance scalar index. The approach is very similar to unsupervised learning (also in this case there are not given input-output pairs), reward or punishment signals are adopted for the prediction of output parameters . In most cases, the unsupervised learning allows to ensuring lower cost function. Three different methods, usually considered to be supervised learning methods, are described in this work: Quick Propagation (QP), Conjugate Gradient (CG) and Levenberg-Marquardt algorithm (LM).
QP is a heuristic modification of the standard back propagation, the output of the
All network weights are updated after presenting each pattern from the learning data set.
As far as concern CG method, the learning algorithm starts with a random weight vector that is iteratively updated according the direction of the greatest rate of decrease of the error evaluated as
LM algorithm is also adopted, which blends the ‘Steepest Descent’ method and the ‘Gauss-Newton’, therefore it can converge well even if the error surface is much more complex than the quadratic situation; ensuring, in many cases, speed and stability. LM algorithm can be presented as:
4. Case study: “Prediction of the Vickers microhardness and ultimate tensile strength of AA5754 H111 friction stir welding butt joints using artificial neural network”
Among the artificial neural networks applications to the production processes, this section describes the research of De Filippis et al.  in which a simulation model was developed for the monitoring, controlling and optimization of a particular solid-state welding process called friction stir welding (FSW). The approach based on the use of neural networks, using the FSW technique, has allowed identifying the relationships between the process parameters (input variable) and the mechanical properties (output responses) of the AA5754 H111 welded joints. The optimization of the technological parameters has been developed with the aim to produce a stable welding process that can provide welded joints with no defects. The experimental plans that were tested have been constructed by varying the following parameters:
Tool rotation speed;
Travel tool speed;
Position of the samples extracted from the weld bead;
Thermal data, detected with thermographic techniques for on-line control of the joints.
The quality of welded joints was evaluated through the following destructive and non-destructive tests:
Macro graphic analysis;
Indentation Vickers hardness tests
The simulation model was based on the adoption of artificial neural networks (ANNs) using a back-propagation learning algorithm. Different types of architecture were analysed, which were able to predict with good reliability the FSW process parameters for the welding of the AA5754 H111 in Butt-Joint configuration.
4.1. About the friction stir welding process
The process of friction stir welding (FSW) is a solid-state welding method based on frictional and stirring phenomena, which was discovered and patented by the Welding Institute of Cambridge in 1999. In this process, a rotating non-consumable tool that plunges into the work piece and moves forward produces the heat necessary to weld the parts together. Therefore, given the particular geometry of the tool used, as shown in Figure 5 , the following actions are performed in the process:
The tool shoulder generates heat with the base material,
The tool pin generates plastic deformation and mixing of the material.
The much lower temperatures compared with those achieved in traditional welding processes by melting, determine the main advantages of this process. In fact, there is minimal mechanical distortion, with minimal Heat Affected Zone (HAZ), and an excellent surface finish.
No crack formation and porosity right after welding thanks to the low input of total heat.
The main parameters of the friction stir welding process are the tool rotation speed (
4.2. ANN simulation model for the monitoring of the friction stir welding process
The research interest to developing new technological tools for the control and optimization of manufacturing processes is growing. Such developments are crucial elements for the production engineering. Within the FSW process, many experiments are needed to understand the process-related dynamics and to control all the significant variables and the thermographic techniques are a valuable help but it is necessary to increase and optimize control techniques with new information tools for enhancing the quality of manufacturing systems. The reduction in time and cost of the experiments can be reduced by the implementation of numerical and analytical models. Thus the relationship between the process parameters and the quality of the weld can be easily identified by a model based on the adoption of one or more artificial neural networks (ANNs). Neural networks software packages are very common among scientists and manufacturing researchers. In particular, their applications in the field of welding have showed good success. As far as concern the FSW process, in scientific literature, there are only few papers that discuss the modelling of this welding process by a neural network [75, 76, 77, 78, 79]. In particular, a very interesting work is the study of Shojaeefard et al.  who studied the adoption of the neural network trained with Particle Swarm Optimization (PSO) for modelling and forecasting of the mechanical properties of the friction stir welding butt joints in AA7075/AA5083. A further contribution is provided by Asadi et al. , which with the use of ANN found a relationship between the grain size and the hardness of nanocomposites in FSW process. In this particular case study, an effective simulation model was developed for predicting, monitoring and controlling the mechanical properties of welded AA5754 H111 plates, using the ANNs with the FSW process parameters as input variables. The data set for training, testing and validation of the ANN were the results obtained by experimental cases [82, 83, 84], in which all welded joints were performed by non-destructive (visual inspection) and destructive testing (macrographic tests). These tests have been useful for detecting macro defects present on the surface and within the welded area. An accurate quantitative analysis of the FSW process was carried out using the results of the destructive tests of each welded specimen in terms of ultimate tensile strength (UTS) and Vickers micro hardness. Thermographic techniques were used to study the thermal behaviour of FSW process. In the thermal analyses, two thermal parameters were considered: the maximum temperature and the slope of the heating curve measured during the FSW process, along the two sides of the weld (MSHCRS and MSHCAS, respectively). The analysis established that there is a correlation between the data derived from the thermographic controls and the quality of the welded joints, in terms of UTS. Thus this work defines the importance and effectiveness of the use of infrared technology for monitoring the FSW process in a quantitative manner, giving important information on the thermal behaviour of joints during the process. Finally, the purpose of this case study was to correlate the mechanical properties of welded joints in terms of UTS and microhardness to the thermal parameters with the use of the ANN. The results obtained have defined a model with the use of the neural networks that can predict quantitatively the mechanical behaviour of the FSW joints, as shown in Figure 6 .
The results of all tests are summarized in Table 2 and the same data were used to train the ANN.
In order to establish a relationship between the mechanical properties of the FSW joints and the process parameters, a simulation model was developed. In the development of the model two different ANNs were used, as follows: in the first network, called “ANNHV”, was used as output variable, Vickers micro hardness of HAZ; in the second network, called “ANNUTS” was used as the output variable the ultimate tensile strength. Both have used process parameters as inputs. The ANNs were implemented using Alyuda NeuroIntelligence™-Neural networks software (2.2, Alyuda Research Company, LLC., Cupertino, CA, USA). The first network ANNHV was developed with five input nodes (
The second network (ANNUTS) was developed with six input nodes (
The reliability of the estimation of the mechanical properties predicting by the ANN simulation model was evaluated by comparing the data with the experimental results. For this purpose, the mean absolute percentage error (MAPE) was calculated for the two ANNs modelled. Table 3 summarizes the results of this analysis that demonstrated that the values derived from ANN simulation have a higher level of reliability. Therefore, the neural networks were able to predict, with significant accuracy, the mechanical properties of the friction stir welding joints, under a given set of welding conditions.
|ANN model||Input parameters||Output parameters||MAPE (%)|
Vickers micro hardness of the Heat Affected Zone
|ANNUTS (in cascade)||- |
ultimate tensile strength
Starting from the results obtained in previous researches  where the most significant FSW process parameters for AA5754 H111 plates were identified, the development of this ANN model could be used to identify the optimal process parameter setting in order to achieve the desired welding quality.
The interest in ANN is growing so much that its models and algorithms are becoming standard tools in computer science and information engineering. This highlights the fact that after a long and productive youth, neural networks have formed a robust set of computation procedures with a robust theoretical base and undeniable effectiveness in solving real problems in different fields of information processing. In case study discussed in this work, the analysis performed has shown that the ANN simulation model can be used as a further effective method for predicting the FSW process. The MAPE obtained for the outputs micro hardness (HAZ) and ultimate tensile strength (UTS) were, respectively, 0.29% and 9.57%; R2 values were, in all cases, bigger than 0.90. Although the prediction of UTS was characterized by more high level of MAPE, if it is compared to HAZ estimated value, it was considered acceptable to ensure a model characterized by high reliability. The adoption of the simulation model can be very useful for the friction stir welding process. In fact, the use of tools for predicting the mechanical properties of the welds and for controlling the welding process, allows the production of welds with fewer defects. This reduces the number of repairs and costs, associated with the reiteration of the process. In the context of neural networks, the biggest question is “are we currently capable of building a human brain?” . Undoubtedly, the achievement of these challenges is very ambitious, considering that the human brain has around 90 billion neurons shaping an extremely complex network, but in many cases the ANN can support the human behaviour, simplifying the decision making process, increasing the level cognition, under stress conditions, and increasing the capacity in evaluation and analysis of complex processes.