Open access

Introductory Chapter: Advanced Analytics and Artificial Intelligence Applications

Written By

Ali Soofastaei

Published: November 13th, 2019

DOI: 10.5772/intechopen.89784

Chapter metrics overview

840 Chapter Downloads

View Full Metrics

“The key challenge is not so much globalization. It is what I call the fourth industrial revolution. Because its technology which creates major changes in our daily lives. It’s a technology that creates fears. What we want to do is make the world much more aware. On the one hand of the opportunity of the new technology but on the other hand the risks and dangers we encounter”.

—Klaus Schwab


1. Introduction

The opportunities and complexities associated with the digital era can be overwhelming to industries and markets, which face an enormous amount of potential information in each transaction. Being aware of trends in the data pool and benefiting from hidden information has created a new paradigm, redefining the meaning of corporate power. Access to information can make organizations more effective and help them to reach their goals. Big data analytics (BDA) enables industries to describe, diagnose, predict, prescribe, and find hidden growth opportunities, potentially increasing business value. BDA uses advanced analytical techniques to enhance knowledge and improve decision-making by reducing the complexity of exponentially increasing amounts of data. BDA uses novel and sophisticated algorithms to analyze real-time data, resulting in highly accurate analytics. Depending on the problem being solved, these complex algorithms can be allocated to either deep learning or machine learning (ML) approaches.

A significant consequence of the digital world is the creation of bulk raw data. Managers are responsible for managing this valuable capital, with its various shapes and sizes, on the basis of organizational needs. Big data has the power to affect all aspects of society, from social to educational. As the volume of raw data increases, particularly in technology-based companies, the issue of managing it becomes more critical. The variety, velocity, and volume of raw data warrant the use of advanced tools to overcome its complexity and to reveal the hidden information embedded in it. Thus, BDA has been proposed as a means of experimentation, simulation, data analysis, and monitoring. One BDA tool, advanced analytics (AA), can provide the foundation for predictive analysis on the basis of supervised and unsupervised data input. A reciprocal relationship exists between the power of AA and data input—the more precise and accurate the input data, the more effective the analytical performance. Additionally, ML, artificial intelligence (AI) and deep learning as subfields of AA can be used to extract knowledge from hidden data trends [1].

The growing rate of data production in the digital era has introduced the concept of big data, which is defined by its significant volume, variety, veracity, velocity, and high value. Big data has created challenges for analysis, requiring organizations to deploy new analytical approaches and tools to overcome the complexity and magnitude of different data types (structured, semi-structured, and unstructured). Thus, BDA offers a sophisticated technique that can analyze an enormous volume of data and manage its complexity.

BDA can be used to support projects in innovation, productivity, and competition [2] by examining, processing, discovering, and exhibiting results to uncover hidden patterns and provide insights into interesting contextual relationships [3]. Complexity reduction and managing the cognitive burden of a knowledge-based society are key benefits of BDA. The most critical contributor to the success of BDA is feature identification, which defines the most crucial elements affecting results. This is followed by identifying correlations between inputs and a dynamic given point, which can change from time to time [3].

As a result of the rapid evolution of BDA, e-commerce and global connectivity have flourished. Governments have also taken advantage of BDA to provide improved services to their citizens [3]. Specific applications of BDA for the management and analysis of big data include business and social media. BDA can improve understanding of customer behaviors and handling of the five features of big data—volume, velocity, value, variety, and veracity. BDA not only provides businesses with a comprehensive view of consumer behavior but also enables organizations to be more innovative and effective in deploying strategies. Small- and medium-sized companies can use BDA to mine semi-structured big data, improving the quality of product recommendation systems and website design [4]. As suggested by [5], the use of BDA technology and techniques for large volumes of data can improve firm performance.

AA, AI, ML, predictive and prescriptive analytics, optimization models, decision-making algorithms, natural language processing, and robotic process automation have been popular keywords in various industrial studies in recent years. Industry managers and key decision-makers who are aware of the ability of AA to solve business problems are now competing for intelligent technologies and experts to operate them. However, investments in technology and data scientists do not guarantee success. Industrial managers must develop a strong foundation by embedding an understanding of AA within their companies. Three factors contribute to a thriving AA culture: people, strategy, and technology. The knowledge of individuals in companies plays a critical role in the success of the analytics revolution because there are no practical solutions for applying AA when a company is faced with unacceptable levels of knowledge and experience. Management strategies can be implemented to ensure that project teams are sufficiently flexible to adapt to solutions to work processes suggested by the AA. The level of technology is also essential for the accuracy of analytics and can be a critical parameter when the outcomes of AA are used to improve business processes.

AA and AI, defined as intelligence demonstrated by machines, have many applications. AA has been applied in many fields and industries, including agriculture [6, 7], oil and gas [8], aviation [9, 10, 11, 12, 13, 14, 15], computer science [16], deepfake [17, 18], education [19, 20, 21], finance [22], government, heavy industry, history [23], telecommunication maintenance, toys and games [24], hospitals and medicine [25, 26, 27], recruiting, human resources and job search engines [28, 29], military [30, 31], news services [32, 33], writing and publishing, online conference services [34, 35, 36], power electronics [37], sensors [38], and transport [39, 40].


2. Artificial intelligence: applications and challenges

AI capabilities are rapidly evolving, and it is essential to build a framework to model the AI application process from study to implementation. Figure 1 illustrates the general application of AI.

Figure 1.

Steps in applying artificial intelligence (AI).

The critical challenge for using AI in industrial projects is to demonstrate the value of processed data in making intelligent predictions and optimizing decision-making. Overall, there are four significant challenges for the employment of AI in different industries: data, speed, high reliability, and interpretability.

2.1 Data

Industrial systems produce large volumes of data, and advanced engineering is undoubtedly a big data environment. Collected data are typically structured. However, when faced with a low-quality dataset, industrial operations can generate data with “3B” (bad, broken, and illogical background) issues. 3B issues can potentially create challenges in implementing ML and AI solutions for solving business problems. In some industries, the quality of data is insufficient to train and validate sophisticated algorithms such as deep learning models. This problem has been a major challenge for data scientists and data analysis in the development of prediction and optimization applications.

Moreover, real collected data from sites cannot cover all requirements and there are many gaps in datasets. A lack of data can be a crucial problem when data scientists are seeking comprehensive datasets to cover all working conditions. Further, AI solutions should be generated based on reliable historical information; however, in many cases, there is a lack of available data to make sustainable models.

2.2 Speed

With the evolution of technology across various industries, operational processes can rapidly produce large amounts of information. The use of intelligent applications is essential for working with enormous amounts of generated real-time data to reduce resource waste and operational risk. Nowadays, key industries use cloud-based approaches not only to store data but also to improve ease of access to information. However, these approaches still fail to meet the specific requirements for calculation effectiveness.

2.3 High reliability

AI solutions are strongly related to background processes and collected datasets. In other words, the reliability of AI applications depends on the quality of historical information. AA applications usually deal with critical challenges related to security, maintenance, operations, energy consumption, and safety. Dissatisfaction with prediction, optimization, or decision-making algorithms may lead to negative outcomes and discourage users from relying on AA approaches such as AI systems.

2.4 Interpretability

AI can help improve the accuracy and reliability of prediction and optimization of industrial applications. However, interpreting the results is a significant challenge for experts and managers when using AI to solve business problems. A practical solution for industry may be to train experts, specialists, and managers to operate the analytics and provide root cause analysis for anomalies. This implies that during the development of applications, data scientists should work with experts and managers to include domain knowledge in algorithm expansion processes and ensure that models can adaptively learn and accumulate knowledge.


3. Predictive models

Predictive modeling is the term used for the process of utilizing data mining and probability to forecast future outcomes. A predictive (forecasting) model uses several independent variables or predictors that are likely to influence the desired dependent variable (forecasting output). Once data have been collected for the relevant predictors, a statistical algorithm is deployed. This algorithm may be a simple linear equation or a sophisticated ML algorithm such as a neural network. Predictive modeling mainly overlaps with the field of ML, and many of the algorithms utilized in forecasting models are found in the context of ML and AI.

Predictive modeling is often associated with weather forecasting, online advertising, and marketing. However, it also has applications in mining engineering.

One of the most frequently overlooked challenges of the forecasting model is obtaining suitable data to apply when creating algorithms. Data collection and preparation is the most challenging step in developing a predictive model, and it is essential to locate the best predictors to feed into the model. A descriptive analysis of the data and data treatment, including missing values and outlier fixing, is a crucial task that consumes most of the time needed in predictive modeling.

Once the data have been collected, the next step is to select an appropriate model. Linear regressions are among the most accessible models for predictive algorithms, but other multifaceted AI models are available. The complexity of the model does not guarantee the performance of the prediction. Model selection should be considered in relation to data availability and quality and the forecasting period.

After modeling, a production estimation should be provided to measure the accuracy of the model.

Some well-known predictive modeling methods widely used in industrial applications are regression, time series algorithms, deep learning, and ML.


4. Optimization methods

Many different practical optimization methods have been used in critical industries. Generally, the aim of optimization is to increase productivity, energy and cost efficiency, and safety. Prior to the data revolution, traditional optimization models were used for practical business solutions. Currently, the quantity and quality of collected data in many industries have created an opportunity to use innovative optimization solutions to achieve better outcomes. Of all the current optimization approaches, genetic algorithm, particle swarm, ant colony, bee colony, firefly algorithm (FA), and tabu search are the most prevalent in critical industries.

The aforementioned AA, BDA, and AI applications for prediction, optimization, and decision-making may help industries increase efficiency across various dimensions as well as take action to solve global environmental and energy consumption problems. The case studies presented in the following chapters illustrate the possibilities for using AA, BDA, and AI to solve business problems across different industries.


  1. 1. Jan B et al. Deep learning in big data analytics: A comparative study. Computers and Electrical Engineering. 2017;2(3):542-551
  2. 2. Esposito C et al. A knowledge-based platform for Big Data analytics based on publish/subscribe services and stream processing. Knowledge-Based Systems. 2015;79:3-17
  3. 3. Iqbal R et al. Big data analytics: Computational intelligence techniques and application areas. Technological Forecasting and Social Change. 2018;1(2):416-427
  4. 4. Gandomi A, Haider M. Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management. 2015;35(2):137-144
  5. 5. Chen J-L. The synergistic effects of IT-enabled resources on organizational capabilities and firm performance. Information & Management. 2012;49(3-4):142-150
  6. 6. Intel. The Future of AI in Agriculture. 2019. Available from:
  7. 7. Sennaar K. AI in Agriculture—Present Applications and Impact. 2019. Available from:
  8. 8. Aminzadeh F. Applications of AI and soft computing for challenging problems in the oil industry. Journal of Petroleum Science and Engineering. 2005;47(1-2):5-14
  9. 9. Gallagher S. AI bests Air Force combat tactics experts in simulated dogfights. ARC Technica. 2016;2(1):94-107
  10. 10. Jones RM et al. Automated intelligent pilots for combat flight simulation. AI Magazine. 1999;20(1):22-27
  11. 11. Group KBS. AIDA: Artificial Intelligence Supported Design of Aircraft. 2017. Available from:
  12. 12. Tomayko JE. The Story of Self-Repairing Flight Control Systems. 2018. Available from:
  13. 13. Schweikhard KA. Flight demonstration of X-33 vehicle health management system components on the F/A-18 systems research aircraft. Technical Information Service Journal. 2008;6(1):5-15
  14. 14. Adams E. AI Wields the Power to Make Flying Safer—And Maybe Even Pleasant. 2017. Available from:
  15. 15. Haitham Baomar PJB. An intelligent autopilot system that learns flight emergency procedures by imitating human pilots. IEEE Symposium Series on Computational Intelligence. 2016;1(1):1-9
  16. 16. Independent T. Google AI Creates its Own “Child Bot”. 2017. Available from:
  17. 17. Nießner M. Face2Face: Real-Time Face Capture and Reenactment of RGB Videos. 2019. Available from:
  18. 18. Will ‘Deepfakes’ Disrupt the Midterm Election? 2019. Available from:
  19. 19. Sears A. The Role of Artificial Intelligence in the Classroom. 2018. Available from:
  20. 20. Quan-Haase A. Technology and society: Social networks, power, and inequality. Oxford University. 2016;2(1):43-44
  21. 21. Richtel M. Growing up Digital, Wired for Distraction. 2010. Available from:
  22. 22. Chen J. Algorithmic trading. Investopedia. 2019;2(1):45-58
  23. 23. Robotics IIFO. World Robotics 2015 Industrial Robots. 2019. Available from:
  24. 24. Lennihan M. How Artificial Intelligence Is Moving from the Lab to your Kid’s Playroom. 2016. Available from:
  25. 25. Reed TRR, Nancy E, Fritzson P. Heart sound analysis for symptom detection and computer-aided diagnosis. Simulation Modelling Practice and Theory. 2004;12(2):129-146
  26. 26. Yorita AK, Naoyuki. Cognitive development in partner robots for information support to elderly people. IEEE Transactions on Autonomous Mental Development. 2011;3(1):64-73
  27. 27. Luxton DD. Artificial intelligence in psychological practice: Current and future applications and implications. Professional Psychology: Research and Practice. 2014;45(5):332-339
  28. 28. Strauss K. The Role of Artificial Intelligence in the Future of Job Search. 2018. Available from:
  29. 29. Chamberlain A. What’s Ahead for Jobs? Five Disruptions to Watch in 2018. 2017. Available from:
  30. 30. Requena G et al. Melomics music medicine (M3) to lessen pain perception during pediatric prick test procedure. Pediatric Allergy and Immunology. 2014;25(7):721-724
  31. 31. Souppouris A. Google’s ‘Magenta’ Project Will See if AIs Can Truly Make Art. 2019. Available from:
  32. 32. Meehan JR. Tale-spin, an interactive program that writes stories. In: Proceedings of the 5th International Joint Conference on Artificial Intelligence; Vol. 1, Issue 2; 1977. pp. 91-98
  33. 33. Olewitz C. A Japanese A.I. Program Just Wrote a Short Novel, and it Almost Won a Literary Prize. 2016. Available from:
  34. 34. Kongthon A et al. Implementing an online help desk system based on conversational agent. In: Proceedings of the International Conference on Management of Emergent Digital EcoSystems Article No. 69; Vol. 1, Issue 1; 2009. pp. 450-454
  35. 35. O’Brien SA. Is this App the Call Center of the Future? 2016. Available from:
  36. 36. Clark J. New Google AI Brings Automation to Customer Service. 2016. Available from:
  37. 37. Dragičević T, Wheeler P, Blaabjerg F. Artificial intelligence aided automated design for reliability of power electronic systems. IEEE Transactions on Power Electronics. 2019;34(8):7161-7171
  38. 38. Ideacuria Inc. AI Sensor Technology. 2019. Available from:
  39. 39. Johns RC. Artificial Intelligence in Transportation Information for Application. 2007. Available from:
  40. 40. Hallerbach S et al. Simulation-based identification of critical scenarios for cooperative and automated vehicles. SAE International. 2018;1(2):93-106

Written By

Ali Soofastaei

Published: November 13th, 2019