Introductory Chapter: Advanced Analytics and Artificial Intelligence Applications

“The key challenge is not so much globalization. It is what I call the fourth industrial revolution. Because its technology which creates major changes in our daily lives. It’s a technology that creates fears. What we want to do is make the world much more aware. On the one hand of the opportunity of the new technology but on the other hand the risks and dangers we encounter”.

—Klaus Schwab

1. Introduction

The opportunities and complexities associated with the digital era can be overwhelming to industries and markets, which face an enormous amount of potential information in each transaction. Being aware of trends in the data pool and benefiting from hidden information has created a new paradigm, redefining the meaning of corporate power. Access to information can make organizations more effective and help them to reach their goals. Big data analytics (BDA) enables industries to describe, diagnose, predict, prescribe, and find hidden growth opportunities, potentially increasing business value. BDA uses advanced analytical techniques to enhance knowledge and improve decision-making by reducing the complexity of exponentially increasing amounts of data. BDA uses novel and sophisticated algorithms to analyze real-time data, resulting in highly accurate analytics. Depending on the problem being solved, these complex algorithms can be allocated to either deep learning or machine learning (ML) approaches.

A significant consequence of the digital world is the creation of bulk raw data. Managers are responsible for managing this valuable capital, with its various shapes and sizes, on the basis of organizational needs. Big data has the power to affect all aspects of society, from social to educational. As the volume of raw data increases, particularly in technology-based companies, the issue of managing it becomes more critical. The variety, velocity, and volume of raw data warrant the use of advanced tools to overcome its complexity and to reveal the hidden information embedded in it. Thus, BDA has been proposed as a means of experimentation, simulation, data analysis, and monitoring. One BDA tool, advanced analytics (AA), can provide the foundation for predictive analysis on the basis of supervised and unsupervised data input. A reciprocal relationship exists between the power of AA and data input—the more precise and accurate the input data, the more effective the analytical performance. Additionally, ML, artificial intelligence (AI) and deep learning as subfields of AA can be used to extract knowledge from hidden data trends [1].

The growing rate of data production in the digital era has introduced the concept of big data, which is defined by its significant volume, variety, veracity, velocity, and high value. Big data has created challenges for analysis, requiring organizations to deploy new analytical approaches and tools to overcome the complexity and magnitude of different data types (structured, semi-structured, and unstructured). Thus, BDA offers a sophisticated technique that can analyze an enormous volume of data and manage its complexity.

BDA can be used to support projects in innovation, productivity, and competition [2] by examining, processing, discovering, and exhibiting results to uncover hidden patterns and provide insights into interesting contextual relationships [3]. Complexity reduction and managing the cognitive burden of a knowledge-based society are key benefits of BDA. The most critical contributor to the success of BDA is feature identification, which defines the most crucial elements affecting results. This is followed by identifying correlations between inputs and a dynamic given point, which can change from time to time [3].

As a result of the rapid evolution of BDA, e-commerce and global connectivity have flourished. Governments have also taken advantage of BDA to provide improved services to their citizens [3]. Specific applications of BDA for the management and analysis of big data include business and social media. BDA can improve understanding of customer behaviors and handling of the five features of big data—volume, velocity, value, variety, and veracity. BDA not only provides businesses with a comprehensive view of consumer behavior but also enables organizations to be more innovative and effective in deploying strategies. Small- and medium-sized companies can use BDA to mine semi-structured big data, improving the quality of product recommendation systems and website design [4]. As suggested by [5], the use of BDA technology and techniques for large volumes of data can improve firm performance.

AA, AI, ML, predictive and prescriptive analytics, optimization models, decision-making algorithms, natural language processing, and robotic process automation have been popular keywords in various industrial studies in recent years. Industry managers and key decision-makers who are aware of the ability of AA to solve business problems are now competing for intelligent technologies and experts to operate them. However, investments in technology and data scientists do not guarantee success. Industrial managers must develop a strong foundation by embedding an understanding of AA within their companies. Three factors contribute to a thriving AA culture: people, strategy, and technology. The knowledge of individuals in companies plays a critical role in the success of the analytics revolution because there are no practical solutions for applying AA when a company is faced with unacceptable levels of knowledge and experience. Management strategies can be implemented to ensure that project teams are sufficiently flexible to adapt to solutions to work processes suggested by the AA. The level of technology is also essential for the accuracy of analytics and can be a critical parameter when the outcomes of AA are used to improve business processes.

AA and AI, defined as intelligence demonstrated by machines, have many applications. AA has been applied in many fields and industries, including agriculture [6, 7], oil and gas [8], aviation [9, 10, 11, 12, 13, 14, 15], computer science [16], deepfake [17, 18], education [19, 20, 21], finance [22], government, heavy industry, history [23], telecommunication maintenance, toys and games [24], hospitals and medicine [25, 26, 27], recruiting, human resources and job search engines [28, 29], military [30, 31], news services [32, 33], writing and publishing, online conference services [34, 35, 36], power electronics [37], sensors [38], and transport [39, 40].

2. Artificial intelligence: applications and challenges

AI capabilities are rapidly evolving, and it is essential to build a framework to model the AI application process from study to implementation. Figure 1 illustrates the general application of AI.

Figure 1.

Steps in applying artificial intelligence (AI).

The critical challenge for using AI in industrial projects is to demonstrate the value of processed data in making intelligent predictions and optimizing decision-making. Overall, there are four significant challenges for the employment of AI in different industries: data, speed, high reliability, and interpretability.

2.1 Data

Industrial systems produce large volumes of data, and advanced engineering is undoubtedly a big data environment. Collected data are typically structured. However, when faced with a low-quality dataset, industrial operations can generate data with “3B” (bad, broken, and illogical background) issues. 3B issues can potentially create challenges in implementing ML and AI solutions for solving business problems. In some industries, the quality of data is insufficient to train and validate sophisticated algorithms such as deep learning models. This problem has been a major challenge for data scientists and data analysis in the development of prediction and optimization applications.

Moreover, real collected data from sites cannot cover all requirements and there are many gaps in datasets. A lack of data can be a crucial problem when data scientists are seeking comprehensive datasets to cover all working conditions. Further, AI solutions should be generated based on reliable historical information; however, in many cases, there is a lack of available data to make sustainable models.

2.2 Speed

With the evolution of technology across various industries, operational processes can rapidly produce large amounts of information. The use of intelligent applications is essential for working with enormous amounts of generated real-time data to reduce resource waste and operational risk. Nowadays, key industries use cloud-based approaches not only to store data but also to improve ease of access to information. However, these approaches still fail to meet the specific requirements for calculation effectiveness.

2.3 High reliability

AI solutions are strongly related to background processes and collected datasets. In other words, the reliability of AI applications depends on the quality of historical information. AA applications usually deal with critical challenges related to security, maintenance, operations, energy consumption, and safety. Dissatisfaction with prediction, optimization, or decision-making algorithms may lead to negative outcomes and discourage users from relying on AA approaches such as AI systems.

2.4 Interpretability

AI can help improve the accuracy and reliability of prediction and optimization of industrial applications. However, interpreting the results is a significant challenge for experts and managers when using AI to solve business problems. A practical solution for industry may be to train experts, specialists, and managers to operate the analytics and provide root cause analysis for anomalies. This implies that during the development of applications, data scientists should work with experts and managers to include domain knowledge in algorithm expansion processes and ensure that models can adaptively learn and accumulate knowledge.

3. Predictive models

Predictive modeling is the term used for the process of utilizing data mining and probability to forecast future outcomes. A predictive (forecasting) model uses several independent variables or predictors that are likely to influence the desired dependent variable (forecasting output). Once data have been collected for the relevant predictors, a statistical algorithm is deployed. This algorithm may be a simple linear equation or a sophisticated ML algorithm such as a neural network. Predictive modeling mainly overlaps with the field of ML, and many of the algorithms utilized in forecasting models are found in the context of ML and AI.

Predictive modeling is often associated with weather forecasting, online advertising, and marketing. However, it also has applications in mining engineering.

One of the most frequently overlooked challenges of the forecasting model is obtaining suitable data to apply when creating algorithms. Data collection and preparation is the most challenging step in developing a predictive model, and it is essential to locate the best predictors to feed into the model. A descriptive analysis of the data and data treatment, including missing values and outlier fixing, is a crucial task that consumes most of the time needed in predictive modeling.

Once the data have been collected, the next step is to select an appropriate model. Linear regressions are among the most accessible models for predictive algorithms, but other multifaceted AI models are available. The complexity of the model does not guarantee the performance of the prediction. Model selection should be considered in relation to data availability and quality and the forecasting period.

After modeling, a production estimation should be provided to measure the accuracy of the model.

Some well-known predictive modeling methods widely used in industrial applications are regression, time series algorithms, deep learning, and ML.

4. Optimization methods

Many different practical optimization methods have been used in critical industries. Generally, the aim of optimization is to increase productivity, energy and cost efficiency, and safety. Prior to the data revolution, traditional optimization models were used for practical business solutions. Currently, the quantity and quality of collected data in many industries have created an opportunity to use innovative optimization solutions to achieve better outcomes. Of all the current optimization approaches, genetic algorithm, particle swarm, ant colony, bee colony, firefly algorithm (FA), and tabu search are the most prevalent in critical industries.

The aforementioned AA, BDA, and AI applications for prediction, optimization, and decision-making may help industries increase efficiency across various dimensions as well as take action to solve global environmental and energy consumption problems. The case studies presented in the following chapters illustrate the possibilities for using AA, BDA, and AI to solve business problems across different industries.