Artificial Intelligence and Big Data Analytics in Vineyards: A Review

Advances in remote-sensing, sensor and robotic technology, machine learning, and artificial intelligence (AI) – smart algorithms that learn from patterns in complex data or big data - are rapidly transforming agriculture. This presents huge opportunities for sustainable viticulture, but also many challenges. This chapter provides a state-of-the-art review of the benefits and challenges of AI and big data, highlighting work in this domain being conducted around the world. A way forward, that incorporates the expert knowledge of wine-growers (i.e. human-in-the-loop) to augment the decision-making guidance of big data and automated algorithms, is outlined. Future work needs to explore the coupling of expert systems to AI models and algorithms to increase both the usefulness of AI, its benefits, and its ease of implementation across the vitiviniculture value-chain.


Introduction
Viticulture is at the front line of climate change as grape production is highly sensitive to changing environmental conditions. Growers, producers, and investors plan and anticipate risks far into the future with long time horizons (i.e., 7-11 years or more) for investing, establishing, and attaining positive net income and returns on investment. Growers are grappling with unpredictable, rapidly changing weather patterns and more frequent and intense extreme events such as spring frosts, floods, droughts, heatwaves, and wildfires. Seasonal climate changes of hotter and longer summers and warmer winters are shifting areas suitable for growing grapes further north in the Northern Hemisphere (NH), and south in the Southern Hemisphere (SH), from historical cultivation latitudes of 4°and 51°(NH) and 6°and 45°(SH) [1]. This is driving wine makers to move vineyards to higher elevations that provide colder nighttime temperatures and less frequent and intense peak daytime temperatures to ripen grapes, while preventing over-ripening [2,3]. Climate change warming scenarios project that grape cultivar diversity may buffer wine-growing regions from losses resulting from both the reduction of suitable areas for growing grapes and attainable yields. In a recent global study using data on long-term French records to extrapolate globally for 11 cultivars (varieties), increasing cultivar diversity more than halved future, projected losses of current wine-growing areas and decreasing areas lost (56 to 24%) under a 2°C warming scenario, and reducing areas lost by a third (85% versus 58%) under a 4°C warming scenario [4]. These warming scenarios combine daily temperature and precipitation from a large ensemble of the Community Earth System Model (CESM), alongside winegrape phenology and global variety-level planting data [5,6], projecting geographical shifts of areas suitable for grape varieties as well as phenological shifts in the timing of grape ripening (veraison). The resulting loss of suitability of areas is primarily attributed to shifting temperature regimes, and greater accumulations of temperatures above 25°C, and number of days above 40°C. Precipitation was found to have a buffering effect, both reducing the number of varieties that were lost over time, while increasing the capacity for cultivar turnover [4]. While growing diverse cultivars that are more heat-tolerant and drought-resistant can reduce area and yield loss due to climate change impacts, the industry still faces the uncertainty and complexity associated with fulfilling the stringent consumer demands for quality, novelty, cost and sustainability of this agricultural product.
Big data (BD) is data that is machine-readable as opposed to human-readable. There is no official size that makes data "big". It consists of massive amounts of digital information, collected from all sorts of sources that are too large, raw, or unstructured for analysis using conventional relational database and techniques. The internet-of-things (IoT) (i.e., the network of physical objects that exchanging data between devices, software, and systems over the Internet) continues to create BD and expand globally. Artificial intelligence (AI) refers to the simulation of human intelligence in machines that are programmed to think, learn and problemsolve like humans and mimic their actions. Machine learning (ML) is a sub-set of AI where machines learn from data without being explicitly programmed. Deep learning (DL) is a subset of ML in which artificial neural networks (ANNs) mimics the structure of the human brain, to adapt and learn from vast amounts of data. Algorithms are procedures that are implemented in computer code that use data, and are, in general, distinguished from models, which comprise many algorithms. BD needs to be of sufficient high quality to reliably train, validate, and independently test and/or reproduce algorithmic and model output at reported levels of accuracy and reliability. Here the goal is to design AI algorithms with a fast and efficient learning speed, fast convergence to a solution, good generalization ability and ease of implementation.

Review objective and methodology
This review explores the benefits and challenges of BD and AI to sustainable viniviticulture through the lens of recent research findings and insights. Detailing all the different AI methodologies and their implementation is beyond the scope of this review that focuses on their domain application. For background reading of state-of-the-art AI methods and solution techniques, we direct interested readers to an article that features how vineyards are making use of BD [7], a recent introductory methodological reviews of ML in agriculture [8], and DL [9]. In the review conducted and reported here, recently published and highly relevant scientific journal articles were searched and selected using the University of Victoria (UVic)'s Summons 2.0 search engine, which includes a wide range of scientific databases, including the Scopus, ScienceDirect and PubMed databases. A total of 59 articles were selected that met the required, minimal criteria that they assessed, applied, adapted, or developed an AI method/algorithm and addressed a main aspect linked with viniviticulture. This search approach was selective rather than exhaustive or systematic. The resulting sample size is similar to the 40 articles selected as part of another recent AI review which also employed online search of major scientific databases [8].
A systems overview of vitiviniculture interactions and drivers of change was first constructed. This was used to distinguish 10 major aspects under which a range of use-cases could be identified and linked across the selected works. This was informed, in part, by a broad review of vineyard ecosystems, their multifunctionality, and ecosystem services, applied the Common International Classification of Ecosystem Services (CICES) highlights the need to better identify and understand interactions within vineyards, identifying six ecosystem services (or aspects) that are most studied, namely: i) cultivated crops, ii) filtration and sequestration, iii) storage and accumulation, iv) pest and disease control, v) heritage and cultural services, and vi) scientific services (e.g., studying vineyard agronomy) [10]. Challenges identified and described within the selected articles were next extracted, compiled, and synthesized into a summary Table. A depiction or simplified design of a novel BD value chain informed by an ES comprising expert knowledge and providing an ES system with an ability to learn is presented. This is structured to encompass all the identified aspects and potentially capable of addressing current research challenges.

AI in Vitiviniculture
Viticulture is at the front line of technological disruption driven by automated, AI algorithms that integrate and learn from large complex data obtained from diverse sources both old and new. New technologies and data sources include satellite and drone remote-sensing, field sensors, and automated weather stations which are increasingly being deployed and used to enhance decision-making because of their increased availability, affordability, and reliability. For example, Palmaz vineyards in California's Napa Valley are early-adopters of BD and AI, bringing innovation and invention to the ancient art of making wine. They use monitoring and geospatial technology for guidance and decision support. This includes VIGOR (Vineyard Infrared Growth Optical Recognition) to monitor and adjust conditions in the vineyard and an intelligent wine-making assistant, FILCS (Fermentation Intelligent Logic Control System), nicknamed Felix, and STAVES (Sensory Transambiental Variance Experiment) to monitor wines as they age in the barrel [11]. New decision-support tools have also been developed that use BD and AI technology provided by Sippd TM and Vitiapp TM [12,13]. There are aspirations even to build an AI system (i.e., a Turing AI taster) that can out-perform a wine expert? [14]. Sippd offers a commercially-available, personal sommelier that uses AI to help consumers discover wines based on taste and budget, with personalized wine recommendations. VitiApp TM is a pre-commercial web-based application for supporting decisions about vineyard management. It includes environmental data (weather, soil) to describe conditions influencing grape yield and fruit composition, cloud computing to integrate multiple data streams from a diversity of vineyard sensors and weather forecast data. It provides vineyard patch-specific awareness of weather-based risks for each selected management issue: botrytis/powdery/downy disease, and frost/chilling/heat accumulation, wind, rainfall, soil moisture and/or spraying conditions. While often used interchangeably, viti-culture refers to the science, study, and production of grapes, whereas vini-culture is specific to grapes for winemaking; when combined is vitiviniculture. According to the International Organization of Vine and Wine (OIV), sustainable vitiviniculture is a â€oeglobal strategy on the scale of the grape production and processing systems, incorporating at the same time the economic sustainability of structures and territories, producing quality products, considering requirements of precision in sustainable viticulture, risks to the environment, products safety and consumer health and valuing of heritage, historical, cultural, ecological, and landscape aspectsâ€ (see [15] and references therein). While sustainable wines are currently a niche market, they are increasing in number, and consumers are willing to pay a premium for sustainably produced wines. Actions and guidance need to incorporate uncertainty and be fine-tuned to the local conditions and impacts. Grapevines phenotype (terroir), canopy microclimate, vine growth and physiology, yield, and berry composition all contribute various attributes to wine and the degree to which it reflects its varietal origins and signature characteristics or typicity [1]. Vitiviniculture management is likely to become more complex. There are also stringent rules and regulations linked with production certification schemes and labelling systems for vineyards that apply organic, sustainable, biodynamic practices that include reducing environmental risks. The Summerhill Pyramid Winery based in Kelowna, British Columbia, Canada, for example, was certified in both organic under Canadian organic standards (PACS # 16-077, COR Section 345) in 1988 and Demeter biodynamic certification in 2012. Timely, suitable, and cost-effective adaptation strategies and enhanced foresight are crucial to support the complex dynamics and management of vitiviniculture.

AI learning algorithms and model types
There are three main types of learning: supervised that learns known patterns, unsupervised that learns unknown or hidden patterns, and reinforced that learns rules or actions in data to learn a pattern or decision process and can be value-, policy-, or model-based in how it optimizes its solution to a given complex problem. Classification and regression problems are supervised, clustering and anomaly detection are unsupervised. Learning algorithms differ according to the problem and their ability to be trained on different types and amounts of data without being overfitted. Overfitting is a concept in AI and data science, which occurs when a statistical model fits exactly against its training data because it memorizes the noise and fits too closely. Deep double descent is the phenomenon where performance improves, then gets worse as the model begins to overfit, and then finally improves more with increasing model size, data size, or training time. Essentially, there is a given level of complexity where models are more prone to overfitting, but if enough complexity is captured in the model, the larger the model and data, the better. Learning can be sequential, in which one part of a task is learnt before the next, or incremental, in which an algorithm learn from scratch and gradually obtains more knowledge with an increasing amount of training inputs or examples by adjusts weights of an observation based on the last classification. How algorithms are trained on data differs as well. Bagging (i.e., bootstrap aggregating) generates additional data for training a model by resampling a given dataset through repeatedly re-combinations to produce multi-sets of the original data. Learning can also be ensemble-based (termed batch learning or stacking) that combines several base models in order to produce one optimal predictive model. Bagging is suitable for high variance, low bias problems, boosting is suitable for low variance, high bias problems, and stacking combines different models to learn some parts of a problem, in solving the whole space of a complex problem. Popular ML algorithms differ in terms of how they find solutions and partition a given problem space. A Support Vector Machine (SVM) uses hyperplane partitioning, Random Forest (RF) uses tree-based ensemble partitioning, and Gradient Boosting (GB) use an ensemble of weak prediction decision trees. Adaboost or Adaptive Boosting assigns higher weights to incorrectly classified data and Stochastic Gradient Boosting uses statistical bootstrapping of data to generate samples for implementing boosting. XGBoost is a boosting algorithm that benefit from 'regularization' that penalizes various parts of the algorithm to improve its performance by reducing overfitting.
ANNs comprise a collection of connected units or nodes called artificial neurons aggregated into different layers which transmit and process signals between their connections (edges). The signal of a given node is prescribed by a mathematical 'activation' function. Signals travel from a first 'input' layer, through one or more intermediate or 'hidden' layers, to an 'output' layer. Nodes in the hidden layer have values that are unknown and determined mathematically from their input and output signals as a network learns. Different layers may perform different transformations on their inputs. Connections can exist between nodes in different layers or between nodes within a given layer. Feedforward neural networks (FNNs) are a type of ANN having no memory, whereby signals only move in one direction from the input through to the output layer, never being processed by a node more than once. An extreme learning machine (ELM) is a FNN with a one or many hidden layers whose nodes can signal randomly, never update, or inherit previous signals without requiring any tuning of the mathematical function parameters of its node activation functions, or the weight values that alter the strength of how its inputs are connected within the network. A wide range of different DL model structures have evolved from FNNs. Recurrent neural networks (RNNs) are FNNs with memory whose nodes process signals in loops/feedbacks/cycles that considers current inputs and also what it has learned from previous inputs. Long-short-termmemory (LSTM) are a type of RNN that uses special units that include a â€~memory cellâ€™ that maintains information in memory for longer periods of time. Convolutional neural networks (CNNs) have several layers whose nodes are sparsely connected (i.e., nodes are not fully connected) whose flexibility is particularly useful for image recognition and object classification. A CNN typically comprises four types of layers, namely, the convolution layer, rectifier (ReLU) layer, pooling, and fully connected layers. Every layer has its own functionality and performs feature extractions and discovers hidden patterns in input data. RNNs can use sequential information, while CNNs cannot.
Restricted boltzman machines (RBM) consist of a two-layer network of fully connected nodes with both forward and backwards connections (i.e., a cycle) that can share weights (i.e., bidirectional). This two-layer network was originally designed to better determine good starting weights (i.e., pretraining) of FNNs. A deep belief network (DBN) consists of RBMs which are sequentially connected, comprising multiple hidden layers, with connections between hidden units are in separate layers. Deep q-learning networks (DQLNs) use reinforcement learning to make a sequence of decisions through trial and error within an interactive environment involving 'agents' that have 'states' that change, learn, and adapt over time. Q-learning is a specified form of reinforcement learning (i.e., values-based learning) that is model-free i.e., does not require a model of the environment. It learns expected values of future rewards for actions of agents that are in a given state with a given 'value'. It uses q-learning (i.e., learning from delayed rewards) based on Bellman's Equation that decomposes the value of an agent's state into an immediate reward and the value of a cumulative set of successor states according to a discount factor that determines the importance of future rewards. Bayesian learning (or belief) networks (BLNs) are a type of network model that is stochastic or probabilistic and involves 'priors'. Prior is short for 'prior probability distribution' and is the probability distribution that express one's beliefs about an uncertain quantity before some data or further evidence is taken into account. They are used to represent spatial or temporal dependence (represented by conditional probability distribution functions) between multiple stochastic variables (i.e., nodes), describing how the variables depend on each other in terms of cause-and-effect or causality (i.e., connections or arcs between nodes). Variables can be discrete or continuous. BLNs can be prepared by experts or learned from data, then used for inference to estimate the probabilities for causal or subsequent events. Copula bayesian networks (CBNs) use a tailored mathematical function called a copula that provides an efficient way to represent and compute the joint probability represented by such networks along with how its variables depend on each other.
New methods and frameworks to use and integrate BD and AI for complex problem-solving and enhanced decision making will, very likely, be needed to support sustainable vitiviniculture. Such approaches will need to consider complex interactions between climate, biotic, and abiotic drivers, stressors, and risks within vineyards, influencing grape and wine production, and value-chain resiliency and sustainability (Figure 1).

AI use-cases and knowledge gaps
Structured data is highly organized and easily understood by machine language, whereas unstructured data is often categorized as qualitative data that cannot be processed and analyzed using conventional tools and methods and includes text, video files, audio files, mobile activity, social media posts, and satellite imagery. BD can include also vague and imprecise information, qualitative data, and rule-based logic. An expert system (ES) is a computer program, model, or algorithm that uses AI to simulate the judgment and behavior of a human or an organization that has expert knowledge and experience in a particular domain or field. It provides supervision for AI algorithms by human experts termed human-in-the-loop (HITL), whereby a model requires human interaction and intervention and is not fully automated or self-reliant. AI in winemaking based on an ES approach was explored in 2000 [16], with limited research on ES, and closely associated, fuzzy inference systems (FIS) in viniviticulture. Fuzzy theory and FIS represent vagueness and imprecise information often used in making decision in a mathematical way using

Overview of the interactions of major climate, biotic, and abiotic drivers, stressors, and risks within vineyards.
fuzzy sets and rule-based logic. Several leading examples are noteworthy. An ES for automated forecasting of optimal grape ripeness dates using data gathered from a vineyard wireless sensor network (WSN) has been developed and tested, but uses the Holt method (exponential adaptive forecasting for trended data) instead of ML or DL models/algorithms [17]. Also, an FIS that enables automating the classification of grape quality at harvest for grape growers has been developed and tested [18]. An ES for evaluating the sustainability of vineyards based on their management called Vigneto uses a fuzzy logic indicator [19]. A decision support system called FGRAPEDBN that uses fuzzy logic and expert knowledge is able to predict grape berry maturity. Berry maturity is measured as sugar concentration that increases rapidly, and acidity concentration, that decreases along with pH levels as berry mature. This ES attains high predictive accuracy (i.e., a root-mean-squarederror (RMSE) of 7 g/l (i.e., 0.44 g/l or 0.11 g/kg) [20]. The coupling of ES to AI (i.e., ML and DL models/algorithms) in viticulture, or agriculture in general, is still unexplored and in its infancy. Also, ES systems generally have no ability to learn decision rules, so could benefit also from being informed by AI/ML analytics and predictive insights.
A wide array of applications and use-cases of AI in vitiviniculture are evident, and are summarized in Table 1. This shows that there is substantial interest, applied expertise, and future potential in developing such approaches to help mitigate and adapt to climate change, address inter-related risks, and enhance decision-making and foresight. Current AI work is, however, concentrated heavily on grapevine yield prediction and grape variety classification using on the pattern recognition, detection, counting, and clustering of grape berries and bunches in imagery collected by observers, unmanned aerial vehicles (UAVs), and/or robots. Such imagery differs based on vineyard environmental conditions and grape variety altering illumination, occlusions, colors and contrast in images. Existing research limitations and challenges point to the need for robotics and mobile sensing platforms, the combination or fusion of both fine-scale hyperspectral and coarser-scale multispectral imagery data, as well as spatially-distributed sampling within vineyards to better measure and assess micro-climate variability linked with meso-and macroclimate and landscape suitability requirements that are changing with climate change.
Suitability requirements for vineyards would benefit from other AI/ML techniques to explore geospatial data and cross-validate geographical locations determined from CNN models applied to identify vineyards in satellite data. A wide range of different models for disease and pest control (i.e., a hybrid BLN, CNN, RF, GB) have been applied, and these multiple AI approaches could be coupled to provide a fully-integrated solution for processing field imagery, conducting data mining and analytics, and forecasting of disease risk in vineyards. Vineyard management is already exploring decision rule applications via case-based reasoning, and sequential methods of AI, but in isolation, and such work could greatly benefit from being coupled together to accelerate advancement. This would enable them to be tested on a broader set of vineyard data and to better identify best management practices, rather than a more incremental, siloed approach. Much more work is needed to explore opportunities and potential of BD and AI in vineyard biotic and abiotic factors and stress. Only a handful of studies have explored the use of satellite remote-sensing (i.e., Earth Observation or EO) data for detecting and mapping water and heat stress, yet large amounts of data for training and validating AI models now exists from EO data centers and providers. This could help to validate whether satellite indices can reliably detect and map stress variability in vineyard, what data fusion and satellite indices perform best, to port such BD and capabilities to support stakeholders proactive decision making ahead of extreme weather  [56][57][58][59][60][61][62] Refer to abbreviation list for model/algorithms. impacts like heatwaves. Most work on wine aroma and sensory profiling still employs traditional statistical techniques and clustering with limited work on global optimization (GO). While decision tools already exist in the market to track the wine preferences of consumers, they could be better informed from AI analysis and prediction that links more objective, scientific data on new varieties, wine constituents, alternative wine blends and new wine grown in newly establish vineyards in more suitable areas as climate change shifts grape and wine suitability. The application of BD and AI in traceability, authenticity, and protection also relies on more traditional statistical methods, rather than BD and AI. This is surprising and was not expected before conducting this review, as this area involves large extents of the value-chain and major business risk. Here, government could play a vital role to codesign and pilot test new solutions alongside experts in BD and AI, as developing broad-based solutions in this aspect likely require broad collaboration, multidisciplinary expertise, substantial BD collection and sharing, and industry wide involvement, adoption, and deployment.

Proposed BD and AI framework
An existing ontology framework called the Agri-Food Experiment Ontology (AFEO) has been developed to guide the integration of data in a way that provides researchers with the information necessary to address extended research questions [63]. It contains 136 concepts spanning viticulture practices, wine-making products, and operations. It utilizes the Resource Description Framework (RDF) format, a standard model for relational data queries, interchange, and metadata processing, to represent these data in a standard format. Based on this review, an analytical framework is proposed that integrates BD analytics and AI prediction as part of a BD value-chain using expert knowledge as HITL intervention and guidance is outlined in Figure 2.
BD is distributed across different remote-sensing platforms (e.g., drone and satellite), across vineyards (e.g., networks of AI and climate-smart vineyards), and within vineyards (e.g., field sensor networks), and across data centers and providers (e.g., long-term climate stations and weather monitoring networks providing Depiction of a vineyard BD value-chain that incorporates diverse, distributed vineyard data alongside an expert system. This system integrates traditional, cultural perspectives, knowledge, and reasoning of grape growers, viticulture specialists, and other wine industry stakeholders. both historical climate and near-real-time weather station data). Using a distributed cloud approach, an application of cloud computing technology, BD can be interconnected with public and private applications served from varied geographical locations for preprocessing quality control, data quality checks, model identification (i.e., variable selection, quantile classification), indicator model benchmarking, and the development of risk forecast models using AI. An ES system comprising conditional, decision rules provides traditional and expert knowledge, while informing AI model training and validation. An AI model then also learns by selecting rules from the master ES ruleset, adjusting and updating rules as it learns. In this way, the framework is agile and scaleable to address a wide range of stakeholder needs along the value-chain. This includes life-cycle assessment (LCA), providing data to support monitoring and tracking of vineyard sustainability indicators, and providing forecasts (i.e., foresight) to better anticipate future impacts, having additional lead time to mitigate and safeguard operations in time, and deciding between different possible actions and interventions to climate change (i.e., irrigation needs and limitations, disease outbreaks, extreme weather events) risks for more informed vineyard management scheduling and planning. Weather and climate transformed into tailored information and knowledge that vineyard stakeholders and users need and require are provided through customized Climate Information Services (CIS) help to drive forecasts of relevant vineyard indicators. This could integrate sub-seasonal and seasonal forecasting, alongside longer-term, downscaled inter-annual and decadal scenario projections. The quantification of risk (i.e., levels and associated uncertainties) is essential to determine an appropriate response. With an approach that can be scaled up to the entire vitiviniculture value-chain the adoption of BD and AI can be accelerated. This would enable all stakeholders to co-learn and collaborate in evidence-based and model-tested design tactics and strategies. Such an approach can ensure mitigation and adaptation actions and interventions are enabling, rather than inhibiting, to maximize perceived benefits and organizational readiness, while minimizing external pressures [64].

Conclusions
Vineyards that are certified organic and biodynamic, however, are not necessarily the same ones that are early-or significant-adopters of latest BD and AI technology that can accelerate and support the wider transformation from conventional to sustainable vitiviniculture practices. As discussed, this is because of a disconnect that exists between the path to adoption of sustainable practices and the path to adoption of BD and AI technology. This could be addressed by providing a way to structure and integrate an expert knowledge and insights from all stakeholders into an ES embedded within an overarching analytical framework. The majority of research challenges identified in this review, which span a wide range of aspects of viniviticulture, also point to the need for including expert knowledge to provide context and rules to design AI algorithms and their automated learning, while helping to structure data, obtain high-quality data for training AI models, and validate the use and adoption of new BD types and sources. Aligning the existing AFEO ontology that links vitiviniculture objects and experimental activities to an analytical BD and AI modeling, could accelerate the advancement of sustainable vitiviniculture. This would also provide the ES methodology with an ability to learn from experience which most systems cannot do currently. ML and DL models and algorithms need to be trained and informed by an ES that integrates imprecise and vague information as well as qualitative data and decision rule-based logic that is used in stakeholder decision making. This will require linking the scientific and expert knowledge on climate and weather risks pertaining to drivers and interactions, the BD value chain, to address the identified research challenges outlined here. Future work will aim to synthesize knowledge and insights from the wide array of applications of ES, to design a representative ES for the proposed BD value chain.