Hyperspectral Imaging and Their Applications in the Nondestructive Quality Assessment of Fruits and Vegetables

Abstract

Over the past decade, hyperspectral imaging has been rapidly developing and widely used as an emerging scientific tool in nondestructive fruit and vegetable quality assessment. Hyperspectral imaging technique integrates both the imaging and spectroscopic techniques into one system, and it can acquire a set of monochromatic images at almost continuous hundreds of thousands of wavelengths. Many researches based on spatial image and/or spectral image processing and analysis have been published proposing the use of hyperspectral imaging technique in the field of quality assessment of fruits and vegetables. This chapter presents a detailed overview of the introduction, latest developments and applications of hyperspectral imaging in the nondestructive assessment of fruits and vegetables. Additionally, the principal components, basic theories, and corresponding processing and analytical methods are also reported in this chapter.

1. Introduction

In recent years, consumer demand for fruits and vegetables tends to be diversified, and more attention has been paid to the external quality of apples. Generally, such attributes include its ripeness, size, weight, shape, color, condition, or presence/absence of defects, stems or seeds, as well as a series of internal properties such as sweetness, acidity, texture, hardness, among others [1]. Consequently, the accurate, rapid, and objective assessment system in the processing stage is essential to ensure the quality of fruits and vegetables during processing operations. Food process control necessitates real-time monitoring at critical processing points [2].

Traditional optical sensing techniques, such as imaging and spectroscopy, have limitations to acquire adequate spatial and spectral information for nondestructive evaluation of food and agricultural products. Generally, conventional imaging cannot acquire spectral information and spectroscopy measurement cannot cover large sample area. In general, the frequently-used vision systems for fruits and vegetables sorting are based on color video camera that imitates the vision of the human eye by capturing images using three filters centered on red, green and blue (RGB) wavelengths [3, 4]. Thus, they are limited to observing scenes and are usually not able to obtain much information about the external or internal composition of the products or to detect some defects or alteration whose color is similar to the color of the sound skin. In addition, traditional methods of fruits and vegetables monitoring involving analytical techniques are too time consuming, expensive and require sample destruction.

Over the past decades, with the rapid development of information science, image processing and pattern recognition technology, optical sensing technologies have been emerged as scientific tools for nondestructive assessment for quality of fruits and vegetables. Spectral imaging technology, combining conventional imaging and spectroscopy techniques, can acquire spatial and spectral information from the target, which is used for evaluating individual food products. In particular, hyperspectral imaging has been widely researched and developed by integrating spectroscopy and imaging techniques into a system that can obtain a spatial map of spectral variation, resulting in many successful applications in the quality assessment of fruits and vegetables. A typical spectral image is composed of a set of monochromatic images corresponding to certain wavelengths, and hyperspectral image systems have the natural advantage compared to the traditional computer vision, even the human vision [2]. Hyperspectral imaging systems can make it possible to extract some appearance features that are difficult or impossible with the traditional computer vision systems.

This chapter focuses on hyperspectral imaging technologies in the quality nondestructive assessment of fruits and vegetables. In the second section, overview, components, and different image acquisition technologies of hyperspectral imaging are explained and discussed. Hyperspectral images generate a large amount of information that can be processed using different statistical techniques [1]. In the third section, varying nondestructive processing and analysis methods are illustrated in detail. Finally, applications of this technology are discussed, and conclusions are given.

2. Hyperspectral imaging technique

2.1. Overview of hyperspectral imaging

Hyperspectral imaging, known also as chemical or spectroscopic imaging, is an emerging technique that integrates conventional imaging and spectroscopy to simultaneously collect spatial and spectral information from an object. The term “hyperspectral imaging” was derived from works in remote sensing first mentioned by Goetz et al. in [5] to make a direct identification of surface materials in the form of images. Although originally developed for remote sensing, hyperspectral imaging system is gradually found to have natural advantages over the traditional computer vision systems [2] in such diverse fields as agriculture [6, 7, 8, 9]. With the development of optical sensing and imaging techniques, hyperspectral imaging has recently emerged as a scientific and efficient inspection and assessment tool for quality of fruits and vegetables. The goal of hyperspectral imaging is to obtain the spectrum for each pixel in the image of a scene, with the purpose of finding objects, identifying materials, or detecting processes [10]. To obtain high spectral resolution and narrow band image data, hyperspectral imaging is generally combined with spectroscopic technique, two-dimensional geometric space and one-dimensional spectral information detection.

2.2. Components of hyperspectral imaging system

Figure 1 shows the schematic of the hyperspectral imaging system commonly used in our research. As shown in Figure 1, a typical hyperspectral imaging system usually consists of the following components: a light source (illumination), a wavelength dispersion device (spectrograph), an area detector (camera), a transportation stage and a computer with corresponding software [11].

Figure 1.

A schematic of the hyperspectral imaging system.

Light source for spectral imaging applications can generally be classified into two categories: illumination and excitation source. Broadband lights are generally used as the illumination sources for reflectance and transmittance imaging while narrowband lights are commonly used as the excitation sources. Therefore, illumination is a crucial part of the hyperspectral imaging system. Compared with the naked eyes, vision systems are affected by the level and quality of illumination. Illumination devices generate light that illuminates the inspected target objects; thus, the performance of the illumination system can greatly influence the quality of images and plays an important role in the overall efficiency and accuracy of the system [12]. Good illumination can help to improve the success of the image processing and analysis by reducing noise, shadow, reflection, and enhancing image contrast [2]. In addition, the positions, types of lamps, and color quality of the illumination are all considered when choosing the most suitable illumination. Incandescent lamps, fluorescent lamps, lasers, and infrared lamps are the commonly used light sources [13].

The wavelength dispersion device is one of the key components of hyperspectral imaging system. Filter, grating and prism are three typical wavelength dispersion devices. These optical devices are used to disperse broadband light into different wavelengths and project the scattered light onto the area detector. The principles of prism and diffraction grating are illustrated in Figure 2. In a word, filter is always used in the multispectral imaging system, while prism and grating are widely used in the hyperspectral imaging system [2]. Besides, the efficiencies of the transmission components (e.g., prisms) are generally lower than those of the reflective optical component (e.g., mirrors). An optical wavelength dispersion device includes [14, 15]: a first substrate; an input unit formed on the first substrate having a slit for receiving an optical signal; a grating line formed on the first substrate for generating a diffracted light beam of the optical signal; a first optical reflector formed on the first substrate to the reflected output beam from the diffraction grating for the output; and a second substrate covered on the top of the input unit and the grating. The wavelength dispersion is capable to disperse broadband light into varying wavelengths. Typical examples include filter wheels, imaging spectrographs, acousto-optic tunable filters, liquid crystal tunable filters, Fourier transform imaging spectrometers, and single shot imagers [16].

Figure 2.

Operating principles of diffraction grating and prism.

The camera, which is one of the image acquisition devices, is another core component of the hyperspectral imaging system. It is the carrier of the physical or chemical information and the light generated from the light source. Other image acquisition devices used in food applications are computed tomography (CT), magnetic resonance imaging (MRI), ultrasound and electrical tomography [17]. Charge coupled device (CCD) and complementary metal oxide semiconductor (CMOS) image sensors are two different means to generate the image digitally [2]. A CCD is a device for the movement of electrical charge, generally from within the device to an area where the charge can be manipulated. In the CCD image sensor, pixels are represented by P-doped metal oxide semiconductor (MOS) capacitors. When image acquisition starts, these capacitors are biased above the threshold for inversion, allowing the conversion of incoming photons into electron charges at the semiconductor-oxide interface [18]; then, the CCD is used to read out these charges. The CMOS image sensor consists of millions of pixel sensors, each of which includes a photo detector. As light enters the camera through the lens, it strikes the CMOS image sensor, allowing each photo detector to accumulate an electric charge based on the amount of light that strikes it. CMOS is also sometimes referred to as complementary-symmetry metal–oxide–semiconductor (COS-MOS). In general, the CMOS image sensor is used in applications with less exacting quality demands, and the CCD image sensor is widely used in medical, scientific and professional applications where high-quality image data are required.

Compared with the traditional computer vision system, a wavelength dispersion device and a transportation stage are additional components of hyperspectral or multispectral computer vision systems. The translation stage is used to move the sample past the objective lens when the camera captures only a line of the illuminated object.

The computer is not only used to control the hyperspectral imaging system for data acquisition, processing and analysis of image and spectral data for specific application, but also can provide storage space for hyperspectral image. By scanning the entire surface of the specimen, a complete hyperspectral image is created and displayed by the computer [19].

2.3. Generation of hyperspectral images

Hyperspectral image is three-dimensional hyperspectral cube, composed of two spatial and one wavelength dimension [20]. There are three approaches to build hyperspectral images based on the method by which spatial information is acquired as whiskbroom, pushbroom, and tunable filter known as point scanning, line scanning, and area scanning, respectively [21], as illustrated in Figure 3. The point-scan method (Figure 3a) is a basic spectroscopic approach, where a single point is scanned along two spatial dimensions by moving the sample or the detector. When a single point is scanned, the sample moves to the next measurement point and another spectrum is captured. By moving the sample systematically in two spatial dimensions, a complete hyperspectral image can be obtained. However, it is not suited for fast image acquisition because the scan of many points for two spatial dimensions is a time-consuming process. The line-scan method (Figure 3b) can be considered as an extension of point scanning method. In the line-scan method, a slit of spatial information and full spectral information for each spatial point in the linear field of view can be acquired simultaneously. But the line-scan method requires the use of an imaging spectrometer, in which a diffraction grating disperses light entering through a thin slit and projects. Food commodities normally are moved linearly along a production line [11]. Consequently, the line-scan method is appropriate for online inspection of individual food. The area-scan method (Figure 3c) does not require the relative movement between the sample and the detector and is usually used to collect images from the fixed scene. The line-san camera holds an advantage over area-scan camera. Unlike these area-scan cameras, a line-scan camera can expose a new image while the previous image is still reading out its data. A detailed description of data preprocessing methods can be found in the literature [22, 23].

Figure 3.

Three different approaches to generate a hyperspectral image. (a) The point-scan method. (b) The line-scan method. (c) The line-scan method.

As shown in Figure 4, hyperspectral imaging system is generally carried out in reflectance, transmittance or interactance modes according to the specific light-output captured by hyperspectral imaging system [24]. In the external quality inspection of fruits and vegetables, the reflectance mode is considered to be the most suitable approach. Position of light source and the optical detector (cameral, spectrograph, and lens) are different for each acquisition mode [21]. In the external quality inspection of fruits and vegetables, the reflectance mode (Figure 4a) is considered to be the most suitable approach. In reflectance mode, to avoid specular reflection, the detector captured the reflected light from the illuminated sample in a specific conformation. The transmitted light captured through the sample is often very weak but carries more valuable information and the detector is located on the opposite side of the light source. Transmittance mode (Figure 4b) is usually used to determine internal component concentration and detect internal defects of relative transparent materials [16]. Interactance mode (Figure 4c) is a combination of reflectance and transmittance where both light source and the detector are located in the same side of sample and parallel to each other.

Figure 4.

Three different modes to generate a hyperspectral image. (a) The reflectance mode. (b) The transmittance mode. (c) The interactance mode.

2.4. Characteristics of the hyperspectral images

In the conventional RGB images, some unobvious quality character, which is even not visible to the human eyes, is impossible or difficult to detect. Unlike the conventional RGB images, whose spectrum information is very limited, the hyperspectral images contain extensive monochromatic image [2]. In one or several monochromatic images, the unobvious external quality characters can be very clear or easy to detect. Hyperspectral images are composed of numerous continuous wavebands for spatial position of an object studied.

Figure 5 illustrates the conceptual view of a hyperspectral image, which contains a stack of two-dimensional images one behind each other at different wavelengths and can be described as I(x, y, λ) [21]. The diagram shows that the raw hyperspectral cube consists of a series of contiguous sub-images one behind each other at different wavelengths [16], and each sub-image provides the spatial distribution of the spectral intensity at a certain wavelength. The hyperspectral images can be viewed either as a spectrum I (λ) at each individual pixel (x, y) or as an image I (x, y) at individual wavelength λ. Each image acquires spatially distributed spectral information at pixel level and can be used to analyze the biochemical constituent of a sample according to the spatial information. Each pixel containing a complete spectrum can be used to characterize the composition of that particular pixel.

Figure 5.

The conceptual view of a hyperspectral image with spectral and spatial domains.

2.5. Calibration of hyperspectral images

The hyperspectral imaging is a useful tool to acquire and record the raw hyperspectral information of fruits and vegetables. However, due to the differences in camera quantum and physical configuration of imaging systems, the uncorrected radiance for the different systems, even for the same system used in different times, might be very different for the same sample taken under the same condition [25]. Therefore, accurate calibrations for a hyperspectral imaging system are necessary to guarantee the stability and acceptability of the extracted hyperspectral image data and the consistent performance of the system. The original hyperspectral images can be calibrated into the reflectance mode based on black and white reference images. The hyperspectral reflectance images $R$ for a spatial pixel ($i$) at a given wavelength was calculated by using the following equation [26, 27].

$$R_i=\left(\frac{R{S}_i-R{D}_i}{R{W}_i-R{D}_i}\right)\times 100\%$$E1

where RS, RD, and RW are respectively the raw intensity values of identical pixels from the sample image, dark reference image, and white reference images. R_i is the calibrated hyperspectral image in a unit of relative reflectance. The dark reference image RD (with ~0% reflectance), which can be obtained with the light source turned off completely and the camera lens covered completely with its nonreflective opaque black cap, is used to remove the dark current effect of the area detectors [28]. The white reference image RW (with ~99% reflectance) represents the highest intensity values. RW can be acquired from a Teflon white surface under the same condition of the raw image.

3. Nondestructive assessment methods

The spectrum may be complicated by instrumental noise, complex chemical composition of products, environmental factors and other sources of variability [19]. As a consequence, spectral and image preprocessing and correction are necessary to improve the quality of the data before data analysis [29]. Moreover, the chemometrics is crucial for information extraction and better interpretation of the acquired data. The methods for spectral preprocessing and correction, optimal wavelength selection, and imaging processing and analysis models are introduced in detail in the following sections, as illustrated in Figure 6.

Figure 6.

The flowchart of analyzing methods with hyperspectral image.

3.1. Spectral analysis methods

3.1.1. Spectral preprocessing methods

The spectra of solid and scattering samples such as vegetables are influenced by physical properties such as shape, size, etc. This creates baseline shifts and noises in the spectra with broad wavelength regions when analyzing quality parameters [30]; thus, preprocessing of near-infrared (NIR) spectral data has become an integral part of chemometrics modeling. The goal of the preprocessing is to remove physical effects in the spectra in order to improve the subsequent multivariate regression, classification model or exploratory analysis. Selecting suitable preprocessing methods should always be considered in relation to the successive modeling stage. The whole data processing generally consists of the following several steps: spectral preprocessing, calibration model and model validation. A detailed description of data preprocessing methods can be found elsewhere [24, 31]. Some of the preprocessing methods are presented in the following sections.

3.1.1.1. Averaging

Averaging over spectra is generally performed during the acquisition spectrum to reduce the thermal noise of the detector. The number of scans depends on the application: the PDA spectrophotometer operates at a typical acquisition time of less than 50 ms, with almost no time to get multiple scans in the online classification, while the PDA spectrophotometer measurement time is less critical and can average several spectra without affecting the measurement throughput rate in the laboratory [32]. Averaging over wavelengths is used to smooth the spectrum or to reduce the number of wavelengths. Overall, most spectrophotometers may provide a better spectral resolution than the actual optical resolution.

3.1.1.2. Centering

For all practical purposes, it is recommended that data be centered or mean centered. The first stage in centering is often to subtract the average from each variable. The objective of centering is to ensure that all results will be interpretable in term of variation around the mean [32]. This is especially crucial if the variables differ significantly in their relative magnitudes, as the values with the greatest variance will be favored in regression analysis.

3.1.1.3. Smoothing

Smoothing is used to reduce high-frequency noise from the spectral data and signal-to-noise without reducing the number of spectral variables. Its principle is to acquire an optimal estimation value by averaging or fitting several points in a window. The broader the window is, the lower the spectral resolution would be [24]. Consequently, it is important to choose the window width properly. Smoothing improves the vision of the original spectra in addition to remove the useless information. Based on different smoothing fit methods, smoothing could be divided into moving average smoothing, Gaussian filter smoothing, median filter smoothing and Savitzky-Golay smoothing (S-G smoothing) [33, 34]. Different smoothing algorithms are adapted to different specific types of noise models. In other words, the appropriate smoothing algorithm should be selected flexibly according to the noise situation contained in the actual image.

3.1.1.4. Standard normal variate

Standard normal variate (SNV) is a row-oriented transformation which is capable of removing the multiplicative interferences from spectral caused by scatter and particle size effects from spectral data. SNV removes scatter effects by centering and scaling each individual spectrum [35, 36]. The method assumes that the absorbance of each wavelength point in the spectrum meets some certain distribution such as Gaussian distribution. Each spectrum can be calibrated based on this assumption. Firstly, the average value of a spectrum is subtracted from the original spectrum, and then the result is divided by the standard deviation [24]. This method is widely used when the variables are measured in different ranges or in different units, and it cannot be used for NIR spectroscopy because the noise from variables with small standard deviations may explode and lead to unreliable or incorrect models.

3.1.1.5. Multiplicative scatter correction

Multiplicative scatter correction (MSC) is a transformation method used to compensate for additive or multiplicative effects in spectral data [36, 37]. It is performed by correcting the scatter level of each to the level of an average spectrum. Similar to SNV, the objective of MSC is to eliminate the deviations caused by particle size and scattering [36]. The difference is that MSC standardizes every spectrum using the mean spectrum of all spectra while SNC use only the data from that spectrum. Therefore, for MSC effects on each spectrum alone, the correction capability of MSC is usually weaker than that of SNV. In SNV correction, each individual spectrum is normalized to zero mean and unit variance [32].

3.1.1.6. Derivative correction

Derivative is used to remove overlapping peaks and baseline shifts induced by the variation of particle sizes and instrumental conditions, so that more details within the spectra can be revealed [31, 32]. The first derivative of a spectrum is simply a measure of the slope of the spectral curve at every point [38, 39]. The slope of the curve is not affected by baseline offsets in the spectrum, and thus, the first derivative is a very effective method for removing baseline offsets. However, peaks in raw spectra usually become zero-crossing points in first derivative spectra, which can be difficult to interpret. The second derivative is a measure of the change in the slope of the curve. In addition to ignoring the offset, it is not affected by any linear that may exist in the data, and is therefore a very effective method for removing both the baseline offset and slope from a spectrum. The second derivative can help resolve nearby peaks and sharpen spectral features. Peaks in raw spectra change sign and turn to negative peaks with lobes on either side in the second derivative. Two commonly used spectral derivative approaches are Gap-Segment derivative and Savitzky-Golay (S-G) derivative [24].

3.1.1.7. Transformation

In spectral analysis, Fourier transformation (FT) and Wavelet transformation (WT) are often used for data compression, smoothing and filtering, as well as for the extraction of effective information. FT is a very important signal processing technique, which can realize the transformation between time domain functions and frequency domain functions. The principle of it is to decompose the original spectrum into the sum of sinusoidal waves of many varying amplitudes, frequencies and directions. WT is based on the idea of decomposing chemical signals into scale compositions according to their different frequencies by applying a basis function [24]. WT is similar to FT with a completely different merit function. The main difference is that FT decomposes the signal into sines and cosines; in contrary, WT uses functions that are localized in both the real and Fourier space [40].

3.1.2. Optimal wavelength selection methods

Due to the high resolution of modern spectroscopy instrumentations, the acquired spectral data set may have thousands of variables/wavelengths and hundreds or thousands of samples [41, 42]. Thus the hyperspectral imaging inspection algorithm will be very time-consuming due to the large-scale massive data. In order to simplify the complexity of computation, improve the efficiency of the detection, and meet the inspection speed required by the industry, variable selection (wavelength selection) is the most necessary and important step to select the optimal variables and remove the highly calibrated variables [43]. Many methods based on different criteria have been developed for this purpose. Some of them include competitive adaptive reweighted sampling (CARS), random frog (RF), successive projections algorithm (SPA), genetic algorithm (GA) and uninformative variables elimination (UVE) which can be implemented prior to the construction of both regression and classification models.

3.1.2.1. Competitive adaptive reweighted sampling (CARS)

Competitive adaptive reweighted sampling (CARS) is a novel wavelength selection algorithm employing the “survival of the fittest” principle from Darwin’s evolution theory [44]. It is originally developed to select informative wavelengths from contiguous spectral data, specifically applied for the first time to NIR spectroscopy. The method selects wavelength subsets sequentially from the sampling runs in an iterative manner. It basically consists of a number of iterations involving [45]: (1) Monte Carlo (MC) model sampling, (2) wavelength reduction by exponentially decreasing function (EDF), (3) wavelength reduction by adaptive reweighted sampling (ARS), and (4) model building with each subset of selected variables and CV to calculate prediction error. Figure 7 shows the scheme of the CARS algorithm. For each MCS run or iteration, the four steps mentioned above will be repeated, obtaining an error for each one. Finally, the subset with the lowest RMSECV value will be determined as the optimal subset [46]. The key wavelengths selected by CARS are considered as the wavelengths with the large absolute regression coefficients in a multivariate linear regression model. The exponential decay function is used to control the retention rate of variable in the algorithm, and it has the potential to select an optimal combination of the wavelengths.

Figure 7.

The flowchart of the competitive adaptive reweighted sampling algorithm.

3.1.2.2. Random frog (RF)

Random frog (RF) algorithm is a useful wavelength selection technique based on the framework of reversible jump Markov chain Monte Carlo (MCMC) or the multiple decision trees. Like CARS, it works in an iterative manner; meanwhile, it calculates the selection probability for each variable. Briefly, random frog works in three steps [47, 48]: (1) initializing randomly a variable subset V₀ containing Q variables; (2) generating a candidate variable subset V* including Q* variable; accept V* as V₁ with a certain probability and let V₀ = V₁; repeat the above procedures until N iterations are finished; and (3) computing a selection probability of each variable which can be used as a measure of variable importance. The schematic is shown in Figure 8. The advantage of random frog is that it does not require any rigorous mathematical formula. And it do not need to use the previous distribution in formal reversible jump MCMC methods, which makes it easier to implement and compute. There are five tuning parameters to control the RF performance, which can be optimized in the routine. The two most important parameters are the number of variables contained in the number of iterations and the initial variable set.

Figure 8.

The flowchart of the random frog algorithm.

3.1.2.3. Successive projections algorithm (SPA)

The successive projections algorithm (SPA), a forward selection method which uses simple operations in a vector space to minimize variable collinearity, is a novel variable selection strategy in hyperspectral image analysis for multivariate calibration [49, 50]. The main purpose of SPA is to select wavelengths with minimal redundancy [43]. In summary, the steps to execute SPA are: (1) carrying out projections on the N matrix and generating K chains of M variables each, (2) evaluating candidate subsets of variables extracted from the chains generated in the first phase, and (3) eliminating procedures aimed at discarding uninformative variables without significant loss of prediction capability. Many successful applications have proven SPA to be an outstanding variable selection approach.

3.1.2.4. Genetic algorithm (GA)

The Genetic algorithm (GA) is an effective globe searching algorithm. Based on a fitness function, GA is an iterative process starting from a population of randomly generated individuals and achieves optimal solutions through genetic operations including crossover, selection and mutation [24]. The steps of GA involved are [51]: (1) building an initial population of variable sets by setting bits for each variable randomly, (2) fitting a PLS regression model to each variable set and computing the performance, (3) a collection of variable sets with higher performance are selected to survive, (4) crossover and mutation, (5) the surviving and modified variable sets from the population. Through such operation, irrelevant spectral information is eliminated and the number of spectral variables is reduced.

3.1.2.5. Uninformative variable elimination (UVE)

The uninformative variable elimination (UVE) is a method for variable selection based on an analysis of regression coefficient of PLS. The UVE method was employed by Sun et al. Readers are referred to the corresponding references for details about many effective variable selection methods [52]. The method builds a large number of models with randomly selected calibration samples at first, and then each variable is evaluated with a stability of the corresponding coefficients in these models. Variables with poor stability are known as uninformative variable and are eliminated [53].

3.1.3. Calibration models

Multivariate regression techniques (quantitative analysis) aim to establish a relationship between the observed response values and spectral matrix. In our research, partial least squares (PLS) regression is a common multivariate method used in calibration of spectroscopy data. The principle of PLS is to use a linear least squares fitting technique. It builds linear models between an independent matrix X (spectral data) and a dependent matrix Y and estimate the regression coefficient matrix using least squares fitting techniques. Least squares support vector machines (LS-SVM) can deal with nonlinear relationships between variables.

3.1.3.1. Partial least squares (PLS)

Partial least squares (PLS) analysis is widely used for calibration in present chemometric analysis. It is an unsupervised statistical method used when not only a data array coming from X data is available but also a Y array that we want to predict from our X data [32]. Normally, there are two variable selection methods using PLS regression: using variable importance in projection scores and using regression coefficients estimated by PLS regression [54, 55]. The aim of PLS analysis is to find a latent variables linear regression model by projecting the X variables and the Y variables into a new latent space, where the covariance between these latent variables is maximized [1]. PLS analysis can be performed to establish the regression model leading to the content prediction of chemical components. PLS considers simultaneously the variable matrix Y (the values of SSC, pH) and the variable matrix X (the spectral data). Generally, the first step in PLS is to decompose the matrix, and the model is given:

$$X=\mathit{TP}+E$$E2

$$Y=\mathit{UQ}+F$$E3

In these equations, X is a n × m spectral matrix (n is the number of samples, m is the number of wavelengths), T and U are the score matrices of X matrix and Y matrix, P is the m × k matrix of X matrix and Q is the loading (l × k), and y is the reference data (n × l) that needs to be predicted from X (k is the number of latent variables), and E and F are the errors which come from the process of PLS regression [43]. The second step is that T and U are processed by linear regression. It must build the following linear correlation:

$$U=\mathit{BT}+E$$E4

where B represents the internal relations between U and T. In order to reach this object this object, the coordinate of T is rotated.

3.1.3.2. Least square support vector machine (LS-SVM)

Least square support vector machine (LS-SVM) is a set of related supervised learning method that analyzes data and recognizes patterns, and is used for classification and regression analysis. PLS method can only handle linear problems and builds a linear relationship between spectral variables and target chemical response such as SSC value. However, some researchers reported that the latent nonlinear information might be existed in the spectra data of fruit and the nonlinear models were better than linear models. The computational complexity and quality of the SVM does not directly depend on the dimension of input data. Therefore, least square support vector machine (LS-SVM) was applied to build a nonlinear model for a comparison of the prediction performance with linear PLS models. LS-SVM is widely applied in pattern recognition and function regression for the advantage of limited over-fitting, high predictive reliability and strong generalization ability [24]. More details of LS-SVM method can be found in the paper [56, 57]. The final LS-SVM regression model can be expressed as:

$$y\left(x\right)={\displaystyle \sum _{k=1}^N}{a}_{k}K\left(x,x_k\right)+b$$E5

where K(x, x_k) is the kernel function, x_k is the input vector, α_k is the Lagrange multiplier called support value, and b is the bias. The radial basis function (RBF), which is a frequently used kernel function K(x, x_k), is used in this study and defined as follows:

$$K\left(x,x_k\right)=\mathit{exp}\left(-\frac{{‖x_k-x‖}^2}{2{\sigma }^2}\right)$$E6

In the equation, ‖x_k-x‖ represents the distance between input vector and threshold vector, and σ is a width vector. Generally, the selected variables by wavelength selection methods could be used as the inputs to build the LS-SVM models.

3.1.4. Model validation

Validation procedures are crucial to assess the accuracy of the calibration and to avoid over-fitting. The prediction ability of a calibration model can be evaluated by the correlation coefficient (r), root mean square error of prediction (RMSEP) and calibration (RMSEC) between the predicted value and the measured value in validation set [24]. In order to establish useful calibration models, different methods in spectral preprocessing and calibration modeling as mentioned above should be investigated. When cross validation is employed, the prediction performance could also be assessed by the root mean square error for cross validation (RMSECV). These indices are defined as follows:

$$r=\sqrt{\frac{\sum _{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2}{\sum _{i=1}^n{\left({\widehat{y}}_i-y_{\mathit{\text{mean}}}\right)}^2}}$$E7

$$\mathit{\text{RMSEC}}=\sqrt{\frac{1}{n_c}{\displaystyle \sum _{i=1}^{n_c}}{\left({\widehat{y}}_i-y_i\right)}^2}$$E8

$$\mathit{\text{RMSEP}}=\sqrt{\frac{1}{n_p}{\displaystyle \sum _{i=1}^{n_p}}{\left({\widehat{y}}_i-y_i\right)}^2}$$E9

$$\text{bias}=\frac{1}{n}{\displaystyle \sum _{i=1}^n}\left({\widehat{y}}_i-y_i\right)$$E10

where, ${\widehat{y}}_i$ is the predicted value of the ith observation, y_i is the measured value of the ith observation, y_m is the mean value of the calibration or prediction set, n, n_c, and n_p are respectively the number of observations in the data set, calibration and prediction set. Generally, a good model should have higher correlation coefficients, lower RMSEC, RMSEP, and bias values [58, 59].

3.2. Image processing and analysis techniques

Image processing and image analysis are considered to be the core of the hyperspectral imaging system with various algorithms and methods available to complete the specific classification and measurement. As illustrated in Figure 9, image processing and analysis are performed in three levels. The low level processing is the basic processing of image, which involves image acquisition and image preprocessing, and is the important step in image processing and analysis, which involves image segmentation, feature extraction, representation, and description [60]; the high level processing is the key step of image analysis, which involves recognition, interpretation and classification [2].

Figure 9.

Different levels of image processing.

3.2.1. Image processing methods

The assessment accuracy of fruits and vegetables quality is highly related to the images. However, owing to the imperfections of the image acquisition systems, the images acquired are subject to various defects that will need subsequent processing. Image processing plays an important role in hyperspectral data analysis. The image processing involves a series of steps, which can be divided into three major steps: image preprocessing, segmentation and feature extraction [61].

The purpose of image preprocessing and calibration is to improve the quality of the obtained images by removing the noise, increasing the contrast and correcting the distortion [2]. Generally, the frequently used preprocessing methods include basic point operations (intensity mappings) and histogram equalization [43]. Basic point operations, such as luminance inversion and multiplicative brightness scaling, can improve by stretching the brightness levels into a mapping between the input level and the output level. Histogram equalization provides a sophisticated method for modifying the dynamic range and contrast of an image by changing the image so that its intensity histogram has a desired shape. Histogram model use nonlinear and nonmonotonic transfer functions to map the pixel intensity values of input and output images. Other typical image preprocessing techniques include filtering, transformation and arithmetic operations.

Image segmentation is the most vital and challenging step to partition the image into regions of interest (ROI). The goal of image segmentation is aimed at simplifying and altering the representation of an image into something more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries (lines, curves, etc.) in images. The accuracy of image segmentation plays an important role in the subsequent image processing. Threshold-based segmentation, edge-based segmentation, region-based segmentation, and classification-based segmentation are four major types of segmentation methods [62, 63, 64].

Feature extraction is a key step in connecting image processing and analysis, which converts image data or segmented regions into a set of feature vectors. In image processing, feature extraction builds features intended to be informative and nonredundant, facilitating the subsequent learning and generalization step [65]. When the image segmentation is successfully performed, if the data in ROI to an algorithm is too large to be processed, it can reduce its dimensionality. Feature extraction is related to dimensionality reduction. Thus, feature extraction is crucial to the accuracy of quality assessment. In general, shape features, texture features, color features and size features of the target are typically extracted for quality assessment.

3.2.2. Image analysis methods

Image analysis is a nondestructive method of calculating measurements and statistics based on the interesting values of images’ pixels, and their corresponding spatial location. The image analysis is performed on the feature extracted from the image, and interprets the results. It uses intuitive explanations to display images and mathematically processing images, helping to solve the problem of computer vision. Vision measurement and pattern classification are the most crucial parts of image analysis.

Vision measurement is a quantitative analysis method in the image analysis. Visual measurement is the process of quantifying the parameters of interest from the features extracted from the image. It is the process of quantitative measurement of interest parameters based on the characteristics extracted from the image [66]. The computer vision systems can achieve different types of measurements. Generally, typical measurements include the size, texture and color.

Pattern classification, also known as pattern recognition, is a method for qualitative analysis in the image analysis. It is the science of reasoning based on measurement characteristics through probabilistic, statistical, computational geometry, multivariate analysis and algorithm design techniques. The classification techniques can be divided into two types: supervised methods and unsupervised methods. In the image analysis, the supervised methods are the most widely used. In most cases, the supervised classification method aims to build a model or a classifier for classification of labels according to the corresponding characteristics, while the unsupervised classification method is mainly used to classify image by finding out similarity between the selected features and using clustering algorithm. The widely used pattern classification methods in image analysis include Artificial Neural Network (ANN), Support vector machine (SVM), K-Nearest Neighbor (KNN), Adaptive Boosting, and decision tree. ANN is a nonlinear statistical data modeling tool that attempts to mimic the fault tolerance and capacity to learn biological neural systems by modeling the low-level structure of the brain [1]. ANN is widely used in hyperspectral image analysis, because it can handle a large amount of heterogeneous data with considerable flexibility and nonlinearity. It is composed of a set of interconnected artificial neurons, which are like a parallel system that capable of resolving the paradigm that linear computing cannot. SVM is a supervised nonparametric statistical learning model with associated learning algorithms that analyze data and recognize patterns, used for classification and regression analysis. In addition to performing linear classification, SVM can use the so-called kernel technique to efficiently perform nonlinear classification and map its inputs implicitly into high-dimensional feature spaces. As SVM, AdaBoost is one of the most successful supervised classification methods with the aim to maximize the minimum margin of a training sample [2]. KNN is another unsupervised classification method which is able to predict the response of the new sample by analyzing a certain number of the nearest neighbors in the feature space of the sample. In KNN, dataset is classified by minimizing the sum of squares of distances between each category and the corresponding cluster centroid [67]. Decision trees are commonly used in hyperspectral image analysis, to help identify a strategy that is most likely to reach a goal.

4. Applications in the quality assessment of fruits and vegetables

4.1. Applications of surface defect detection

The presence of surface defects influences the quality and price of fruits and vegetables, and weeding out the fruits and vegetables with serious defects early can prevent the infection of the whole patch. Therefore, detection of surface defects is the most commonly extended application of image and spectral analysis to the external quality inspection of fruits and vegetables.

Visual inspection of fruits and vegetables with respect to color, texture, size, and shape by traditional computer vision is already automated in the commercial sorting machines. However, sorting by defects is still a challenging task due to the high variance of defect types and existence of stem/calyx concavities [68]. The color, texture, or internal components of defects may be different from that of the sound; therefore, color, texture, or spectral reflectance are usually selected as the defect features to discriminate the defects from the sound peel. Many applications aimed to detect defects based on these features have been described by using hyperspectral or multispectral imaging system.

Due to lack of spectral information in conventional color images, traditional computer vision system is not efficient for the inspection of some defects with similar color and texture as sound peel, such as bruises, rottenness, or chilling injury. Hyperspectral and multispectral imaging systems provide powerful tools not only to detect skin defects but also to differentiate between a variety of defects that have similar color and texture or even to detect some defects that are not clearly visible [1]. Bruising is one of the familiar defects occurring on fruits and vegetables during post-harvest handling and processing stage. The existing commercial sorting machines are still not available in detecting bruises [69, 70]. An experiment of using a hyperspectral imaging system for bruise detection on apples was conducted by Xing et al. [70]. PCA and PLSDA were used to extract the spectral and spatial features from the hyperspectral images in the region between 400 and 1000 nm. Their experiment proved that combination of image processing and chemometric tools had a potential in detecting the bruises on apples. In order to detect the early bruises in apples, Baranowski et al. [69] proposed a system that included hyperspectral cameras equipped with sensors working in the visible and near-infrared (400–1000 nm), short-wavelength infrared (1000–2500 nm) and thermal imaging camera in mid-wavelength infrared (3500–5000 nm) ranges. Results showed that the principal components analysis (PCA) and minimum noise fraction (MNF) analyses of the images could make it possible to distinguish between areas with defects in the tissue and the sound ones, and the fast Fourier analysis of the image sequences after pulse heating of the fruit surface could provide additional information not only about the position of the area of damaged tissue but also about its depth. As unsupervised methods, the class number and the color or intensity value for each class are always randomly assigned by PCA and MNF. The robustness and stability of their algorithms are needed to be tested in inline inspection situation.

Decay is another common defect with great potential risk for consumers, sellers and growers. To fast detect and visualize the early decay in citrus, Li et al. [71] developed multispectral image processing method with mean normalization reducing spectral variability due to spherical fruit. The overall accuracy of 98.6% for test set with no false negatives was achieved. Their idea behind the proposed algorithm can be extended to detect the nonvisible damages of other fruit. Gómez-Sanchis et al. [72] presents the development of a hyperspectral system based on two liquid crystal tuneable filters for the acquisition of images of spherical fruits. They also designed a system that allows the filters to be exchanged quickly and without altering the acquired scene. The system and decay segmentation results are shown in Figure 10. Correctly classifying 98% of pixels as rotten or nonrotten tissues were achieved; however, changing the filters frequently decreases the detection efficiency, especially when working in the sorting line, the rotating products might cause the acquired scene vary with each of the filters.

Figure 10.

The system and decay segmentation results proposed by Gómez-Sanchis et al. (a) Hemispherical illumination chamber used to illuminate spherical objects (b) System created to facilitate the swap of two LCTF filters (c) RGB images and segmented images of mandarins with decay lesions [72].

Chilling injury is a common defect occurring during the storage and transportation at low temperatures. Liu et al. [73] developed a hyperspectral imaging system to detect the chilling injury in cucumber by using band ratio and PCA methods. Results revealed that either band ratio algorithm (Q811/756) or PCA transform in a spectral region between 733 and 848 nm could detect the chilling injury with an accuracy of over 90%. Ariana and Lu [74] found that the hyperspectral imaging under transmittance mode has shown potential for detecting internal defect. However, the technique still cannot meet the online speed requirement because of the need to acquire and analyze a large amount of image data. They determined up to four-waveband subsets by a branch and bound algorithm combined with the k-nearest neighbor classifier. The highest classification accuracies of 94.7 and 82.9% were achieved for cucumbers and whole pickles, respectively.

However, the acquisition and processing of the hyperspectral images is time-consuming, and the redundancy data makes the hyperspectral imaging system impossible to be used in-line or real-time. Actually, the hyperspectral imaging is always used for analysis and determining the effective wavelengths for a multispectral imaging system. Based on hyperspectral images and PCA, four efficient wavelengths (558, 678, 728, and 892 nm) were selected, and then a multispectral imaging system was developed by Xing et al. [75] to detect the bruises on apples. An overall accuracy of about 86% was obtained with their systems and algorithms. A near commercial multispectral imaging prototype for inline bruise detection was developed by Huang et al. [76] in NERCITA, China. Segmented principal component analysis (PCA) was conducted to eliminate data redundancy and select optimal wavelengths. Two dichroic beamsplitters, two band-pass filters with the center at selected wavelengths and three prism-based 2CCD multispectral progressive area scan cameras were used to develop the multispectral imaging system. Static and online tests were evaluated by their system, and 91.5% and 74.6% overall accuracy were achieved for static and online detection, respectively.

Table 1 shows a detailed summary of studies about the defect detection of fruits and vegetables by using hyperspectral imaging systems.

Products	Species	Applications	Types of CVS	Methods	Accuracy	Reference
Fruits	Apple	Quality grading	MIS	Statistical and syntactical classifiers	93.5%	[81]
	Apple	Defect segmentation	MIS	ANN	–	[68]
	Apple	Quality evaluation	MIS	Flat-field correction	95%	[82]
	Apple	Bruise detection	HIS	PCA, MNF, SIMCA, LDA, SVM	–	[69]
	Apple	Defect detection	HIS	ASD	–	[83]
	Apple	Defect and feces detection	MIS	_	–	[84]
	Apple	Bruise detection	HIS	PLS, SDA	93.95%	[85]
	Apple	Efficient wavelength selection	MIS	QDA	–	[86]
	Apple	Rottenness detection	HIS	LDA, CART	91.2%	[87]
	Apple	Bitter pit detection	HIS	PLS	–	[88]
	Apple	Defect detection	MIS	ANN	95.4%	[89]
	Apple	Defect detection	HIS	BR	99.5%	[90]
	Apple	Defect detection	HIS	SD, PCA	–	[83]
	Apple	Decayed spot, wound and rot detection	MIS	BR	92.42%	[91]
	Apple	Bruise detection	HIS	PCA, PLSDA	86%	[70]
	Apple	Bruise detection	MIS	PCA, MT	86%	[75]
	Apple	Bruise detection	HIS	PCA	>77.5%	[75]
	Apple	Bruise detection	HIS	PCA, MNT	88%, 94%	[92]
	Apple	Defect detection	MIS	Rotating	90%	[93]
	Apple	Defect detection	MIS	PCA, ANN	79%	[94]
	Apple	Chilling injury detection	HIS	ANN	98.4%	[95]
	Citrus	Canker detection	MIS	BR, T	95.3%	[96]
	Citrus	Skin damage detection	MIS	Bayesian discriminant analysis	86%	[97]
	Citrus	Common defect detection	HIS	PCA	93.7%	[98]
	Citrus	Light correction	HIS	Light correction	–	[87]
	Citrus	Rottenness detection	HIS	ANN, DT	98%	[99]
	Pear	Bruise detection	HIS	PCA, MLC, EDC, MDC, SAM	93.8–95%	[100]
	Strawberry	Bruise detection	HIS	LDA, ND, ANN	100%	[101]
	Cherry	Pit detection	HIS	NN	97%	[102]
	Jujube	Insect infestation detection	HIS	JMP, MA	97%	[103]
Vegetables	Cucumber	Bruise detection	HIS	PCA, BR	75-95%	[89]
	Cucumber	Chilling injury detection	HIS	PCA, FLD	91%	[104]
	Cucumber	Chilling injury detection	HIS	BR, PCA	>90%	[75, 105]
	Mushroom	Bruise detection	HIS	PCA	79-100%	[106]
	Mushroom	Freeze damage detection	HIS	PCA, LDA	95%	[107]
	Mushroom	Enzymatic browning	HIS	PLS-DA	–	[108]
	Onion	Sour skin disease detection	HIS	MS	–	[109]

Table 1.

Summary of studies about the defect detection of fruits and vegetables.

HIS: hyperspectral imaging system; MIS: multispectral imaging system; BR: band ratio; MS: mean reflectance spectra; ASD: asymmetric second difference; MT: moments thresholding; and T: thresholding.

4.2. Applications of internal quality parameter measurement

4.2.1. Soluble solids content (SSC)

Soluble solids content, also named total soluble solids (TSS) content, is a collective index for sweetness measurement [77]. In the preharvest period, SSC profoundly dominates the optimal harvest time for various fruits and vegetables, whereas changes of SSC during the shelf-life period after harvesting would lead to quality fluctuation of fruits and vegetables [77]. Therefore, soluble solids content is an important internal quality attribute in determining fruit maturity and harvest time, and in assessing and grading post-harvest quality of apples [78].

In the past 20 years, many studies have been reported on predicting SSC in fruits using near-infrared spectroscopic technique. Leivavalenzuela et al. [79] made a report on the application of hyperspectral imaging technique for predicting the SSC of blueberries in the visible and short-wave near-infrared region of 500–1000 nm. In this study, Calibration models using partial least squares method were developed to predict the SSC, and the effect of fruit orientation on the model performance was evaluated. Results showed that hyperspectral imaging is promising for online sorting and grading of blueberries for firmness and perhaps SSC as well. Mendoza et al. [80] developed two different hyperspectral imaging systems: a stationary hyperspectral imaging system and a prototype on-line hyperspectral imaging system to evaluate the SSC in apples. The work used several methods, including discrete and continuous wavelet transform and conventional image texture analysis. Finally, the results showed that the integration of spectral and image features for hyperspectral scattering technique significantly improved firmness and SSC prediction (by the t-test) for all three cultivars but with a lesser degree of pronouncement for SSC.

Peng et al. [78] did a research on the hyperspectral imaging system for predicting soluble solids content (SSC) of “Golden Delicious” apples which was calibrated both spectrally and spatially. Their proposed methods, evaluating and comparing different mathematical models for describing the hyperspectral scattering profiles over the spectral region between 450 nm and 1000 nm coupled with the scattering profile correction methods, could improve the hyperspectral scattering technique for measuring fruit quality; and the study also showed the modified Lorentzian distribution function with three parameters without including the parameter for the asymptotic value which was most appropriate for predicting both fruit firmness and SSC. Rajkumar et al. [110] at three different temperatures used a hyperspectral imaging technique in the visible and NIR regions (400–1000 nm) to study bananas’ SSC. Some quality parameters like moisture content were also determined and correlated with the spectral data using PLS. Their proposed methods, coupled with the scattering profile correction methods, could improve the hyperspectral scattering technique for measuring banana fruit quality.

Applications of hyperspectral imaging in fruit and vegetable SSC measurement could also be found in other crop products, such as strawberries, pears and so on [111, 112].

4.2.2. Firmness

Firmness is an important textural attribute for fruits and directly influences their shelf life and consumer acceptance, and it is an important internal quality attribute in determining fruit maturity and harvest time, and in assessing and grading post-harvest quality of apples. Thus, nondestructive sensing of fruit firmness would provide the fruit industry with a mean to ensure the quality and consistency of individual fruit, increase consumer satisfaction, and thus improve industry profitability [113].

Peng and Lu [113] proposed ten modified Lorentzian distribution with three parameters to characterize spatial scattering profiles from scattering images for Golden Delicious apples. A multilinear regression analysis was performed to predict the relationship between parameters of the scattering profile and the firmness of apples. This new method, coupled with the scattering profile correction methods, improved the hyperspectral scattering technique for measuring fruit and vegetable quality. Fan et al. [114] acquired hyperspectral reflectance image from each pear in visible and near-infrared (400–1000 nm) regions by employing the hyperspectral imaging system to determine SSC and firmness of pears. In this study, the variables selected by SPA, CARS and the combination of CARS and SPA were used for PLS regression. The overall results indicated that the CARS-SPA was an effective way for the selection of effective variables and the hyperspectral imaging system together with CARS-SPA-PLS model could be applied as a fast and potential method for the determination of SSC and firmness of pear. Qin et al. [115] measured the absorption and reduced scattering coefficients of apples through a spatially-resolved hyperspectral imaging technique and related them to fruit firmness. This research demonstrated the potential of using spectral absorption and scattering properties to evaluate internal quality attributes of horticultural products.

4.2.3. Acidity/pH

The acid content is often determined by a titratable method. A common method used for measuring ethylene production is to extract a gas sample from the internal core space of fruit or from a sealed container, in which fruits have been kept for a period of time and then analyzed using gas chromatograph [116]. The quality of fruit or vegetables is determined by a series of properties, such as acidity, which makes them attractive to consumers, is very crucial.

Cayuela et al. [117] described a portable AOTF-NIR spectrophotometer with a wide spectral range between 1100 and 2300 nm, which was equipped with a reflectance post-dispersive optical configuration and an InGaAs detector used for NIR prediction of fruit moisture content and free acidity. ElMasry et al. [111] determined acidity in strawberries by feat of a visible/NIR hyperspectral imaging system (400–1000 nm). It was found that the spectral pretreatments of mean-centering and automatic baseline correction enhanced PLS calibration model when compared with others pretreatments, such as Savitzky-Golay smoothing, MSC, and first and second derivatives.

Rungpichayapichet et al. [118] proposed a new hyperspectral imaging technology using a newly developed frame camera which was applied to determine internal properties of mango fruits including firmness, total soluble solids (TSS) and titratable acidity (TA). In their study, prediction models were developed using spectral data from relative surface reflectance of 160 fruits in the visible and NIR region of 450–998 nm analyzed by PLS regression. From their results, HSI can be used as a nondestructive technique for determining the quality of fruits which could potentially enhance grading capabilities in the industrial handling and processing of mango. Baiano et al. [119] carried out acidity determination in 7 cultivars of table grapes using NIR HSI with PLS models performing on the mean-centering correction spectra, and they achieved the coefficients of determination for predicting titratable acid and pH of red grapes and white grapes. They concluded that spectra information was not correlated with the sensory data, making hard prediction of attribute perception.

In addition to these fruits, the application of hyperspectral images acidity with broader range of 1000–2300 nm was acquired for the determination of total fat in beef cuts with good prediction abilities [120]. In other study, Abdel-Nour et al. [121] applied hyperspectral transmittance imaging (900–1700 nm) to classify eggs into three types with different docosahexaenoic acid contents using K-means analysis, resulting in 100% classification accuracy. Liu and Ngadi [122] detected fertility and early embryo development of chicken eggs using near-infrared hyperspectral imaging.

4.2.4. Moisture/water content

A fruit or vegetable consists of many different constituents, where water is the major component in fruits and vegetables [16]. Moisture content influences the taste, texture, weight, appearance, and shelf life of fruits and vegetables. Therefore, even a slight deviation from a defined standard can adversely impact the physical properties of a fruit or vegetable. For these reasons, the analysis to the moisture content of food products has a critical impact on quality and safety features [123].

Recently, hyperspectral imaging has also been used for determining the water content of other large variety of fruits and vegetables. Mollazade et al. [124] evaluated the potential of hyperspectral imaging combined with artificial neural networks to predict the moisture content in tomato fruit and to obtain spatial distribution maps. Their works displayed the spatial distribution of moisture content as a color map, where colors represent different values of predicted attribute. Finally, result showed that the feasibility of the method for characterizing the spatial distribution of an attribute in horticultural produce. Dong and Guo [125] proposed a hyperspectral reflectance imaging technology in near-infrared regions (900–1,70,002 nm) to evaluate soluble solids content (SSC), firmness, moisture content, and pH values of “Fuji” apples. They employed PLS regression, LS-SVM and back propagation (BP) network modeling methods to establish models to predict SSC, firmness, MC, and pH of apples, respectively. Results indicated that the moisture content could be predicted exactly by all developed models.

Firtha et al. [126] described an approach for the prediction of moisture content in carrot tissue. The work reduced the data load of hyperspectral experiments by using sample-specific vector-to-scalar operators for real-time feature extraction and a systematic procedure for compensating for pixels in the NIR sensor. Results demonstrated that the approach to predict the moisture content of carrots is feasible. Except what we mentioned above, hyperspectral imaging can be applied on the moisture content of all kinds of fruits and vegetables such as strawberries and soybean [24, 127].

4.2.5. Starch content

Starch is the main form of carbohydrate in our food, which is present in a variety of grains, vegetables and fruits. During the ripening of fruit, starch is changed into sugar, which gives sweetness to ripe fruits [128]. The harvest time of fruits, matching the desired commercial characteristics, is assessed through starch-iodine test in practice [129].

Peirs et al. [130] employed a threshold value of the first principal component score image to measure the starch distribution and starch index of apple fruit during maturation. Results showed that the starch concentration obtained in each position of the fruit was continuously measured compared with the discrete values obtained with the traditional technique. The method that they are proposed will speed up the application while the purchase costs decrease considerably and can be considered as a model system to map quality attributes of fruits. Menesatti et al. [129] researched the relationships of near-infrared (NIR) spectral images, starch/starch-free patterns visually assessed and RGB color images through PLS-DA to assess the starch index of apples. They studied the spectral region between 1000 and 1,70,002 nm through PLS-DA to assess the starch index in apples. Their proposed methods, avoiding expert’s subjective interpretation of starch index assignment, show the feasibility of NIR imaging spectroscopy as a tool for fruit maturity determination.

Chen et al. [131] studied nondestructive detection of starch content in potatoes using the SPA-MLR model and SPA-PLSR model, respectively. Results showed that the effect of the SPA-MLR model was superior to that of the SPA-PLSR model. Trong et al. [132] employed the starch index to estimate the optimal cooking time of potatoes. The changes caused by the microstructure and composition of starch affected the interaction of light with the starch granules at different regions inside the potatoes. In their research, the potential of hyperspectral imaging in the wavelength range of 400 nm to 1000 nm in combination with chemometric tools and image processing for contactless detection of the cooking front in potatoes has been investigated.

4.2.6. Ripening/maturity stages

The definition of apple maturity corresponding to the stage of fruit development, giving minimum acceptable quality to the ultimate consumer, implies measurable points in the commodity’s development and the need for techniques to measure maturity [133]. In addition, concerning the internal quality, maturity is extremely important to determine the harvest time and optimize the post-harvest treatment and environment [1, 16].

In recent years, many works on the determination of the maturity of fruits have been reported. An example of such studies is that of Rajkumar et al. [110] who studied banana fruit quality and maturity stages at three different temperatures by using hyperspectral imaging technique in the visible and near-infrared (400–1000 nm) regions to determine the quality parameters like moisture content. In their research, they concluded that the change in TSS and firmness of banana fruits stored at different temperatures during the ripening process followed the polynomial relationships and the change in moisture content followed a linear relationship at different maturity stages. And Garridonovell et al. [134] evaluated the potential of RGB digital imaging and hyperspectral imaging for discriminating maturity level in apples. In their research, segmentation, preprocessing and PLS-DA are applied to hyperspectral data analysis, while illumination correction, dimensionality reduction and linear discriminant analysis (LDA) are applied to RGB data analysis. Finally, they concluded that hyperspectral discrimination classified different storage regimes better than RGB.

Herrerolangreo et al. [135] developed an automatic procedure which is able to classify commercial peaches according to their maturity stage through multispectral imaging techniques. They proposed and validated the process of evaluating peach maturity through spectral imaging, which is very crucial for ensuring its quality of optimum peach ripeness. The proposed method is nondestructive and quick, and thus, it will have a good perspective for its application in fresh fruit packing lines. Girod et al. [136] introduced a nondestructive and quick technique that can measure the DM content to assess the maturity of avocados. The work analyzed avocado fruits at different maturity stages through hyperspectral imaging in reflectance and absorbance modes. The proposed method indicated that the reasonably accurate models could be obtained for DM content with the entire spectral range. Applications of hyperspectral imaging to measure maturity stages of fruit and vegetable could also be found in pawpaws, tomatoes and grapes [1, 137, 138].

5. Conclusions

Over the past decades, hyperspectral imaging technique has been rapidly developing and widely applied in nondestructive fruit and vegetable quality assessment. This chapter provides the principles, developments and applications of hyperspectral imaging technology in the quality inspection of fruits and vegetables. The principal components, basic theories and corresponding processing and analytical methods are also reported in this chapter. Looking into the future of fast inline sorting industry, hyperspectral imaging faces both challenges and opportunities. The challenges include the influence of physical and biological variability, whole surface detection, discrimination between defects and stems/calyxes, unobvious defect detection, robustness of the features and algorithms, as well as rapid multispectral imaging system development. Though many solutions have been presented to solve the challenging problems in fruit and vegetable quality inspection by using hyperspectral imaging technique in previous studies by the scientific researchers worldwide, the challenges presented above will continue to be intractable problems for a long time.