Optical Methods for Firmness Assessment of Fresh Produce: A Review

Abstract

This chapter is devoted to a review of optical techniques to measure the firmness of fresh produce. Emphasis is placed on the techniques that have a potential for online high-speed grading. Near-infrared spectroscopy (NIRS) and spatially resolved reflectance spectroscopy (SRRS) are discussed in detail because of their advantages for online applications. For both techniques, this chapter reviews the fundamental principles as well as the measured performances for measuring the firmness of fresh produce, particularly fruit. For both techniques, there have been studies that show correlations with penetrometer firmness as high as r = 0.8 − 0.9. However, most studies appear to involve bespoke laboratory instruments measuring single produce types under static conditions. Therefore, accurate performance comparison of the two techniques is very difficult. We suggest more studies are now required on a wider variety of produce and particularly comparative studies between the NIRS and SRRS systems on the same samples. Further instrument developments are also likely to be required for the SRRS systems, especially with an online measurement where fruit speed and orientation are likely to be issues, before the technique can be considered advantageous compared to the commonly used NIRS systems.

1. Introduction

Firmness is a major quality parameter in grading fresh produce, governed by the mechanical and structural properties of fruit. For producers, it can indicate ripeness and/or storage potential, and for consumers, it directly influences consumer acceptance and satisfaction. The industry standard instrument for firmness assessment is a penetrometer, which drives a metal plunger into the fruit flesh and records the maximum resistance force. This technique has three main drawbacks [1]: it is destructive, leaving the fruit unsaleable, measurements are highly variable (up to 30%) and it cannot be used in online situations. A fast and nondestructive technique would be desirable for the fresh fruit industry as it offers the benefit of grading and sorting each individual fruit.

Firmness has been a difficult parameter to measure by fruit graders. To date, no commercially successful nondestructive system has been created on a high-speed grader. Most prior research has focused on mechanical methods such as acoustic resonance, impact response, and force-deformation [2–6]. Most of the mechanical methods require contact with the fruit, which limits the grading speed due to the difficulty of achieving reliable physical contact and consistent fruit compliance at high speeds. It also potentially causes physical damage to the fruit. Moreover, mechanical methods are sensitive to each method’s specific mechanical property such as deformation force, so they often do not correlate well or consistently with the penetrometer. For this reason, the industry is reluctant to adopt these methods [7]. This has led to more research into the use of optical methods, which have the unique feature of being noncontact. Modern high-speed fruit-grading systems run at speeds in excess of 10 fruit per second and noncontact methods will be advantageous in such circumstances.

This chapter reviews the current optical techniques for firmness measurement. Among these techniques, near-infrared spectroscopy (NIRS) [8–10] and spatially resolved reflectance spectroscopy (SRRS) [11, 12] have been investigated more commonly in recent years, and are more suitable for high-speed operation.

2. Principle of optical methods for measuring firmness

Optical techniques are based on light interactions with fruit tissue. In the visible to near-infrared (Vis/NIR) range of the electromagnetic spectrum, fruit can be considered as semi-transparent or turbid. There are two optical phenomena that describe how light interacts with turbid biological material: absorption and scattering (Figure 1). Absorption is primarily due to the chemical composition of the tissue (pigments, chlorophylls, water, etc.). Scattering depends on microscopic changes in refractive index caused by the tissue density, cell composition, and extra- and intra-cellular structure of the fruit, and thus may be useful for assessing textural properties such as firmness. The light transportation in fruit can be characterized by fundamental optical properties of absorption, scattering and refraction, which are defined by the absorption coefficient (μ_a), scattering coefficient (μ_s), refractive index (n), and anisotropy factor (g).

Figure 1.

Distribution of incident light in fruits: (1) surface/specular reflectance, (2a) diffuse reflectance, (2b) transmittance, and (3) absorption [13] (Copyright 2016 American Society of Agricultural and Biological Engineers. Used with permission).

Cen et al. [14] used a hyperspectral backscattering system to measure optical properties of “Golden Delicious” and “Granny Smith” apples over 30 days’ storage time. The optical properties from 300 to 1000 nm were compared with acoustic and impact firmness. They found the scattering coefficient generally decreased as the fruit softened (r> 0.9 for both mechanical properties). Absorption coefficients also had high correlations with firmness (r~ 0.9 for “Golden Delicious”) in the wavelength range that associated with chlorophyll and anthocyanin absorption.

The inverse adding-doubling (IAD) technique was used to measure optical properties between 400 and 1050 nm in another study on apples [15]. The reduced scattering coefficient between 550 and 900 nm had an average correlation r= −0.68 with penetrometer firmness. Changes in optical properties at carotenoid (400–500 nm) and chlorophyll-a (680 nm) wavelengths correlated with penetrometer firmness with r= −0.69 and 0.52, respectively. However, Tomer et al. [16] found that the IAD technique could be quite inaccurate for absorption coefficient measurements on fresh produce at 785 nm, reporting a coefficient for fresh onions that was five times larger than that required for light transport modeling on onions.

There have been some studies based on other optical principles. For example, Costa et al. [17] used a biospeckle laser system to measure the biospeckle images on the Acrocomia aculeatafruit pulp. The calculated biological activity (BA) had a negative correlation with penetrometer firmness. The correlation varied depending which tree was evaluated, the highest was r= −0.903. Skic et al. [18] used a similar approach for “Ligol” and “Szampion” apples. They achieved a correlation of about r= −0.5 for both cultivars. Peña-Gomar et al. [19] used a technique called laser reflectometry near the critical angle (LRCA) to measure the refractive index of mango pulp, which was expected to correlate with acoustic firmness. Their results showed some correlation but the authors did not report the correlation coefficient, and only six fruits were measured.

3. Optical techniques for firmness measurements

Optical methods are noncontact; a feature that distinguishes them from most mechanical methods. In the past two decades, the most common optical sensing method for produce grading is NIRS. Grading lines equipped with NIR sensors are now commercially available from many manufacturers. Firmness is not an attribute commonly assessed using industrial NIR sensors [1], but it has been studied in a number of research applications (Table 1).

In theory, the optical scattering properties are more directly related to firmness than absorption properties and have been reported to correlate with firmness, as discussed in Section 2 [14, 15]. Optical techniques that can measure optical properties of biological materials have been studied more recently, aiming to provide a more accurate and robust technique compared to NIRS. These techniques may be divided into three main categories: time resolved, frequency domain, and spatially resolved. Time-resolved and frequency domain techniques have been extensively researched in the biomedical area, but they may not be suitable for applications on a grader line because of expensive instrumentation, slow speed, and the requirement of good contact between the sample and detector [20]. Spatially resolved techniques, and more specifically SRRS, have been researched more commonly for such applications as it can overcome many of those deficiencies.

3.1. Near-infrared spectroscopy

NIRS is widely used to determine fruit quality parameters, particularly compositional parameters such as soluble solids or dry matter content [4, 21]. Standard NIRS measures the spectral pattern of light transmitted through a representative portion of the flesh, and chemometric analysis methods are generally used to interpret the resulting absorbance spectra in terms of the parameters of interest. The disadvantage is that this technique relies on a prior extensive training exercise to develop a predictive model, based on the careful selection and measurement of a representative calibration data set from a suitable population. The model also needs to be checked and updated constantly.

For measuring fruit firmness, the NIRS method is limited in theory because it involves measurement of the apparent light absorbing power of a sample, which does not segregate scattering and absorption properties. However previous studies have suggested firmness may affect the apparent light-absorbing power through chemical changes associated with cell wall degradation, physical changes in intercellular structure and/or indirectly through correlated pigment absorption changes such as a chlorophyll decrease on ripening [14, 15].

3.1.1. Basic concepts

Near-infrared radiation covers the range of the electromagnetic spectrum between 780 and 2500 nm. Often wavelengths below 780 nm are also included in the analysis as these regions contain valuable information on absorbing pigments within the fruit flesh and skin [15]. Therefore, this technique is often referred to as Vis/NIR spectroscopy.

The typical NIRS set-up uses a broadband light source to illuminate the sample and the transmitted or reflected light is measured using a spectrometer. In the design process, it is useful to know that the NIR light intensity decreases exponentially with depth. One study [22] showed that the light intensity dropped to 1% of the initial intensity at a depth of 25 mm inside an apple in the 700–900 nm range. The depth was less than 1 mm in the 1400–1600 nm range. Therefore, the optical arrangement and the effective optical path length for the light are crucial elements to consider in order to collect spectra containing relevant information from the sample. This also explains why NIRS is suited for use with thin-skinned fruit, the thicker skins limiting light penetration [23].

In practice, three measurement set-ups are used (Figure 2). In reflection mode, light source and spectrometer are on one side but at a specific angle to avoid specular reflection, while in transmission mode the light source and detector are on opposite sides. Interactance requires a special optical arrangement so that specular and surface reflection cannot directly enter the detector.

Figure 2.

Three different set-ups: (a) reflectance, (b) transmittance, and (c) interactance. (i) Is the light source, (ii) is the sample, (iii) is the detector, (iv) is a light barrier, and (v) is the mechanical support [21] (Used with permission from Elsevier).

Transmission measurement has the advantages of exploring the largest volume of the internal flesh and all the light measured has interacted with the flesh. Thus it is suitable to find internal defects, but the transmitted light might also contain information of the two layers of skin (front entrance and back exit), and the core of the fruit. For firmness measurement, although light penetration is limited and one skin layer is still present, reflection and interactance set-ups will be more desirable as the light interacts with some portion of flesh without interference from the core. Schaare and Fraser [24] compared reflectance, interactance and transmission measurements for measuring soluble solid content (SSC), density and internal flesh color of kiwifruit and concluded that interactance measurements provided the most accurate results.

3.1.2. Firmness applications

Sensors based on NIRS techniques have been mainly developed for chemical compositions such as SSC, and most of the studies have been carried out under static conditions. The industry is taking the lead in the development of online systems, but there is little scientific evidence of their accuracies [21]. Attempts to use NIRS for fruit firmness prediction have met with varying degrees of success with some studies reporting correlations as high as R~ 0.8 − 0.9. Table 1 gives an overview of NIRS applications that measure firmness of fruits and vegetables.

Most reported scientific studies consider only a single NIR instrument format for fruit assessment. For example, McGlone et al. [25] used an interactance mode (Figure 3). The system contained a broadband light source (50 W quartz halogen, RJL 5012 FL, Radium, Germany) and a nonscanning polychromatic diode array spectrometer (Zeiss MMS1-NIR, Germany). Fruits were placed on a holder with stem-calyx horizontal. Measurements were generally taken on two opposite sides around the circumference, taking care to avoid any obvious surface defects. The absorbance spectrum measured was the average of 5 contiguous acquisitions at 175 ms integration time.

Figure 3.

The benchtop NIRS system [25].

The wavelength range used varies among the reported literature studies (Table 1). Walsh [23] suggested that restricted wavelength ranges could improve the robustness of a model and allow for the development of lower cost “multispectral” measurement systems. Prediction performance was generally determined by dividing the fruits randomly into a calibration and a validation set for model development. Walsh [23] also reported that such a model will predict the attribute of interest within the population, but it is likely to fail spectacularly on a new, independent set.

3.2. Spatially resolved reflectance spectroscopy (SRRS)

Figure 1 illustrates two types of light reflectance: surface reflectance and diffuse reflectance. Surface reflectance contains information about the object surface such as color. Only 4–5% of incident light is reflected by surface reflection and external diffuse reflectance, so most reflected light contains the diffuse reflected/backscattered photons that carry information of the internal tissue properties [11].

Figure 4 shows a small continuous-wave light beam perpendicularly illuminates the sample’s surface, and the reflected light is measured at different distances from the light source, forming the spatial profile (Figure 3). Optical properties/parameters can be obtained by using a phenomenological diffusion model and/or a heuristic modified Lorentzian model from the measured one-dimensional scattering profile. Mollazade et al. [28] used texture-based features methods to build models to predict mechanical properties of various produce. Instead of looking at a single 1D scattering profile, this technique analyzed the entire 2D images, which was expected to improve the correlation to firmness.

Figure 4.

Measuring principle for spatially resolved reflectance spectroscopy (SRRS) [20] (Used with permission from the author).

The extracted parameters can then be used to predict firmness using statistical models such as multiple linear regression (MLR) and artificial neural network (ANN). Typically, images are first processed to reduce noise and then converted into one-dimensional profile [29]. Figure 5 illustrates the process used by Sun et al. [30] for measuring apple firmness. The scattering image was first processed to find the center of the illuminated area (Figure 5(a)). Then a process called ring/radial averaging was performed. The distance to each pixel was calculated and rounded to the nearest whole number (Figure 5(b)). All pixels at each of these integer radii were grouped and averaged providing a vector of intensity values that correspond to single pixel rings expanding out from the center point (Figure 5(d)). The intensity profile (Figure 5(c)) was finally produced.

Figure 5.

Imaging processing used by Sun et al. [30] for apple firmness measurements: (a) finding center in the raw image, (b) ring average, and (c) & (d) producing the spatial profile.

3.2.1. Parameters extraction

In turbid material, a diffusion equation is often used as an approximation of the transport of the light. For SRRS under the assumption of inexistence of photon source in the medium, the diffusion equation can be simplified to an equation consisting of three variables: r(source-detector distance), μ_a and μ_s′ [11, 31]. Unknown optical properties μ_a and μ_s′ can be obtained by applying a curve fitting procedure with respect to r.

Researchers have also used statistical distribution functions to fit scattering profiles as a function of scattering distance. Peng and Lu [32] investigated a number of variations of modified Lorentzian functions aiming to find one suitable for firmness and SSC measurements. They concluded Eq.(1) was the best performing equation, which was also used in other studies for firmness applications [7, 29, 30]:

$I\left(x\right)=a+\frac{b}{1+{\left(\frac{\left|x\right|}{c}\right)}^d}$E1

where Iis the intensity along a radial intensity profile, ais the asymptotic value of light intensity when x(distance to center of the light spot) approaches infinity, bis the peak value corresponding to the intensity at the center of the image, cis the full width half maximum (FWHM) of the intensity profile, and dis related to the slope of the profile in the FWHM region.

3.2.2. Hardware

A SRRS system consists of two essential components: light source and imaging system. All the systems can be divided into three types according to the light source and operating wavelength range: laser light backscatter imaging (LLBI), multispectral light backscatter imaging (MLBI), and hyperspectral light backscatter imaging (HLBI).

The LLBI technique requires a small illumination spot on the target fruit, and measurement scattering areas of 25–30 mm diameter have been used for beam diameters of 0.8–1.5 mm by Lu [33] and Peng and Lu [32], respectively. Lasers are particularly suitable for this purpose since lasers can produce focused high-irradiance illumination spots on the fruit, which allows for deeper light penetration and fast image acquisition (shorter integration time). Moreover, LLBI systems are more robust and cost-effective than MLBI and HLBI. Overall, LLBI systems are potentially suitable for online high-speed operations. One of the drawbacks of LLBI systems is the limited operating wavelength. One to four lasers are typically used [28, 30, 34].

In the MLBI and HLBI systems, the light source is a tungsten-halogen lamp. The light usually passes through an optical fiber and then focuses on the fruit by a collimating lens, as shown in Figure 6. One exception is the system developed by Van Beers et al. [35] where a super-continuum laser and a monochromator were used for the hyperspectral measurements.

Figure 6.

Hyperspectral system (HLBI) for measuring the firmness of peach [36] (Used with permission from Elsevier).

The scattering profiles can be measured using multiple spectrometers at different source-detector distances. The advantage of using a spectrometer is that multiple wavelengths or a specific spectral region can be obtained simultaneously. However, it requires a good contact/focus between the probes and the sample, which will not be suitable for online operations. A CCD camera is more commonly used as it is noncontact, which has been a dominant format in all three types of systems (Table 2), except that Sun et al. [30] used a CMOS camera. CCD and CMOS cameras allow only single wavelength operation, but an imaging spectrograph has been used in HLBI systems to provide spectral and spatial information on a single image (Figure 6). Filters were also used in MLBI system to enable the image acquisition at specific wavelength [33].

Species	Cultivar	Acquisition mode	Spectral range	Prediction	Reference
Apple	“Royal Gala”	Interactance	500–1100 nm	R= 0.77; RMSEP = 7.5 N	McGlone et al. [25]
Apple	“Gala”; “Red Delicious”	Reflectance	400– 1800 nm	R= 0.88	Park et al. [26]
Mandarin	Satsuma	Reflectance	350–2500 nm	R= 0.83	Gómez et al. [10]
Pear	“Conference”	Reflectance	780– 1700 nm	R= 0.59	Nicolaï et al. [9]
Capsicum annuum	Bell pepper	Reflectance	780– 1690 nm	R= 0.6; RMSEP = 7.2 N	Penchaiya et al. [8]
Cherry	“Hedelfinger; Sam”	Reflectance	800–1700 nm	R= 0.8 & 0.65; SEP = 0.79 and 0.44 N	Lu [27]

Table 1.

Overview of applications of NIR spectroscopy in firmness measurements.

Species	Cultivar	Light source	Detector	Spectral range	Prediction	Reference
Apple	“Golden Delicious”; “Jonagold Delicious”	Halogen lamp	CCD camera with spectrograph	500–1000 nm	r= 0.84 − 0.95; SEP = 5.9 − 8.7 N	Mendoza et al. [37]
Apple	“Golden Delicious”	Halogen lamp	CCD camera with spectrograph	450–1000 nm	r= 0.894; SEP = 6.14 N	Peng and Lu [32]
Apple	“Braeburn”	Halogen lamp	CCD camera with spectrograph	500–1000 nm	r= 0.84; RMSEP = 9.68 N	Nguyen Do Trong et al. [38]
Apple	“Braeburn”	Supercontinuum laser with monochromator	CCD camera	550–1000 nm	r= 0.8; RMSEP = 0.75 kg cm^-2	Van Beers et al. [35]
Peach	“Red Haven”; “Coral Star”	Halogen lamp	CCD camera with spectrograph	500–1000 nm	r= 0.76 − 0.88	Lu and Peng [36]
Apple	“Red Delicious”; “Golden Delicious”	Quartz tungsten halogen lamp	NIR enhanced CCD camera with a liquid crystal tunable filter	680, 700, 740, 800, 820, 910, & 990 nm	r= 0.898; SEV = 6.14 − 6.41 N	Peng & Lu [29]
Apple	“Elstar”; “Pinova”	Laser diode	CCD camera	680,780,880,940, & 980 nm	r= 0.89 − 0.81; RMSEC V= 4.71 − 5.48 N/cm^2	Qing et al. [34]
Apple	“Royal Gala”	Laser diode	CMOS camera	685,850,904, & 980 nm	r= 0.87; RMSECV= 7.17 N	Sun et al. [30]
Apple; Plum; Tomato; mushroom	“Pinova” & “Elstar”; “Jojo” & “Tophit”; “Pannovy”	Laser diode	CCD camera	660 nm	r= 0.887 − 0.919	Mollazade et al. [28]
Apple	“Golden Delicious”	Laser diode	CCD camera with spectrograph	408 nm	R= 0.75; SEP = 8.57 N	Noh & Lu [39]

Table 2.

Overview of applications of SRRS in firmness measurements.

3.2.3. Applications

An overview of SRRS to measure the firmness of fruits and vegetables is given in Table 2. The studies show that SRRS achieves similar performance compared with NIRS. The correlations with penetrometer firmness are often in the range of r= 0.8 − 0.9. It is not clear which type or instrument format of SRRS is more advantageous. Most studies evaluated the potential of SRRS systems for firmness measurements on static fruit and have not considered the practical challenges of applying SRRS to online situations. Unlike NIRS, there have been no commercially available sensors based on SRRS. All the studies listed in Table 2 are laboratory systems specifically constructed for measuring stationary fruits. Lu and Peng [7] developed a real-time LLBI system for measuring the firmness of apples on a belt conveyor and achieved a correlation of r= 0.86. They claimed that the LLBI system could be integrated into existing grader lines without significant modification. However, their measurements were taken when the conveyor speed was only two fruit per second which is well below the maximum speed of a modern grader. Also, the fruit was manually positioned so that the scattering images could be captured from the equatorial areas of the fruit. The authors suggested the lasers and CCD camera should allow faster acquisition of the scattering images, but the algorithm for processing the images was the bottleneck. Overall, fruit orientation and data processing speed are the main challenges for applying SRRS in online systems.

4. Conclusion

For the main two optical techniques discussed here, NIRS and SRRS, there have been prior studies showing correlations with penetrometer firmness as high as r= 0.8 − 0.9. Both techniques can come in many instrument formats, so it is hard to judge from the literature which instrument is more advantageous. A direct comparison of the NIRS and SRRS methods has not been performed on the exact same fruit samples commonly. Sun et al. [30] compared an interactance mode NIRS system with an LLBI system using “Royal Gala” apples. The two systems had similar correlations with penetrometer firmness of about r= 0.9. By contrast, a comparison of a reflectance mode NIRS system and an MLBI system using “Red Delicious” and “Golden Delicious” was conducted by Lu and Peng [40]. Their MLBI system outperformed NIRS system with r= 0.82 and 0.81 for two apple cultivars, versus r= 0.5 and 0.48 from the NIRS system.

It has been suggested r= 0.94 (r² = 0.89) be considered as a minimum for any useful sorting/grading purposes [41]. Although sometimes very close to that mark, the correlations reported here and in most previous studies are lower. Moreover, NIRS sensors are likely to perform worse across grader lines and seasons because of the low robustness of the calibration models. These may explain why there are no optical sensors for firmness measurements yet commercially available. For SRRS, another concern is the feasibility of online applications; most studies discussed here are bespoke laboratory systems for measuring static fruits. Fruit speed and orientation are normally not a problem for NIRS but might be an issue for the online application of SRRS.

NIRS is a relatively mature technique for quality grading of fruits and vegetables, though not commonly used for firmness. SRRS might well be a better method for firmness, being more robust in practice as it is more directly linked to the optical scattering properties that are presumed to be directly affected by changes in texture properties. However, the SRRS systems will have to be improved and demonstrate better performance than has been achieved to date before they can be considered for commercial implementation. We recommend further research across a wider variety of fruits in the future, and feasibility studies to assess the potential of SRRS for online applications.