Using Near-Infrared Spectroscopy in Agricultural Systems

Abstract

This chapter provides a review on the state of art of the use of the visible near-infrared (vis-NIR) spectroscopy technique to determine mineral nutrients, organic compounds, and other physical and chemical characteristics in samples from agricultural systems—such as plant tissues, soils, fruits, cocomposted sewage sludge and wastes, cereals, and forage and silage. Currently, all this information is needed to be able to carry out the appropriate fertilization of crops, to handle agricultural soils, determine the organoleptic characteristics of fruit and vegetable products, discover the characteristics of the various substrates obtained in composting processes, and characterize byproducts from the industrial sector. All this needs a large number of samples that must be analyzed; this is a time-consuming work, leading to high economic costs and, obviously, having a negative environmental impact owing to the production of noxious chemicals during the analyses. Therefore, the development of a fast, environmentally friendly, and cheaper method of analysis like vis-NIR is highly desirable. Our intention here is to introduce the main fundamentals of infrared reflectance spectroscopy, and to show that procedures like calibration and validation of data from vis-NIR spectra must be performed, and describe the parameters most commonly measured in the agricultural sector.

1. Introduction

One of the challenges of the twenty-first century is to achieve a more productive agriculture, while improving the safety and quality of food. The food industry has to feed a population that is in continuous increase, bearing in mind that these systems have to respect the environment, should optimize natural resources in each area, and anticipate changes in temperature and rainfall that will occur in the future as a result of climate change. Proper soil management and fertilization of crops will be crucial to increasing the capacity of agriculture, to the provision of products of high added value, and to the protection of crops against pests and diseases. To do this, in each of the steps ranging from the production of fruits and vegetables in the field to the development of industrial products, it is necessary to determine a great number of physical and chemical parameters in the soil, plants, fruits, compost, and byproducts from food processing industries. Currently, the traditional techniques of analysis of such samples are being replaced by spectroscopic techniques—one of which is visible near-infrared spectroscopy (vis-NIRS). This technique has a number of advantages over the traditional methods, as it (i) is a method of nondestructive analysis, (ii) does not pollute the environment, because it does not use chemical reagents, (iii) is cheap and fast, (iv) measures many parameters in a single analysis, and (v) can perform analyses in situ and online for a large number of samples per minute.

The aim of this chapter is to provide an updated review of the current state of vis-NIRS as a technique for the estimation of physical and chemical parameters in samples derived from agricultural systems, such as soils, plants, fruit, compost, and products derived from food processing industries. The chapter starts by describing the basic principles of this technique and the different ways in which the equipment can be calibrated, detailing the statistical tools that are useful to establish that the calibration and the estimation of the desired parameters are valid. We will describe the parameters that can be measured by vis-NIRS in samples, with the emphasis on soil, plants, fruit, compost, and byproducts from the industrial sector that processes the output of agricultural systems. A basic explanation of the parameters measured in these samples will be given, together with a description of how they are measured and the mathematical tools used, focusing on the most novel issues.

2. Fundamentals of infrared diffuse reflectance spectroscopy

Spectroscopy in the near infrared or NIRS (near-infrared reflectance spectroscopy) is a tool that has been used widely for the rapid determination of organic components. For example, NIRS readout for nutrient level estimation on citrus leaves, using FT-NIR spectrometer and 64 scans per sample, takes 1–2 min. The only pretreatments of the sample required prior to analysis are drying, crushing, and mixing, in the case of solid matrices. Samples can also be scanned when fresh, as in the work of Huang et al. [28]. All this bestows on this technique several advantages over other, more sophisticated spectroscopic or analytical methods. The operating principle of the NIRS technique requires that the energy absorbed in the near-infrared region by a sample causes covalent bonds of C-H, O-H, and N-H, important components of organic substances, to vibrate in different forms [1]. Within the field of NIRS, two main types of fundamental vibrations are considered: stretching, which involves a change in the length of a bond, and bending, which involves a change in the angle between two bonds. Overtones appear when a vibrational mode is excited at a frequency higher than that of the fundamental vibration.

This infrared fraction comprises wavelengths between 780 and 2500 nm (12,500–4000 cm⁻¹, expressed as a wavenumber, Table 1).

Group	Aliphatic hydrocarbons	Aromatic hydrocarbons	Carboxylic acid	Amines	Water
Frequency range (cm^−1)	9100–7800 (overtone of CH-stretching)	ca. 9000 (overtone of CH-stretching)	ca. 6900 (overtone of CH-stretching)	7000–6500 (overtone of NH-stretching)	7500–6400 (overtone of OH-stretching)
	7700–6900 (combination)	7300–6900 (combination)	ca. 5250 (overtone of CO-stretching)	5200–4500 (combination)	5400–4900 (combination)
	6300–5500 (overtone of CH-stretching)	ca. 6000 (overtone of CH-stretching)	4900–4600 (combination)
	5000–4100 (combination)	4700–4000 (combination)

Table 1.

Absorbance signals in the near infrared for the major chemical groups present in organic matter.

There is a relationship, both quantitative and qualitative, between the chemical composition and the spectrum recorded in the near-infrared. Hence, samples having different organic compositions have different infrared spectra. But, interpretation of the spectra is tremendously complex, although the spectral characteristics of each compound are unique, as their amplitudes sometimes overlap.

Before the NIR spectrum of a sample can be used for the determination of a compound or specific element, a calibration for this compound or element must be developed. In an NIRS spectrum, the various constituents of the sample have some overlapping peaks; thus, the measurements made with NIRS must be calibrated with samples of known chemical composition in order to extract the desired information using NIRS [2].

3. Calibration and validations of data from NIRS

Chemometrics includes all methods of multivariate calibration in the field of analytical chemistry. Unlike univariate calibration, where a spectral peak (height or area) is correlated with the reference concentration, multivariate calibration uses the entire spectrum structure with a large amount of spectral information to correlate with the reference concentration.

The establishment of a model for the use of NIRS data in the analysis of samples consists of the following steps: (1) introduction of the spectral and concentration data; (2) preprocessing of the spectral data; (3) definition of the appropriate frequency range; (4) validation and optimization of the method; (5) definitive calibration; and (6) routine analysis.

(1) Introduction of the spectral and concentration data. This process begins with the selection of the group of samples for calibration, which must be well defined statistically, and pretreatment of the samples to assess measurement errors. The dispersion of incident radiation, also known as the scatter effect, produces a low selectivity (quality of being able to tune in to one particular frequency while blocking out other unwanted frequencies) of the NIR spectral information [3]. This is due to physical phenomena—such as the texture, size, and geometry of the particles that make up the sample [4, 5]—and to changes in the refractive index of the material which interacts with the radiation, causing numerous unwanted variations in the NIR spectral data [6–9]. Depending on the complexity of the samples, between 20 and 200 samples are necessary to develop a multivariate calibration method. The greater the number of samples, the more representative is the calibrations achieved. The samples should have a normal distribution, cover the entire range of concentrations of the parameters that are to be estimated by NIRS, and should not have areas where uncertainty is high and errors can be significant. For instance, the NIR spectrum of water (transmission measurement, optical path length: 2 mm) shows a total absorption between 5200 and 4000 cm⁻¹ and below 4000 cm⁻¹ a strong contribution of spectral noise. Finally, for each sample a classical analysis of the desired components is carried out, to obtain the so-called reference values, and its NIR spectrum is obtained.

(2) Preprocessing of the spectral data. The spectral pretreatment that improves the signal/noise ratio must be chosen. For example, the problems of baseline displacement need to be eliminated. The procedures for preprocessing of the NIRS spectrum, to obtain a good correlation between the spectral data and the concentration, include: no data preprocessing (NDP), first derivate (FD), application: it is used to emphasize pronounced, but small features compared to enormous broad-banded structures or on the evaluation of broad bands that get a steeper shape, so it can be evaluated more easily; second derivate (SED), application: similar to first derivative, where even extremely flat structures can be evaluated, but the spectral noise is enhanced as well, the most widely used methods here are the Savitzky-Golay [10] and Norris [11] methods; standard normal variate (SNV [6]; multiplicative scatter correction (MSC [3, 12]), application: it is used for measurements in diffuse reflection; detrending (DT), which is usually applied in conjunction with SNV; spectral smoothing (SS), for which the most used are the Savitzky-Golay [10] and Fourier transformation [13] or vector normalization (VN), application: in a measurement in diffuse reflection, the interfering influences of different material densities or particles sizes can often be minimized; maximum-minimum normalization (MMN), application: similar to vector normalization; subtraction of a straight line (SSL), application: a linear tilt of the baseline shift is eliminated; linear offset subtraction (LOS), application: linear baseline shifts are eliminated. The optimum method depends on the system to be analyzed. Generally, SSL, VN, or FD leads to better calibration.

(3) Definition of the appropriate frequency range. Once the calibration samples have been selected and then analyzed by the reference method and NIRS, a correlation between the spectral and analytical data is searched for [14]. For this purpose different statistical treatments are used, such as multiple linear regression (MLR [15]), principal components regression (PCR), and partial least squares regression (PLSR) as linear methods and use of artificial neural networks (ANN) as a nonlinear method. PLSR is the one most commonly used [16, 17]. The best correlated frequency ranges are assessed and then selected based on the coefficient of determination, R², and a corresponding low error of analysis (root mean square error of cross validation/root mean square error of prediction (RMSECV/RMSEP), see equations (4) and (5)). Typically, an R² value of 0.75–1.0 indicates an acceptable correlation and it depends on the type of sample. Good values for R² are larger than 0.90 for solids and larger than 0.99 liquid measurements. The total absorption of water (frequency range 5200–4000 cm⁻¹) yields relative absorbance values (A) greater than 2.5. The use of dried samples prevents interference of water in the aforementioned frequency range. Spectral noise is usually found below 4000 cm⁻¹ and gives relative absorbance values lower than 0.7. Thus, this region should not be included to establish a calibration. Values of A between 0.7 and 1.0 generally give better results. Besides, modern FT-spectrometers allow the use of absorbance values of up to 2.5 for the calibration.

(4) Validation and optimization of the method. To choose the best calibration for the regression equation with linear models (PLS algorithm, for instance), the instrumental software combines different methods of data pretreatment and frequency ranges. Then, it provides as output the corresponding mean error of prediction and R² for a given number of factors. The quality of the calibration is evaluated by the validation, which consists of comparing the concentrations predicted by the calibration with the reference values of samples not used in this calibration [18]. There are two types of validation: internal or cross validation and external validation. In internal validation, a sample, or group of samples, is taken from the set of samples. With the calibration obtained using the remaining samples, the concentrations in the previously separated samples are predicted. The samples are interchanged until all have been used once for the validation. In external validation, all samples are used for calibration and prediction is performed for additional samples [19]. Since optimal frequency windows and pretreatments of signals cannot be anticipated, they are generally determined empirically by trial and error. These values are calculated for a growing number of factors. The concentration and spectral data are encoded in matrix form and reduced to a small number of factors called “rank.” To some extent, the factors or principal components are “information units,” as may be the case for the concentration of a sample component. In many cases, there are several combinations of frequency window and pretreatment of spectral data of comparable quality for the prediction of analytical results. In these cases the combination that has fewer factors is recommended, as it generally will be more stable (Table 2). The optimum method is number 2 (mean error of prediction 0.07% and optimum rank 6). However, it is possible to manually set a lower rank in order to get a better result.

Number	Data preprocessing	Frequency ranges (cm^−1)	Optimum rank	Coefficient of determination (R^2)	Mean error of prediction (%)
1	NDP	9000–5200	9	0.998	0.16
2	SSL	9000–5200	6	>0.999	0.07
3	VN	9000–5200	8	0.996	0.42
4	SSL	7000–5200	8	>0.999	0.07

Table 2.

Method optimization using the PLS algorithm for NIRS analysis of CH₃OH concentration in a mixture of CH₃OH, C₂H₅OH, and C₃H₇OH.

No data preprocessing (NDP), subtraction of a straight line (SSL), and vector normalization (VN).

There will be anomalous samples and spectra, called “outliers,” which affect the predictive ability of the NIRS equations obtained [20]. It is important not to remove them until one has a clear explanation; to make this decision one has to take into account the t-test (“Student's t-test” Eq. 1) and H (Mahalanobis distance, Eqs. 2 and 3) values, among others.

t-Test applied to each wavelength gives an idea about the weight of each wavelength in the calibration. The higher the value in the t-test, the more important it is. If it is higher than 10, it is considered essential to take part in the calibration equation.

$t=\frac{Y_{\text{pred}}-Y_{\text{ref}}}{SEC*\sqrt{1-H}}$E1

SEC is the standard error of the calibration, and H is the spectral error. In NIRS, t > 2.5 are considered significative and kept in the calibration.

Chemical outliers can be recognized after applying a t-test since they present significative differences between the composition value provided by the reference method and the regression model.

To detect spectral outliers, the Mahalanobis distance is particularly useful. For MLR models it is calculated as follows:

$H=\frac{K}{n}$E2

For the models PLSR and PCR, it is expressed with the following equation:

$H=\frac{K+1}{n}$E3

n is the number of spectra in the dataset and K is the number of selected wavelengths. H < 3 means that the samples belong to the population.

The statistics used in the evaluation, selection, and validation of the calibration equations are as follows:

- Determination coefficient of the calibration (R²_c)/Determination coefficient of the cross validation (R²_v). This establishes a correlation between the analytical data obtained in the laboratory and those predicted by the calibration equations for each of the components analyzed. As mentioned above, an R² value of 0.75–1.0 indicates an acceptable correlation. Some calibrations with an R² value <0.75 may be useful for monitoring purposes. Thus, an R² value of 0.50–0.69 distinguishes between low, medium, and high values; an R² of 0.30–0.049 distinguishes between low and high values; and with R² < 0.29, it is better not to analyze [21, 22].

- Root mean square error of estimation (RMSEE). This is the error associated with the differences between the analyses performed in the laboratory using the reference methods and the results of the analysis by NIRS technology, for each of the parameters determined in the samples used in the calibration. This value of this statistical parameter should be as low as possible. It is calculated using the formula:

$RMSEE=\sqrt{\frac{1}{M-R-1}SSE}$E4

where M is the number of samples in the calibration set, R is the number of principal components (factors), and SSE is the squared sum of the differences between the actual and estimated values. It is preferable to compare this type of error with the error that can occur with traditional methods of analysis and decide whether the error is acceptable for routine use. The prediction error (P) is the accumulation of the errors of the reference concentrations (R), of the NIRS data, and of the calibration. The lower the ratio of the errors (P/R), the greater the accuracy of the NIRS model obtained: P/R = 1–1.5, excellent; P/R = 2–3, good; P/R = 4, moderate; and P/R = 5, poor [23].

- Root mean square error of cross validation/Root mean square error of prediction (RMSECV/RMSEP). Following calibration the cross validation error is obtained. This error is the one that should be taken into account most closely when evaluating the calibration. To calculate it, considering the number of samples in the set and the differences between the estimated values and those obtained by standard methods of analysis, the following formula is used:

$RMSECV=\sqrt{\frac{1}{M}·\sum _{i=1}^M{\left(Diffe{r}_i\right)}^2}$E5

- Residual prediction deviation (RPD). This is defined as the ratio between the standard deviation of the reference data and the RMSEE/RMSECV. One researcher [24] provided a guide to evaluate calibrations performed with environmental samples, based on the R² and the RPD, as follows: excellent, R² > 0.95 and RPD > 4; good, R² = 0.9–0.95 and RPD = 3–4; quite good, R² = 0.8–0.9 and RPD = 2.25–3; quite useful, R² = 0.7–0.8 and RPD = 1.75–2.25. RPD is of the same significance as R² explained variance. The R² also allows a qualitative evaluation of the error of prediction during the validation process.

- Bias. This is the difference between the mean value predicted by FT-NIRS and the mean value of the reference predictive model and the residual prediction deviation (RPD, [20, 25, 26]): M is the number of samples used in the calibration, xi is the result obtained by NIRS, and yi is the result obtained by the reference method for sample i:

$Bias=\frac{1}{M}{\sum }_{i=1}^{i=M}(xi-yi)$E6

In the presence of laborious and troublesome datasets, it is possible to ask for high-performance external NIR calibration services such as those provided by private companies to optimize and validate the method.

(5) Definitive calibration. After all the “outliers” have been eliminated and the optimal parameters determined (for example, R², RMSEE, and RPD for the calibration and R², RMSEP, RPD, and bias for the validation), the final calibration model is evaluated for the analysis of new samples.

(6) Routine analysis. Here the optimum chemometric model is used to analyze quickly unknown samples. The Mahalanobis distance can alert one to samples (“outliers”) that are outside the calibration range or do not fit the model well.

4. Mineral nutrients and organic compounds in different samples from agricultural systems

4.1. Plants

Fruit and vegetable crops, in order to achieve good vegetative growth and maximum production with good quality fruit, require a good nutritional status, maintaining a proper balance of nitrogen, potassium, phosphorus, and trace elements such as manganese, boron, copper, and magnesium. The guide to the nutritional status of crops is based on the method known as “sufficiency range” (SR), which establishes—for each nutrient—the ranges of values considered to be normal and to represent deficiency or toxicity [27]; or it can be based on the establishment of indices of dependent nutrients, in which each index includes two or more nutrients, the so-called “Integrated System of Diagnosis and Recommendation (DRIS).” But, a good fertilization program also should pay attention to the changes that occur in the mineral status of plants in their different phenological stages so that the application of fertilizers can be adapted to the requirements of the plants at all times. To obtain this information, it is necessary to perform mineral analyses of leaves by ICP-OES or AA after acid digestion of the samples, as well as analysis of the C/N ratio, which involves the analysis of a large number of samples with all that this entails. Currently, the available knowledge of reflectance spectroscopy in the near-infrared (NIR) part of the spectrum can be used to determine the nutritional status of crops quickly and cheaply. The mineral composition of an organic matrix can be estimated by NIRS, from the spectra in the range 700–2500 nm, due to the association between the minerals and the organic functional groups or the organic matrix itself [28]. There are no infrared absorption bonds in the mineral species of macro- and micronutrients, but NIRS determines bonds within organic compounds that are negatively related to inorganic materials. If mineral matter is bound to organic compounds, the distortion of the spectrum is detectable at certain wavelengths, suggesting that NIRS can quantify inorganic materials using their ratio to the organic matter [29].

Numerous studies show that the NIRS technique, together with multivariate analysis and partial least squares regression (PLSR), provides a powerful tool for the interpretation and analysis of spectra. For example, NIRS technology has been used successfully to predict the nutritional status of leaves of apple [30], alfalfa [31], sugar cane [32], root crops [33], yerba mate [34], and citrus [35, 36]. It has been observed in citrus leaves of different varieties including lemon, mandarin, orange, and grapefruit—high accuracy regarding the estimation of N (R = 0.99) and Ca (R = 0.98) as well as acceptable estimates for K, Mg, Fe, and Zn [37]. However, good calibrations for the estimation of P, B, Cu, and Mn were not obtained. Furthermore, the concentrations of nutrients could be estimated with a single calibration model, regardless of the variety of citrus analyzed. In yerba mate plants, the prediction was good for P and Cu but not for K, Ca, Na, Mn, or Zn [34]. These data show that the NIR spectral response depends on the species studied, so for each species it is necessary to make the appropriate calibrations—but these are valid for different cultivars of the same species.

4.2. Soils

Soil is a natural resource that is vital in agriculture for the production of food, fiber, and energy; but it serves also as a platform for human activities, constitutes an element of the landscape, is an archive of cultural heritage, and plays a central role as a habitat and gene pool. It stores, filters, and transforms many substances, including water, nutrients, and carbon (C). In fact, it is the largest C “store” in the world (1500 gigatonnes). All these functions must be protected because of their socioeconomic and environmental importance.

Fundamentally, the soil is a complex matrix of organic matter, minerals, water, air, and microorganisms. The soil organic matter is only a small part, but it plays a big role in both the physical and chemical properties of soil as well as in the development of crops. This soil fraction comprises humus (material that is decomposed, dark, and colloidal in nature) and materials such as the roots and aerial parts of plants and the bodies of insects and other animals that are deposited on the ground. The content of organic matter normally found in the soil is small, only about 1–5% by weight, of which 85–90% is humus and only a small part is the nonhumified remains. The mineral phase is a mixture of materials that differ in their composition and properties. Typically, this fraction is characterized by particle size. Stones, gravel, and sand represent the coarse fraction, while smaller particles like silt and clay constitute the fine fraction of the soil. The clays also can be classified according to the negative charges on their surfaces—some minerals are more negative than others—and this property also influences the chemical characteristics of the soil. There is a parameter that is very useful for measuring these chemical properties, namely, the cation exchange capacity (CEC)—defined as the maximum amount of cations that a soil can fix. Water is another significant fraction of the soil, and its content depends on the amount and size of the pores in the soil. In plants, water is the major constituent of protoplasm (85–95%) and is essential for physiological processes such as photosynthesis, nutrient transport, and maintenance of turgor. The air is another important fraction of the soil. Its oxygen is essential for the respiration of roots and microorganisms. When aeration is poor, organic matter is oxidized slowly, the activity of aerobic microorganisms is paralyzed, and only anaerobes are active, giving rise to reduced forms of elements that are usually toxic to plants. The most common microorganisms in soils are nematodes, protozoa, and rotifers—whose activities also determine the physiochemical soil characteristics, as they have the ability to degrade highly resistant organic compounds such as cellulose and lignin, and can even degrade minerals, thus releasing plant nutrients.

The study of soil should take into account the different phases of the soil and must be directed toward two main objectives: (i) consideration of its various properties, with special emphasis on plant productivity (that is, practical or applied aspects); and (ii) scientific, especially chemical, study—to determine the variation of productivity and find ways for soil conservation and improvement. In recent years, it has been observed that NIRS (using visible-near infrared) can be very useful for characterization of soils. This technique has many advantages: sample preparation is easy as it only requires the drying and grinding of the soil, reagents nontoxic to the environment are used, measurements are made in a few seconds, a single scan can show multiple properties, and the technique can be used both in the laboratory and in situ. The parameters that can be measured in the soil using NIRS are described below.

4.2.1. Soil organic matter (SOM)

In the laboratory, it is difficult to separate the organic and inorganic material of a soil, so an estimate of the content of organic matter is obtained indirectly through the analysis of an element that is a constituent of all organic substances, namely C. Once the amount of organic C present in a soil sample is known, the percentage of organic matter with respect to the total weight of the soil can be estimated indirectly. Classically, it has been determined by various methods—such as calcination of the soil sample, oxidation with potassium dichromate, or oxidation with hydrogen peroxide. With regard to NIRS, a large number of studies have shown that this analytical technique is very useful for estimating SOM. For this, absorption bands in the NIR region which result from the stretching and bending of NH, CO, and CH groups, that form part of the organic material, are used [38].

The 1990s saw the identification of the absorption bands of wavelength (nm) 1100, 1600, 1700–1800, 2000, and 2200–2400 as the most useful for measuring organic C. Since 2000, the technique of NIRS has been perfected and adapted to the soil and climatic conditions of each area. Thus, in Australian soil, it has been also observed that if the absorption spectrum was made in the vis-NIR region—that is, including the visible region—better results were obtained than with NIR alone [39]. Another problem that researchers have faced in achieving good calibrations has been that the spectral response can change depending on the mineral fraction of the soil, composition of organic matter, texture, and soil moisture content [40–42]. All these problems have been solved by optimizing the way of taking samples (local and regional scales) [43, 44], choosing the most appropriate mathematical models for the calibration [45–48], establishing covariance models, or eliminating certain factors that make the model weak [49, 50]. For example, in saline soils of El-Tina Plain (Egypt), it has been compared several regression techniques to estimate the organic matter content of soils [51]. Specifically, they used PLSR, support vector regression (SVR), and multivariate adaptive regression splines (MARS) and found that the best calibration was obtained with MARS with continuum removed reflectance preprocessing (R² and RMSE were 0.89 and 0.19, respectively). The calibration model to estimate the organic matter in an area of the Grand-Duchy of Luxembourg have been improved, taking into account the amount of water in the soil [52]; it was also considered by NIRS, using the reflectance values at 1800 and 2119 nm and calculating the normalized soil moisture index (NMSI).

4.2.2. Soil mineralogy

The mineral fraction of the soil occupies almost half of the soil volume. Its composition and concentration as well as the proportion of different minerals determine important properties such as texture, structure, and CEC. These properties also determine other soil characteristics such as the availability of nutrients to agricultural crops. Classical methods for the determination of clay minerals are qualitative and are based on XRD (XTR). However, some researchers [53, 54] made the first tests to see if NIRS could be used to estimate soil minerals; later, these same authors [55] compared the NIRS results with XRD analysis, concluding that NIRS is an effective method to determine the mineralogy of the soil. This is because soil minerals absorb light in the UV, visible, vis-NIR, and mid-NIR parts of the electromagnetic spectrum. For example, Fe oxides absorb UV radiation while phyllosilicates (clay minerals) have varied spectra in the vis-NIR. Overall, this technique has been used to estimate the Fe oxides goethite (α-FeOOH) and hematite (α-Fe2O3), clays of the kaolinite, illite, and smectite types, and carbonates [56, 57].

4.2.3. Soil texture and CEC

The water dynamics and aeration of a soil depend on its structure and texture, and these parameters are important for the development of both plants and microorganisms, so they need to be evaluated. These parameters also determine the leaching of fertilizers and pesticides in agricultural soils. Generally, soil texture is defined as the ratio (in percentage by weight) of particles smaller than 2 mm in diameter and classified as sand (2–0.02 µm), silt (0.02–0.002 µm), or clay (0.002 µm). Ben-Dor and Banin [58] found that the clay content may be estimated by analyzing the absorption bands of O-H in water, and those of Mg-, Al-, and Fe-OH in the mineral fraction of the soil. Curcio [59] used visible and near-infrared (VNIR, 400–1200 nm) and shortwave infrared (SWIR, 1200–2500 nm) reflectance domains to estimate soil texture in three agricultural areas of Italy (Bompensiere, Dirillo, and Pietranera), and obtained a good calibration by using the PLSR method, the accuracy being good for the clay fraction (RMSE = 5.8%, R²=0.87) and satisfactory for sand (RMSE = 7.7%, R²=0.80) and silt (RMSE = 7.2%, R²=0.60).

The CEC is traditionally measured by the method of Chapman [60], based on saturating the soil with sodium. However, the vis-NIR technique can also estimate this parameter in soils accurately, if methods of multivariate regression are used instead of simple bivariate relationships, and it is suitable for measurements of peak intensities in the mid-IR range. Recently, Ulusoy et al. [61] obtained a good prediction of CEC using an analysis of PLSR, both in the laboratory and for online measurements in the field—although the calibration was much better for the data obtained in the laboratory.

4.2.4. Plant nutrients

Due to the importance of the mineral nutrition of plants in the yield and quality of fruit and vegetables, one of the most common practices in agriculture is the analysis of the soil content of N, P, K, Fe, Ca, and Mg. This information is particularly important when optimizing fertilization programs. In most farming systems N is the element most commonly applied, followed by K, P, Ca, and micronutrients. These nutrients do not have a specific absorption spectrum in the vis-NIR region. Generally, the correlations between the “real” concentrations of these nutrients and those estimated by NIR are highly variable, the variability coefficients (R²) being in the following range: N (0.11–0.55), available potassium (0.56–0.83), exchangeable potassium (0.11–0.55), Ca (0.75–0.89), Fe (0.64–0.91), Na (0.09–0.44), Mg (0.53–0.82), and P (0.23–0.92) [40]. This variability may depend on many factors, so local-scale calibrations are recommended to achieve greater accuracy.

4.2.5. pH

Soil pH is an important regulator of fertilization. The solubility of the nutrients is pH-dependent, and plants may decrease nutrient uptake if the soil pH is not suitable. Other pH-dependent factors include the biological activity, decomposition of organic matter, and mineralization. Generally, the ideal soil pH for plants is between 5.5 and 6.5. Soil pH, or more specifically the H⁺ ions, has no direct response to NIR but its value can be estimated well with this technique if the appropriate covariations are applied to components that do exhibit activity in the NIR, such as organic matter and clays [62]. In different experiments it has been found that pH calibration gives R² values between 0.55 and 0.77 and an RMSE of 0.3–0.5 pH units. These parameters could be further improved if specific calibrations were made at the local scale, while studying in detail which covariance parameters have a direct influence on the NIR and thus should be used.

4.2.6. Heavy metals and other soil contaminants

Heavy metals are potential pollutants of air, water, and soil and of plants when taken up in sufficiently high amounts; this pollution will also affect other links in the food chain. In most agricultural soils, there are small amounts of As, B, Cd, Co, Cr, Cu, Mo, Mn, Ni, Se, and Zn, but when normal values are exceeded this can cause soil pollution and phytotoxicity, negatively impacting the agronomic performance. Usually, heavy metals are measured by atomic absorption or ICP. In the vis-NIR region these metals do not absorb energy, but their concentrations can be estimated if used as covariates with other components that do possess absorption spectra [40]. For example, they can be related to the organic matter, hydroxides, sulfides, carbonates, oxides, clay minerals, or soil texture (Stenberg et al. [40]). Todorova et al. [63] investigated the use of NIRS to estimate the concentration of heavy metals (Zn, Cu, Pb, Cr, and Ni) in various soils of Stara (Zagora Region, Bulgaria), using the PLS type of calibration. The best validation of the method was observed with Cu, while it allowed estimation of whether the concentrations of Zn, Pb, and Ni were low or high; however, Cr gave the weakest validation. These authors also noted that as the number of samples in the validation process increased, the RMESP values decreased.

In soils, hydrocarbons can also be measured with the vis-NIR technique, to establish the degree of contamination of soils that have suffered spills of petroleum products. Okparanma [64] used this technique to make soil maps in which the concentration of polycyclic aromatic hydrocarbons and their equivalent toxic concentrations in soil from Niger (Nigeria) were indicated. The data of this study revealed that the elaboration of soil spectra between 300 and 2500 nm, together with a PLS calibration, permitted the estimation of the concentration of hydrocarbons without significant differences from the results obtained by the conventional method of gas chromatography-mass spectrometry.

4.2.7. Soil moisture

There are a multitude of reasons to measure the water content in agricultural soils because water is fundamental to the development of plants and for soil biology, besides regulating important processes in the soil such as nitrification/denitrification, leaching, and erosion. In the laboratory, although there are many methods of measurement, the traditional one is based on weighing the sample fresh and then after drying, calculating the percentage of water relative to the dry weight of soil. Water produces an absorption spectrum in the NIR due to expansions and stresses of O-H bonds. Water incorporated into the soil in clay minerals absorbs at wavelengths around 1400 and 1900 nm. The main problem researchers have had in the calibration of this parameter is due to the fact that the water found on the surface of the minerals in thin layers and in the pores tends to decrease the albedo (the percentage of radiation that the soil reflects), changing the refractive index. As the porosity and refractive index of soil particles vary between soil types, a relationship between the albedo and water content cannot be given. However, some authors have successfully used a multivariate calibration with data from the NIR spectral bands to estimate the water content. For example, Bullock et al. [65] found a good correlation using a PLSR of the regions of 1100–2500 nm, and Ben-Dor and Banin [58] produced a good regression for samples having a water content of 0.2–11.6% using an MLR (multiple linear regression) calibration. The problem with all these calibrations appears when one wants to have a single calibration for soils of very different geological origins; thus, calibrations at the local scale are recommended [62, 66].

4.3. Fruits

Fruits and vegetables from agricultural plantations must maintain their optimum quality, whether they are destined for fresh consumption or for processing. As quality rises so do prices, so it is necessary to determine the intrinsic characteristics and external appearance of the fruits and vegetables. This information can be used to exclude fruits of poor quality, and can be provided to the consumer/industry to inform them of the added value of the product that is being offered. External defects—such as bruises, injuries from cold and wind, cracks in the skin, and contamination by pathogens—cause significant economic losses. Parameters such as total soluble solids content (TSS), acidity, and water content, which are related to the flavor and aroma of fruit and vegetables, serve to define their organoleptic quality. Therefore, currently, analytical methods are being developed that allow accurate, fast, and noninvasive determination of the qualities of agricultural products. In the case of the appearance of the fruits, computer vision technology—which integrates data acquisition, processing, and analysis—has great potential for the automatic inspection of the appearance of the products. The internal quality of the fruits can be estimated accurately by spectroscopy in the visible and infrared (vis-NIR), because most of the organoleptic characteristics are related to functional groups of the type C-H, N-H, and O-H. What follows is a brief summary of the most significant parameters that can be measured with these spectrometric techniques.

4.3.1. Total soluble solids

This parameter is used to measure the approximate amount of sugars in fruit juices, wine, or liquids processed in the agri-food industry, and is used to track in situ the evolution of ripening of the fruits and their optimal harvest time. The determination is made by refractometry and expressed in Brix, equivalent to grams of sugar per 100 ml of juice. Using the vis-NIR technique with a calibration model based on PLS, many researchers have obtained high correlation coefficients (R) (0.80–0.95) for fruits of mango, strawberry, apple, table grape, banana, blueberry, and bell pepper [67]. Recently, it has been shown that this technique can measure online the TSS of pear fruits (five fruits per second) as they pass along a transport chain [68]. The authors noted that the relationship between the wavelengths of 681 and 822 nm and a PLS calibration model allowed estimation of both the TSS and the healthy pears that had no apparent surface damage; therefore, they suggested that this technique could be integrated into industrial processes to select good quality fruit quickly, thereby reducing the labor required for both processing and laboratory analysis. Also, with portable vis-NIR equipment, the state of maturation of the grape variety Sangiovese could be determined in situ, by calculating the index of absorbance difference (IAD) from the values at wavelengths of 560 and 640 nm. These values correlate with TSS as well as with parameters such as titratable acidity (TA), firmness (DI), and anthocyanins—which allows one to know quickly and accurately the date on which the fruit should be harvested [69].

4.3.2. Total titratable acidity

This parameter describes the total concentration of acids in food, vegetables, or fruit. It is determined by an acid-base titration (soluble acids determined as citric, malic, lactic, oxaloacetic, succinic, glyceric, phosphoric, hydrochloric, fumaric, galacturonic, glyceric, tartaric acids, etc.). Acidity influences the taste of food (roughness), the color, the microbial stability, and the quality of conservation, and is determined by an acid-base titration using 0.1 N NaOH as the base and phenolphthalein as the indicator. Estimations by NIRS of this parameter and the pH of the fruit are as good as those found for the TSS. Thus, with the corresponding calibration data obtained by NIR, values of R between 0.80 and 0.82 have been observed in Chinese bayberry, apple, strawberry, table grape, and grape using wavelengths between 320 and 1650 nm [67]. In Spain, acidity has been studied using online NIRS combined with chemometric techniques (PCA, LDA, and PLSR) in fruits of different olive varieties; this gave good estimates of the free acidity (R² = 0.83), water content (R² = 0.76), and fatty acid content (R² = 0.83). For the calibration, a reflectance spectrum of intact olive fruits in the wavelength range 1000–2300 nm was produced and then samples of a paste prepared from these fruits were analyzed in the traditional manner. The estimation of these parameters improved when the calibration was performed for each stage of ripeness [70].

4.3.3. Contents of water and dry matter

For the food industry, the moisture content is an important quality factor of fruit and vegetables, whether fresh or processed, and influences their conservation and deterioration. The dry matter content is obviously very important when calculating the contents of other constituents of fruit and vegetables on the basis of the dry matter, which is uniform and less variable than the fresh weight. Water is the major component of all fruit and vegetables, representing between 60 and 96% by weight. The methods used most commonly for its determination are drying methods; the percentage water content is calculated as the loss in weight due to elimination by heating under standard conditions. Pu et al. [67] stressed that the vis-NIR technique is useful for measuring the water content in fruits of mango, strawberry, mushroom, banana, and soybean using spectra in the range of 400–1000 nm.

4.4. Grains and seeds of cereals, grasses species, and legumes

Cereals are important in animal feed not only because they provide energy but also since they contribute 70% of the protein in the diet and are especially important for pig nutrition. Similarly, soya is a major source of vegetable protein in animal feed formulations. It is important that the animal diets have a proper balance regarding the contents of amino acids.

The main nutritional parameters in the grains and seeds of these crop plants determined by the NIRS technique with regard to animal feed are the moisture and protein contents, representing the biological value. Other parameters analyzed are the contents of lipids, carbohydrates, and ash.

4.4.1. Amino acids

In 1978, Rubenthaler and Bruinsma [71] developed the first calibration equations for the determination of lysine in wheat and barley. Subsequently, Fontaine et al. [72] determined the total contents of methionine (Met), cysteine (Cys), lysine (Lys), threonine (Thr), tryptophan (Trp), and other essential amino acids in a population of cereal and sorghum samples. The spectra were first treated with SNV (recommended for samples with <15% moisture) and trend to reduce differences in the spectra that are caused by particle size effects only and not by changes in the constituents. In this way, the validation of the calibration equations showed that 70–98% of the variance of the amino acids in the samples could be explained using the NIRS technique, especially for Lys and Met—the amino acids most limiting to animal nutrition. Also, Kovalenko et al. [73], in the analysis of soybean samples, applied the MSC mathematical treatment (let to remove background spectroscopy) to the spectral data together with the PLSR regression model and obtained a determination coefficient (r²) of 0.91 for Lys. However, the concentrations of Cys and Trp did not exhibit a good correlation with the spectral information, the r² value for Trp being 0.04.

4.4.2. Other organic matter

With respect to dry matter, lipids, total protein, carbohydrate, and ash, Ferreira et al. [74] and Wang et al. [75] established models to determine the protein and lipid contents in both soya and fava beans. These authors obtained high R² values for protein (0.81 vs. 0.94); however, for lipids the values were slightly lower (0.71 vs. 0.66). Both groups used as a mathematical treatments: standard normal variate transformation (SNV; let to correct scattering effects caused by physical differences between samples) and first derivate. For the rest of the components, Ferreira et al. [74] obtained calibrations giving high predictability for dry matter, ash, and carbohydrates (RMSEP of 0.38–3.71%), the prediction being poorest for carbohydrates (R²_c = 0.50 and RPD_c = 0.83). Wang et al. [75] found RPD values of 2.95 and 2.50 for starch and total polyphenol content, respectively.

4.4.3. Toxic substances

Some seeds may contain substances that are antinutritional or toxic in nature, such as L-canavanine in seeds of one-flowered vetch. This is a toxic nonprotein amino acid that can cause a reduction in food intake, particularly in nonruminant animals [76]. These authors determined the content of L-canavanine in one-flowered vetch seeds by NIRS: the calibration equation obtained showed a correlation (r²) of 0.72, which, according to these authors, only served to separate the samples into groups of low, medium, and high L-canavanine content. However, the equation obtained for the total protein was able to predict with an accuracy similar to that of the reference method, showing a correlation (r²) of 0.95. Berardo et al. [77] studied the rapid detection of mycotoxins, mainly produced by the fungus Fusarium verticillioides, in maize samples. The best predictive ability for the overall rate of infection and F. verticillioides was obtained using MPLSR in samples consisting of grains of maize (r² = 0.75 and SECV = 7.43) and in samples of maize flour (r² = 0.79 and SECV = 10.95). These authors before the development of the calibration equations applied MSC (multiplicative scatter correction) to remove additive multiplicative effects in spectroscopic data to prevent them from dominating the information signal in the data.

4.5. Forage and silage

The production systems of ruminants are based on forage resources. These forages and—more particularly—maize, wheat, and alfalfa can be conserved as silage. Therefore, quick and reliable knowledge of the quality of forage and silage is very important for technicians and producers. The quality of silage depends—on the one hand—on its nutritional value, which is directly linked to its chemical composition (fiber fractions, nitrogenous materials, minerals, carbohydrates), and—on the other hand—on the quality of its conservation, which is defined by the end-products (lactic, acetic, and butyric acids, ammoniacal nitrogen, soluble nitrogen, etc.) of the fermentation processes. Currently, the main constituents determined by the NIRS technique in forage and silage, and which are important in the feeding of ruminants, are the total protein and protein fractions, soluble and structural carbohydrates, and digestibility of the forage, the latter depending on the content of structural carbohydrates.

Thus, Volkers et al. [78] established calibration equations for samples from different parts of a forage maize crop to predict the crude protein content, obtaining coefficients of determination (R²) of 0.86–0.96, except for samples of the cobs—which had an R² of 0.56. For net energy, the prediction was good—with an R² of 0.93 and 0.84 for the entire plant without the cobs and stalks, respectively.

With respect to the nitrogenous fractions, the nonprotein nitrogen/total nitrogen (NPN/TN) ratio in silage is very important for animal nutrition, since it indicates whether the silage has overheated; if the value is greater than 12%, it is considered to have occurred [79]. The acid detergent insoluble nitrogen (ADIN) is the fraction of the total nitrogen that is bound to the cell wall lignin of the plant. This component has low bioavailability but indicates the quality of the silage, as its abundance increases with overheating. Normally, it is not routinely analyzed due to the slowness of the official methods. Consequently, Hermida et al. [79] developed calibration equations by MPLS using first and second derivatives with smoothing average, which led to the removal of spectral noise that makes it difficult to extract relevant information, for determining the TN, soluble nitrogen (SN), NPN, and ADIN in different samples of grass silage. The R² values obtained in samples belonging to the validation set were 0.94, 0.92, 0.90, and 0.48 for TN, SN, NPN, and ADIN, respectively. The latter value indicates that NIRS is an acceptable method for the semiquantitative determination of the ADIN fraction.

Nie et al. [80] established calibration equations to predict the total crude protein (CP), true protein (TCP), neutral detergent insoluble protein (NDFCP), and acid detergent insoluble protein (ADFCP) contents in samples of alfalfa. For CP the statistical parameters were R²_p = 0.96 and RPD_p = 5.07; for TCP they were R²_p = 0.91 and RPD_p = 3.31. However, for NDFCP (R²_p = 0.75, RPD_p = 1.98) and ADFCP (R²_p = 0.83, RPD_p = 2.42), the prediction was less precise. With these results, the NIRS technique was able to simplify the long and tedious process that determination in the laboratory entails, and predict quickly and empirically the degradability of the alfalfa protein in the rumen; also, these results could be extrapolated to proteins from other forage.

With respect to carbohydrates, Nousiainen et al. [81] established calibration equations to predict neutral detergent fiber (NDF), indigestible neutral detergent fiber (INDF), and digestible neutral detergent fiber (DNDF). For the development of the equations the authors used the MPLSR model and the mathematical treatments: standard normal variate transformation, detrending (SNV-D), and first-order derivatization. The SNV transformation removed scatter effects from spectral data, and corrected scattering effects caused by physical differences between samples. In these parameters, a scatter correction with the standard normal variate transformation combined with detrend eliminated background spectroscopy. The statistics obtained in the cross validation were R²_cv ranging from 0.82 to 0.91 and an RPD_cv between 2.39 and 3.33. These authors concluded that the NIRS technique has great potential to predict INDF in grass silage.

Cozzolino et al. [82] developed equations to predict the organic matter, dry matter (DM), acid detergent fiber (ADF), NDF, CP, pH, and in vitro organic matter digestibility (DOM) in samples of ensiled whole plants of maize, using second derivative with SNV-D and MSC, which eliminated background spectroscopy. The best statistics obtained in the cross validation were for DM, CP, and ADF, with R² values of 0.85, 0.91, and 0.86, respectively. However, for DOM, NDF, and pH the R² values showed poor predictive ability, being 0.53, 0.60, and 0.51, respectively. A study by Fassio et al. [83] of samples of ensiled maize kernels found similar values of R² for DM, CP, and ADF; however, the R² values for DMO and pH were higher (0.84 and 0.90, respectively). These authors also obtained an R² of 0.90 for the prediction of the content of ammonia nitrogen (NH₃-N). In this work, the use of the jack-knifing method improved the calibration models obtained. It is used to evaluate the stability of the calibrations and to eliminate nonsignificant wavebands in the calibration.

For biological parameters such as the in vitro digestibility (IVD) and metabolizable energy (ME) in pastures, Lobos et al. [84] established prediction equations with RMSEP values of 3.06 and 0.06 and R²_p values of 0.90 and 0.94, respectively. The reliability of prediction of these NIRS parameters may be affected by the particle size, the effect of drying the sample prior to analysis, and the residual moisture in the samples after drying. Lovett et al. [85] studied such effects for samples of maize silage, with regard to the prediction of these parameters, concluding that the particle size was the most important factor, followed by the drying process and finally the presence of residual moisture. These authors used three statistical treatments—PLS, MPLS, and PCR—with two standard data preprocessing methods: standard normal variate (SNV) followed by detrending and first derivative. These authors used three statistical treatments—PLS, MPLS, and PCR—with two standard data preprocessing methods: standard normal variate (SNV) followed by detrending and first derivative.

Other authors have developed equations using NIRS to determine quality parameters that indicate whether the silage fermentation has been correct, and for quantification of fatty acids in forages. Thus, Sorensen et al. [86] developed prediction models (using PLS on full scan mean spectra after scatter correction with the standard normal variate (SNV) transformation combined with detrend and applying a second derivative) for the determination of lactic acid (Lac), acetic acid (HAc), pH, and NH3-N in maize silage. The RMSECV values were 4.7, 1.9, 2.4, and 2.9, respectively, and 4.0 for ethanol (EtOH). These authors showed that the NIRS technique is less accurate for HAc, but provides an estimate of its concentration. With regard to the quantification of fatty acids in fodders, Foster et al. [87] obtained high coefficients of determination for calibration (0.93–0.99) and cross-validation (0.89–0.98). The SEC and SECV were 20% lower compared to the mean. The RPD_CV was greater than 3 for all fatty acids except C12:0 (2.6) and C14:0 (2.9). The reliability of the prediction was lower, but acceptable for C12:0, C14:0, C18:0, C16:1, and C18:1. In this study, two limits were used for the validation of the prediction equations: GH (global spectral distance) and NH (neighborhood spectral distance) to determine if significant bias occurs and if there is a significant increase in unexplained error.

Finally, undesirable substances of a toxic nature can be found in animal forages. Fox et al. [88] established calibration equations for the estimation of hydrogen cyanide in forage sorghum. The equations developed by MLR gave a coefficient of determination (R²) of 0.847 and an SEC value of 0.050%, with R² and SEP values for the validation of 0.829 and 0.057%, respectively. These authors found two important wavelengths for the prediction: 2034 and 2458 nm, associated with the former C=O carbonyl stretch and the latter associated with C-N-C stretching.

4.6. Organic residues and compost

The addition of value to wastes that are organic in nature is required to help reduce the increasing pollution, to optimize the use of available resources, and to offset the increasing energetic and economic costs of synthetic fertilizers. For this it is essential to know in detail the nature and type of such wastes, which can be used both fresh and stabilized. In this section we focus on fresh organic waste and stabilized materials—compost—resulting from the composting process. The origin of these organic wastes can be varied, but generally they originate from urban solid waste, sewage sludge generated in waste water treatment plants, and the agro-industrial sector.

The agri-food industry is one of the most important sectors in Europe and therefore the wastes it generates pose a serious environmental problem. Most of these organic wastes are considered biodegradable and derive from plants and animals. They include fruit and vegetable remains and crop pruning waste. Since these materials are organic or have a high organic matter content, usually with significant amounts of macronutrients as well, it is convenient to use them as organo-mineral fertilizers. This represents a double energy saving: first, waste is eliminated and, second, the need for synthetic fertilizers, whose cost has increased in recent years, is diminished [88].

In the treatment of organic wastes their possible uses must be taken into account, as the treatment determines the characteristics of the final product obtained. The fate of these organic wastes has been and is still very varied, depending on geographical location, activities taking place in a region, the population, facilities for reuse, and current regulations governing their handling and use. They can be used as soil conditioners, allowing long-term improvement of the physical properties of soils, reducing erosion, and helping the recovery of unproductive marginal areas. Another possibility is their use as substrates for the production of ornamental and horticultural plants. This requires improvement of the physical characteristics of the sludge or residue in question, which is achieved by composting [89].

Fresh organic wastes can be recycled by composting, a controlled bio-oxidative process involving numerous and varied microorganisms and requiring adequate moisture and heterogeneous organic substrates in the solid state. It involves a thermophilic stage and a temporary production of phytotoxins, giving—as the end-products of the degradation processes—carbon dioxide, water, minerals, and stabilized organic matter, free of phytotoxins and ready for use in agriculture without the risk of adverse phenomena. Finally, the compost, that can be defined as the product resulting from the composting process and maturation and that consists of stabilized organic matter like humus, is obtained. It has little resemblance to the original organic material as it will have been degraded, resulting in finer and dark particles. It is a product that is safe and free of phytotoxic substances, whose application to the soil will not cause damage to plants and which can be stored without further treatment or alterations [90, 91].

Thus, NIRS is used to predict different parameters and/or mineral elements in different organic residues and compost. The NIRS calibration results used successfully by different authors working with different types of organic matrices (industrial compost, compost of various animal manures, compost based on sludge and vegetable waste, compost based on winery and agro-industrial waste, compost derived from tofu waste and sewage sludge) show the great interest and the extent of use of this technique in the study of different variables in this type of organic material. In this regard there are several studies of compost or organic waste which highlight that the information generated with NIRS can increase the effectiveness of composting as a management method, due to the advantages that this technique presents as we have already discussed throughout this chapter. Thus, NIRs has been used to determine the next parameters.

4.6.1. Contents of mineral nutrients in compost

Mineral analysis of the materials at the start, during, and at the end in the composting process is desirable to ensure that the input materials are within acceptable ranges. To determine the mineral nutrient content it is necessary to make a mineral analysis of the compost material by ICP-OES and AA. In addition, nitrogen is one of the most important nutrients in the compost. When we analyze its total content, we refer to the sum of inorganic forms (ammonium, nitrate, and nitrite; NH₄⁺, NO₃^-, NO₂^-, respectively) and organic (amino acids, proteins, nucleic acids, and other organic compounds having nitrogen in their structure). The usual methods for its analysis have been the Kjeldahl method (wet digestion) and the Dumas method (dry digestion). Although these conventional methods have been optimized, the digestion of the sample is still carried out with sulfuric acid and a series of catalysts which causes problems such as the emission of acid gases into the environment. Currently, near-infrared reflectance spectrometry technique (NIR) is available today and can be used to determine mineral nutrient content in organic residues and compost [91, 92]. Malley et al. [91] used a field-portable Corona 45 VIS NIR (visible/near-infrared) spectrometer (Carl Zeiss, Germany) from 360 to 1690 nm to measure total N, ammonium-N, organic N, P, K, Ca, Mg, S, Mn, Zn, and Cu in manure coming from beef cattle manure, stockpiled manure, and compost. The calibrations were developed for each constituent separately by using PCA/PLS1 in The Unscrambler. The calibrations were successfully developed for all parameter measurements (except available P, nitrate + nitrite, or Na). Therefore, field-portable NIRS offers a considerable advance over existing field and laboratory methods by providing rapid, comprehensive compositional analysis when and where the information is required to assist with management of the nutrients of cattle manure.

Usually, heavy metals are also measured in the composts as they can cause toxicities in the plants (Cu, Hg, Cd, Ni). In several publications, it has been observed that NIR technique can be successful to measure these metals in compost coming from raw material rich in these metals [88, 93]. An interesting study was carried out by Shen et al. [94], who investigated the use of NIR to detect copper (Cu) in animal manure. A total of 118 pig manure samples were collected from four provinces in China, and spectra were acquired in the range of 10,000–4000 cm^-1. Results showed that the prediction of Cu concentration in pig manure was feasible (r² = 0.84, RMSE = 198 mg/kg; SE/SD = 2.4). Although the heavy metals in the vis-NIR region do not absorb energy, Cu in pig manure can be detected by NIR spectroscopy because a high percentage of the Cu is complexed with CONH₂ or CONHR functional groups of organic ligands such as protein, urea, amino acids, and other amide compounds.

4.6.2. Organic matter and total organic carbon

Compost maturity has often been associated with the degree of compost humification. Compost stability refers to the degree to which composts have been decomposed to more stable organic materials. Various global parameters have been currently used to assess both maturation process and quality of the final product, including physicochemical properties, such as C:N ratio, humified organic and water-soluble carbon, and cation exchange capacity. Methods for measuring total C in soils, such as wet combustion or dry combustion, are generally very accurate, but too slow or costly for everyday analysis. It has been measured by NIR the contents of carbon and nitrogen in sewage sludge and green waste compost [95], and sewage sludge [96] with successful results. So, Albrecht et al. [95] analyzed changes in composts of sewage sludges and green wastes by NIRS technique of six stages of composting: 8, 20, 35, 75, 135, and 180 days. Maturity of compost was assessed through changes in C:N ratio. Results of spectroscopic properties (200 wavelengths) were studied with several multivariate analyses showing a precise calibration models between spectral data, the C, N, C:N values, and composting time were build using partial least square regression (r² > 0.95). Together, these results show the efficiency of NIRS to predict chemical changes and the stage of transformation of organic matter during the composting process.

Humic acids from sewage sludge. Humic acids are part of the stable organic matter fraction in soils and composts. Due to their favorable properties for soils and plants, and their role in carbon sequestration, they are considered a quality criterion of composts. The traditional methodology for determining the content of humic acids is based on the solubility of the humic substances in aqueous media of different pH, i.e., humins are insoluble in any pH range, humic acids are insoluble in acid medium, and fulvic acids are soluble throughout the pH range. In the determination of humic acids by NIR in compost, a correlation coefficient of 0.94 and a standard error of estimation of 0.28 were obtained, values that can be considered very acceptable [97]. Other publications in mushroom compost [98], manure [99], and fattening pig manure [100] have given excellent results to characterize the humic acids in this material.

In summary, numerous studies of compost or organic wastes using NIRS have demonstrated the efficacy of this methodology. For all the above reasons, this spectroscopic tool is an emerging technique in the analysis of environmental parameters. It offers several advantages over traditional analytical techniques, such as rapidity, ease of preparation and handling of samples (no reagents are required), and low cost.

Nomenclature

ANN

Artificial Neural Networks

HAc

Acetic acid

ADF

Acid detergent fiber

ADIN

Acid detergent insoluble nitrogen

ADFCP

Acid detergent insoluble protein

CEC

Cation exchange capacity

R²/r²

Coefficient of determination

R²c/R²p/R²cv

Coefficient of determination of the calibration/prediction/cross-validation

CP

Crude protein

Cys

Cystine

DT

Detrending

DM

Dry matter

DDM

Digestible dry matter

DNDF

Digestible neutral detergent fiber

DOM

Digestible organic matter

EtOH

Ethanol

FD

First derivate

GH

Global spectral distance

INDF

Indigestible neutral detergent fiber

IVD

In vitro digestibility

IAD

Index of Absorbance Difference

DRIS

Integrated System of Diagnosis and Recommendation

Lac

Lactic acid

LOS

Linear offset subtraction

Lys

Lysine

H

Mahalanobis distance values

MMN

Maximum-minimum normalization

ME

Metabolizable energy

Met

Methionine

MPLSR/MPLS

Modified partial least-squares regression

MLR

Multiple linear regression

MSC

Multiplicative scatter correction

MARS

Multivariate adaptive regression splines

NIRS

Near infrared reflectance spectroscopy

NH

Neighborhood spectral distance

NDF

Neutral detergent fiber

NDFCP

Neutral detergent insoluble protein

NDSC

Neutral detergent-soluble carbohydrates

NDSF

Neutral detergent-soluble fiber

NDP

No data preprocessing

NPN

Nonprotein nitrogen

NMSI

Normalized soil moisture index

PLSR/PLS

Partial least squares regression

PCR

Principal components regression

A

Relative absorbance values

RPD

Residual prediction deviation

RPDc/RPDp/RPDcv

Residual prediction deviation of the calibration/prediction/cross-validation

RMSECV

Root mean square error of cross validation

RMSEE

Root mean square error of estimation

RMSEP

Root mean square error of prediction

SED

Second derivate

SWIR

Shortwave infrared

SN

Soluble nitrogen

TSS

Soluble solids content

SOM

Soil organic matter

SS

Spectral smoothing

SEC

Standard error of calibration

SECV

Standard error of cross-validation

SEP

Standard error of prediction

SNV/SNV-D

Standard normal variate, detrending

SR

Sufficiency range

SSE

Squared sum estimation

t-test

Student's t-test

SSL

Subtraction of a straight line

Thr

Threonine

TA

Titratable acidity

TN

Total nitrogen

TCP

True protein

Trp

Tryptophan

VN

Vector normalization