Absorbance signals in the near infrared for the major chemical groups present in organic matter.
This chapter provides a review on the state of art of the use of the visible near-infrared (vis-NIR) spectroscopy technique to determine mineral nutrients, organic compounds, and other physical and chemical characteristics in samples from agricultural systems—such as plant tissues, soils, fruits, cocomposted sewage sludge and wastes, cereals, and forage and silage. Currently, all this information is needed to be able to carry out the appropriate fertilization of crops, to handle agricultural soils, determine the organoleptic characteristics of fruit and vegetable products, discover the characteristics of the various substrates obtained in composting processes, and characterize byproducts from the industrial sector. All this needs a large number of samples that must be analyzed; this is a time-consuming work, leading to high economic costs and, obviously, having a negative environmental impact owing to the production of noxious chemicals during the analyses. Therefore, the development of a fast, environmentally friendly, and cheaper method of analysis like vis-NIR is highly desirable. Our intention here is to introduce the main fundamentals of infrared reflectance spectroscopy, and to show that procedures like calibration and validation of data from vis-NIR spectra must be performed, and describe the parameters most commonly measured in the agricultural sector.
- vis-NIR spectroscopy
- calibration and validation methods
- plant mineral analysis
- fruit analysis
- soil analysis
- fruit organoleptic characteristics
One of the challenges of the twenty-first century is to achieve a more productive agriculture, while improving the safety and quality of food. The food industry has to feed a population that is in continuous increase, bearing in mind that these systems have to respect the environment, should optimize natural resources in each area, and anticipate changes in temperature and rainfall that will occur in the future as a result of climate change. Proper soil management and fertilization of crops will be crucial to increasing the capacity of agriculture, to the provision of products of high added value, and to the protection of crops against pests and diseases. To do this, in each of the steps ranging from the production of fruits and vegetables in the field to the development of industrial products, it is necessary to determine a great number of physical and chemical parameters in the soil, plants, fruits, compost, and byproducts from food processing industries. Currently, the traditional techniques of analysis of such samples are being replaced by spectroscopic techniques—one of which is visible near-infrared spectroscopy (vis-NIRS). This technique has a number of advantages over the traditional methods, as it (i) is a method of nondestructive analysis, (ii) does not pollute the environment, because it does not use chemical reagents, (iii) is cheap and fast, (iv) measures many parameters in a single analysis, and (v) can perform analyses
The aim of this chapter is to provide an updated review of the current state of vis-NIRS as a technique for the estimation of physical and chemical parameters in samples derived from agricultural systems, such as soils, plants, fruit, compost, and products derived from food processing industries. The chapter starts by describing the basic principles of this technique and the different ways in which the equipment can be calibrated, detailing the statistical tools that are useful to establish that the calibration and the estimation of the desired parameters are valid. We will describe the parameters that can be measured by vis-NIRS in samples, with the emphasis on soil, plants, fruit, compost, and byproducts from the industrial sector that processes the output of agricultural systems. A basic explanation of the parameters measured in these samples will be given, together with a description of how they are measured and the mathematical tools used, focusing on the most novel issues.
2. Fundamentals of infrared diffuse reflectance spectroscopy
Spectroscopy in the near infrared or NIRS (near-infrared reflectance spectroscopy) is a tool that has been used widely for the rapid determination of organic components. For example, NIRS readout for nutrient level estimation on citrus leaves, using FT-NIR spectrometer and 64 scans per sample, takes 1–2 min. The only pretreatments of the sample required prior to analysis are drying, crushing, and mixing, in the case of solid matrices. Samples can also be scanned when fresh, as in the work of Huang et al. . All this bestows on this technique several advantages over other, more sophisticated spectroscopic or analytical methods. The operating principle of the NIRS technique requires that the energy absorbed in the near-infrared region by a sample causes covalent bonds of C-H, O-H, and N-H, important components of organic substances, to vibrate in different forms . Within the field of NIRS, two main types of fundamental vibrations are considered: stretching, which involves a change in the length of a bond, and bending, which involves a change in the angle between two bonds. Overtones appear when a vibrational mode is excited at a frequency higher than that of the fundamental vibration.
This infrared fraction comprises wavelengths between 780 and 2500 nm (12,500–4000 cm−1, expressed as a wavenumber, Table 1).
|Group||Aliphatic hydrocarbons||Aromatic hydrocarbons||Carboxylic acid||Amines||Water|
|Frequency range (cm−1)||9100–7800 (overtone of CH-stretching)||ca. 9000 (overtone of CH-stretching)||ca. 6900 (overtone of CH-stretching)||7000–6500 (overtone of NH-stretching)||7500–6400 (overtone of OH-stretching)|
|7700–6900 (combination)||7300–6900 (combination)||ca. 5250 (overtone of CO-stretching)||5200–4500 (combination)||5400–4900 (combination)|
|6300–5500 (overtone of CH-stretching)||ca. 6000 (overtone of CH-stretching)||4900–4600 (combination)|
|5000–4100 (combination)||4700–4000 (combination)|
There is a relationship, both quantitative and qualitative, between the chemical composition and the spectrum recorded in the near-infrared. Hence, samples having different organic compositions have different infrared spectra. But, interpretation of the spectra is tremendously complex, although the spectral characteristics of each compound are unique, as their amplitudes sometimes overlap.
Before the NIR spectrum of a sample can be used for the determination of a compound or specific element, a calibration for this compound or element must be developed. In an NIRS spectrum, the various constituents of the sample have some overlapping peaks; thus, the measurements made with NIRS must be calibrated with samples of known chemical composition in order to extract the desired information using NIRS .
3. Calibration and validations of data from NIRS
Chemometrics includes all methods of multivariate calibration in the field of analytical chemistry. Unlike univariate calibration, where a spectral peak (height or area) is correlated with the reference concentration, multivariate calibration uses the entire spectrum structure with a large amount of spectral information to correlate with the reference concentration.
The establishment of a model for the use of NIRS data in the analysis of samples consists of the following steps: (1) introduction of the spectral and concentration data; (2) preprocessing of the spectral data; (3) definition of the appropriate frequency range; (4) validation and optimization of the method; (5) definitive calibration; and (6) routine analysis.
(1) Introduction of the spectral and concentration data. This process begins with the selection of the group of samples for calibration, which must be well defined statistically, and pretreatment of the samples to assess measurement errors. The dispersion of incident radiation, also known as the scatter effect, produces a low selectivity (quality of being able to tune in to one particular frequency while blocking out other unwanted frequencies) of the NIR spectral information . This is due to physical phenomena—such as the texture, size, and geometry of the particles that make up the sample [4, 5]—and to changes in the refractive index of the material which interacts with the radiation, causing numerous unwanted variations in the NIR spectral data [6–9]. Depending on the complexity of the samples, between 20 and 200 samples are necessary to develop a multivariate calibration method. The greater the number of samples, the more representative is the calibrations achieved. The samples should have a normal distribution, cover the entire range of concentrations of the parameters that are to be estimated by NIRS, and should not have areas where uncertainty is high and errors can be significant. For instance, the NIR spectrum of water (transmission measurement, optical path length: 2 mm) shows a total absorption between 5200 and 4000 cm−1 and below 4000 cm−1 a strong contribution of spectral noise. Finally, for each sample a classical analysis of the desired components is carried out, to obtain the so-called reference values, and its NIR spectrum is obtained.
(2) Preprocessing of the spectral data. The spectral pretreatment that improves the signal/noise ratio must be chosen. For example, the problems of baseline displacement need to be eliminated. The procedures for preprocessing of the NIRS spectrum, to obtain a good correlation between the spectral data and the concentration, include: no data preprocessing (NDP), first derivate (FD), application: it is used to emphasize pronounced, but small features compared to enormous broad-banded structures or on the evaluation of broad bands that get a steeper shape, so it can be evaluated more easily; second derivate (SED), application: similar to first derivative, where even extremely flat structures can be evaluated, but the spectral noise is enhanced as well, the most widely used methods here are the Savitzky-Golay  and Norris  methods; standard normal variate (SNV ; multiplicative scatter correction (MSC [3, 12]), application: it is used for measurements in diffuse reflection; detrending (DT), which is usually applied in conjunction with SNV; spectral smoothing (SS), for which the most used are the Savitzky-Golay  and Fourier transformation  or vector normalization (VN), application: in a measurement in diffuse reflection, the interfering influences of different material densities or particles sizes can often be minimized; maximum-minimum normalization (MMN), application: similar to vector normalization; subtraction of a straight line (SSL), application: a linear tilt of the baseline shift is eliminated; linear offset subtraction (LOS), application: linear baseline shifts are eliminated. The optimum method depends on the system to be analyzed. Generally, SSL, VN, or FD leads to better calibration.
(3) Definition of the appropriate frequency range. Once the calibration samples have been selected and then analyzed by the reference method and NIRS, a correlation between the spectral and analytical data is searched for . For this purpose different statistical treatments are used, such as multiple linear regression (MLR ), principal components regression (PCR), and partial least squares regression (PLSR) as linear methods and use of artificial neural networks (ANN) as a nonlinear method. PLSR is the one most commonly used [16, 17]. The best correlated frequency ranges are assessed and then selected based on the coefficient of determination,
(4) Validation and optimization of the method. To choose the best calibration for the regression equation with linear models (PLS algorithm, for instance), the instrumental software combines different methods of data pretreatment and frequency ranges. Then, it provides as output the corresponding mean error of prediction and
|Number||Data preprocessing||Frequency ranges (cm−1)||Optimum rank||Coefficient of determination (||Mean error of prediction (%)|
There will be anomalous samples and spectra, called “outliers,” which affect the predictive ability of the NIRS equations obtained . It is important not to remove them until one has a clear explanation; to make this decision one has to take into account the
Chemical outliers can be recognized after applying a
To detect spectral outliers, the Mahalanobis distance is particularly useful. For MLR models it is calculated as follows:
For the models PLSR and PCR, it is expressed with the following equation:
The statistics used in the evaluation, selection, and validation of the calibration equations are as follows:
In the presence of laborious and troublesome datasets, it is possible to ask for high-performance external NIR calibration services such as those provided by private companies to optimize and validate the method.
(5) Definitive calibration. After all the “outliers” have been eliminated and the optimal parameters determined (for example,
(6) Routine analysis. Here the optimum chemometric model is used to analyze quickly unknown samples. The Mahalanobis distance can alert one to samples (“outliers”) that are outside the calibration range or do not fit the model well.
4. Mineral nutrients and organic compounds in different samples from agricultural systems
Fruit and vegetable crops, in order to achieve good vegetative growth and maximum production with good quality fruit, require a good nutritional status, maintaining a proper balance of nitrogen, potassium, phosphorus, and trace elements such as manganese, boron, copper, and magnesium. The guide to the nutritional status of crops is based on the method known as “sufficiency range” (SR), which establishes—for each nutrient—the ranges of values considered to be normal and to represent deficiency or toxicity ; or it can be based on the establishment of indices of dependent nutrients, in which each index includes two or more nutrients, the so-called “Integrated System of Diagnosis and Recommendation (DRIS).” But, a good fertilization program also should pay attention to the changes that occur in the mineral status of plants in their different phenological stages so that the application of fertilizers can be adapted to the requirements of the plants at all times. To obtain this information, it is necessary to perform mineral analyses of leaves by ICP-OES or AA after acid digestion of the samples, as well as analysis of the C/N ratio, which involves the analysis of a large number of samples with all that this entails. Currently, the available knowledge of reflectance spectroscopy in the near-infrared (NIR) part of the spectrum can be used to determine the nutritional status of crops quickly and cheaply. The mineral composition of an organic matrix can be estimated by NIRS, from the spectra in the range 700–2500 nm, due to the association between the minerals and the organic functional groups or the organic matrix itself . There are no infrared absorption bonds in the mineral species of macro- and micronutrients, but NIRS determines bonds within organic compounds that are negatively related to inorganic materials. If mineral matter is bound to organic compounds, the distortion of the spectrum is detectable at certain wavelengths, suggesting that NIRS can quantify inorganic materials using their ratio to the organic matter .
Numerous studies show that the NIRS technique, together with multivariate analysis and partial least squares regression (PLSR), provides a powerful tool for the interpretation and analysis of spectra. For example, NIRS technology has been used successfully to predict the nutritional status of leaves of apple , alfalfa , sugar cane , root crops , yerba mate , and citrus [35, 36]. It has been observed in citrus leaves of different varieties including lemon, mandarin, orange, and grapefruit—high accuracy regarding the estimation of N (
Soil is a natural resource that is vital in agriculture for the production of food, fiber, and energy; but it serves also as a platform for human activities, constitutes an element of the landscape, is an archive of cultural heritage, and plays a central role as a habitat and gene pool. It stores, filters, and transforms many substances, including water, nutrients, and carbon (C). In fact, it is the largest C “store” in the world (1500 gigatonnes). All these functions must be protected because of their socioeconomic and environmental importance.
Fundamentally, the soil is a complex matrix of organic matter, minerals, water, air, and microorganisms. The soil organic matter is only a small part, but it plays a big role in both the physical and chemical properties of soil as well as in the development of crops. This soil fraction comprises humus (material that is decomposed, dark, and colloidal in nature) and materials such as the roots and aerial parts of plants and the bodies of insects and other animals that are deposited on the ground. The content of organic matter normally found in the soil is small, only about 1–5% by weight, of which 85–90% is humus and only a small part is the nonhumified remains. The mineral phase is a mixture of materials that differ in their composition and properties. Typically, this fraction is characterized by particle size. Stones, gravel, and sand represent the coarse fraction, while smaller particles like silt and clay constitute the fine fraction of the soil. The clays also can be classified according to the negative charges on their surfaces—some minerals are more negative than others—and this property also influences the chemical characteristics of the soil. There is a parameter that is very useful for measuring these chemical properties, namely, the cation exchange capacity (CEC)—defined as the maximum amount of cations that a soil can fix. Water is another significant fraction of the soil, and its content depends on the amount and size of the pores in the soil. In plants, water is the major constituent of protoplasm (85–95%) and is essential for physiological processes such as photosynthesis, nutrient transport, and maintenance of turgor. The air is another important fraction of the soil. Its oxygen is essential for the respiration of roots and microorganisms. When aeration is poor, organic matter is oxidized slowly, the activity of aerobic microorganisms is paralyzed, and only anaerobes are active, giving rise to reduced forms of elements that are usually toxic to plants. The most common microorganisms in soils are nematodes, protozoa, and rotifers—whose activities also determine the physiochemical soil characteristics, as they have the ability to degrade highly resistant organic compounds such as cellulose and lignin, and can even degrade minerals, thus releasing plant nutrients.
The study of soil should take into account the different phases of the soil and must be directed toward two main objectives: (i) consideration of its various properties, with special emphasis on plant productivity (that is, practical or applied aspects); and (ii) scientific, especially chemical, study—to determine the variation of productivity and find ways for soil conservation and improvement. In recent years, it has been observed that NIRS (using visible-near infrared) can be very useful for characterization of soils. This technique has many advantages: sample preparation is easy as it only requires the drying and grinding of the soil, reagents nontoxic to the environment are used, measurements are made in a few seconds, a single scan can show multiple properties, and the technique can be used both in the laboratory and
4.2.1. Soil organic matter (SOM)
In the laboratory, it is difficult to separate the organic and inorganic material of a soil, so an estimate of the content of organic matter is obtained indirectly through the analysis of an element that is a constituent of all organic substances, namely C. Once the amount of organic C present in a soil sample is known, the percentage of organic matter with respect to the total weight of the soil can be estimated indirectly. Classically, it has been determined by various methods—such as calcination of the soil sample, oxidation with potassium dichromate, or oxidation with hydrogen peroxide. With regard to NIRS, a large number of studies have shown that this analytical technique is very useful for estimating SOM. For this, absorption bands in the NIR region which result from the stretching and bending of NH, CO, and CH groups, that form part of the organic material, are used .
The 1990s saw the identification of the absorption bands of wavelength (nm) 1100, 1600, 1700–1800, 2000, and 2200–2400 as the most useful for measuring organic C. Since 2000, the technique of NIRS has been perfected and adapted to the soil and climatic conditions of each area. Thus, in Australian soil, it has been also observed that if the absorption spectrum was made in the vis-NIR region—that is, including the visible region—better results were obtained than with NIR alone . Another problem that researchers have faced in achieving good calibrations has been that the spectral response can change depending on the mineral fraction of the soil, composition of organic matter, texture, and soil moisture content [40–42]. All these problems have been solved by optimizing the way of taking samples (local and regional scales) [43, 44], choosing the most appropriate mathematical models for the calibration [45–48], establishing covariance models, or eliminating certain factors that make the model weak [49, 50]. For example, in saline soils of El-Tina Plain (Egypt), it has been compared several regression techniques to estimate the organic matter content of soils . Specifically, they used PLSR, support vector regression (SVR), and multivariate adaptive regression splines (MARS) and found that the best calibration was obtained with MARS with continuum removed reflectance preprocessing (
4.2.2. Soil mineralogy
The mineral fraction of the soil occupies almost half of the soil volume. Its composition and concentration as well as the proportion of different minerals determine important properties such as texture, structure, and CEC. These properties also determine other soil characteristics such as the availability of nutrients to agricultural crops. Classical methods for the determination of clay minerals are qualitative and are based on XRD (XTR). However, some researchers [53, 54] made the first tests to see if NIRS could be used to estimate soil minerals; later, these same authors  compared the NIRS results with XRD analysis, concluding that NIRS is an effective method to determine the mineralogy of the soil. This is because soil minerals absorb light in the UV, visible, vis-NIR, and mid-NIR parts of the electromagnetic spectrum. For example, Fe oxides absorb UV radiation while phyllosilicates (clay minerals) have varied spectra in the vis-NIR. Overall, this technique has been used to estimate the Fe oxides goethite (α-FeOOH) and hematite (α-Fe2O3), clays of the kaolinite, illite, and smectite types, and carbonates [56, 57].
4.2.3. Soil texture and CEC
The water dynamics and aeration of a soil depend on its structure and texture, and these parameters are important for the development of both plants and microorganisms, so they need to be evaluated. These parameters also determine the leaching of fertilizers and pesticides in agricultural soils. Generally, soil texture is defined as the ratio (in percentage by weight) of particles smaller than 2 mm in diameter and classified as sand (2–0.02 µm), silt (0.02–0.002 µm), or clay (0.002 µm). Ben-Dor and Banin  found that the clay content may be estimated by analyzing the absorption bands of O-H in water, and those of Mg-, Al-, and Fe-OH in the mineral fraction of the soil. Curcio  used visible and near-infrared (VNIR, 400–1200 nm) and shortwave infrared (SWIR, 1200–2500 nm) reflectance domains to estimate soil texture in three agricultural areas of Italy (Bompensiere, Dirillo, and Pietranera), and obtained a good calibration by using the PLSR method, the accuracy being good for the clay fraction (RMSE = 5.8%,
The CEC is traditionally measured by the method of Chapman , based on saturating the soil with sodium. However, the vis-NIR technique can also estimate this parameter in soils accurately, if methods of multivariate regression are used instead of simple bivariate relationships, and it is suitable for measurements of peak intensities in the mid-IR range. Recently, Ulusoy et al.  obtained a good prediction of CEC using an analysis of PLSR, both in the laboratory and for online measurements in the field—although the calibration was much better for the data obtained in the laboratory.
4.2.4. Plant nutrients
Due to the importance of the mineral nutrition of plants in the yield and quality of fruit and vegetables, one of the most common practices in agriculture is the analysis of the soil content of N, P, K, Fe, Ca, and Mg. This information is particularly important when optimizing fertilization programs. In most farming systems N is the element most commonly applied, followed by K, P, Ca, and micronutrients. These nutrients do not have a specific absorption spectrum in the vis-NIR region. Generally, the correlations between the “real” concentrations of these nutrients and those estimated by NIR are highly variable, the variability coefficients (
Soil pH is an important regulator of fertilization. The solubility of the nutrients is pH-dependent, and plants may decrease nutrient uptake if the soil pH is not suitable. Other pH-dependent factors include the biological activity, decomposition of organic matter, and mineralization. Generally, the ideal soil pH for plants is between 5.5 and 6.5. Soil pH, or more specifically the H+ ions, has no direct response to NIR but its value can be estimated well with this technique if the appropriate covariations are applied to components that do exhibit activity in the NIR, such as organic matter and clays . In different experiments it has been found that pH calibration gives
4.2.6. Heavy metals and other soil contaminants
Heavy metals are potential pollutants of air, water, and soil and of plants when taken up in sufficiently high amounts; this pollution will also affect other links in the food chain. In most agricultural soils, there are small amounts of As, B, Cd, Co, Cr, Cu, Mo, Mn, Ni, Se, and Zn, but when normal values are exceeded this can cause soil pollution and phytotoxicity, negatively impacting the agronomic performance. Usually, heavy metals are measured by atomic absorption or ICP. In the vis-NIR region these metals do not absorb energy, but their concentrations can be estimated if used as covariates with other components that do possess absorption spectra . For example, they can be related to the organic matter, hydroxides, sulfides, carbonates, oxides, clay minerals, or soil texture (Stenberg et al. ). Todorova et al.  investigated the use of NIRS to estimate the concentration of heavy metals (Zn, Cu, Pb, Cr, and Ni) in various soils of Stara (Zagora Region, Bulgaria), using the PLS type of calibration. The best validation of the method was observed with Cu, while it allowed estimation of whether the concentrations of Zn, Pb, and Ni were low or high; however, Cr gave the weakest validation. These authors also noted that as the number of samples in the validation process increased, the RMESP values decreased.
In soils, hydrocarbons can also be measured with the vis-NIR technique, to establish the degree of contamination of soils that have suffered spills of petroleum products. Okparanma  used this technique to make soil maps in which the concentration of polycyclic aromatic hydrocarbons and their equivalent toxic concentrations in soil from Niger (Nigeria) were indicated. The data of this study revealed that the elaboration of soil spectra between 300 and 2500 nm, together with a PLS calibration, permitted the estimation of the concentration of hydrocarbons without significant differences from the results obtained by the conventional method of gas chromatography-mass spectrometry.
4.2.7. Soil moisture
There are a multitude of reasons to measure the water content in agricultural soils because water is fundamental to the development of plants and for soil biology, besides regulating important processes in the soil such as nitrification/denitrification, leaching, and erosion. In the laboratory, although there are many methods of measurement, the traditional one is based on weighing the sample fresh and then after drying, calculating the percentage of water relative to the dry weight of soil. Water produces an absorption spectrum in the NIR due to expansions and stresses of O-H bonds. Water incorporated into the soil in clay minerals absorbs at wavelengths around 1400 and 1900 nm. The main problem researchers have had in the calibration of this parameter is due to the fact that the water found on the surface of the minerals in thin layers and in the pores tends to decrease the albedo (the percentage of radiation that the soil reflects), changing the refractive index. As the porosity and refractive index of soil particles vary between soil types, a relationship between the albedo and water content cannot be given. However, some authors have successfully used a multivariate calibration with data from the NIR spectral bands to estimate the water content. For example, Bullock et al.  found a good correlation using a PLSR of the regions of 1100–2500 nm, and Ben-Dor and Banin  produced a good regression for samples having a water content of 0.2–11.6% using an MLR (multiple linear regression) calibration. The problem with all these calibrations appears when one wants to have a single calibration for soils of very different geological origins; thus, calibrations at the local scale are recommended [62, 66].
Fruits and vegetables from agricultural plantations must maintain their optimum quality, whether they are destined for fresh consumption or for processing. As quality rises so do prices, so it is necessary to determine the intrinsic characteristics and external appearance of the fruits and vegetables. This information can be used to exclude fruits of poor quality, and can be provided to the consumer/industry to inform them of the added value of the product that is being offered. External defects—such as bruises, injuries from cold and wind, cracks in the skin, and contamination by pathogens—cause significant economic losses. Parameters such as total soluble solids content (TSS), acidity, and water content, which are related to the flavor and aroma of fruit and vegetables, serve to define their organoleptic quality. Therefore, currently, analytical methods are being developed that allow accurate, fast, and noninvasive determination of the qualities of agricultural products. In the case of the appearance of the fruits, computer vision technology—which integrates data acquisition, processing, and analysis—has great potential for the automatic inspection of the appearance of the products. The internal quality of the fruits can be estimated accurately by spectroscopy in the visible and infrared (vis-NIR), because most of the organoleptic characteristics are related to functional groups of the type C-H, N-H, and O-H. What follows is a brief summary of the most significant parameters that can be measured with these spectrometric techniques.
4.3.1. Total soluble solids
This parameter is used to measure the approximate amount of sugars in fruit juices, wine, or liquids processed in the agri-food industry, and is used to track
4.3.2. Total titratable acidity
This parameter describes the total concentration of acids in food, vegetables, or fruit. It is determined by an acid-base titration (soluble acids determined as citric, malic, lactic, oxaloacetic, succinic, glyceric, phosphoric, hydrochloric, fumaric, galacturonic, glyceric, tartaric acids, etc.). Acidity influences the taste of food (roughness), the color, the microbial stability, and the quality of conservation, and is determined by an acid-base titration using 0.1 N NaOH as the base and phenolphthalein as the indicator. Estimations by NIRS of this parameter and the pH of the fruit are as good as those found for the TSS. Thus, with the corresponding calibration data obtained by NIR, values of
4.3.3. Contents of water and dry matter
For the food industry, the moisture content is an important quality factor of fruit and vegetables, whether fresh or processed, and influences their conservation and deterioration. The dry matter content is obviously very important when calculating the contents of other constituents of fruit and vegetables on the basis of the dry matter, which is uniform and less variable than the fresh weight. Water is the major component of all fruit and vegetables, representing between 60 and 96% by weight. The methods used most commonly for its determination are drying methods; the percentage water content is calculated as the loss in weight due to elimination by heating under standard conditions. Pu et al.  stressed that the vis-NIR technique is useful for measuring the water content in fruits of mango, strawberry, mushroom, banana, and soybean using spectra in the range of 400–1000 nm.
4.4. Grains and seeds of cereals, grasses species, and legumes
Cereals are important in animal feed not only because they provide energy but also since they contribute 70% of the protein in the diet and are especially important for pig nutrition. Similarly, soya is a major source of vegetable protein in animal feed formulations. It is important that the animal diets have a proper balance regarding the contents of amino acids.
The main nutritional parameters in the grains and seeds of these crop plants determined by the NIRS technique with regard to animal feed are the moisture and protein contents, representing the biological value. Other parameters analyzed are the contents of lipids, carbohydrates, and ash.
4.4.1. Amino acids
In 1978, Rubenthaler and Bruinsma  developed the first calibration equations for the determination of lysine in wheat and barley. Subsequently, Fontaine et al.  determined the total contents of methionine (Met), cysteine (Cys), lysine (Lys), threonine (Thr), tryptophan (Trp), and other essential amino acids in a population of cereal and sorghum samples. The spectra were first treated with SNV (recommended for samples with <15% moisture) and trend to reduce differences in the spectra that are caused by particle size effects only and not by changes in the constituents. In this way, the validation of the calibration equations showed that 70–98% of the variance of the amino acids in the samples could be explained using the NIRS technique, especially for Lys and Met—the amino acids most limiting to animal nutrition. Also, Kovalenko et al. , in the analysis of soybean samples, applied the MSC mathematical treatment (let to remove background spectroscopy) to the spectral data together with the PLSR regression model and obtained a determination coefficient (
4.4.2. Other organic matter
With respect to dry matter, lipids, total protein, carbohydrate, and ash, Ferreira et al.  and Wang et al.  established models to determine the protein and lipid contents in both soya and fava beans. These authors obtained high
4.4.3. Toxic substances
Some seeds may contain substances that are antinutritional or toxic in nature, such as L-canavanine in seeds of one-flowered vetch. This is a toxic nonprotein amino acid that can cause a reduction in food intake, particularly in nonruminant animals . These authors determined the content of L-canavanine in one-flowered vetch seeds by NIRS: the calibration equation obtained showed a correlation (
4.5. Forage and silage
The production systems of ruminants are based on forage resources. These forages and—more particularly—maize, wheat, and alfalfa can be conserved as silage. Therefore, quick and reliable knowledge of the quality of forage and silage is very important for technicians and producers. The quality of silage depends—on the one hand—on its nutritional value, which is directly linked to its chemical composition (fiber fractions, nitrogenous materials, minerals, carbohydrates), and—on the other hand—on the quality of its conservation, which is defined by the end-products (lactic, acetic, and butyric acids, ammoniacal nitrogen, soluble nitrogen, etc.) of the fermentation processes. Currently, the main constituents determined by the NIRS technique in forage and silage, and which are important in the feeding of ruminants, are the total protein and protein fractions, soluble and structural carbohydrates, and digestibility of the forage, the latter depending on the content of structural carbohydrates.
Thus, Volkers et al.  established calibration equations for samples from different parts of a forage maize crop to predict the crude protein content, obtaining coefficients of determination (
With respect to the nitrogenous fractions, the nonprotein nitrogen/total nitrogen (NPN/TN) ratio in silage is very important for animal nutrition, since it indicates whether the silage has overheated; if the value is greater than 12%, it is considered to have occurred . The acid detergent insoluble nitrogen (ADIN) is the fraction of the total nitrogen that is bound to the cell wall lignin of the plant. This component has low bioavailability but indicates the quality of the silage, as its abundance increases with overheating. Normally, it is not routinely analyzed due to the slowness of the official methods. Consequently, Hermida et al.  developed calibration equations by MPLS using first and second derivatives with smoothing average, which led to the removal of spectral noise that makes it difficult to extract relevant information, for determining the TN, soluble nitrogen (SN), NPN, and ADIN in different samples of grass silage. The
Nie et al.  established calibration equations to predict the total crude protein (CP), true protein (TCP), neutral detergent insoluble protein (NDFCP), and acid detergent insoluble protein (ADFCP) contents in samples of alfalfa. For CP the statistical parameters were
With respect to carbohydrates, Nousiainen et al.  established calibration equations to predict neutral detergent fiber (NDF), indigestible neutral detergent fiber (INDF), and digestible neutral detergent fiber (DNDF). For the development of the equations the authors used the MPLSR model and the mathematical treatments: standard normal variate transformation, detrending (SNV-D), and first-order derivatization. The SNV transformation removed scatter effects from spectral data, and corrected scattering effects caused by physical differences between samples. In these parameters, a scatter correction with the standard normal variate transformation combined with detrend eliminated background spectroscopy. The statistics obtained in the cross validation were
Cozzolino et al.  developed equations to predict the organic matter, dry matter (DM), acid detergent fiber (ADF), NDF, CP, pH, and
For biological parameters such as the
Other authors have developed equations using NIRS to determine quality parameters that indicate whether the silage fermentation has been correct, and for quantification of fatty acids in forages. Thus, Sorensen et al.  developed prediction models (using PLS on full scan mean spectra after scatter correction with the standard normal variate (SNV) transformation combined with detrend and applying a second derivative) for the determination of lactic acid (Lac), acetic acid (HAc), pH, and NH3-N in maize silage. The RMSECV values were 4.7, 1.9, 2.4, and 2.9, respectively, and 4.0 for ethanol (EtOH). These authors showed that the NIRS technique is less accurate for HAc, but provides an estimate of its concentration. With regard to the quantification of fatty acids in fodders, Foster et al.  obtained high coefficients of determination for calibration (0.93–0.99) and cross-validation (0.89–0.98). The SEC and SECV were 20% lower compared to the mean. The RPDCV was greater than 3 for all fatty acids except C12:0 (2.6) and C14:0 (2.9). The reliability of the prediction was lower, but acceptable for C12:0, C14:0, C18:0, C16:1, and C18:1. In this study, two limits were used for the validation of the prediction equations: GH (global spectral distance) and NH (neighborhood spectral distance) to determine if significant bias occurs and if there is a significant increase in unexplained error.
Finally, undesirable substances of a toxic nature can be found in animal forages. Fox et al.  established calibration equations for the estimation of hydrogen cyanide in forage sorghum. The equations developed by MLR gave a coefficient of determination (
4.6. Organic residues and compost
The addition of value to wastes that are organic in nature is required to help reduce the increasing pollution, to optimize the use of available resources, and to offset the increasing energetic and economic costs of synthetic fertilizers. For this it is essential to know in detail the nature and type of such wastes, which can be used both fresh and stabilized. In this section we focus on fresh organic waste and stabilized materials—compost—resulting from the composting process. The origin of these organic wastes can be varied, but generally they originate from urban solid waste, sewage sludge generated in waste water treatment plants, and the agro-industrial sector.
The agri-food industry is one of the most important sectors in Europe and therefore the wastes it generates pose a serious environmental problem. Most of these organic wastes are considered biodegradable and derive from plants and animals. They include fruit and vegetable remains and crop pruning waste. Since these materials are organic or have a high organic matter content, usually with significant amounts of macronutrients as well, it is convenient to use them as organo-mineral fertilizers. This represents a double energy saving: first, waste is eliminated and, second, the need for synthetic fertilizers, whose cost has increased in recent years, is diminished .
In the treatment of organic wastes their possible uses must be taken into account, as the treatment determines the characteristics of the final product obtained. The fate of these organic wastes has been and is still very varied, depending on geographical location, activities taking place in a region, the population, facilities for reuse, and current regulations governing their handling and use. They can be used as soil conditioners, allowing long-term improvement of the physical properties of soils, reducing erosion, and helping the recovery of unproductive marginal areas. Another possibility is their use as substrates for the production of ornamental and horticultural plants. This requires improvement of the physical characteristics of the sludge or residue in question, which is achieved by composting .
Fresh organic wastes can be recycled by composting, a controlled bio-oxidative process involving numerous and varied microorganisms and requiring adequate moisture and heterogeneous organic substrates in the solid state. It involves a thermophilic stage and a temporary production of phytotoxins, giving—as the end-products of the degradation processes—carbon dioxide, water, minerals, and stabilized organic matter, free of phytotoxins and ready for use in agriculture without the risk of adverse phenomena. Finally, the compost, that can be defined as the product resulting from the composting process and maturation and that consists of stabilized organic matter like humus, is obtained. It has little resemblance to the original organic material as it will have been degraded, resulting in finer and dark particles. It is a product that is safe and free of phytotoxic substances, whose application to the soil will not cause damage to plants and which can be stored without further treatment or alterations [90, 91].
Thus, NIRS is used to predict different parameters and/or mineral elements in different organic residues and compost. The NIRS calibration results used successfully by different authors working with different types of organic matrices (industrial compost, compost of various animal manures, compost based on sludge and vegetable waste, compost based on winery and agro-industrial waste, compost derived from tofu waste and sewage sludge) show the great interest and the extent of use of this technique in the study of different variables in this type of organic material. In this regard there are several studies of compost or organic waste which highlight that the information generated with NIRS can increase the effectiveness of composting as a management method, due to the advantages that this technique presents as we have already discussed throughout this chapter. Thus, NIRs has been used to determine the next parameters.
4.6.1. Contents of mineral nutrients in compost
Mineral analysis of the materials at the start, during, and at the end in the composting process is desirable to ensure that the input materials are within acceptable ranges. To determine the mineral nutrient content it is necessary to make a mineral analysis of the compost material by ICP-OES and AA. In addition, nitrogen is one of the most important nutrients in the compost. When we analyze its total content, we refer to the sum of inorganic forms (ammonium, nitrate, and nitrite; NH4+, NO3-, NO2-, respectively) and organic (amino acids, proteins, nucleic acids, and other organic compounds having nitrogen in their structure). The usual methods for its analysis have been the Kjeldahl method (wet digestion) and the Dumas method (dry digestion). Although these conventional methods have been optimized, the digestion of the sample is still carried out with sulfuric acid and a series of catalysts which causes problems such as the emission of acid gases into the environment. Currently, near-infrared reflectance spectrometry technique (NIR) is available today and can be used to determine mineral nutrient content in organic residues and compost [91, 92]. Malley et al.  used a field-portable Corona 45 VIS NIR (visible/near-infrared) spectrometer (Carl Zeiss, Germany) from 360 to 1690 nm to measure total N, ammonium-N, organic N, P, K, Ca, Mg, S, Mn, Zn, and Cu in manure coming from beef cattle manure, stockpiled manure, and compost. The calibrations were developed for each constituent separately by using PCA/PLS1 in The Unscrambler. The calibrations were successfully developed for all parameter measurements (except available P, nitrate + nitrite, or Na). Therefore, field-portable NIRS offers a considerable advance over existing field and laboratory methods by providing rapid, comprehensive compositional analysis when and where the information is required to assist with management of the nutrients of cattle manure.
Usually, heavy metals are also measured in the composts as they can cause toxicities in the plants (Cu, Hg, Cd, Ni). In several publications, it has been observed that NIR technique can be successful to measure these metals in compost coming from raw material rich in these metals [88, 93]. An interesting study was carried out by Shen et al. , who investigated the use of NIR to detect copper (Cu) in animal manure. A total of 118 pig manure samples were collected from four provinces in China, and spectra were acquired in the range of 10,000–4000 cm-1. Results showed that the prediction of Cu concentration in pig manure was feasible (
4.6.2. Organic matter and total organic carbon
Compost maturity has often been associated with the degree of compost humification. Compost stability refers to the degree to which composts have been decomposed to more stable organic materials. Various global parameters have been currently used to assess both maturation process and quality of the final product, including physicochemical properties, such as C:N ratio, humified organic and water-soluble carbon, and cation exchange capacity. Methods for measuring total C in soils, such as wet combustion or dry combustion, are generally very accurate, but too slow or costly for everyday analysis. It has been measured by NIR the contents of carbon and nitrogen in sewage sludge and green waste compost , and sewage sludge  with successful results. So, Albrecht et al.  analyzed changes in composts of sewage sludges and green wastes by NIRS technique of six stages of composting: 8, 20, 35, 75, 135, and 180 days. Maturity of compost was assessed through changes in C:N ratio. Results of spectroscopic properties (200 wavelengths) were studied with several multivariate analyses showing a precise calibration models between spectral data, the C, N, C:N values, and composting time were build using partial least square regression (
Humic acids from sewage sludge. Humic acids are part of the stable organic matter fraction in soils and composts. Due to their favorable properties for soils and plants, and their role in carbon sequestration, they are considered a quality criterion of composts. The traditional methodology for determining the content of humic acids is based on the solubility of the humic substances in aqueous media of different pH, i.e., humins are insoluble in any pH range, humic acids are insoluble in acid medium, and fulvic acids are soluble throughout the pH range. In the determination of humic acids by NIR in compost, a correlation coefficient of 0.94 and a standard error of estimation of 0.28 were obtained, values that can be considered very acceptable . Other publications in mushroom compost , manure , and fattening pig manure  have given excellent results to characterize the humic acids in this material.
In summary, numerous studies of compost or organic wastes using NIRS have demonstrated the efficacy of this methodology. For all the above reasons, this spectroscopic tool is an emerging technique in the analysis of environmental parameters. It offers several advantages over traditional analytical techniques, such as rapidity, ease of preparation and handling of samples (no reagents are required), and low cost.
Artificial Neural Networks
Acid detergent fiber
Acid detergent insoluble nitrogen
Acid detergent insoluble protein
Cation exchange capacity
Coefficient of determination
Coefficient of determination of the calibration/prediction/cross-validation
Digestible dry matter
Digestible neutral detergent fiber
Digestible organic matter
Global spectral distance
Indigestible neutral detergent fiber
In vitro digestibility
Index of Absorbance Difference
Integrated System of Diagnosis and Recommendation
Linear offset subtraction
Mahalanobis distance values
Modified partial least-squares regression
Multiple linear regression
Multiplicative scatter correction
Multivariate adaptive regression splines
Near infrared reflectance spectroscopy
Neighborhood spectral distance
Neutral detergent fiber
Neutral detergent insoluble protein
Neutral detergent-soluble carbohydrates
Neutral detergent-soluble fiber
No data preprocessing
Normalized soil moisture index
Partial least squares regression
Principal components regression
Relative absorbance values
Residual prediction deviation
Residual prediction deviation of the calibration/prediction/cross-validation
Root mean square error of cross validation
Root mean square error of estimation
Root mean square error of prediction
Soluble solids content
Soil organic matter
Standard error of calibration
Standard error of cross-validation
Standard error of prediction
Standard normal variate, detrending
Squared sum estimation
Subtraction of a straight line