Hyperspectral imaging is a technique of analysis that associates to each pixel of the image the spectral content of the radiation coming from the scene. This content can be helpful to recognize the chemical nature of the materials within the scene or to calculate their colours under particular conditions. Different solutions of hyperspectral imager have been realized with different spatial resolution, spectral resolution and range in the electromagnetic spectrum. In particular, improving the spectral resolution allows discriminating smaller features in the spectrum and the unambiguous detection of the absorption bands characteristic of superficial materials. Hyperspectral imagers based on interferometers have the advantage of having a spectral resolution that can be varied according to the needs by changing the optical path delay of the interferometer. A spectrum for each pixel is obtained with an algorithm based on the Fourier transform of the calibrated interferogram. We present the results of the application of a hyperspectral imager based on Fabry‐Perot interferometers to the field of cultural heritage. On different artworks, the hyperspectral imager has been used for pigment recognition, for colour rendering elaborations of the image with different light sources or standard illuminants and for calculating the chromatic coordinates useful for specific purposes.
- hyperspectral imaging (HSI)
- fibre optics reflectance spectroscopy (FORS)
- cultural heritage
- colour rendering
- Fourier transform
Hyperspectral imaging (HSI) is a powerful technique of analysis where each pixel of the image is associated with the spectral content of the radiation coming from the scene in the spectral band of interest. Different solutions with different spectral resolutions have been adopted to separate the spectral content of the radiation impinging on the pixel: starting from the three bands of the Bayer filter camera , to the tens of bands with fixed bandpass filters like in the OSIRIS camera on Rosetta spacecraft  or hundreds or even thousands of bands of imagers based on dispersive means with gratings or prism like on the VIRTIS camera on Rosetta spacecraft . In this work, we are interested in hyperspectral imagers based on interferometers: an interferometer is placed in the optical system in front of the camera, and while the optical path delay (OPD) of the interferometer is varied, the interferogram for each pixel is acquired by the camera and the spectrum is calculated with an algorithm based on the Fourier transform. The final attainable resolution in principle is only limited by the maximal optical path delay of the interferometer. HSI based on Michelson interferometers have been implemented with success in commercial instruments by Bruker  and Telops  ensuring more than 500 bands in the infrared region and reaching a resolution of less than 1 cm−1. At INRIM, we have realized a different concept of HSI based on Fabry‐Perot interferometer (FPI) and we have validated it in different regions of the spectrum: in the UV , in the visible  and in the near infrared  and in different applications .
In paragraph 2 of this chapter, we will describe the principle of the reflectance spectra calculation based on the Fourier transform. In paragraph 3, we will show the application of this technique to the field of cultural heritage in collaboration with Centre for Conservation and Restoration
2. Fabry‐Perot hyperspectral imager
Our hyperspectral imager (HSI) is based on a Fabry‐Perot interferometer (FPI), the optical system is represented in Figure 1 where the scene is firstly imaged in the FPI so that the transmitted intensity is modulated by the interference, and the second image is then formed on the camera sensor by means of the relay lens.
A sequence of frames carrying the interference fringe information is acquired synchronized with the scan of the optical path delay (OPD) between the mirrors of the FPI, from contact to the maximal distance of the mirror. For each pixel of the image, the interferogram is extracted from the acquired video: as an example, we present the interferogram of a light‐emitting diode (LED) at about 635 nm in Figure 2 where it is evident the non‐linearity of the actuators used to vary the OPD near the contact of the mirrors. The
The first points of the interferograms are missing due to the penetration depth of mirror coatings, and according to Fourier transform theory, they correspond to the cosine contributions having the longest period in the spectra calculation. By inserting a bandpass filter in the optical setup and using the information that the spectrum has to be zero in certain regions of the electromagnetic spectrum, it is possible to find the amplitude of the missing points of the interferogram and reconstitute the original spectrum by applying the discrete Fourier transform (DFT) . In Figure 4 , we present the spectrum obtained from the DFT applied to the calibrated interferogram in Figure 3 using the Hanning apodization function. The DFT spectrum is expressed in the frequency domain, and the spectrum of interest is in the band 400–720 nm (416–750 THz) according to the bandpass filter. The peak of the LED is at about 472 THz, corresponding to 635 nm, and since the base of the FPI is the Airy function, and not the cosine as in the Michelson interferometer, DFT creates the harmonics of the peak at 472 THz that decease as
Once a spectrum for each pixel is calculated, all the spectral information are stored in a hyperspectral cube, a three dimensional array with the spatial information of the scene on the
3. Applications in cultural heritage
In the field of cultural heritage, the study of the artwork intended as physical object is very important for knowing constitutive materials, state of preservation and degradation phenomena. It is surely helpful for taking measurements suitable for the artwork conservation during the exposition, that is, in museums or art galleries, but also during possible conservation treatment. Moreover, it may provide precious information about the object's history and provenience, sometimes affecting its economic value.
This study normally implies the development of a diagnostic campaign through different analytical techniques, from the so‐called
Since the artwork is a unique piece, it has to be fully preserved and therefore the use of noninvasive diagnostic techniques is always preferred: on the contrary, the sample taken for analysis has to be limited because it is inevitably an irreversible operation. Luckily, from the interaction between artwork and radiation, many data about materials can be collected in a completely noninvasive manner.
Going beyond the traditional multispectral analyses, where few radiation bands are used for investigating materials, the HSI techniques represent the innovation so that they are making inroads as diagnostic tools [12, 13]: the possibility of combining the painting’s image with the spectral information of each pixel has evident advantages in comparison with punctual analyses. The punctual analysis corresponding to the HSI is the fibre optic reflectance spectroscopy (FORS) [14–18]: for this reason, in the next case studies and applications, we will report the comparison between the spectra extracted from the hyperspectral cube to the ones coming from the spectrophotometer. In both kinds of spectra, it is possible to reveal characteristic peaks, shoulders or absorption bands ; the performance of the two instruments in material identification deeply varies depending on the wavelength range used [20, 21].
In the examples below, concerning artworks studied at CCR, FORS analyses were carried out in the 350–1000 nm band with 0.5 nm spectral resolution by means of an Ocean Optics HR2000+ES spectrophotometer, bounded by optical fibres of 400 µm in diameter to an Ocean Optics HL2000 halogen lamp. Spectralon© 99% was used as white reference. Measures were acquired with a probe in 45°x/0° geometry, so following the standard illuminating/viewing geometry defined by CIE (
3.1. Study of the spatial resolution of HSI
As for other imaging techniques, the application of the HSI requires to choose the right distance between the camera and the object, an important parameter that inevitably determines the quality of the result.
In order to evaluate the spatial resolution performance of the HSI device here discussed , a small Flemish painting on copper (34 cm wide and 49 cm high, Figure 5a: “Passaggio del Mar Rosso”, inv. 299,
Interesting points/pixel areas of the painting were thus selected from the HSI cube, even in correspondence of very small paintbrushes, for extracting the interferogram and calculating the reflectance spectra; FORS analyses on the same points were carried out and used as comparison means for validating the results.
Placing the painting 120 cm distant from the HSI camera and illuminating the scene at 45° angle on the right by means of a halogen lamp, as for the FORS analyses, we acquired videos framing about 10 cm x 10 cm painting areas. The image size is about 1000 pixel x 1000 pixel due to the aperture of the FPI.
Without changing the camera‐object distance, we chose various scenes of the painting portrayed in different views in order to analyse paintbrushes of different size: for example, the main central scene displaying Moses (Figure 5b) has bigger homogeneous areas of paint. The crowed scene on the left is portrayed in the background and so it presents very small brushes (Figure 6a); the lateral scene on the right, portrayed in the foreground, has medium or small size details (Figure 6b). Reflectance spectra were extracted from the hyperspectral cube, attempting to select as most as possible homogeneous coloured pixel areas or paintbrushes. The examples in Table 1, reported as blue, green, yellow, orange, red and purple, called p1, p2, etc., are from 25 to 279 pixel areas. The smallest pixel area corresponds to a paintbrush that is about 0.5 mm x 0.5 mm. These videos took 180 s each, spatial resolution is about 100 µm corresponding to about 250 ppi, spectral resolution is 10 THz.
|Painting detail||Selected areas||Image coordinates||Area size||Area size|
|[pixel]||[mm x mm]|
|Main central scene ( Figure 5b )||Red (p1)||325–335||775−790||176||1.1 x 1.6|
|Blue (p2)||122–130||990−1020||279||0.9 x 3.1|
|Yellow (p10)||574–580||999−1006||56||0.7 x 0.8|
|Lateral scene on the left ( Figure 6a )||Red (p32)||1131–1139||715–719||45||0.9 x 0.5|
|Purple (p34)||1245–1249||835–870||180||0.5 x 3.6|
|Green (p35)||1065–1080||588–591||64||1.6 x 0.4|
|Lateral scene on the right ( Figure 6b )||Orange (p8)||317–321||288–297||50||0.5 x 1|
|Red (p21)||294–301||406–415||80||0.8 x 1|
|Green (p24)||156–160||314–318||25||0.5 x 0.5|
|Graysh blue (p26)||258–264||94–103||70||0.7 x 1|
FORS analyses were performed on the same selected areas (p1, p2, etc.) to verify the reliability of the results of the Fabry‐Perot hyperspectral device when used as diagnostic tool. Figures 7–9 report some examples: the correspondence among calculated HSI and FORS spectra is evident. In red areas of paint (p1 and p32), it is possible to recognize the flex at around 585 nm ascribable to the red pigment
Actually, the spectrum of the green area p24 comes from the smallest pixel area, a paintbrush that on the painting is 0.5 mm × 0.5 mm, and so it defines the spatial resolution performance of the HSI system: it was possible to use this HSI instrument as diagnostic tool for pigment identification selecting areas of just 25 pixels.
This feature will be helpful in the diagnostic investigation on Flemish, Divisionist or Pointillist paintings that present very small paintbrushes or glazing, but also on three‐dimensional surfaces that may be difficult to reach with punctual contact instruments.
3.2. Study of the pigment recognition
Here we report two important case studies where the use of HSI for the comprehension about materials was fundamental. The first one refers to the noninvasive study of the “Croce astile” (
Concerning the wooden cross, it presents polychrome areas and gold leaf gilding. Figures of the saints are portrayed within rounded medals, one per each extreme of the cross on the front and on the back; mouldings of the wooden support are along the perimeter and all around the medals ( Figure 10 ).
Because of the discontinuities of the support, the pigment identification by FORS, in particular on some lateral details nearby the moulding, would have been complicated since this technique normally requires putting the probe (usually of few centimetres in diameter) in contact with the artwork surface. Therefore, the hyperspectral imaging investigation was decisive.
Reflectance spectra extracted and calculated from the HSI cubes showed to be very helpful for pigment identification. The comparison and the integration with other complementary analyses, as required by an efficient diagnostic investigation, allowed the identification of the pictorial palette used for the cross. Figure 11 reports the example showing the use of azurite in the blue backgrounds (blue curves), red lake in the pink stoles (red curves), Sienna earth in the brown hair (orange curves), green copper‐based pigments in the green gowns (green curves), and ochre‐based mixtures in the representation of the skin (grey curve).
Concerning the Egyptian coffins, the hyperspectral investigation allowed the rapid identification of the palette and the total pigment mapping over the surface thanks to the implementation of an algorithm built for recognizing similar reflectance spectra within the hyperspectral cube.
As known, paints of the same hue can have different saturation levels due to various factors, such as different amount of white pigment in the mixture or differences in the state of preservation. Therefore, the resulting reflectance spectra may change even if the main “coloured” pigment is the same. In fact, the possibility of identifying the pigment lies in the fact that the trend of the reflectance curve is recognizable, since it maintains the position of the absorption bands fixed in the spectrum, while the percent reflectance values over the spectrum can vary more or less uniformly. Figure 12 shows the example of two red ochre‐based mixtures containing 50% and 90% in weight of white pigment. Converted in colorimetric coordinates expressed in the CIELAB 1976 colour space , L* (lightness) increases when the paint is clearer, whereas a* and b* (hue) components maintain approximately constant values. From 50% to 90% of white, L* passes from 33.6 to 38.9, a* from 34.4 to 34.1 and b* from 32.7 to 31.3 (measures obtained by a Konica Minolta CM700d colorimeter in D65, SCE and d/8° condition).
In imaging acquisition as in HSI, moreover, the intensity of the light source may be not so uniform over the artwork, so that the reflectance spectra have a vertical scaling. Aiming at obtaining the pigment clustering and recognition from the spectrum analysis, independently of the light intensity, a metric has to be defined in order to measure the similarity of two spectra, represented as vectors in the vector space formed by the spectral components. A vertical scaling of the spectrum would change the length of the vectors without changing the direction in the vector space. We consider the metric, and the closeness of two vectors, as the angle between the two vectors using the spectral angle mapper (SAM) technique . Figure 13 shows the false colours elaboration coming from hyperspectral cube acquired on the lid of an anthropoid coffin (Museo Egizio, S. 05239): red and yellow ochres/earths, Egyptian blue and copper green, commonly used in Ancient Egypt, have been identified and their distribution over the coffin surface is highlighted by the false colours. The lower part of the figure shows the red, green, blue (RGB) image composed of two HSI cubes and rendered for a standard illuminant (D65).
3.3. Study of the colour rendering
Another important advantage of the hyperspectral imaging is the possibility of using the HSI data for colour rendering operations. This means that the RGB image of the artwork may be calculated starting from the reflectance spectrum of each pixel, as shown previously.
If necessary, by correcting the spatial non‐uniformity (in intensity) of the illumination, it is possible to obtain “hyperspectral images” of the artworks where each spectrum depends only on the pictorial material's characteristics, and it is not affected by the light source (neither the quality, nor the intensity) used during the acquisition.
In this way, it is possible to calculate and render the hyperspectral (HS) image for the lamp you prefer, simply by choosing the emission spectrum for the illumination you want to simulate (i.e. CIE standard illuminants, or commercial light sources, depending on the purpose).
In the conservation field, the possibility of studying how the artwork—or more generally, a pictorial material—would appear under different illumination conditions represents a real advantage not only for the aspects linked to the exhibition lightening but also for the choice of pigments for conservation. In fact, the pictorial retouching and the inpainting made in laboratory shall appear different outside it, losing their harmony with the original painting.
The case study of the painting “Gesù tra i dottori” attributed to Giovanni Battista Beinaschi (1636–1688), belonging to the
It is not so easy to find pigments “suitable” for the inpainting treatment. Light plays an important role, obviously, also on the in‐painting's colour appearance, and often we know very few about the lighting system of the museum.
Based on some selected pigments and light sources, we exploited the HSI data for giving an evaluation of the colour appearance variability. Just to make an example, Figure 15 shows the HSI images of two mockups, the one on the left made of copper blue (Kremer Pigmente GmbH & Co. KG, n° 45364), the one on the right made of Zirconium Cerulean Blue (n° 45400). Images are rendered with one halogen lamp (3000 K) and with some different, from “warm” to “neutral” white, LED sources (LED 3000K, typical light source in art galleries or museums simulating halogen lamp; LED 2700 K and LED 4000 K).
Beyond the visual comparison that would be surely affected by many factors (first of all by the computer screen), the HSI images allows to extract the colorimetric values of the mockups for each lamp and to find which one has the minimum chromatic variability. In this case, the zirconium cerulean blue presents a slightly lower variability with respect to the copper blue and relatively to the four lamps tested ( Table 2 ).
|MOCKUPS||Copper blue in PVAc||Copper blue in oil||Zirconium blue in PVAc||Zirconium blue in oil|
|halogen 3000 K||52.1||−23.3||−29.0||44.6||−23.7||−26.1||67.3||−20.6||−6.3||56.7||−23.5||−5.4|
|LED 3000 K||53.2||−22.0||−27.8||45.6||−22.6||−24.8||67.7||−20.2||−5.3||57.1||−23.3||−4.4|
|LED 2700 K||51.3||−21.8||−30.7||43.8||−22.4||−27.6||66.5||−20.8||−7.5||55.8||−23.8||−6.8|
|LED 4000 K||53.5||−19.2||−27.2||45.9||−20.1||−24.1||67.9||−19.0||−4.9||57.3||−22.0||−3.9|
Obviously, only the precise knowledge of the museum lighting system, that will host the artwork, would be the key for the right consideration about materials, to prefer in the conservation treatment. We think this example may show the system potentiality and its precious contribution in the choice of pigments and binders for restoration and in promoting interventions that might consider not only the chemical properties of the materials but also the colour appearance.
4. Future work
The opportunity of collecting, in very little time and in noninvasive and contactless modality, the spectral information of the entire artwork is undoubtedly an advantage of the HSI technique with respect to punctual analyses. When the aim is the material identification, the wavelength range used for the investigation surely affects the system performance. In this case, we worked in two bands, covering the range from 420 to 1000 nm, but the use of wider ranges can surely improve the performance of the HSI for identifying also organic materials, such as consolidating materials , natural polymers and resins , or to map paint binders in situ .
Moreover, HSI offers the possibility of calculating the colours of the artwork in a more controlled way: some attempts have been made for applying this technique to the digital documentation of the artworks through the production of high‐quality images , maybe comparable to professional photographs.
Probably the future, natural development of the hyperspectral imaging may be the integration of the HSI in 3D acquisition systems that will solve the problem of not flat surfaces.
Hyperspectral imaging (HSI) based on a Fabry‐Perot interferometer showed to be a powerful technique of analysis in the field of cultural heritage, providing at the same time the artwork's image and the spectral behaviour of each pixel. This has evident advantage with respect to punctual analysis as fibre optics reflectance spectroscopy (FORS) in terms of time spending for pigment recognition. Nevertheless, the system performance in the identification of materials strictly depends on the wavelength range used for the analysis: aware of that, the most recent research is intended to enlarge the investigation range in order to study in imaging modality specific features in particular of organic materials. Further HSI applications were tested on the artworks at CCR for elaborating the images rendered with different light sources or illuminants, in order to enhance particular aspects of the materials, such as for highlighting the areas with pictorial retouching. The hyperspectral data were shown to be useful also for calculating colorimetric coordinates for different light sources permitting a better evaluation of the pictorial materials to use in the conservation treatment.
The authors like to thank the
Dillon P L, Lewis D W, Kaspar F G: Color imaging system using a single CCD area array. IEEE Transactions on Electron Devices. 1978; 25:102–107.
Keller HU et al.: OSIRIS–The scientific camera system onboard Rosetta. Space Science Reviews. 2007; 128:433–506.
Coradini A et al: VIRTIS: An imaging spectrometer for the Rosetta mission. Space Science Reviews. 2007; 128:529–559.
Braun R, Harig R. Identification and mapping of spilled liquids by passive hyperspectral imaging. Proc. SPIE 8546, Optics and Photonics for Counterterrorism, Crime Fighting, and Defence VIII, 85460F (October 30, 2012); doi:10.1117/12.974496.
Kastek M et al. Method of gas detection applied to an infrared hyperspectral sensor. Photonics Letters of Poland. 2012; 4(4):146–148.
Zucco M, Caricato V, Egidi A, Pisani M. A hyperspectral camera in the UVA band. IEEE Transactions on Instrumentation and Measurement. 2015; 64(6):1425–1430.
Pisani M, Zucco M. Compact imaging spectrometer combining Fourier transform spectroscopy with a Fabry‐Perot interferometer. Optics Express. 2009; 17(10):8319–8331.
Pisani M, Bianco P, Zucco M. Hyperspectral imaging for thermal analysis and remote gas sensing in the short wave infrared. Applied Physics B. 2012; 108(1):231–236.
INRIM. http://www.inrim.it/res/hyperspectral_imaging, accessed date 28/11/2016.
Pisani M, Zucco M: Fourier transform based hyperspectral imaging,in Fourier Transforms - Approach to Scientific Principles, Prof. Goran Nikolic (Ed.), InTech, 2011, DOI: 10.5772/15464.
Zucco M, Pisani M, Caricato V, Egidi A. A hyperspectral imager based on a Fabry‐Perot interferometer with dielectric mirrors. Optics Express. 2014; 22(2):1824–1834.
Liang H. Advances in multispectral and hyperspectral imaging for archeology and art conservation. Applied Physics A. 2012; 106(2):309–323.
Legrand S et al. Examination of historical paintings by state‐of‐the‐art hyperspectral imaging methods: from scanning infra‐red spectroscopy to computed X‐ray laminography. Heritage Science. 2014; 2(1):2–13.
Bacci M et al. Non‐invasive spectroscopic measurements on the Il ritratto della figliastra by Giovanni Fattori: identification of pigments and colourimetric analysis. Journal of Cultural Heritage. 2003; 4:329–33.
Casini A et al. Fiber optic reflectance spectroscopy and hyper‐spectral image spectroscopy: two integrated techniques for the study of the Madonna dei Fusi. Proc. SPIE 5857, Optical Methods for Arts and Archaeology, ed Salimbeni and Pezzati, Munich, Germany, 58570M (August 12, 2005); doi:10.1117/12.611500.
Poldi G. Ricostruire la tavolozza di Pellizza da Volpedo mediante spettrometria in riflettanza. In: l colore dei Divisionisti. Tecnica e teoria, analisi e prospettive di ricerca; ed A. Scotti, Tortona (It), 30 Sept. 2005, 2007. p. 113–128.
Montagner C et al. Library of UV‐Vis‐NIR reflectance spectra of modern organic dyes from historic pattern‐card coloured papers. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy. 2011; 79(5):1669–1680.
Depuis G, Menu M. Quantitative characterization of pigment mixtures used in art by fiber‐optics diffuse‐reflectance spectroscopy. Applied physics A. 2006; 83:469–474.
Johnston‐Feller R. Color science in the examination of museum objects: nondestructive procedures. The Getty Consetrvation Institute, Los Angeles, Publications ed. 2001.
Berns R, Imai F. The use of multi‐channel visible spectrum imaging for pigment identification. In: ICOM Committee for Conservation, editor. 13th Triennial Meeting Rio de Janeiro, Vol 1; 2002. p. 217–222.
Kubik M. Hyperspectral Imaging: a new technique for the non‐invasive study of artworks. Physical Techniques in the Study of Art, Archeology and Cultural Heritage. 2007; 2:199–259.
Oleari C. Misurare il colore, pp 53–60, Hoepli, Florence Italy. 2008.
Caricato V, Egidi A, Pisani M, Zucco M. A device for efficient hyperspectral imaging for color analysis. In: Rossi, Maggioli, editors. Colour and Colorimetry: Multidisciplinary Contribution, 10th Conferenza del Colore, Vol. X B; 2014. p. 13–18.
Bisulca C et al. UV‐VIS‐NIR reflectance spectroscopy of red lakes in paintings. In: 9th international conference on NTD of Art, Jerusalem Israel. 2008.
Gulmini M et al. Identification of dyestuffs in historical textiles: strong and weak points of a non‐invasive approach. Dyes and Pigments. 2013; 98(1):136–145.
Gil M et al. Blue pigments colors from wall paintings churches in danger (Portugal 15th to 18th century): identification, diagnosis and color evaluation. Applied Spectroscopy. 2011; 65(7):782–789.
Dik J. Scientific analysis of historical paint and the implications for art history and art conservation: the case studies of naples yellow and discoloured smalt, Thesis of Van ‘t Hoff Institute for Molecular Sciences, Amsterdam Holland; 2003.
Kruse FA et al. The spectral image processing system (SIPS) ‐ interactive visualization and analysis of imaging spectrometer data. Remote Sensing of Environment. 1993; 44(1):145–163.
Bonifazi et al. Study of consolidant materials applied on wood by hyperspectral Imaging. In: International Society for Optics and Photonics, editors. SPIE Commercial+ Scientific Sensing and Imaging; 2016. pp. 98620I–98620I.
Vagnini M et al. FT‐NIR spectroscopy for non‐invasive identification of natural polymers and resins in easel paintings. Analytical and bioanalytical chemistry. 2009; 395(7):2107–2118.
Ricciardi P et al. Near infrared reflectance imaging spectroscopy to map paint binders in situ on illuminated manuscripts. Angewandte Chemie International Edition. 2012; 51(23):5607–5610.
Cucci C et al. A hyper‐spectral scanner for high quality image spectroscopy: Digital documentationand spectroscopic characterization of polychrome surfaces. In: ART11‐10th International Conference on Non‐destructive Investigations and Microanalysis for the Diagnostics and Conservation of Cultural and Environmental Heritage; 2011.