Fourier Transform Hyperspectral Imaging for Cultural Heritage Fourier Transform Hyperspectral Imaging for Cultural Heritage

Hyperspectral imaging is a technique of analysis that associates to each pixel of the image the spectral content of the radiation coming from the scene. This content can be helpful to recognize the chemical nature of the materials within the scene or to calculate their colours under particular conditions. Different solutions of hyperspectral imager have been realized with different spatial resolution, spectral resolution and range in the elec ‐ tromagnetic spectrum. In particular, improving the spectral resolution allows discrimi ‐ nating smaller features in the spectrum and the unambiguous detection of the absorption bands characteristic of superficial materials. Hyperspectral imagers based on interfer ‐ ometers have the advantage of having a spectral resolution that can be varied according to the needs by changing the optical path delay of the interferometer. A spectrum for each pixel is obtained with an algorithm based on the Fourier transform of the calibrated interferogram. We present the results of the application of a hyperspectral imager based on Fabry‐Perot interferometers to the field of cultural heritage. On different artworks, the hyperspectral imager has been used for pigment recognition, for colour rendering elaborations of the image with different light sources or standard illuminants and for calculating the chromatic coordinates useful for specific purposes.


Introduction
Hyperspectral imaging (HSI) is a powerful technique of analysis where each pixel of the image is associated with the spectral content of the radiation coming from the scene in the spectral band of interest. Different solutions with different spectral resolutions have been In paragraph 2 of this chapter, we will describe the principle of the reflectance spectra calculation based on the Fourier transform. In paragraph 3, we will show the application of this technique to the field of cultural heritage in collaboration with Centre for Conservation and Restoration La Venaria Reale (CCR). Reflectance spectra indeed contain information useful for identifying pigments and dyes and thus for discriminating original and possibly superimposed materials (e.g. pictorial retouching) that is one of the main aims of a diagnostic campaign intended at preserving artworks and guiding the conservation treatment. Moreover, reflectance spectra can be used for rendering the artworks' colour appearance under different lights in order to choose the light source most suitable for enhancing some aesthetical aspects of the objects, for increasing the visitors' satisfaction, in the meantime taking into account the preventive conservation principles and the standard recommendations for lighting in museums. On the other hand, the possibility of studying the pigments' colour appearance can be of some help when choosing materials for the conservation treatment. Finally, spectra can be converted in colorimetric values for different light sources or standard illuminants useful for calculating chromatic differences for specific purposes. Results here presented concern some real artworks of different art periods (e.g. coffins from Ancient Egypt, Italian Renaissance polychrome artworks) and some mockups used as references made with known pigments and binders.

Fabry-Perot hyperspectral imager
Our hyperspectral imager (HSI) is based on a Fabry-Perot interferometer (FPI), the optical system is represented in Figure 1 where the scene is firstly imaged in the FPI so that the transmitted intensity is modulated by the interference, and the second image is then formed on the camera sensor by means of the relay lens.
A sequence of frames carrying the interference fringe information is acquired synchronized with the scan of the optical path delay (OPD) between the mirrors of the FPI, from contact to the maximal distance of the mirror. For each pixel of the image, the interferogram is extracted from the acquired video: as an example, we present the interferogram of a light-emitting diode (LED) at about 635 nm in Figure 2 where it is evident the non-linearity of the actuators used to vary the OPD near the contact of the mirrors. The x axis of the interferogram is calibrated by using a laser in the optical setup. More details of the calibration technique are described in the previous work [10]. In Figure 3, we present the calibrated, resampled and rescaled interferogram.  The first points of the interferograms are missing due to the penetration depth of mirror coatings, and according to Fourier transform theory, they correspond to the cosine contributions having the longest period in the spectra calculation. By inserting a bandpass filter in the optical setup and using the information that the spectrum has to be zero in certain regions of the electromagnetic spectrum, it is possible to find the amplitude of the missing points of the interferogram and reconstitute the original spectrum by applying the discrete Fourier transform (DFT) [10]. In Figure 4, we present the spectrum obtained from the DFT applied to the calibrated interferogram in Figure 3 using the Hanning apodization function. The DFT spectrum is expressed in the frequency domain, and the spectrum of interest is in the band 400-720 nm (416-750 THz) according to the bandpass filter. The peak of the LED is at about 472 THz, corresponding to 635 nm, and since the base of the FPI is the Airy function, and not the cosine as in the Michelson interferometer, DFT creates the harmonics of the peak at 472 THz that decease as R n /n, where R is the reflectance of the mirrors and n is the order of the harmonic [11]. The interferogram in Figure 3 is obtained with a maximal OPD of about 30 µm that corresponds to a spectral resolution of about 10 THz. The frequency interval in the spectrum, as visible in the inset of Figure 4, is decreased below the spectral resolution by using the zero padding method [10]. A phase correction has been applied to the spectrum calculation in order to take into account the phase dispersion of the mirror coatings [11]. In the right side of spectrum are evident the aliases of the LED peak, artefacts of the DFT. The effect of aliases on the original spectrum is decreased by increasing the number of points per fringe and decreasing the reflectivity of the mirrors. Since the aliases of the LED peak have a phase dispersion that is not corrected they have deformed peaks.
Once a spectrum for each pixel is calculated, all the spectral information are stored in a hyperspectral cube, a three dimensional array with the spatial information of the scene on the x and y axis, and the spectral content on the third axis. The hyperspectral cube contains all the spectral information that can be used for the applications of the next sections.

Applications in cultural heritage
In the field of cultural heritage, the study of the artwork intended as physical object is very important for knowing constitutive materials, state of preservation and degradation phenomena. It is surely helpful for taking measurements suitable for the artwork conservation during the exposition, that is, in museums or art galleries, but also during possible conservation treatment. Moreover, it may provide precious information about the object's history and provenience, sometimes affecting its economic value.
This study normally implies the development of a diagnostic campaign through different analytical techniques, from the so-called imaging techniques to the punctual-noninvasive or micro-invasive, destructive or not destructive-chemical analyses. The main aims are to map  and chemically identify in the artwork both the original materials, with connected alteration and degradation phenomena, and the substances possibly superimposed during past interventions, such as pictorial retouching or inpainting.
Since the artwork is a unique piece, it has to be fully preserved and therefore the use of noninvasive diagnostic techniques is always preferred: on the contrary, the sample taken for analysis has to be limited because it is inevitably an irreversible operation. Luckily, from the interaction between artwork and radiation, many data about materials can be collected in a completely noninvasive manner.
Going beyond the traditional multispectral analyses, where few radiation bands are used for investigating materials, the HSI techniques represent the innovation so that they are making inroads as diagnostic tools [12,13]: the possibility of combining the painting's image with the spectral information of each pixel has evident advantages in comparison with punctual analyses. The punctual analysis corresponding to the HSI is the fibre optic reflectance spectroscopy (FORS) [14][15][16][17][18]: for this reason, in the next case studies and applications, we will report the comparison between the spectra extracted from the hyperspectral cube to the ones coming from the spectrophotometer. In both kinds of spectra, it is possible to reveal characteristic peaks, shoulders or absorption bands [19]; the performance of the two instruments in material identification deeply varies depending on the wavelength range used [20,21].
In the examples below, concerning artworks studied at CCR, FORS analyses were carried out in the 350-1000 nm band with 0.5 nm spectral resolution by means of an Ocean Optics HR2000+ES spectrophotometer, bounded by optical fibres of 400 µm in diameter to an Ocean Optics HL2000 halogen lamp. Spectralon © 99% was used as white reference. Measures were acquired with a probe in 45°x/0° geometry, so following the standard illuminating/viewing geometry defined by CIE (Commission Internationale de l'Eclairage) [22], on areas of fixed dimension (approximately 3 mm in diameter).

Study of the spatial resolution of HSI
As for other imaging techniques, the application of the HSI requires to choose the right distance between the camera and the object, an important parameter that inevitably determines the quality of the result.
In order to evaluate the spatial resolution performance of the HSI device here discussed [23], a small Flemish painting on copper (34 cm wide and 49 cm high, Figure 5a: "Passaggio del Mar Rosso", inv. 299, Musei Reali di Torino: Galleria Sabauda), characterized by crowded and analytic composition with brilliant colours and very subtle brushes typical of the Flemish School was chosen for the test.
Interesting points/pixel areas of the painting were thus selected from the HSI cube, even in correspondence of very small paintbrushes, for extracting the interferogram and calculating the reflectance spectra; FORS analyses on the same points were carried out and used as comparison means for validating the results.
Placing the painting 120 cm distant from the HSI camera and illuminating the scene at 45° angle on the right by means of a halogen lamp, as for the FORS analyses, we acquired videos framing about 10 cm x 10 cm painting areas. The image size is about 1000 pixel x 1000 pixel due to the aperture of the FPI.
Without changing the camera-object distance, we chose various scenes of the painting portrayed in different views in order to analyse paintbrushes of different size: for example, the main central scene displaying Moses (Figure 5b) has bigger homogeneous areas of paint. The crowed scene on the left is portrayed in the background and so it presents very small brushes (Figure 6a); the lateral scene on the right, portrayed in the foreground, has medium or small size details (Figure 6b). Reflectance spectra were extracted from the hyperspectral cube, attempting to select as most as possible homogeneous coloured pixel areas or paintbrushes.
The examples in Table 1, reported as blue, green, yellow, orange, red and purple, called p1, p2, etc., are from 25 to 279 pixel areas. The smallest pixel area corresponds to a paintbrush that is about 0.5 mm x 0.5 mm. These videos took 180 s each, spatial resolution is about 100 µm corresponding to about 250 ppi, spectral resolution is 10 THz.
FORS analyses were performed on the same selected areas (p1, p2, etc.) to verify the reliability of the results of the Fabry-Perot hyperspectral device when used as diagnostic tool. Figures 7-9 report some examples: the correspondence among calculated HSI and FORS spectra is evident. In red areas of paint (p1 and p32), it is possible to recognize the flex at around 585 nm ascribable to the red pigment cinnabar (Figure 7). In the green area (p24), spectra suggest the presence of a green copper-based pigment (Figure 8); in purple area (p34), spectra show the double absorption band at around 530 and 560 nm ascribable to the use of red lake (Figure 8), probably coming from cochineal [24,25]. In blue areas (Figure 9), spectra acquired in two  bands report the double absorption band at around 595 and 650 nm (p26) suggesting the presence of the smalt pigment [26,27].
Actually, the spectrum of the green area p24 comes from the smallest pixel area, a paintbrush that on the painting is 0.5 mm × 0.5 mm, and so it defines the spatial resolution performance of the HSI system: it was possible to use this HSI instrument as diagnostic tool for pigment identification selecting areas of just 25 pixels. Lateral scene on the right (Figure 6b) This feature will be helpful in the diagnostic investigation on Flemish, Divisionist or Pointillist paintings that present very small paintbrushes or glazing, but also on three-dimensional surfaces that may be difficult to reach with punctual contact instruments.

Study of the pigment recognition
Here we report two important case studies where the use of HSI for the comprehension about materials was fundamental. The first one refers to the noninvasive study of the "Croce astile"    and on the back; mouldings of the wooden support are along the perimeter and all around the medals (Figure 10).
Because of the discontinuities of the support, the pigment identification by FORS, in particular on some lateral details nearby the moulding, would have been complicated since this technique normally requires putting the probe (usually of few centimetres in diameter) in contact with the artwork surface. Therefore, the hyperspectral imaging investigation was decisive.
Reflectance spectra extracted and calculated from the HSI cubes showed to be very helpful for pigment identification. The comparison and the integration with other complementary analyses, as required by an efficient diagnostic investigation, allowed the identification of the pictorial palette used for the cross. Figure 11 reports the example showing the use of azurite in the blue backgrounds (blue curves), red lake in the pink stoles (red curves), Sienna earth in the brown hair (orange curves), green copper-based pigments in the green gowns (green curves), and ochre-based mixtures in the representation of the skin (grey curve).
Concerning the Egyptian coffins, the hyperspectral investigation allowed the rapid identification of the palette and the total pigment mapping over the surface thanks to the implementation of an algorithm built for recognizing similar reflectance spectra within the hyperspectral cube.
As known, paints of the same hue can have different saturation levels due to various factors, such as different amount of white pigment in the mixture or differences in the state of preservation. Therefore, the resulting reflectance spectra may change even if the main "coloured" pigment is the same. In fact, the possibility of identifying the pigment lies in the fact that the trend of the reflectance curve is recognizable, since it maintains the position of the absorption bands fixed in the spectrum, while the percent reflectance values over the spectrum can vary more or less uniformly. Figure 12 shows the example of two red ochre-based mixtures containing 50% and 90% in weight of white pigment. Converted in colorimetric coordinates expressed in the CIELAB 1976 colour space [22], L* (lightness) increases when the paint is clearer, whereas a* and b* (hue) components maintain approximately constant values. From 50% to 90% of white, L* passes from 33.6 to 38.9, a* from 34.4 to 34.1 and b* from 32.7 to 31.3 (measures obtained by a Konica Minolta CM700d colorimeter in D65, SCE and d/8° condition).
In imaging acquisition as in HSI, moreover, the intensity of the light source may be not so uniform over the artwork, so that the reflectance spectra have a vertical scaling. Aiming at obtaining the pigment clustering and recognition from the spectrum analysis, independently of the light intensity, a metric has to be defined in order to measure the similarity of two spectra, represented as vectors in the vector space formed by the spectral components. A vertical scaling of the spectrum would change the length of the vectors without changing the direction in the vector space. We consider the metric, and the closeness of two vectors, as the angle between the two vectors using the spectral angle mapper (SAM) technique [28]. Figure 13 shows the  false colours elaboration coming from hyperspectral cube acquired on the lid of an anthropoid coffin (Museo Egizio, S. 05239): red and yellow ochres/earths, Egyptian blue and copper green, commonly used in Ancient Egypt, have been identified and their distribution over the coffin surface is highlighted by the false colours. The lower part of the figure shows the red, green, blue (RGB) image composed of two HSI cubes and rendered for a standard illuminant (D65).

Study of the colour rendering
Another important advantage of the hyperspectral imaging is the possibility of using the HSI data for colour rendering operations. This means that the RGB image of the artwork may be calculated starting from the reflectance spectrum of each pixel, as shown previously.
If necessary, by correcting the spatial non-uniformity (in intensity) of the illumination, it is possible to obtain "hyperspectral images" of the artworks where each spectrum depends only Figure 11. Details of the wooden cross "Croce astile" attributed to Raffaello Sanzio (Museo Poldi Pezzoli, Milan). Hyperspectral images calculated for D65 illuminant and 10° standard observer, with reflectance spectrum extracted from the HS cubes.
on the pictorial material's characteristics, and it is not affected by the light source (neither the quality, nor the intensity) used during the acquisition.
In this way, it is possible to calculate and render the hyperspectral (HS) image for the lamp you prefer, simply by choosing the emission spectrum for the illumination you want to simulate (i.e. CIE standard illuminants, or commercial light sources, depending on the purpose).
In the conservation field, the possibility of studying how the artwork-or more generally, a pictorial material-would appear under different illumination conditions represents a real advantage not only for the aspects linked to the exhibition lightening but also for the choice of pigments for conservation. In fact, the pictorial retouching and the inpainting made in laboratory shall appear different outside it, losing their harmony with the original painting.
The case study of the painting "Gesù tra i dottori" attributed to Giovanni Battista Beinaschi (1636-1688), belonging to the Castle of Racconigi (province of Cuneo, Italy), is particularly significant for this aspect. The surface presents many lacunae, at present filled with stucco, that have to be inpainted at CCR. In Figure 14, we present two different details from the painting calculated from the HSI cube, the detail on the left is acquired using a xenon lamp and rendered with a D65 illuminant, with standard CIE 10° observer, on the right a particular acquired using a halogen lamp and rendered with an LED4000K illuminant, with standard CIE 10° observer.
It is not so easy to find pigments "suitable" for the inpainting treatment. Light plays an important role, obviously, also on the in-painting's colour appearance, and often we know very few about the lighting system of the museum.
Based on some selected pigments and light sources, we exploited the HSI data for giving an evaluation of the colour appearance variability. Just to make an example, Figure 15 shows  Images are rendered with one halogen lamp (3000 K) and with some different, from "warm" to "neutral" white, LED sources (LED 3000K, typical light source in art galleries or museums simulating halogen lamp; LED 2700 K and LED 4000 K). Beyond the visual comparison that would be surely affected by many factors (first of all by the computer screen), the HSI images allows to extract the colorimetric values of the mockups for each lamp and to find which one has the minimum chromatic variability. In this case, the zirconium cerulean blue presents a slightly lower variability with respect to the copper blue and relatively to the four lamps tested ( Table 2).    Obviously, only the precise knowledge of the museum lighting system, that will host the artwork, would be the key for the right consideration about materials, to prefer in the conservation treatment. We think this example may show the system potentiality and its precious contribution in the choice of pigments and binders for restoration and in promoting interventions that might consider not only the chemical properties of the materials but also the colour appearance.

Future work
The opportunity of collecting, in very little time and in noninvasive and contactless modality, the spectral information of the entire artwork is undoubtedly an advantage of the HSI technique with respect to punctual analyses. When the aim is the material identification, the wavelength range used for the investigation surely affects the system performance. In this case, we worked in two bands, covering the range from 420 to 1000 nm, but the use of wider ranges can surely improve the performance of the HSI for identifying also organic materials, such as consolidating materials [29], natural polymers and resins [30], or to map paint binders in situ [31].
Moreover, HSI offers the possibility of calculating the colours of the artwork in a more controlled way: some attempts have been made for applying this technique to the digital documentation of the artworks through the production of high-quality images [32], maybe comparable to professional photographs.
Probably the future, natural development of the hyperspectral imaging may be the integration of the HSI in 3D acquisition systems that will solve the problem of not flat surfaces.

Conclusion
Hyperspectral imaging (HSI) based on a Fabry-Perot interferometer showed to be a powerful technique of analysis in the field of cultural heritage, providing at the same time the artwork's image and the spectral behaviour of each pixel. This has evident advantage with respect to punctual analysis as fibre optics reflectance spectroscopy (FORS) in terms of time spending for pigment recognition. Nevertheless, the system performance in the identification of materials strictly depends on the wavelength range used for the analysis: aware of that, the most recent research is intended to enlarge the investigation range in order to study in imaging modality specific features in particular of organic materials. Further HSI applications were tested on the artworks at CCR for elaborating the images rendered with different light sources or illuminants, in order to enhance particular aspects of the materials, such as for highlighting the areas with pictorial retouching. The hyperspectral data were shown to be useful also for calculating colorimetric coordinates for different light sources permitting a better evaluation of the pictorial materials to use in the conservation treatment.