0 Real-Time Heterodyne Interferometry with Correlation Image Sensor

Optical heterodyne interfereometry (HI) (R. Dändliker, 1973; N.A. Massie, 1979) is an interferometric technique that employs two interfering beams of slightly different frequencies. Interference of these beams yields a time-varying sinusoidal intensity signal whose frequency equals the difference between the frequencies of the two beams. The signal and frequency are called beat signal and beat frequency, respectively. One of the most significant properties of HI is that the phase difference between the two beams, which is usually the most important quantity in interferometry, equals the phase of the beat signal. In conventional HI, the phase of the beat signal is measured by a single photodetector and a demodulation circuit or instrument. It is said that accuracy of phase measurement in HI reaches 2π/1000 rad. Because of the nature of point-wise measurement, however, this method requires two-dimensional (2-D) scanning of the measured point to realize 2-D measurement, resulting in a trade-off between temporal and spatial resolution.


Introduction
Optical heterodyne interfereometry (HI) (R. Dändliker, 1973;N.A. Massie, 1979) is an interferometric technique that employs two interfering beams of slightly different frequencies.Interference of these beams yields a time-varying sinusoidal intensity signal whose frequency equals the difference between the frequencies of the two beams.The signal and frequency are called beat signal and beat frequency, respectively.One of the most significant properties of HI is that the phase difference between the two beams, which is usually the most important quantity in interferometry, equals the phase of the beat signal.In conventional HI, the phase of the beat signal is measured by a single photodetector and a demodulation circuit or instrument.It is said that accuracy of phase measurement in HI reaches 2π/1000 rad.Because of the nature of point-wise measurement, however, this method requires two-dimensional (2-D) scanning of the measured point to realize 2-D measurement, resulting in a trade-off between temporal and spatial resolution.
The objective of this article is to introduce a real-time 2-D HI technique.The key idea is the use of the time-domain correlation image sensor (CIS) (S.Ando, 2003) as a 2-D array of phase-demodulating pixels for incident heterodyne beams (A.Kimachi, 2007;2010a).The CIS produces temporal correlations between the intensity signal of incident light and global reference signals over a frame period and outputs them as images at an ordinary frame rate.This property enables the CIS to simultaneously demodulate the amplitude and phase of the wavefront of incident heterodyne beams at each pixel in every frame using sinusoidal reference signals of the beat frequency.Consequently, as the most significant advantage, this real-time 2-D HI can ensure both maximal temporal and spatial resolution of the image sensor and high accuracy of phase measurement.Note that other than real-time 2-D HI, the CIS has opened a very wide range of novel applications such as image feature extraction (A.Kimachi, 1998), three-dimensional measurement (A.Kimachi, 2001;T. Kurihara, 2003;S. Ando, 2008a;A. Kimachi, 2009), optical flow measurement (S.Ando, 2008b), spectral matching imaging (A.Kimachi, 2002;2004;2010b), magneto-optic leakage flux imaging (S.Ando, 2007), and so forth.
The remainder of this article is organized as follows.Section 2 reviews previous studies on real-time 2-D HI and phase-shifting interferometry (PSI).Section 3 describes the principle of the CIS and the real-time 2-D HI.Section 4 shows experimental results of the real-time 2-D HI for a polarizing Michelson interferometer and speckle pattern interferometers for in-plane and out-of-plane deformation measurement.Section 5 briefly describes ongoing research on a networked sensing system for health monitoring of large-scale structures based on multi-zero beams and the real-time 2-D HI.Finally, Section 6 concludes this article.

Review of real-time 2-D heterodyne and phase-shifting interferometry
A great number of studies have been devoted to the problem with HI mentioned in Section 1. Realization of real-time 2-D HI can be divided into two categories: 1) development of real-time PSI methods using image sensors, and 2) integration of phase-demodulating sensors into 2-D arrays.The following subsections briefly review studies belonging to each category.

Real-time PSI using image sensors
PSI, a widely-used interferometry technique realized with a commercial image sensor, is easier to deal with than the development of 2-D arrays of phase-demodulating sensors.PSI captures a sequence of fringe images formed by two interfering beams of the same frequency, while the phase difference between the beams is shifted stepwise with a piezoelectric actuator or a variable retarder in each frame (K.J. Gåsvik, 2002;J.H. Bruning, 1974).The original phase difference is estimated by fitting the intensity of the image sequence to a sine wave individually at each pixel (C.J. Morgan, 1982).PSI is used not only in ordinary interferometry for specularly reflective objects or transparent objects, but also in speckle pattern interferometry (SPI) (K.Creath, 1985;J. Kato, 1993;M. Takeda, 1994), optical coherence tomography (M.Akiba, 2003), or holography (I.Yamaguchi, 1997) for objects with rough surfaces or made of turbid media.PSI requires three images at the minimum, and frequently employs four images with a phase shift increment by 90 • .Higher accuracy is achieved for phase demodulation with a larger number of images; however, it results in a longer time for image capture.In PSI, temporal resolution and measurement accuracy are thus in a trade-off.
It seems that this problem with PSI could be solved by capturing fringe images with different phase shifts at the same time.North-Morris et al. (M. North-Morris, 2005) developed a phase mask consisting of an array of 2 × 2-pixel retarder matrices with π/2 steps of retardance for phase-shifting SPI.Four-step PSI is thus realized from a single-frame image by attaching this mask in front of an image sensor.A similar phase mask was proposed for phase-shifting holography by Awatsuji et al. (Y. Awatsuji, 2004).Kiire et al. (T. Kiire, 2008) proposed a method for simultaneously producing four separate images of speckle interference for different phase shifts on an image sensor using a polarizing prism, a diffraction grating, and two lasers of different wavelengths.The price for these single-frame PSI methods is, of course, loss of spatial resolution down to half of the image sensor.Akiba et al. (M. Akiba, 2003) proposed a system of optical coherence tomography using two pairs of a liquid-crystal shutter and a CCD (charge coupled device) camera.The beat signal of low coherence interference exerted by a translating mirror is sampled in synchrony by the two shutters and integrated over a frame period of the CCDs at each pixel.This system captures a pair of phase-shifted fringe images at a video frame rate, though it may be categorized into 2-D discrete sampling of a continous beat signal as discussed in the next subsection rather than phase shifting.The phase of the beat signal can be obtained from the pair of integrated images and a dc component image captured beforehand, though phase demodulation was not performed in the paper.The accuracy of this method, however, can be degraded because of temporal sampling with a rectangular window, which has a certain degree of sensitivity to higher-order harmonic components in the beat signal.Moreover, the two shutter-CCD pairs make the system rather bulky and demand cumbersome optical alignment.

Integration of 2-D phase-demodulating pixel arrays
Other studies focus on integrating real-time phase-demodulating pixels for HI into 2-D arrays.It is a straightforward idea to employ high-speed cameras as 2-D phase-demodulating arrays for sampling continuous beat signals at discrete time intervals (P.Haible, 2000; M.V. Aguanno, 2007).The higher the frame rate, the higher the accuracy of phase demodulation because of a larger number of samples, while the spatial resolution of the image sensor is maintained.The use of a high-speed camera, however, must be accompanied by a high-power laser source; otherwise it degrades the accuracy of phase demodulation because of shorter exposure time, lower light intensity, and hence lower signal-to-noise ratio (SNR) in the interference images.Moreover, high-speed cameras require high-speed frame grabber interface with large memory, which gives rise to a problem in terms of commercial availability.
Another group of studies employ 2-D arrays of phase-demodulating sensors developed on their own.A type of phase-demodulating image sensor called lock-in CCD, originally proposed by Spirig et al. (T. Spirig, 1995), outputs a set of discrete samples of the incident beat signal from each pixel, instead of a sequence of fringe images.The multiple samples are transferred to separate bins within a pixel as charge packets and accumulated over a frame period.The phase of the beat signal is demodulated from the set of samples at each pixel on a computer by the same algorithm as used in PSI.Dändliker et al. (R.D. Dändliker, 1998) used this sensor for distance interferometry.Earlier than these reports Povel et al. (H. Povel, 1990) devised a similar sensor for imaging polarimetry, though it was not named lock-in CCD.
It follows that in lock-in CCDs, the accuracy of phase demodulation, which increases for a larger number of samples or a higher sampling rate, is in a trade-off with spatial resolution because storing a large number of samples requires more space within the pixel on the sensor chip.Note also that the nature of discrete temporal sampling in lock-in CCDs as well as in high-speed cameras, represented by a rectangular temporal window function, again brings about the problem of sensitivity to higher-order harmonic components in the incident beat signal.This degradation becomes severer for a smaller number of samples or a lower sampling rate.
Another type of 2-D phase-demodulating pixel array was developed by Bourquin et al. for optical coherence tomography (S.Bourquin, 2001a).The pixel consists of a photodetector followed by a feedback circuit with a low-pass filter (LPF) and a rectifier (S.Bourquin, 2001b).For an incident beat signal of a frequency above the LPF cutoff frequency, the pixel can demodulate the signal amplitude as a result of open-loop operation of the feedback circuit.This sensor, however, cannot perform phase demodulation because it does not have the circuit for it.Now the real-time 2-D HI presented in this article is compared to the techniques discussed so far.The CIS used in the real-time 2-D HI is exactly an ideal type of 2-D phase-demodulating pixel array.As a significant advantage over other techniques employing 2-D phase-demodulating pixel arrays and high-speed cameras, the CIS demodulates the phase of a continuous beat signal by continuous operation on analog circuitry under an ordinary frame rate, instead of discrete sampling and off-line computation.This property makes the real-time 2-D HI immune to the problems of rectangular windowing, light SNR, and frame

Principle
This section first describes the principle of the CIS, and then explains how the CIS demodulates amplitude and phase images of heterodyne beams in real time.

Correlation image sensor
Figure 1(a) depicts the architecture of the CIS, which consists of a pixel array and three external inputs of reference signals supplied to all of the pixels.The pixel circuit for passive readout is illustrated in Figure 1(b) (S.Ando, 2003).The photodiode (PD) converts the incident light into a photocurrent I(t) in proportion to its intensity f ij (t) at pixel (i, j).
) that flow through three NMOS transistors Q k , and accumulated in three capacitors connected to the drains of Q k over a frame period T. The balance among I k (t) depends on the gate voltages V k (t) as where  over T while redefining ρg k (t) as g k (t) and replacing I(t) with f ij (t) yields an expression for the accumulated charges Q k (i, j) as where • denotes an integral over T.T h et e r m f ij (t) , obtained by is proportional to the average light intensity, as in conventional image sensors.The most significant difference from them is the term which represents the temporal correlation between the intensity of the incident light f ij (t) and each of the reference signals g k (t).The three temporal correlations ∆Q k (i, j) have two degrees of freedom under a constraint ∑ 3 k=1 ∆Q k (i, j)=0c a u s e db y∑ 3 k=1 g k (t)=0.The charges Q k (i, j) are read out either passively through PMOS switches as shown in Figure 1(b) (S.Ando, 2003), or actively with voltage source followers (S.Han, 2010).Figure 2 shows photographs of CIS cameras with 64 × 64-pixel (S.Ando, 2003) and 200 × 200-pixel (S.Ando, 2007) CMOS chips of passive type.

Real-time amplitude and phase demodulation of heterodyne beams
Consider two heterodyne coherent beams with complex wavefronts u ij , v ij and a beat frequency ∆ν to interfere on the focal plane of the CIS.The intensity f ij (t) of the interfering beams is given by 5) where i denotes the imaginary unit (i 2 = −1) and background illumination n ij (t) is taken into account.The beat frequency ∆ν is chosen to be a multiple of the frame rate of the CIS, ∆ν = mT −1 (m: integer), or much higher than the frame rate, ∆ν ≫ T −1 .The interference amplitude A ij and phase θ ij are simultaneously demodulated from single-frame temporal correlation images ∆Q k (i, j), by supplying the CIS with three-phase reference signals g k (t) of the same Noting that the average intensity B ij and the background illumination n ij (t) are both uncorrelated with the sinusoudal reference signals g k (t),i .e .g k (t)B ij = g k (t)n ij (t) = 0, substitution of Eqs. ( 5) and ( 6) into Eq.( 4) under the condition ∆ν = mT −1 or ∆ν ≫ T −1 yields the single-frame temporal correlation images ∆Q k (i, j) as where noise contributions ξ k (i, j) are taken into account.The interference amplitude A ij and phase θ ij are obtained from ∆Q k (i, j) by least-squares estimation as where (i, j) is dropped for notational symplicity.

Experimental results
The theory on real-time 2-D HI using the CIS described in Section 3 are verified on two types of interferometer-a Michelson interferometer and speckle pattern interferometers for in-plane and out-of-plane deformation measurement.

Real-time heterodyne Michelson interferometry
To study the feasibility of the proposed real-time 2-D HI, a heterodyne polarizing Michelson interferometer was constructed with the CIS as depicted in Figure 3(a) (A.Kimachi, 2007).A pair of orthogonally-polarized heterodyne beams coaxially enter the interferometer through a beam expander L1.The beams are split by the polarizing beam splitter PBS1 and reflected on the mirrors M1 and M2 while passing through the quarter wave plates QWP1 and QWP2 twice, which rotate the polarization of each reflected beam by 90 • .The reflected beams are merged by PBS1 and interfere on the CIS through an analyzer POL1 and an imaging lens L2.
A reference beat signal is generated by taking out a small portion of the heterodyne beams with a beam splitter BS and detecting the interference intensity with a photodiode PD.This signal is converted into three-phase sinusoidal reference signals for the CIS.The heterodyne beams are emitted from a commercial two-frequency He-Ne laser source shown in Figure 3 The 64 × 64-pixel CIS in Figure 2(a) was used in the interferometer in Figure 3(a), operating at a frame rate of T −1 = 30 fps (≪ ∆ν). Figure 4 shows an example of average intensity, demodulated amplitude, and demodulated phase images computed from the raw output images Q k (i, j) with Eqs.(3), ( 4), ( 8) and (9) (A.Kimachi, 2007).The demodulated phase θ ij in Figure 4(c) is coded in gray scale as depicted in Figure 4(d).The abrupt edge from black to white in Figure 4(c) corresponds to a 2π jump.Figure 5 shows the results when the mirror M1 was moved by the deformation of the piezoelectric actuator for changes in voltage (A.Kimachi, 2007).Figure 5(a) shows a sequence of demodulated phase images for voltages from 30.0 V to 36.5 V in 0.5-V steps.The difference each phase image from that for 30.0 V is shown in Figure 5(b).The phase difference for the pixel at the image center is unwrapped and plotted in Figure 5(c) up to 50.0 V, along with a linear fit.The plot in Figure 5(c) exhibits a good linear response to the applied voltage, which is ideally linear to the displacement of M1, despite the open-loop control of the piezoelectric actuator.The slope of the linear fit in Figure 5(c) amounts to 65 nm/V in terms of optical path length, which is close to the characteristic of the piezoelectric actuator (61 nm/V).
The real-time heterodyne Michelson interfereometer in Figure 3 was applied to monitoring dynamic change in the interference phase of the heterodyne beams, which was caused by silicone oil inserted in front of the mirror M1 as depicted in Figures 6(a) and (b). Figure 6(c) shows a real-time sequence of phase difference images in every 0.2 s, i.e. every six frames, with respect to the initial state, while the silicone oil was heated by the heating cable (A.Kimachi, 2007).It is observed that gradation is buidling up near the surface of the silicone oil as time passes, suggesting changes in refractive index due to increase in temperature as well as convection that brings heated silicone oil up toward the surface.

Real-time heterodyne speckle pattern interferometry
The real-time 2-D HI was also applied to SPI. Figure 7(a) depicts a real-time heterodyne SPI system for in-plane deformation measurement (A.Kimachi, 2010a).The heterodyne beams are generated by a He-Ne laser and an electro-optic modulator (EOM), which shifts the phase between a pair of orthogonally-polarized coaxial beams from 0 to 2π linearly to time at a frequency of ∆ν = 100 Hz by a sawtooth-shaped driving voltage signal.The heterodyne beams are split by the polarizing beam splitter PBS and projected onto the object from symmetrically opposite directions with an inclination angle α = 45 • .The interference speckle pattern generated on the object is imaged through an analyzer POL onto the CIS, which is supplied with three-phase reference signals of the same frequency ∆ν as that of the EOM driving signal.The CIS demodulates the phase of the interference speckle pattern from single-frame output images, yielding a random pattern.The difference of the demodulated phase from that of the object at an initial state changes linearly to the in-plane deformation δx flat object with a rough surface while it was moved in an in-plane direction in 40-nm steps by a nano-scale linear stage (A.Kimachi, 2010a).Figure 8(a) shows a sequence of phase difference images with respect to the initial position of the object, smoothed with a 5 × 5-pixel Gaussian filter.The value of phase difference is coded again as in Figure 4(d).The images exhibit gradual changes in phase difference in accordance with the in-plane displacement.The average and standard deviation of unwrapped phase difference over the central 50 × 50-pixel region are plotted in Figure 8(b) along with a theoretical response, up to the displacement of 2 µm.The plot of the average phase difference agrees well with the theoretical response.The standard deviation was 0.13 rad, or 9 nm in terms of displacement, in average over the plotted range.
A similar experiment was carried out as well for the real-time heterodyne SPI system for out-of-plane deformation measurement depicted in Figure 7(b), which is based on a Michelson interferometer with the reflection mirrors replaced by the object and a reference plate with a rough surface (A.Kimachi, 2010a).The same 200 × 200-pixel CIS camera, heterodyne laser source, and flat object were used as in the system in Figure 7(a).Figure 9(a) shows a sequence of phase difference images with respect to the initial position of the object, while it underwent out-of-plane displacement along the optical axis in 40-nm steps.The images exhibit gradual theoretical plots is not as good as that for the in-plane displacement in Figure 8(b), exhibiting a drift of the experimental plot from the theoretical one.These observations can be attributed to the nature of the Michelson interferometer the SPI system in Figure 7(b) is based on, which involves two separate reflective objects and is thus more sensitive to disturbance than the SPI system for in-plane deformation in Figure 7(a).
The real-time heterodyne SPI system for in-plane deformation in Figure 7(a) was employed for monitoring an object under dynamic deformation.The object is a piezoelectric actuator placed horizontally and attached to a fixed frame, as depicted in Figure 10(a).Figure 10(b) shows a real-time sequence of phase difference images in every 0.12-s frame with respect to the initial state, featuring the junction between the object and fixed frame.The profile of the images along the horizontal white line is plotted in Figure 10(c).The left-hand side of the plot, corresponding to the actuator, exhibits a continuous increase in the slope as well as a gradual upward shift.The increasing slope indicates that the actuator is expanding toward the left, whereas the upward shift implies that the actuator itself also slides to the left.The right-hand side of the plot, corresponding to the fixed frame, exhibits a gradual downward shift of much smaller steps than that for the actuator.This suggests the existence of a strong force by the actuator that slightly pushes the fixed frame in the opposite direction to its expanding direction.

Application to a health monitoring system for large-scale structures
The real-time 2-D HI described in the previous sections is particularly effective in applications that require not only high accuracy but also real-time 2-D phase measurement.As one of such applications, a research project on health monitoring of large-scale structures has been promoted by Ando et al. (S. Sato, 2008a;b;c;S. Ando, 2009).This project aims at developing a networked sensing system that can measure relative deformation of large-scale structures such as bridges, tunnels, industrial plants and tall buildings in real time and at high accuracy over three axes of displacement and three axes of rotation.This kind of sensing system is drawing increasing attention because large-scale structures can cause catastrophic disasters when they are broken down from aging, fatigue, or defects.Health monitoring of large-scale structures, however, is not easy because they are located outdoors in tough enviroment and their size makes it very laborious to inspect the whole entity for defects.
As a solution to this problem, a networked sensing system that employs the real-time 2-D HI has been proposed (S.Sato, 2008a;b;c).The sensor network is comprised of a train of sensing units placed over a large-scale structure.The sensing unit consists of a pair of a CIS chip and a heterodyne laser source, and transmits coaxial heterodyne laser beams to one neighboring unit while receiving coaxial heterodyne laser beams from another neighboring unit.The zeros distributed over the beam area.The relative deformation can be known from a change in the amplitude and phase pattern of a multi-zero wavefront, which is easily detected on the CIS chip by the real-time 2-D HI at high accuracy.The multi-zero beams, which are generated by multiplexing Laguerre-Gaussian (LG) beams of different orders, have been found from extensive studies to possess many characteristics suitable for measurement and sensing (S.Sato, 2008a;b;c).
Based on these ideas, several experimental systems have been so far developed and tested in experiments on real-time tracking of multi-zeros (S.Ando, 2009).This project is currently at the stage of the development of a prototype system and the experimental evaluation on actual large-scale structures.

Conclusion
The real-time 2-D HI technique presented in this article is based on the use of the CIS as a two-dimensional array of pixels, each of which simultaneously demodulate the amplitude and phase of incident heterodyne beams at an ordinary frame rate.As the most significant advantage over other real-time 2-D HI or PSI techniques, the CIS-based technique ensures both maximal temporal and spatial resolution and high accuracy of phase measurement.The experimental results obtained on a polarizing Michelson interferometer and speckle pattern interferometers for in-plane and out-of-plane deformation measurement confirm the feasibility and usefulness of the real-time 2-D HI.The real-time 2-D HI is expected to find many application areas such as the health monitoring system for large-scale structures based on multi-zero beams.

382Interferometry-
Research and Applications in Science and Technology www.intechopen.comangular frequency ω expressed by

Fig. 5 .Fig. 6 .
Fig. 5. Results of the real-time heterodyne Michelson interferometer for changes in the piezoelectric actuator's voltage (A.Kimachi, 2007).(a) Demodulated phase images.(b) Images of phase difference with respect to 30.0 V. (c) Plot of the unwrapped phase difference at a pixel at the image center.at each pixel, as long as the deformation does not exceed the average speckle size to escape speckle decorrelation.The 200 × 200-pixel CIS camera in Figure 2(b) was used in the SPI system in Figure 7(a), operating at a frame rate of T −1 = 8.33 fps (= ∆ν/12).Figure 8 shows the results for a

Fig. 7 .Fig. 8 .
Fig.7.Schematics of the real-time heterodyne SPI systems with the CIS for (a) in-plane, and (b) out-of-plane deformation measurement (A.Kimachi, 2010a).changes in phase difference, as observed in Figure8(a), though they look a little noisier.The average and standard deviation of unwrapped phase difference over the central 50 × 50-pixel region are plotted in Figure9(b), along with a theoretical response.The standard deviation was 0.33 rad, or 17 nm in terms of displacement, in average over the plotted range, which is larger than the plot in Figure8(b).Moreover, the agreement between the experimental and

Fig. 9 .
Fig. 9. Results on the real-time heterodyne SPI system for out-of-plane displacement of a flat object with a rough surface (A.Kimachi, 2010a).(a) Sequence of phase difference images with respect to the initial position of the object, for displacement increasing in 40-nm steps.(b) Average and standard deviation of the images of unwrapped phase difference within a central 50 × 50-pixel region, along with a theoretical response.

Table 1
. Comparison of 2-D HI and PSI techniques.The boxes with an asterisk in each row indicate a trade-off relation.grabberinterface.Moreover, spatial resolution can be maximally utilized as far as the pixel size permits, without any loss by sharing among multiple phase-shifted images, as is the case with the real-time PSI methods.The real-time 2-D HI and PSI techniques discussed above are summarized in Table1.
,a n dρ is a constant.The signals g k (t), called reference signals, are supplied to the gates of Q k in each pixel under a constraint ∑ 3