Fusion of Optical and Thermal Imagery and LiDAR Data for Application to 3-D Urban Environment and Structure Monitoring

For many years, panchromatic aerial photographs have been the main source of remote sensing data for detailed inventories of urban areas. Traditionally, building extraction relies mainly on manual photo-interpretation which is an expensive process, especially when a large amount of data must be processed (Ameri, 2000). The characterization of a given object bases on its visible information, such as: shape (external form, outline, or configuration), size, patterns (spatial arrangement of an object into distinctive forms), shadow (indicates the outlines, length, and is useful to measure height, or slopes of the terrain), tone (color or brightness of an object, smoothness of the surface, etc.) (Ridd 1995). Automated assessment of urban surface characteristics has been investigated due to the high costs of visual interpretation. Most of those studies used multispectral satellite imagery of medium to low spatial resolution (Landsat-TM, SPOT-HRV, IRS-LISS, ALI and CHRIS-PROBA) and were based on common image-analysis techniques (e.g. maximum likelihood (ML) classification, principal components analysis (PCA) or spectral indices (Richards and Jia 1999)). The problems of limited spatial resolution over urban areas have been overcome with the wider availability of space-borne systems, which characterized by large swath and high spatial and temporal resolutions (e.g. Worl-View2). However, the limits on spectral information of nonvegetative material render their exact identification difficult. In this regard, the hyperspectral remote sensing (HRS) technology, using data from airborne sensors (e.g. AVIRIS, GER, DAIS, HyMap, AISA-Dual), has opened up a new frontier for surface differentiation of homogeneous material based on spectral characteristics (Heiden et al. 2007). This capability also offers the potential to extract quantitative information on biochemical, geochemical and chemical parameters of the targets in question (Roessner et al. 1998).


Introduction
For many years, panchromatic aerial photographs have been the main source of remote sensing data for detailed inventories of urban areas. Traditionally, building extraction relies mainly on manual photo-interpretation which is an expensive process, especially when a large amount of data must be processed (Ameri, 2000). The characterization of a given object bases on its visible information, such as: shape (external form, outline, or configuration), size, patterns (spatial arrangement of an object into distinctive forms), shadow (indicates the outlines, length, and is useful to measure height, or slopes of the terrain), tone (color or brightness of an object, smoothness of the surface, etc.) (Ridd 1995). Automated assessment of urban surface characteristics has been investigated due to the high costs of visual interpretation. Most of those studies used multispectral satellite imagery of medium to low spatial resolution (Landsat-TM, SPOT-HRV, IRS-LISS, ALI and CHRIS-PROBA) and were based on common image-analysis techniques (e.g. maximum likelihood (ML) classification, principal components analysis (PCA) or spectral indices (Richards and Jia 1999)). The problems of limited spatial resolution over urban areas have been overcome with the wider availability of space-borne systems, which characterized by large swath and high spatial and temporal resolutions (e.g. Worl-View2). However, the limits on spectral information of nonvegetative material render their exact identification difficult. In this regard, the hyperspectral remote sensing (HRS) technology, using data from airborne sensors (e.g. AVIRIS, GER, DAIS, HyMap, AISA-Dual), has opened up a new frontier for surface differentiation of homogeneous material based on spectral characteristics (Heiden et al. 2007). This capability also offers the potential to extract quantitative information on biochemical, geochemical and chemical parameters of the targets in question (Roessner et al. 1998).
The most common approach to characterizing urban environments from remote sensing imagery is land-use classification, i.e. assigning all pixels in the image to mutually exclusive classes, such as residential, industrial, recreational, etc. (Ridd 1995, Price 1998. In contrast, mapping the urban environment in terms of its physical components preserves the 30 heterogeneity of urban land cover better than traditional land-use classification (Jensen & Cowen, 1999), characterizes urban land cover independent of analyst-imposed definitions and more accurately captures changes with time (Rashed et al. 2001).
Hyperspectral thermal infrared (TIR) remote sensing has rapidly advanced with the development of airborne systems and follows years of laboratory studies (Hunt & Vincent 1968, Conel 1969, Vincent & Thomson 1972, Logan et al. 1975, Salisbury et al. 1987). The radiance emitted from a surface in thermal infrared (4-13μm) is a function of its temperature and emissivity. Emittance and reflectance are complex processes that depend not only on the absorption coefficient of materials but also on their reflective index, physical state and temperature. Most urban built environment studies are taking into account both temperature and emissivity variations, since these relate to the targets identification, mapping and monitoring and provide a mean for practical application.
The hyperspectral thermal imagery provides the ability for mapping and monitoring temperatures related to the man-made materials. The urban heat island (UHI) has been one of the most studied and the best-known phenomena of urban climate investigated by thermal imagery (Carlson et al., 1981;Vukovich, 1983;Kidder & Wu, 1987;Roth et al., 1989;Nichol, 1996). The preliminary studies have reported similarities between spatial patterns of air temperature and remotely sensed surface temperature (Henry et al., 1989;Nichol 1994), whereas progress studies suggest significant differences, including the time of day and season of maximum UHI development and the relationship between land use and UHI intensity (Roth et al., 1989). The recent high-resolution airborne systems determine the thermal performance of the building that can be used to identify heating and cooling loss due to poor construction, missing or inadequate insulation and moisture intrusion.
The spectral (reflective and thermal) characteristics of the urban surfaces are known to be rather complex as they are composed of many materials. Given the high degree of spatial and spectral heterogeneity within various artificial and natural land cover categories, the application of remote sensing technology to mapping built urban environments requires specific attention to both 3-D and spectral domains (Segl et al. 2003). Segl confirms that profiling hyperspectral TIR can successfully identify and discriminate a variety of silicates and carbonates, as well as variations in the chemistry of some silicates. The integration of VNIR-SWIR and TIR results can provide useful information to remove possible ambiguous interpretations in unmixed sub-pixel surfaces and materials. The image interpretation is based on the thematic categories (Roessner et al. 2001), which are defined by the rules of urban mapping and land-uses.
The ultimate aim in photogrammetry in generating an urban landscape model is to show the objects in an urban area in 3-D (Juan et al. 2007). As the most permanent features in the urban environment, an accurate extraction of buildings and roads is significant for urban planning and cartographic mapping. Acquisition and integration of data for the built urban environment has always been a challenge due to the high cost and heterogeneous nature of the data sets (Wang 2008). Thus, over the last few years, LiDAR (LIght Detection And Ranging) has been widely applied in the field of photogrammetry and urban 3-D analysis (Tao 2001, Zhou 2004. Airborne LiDAR technique provides geo-referenced 3-D dense points ("cloud") measured roughly perpendicular to the direction of flight over a reflective surface on the ground. This system integrates three basic data-collection tools: a laser scanner, a global positioning system (GPS) and an inertial measuring unit (IMU). The position and altitude of it determined by GPS/INS, therefore, the raw data are collected in the GPS reference system WGS 84.
Generally, 3-D urban built environment models are created using CAD (computer-aided design) tools. There have been many successful projects which have produced detailed and realistic 3-D models for a diverse range of cities (Dodge et al. 1998, Bulmer 2001, Jepson et al. 2001. These city models were created with accurate building models compiled with orthophotographs and exhibited an impressive, realistic urban environment (Chan et al. 1998). However, the creation of 3-D city models using CAD tools and orthophotographs faces some challenges: it is time-consuming and expensive.
The analysis of InSAR (Interferometric Synthetic Aperture Radar) and SAR (Synthetic Aperture Radar) data for urban built targets has several important benefits, such as the ability to adopt numerical tools, and the ability to provide results resembling the real-world situation. In addition, a relation can be found between target geometry and the measured scattering, and according to target-scattering properties, height-retrieval algorithms can be developed. The limitation of this method is that the targets in urban models have to be as detailed as possible; otherwise the results obtained in the modeled environment will be not reliable (Margarit et al. 2007).
The use of 3-D high-spatial-resolution applications in urban built environments is a mainstay of architecture and engineering practice. However, engineering practices are increasingly incorporating different data sets and alternative dissemination systems. Understanding, modeling and forecasting the trends in urban environments are important to recognize and assess the impact of urbanization for resource managers and urban planners. Many applications are suitable sources of reliable information on the multiple facets of the urban environment (Jensen & Cowen 1999, Donnay et al. 2001, Herold et al. 2003. These models have provided simulations of urban dynamics and an understanding of the patterns and processes associated with urbanization (Herold et al. 2005). However, the complexity of urban systems makes it difficult to adequately address changes using a single data type or analysis approach (Allen & Lu 2003).
This chapter presents techniques for data fusion and data registration. The ability to include an accurate and realistic 3-D position, quantitative spectral information, thermal properties and temporal changes provides a near-real-time monitoring system for photogrammetric and urban planning purposes. The method is focusing on registration of multi-sensor and multitemporal information for 3-D urban environment monitoring applications. Generally, data registration is a critical pre-processing procedure in all remote-sensing applications that utilizes multiple sensors inputs, including multi-sensor data fusion, temporal change detection, and data mosaicking. The main objective of this research is a fully controlled, nearreal-time, natural and realistic monitoring system for an urban environment. This task led us first to combine the image-processing and map-matching procedures, and then to incorporate remote sensing and GIS tools into an integrative method for data fusion and registration. To support this new data model, traditional spatial databases were extended to support 5-D data.
This chapter is organized as follows. Section 2 describes the materials and methods, which are implemented in the 3-D urban environment model presented in Section 3. Section 4 addresses to the generic 3-D urban application, which involves data fusion and contextual information of the environment.

www.intechopen.com
Remote Sensing -Advanced Techniques and Platforms 32 2. Materials and methods

Study area
Two separate datasets were utilized in this study. The first dataset was acquired over the suburban Mediterranean area on 10 Oct 2006 at 03h37 UTC and at 11h20 UTC. This area combines natural and engineered terrains (average elevation of 560m above sea level), a hill in the north of the studied polygon area and a valley in the center. The entire scene consists of rows of terraced houses located at the center of the image. The neighborhood consists of cottage houses (two and three floors) with tile roofs, flat white-colored concrete roofs and balconies, asphalt roads and parking lots, planted and natural vegetation, gravel paths and bare brown forest soil. The height of large buildings ranges from 8 to 16 m. A group of tall pine trees with various heights and shapes are located on the streets and the Mediterranean forest can be found in the corner of the scene.
The second dataset was acquired over urban settlement, on 15 Aug 2007 at 02h54 UTC and at 12h30 UTC. This area combines natural, agriculture and engineered terrains (average elevation of 30m above sea level). The urban settlement consists of houses (two and three floors) and public buildings (schools and municipalities buildings) with flat concrete, asphalt or whitewash roofing, asphalt roads and parking lots, planted and natural vegetation, gravel paths, bare brown reddish Mediterranean and agriculture soils, greenhouses and whitewash henhouse roofing. The height of large buildings ranges from 3 to 21 m.

Data-acquisition systems
The research combines airborne and ground data collected from different platforms and different operated systems. The collected imagery data were validated and compared to the ground truth in situ measurements collected during the campaigns.
The first airborne platform carries AISA-Dual hyperspectral system. The airborne imaging spectrometer AISA-Dual (Specim Ltd.) is a dual hyperspectral pushbroom system, which combines the Aisa EAGLE (VNIR region) and Aisa HAWK (SWIR region) sensors. For the selected campaigns, the sensor simultaneously acquired images in 198 contiguous spectral bands, covering the 0.4 to 2.5 µm spectral region with bandwidths of ~10 nm for Aisa EAGLE and ~5 nm for Aisa HAWK. The sensor altitude was 10,000 ft, providing a 1.6 m spatial resolution for 286 pixels in the cross-track direction. A standard AISA-Dual data set is a 3-D data cube in a non-earth coordinate system (raw matrix geometry).
The second airborne platform carries hyperspectral TIR system, which is a line-scanner with 28 spectral bands in the thermal ranges 3-5 μm and 8-13 μm. It has 328 pixels in the crosstrack direction and hundreds of pixels in the along-track direction with a spatial resolution of 1.4m.
The third airborne platform carries the LiDAR system. This system operates at 1500 nm wavelength with a 165 kHz laser repetition rate and 100 Hz scanning rate and provides a spatial/footprint resolution of 0.5 m and an accuracy of 0.1 m. The scanner has a multi-pulse system that could record up to five different returns, but in this study, only the first return was recorded and analyzed.
The ground spectral camera HS (Specim Ltd.) is a pushbroom scan camera that integrate ImSpector imaging spectrograph and an area monochrome camera. The camera's sensitive high speed interlaced CCD (Charge-Coupled Device) detector simultaneously acquires images in 850 contiguous spectral bands and covers the 0.4 to 1 µm spectral region with bandwidths of 2.8 nm. The spatial resolution is 1600 pixels in the cross-track direction, and the frame rate is 33 fps with adjustable spectral sampling.
The ground truth reflectance data were measured for the calibration/validation targets by the ASD "FieldSpec Pro" (ASD.Inc, Boulder, CO) VNIR-SWIR spectrometer. Internally averaged scans were 100 ms each. The wavelength-dependent signal-to-noise ratio (S/N) is estimated by taking repeat measurements of a Spectralon white-reference panel over a 10min interval and analyzing the spectral variation across this period. For each sample, three spectral replicates were acquired and the average was used as the representative spectrum. The ground truth thermal data were collected by a thermometer and thermocouples installed within calibration/validation targets (water bodies) and a thermal radiometer infrared camera (FLIR Systems, Inc.).

Data processing
This research integrates multi-sensor (airborne sensor, ground camera and field devises) and multi-temporal information into fully operational monitoring application. The aim of this sub-paragraph is to present several techniques for imagery and LiDAR data processing.
The classification approaches for airborne and ground hyperspectral imagery are firstly presented. The radiance measured by these sensors strongly depends on the atmospheric conditions, which might bias the results of material identification/classification algorithms that rely on hyperspectral image data. The desire to relate imagery data to intrinsic surface properties has led to the development of atmospheric correction algorithms that attempt to recover surface reflectance or emission from at-sensor radiance. Secondly, the LiDAR data are processed by applying the surface-based clustering methods.

Hyperspectral airborne and ground imagery
Accurate spectral reflectance information is a key factor in retrieving correct thematic results. In general, the quality of HRS sensors varies from very high to moderate (and even very poor) in terms of signal-to-noise ratio, radiometric accuracy and sensor stability. Instability of the sensors' radiometric performance (stripes, saturation, etc.) might be caused by either known or unknown factors encountered during sensor transport, installation and/or even data acquisition. As part of data pre-processing, these distortions have to be assessed and quantified for each mission.
A full-chain atmospheric calibration SVC (supervised vicarious calibration) method (Brook & Ben-Dor 2011a) is applied to extract reflectance information from hyperspectral imagery. This method is based on a mission-by-mission approach, followed by a unique vicarious calibration site. In this study, the acquired AISA-Dual and HS images were subjected to the SVC method, which includes two radiometric recalibration techniques (F1 and F2) and two atmospheric correction approaches (F3 and F4). The atmospheric correction incorporate deshadow algorithm, which is applied o the map provided by the boresight ratio band (Brook & Ben-Dor 2011b).
The hyperspectral reflectance images are subjected to the data processing stage, which is operated in four steps (Figure 1). First step is a general coarse classification. Each "pure" pixel is assigned to a class in order to predefine the threshold of the probabilistic output of a support vector machine (SVM) algorithm, or remains unclassified (Villa et al., 2011). The unclassified pixels might associate with mixed spectra pixels, thus their classification is addressed at the third stage by the unmixing method in order to obtain the abundance fraction of each endmember class. Prior to this step, a second step is applied, where spectral data are reduced by the selected algorithm. The input variables in terms of absorption features can be reduced through a sequential forward selection (SFS) algorithm (Whitney, 1971). This method starts with the inclusion of feature sets one by one to minimize the prediction error of a linear regression model and focuses on conditional exclusion based on feature significance (Pudil et al., 1994). This step is proven to enhance overall performance of spectral models. The nonnegative matrix factorization (NMF) was offered as an alternative method for linear unmixing (Lee et al., 2000). This algorithm search for the source and the transform by factorizing a matrix subject to positive constraints based on gradient optimization and Euclidean norm designation (Pauca et al, 2006;Robila & Maciak, 2006). We generated an algorithm that starts with the random linear transform to the nonnegative source data. The algorithm is continuously computing scalar factors that are chosen to produce the "best" intermediate source and transform. At each step of the algorithm the source and transform should remain positive. The final stage is a method for image segmentation combined with a Markov random field (MRF) model under a Bayesian framework (Yang & Jiang, 2003).
The validation of the thematic map is performed by comparing ground truth and image reflectance data of the selected targets. The ten well-known targets (areas of approximately 30-40 pixels) were spectrally measured (using ASD SpecPro) and documented. The overall accuracy for the Ma'alot Tarshiha images was 96.8 and for the Qalansawe images it was 97.4. The exact location of each target within the scenes was captured using aerial orthophoto and ground truth field survey. The confusion matrices (Tables 1 and 2) and ROC (receiver operating characteristic) curve (Table 3) were calculated by comparison between number of pixels in each class (concrete, asphalt, scuffed asphalt) and ground truth maps. The overall accuracy of both images stands in good agreement, thus it can be concluded that the suggested classification algorithm (Figure 1)

Thermal airborne and ground imagery
Atmospheric correction is a key processing step for extracting information from thermal infrared imagery. The ground-leaving radiance combined with temperature/emissivity separation (TES) algorithms are generated and supplied to in-scene atmospheric compensation ISAC 1 (Young et al., 2002). This model requires only the calibrated, ataperture radiance data to estimate the upwelling radiance and transmittance of the atmosphere. It is an effective atmospheric correction that produces spectra that compare favorably to the Planck function.
The ground truth must include several targets as water, sand or soil continuously measured by installed thermocouples. The generating atmospheric data cube may be used as an input to a temperature emissivity separation algorithm (normalized emissivity method). The proposed thermal classification method follows the same four stages of data processing (SVM's probabilistic map; data reduction; umnixing; classification) applied to the preprocessed emissivity imagery (Figure 2).

Fig. 2. Flow chart scheme of the thermal airborne and ground data preprocessing
From the physical definition, the spectral characteristics of urban materials in the reflective and thermal ranges are related. Segl (Segl et al., 2003) showed that materials with high albedos in the reflective range produce low albedos in the thermal range and vice versa, due to a better energy absorption in the reflective region. However, it is reported that bitumen roofing and asphalt pavement generate distinct spectral differences in the thermal wavelength range. The thermal measurements remain a compelling focus on a climate research in the built urban areas. However, the thermal airborne and ground imagery permit definition of UHI (for the ground surface) and resolve streets, roofs and walls. The successful numerical model of the urban areas is acquired during night-time conditions, when solar shading is absent and turbulent interactions are minimal.
The validation of the thematic map is performed by comparing ground truth and image emissivity data. The five targets (concrete, sand lot, bitumen, tile roof and polyethylene) were measured and documented. The resulting emissivity signatures are in good agreement with ground-truth data (two examples in Figure 3A and 3B). The results presented here confirm the robustness and stability of the suggested algorithm.

Airborne LiDAR data
LiDAR data provides precise information about the geometrical properties of the surfaces and can reflect the different shapes and formations in the complex urban environment. The point cloud (irregularly spaced points) was interpolated into the digital surface model (DSM) by applying the Kriging technique (Sacks et al. 1989). The Kriging model has its 1 ISAC (in-scene atmospheric compensations) model is implemented in ENVI® Airborne Thermal Ground Thermal Input Images: Preprocessing: Radiometric correction by system operator Atmospheric correction Tem p era t u re Im a g e Emissivity Image Fig. 3. Emissivity calculated from the thermal radiance. A is a tile roof and B is a bitumen roof origins in mining and geostatistical applications involving spatially and temporally correlated data (Cressie 1993).
The surface analysis (Figure 4) is first represented as a DEM (digital elevation model) of the scanned scene, where data are separated into on-terrain and off-terrain points (Masaharu and Ohtsubo 2002). In this study, the Kriging Gaussian correlation function was utilized to visualize and illustrate the edited DEM as a surface-response function. Note that the interpolation converts irregularly spaced LiDAR data to a self-adaptive DSM.
The DTM (digital terrain model) was created by a morphological scale-opening filter, using square structural elements (Rottensteiner et al., 2003). Then, according to the filter, the slope  map is estimated. The next stage is to fragment a surface model convolved with highly heterogeneous terrain slopes into subareas with fixed slope (Zhang et al., 2003;Shan & Sampath 2005). At this stage, the terrain is uniformly normalized and the separation between on-and off-terrain points is applicable.
The building boundary is determined by a modified convex hull algorithm (Jarvis 1973) which classifies the cluster data into boundary (contour/edge) and non-boundary (intershape) points (Jarvis 1977). Separating points located on buildings from those on trees and bushes, is a difficult task (Wang & Shan 2009). The common assumption is that the building outlines are separated from the trees in terms of size and shape. The dimensionality learning method, proposed by Wang and Shan (2009), is an efficient technique for this purpose.
In relatively flat urban areas, the roads, which have the same elevation (height) as a bare surface, can be extracted by arrangement examination. The simple geometric and topological relations between streets might be used to improve the consistency of road extraction. First, the DEM data are used to obtain candidate roads, sidewalks and parking lots. Then the road model is established, based on the continuous network of points which are used to extract information such as centerline, edge and width of the road (Akel et al. 2003;Hinz & Baumgartner 2003;Cloude et al., 2004).

Data registration: Automatic and manual approaches
The optical and thermal imagery and LiDAR data have fundamentally different characteristics. The LiDAR data (monochromatic NIR laser pulse) provides terrain characteristics; hence, optical imagery (radiation reflected back from the surface at many wavelengths) provides ability for in situ, easy, rapid and accurate assessment of many materials on a spatial/spectral/temporal domain, and thermal imagery determines temperatures and radiance signature of urban materials and land covers. Since all these datasets ( Figure 5) are crucial for the assessment and classification of the urban area, a novel method for automatic registration and data fusion is needed.
Data fusion techniques combine data from multiple sensors and related information from associated databases. The integrated data set achieves higher accuracy and more specific inferences that might be obtained by the use of single sensor alone. In general, data registration is a critical preprocessing procedure in all remote-sensing applications that utilizes multiple sensor inputs, including multi-sensor data fusion, temporal change www.intechopen.com detection, and data mosaicking (Moigne et al., 2002). In manual registration, the selection of control points (CPs) is usually performed by a human operator. This has proven to be inaccurate, time-consuming, and unfeasible due to data complexity, which makes it cumbersome or even impossible for the human eye to discern the suitable CPs. Therefore, researchers have focused on automating feature detection to align two or more data sets with no need for human intervention.
The automatic registration of data sets has generated extensive research interest in the fields of computer vision, medical imaging and remote sensing. Comprehensive reviews have been published by Brown (1992) and Zitova and Flusser (2003). Many proposed schemes for automatic registration employ a multi-resolution process (Viola and Wells 1997, Wu & Chung 2004, Fan et al., 2005, Zavorin & Moigne 2005, Xu & Chen 2007. The existing automatic data-registration techniques based on spatial information fall into two categories: intensity-based and feature-based (Zitova and Flusser 2003). The featurebased technique extracts salient structures from sensed and reference data sets by accurate feature detectin and by the overlap criterion. As the relevant objects of intereset (e.g., roofs) and lines (e.g., roads) are expected to be stable in time at a fixed position, the feature-based method is more suitable for multi-sensor and multi-data set fusion, change detection and mosaicking. The method generally consists of four steps (Jensen, 2004): (1) CP extraction; (2) transformation-model determination; (3) image transformation and resampling, and (4) assessment of registration accuracy. The first step is the most complex, and its success essentially determines registration accuracy. Thus, the detection method should be able to detect the same features in all projections and in different data, regardless of the particular image/sensor/data type deformation. Despite the achieved performance, the existing methods operate directly on gray intensity values and hence they are not suited for handling multi-sensor and multi-type data sets.
The suggested algorithm is an adapted version of the four stages AIRTop ( Figure 6) algorithm (Brook & Ben-Dor 2011c). First, the significant features are extracted from all input data sets and converted to a vector format. Since the studied scene has a large area, regions of interest (ROI) with relatively large variations are selection. The idea of addressing the registration problem by applying a global-to-local level strategy (the whole image is now divided into regions of interest which are treated as an image) proves to be an elegant way

Input
Hyperspectral / Thermal Airborne Ground Imagery of speeding up the whole process, while enhancing the accuracy of the registration procedure (Chantous et al. 2009). Thus, we expected this method to greatly reduce false alarms in the subsequent feature extraction and CP identification steps .

Input
To select the distinct areas in the vector data sets, a map of extracted features is divided into adjacent small blocks (10% × 10% of original image pixels with no overlap between blocks). Then, the significant CPs extraction has been performed by applying the SURF algorithm (Brown & Lowe, 2002). First the fast-Harris corners Detector (Lindeberg, 2004), which based on an integral image, was performed. The Hessian matrix is responsible for primary image rotation using principal points that identified as "interesting" potential CPs in the block. The local feature representing vector is made by combination of Haar wavelet response. The values of dominant directions are defined in relation to the principal point. As the number of interesting points tracked within the block is more than the predefined threshold, the block is selected and considered a suitable candidate for CPs detection.
The spatial distribution and relationship of these features are expressed by topology rules (one-to-one) and they are converted to potential CPs by determining a transformation model between sensed and reference data sets. The defined rules for a weight-based topological map-matching (tMM) algorithm manage (Velaga et al. 2009), transform and resample features of the sensed goereferenced LiDAR data according to a non georeferenced imagery in order to reserve original raw geometry, dimensionality and imagery matrices (imagery pixels size and location).

Fast-Harris
In the proposed 3-D urban application, the manual registration is used to register facades imagery and thematic mapping acquired by ground sensors and simplify buildings model extracted from LiDAR. This method is executed by a human operator, who identifies a set of corresponding CPs from the images and referenced control building model. Despite the fact that manual registration has been proven inaccurate and time-consuming due to data complexity, this method is still the most widely used technique. We found that for the current data sets, manual registration is the easiest and most accurate solution.

3-D urban environment model
The urban database-driven 3-D model represents a realistic illustration of the environment that can be regularly updated with attribute details and sensor-based information. The spatial data model is a hierarchical structure (Figure 7), consisting of elements, which make up geometries, which in turn composes layers. Fig. 7. The 3-D urban environment application's conceptual architecture A fundamental demand in non-traditional, multi-sensors and multi-type applications is spatial indexing. A spatial index, which is a logical index, provides a mechanism to limit searches based on spatial criteria (such as intersection and containment). Due to the variation of data formats and types, it is difficult to satisfy the frequent updating and extension requirements for developing urban environments.
An R-tree index is implemented on spatial data by using Oracle's extensible indexing framework (Song et al. 2009). This index approximates the geometry with a single rectangle that minimally encloses the geometry (minimum bounding rectangle MBR). A bounding volume is created around the 3-D object, which equals the bounding volume around the solid. The index is helpful in conducting very fast searches and spatial analyses over large 3-D scenes.
CityGML 2 is an application based on OGC's (open geospatial consortium) GML 3.1. This application not only represents the graphical appearance but in particular, it takes care of the semantic properties (Kolbe et al. 2005), such as the spectral/thematic properties, and model evaluations. The main advantage is the ability to maintain different levels of detail (Kolbe & Bacharach 2006). The underlying model differentiates three levels of detail, for which objects become more detailed as the level incise.
The 3-D urban application is based on an integrated data set: spectral models, ground camera and airborne images, and LiDAR data. The system requirements are defined to include geo-spatial planning information and one-to-one topology. The concept architecture diagram is presented in Figure 4. As the model consist visualization and interactivity with maps and 3-D scenes, the interface includes 3-D interaction, 2-D vertical and horizontal interactions and browsers that contain spectral/thematic temporal information. The 3-D urban application provides services such as thematic mapping, and a complete quantitative review of the building and it's surrounding with respect to temporal monitoring. The design of the application shows the possibilities of delivering integrated information and thus holistic views of whole urban environments in a freeze-frame view of the spatiotemporal domain.
The self-sufficient/self-determining levels of the integrated information contribute different parts to this global urban environmental application. The first level (Figure 8), termed "City 3-D", supplies three different products: 1) integrated imagery and LiDAR data, 2) 3-D thematic map, and 3) 2-D thematic map (which includes 3-D analysis layers such as terrain properties, spatial analysis, etc.). The second level, termed "Building Model" (Figure 9), focuses on a single building in 3-D and provides two additional products: 1) integrated imagery and building model extracted from LiDAR data set, and 2) 3-D thematic map for general materials classification, and quantitative thematic maps implemented by spectral models.
The most specific and localized level is the third level, termed "Spectral Model" ( Figure  9). The area of interest in this level is a particular place (a patch) on the wall of the building in question. The spatial investigation at this level is a continuation of the previous level; yet, the data source consists of spectral models that are evaluated for spectral in-situ point measurements. This level does not provide any integrated and rectified information, but provides geo-referencing of the results of the spectral models in realistic 3-D scale. This level completes the database of the suggested 3-D urban environment application.

Spatial selection
Data processing

3-D urban environment application
The 3-D monitoring built urban environment application, up to this point, employs single processing algorithms applied on imagery or LiDAR data, without taken into account contextual information. The data fusion application must provide fully integrated information, both of the classification products and the context within the scene. In the proposed application, a complete classification and identification task consist of subtasks, which have to operate on material and object characteristic/shape levels provided by accurately registered database. Moreover, the final fused and integrated application should be operated on objects of different sizes and scales, such as a single building detected within the urban area or a selected region on a building facade. The multi-scale and multi-sensor data fusion is possible with the eCognition procedure (user guide eCognition, 2003), when the substructures are archived by a hierarchical network.

Spatial selection
Data processing

2-D thematic analysis/interaction
Layer Spectra model

Quality Map
The results of spectral/thermal classification processes are by far not only a spectral/thematic aggregation of classes converted to polygons or polylines (in vector format), but also a spatial and semantic structuring of the scene content (example of roofs extraction in Figure 11). The resulting network of extracted and identified objects can be seen as a spatial/semantic network of the scene. The local contextual information describes the joint relationships and meaningful interactions between those objects in the build urban environment and linked multi-scale and multi-sensor products. This hierarchy in the rulebase design allows a well-structured incorporation of knowledge. Fig. 11. Hierarchical rule-base structure in eCognition In fact, now each object is identified not only by its spectral, thermal, textural, morphological, topological and shape properties, but also by its unique information linkage and its actual neighbors. The data is fused by mutual dependencies within and between objects that create a semantic network of the scene. To assure high level accuracy and operational efficiency the input products are inspected by the basic topological rule, which obligates that object borders overlay borders of objects on the next layer. Therefore, the multi-scale information, which is represented concurrently, can be related to each other.
The semantic network of fuzzy logic is an expert system that quantities uncertainties and variations of the input data. The fuzzy logic, as an alternative approach for the Boolean statements, avoids arbitrary thresholds and thus, it is able to estimate a real world environment (Benz et al., 2004). The implemented rules are guided by the reliability of class assignments, thus the solution is always possible, even if there are contradictory assignments (Civanlar & Trussel, 1986). This logic proposes a deliberate choice and parameterization of the membership function that established the relationship between object features and acceptable characteristics. Since the design is the most crucial step to

Roof Level 1 Segmentation
Hierarchical rule-base structure in eCognition: introduce expert knowledge and information into the logic, the better and detailed the description of the real world environment are modeled by the membership function, the better the data fusion.
The operational system controls that a first class hierarchy will be loaded and used in the next step for data integration. Based on this preliminary fusion, first objects of interest are created from object primitives by thematic-based fusion. The same steps are performed until the final information (spectral quantitative model) is applied. The results are registered and integrated information is followed by the reliability map, which is established by the primary accuracy and classification confidence of each input data. The reliability map is important for post-processing inspection and testing routines; objects with low reliability must be assigned manually because no decision is possible. The suggested application involves semi-automatic or even manual stages, which have proven to be time-consuming operations. Yet, due to the expert system support, it is a time efficient application that produces highly accurate and reliable merged information.

Discussion
In this chapter, we present techniques for data fusion and data registration in several levels.
Our study focused on the registration and the integration of multi-sensor and multitemporal information for a 3-D urban environment monitoring application. For that purpose, both registration models and data fusion techniques were used.
The 3-D urban application satisfies a fundamental demand for non-traditional, multi-sensor and multi-type data. The frequent updating and extension requirement is replaced by integrating the variation in data formats and types for developing an urban environment. The main benefit of 3-D modeling and simulation over traditional 2-D mapping and analysis is a realistic illustration that can be regularly updated with attribute details and remote sensor-based quantitative/thermal information and models.
The proposed application offers an advanced methodology by integrating information into a 5-D data set. The ability to include an accurate and realistic 3-D position, quantitative information, thermal properties and temporal changes provide a near-real-time monitoring system for photogrammetric and urban planning purposes. The main objectives of many studies are linked to, and rely on a historical set of remotely sensed imagery for quantitative assessment and spatial evolution of an urban environment (Jensen and Cowen 1999, Donnay et al. 2001, Herold et al. 2003. The well-known methodology is pattern observation in the spatiotemporal and spectral domains. The main objective of this research is a fully controlled, near-real-time, natural and realistic monitoring system for an urban environment. This task led us first to combine the image-processing and map-matching procedures, and then incorporate remote sensing and GIS tools into an integrative method for data fusion and registration.
The proposed application for data fusion proved to be able to integrate several different types of data acquired from different sensors, and which are additionally dissimilar in rotation, translation, and possible scaling. The data fusion operated by fuzzy logic is a final product of the application. This approach is an important stage for quality assurance and validation but furthermore for information fusion in current and future remote sensing systems with multi-sensor sources.
The multi-dimensionality (5-D) of the developed urban environment application provides services such as thematic and thermal mapping, and a complete quantitative review of the building and its surroundings. These services are completed by providing the ability for accurate temporal monitoring and dynamic changes (changed detection) observations. The application design shows the possibility of delivering integrated information, and thus holistic views of whole urban environments, in a freeze-frame view of the spatio-temporal domain.

Conclusion
In conclusion, the suggested application may provide the urban planners, civil engineers and decision makers with tools to consider quantitative spectral information and temporal investigation in the 3-D urban space. It is seamlessly integrating the multi-sensor, multidimensional, multi-scaling and multi-temporal data into a 5-D operated system. The application provides a general overview of thematic maps, and the complete quantitative assessment for any building and its surroundings in a 3-D natural environment, as well as, the holistic view of urban environment.

Acknowledgment
This research work is supported by Discovery Grand (3-8163) from the Ministry of Science of Israel. The authors would like to express their deepest gratitude for this opportunity.