Modes and scenarios of the FCUGM.
The dynamics of urban growth are the direct consequence of the actions of individuals, and public and private organisations, which act to change the urban landscape simultaneously over space and time. Since previous urban form has a strong influence on the present, a prime concern of urban planners, spatial scientists and government authorities is to understand how a city has grown in the past in order to predict the growth of the city in the future. This requires flexible tools that allow planners to examine the impacts and potential consequences of applying different development policies, strategies and future plans . However, traditional linear, static and top-down models are unable to adequately capture the processes underlying urban change. The non-linearity of spatial and temporal relationships and irregular, uncoordinated and uncontrolled local decision-making gives rise to seemingly coordinated global patterns that define the size and shape of cities in familiar ways [2-7]. Cities are now increasingly recognized as complex systems and display many of the characteristic traits of complexity, i.e. non-linearity, self-organization and emergence. Cellular Automata (CA) offer a modeling framework and a set of techniques for modelling the dynamic processes and outcomes of self-organizing systems [8-13]. Since the late 1980s they have demonstrated significant potential benefits for urban modelling through their simplicity, flexibility and transparency [8, 14-17]. CA are capable of generating complex patterns in aggregate form by using relatively simple local transition rules, i.e. by recursive development decisions being made at individual cells or sites [2, 15, 18]. However, cities are also influenced by global factors representing government polices (such as broader social, economic and technological factors). This has led to a number of hybrid-type urban growth models, which take into consideration local, regional and global factors [19-22]. When integrated with other technologies such as GIS and remote sensing, the potential of CA for geospatial, temporal and sectoral studies increases significantly through the ability of CA to utilise physical, environmental, social and economic data in their simulations . For example, remote sensing and GIS can be integrated with CA for providing detailed land use information as well as information on other characteristics of cities to produce realistic simulations of urban change .
A current challenge facing CA urban growth models is the lack of rigorous calibration procedures [21, 25-27]. Progress in the evolution of algorithms, particularly from artificial intelligence (AI), has, however, created many new options for calibrating these complex models. For example,  suggested that heuristic-based searches using AI would be an effective approach for optimising spatial problems, since they offer many advantages for model calibration compared to traditional methods. An example of an urban growth CA model calibrated using AI was developed in [29-31]. They presented an urban planning tool for the city of Riyadh, Saudi Arabia, which is one of the world’s major cities undergoing rapid development. At the core of the system is a Fuzzy Cellular Urban Growth Model (FCUGM), which is capable of simulating and predicting the complexities of urban growth. This model was shown to be capable of replicating the trends and characteristics of an urban environment during three periods: 1987-1997, 1997-2005 and 1987-2005.
Along with calibration, one of the most significant aspects of any model is to verify, validate and assess its performance. This is normally undertaken by verifying the model’s output against the real-world system through evaluation of goodness-of-fit tests. Validation can be defined as ‘a demonstration that a model within its domain of applicability possesses a satisfactory range of accuracy consistent with the intended application of the model’ . In terms of urban CA models, the validation process refers to the approach by which the performance of the model is assessed by comparing the simulated map (one generated by the model) with the observed map (based on ground truth). The observed map should be accurate and shape the benchmark for comparison. A good performing urban CA model generates outcomes that capture the basic features of urban forms between simulated and observed spatial patterns . Researchers have utilised a combination of different methods for validating CA models. For example, in , thirty-three urban CA models were reviewed and compared using a number of different criteria including the types of validation method employed. In some cases no validation method was used since the models were largely hypothetical or idealized, while in other models, a range of different methods were employed including one or a combination of the following approaches: visual comparison, confusion matrices , statistical measures , a fractal index and analysis [8, 21, 34], landscape metrics , spatial statistics, for example, Moran’s I index [8, 25] and structural measurements such as the Lee-Sallee index . It it clear from the review , however, that there is no consensus on how CA models of urban growth should be validated and research in this area has not progressed that much [26-27].
The focus of this chapter is on the techniques used to validate the performance of the FCUGM model; however these approaches are applicable to urban CA models more generally. A brief overview of the fuzzy cellular urban growth model (FCUGM) for the city of Riyadh is first provided. We then present seven different validation metrics including visual inspection, accuracy and spatial statistics, metrics for spatial pattern and district structure detection as well as spatial multi-resolution validation. Results of these methods are given followed by a discussion of the usefulness of the different validation approaches in relation to the assessment of the FCUGM.
2. The Fuzzy Cellular Urban Growth Model (FCUGM) for the City of Riyadh
The Fuzzy Cellular Automata Urban Growth Model (FCUGM) is driven by the following simple rule of development:
where a new urban cell, , is created at time t+1 if the cell’s development possibility (DP) is greater than or equal to a transition threshold parameter, λ, which is determined through the calibration process. The DP is a function of the development suitability of a cell and a stochastic disturbance factor. The development suitability is, in turn, a function of four driving forces, i.e. transportation , urban agglomeration and attractiveness , topographical constraints and a factor that encompasses planning policies and regulations :
The four driving forces of urban growth (TSF, UAAF, TCF and PRF) are themselves functions of a series of fuzzy input variables expressed as follows:
where the TSF is a function of Accessibility to Local Roads (ALR), Accessibility to Main Roads (AMR) and Accessibility to Major Roads (AMJR); the UAAF is determined by a combination of Urban Density (UD), Accessibility to Town Centres (ATC) and Accessibility to Employment Centres and Socio-Economic Services (AECSES); the TCF is a function of Gradient (G) and Altitude (A); and the PPRF takes Planned Areas (PA) and Excluded Areas (EA) into account. These drivers of urban growth are integrated via a fuzzy rule base, where the membership functions and the rules are determined through calibration. A fuzzy inference engine is used to process the fuzzy rules and produce a fuzzy development suitability score at each cell. These fuzzy values are then defuzzified and used in combination with the stochastic disturbance factor and the transition threshold to determine whether a given cell becomes an area of further urban development. The full details of the model are provided in [29, 31].
To calibrate the model, a stratified random sample consisting of 60% urban and 40% non-urban cells was utilised in combination with a genetic algorithm (GA) where a single objective function consisting of the mean squared error and the root mean squared error was employed. The use of these two measures together was designed to penalise model instances in which the parameters fell outside of an acceptable range. Nine different model instances were developed, which are listed in Table 1. These nine instances were based on different complexities of fuzzy rule (the modes) and different drivers (the scenarios). Mode 1 included fuzzy rules with only a single driver, e.g. transportation or topography while modes 2 and 3 had multiple drivers connected by the AND operator in the fuzzy rules. Scenarios considered different combinations of drivers in order to determine how well the different drivers were able to explain the observed urban development on their own and in combination. Thus M3-S1 is the most complex of the FCUGM instances. The top three performing models were M1-S4, M2-S4 and M3-S1, which clearly indicates that all the drivers are important in explaining urban growth in the city of Riyadh. These three simulations will be the focus of the validation process in this chapter.
|Mode/Scenario||Acronym||Name of Simulation|
|Mode 1 – Scenario 1||M1-S1||Transportation|
|Mode 1 – Scenario 2||M1-S2||Urban density-attractiveness|
|Mode 1 – Scenario 3||M1-S3||Topography|
|Mode 1 – Scenario 4||M1-S4||Transportation, urban density-attractiveness and topography|
|Mode 2 – Scenario 1||M1-S1||Transportation and topography|
|Mode 2 – Scenario 2||M2-S2||Transportation and urban density-attractiveness|
|Mode 2 – Scenario 3||M2-S3||Topography and urban density-attractiveness|
|Mode 2 – Scenario 4||M2-S4||Transportation, urban density-attractiveness and topography|
|Mode 3 – Scenario 1||M3-S1||Transportation, urban density-attractiveness and topography|
Once calibrated, the FCUGM was used to simulate the Urban Growth Boundary (UGB) in the city of Riyadh for the following three periods: UGB I (1987–1997), UGB II (1997–2005) and UGB I+II (1987–2005) using the calibrated weights and parameters derived from the GA. Figures 1 to 3 show the simulations for the three time periods respectively for the three top performing simulations, i.e. M1-S4, M2-S4 and M3-S1. The new urban developments that are simulated by the model are shown in red while blue cells indicate those areas that have already been developed. For UGB I (1987-1997), simulation M1-S4 shows more compact urban patterns compared with the other two simulations (M2-S4 and M3-S1), where the latter show more urban development across the peripheral areas, in particular for M3-S1. This might be attributed to the high weight assigned to the urban density variable for M1-S4 and to the form of the distance decay effect captured through the membership functions. However, the morphology of the simulated urban spatial structure that is located to the north and north east shows quite some dispersed and scattered development. Generally, development sites are more linked in order to provide necessary infrastructure and service facilities. However, dispersed development is one of the characteristics of Riyadh’s urban pattern. Typically, urban sprawl is produced by the three simulations regardless of the overall macroscopic pattern. This sprawl might be attributed to a lack of implementation of a policy to limit urban growth, which the government introduced to prevent chaotic development. In addition, this sprawl mimics the non-continuous or leap-frog pattern of urban growth characteristic of this period.
Figures 1 to 3 also show that the direction of growth is generally radial, where urban growth takes place around most of the already developed lands. In particular, most of the growth is to the south west and to the east of the city, while only moderate growth is simulated in the top south eastern part. Growth also rarely occurs to the west of the city. The pattern of growth might be a result of the government’s free grant program. Most of the lands in these two areas were granted by the government to households with low incomes. Another reason may be the lower price of this land compared with the high price of land located to the north of the city. Moreover, moderate growth in the south east of the city could be due to the concentration of heavy industry in this part of the city and to the low urban environmental quality due to proximity to industrial zones and the oil refinery. It can also be seen that there is almost no urban growth simulated to the west of the city, where areas are either steep or located at higher altitudes, indicating that topographical constraint factors have confined growth in such areas. Topographical characteristics have also constrained growth in the south western part of the city, where the steep areas located between the two big urban clusters are simulated as non-urban.
In UGB II (1997-2005), the simulated urban pattern contrasts with that shown in UGB I (1987-1997) where the pattern showed compact development around those areas already developed, and dispersed in the outskirts of the city and peripheral areas. During this second period (UGB II), the simulated pattern followed an in-filling strategy, where most of the development took place within already developed lands and no development occurred beyond the boundary of the developed areas. This can be seen where small simulated clusters (shown in red) are located within the existing urban areas (shown in blue). This is also an expected finding, since during this historical period, the planning authority in Riyadh strictly applied a policy to limit urban growth to avoid further urban sprawl that characterised the period UGB I. As a result of this policy, most of the development occurred on vacant land with the greatest development possibility occurring within existing developed areas. This particular pattern was simulated by all three model instances.
In contrast to the two individual periods (UGB I and UGB II), the simulated urban growth over the combined period UGB I +II shows a more consistent pattern in terms of trend and direction of growth. This is not surprising since the simulation is for a period of 18 years, where more macroscopic urban growth patterns can be identified. The three model simulations produced a broadly similar direction of urban growth where the highest growth took place to the east of the city followed by a moderate growth to the south west and south east and a low growth to the north and south for the reasons noted above. However, there is a notable variation between the three scenarios in terms of urban morphological pattern. M1-S4 produced highly compact urban patterns while M2-S4 and M3-S1 both generated more dispersed patterns. The patterns produced by M3-S1 contained less noise (i.e. unrealistic scattered urban lands) compared to M2-S4, which can be clearly viewed in the north eastern part of the city. However, the non-uniform dispersal of lands, as shown in these simulations, is one of the characteristics of Riyadh’s historical pattern of urban growth.
Overall the model outputs verify that the model is replicating the main processes and drivers as would be expected given knowledge of policies and city structure in the past. In the following sections, more formal methods of model validation are considered.
3. Methods of CA Validation
Seven different methods are described in this chapter; these approaches have all been used to validate the FCUGM model for the city of Riyadh. These include: visual validation; measures of accuracy; urban cell correspondence; the Lee-Sallee index; a spatial pattern measure; a spatial district measure; and multi-resolution validation. The first method, or visual validation, compares the observed results and simulated images by overlaying one image on top of the other and comparing the patterns qualitatively. Such an approach has been used in a number of studies to compare the overall spatial distribution and urban patterns of observed and simulated images, see e.g. [25, 36-39]. Visual comparison by itself may be prone to bias as it is based on the judgment of the researcher or planner. For this reason, more objective methods are required such as those described below. However, visual examination is still an essential part of the validation process since the human brain is particularly good at recognising spatial patterns (and highlighting missing ones), which a more automated or global method would not adequately capture .
One of the most common methods for assessing the performance of urban CA models quantitatively is through the calculation of an error or confusion matrix. This approach has been widely used by several authors to compare simulated results against the actual ones for urban CA models [21, 25, 38-39, 41]. The error matrix is a square array, where the rows and columns represent the number of categories whose classification accuracies are being assessed. Typically, the columns represent the observed data and the rows indicate the simulated data. Table 2 shows the error matrix for evaluating the FCUGM where the cells that are categorized in agreement with their observed data are located along the major diagonal of the matrix from the upper left to the lower right. These include urban areas that were simulated and are also observed, i.e. the true urban areas (TU) and areas that are not urban in both the observed and simulated data (true not urban or TNU). The cells off the diagonal represent errors that are underestimated (FNU or false non-urban) or overestimated (FU or false urban) in the simulated image when compared to the observed image.
From this error matrix, the accuracy can be calculated, which assesses the overall performance of the model by calculating the proportion of the total number of simulated cells that match the corresponding ones in the observed image using Equation 7. In addition the percentages of agreement and disagreement can be calculated as expressed in Equations 8 and 9:
However, if the study area includes a large number of non-urban cells and a small number of urban cells, the accuracy measure might overstate the model performance due to the high number of non-urban simulated cells that match the non-urban observed ones (i.e. true non-urban (TNU) in Table 2). Such a situation renders it difficult to differentiate between the true performances of different simulations as they might yield similarly high values of accuracy. A validation measure which overcomes this problem is the urban cell correspondence (UCC), since it considers only the True Urban and False Urban cells from the error matrix, as outlined in Equation 10:
Another problem with the error matrix is that it is not able to assess and estimate the form and shape of patterns because it is based on independent comparisons between pairs of cells. Once such measure that does take shape into account and which has been used frequently for assessing the urban shape produced by CA models is the Lee-Sallee Index (LSI) [18, 38-39, 42-43]. The LSI is calculated as the ratio of the intersection between the observed and simulated urban areas against the union of these areas in the two images as follows:
where is a simulated urban cell and is an observed urban cell.
Another validation measure that considers shape is the spatial pattern measure (SPM). Most cell-by-cell based analyses like those described above ignore the underlying presence of neighbourhoods. In the case of the SPM, a cell is regarded as erroneous if the category in the observed map differs from the category in the simulated map, irrespective of whether the category is found in the neighbouring cell or nowhere near the cell. In this sense, the SPM evaluates the performance based on the agreement within a neighbourhood. If a simulated cell and its corresponding observed urban cell have the same number of adjacent urban neighbours within a predefined neighbourhood, then the cell in question gets a value of 1, indicating that this simulated cell and its neighbours have the same simulated spatial pattern as the observed one. Finally, the total number of correct cells is summed and compared against the results generated by the cell-by-cell analysis. In practice, a pre-designed kernel matrix is moved across the whole study area which simultaneously compares the number of neighbours for each cell both in the simulated and observed images. When the number of neighbours (cells) is the same for this particular cell, a value of 1 is assigned to the output image. This can be expressed mathematically as shown below:
where Ω Sij is the number of simulated urban cells ij within a neighbourhood Ω; and Ω Oij is the number of observed urban cells ij within a neighbourhood Ω. To calculate this measure, a special kernel matrix is designed as a neighbourhood measure to mimic the common urban block shape in Riyadh. The general urban pattern can be characterised as a grid-iron pattern. The most common shape and size of urban blocks in the contemporary and future districts of Riyadh are rectangular shapes of 180m length and 60m width. In the FCUGM (with a cell size of 20m), this is equivalent to 9 cells in length and 3 cells in width. Thus, a neighbourhood with a rectangular shape of 180m in length and 60m in width is used to validate the performance of the model in terms of spatial pattern. The SPM compares the number of developed land cells within this neighbourhood shape and size in both the simulated and observed images.
A measure that captures the spatial district structure (SDS) is also used to validate the structural similarity between the simulated and observed urban growth in terms of urban neighbourhood (Figure 4a). It would also be possible to assess this in terms of urban sub- neighbourhood (Figure 4b) and urban block (Figure 4c), where the boundaries of these zones are shown for the city of Riyadh in Figure 4. Figure 5 shows what these structures look like when zooming into a section of the city. In this chapter, only the spatial structure of the urban neighbourhood is examined.
The final validation method considers the effect of spatial scale or resolution on the model results. The effects of scale have been considered in previous studies of urban growth modelling by [40, 44-45]. For example, in , a multiple-resolution comparison was conducted between the reference and modelled images by demonstrating a pixel aggregation procedure by which four adjacent pixels were averaged at increasingly coarser levels of resolution. To investigate the influence of spatial resolution on the outputs from the FCUGM model, a similar multiple-resolution validation experiment was conducted to that of . The model output from simulation M3-S1 over the period UGB I+II and the observed image for the corresponding period were aggregated from higher to lower levels of spatial resolution whereby four neighbouring pixels were averaged at each coarser resolution. Thus, cells at the next level up had twice the width and height of the previous cell size. The initial cell size was 20 m and the experiments were conducted for 40, 80, 160, 320 and 640 m pixel sizes.
This section provides the results from the application of the seven validation methods as described in section 3.
4.1. Visual Validation of Urban Growth Patterns
The simulated images were overlaid on the observed patterns of development for each of the three time periods and for the three model simulations. Figure 6 shows the overlaid outputs for the period UGB I (1987-1997) for M1-S4, M2-S4 and M3-S1 while Figures 7 and 8 show the same comparison but for UGB II (1997-2005) and UGB I+II (1987-2005) respectively. In the comparison of the images, four main categories were mapped:
non-urban match (non-urban in observation and simulation);
urban match (urban in observation and simulation);
underestimated (urban in observation but non-urban in simulation); and
overestimated (non-urban in the observation but urban in the simulation).
The first two classes indicate that the simulation is correct while the latter two are incorrect. Two other classes have been added to facilitate the comparison:
starting urban (i.e. already developed lands before the year of the simulation); and
For the period UGB I (1987 – 1997), the urban development for the three scenarios in most areas of the city such as north, north east or south west is relatively well estimated (as shown in red). However, areas located at the immediate edges of boundaries of urbanised areas are overestimated (as shown in yellow). This is not surprising because those cells are adjacent to urban land and nearby to attractions, which are more likely to be urban than non-urban. Although the model was able to simulate the pattern or distribution of the developed land of the city reasonably well, it is clear that the FCUGM was not able to reproduce all of the actual urban development that took place, for example, at the extreme south eastern edge of the city (right-bottom corner of Figure 6, coloured in black), which resulted in an underestimation of these lands. Additionally, some small clusters at the extreme edge of the mid-east, west and south west of the city are also underestimated. This underestimation could be attributed to the fact that these areas are widely scattered from one another and from the boundaries of the other urbanised areas, and they are located at some distance from most attractions (e.g. the town centre, developed lands and other services), which in turn were assigned a low possibility of being developed. Thus these areas would have been simulated as non-urban. Another possible explanation is misclassification of the satellite images during the image processing procedure. However, this underestimation is reasonably small, indicating that the model was able to capture the majority of chaotic and fragmented development that occurred during this period.
In contrast to UGB I (1987 – 1997), during the period UGB II (1997 – 2005) (as shown in Figure 7), the correctly estimated urban areas are hard to distinguish and detect, because most of the developments are in small urbanised clusters located within the boundaries of already developed areas. Moreover, this particular period was characterised by significant levels of ‘leapfrog’ development, which might explain why most of the areas in the maps are coloured blue (starting urban) and the urban match that is coloured in red is marginal and scarcely to be seen. However, M1-S4 and M3-S1 seem to have estimated the urban development reasonably well. It is very hard to detect any urban matching in the M2-S4 simulation, which might suggest that the topographical constraints factor can be considered as a significant influence during this period. This is corroborated by the fact that each of the simulations that included this factor (such as M1-S4 and M3-S1) produced better results than M2-S4.
With respect to UGB I+II as shown in Figure 8, the main result of this analysis is that there is a good visual similarity between the maps, and the simulation results resemble the real city. It can be noted that urban development is largely estimated by the three simulations where M3-S1 has estimated most of the urban development followed by M2-S4 and M1-S4. However, some clusters of land cells are underestimated, mainly in the peripheral areas, showing a different shape in comparison with the actual city. This particular area (coloured in black) is under-estimated highly, moderately and slightly by M1-S4, M2-S4 and M3-S1 respectively. The locations of these cells are very difficult to model since they are located in a highly non-linear and chaotic pattern (e.g. they are far from already developed lands, distant from attractions and services, etc.). Furthermore, it can be noted that M3-S1 was capable of reproducing such complex features to a large extent. This can be attributed to that fact that in this particular simulation, the three urban growth driving forces (TSF, UAAF and TCF) are embedded in each single fuzzy rule, while in M2-S4 and M1-S4 only two and one of these factors are embedded, respectively. This explains why M3-S1 performed well over all periods. It can also be noted that few cells are overestimated compared with the other two periods (i.e. UGB I, UGB II).
Thus overall, the visual analysis of the simulated images shows that they compare well with the patterns that actually occurred in Riyadh during these periods for most of the three simulations, which is a positive reflection of the model’s ability to simulate urban growth in the past.
4.2. Accuracy Assessment and Spatial Statistical Measures
An accuracy assessment is another commonly used validation method where the first step is to calculate the error matrix as shown in Table 3 for the three simulations over the three time periods. It can be seen from Table 3-A that the observed urban development during the period UGB I (1987-1997) was about 261,000 cells. The FCUGM simulated around 265,000, 303,000 and 269,000 urban cells in M1-S4, M2-S4 and M3-S4, respectively. Amongst those simulated cells, about 135,000, 152,000 and 142,000 cells were correctly simulated and matched the observed image. However, 129,000, 151,000 and 128,000 were overestimated, and approximately 125,000, 109,000 and 119,000 were underestimated. For the period UGB II (1997-2005), the results were less good as shown in Table 3-B. The simulated urban cells that were generated by the simulations M1-S4, M2-S4 and M3-S4 were only 82,000, 10,000 and 108,000 compared with 237,000 observed ones. In contrast, the simulated results for both periods together UGB I+II (1987-2005) (Table 3-C) showed an improvement and resulted in a higher correspondence of urban cells compared with the two preceding periods (UGB I and UGB II). The urban cells that were correctly matched reached 222,000, 343,000 and 355,000 compared with 464,000 urban observed cells.
From the error matrix, the accuracy measures and the UCC were calculated as shown in Table 4. The LSI is also provided. The results show that the overall accuracy of all simulations is quite high, ranging between 0.890 for simulation M2-S4 UGB II and 0.937 for scenario M3-S1 UGB I+II. However, these high values of accuracy are mainly achieved through the high matching of non-urban cells, of which there are a very large number in this test area (i.e. it ranges between 3,250,000 and 3,450,000). This implies the need to use a measure that allows for better discrimination between the different simulations, i.e. the UCC, which considers only the matching of the urban cells. Note that the accuracy drops across all simulations, resulting in 0.053 (lowest) and 0.743 (highest) for M2-S4 during UGB II and M3-S1 during UGB I+II, respectively. The UCC measure reveals that the FCUGM simulated the urban growth more accurately over the period UGB I+II (ranging between 0.635 – 0.743) followed by UGB I (0.500 – 0.525), while over the period UGB II, the model produced the poorest results (0.053 – 0.376).
The M3-S1 simulation over all periods yielded the most accurate results compared to the other two simulations, achieving UCC accuracies of 0.743, 0.525 and 0.376 for the periods UGB I+II, UGB I and UGB II, respectively. In contrast, M2-S4 produced the poorest performance across the two periods UGB I and UGB II with a UCC accuracy of 0.500 and 0.053, respectively, while over the period UGB I+II, this simulation performed better than M1-S4 but worse than M3-S1. The M1-S4 simulation produced moderately accurate results with UCC values of 0.635, 0.511 and 0.373 for the three periods UGB I+II, UGB I and UGB II, respectively.
With respect to the agreement between the shape of the simulated and observed images in the form of the LSI, Clark and Gaydos (1998) reported that the practical accuracy of the LSI is only around 0.3 while Cheng and Masser (2004) reported values of 0.383 for their model simulations. However, LSI values of greater than 0.35 were achieved in six out of nine simulations from the FCUGM, indicating a better performance than other CA urban growth models. It can also be noted that the LSI and UCC are highly related to one another where a correlation coefficient of 0.969 was obtained between the two measures for all simulations. Thus, simulations with high UCC are more likely to achieve high LSI, indicating a consistency in performance and stability between the spatial shape measure and the cell-by-cell accuracy measure.
The accuracy of the LSI produced by the FCUGM was relatively good with the majority above 0.35. During the period UGB I+II, the model was reasonably good at capturing the shape of the simulated urban areas with values ranging between 0.375 and 0.602. In contrast, the poorest LSI was produced for the period UGB II with values falling to between 0.024 and 0.259. The LSI during the period UGB I was acceptable, ranging between 0.347 and 0.364. The simulation M3-S1 during the period UGB I+II generated the highest shape matching, whilst simulation M2-S4 over the period UGB II showed the poorest performance.
|(a): UGB I||Observed|
|(b): UGB II||Observed|
|(c): UGB I+II||Observed|
|M1-S4 UGB I||93.1||6.9||0.931||0.511||0.347|
|M2-S4 UGB I||92.9||7.1||0.929||0.500||0.367|
|M3-S1 UGB I||93.3||6.7||0.933||0.525||0.364|
|M1-S4 UGB II||91.6||8.4||0.916||0.373||0.259|
|M2-S4 UGB II||89.0||11.0||0.890||0.053||0.024|
|M3-S1 UGB II||92.1||7.9||0.921||0.376||0.218|
|M1-S4 UGB I+II||90.0||10.0||0.900||0.635||0.375|
|M2-S4 UGB I+II||93.3||6.7||0.933||0.733||0.582|
|M3-S1 UGB I+II||93.7||6.3||0.937||0.743||0.605|
4.3. Spatial Pattern Measure (SPM)
Tables 5 and 6 show the error matrix and statistical indices for the spatial pattern measure for the three FCUGM simulations over the three time periods. These measures include the percentage of agreement, disagreement, accuracy, UCC and LSI when considering the underlying neighbourhood (Equation 12). It can be seen from Tables 5 and 6 that the performance of the FCUGM taking the spatial pattern of neighbourhoods into account shows relatively positive results. For example, the percentage of agreement across all simulations lies between 88% and 94%. The UCC indicates a very high degree of matching between the simulated and observed urban lands with the UCC accuracy as high as 0.765 generated by simulation M3-S1 over the period UGB I+II, and a value of 0.542 generated by simulation M1-S4 and M2-S4 over the period UGB II. The least satisfactory performance was generated by M2-S4 during the period UGB II. In terms of the shape index, this measure shows fairly consistent results similar to the accuracy and the UCC. Those results with high values of UCC and accuracy also generated high shape agreements similar to the findings in section 4.2 where the underlying neighbourhood was not taken into account.
The performance of the different simulations based on both cell-by-cell and spatial pattern methods of validation are provided in Figure 9. If the validation measures from applying the SPM method produced better results than the cell-by-cell ones, this would be understandable since it is extremely difficult to simulate and predict the precise location of urban lands due to the complexity of the urban system. However, the results in Figure 9 indicate a high degree of consistency and stability in the model. From this it can be inferred that the FCUGM has simulated urban growth based on both local and neighbourhood configurations to a large extent.
|(a) UGB I||Observed|
|(b) UGB II||Observed|
|(c) UGB I+II||Observed|
|M1-S4 UGB I||93.7||6.3||0.937||0.542||0.364|
|M2-S4 UGB I||93.2||6.8||0.932||0.542||0.343|
|M3-S1 UGB I||93.6||6.4||0.936||0.520||0.347|
|M1-S4 UGB II||92.1||7.9||0.921||0.343||0.218|
|M2-S4 UGB II||88.7||11.3||0.887||0.004||0.002|
|M3-S1 UGB II||91.6||8.4||0.916||0.454||0.259|
|M1-S4 UGB I+II||90.2||9.8||0.902||0.478||0.375|
|M2-S4 UGB I+II||93.8||6.2||0.938||0.738||0.582|
|M3-S1 UGB I+II||94.1||5.9||0.941||0.765||0.605|
4.4. Spatial District Structural Measure
Figure 10 presents the results of the spatial structure indicator, which plots the number of developed areas in each district against the observed ones for the city of Riyadh over the three periods UGB I, II and I+II producing a profile of development by district ID. During the periods UGB I+II and UGB I, the simulation results generated similar patterns to the observed while, in contrast, the simulation results over the period UGB II underestimated large areas for districts with IDs between 47-50, 75-90 and 110-150.
With respect to the simulations, the model results generated from simulations M2-S4 and M3-S1 show good matching with the observed data, while M1-S4 moderately underestimated some actual developed areas. During the period UGB II, simulation M3-S1 performed better than the other two simulations (M1-S4 and M3-S1), while over the period UGB I, the three simulations produced similar moderate levels of urban matching.
4.5. Spatial Multi-Resolution Validation
Table 7 shows the error matrix for the FCUGM for simulation M3-S1 over the period UGB I+II (1987 – 2005) at the original and five increasingly coarser spatial resolutions while Table 8 presents the statistical indicators derived from the error matrix.
|Cell Size (m)||Agreement (%)||Disagreement (%)||Accuracy||UCC||LSI|
The results show that there are improvements to all the measures reported in Table 8 as the cell size increases from 20 m to a higher resolution. However, some of these improvements are very small and they remain relatively stable as the resolution continues to increase. For example, the accuracy at a 20m resolution is 93.7%, while accuracies at higher resolutions are all around 94%. The only exception is at 80 m where the performance according to all measures is the highest. In terms of urban cell matching, the lowest performance (0.841) was found at 40 and 160 m while moderate UCC accuracies (0.856 and 0.847) were found at spatial resolutions of 320 and 640 m respectively. Thus, the UCC does not appear to improve very much with a coarser spatial resolution and is likewise quite stable at higher resolutions. With respect to matching the shape between the output of the model and the actual urban image, the LSI also indicates similar values at the higher resolutions with the exceptional performance at a resolution of 80 m. Overall the results suggest that the simulated urban images produced by the FCUGM are not that sensitive to spatial resolution, which indicates that a significant feature of the model is its stability and consistency of accuracy over various cell sizes.
5. Discussion and Conclusions
Simulating the main processes and drivers of urban growth is a challenging area; researchers are increasingly turning to individual-based models to handle the complexity of these systems. To have any confidence in the outputs of these models, rigorous calibration and validation tests need to be applied. Within this chapter, a series of different measures were used to validate the FCUGM, a complex CA model, for the city of Riyadh. While no one validation method was found to ‘outperform’ the others, there was great benefit in using a combination of several approaches. Three different simulations of the FCUGM applied to three different time periods of urban growth were considered. It is clear from the results that the characteristics and patterns of urban development over a particular time period have a large influence on the performance of the model and the resulting accuracy of a given simulation. For example, over UGB II, urban development has mainly followed a pattern of infilling of urban growth, i.e. the non-urban areas surrounded by urban areas were converted to urban, while very limited development took place on the margins or fringe areas of the city. This type of development exhibits a highly non-linear pattern, where the new potential developed land occurs in very small clusters that are surrounded by very large urban clusters. Consequently, the simulation results over this period were the least satisfactory when compared with the other two time periods. It is worth noting that this pattern was generated as a result of applying the urban growth limit regulations (as advocated by the planning local authority of Riyadh) to prevent urban sprawl. The urban growth pattern over the periods UGB I and UGB I+II can be characterised by a pattern of edge-expansion, where the newly developed urban areas spread out from the fringes or margins of existing urban patches. This feature was modeled in a satisfactory manner during these two periods of growth.
Similarly, the characteristics of the simulation are another factor that can have a significant impact on the results, which was clearly supported by consistency across the different validation measures when examining the three simulations, i.e. simulation M3-S1 produced the best spatial simulation over all of the periods followed by M1-S4 and M2-S4. It is worth noting that the three urban growth factors, i.e. transportation, urban density and attractiveness, and topographical constraints, were part of all three simulations M1-S4, M2-S4 and M3-S1. However, the difference between these model instances has to do with the form of the fuzzy rules and how many factors are combined in each rule. M1-S4 embeds only one factor, M2-S4 embeds two factors and M3-S1 combines all three factors in each fuzzy rule. Embedding all factors into the fuzzy rules and combining these via the AND operator appears to have produced the best performing model. However, M1-S4, with only one factor per fuzzy rule, generally outperformed M2-S4 with two factors in each rule but containing all three factors in the model with more rules needed to capture all the possible pairs of factors. Perhaps restricting the model to rules with only two factors produced a model that was actually more complex than the simple M1-S4 and even the M3-S1 simulation, but less able to capture urban growth as adequately.
Overall there was consistency between the measures regarding which model instance performed better and for which growth periods. The visual inspection provided an overall qualitative assessment that would not have been possible using any of the quantitative measures and is therefore always recommended as a method of model validation. The accuracy measures are very sensitive to the number of non-urban cells and should mostly likely not be used or reported in conjunction with the UCC, which took only urban cells into account. This measure provides a much better assessment of model performance. The measures that took shape or underlying neighbourhood into account are also valuable. In this case, they provided a consistent message regarding model performance but they could help to identify models that are good global predictors but are not spatially or locally very good. Finally the analysis at multiple resolutions provides a good indication of model stability across spatial scales and should be implemented as a minimum measure of validation as advocated in .
While the validation techniques used in this work provided a comprehensive assessment of the model outputs, there are other techniques available, e.g. fractal dimensional analysis [34, 46]. However, this approach has limitations, e.g. two maps that seem different may have identical fractal dimensions. Thus, this measure tells us very little about how similar the two maps may be in terms of local structures. Although the approach reflects how much space is filled correctly across a range of scales, it does not seem to be valid when dealing with non-urban situations . However, other approaches involving comparison with null models require further investigation . What remains clear from this study and the current state of validation approaches in the CA urban modelling literature is that there is no one best method or set of approaches available for validating CA urban growth models. Many different methods are available and the best approach appears to be validation using multiple measures. Ultimately, these measures must be linked to confidence in the model performance and the ability to simulate future growth especially when they move from an academic and experimental environment to real world applications by planners.