Measuring Urban Development and City Performance

Abstract

Cities represent the driving force of development in economic, social, and cultural life, reflecting also the spatial organization of human society. Taking into account the fact that cities are becoming generators of economic development and a source of growth for the national economy, there is an increasing urge to identify the stages of development and to establish a system for the ranking and positioning of cities and regions in this process (the level of categorization). This will allow the preparation of appropriate strategic and development guidelines for cities and urban regions to take place. In order to be able to compare the level of their efficiency in fostering development, there is an intensifying need to develop indicators that measure the performance of cities, are representative and comparable between countries, and allow verification to others. At present, there are many different urban indicators and institutions that compile and analyze them. Performance measurement systems, developed for internal use in some cities, already show a degree of measurement feasibility. The fundamental problem is that this variety of indicators lacks consistency and comparability (over time and between compared cities). Therefore, their use cannot be approved in a wider context (benchmark) of comparative situations. Upon the case of medium-sized cities, we consequently have to question the applicability of the methodology and indicators, used mostly in cases of large, global cities by internationally recognized institutions. With the established set of indicators and assistance of computer programs for multiparameter decision-making processes (analytic hierarchical process [AHP]), this paper also seeks to investigate comparisons between performance of selected European cities (on a qualitative basis).

1. Introduction

Existing methodologies in comparison to performance and quality of urban city structure affect more or less a wider field of urban and regional disparities, while specific approaches cover only limited areas. [34] focuses exclusively on infrastructure impacts, Callois and Aubert (2007) empirically analyze the impact of social capital on regional development. The advantage of quoted approaches represents limited number of variables involved in analysis. In the area of measuring the quality of life, [53] provide an overview of indicators of sustainable development, as well as [54] and [55], but the interpretation of the indicators of quality of life is missing. In the field of competitiveness, [62] presents the synopsis of indicators measuring urban competitiveness on a European scale, while [39] indicate the multicast nature of sustainable development that consequently leads to the unclear definition of the measuring indicator. Missing thematic indicators can also be found in the context of measuring regional disparities, both at the level of the broader European countries [58] and in the narrow sense of the regions [36]. Comparing cities by the use of indicators that represent diverse aspects of urban life is only possible with the meaningful structured set system; easily adding a large number of indicators to achieve a single index may result in criticism of uncertainty and noticeable limitation of its interpretation. Similar effects can also be achieved by using a larger set of nonaggregated indicators; therefore, identification of appropriate, small number of relevant indicators is crucial. In the process of establishing the set of indicators, the inclusion of indicators with higher impact on the general differences between selected cities in different countries is necessary; an additional assumption incorporates the integration of indicators from the field of environmental, human, and social capital as well as the demographic point of view.

2. Theoretical background and applied practices

When searching for the most relevant performance indicators of city development, we proceed from the fact that more than two-thirds of the population live in urban areas. The urban environment provides a fertile ground for the development of science and technology, culture, and innovation. On the other hand, in cities, there is also more emphasis on the problems such as unemployment, discrimination, segregation of society, and poverty. The cities are also faced by challenges, associated with mitigating the effects of climate change, job creation, prosperity, and quality of life. Therefore, the development of cities has a decisive impact on the future of the economic, social, and territorial development. As highlighted by the recent European Commission survey entitled “Cities of the Future—Challenges, Ideas and Expectations” (EC, 2011), a phase of urban sprawl in recent decades has shown serious problems associated with the deterioration of urban areas due to lack of infrastructure construction and basic services. Promoting urban renewal as the driving force of prosperity and creating opportunities together with strengthening the link between cities and development, and between urban centers and surrounding areas, are the main challenges to provide stable economic growth.

Establishing a system of indicators for measuring performance development of selected cities included the consideration of contemporary city’s complex aspects with reference to (a) the 72 attributes of a smart city,

http://www.smart-cities.eu.

(b) the performance of the city, and (c) urban status or urban sustainability.

The case study example of Glasgow classifies city performance indicators as follows: (1) population (mortality, fertility, population projections), (2) economic participation (employment, unemployment, vacancies), (3) poverty (access to bank accounts, children poverty, financial hardship, low-income households), (4) health (life expectancy, inability to work), (5) social capital (social inclusion, social networks, trust and reciprocity, civic participation), (6), environment (green environment, open space, air quality, recycling), (7) transport (transport volume, journeys to work and school, traffic accidents, cycling), (8) education (children education, the highest qualification obtained, the qualification of the working population, training of young people), (9) safety of local communities (overall level of crime, antisocial behavior, violence, unintentional injuries), (10) lifestyle, (11) cultural vitality (involvement in sport and cultural events), and (12) mind-set (religion, politics, involvement in the community, trust, national identity).

Indicators of sustainable development reflect the complex and dynamic structure of the urban environment. With the adoption of Agenda 21 (1992), this type of indicator was developed by a number of institutions, including the World Bank (UN—Urban Indicators Programme), followed by indicators of the World Health Organization (WHO), as the analytical tools for studying population health and quality of life in urban environment. A wider set of indicators also includes project SUD-LAB EC (European Commission) with an expanded database of European cities, where indicators are divided into the following categories: (a) air quality, (b) composed environment, (c) cultural endowments, (d) social disparities, (e) quality of transport, (f) urban management, and (g) waste management. For each of listed categories, a set of indicators is reflecting the level of urban functionality. Indicators of the EU-TISSUE Programme, in use in 15 European countries, relate to the areas of sustainable urban management (descriptive indicators), sustainable urban transport, sustainable urban construction, and sustainable urban design [2].

In accordance with the Charter on European Sustainable Cities and Towns, [60] lists six key areas of sustainable development and urban transformation: (1) active city/town, (2) beautiful town, (3) green city/town, (4) town with a better environment, (5) cooperation for a better city, and (6) town catalogue. The strategy of urban sustainability consequently includes urban sustainability performance indicators such as (1) local involvement (citizen’s participation); (2) employment; (3) city deficit; (4) economic growth; (5) urban mobility; (6) urban metabolism, resources, and consumption; (7) environment and social expenditure; (8) urban safety; (9) public health; (10) social justice; and (11) global change.

Among indicators of central city area development, Niţulescu (2000) includes the following: (1) types of land using (constructions, green areas), (2) green areas surface from the total town center’s surface, (3) percent of residential buildings from the total number of buildings from the center of the town, (4) percent of trade buildings from the total number of buildings from the town center, (5) percent of central functions buildings (administrative, international, unique endowment) from the total number of buildings from the center of the town, (6) built areas of public utility related to then inhabited areas, (7) employment density (number of working places related to the town center surface), (8) rate of employed population for each sector (industry, trade, services), (9) number of crossroads for the surface of the town center, (10) surface of pedestrian circulation for the surface of the town, and (11) surface of pedestrian circulation for the surface of roadway [2].

Among indicators of urban status ranks, Şuler’s (2005) category of population and labor force indicators are as follows: (1) number of inhabitants, (2) population density (per hectare), (3) working places/1000 inhabitants, and (4) proportion of the population employed in the service sector. The category of living and quality of life indicators include the following: (5) number of residential buildings per 1000 inhabitants, (6) houses equipped with plumbing (% of buildings), (7) number of personal cars per 1000 inhabitants, (8) number of houses with bathrooms inside the building, (9) number of hospital beds per 1000 inhabitants, (10) number of doctors per 1000 persons, (11) financial/banking institutions (headquarters, working points), and (12) accessibility to lines of communication (railway station, bus station). The indicators of category society, culture, and leisure include the following: (13) education units (high school, secondary, postsecondary school), (14) secondary school in primary and secondary educational units (%), (15) cultural and sports endowments (theaters, public libraries, gyms, auditorium, stadium), and (16) accommodation places/1000 inhabitants. Indicators of the urban network are specified as follows: (17) modernized streets (%), (18) streets with water pipes (%), (19) waste water treatment, (20) household gas distribution pipes (%), (21) sanitation motor vehicles for 100 km of street, (22) scavengers for 1000 inhabitants, and (23) green area surface m²/inhabitant.

[2] defines an index of local development as an integrated indicator, including the importance (weights) of individual elements as category of infrastructure (4), followed by the economy (3), community (2), and the public administration (1):

$$I_{dl}=\left[\left(I_i\times 4\right)+\left(I_e\times 3\right)+\left(I_{mc}\times 2\right)+\left(I_{ap}\times 1\right)\right]/10,$$E1

where I_dl is the local development index, I_i is the infrastructure index, I_e is the local economy index, I_mc is the local community index, and I_ap is the public administration index.

Category infrastructure includes utilities, transport infrastructure, health infrastructure, natural resources, and natural environment. Economy includes financial services and insurance, labor, and public budget. Public administration includes public administration, services and support to small and medium-sized enterprises, urban planning, communication, and information dissemination. Among the indicators of development, Bӑnicӑ (2010) introduces the community spirit, safety of citizens, tourist attractions, cultural/sports facilities, and cultural/historical heritage.

[63] formed an indicator of the public urban transport quality using available Eurostat database indicators, including the following subindicators: (1) the proportion of journeys to work by public transport, (2) the length of the public transport network, (3) the number of stops of public transport per km², (4) the price of a monthly public transport ticket, (5) the number of stops per 1000 population, (6) the number of stops per 1 km of public transport network, (7) the ratio between the public transport network on fixed infrastructure and flexible connections, and (9) the proportion of land for transport use.

World Bank Urban Development Indicators include the following: (1) proportion of urban population with access to improved health services, (2) proportion of urban population with access to water resources, (3) number of motor vehicles per 1000 population, (4) number of passenger cars per 1000 inhabitants, (5) emissions of PM10 (micrograms per m³), (6) proportion of poverty, (7) fuel prices, (8) fuel consumption per capita, and (9) the percentage of the urban population.

Significant importance among urban sustainability indicators belongs to European Common Indicators (ECI), first established in 1999–2003, by the research institute Ambiente Italia. Within a base with more than 1000 indicators, 10 key indicators, reflecting the development trends of European cities in accordance with the principles of social inclusion, local governance and democracy, local/global integration of the city, local economy, environment, cultural heritage, and quality of the institutional environment was selected: (1) citizens’ satisfaction with the local community, (2) local contribution to global climate change (CO₂ emissions per capita), (3) local mobility and passenger transportation, (4) availability of local public open areas and services, (5) quality of the air (emissions of PM₁₀), (6) children’s journeys to and from school, (7) sustainable management of the local authority and local enterprises, (8) noise pollution, (9) sustainable land use, and (10) products promoting sustainability.

With reference to the cited attributes of a smart city,

Specifically explained in Section 6.

city performance, and urban sustainability, a system of indicators, whose structure is presented in the following text, for measuring performance development of the city was developed. Areas of measurement, enabling the international comparison of cities, covered six areas: (1) demography, labor market, and economy; (2) quality of life; (3) society, culture, and leisure activities; (4) research and development; (5) accessibility of urban networks and international connectivity; and (6) management of sustainable resources. Within listed areas of measurement system, categories enabling grouping of individual indicators and appropriate weighting of the their relative importance were set. Relevant indicators resulted from knowledge of current topics and problems of urban development as well as the renewal priorities of local development model. The indicator system, based on current challenges of a multicultural society, was reaching the areas within the sphere of local communities, trust in institutions, prosperity, quality of life, environmental change, social exclusion, unemployment, poverty, polarization, and demographic changes. From this perspective, it can be regarded as a dynamic system, where 53 selected indicators serve as a basis, always possible to upgrade and adapt to the situation and degree of urban development.

3. Selection of indicators

The selection of appropriate indicators included research and exploration, evaluation, and selection of relevant databases, through which adequate indicators of measurement as a basis for determining the level of the city performance development and consequently a useful tool for ranking of comparable medium-sized European cities was obtained. Indicators in the study were selected on the basis of following assumptions: (1) objectivity (clear, easy to understand, precise, and unambiguous); (2) relevance, measurability, and reproducibility (quantitative, systematic observable); (3) validity (with the possibility of verification and data quality control); (4) statistical representativeness (at the city level); (5) comparability/standardization—longitudinal (over time) and transverse (between cities); (6) flexibility (with the possibility of continuous improvement); (7) efficiency/performance (as decision making and local management planning tool); (8) accessibility (available databases, use of existing data); (9) interaction (social, environmental, economic); and (10) consistency and temporal stability. Last but not least, the selection of appropriate indicators was also related to the concept of data homogeneity. In searching for the relevant data, many of the existing semantic information about the state of the city and urban region were expected to be available; therefore, the data credibility was highlighted.

4. Selection of cities

In Europe, more than 600 cities and urban regions are classified as medium-sized with a population between 100,000 and 500,000 inhabitants (selection criteria). In the case of a single manual data collection, the data processing for such number of cities are practically impossible. Therefore, the reselection of urban sample in terms of a data source (all selected cities should be covered by a specific source, e.g., Urban Audit) was necessary to eliminate the risk of the diverse resources’ use, related to the area and the region of the city, induced by the dimension of the selected city sample. In case of insufficient data, the use of different spatial levels (Eurostat database is corresponding to NUTS2, representing regions and provinces, while the Eurobarometer data correspond to NUTS0/national level) was imminent. Quoted databases focus on the European cities’ profiles, which further narrowed the selection frame. The final selection of cities was defined (Table 2) on the basis of the following: location (criterion 1: all selected cities are located in Europe), database (criterion 2: inclusion in the database Urban Audit), definition in terms of a smart city (criterion 3: placed in the “Smart Cities” base), comparability in terms of urban size (criterion 4: comparable population size: medium-sized cities with the range of 100,000 to 200,000 inhabitants), and regional significance (criterion 5: capital region or important regional center). With reference to fulfilled criteria, research cities represented Maribor (Slovenia), Pleven (Bulgaria), Linz (Austria), Erfurt (Germany), Trieste (Italy), and Brugge (Belgium).

Table 1.

City performance measurement indicators

City	NUTS0	NUTS1	NUTS2	NUTS3
Maribor	Slovenia	SI0: Slovenia	SI02: West Slovene region	SI012 Podravje region
Pleven	Bulgaria	BG3: North I SE Bulgaria	BG31: Northwest	BG314 Pleven
Linz	Austria	AT3: Westösterreich	AT31: Oberösterreich	AT312 Linz-Wels
Erfurt	Germany	DEG: Thüringen	DEG0: Thüringen	DEG01 Erfurt
Trieste	Italy	ITH: North-East	ITH4: Friuli-Venezia Giulia	ITH44 Trieste
Brugge	Belgium	BE2: Vlaams Gewest	BE25: Prov. West-Vlaandeeren	BE251 Arr.Brugge

Table 2.

Selected cities and the corresponding regions (NUTS classification)

Source: http://ec.europa.eu/eurostat/ramon/nomenclatures

5. Data structure and categorization

5.1. Database

The database of the research was largely represented by an Urban Audit indicator set for core cities, available as a part of a broader Eurostat collection. The base of data analysis (accessed February 2012) covered 30 countries: the EU-27, along with Turkey, Switzerland, Norway, and 372 urban units (city and the wider urban area) and specific metropolitan areas. The temporal span, used in the research, included periods 2010–2012, 2007–2009, and 2003–2006, exceptionally 1999–2002, but only to illustrate the missing measurement in the time series. In addition to Urban Audit, research also implied regional databases of EUROSTAT (appsso.eurostat.ec.europa.eu), and the index of quality of life in each country was defined by using ranking of International Living survey. Taking into account the selection of cities from different countries in terms of validity and international comparability, and to avoid inaccuracies due to diverse methodological approaches, the research additionally incorporated data from the Statistical Office of the Republic of Slovenia (www.stat.si), Austria (Statistik Austria; www.statistik.at), Italy (SISTAN Sistema statistico nazionale; www.sistan.it and www.istat.it), Germany (www.destatis.de), Belgium (statbel.fgov.be), and Bulgaria (www.nsi.bg). Urban Audit database, used in 75.47% of cases, was followed by Eurostat database with 22.64% and other data sources (1.89%); overall data coverage rate reached 87%. Limitations of the research referred to the missing data; the inclusion of the secondary databases that would otherwise fill out the data gap could be due to the chosen methodology of data collection and evaluation, which will result in the reduced data comparability of data and furthermore between cities within individual indicators. Dropping variables was potentially admissible in cases of minor influence on the dependent variable (y), which, in most cases, proved to be the best choice since it pointed out the problems associated with data collection (listwise/casewise deletion of missing data of the valuation criterion). Options of replacing missing data represented single imputation as the arithmetic mean (overall mean) or multiple imputation methods (e.g., program Amelia II). When using programs of multicriteria decision making (Expert Choice) in research, only indicators without data gaps were evaluated.

6. Criteria weighting

6.1. Determining the weights of indicators: different approaches

Weighting of indicators emphasized the suitability requirements, with the value of the weight indicating the impact of each criterion on the final goal (objective). Weighting methods are different, are very widely used, and are scalable in many cases applied, where 0 equals the insignificant impact of the indicator, range 1–3 represents a significantly less important indicator, range 4–7 represents a little less important indicator, and range 8–10 represents an equally important indicator in terms of the relative importance with the most important criteria [1, 30]. In the case of a clearly defined target group, the determination of relevant weightings is also possible by using the questionnaire survey. Stepwise methods label 5–6 as low importance indicators (complementary, supplementary, secondary, incidental, indirect, and no impact), 7–8 as average significant indicators (imperative, mandatory, or required indicator), and 9–10 as high importance indicator (fundamental, essential, decisive, definitive, and guidelines).

The weighting is also possible with the prioritization of functional variables in the form of a matrix (CICAPSO, 2012), consisting of the power zone with a low dependence of variable x (abscissa axis) and a high impact y (ordinate axis); connection zone, linked with a high dependence of x and a high influence of y; isolated zone, with a low dependence of x and a low impact of y; and exit zone, with the high dependence of x and low impact of y. The weightings in the power zone are the most important, influential, and less dependent; those identified in the connection zone are often associated with conflicts, relevant by influence, but at the same time very dependent. In the isolated zone, the weights with low or no dependence and influence on other, mostly useful at the end of the evaluation, can be found. As last in the exit zone, weights of minor importance and high dependence, with the purpose for understanding the power and connection zone, are located.

Table 3.

Matrix of weights.

Conceptualization of the system of indicators in research was based on the relevance of the individual categories, taking into account the relative importance of weights on the objective measurement: performance development of selected cities. Considering that the system of indicators represents a baseline tool, the weighting depends on the purpose of the decision maker in terms of defining the specific goal of measuring and monitoring. Comparability of the indicators was previously reached by using available, credible databases (Section 5.1).

In the case of the desk research data collection, the z-transformation method, which provides standardization and data aggregation, is suggested. In the concept of the “smart city,” establishing a standardized indicator value of each city was followed by determining the weightings in accordance with the coverage degree of indicators (lower weightings indicated that values of indicators were not covering all cities). Indicators were assumed to have equal influence on a particular category (currently 70 cities with 74 indicators represent 87% level of coverage). Indicators of “smart economy” include innovative spirit (3 indicators with a weight of 0.17), entrepreneurship (2 indicators with a weight of 0.17), economic image/trademark (1 indicator with a weight of 0.17), productivity (1 indicator with a weight of 0.17), and flexibility of labor market (2 indicators with a weight of 0.17). “Smart mobility” indicators represent the following: local accessibility (3 indicators with a weight of 0.25), (inter-)national accessibility (1 indicator with a weight of 0.25), availability of ICT infrastructure (2 indicators with a weight of 0.25), and sustainable, innovative, and safe transport systems (3 indicators with a weight of 0.25). Among indicators of “smart environment,” the attractiveness of natural conditions (2 indicators with a weight of 0.25), pollution (3 indicators with a weight of 0.25), environmental protection (2 indicators with a weight of 0.25), and sustainable resource management (3 indicators with weighting of 0.25) are considered. Indicators of the category “smart people” include the following: level of qualification (4 indicators with a weight of 0.14), affinity to lifelong learning (3 indicators with a weight of 0.14), social and ethnic plurality (2 indicators with a weight of 0.14), flexibility (1 indicator with a weight of 0.14), creativity (1 indicator with a weight of 0.14), cosmopolitanism/open-mindedness (3 indicators with a weight of 0.14), and participation in public life (2 indicators with a weight of 0.14). Indicators of the category “smart life” represent the following: cultural facilities (3 indicators with a weight of 0.14), health conditions (4 indicators with a weight of 0.14), individual safety (3 indicators with a weight of 0.14), housing quality (3 indicators with a weight of 0.14), education facilities (3 indicators with a weight of 0.14), touristic attractiveness (2 indicators with a weight of 0.14), and social cohesion (2 indicators with a weight of 0.14). Participation in decision-making processes (4 indicators with a weight of 0.33), public and social services (3 indicators with a weight of 0.33), and transparent governance (2 indicators with a weight of 0.33) form the category of “smart regulation” indicators.

By determining the adequate weighting, the research in this section also considered weighting of indicators, measuring the competitiveness of cities in the context of the knowledge economy, where the greatest importance was given to categories of quality of life (weighting 0.20) and knowledge base (weighting 0.20), followed by the categories of innovation (weighting 0.10) accessibility (weighting 0.10), urban diversity (weighting 0.10), productivity (weighting 0.10), and social connectivity (weighting 0.10). Areas of agglomeration and economic heritage were defined with a weight of 0.05 [62].

With reference to quoted concepts, the largest weighting importance in research was assigned to the categories of quality of life, environment, lifelong learning, development of information, and communication technology and city brand (weighting 0.20), followed by labor market, productivity, entrepreneurship, innovation, and mobility (weighting 0.15). The importance of social cohesion, governance, and urban diversity was defined with a weight of 0.10; a minimum effect on the performance development measurement was attributed to demographic categories (weighting 0.05). Weightings for individual categories of indicators 1–53 are presented in Table 1.

In terms of weighting credibility, the study also considered Mercer’s classification and evaluation indicators (weights) of quality of life (Quality of Living Report) in European cities (Urban Audit database, benchmarking analysis of 246 European cities). The study of 10 dimensions, namely, (1) quality of life, economic environment, (2) political and social environment, (3) sociocultural environment, (4) health and medicine, (5) schools and education, (6) public services and transport, (7) recreation, (8) consumer goods, (9) housing possibilities, (10) natural environment, and 39 quality of life indicators showed a certain degree of area similarity to the selected indicators’ system in the research (demography, labor market, economy, quality of life, society, culture and leisure activities, and R & D). Mercer’s weights in specific areas are defined as follows: political and social environment (weighting 0.283); economic environment (0.048), which includes employment in the services sector (NACE classification J-K); area of health and medicine (0.229), which also includes life expectancy in years; schools and education (ISCED with weight of 0.041); public services and transportation (0.157), including air passengers using nearest airport; recreation (0.109); housing possibilities (0.062); and the natural environment (0.071), including rainfall [33].

7. Multiattribute decision models and system of indicators’ simulation with computer-based programs

7.1. Methods for decision support

After the system of indicators for monitoring performance development of the city had been set, the purpose of the study was to enable quality decision making in a systematic, organized manner. The preparation of scenario and the selection of the chosen strategy involved either verbal or numerical representation of inputs in principle, which required the inclusion of artificial intelligence. Multicriteria models represent a useful tool to support decision making in complex decision situations, where a large number of factors and variants affect the final decision. Supporting software tools in designing a decision model evaluate variants and offer a range of different analyses for detailed decision’s verification and justification [6, 7].

Systematic decision-making processes for supporting smart decisions should be based on combining normative theories and cognitive aspects, forming an integral part of decision making in practice. According to [23], problem solving can be done in several ways: intuitively, routinely—by adopting the past used procedures, or random selection—by systematic rational thinking using relevant information. In the latter, the decision maker measures the values of alternatives by each individual criterion or by multiple criteria simultaneously [11].

The general approach of decision analysis originates from axioms of the game theory, by John von Neumann and Oskar Morgenstern. The main steps represent problem structuring, estimating the likelihood of possible outcomes, determining their utility, evaluating alternatives and selecting strategies. Briefly, the process of multiattribute decision making involves problem identification and its structuring, the model building, and activities of problem solving planning, wherein[5] have foreseen also returning from each following to the previous phase [11].

The major role in decision making according to multiple criteria goes to classification or ranking. Identifying the decision maker’s relative importance of each criterion can be expressed as a priority (the criterion is more important than the other) or weighting, which declares the relative importance of the various criteria [10].

In the research, comparison of the cities’ development performance was carried out using the analytic hierarchical process (AHP) method, developed by Thomas L. Saaty. In accordance with the theory of AHP, multicriteria problems are initially presented in the form of a hierarchical model. Several papers demonstrated the AHP efficiency in different areas [19, 21, 26, 31, 32, 37, 51, 52, 59, 64]. The oldest reference we have found dates from 1972 [41]. After this, a paper in the Journal of Mathematical Psychology [42] precisely described the method [26].

The method’s basis represent pairwise comparisons of the two criteria at the same level in relation to the element on the next (higher) level. In order to help the decision maker to provide the pairwise comparisons, Saaty created a 9-point intensity scale of importance between pair of elements (Table 4). If the estimation a.., is assigned to criterion i in comparison with j, then to criterion j when compared with i, the reciprocal value is assigned [44, 48, 50].

Weighting criteria and priorities to alternatives are not assigned directly by decision makers; they are calculated from the judgments, entered by comparing the importance of criteria and preferences of alternatives in pairs in verbal, graphic, or numerical manner [10].

A top-down approach of AHP method leads from the goal to the alternatives, while the bottom-up approach represents expression of judgments about alternatives before expressing judgments on the criteria [16, 38].

Intensity of importance	Definition/explanation
1	The criteria are equally important; alternatives are equally preferred/equally; contribute to the objective.
3	Experience and judgment slightly favor one activity over another. The criterion is moderately/slightly more important than the comparable criterion.
5	Experience and judgment strongly favor one activity over another. The criterion is strongly more important than the comparable criterion; alternative is strongly more preferred.
7	Very strong or demonstrated importance. Criterion is powerfully more important than the comparable criteria.
9	The criterion is extremely more important than the comparable criterion; alternative is extremely more preferred, highest possible favoring of one criterion over another.

Table 4.

The fundamental scale for pairwise comparisons.

Source: [10].

Evaluating the importance of criteria and preference of alternatives, according to individual criteria, includes a criteria importance estimation by setting the appropriate weights; for AHP, the sum of the weights for each group of criteria is considered equal to 1 (hierarchical manner of determining the weights).

Attributes (criteria at the lowest hierarchical level) are represented as follows [10]:

$$z_1,z_2,\dots ,z_k,$$ while

$$w_1,w_2,\dots ,w_k{w}_k$$ represent the weights.

It is assumed that:

$$w_1+w_2+\dots w_k=1;w_m\ge 0;m=1,2,\dots ,k,$$E2

$$w={\left[w_1{w}_2\mathrm{...}{w}_k\right]}^T$$ and

$$a_{ij}=\frac{w_i}{w_j};i=1,2,\dots ,k;j=1,2,\dots ,k,\text{ meaning:}$$E3

attribute z_i is a_ij- times as important as the attribute z_j.

By calculating the values of alternatives with respect to each attribute is:

v_m (A_i) value of alternative A_i with respect to the attribute z_m and

v_m (A_j) value of alternative A_j with respect to the attribute z_m.

Given n objects, e.g., attributes or alternatives, we suppose that the decision maker is able to compare any two of them. In preference modelling, this assumption is called comparability. For any pairs (i, j); i, j = 1, 2,..., n, the decision maker is requested to tell how many times the i-th object is preferred (or more important) than the j-th one, which result is denoted by a_ij:

$$a_{ij}=\frac{v_m\left(A_i\right)}{v_m\left(A_j\right)};i=1,2,\dots ,n;j=1,2,\dots ,n,$$E4

ratios of values of alternatives, indicating that alternative A_i is with respect to attribute z_m a_ij-times as good as alternative A_j [10].

By pairwise comparison, regarding the importance of the criteria, a square matrix A whose elements are the ratios of a_ij criteria weights [10, 15, 22] can be composed as follows:

$$A=\left[a_{ij}\right].$$E5

The characteristics of the matrix are as follows [10]:

$$a_{ij}>0,a_{ij}=\frac{1}{a_{ji}},a_{ii}=1\text{ and}$$E6

$$a_{im}{a}_{mj}=a_{ij},i,m,j=1,2,\dots ,k.$$E7

The consistency of matrix is confirmed in the case of:

$$Aw=kw.$$E8

In practice, the consistency is usually incomplete; therefore,

$$Aw=\lambda w,$$E9

where λ represents the eigenvalue and w the eigenvector of the matrix A, which belongs to the eigenvalue λ. Only if k = λ, the consistency of the decision maker is complete. [5, 10] defined a measure of consistency or consistency index (CI) as follows:

$$\text{CI}=\frac{\lambda \max -k}{k-1}$$E10

where λ_max represents the principal eigenvalue and k the size of matrix.

The calculation of the consistency of the decision maker is defined as follows [10, 50]:

$$CR=\frac{CI}{R}$$E11

where CR is the consistency ratio, and R represents the random consistency index.

The consistency index is compared to a value, derived by generating random reciprocal matrices of the same size, to give a consistency ratio (CR), which is meant to have the same interpretation, regardless the size of the matrix. The comparative values from random matrices are as follows for 3 ≤ k ≤ 10 [5].

Size of matrix k	3	4	5	6	7	8	9	10
Comparative value R	0.58	0.90	1.12	1.24	1.32	1.41	1.45	1.49

Table 5.

Comparative values.

Source: [42, 5 and [10].

A consistency ratio of 0.1 or less is generally considered to be acceptable. Evaluating the importance of the criteria results [15, 22] in:

$$\left[\begin{array}{ccc}a12& a13& a1k\\ & a23& a2k\\ & & ak-1,k\end{array}\right]$$E12

Advantages of the method include (1) unity (a single, comprehensive, and flexible model for unstructured problems), (2) interdependence (of the system elements), (3) complexity (combining deductive and systemic approaches to problem solving), (4) hierarchical structure, (5) measurement (descriptive expressed properties by corresponding scale), (6) consistency (foresees the consistency of judgments for determining priorities), (7) synthesis, (8) exchange (considers relative priorities and enables selection of the best alternative), (9) judgment and consensus (combining various judgments in the result), and (10) reiteration (allows reconsideration of the problem definition, correction of judgments, and improved understanding of the problem) [48].

[10] classifies activities of solving the multicriteria decision-making problem as (a) structuring the problem (criteria tree), (b) determining weights of the criteria, (c) calculating aggregated values of alternatives, (d) alternatives ranking, and (e) sensitivity analyses.

In accordance with the method of AHP, by using leading supporting software Expert Choice, research compared previously selected cities (Table 1), with the aim to identify the performance of urban development, using the criteria (indicators) and alternatives (variants), arranged in a hierarchical model. Synthesis results replied to the question of the performance development effectiveness of selected national city compared to chosen European cities.

8. Problem modeling

The structuring of a decision making process started by defining the global objective (goal setting)—selecting the most development successful among six preferential cities, followed by entry of criteria, which represent six areas: (1) demography, labor market, and economy; (2) quality of life; (3) society, culture, and leisure activities; (4) research and development; (5) accessibility, urban networks, and international connectivity; and (6) management of sustainable resources. The process continued with defining alternatives (cities: Maribor, Pleven, Linz, Erfurt, Trieste, and Brugge) and structuring the problem-specific criteria and subcriteria entry.

The chosen indicators were derived from the set of 53 indicators (Table 1), where selection was narrowed to 24 indicators (3, 6, 7, 9, 13, 15, 20, 22, 23, 25, 26, 31, 33, 38, 39, 42, 43, 44, 45, 46, 47, 48, 50 and 52) due to the availability and completeness of data (no data gaps) for all criteria and all alternatives, thus providing credible values regarding to the attribute and the global objective. In addition to the presented weighting approaches (Chapter 6), the importance of each criterion in comparison with the importance of other criteria of an area (1 to 6, a total of 24 indicators = criteria) following the concept of classifying indicators (Table 3) was introduced. Weights in the power zone are the most important and influential (indicators: 3, 7, 15, 22, 25, 26, 42, 43, 45, 46), those identified in the connection zone are important regarding the influence, but at the same time significantly dependent from others (indicators: 9, 20, 31, 39, 44, 47, 48), while weights located in the isolated zone, with small influence above others, are the most useful at the end of the estimation (indicators: 6, 13, 23, 33, 38, 50, 52).

Figure 1 demonstrates the process of problem structuring using criteria tree. Weights are based on available data and methods for calculating the factor weights (Saaty). At each node of the hierarchy, a matrix will collect the pairwise comparisons of the decision maker for the criteria and subcriteria, e.g., subcriterion of the total working population is three times more important than the proportion of the population employed in the service sector, equally important as the unemployment rate, and 1.5-times more important than average disposable income (Figure 2).

Figure 1.

Structuring the problem (criteria tree).

The total working population includes employment not only in the services but also in other sectors (agriculture, hunting, forestry, fishing, mining, manufacturing, construction, etc.); consequently, the importance assigned is greater. Compared with the rate of unemployment, its importance is equal, owing to the fact that the entire working population and unemployment rate represent an important factor of social inclusion. Confirming the strength of the importance judgment, theoretical principles define “labour force participation rate,” expressed as [3]:

Pop = total population LF = labor (“labour force”) = U + E

LF_pop = total working population p = participation rate = LF / LF_pop

(males and females 15–64)

E = number of employees (“employed”) e = rate of employment = E / LF

U = number of unemployed persons u = unemployment rate = U / LF.

The increase of the unemployment rate can be simultaneously reflected by the increase of employment, e.g., if a larger number of new workers are entering the workforce segment, but only a small fraction actually becomes employed, an increase in the number of unemployed exceeds the growth in employment. The rate of presence in the labor market is therefore a key component of long-term economic growth, almost as important as productivity [3].

Figure 2.

Comparison matrix of the first node (graphical scale).

One of the AHP’s strengths is the possibility to evaluate quantitative as well as qualitative criteria and alternatives on the same preference scale of nine levels, also verbal (Figure 3). The subcriterion - number of students in upper and further education ISCED 3-4 per 1000 resident population, is according to the criterion of society, culture, leisure activities, equally important as subcriterion - number of students in higher education ISCED 5-6 per 1000 resident population. The strength of the given importance judgment is based on the results of the research about skills, needed in Europe by focusing on the 2020 objectives, carried out by European Centre for the Development of Vocational Training in 2008.

Defining future employment opportunities, the research highlighted the importance of qualifications ISCED 3-4 and 5-6 against the others (trend of “replacement demand”). Forecasts include 105 million new jobs by 2020 (2006–2020); among them, 41 million require a high level of qualification (ISCED 5-6). The existing qualification structure must be, in accordance with quoted, necessarily transformed as the foreseen remaining 55 million new jobs expectedly require medium-level education (ISCED 3-4), which traditionally include vocational qualifications, while less than 10 million new jobs include lower qualification levels (CEDEFOP, 2008, p. 13).

Figure 3.

Judgment scale (verbal).

Pairwise numerical criteria comparisons (Figure 4) showed that criterion employment in high-tech industries and knowledge-intensive sectors (NUTS 2) is three times more important than the R & D expenditure in % of GDP (NUTS 2). The strength of the criterion importance judgment was expressed on the basis of the United Nations Educational, Scientific, and Cultural Organization (UNESCO) survey measuring R & D personnel, carried out by Institute for Statistics in 2009. Salaries of researchers in high-tech and knowledge-intensive sectors represent a significant part of expenditures for research and experimental development, taking into account the total R & D personnel by sector and occupation as well as the level of qualification and full-time employment (“FTE method”). The consideration of the head count (HC) methodology, half-, part-, and full-time FTE, consequently led to overestimated expenditure (research and experimental development) in % of GDP, which again reinforced the validity of the criterion importance judgment (UNESCO, 2009). The matrix was shown as perfectly consistent (inconsistency ratio = 0.00).

Figure 4.

Judgment scale (numerical).

When calculating the final value of alternatives, in the synthesis, where local priorities change to global, the additive model is used in a research. Expert Choice allows two modes of synthesis: (a) the distributive (the sum of the priorities on each level equals 1), used in the case of the desired alternative selection, better in relation to the other, and (b) an ideal mode, used in the case of obtaining only the best variant, regardless to any other [10]. If the priorities are already known, the distributed mode is the only approach, which retrieves these priorities. Introducing or removing (Troutt, 1988) a copy [4] or a near copy [17] of an alternative results in a rank reversal of the appeared alternatives. The latest was subject to criticism [17, 27, 28, 56] but also legitimization [24, 40, 43, 46, 47, 49, 59]. In accordance to Wang and Luo (2009), the rank reversal is not unique to AHP but to all additive models [29]. In this study, the distributive mode was selected; adding or removing alternatives was reflected in the adjustment of ratios and rankings.

The final values of alternatives to the objective (main goal) of “the best city performance development” (Figure 5) were as follows: Erfurt (0.191), Linz (0.188), Brugge (0.180), Trieste (0.159), Pleven (0.142), and Maribor (0.134).

Figure 5.

Selection of the best city performance development: the final values of alternatives.

8.1. Interpretation of results

By analyzing the evaluation results (Table 6) using the criterion of demography, labour market—employment, economy (and its subcriteria), the city of Maribor reached a value of 0.120, reflecting the weakest result in comparison with other cities, with 57.97% realization of “the best city performance development” main goal, as compared to Linz. Trieste reached this objective by 95.17, Brugge by 89.37, and Erfurt by 67.63%. According to the criterion quality of life, Maribor reached a rating of 3 by 59.84% realization of the main objective; its position worsened with a rating of 5 according to the criterion society, culture, and leisure activities (56.83% of main goal accomplishment). Improved classification (rating 4) was achieved in the field of research and development. However, by the criteria of accessibility of urban networks and international connectivity (46.70% as compared with the leading Trieste and the value of 0.212) and management of sustainable resources (51.11%), the weakest goal realization was recorded. Considering all quoted (sub)criteria of areas 1–6, the latter was most successfully reached by the city of Erfurt, followed by Linz, Brugge, Trieste, and Pleven, with last rating belonged again to Maribor (realization of 70.16%).

Scope/rating (ratings)	1	2	3	4	5	6
Demography, labor market—employment economy	Linz; (0.207)	Trst; (0.197)	Brugge; (0.185)	Pleven; (0.151)	Erfurt; (0.140)	Maribor; (0.120)
Quality of life	Pleven (0.249)	Brugge (0.220)	Maribor (0.149)	Erfurt (0.142)	Trst (0.133)	Linz (0.106)
Society, culture, leisure activities	Linz (0.271)	Brugge; (0.184)	Erfurt*; (0.155)	Trst* (0.155)	Maribor; (0.154)	Pleven (0.081)
Research and development	Erfurt (0.283)	Linz (0.218)	Trst (0.150)	Maribor (0.142)	Brugge (0.132)	Pleven (0.075)
Accessibility of urban networks and international connectivity	Trst (0.212)	Pleven (0.201)	Linz (0.171)	Erfurt (0.165)	Brugge (0.153)	Maribor (0.099)
Management of sustainable resources	Erfurt (0.225)	Linz (0.200)	Brugge (0.184)	Trst (0.158)	Pleven (0.118)	Maribor (0.115)
Total 1–6 to the objective of “the best city performance development”	Erfurt (0.191)	Linz (0.188)	Brugge (0.180)	Trst (0.159)	Pleven (0.148)	Maribor (0.134)

Table 6.

Comparison of the evaluation results.

*Rating 3 is shared by Erfurt and Trieste (equal values).

With the purpose of determining the stability of the resulting solutions [12], respectively, the sensitivity of the result by varying criteria weights (the latter identifies in changing values and the order of the alternatives), the sensitivity analysis in forms of “performance,” “dynamic” “gradient,” “two-dimensional (2D) plot,” and “head to head” (between two alternatives) was performed.

Performance sensitivity graph in the Figure 6 indicates which alternatives are better or weaker at a particular criterion (Čančer, 2009), e.g., Erfurt is the best according to the criteria of research and development and sustainable resource management. Pleven is the best according to the criterion quality of life and weakest regarding society, culture, leisure activities, and research and development. Maribor is the weakest in terms of the criteria demography, labour market—employment, and economy; the accessibility of urban networks; and the sustainable resource management. Trieste is the best regarding the accessibility of the urban network.

The gradient analysis enabled to identify influence on the final value of alternatives due to individual criteria weightings alterations [12]. Dynamic sensitivity analysis (Figure 7) indicates the weight increase of the criterion of society, culture, and leisure activities from 0.150 to 0.164 or more for the second-ranked alternative to become the best one.

Head-to-head analysis, by comparing two alternatives, clearly demonstrated the superior one by accomplishing individual criterion and global goal (Čančer, 2009). As apparent from the Figure 6, the city of Pleven was more successful according to the criterion of demography, labor market—employment, economy, quality of life criterion and the criterion of accessibility of urban network, while the city of Maribor indicated better performance according to the criteria society, culture, and leisure activities, as well as research and development. Pleven gathered higher final value, namely, for 0.0144.

Analysis of the 2D plot led to identifying the dominated and nondominated alternatives regarding the pair of selected criteria [12]. As shown in Figure 6, according to the criteria of demography, labor market—employment, economy, and quality of life, Linz, Trieste, Brugge, and Pleven represent the nondominated alternatives, while Erfurt and Maribor belong to dominated alternatives.

Figure 6.

Sensitivity analysis.

Figure 7.

Change in weightings.

9. Conclusion and future development

The research aimed at testing the development efficiency of such a methodology for measuring performance success of urban development, which would be useful within the national as well as international (European) comparable city sample. For the testing purposes, the selection of cities followed certain criteria. The determination of adequate measurement indicators, closely associated with evaluation of known methodological concepts (Smart City, city performance, and urban status and sustainability) and relevant databases, resulted in obtained useful tool: a system of 53 selected indicators by field measurement, meaningfully divided into six areas and added categories to enable ranking of comparable medium-sized European cities.

Using AHP and its supporting Expert Choice program tool for quantitative data analysis, which included narrowed set of 24 indicators (no data gaps), the research sought out for the confirmation of selection decision possibility in quoted city sample. AHP evaluation of alternatives provided clarity in multiattribute decision-making process, resulting in ranking in accordance with a defined hierarchy and relative importance of decision criteria (criteria tree and weightings). Achieving the best possible decision due to complex problem structure therefore demands a trade-off between prefect modeling and usability of the model.

Meanwhile, advantages of the hierarchical problem modeling included the possibility to adopt verbal judgments and the verification of the consistency, Expert Choice incorporated intuitive graphical user interfaces, automatic calculation of priorities and inconsistencies, as well as several ways to process a sensitivity analysis. It has to be pointed out that, beside the traditional application of AHP and supporting software, a new trend in use, namely combining others methods, e.g., neural networks, SWOT analysis, and others, is emerging, as AHP is still part of certain theoretical discussions, resulting from the limitation due to assumed criteria independence and hierarchy problems due to appropriate judgment scale.