City performance measurement indicators
Cities represent the driving force of development in economic, social, and cultural life, reflecting also the spatial organization of human society. Taking into account the fact that cities are becoming generators of economic development and a source of growth for the national economy, there is an increasing urge to identify the stages of development and to establish a system for the ranking and positioning of cities and regions in this process (the level of categorization). This will allow the preparation of appropriate strategic and development guidelines for cities and urban regions to take place. In order to be able to compare the level of their efficiency in fostering development, there is an intensifying need to develop indicators that measure the performance of cities, are representative and comparable between countries, and allow verification to others. At present, there are many different urban indicators and institutions that compile and analyze them. Performance measurement systems, developed for internal use in some cities, already show a degree of measurement feasibility. The fundamental problem is that this variety of indicators lacks consistency and comparability (over time and between compared cities). Therefore, their use cannot be approved in a wider context (benchmark) of comparative situations. Upon the case of medium-sized cities, we consequently have to question the applicability of the methodology and indicators, used mostly in cases of large, global cities by internationally recognized institutions. With the established set of indicators and assistance of computer programs for multiparameter decision-making processes (analytic hierarchical process [AHP]), this paper also seeks to investigate comparisons between performance of selected European cities (on a qualitative basis).
- urban development
- measurement systems
- development of urban indicators
- city ranking
- quality of life
Existing methodologies in comparison to performance and quality of urban city structure affect more or less a wider field of urban and regional disparities, while specific approaches cover only limited areas.  focuses exclusively on infrastructure impacts, Callois and Aubert (2007) empirically analyze the impact of social capital on regional development. The advantage of quoted approaches represents limited number of variables involved in analysis. In the area of measuring the quality of life,  provide an overview of indicators of sustainable development, as well as  and , but the interpretation of the indicators of quality of life is missing. In the field of competitiveness,  presents the synopsis of indicators measuring urban competitiveness on a European scale, while  indicate the multicast nature of sustainable development that consequently leads to the unclear definition of the measuring indicator. Missing thematic indicators can also be found in the context of measuring regional disparities, both at the level of the broader European countries  and in the narrow sense of the regions . Comparing cities by the use of indicators that represent diverse aspects of urban life is only possible with the meaningful structured set system; easily adding a large number of indicators to achieve a single index may result in criticism of uncertainty and noticeable limitation of its interpretation. Similar effects can also be achieved by using a larger set of nonaggregated indicators; therefore, identification of appropriate, small number of relevant indicators is crucial. In the process of establishing the set of indicators, the inclusion of indicators with higher impact on the general differences between selected cities in different countries is necessary; an additional assumption incorporates the integration of indicators from the field of environmental, human, and social capital as well as the demographic point of view.
2. Theoretical background and applied practices
When searching for the most relevant performance indicators of city development, we proceed from the fact that more than two-thirds of the population live in urban areas. The urban environment provides a fertile ground for the development of science and technology, culture, and innovation. On the other hand, in cities, there is also more emphasis on the problems such as unemployment, discrimination, segregation of society, and poverty. The cities are also faced by challenges, associated with mitigating the effects of climate change, job creation, prosperity, and quality of life. Therefore, the development of cities has a decisive impact on the future of the economic, social, and territorial development. As highlighted by the recent European Commission survey entitled “Cities of the Future—Challenges, Ideas and Expectations” (EC, 2011), a phase of urban sprawl in recent decades has shown serious problems associated with the deterioration of urban areas due to lack of infrastructure construction and basic services. Promoting urban renewal as the driving force of prosperity and creating opportunities together with strengthening the link between cities and development, and between urban centers and surrounding areas, are the main challenges to provide stable economic growth.
Establishing a system of indicators for measuring performance development of selected cities included the consideration of contemporary city’s complex aspects with reference to (a) the 72 attributes of a smart city,
The case study example of Glasgow classifies city performance indicators as follows: (1) population (mortality, fertility, population projections), (2) economic participation (employment, unemployment, vacancies), (3) poverty (access to bank accounts, children poverty, financial hardship, low-income households), (4) health (life expectancy, inability to work), (5) social capital (social inclusion, social networks, trust and reciprocity, civic participation), (6), environment (green environment, open space, air quality, recycling), (7) transport (transport volume, journeys to work and school, traffic accidents, cycling), (8) education (children education, the highest qualification obtained, the qualification of the working population, training of young people), (9) safety of local communities (overall level of crime, antisocial behavior, violence, unintentional injuries), (10) lifestyle, (11) cultural vitality (involvement in sport and cultural events), and (12) mind-set (religion, politics, involvement in the community, trust, national identity).
Indicators of sustainable development reflect the complex and dynamic structure of the urban environment. With the adoption of Agenda 21 (1992), this type of indicator was developed by a number of institutions, including the World Bank (UN—Urban Indicators Programme), followed by indicators of the World Health Organization (WHO), as the analytical tools for studying population health and quality of life in urban environment. A wider set of indicators also includes project SUD-LAB EC (European Commission) with an expanded database of European cities, where indicators are divided into the following categories: (a) air quality, (b) composed environment, (c) cultural endowments, (d) social disparities, (e) quality of transport, (f) urban management, and (g) waste management. For each of listed categories, a set of indicators is reflecting the level of urban functionality. Indicators of the EU-TISSUE Programme, in use in 15 European countries, relate to the areas of sustainable urban management (descriptive indicators), sustainable urban transport, sustainable urban construction, and sustainable urban design .
In accordance with the Charter on European Sustainable Cities and Towns,  lists six key areas of sustainable development and urban transformation: (1) active city/town, (2) beautiful town, (3) green city/town, (4) town with a better environment, (5) cooperation for a better city, and (6) town catalogue. The strategy of urban sustainability consequently includes urban sustainability performance indicators such as (1) local involvement (citizen’s participation); (2) employment; (3) city deficit; (4) economic growth; (5) urban mobility; (6) urban metabolism, resources, and consumption; (7) environment and social expenditure; (8) urban safety; (9) public health; (10) social justice; and (11) global change.
Among indicators of central city area development, Niţulescu (2000) includes the following: (1) types of land using (constructions, green areas), (2) green areas surface from the total town center’s surface, (3) percent of residential buildings from the total number of buildings from the center of the town, (4) percent of trade buildings from the total number of buildings from the town center, (5) percent of central functions buildings (administrative, international, unique endowment) from the total number of buildings from the center of the town, (6) built areas of public utility related to then inhabited areas, (7) employment density (number of working places related to the town center surface), (8) rate of employed population for each sector (industry, trade, services), (9) number of crossroads for the surface of the town center, (10) surface of pedestrian circulation for the surface of the town, and (11) surface of pedestrian circulation for the surface of roadway .
Among indicators of urban status ranks, Şuler’s (2005) category of population and labor force indicators are as follows: (1) number of inhabitants, (2) population density (per hectare), (3) working places/1000 inhabitants, and (4) proportion of the population employed in the service sector. The category of living and quality of life indicators include the following: (5) number of residential buildings per 1000 inhabitants, (6) houses equipped with plumbing (% of buildings), (7) number of personal cars per 1000 inhabitants, (8) number of houses with bathrooms inside the building, (9) number of hospital beds per 1000 inhabitants, (10) number of doctors per 1000 persons, (11) financial/banking institutions (headquarters, working points), and (12) accessibility to lines of communication (railway station, bus station). The indicators of category society, culture, and leisure include the following: (13) education units (high school, secondary, postsecondary school), (14) secondary school in primary and secondary educational units (%), (15) cultural and sports endowments (theaters, public libraries, gyms, auditorium, stadium), and (16) accommodation places/1000 inhabitants. Indicators of the urban network are specified as follows: (17) modernized streets (%), (18) streets with water pipes (%), (19) waste water treatment, (20) household gas distribution pipes (%), (21) sanitation motor vehicles for 100 km of street, (22) scavengers for 1000 inhabitants, and (23) green area surface m2/inhabitant.
 defines an index of local development as an integrated indicator, including the importance (weights) of individual elements as category of infrastructure (4), followed by the economy (3), community (2), and the public administration (1):
Category infrastructure includes utilities, transport infrastructure, health infrastructure, natural resources, and natural environment. Economy includes financial services and insurance, labor, and public budget. Public administration includes public administration, services and support to small and medium-sized enterprises, urban planning, communication, and information dissemination. Among the indicators of development, Bӑnicӑ (2010) introduces the community spirit, safety of citizens, tourist attractions, cultural/sports facilities, and cultural/historical heritage.
 formed an indicator of the public urban transport quality using available Eurostat database indicators, including the following subindicators: (1) the proportion of journeys to work by public transport, (2) the length of the public transport network, (3) the number of stops of public transport per km2, (4) the price of a monthly public transport ticket, (5) the number of stops per 1000 population, (6) the number of stops per 1 km of public transport network, (7) the ratio between the public transport network on fixed infrastructure and flexible connections, and (9) the proportion of land for transport use.
World Bank Urban Development Indicators include the following: (1) proportion of urban population with access to improved health services, (2) proportion of urban population with access to water resources, (3) number of motor vehicles per 1000 population, (4) number of passenger cars per 1000 inhabitants, (5) emissions of PM10 (micrograms per m3), (6) proportion of poverty, (7) fuel prices, (8) fuel consumption per capita, and (9) the percentage of the urban population.
Significant importance among urban sustainability indicators belongs to European Common Indicators (ECI), first established in 1999–2003, by the research institute Ambiente Italia. Within a base with more than 1000 indicators, 10 key indicators, reflecting the development trends of European cities in accordance with the principles of social inclusion, local governance and democracy, local/global integration of the city, local economy, environment, cultural heritage, and quality of the institutional environment was selected: (1) citizens’ satisfaction with the local community, (2) local contribution to global climate change (CO2 emissions per capita), (3) local mobility and passenger transportation, (4) availability of local public open areas and services, (5) quality of the air (emissions of PM10), (6) children’s journeys to and from school, (7) sustainable management of the local authority and local enterprises, (8) noise pollution, (9) sustainable land use, and (10) products promoting sustainability.
With reference to the cited attributes of a smart city, Specifically explained in Section 6.
Specifically explained in Section 6.
3. Selection of indicators
The selection of appropriate indicators included research and exploration, evaluation, and selection of relevant databases, through which adequate indicators of measurement as a basis for determining the level of the city performance development and consequently a useful tool for ranking of comparable medium-sized European cities was obtained. Indicators in the study were selected on the basis of following assumptions: (1) objectivity (clear, easy to understand, precise, and unambiguous); (2) relevance, measurability, and reproducibility (quantitative, systematic observable); (3) validity (with the possibility of verification and data quality control); (4) statistical representativeness (at the city level); (5) comparability/standardization—longitudinal (over time) and transverse (between cities); (6) flexibility (with the possibility of continuous improvement); (7) efficiency/performance (as decision making and local management planning tool); (8) accessibility (available databases, use of existing data); (9) interaction (social, environmental, economic); and (10) consistency and temporal stability. Last but not least, the selection of appropriate indicators was also related to the concept of data homogeneity. In searching for the relevant data, many of the existing semantic information about the state of the city and urban region were expected to be available; therefore, the data credibility was highlighted.
4. Selection of cities
In Europe, more than 600 cities and urban regions are classified as medium-sized with a population between 100,000 and 500,000 inhabitants (selection criteria). In the case of a single manual data collection, the data processing for such number of cities are practically impossible. Therefore, the reselection of urban sample in terms of a data source (all selected cities should be covered by a specific source, e.g., Urban Audit) was necessary to eliminate the risk of the diverse resources’ use, related to the area and the region of the city, induced by the dimension of the selected city sample. In case of insufficient data, the use of different spatial levels (Eurostat database is corresponding to NUTS2, representing regions and provinces, while the Eurobarometer data correspond to NUTS0/national level) was imminent. Quoted databases focus on the European cities’ profiles, which further narrowed the selection frame. The final selection of cities was defined (Table 2) on the basis of the following: location (criterion 1: all selected cities are located in Europe), database (criterion 2: inclusion in the database Urban Audit), definition in terms of a smart city (criterion 3: placed in the “Smart Cities” base), comparability in terms of urban size (criterion 4: comparable population size: medium-sized cities with the range of 100,000 to 200,000 inhabitants), and regional significance (criterion 5: capital region or important regional center). With reference to fulfilled criteria, research cities represented Maribor (Slovenia), Pleven (Bulgaria), Linz (Austria), Erfurt (Germany), Trieste (Italy), and Brugge (Belgium).
|Maribor||Slovenia||SI0: Slovenia||SI02: West Slovene region||SI012 Podravje region|
|Pleven||Bulgaria||BG3: North I SE Bulgaria||BG31: Northwest||BG314 Pleven|
|Linz||Austria||AT3: Westösterreich||AT31: Oberösterreich||AT312 Linz-Wels|
|Erfurt||Germany||DEG: Thüringen||DEG0: Thüringen||DEG01 Erfurt|
|Trieste||Italy||ITH: North-East||ITH4: Friuli-Venezia Giulia||ITH44 Trieste|
|Brugge||Belgium||BE2: Vlaams Gewest||BE25: Prov. West-Vlaandeeren||BE251 Arr.Brugge|
5. Data structure and categorization
The database of the research was largely represented by an Urban Audit indicator set for core cities, available as a part of a broader Eurostat collection. The base of data analysis (accessed February 2012) covered 30 countries: the EU-27, along with Turkey, Switzerland, Norway, and 372 urban units (city and the wider urban area) and specific metropolitan areas. The temporal span, used in the research, included periods 2010–2012, 2007–2009, and 2003–2006, exceptionally 1999–2002, but only to illustrate the missing measurement in the time series. In addition to Urban Audit, research also implied regional databases of EUROSTAT (appsso.eurostat.ec.europa.eu), and the index of quality of life in each country was defined by using ranking of International Living survey. Taking into account the selection of cities from different countries in terms of validity and international comparability, and to avoid inaccuracies due to diverse methodological approaches, the research additionally incorporated data from the Statistical Office of the Republic of Slovenia (www.stat.si), Austria (Statistik Austria; www.statistik.at), Italy (SISTAN Sistema statistico nazionale; www.sistan.it and www.istat.it), Germany (www.destatis.de), Belgium (statbel.fgov.be), and Bulgaria (www.nsi.bg). Urban Audit database, used in 75.47% of cases, was followed by Eurostat database with 22.64% and other data sources (1.89%); overall data coverage rate reached 87%. Limitations of the research referred to the missing data; the inclusion of the secondary databases that would otherwise fill out the data gap could be due to the chosen methodology of data collection and evaluation, which will result in the reduced data comparability of data and furthermore between cities within individual indicators. Dropping variables was potentially admissible in cases of minor influence on the dependent variable (
6. Criteria weighting
6.1. Determining the weights of indicators: different approaches
Weighting of indicators emphasized the suitability requirements, with the value of the weight indicating the impact of each criterion on the final goal (objective). Weighting methods are different, are very widely used, and are scalable in many cases applied, where 0 equals the insignificant impact of the indicator, range 1–3 represents a significantly less important indicator, range 4–7 represents a little less important indicator, and range 8–10 represents an equally important indicator in terms of the relative importance with the most important criteria [1, 30]. In the case of a clearly defined target group, the determination of relevant weightings is also possible by using the questionnaire survey. Stepwise methods label 5–6 as low importance indicators (complementary, supplementary, secondary, incidental, indirect, and no impact), 7–8 as average significant indicators (imperative, mandatory, or required indicator), and 9–10 as high importance indicator (fundamental, essential, decisive, definitive, and guidelines).
The weighting is also possible with the prioritization of functional variables in the form of a matrix (CICAPSO, 2012), consisting of the
Conceptualization of the system of indicators in research was based on the relevance of the individual categories, taking into account the relative importance of weights on the objective measurement: performance development of selected cities. Considering that the system of indicators represents a baseline tool, the weighting depends on the purpose of the decision maker in terms of defining the specific goal of measuring and monitoring. Comparability of the indicators was previously reached by using available, credible databases (Section 5.1).
In the case of the desk research data collection, the
By determining the adequate weighting, the research in this section also considered weighting of indicators, measuring the competitiveness of cities in the context of the knowledge economy, where the greatest importance was given to categories of quality of life (weighting 0.20) and knowledge base (weighting 0.20), followed by the categories of innovation (weighting 0.10) accessibility (weighting 0.10), urban diversity (weighting 0.10), productivity (weighting 0.10), and social connectivity (weighting 0.10). Areas of agglomeration and economic heritage were defined with a weight of 0.05 .
With reference to quoted concepts, the largest weighting importance in research was assigned to the categories of quality of life, environment, lifelong learning, development of information, and communication technology and city brand (weighting 0.20), followed by labor market, productivity, entrepreneurship, innovation, and mobility (weighting 0.15). The importance of social cohesion, governance, and urban diversity was defined with a weight of 0.10; a minimum effect on the performance development measurement was attributed to demographic categories (weighting 0.05). Weightings for individual categories of indicators 1–53 are presented in Table 1.
In terms of weighting credibility, the study also considered Mercer’s classification and evaluation indicators (weights) of quality of life (Quality of Living Report) in European cities (Urban Audit database, benchmarking analysis of 246 European cities). The study of 10 dimensions, namely, (1) quality of life, economic environment, (2) political and social environment, (3) sociocultural environment, (4) health and medicine, (5) schools and education, (6) public services and transport, (7) recreation, (8) consumer goods, (9) housing possibilities, (10) natural environment, and 39 quality of life indicators showed a certain degree of area similarity to the selected indicators’ system in the research (demography, labor market, economy, quality of life, society, culture and leisure activities, and R & D). Mercer’s weights in specific areas are defined as follows: political and social environment (weighting 0.283); economic environment (0.048), which includes employment in the services sector (NACE classification J-K); area of health and medicine (0.229), which also includes life expectancy in years; schools and education (ISCED with weight of 0.041); public services and transportation (0.157), including air passengers using nearest airport; recreation (0.109); housing possibilities (0.062); and the natural environment (0.071), including rainfall .
7. Multiattribute decision models and system of indicators’ simulation with computer-based programs
7.1. Methods for decision support
After the system of indicators for monitoring performance development of the city had been set, the purpose of the study was to enable quality decision making in a systematic, organized manner. The preparation of scenario and the selection of the chosen strategy involved either verbal or numerical representation of inputs in principle, which required the inclusion of artificial intelligence. Multicriteria models represent a useful tool to support decision making in complex decision situations, where a large number of factors and variants affect the final decision. Supporting software tools in designing a decision model evaluate variants and offer a range of different analyses for detailed decision’s verification and justification [6, 7].
Systematic decision-making processes for supporting smart decisions should be based on combining normative theories and cognitive aspects, forming an integral part of decision making in practice. According to , problem solving can be done in several ways: intuitively, routinely—by adopting the past used procedures, or random selection—by systematic rational thinking using relevant information. In the latter, the decision maker measures the values of alternatives by each individual criterion or by multiple criteria simultaneously .
The general approach of decision analysis originates from axioms of the game theory, by John von Neumann and Oskar Morgenstern. The main steps represent problem structuring, estimating the likelihood of possible outcomes, determining their utility, evaluating alternatives and selecting strategies. Briefly, the process of multiattribute decision making involves problem identification and its structuring, the model building, and activities of problem solving planning, wherein have foreseen also returning from each following to the previous phase .
The major role in decision making according to multiple criteria goes to classification or ranking. Identifying the decision maker’s relative importance of each criterion can be expressed as a priority (the criterion is more important than the other) or weighting, which declares the relative importance of the various criteria .
In the research, comparison of the cities’ development performance was carried out using the analytic hierarchical process (AHP) method, developed by Thomas L. Saaty. In accordance with the theory of AHP, multicriteria problems are initially presented in the form of a hierarchical model. Several papers demonstrated the AHP efficiency in different areas [19, 21, 26, 31, 32, 37, 51, 52, 59, 64]. The oldest reference we have found dates from 1972 . After this, a paper in the
The method’s basis represent pairwise comparisons of the two criteria at the same level in relation to the element on the next (higher) level. In order to help the decision maker to provide the pairwise comparisons, Saaty created a 9-point intensity scale of importance between pair of elements (Table 4). If the estimation a.., is assigned to criterion
Weighting criteria and priorities to alternatives are not assigned directly by decision makers; they are calculated from the judgments, entered by comparing the importance of criteria and preferences of alternatives in pairs in verbal, graphic, or numerical manner .
A top-down approach of AHP method leads from the goal to the alternatives, while the bottom-up approach represents expression of judgments about alternatives before expressing judgments on the criteria [16, 38].
|1||The criteria are equally important; alternatives are equally preferred/equally|
contribute to the objective.
|3||Experience and judgment slightly favor one activity over another. The criterion is moderately/slightly more important than the comparable criterion.|
|5||Experience and judgment strongly favor one activity over another. The criterion is strongly more important than the comparable criterion; alternative is strongly more preferred.|
|7||Very strong or demonstrated importance. Criterion is powerfully more important than the comparable criteria.|
|9||The criterion is extremely more important than the comparable criterion; alternative is extremely more preferred, highest possible favoring of one criterion over another.|
Evaluating the importance of criteria and preference of alternatives, according to individual criteria, includes a criteria importance estimation by setting the appropriate weights; for AHP, the sum of the weights for each group of criteria is considered equal to 1 (hierarchical manner of determining the weights).
Attributes (criteria at the lowest hierarchical level) are represented as follows :
represent the weights.
It is assumed that:
By calculating the values of alternatives with respect to each attribute is:
Given n objects, e.g., attributes or alternatives, we suppose that the decision maker is able to compare any two of them. In preference modelling, this assumption is called comparability. For any pairs (
ratios of values of alternatives, indicating that alternative
The characteristics of the matrix are as follows :
The consistency of matrix is confirmed in the case of:
In practice, the consistency is usually incomplete; therefore,
The consistency index is compared to a value, derived by generating random reciprocal matrices of the same size, to give a consistency ratio (CR), which is meant to have the same interpretation, regardless the size of the matrix. The comparative values from random matrices are as follows for 3 ≤
|Size of matrix ||3||4||5||6||7||8||9||10|
|Comparative value ||0.58||0.90||1.12||1.24||1.32||1.41||1.45||1.49|
Advantages of the method include (1) unity (a single, comprehensive, and flexible model for unstructured problems), (2) interdependence (of the system elements), (3) complexity (combining deductive and systemic approaches to problem solving), (4) hierarchical structure, (5) measurement (descriptive expressed properties by corresponding scale), (6) consistency (foresees the consistency of judgments for determining priorities), (7) synthesis, (8) exchange (considers relative priorities and enables selection of the best alternative), (9) judgment and consensus (combining various judgments in the result), and (10) reiteration (allows reconsideration of the problem definition, correction of judgments, and improved understanding of the problem) .
 classifies activities of solving the multicriteria decision-making problem as (a) structuring the problem (criteria tree), (b) determining weights of the criteria, (c) calculating aggregated values of alternatives, (d) alternatives ranking, and (e) sensitivity analyses.
In accordance with the method of AHP, by using leading supporting software Expert Choice, research compared previously selected cities (Table 1), with the aim to identify the performance of urban development, using the criteria (indicators) and alternatives (variants), arranged in a hierarchical model. Synthesis results replied to the question of the performance development effectiveness of selected national city compared to chosen European cities.
8. Problem modeling
The structuring of a decision making process started by defining the global objective (goal setting)—selecting the most development successful among six preferential cities, followed by entry of criteria, which represent six areas: (1) demography, labor market, and economy; (2) quality of life; (3) society, culture, and leisure activities; (4) research and development; (5) accessibility, urban networks, and international connectivity; and (6) management of sustainable resources. The process continued with defining alternatives (cities: Maribor, Pleven, Linz, Erfurt, Trieste, and Brugge) and structuring the problem-specific criteria and subcriteria entry.
The chosen indicators were derived from the set of 53 indicators (Table 1), where selection was narrowed to 24 indicators (3, 6, 7, 9, 13, 15, 20, 22, 23, 25, 26, 31, 33, 38, 39, 42, 43, 44, 45, 46, 47, 48, 50 and 52) due to the availability and completeness of data (no data gaps) for all criteria and all alternatives, thus providing credible values regarding to the attribute and the global objective. In addition to the presented weighting approaches (Chapter 6), the importance of each criterion in comparison with the importance of other criteria of an area (1 to 6, a total of 24 indicators = criteria) following the concept of classifying indicators (Table 3) was introduced. Weights in the power zone are the most important and influential (indicators: 3, 7, 15, 22, 25, 26, 42, 43, 45, 46), those identified in the connection zone are important regarding the influence, but at the same time significantly dependent from others (indicators: 9, 20, 31, 39, 44, 47, 48), while weights located in the isolated zone, with small influence above others, are the most useful at the end of the estimation (indicators: 6, 13, 23, 33, 38, 50, 52).
Figure 1 demonstrates the process of problem structuring using criteria tree. Weights are based on available data and methods for calculating the factor weights (Saaty). At each node of the hierarchy, a matrix will collect the pairwise comparisons of the decision maker for the criteria and subcriteria, e.g., subcriterion of the total working population is three times more important than the proportion of the population employed in the service sector, equally important as the unemployment rate, and 1.5-times more important than average disposable income (Figure 2).
The total working population includes employment not only in the services but also in other sectors (agriculture, hunting, forestry, fishing, mining, manufacturing, construction, etc.); consequently, the importance assigned is greater. Compared with the rate of unemployment, its importance is equal, owing to the fact that the entire working population and unemployment rate represent an important factor of social inclusion. Confirming the strength of the importance judgment, theoretical principles define “labour force participation rate,” expressed as :
Pop = total population LF = labor (“labour force”) = U + E
LFpop = total working population p = participation rate = LF / LFpop
(males and females 15–64)
E = number of employees (“employed”) e = rate of employment = E / LF
U = number of unemployed persons u = unemployment rate = U / LF.
The increase of the unemployment rate can be simultaneously reflected by the increase of employment, e.g., if a larger number of new workers are entering the workforce segment, but only a small fraction actually becomes employed, an increase in the number of unemployed exceeds the growth in employment. The rate of presence in the labor market is therefore a key component of long-term economic growth, almost as important as productivity .
One of the AHP’s strengths is the possibility to evaluate quantitative as well as qualitative criteria and alternatives on the same preference scale of nine levels, also verbal (Figure 3). The subcriterion - number of students in upper and further education ISCED 3-4 per 1000 resident population, is according to the criterion of society, culture, leisure activities, equally important as subcriterion - number of students in higher education ISCED 5-6 per 1000 resident population. The strength of the given importance judgment is based on the results of the research about skills, needed in Europe by focusing on the 2020 objectives, carried out by European Centre for the Development of Vocational Training in 2008.
Defining future employment opportunities, the research highlighted the importance of qualifications ISCED 3-4 and 5-6 against the others (trend of “replacement demand”). Forecasts include 105 million new jobs by 2020 (2006–2020); among them, 41 million require a high level of qualification (ISCED 5-6). The existing qualification structure must be, in accordance with quoted, necessarily transformed as the foreseen remaining 55 million new jobs expectedly require medium-level education (ISCED 3-4), which traditionally include vocational qualifications, while less than 10 million new jobs include lower qualification levels (CEDEFOP, 2008, p. 13).
Pairwise numerical criteria comparisons (Figure 4) showed that criterion employment in high-tech industries and knowledge-intensive sectors (NUTS 2) is three times more important than the R & D expenditure in % of GDP (NUTS 2). The strength of the criterion importance judgment was expressed on the basis of the United Nations Educational, Scientific, and Cultural Organization (UNESCO) survey measuring R & D personnel, carried out by Institute for Statistics in 2009. Salaries of researchers in high-tech and knowledge-intensive sectors represent a significant part of expenditures for research and experimental development, taking into account the total R & D personnel by sector and occupation as well as the level of qualification and full-time employment (“FTE method”). The consideration of the head count (HC) methodology, half-, part-, and full-time FTE, consequently led to overestimated expenditure (research and experimental development) in % of GDP, which again reinforced the validity of the criterion importance judgment (UNESCO, 2009). The matrix was shown as perfectly consistent (inconsistency ratio = 0.00).
When calculating the final value of alternatives, in the synthesis, where local priorities change to global, the additive model is used in a research. Expert Choice allows two modes of synthesis: (a) the distributive (the sum of the priorities on each level equals 1), used in the case of the desired alternative selection, better in relation to the other, and (b) an ideal mode, used in the case of obtaining only the best variant, regardless to any other . If the priorities are already known, the distributed mode is the only approach, which retrieves these priorities. Introducing or removing (Troutt, 1988) a copy  or a near copy  of an alternative results in a rank reversal of the appeared alternatives. The latest was subject to criticism [17, 27, 28, 56] but also legitimization [24, 40, 43, 46, 47, 49, 59]. In accordance to Wang and Luo (2009), the rank reversal is not unique to AHP but to all additive models . In this study, the distributive mode was selected; adding or removing alternatives was reflected in the adjustment of ratios and rankings.
The final values of alternatives to the objective (main goal) of “the best city performance development” (Figure 5) were as follows: Erfurt (0.191), Linz (0.188), Brugge (0.180), Trieste (0.159), Pleven (0.142), and Maribor (0.134).
8.1. Interpretation of results
By analyzing the evaluation results (Table 6) using the criterion of demography, labour market—employment, economy (and its subcriteria), the city of Maribor reached a value of 0.120, reflecting the weakest result in comparison with other cities, with 57.97% realization of “the best city performance development” main goal, as compared to Linz. Trieste reached this objective by 95.17, Brugge by 89.37, and Erfurt by 67.63%. According to the criterion quality of life, Maribor reached a rating of 3 by 59.84% realization of the main objective; its position worsened with a rating of 5 according to the criterion society, culture, and leisure activities (56.83% of main goal accomplishment). Improved classification (rating 4) was achieved in the field of research and development. However, by the criteria of accessibility of urban networks and international connectivity (46.70% as compared with the leading Trieste and the value of 0.212) and management of sustainable resources (51.11%), the weakest goal realization was recorded. Considering all quoted (sub)criteria of areas 1–6, the latter was most successfully reached by the city of Erfurt, followed by Linz, Brugge, Trieste, and Pleven, with last rating belonged again to Maribor (realization of 70.16%).
|Demography, labor market—employment economy||Linz|
|Quality of life||Pleven (0.249)||Brugge (0.220)||Erfurt (0.142)||Trst (0.133)||Linz (0.106)|
|Society, culture, leisure activities||Linz (0.271)||Brugge|
|Research and development||Erfurt (0.283)||Linz (0.218)||Trst (0.150)||Brugge (0.132)||Pleven (0.075)|
|Accessibility of urban networks and international connectivity||Trst (0.212)||Pleven (0.201)||Linz (0.171)||Erfurt (0.165)||Brugge (0.153)|
|Management of sustainable resources||Erfurt (0.225)||Linz (0.200)||Brugge (0.184)||Trst (0.158)||Pleven (0.118)|
|Total 1–6 to the objective of “the best city performance development”||Erfurt (0.191)||Linz (0.188)||Brugge (0.180)||Trst (0.159)||Pleven (0.148)|
With the purpose of determining the stability of the resulting solutions , respectively, the sensitivity of the result by varying criteria weights (the latter identifies in changing values and the order of the alternatives), the sensitivity analysis in forms of “performance,” “dynamic” “gradient,” “two-dimensional (2D) plot,” and “head to head” (between two alternatives) was performed.
Performance sensitivity graph in the Figure 6 indicates which alternatives are better or weaker at a particular criterion (Čančer, 2009), e.g., Erfurt is the best according to the criteria of research and development and sustainable resource management. Pleven is the best according to the criterion quality of life and weakest regarding society, culture, leisure activities, and research and development. Maribor is the weakest in terms of the criteria demography, labour market—employment, and economy; the accessibility of urban networks; and the sustainable resource management. Trieste is the best regarding the accessibility of the urban network.
The gradient analysis enabled to identify influence on the final value of alternatives due to individual criteria weightings alterations . Dynamic sensitivity analysis (Figure 7) indicates the weight increase of the criterion of society, culture, and leisure activities from 0.150 to 0.164 or more for the second-ranked alternative to become the best one.
Head-to-head analysis, by comparing two alternatives, clearly demonstrated the superior one by accomplishing individual criterion and global goal (Čančer, 2009). As apparent from the Figure 6, the city of Pleven was more successful according to the criterion of demography, labor market—employment, economy, quality of life criterion and the criterion of accessibility of urban network, while the city of Maribor indicated better performance according to the criteria society, culture, and leisure activities, as well as research and development. Pleven gathered higher final value, namely, for 0.0144.
Analysis of the 2D plot led to identifying the dominated and nondominated alternatives regarding the pair of selected criteria . As shown in Figure 6, according to the criteria of demography, labor market—employment, economy, and quality of life, Linz, Trieste, Brugge, and Pleven represent the nondominated alternatives, while Erfurt and Maribor belong to dominated alternatives.
9. Conclusion and future development
The research aimed at testing the development efficiency of such a methodology for measuring performance success of urban development, which would be useful within the national as well as international (European) comparable city sample. For the testing purposes, the selection of cities followed certain criteria. The determination of adequate measurement indicators, closely associated with evaluation of known methodological concepts (Smart City, city performance, and urban status and sustainability) and relevant databases, resulted in obtained useful tool: a system of 53 selected indicators by field measurement, meaningfully divided into six areas and added categories to enable ranking of comparable medium-sized European cities.
Using AHP and its supporting Expert Choice program tool for quantitative data analysis, which included narrowed set of 24 indicators (no data gaps), the research sought out for the confirmation of selection decision possibility in quoted city sample. AHP evaluation of alternatives provided clarity in multiattribute decision-making process, resulting in ranking in accordance with a defined hierarchy and relative importance of decision criteria (criteria tree and weightings). Achieving the best possible decision due to complex problem structure therefore demands a trade-off between prefect modeling and usability of the model.
Meanwhile, advantages of the hierarchical problem modeling included the possibility to adopt verbal judgments and the verification of the consistency, Expert Choice incorporated intuitive graphical user interfaces, automatic calculation of priorities and inconsistencies, as well as several ways to process a sensitivity analysis. It has to be pointed out that, beside the traditional application of AHP and supporting software, a new trend in use, namely combining others methods, e.g., neural networks, SWOT analysis, and others, is emerging, as AHP is still part of certain theoretical discussions, resulting from the limitation due to assumed criteria independence and hierarchy problems due to appropriate judgment scale.
- Specifically explained in Section 6.