Evaluating Management Effectiveness of National Parks as a Contribution to Good Governance and Social Learning

1. Introduction

From the perspective of sustainability, protected areas (PA) should serve several aims such as biodiversity conservation, regional economic development, social inclusion and sharing of benefits from conservation as put forward by the Convention in Biological Diversity (CBD; see Secretariat of the CBD, 2005). As such, protected areas certainly contribute to a sustainable development from various perspectives (Getzner and Jungmeier, 2012). However, as several analyses show, many protected areas can be considered as “paper parks” (Brandon et al., 1998; Bruner et al., 2001) for which no effective regulatory and management system is in place.

The ineffectiveness of “paper parks” may also constitute a reason why studies also found no significant difference in management effectiveness of land inside and outside of parks (Hayes, 2006).

“Paper parks” not only appear in developing countries (cf. Bonham et al., 2009) but are also regular in industrialized high-income countries. For instance, the European Union’s Natura 2000 network of protected areas according to the FFH (flora-fauna-habitat directive; European Council, 1992) has been estimated to require at least about EUR 5.8bn per year to be efficiently and effectively managed to fulfill its aims and objectives (Gantioler et al., 2010). Incomplete implementation frameworks, “bad governance”, and the lack of resources are most prominent in Natura 2000 regions, especially in Central and Eastern Europe (cf. Kirchmeir et al., 2012).

For fulfilling the aims and objectives of protected areas, it is of great importance that property rights are well defined, the legal, institutional, and managerial frameworks are in place, and sufficient resources are provided for funding efficient and effective protected area management bodies. In comparison to the status quo, even some basic funding might significantly increase biodiversity conservation in “paper parks” (Joppa et al., 2008). Evaluation and monitoring tools generally provide the basis for assessing these frameworks both in terms of efficiency (e.g. wise use of resources relative to outputs and results), effectiveness (e.g. achievement of ecological objectives), and social and distribution issues (e.g. benefit sharing) (see Pomeroy et al., 2005) with the aim to improving adaptive multi-dimensional management. However, merely assessing management effectiveness by applying evaluation and/or monitoring tools, while necessary, might not be sufficient for achieving the protected area’s objectives. For increasing PA management effectiveness, the involvement and the contributions of several stakeholder groups might be crucial. For instance, involving even visitors and local residents can contribute to the effectiveness of ecological management since illegal behavior is reduced by increasing the understanding of and support for ecosystem management (Powell and Vagias, 2010). In this context, evaluating management effectiveness in protected areas might not only serve as an information and management tool for the PA management authority but also as an instrument for informing stakeholders and collecting tacit knowledge. Such instruments and methods thus also may contribute to “good governance” of protected areas, and lead to social learning experiences of all stakeholders. Some broad acceptance of stakeholders is thus considered to constitute an important pillar for effective (adaptive and integrative) PA management.

The current chapter therefore aims at discussing the role and function of evaluating management effectiveness as an integral part of the whole set of management tools and strategies of the PA management authority, as an important ingredient of the governance system of protected areas, and as a tool for social learning and inclusion. As a case study, we use an evaluation tool that has formerly been developed by the Nature Conservancy (2004) (Parks in Peril Site Consolidation Scorecard). This tool was transformed and adapted to the European context by Pfleger (2007a and 2008) and renamed as “European Site Consolidation Scorecard” (ESCS, Pfleger, 2007b). The most important characteristics, described in more detail below, are the strategic and long-term assessment of relevant management plans, instruments and resources, the selective inclusion of several stakeholder groups internal and external to the protected area, and the built-in feed-back and commentary sections in the evaluation report. As an example for applying the ESCS to a concrete protected area, we chose the Gesäuse National Park (Austria), established in 2002 and evaluated in 2007/2008.

The structure of the chapter is the following: Section 2 presents a brief review of the international debate concerning the application of diverse PA management effectiveness evaluation frameworks, and discusses selected aspects of evaluation in PAs. Section 3 describes the linkages between the ecological system, the economy, PA management, and the function of evaluations such as the ESCS. Section 4 presents the effectiveness indicators of the European Site Consolidation Scorecard (ESCS) and discusses the dimensions of evaluation along the empirical case study of the Gesäuse National Park (Austria). Section 5 finally discusses the importance of evaluation instruments, specifically the ESCS, for good governance and social learning, summarizes the results, and concludes with a range of further recommendations for improving the existing evaluation and monitoring frameworks.

2. Overview of frameworks for evaluating management effectiveness in protected areas

Since about the last 20 years, a broad range of frameworks (tools) for measuring management effectiveness of protected areas have been developed. Hockings et al. (2006, 1) define the evaluation of management effectiveness as “the assessment of how well protected areas are being managed – primarily the extent to which management is protecting values and achieving goals and objectives.”

Evaluations in general may fulfill several aims and objectives in modern societies. It is not only necessary to evaluate products (outputs) but to assess also outcomes in terms of legitimacy of public activities, and to provide steering mechanisms for public decision-makers (cf. Stockmann, 2007; Diller, 2008).

Evaluation management effectiveness of protected areas is thus discussed by Hockings et al. (2006, 1) in three dimensions;

1. “design issues relating to both individual protected areas and protected area systems;

2. adequacy and appropriateness of management systems and processes; and

3. delivery of protected area objectives including the conservation of value.”

An ideal assessment methodology and framework tries to account for the whole evaluation/management cycle (PAME, Protected Areas Management Effectiveness) with the elements of “context”, “planning”, “inputs”, “processes”, “outputs”, and “outcomes”.

Effectiveness of PA management in conserving biodiversity thus addresses basically the following two questions (Chape et al., 2005, 443):

1. “Effectiveness of coverage: how much and what biodiversity is included within protected areas?

2. Effectiveness in achieving conservation objectives: are protected areas being managed effectively?”

Leverington et al. (2010a) describe about 40 different approaches to evaluation, ranging from rapid assessments to in-depth monitoring tools. A most recent overview of management effectiveness assessment tools lists about 50 different methodologies (WDPA, 2012), of which the most widely internationally used are the following (Leverington et al., 2010b; Stoll-Kleemann, 2010; see Table 1):

1. RAPPAM (Rapid Assessment and Prioritization of Protected Area Management) was developed by WWF and has widely been used for a quick assessment of strengths and weaknesses of PA management, and PA systems and networks. By allowing for comparisons between protected areas, the methodology can also be used for setting policy priorities both of governments and NGOs.

2. METT (Management Effectiveness Tracking Tool) was developed to quickly assess management effectiveness at the site level using scorecards. METT is internationally used by WWF, the World Bank, and GEF (Global Environment Facility).

3. The US-based Nature Conservancy (TNC) developed a specific assessment tool for its PiP (Parks in Peril) program, the Parks in Peril Site Consolidation Scorecard (The Nature Conservancy, 2004), to explore the existing and needed resources for effective management of parks that may only exist “on paper”.

The tools provided or promoted by big international institutions have become standards even if some other solutions may also have their advantages or justifications (e.g. IPAM-toolbox [www.ipam.info], or the ESCS as presented in this paper).

The European Site Consolidation Scorecard (ESCS) was developed on the basis of the TNC-PiP assessment methodology and may be described as a comprehensive assessment of a specific park and its long-term policies by means of a range of indicators measured and described along a five-point scorecard (Pfleger, 2007b and 2010). Table 2 presents an overview of the indicators of the ESCS to be measured during the evaluation process.

Methodology	Abbreviation	No. of assessment with the methodology
Rapid Assessment and Prioritization of Protected Area Management	RAPPAM	939
Management Effectiveness Tracking Tool	METT	865
New South Wales State of Parks (Australia)	NSW SOP	639
Monitoring Important Bird Areas	BirdLife	506
PROARCA/CAPAS scorecard evaluation	PROARCA/CAPAS	483
TNC Parks in Peril Site Consolidation Scorecard	PiP SCSC	300
Victoria’s State of Parks	Victorian SOP	102
AEMAPPS: PAME with Social Participation-Colombia	AEMAPPS	89
Parks profiles	Parks profiles	62

Table 1.

Recent assessments of protected areas by the most prominent management effectiveness evaluation tools

No.	Indicator
A.	Strategic planning
A.1	Project area zoning
A.2	Site‐based long‐term management plan
A.3	Science and information needs assessment for project area
A.4	Monitoring plan development and implementation for project area
B.	Basic protection activities
B.1	Physical infrastructure for project area
B.2	On‐site personnel
B.3	Training Plan for On‐site Personnel
B.4	Land tenure and land use issues within the project area
B.5	Threats analysis for the project area
B.6	Official declaration of protected area status for the project area
B.7	Organisational structure
C.	Long‐term financing
C.1	Long‐term financial plan for sites in the project area
D.	Site constituency
D.1	Broad‐based management committee/technical advisory committee for project area
D.2	Institutional Leadership for the project area
D.3	Common Leadership for the project area
D.4	Community involvement in compatible resource use at the project area
D.5	Stakeholder and Constituency Support for Project Area
D.6	Policy agenda development at national/regional/local levels for project area
D.7	Communication plans for the project area
D.8	Environmental education plans for the project area
D.9	Cooperation with other organizations
D.10	Integration in an ecological network

Table 2.

Summary of scorecard indicators of the European Site Consolidation Scorecard (ESCS)

Deciding on the “right” evaluation tool, of course, has to be done in close reference to the aims of the evaluation, and also to the efforts necessary to achieve the evaluation objectives. Comparing different evaluation methods may thus be done according to the benefit-cost ratio of such instruments. Figure 1 presents a rough picture of the benefits of the ESCS in relation to the efforts (costs) involved in comparison to two prominent evaluation tools mentioned above (RAPPAM, METT). While RAPPAM is considered to be a fast evaluation providing a quick picture of PA effectiveness, METT goes a little bit more into detail. The ESCS is an evaluation tool going into much more detail. However, depending on the requirements of a specific evaluation (e.g. self-assessment vs. in-depth external evaluation), its application range is very flexible and so are the necessary efforts.

Figure 1.

Cost-benefit ratios of different evaluation tools

Management effectiveness evaluation is generally considered to constitute an important ingredient and pre-requisite for adaptive management since the changing ecological, social and economic environments call for flexible management frameworks. Effective management in this sense has to focus on adaption and learning in order to hold the developing PA on a sustainable path. Thus, evaluation not only measures results (outputs) but relates outcomes to resource inputs (e.g. budgets) and also may form the basis of a monitoring system. In addition, management effectiveness instruments provide an overview of different dimensions of protected areas (ecological, social) which are not considered in mere economic valuation methods. Figure 2 presents the “management effectiveness triangle”. Throughout the phases of management (the life-cycle of the protected area), certain measures and policies can be implemented. The ESCS concentrates on contexts to processes (the first four phases of management) since these are the most important to achieve respective outputs and outcomes (results). The figure also demonstrates that the decision on effective frameworks and processes is of utmost importance for future results. Policies that might try to change outputs and outcomes only have little impacts if contexts or processes are ineffective and inefficient.

Figure 2.

Long-term impacts of management decisions and room for improvements in a lock-in situation (management effectiveness triangle)

Regarding “paper parks” described above, there are examples of such parks that do not lack financial resource (e.g. international funding) but are designated without further activities. Evaluating effectiveness thus provides also a firm basis for discussing the financial inputs and their efficiency.

3. Linking PA effectiveness evaluation with an economic-ecological model of PA management, and governance and social learning

Evaluation of management effectiveness may be an important, even essential part in the management cycle of a protected area in particular with a focus on adaptive management strategies (Getzner et al., 2010). However, the current paper not only presents details of an evaluation of a national park (see section 4) but also extends the perspective of evaluation with respect to governance and social learning. Before discussing these aspects along the empirical case-study, we want to address evaluation from a systemic viewpoint linking evaluation with an economic-ecological model in a conceptual perspective.

Figure 3 presents such a conceptual model of the linkages between evaluation and monitoring by means of the European Site Consolidation Scorecard (ESCS) and the whole governance and financing system of a protected area. An ecological model of the ecosystem lies at the center of the conceptual framework describing the interdependencies between species (1), the habitats (2), and the abiotic features of the site. For simplicity, we assume that the species is an endemic animal species that may be observed in the habitat but is closely reliant on the quality and the size of the habitat since it cannot easily migrate to other habitats. In this ecological model (cf. Behrens et al., 2009), the species uses the habitat for food and reproduction, and underlies its own dynamics in terms of reproduction and decline which may seasonally change due to changing environments (provision of nutrients, water). This includes that the species is in competition with other species, and might also be a prey for natural enemies. We assume that protecting the animal species is crucial from a nature conservation perspective (e.g. endemic, or protected and listed in the species “red list”), and may easily be negatively affected by disturbing visitors. PA management may try to improve the reproduction of the species e.g. by temporal or permanent access bans to breeding grounds. The habitat itself has its own dynamics in terms of natural growth (including the abiotic elements) for which we assume that there exists a natural maximum size of the habitat in a certain area. The habitat may be impacted by visitors and their damages, and, again, by visitor management policies of the PA management.

Figure 3.

Linking evaluation and monitoring according to the European Site Consolidation Scorecard (ESCS) with the protected area governance and financing system

The second part of the conceptual model may be labeled “economic” model since it refers to the economic use of the habitat in terms of demand for visits to the area (3). Visitors may be attracted to the area by a mixture of landscape (habitat) and species attributes (e.g. observation of wildlife). In addition, PA management can influence demand by providing or not providing (de-marketing) respective visitor infrastructure and facilities, and by education and information policies.

PA management, described by box (4), can steer (manage) the ecological, social and economic systems by a range of management policies such as ecological management for the habitat and species, and by aiming at the wide range of PA objectives such as visitor management and infrastructure provision, by information and education, and by production of scientific knowledge. Depending on the strength of the different objectives, PA management may have a certain objective function to be fulfilled (e.g. maximizing visitor benefits while sustaining a viable balance in the ecological system). In addition, PA management may also include the provision of non-use values (e.g. existence value of species/habitats) into its objective function.

The ecosystem, visitors, and the PA management – the latter including the whole range of property rights, relevant policies, decisions bodies and authorities – constitute the governance system of the protected area. Stakeholders (5) may partially be part of the governance system, for instance, by their inclusion in decision bodies of the protected area. However, we assume that the general public is outside the governance system.

The establishment and effective and efficient management of protected areas consumes resources, i.e., a range of costs have to be financed. On the one hand, the designation of a certain plot of land to be protected defines and changes property rights, and thus also alters the economic use of the area

In the case of a national park according to IUCN’s category II, all consumptive (extractive) uses of the area are prohibited.

. The costs associated with the designation of a protected area are therefore the opportunity costs (foregone benefits of using the area in an alternative way). In addition, “out-of-pocket” expenses such as personnel and equipment costs, investments for infrastructure, and other operating costs, have to be financed.

The efficiency and effectiveness of PA management, together with information provided, leads to a certain public support for the protected area in terms of willingness-to-pay for the financing the park (6). Willingness-to-pay may be measured by expenses of visitors (e.g. travel costs), and by the whole range of non-use values (e.g. existence values) of the general public. A positive attitude towards the protected area may also lead to a different regional economic development (7) which also may be a source of funding for PA management. Altogether, public authorities and private bodies (households, companies) pay for the establishment and operation of the protected area in terms of a range of contributions (e.g. taxes and fees, entry fees, sponsoring, donations) (8).

For understanding the implications of this conceptual model, it is important to consider that efficiency and effectiveness of PA management is paramount for securing funding. One the one hand, the private sector is only willing to pay in terms of fees, travel costs, regional labeled products, etc., if PA management is effective and also sufficiently informs the general public. On the other hand, for securing public funds, national tax payers have to be convinced to provide sufficient financing.

Evaluation and current monitoring instruments thus may play a vital role in this conceptual model of managing and financing protected areas. The grey arrows and the letters (A to D) refer to the grouping of indicators of the European Site Consolidation Scorecard (ESCS) listed and described above (see Table 2). The different PA management policies are evaluated by different indicators, such as ecological management, visitor management, and education and information policies. However, as the ESCS is not only a tool for assessment based on data and statistics, the inclusion of several stakeholder groups is of specific importance. Stakeholder groups such as local decision-makers, experts, visitors, and PA business partners, in this context provide, first of all, information for attributing “correct” values to the indicators. Second, as stakeholders are involved by means of talks, discussions, and group work, the different perspectives become apparent. Third, the ESCS is also an information instrument by itself. By communicating the evaluation process and the results, the general public is informed about the use of public (and private) funds, and about the effectiveness of PA management.

In this conceptual model, evaluation thus becomes part of the governance system as an instrument of stakeholder involvement (participation), as well as an instrument of information. Furthermore, as the empirical case study will demonstrate, the ESCS process is a contribution to social learning and to “good governance”. In general, accounting for stakeholder participation rests on the normative assumption that participatory frameworks lead to more efficient and effective park management. However, as governance systems including regulatory frameworks are different between regions and between parks, it is not straight forward to clarify the impacts of more or less participation. Hayes (2006) discusses this issue by presenting evidence both in favor and against participation of stakeholders by stressing that the “traditional” model of PA planning has recently regained more weight in effective park management. Schultz et al. (2011) argue in the same direction by cautiously describing how park management effectiveness might gain from participation of certain stakeholder groups (such as scientists, volunteers, and local inhabitants). The current chapter does not judge upon this question but rather shows that evaluation exercises might benefit from certain and well-defined instruments for stakeholder inclusion. As Raymond et al. (2010) point out, it is of specific importance in which frameworks local knowledge is collected and used for environmental management because evaluating management effectiveness might partially rely on tacit (local) knowledge for which stakeholder involvement is crucial.

Social learning in the context of (collaborative) resource management is defined as “learning that occurs when people engage one another, sharing diverse perspectives and experiences to develop a common framework of understanding and basis for joint action” (Schusler et al., 2003, 312). This general understanding of social learning emphasizes that – while the “learning unit” is the individual – learning is done through observation of or interaction with others in the social context, for instance, in a process of deliberation which “includes any process to communicate, raise and collectively consider issues, increase understanding, and arrive at substantive decisions” (Schusler et al., 2003, 312). Furthermore, social learning includes “learning by social aggregates, learning pertaining to social issues, and learning that results in recognizable social entities such as collective decision making procedures” (Maarleveld and Dangbégnon, 1999, 268). Even simple face-to-face communication has already proved to reduce resource consumption in experimental settings (Anderies et al., 2011), thus providing a fruitful starting point for social learning perspectives. Borowski et al. (2008) present a conceptual framework regarding the perspective of social learning in participatory resource management. They find that social learning may take place at many places in the governance and management processes (such as context, inputs, processes, outputs, outcomes) and provide indicators for the extent and effectiveness of social learning (see also Cheng et al., 2011). Consequently, social learning is supported by a range of different participatory instruments (e.g. information, stakeholder forums). As evaluation exercises provide several indicators for the efficiency and effectiveness of PA management, involvement of stakeholders in the evaluation process thus may also account for different forms of participation, from face-to-face discussions to workshops, and to comments and written critique in the evaluation report.

The ESCS methodology rests on a specific structure of indicators, and recommends a certain process how the information on each of the indicators is collected, discussed with stakeholders, condensed and assessed, and summarized to a certain score value. Insofar, evaluation does not seem to provide a major contribution to social learning allowing for extensive deliberative processes. While the ESCS may not be able to make up for missing deliberation and participation in the governance structure of the PA management – the lack of such participatory structures should become apparent by the evaluation itself – the evaluation process might also constitute an element of deliberation due to the inclusive nature of the indicator assessment, with the potential aim to encourage “social partnerships” among stakeholders and the park management (Benn, 2010). As such, an evaluation of management effectiveness in parks (e.g. by means of the ESCS) may also form one part of a governance structure of a protected area. Governance in the context of protected areas is usually defined as the “interactions among structures, processes and traditions that determine how power is exercised, how decisions are taken, and how citizens or other stakeholders have their say” (Graham et al., 2003, 2f.). Decision-making rules, including responsibility and accountability, are clearly part of the governance system of protected areas. While the leeway within an evaluation exercise is certainly limited since evaluation has to be based on a certain methodology and structure which cannot be changed during the evaluation process without loss of credibility and focus, the applied evaluation instruments allow for different extents of participation and deliberation.

Whether evaluation instruments in principle contribute to “good” governance is an additional question. The positive question of research concerning the role of evaluation instruments in the governance system would be generally answered by assessing how the governance framework is set up (system of rules, regulatory structure; cf. Mayntz, 2005; Ostrom, 1990), what the roles the different stakeholders have, and how the system works. The normative question refers to the quality of governance and potential recommendations for improving the frameworks. As Getzner et al. (2012) write, “good” governance might “crucially influence whether the protected area can achieve its objectives, is able to fairly share benefits and costs, and seeks and gains sufficient support from local communities and stakeholders” (cf. Lockwood, 2010).

Building up social capital by learning due to appropriate stakeholder involvement is nevertheless one important ingredient in participatory resource management (Enengel et al., 2011).

4. Evaluating management effectiveness at Gesäuse national park (Austria)

4.1. Choice of methodology and case study area

Based on a feasibility studie (Jungmeier & Velik, 1999) and an intensive regional public debate the Gesäuse national park was established in the Austrian federal state of Styria in 2002 as a national park conforming to IUCN’s category II of protected areas. Only in 2003, the national park was officially acknowledged by IUCN to conform to category II. The landscape is alpine with deep gorges formed by the Enns river and its contributing streams with small patches of high-diversity wetlands adjacent to the river. The area of the national park amounts to around 11,000 hectares of which 86% are core zones (the rest consists of buffer zones and sustainable pastures). The park stretches from around 500 to over 2,300 meters above sea level, with about 63% of the area consisting of forests, 30% high alpine areas, and the rest water bodies and pastures. The land is almost entirely owned by the public (federal state of Styria). Four rural municipalities form the core of the “Gesäuse national park region”. The national park administration is also responsible for managing the area according to the European Union’s Natura 2000 network.

The park was established in 2002, based on an agreement between the federal state of Styria and the Republic of Austria (Ministry of the Environment) sharing the management and financial responsibilities. While this agreement – codified in a regional law – comprehensively deals with all aspects including IUCN management criteria, it also provides the legal basis for regularly evaluating management effectiveness of the park. According to the law, comprehensive evaluations have to take place once every five years with the aim to assess management effectiveness in all dimensions of the park’s organizational and managerial fields, such as the organization, financial and activities planning, ecological management, visitor policies, and scientific research.

The current evaluation project was commissioned in 2007 by the national park authority to an interdisciplinary group consisting of M. Jungmeier (ecologist and nature conservation planner for over 20 years), B. Pfleger (environmental engineer and conservation planner/manager), W. Scherzinger (professor of ecology and nature conservation), and M. Getzner (ecological economist). The framework of evaluation was set up by the European Site Consolidation Scorecard (ESCS) with the explicit recommendation to consult all relevant stakeholders during the evaluation process, and the request by the park management to present a very detailed assessment of the park.

The ESCS evaluates the effectiveness of management in a broad perspective, addressing all relevant issues of a park’s management, and measuring the success or failure by means of detailed scores for each indicator. A brief description of the methodology can be found in Pfleger, 2010. Table 2 above presents an overview of the indicators of this evaluation tool.

Each indicator is evaluated according to a pre-defined scale; for instance, the indicators measure whether the financial basis of the park is sustainable and secured for the future, and whether effective management plans are developed and implemented. Five different levels of implementation and effectiveness are described, explained, and can be selected. In addition, management tools, references and best practice examples for each indicator are included in the description of the ESCS framework and instrument. Finally, there is a comprehensive documentation section for recording information, and, among others, for verifying the results and helping to implement necessary measures and policies.

The measurement of indicators along a scorecard is probably not the only significant contribution of the ESCS evaluation tool. While the assigning of certain scores to the indicators was done by the research team, the concrete scores for the indicators are not the most relevant element of the evaluation process. The first step of the evaluation was to collect a comprehensive informational basis for assessing management effectiveness which was done by contacting all relevant stakeholders; at the beginning, information was, of course, collected with the national park authorities including the management, and the local nature conservation authorities. The informational basis was, as objectively as possible, described and condensed for each indicator. The second step was to collect stakeholders’ opinions regarding the current state of each indicator. For some indicators, different stakeholder groups presented their assessment of the current situation which was documented in the ESCS report, too. The third working step included an assessment from the viewpoint of the expert (evaluation) team, and the final (fourth) section of each indicator assessment included a comprehensive list of recommendations for further improving management effectiveness.

It is of high importance to note that the evaluation of management effectiveness by means of evaluation tools, especially in the case of the ESCS, is concentrated towards the long-term development of the park. In the case of the Gesäuse national park, the evaluation of the long-term sustainability of current policies, including the frameworks mentioned above, is particularly significant. For instance, current policies in transforming the composition of tree species in the diverse forests of the park can only be assessed in the far future due to the slow development of forests towards a natural composition and ecological balance. As national parks secure and partially (re-) introduce natural dynamics to the ecosystems, these processes are inherently stretching far into the future. A basic assessment such as the ESCS therefore cannot, and does not want to, assess management effectiveness in the sense of measuring the current benefits of policies (outputs and outcomes), as this would often require long-term monitoring and research. Instead, the ESCS has the aim to review current policies with the perspective of the sustainability and future achievements of PA authorities. Based on the assumption, that changes in the first phases of the management cycle (see Figure 2) have far more significant impacts on the effectiveness of a protected area than improvements in the latter ones, the ESCS focuses on the first four phases (context, planning, inputs, processes), rather than on concrete outcomes. The last phases are indirectly addressed by the respective indicators for comprehensive monitoring which measures the objectives of the park.

As mentioned above, a particularly strong focus of the ESCS was the involvement of local and regional stakeholders. At the time of evaluating the park, park policies were subject to many emotionalized debates in the region. The need to identify and integrate critical perceptions into the evaluation was obvious, but a clear distinction between opinions and facts was considerably difficult. To secure the credibility of the process and the whole evaluation efforts, the discussion with stakeholders was structured according to the “Regional Timeline Method” (Jungmeier et al., 2010). Based on individual events and incidents, stakeholders created a joint emotional timeline for particular periods from the inauguration of the park until the evaluation exercise. This method thus complemented the evaluation process by providing “screenshots” of stakeholders’ views.

4.2. Results of the evaluation

As mentioned before, evaluating the Gesäuse national park is included in the governance framework of the park which is itself codified in the regional national park law. The evaluation was carried out based on international standards with reference to the existing evaluation frameworks while applying a newly developed, participatory evaluation methodology called “European Site Consolidation Scorecard” (ESCS). It is safe to say that this evaluation exercise has been by far the most in-depth evaluation for Austrian parks to date. The analysis was based on all important documents, personal talks, several workshops in the region, and an assessment of the regional economic impacts. The evaluation report summed up the results in the following way. The Gesäuse national park management can be described as “exceptionally positive. The foundations that are laid down, the developed structures in the park, processes and activities certainly compare favorably in the international context. Strategic decisions of the park management are well argued, the documentation is comprehensive, and the overall direction of the park is sustainable.” (Getzner et al., 2008; translation from German).

In particular, the following fields and projects of the national park management are exceptional:

1. Innovative and high-quality, partially unique offers for visitors regarding education and information (visitor center, exhibitions, events);

2. Comprehensive and systematic public relations work within and outside the region;

3. Highly qualified, motivated and effective team members and team work;

4. Comprehensive basic ecological inventory, excellent scientific research and planning work;

5. First already verifiable impacts on the regional economy due to increases in visitor numbers.

On the other hand, the evaluation highlighted some aspects that might need further concentration of resources:

1. Reduction of the barrier effect of roads, rail tracks, forestry and provisioning paths;

2. Assurance of financial sustainability by valorization of public budgets based on the consumer price index;

3. Further development of the legal frameworks to stringently apply IUCN criteria;

4. Improvement of participatory processes for stakeholders and vested interests inside and outside the region;

5. Development of a comprehensive and inclusive management plan with the park’s vision, objectives, and implementation strategies;

6. Improvement of collaboration between the land owner (Styrian Forests, a publicly owned company) and the national park authority;

7. Improvement of the boundaries of the national park in the light of ecological dynamics.

All in all, the existing evaluation of the Gesäuse national park has certified that the park’s management has decided upon sustainable strategies for improving and securing the natural dynamics in the ecosystems while providing excellent infrastructure for visitors and information for stakeholders and the general public.

5. Discussion, summary and conclusions

During the life-cycle of a protected area (PA), the evaluation of management effectiveness of the PA administrating and managing authorities including the whole PA context becomes increasingly important, both for securing and improving the conservation of biodiversity, but also for the acceptance of stakeholders and funding bodies. Currently, there are numerous approaches to evaluating management effectiveness of parks; many international institutions have drafted and implemented such evaluation instruments.

The current paper has focused on a specific evaluation instrument that emphasizes the process of evaluation from the perspective of a range of important stakeholders. The “European Site Consolidation Scorecard” (ESCS) is an evaluating instrument with four large groups of indicators dealing with strategic planning, conservation activities and results, long-term financing, and site constituency. The instrument thus has become part of the governance system of the park contributing to social learning both of the PA staff as well as local stakeholders in manifold dimensions.

At the beginning of any evaluation exercise, PA management and policy makers have to be convinced of the usefulness of the evaluation to increase the probability that recommendations will indeed be implemented (Pfleger, 2008). The implementation of the evaluation results is crucial since the evaluation might lead to higher costs than benefits since the frustration of those involved in the evaluation process might be significant (Hockings et al., 2006). However, the lack of implementing evaluation results is, at the moment, one of the biggest problems in assessing protected area management effectiveness. Evaluations are carried out, but recommendations are not implemented in daily management (Steindlegger, 2007).

In general, external evaluation is important because it addresses “uncomfortable” topics such as the organizational structure, enhances credibility of park management, and forms the basis for policy changes. However, it is useless without sufficient discussion and participation of the PA management because the administration of the protected area has the responsibility to implement the recommendations. Without understanding evaluation results in a respective social learning environment, PA management will not follow recommendations and adapt management policies.

Integrating stakeholders’ opinions and viewpoints is necessary especially for collecting information about PA policies and impacts. In addition, it is of high importance to present stakeholders with the opportunity to express their opinions with the aim to making PA management more effective, and to increase the acceptance and support of the evaluation results. Suitable instruments may be workshops, and personal talks and interviews as an addition to the existing governance framework. However, once stakeholders are involved, their opinion has to be accounted for in a transparent way. In addition, evaluation results have to be made available for stakeholders (as well as the general public) to receive a sufficient acceptance of evaluation results. Publicizing evaluation results has to be done in a sensitive way because presenting bare scorecard figures to the general public may lead to misinterpretations.

As the European Site Consolidation Scorecard (ESCS) focuses on the first four phases of the management cycle and not on outputs and outcomes, the evaluation method cannot assess the efficiency of PA management in terms of relating outcomes to the costs of the park. The ESCS thus has to be complemented by long-term monitoring methods (however, the ESCS indirectly addresses long-term developments and outcomes by including an indicator assessing the PA’s monitoring plan). During the detailed ESCS evaluation process in the Gesäuse national park some crucial elements of effective PA management such as the missing management plan or insufficient stakeholder involvement, were soon obvious. As a methodological conclusion of our evaluation study, it is not necessary to employ in-depth evaluation tools for discovering the most obvious weaknesses of PA management which might be evident by using a simple scorecard with some stakeholder involvement. Thus, a first quick evaluation might exhibit crucial issues with relative low efforts if the evaluation concentrates on the first four management phases. If results should be more comprehensive, and provide more detailed recommendations, costly evaluation and long-term monitoring tools are necessary.

Whether protected area management effectiveness evaluations really contribute to good governance and social learning depends less on the methodology chosen (whether e.g. it is a rapid assessment or an in-depth evaluation). Instead, the evaluation process itself is of much more importance. A “secret” self-assessment done by the protected area management body will hardly be useful whereas an integrated approach that also includes external experts as well as relevant stakeholders, and widely publishes the results, brings far more benefits than just management recommendations.

The debate on appropriate indicators, evaluation tools, scorecards and checklists, has intensified in the last decade but remains at a rather technical level. However, any learning and progress in and about protected areas management is clearly connected to the intensity of the public debate provoked by the evaluation process. The evaluation of the Gesäuse national park as part of the park’s governance had mainly three results. First, a written evaluation report including the scorecard and the recommended policies; second, a broad range of personal viewpoints and findings of over 50 staff members and stakeholders directly involved in the evaluation exercise in the sense of social learning experiences; third, a broader understanding of the PA staff for the principles of adaptive management that may quickly respond to changing needs and requirements. Summing up, the authors of the current study consider the first result – while the only “tangible” output of the evaluation – to be the least important one.

Acknowledgement

We are thankful to the editor and the reviewers for their comments and suggestions. In addition, we would like to thank W. Scherzinger, W. Franek, and the Gesäuse national park staff, for their collaboration and support during the evaluation of the national park. All errors are, of course, the responsibility of the authors.