Analysis of the Current German Benchmarking Approach and Its Extension with Efficiency Analysis Techniques

A first similarity in most European countries is that local governments are the responsible bodies for providing water services. They can decide if they want to perform the service themselves or if they contract it out to private companies. Figure 1 shows that they predominantly transfer the task to publicly owned companies. It is worth noting that Figure 1 displays the percentage of population served by either a public, a private or a mixed operator. If it would show the number of companies then the percentage of private companies in relation to all would diminish drastically. The same holds true if the term “privatization” would be specified more clearly. In Germany, for example, it ought to be distinguished if a company is only formally privatized, which means that the shareholders remain public, or if a company is really materially privatised.


Introduction
The analysis of the current German Benchmarking approach and an extension with efficiency analysis techniques seems to be a very specific topic at first sight. However, at second sight it turns out that this question is relevant for many European countries. The reason is that many countries share a similar structure of their water sector which then implies similar challenges.
A first similarity in most European countries is that local governments are the responsible bodies for providing water services. They can decide if they want to perform the service themselves or if they contract it out to private companies. Figure 1 shows that they predominantly transfer the task to publicly owned companies. It is worth noting that Figure  1 displays the percentage of population served by either a public, a private or a mixed operator. If it would show the number of companies then the percentage of private companies in relation to all would diminish drastically. The same holds true if the term "privatization" would be specified more clearly. In Germany, for example, it ought to be distinguished if a company is only formally privatized, which means that the shareholders remain public, or if a company is really materially privatised.
A second observation for the various European countries is that the water sector is very fragmented. 1 EUREAU (2009, p. 94) is counting 600,000 jobs for more than 70,000 water services operators. On average a water service provider would employ less than 10 people. The structure is thus mainly publicly organized and rather fragmented.
At the same time, tariffs are now supposed to cover all costs. 2 In addition, immense investments will be needed in many European countries to fulfill the various European Directives. 3 Both developments will lead to significantly higher water prices in the future. It can be expected that the public in many European countries will hold the companies accountable to show that they are performing efficiently.  (EUREAU, 2009, p. 93) The provision of water supply is a natural monopoly which implies that companies are not sanctioned if they are inefficient. A water utility regulator and the economic regulation of water supply and wastewater companies is thus an important issue which is discussed internationally. 4 The problem, however, is that in countries with a very fragmented structure of the industry a complex economic regulatory framework, like in England & Wales or Scotland, is not applicable for the majority of providers. At the same time, privatisation is not always worth considering: a precondition for a successful privatisation is that public authorities have the knowledge and the data to supervise the private service provider. Otherwise, a public monopoly is only transferred into a private one (Newberry, 2003, p.4). Many European countries are thus considering a third approach: benchmarking. Such a benchmarking system compares companies with one another according to certain indicators. It generally serves two purposes: First, it shall be a measure to increase transparency in the sector. Displaying reports which are publicly available are supposed to enhance accountability of companies. Second, performance benchmarking systems evolve which analyze certain processes within the company in more detail and give, therefore, insight to companies where they could enhance their efficiency. 5 The main question for this paper is to analyze the potential of benchmarking. We will use the German experiences with the current approach and will answer the question if the current system can be enhanced by applying efficiency analysis techniques. Can we expect that such an enhanced benchmarking will imply that a regulator is redundant?
The remainder of this article is structured as follows. In the following, second, section we will briefly present the concept of benchmarking as well as the current use of benchmarking in Germany. Its deficiencies imply the need, in the third section, to portray alternative 4 International regulatory approaches for water and wastewater services are portrayed in Marques (2010). 5 For a short portray of European benchmarking approaches see Marques/De Witte (2007). techniques like Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis (SFA) as means to better analyze data. In the fourth section we introduce the employed database. Section five displays the best models to explain cost differences for small, middle and large water service providers in the distribution of water in Germany -something which has never been done for the German water supply sector. In section six we practically describe what kind of information a company, which is participating in such an enhanced benchmarking approach, can expect. The paper ends with a brief conclusion and outlook.

Current benchmarking in the German water supply sector
Benchmarking can be defined as "the process whereby a company compares and improves its performance by learning from the best in a selected group" (BDEW, 2010, p. 4). 36 of such, so called process benchmarking projects are carried out in the German water and wastewater sector (ATT et al., 2011, p. 94ff.). Parts of the value chain are analyzed in detail mainly between a limited number of companies. Up to 20 companies are participating in the various projects (ATT et al., 2011, p. 94ff.). The concept is displayed in the following figure. The process starts out with a comparison of key indicators. For each single company the deviation between its actual value and the benchmark is determined. The different factors which may explain the difference are then intensely discussed between the specialists of the companies for the particular process. Quantifiable measures which are then implemented shall diminish the gap between own value and benchmark. The relative efficiency of the company within this particular process increases. Process benchmarking is, therefore, characterized by a continuous process to learn from the best.
The 36 water and wastewater programmes have approximately 12 participants, on average. Very often the same companies take part in several projects covering different processes. For

www.intechopen.com
Current Issues of Water Management 274 the 16 water supply projects 100 different companies might participate. Compared to more than 6,000 German operators, this number is quite negligible.
The question, therefore, arose of how to activate more companies to participate. Particularly, since the Federal Government and the Bundestag have submitted its so called "modernization strategy", metric benchmarking projects increased in number. 6 The "modernization strategy" -approved by the German Parliament on the 22 nd of March 2002 -acknowledged the benchmarking concept and asked the German water associations to continue implementing them in the various Bundesländer. Benchmarking projects in the water supply sector are now performed in each of them (see Figure 3). Public reports are available for 12 Fig. 3. Metric benchmarking in German Bundesländern (BDEW, 2010, p.9) 6 A metric benchmarking system is not going into such detail as a process benchmarking system does. It merely compares companies by employing key performance indicators. The link to the German "modernization strategy", however, does not imply that metric benchmarking, as such, is a new invention. The Betriebsvergleich kommunaler Versorgungsunternehmen (Benchmarking of public water supply utilities), run by the German Water Association VKU, was first installed about 50 years ago.
out of the 16 Bundesländer (ATT et al., 2011, p. 92f.). Based on drinking water quantities 30 % (Baden-Württemberg and Bavaria) up to 100 % (city states of Berlin, Hamburg and Bremen) are covered by the projects. Based on the number of companies it is far less.
However, these kinds of metric benchmarking projects are also activating a number of additional water service providers which are not participating in the very intense performance benchmarking projects. Its main intention is to give the companies a first insight of how good they actually seem to perform. Similar to the performance benchmarking projects, discussions between the relative good and bad companies are intended to take place. The problem, however, is to distinguish a good and a bad company. The current approach shows that, where ones costs in a certain part of the value chain are solely compared with those of others without actually taking into account differences in basic conditions, benchmarking is not as efficient as it could be. Due to very different conditions for companies to deliver water services, costs can be very different between companies for good reason. A company with rather unfavorable conditions and higher costs might be more efficient than another one with more favorable conditions and lower costs.
As a result, current metric benchmarking projects seem not to fullfill the high expectations.
In nearly all of the metric benchmarking projects the number of participating companies remains either constant, over time, or diminishes (ATT et al., 2011, p. 90ff.).
Current metric benchmarking approaches should, therefore, employ techniques which are able to assess costs, taking into account the relevant environmental conditions in which the company actually operates. The Data Envelopment Analysis (DEA) and the Stochastic Frontier Analysis (SFA) are the scientifically established tools, which are giving a good indication about the relative efficiency of a company. Water suppliers which are performing badly according to both DEA and SFA -given their particular, not influenceable environmental conditions -ought to have potentials to improve efficiency. Such an enhanced benchmarking can thus improve the information a company may receive from participating in a benchmarking project.
It is worth noting that such an enhanced metric benchmarking project is better displaying the relative performance of a company. It is, however, not giving advice on how a company might increase its efficiency. In order to determine the correct measures a company might participate in a process benchmarking project, install certain working groups within its company or employ consulting companies. A metric benchmarking project is thus very often a necessity for a company to deal with its own performance relative to others. After detecting certain inefficiencies, the company should encounter incentives to install programs which help in improving their performance. Time series data of a company's performance should thus be collected.
All European countries which are employing metric benchmarking systems will, therefore, sooner or later face the necessity to decide which kind of information they want to display publically and whether companies should be obliged to participate. The Netherlands, for example, made it compulsory to take part in such programs whereas Germany is very reluctant to do so.
There are also, however, other means to give incentives to companies to participate in enhanced metric benchmarking systems. Those German water suppliers, which are setting prices, are currently under the supervision of cartel offices. Currently, these regulatory www.intechopen.com

Current Issues of Water Management 276
institutions are investigating those companies which have high prices per m³. This is particularly ridiculous because, due to very different conditions, a company with high water prices might be much more efficient than a company with low ones. An incentive for companies to participate in metric benchmarking projects could, therefore, be to either start investigations in companies which are not participating in metric benchmarking projects at all or which seem to be relatively inefficient at first sight. For other European countries it might be worth considering attaching the granting for subsidies to a successful participation in benchmarking projects.

Brief introduction into efficiency analysis techniques
Scientific efficiency and productivity analysis can be differentiated into parametric and nonparametric methods (Coelli et al., 2005). Parametric approaches, like Ordinary Least Squares (OLS) or Stochastic Frontier Analysis (SFA), estimate cost or production functions and an (in-) efficiency value per observation. Therefore, one has to specify a functional form (like log-linear, Cobb Douglas or Translog). This, indeed, leads to implicit assumptions about the underlying production technology (Jamasb and Pollitt, 2003), for instance, about factor substitution etc. A major advantage of parametric methods is that they allow for statistical inference and their robustness against outliers and statistical noise (Coelli et al, 2003). Nonparametric techniques like the Data Envelopment Analysis (DEA) rather calculate than estimate multi-input/multi-output productivities. The major advantage of Data Envelopment Analysis is its flexibility, i.e. that the analyst does not have to specify a functional form (Coelli et al, 2003). This section briefly discusses the different methods of productivity analysis. 7 The statistical method of Ordinary Least Squares (OLS) is a parametric method estimating the explanatory power of so called exogenous variables (regressors) on an endogenous variable (regressand). The parameters are estimated by minimizing the squared deviances of modeled to actual values (sum of squared residuals). A widespread application of this relatively easy method is the linear regression analysis. The central problem of the linear regression model is, however, that the deviation of one firm's value to the regression line is declared to result from relative efficiencies, which does not always have to be the case.
But, even if the linear regression analysis provides substantially better information to a firm than the average cost approach used up until now, further improvement in efficiency evaluation is in order. For "operational distribution costs", as well as for "total costs" and the other most important costs along the value chain "operational costs production and treatment", "administrative costs" and "capital costs", two additional analyses should be employed to make the linear regression results more robust when analyzed in detail.
Stochastic Frontier Analysis (SFA) is another parametric method to determine the efficiency frontier and an advancement of the OLS method in some ways. It requires assumptions about the functional form of the relationship between costs and output values. 8 Essentially, the actual costs of one firm are compared to the minimum (efficient) costs of another firm.
Here, in contrast to the linear regression model, the deviation from the optimum need not be resulting purely from inefficiencies, but also from so called "White Noise". Hence, interpreting these deviations purely as efficiency potentials may be misleading and should be avoided.
The aim of the Data Envelopment Analysis (DEA) is also to measure the efficiencies of respective firms relative to a threshold firm. The productivity of single entities is compared to an efficiency frontier, which is derived from a linear connection between efficient firms (so called "peers"). The DEA is a non-parametric method so that the efficiency frontier is not estimated empirically but calculated by a linear optimization program.
In other grid-reliant sectors (like electricity, gas, telecommunications and even water supply in other countries) the DEA and SFA methods are well established, while the linear regression model does not provide robust and consistent results.

Data set
We The sample is as close in line with the overall structure of the German water supply sector as possible. However, Figure 4 shows that the distribution, according to the size of the companies between our sample and the overall situation, differs. 30.2 % of approximately 6,400 water supply utilities (ATT et al., 2008, p. 12) in the German water sector supply more than 500,000 m³ annually. In our sample this percentage of companies, which supply more than 500,000 m³ annually, is nearly 80 %. In the whole German water supply sector 92.6 % of water output is supplied by companies with an annual water delivery of more than 500,000 m³. The figure for our sample is nearly 99 %. This implies that our sample contains relatively bigger companies than the overall German average. 9 Panel data might be interesting in the future to follow the efficiency development of a single company over time. 10 Rödl & Partner have been very cautious to crosscheck, in particular, all cost data. No inconsistencies were found. Over time however, the set of data slightly changed. Particular older observations with lacking structural variables were, therefore, removed from the data set. 11 For our calculations in the production/treatment segment we deduct the water abstraction charges. DEA and SFA for total costs imply that concession levy, water abstraction charges and compensatory payments for agriculture have to be subtracted.

Methodological approach
To achieve robust modelling results, we follow a three step approach. First, we cluster the observations with regard to their size. By that, we implicitly assume that small companies have a different production technology than larger ones. Secondly, we perform some theoretical and empirical analyses on the potential variables to develop reasonable inputoutput combinations for our latter modelling. This is then performed in the third subsection on section 5.

Clustering
Rödl & Partner, the consultancy which performs the benchmarking for several German Bundesländer and which provided the data for calculating the efficiencies, has been clustering all participating companies according to the annual accounted water. In workshops with water supply companies they agreed to form three groups. The first cluster comprises 38 companies with a water delivery of 500,000 m³ annually, the second one comprises 97 companies with water delivery between 500,000 m³ and 2,500,000 m³ and for the last one, all remaining companies with annual water delivery up to 50,000,000 m³ (61 companies).
Such a differentiation, according to the size of companies, is extremely important. Our models will later reveal that the production functions of the three different groups vary.
Thus, a data set should always contain enough observations in order to be able to form groups.

Analysis of the Current German Benchmarking
Approach and Its Extension with Efficiency Analysis Techniques 279

Variables and cost driver analysis
We assume that the objective of a water service provider is not solely to produce drinking water. The objective rather is to provide the option for clients to use as much drinking water as they wish at any time. This implies that the set-up of the network with transportation and distribution pipes, tanks, pumps, valves and service areas should also be considered as outputs, which at least in the short-run cannot be influenced by the company.
Correlation analysis provides an initial determination of the statistical relationship between costs on the one hand and potential explanatory variables reflecting the specific frameworks faced by each water supplier on the other hand. Such correlation analyses are the basis for estimating costs as a function of multiple drivers, i.e., regressors, as they help specifying the efficiency-analyzing models later on. In this step it is made sure that the exogenous variables, like outputs and cost drivers, explain the endogenous variable, costs, sufficiently. 12 Analyses revealed, particularly for the bigger companies, that the five variables of group one in Table 1 are highly correlated, both with operational distribution costs as well as with one another. Both the technical common understanding and the analysis of the empirical literature stress the explanatory power of these variables. 13 It thus makes sense to always have at least one of these variables in the DEA-or SFA-functions in a cost or production function model. Variables of group two to four were tested for additional explanatory value. Walter et al. (2010, p. 228) refer to a number of studies which display the significance of "water losses" as an explanatory variable. For countries like Brazil, Spain or Peru this might certainly be of importance due to high variations in the quality of the network. For a country like Germany however, where the level of water losses is only about 6.5 % (ATT et al., 2011, p. 56) on average, 14 water losses cannot serve as a good proxy for the quality of the network or associated operational costs, respectively.
The two variables "downturn of demand since 1992" and "downturn of demand since 1998" are surely interesting for explaining the development of prices. Many companies, which face a significant decrease of demand due to various reasons, need to increase prices if they lack the appropriate tariff models. Too often only a minor share of the total fixed costs is actually covered by earnings, which are independent from actual demand. However, for a cost benchmarking -particularly the operational distributional costs -these variables are insignificant.
Whereas all variables of the fourth group were not taken into account any longer, the variables of the third group were tested in DEA-and SFA-functions, where a certain combination of variables made sense from a technical water perspective. Particularly, the client structure ("Household supply relative to accounted water") is quite often used to explain differences in both operational distribution costs as well as total costs. We, however, 12 Tests for heteroskedasticity (Breusch-Pagan/Cook-Weisberg test) and multicollinearity (Variable Inflation Factor, VIF) have been applied to fulfill general conditions of multivariate regression analysis and specifically Ordinary Least Squares conditions. 13 Besides the literature discussed in Walter et al. (2010) also see Lin (2005), Picazo-Tadeo et al. (2009) and Coelli & Walding (2006). All of them, however, only apply either DEA or SFA. Due to rather bad data quality they were also not able to analyze other than total costs. 14 For more detailed data on German water losses see IGES (2010, p.30). encountered that this criterion did not have any significant explanatory power. The reason for it might be that the companies within our three groups are actually quite homogenous, which again stresses our hypothesis that companies need to be analyzed according to groups. Our findings might have been different if we would have followed the same path as other researchers which have not had such a detailed database, both quality and quantity wise, and therefore were not able to cluster their observations.

Methods
Based on the definition of relevant cost drivers we apply a parametric and a non-parametric benchmarking approach, namely SFA and DEA (compare Section 3). Because DEA is sensitive towards extreme values, an outlier analysis is applied in addition. Therein, firms that are most efficient in many of the observations are iteratively taken out of the sample and, hence, the efficiency analysis. The process stops when the average value of efficiency of all transmission system operators, including the potential outliers, is statistically indifferent (at 95% confidence) to the average value of efficiency excluding the potential outliers. A t-test (according to Satterthwaite) is used to compare the expected values. Identified outliers are removed from the sample.
Multiple specifications of SFA models are estimated to compare specifications given by similar correlation coefficients in earlier phases of the analysis. To conclude on an improved goodness of fit of one specification against the other, Akaike's and Schwarz's information criteria are used as well as a comparison of the log-likehood values. Given insignificant parameters, a likelihood ratio test is performed. Also, we test different functional forms: Cobb-Douglas, Translog, and log-linear models.

Results
The best model for the largest companies is displayed in the following table.  The results of the DEA-and SFA-analysis have shown that a combination of the variables Distribution pipes, Distribution pipes per household connection and Distribution pipes to accounted water (excl. re-distribution) suits particularly well for an efficiency evaluation of operational distribution costs in the group of the largest water suppliers (2.5 mio. m³ up to 50 mill. m³ annual supply): All three indicators are significant with a minimum confidence level of 99%. Besides, the combination of those three variables explains about 70% of operational distribution costs in this group of firms (R² = 0,706). The English water regulation authority OFWAT, in comparison, uses models sometimes with less than 30% explanatory power. Last, but not least, the sign on the coefficients for Distribution pipes and Distribution pipes per household connection is positive, as expected. Increasing absolute, as well as relative grid length, independently increases costs. Only the regressor Distribution pipes to accounted water (excl. re-distribution) is expected to have a negative influence on costs as it increases with population per km². Because of simultaneous modeling of Distribution pipes per household and Distribution pipes to accounted water (excl. redistribution) the result can additionally be interpreted in the way that costs of a one unit increase in grid length overcompensate the grid density advantages.

Variable
www.intechopen.com 1,390*** 0,048 ln (Distribution pipes to accounted water (excl. re-distribution)) 0,509*** 0,186 Table 4. SFA-Model Small Companies (< 0.5 mill. m³ per year) for operational distribution costs The DEA and SFA results for the largest group of companies are plotted in the figure below to verify consistency of the efficiency analyses. Because the efficiency measures are not always comparable due to the different methods used, the rank correlation of the results are determined (according to Spearman) and plotted. A value of, for example, 0,78, means that the ranks of a firm resulting from DEA and SFA analyses correlate with 78%. 15

Additional value for companies
The problem of current metric benchmarking has been to distinguish between a good and a bad company. So far a water supplier with low costs per m³ is supposed to be a good company. Such a company might however face very favourable conditions which would actually be the reason for this low relative costs compared to other operators. Applying the concept "Learning from the Best" is thus misleading. As mentioned earlier the same holds true for the classical OLS. If a company is, however, good according to both the DEA as well as the SFA (in Figure 5: the company marked by the bottom arrow) this operator would quite surely be an interesting candidate for a discussion with those laggards which should have rather high efficiency potentials (in Figure 5 the company marked by the upper arrow).
Besides, in better identifying a good and a bad company, this approach also helps to quantify an efficiency potential. This potential can of course be displayed in various ways. It is possible to list the DEA-, the SFA-or for example the average of DEA-and SFA-result. The We would certainly always suggest to not only analyze the results of the "operational distribution costs". It is worth doing the same calculations for "total costs" and the remaining sub-categories "operational production costs", "capital costs" and "administration costs". In such a way potential trade-offs between, for example, operational and capital costs can be observed and interpreted. In addition, an analysis would also need to take into account different quality provision between companies which would need to be backed by willingness-to-pay-studies.

Summary and outlook
Benchmarking is an already well established concept in the German water supply sector. It is performed in all of the German Bundesländer. However, if we compare the number of companies which take part in such a metric benchmarking with the total number of water suppliers the percentage will be less than 2 %. One reason for the companies to not participate might be the quality of the feedback they receive. Currently, the structural variables which a company faces are not taken sufficiently into account. Many reasons, besides inefficiency, can explain why one company encounters higher costs than another water supplier.
Efficiency analysis techniques, which imply not only a comparison of companies according to key performance indicators but a measurement of performance, are tools which better analyze the existing benchmarking data. They take into account differing structural conditions of the companies and, therefore, give a more valuable feedback to a company in which parts of the value chain they have potentials to increase their efficiency. Using the data of Rödl & Partner, the most prominent metric benchmarking consultant in the German water supply industry, we showed for the first time that these methods can be easily applied. Our results show that the rank correlations between DEA-and SFA-results are much higher than in other network sectors and other countries. They also display that companies should be clustered. A small and a very large company seem to have different production functions and can hardly be compared with one another. It, therefore, makes sense to cluster the companies according to groups. Overall, we may conclude that the enhancement of the current metric benchmarking systems by performance measurement is displaying if a company seems to have potentials to increase its efficiency in the operational distribution costs. It may also give a quantitative indication of the extent of inefficiency.
The analysis of the operational distribution cost, which has been performed here, certainly is only the first step. The feedback for the companies will further increase if a performance measurement is offered for all parts of the water supply value chain and the total cost. 16 An introduction of performance measurement into the current benchmarking is, therefore, a big leap forward. We may hope that such an improved benchmarking system is giving incentives for more German companies to participate voluntarily.
Otherwise, the question certainly arises of how to proceed. Due to the situation, that companies in a natural monopoly sector are particularly accountable to the public, the German water sector faces similar questions than those ones in other European countries which are employing metric benchmarking systems. Sooner or later the countries face the necessity to decide which kind of information they want to display publically and whether companies should be obliged to participate in benchmarking. The Netherlands, for example, made it compulsory to take part in such programs. Every three years they are publishing reports which also display the performance development of a water supply company both in relation to all other companies as well as over time. By publishing these data, companies did not only detect efficiency potentials but also faced the public pressure to use measures to actually improve their performance. Between 1997 and 2005 the average efficiency of a Dutch water supply company increased by 23 % (Dijkgraaf et al., 2006, p. 8). Such incentives in benchmarking systems to actually increase performance are essential that benchmarking could be regarded as an alternative to an introduction of an economic water utility regulator. 17 16 First, preliminary results for the total cost calculations reveal that the rank correlations between DEA and SFA-results are even higher than for the operational distribution costs. 17 Cross-country comparisons of benchmarking systems in the drinking water sector in the Netherlands, England and Wales, Australia, Portugal and Belgium recently revealed that the average efficiency correlates with the incentives of such benchmarking systems (De Witte/Marques, 2010).