Technical Performance Based Earned Value as a Management Tool for Engineering Projects

1. Introduction

In the project management, one of the keys for the project success is the control of the project in terms of performance, progress and cost. The first practice to control a project is just checking the financial reporting and therefore the project control is only based on the cost control (budgeted versus actual incurred cost). But the experience, especially from projects that failed, shows that a more efficient project management requires controlling also the project performance and scheduling besides the cost.

The Earned Value Management (EVM) is a methodology that integrates the management of project scope, schedule and cost. It has been widely and successfully used for over 40 years and it should apply to any project, large or small, at any industry. The interest about the EVM has increased in the last ten years and a substantial amount of research has been carried out regarding the EVM, generating new extensions of the methodology and specific applications. In parallel, many guidelines and rules have been published to implement properly the EVM, coming mainly from the U.S. Government who originally developed the EVM, but not only, because also the ANSI organization has established a standard defining the EVM.

The contribution of this work is twofold. First of all, it aims to identify the principal lines of research in the project management control using earned value management across the academic research and organizations practice. Following, the attention is focused on the characteristics of one of these research lines, the PBEV. This extension pretends to be not only an EVM parameters enhancement but a complete managing system based on the technical performance, which could be a useful EVM improvement to be used in engines engineering projects where the technical objectives are the main target. Secondly, it lies in the intention of this work to evaluate the applicability and the efficiency of the PBEV with two case studies of two real-life engineering projects of combustion engines development for energy generation applications.

2. Earned value management

2.1. EVM standard

Basically, the EVM requires a fixed point-of-reference, given by the project baseline schedule and the budget at completion (BAC), in order to periodically measure the project performance along the life of the project. Project performance, both in terms of time and costs, is determined by comparing the three key parameters of the EVM, planned value (PV), actual costs (AC) and earned value (EV), resulting in performance variances known as schedule variance (SV=EV-PV) and cost variance (CV=EV-AC) and performance indexes as the schedule performance index (SPI=EV/PV) and the cost performance index (CPI=EV/AC). See [4] and [5]. Figure 1 shows the EVM key parameters.

Figure 1.

EVM key parameters.

For a better understanding of the state of the art of the EVM it is necessary to know its origins, widely commented in [1] and [2], and the official sources of EVM guidelines and standards.

The earned value concept originally came from industrial engineers in factories in the early 1900s who for years have employed a three-dimensional approach to assess true “cost-performance” efficiencies. To assess their cost performance, they compared their earned standards (the physical factory output) against the actual cost incurred. Then, they compared their earned standards to the original planned standards (the physical work they planned to accomplish) to assess the schedule results. These efforts provided earned value in its most basic form. Most important, the industrial engineers defined a cost variance as the difference between the actual costs spent and the earned standards in the factory. This definition of a cost variance is perhaps the indication to determine whether one uses the earned-value concept. Later on, in 1965 the United States Air force acquisition managers defined 35 criteria which capture the essence of earned value management. Two years later the U.S. Department of Defense (DoD) adopted these same criteria as part of their Cost/Schedule Control Systems Criteria (C/SCSC). Then, in 1996, after a rewrite of the C/SCSC 35 criteria by private industry, the DoD accepted the rewording of this criteria under a new title called Earned Value Management System (EVMS), and the total number of criteria was reduced to 32. In 1998, National Defense Industrial Association (NDIA) obtained acceptance of the Earned Value Management System in the form of the American National Standards Institute, termed the ANSI/EIA-748 Standard, see [3]. From this on, many research lines have been developed trying to improve the EVM, and they are commented in the following point.

2.2. EVM further research lines

As mentioned in the introduction, the use of EVM and parts of it, or tailoring it to specific situations continue to grow and a substantial amount of research is being carried out. The main investigations could be grouped at least in six big research lines as follows.

1. EVM and fuzzy determination of EV. One of the major difficulties in the determination of EV is the evaluation of in-process work. There are some techniques to quantify the accomplished work and some of them propose fuzzy techniques. As explained in [6] and [7].

2. EVM forecast accuracy. Some studies about EVM are focusing on the improvement of the accuracy of EVM to calculate forecasts of project cost and schedule at completion. See references [8], [9] and [10]

3. EVM and Earned Schedule. Another object of studies is an enhancement to Earned Value Management, called Earned Schedule that appeared in 2003. This method propose the parameters of the EVM in terms of time instead of cost, and it could be postulated to give more forecasting accuracy in the latest stages of the project. See [11]. The author of this research line criticized the use of the classic SV and SPI metrics since they give false and unreliable time forecasts near the end of the project. Instead, he provided a time-based measure to overcome this unreliable behavior of the SV and SPI indicators. This earned schedule method relies on similar principles than the earned value method, but translates the monetary metrics into a time dimension. He reformulated the time performance measures SV and SPI to SV(t) and SPI(t) (where the t between brackets is used to distinguish with the original performance measures) and has shown that they have a reliable behavior along the whole life of the project. Since its introduction, a considerable stream of publications on the project time performance measurement and/or the earned schedule concept has been published in the academic literature. Statistical validation of the time performance indicators and stability studies on empirical data can be found in [12].

4. EVM to integrate risk management. There are project tools for risk analysis mainly focused on network based techniques, such as, CPM (Critical Path Method) and PERT (Program Evaluation and Review Technique) and others based in statistical probabilities like Montecarlo Simulations. Nevertheless, EVM contains no guidelines on risk management. Therefore, some investigations are conducting to integrate risk analysis in the EVM. First, including risk management activities in the WBS (Work Breakdown Structure) and second considering that Estimated at Completion (EAC) should be based on both project performance and quantified risk assessments. Other investigations try to combine the risk analysis tool results, for example Montecarlo simulations, with EVM metrics. See [13].

5. EVM to integrate quality. The quality of the product must be tracked in an engineering project to evaluate the success or failure of the product. Some studies are proposing to apply a quality factor to the EV, such as, reducing the EV when poor quality occurs. See [14].

6. EVM to integrate technical performance. In engineering projects not only cost and schedule must be tracked but also the technical performance. Further investigations are being done to integrate technical requirements accomplishment with EVM. One proposal is an EVM extension called Performed-Based Earned Value (PBEV). See [15-18].

This last extension of the EVM integrating the technical performance, the PBEV, will be deeply analyzed in the following points due to it is not only a new manner of measuring the earned value but it is an entire management system which include concepts and procedures to manage the technical issues that could enhance the engineering projects in general. Therefore, it will be studied its suitability to engine engineering projects.

Figure 2.

EVM evolution and research lines.

3. PBEV: The EVM extension focusing on technical performance

3.1. PBEV introduction

The Performance Based Earned Value (PBEV) is an enhancement to the EVM standard that overcomes the standard´s shortcomings with regard to measuring technical performance and quality because it is based on standards and models for systems engineering, software engineering, and project management. The distinguishing feature of PBEV is its focus on the customer requirements. PBEV provides principles and guidance for cost effective processes that specify the most effective measures of cost, schedule, and product quality performance.

The PBEV has been formulated by the author Paul J. Solomon, see [16], whom experience at programs management and concretely with the EMVS explains the origin of the PBEV. Solomon manages the EVM for Northrop Grumman Corporation, Integrated Systems and he has participated in programs for the U.S. Government as the B-2 Bomber, Global Hawk and F-35 Joint Strike Fighter. He is on the board of the National Defense Industrial Association, Program Management Systems Subcommittee that authored ANSI/EIA-748.

3.2. EVM shortcomings

The cancellation of the US Navy’s A-12 Avenger stealth aircraft program in January 1991 resulted in research during the 1990s, which investigated the reliability of EVM cost prediction and the project management failures in accurately reporting project performance, using U.S. Department of Defense (DOD) project data. These research findings have come to be regarded as generally applicable across all project types using EVM across multiple industry sectors. The U.S. Government Accountability Office (GAO), for example, studied failures in acquisition of weapons systems and information technology systems. Some report of the GAO concluded that DOD paid billions in award and incentive fees regardless the contractors not held cost goals, schedule goals and the program did not capture early on the warnings needed to effectively manage program risk. The GAO concluded that, if EVM is not properly implemented, the data may be inaccurate and misleading.

The EVM standard, which is formulated in the ANSI/EIA Standard-748-A-1998, has significant shortcomings with regard to measuring technical performance and quality. First, the EVMS standard states that EV is a measurement of the quantity of work accomplished and that the quality and technical content of work performed are controlled by other processes. A program manager should ensure that EV is also a measurement of the product quality and technical maturity of the evolving work products instead of just the quantity of work accomplished. Second, the EVMS principles address only the project work scope. EVMS ignores the product scope and product requirements. Third, EVM is perceived to be a risk management tool. However, EVMS was not designed to manage risk and does not even mention the subject.

Thus, the EVM will provide more reliable information for analysis and decision-making if the EVM guidelines are augmented by guidance regarding maintaining the technical baseline, measuring technical performance, and managing risk.

3.3. U.S. DOD policy and industrial standards

Most of the guidelines and best practices for planning and measuring the technical performance considering also the EVM are provided by two sources: the U.S. Department of Defense (DOD) and some industrial standards.

First of this sources, the U.S. DOD, issued an acquisition policy which states that programs implement Systems Engineering Plans (SEP). DOD also published guides to implement the policy, such as:

1. The Defense Acquisition Guidebook (DAG).

2. The Systems Engineering Plan Preparation Guide (SEPPG).

3. The Work Breakdown Structure Handbook (MIL-HDBK-881A (WBS))

4. The Integrated Master Plan (IMP).

5. The Integrated Master Schedule (IMS) Preparation and Use Guide.

6. The DOD guides refer also to EVMS. See [19].

These guides show the enormous importance that the U.S. DOD gives to measuring technical performance, hence, according to U.S. DOD it should be one of the parameters to control during project progress. See [20] and [21].

The second source of guidelines for planning and measuring the technical performance are some industry and professional standards, that have been incorporating the technical performance accomplishment for the project control and they have been at the same time sources themselves for defining the DOD policy, as well as:

1. The Institute of Electrical and Electronics Engineers (IEEE) 1220.

2. EIA 632.

3. “A Guide to the Project Management Body of Knowledge” (PMBOK). See [22] and [23].

4. Capability Maturity Model Integration (CMMI). See [24].

In the following points are explained the main concepts defined by the U.S. DOD guidelines and the industrial standards that are used by the PBEV. Figure 3 shows the guidelines and standards and their sources for technical performance treatment at projects.

Figure 3.

Guidelines and standards to manage technical performance.

3.4. PBEV guidelines

The PBEV is a set of principles and guidelines that specify the most effective measures of cost, schedule, and product quality performance. It has several characteristics that distinguish it from standard EVMS:

• Plan is driven by product quality requirements, not work requirements.

• Focuses on technical maturity and quality, in addition to work.

• Focuses on progress toward meeting success criteria of technical reviews.

• Adheres to standards and models for SE, software engineering, and project management.

• Provides smart work package planning.

• Enables insightful variance analysis.

• Ensures a lean and cost-effective approach.

• Enables scalable scope and complexity depending on risk.

• Integrates risk management activities with the performance measurement baseline.

• Integrates risk management outcomes with the Estimate at Completion.

PBEV augments EVMS with 4 additional principles and 16 guidelines. The following are PBEV principles that set it apart from EVMS:

1. Product scope and quality. Integrate product scope and quality requirements into the performance measurement baseline. This principle focuses on customer satisfaction, which is based on delivery of a product that meets its quality requirements and is within the cost and schedule objectives.

2. Product quality requirements. Specify performance toward satisfying product quality requirements as a base measure of earned value. A product quality requirement is a characteristic of a product that is mandatory in order for the product to meet verified customer needs.

3. Risk management integration. Integrate risk management with EVM.

4. Tailor PBEV. Tailor the application of PBEV according to the risk.

And the PBEV guidelines are listed following.

1. Establish product quality requirements and allocate these to product components.

2. Maintain bidirectional traceability of product and product component quality requirements among the project plans, work packages, planning packages, and work products.

3. Identify changes that need to be made to the project plans, work packages, planning packages, and work products resulting from changes to the products quality requirements.

4. Define the information need and objective to measure progress toward satisfying product quality requirements.

5. Specify work products and performance-based measures of progress for satisfying product quality requirements as base measures of earned value.

6. Specify operational definitions for the base measures of earned value, stated in precise, unambiguous terms that address communication and repeatability.

7. Identify event-based success criteria for technical reviews that include development maturity to date and the products´s ability to satisfy product quality requirements.

8. Establish time-phased planned values for measures of progress toward meeting product quality requirements, dates of frequency for checking progress, and dates when full conformance will be met.

9. Allocate budget in discrete work packages to measures of progress toward meeting product quality requirements.

10. Compare the amount of planned budget and the amount of budget earned for achieving progress toward meeting product quality requirements.

11. Use Level of Effort method to plan work that is measurable, but is not a measure of progress toward satisfying product quality requirements, final cost objectives, or final schedule objectives

12. Perform more effective variance analysis by segregating discrete effort from Level of Effort.

13. Identify changes that need to be made to the project plans, work packages, planning packages, and work products resulting from responses to risks.

14. Develop revised estimates of costs at completion based on risk quantification.

15. Apply PBEV coverage to the whole work breakdown structure or just to the higher risk components.

16. Apply PBEV throughout the whole system development life cycle or initiate after requirements development.

3.5. PBEV concepts

The PBEV feeds on concepts from the U.S. DOD guidelines and industrial standards mentioned above. In the following points are explained in detail the main concepts used in the PBEV coming from those sources, such as, the Systems Engineering Plan (SEP), the products metrics and quality, the success criteria, the Technical Performance Measurements (TPM) and Capability Maturity Model Integration (CMMI). The concepts that feed the PBEV are shown in figure 4.

Figure 4.

The concepts that feeds the PBEV

1. Systems Engineering Plan (SEP)

The purpose of the Systems Engineering Plan (SEP) is to help programs develop their Systems Engineering (SE) approach, providing a firm and well-documented technical foundation for the program. The SEP is a living document in which periodic updates capture the program’s current status and evolving SE implementation and its relationship with the overall program management effort. Although the detailed content of each SEP is customized according to the particulars of a program and each update may vary depending on the program’s acquisition phase, using a common framework encourages sound technical planning throughout the program’s life cycle. The emphasis should be on the rigor of the technical planning as captured in the SEP, not on the SEP itself. The SEP also serves as a common reference to achieve shared stakeholder insight regarding a program’s planned technical approach. It provides a documented understanding of how the program will accommodate cost, schedule, performance, and sustainment trades; the expected products of the SE effort; and how these products will contribute to program decision making.

• Products metrics and quality

The IEEE 1220 and the EIA 632 have similar guidance regarding product metrics and quality. Product metrics allow assessment of the product´s ability to satisfy requirements and to evaluate the evolving product quality against planned or expected values. Of equal importance are a disciplined requirements traceability process and a requirements traceability database.

• Success criteria

The standards discuss the importance of holding technical reviews at various stages of development to assure that all success criteria have been met. IEEE 1220 provides success criteria to be used at major technical reviews. For example some of the success criteria for a preliminary design review are the following:

1. Prior completion of subsystem reviews.

2. Determine whether total system approach to detailed design satisfies the system baseline.

3. Unacceptable risks are mitigated.

4. Issues for all subsystems, products, and life-cycle processes are resolved.

The success criteria should be defined in a SEP or other technical plan. The customer should review this plan with the supplier and reach agreement on the success criteria to be used at technical reviews.

• Technical Performance Measurement (TPM)

Technical Performance Measurement (TPM) are defined and evaluated to assess how well a system is achieving its performance requirements. TPM uses actual or predicted values from engineering measurements, test, experiments, or prototypes. IEEE 1220, EIA 632 and “A Guide to the Project Management Body of Knowledge” (PMBOK guide) provide similar guidance for TPM planning and measurement and for integrating TPM and EVM.

• Capability Maturity Model Integration (CMMI)

The CMMI provides many practices that augment the EVMS guidelines. CMMI also lists Typical Work Products (TWPs) within process areas. To ensure traceability of product quality requirements to work tasks and work products, these TWPs, or similar artifacts, should be the outcome of work packages. Here are some TWPs in CMMI.

TWPs include the following:

1. Product-component requirements.

2. Activities diagrams and use cases.

3. Verification criteria used to ensure requirements have been achieved.

4. Exit and entry criteria for work products.

3.6. PBEV pros and cons

Following are listed the advantages and disadvantages of the PBEV that it has been used also in the following case studies analyzed during this work.

The main PBEV pros are:

• It integrates effectively the technical performance with the cost and schedule control.

• It collects the best practices of the U.S. DOD guides and industry and professional standards related with project management and then apply the EVM.

• It is based on client or product requirements.

By the other hand, the PBEV could have the following cons:

• It gives a very large methodology to implement to be able to integrate the technical performance in the project management control but there is no mention to any parameter, or metric definition to implement easily that allows calculating EVM variances or indexes with results of technical performance status.

• It only influences somehow the work breakdown structure but not the EVM metrics.

• The customer should review the SE plan with the supplier and reach agreement on the success criteria to be used at technical reviews. It would be more objective to define a technical parameter to evaluate success criteria, as for example, the strength safety margin of a mechanical part.

4. PBEV model adaptation to industrial production environment

They have been performed two case studies of the applicability and the efficiency of the EVM standard and one of its extensions, the PBEV which is based on technical performance, for the engines engineering projects control in the energy field.

4.1. Type of projects analyzed

They have been analyzed two real-life projects of engine engineering in the energy field performed between 2004 and 2005 by a company dedicated to the development and manufacturing of combustion engines for power generation.

These projects are characterized by including the typical phases of the industrial engineering projects, such as, the concept design phase, detailed design phase, simulation, prototyping, testing and launch mass production, where the level of compliance of the technical objectives is strongly important besides the cost and scheduling control. The two projects analyzed are also featured by the fact that they are large projects in the energy field as they have budgets of several million euros, durations between 2-4 years and the risk of developing new products for the market. The large size of the projects makes the breakdown structure of the tasks (WBS) to be also quite large. Another property that characterizes these two projects, and in general all the engine development projects in the energy field, is that they are pretty similar to engine development projects in the automotive field since the product is similar. Therefore, the engines made for energy field are fed with many of the technologies used in automotive field, as well as, with the methodologies and requirements to create new products that meet the market requirements. The competitiveness for obtain improved technical results makes that in both, energy projects and automotive projects, the technical objectives are a critical parameter to control for the project, just with the costs and the schedule. Hence, the PBEV could be a suitable and useful tool for integrating scope, schedule, cost and technical performance in these projects. Moreover, in both fields, energy and automotive, the continuous technology evolution makes the technical objectives to be overpassed from one project to the following. For that reason is a real-life practice in these fields that at new product development projects, with the existing technology in that moment, that, if the technical objectives are not met but they are inside of an allowable range, the project could be accepted with penalties.

In summary, the projects analyzed with PBEV are characterized for being classical industrial engineering projects with large size of the WBS, also for having the technical performance as a key parameter to control where the technical objective could be in a range of tolerance for accept the successful of the project. The project number 1 consists on the development of a new engine of higher efficiency and power than the existing ones in the company and the project number 2 was dedicated to development a new injection system for the current engines. Figure 5 shows the type of engine developed in project 1 and 2.

Figure 5.

Type of projects analysed: engine engineering projects

4.2. Project information

The project 1 began in January 2004 and was delayed one year by material procurement problems but once they were negotiated new deadlines with suppliers in 2005 was launched again and began an overall tracking of the technical targets, cost and schedule. In this project, the engineering consultancy delivered a design that was slightly below the technical targets of engine performance and efficiency, which was penalized in his fees. The engine efficiency target was 42% but an engine with 40% was obtained. Finally, the project was completed in January 2008, bringing to the market the new engine with a very good acceptance and overall rating of satisfactory.

Regarding the project 2, it began also in 2005 and although the design and material procurement were on time, the project turned 180º in 2007 when it was found that to accomplish the technical targets of the new injection system it was necessary to implement a technology 10 times more expensive than originally planned. This event made management direction to stop the project after making a rough estimation of the overall cost of the project with the implementation of the new technology.

From these two project data, here is proposed to apply the PBEV to analyze if it is possible to report the status of the project in time, cost and technical compliance, to predict future states.

In the two projects analyzed the available information during the duration of all the project was the following documents and reports.

1. A project specification document which collects all the technical specifications to meet with contractual nature.

2. A Statement Of Work (SOW) document which collects the definition of all the tasks to be performed in the project and has contractual nature.

3. The initial baseline schedule in an MS project file. It is detailed with several tasks levels and including the starting and finishing dates, as well as, the dependences between them.

4. The total budget of the project made at the beginning of the project and broken down in general spending issues. The general spending issues are the following six: material costs, the tooling investments, engineering hours, outsourcing expert consultancy support hours, testing costs, and trips. Intermediate budget estimation was not performed during the project progress.

5. The general accounting of the project with the invoiced costs per month. They are available the figures of monthly costs of the general issues.

6. A monthly report with the technical, economical and scheduling tracing. The technical part of the report included a list of all manufacturing drawings and the testing results. Figure 6 shows engine control units screen where technical parameters are monitored, as for example, the engine performance. These technical values are the reference to check the technical accomplishment. The economical part of the report included cost monthly figures of the general spending issues collected from the invoices. The scheduling part of the report included approximate deadlines for the critical tasks but not a detailed scheduling track.

Figure 6.

Engine control unit monitoring engine performance

4.3. Methodology

Due to fact that the available data in the two projects analyzed are the initial budget, accumulated costs and the engineering hours accumulated only for the higher level of the WBS, the EVM is applied only at this top level. This way of controlling is called top-down project control. According to reference [13], the EVM offers calculation methods yielding reliable results on higher WBS levels, which greatly simplify final duration and completion date forecasting. These early warning signals, if analyzed properly, define the need to eventually go down into lower WBS levels when action thresholds are exceeded. In conjunction with the project schedule, it allows taking corrective actions on those activities which are in trouble (especially those tasks which are on the critical path).

The methodology used to implement the EVM based on technical performance in the two projects under study has two steps. The first step is to apply the EVM standard and check if the results are consistent. The second step is to apply the PBEV through the use of penalties in the obtained EV when the technical requirements are not met and check if the results are consistent with the reality of the project. An objectively way of establish the penalties depends on the technical parameter selected to control and on the technical performance measurement, therefore, it depends on the internal mechanisms of each company to evaluate the objectives compliance as explained in reference [16].

To carry on the first step, this is to apply the EVM standard on the two projects, from the available data, one begins by determining the values of the three basic parameters of the EVM, and this is, PV, AC and EV. Because PV is not available monthly, it is considered the hypothesis of distributing uniformly the initial budget between the months of the project, and then the PV accumulated is lineal. The figures of the monthly AC are available in the project data. Finally, to calculate the EV, the engineering hours performed are taken as the indicator of work accomplished. And from the 3 basic parameters they are calculated the Cost and Schedule Variances (CV and SV) and the Cost and Schedule Performance Indices (CPI and SPI) to check if the results of de EVM are consistent.

To implement the second step, this is to calculate the PBEV, a reduction of the EV is applied in the monthly work accomplished when the technical objectives are not met, so in this way it is shown that the project is farther from the objectives and then from finish than planned.

Figure 7.

Methodology to apply PBEV to engine engineering projects.

5. Case studies: Applying PBEV to engine engineering projects

5.1. Research questions

The two case studies carried out with engine engineering projects have been raised as a series of research questions to be answered in order to draw conclusions. The research questions are the following:

• Q1: Is it possible to apply the EVM standard to engines engineering projects only with the data that is usually controlled in such projects?

• Q2: Is it possible to apply the PBEV to engines engineering projects?

• Q3: Does the PBEV capture the real project progress in engines engineering projects with only the project information available?

• Q4: Is it useful the information resulting from the PBEV to take decisions in the engine engineering projects?

5.2. Results

In this point are summarized the two case studies results. Table 2 shows the project 1 cumulative cost data. The Budget At Completion (BAC) for project 1 was 2.427.950 euros. The costs and BAC are used as input to calculate the EVM and the PBEV parameters which are shown in table 2.

PROJECT 1 - CUMULATIVE COST DATA (€)
ITEM	Oct-07	Jun-08	Aug-08	Nov-08	Dec-08
MATERIALS	416.686	444.229	445.059	471.645	472.736
INVESTMENT	279.356	279.356	279.356	279.356	279.356
LABOR HOURS	308.880	400.650	424.890	474.750	487.560
TESTING HOURS	2.935	4.036	4.353	4.688	4.688
ENGINEERING OUTSOURCING	1.443.296	1.522.324	1.522.605	1.526.015	1.526.015
TRIPS	13.021	16.435	16.435	22.524	22.730
TOTAL	2.464.174	2.667.030	2.692.698	2.778.978	2.793.085

Table 1.

Project 1 cumulative cost data in euros.

PROJECT 1 - EVM AND PBEV CALCULATED PARAMETERS
PARAMETER	Oct-07	Jun-08	Aug-08	Nov-08	Dec-08
PV	1.824.282	2.169.235	2.255.473	2.384.831	2.427.950
AC	2.464.174	2.667.030	2.692.698	2.778.978	2.793.085
EV	1.538.160	1.995.156	2.115.866	2.364.159	2.427.950
CV	-926.014	-671.874	-576.832	-414.819	-365.135
SV	-286.122	-174.079	-139.607	-20.672	0
CPI	0,62	0,75	0,79	0,85	0,87
SPI	0,84	0,92	0,94	0,99	1,00
PBEV	1.300.000	1.600.000	1.700.000	1.900.000	2.000.000

Table 2.

Project 1 EVM and PBEV calculated parameters.

Figure 8 shows the calculated EVM parameters and figure 9 shows the calculated PBEV against the EVM for project 1.

Figure 8.

Project 1 calculated EVM parameters AC, PV, EV.

Figure 9.

Project 1 EVM versus PBEV.

Following is explained the calculation of the points of figure 9. Let instant time June 2008, for example, the Actual Cost, AC, is 2.667.030€, which is a project data defined in table 1. The Planned Value, PV, is the proportional part of the total budget, BAC, corresponding to the month 30, that is June 2008, over the total number of months of the project which is 36 from December 2005 to December 2008. Previously, to the initial BAC of the 2004 which was a project input data of 2.427.950€ is subtracted the costs from 2004 to 2005 of 875.661€ to consider the initial BAC updated to 2005 because in this year the project was launched again after a delay an all the project data start in 2005. The Earned Value, EV, is calculated with the rate labor hours at June 2008 over the total labor hours multiply by the BAC, resulting in 1.995.156 €. For calculating the PBEV it is considered a penalty of 25% in the EV because the engine efficiency target of 42% is not achieved and instead of that is obtained 40%.

Table 3 shows the project 2 cumulative cost data. The Budget At Completion (BAC) for project 2 was 2.616.831 euros. The costs and BAC are used as input to calculate the EVM and the PBEV parameters which are shown in table 4.

PROJECT 2 - CUMULATIVE COST DATA (€)
ITEM	Jan-07	Feb-07	Mar-07	Apr-07	May-07
MATERIALS	297.018	298.606	315.487	315.742	324.575
INVESTMENT	54.467	54.467	54.467	54.467	54.467
LABOR HOURS	307.170	320.940	338.220	346.530	366.480
TESTING HOURS	111.854	126.781	138.642	147.510	159.630
ENGINEERING OUTSOURCING	139.290	139.290	139.290	139.290	139.290
TRIPS	18.770	18.770	18.770	18.770	19.370
FUEL	2.822	3.000	3.256	3.408	3.986
TOTAL	931.391	961.854	1.008.132	1.025.717	1.067.798

Table 3.

Project 2 cumulative cost data in euros.

PROJECT 2 - EVM AND PBEV CALCULATED PARAMETERS
PARAMETER	Jan-07	Feb-07	Mar-07	Apr-07	May-07
PV	2.030.091	2.176.776	2.323.461	2.470.146	2.616.831
AC	931.391	961.854	1.008.132	1.025.717	1.067.798
EV	307.170	320.940	338.220	346.530	366.480
CV	-624.221	-640.914	-669.912	-679.187	-701.318
SV	-1.722.921	-1.855.836	-1.985.241	-2.123.616	-2.250.351
CPI	0,33	0,33	0,34	0,34	0,34
SPI	0,15	0,15	0,15	0,14	0,14
PBEV	300.000	300.000	300.000	300.000	300.000

Table 4.

Project 2 EVM and PBEV calculated parameters.

Figure 10 shows the calculated EVM parameters and figure 11 shows the calculated PBEV against the EVM for project 2.

Figure 10.

Project 2 calculated EVM parameters AC, PV, EV.

Figure 11.

Project 2 EVM versus PBEV.

Now, it is possible to give answers to the proposed questions.

• Q1: The results presented in figures 8 and 10 show that it is possible to apply the EVM to the higher level of the WBS in projects where the only information is the monthly accounting and the work progress, so the EVM parameters could be obtained and a project control with EVM is possible. For checking that the obtained values for AC, PV and EV are consistent with the EVM, it could be checked that in general when a project is finished satisfactorily the SV is zero and the SPI is 1, due to the fact that at the end of the project the earned value will match with the planned value. For the project 1, which finished satisfactorily, at the end of the project the EV fitted with the PV and therefore the SV is zero and the SPI is 1, as can be seen in table 2. By the other hand, for the project 2, which was intentionally terminated by the managing director, the EV does not reach the PV and thus, the SV never becomes zero and the SPI does not tend to 1. The EVM application in both projects is consistent with EVM rules and with the reality of the projects.

• Q2: The result of PBEV presented in figures 9 and 11 shows that, once the EVM is applied, is also applicable the PBEV if they are defined penalties for when the technical targets are not fully accomplished. The manner to quantify the penalties depends on each company’s internal procedure and his technical performance measurements, as it is mentioned in [16]. For the project 1, after applying the PBEV criteria, which means a reduction in the EV value because in this project was established an engine efficiency of 42% and it reached only to 40%, the obtained PBEV is still inside an admissible range of objectives accomplishment and thus it is accepted the engine with performance 40% to be released. In the case of the project 2, the application of the PBEV is also possible but it has not much influence because of the project circumstance of have been stopped by Managing Director. Thus, this project had not time to correct the situation even the PBEV gives a more realistic status of objectives accomplishment.

• Q3: As the PBEV using the EV with penalty adjust the real situation of the project integrating technical targets, cost and schedule, it captures properly the real situation in an engine development project where the accomplishment of the technical targets is a key parameter.

• Q4: The PBEV gives useful information for the project control and helps to the decision making process regarding the technical targets compliance besides the cost and scheduling control. The proposed EV with penalty in the PBEV reduce the EV near the end of the project and this implies to take a decision between accepting the project finish with the obtained technical results under the objectives or the necessity of continue the project until the objectives are achieved.

6. Conclusions

The following conclusions could be extracted from the present work.

• The EVM method to project control has been widely extended in the last ten years and a substantial amount of research has been generated in academic literature and industrial guidelines to evaluate the method and proposing extensions to improve it.

• The research regarding the EVM could be classified in six main research lines according with this work. The research lines try to enhance the EVM adding to it concepts, such as, the product quality, the project risk analysis, the technical performance, a fuzzy determination of the EV, or analysing for example the EVM forecast accuracy, or even, there is a research line, the Earned Schedule, that proposes another point of view for the EVM in terms of time instead of costs.

• One of the most complete extensions of the EVM is the PBEV, that is not only a new EVM metric to evaluate the technical performance but a complete system of guidelines for project management that integrates the technical performance measurement in all the project management procedures including the EVM.

• The PBEV could be applied to industrial projects such as engine engineering projects in the energy field and it could be applied at the top level of the project WBS.

• The case study number 1 shows that PBEV allows capturing the real project status including the technical performance. In this case, the PBEV reduce the EV near the end of the project and this implies to take a decision between accepting the project finish with the obtained technical results under the objectives or the necessity of continue the project until the reach the objectives.

• The case study number 2 shows that PBEV could be applied to a project that finish dramatically but it does not contribute significantly to give early warnings of the health of the project if the project finished by other issue regardless the technical performance.

7. Future research

In a complementary way to the PBEV that considers EV with penalty when the technical target is not achieved, it could be proposed other penalties in the PBEV variables, such as, the penalty to the AC to capture the reduction in supplier’s fees when the technical targets are not met or when the project suffers an unexpected increase in the cost of any task or component. This AC with penalty could be an improvement in the project management as a tool also for risk management.

The future research in general will include probably new metrics in the EVM methodology to take into account issues like risk analysis or quality and technical performance for a more efficient project control. By the moment, these issues depend on the practitioners, to be included in the project scheduling by means of milestones which is quite subjective.

Moreover, the combination of other project management techniques with EVM is also a good practice that is under developing by some authors.

Which it is sure is that the EVM will take place as a common tool in the project management as Gantt did in the past because it is easy, quick, an efficient.