Open access peer-reviewed chapter

Evaluation of the Project Management Team Members by Using the MCDM

By Blanka Bazsova

Submitted: September 19th 2016Reviewed: April 14th 2017Published: August 30th 2017

DOI: 10.5772/intechopen.69229

Downloaded: 1049

Abstract

Modern trends in human resources show the necessity to quantitatively express employees’ evaluation and their complex assessment. The competency models are more and more involved in the evaluation process. They can significantly contribute to the objectification of the rewarding system. The evaluation process was introduced to the project team members from one of the high‐tech companies. The company’s primary focus is on a creation of the web sites and e‐commerce. The competency models were tested for various positions, such as a project manager, graphic designer, web developer and tester, with an application of the multi‐criteria decision‐making methods (MCDMs). Using Saaty’s method based on expert evaluation, groups of competencies were evaluated.

Keywords

  • competences
  • evaluation
  • MCDM
  • project management
  • Saaty’s matrix

1. Introduction

Evaluation of the work is an integral part of human resource management in the organization. According to Duchon and Safrankova [1], the most common forms of evaluation include evaluation by direct supervisor, evaluation by staff and self‐assessment. The best known forms of evaluation include verbal description or questionnaire, comparison with the objectives set, comparison with other workers and an assessment based on critical cases, as stated by Duchoň and Šafránková [1]. Evaluation by the supervisor and self‐assessment can cause distorted or subjective picture of the work done and the qualities of the worker, while the evaluation by other workers appears to be more objective. For all the above methods of the work evaluation and especially when compared to actually performed work with the stated objectives, it is useful to create a system of assumptions—competences—which serve to measure the quantity and the quality of the work. An important issue for all project managers is whether they are competent to manage large and complex projects. Competence expresses qualification or ability to perform a specific job, namely to hold the position. This is an evaluation of behaviour of the employee to perform job duties. Hronik [2] understands competence as the qualification for a particular performance. The aim of the evaluation by competences is to determine whether the organization is poised to implement its outlined strategy. Evaluation by competencies reflects comparison of expectations of the manager with the actual behaviour of the employees. Based on the comparison of assumptions for the work performance and the actual achieved behaviour, there is a differentiation in remuneration. According to Kleibl et al. [3], analysis of the work focused on the task doesn’t say about skills and personal characteristics of the ideal worker. As asserted by Hronik [2], the performance of the company depends on the processes and people in the organization. The proper competency model should recognize the level of knowledge and skills of the employees; the time needed for training and education of employees should be provided, the model should match the goals of the organization and focus on the performance of the organization as such Redmondova [4]. In the literature, there is no uniform classification of competencies. Generally, competences can be divided into general, expertise and specific. To hold a certain position, it is important to have a choice from these three categories. The unique combination of these sub‐competencies creates the so‐called competency model. Competency model for a specific position should be described in a written form. Developing of the employees’ competences also becomes an opportunity for the personal and professional growth and development of the employee. Wagnerova [5] provides the skills, knowledge, experience and motivation as the so‐called ‘personal determinants of performance’. The unique combination of competencies forms the culture of an organization that promotes learning and education seemed as an important business process by the top‐level and line managers that are committed to it and that are permanently engaged in it, as stated by Armstrong [6].

Advertisement

2. Assessment of the competences

The Project Management Institute (PMI) was founded in the 1970s to cover project management and standardization of procedures and project management methodologies. The PM institute defines the project management as project management is the application of knowledge, skills, tools and techniques to project activities (as stated by Caupin et al. [7]). According to Rehacek [8], human resources are one of the areas, which the project management is aimed at. Human resources describe the processes for the efficient use of labour (organizational planning, personnel and project teams) to meet the project requirements. IPMA (International Project Management Associations) defines three groups of competences—contextual, technical and behavioural. The contextual competences group includes project, programme and portfolio orientation, also knowledge of finances, laws, knowledge of project, implementation, a programme and a portfolio, and so on. The technical competences group includes the ability of the success of the portfolio, project organization scope and deliverables, time and project phases, resources, start‐up, close‐out and communication. Behavioural competencies include leadership, self‐control, assertiveness, openness, creativity, results orientation, reliability, efficiency and so on (according to Caupin et al. [7]).

Project managers are responsible for a wide range of processes, from planning the work (selection of potential spectrum of suppliers, management of contractual relations, determination, risk valuation, risk response creation) through the implementation process (the implementation of the project plan, spreading of information, acquisition of bids and team development), control and management processes and ultimately closing processes (termination of contractual relations and archiving of project documentation) [9]. Project manager competency model will be designed in accordance with CSN ISO 21500 and PM Book (Project Management Book) (as stated by Rehacek [10]).

In practice, a series of competency models were created. Most of them are based on three pillars—focus on the product, customer and service and system. Krenar and Taraba [11] deal with the assessment of holistic competencies in the management of multinational corporations. They argue that the holistic competencies can increase the efficiency of functioning of the international management teams. A crisis manager competency model evaluating especially managerial competencies was created [12].

3. Multi‐criteria decision‐making

The present complexity of problems requires analysing and taking into consideration not only one but several important aspects (points of view), according to which we evaluate a certain problem. Whereas the decision‐making problem is such a problem, in which we are making a decision between at least two versions [13]. One of the most used groups of methods for solving problems of multi‐criteria decision making are the methods that are based on calculation of the rate of benefit. The higher benefit from the given version we have, the more suitable this version for solving the problem is.

The number of authors quote the classification of multiple‐criteria decision methods (MCDMs), for example, Brozova et al. [13] and Zonkova [14]. They are subdivided according to different points of view. The TOPSIS method is ranked among the methods based on minimization of the distance from the ideal version. Further, we differentiate methods based on the evaluation of the preferential relation (Electre and Promethee), methods based on the threshold rate of substitution, methods that require an aspirating level of criteria (conjunctive method, disjunctive method and PRIAM method) and methods that require ordinal information (lexographical method, permutation method and ORESTE method).

To evaluate competences of the project team members, we are going to use one of the multiple‐criteria methods—Saaty method. Saaty’s matrix is useful method for evaluation of the criteria, which are difficult to express quantitatively. Saaty’s matrix evaluates manager’s preferences among criteria. Bazsova [15] states that the criteria reflect the way, how to solve problems which means the way of goal achievement. The criteria can be expressed qualitatively and quantitatively [16]. We compare preferences of criteria with each other. We can obtain information about preferences and criteria’s weight.

Saaty´s matrix is the quadrat matrix, S= {sij}, where i, j= 1, 2, …, n, of size cnxcn, where c1, … cnare the individual criteria [17, 18, 19, 20].

c1c2c3cn(c1c2c3cn1s12s13s1n1s121s23s2n1s131s231s3n1s1n1s2n1s3n1)E1

We compare each criterion and determine the preferences [15]. The element of the matrix sijexpresses the weights of ith and jth criterion. On the diagonal are displayed notes 1

sijwiwjE2

where wiis the weight of ith criterion and wjis the weight of jth criterion.

Saaty uses the nine‐escalate scale of the criteria evaluation, where value 1 means that criteria are equally significant, value 3 means that the first criteria are slightly more important than second criteria, value 5 means that the first criteria are strongly more important than the second criteria, value 7 means that the first criteria are of much greater importance than the second criteria and value 9 means that the extreme importance of the first criteria than the second criteria [18]. We determine the weight of each criterion according to the geometric mean

wi=(j=1nsij)1nj=1n(sij)1n E3

The final rating is then expressed in the following relationship:

Ui=j=1kuij×wjE4

where Uirepresents the overall significance of the variant with respect to the objective of the decision‐making process, uijexpressed the significances of the variants for the individual criteria, and wjexpresses the significance jof that criterion.

On the basis of an expert assessment of the superior manager, the significance for each group of competencies, and also within each group, is determined. Further, by means of Saaty’s matrix, the significance calculation and evaluation of three project managers in the Czech IT company engaged in designing websites is performed. Its organizational structure is formed by the project teams managed by three project managers and graphic designers, web developers and testers (see Figure 1). Every team works on many projects.

Figure 1.

Organizational structure of the project team in IT company.

Table 1 shows the professional duties of the work of project team members. In the project team, there are project managers, graphic designers, web developers and testers.

Project managerGraphic designerWeb developerTester
Web pages graphic proposalxx
Web pages contentxxx
HTML, CSS and JS codingxx
Change managementx
Modules and functionality developmentxx
Scripts developmentx
Functionality testingx
Fixing the bugsxx
Implementation and consultationx
User documentationxxx
Contract with costumerx
Handling with extern usersx
Revision list managementx

Table 1.

Requirements for expertise in the project team of IT firms.

4. Case study

Now, we create and evaluate the competency model of each team member using multi‐criteria decision‐making method, concrete Saaty’s method by evaluation of the criteria.

4.1. Project manager competency model

Each manager is assigned to one or more projects. According to the qualification standard, the following expert knowledge is important for the project manager:

  1. Orientation in related terminology,

  2. Definition of reasons of the project creation, targets and benefits of a particular project, understanding the differences between the target and the benefit of the project,

  3. Applications of the triple imperative of the project,

  4. Practical knowledge (application) of the project cycle,

  5. Applications of the logical framework of the project,

  6. Application of the management of the interested parties (stakeholders),

  7. Application of the project plan,

  8. Application of the principles of the formation of the organizational structure of the project.

In addition to these basic skills, the project manager must be able to coach the time frame of the project, the quality of the project, risks, the project scope, the project changes, resources, and also to be able to process the information and documentation in the project.

Based on the above skills, five groups of competences were created—professional, managerial, general, computer and language (see Figure 2). For the project manager, the professional competencies are important, that is, competencies relating to the application of knowledge of the project management. For this purpose, the so‐called competency model serves. The level of importance is adapted to the needs and requirements of each organization. The competency model should be simple, not too long, meaningful and understandable.

Figure 2.

Project manager competences structure.

In the proposed competency model (see Figure 2), five groups of competences were created, and individual partial competences that should be possessed by the project manager to handle the complex project management and effectively manage the projects of the highest quality were formulated within these groups. The results verified the suitability of Saaty’s matrix for the project manager position.

The professional competences 3.1–3.5 (knowledge of project management principals, application of project life cycles, knowledge of ISO standards (Internal Organization for Standardisation), risk management, budgeting and costing) have been evaluated with the same weight value 0.2.

Results show that the most important for the project manager are the general competences (weight 0.5232) and the managerial competences (weight 0.2777) (see Table 2).

1.2.3.4.GeomeanWeight
1.13372.817313250.5232
2.1/31351.495348780.2777
3.1/31/3130.759835690.1411
4.1/71/51/310.312393990.0580

Table 2.

Criteria evaluation of the main groups project managers’competences (own calculation).

The evaluation of the competences has been done inside of each group (see Tables 36). Inside the group of general competences (see Table 3) are the most important interpersonal skills (weight 0.3676) and responsibility (weight 0.2974).

1.1.11.21.31.4GeomeanWeight
1.1131/571.431569120.2847
1.21/31351.495348780.2974
1.351/3171.848147790.3676
1.41/71/51/710.252760080.0503

Table 3.

Evaluation of the general competences (own calculation).

2.2.12.22.32.4GeomeanWeight
2.111/7351.20990.2214
2.271533.20110.5857
2.31/31/5130.66870.1223
2.41/51/31/310.38610.0706

Table 4.

Evaluation of the managerial competences (own calculation).

4.4.14.2GeomeanWeight
4.1152.23610.8333
4.21/510.44720.1667

Table 5.

Evaluation of the computer competences (own calculation).

5.5.15.25.35.4GeomeanWeight
5.117533.27110.6300
5.21/71530.89390.1722
5.31/51/5130.58860.1134
5.41/31/31/310.43870.0845

Table 6.

Evaluation of the language competences (own calculation).

For the group of managerial competences, the most important competences are team leadership (weight 0.5857) and workflow management (0.2214; seeTable 4). The group of the professional competences has not been proved by using Saaty’s matrix because the superior marked the same weight inside the group (weight, 0.2). The most important competence inside the group computer competences is knowledge of software MS Project with weight 0.8333 (see Table 5). The most important competence inside the group language competences is English, weight is 0.63 (see Table 6).

The superior performed the evaluation of three project managers. As we can see, the best evaluation is project manager No. 3. (see Table 7).

1.WeightManager 1Manager 2Manager 3MAX
10.13774.129827814.129827815.506437085.50643708
20.26216.551469157.8617629811.7926444611.7926445
30.450411.260481315.764673818.0167700118.01677
40.05161.548335931.806391911.8063919131.80639191
50.098253.929994383.438745082.4562464863.92999438
Total27.420108533.001401539.5784899541.0522378
Percentage66.79%80.39%96.41%

Table 7.

Evaluation of the project managers by using main groups of competences (own calculation).

4.2. Graphic designer competency model

The other member of the project team is the graphic designer, whose competences are to be evaluated. On defining the competence groups, we go from these groups: general, managerial, professional and language ones. Among the general competences, we consider reliability, work under pressure, responsibility and interpersonal skills. Among the managerial competences, we consider workflow management, cooperation in team, problems handling and conflict solving. As for the professional and computer ones, we consider, mainly, knowledge of Adobe Photoshop, knowledge of Adobe Fireworks and knowledge of GIMP (General Image Manipulation Program). Among the language competences, we involve English and German (see Figure 3 andTables 812).

Figure 3.

Graphic designer competences structure.

1.2.3.4.GeomeanWeight
1.131/571.431569120.2464
2.1/311/550.759835690.1308
3.55153.343701520.5755
4.1/71/51/510.274941620.0473

Table 8.

Evaluation of the main competences (own calculation).

1.1.11.21.31.4GeomeanWeight
1.1131/571.4315691230.2847
1.21/31351.4953487810.2974
1.351/3171.848147790.3676
1.41/71/51/710.2527600770.0503

Table 9.

Evaluation of the general competences (own calculation).

2.2.12.22.32.4GeomeanWeight
2.111/7351.20990.2214
2.271533.20110.5857
2.31/31/5130.66870.1223
2.41/51/31/310.38610.0706

Table 10.

Evaluation of the managerial competences (own calculation).

3.3.13.23.3GeomeanWeight
3.11532.46620.6370
3.21/511/30.40550.1047
3.31/3311.00000.2583

Table 11.

Evaluation of the professional competences (own calculation).

4.4.14.2GeomeanWeight
4.1152.23610.8333
4.21/510.44720.1667

Table 12.

Evaluation of the language competences (own calculation).

The most important group of competences are the professional competences, weight 0.5755 (see Table 8). Inside the group of professional competences is the most important knowledge of Adobe Photoshop, weight 0.6370 (see Table 11).

4.3. Web developer competency model

For a web developer, we evaluate general, managerial, professional and language competences. The professional competences include computer knowledge of languages of Java Script, CSS, knowledge of HTML (Hypertext Markup Language), knowledge of PHP (Hypertext Preprocessor—one of the script languages), knowledge of JQuery and knowledge of databases (My SQL) (see Figure 4). Evaluation of the competences is calculated in Tables 1317.

Figure 4.

Web developer competences structure.

1.2.3.4.GeomeanWeight
1.131/51/30.66874030.1162
2.1/311/51/70.312393990.0543
3.55153.343701520.5809
4.371/511.431569120.2487

Table 13.

Evaluation of the main groups of competences (own calculation).

1.1.11.21.31.4GeomeanWeight
1.1131/551.316074010.2532
1.21/311/531.495348780.2877
1.355131.626576560.3129
1.41/51/31/310.759835690.1462

Table 14.

Evaluation of the general competences (own calculation).

2.2.12.22.32.4GeomeanWeight
2.111/31/71/50.31240.074483178
2.21/31351.49530.356531595
2.371/3131.62660.387819848
2.451/51/310.75980.181165379

Table 15.

Evaluation of the managerial competences (own calculation).

3.3.13.23.33.43.53.6GeomeanWeight
3.111/31/35311.0888670.1415
3.2311/35311.5704180.2041
3.33315733.1326030.4071
3.41/51/51/51530.7023120.0913
3.51/31/31/71/511/50.293170.0381
3.6111/31/3510.9066810.1178

Table 16.

Evaluation of the professional competences (own calculation).

4.4.14.2GeomeanWeight
4.1152.23610.8333
4.21/510.44720.1667

Table 17.

Evaluation of the language competences (own calculation).

The most important are groups of professional competences, with weight 0.5809 (see Table 13). In the group of professional competences is the most important knowledge of HTML, weight 0.4071 (see Table 16).

4.4. Tester competency model

For a tester, there are also important general, managerial, professional and language competences. The professional competences include knowledge of testing tools (Sabi, Silenium or EggPlant) and knowledge of methodics of the test types (see Figure 5). We evaluate competences and groups of competences again (see Tables 1822).

Figure 5.

Tester competences structure.

1.2.3.4.GeomeanWeight
1.131/571.431569120.2510
2.1/311/51/30.38609740.0677
3.55153.343701520.5864
4.1/71/5310.541082270.0949

Table 18.

Evaluation of the main groups of competences (own calculation).

1.1.11.21.31.4GeomeanWeight
1.1131/571.431569120.2847
1.21/31351.495348780.2974
1.351/3171.848147790.3676
1.41/71/51/710.252760080.0503

Table 19.

Evaluation of the general competences (own calculation).

2.2.12.22.32.4GeomeanWeight
2.111/7351.20990.2214
2.271533.20110.5857
2.31/31/5130.66870.1223
2.41/51/31/310.38610.0706

Table 20.

Evaluation of the managerial competences (own calculation).

3.3.13.2GeomeanWeight
3.111/30.577350.2500
3.2311.7320510.7500

Table 21.

Evaluation of the professional competences (own calculation).

4.4.14.2GeomeanWeight
4.1152.23610.8333
4.21/510.44720.1667

Table 22.

Evaluation of the language competences (own calculation).

The most important are the professional groups of competences, weight 0.5864 (see Table 18). Inside the group of professional competences is most important knowledge of methodics of the tests types, weight 0.75 (see Table 21).

5. Results

In this research, a mathematical approach for evaluating of the competences was considered. This approach compares each competency with each other and each group of competences with other groups of competences. By applying this approach, we can determine the importance of the competences and groups of competences at the superior. This means that we transform and evaluate qualitative criteria into the quantitative ones. This study enables to evaluate competences of all the team members. Evaluation of the competences can be involved in the complex employee evaluation. Contemporary assessments require not only performance evaluation but also the competence evaluation in the companies. Modern trends in the measurement of organizations targets are currently based on a performance assessment of employees based on and assessment of employees’ competence. Organization that wants to achieve a success must know the requirements on the competences on each work position and must develop them by the employees.

The companies primarily focused on the creation of the websites and e‐commerce. According to the results, we can say that the professional competences are the most important for job positions of graphic designer, web developer and tester. For the position, project managers are the most important general competences with weight 0.5232 (see Table 2), especially interpersonal skills. For the work position, graphic designers are the most important professional competences with weight 0.5755 (see Table 8), especially knowledge of Adobe Photoshop. For the position, web developers are the most important professional competences with weight 0.5809 (see Table 13), especially knowledge of HTML. For the position, testers are the most important professional competences with the weight 0.5864 (see Table 18), especially knowledge of methodics of the tests types. Current company management needs workers, not only professionally equipped but also self‐capable of solving the assignments on time, reliable and team‐oriented employees who can resist stress situations and problems that require quick and professional solution.

The multiple‐criteria evaluation methods were applied for the solution of complicated decision‐making problem in the small IT company. If we compare the evaluation of the project managers, we can reveal very significant impact in the language competences. The high demands on project managers in general competences, managerial competences and language competences are recognized. It is learnt that organization is focused on the international project, international customers and handling with them.

If we compare the competency models of the subordinates—graphic designer, web developer and tester—the professional competences are the most important for these positions. This fact is very significant. If we reveal the background, it is clear that the next step of the superior will be to conduct training courses for the reinforcing professional competences.

Advertisement

6. Conclusion

Any such organizational structure is based on direct responsibility for results. The project manager is responsible for the whole IT contract. He/she is responsible for a set of goals for all aspects of the work, as well as changes that were previously discussed with the customer. Project manager delegates tasks, which emerge within the project cycle. Created competency models can be an integral part of the modern complex system of evaluation. On the basis of application of the manager’s multiple‐criteria decision‐making methods, we particularly used Saaty’s method for criteria evaluation, as well as the benefit maximization method. To illustrate it, we performed a selection between three candidates for the job position of a project manager. It may be helpful not only for organizations implementing project management within the existing organizational structure but in particular also for organizations focused on management by projects to improve the quality of projects and the project team performance.

Acknowledgments

This chapter was supported within Operational Programme Education for Competitiveness—Project No. CZ. 1.07/2.3.00/20.0296.

© 2017 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite and reference

Link to this chapter Copy to clipboard

Cite this chapter Copy to clipboard

Blanka Bazsova (August 30th 2017). Evaluation of the Project Management Team Members by Using the MCDM, Key Issues for Management of Innovative Projects, Bernardo Llamas Moya, M. Dolores Storch de Gracia and Luis F. Mazadiego, IntechOpen, DOI: 10.5772/intechopen.69229. Available from:

chapter statistics

1049total chapter downloads

More statistics for editors and authors

Login to your personal dashboard for more detailed statistics on your publications.

Access personal reporting

Related Content

This Book

Next chapter

Handling Innovative People

By M. Dolores Storch de Gracia, Luis Mazadiego and Bernardo Llamas

Related Book

First chapter

The New Panama Canal

By Ana Belén Berrocal Menarguez and Juan Pous de la Flor

We are IntechOpen, the world's leading publisher of Open Access books. Built by scientists, for scientists. Our readership spans scientists, professors, researchers, librarians, and students, as well as business professionals. We share our knowledge and peer-reveiwed research papers with libraries, scientific and engineering societies, and also work with corporate R&D departments and government entities.

More About Us