Qualitative Analysis of Different CRM Evaluation Models

Customer relationship management (CRM) systems help companies to manage their business processes. Specially for sales, campaigns, contact management, etc. The simpler and more efficient the business processes are executed, the more profitable these companies can be. Therefore, the process of selecting and evaluating a CRM system is an important success factor for each company in every industry and in every company size. The qualitative analysis of CRM evaluation models examines the necessary phases and activities for selecting a new CRM system. It is important to go through the relevant phases in the selection in order to be able to make a decision in a structured manner.


Introduction
A customer relationship management (CRM) system is a business management software [1]. CRM software solutions usually include relevant modules for managing and executing business processes in a company such as operational CRM processes (marketing processes, sales processes, or service processes), analytical CRM processes (target group analysis, cross-selling analysis, or customer risk analysis), comprehensive processes (customer value analysis, customer segmentation, or customer characterization), or strategic processes (strategic analysis, strategic conception, or strategic controlling) [2]. A CRM system helps various parts of sales and marketing departments of an organization to organize and share data and knowledge, reduce costs, and improve the management of their business processes [3].
Various architectures of CRM systems exist on the market: proprietary CRM systems, modules as a part of ERP systems, and original equipment manufacturer (OEM) systems in enterprise system landscapes. CRM vendors and implementation partners offer similar bundles of functionalities in their software-products: a set of application modules that fit together. Each module includes a variety of functions [3].
There are different approaches for evaluating and implementing a CRM system. Thus, software evaluation is not a simple technical activity; it is a decision process including subjectivity and uncertainty with no possibility of arbitrary reduction [4]. As listed in the Appendix, several authors developed evaluation models with different methods between 1998 and 2017. These models are structured in different phase sequences and are used as a base for this paper.

Methodology
The methodology approach is structured in two phases: 1. Literature analysis 2. Qualitative content analysis (word analysis) In the first phase (literature analysis), different models in evaluating CRM systems are identified and used as a basis to develop a new CRM evaluation model. Twentyone different CRM evaluation models have been identified (see Appendix: Table 16).
In the second phase, all identified papers are processed via a qualitative content analysis (word analysis). In this analysis, sources, levels, phases, activities, and tasks are qualitatively coded. Identical or similar phrases are combined and derived [5] as shown in Table 1.

Different ways of sequencing
Based on the word analysis, the sequences of phases and activities are determined. Three different calculation types are used to determine the sequences: 1. The phases or activities were normalized in each evaluation model, and an average value was calculated.
2. Average value is given by the sum of the multiplication of nominations per phase/activity number, and the frequency of the mentions in this phase/ activity is divided by the amount of nominations per phase/activities.
3. The mean is determined from the occurrence of the identified first and last phase or activities of the analyzed models. B: sum of (amount of nominations per phase * phase number)/amount of nominations per phase/activity. C: normalized phase/activity (see normalization formula in the appendix: Table 17).

Analysis
The number of phases within the different CRM evaluation models varies between two and eight phases. In this analysis, only phases which are listed in minimum three of the 21 papers mentioned before were considered. On average 4,48 phases are present. Table 2 shows the considered phases.
The following terms are mentioned in the papers on the one hand as an activity in a selection phase and on the other hand as a phase: definition of requirements, longlist, market information, presentation, negotiation and contract, and decision.
Each phase is structured with different activities. In this analysis, only activities which are listed at least two times in the phases mentioned before are considered. In the 21 papers, these activities are called sub-phases or detail descriptions. Further identified and analyzed activities will be added after each phase.

P1 condition phase
The main activity in the first phase is clarification and definition of the strategy. This concerns the corporate strategy as well as the IT strategy and CRM strategy (

Other activities
It is important to differentiate the CRM evaluation project from other projects in the company and to focus on the selection and implementation of CRM processes with a new CRM system. Furthermore, basic requirements are clarified.

P2 project initialization phase
In Table 4, four activities are identified. At the beginning of the project initiation, a project manager is appointed by the management. The project manager defines the project and elaborates the project assignment in detail. One focus is detailing the goals, which have been defined by the management in the condition phase. Following this, the project will be launched.

Other activities
A project sponsor is set by the management to give the project the necessary importance in the company. The project is justified by the project manager who defines the project team. The detailed project scope is agreed with the project team and the project client. Another activity includes conducting a preliminary study on the feasibility of the CRM project. A kickoff is prepared and carried out. A detailed risk analysis is done and a risk management for the project is established. On the one hand the project budget is defined and on the other hand a exact timetable is defined and formulated.

P3 (business) process analysis phase
In the process analysis phase, all necessary business processes are defined. This phase includes identification, documentation, and evaluation of all CRM processes. In further step criteria for the evaluation of CRM systems, CRM providers and CRM business processes are defined ( Table 5).

Other activities
In business process analysis, processes can be further developed and modeled.

Activities
Nomination in phase

P4 requirement definition phase
In the requirement definition phase, the requirements (business and technical view) are determined, analyzed, defined, and prioritized. Specifications are developed for detailed requirements. These requirements affect the CRM processes, CRM systems, and CRM providers. The detailed requirements are listed in a requirement catalog, which is the basis for the specifications ( Table 6).

Other activities
Selection criteria are weighted, and the decision criteria are prioritized. CRM processes are developed. Target processes for the processing of CRM activities are defined.

P5 analysis phase
An actual analysis of all relevant CRM business processes must be carried and worked out. This includes all needs for functions and processes of the CRM systems. Furthermore, the future (CRM) processes will be developed, and the selection criteria will be evaluated ( Table 7).

Other activities
The following points are defined: • All necessary (CRM) processes evaluation criteria • Quantity structure such as number of offers, customers, etc.
• IT infrastructure • CRM improvement potentials

P6 market information phase
Within the market information phase, not only CRM systems but also CRM vendors and CRM implementation partners are determined and structured in an adequate market overview ( Table 8).

Other activities
In P6, a first longlist is created. This is done based on the evaluation of the market data. Detailed information is collected from the providers in the form of a specification.

Other activities
A longlist of CRM systems and CRM provider (vendor and implementation partner) is created ( Table 9). All possible CRM systems and CRM providers are listed. Based on the defined criteria, the CRM systems and CRM providers are evaluated.

Other activities
In addition to the (CRM) business processes (Table 10), the IT infrastructure is also evaluated.

P9 selection phase
The selection can be done over 1-n stages (basic selection or preselection, fine selection, and final selection). Depending on size and/or complexity of the evaluation, all three or even two sub-phases can be necessary (Figure 1).
The proposals from the selection phase are compared and evaluated according to predefined criteria. Presentations and workshops will be held for the decisionmaking of CRM software and CRM providers. An implementation plan is created ( Table 11).

Other activities
In the selection phase, it may be necessary to carry out market research again. Possible first tests of CRM systems are completed. Visits to reference customers of the CRM providers are prepared and carried out. For this, questions are prepared, and potential participants are determined. CRM business cases are defined. Possible CRM providers are contacted and the answered specifications pre-evaluated. Contract negotiation contracts are drafted, and initial negotiations are carried out. Detailed requirements are determined, the essential selection criteria are defined, and, if necessary, the criteria are weighted.   Table 11. Activities in the selection phase.

P10 presentation phase
During P10 phase, 1-n presentations will be held. This can also be considered as a proof of concept (Table 12).

Other activities
An agenda must be created for vendor presentations. The vendor presentations should be moderated. A documentation (e.g., minutes) of each vendor presentation is created. Each vendor presentation is rated by a defined Team via defined questionnaire. For the vendor presentations, data for the prototype is provided. The activities for the vendor presentations are also valid for the proof of concept.

Other activities
A shortlist of CRM systems and CRM providers (vendors and implementation partners) must be created ( Table 13). All CRM systems and CRM providers from the reduced longlist are collected in the list. Based on previous defined detailed criteria, the CRM systems and CRM providers are evaluated and reduced to a shortlist.

P12 negotiation and contract phase
The focus in this phase is to design the contracts (project contract, software contract, maintenance contract) and conduct contract negotiations (Table 14).

Other activities
The negotiation strategy is defined, and the following important contract elements are identified: • Costs for licenses, consulting, development • Travel expenses • Liability • Warranty • Delay • Approval conditions • Service-Level-Agreements (SLA) • etc.

P13 decision phase
In the decision phase, all decisions are collected, and a management summary is given. Final tests provide information and form the final basis for a definite decision ( Table 15).

Other activities
Possible reference customers (clients) of the final vendor are identified, contacted, and visited. During reference visits, the implementation project, the industry experience of the consultants, and the ongoing support will be discussed. The final contract negotiations are carried out. In the final phase, the most suitable system and the vendor or implementation partner are selected.

Conclusion
The study is interesting since viewpoints from a literature perspective as well as from a business side have been considered. The results of the qualitative content analysis (Figure 2) offer the possibility of the individual design of a procedure model for the selection of CRM systems.
The selection model can be changed or adjusted if necessary. This applies to the analyzed phases as well as to the respective activities within the phases.

Future work
In the qualitative analysis of CRM evaluation models, 13 phases including specific activities were identified. Based on this literature review, it is planned to conduct expert interviews. On the basis of these expert interviews, a new CRM evaluation model will be defined an afterwards tested via a quantitative analysis to