Weighing Price and Performance for Decisions for Multisource Pharmaceutical Bidding in Public Hospitals in Thailand

Abstract

Following a national law introduced in 2017 in Thailand, the selection of winning bidders for multisourced pharmaceuticals and medical supplies in public hospitals must reflect “price-performance” aligned with the principles of worthiness, transparency, efficiency, effectiveness and accountability. We describe how a practical tool using Multiple Criteria Decision Analysis (MCDA) for evidence-based decision making in hospital bidding (tender) was developed through a multi-stakeholder workshop format. The local leader of the initiative together with 2 international advisors guided the 37 workshop participants through five interactive steps for local adaptation of the previously developed and validated global MCDA-tool: (1) Criteria selection, (2) Scoring definition, (3) Weighting of price criterion, (4) Definition of cut-off point for price criterion, (5) Ranking and weighting of remaining criteria. All consensus judgments were imported to the decision tool which can later be used in the real-world situation in the hospitals to support the selection and document the underlying rationale. The final list of criteria differs from the previously suggested international template and now reflects the Thai decision priorities and current decision processes. In the book chapter, the resulting model will be presented and a pathway for implementation will be discussed.

1. Introduction

Pharmaceutical procurement in Thailand has a long history of deconcentration of procurement management and decisions to the Provincial Health Office (PHO) and all public hospitals. This includes the delegation of power to generate, retain, and use financial revenues according to regulations and subject to regular audits by the auditor general [1]. Thus, purchasing for hospital pharmaceuticals is strongly decentralized. Before the deployment of the Public Procurement Act BE 2560 (AD 2017) in 2017, the single selection criterion in the tender or bidding, as called in Thailand, was the lowest price. Since the establishment of Public Procurement Act, the bidder selection for multisourced supplies, including pharmaceutical and medical products, has been expanded beyond “price” to “price-performance” in order to align with the principles of the Act concerning worthiness, transparency, efficiency, effectiveness, and accountability. While public hospitals are encouraged to use performance criteria to determine the suppliers for pharmaceutical products, there is still a lack of a standard definition of what these criteria encompass and how important each of them is in making the decision. This may lead to a high level of variation between the hospitals on the formulary composition and in the methods used to shape the specific bidding process. To increase the overall quality and transparency based on the Public Procurement Act BE 2560 (AD 2017), the government is now requiring a solid rational and transparent documentation of hospital purchasing decisions.

Multiple-criteria decision analysis (MCDA) is a method which has been suggested as a tool for the evidence-based assessment of multisource pharmaceuticals in developing countries [2]. MCDA can help to consider multiple and sometimes conflicting criteria in the evaluation of the available alternatives [3].

By considering multiple criteria, individuals or groups can follow an explicit structure for arriving at a decision that best fulfills the criteria [4]. In 2016, a task force of the International Society for Pharmacoeconomics and Outcomes Research (www.ispor.org) set out to give guidance for the best practices of MCDA in healthcare decision-making [5, 6]. In this, the development and use of MCDA are described as an iterative process containing several elements, which may be adapted to the specific use by the key stakeholders involved in the decision. At the start, it is important to define the decision problem with the objective, the stakeholders, the expected alternatives, and the expected output. Based on this, criteria for evaluation can be determined which reflect the important attributes or drivers for the success in the decision. For each of the criteria, a scoring scale or graduation needs to be determined. If the criteria (or attributes) have a different importance in reaching the overall objective, the criteria will have to be weighted according to their importance [5, 6].

In the actual evaluation of the alternatives, the performance in each criterion is scored separately for the available alternatives and contributes with the predetermined weight, according to its relative importance, to the composite score reflecting the overall performance of the alternative.¹ When comparing alternatives, the MCDA will result in a “score profile” for each alternative and a composite score, which is generated by the MCDA model. The result is not the decision but structured information to better inform the decision to be made. MCDA is being used widely to inform decision-making in healthcare, including benefit-risk assessment of medicine, formulary listing, or reimbursement decisions [5, 7]. Examples for using MCDA in decision-making for multisource medicines in developing countries are emerging in several countries such as China, Thailand, or Egypt [8, 9]. MCDA could be a solution for hospitals in Thailand to select those products which best meet the needs of the patients, providers, and the national policy makers for healthcare.

Thailand has a strong history of using multiple-criteria decision analysis (MCDA), considering the value of pharmaceuticals as an important component in pharmaceutical policy planning, price negotiation, development of clinical practice guidelines, and communication with health professionals [10, 11]. It has been recognized that MCDA enhances the legitimacy of policy decisions by increasing the transparency, systematic nature, and inclusiveness of the process [10]. Examples for using the MCDA method on a national level for rational, transparent, and fair priority setting in the context of single-source drugs have been described [12].

The objective of this initiative was to develop a simple tool for improving decision-making in the hospital bidding setting, based on the MCDA methodology, through a multi-stakeholder workshop format to attain consensus. This tool should integrate a set of standard decision criteria, which (1) can be used by hospital purchasers to base bidding decisions on both performance and price, (2) would be applicable across diverse hospitals and institutions, but (3) would also allow for adaptation to local priorities.

2. Methods

On June 29, 2018, key stakeholders and experts in pharmaceutical bidding policies in Thailand came together on invitation by the Pharmaceutical Association of Thailand under Royal Patronage (PAT). The 37 active workshop participants represented multiple perspectives in Thailand (24 pharmaceutical purchasing (12 of these from leading hospitals), 7 academic pharmacy education leaders, 4 from the Ministry of Health, 1 from PAT, and 1 from an industry association) in addition to 2 observers from the regulatory perspective. During the 1-day workshop, all active participants were involved in developing an MCDA tool which can be used in making decisions in the hospital bidding setting.

Two international health policy advisors moderated the workshop following a validated MCDA calculation model and process for local adaptation [13]. Together with the local leader of the initiative, the international experts used a structured process as described in Figure 1 to prepare the workshop, align the participants’ expectations and knowledge at the workshop, and to guide the workshop participants through five steps for the local adaptation of the MCDA format. The international advisors conducted the workshop in English language. However, to ensure that all participants could follow the discussions and freely express their experiences and opinions at all times, independent of their knowledge of either Thai or English, the workshop was supported by a two-way simultaneous translation.

Figure 1.

Description of the entire process for developing a value-based decision tool for multisource pharmaceutical bidding in Thai hospitals.

The workshop started by defining all non-price criteria which may be relevant in the Thai decision process. These were defined starting from the basic decision criteria proposed by international health policy thought leaders [2] and an adapted set of these criteria which had gone through a preliminary adaptation to current Thai decision priorities before the workshop, by the local leadership team of the initiative (Step 1). This involved a detailed moderated discussion of each of the criteria and of the measures used for scoring each of the criteria (Step 2).

Subsequently, the participants determined the weight of the price criterion (Step 3) in the overall decision and the acceptable price range and cutoff point qualifying a product for positive ratings on the price criterion (Step 4). After this, the relative importance of each of the criteria in the overall decision was determined following the modified simple multi-attribute rating technique (SMART) method [9] for ranking and swing weighting of the criteria (Step 5). Steps 3–5 included anonymous voting by the participants using an audience response system (Ombea® with OMBEA ResponsePad™). The results of each voting were shown directly to the audience. In case of large variations or disagreements between the voters, the arguments of the participants in support of their votes were deliberated in open discussion followed by a second voting. For the voting on price and the cutoff point, the result was computed by assessing the median value. For the ranking of the criteria, the majority vote was used in repeated voting rounds to select the most important of the remaining criteria.

An important step after the workshop will be the testing and validation of the tool in a realistic setting (piloting) with monitoring of the results, the revision based on the learnings during the pilot, and, finally, the full implementation as summarized in the right part of Figure 1.

3. Results of the workshop

The discussion among the participants confirmed that currently there is no uniform evaluation method applied to bidding decision-making in hospitals and that there is a need for more consistency and better decision documentation on one side but also a need to adapt the weighting or criteria to local situations in cases where there are special environmental conditions. In addition, there was a general agreement that the decision should not solely be based on price, because major differences relating to quality and reliability or other factors with the healthcare impact are observed in real life between the products offered by different suppliers in Thailand. The advantages of using a consistent approach involving the MCDA methodology would be, on the one hand, the improved decision consistency and equity and, on the other hand, the high transparency and documentation of decisions versus all stakeholders with interest in the decision (e.g., manufacturers, government agencies, quality control, hospital administration, and providers).

3.1 Pre-workshop preparation

Based on experience with the local legal-structural setting, desk research, and a pre-workshop survey among the workshop participants, the leadership team (Thai academic pharmacy experts with international advisors) described 11 relevant criteria, including:

• Six product quality criteria, equivalence with the reference (original) product, stability and drug formulation, product quality determined by the Certificates of Analysis (CoA) of both the finished product and the active pharmaceutical ingredient (API), and the product specifications of both the finished product and the API

• Three criteria relating to the manufacturer quality, the manufacturing standard of both the finished product and the API, as well as the reliability of drug supply, pharmacovigilance, and added value service related to the product.

3.2 Workshop

3.2.1 Step 1: selection of non-price criteria

At the beginning of the interactive part of the workshop, the participants discussed and selected the most important non-price criteria which should be considered for determining the value of multisource pharmaceuticals starting from the set of criteria resulting from the pre-workshop preparation. During this discussion, several alterations were adopted so that it finally resulted in 10 non-price selection criteria, of which:

• Five relate specifically to the product (equivalence with the reference (original) product, stability, and drug formulation, the product quality determined by the CoA of the finished product, and the product specifications of both the finished product and the API).

• Three relate to the manufacturer (the manufacturing standard of both the finished product and the API as well as the reliability of drug supply).

• Two relate to additional value beyond the actual product (added value services at the hospital level and macroeconomic benefit in terms of local investments by the manufacturer).

Two other criteria have been considered but were not adopted to the final essential list of decision criteria: The Certificate of Analysis for the API was considered a prerequisite to enter the bidding and, therefore, would not be relevant for further differentiation between the products; pharmacovigilance was also not considered relevant for the multisource pharmaceuticals used in the hospital setting. In addition, it was warned that this criterion might introduce an unfair bias toward the originator products who are usually the only ones pursuing a pharmacovigilance database on the national or international level.

3.2.2 Step 2: criteria scoring

For all selected criteria, the measurement scales were discussed in some cases; the previously suggested rating were adapted by the participants as considered more appropriate in the Thai hospital setting. The detailed descriptions of the criteria scoring are listed in Tables 1 and 2.

Criterion name	Scoring (possible outcomes)	Score
Equivalence with the reference (original) product	No data on pharmaceutical equivalence	0%
	Pharmaceutical equivalence	10%
	Bioequivalence proven in compliance with the Thai FDA	30%
	Bioequivalence approved by the Thai FDA and with the European EMA or US FDA standard	70%
	Bioequivalence approved by the Thai FDA and with the European EMA or US FDA approval	80%
	Therapeutic efficacy or equivalence proven in a clinical trial	100%
Stability and drug formulation	No data on product expiry or stability	EXCL
	Have data (1) long-term study (full shelf life), but do not follow the ASEAN guidelines	10%
	Have data (1) long-term study (full shelf life) and follow the ASEAN guidelines	50%
	Have data (1) and (3) latest yearlong-term stability study or (4) in-use stability data for the drug which needed to be mixed before use (drug to be mixed before use must have “in-use stability data”) but do not follow the ASEAN guidelines or have only data (1) which follow the ASEAN guidelines	75%
	Have data (1) and (3) or (4) completely follow the ASEAN guidelines	100%
Quality: manufacturing standard finished product	Limited information on quality assurance	EXCL
	Country of origin GMP quality assurance	33%
	WHO GMP certification	67%
	EU or PIC/S GMP	100%
Quality: certificate of analysis (CoA) finished product	Not comply with registered finished product specification	EXCL
	Partially comply with registered finished product specification	EXCL
	Comply with registered finished product specification	100%
Quality: product specification (finished product)	Do not comply with registered specification	EXCL
	Follow the previous pharmacopeia version or in-house specification with topics not aligned with the general chapters	50%
	Follow updated pharmacopeia or in-house specification with topics recommended by pharmacopeia	100%

Table 1.

List of criteria with consensus scoring (qualitative descriptive) (Part 1).

Criterion name	Scoring (possible outcomes)	Score
Quality: manufacturing standard API	Limited information on quality assurance	EXCL
	Country of origin GMP quality assurance	33%
	WHO GMP certification	67%
	EU or PIC/S GMP	100%
Quality: product specification API	Not comply with registered specification	EXCL
	Follow the previous pharmacopeia version or in-house specification with topics not aligned with the general chapters	50%
	Follow updated pharmacopeia or in-house specification with topics recommended by pharmacopeia	100%
Added value service on the hospital level	No program or service	0%
	Low value (meets one criterion)	33%
	Moderate value (meets two criteria)	66%
	High value (meets three criteria)	100%
Macroeconomic benefit	The manufacturer has no local investment in the country	0%
	The manufacturer has minor local investment in the country	33%
	The manufacturer has moderate local investment in the country	67%
	The manufacturer has significant local investment in the country	100%
Reliability of drug supply	Major and multiple problems in the last 2 years	0%
	Minor and occasional problems in the last 2 years	20%
	Single precedence of supply problems in the last 2 years	50%
	No precedent of supply problems in the last 2 years	80%
	Manufacturer is financially capable and willing to guarantee supply	100%

Table 2.

List of criteria with consensus scoring (qualitative descriptive) (Part 2).

3.2.3 Step 3: weight of price criterion

Subsequently, the relative importance of the price criterion was determined by voting and was determined to be 40% of the overall decision, which is already established as a general ratio for chemical pharmaceutical products.

3.2.4 Step 4: scoring of price criterion

To enable a quantitative scoring function for the price criterion, the participants had to determine the cutoff point for the price. This median cutoff point was voted to be an excess price of 100% based on the acceptance threshold defined by the current guideline of Comptroller General’s Department. As shown in Figure 2, this means that all products with prices which are 100% or higher than the lowest price offered in the bidding would receive a score of zero for the pricing criterion in the evaluation.

Figure 2.

Graphic representation of the scoring for the procurement price difference in comparison to the lowest price product. The cutoff point determined in the workshop was an excess price of 100%. All prices higher than this cutoff point receive a score of 0%.

3.2.5 Step 5: ranking and weighting of non-price criteria by “SMART and swing” method

Finally, the selected criteria were ranked and rated for their weight in the final decision round [7]. The results are summarized in Table 3 in the column “Final weights.” The impact of each criterion on the final decision is shown in Figure 3.

Criterion	Measures	Rank (importance)	Final weights* (%)
Price	Quantitative	1	40
Equivalence with the reference (original) product	Qualitative	2	12.2
Product quality: certificate of analysis (CoA) finished product	Yes/no (no = exclusion)	3	8.7
Manufacturer quality: manufacturing standard finished product	Qualitative	4	8.7
Stability and drug formulation	Qualitative	5	7.3
Product quality: product specification (finished product)	Qualitative	6	5.8
Quality: product specification API	Qualitative	7	4.9
Quality: manufacturing standard API	Qualitative	8	4.0
Added value service on the hospital level	Qualitative	9	3.1
Reliability of drug supply	Qualitative	10	2.8
Macroeconomic benefit	Qualitative	11	2.5

Table 3.

Results of the consensus workshop for the relative importance of the evaluation criteria and their weight in the final score for each option.

Figure 3.

Impact of each decision criterion in the evaluation on the final decision (top axis: impact percentage). Abbreviations: CoA = certificate of analysis, Std. = standard, API = active pharmaceutical ingredient.

3.3 Workshop follow-up

Finally, all participants agreed that the resulting model seemed appropriate to be used for selecting bidding winners in Thai hospitals and that it should be tested in real-life pilot applications. Hence, after the MCDA model has in this workshop been adapted to the Thai hospital decision context by Thai stakeholders from a broad range of healthcare-related institutions, two additional steps are important to ensure applicability in the hospital setting: (1) piloting and validating in real-life decision processes and (2) refinement based on the experiences in the piloting in selected hospitals. Realizing such a pilot application will require involvement of all functions concerned in the specific hospital decision process and their agreement. This will be facilitated through support from the local leader of the initiative.

4. Discussion

In this report we have described a structured process to adapt the template of a validated international multiple-criteria scoring decision format to the specific setting of making performance-based decisions for public purchasing in Thai hospitals. The involvement of a broad stakeholder group in the design process is critical for the acceptance and subsequent implementation of the methodology. In this workshop, there were 37 participants who represented the user perspectives as well as the administrative or regulatory perspectives, the academic expertise and the perspectives of the pharmacist profession through PAT and of the industry by representation of the Pharmaceutical Research and Manufacturers Association (PReMA, http://www.prema.or.th).

Although using a standardized process for the workshop and a previously designed Excel-based model template [13], the participants were involved in each step of designing the specific Thai decision tool during a 1-day workshop. Continuing the participatory process by involving the important purchasing stakeholders in the pilots and the evaluation will further foster full transparency and improvement through user feedback, and, finally, it should support endorsement of the process in the specific Thai hospital bidding decision context. The participants agreed to the approach and considered the resulting MCDA tool to be suitable to improve the transparency and consistency of decision-making for multisource pharmaceuticals in Thai hospitals.

The MCDA model is a living instrument which can be revised when the priorities and needs in the healthcare system and policies change. Therefore, criteria can be included, excluded, or adapted at a later stage once a new consensus on the importance and the transparent measures for qualification is reached among the users of the instrument due to new developments and experiences. For example, it has been proposed by some participants that some flexibility might be advisable for the weighting of the price criterion when evaluating a specific type of product such as lifesaving medicines or a stricter scoring of the quality criteria when it comes to narrow therapeutic window drugs. An adaptation of the price weight depending on such considerations is possible on the hospital level if required. Another point for reconsideration after testing the tool in the real-life situation resulted from the discussion of the criterion of the “Certificate of Analysis for the finished product”: in the final model, the scoring was determined as either complying with the specifications (=100%) or not complying (exclusion). Thus, this criterion may be considered as another prerequisite to enter the bidding instead of a MCDA decision criterion.

The final list of criteria selected in this initial workshop for the resulting MCDA model shows some deviations from the criteria which were previously suggested by an international expert group [2] and which were selected in other countries which adapted the tool to their settings [13]. This reflects the active engagement and contribution of the participants who critically questioned and deliberated each of the proposed criteria in comparison to their current decision processes.

After successful piloting, evaluation, and refinement of the model based on the real-life experience, a roadmap for further dissemination and implementation should be developed.

The process presented here for the adaption of a multiple-criteria scoring format to the specific decision problem in Thailand follows the general process as suggested by the ISPOR task force [5, 6]. The core elements in this process were addressed with a group of Thai stakeholders in the hospital purchasing processes, who represented a range of hospitals.

While the selection of criteria, the ranking, and the weighting require adaptation to the specific decision problem and policy framework, the process itself can be generalized and transferred to other countries or organizations. The foundation for the course of work steps in preparation of the workshop, conduct of the workshop, and follow-up has been formed through the experience from three countries, Indonesia, Kazakhstan, and Vietnam [13, 14]. In each of these countries, different types of purchasing or tender decision problems (national purchasing, public tender) had been addressed. In this book, another example is presented, where the process was followed to develop a decision analysis tool to help provincial policy makers with the comparison of alternative insurance policies in China [15]. For that, a new set of decision criteria had to be compiled, which reflected the needs to be addressed by a policy change from the stakeholder perspective. However, despite that the objective to select the optimal future insurance policy is very different from the objective which guided the Thai initiative, the same process was followed in China: preparation with desk research and discussions with local stakeholders, workshop with consensus on the purpose of the tool, selection of the criteria, prioritization and ranking of the criteria, and follow-up with testing and piloting. The most important element is the engagement with and of those stakeholders who are concerned by the decision. How each of the procedural elements is shaped in the specific local application will strongly depend on the local preferences and needs. If the participating stakeholders are already familiar with the principles of MCDA, such as in Thailand, a 1-day workshop format may suffice. In Indonesia and Kazakhstan, a 2-day format was preferred which allowed for more presentation of the technical and methodological information before entering the interactive workshop parts. In all cases, we saw that the discussion at each step throughout the workshop is essential for building consensus.

Another important consideration should be that the current values and daily routines are considered when selecting the criteria. For example, the original list of internationally validated criteria was modulated in Thailand to satisfy the traditionally high use of specific quality measures. If the MCDA process shall be used for tender decisions, it will be important to train the users who may previously only have used the price or a very limited amount of information to select the winning bid. A standard template for dossier submission may facilitate the targeted supply of data and information for the manufacturers, a standard template for data as has been proposed by Brixner et al. in consequence of the experiences in the previous workshops [16]. Increasing experience with the implementation for further applications in CEHCs and ongoing evaluation and communication will help in the efficient implementation of new initiatives.

A limitation of the approach presented here for developing a MCDA tool to be used for hospital purchasing may be that the initial design is limited to the number of participants and the breadth of stakeholder groups involved in the design workshop. However, further involvement will be achieved throughout the piloting through communication of the experiences after each step of the process and through updating of the tool based on the practical experience.

5. Conclusions

The present paper describes how MCDA can be easily adapted to different countries and decision-making settings to improve the efficiency and transparency of the decision-making process, in the case of the undertaking of pharmaceutical bidding. The approach described here can be easily adapted to other countries and decision-making settings.

A short explanation of the principles of multiple-criteria decision analysis and the use in decisions on pharmaceuticals can be viewed at https://www.youtube.com/watch?v=7OoKJHvsUbo.

Conflict of interest

The research underlying this methodology was partially financed by Abbott Products Operations AG, Switzerland. The workshop was hosted by the Pharmaceutical Association of Thailand under Royal Patronage (PAT) who received funding by an unrestrictive educational grant. The international facilitation and the open-access publication fee were also financed by Abbott Products Operations AG, Switzerland.

Notes/thanks/other declarations

All workshop participants have actively contributed to the results of the workshop. We would specially thank the following participants, who have supported us during the preparation and the workshop and in the writing of this manuscript:

Ms. Jutatip Meepadung, Buddhachinaraj Hospital, Buddhachinaraj Hospital 90 Srithammaratipitak Road Amphoe Mueang Phitsanulok, Chang Wat Phitsanulok 65000.

Ms. Warawan Chungsivapornpong, Veterans General Hospital, Veterans General Hospital 123 Vibhavadi Rangsit Road, Samsennai Phayathai Bangkok 10400.

Mrs. Patcharin Suvanakoot, Ramathibodi Hospital, Bangkok.

Mrs. Montakarn Rahong; Bhumibol Adulyadej Hospital, Khlong Toei.

Mrs. Kannika Pongthranggoon, Thammasat University Hospital, Khlong Luang.

Dr. Suriyan Thengyai, School of Pharmacy, Walailak University, Nakhonsithammarat.

Mr. Thurdsak Piriyakakul, Ratchaburi Hospital, Ratchaburi.

Assoc. Prof. Payom Wongpoowarak, Faculty of Pharmacy, PSU (Prince of Songkla University), Songkhla.

Mr. Hatairat Panparipat, Rayong Hospital, Rayong.

Mr. Thanapoom Kiewchaum, Faculty of Pharmacy Chiang Mai University, Chiang Mai.

Mrs. Patcharawan Meesilp, Faculty of Pharmacy Chiang Mai University, Chiang Mai.

Mrs. Surirat Tangsangasaksri, Hatyai Hospital; Songkhla.