Codified Knowledge and Decisions in a Major eHealth Project: Efforts to Introduce the Electronic Health Record in Quebec

1. Introduction

Electronic health records (EHR) are being implemented in countries around the world, as the hope is that they will allow health personnel to have more timely access to information about patients and their medical history, and to improve quality of care. These objectives have translated into major expenditures and effort in order to try to implement national EHRs. The United States, for example, passed the Health Information Technology for Economic and Clinical Health (HITECH) provisions of the American Recovery and Reinvestment Act of 2009 [1]. In England, from 2003 to 2010, the government spent $20.6 billion on the National Programme for Information Technology (NPfIT) [2]. In Canada, it has been reported that $1.6 billion has been alloted to this effort [3].

Along with expenditures and efforts to implement these systems, there has recently been an interest in the evaluation of national projects to implement an EHR, both in terms of the results that have been achieved and the road that has been taken [2-4]. Although evaluations of the implementation of national EHRs are still relatively scarce, there have been some remarkable efforts. For example, in the case of England, Greenhalgh et al. [2] note that the national EHR project was expensive and well behind its implementation schedule, and indicate that its success was limited. Furthermore, these authors point out that "policymakers appeared to overlook many of their recommendations and persisted with some of the NPfIT’s most criticized components and implementation methods" [2](p. 533). To explain the limited uptake of their recommendations, they note that policymakers generally failed to learn from the evaluations that had been commissioned.

For their part, Stroetmann et al. [4] identify good practices and lessons learned in European countries, while noting that progress has not been simple: "... implementing them (eHealth strategies) has proven to be much more complex and time-consuming than initially anticipated. In addition, the complexity of eHealth as a management challenge has also been vastly underestimated" [4](p.1349).

In Canada, a similar broad assessment effort has also occurred recently [3]. A short explanation of certain aspects of the Canadian health system, as they relate to the EHR, could be useful at this point. Canada is a federation of provinces and territories, which are primarily responsible for providing health services to the citizens living in their geographic territory. The individual health organizations (private clinics, public hospitals) have a main role in maintaining the patient's record, although each provincial Ministry of Health has oversight over the public hospitals and, to a lesser extent, some influence on private physician's practices. The provincial and federal Ministries of Health set policy and decide if they want to support EHR projects. However, at the turn of the century (2001), the federal government wanted to insure that patient record systems would be compatible so that a citizen who moved from one province to another could still provide a record of treatment to his or her new physician. This resulted in an organization (Infoway) and a large federal fund to facilitate the development of interoperable systems. This initiative will be further described in a particular section below.

If we return now to the recent analysis of this federal initiative (Infoway), Rozenblum et al. [3] observe that "Canada continues to lag behind other Western countries in adopting electronic health records" [3]. These authors report that, across the provinces and for 2009, only 36% of physicians were using an EHR, and that this was much lower than for countries such as Australia, the United Kingdom, New Zealand, and the Netherlands. The authors carried out a set of telephone interviews of key informants from three provinces across Canada (not including Quebec), and these informants indicated a number of reasons for the poor rate of implementation: lack of an e-health policy (federally and in concert with the provinces), inadequate involvement of clinicians, failure to establish a business case for using electronic health records, a focus on national rather than regional interoperability, and inflexibility in approach.

In this chapter we wish to take a different approach, by examining decisions that have been made during the process of implementing a provincial EHR and the influence that codified knowledge may have had on those decisions. Our general goal is to document, and if possible, gain deeper understanding of the reasons for which scientific evidence in particular may be considered or not in EHR projects. We will argue that in Quebec, but probably not unique to it, key decisions regarding EHR implementation have not always been based on available knowledge, and this could help to explain the limited success of EHR projects here and perhaps elsewhere.

Currently, under the label of evidence-based medicine, there is substantial interest in, and effort by, health researchers to examine empirical evidence in order to identify actions and conditions that are most likely to result in positive outcomes for patients. This chapter builds on this strategy. Where possible, we will point out certain research results, expertise, and evaluation conclusions (which we will call the codified knowledge) that were available to decision-makers at the time of their decisions. We will identify specific occasions during which the available knowledge influenced decision-makers (or was available and did not have an influence), and we will explore the reasons for which this may or may not have occurred.

2. Methodology

Our presentation and analysis centre on decisions, projects, and events, along with aspects of the political, technological, and social context in Quebec, Canada. As we have indicated earlier, because of provincial jurisdiction in the area of health in Canada, significant decisions were made in this area by the provincial government, and for this reason, the province is a useful unit of analysis. The Federal government also funded some projects, and these too represented significant decisions. These are the main decisions that will be considered here, although we also recognize that many other decisions took place and likely affected the projects, such as the choice of partners, the creation and composition of steering committees, the project objectives, the technology used, as well as implementation, training, and recruitment strategies.

The analysis is based on a multiple case study approach [5] which examined the evolution of representative EHR projects in the Province of Quebec from the first large scale pilot project at the beginning of the 1990’s until just before the development of the current Quebec EHR project in the late 2000s (le Dossier santé du Québec, or DSQ). Although this approach limits the amount of detail that we can provide for any one project or event, we believe that this longer historical view is also useful. Because of the scope of the current project (the DSQ) and ongoing analysis of it (for example, annual reports by the Auditor General of Quebec), we will not include it here, except for an early initiative that was part of it. Further analysis may eventually find that many of our conclusions also apply to it.

For the analysis here and because of space limitations, a subset of projects or actions were chosen for this chapter which are either representative of similar projects or which, retrospectively, can be considered to have been a significant event (see Table 1). The choice of projects and activities that are discussed here was reached by consensus among the authors. The projects selected also allow us to present and discuss several factors which influenced the uptake of codified knowledge by the EHR projects in Quebec.

By codified knowledge, we mean knowledge that has been written down in the form of reports or articles. The particular type that we are mostly concerned with here is scientific evidence that has been produced by evaluation studies of projects, or research into the actual benefits of an EHR, or factors that affect the implementation of EHR projects. Other pertinent codified knowledge may have been produced in fields other than scientific research. In the case of the EHR, an important area of knowledge consists of an understanding of legislation that controls the storage or transmission of a patient's clinical information. Rather than to stretch the notion of scientific evidence in order to accommodate this type of knowledge, we prefer the more general expression of codified knowledge.

For each case study, we analyzed public descriptions, evaluations, and reports concerning the set of projects and events. This information was supplemented by semi-structured interviews with decision-makers, which allowed us to explore the context in which decisions were made, how available knowledge was actually used (or not) to make decisions, and factors which influenced its use. A total of 31 interviews were completed with key decision-makers who were selected because of their involvement in EHR projects either as policy-makers, evaluators, or project leaders. For the first case study, one of the authors (JPF) co-directed the evaluation of it. In relation to the final case study, two of the authors (MPG and JD) observed several meetings between December 2007 and March 2009 and conducted informal interviews with members of a consulting team.

For each project or activity discussed, we will briefly note its objectives, the major decisions that were made, whether certain forces may have influenced the decisions, the types of difficulties the project or activity may have experienced, and the knowledge that was gained from it. The following table identifies the projects and activities that will be presented.

Project/Activity	Approximate Time Period
The Rimouski Smart Health Card Project	1993-1995
The Role of the Commision d'accès à l'information (CAI)	1993 -
Canada Health Infostructure Partnerships Program	2001-2003
Changes in the Provincial Laws	2001, 2005
Canada Health Infoway (inclusion of Quebec)	2004 -
The RSVP: a change management strategy	2007-2009

Table 1.

Major Projects and Activities in Quebec, on the Road to an EHR

3. Presentation of the case studies

3.1. The Rimouski smart health card project

One of the earliest electronic health record projects in Quebec took place in a small city in Quebec, Rimouski, from 1993 to 1995. This was one of the first large-scale experiments of this type in the world, although it built on a similar concept that was active in France at the time (for a later description of a similar concept, see [6]). The Quebec government decided to encourage the project, provided financial assistance to it (4 million dollars Canadian), and also sponsored an evaluation [7]. The RAMQ (the provincial organization which manages health insurance, pays private physicians for their interventions, and issues health cards) was the main project manager and associated several stakeholders such as professional associations. Proponents of the project hoped that it would help to meet the objectives of the dominant ideology of the time, which was a shift to ambulatory care (see, for example [8]), and the creation of a better continuum of services by the set of professionals that a patient could see. A report from an important consultation on the reform of health care in Quebec (known as the Commission Rochon [9]) had also recommended that a trial be carried out with the card. Two implicit sub-objectives of the experiment were to decrease drug interactions, and to decrease the duplication of lab tests.

A group of academic experts was set up in order to plan the implementation of the pilot smart health card project. They consulted the literature (which was scarce at the time), along with a research group that had implemented the technology in France [10]. The group in Quebec included a variety of researchers from various fields (health services, political science, evaluation, information technology), as well as medical experts. Significant consideration was given to clinical processes and inter-professional collaboration. A local physician was also involved in the project team as a strategy in order to increase the acceptance by the medical community. The project team used knowledge gained from European projects in particular, to inform the implementation of this pilot project. The project was envisioned to be a way to produce new knowledge about the benefits of the smart card for the health care system, but also to help develop strategies that would be useful for a future large scale implementation.

A specific health card was used for this project, and had a smart chip on it that could record information. The concept was essentially to create a portable personal medical record. For example, patients’ allergies or reactions to medication could be noted on the card. Citizens who wanted to participate had to apply and consent to the project (this consent could be withdrawn later), and an office was created to facilitate this. Health professionals needed a personal identification number (PIN) in order to access information on the card, as well as a card reader and IBM microcomputer. It is useful to remember that, at the time, pharmacists often had experience using a computer, but this was not the case for physicians or nurses. When a patient went to visit a health professional, they could present the card and so the information on the card could be shown to him or her, and if suitably equipped and inclined, the professional could in turn add to it. As well, a computer application was offered to pharmacists so that they could check for potential drug interactions. At the end of the project, there were 7248 users (a portion of the city's population), along with 299 physicians, pharmacists, and nurses (almost all of the health professionals who had been identified as potential users) [7].

Overall, 73% of the population who had a card used it. Among these patients, about half used it with a pharmacist and a physician [7]. One of the decision aids that was most used (often by pharmacists) was an application that checked for potential drug interactions. The rate of use increased until July 1994, and then decreased, and this coincided with a decision to not buy more smart cards for the project. As well, there were indications that the use of two different computer systems was an irritant, that use of the card increased the time needed for a consultation, and there were occasional problems updating information on the card. In the summer of 1995, a sample of patients and health professionals were polled, and they generally indicated that the card facilitated the communication of health information, in a secure manner. In spite of the problem related to the need to update information, more than half of the health professionals felt that it provided quicker access to information, allowed them to provide better recommendations, and were satisfied with its quality. As well, many indicated that certain characteristics of the project, in particular the slowness of the system, did not facilitate its systematic use.

Some of the professionals felt that the smart health card provided benefits that out-weighed the irritants and disadvantages, whereas for others, the advantages were not sufficient. Most found it relatively easy to use, although the system was found to be too slow. In general, the physicians and pharmacists indicated that the system contributed to a decrease in drug interactions [7].

The evaluation report included a number of recommendations. For example, such a system needed to be compatible with a computer system in a physician's office or a pharmacy, so as to minimize work for the professional. As noted previously, the information provided by a system should also be up-to-date, and consideration should be given to ways to encourage patients to adopt the behavior that project planners hoped they would have, such as presenting the card (this affected whether the information was indeed up-to-date). Local support for the users was also indicated to be a critical factor, as well as the implication of a local leader. As well, the confidentiality of the health information had to be insured.

In a later analysis of the project, Aubert and Bernard [11] add that the training requirements of the health professionals was a significant aspect of the project. As well, they note that physicians' offices were generally not equipped with computers. In addition, according to these authors, reflection about changes in clinical practices that could maximize the use of such technology seemed to be absent (although it was clear that the project hoped to decrease drug interactions, and reduce the number of lab exams).

Although the evaluation included the identification of actions which could contribute to the generalization of the project elsewhere, there was no such immediate follow-up. A later project in the health region of Laval (north of Montreal), from 1999 to 2001, used another smart card, although patient health information was stored in a database managed by the RAMQ. The Quebec government considered the smart card to be a viable approach, and continued to consider its potential use as late as 2002. In particular, the RAMQ announced in 2001 that within three years it would provide the card to 80% of the population [11].

3.2. The role of the Commision d'Accès à l'Information (CAI)

In Quebec, the CAI (Commission of Access to Information; our translation) was created in 1982, and had a significant influence on events and practices in relation to the EHR. In this section we will describe this organization and some of its activities in relation to electronic health records, and briefly comment on its influence in relation to the development of the EHR in Quebec. From the outset, we should note that this organization was not associated with a single electronic patient record project, but rather several of them. As well, the information it produced was not scientific evidence in the sense that we are generally using it here, but could be considered to have produced pertinent codified knowledge. As we hope to show, the organization produced information of a particular kind, but nevertheless, information that project proponents and the provincial government had to consider.

In 1982, a law was passed in Quebec, which had an objective of instituting better management and protection of information concerning individuals, which was held by the government. An organization was created at the same time to administer this law (the CAI) and it was "responsible for overseeing compliance with the obligations imposed upon public bodies concerning the collection, storage, use and communication of personal information" (http://www.cai.gouv.qc.ca/index-en.html). Health and social service institutions, which created databases that contained health information, fell under the mandate of this organization. Furthermore, the CAI was authorized to comment on draft bills in this area and to assess pilot projects.

Because of this mandate, the CAI decided to analyze various EHR pilot projects that took place in Quebec from 1990 to approximately 2005, as well as on proposed provincial legislation. To be clear, it was a decision-maker in this way, since it opted to comment on the EHR projects and hoped to steer them in a way that would allow them to be compliant with various laws, but it did not have a direct role in the selection of projects which were carried out. As an example of one activity, it wrote a report on the Smart Card and its trial in Rimouski [11]. Although the CAI considered the approaches taken in this project to be useful, problems with the implementation of the card were also noted, such as allowing nurses and pharmacists to share a card, and the fact that it consisted of a partial patient record.

A later, more general study carried out by the CAI in 2001 summarizes their examination of several pilot projects and outlines their general concern: "The transfer of clinical information and the right to privacy are not contradictory. However, their conciliation is complex and fragile, especially when they are implemented" [12](our translation, p. 2). A particular concern was also presented in this report, in that the authors indicate that the provincial health insurance organization, the RAMQ, should not be the organization that handles a centralized health record, and it discussed the reasons that motivated it to take this position.

This 2001 report also summarized the state of legislation in this area: "In relation to the laws that relate to the protection of personal health information, these are numerous, haphazard, sometimes contradictory, and often insufficient in relation to these new health technologies" [12](p. 35). An analysis of the state of the current legislation (in 2001) was also carried out. At the end of this analysis, several questions were raised about the legislative foundation for certain practices concerning the communication of information, such as who would be responsible for the information that is collected, and the regulations that would control this. The obvious call for further legislation which was contained in this report eventually lead to some proposed laws, and to subsequent analysis by the CAI of these proposed laws.

In general, one could conclude that the work of the CAI had an influence on the legislation that framed the creation of the current provincial EHR project. For example, in 2004, a law was proposed that would have modified the main law in Quebec that governs health and social services, and the CAI provided an analysis of this [13]. One of the recommendations in it was that staff in a health organization should not be able to provide patient information to staff in other health organizations without the consent of the patient. As well, the authors recommended that the provincial health insurance organization (RAMQ) should not create a database that would document the medications used by citizens.

The fact that the CAI examined several projects and proposed laws over an extended period of time indicates that senior managers in the CAI decided that this was an area that needed to be examined and monitored. It was also clear that this organization was sufficiently independent from the elected government in order to critically examine the activities of the government. Since many managers of pilot projects informed the CAI of their projects and actively solicited its opinion about a project, this suggests that these project managers were concerned that their projects could potentially contravene existing legislation. At the time, managers felt that the rules were ambiguous about what a project needed to do in order to conform to existing legislation, and many meeting hours were spent discussing what to do about this problem.

3.3. Canada Health Infostructure Partnerships Program

At the turn of the millennium (2000-2002), the Canada Health Infostructure Partnerships Program (CHIPP) was a two-year, $80 million, 50-50 shared-cost program with an objective of supporting collaboration, innovation, and renewal in health care delivery through the use of information and communication technologies. One of the specific areas that was funded was the creation of EHRs. Project applicants were required to complete a literature review, and to include an evaluation.

In Quebec, three EHR projects were funded through the CHIPP program. One of these, called MOXXI, or the Medical Office of the XXI century, was a research project to examine the potential benefits of an electronic prescription and drug management system for primary care physicians [14]. Two of the project objectives were to streamline the prescription process and to improve patient safety. Another aspect examined was the time requirements and other ergonomic issues related to the use of the system. One of the interesting questions implicit in this project was whether such a computerized system would be a help or a hindrance for the physician. This sort of question had generally not been asked before, and is still very rare in Canada. As one of the outcomes of the project, the authors concluded [15]: "However, there are considerable barriers to developing the interfaces necessary with community-based pharmacies to permit exchange of information between physicians and pharmacists" [15].

A second CHIPP project called RIGIC was essentially a project to create a networked EHR system which would be shared between two hospitals, for oncology patients. One of the outcomes of the project was that shared protocols (between the hospitals) had to be established in terms of access and confidentiality procedures. New policies and procedures also had to be developed in relation to this, including the development of a consent management system. As well, the participants became more aware of the importance of change management, and the need for recurrent funding in order to transition the research project into an ongoing operation.

A third project, the SI-RIL project, consisted of the development of an integrated network system in a health region (the health region of Laval, north of Montreal), for ambulatory patients. This system was designed to enable the sharing of physician orders and patient health information among health workers in the region, with a view to regional co-ordination of care. The participants were: the local hospital, four community health centres (CLSCs), and more than one hundred general physicians.

During the course of the project, a number of factors were observed which influenced its development [16]. For example, the Minister of Health changed repeatedly during this period, and it was stated that these changes increased the level of uncertainty. As well, health professionals were not necessarily available to participate in the project, so consulting them was not an easy matter.

One of the outcomes was a set of standardized clinical forms which were agreed to by health professionals, such as a form to assess patient autonomy. Another was that project proponents realized that it was difficult to comply completely with certain outdated laws, and they communicated this problem to provincial authorities. Still another outcome was that participants noted that significant resources were required in order to manage the change process. Observations were gathered about impacts on the work processes, and a recommendation was made to the effect that these had to be considered. Quite a few documents were produced by the project, which can be taken to be an indicator of a high amount of consultation, discussion, and analysis which took place during the project.

It would appear that the SI-RIL project was one of the most advanced primary care EHR projects in Canada at the time. All other networked EHR projects in Quebec involved a small group of hospitals (there were a couple of networked hospital projects at the time). An evaluation of part of the SI-RIL project was also carried out by a research group [17]. For example, the authors highlighted the fact that these were very complex projects, and that any one of three types of problems (technological, organizational, clinical) could lead to the end of the project [17]. In addition, also noted in the report, was the desire of project participants to find a way to avoid illegal transmission or storage of patient information.

In the case of the RIGIC and SI-RIL projects, the regional health authority was closely integrated in the project planning and execution. We mention this since the regional health authority has regular communication with the provincial Ministry of Health. This highlights the fact that the provincial government was fully informed of the difficulties met by the proponents of the three projects, along with the outcomes, at least by 2002, when the project funding ended.

Another significant dimension of these projects was that all of them were based on networked health information systems. Patient information was being made available to professionals in different organizations, without patient intervention. This fact contrasts with a concomitant provincial plan to develop an electronic patient card.

3.4. Changes in the provincial laws

In December 2001, the provincial government proposed a law that would have created a provincial smart card and replaced the existing health card. It should be pointed out that the Laval region had also been involved in a second electronic health card project, which began in 1999, and this project was again sponsored by the RAMQ [12]. For those who proposed the law, this project provided an example of the type of technology that would be used. One of the technological differences with the Rimouski project was that a patient's clinical information was centralized in a databank managed by the RAMQ.

The purpose of the proposed law was to allow the development of a provincial electronic patient record, enabled by the patient's use of a smart card, although the patient record would have only been a summary. A law was also necessary in order to allow easier communication of health information between health organizations, which were subject to restrictions because of laws that were active at the time. With the proposed law, the RAMQ would have become the government's manager of the smart card, and the card would have included both administrative information (the name, address, and date of birth of the patient), and health information. In contrast with the Rimouski project, this information was to be centralized in a database. As part of the legislation development process, the government included a public consultation on the law.

More than 50 groups and individuals responded with written comments about the proposed law [18]. There was a public debate about the law in the media, which gave rise to what one commentator considered to be a sterile debate often based on misconceptions [18]. Professional organizations, such as the provincial Nurses Association, Association of Pharmacists, and an association of physicians, also joined this debate. One of the specific criticisms of the physicians was that a centralized database should not be created, and that a government organization should not manage health summaries (a comment directed at the RAMQ) [19]. The CAI also questioned the inclusion of administrative and clinical information together in one database, and the management of this database by the RAMQ. They also reminded the government of the ongoing need (implicitly beyond this law), to widen the legislation that would frame the management and access to health information. It highlighted the recent creation of family medicine groups, and suggested that such groups could be the focus of efforts to computerize health information, since most health services were provided by primary care services. Finally, it issued a clear warning about the project:

... the law that the government has tabled entails solutions that could jeopardize the confidentiality of health information. The text has a number of grey zones which do not allow us to know with precision the orientations that will be followed. [20] (p. 28, our translation)

In one of our interviews, a government official had this to say about the public debate at the time:

We could say that the proposed law was a major error, the law that was tabled became the object of several controversies; several philosophers, the citizens thought that we wanted to create a 'big brother', that it was a way for the RAMQ to gain power over the health system; the physicians did not want to be controlled and were concerned about the way it was developing. (government official)

The result of the controversy and the questions that were raised was that the government withdrew the proposed legislation. In theory, it could have revised the proposed law, but perhaps because of the extent and range of the opposition, it decided to abandon it completely.

Two and a half years later, and under a new government, another law was proposed in December 2004 (proposed law number 83). Although the CAI and other organizations again tabled reports about the proposed law, there was relatively little public opposition to it, and it was adopted almost a year later. It was clear, however, that it had taken many years for the government to respond to the request from the CAI to enact such a law. With this law, clinical information would now be managed by regional health organizations (there are 18 health regions in Quebec), and this regional organization, even though it was part of the government, was separate from the RAMQ.

3.5. Canada Health Infoway

In 2001, Canada Health Infoway (Infoway), was created in order to accelerate the adoption of compatible electronic health records across the country. This organization was funded by the federal government, but accountable also to the provincial and territories' health ministers, all of whom were represented on a Board of Directors. Projects in the provinces were generally funded equally by Infoway and provincial governments. Over ten years, $1.6 billion Canadian was allocated to this project [3].

One of the main priorities of Infoway was to incite provinces to adopt technologies which would allow patient records in one province to be available to health providers in other provinces. Another rationale was that it would be possible to save money and time by having common technologies deployed across provinces.

In a report provided in 2005, consultants to Infoway proposed a three-phase approach to health technologies. The first phase would consolidate patient data for viewing by health professionals, the second phase would allow the documentation of care and provide basic decision support, and the third phase would allow physician orders and decision support. The estimated cost at that time of reaching the 3rd phase was $10 billion.

In was not until 2004 that the Quebec government decided to join Infoway. Informants indicated that Quebec did not participate in Infoway from 2001 to 2004, and at the end of this period, the situation was such that it was the only province which had not yet joined the project. One senior official indicated that the discussions during this period were "very, very political." The decision to join Infoway was made in 2004, and was taken by a newly elected Liberal provincial government (the previous ruling party was the Parti Québécois, a sovereignist party). This decision implied that Quebec would need to harmonize its EHR plans with those of Infoway and have its projects and deliverables approved by Infoway in order to receive the funding contribution by the federal government. In particular, local companies which had been supplying technology to Quebec health institutions, would have to be compliant with the standards defined by Infoway (for an analysis of the development of standards in Canada, see [21]).

One of the influences that seemed to affect the Quebec government's decision to join Infoway, seemed to be the possibility of obtaining financial advantages. As previously noted, the federal government provided a significant grant to Quebec (and other provinces) as a financial incentive. A form of group purchase of certain technologies also became possible. As a government official commented: "In the end, Quebec chose the same solution (PACS technology) which saved us about 16 million, because they (the technology company) gave us a discount".

3.6. The RSVP: An evidence-based change management strategy

In the subsequent years, work continued with the planning and development of a province-wide EHR, and culminated in the launch of another pilot project in Quebec City, in May 2008. In this pilot project, a patient record system was used that was to be expanded across the province, the DSQ or Dossier santé du Québec. The DSQ had been developed within the Infoway framework, thus the federal government funded about half of it.

From 2007 to 2009, the DSQ project proponents carried out an implementation strategy, which was called the Network for Peer Support, or RSVP (Réseau de soutien et de validation par les pairs). The RSVP strategy aimed to facilitate the appropriation of the DSQ throughout the province through the mobilization of an interdisciplinary group of healthcare professionals (general practitioners (GPs), specialists, nurses and pharmacists). These groups were represented by clinical champions for each of the 18 health regions of the province. Although the pilot project was only taking place in Quebec City, the belief behind the RSVP activity was that the project would soon be extended to the rest of the province, and so it was necessary to include professionals from all the regions. The selection of these champions was made by the various regional health agencies according to a set of criteria, including their credibility, recognition within their peer group, and their interest in informatics. The stated roles of the RSVP members [22] were to:

1. Participate in the elaboration and the realization of the change management strategy at the provincial level, by validating and adapting the promotion tools developed by the DSQ team to the needs and expectations of the professional groups that they represented;

2. Promote the appropriation of the DSQ within their provincial professional association;

3. Identify the challenges and obstacles to DSQ implementation in their region and help to identify solutions;

4. Exchange information with other RSVP participants regarding the DSQ in their region, provide tips for facilitating appropriation by their peers.

5. Ensure constant communication with their peers regarding the DSQ and answer their questions;

6. Work in collaboration with other RSVP members, the regional DSQ group, and other local groups in order to find solutions to challenges and difficulties identified by the DSQ change management team or their peers;

7. Make recommendations and participate in decisions which concern all aspects of the appropriation of the DSQ in their region.

In the DSQ change management documentation [23], it was clear that this strategy was directly informed by evidence in the literature related to the importance of change management and consideration of the critical role that clinical leaders play in the implementation of large scale EHR projects. As well, the managers of the RSVP had requested and received presentations by Quebec experts on change management. The DSQ team had contracted a consulting group, and this group proposed the RSVP strategy. For example, RSVP members were offered training and support in order to prepare a promotion plan for the DSQ in their region. RSVP members were also invited to answer a questionnaire, which measured various aspects pertinent to change management.

Over the course of the RSVP strategy, the need for an interdisciplinary team was expressed but once this team was created, new challenges arose. The rationale for an interdisciplinary team was never directly expressed in the written internal documents, but interviews with the change management team indicated that this decision was based mainly on the perception that resistance from nurses and pharmacists could become an important obstacle, if the promotion of the DSQ was left solely to physicians.

However, important limits to the interdisciplinary team started to appear after a first large group meeting. The physicians who had already had extensive discussions about the project, had a more extensive understanding of the project, compared to the nurses and pharmacists who were new to the group. The large number of participants at the interdisciplinary meetings (more than 50 RSVP members plus the DSQ management group and invited speakers) made it much more formal than the earlier meetings which had only included physicians. RSVP members from the three professional groups agreed that it was extremely difficult to have constructive exchanges within such a large group. They also believed that issues specific to each profession (remuneration for physicians, access to diagnostic data for pharmacists, or responsibility for data entry for nurses) could not be easily addressed in an interdisciplinary group. A solution that was put in place was to alternate meetings specific for nurses, pharmacists, and physicians, and the interdisciplinary meeting. This strategy was costly given the fact that the DSQ team had to reimburse participants’ travel and time. A teleconference was then used, given these constraints, but many RSVP members found that the exchange did not progress as well as with the in-person meetings.

This project was one of the first to formally include a team of experts in change management, and to develop an explicit change strategy. Still, the impact expected from the RSVP did not occur. This was mainly due to factors unrelated to the implementation strategy, such as the delays in the DSQ technology. In fact, the pilot project had revealed several problems, notably the slowness of the system. Therefore, the plan of deploying other pilot projects in other regions had to be altered during the course of the RSVP. Participants were kept informed about the global DSQ strategy but, according to our interviews, it became very difficult for them to know exactly what was going on at the provincial level, and to play their role of DSQ representative within their region. According to one physician, it was “impossible to know who was holding the steering wheel and to know where we were going”.

The RSVP was an exemplar of a provincial government strategy informed by best practices in change management and the implementation of innovations. However, it took place in an environment that was extremely complex and was based on a technology that was not mature. The change from a group composed mainly of physicians to an interdisciplinary group led to a profound change in the social dynamics, which also hampered the realization of the RSVP’s mission.

4. Discussion of the case studies

An analysis of the projects and events described above will now be carried out with a view to highlighting the possible influence of codified knowledge in relation to decisions that determined the projects and events described above. The main decisions that we have highlighted are the ones in which a government or government organization (example, Health Canada) decided to fund a particular project, or to propose or withdraw a proposed law. In the case of the CAI, several decisions were made to examine and to comment on the patient information practices of various projects.

However, when presenting several of the various case studies, it also became apparent that a host of actors were also making various other decisions that affected the projects. Various actors, both individuals and organizations, had to decide whether to become active partners, what steps would be taken to insure the confidentiality of patient information, what technologies would be used and which companies would be hired, whether they would resist or support a project, how potential participants would be addressed so as to encourage their participation, and so on. Many decisions were made in relation to which codified knowledge could have played a role or not. At this point we can only indicate that we are cognizant of all these other decisions that were taking place, and which collectively also determined the outcome of any given project.

Decisions concerning which projects to fund were most likely being made by senior government officials or, perhaps in some cases, elected representatives who were in cabinet (the cabinet is formed by the government ministers). Elected government officials would certainly have been involved in the decision to propose or withdraw a given law. At the project definition phase, however, key decisions would have been made by other decision-makers. For example, the Rimouski project required a coalition of local physicians, local health officials, private companies, along with support from the RAMQ and certain officials within the Ministry of Health. The contributions to the project description (and the decision of these actors to participate in it) would have been influenced by the knowledge, beliefs, and interests of these participants. On the other hand, the decision to fund the project was likely taken by the Council of Ministers. In this regard, we should note that we did not have direct access to either of these groups at the time when decisions were being made (with the exception of the RSVP project). This could be an area for further historical research (i.e., what information did senior officials provide to the Minister of Health for the projects and activities discussed here). Still, a certain number of general observations can be made about the possible influence of codified knowledge on certain decisions that were taken.

In general, a series of real-life, in-situ pilot projects were an integral part of the 16-year period we are analyzing here (1993-2009). These projects allowed the provincial government to develop extensive experience with the challenges and issues associated with EHR projects. The projects were not part of a coordinated plan, but were rather local initiatives which were financed by the provincial (in the case of the Rimouski project), or primarily federal government (CHIPP projects), or both (DSQ project). Although the primary goal of these projects was not to develop "original knowledge", the goal was certainly to create large-scale, real-life tests of technology in an actual clinical context. The hope was that if the project was a success, that it would continue and the technology would become integrated into daily use. If it was not a success, it was implicit that participants would learn from the experience. Some thought had been given to how to maximize learning or knowledge development, since an evaluation was a part of most if not all of the projects.

One general observation that can be drawn from the various experiments is that project promoters seemed to believe that the technological systems that had been developed were ready for real-world experimentation. Smaller scale tests to simulate the real-world demands on the systems seemed to be either absent, or inadequate. This could have been learned with the Rimouski project. In some of the projects, the project proponents appeared to think that the technology would "sell itself", and be relatively easy to implement, although the importance of attending to the users' reactions to technology and to change management did not seem to have been assimilated until the time of the RSVP project. This can be inferred especially from the CHIPP project reports about the "lessons learned". Project promoters seemed to be surprised that medical personnel would require computer systems that responded quickly to requests for information, that users would need extensive training and technical support, and that there could be mixed reactions to the technology on the part of patients, citizens, and personnel.

These sorts of warnings were not new, even in the 1990s, and we would argue that these issues were common knowledge at the time in fields as diverse as management of information systems (for example [24]), computer supported cooperative work [25], and the design and implementation of IT [26]. Our interviews indicated that not all project promoters consulted with experts from these fields, and this was an obvious shortcoming. For their part, we found little evidence of senior government officials or decision-makers having consulted with those who had implemented IT systems in hospitals. Although the systems in place in hospitals at the time may have been simpler than the ones developed for the projects described here, lessons from the implementation of hospital systems could probably also have been applied in these projects.

In the case of the Rimouski project, little scientific evidence about the benefits of EHR or electronic health card projects existed in the early 1990s. So, other than knowledge that was available in other fields or from experiments such as the one in France, it was clear that project planners had relatively little scientific evidence from health researchers that could have been used to orient the implementation strategy. Indeed, $4 million dollars was spent on an innovation that did not have a track record in the health field. Very little information was available concerning the clinical impact of the innovation. It is useful to remember that, at this time, the few scientific articles on the subject generally reported on individual hospital systems [27, 28], or announced the advent of medical computerization and its benefits [29, 30]. For example, the latter write in the summary of their article: "The electronic medical record in the ambulatory setting will be a force to be reckoned with during the 1990s. The increased quality of the medical record will lead to an increased quality of patient care at a fairly level cost (no long-term increase)." Still, the state of the literature at the time did not provide solid support for the hope, on the part of project proponents and government decision-makers, that an EHR system could be readily introduced into a whole city, or would lead to clinical improvements. One of the purposes of the project was precisely to create better knowledge about the feasibility and the requirements for such a system. Even today, although several countries have managed to attain a high level of use of an EHR by clinicians, there is still debate about its clinical impact [31].

Although the evaluation report about the Rimouski project included analysis of use by health personnel, and analysis of perceptions about the usefulness of the information that was made available through the system, it did not analyze clinical changes that could have been used to assess this project or to inform later ones. On the other hand, and in contrast with later projects, a significant number of dissertations were produced in relation to the project. The authors of the evaluation report also identified factors that would likely affect any further projects to introduce the smart card on a wider scale.

Another surprising event was that the concept and technology that was active in the Laval electronic health card project became the basis for the law that was proposed in late 2001. According to one account, even more that a year after the project began (in May, 2001), only 290 patients out of the 1678 who had received a card, actually used it [12]. In spite of results which were not favourable, and in spite of clear concerns on the part of the CAI with respect to the Laval project [12], the provincial cabinet decided to propose legislation which used this project as a model for a province-wide system. The low rate of adoption by patients could be considered to be empirical evidence that should have lead officials and cabinet to question the concept on which the proposed legislation was based. A similar viewpoint was expressed publicly in a newspaper article that appeared in the newspaper Le Devoir, May 7th 2001, well before the legislation was tabled: "The trial of the electronic health card [in Laval], which Quebec wants to extend to the whole province, was not a clear success."

As well, the simultaneous experimentation with a different network technology in the RIGIC project and two other simultaneous projects to create a patient record shared between hospitals, which we have not discussed here, obviously did not alter the government's plans at the time. One of these was called Arc-en-ciel, and was a project financed by the Quebec Ministry of Health, in order to create an EHR shared by three hospitals. Thus the Ministry of Health must have been aware that certain projects were experimenting with another type of EHR technology, and that knowledge about this type of technology was in the process of being developed. For whatever reason, perhaps because of concerns about the cost of this technology, it decided that it was not worthwhile to wait for the results of these projects.

In 2001, the Quebec government decided to not participate in Infoway, then later, after the change in government in 2004, to participate. The rationale provided by informants for the two decisions suggested that these were not influenced by scientific evidence. The first decision, to not participate, seemed to stem from a desire to not accept the terms and requirements stated by the federal government. The second one, to participate, appeared to be motivated primarily by financial considerations: the opportunity to acquire additional funding for an EHR in Quebec, and to a lesser extent, to obtain better deals from vendors through group purchases. The question of whether or not an EHR could contribute to positive clinical results did not appear to be discussed in either case. The participation of Quebec in Infoway provided opportunities to obtain information about the strategies and progress made elsewhere, required that Quebec document and demonstrate the achievement of intermediate objectives for external observers, and some knowledge transfer may have been an unintended consequence of participation in it.

With the RSVP project, senior government officials demonstrated an understanding of the importance of developing and executing strategies that would facilitate the implementation of an EHR. They had clearly come into contact with and had expended effort to better understand scientific literature on change management and the management of the implementation of technology. The spheres of considered knowledge had expanded to include new strategies for communicating with, consultation of, and preparation of primary actors. The problem that arose was that the communication activities that were undertaken appeared to be premature.

The actors who participated directly in a given project undoubtedly gained practical experience, but it is difficult at this point to evaluate the extent to which expertise developed by individuals in one project may have been exploited in subsequent projects. Still, our impression is that the level of re-employment of personnel was low since new project groups were formed for each of the EHR projects. This is also consistent with the findings of a study that examined information sharing between health informatics projects in Quebec and noted that there was little knowledge exchange between the projects [32].

Other mechanisms for learning or transfer of knowledge and expertise could have been formal and informal meetings. Again, we are not able to determine the extent to which people who were involved in one project may have had contact with those who were planning subsequent projects. It is worthwhile noting though, that a series of annual conferences were held in Quebec on the theme of computer systems and healthcare, and several presentations were made about the projects discussed here. This conference was a forum for the exchange of information. The Quebec Society of Biomedical and Health Informatics (SoQIBS) was created in 2001 and is still organizing annual meetings on topics related to health information technologies (see http://soqibs.org/). This initiative is supported by researchers, clinicians, commercial enterprise, and government.

5. Strengths and limitations

This multiple case study is one of the first studies to examine the development and implementation of an EHR in a jurisdiction over an extended period of time. Although it does not constitute a full historical analysis, which is still needed, it adds to the knowledge base concerning factors that have contributed to the limited success of large-scale EHR implementations [2-4]. We used information from multiple sources, including scientific and technical literature, evaluation reports, policy documents, analysis of proposed laws, newspaper articles, and semi-structured interviews with stakeholders involved in the projects.

We are also keenly aware that we have not been able to fully document the myriad decisions and influences that culminated in a decision to fund any particular project. Someone who wishes to do this would be advised to concentrate initially on a single project. Also, although we have suggested that codified knowledge seemed to have relatively little influence in most projects, one means of partially verifying this would be to obtain initial project descriptions and to check to see what scientific evidence may have been cited in order to justify project objectives and orientations (we did not have such documents). What we suspect is that other forms of "evidence" may have sometimes been used (for example, what people were told by project promoters during short visits to sites which may have had less than a fully implemented EHR, or impressions of how an EHR may change care processes).

The selection of the cases under study was made by consensus among the authors, based on criteria such as the relative novelty of the project or activity and its importance in relation to later developments. However, the study of other EHR cases could potentially bring new insights [2]. Our analysis may also have been limited by the fact that some projects took place up to twenty years ago. We relied on documents that were published, as well as on the recall of individuals who participated in the projects. Thus, our analysis could be limited by the available documents and/or recall bias from interviewees. The presentation of several case studies has necessarily limited the amount of detail that could be provided for any one of these. However, the consideration of an extended period of time, triangulation of information, and the validation of our interpretation of each case with several respondents who represented different perspectives (policy-makers, researchers, and project leaders) likely helped us to gain a richer understanding.

6. Conclusion

This chapter examined decisions that have been made during the process of implementing particular EHR projects in the Province of Quebec, Canada. As we have indicated, a significant number of projects took place over the sixteen years that we have considered here, and these involved a variety of locations, objectives, and technologies.

Perhaps the most surprising characteristic of this period is the limited impact of codified knowledge on projects and activities, and this seems to be a problem that has been experienced in other countries as well [2]. Technological shortcomings and the need to insure that the user has a positive experience, both noted in the evaluation of the Rimouski project, cropped up again in later projects. Observations were repeated from one project to another about the need to invest in change management. Warnings from the CAI about the need to update legislation and to consider the complexity and depth of concerns about approaches to safeguarding health information did not seem to be fully considered. Pertinent knowledge developed outside of the health field was not transferred into it. Large scale pilot projects helped to develop codified knowledge but this was not fully exploited over time. However, a note of caution is required here, as these conclusions are tentative, and further investigation and analysis is required.

In the last two decades, there has been a strong emphasis within the health field concerning, on the one hand, the importance of basing decisions and medical practice on scientific evidence, and on the other, the importance of effective knowledge transfer. Unfortunately, in Quebec and elsewhere, it has also taken two decades for these two philosophies and practices to gradually permeate into the multiple and interconnected dynamics that may eventually produce a provincial and a Canadian EHR.

Acknowledgement

The initial study on which part of this chapter is based was funded by the Canadian Institutes of Health Research (grant # 200603MOP-159757-KTE-CFBA-111141). This study was approved by the institutional review board of the Centre hospitalier universitaire de Québec (#5-06-09-02). We would like to acknowledge the contribution of Dr. Jean-Paul Fortin who provided thoughtful comments on this chapter. We would like to thank all the people who participated in the interviews.