InTechOpen uses cookies to offer you the best online experience. By continuing to use our site, you agree to our Privacy Policy.

Medicine » Obstetrics and Gynecology » "Ovarian Cancer - A Clinical and Translational Update", book edited by Iván Díaz-Padilla, ISBN 978-953-51-1030-9, Published: February 27, 2013 under CC BY 3.0 license. © The Author(s).

Chapter 1

In what Setting Should Women with Ovarian Cancer Receive Care?

By Laurie Elit and Clare Reade
DOI: 10.5772/52936

Article top

In what Setting Should Women with Ovarian Cancer Receive Care?

Laurie Elit1, 2 and Clare Reade2, 3

1. Introduction

In Canada, ovarian cancer affects 2600 women and 1750 women die annually from this disease.[1] The case fatality rate for ovarian cancer is quite high at 0.67 because women usually present with wide-spread disease. Symptoms of ovarian cancer are non-specific, and there is no effective screening test which identifies ovarian cancer early, when the cure rate is highest.[2] When patients present with advanced disease, long term survival is elusive and the goals of care focus on increasing duration of survival and improving quality of life by managing symptoms of disease.

Ovarian cancer is usually managed with a combination of surgery and chemotherapy. The role of surgery is to make a histologic diagnosis, determine the extent of disease spread (staging) and remove as much disease as possible (debulking). The role of chemotherapy is to reverse the vascular permeability of tumour capillaries, thereby decreasing the presence of ascites and pleural effusions, and to cause cellular apoptosis of tumour cells, resulting in disease regression.

Evaluation of the patterns of care provided to patients with ovarian cancer in Ontario, Canada demonstrated a variety of specialists are involved in the delivery of surgery including gynaecologists, general surgeons and gynaecologic oncologists.[3] The delivery of chemotherapy can be provided by medical or gynaecologic oncologists. Surgery and/or chemotherapy can be delivered in low, medium or high volume centres in rural or urban settings and by teaching or non-teaching faculty.[3] This paper addresses the question of whether the context in which a woman receives care for her ovarian cancer affects her outcome.

2. Quality of care

The focus of this chapter falls within the rubric of quality of care. The Institute of Medicine has defined quality of care as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge”.[4] Good quality means providing patients with appropriate services in a technically competent manner, with good communication, shared decision making and cultural sensitivity.[4] Quality assurance can be defined as all those planned and systematic actions necessary to provide adequate confidence that a product or service will satisfy given requirements for quality.[5]

Donabedian originally coined the phrase quality of care.[6,7] He assessed quality of care by looking at the triad of structures, processes and outcomes.

Structure attributes describe the physical and organizational settings in which care is provided and evaluate whether these characteristics are conducive to the kind of care that can be expected to improve health and to be acceptable to patients and the community. Evaluation of the adequacy of facilities, qualifications of medical staff, availability of equipment, and organizational structure and operations of programs within the institution providing care fall under the category of structure of care. Often-evaluated structural variables include the demographic characteristics, training and experience of care providers and the environment in which they work. Other structural variables of interest include access to specific technologies, access to intensive care facilities and nurse-to-patient ratios. The most commonly cited variable used as a surrogate for assessing surgical quality is hospital or physician case volume.

Process of care describes the care the patient actually receives and evaluates the degree to which interventions provided to patients correspond to what is known or believed to be most effective in improving health. This includes: 1) the patient’s activity in seeking care and carrying it out, and 2) the practitioner’s activities in making a diagnosis and recommending and implementing therapy. Whether care is effective can be judged according to the evidence from good studies demonstrating a link between a particular process (ie., debulking surgery) and better outcomes (ie., prolonged survival). Process indicators are easily measured in a timely fashion and can provide actionable feedback for quality improvement initiatives. Other examples of process variables include guidelines for surgery and use of care pathways. These variables are usually used in the context of quality assessment audits.

Outcomes are the actual changes in health and wellbeing obtained by patients and communities, and the degree to which the care provided is acceptable. In other words, outcome is the effect of care on the health status of patients and populations. This may be improvement in patient knowledge, behaviour and satisfaction. Endpoints of interest in ovarian cancer could include 30-day peri-operative mortality, overall survival, and quality of life. Overall survival data takes a long time to mature and it reflects a culmination of many processes and structures that have contributed to care. There is currently a strong focus on outcomes for patients with ovarian cancer, especially in the context of health care payers obtaining high quality care for the health care dollars spent.[8]

To demonstrate the concepts of structure, process and outcomes as ways to measure quality of care in ovarian cancer, we will review population-based studies published over the last 10 years. We have restricted our scope to population-based studies because they provide outcomes for the whole population in a region and avoid biases inherent with single institution studies (ie., related to socioeconomic status, race or comorbidities). As well, population-based studies allow us the opportunity to identify where variations in care may lead to superior outcomes for the population. If these processes and/or structures are incorporated into practice, they may lead to improved health outcomes.

3. Methods

A systematic search of the published English language literature from Jan 1, 2000 to Jun 29, 2012 was undertaken in order to present an unbiased view of the current population-based literature in the field of quality of care. Several key articles were identified[9-11] and MeSH terms from these references were used to create a search strategy for PubMed (Figure 1).


Figure 1.

PubMed search strategy

The search yielded 1178 articles of which 172 were identified as potentially relevant by title and abstract. To be included the article had to include population-based data collection related to primary management of ovarian cancer. The article needed to report on structure or processes of care in relation to outcomes. Articles were excluded if they were reporting on screening for ovarian cancer, pre-cancerous or benign conditions; if they were focused solely on quality of life, biologic therapies, biomarkers and personalized medicine, survivorship or palliative care. We identified two systematic reviews of quality of care indicators.[12,13] However, in both cases the authors did not restrict their study inclusion to population-based reports, therefore these studies are not included in our analysis.

4. Results

The 30 population-based studies in this review represent findings from many high-income countries, including Australia (1), Canada (3), USA (12), Austria (2), Finland (2), Germany (1), Netherlands (3), Norway (1), Switzerland (1), UK (3), and Japan (1). Twenty-five unique studies report the impact of structure on outcomes, and 13 studies report the impact of various processes on outcomes. Included are 91,866 patients.

4.1. Outcomes

The 5-year overall survival rate is the indicator of most interest to clinicians caring for patients with ovarian cancer. Other outcomes of interest include quality of life, patient satisfaction, and cost. However, when 5-year survival rates are so poor, surrogate outcomes, including progression-free survival (PFS), can be used to reflect small changes in outcomes that are important to patients and society. Changes in processes or structures that result in improved surrogate outcomes should eventually be reflected in improved 5-year survival rates. Surrogate outcomes in ovarian cancer include PFS and 30 or 60-day mortality.

4.2. Structure

In 25 unique population-based studies of quality of care in ovarian cancer, structural variables evaluated include a hospital’s annual ovarian cancer surgical volume, physician annual ovarian cancer surgical volume, hospital type (university affiliated vs community hospital), and physician type (gynaecologic oncologist, general gynaecologist, or general surgeon). These studies are listed in Table 1.

Studies evaluating hospital volume demonstrate hospitals with higher volumes of ovarian cancer surgery per year are often associated with better long-term survival (Table 2). The improvement in overall survival did not appear to be a reflection of peri-operative deaths, because the 30 and 60-day mortality was not affected by hospital volume in the studies evaluating those outcomes. The long-term survival advantage produced by high-volume hospitals is due to other differences in structures and processes of care in these institutions.

Studies evaluating physician volume did not demonstrate a uniform improvement in survival when high-volume physicians operated on patients with ovarian cancer (Table 3). Findings were inconclusive for both shorter and longer-term survival.

In just over half of the studies identified, hospitals classified as teaching facilities or university hospitals were associated with better short and long-term survival outcomes (Table 4). Usually these specialized facilities provide access to physicians with expertise in complicated gynaecologic oncology surgical procedures necessary for appropriate surgical management of ovarian cancer patients.

Studies evaluating physician specialization usually compare outcomes for patients operated by gynaecologic oncologists versus general gynaecologists versus general surgeons. Operation by a gynaecologic oncologist was associated in most studies with better outcomes in terms of long-term survival (Table 5). It is likely that general surgeons are more likely to perform emergency surgeries in advanced situations like bowel obstruction. However, the difference in outcomes persisted even after adjusting for prognostic factors like the Charlson comorbidity score.

Study Country Data Source Number of patients Did structure impact survival?
Stockton 2000[14] UKRetrospective database989Yes
Olaitan 2001[15] UKProspective cohort595n/a
Carney 2002[16] USARetrospective database734Yes
Elit 2002[17] CanadaRetrospective database3,815Yes
Grossi 2002[18] AustraliaRetrospective database + chart review434No
Kumpulainen 2002[19] FinlandRetrospective database3,851Yes
Cress 2003[20] USARetrospective database1,088n/a
Harlan 2003[21] USARetrospective database1,167n/a
Ioka 2004[22] JapanRetrospective database2,450Yes
Diaz-Montez 2005[23] USARetrospective database2,417n/a
Bailey 2006[24] UKProspective cohort361No*
Earle 2006[25] USARetrospective database3,067Yes
Elit 2006[11] CanadaRetrospective database2,502No
Engelen 2006[26] NetherlandsRetrospective database + chart review632Yes
Goff 2006 and 2007[27,28] USARetrospective database10,432n/a
Kumpulainen 2006 and 2009[29,30] FinlandProspective cohort275Yes
Oberaigner 2006[31] AustriaRetrospective database911Yes
Paulsen 2006[32] NorwayProspective registry198Yes
Schrag 2006[33] USARetrospective database2,952Yes
Elit 2008[34] CanadaRetrospective database1,341No
Bristow 2009[35] USARetrospective database1,894Yes
Marth 2009[36] AustriaProspective cohort1,948Yes
Vernooij 2009[37] NetherlandsRetrospective cohort1,077Yes
Mercado 2010[38] USARetrospective cohort31,897Yes
Rochon 2011[39] GermanyProspective cohort476No

Table 1.

Studies reporting on structural variables in relation to outcomes for ovarian cancer

[i] - n/a: not applicable—these studies used surrogate outcomes, *the authors of this study reported it was underpowed to find an association between structure and survival

Outcomes Number of studies finding an association between higher volume and improved outcomes Number of studies finding no association between volume and outcomes Total
Overall survival7310
30-day mortality022
60-day mortality011

Table 2.

Relationship between hospital volume and patient outcomes

[i] - DFS: disease-free survival

Outcome Number of studies finding an association between higher volume and improved outcomes Number of studies finding no association between volume and outcomes Total
30-day mortality123
60-day mortality011

Table 3.

Relationship between physician volume and patient outcomes

Outcomes Number of studies finding an association between specialized hospitals and improved outcomes Number of studies finding no association between hospital type and outcomes Total
Overall survival437
30-day mortality112

Table 4.

Relationship between hospital type and patient outcomes

Outcomes Number of studies finding an association between increased physician specialization and improved outcomes Number of studies finding no association between physician specialization and outcomes Total
Overall survival639
30-day mortality112

Table 5.

Relationship between physician specialization and patient outcomes

Several studies have reported a link between structural variables (hospital volume, physician volume, hospital type and physician specialization) and outcomes. Population-based studies published over the past ten years identify more consistent evidence linking increased hospital volume and increased physician specialization with long-term outcomes than for other structural variables. Surgery by a gynecologic oncologist appears to provide superior outcomes in terms of long term survival. These studies pertain to the surgical management of patients with ovarian cancer. The single study looking at chemotherapy for ovarian cancer patients found no association between oncologist volume of chemotherapy and outcomes.[11] Of note, no study demonstrated worse outcomes with higher volumes or specialization of hospitals or physicians. Some jurisdictions have used these findings to implement a strategy of centralization of surgery for ovarian cancer in an effort to improve quality of surgical care and outcomes.[40,41]

There are important limitations in this data. Not all studies were able to obtain individual data to allow adjustment for every important confounding variable which can impact survival. The majority of these studies were retrospective or dependant on accurate data-entry into databases. It is possible some of the advantages observed for type or volume of provider may be due to more diligent data-entry and documentation of patient demographics, stage and treatment received. For example, teaching hospitals may have more accurate and detailed documentation of the surgical procedures provided to patients which may lead to an assumption that they provided more complete surgical care when in fact the differences were in documentation only. The use of re-operation as a surrogate outcome is questionable when discussing physician type, since more specialized physicians are typically the ones making the decision to perform a second operation and this decision is more likely to occur if the primary surgery was performed by a less specialized surgeon.

4.3. Process

Evidence-based guidelines on the surgical care of women with ovarian cancer generally recommend hysterectomy, bilateral salpingo-oophorectomy, and omentectomy. In early-stage disease, staging should be performed, including cytology, peritoneal biopsies, and pelvic and para-aortic lymphadenectomy. In late-stage disease, debulking should be performed, including the removal of all macroscopic tumour. This sometimes requires the use of bowel resection, splenectomy, diaphragmatic and peritoneal stripping.[42-44] Adjuvant or neoadjuvant chemotherapy with a combination of a platinum and a taxane agent has been the standard of care for epithelial ovarian cancers over the past ten years.[45] Appropriate surgery and chemotherapy have a demonstrated impact on outcomes for ovarian cancer patients and represent processes of care indicating quality.

Next we look at whether the processes evaluated in the literature are related to the four structural variables reported, and whether these impact on survival.

Number of studies finding an association between higher volume and improved processes Number of studies finding no association between volume and processes Total
Adequate surgery112
Optimal debulking516
Length of Stay202
Adjuvant chemotherapy101

Table 6.

Relationship between hospital volume and evidence-based processes

[i] - LND: lymph node dissection

Higher hospital volumes of ovarian cancer surgery were associated with better compliance to process steps in the optimal care of women with ovarian cancer (Table 6). These processes included: surgery according to guidelines (optimal debulking, lymph node dissection) and use of adjuvant chemotherapy.

Number of studies finding an association between higher volume and improved processes Number of studies finding no association between volume and processes Total
Optimal debulking101
Length of stay101
Re-operation 202
Adjuvant chemotherapy101
Length of Stay101

Table 7.

Relationship between physician volume and evidence-based processes

[i] - LND: lymph node dissection

Surgery by physicians with higher volumes of ovarian cancer surgeries was also associated with better compliance to process steps such as surgery according to guidelines and use of adjuvant chemotherapy (Table 7).

Number of studies finding an association between specialized hospitals and improved processes Number of studies finding no association between hospital type and processes Total
Optimal debulking303
Adjuvant chemotherapy505

Table 8.

Relationship between hospital type and evidence-based processes

[i] - LND: lymph node dissection

Type of hospital (ie. teaching versus non-teaching, academic versus community) where surgery for ovarian cancer is performed was clearly associated with more appropriate surgery and adjuvant chemotherapy in accordance with guidelines (Table 8).

Processes Number of studies finding an association between increased physician specialization and improved outcomes Number of studies finding no association between physician specialization and outcomes Total
Optimal debulking606
Adjuvant chemotherapy404

Table 9.

Relationship between physician specialization and evidence-based processes

[i] - LND: lymph node dissection

Physician specialization (ie., gynaecologic oncologist vs general gynaecologist vs general surgeon) was also associated with appropriate surgery and adjuvant chemotherapy in accordance with guidelines (Table 9).

In summary, 13 population-based studies involving 22,255 patients across 3 continents linked processes of care to improved survival. The relationship of important processes of care with survival is so clear that this work that has led to defining quality indicators for the treatment of ovarian cancer care. In Ontario, Canada, Gagliardi and colleagues[40] used the Delphi technique to define quality indicators. More recently, Verleye and the EORTC has defined and set surgical benchmarks for quality care in ovarian cancer (Table 11., Appendix).[46]

Study Country Data Source Number of patients Which process variables affected survival?
Bailey 2006[24] UKProspective cohort361X
Chan 2008[47] USARetrospective database8,372X
Elit 2006[11] CanadaRetrospective database2,502XX
Elit 2008[34] CanadaRetrospective database1,341X
Engelen 2006[26] NetherlandsRetrospective database + chart review632X
Fairfield 2010[48] USARetrospective database4,589X
Grossi 2002[18] AustraliaRetrospective database + chart review434X
Hershman 2004[49] USARetrospective database236X
Maas 2005[50] NetherlandsRetrospective database1,116XX
Marth 2009[36] AustriaProspective cohort1,948X
Paulsen 2006[32] NorwayProspective registry198XX
Petignat 2007[51] SwitzerlandRetrospective database50X
GermanyRetrospective database476XX

Table 10.

Studies reporting on process variables in relation to outcomes in ovarian cancer

5. Processes or structures of care that require further evaluation

There are many processes considered by experts to be important in the care of women with ovarian cancer. These process variables have face validity but have not yet been clearly evaluated for their impact on outcomes in ovarian cancer. Additionally, the organizational structure for care provision is a complex construct; it is unclear what components contribute most positively to outcomes. We wish to focus on three variables that may or may not be related to survival but may impact treatment and decision making.

5.1. Pathology assessment

When making a diagnosis of ovarian cancer, the histology may be assessed by a pathologist, a pathologist with interest and experience in gynaecologic malignancies, or a subspecialist gynaecologic pathologist. Heatley[53] defines a pathologist as someone who has completed training and passed the appropriate examinations. A pathologist with a special interest (PSI) is a general pathologist who takes the lead in a subspecialty area within their department such as gynaecological pathology, attending meetings of specialist societies, participating in the appropriate subspecialist external quality assurance scheme, providing specialist opinions for colleagues in the department, and on occasion, neighbouring departments. A subspecialist pathologist is a pathologist with a special interest but who now, possibly after a period working as a general pathologist, devotes all or the vast majority of their time to one area of practice.[53] Subspecialisation leads to standardisation of pathology reports and improved communication of findings, participation in multidisciplinary tumour board meetings, enhanced knowledge and standards, decreased turnaround times, quality assurance of diagnoses, improved quality of resident training, ability to distinguish appropriate variation from the standard of care, and advancement of academic knowledge through participation in research.[54]

In gynaecologic oncology, there are several studies reporting up to a 16.9% discrepancy with the referral diagnosis when a PSI or a subspecialist gynaecologic pathologist provides a review of the original pathology.[55] In 4.7% -12% of cases there is a change in diagnosis which has a major therapeutic or prognostic implication.[55-58] Although these findings were from studies including all gynaecologic malignancies rather than ovarian cancer specifically, they demonstrate subspecialist pathology review has an important role to play in the care of patients with ovarian cancer. Verleye and colleagues[59] found that pathology reports for ovarian cancer surgery originating from high-volume centres and academic hospitals are of higher quality than those originating from lower volume or non-academic centres. The availability of subspecialist gynaecologic pathologists may be one structural aspect of care in these centres leading to better outcomes. The impact of expert pathology review in ovarian cancer needs to be evaluated as a process step that could impact survival.

5.2. Multidisciplinary care

Multidisciplinary care is an integrated team-based approach to cancer care where medical and allied health care professionals consider all relevant treatment options and collaboratively develop an individual treatment and care plan for each patient. Evidence in oncology suggests that multidisciplinary care leads to improved survival and quality of life, satisfaction with treatment, and mental well-being of clinicians.[60] An important component of multidisciplinary care is availability of regularly scheduled tumour board meetings[61] with participation of gynaecologic oncologists, pathologists, radiologists, radiation and medical oncologists, and allied health professionals with a special interest in care of gynaecologic oncology patients. Tumour board conferencing in Auckland City Hospital from 2005-2006 led to a 5.9% rate of major changes in patient management.[62] This resulted from radiologic review (major discrepancy rate 1.4%) and pathology review (major discrepancy rate of 4.5%) which led to identification of major diagnostic discrepancies. However, they could not quantify how the changes in diagnosis and management might impact patient outcomes. Santoso did a comparison of the initial gynecologic cancer diagnosis and management plan to the diagnosis and management plan after discussion at a multidisciplinary tumor board meeting. They showed that 6.9% of cases discussed at tumor board had changes made to the diagnosis or plan, and in 5% there were major changes in treatment.[63] The most convincing research suggesting care by a multidisciplinary team is a process that improves outcomes was published by Junor and colleagues using population-based data from Scotland. In a retrospective analysis of all 533 cases of ovarian cancer diagnosed in Scotland in 1987, referral to a multidisciplinary team was one of five factors significantly associated with improved 5 yr survival after adjusting for patient and disease characteristics (hazard ratio 0.60, p<0.001).[64]

5.3. Institutional participation in clinical trials

Several studies have identified clinical trial participation as an institutional marker of quality care. In 1994, Stiller published a review of several cancer disease sites and found that across disease sites, patients treated as part of a clinical trial had better outcomes.[65] du Bois and colleagues evaluated outcomes in a population-based cohort of patients diagnosed with ovarian cancer in Germany in 2001.[52] After adjusting for disease stage, patients treated in an institution participating in multi-centre clinical trials had improved overall survival (35 months vs 25 months for patients with stage III-IV ovarian cancer treated at participating vs non-participating hospitals).[39,52] Notably, patients treated in participating hospitals had better outcomes even if they were not themselves participating in a trial. Patients treated in hospitals participating in trials were more likely to receive care in accordance with clinical practice guidelines including staging, debulking and combination chemotherapy where appropriate.[39] Trial participation at an individual patient level may indicate good performance status that can, in and of itself, lead to better outcomes. However, it appears all patients treated at hospitals participating in trials may benefit from improved outcomes. This is likely due to differences in processes of care at these institutions.

6. What does all this mean?

When geographic variation in outcomes exist at a population level, there are opportunities to assess whether changes in structures or processes of care could improve outcomes. There have been several strategies to improve outcome. One is to standardize care using evidence-based guidelines and techniques to optimize processes like a structured care path, whether in a paper chart or as part of an electronic medical record. Another approach to try to improve outcomes at a population level has been to centralize care. In some situations where care requires an experienced surgical team and highly developed perio-operative care, such as for the surgical management of pancreatic cancer, there is evidence that centralization of care to high-volume centres decreases 30-mortality.[66] However, not all reports are consistent with this finding.[67] Another strategy to improve outcomes is to focus on improving processes by the involvement of highly regarded opinion leaders providing education. More consistent improvement in processes of care has been noted using the audit and feedback system.[68] These approaches have been variously referred to as quality assessment, quality management quality improvement and knowledge translation. In this paper, we refer to quality assessment as the audit process whereby performance is measured and compared with a reference standard. Quality improvement includes the steps taken to actively change practice to improve adherence to processes and to improve outcomes.

7. Quality assurance and monitoring

Quality assurance and monitoring of outcomes is essential to allow for quality improvement initiatives. Regions, hospitals, and care providers must understand which outcomes are not reaching a targeted standard in order to identify structures and processes which may improve outcomes. Initiatives such as the International Cancer Benchmarking Partnership[69] have used population-based registry data to identify significant discrepancies in survival for women with ovarian cancer based on geographic location. Striking differences were observed, with women in Australia and Canada having significantly longer survival than women in the UK and Denmark after adjusting for stage.[70]

Measuring the quality of surgical procedures has lagged behind quality-assurance initiatives in other areas because of the difficulty in identifying parameters to evaluate.[71] Early studies suggested operative morbidity and mortality, adequacy of resection, local recurrence and survival as parameters to measure surgical quality.[71] However several of these factors are also highly influenced by the use of appropriate adjuvant therapy. Several programs have now begun systematically tracking outcomes for surgical oncology patients in an effort to identify areas where quality improvement measures should be implemented.

One such program is the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP).[72] This is a nationally validated, multi-specialty, risk-adjusted 30-day outcomes measurement program which originated in the Veterans Health Administration in 1991. Since 2004, NSQIP has been expanding and now includes more than 400 hospitals in the US, Canada, Lebanon and the UAE. The aim of the program is to provide institutions and surgeons with 30-day outcomes which can be used to compare performance to other institutions. Risk adjustment incorporates pre-operative comorbidities and intra-operative risk factors using hierarchical modelling. Twice per year, institutions are given a report with their risk-adjusted outcomes in the form of odds ratios, which can be used for bench-marking. After implementation of NSQIP in 10 Tennessee hospitals, significantly fewer surgical site infections, failed grafts and flaps, episodes of acute renal failure, and prolonged ventilation of more than 48 hours was achieved.[73] The ACS evaluated outcomes across all participating institutions from 2005 to 2007 and found 66% of hospitals showed improvement in 30-day mortality and 82% of hospitals achieved a reduction in complications after enrollment in the NSQIP program.[74] These improvements also led to significant cost savings. The remarkable success of this program will likely lead to further expansion.

8. Quality improvement

Quality improvement naturally follows from quality assessment. Review of performance in terms of adherence to best-practices (processes of care) in a methodologically rigorous and transparent manner (quality assessment) can lead to improvement in outcomes if interventions are undertaken to improve areas of weakness in performance. Interventions based on quality assurance data attempt to improve processes of care in order to improve outcomes. A framework for quality improvement could include the following steps:[75]

  1. Debate and select values and goals that will inform the effort

  2. Select a clinical area requiring improvement

  3. Select team members

  4. Select relevant quality markers for improvement

  5. Collect data for selected markers

  6. Select and operationalize interventions to achieve improvements in markers

  7. Re-evaluate, modify and repeat the steps

The American Society of Clinical Oncology (ASCO) initiated the Quality Oncology Practice Initiative (QOPI)[76] for US-based Hematology-Oncology practices in order to improve quality of cancer care by using measurement and feedback and by providing improvement tools. Processes of care indicative of quality were identified by a group of oncologists using consensus and clinical practice guidelines.[77] QOPI provides individual care providers with quality of care benchmarking information twice per year, allowing clinicians to make improvements within their own practices. Implementation of QOPI and sharing results with physicians at one academic oncology centre in the US led to significant improvements in several areas of quality.[78] Although this program is only available to medical oncology practices in the US, it serves as a good example of how measurement and feedback can lead to improvement in quality of care.

A quality management program was implemented in one German academic oncology centre in 2001 with the aim of improving the quality of surgery provided to patients with ovarian cancer.[79] The components of the quality management system included establishment of a prospective tumour registry, creation and training of dedicated surgical teams operating on patients with advanced ovarian cancer, inter-disciplinary surgical care, intra-operative second opinion by another gynecologic oncologist if the first surgeon did not believe debulking to microscopic residual disease was attainable, interdisciplinary management of complications, and quality conferences including assessment and benchmarking of morbidity and survival outcomes. This effort, along with a significant increase in the volume of ovarian cancer surgery performed at this centre over time, led to a significant improvement in processes and outcomes. Debulking to microscopic residual disease increased from 33% in 1997-2000, and 47% in 2001-2003, to 62% in 2004-2008. This led to median survival of 26 months for patients treated in 1997-2000, 37 months in 2000-2003 and to 45 months in 2004-2008 and 5-year survival in 24%, 34% and 36% of patients in the three time periods. Changes in both structures and processes of care were achieved using this quality management system, leading to improved survival for patients.[79]

A quality improvement program for the surgical care of patients with advanced ovarian cancer was implemented at the Mayo Clinic using an audit and feedback approach, with the aim of increasing the proportion of patients debulked to microscopic residual disease.[44] A surgical complexity score was developed to categorize the aggressiveness of the surgical approach.[80] The quality improvement program consisted of weekly conferences where patient outcomes and treatment approaches were discussed, confidential benchmarking allowing individual surgeons to see their rates of complete surgical debulking in comparison to peers, teaching fellows and staff how to perform techniques needed for complete debulking, and intra-operative mentoring of staff and fellows by surgeons experienced in advanced procedures. After the quality improvement program was implemented, rates of debulking to microscopic disease increased from 31% to 43%.

9. Knowledge translation

Knowledge translation is the science of moving knowledge into action.[81] Several studies across various disciplines in medicine have demonstrated many patients do not receive care known to improve outcomes.[81,82] One of the first groups to show this in ovarian cancer was Munstedt and colleagues who found a large proportion of patients treated in Hesse, Germany between 1997 and 2001 did not receive care recommended in national guidelines.[83] Knowledge translation aims to bridge the gap between what is known from research, and implementation of this knowledge in an effort to improve outcomes for patients and efficiency for the health care system.[81]

Knowledge translation has been described as a cycle, where a clinical problem is identified (possibly by quality assurance or monitoring efforts), processes of care are identified from research to address the problem, these processes are adapted to the local context and any barriers to implementation are identified and addressed, and the new processes are implemented. After implementation, adherence to the process is monitored, and final patient outcomes are evaluated.[81] Evaluation of outcomes and monitoring of processes may then identify additional clinical problems. If no evidence-based solution to the problem is identified, this leads to a need for additional research. In this way, new research informs clinical practice, and problems from clinical practice help to identify research priorities.[82]

A major focus of knowledge translation research is finding ways to change clinician and patient behaviour given the results of research. Simply publishing new findings in peer-reviewed journals, a method termed ‘diffusion’, is not adequate for wide-spread adoption of new processes.[84] Other methods that have been investigated include audit and feedback,[85] educational outreach by local opinion leaders,[86] and clinical decision support and reminder systems which can be integrated into computer-based patient-care platforms.[87] Audit and feedback, such as the ACS NSQIP or ASCO QOPI programs, are one of the most effective methods for behaviour change in clinicians.[68,88] An excellent overview of these methods has been published by Brouwers and colleagues, who performed a review of systematic reviews on knowledge translation interventions used in cancer control.[68] The science of knowledge translation is relatively new. As research methods continue to improve, strategies are expected to be refined.


Figure 2.

Flow diagram for study selection

10. Conclusion

Women with ovarian cancer should be treated in institutions providing high quality care. Quality of care can be evaluated by examining the processes and structures of care leading to improved outcomes such as survival and quality of life.

In the US, there is a trend to link reimbursement for hospitals and care providers to clinical outcomes in an effort to improve quality of care.[72] Because of financial pressures in the health care system, this trend is expected to continue, since improvement in several metrics used to identify quality surgical care (such as decreased surgical site infections) can save a significant amount of money. Whether health systems are achieving value for money can only be assessed if performance is measured in a systematic way. Tracking outcomes with the use of population-based registries is an essential component of quality assurance, which allows for comparison of outcomes across jurisdictions.[71] Identifying variations in outcomes can then trigger specific quality improvement initiatives. Knowledge translation is the science of moving knowledge into action, and encompasses both quality assurance and quality improvement. The concepts underlying quality of care are essential information for health care providers caring for women with ovarian cancer given the current global focus on outcomes and value for money in health care systems.


Early-stage epithelial ovarian cancer-Percent of patients with a suspicious ovarian mass undergoing staging laparotomy within 1 month after decision to treat or documented clinical or patient-related reason for delay
-Percent of performed staging laparotomies for an ovarian mass suspected to be malignant performed through a vertical incision
Percent of performed staging laparotomies in which all of the following procedures are included: total hysterectomy, bilateral salpingo-oophorectomy, cytology of the peritoneal cavity, infracolic omentectomy, random peritoneal biopsies and systematic pelvic and para-aortic lymphadenectomy if medium or high risk features
Percent of surgery reports with documented presence or absence of cyst rupture before or during surgery
Percent of surgery reports with documented presence or absence of dense adhesions, percent of dense adhesions biopsied
Primary debulking surgery in advanced-stage epithelial ovarian cancerPercent of patients with advanced-stage ovarian cancer undergoing debulking laparotomy within 31 days after decision to treat or documented clinical or patient-related reason for delay
Percent of patients undergoing debulking surgery with the spread of disease fully assessed for operability at the start of study and initial findings documented in the operation notes
Percent of debulking operations including a hysterectomy, bilateral salpingo-oophorectomy and infracolic omentectomy when the surgeon considers optimal debulking feasible
Percent of debulking operations for advanced ovarian cancer at the end of which complete cytoreduction, defined as no macroscopic residual disease at the end of the operation, was achieved
Percent of debulking operations including a pelvic and para-aortic lymphadenectomy when otherwise complete debulking has been achieved
percent of debulking operations for which the size and location of residual disease at the end of the operation is documented in the operation notes

Table 11.

EORTC benchmarks for quality surgical care in ovarian cancer[46]


1 - Canadian Cancer Society’s Steering Committee on Cancer Statistics. Canadian Cancer Statistics 2012. Toronto, ON: Canadian Cancer Society; May 2012. 0835-2976.
2 - Buys SS, Partridge E, Black A, et al. Effect of screening on ovarian cancer mortality: the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Randomized Controlled Trial. JAMA. Jun 8 2011;305(22):2295-2303.
3 - Elit L, Schultz S, Prysbysz R, et al. Chapter 7: Surgery for Ovarian Cancer. In: Urbach D, Simunovic M, Schultz S, eds. Cancer Surgery in Ontario: ICES Atlas. Toronto: Institute for Clinical Evaluative Sciences; 2008.
4 - National Cancer Policy Board, Institute of Medicine, National Research Council. Ensuring Quality Cancer Care: The National Academies Press; 1999.
5 - Hoyle D. ISO 9000 Quality Systems Handbook. 5th ed. Burlington, MA: Butterworth-Heinemann; 2006.
6 - Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. Jul 1966;44(3):Suppl:166-206.
7 - Donabedian A. Contributions of epidemiology to quality assessment and monitoring. Infect Control Hosp Epidemiol. Mar 1990;11(3):117-121.
8 - Hacker NF. Quality control in ovarian cancer surgery. Ann Oncol. Dec 2011;22 Suppl 8:viii19-viii22.
9 - Hillner BE, Smith TJ, Desch CE. Hospital and physician volume or specialization and outcomes in cancer treatment: importance in quality of cancer care. J Clin Oncol. Jun 2000;18(11):2327-2340.
10 - Elit L, Schultz S, Prysbysz R, et al. Patterns of care in the initial management of women with ovarian cancer in Ontario. Eur J Gynaecol Oncol. 2009;30(4):361-364.
11 - Elit L, Chartier C, Oza A, Hirte H, Levine M, Paszat L. Outcomes for systemic therapy in women with ovarian cancer. Gynecol Oncol. Nov 2006;103(2):554-558.
12 - du Bois A, Rochon J, Pfisterer J, Hoskins WJ. Variations in institutional infrastructure, physician specialization and experience, and outcome in ovarian cancer: a systematic review. Gynecol Oncol. Feb 2009;112(2):422-436.
13 - Giede KC, Kieser K, Dodge J, Rosen B. Who should operate on patients with ovarian cancer? An evidence-based review. Gynecol Oncol. Nov 2005;99(2):447-461.
14 - Stockton D, Davies T. Multiple cancer site comparison of adjusted survival by hospital of treatment: an East Anglian study. Br J Cancer. Jan 2000;82(1):208-212.
15 - Olaitan A, Weeks J, Mocroft A, Smith J, Howe K, Murdoch J. The surgical management of women with ovarian cancer in the south west of England. Br J Cancer. Dec 14 2001;85(12):1824-1830.
16 - Carney ME, Lancaster JM, Ford C, Tsodikov A, Wiggins CL. A population-based study of patterns of care for ovarian cancer: who is seen by a gynecologic oncologist and who is not? Gynecol Oncol. Jan 2002;84(1):36-42.
17 - Elit L, Bondy SJ, Paszat L, Przybysz R, Levine M. Outcomes in surgery for ovarian cancer. Gynecol Oncol. Dec 2002;87(3):260-267.
18 - Grossi M, Quinn MA, Thursfield VJ, et al. Ovarian cancer: patterns of care in Victoria during 1993-1995. Med J Aust. Jul 1 2002;177(1):11-16.
19 - Kumpulainen S, Grenman S, Kyyronen P, Pukkala E, Sankila R. Evidence of benefit from centralised treatment of ovarian cancer: a nationwide population-based survival analysis in Finland. Int J Cancer. Dec 10 2002;102(5):541-544.
20 - Cress RD, O'Malley CD, Leiserowitz GS, Campleman SL. Patterns of chemotherapy use for women with ovarian cancer: a population-based study. J Clin Oncol. Apr 15 2003;21(8):1530-1535.
21 - Harlan LC, Clegg LX, Trimble EL. Trends in surgery and chemotherapy for women diagnosed with ovarian cancer in the United States. J Clin Oncol. Sep 15 2003;21(18):3488-3494.
22 - Ioka A, Tsukuma H, Ajiki W, Oshima A. Influence of hospital procedure volume on ovarian cancer survival in Japan, a country with low incidence of ovarian cancer. Cancer Sci. Mar 2004;95(3):233-237.
23 - Diaz-Montes TP, Zahurak ML, Giuntoli RL, 2nd, et al. Surgical care of elderly women with ovarian cancer: a population-based perspective. Gynecol Oncol. Nov 2005;99(2):352-357.
24 - Bailey J, Murdoch J, Anderson R, Weeks J, Foy C. Stage III and IV ovarian cancer in the South West of England: five-year outcome analysis for cases treated in 1998. Int J Gynecol Cancer. Jan-Feb 2006;16 Suppl 1:25-29.
25 - Earle CC, Schrag D, Neville BA, et al. Effect of surgeon specialty on processes of care and outcomes for ovarian cancer patients. J Natl Cancer Inst. Feb 1 2006;98(3):172-180.
26 - Engelen MJ, Kos HE, Willemse PH, et al. Surgery by consultant gynecologic oncologists improves survival in patients with ovarian carcinoma. Cancer. Feb 1 2006;106(3):589-598.
27 - Goff BA, Matthews BJ, Wynn M, Muntz HG, Lishner DM, Baldwin L-M. Ovarian cancer: Patterns of surgical care across the United States. Gynecologic Oncology. 2006;103(2):383-390.
28 - Goff BA, Matthews BJ, Larson EH, et al. Predictors of comprehensive surgical treatment in patients with ovarian cancer. Cancer. 2007;109(10):2031-2042.
29 - Kumpulainen S, Kuoppala T, Leminen A, et al. Surgical treatment of ovarian cancer in different hospital categories--a prospective nation-wide study in Finland. Eur J Cancer. Feb 2006;42(3):388-395.
30 - Kumpulainen S, Sankila R, Leminen A, et al. The effect of hospital operative volume, residual tumor and first-line chemotherapy on survival of ovarian cancer - a prospective nation-wide study in Finland. Gynecol Oncol. Nov 2009;115(2):199-203.
31 - Oberaigner W, Stuhlinger W. Influence of department volume on cancer survival for gynaecological cancers--a population-based study in Tyrol, Austria. Gynecol Oncol. Nov 2006;103(2):527-534.
32 - Paulsen T, Kjaerheim K, Kaern J, Tretli S, Trope C. Improved short-term survival for advanced ovarian, tubal, and peritoneal cancer patients operated at teaching hospitals. Int J Gynecol Cancer. Jan-Feb 2006;16 Suppl 1:11-17.
33 - Schrag D, Earle C, Xu F, et al. Associations between hospital and surgeon procedure volumes and patient outcomes after ovarian cancer resection. J Natl Cancer Inst. Feb 1 2006;98(3):163-171.
34 - Elit LM, Bondy SJ, Paszat LP, et al. Surgical outcomes in women with ovarian cancer. Can J Surg. Oct 2008;51(5):346-354.
35 - Bristow RE, Zahurak ML, Diaz-Montes TP, Giuntoli RL, Armstrong DK. Impact of surgeon and hospital ovarian cancer surgical case volume on in-hospital mortality and related short-term outcomes. Gynecol Oncol. Dec 2009;115(3):334-338.
36 - Marth C, Hiebl S, Oberaigner W, Winter R, Leodolter S, Sevelda P. Influence of department volume on survival for ovarian cancer: results from a prospective quality assurance program of the Austrian Association for Gynecologic Oncology. Int J Gynecol Cancer. Jan 2009;19(1):94-102.
37 - Vernooij F, Heintz AP, Coebergh JW, Massuger LF, Witteveen PO, van der Graaf Y. Specialized and high-volume care leads to better outcomes of ovarian cancer treatment in the Netherlands. Gynecol Oncol. Mar 2009;112(3):455-461.
38 - Mercado C, Zingmond D, Karlan BY, et al. Quality of care in advanced ovarian cancer: the importance of provider specialty. Gynecol Oncol. Apr 2010;117(1):18-22.
39 - Rochon J, du Bois A. Clinical research in epithelial ovarian cancer and patients' outcome. Ann Oncol. Nov 2011;22 Suppl 7:vii16-vii19.
40 - Gagliardi AR, Fung MF, Langer B, Stern H, Brown AD. Development of ovarian cancer surgery quality indicators using a modified Delphi approach. Gynecol Oncol. May 2005;97(2):446-456.
41 - Munstedt K, von Georgi R, Misselwitz B, Zygmunt M, Stillger R, Kunzel W. Centralizing surgery for gynecologic oncology--a strategy assuring better quality treatment? Gynecol Oncol. Apr 2003;89(1):4-8.
42 - Bristow RE, Tomacruz RS, Armstrong DK, Trimble EL, Montz FJ. Survival effect of maximal cytoreductive surgery for advanced ovarian carcinoma during the platinum era: a meta-analysis. J Clin Oncol. Mar 1 2002;20(5):1248-1259.
43 - Chi DS, Eisenhauer EL, Zivanovic O, et al. Improved progression-free and overall survival in advanced ovarian cancer as a result of a change in surgical paradigm. Gynecol Oncol. Jul 2009;114(1):26-31.
44 - Aletti GD, Dowdy SC, Gostout BS, et al. Quality improvement in the surgical approach to advanced ovarian cancer: the Mayo Clinic experience. J Am Coll Surg. Apr 2009;208(4):614-620.
45 - Ozols RF, Bundy BN, Greer BE, et al. Phase III trial of carboplatin and paclitaxel compared with cisplatin and paclitaxel in patients with optimally resected stage III ovarian cancer: a Gynecologic Oncology Group study. J Clin Oncol. Sep 1 2003;21(17):3194-3200.
46 - Verleye L, Ottevanger PB, van der Graaf W, Reed NS, Vergote I. EORTC-GCG process quality indicators for ovarian cancer surgery. Eur J Cancer. Mar 2009;45(4):517-526.
47 - Chan J, Fuh K, Shin J, et al. The treatment and outcomes of early-stage epithelial ovarian cancer: have we made any progress? Br J Cancer. Apr 8 2008;98(7):1191-1196.
48 - Fairfield KM, Lucas FL, Earle CC, Small L, Trimble EL, Warren JL. Regional variation in cancer-directed surgery and mortality among women with epithelial ovarian cancer in the Medicare population. Cancer. Oct 15 2010;116(20):4840-4848.
49 - Hershman D, Fleischauer AT, Jacobson JS, Grann VR, Sundararajan V, Neugut AI. Patterns and outcomes of chemotherapy for elderly patients with stage II ovarian cancer: a population-based study. Gynecol Oncol. Jan 2004;92(1):293-299.
50 - Maas HA, Kruitwagen RF, Lemmens VE, Goey SH, Janssen-Heijnen ML. The influence of age and co-morbidity on treatment and prognosis of ovarian cancer: a population-based study. Gynecol Oncol. Apr 2005;97(1):104-109.
51 - Petignat P, de Weck D, Goffin F, Vlastos G, Obrist R, Luthi JC. Long-term survival of patients with apparent early-stage (FIGO I-II) epithelial ovarian cancer: a population-based study. Gynecol Obstet Invest. 2007;63(3):132-136.
52 - Du Bois A, Rochon J, Lamparter C, Pfisterer J. Pattern of care and impact of participation in clinical studies on the outcome in ovarian cancer. Int J Gynecol Cancer. Mar-Apr 2005;15(2):183-191.
53 - Heatley MK. Subspecialisation and despecialisation in anatomical pathology. J Clin Pathol. Sep 2010;63(9):761-767.
54 - Chafe S, Honore L, Pearcey R, Capstick V. An analysis of the impact of pathology review in gynecologic cancer. Int J Radiat Oncol Biol Phys. Dec 1 2000;48(5):1433-1438.
55 - Selman AE, Niemann TH, Fowler JM, Copeland LJ. Quality assurance of second opinion pathology in gynecologic oncology. Obstet Gynecol. Aug 1999;94(2):302-306.
56 - Khalifa MA, Dodge J, Covens A, Osborne R, Ackerman I. Slide review in gynecologic oncology ensures completeness of reporting and diagnostic accuracy. Gynecol Oncol. Aug 2003;90(2):425-430.
57 - Santoso JT, Coleman RL, Voet RL, Bernstein SG, Lifshitz S, Miller D. Pathology slide review in gynecologic oncology. Obstet Gynecol. May 1998;91(5 Pt 1):730-734.
58 - Chan YM, Cheung AN, Cheng DK, Ng TY, Ngan HY, Wong LC. Pathology slide review in gynecologic oncology: routine or selective? Gynecol Oncol. Nov 1999;75(2):267-271.
59 - Verleye L, Ottevanger PB, Kristensen GB, et al. Quality of pathology reports for advanced ovarian cancer: are we missing essential information? An audit of 479 pathology reports from the EORTC-GCG 55971/NCIC-CTG OV13 neoadjuvant trial. Eur J Cancer. Jan 2011;47(1):57-64.
60 - Wilcoxon H, Luxford K, Saunders C, Peterson J, Zorbas H. Multidisciplinary cancer care in Australia: a national audit highlights gaps in care and medico-legal risk for clinicians. Asia Pac J Clin Oncol. Mar 2011;7(1):34-40.
61 - Wright F, De Vito C, Langer B, Hunter A, the Expert Panel on the Multidisciplinary Cancer Conference Standards. Multidisciplinary Cancer Conference Standards. Program in Evidence Based Care. 2006. Accessed August 10, 2012.
62 - Cohen P, Tan AL, Penman A. The multidisciplinary tumor conference in gynecologic oncology--does it alter management? Int J Gynecol Cancer. Dec 2009;19(9):1470-1472.
63 - Santoso JT, Schwertner B, Coleman RL, Hannigan EV. Tumor board in gynecologic oncology. Int J Gynecol Cancer. Mar-Apr 2004;14(2):206-209.
64 - Junor EJ, Hole DJ, Gillis CR. Management of ovarian cancer: referral to a multidisciplinary team matters. Br J Cancer. Aug 1994;70(2):363-370.
65 - Stiller CA. Centralised treatment, entry to trials and survival. Br J Cancer. Aug 1994;70(2):352-362.
66 - Simunovic M, Urbach D, Major D, et al. Assessing the volume-outcome hypothesis and region-level quality improvement interventions: pancreas cancer surgery in two Canadian Provinces. Ann Surg Oncol. Oct 2010;17(10):2537-2544.
67 - Massarweh NN, Flum DR, Symons RG, Varghese TK, Pellegrini CA. A critical evaluation of the impact of Leapfrog's evidence-based hospital referral. J Am Coll Surg. Feb 2011;212(2):150-159 e151.
68 - Brouwers MC, Garcia K, Makarski J, Daraz L. The landscape of knowledge translation interventions in cancer control: what do we know and where to next? A review of systematic reviews. Implement Sci. 2011;6:130.
69 - Cancer Research UK. International Cancer Benchmarking Partnership. 2010; Accessed August 9, 2012.
70 - Coleman MP, Forman D, Bryant H, et al. Cancer survival in Australia, Canada, Denmark, Norway, Sweden, and the UK, 1995-2007 (the International Cancer Benchmarking Partnership): an analysis of population-based cancer registry data. Lancet. Jan 8 2011;377(9760):127-138.
71 - Landheer ML, Therasse P, van de Velde CJ. The importance of quality assurance in surgical oncology. Eur J Surg Oncol. Sep 2002;28(6):571-602.
72 - American College of Surgeons. ACS NSQIP. 2012; Accessed August 9, 2012.
73 - Guillamondegui OD, Gunter OL, Hines L, et al. Using the National Surgical Quality Improvement Program and the Tennessee Surgical Quality Collaborative to improve surgical outcomes. J Am Coll Surg. Apr 2012;214(4):709-714; discussion 714-706.
74 - Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY. Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg. Sep 2009;250(3):363-376.
75 - Simunovic M, Baxter N. Quality improvement in surgical oncology - measuring and improving clinical outcomes. Surg Oncol. Sep 2011;20(3):179-183.
76 - ASCO. QOPI: The Quality Oncology Practice Initiative. 2012; Accessed August 9, 2012.
77 - Neuss MN, Desch CE, McNiff KK, et al. A process for measuring the quality of cancer care: the Quality Oncology Practice Initiative. J Clin Oncol. Sep 1 2005;23(25):6233-6239.
78 - Blayney DW, McNiff K, Hanauer D, Miela G, Markstrom D, Neuss M. Implementation of the Quality Oncology Practice Initiative at a university comprehensive cancer center. J Clin Oncol. Aug 10 2009;27(23):3802-3807.
79 - Harter P, Muallem ZM, Buhrmann C, et al. Impact of a structured quality management program on surgical outcome in primary advanced ovarian cancer. Gynecol Oncol. Jun 1 2011;121(3):615-619.
80 - Aletti GD, Santillan A, Eisenhauer EL, et al. A new frontier for quality of care in gynecologic oncology surgery: multi-institutional assessment of short-term outcomes for ovarian cancer using a risk-adjusted model. Gynecol Oncol. Oct 2007;107(1):99-106.
81 - Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. Winter 2006;26(1):13-24.
82 - MacDermid JC, Graham ID. Knowledge translation: putting the "practice" in evidence-based practice. Hand Clin. Feb 2009;25(1):125-143, viii.
83 - Munstedt K, von Georgi R, Zygmunt M, Misselwitz B, Stillger R, Kunzel W. Shortcomings and deficits in surgical treatment of gynecological cancers: a German problem only? Gynecol Oncol. Sep 2002;86(3):337-343.
84 - Farmer AP, Légaré F, Turcot L, et al. Printed educational materials: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews. 2008(3).
85 - Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews. 2012(6).
86 - Flodgren G, Parmelli E, Doumit G, et al. Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews. 2011(8).
87 - Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database of Systematic Reviews. 2009(3).
88 - Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. Feb 2004;8(6):iii-iv, 1-72.