InTechOpen uses cookies to offer you the best online experience. By continuing to use our site, you agree to our Privacy Policy.

Medicine » Mental and Behavioural Disorders and Diseases of the Nervous System » "A Fresh Look at Anxiety Disorders", book edited by Federico Durbano, ISBN 978-953-51-2149-7, Published: September 9, 2015 under CC BY 3.0 license. © The Author(s).

# Anxiety — Is There an App for That? Considering Technology, Psychiatry, and Internet-Assisted Cognitive Behavioral Therapy

By David Gratzer, Faiza Khalid-Khan and Sarosh Khalid-Khan
DOI: 10.5772/60816

Article top

# Anxiety — Is There an App for That? Considering Technology, Psychiatry, and Internet-Assisted Cognitive Behavioral Therapy

David Gratzer1, Faiza Khalid-Khan1 and Sarosh Khalid-Khan2

## 1. Introduction

A problem, the world over: anxiety disorders are common, with the global prevalence of such disorders estimated to be 7.3%, although that number varies highly by region [1]. Across Western countries, more than one-third of people have a mental health disorder over their lifetime; mood and anxiety disorders are the most common [2]. The World Health Organization (WHO) estimates that spending on anxiolytics alone is, on average, $94,880 USD per 100,000 people and sharply higher in wealthier countries,$315,560 USD per 100,000 [3]. There is an alternative to medications: psychological interventions are effective for anxiety. For example, cognitive behavioral therapy (CBT) is as effective as medications for mild and moderate anxiety, and there is evidence that combined psychopharmacology and CBT is superior to either modality alone, suggesting a synergistic effect [4]. CBT is based on the concept that people’s thoughts, emotions, and behaviors influence each other (Beck’s Cognitive Triad). In essence, the patients cannot control the world, but they can take control of how they interpret things in that world.However, CBT requires a major investment of time and resources. Thus, in public health-care systems, patients are often subjected to long wait times, and primary-care physicians and psychiatrists may not offer CBT [5]. This raises the question: can technology address the deficit of psychological interventions for mental illness? Internet therapies (including smart phone apps) have recently been developed to offer CBT and other psychological interventions for anxiety patients. In this review, we focus on Internet-assisted CBT (ICBT) for anxiety and anxiety comorbid conditions.

ICBT allows patients to receive ongoing CBT with simpler and faster access, at reduced cost, and with increased convenience over traditional CBT [6]. In this chapter, we will review evidence from the literature, including randomized control trials and meta-analyses. We will include the young literature on smart-phone apps. While additional studies are needed, the cumulative results strongly support the use of ICBT in clinical practice, especially in combination with ongoing therapist support. This chapter will also highlight governments that are embracing ICBT to better serve patients – experiments that are stretching from the Highlands in Scotland to Queensland in Australia. Overall, ICBT is a viable contribution to address the global mental health crisis.

## 2. Evidence for ICBT

ICBT has been the focus of much research, though the quality of these studies has varied. Early studies suffered from a lack of rigorous experimental design [7]; however, in recent years, improved study designs have generated a robust, reliable body of literature. A PubMed search with the terms “cognitive behavioral therapy” and “Internet” yielded 825 papers, covering everything from the treatment of mood and anxiety disorders to video game “addiction” and tinnitus. We chose the 40 most relevant articles related to anxiety that presented results from randomized controlled trials (RCT), meta-analyses, and reviews. We also incorporated literature on smart phone apps (apps), which is a growing, albeit young, field of study. Many well-controlled, randomized trials of ICBT were identified for various psychiatric conditions.

Although there are several types of ICBT programs, they have many common features: short-term, goal-oriented sessions that are self-guided by the patient [8], typically consisting of 8 to 12 modules [9]. There are RCTs of ICBT for various conditions; however, studies differ with respect to the kind of the control group used (e.g., waitlisted controls, placebo groups, or therapy as usual), the intensity of the online interaction between patients and therapists, the types of communication methods (such as email or text), and the format of online modules [10]. These variables make it difficult to directly compare the results of these studies, since they impact patient participation, adherence, and the efficacy of the approach. However, overall, the quality of evidence supporting integration of ICBT into clinical practice is strong for specific conditions and specific populations.

Focus on Panic Disorder and ICBT. There are a many studies considering ICBT. One of particular interest: Hedman et al. [11] performed a cohort study of panic disorder patients treated with guided ICBT through a Swedish university hospital clinic. The study had several interesting aspects. First, the hospital had created a permanent, dedicated ICBT unit to treat patients. Second, the study evaluated patients over a long period: 4.5 years. Third, patients never met their therapists, but therapists maintained an integral role in treatment (therapist-guided ICBT). Fourth, the results were robust and comparable to traditional, face-to-face CBT. And fifth, the paper was published in a prestigious journal, Acta Scandinavica Psychiatrica.

Patients with panic disorder were enrolled in an ICBT program that was structured as a 10-module self-help program. Each module covered specific principles of CBT treatment, such as interoceptive exposure and cognitive restructuring, and homework assignments were included. The ICBT was supported by a therapist, who monitored patient progress and provided feedback on homework with communication by email, although phone calls were included on an as-needed basis. Patients also had access to an online forum to discuss issues with other patients (though they could post anonymously). The online modules also included self-reported symptom assessments to flag patients who were deteriorating or were at risk of dropping out of the program. Following completion of the treatment program, patients could access the materials for 6 months.

Of the 570 patients included in the study, the majority completed the program (404) and a reasonable data set was generated (451 completed some form of postassessment and 156 provided 6-month follow-up data). Participants showed significant improvement in panic-related anxiety and depressive symptoms, which was maintained at the 6-month follow-up. Additionally, patients showed high satisfaction with the program. The therapists spent, on average, just 11 minutes per patient per week on patient communication (in contrast to, say, an hour per patient per week); in total, therapists spent slightly less than 2 hours per patient over the entire treatment course. The study also charted the effect size over time. Significantly higher effects were seen with patients enrolled in the program further from its inception, suggesting that the effectiveness improved as the therapists gained more experience. The effect size was equivalent to those seen in ICBT RCTs of panic disorder patients and to values estimated from meta-analyses of traditional face-to-face ICBT. Hedman et al. conclude: “As ICBT requires less resources compared with conventional CBT, the most important implication of the results of this study is that ICBT can be used as a mean to increase availability to empirically supported psychological treatments.”

Larger Literature. This panic disorder study is just one example of the many ICBT studies, including well-designed RCTs and meta-analyses. ICBT programs have been developed for many common psychiatric disorders, including obsessive-compulsive disease (OCD), posttraumatic stress disorder (PTSD), depression, panic disorder, and social anxiety disorder [12], with particularly strong evidence supporting the efficacy of ICBT for the last three disorders [13]. For example, one meta-analysis evaluated therapist-guided ICBT versus face-to-face CBT (either group or individual) in a variety of psychiatric disorders, more than half of which were anxiety or anxiety-related conditions. The meta-analysis included 13 RCTs, and the results showed that there were no significant differences in effect size or adherence between ICBT and face-to-face CBT [14]. In an ICBT study for people with social anxiety disorder (therapist-guided and an RCT), patients showed significant improvement at the 1-year follow-up and the benefits were retained at the 5-year follow-up [15].

One quantitative meta-analysis evaluated the effect size of 19 Internet- or computer-based CBT trials for anxiety [10]. Reger et al. showed that ICBT patients had fewer symptoms and had moderate to large effect sizes in all clinical measures compared with placebo or waitlisted assignments. More importantly: the effect size was comparable to in-person CBT. Another meta-analysis of 23 computer-aided psychotherapy RCTs on anxiety disorders was performed [16]. In that study, Cuijpers et al. found that computer-based CBT showed a large effect size when compared to their respective controls (waitlisted, placebo, etc.), and there was no difference observed between face-to-face therapy and computer-based CBT. Another meta-analysis was performed evaluating ICBT’s effectiveness in depression and anxiety symptoms [17]. Spek et al. considered 12 studies evaluating the effect of ICBT compared with controls (waitlisted, placebo, and treatment as usual). For the anxiety studies, both fixed and mixed effects analyses demonstrated high effect size for ICBT compared to controls.

So, to simplify three big meta-analyses covering 19, 23, and 12 trials (all randomized controlled): ICBT rivaled traditional CBT in benefits to patients. These data indicate that, despite requiring far less time and fewer resources, ICBT does not sacrifice quality of care. But a point of disagreement: Reger et al. showed no significant differences between patients receiving ICBT with or without therapist contact; the Culpers and Spek meta-analyses, in contrast, found a difference (small but significant in effect size). So is therapist-guided ICBT superior? More recent studies generally demonstrate improved adherence and outcomes with therapist contact (reviewed in [12]), and we will return to this debate shortly.

Physical Illness. Anxiety is frequently a comorbid condition with physical illnesses – and, potentially, a complicating factor in recovery. CBT has been shown to be effective for this population, but what about ICBT? This e-therapy has shown impressive outcomes in patients with various physical problems, such as chronic pain [18, 19] and cancer [20]. Recently, Dear et al. studied the effectiveness of guided ICBT for chronic pain in the journal Pain [19]. Patients were split into treatment or waitlisted control groups. The ICBT program, named Pain Course, was an 8-week long course specifically adapted for chronic pain and included five online lessons with summaries and homework. One clinical psychologist handled all the patient communication, including both weekly telephone calls (usually limited to 10-15 minutes) and emails. Interestingly, care continued after the modules were completed: the therapist made phone calls every 4–6 weeks posttreatment to evaluate the patient’s progress and answer questions. This study had a remarkable 93% adherence. The treatment group had significantly better posttreatment scores on the Generalized Anxiety Disorder 7-Item (GAD-7) test, and these results were retained at the 3-month follow-up. Overall, a moderate effect size was obtained for anxiety-related symptoms. The therapist spent, on average, approximately 80 minutes per patient over the 8-week course and an additional 30 minutes in the time between the end of the course and the follow-up [19].

That result has been reproduced: in another study focused on chronic pain, ICBT-treated patients were compared to control patients who participated in moderated online discussion forums [18]. The 8-week ICBT treatment consisted of eight sections with therapist support (by email). An interesting aspect of this study: the material was partly customized, with the first and last sections identical for all patients, but the remaining six sections were individualized based on the clinical interview. Buhrman et al. reported that significant effects were observed in the Beck Anxiety Inventory scores and these effects were retained at the 1-year follow-up; however, the effect size was relatively small. In addition to anxiety improvements, the patient showed both statistically and clinically significant improvement in pain catastrophizing [18].

Similarly, feelings of anxiety, helplessness, and despair were reduced in patients with cancer using a program called Cancer Coping Online (discussed in detail later) [20]. These studies provide compelling support for ICBT as a treatment for the emotional distress caused by medical conditions.

Special Populations. A few studies have looked at ICBT for special populations, including children and the elderly. Treating children with anxiety presents unique challenges (including problems with adherence). In randomized, controlled trials, juvenile anxiety patients (ages 8 to 18) were responsive to CBT treatment, and the interventions demonstrated long-term effectiveness (reviewed in [21]). Children are also more inclined to embracing new technology; therefore, there seems to be much potential for ICBT in this population. There are few studies looking into ICBT’s efficacy in children and adolescents. There are studies of BRAVE-ONLINE and MoodGym, used in primary-care and school-based settings, respectively. Both programs showed significant improvement in anxiety symptoms (reviewed in [22]). For example, one study looked at MoodGym as part of the curriculum in 30 different Australian schools. The program was presented in students’ classrooms by their teachers. The authors found that there was significant improvement in anxiety symptoms among high adherence participants compared with waitlisted controls, and the effect was retained at the 6-month follow-up [23].

Another trial evaluated the ICBT program Camp Cope-A-Lot with individual CBT and computer-linked education, support, and attention (CESA) controls [24]. Camp Cope-A-Lot is a dynamic, interactive program that consists of twelve 35-minute “levels” with video game “rewards” for level completion. The first six levels are independent, and the remaining six are therapist-assisted. CESA controls included face-to-face therapist time, access to educational information, and computer access without including CBT protocols. Both individual CBT and ICBT produced significant improvements compared with controls. There was also greater therapist adherence to the ICBT protocol, and the patients showed significantly higher satisfaction with ICBT than with individual CBT.

It could be assumed that the elderly would be less receptive to ICBT because of their reluctance to embrace new technology. However, studies have shown that age is associated with higher completion in a primary-care setting [25, 26]. One paper focused on ICBT for older patients, specifically to determine adherence rates, treatment gains, and enrollment patterns for older versus younger adults [25]. The study used an unguided, 6-lesson ICBT program that incorporated homework following each lesson. Older patients were more likely to complete the program and had similar treatment gains but did not enroll in ICBT more frequently than younger adults. This indicates that ICBT is relevant and effective in older populations – but they do not seek it out. This could possibly be remedied by increasing awareness of the programs’ availability.

Overall, the research strongly suggests that ICBT is an effective treatment for anxiety symptoms and requires fewer therapist resources than face-to-face CBT. A British Medical Journal editorial opined: “ICBT is effective for common psychiatric disorders, is a useful adjunct to face-to-face treatment, and has the potential to substantially increase accessibility to effective psychological treatment [13].”

## 3. Clinical Implementation: Australia

E-therapy is of interest to governments and government bodies; the WHO recently touted Internet-based self-management programs especially for chronic illness [27]. Given the evidence in support of ICBT, a number of national health services have rolled out government-endorsed ICBT programs. Australia, in particular, has made significant strides in developing and implementing e-mental health services, including ICBT.

In recent years, the Australian federal government has committed resources to mental health reforms (the Better Access initiative), as well as funding e-therapy projects like MoodGYM (a web-based program that is free with registration – see below). Based on the promising research in the field, the e-Mental Health Alliance was formed in 2009 to further expand Australia’s e-mental health services. The justification? “It is estimated that, even with improvements in access resulting from the Better Access initiative, only about 46 percent of people with a mental health disorder accessed treatment in 2010” [28]. E-mental health is seen as a way of reaching more people in need. In the words of Mark Butler, who was the Minister of Mental Health and Ageing: “This reform agenda gives us an opportunity and presents us with a challenge: to remain alert and alive to innovative ways to further improve our mental health system. Online mental health services offer such an approach – both as an alternative, and as an adjunct, to face-to-face mental health care.” The rhetoric is matched with ambitious plans to invest tens of millions of dollars over a short period of time in e-therapy, including ICBT, with a virtual clinic to provide service to 50,000 people over 5 years.

Australia is something of a hotbed for ICBT initiatives. THIS WAY UP Clinic, created through a university–hospital partnership, is one example. The e-clinic offers a number of anxiety-related courses including: Worry (for generalized anxiety disorder), Worry and Sadness (for mixed depression and anxiety), Shyness (for social phobias), Obsessions and Compulsions (for OCD), and Panic (for panic and agoraphobia). The courses are available by referral through clinic-registered clinicians – including family doctors but also mental health nurses and psychologists.

THIS WAY UP Clinic makes good use of the Internet: patients can set up an online calendar to provide email reminders; progress reports are provided to referring clinicians, by email; patients have access to “patient recovery stories”; the website is graphically pleasing and uses comic book-style lessons, with vivid colors, to explain key concepts in story form. These stories focus on fictional but realistic characters with mental health issues (a character-based approach, as opposed to the text-based approach of the Swedish panic study). In Lesson 1 on “The Diagnosis” in one course, for example, we are introduced to “a 32 year old woman called Jess” who recounts her struggles with low mood beginning a few years before, she notes: “I’d had a pretty tough year. My boss was always on my case.” This caption is accompanied with a picture of Jess being criticized by her boss. She describes other stresses and then reports: “Life had lost its colour” – with a picture of her sitting on a park bench but with the colors blanched out.

Like other ICBT programs, THIS WAY UP Clinic has lessons, followed by homework with summaries of the lessons and activities to apply the newly learned skills, with patients typically completing course-work over 6-8 weeks. The programs come with a fee: $55 AUS for 90 days’ access, with an additional 90 days of access upon completion. The e-clinic also features free self-help courses. Developed in the Clinical Research Unit for Anxiety and Depression (CRUfAD) at the University of New South Wales and St. Vincent’s Hospital, the courses have been the focus of much study; the CRUfAD website includes more than two dozen papers on ICBT (including a paper they suggest is in press and looks at how e-therapy is here to stay). THIS WAY UP Clinic claims strong results: according to the website, they boast a success rate of 75% for course completion and a 90% satisfaction rate. It is, as they claim, a success. THIS WAY UP Clinic is part of a larger effort, including courses designed to teach school children about mood and anxiety disorders as part of their health curriculum (THIS WAY UP SCHOOLS). And the Clinic has also experimented with different types of service delivery. A unique focus of study: incorporating ICBT into primary care. Other countries involve mental health care directly in primary-care settings. Psychiatrists in Canada and psychologists in Norway are sometimes affiliated with family medicine clinics. Can ICBT be used in primary care, a virtual experiment in Shared Care? The idea was tried [25, 29, 30]. In one study, patients enrolled in the Worry and Sadness course and were assigned either a CRUfAD psychologist or their family doctor. Not surprisingly, the busy family physician was not quite able to motivate and encourage patients the way a dedicated psychologist could: adherence was 41% vs. 89%, respectively – though the effect size was equivalent and dropouts still showed benefit. In a recent 5-year progress update to the Commonwealth Government and National Mental Health Commission [31], the e-Mental Health Alliance took stock of the e-mental health efforts. According to the report, THIS WAY UP is logging an average of 1,000 unique visitors per month, with 80 patients registered for the Clinic. Newby et al. [25] reported that more than 3,600 clinicians were registered with THIS WAY UP as of May 2013. The self-help program alone had over 6,000 registrations since its inception and approximately 275 registrations per month. How effective was treatment? The progress update suggests a cost of$1800 AUS for every Disability Adjusted Life Year (DALY) – to put that figure in context, interventions are generally considered cost-effective under $50,000 AUS per DALY. And THIS WAY UP’s reach extends beyond the doctor’s office and into the classroom: approximately 500 primary and high schools had registered to use the coursework [31]. It is apparent that in five short years, the program is being embraced by the medical community, and it is a valuable resource for the general public. This provides encouraging results for successful ICBT implementation in other countries, especially those with public health-care systems. ## 4. Advantages of ICBT There are three principle advantages to ICBT: patient empowerment, increased clinical efficiency, and cost-effectiveness. Patient empowerment. Patients have the ability to schedule their own care at any time and any location with an Internet connection [7, 16], which allows the treatment to be adjusted around work schedules and other responsibilities. And ICBT programs allow patients to acknowledge and address issues in real time rather than waiting for their next therapist appointment. ICBT also provides a solution for geographic barriers. Patients in remote rural areas with limited access to direct patient care can benefit from ICBT. Primary-care physicians in rural areas of Scotland and Australia have been encouraged to refer anxiety patients to ICBT programs. Efforts have also been made to raise general public awareness of these programs in an effort to increase self-referrals. This strategy has proven effective in clinical trials, and rural patients showed increased adherence in some cases [19, 23, 29, 32]. In one study, rural populations in Scotland were targeted; investigators contacted 56 general practitioners for direct referrals, and also distributed posters in family medicine practices, dentist offices, and emergency departments. Among the individuals who provided pretreatment data, 74% completed treatment, and there were moderate to large effect sizes on most anxiety measures [32]. ICBT may also be well-received by patients who would not otherwise seek CBT. Individuals who are isolated due to physical conditions (pain) or psychological issues (social anxiety) can seek and receive treatment (Box 1) [6]. And ICBT may also help address the significant social stigma associated with mental health care. In a study investigating ICBT patients’ attitudes to psychological treatments, up to 21% of patients expressed some form of skepticism toward psychotherapy, and up to 11% had a self-stigma ideation [33]. (It is likely that these percentages are higher in the general population.) Currently, Australia’s e-Mental Health Services are trying to decrease this stigma through online health promotion and psychoeducation programs [31]. It is hoped that ICBT will help act those with negative attitudes toward therapy to seek professional treatment [34]. The Internet also provides a sense of privacy and anonymity that may also help introverted or depressed patients to be more open and receptive to treatment [7]. Box 1: Applying ICBT to clinical practice: A 25-year-old Toronto man presents to a psychiatrist on referral from his family physician. He has a long history of low mood, and now presents with profound anxiety. Initially,he was able to work despite panic attacks and “dizzy” episodes thata neurologist couldn’t diagnose. Eventually, he left his job, 7 months prior to the initial appointment. He now spends his time in his house, afraid to leave. His social circle is limited to a few high school friends. Although no past suicide attempts are reported, he expresses passive suicidal ideation. He notes that he is particularly afraid of crowds and describes ongoing anxiety about “everything.” “I can’t get my mind to shut off.” He is very negative in his thinking, taking small issues like a disagreement with family and building them into disaster scenarios (total estrangement). We agree to start an SSRI trial, and Seroquel to target his insomnia. He is unable to attend a CBT program through his local hospital’s outpatient center because of his profound anxiety (“I don’t think I can get on the bus”), but he enrolls in the ICBT research study. Through the ICBT, he learns about Beck’s Cognitive Triad. Every Tuesday, he is emailed a module, and he emails back his “homework” by the next Monday afternoon. A therapist is available to answer any questions by email and also gives email comments on his assignment. When he misses a deadline, the therapist sends a reminder email. After completing the 6 sessions, he writes, “This is really helpful. Thank you so much.” He gained insight into his cognitive errors, including his catastrophization. With the medications and the ICBT, he notes a marked reduction in depressive symptoms. Commentary.This case illustrates how ICBT can be beneficial for moderate anxiety, in combination with psychopharmacological treatment. In this case, in-person therapy was not an option. The ICBT was immediately accessible. Increased Clinical Efficiency. ICBT saves time and resources for health-care providers. As mentioned above, guided ICBT generally shows higher adherence than unguided ICBT. This support can include telephone calls and emails, which are less time consuming and require fewer resources than traditional CBT. Such communications can occur asynchronously, so the therapist does not have to be readily available [7]. In an RCT of face-to-face therapy versus ICBT for social phobia, the therapists spend on average 18 minutes per ICBT patients, compared with 240 minutes for face-to-face patients with no significant differences between the groups posttreatment [35]. How much training is required for a therapist who can guide ICBT? In a study on generalized anxiety disorder, the efficacy of clinicians was compared to technicians [36]. In that study, the patients were assisted by either a registered clinical psychologist or a technician with no previous psychological experience who was provided with a script. The technician gave support and encouragement, and answered questions, but was instructed not to give clinical advice. The results demonstrated that a modestly trained individual with no prior therapy or counseling qualifications could guide ICBT with similar outcomes and adherence as a clinician. In a different study on the content of therapist interactions in ICBT, investigators analyzed 490 emails collected from patients receiving treatment for generalized anxiety disorder [37]. Preliminary content analysis identified eight therapist behaviors: task reinforcement, task prompting, empathetic utterance, deadline flexibility, alliance bolstering, psychoeducation, self-disclosure, and self-efficacy shaping. Of those, the first three were correlated with patients completing their modules. These studies, taken together, support the idea of a resource- and cost-effective treatment model with a single clinician supervising and supporting several technicians with communication scripts optimized to improve adherence. (It could be pointed out, though, that the often-used model of therapist-guided care is still much more resource-effective than traditional CBT.) Cost-effectiveness. Several studies investigated the cost-effectiveness of ICBT. On average, the probability of ICBT as a cost-effective treatment was 57% (range 38–96%) compared with waitlisted controls (reviewed in [38]). Hedman et al. performed a cost-effectiveness analysis on data from a social anxiety disorder trial comparing in-person CBT to ICBT [39]. The data included posttreatment and 6-month follow-up evaluations, and the results showed that there was a 79.5% probability that ICBT would be efficacious at a lower cost. In addition, they reported that the societal cost was reduced by$7,046 USD for each patient. Several other studies investigating societal costs showed that for each patient who improved using ICBT, the overall societal cost was reduced by $16,000 USD to$39,000 USD compared with waitlisted controls [38]. These studies clearly indicate that there is a huge cost benefit to implementing ICBT.

Another study considered the cost-effectiveness of translating an existing (English) ICBT program into another language (Norwegian). Data showed that the cost of translating well-researched programs was roughly a quarter of the cost of developing a new one, and implementing it would result in a 21-fold return in investment based on the quality-adjusted life years gained [40]. By using existing, well-tested programs translated into other languages, health services have the opportunity to expand their programs with small investment.

There are three potential disadvantages to ICBT: lack of direct patient monitoring, misdiagnosis/inappropriate treatment, and accessibility issues.

Lack of Direct Patient Monitoring. A number of studies have shown that the lack of direct patient monitoring may contribute to low adherence, both in unguided ICBT and in ICBT managed by primary-care physicians [12, 25, 29]. This is a serious consideration that needs further study and is a major challenge for future program development [31].

Several studies indicated that for anxiety patients, poorer adherence is related to increased baseline symptom severity and baseline psychological distress (reviewed in [41]). For those patients that do not complete their programs, several factors contribute to their lack of adherence. The reasons vary by study, and include time constraints, lack of motivation, technical problems, lack of face-to-face therapy, preference for medication, and skepticism regarding the treatment’s effectiveness (reviewed in [42]).

And while there are many reasons for nonadherence, unguided ICBT seems to have very high dropout rates. Different studies have different designs and interventions, but the pattern is clear: patients enrolled in MoodGym: 37% completed the self-directed program [43]; patients with mild panic symptoms enrolled in Don’t Panic Online: 27% completed the unguided ICBT [44]; patients enrolled in This Way Up Clinic’s Sadness program without therapist support: 33% completed it, compared to 77% in a clinician-assisted version [45]. Adding in the support of primary-care physicians, incidentally, helped, but not profoundly, as noted above (in the context of THIS WAY UP). Primary-care physician-guided ICBT had repeatedly achieved approximately 40–60% adherence, whereas involving a clinical psychologist-guide was something of a game-changer in terms of adherence, with rates rising to 80–90% [29, 30, 41]. (However, conflicting results should be noted. Berger et al., for example, had a unique study in that they directly compared unguided, minimally guided, and flexibly guided ICBT for social phobias, and found all three treatments to be equally effective with no statistical difference in adherence rates [46] – suggesting that other factors are at play in overall adherence, not simply whether or not the therapy involves a therapist.)

The big question is: how much therapist support is needed? Even modest therapist interaction seems to be effective. For example, Klein et al. used emails in ICBT and demonstrated that one email a week increased adherence rates in panic disorder patients, and was as effective as three emails a week [47]. And, as discussed above, this support may not necessarily need to be provided by a clinician.

Misdiagnosis/Inappropriate Treatment. Because the treatment is often based on patient self-assessment and not direct interaction, critical cues (such as a patient’s voice or body language) cannot be assessed, possibly leading to misdiagnosis and inappropriate treatment as signs of comorbidity, distress, or an imminent crisis are missed. However, several studies have shown that well-crafted questionnaires are effective and may in fact highlight potentially overlooked issues (reviewed in [12]). There is also evidence that statistical prediction is superior to a clinician’s judgment in diagnosis and treatment planning [48]. A recent study compared automated pretreatment assessments with clinician diagnoses for ICBT program assignment. In this study, the patients were evaluated and assigned programs for depression, anxiety, or mixed depression and anxiety. Similar outcomes were achieved with automated- and clinician-diagnosed assessments, indicating that the automated assessment was as effective as the clinician diagnosis [49]. It seems likely that this concern may be overstated.

Accessibility. Because roughly 25% of the population in high-income countries does not have Internet access [50] and up to 30% may not own a computer [51], ICBT may inadvertently select against economically disadvantaged, rurally located, or elderly patients, many of whom would most benefit from this resource. In the rural Scotland study, several laptops were provided to patients to give them access to the program [32]. This further underscores that those who benefit most may not have the resources necessary for treatment. Additionally, patients who are not technologically proficient may be hesitant about replacing in person visits with e-treatment. ICBT appears to be preferred by educated, technologically savvy patients (reviewed in [52]). To reach out to those less comfortable with technology, ICBT programs must be user-friendly to be effective and achieve a sustained response (reviewed in [51]).

When considering disadvantages, we must acknowledge that the long-term effects of ICBT have not yet been adequately studied; thus, the specific populations who will benefit and the potential adverse effects of ICBT remain to be established, especially for programs that are administered without therapist support.

## 6. ICBT resources available

Numerous ICBT programs are available, both web-based and app-based. But not all ICBT programs are created equal – obviously, the quality is mixed. Several ICBT programs stand out: some carry formal endorsements of government health services; some belong to clinical groups that hold hospital and academic affiliations; some have been studied in clinical trials with published results; some have been developed with larger development teams involving psychologists and psychiatrists. Fourteen anxiety and anxiety-related ICBT websites/apps are described in Table 1. We highlight a few here:

FearFighter (http://www.fearfighter.com) went live in 2005 and focuses on patients with panic and phobias. It consists of nine steps and includes activity worksheets and progress tracking. FearFighter includes video explanation of core CBT concepts. Step 6, for example, is introduced by a young (reassuring) woman who describes exposure therapy, after quickly reviewing the lessons of the past 5 steps. The approach, then, is therapist-focused, as opposed to the character-focus of THIS WAY UP or the text-based focus of the Swedish panic study. The videos include clear graphics – demonstrating, for instance, diaphragmatic breathing with illustrations of lungs and breathing techniques. Therapist support is offered by phone and in-person, but also through Skype.

FearFighter is endorsed by the United Kingdom’s National Institute for Health and Care Excellence (NICE). The program can be “prescribed” by a family doctor or another health professional (and, for some English patients, the fees are then covered by the NHS), but it can also be self-referred. FearFighter has been the subject of several studies [32, 53-57] with patients typically having moderate to moderately severe symptoms at the beginning and showing improvement equivalent to the effects of traditional CBT – but the dropout rate was higher with this program. FearFighter consistently showed high patient satisfaction [57]. The program was also shown to be effective in rural areas with limited access to psychological resources [32].

Though iTunes offers many ICBT apps, the literature on these programs is limited. One study concluded that there were no outcome differences between web-based and app-based ICBT [58]. One popular app is MoodKit, developed by Pepperdine University psychologists Edrick Dorian and Drew Erhardt, which is available for a small fee. MoodKit has gathered much press and was even featured in Discover magazine. Among its various features: MoodKit offers over 200 activities in five mood-improvement categories that suggests concrete steps to improve mood, and includes examples and tips to help implement the concepts.

MoodKit includes a Thought Checker to identify thoughts that contribute to negative feelings and the situations and patterns that generate them, which allows the user to manage them. The Thought Checker also walks the user through the factors that contribute to their feelings and helps them identify alternative positive or more rational thoughts to alleviate the negative ones. The Mood Tracker charts the mood range of the user throughout the day and records their progress over time. The app has a Journal function to collate the patient’s notes and comes with preformatted templates to ensure productive journaling. Many of these features can be exported for the user’s personal records or can be integrated into professional therapy.

And for those with physical illness? Cancer Coping Online is being developed. This 6-week intervention program targets distress in cancer patients and includes CBT-based worksheets and immediate-feedback quizzes. The modules include topics like communicating with medical professionals and decision making, coping with physical symptoms, coping with emotional distress, interacting with friends and family, and managing life style following remission. In a pilot study, the program achieved reduced anxious preoccupation [20]. High patient satisfaction was reported. It should be emphasized that the data are small and the experimentation is very early: 10 of the 12 participants completed the full program, and the patients stated that the program was easy to use. Negative affect, hopelessness, and anxiety were reduced in patients following completion. The program has moved forward for further testing. Preliminary data from this new trial showed significant, sustained interaction effects for both cancer-specific distress and anxious preoccupation, indicating that type of ICBT may help prevent and treat the anxiety so commonly found in cancer patients [59].

#### Table 1.

ICBT Resources

[i] - 1The list of resources is not fully inclusive of all available sites or applications. Additional resources are available, some of which may be found at http://www.getselfhelp.co.uk/links2.htm.

## 7. Challenges and future directions

Moving forward, there are a number of questions that need to be addressed. What patient-specific factors can help make ICBT successful? What about program-specific factors or therapist-specific factors? And, with the latter, what level of therapist support is optimal for completion and efficacy of ICBT? Is there a role for extra or “booster” sessions of ICBT – and how often (sessions every six months or twelve months)? Not surprisingly, others are wrestling with similar questions. The e-Mental Health Alliance has outlined its own challenges for popularizing and improving e-therapy: increasing awareness of available services and optimizing the programs to increase adherence and maximize outcomes [31].

Current evidence indicates that ICBT has tremendous potential in psychiatric practice. Studies should examine if additional types of therapy, such as pharmacotherapy, or even intermittent face-to-face therapist support are needed in combination with ICBT for optimal benefit. The literature regarding “blended treatments,” a combination of both online and face-to-face therapy, is limited [68, 69]. And what about special populations? Currently, there are a limited number of studies on anxiety and ICBT in children, elderly populations, and rural settings [21, 22, 26, 32]. There is a dearth of literature on several relevant populations including minorities, the economically disadvantaged, and the poorly educated [6]. In many ways, these populations are the ideal demographics of ICBT programs, and ensuring that the programs are effective and accessible to them should be a priority.

Further research should also focus on identifying those who will benefit the most and who may actually be harmed by ICBT – such as those with serious phobias, severe depression, or suicidal thoughts. Similar to traditional CBT, ICBT is not appropriate for patients with severe illnesses, which is reflected in the poorer adherence rates [41]. Those patients with less support or those who feel isolated may react negatively to ICBT due to a lack of in-person visits. This is consistent with the self-reported reasons for noncompletion [42]. For these patients, intermittent clinical visits or support groups may be beneficial. Thus, ICBT may be better-suited for patients with mild to moderate disease and those with sufficient social support [51]. Current evidence indicates that ICBT is most effective when a) an accurate diagnosis has been made before ICBT is initiated, b) ICBT is combined with therapist support, and c) the website or app is user-friendly [52]. Regarding the software, the format, speaking voice, font, organization, and various other factors can influence the overall experience, participation, adherence, and outcomes. There is value in tailoring ICBT modules according to the needs of each patient, which may allow individualized treatment and better outcomes. As discussed above, anxiety is frequently comorbid with other disorders. While there are some studies focused on developing a single program for mixed anxiety and depression [25, 30, 58], there is room for improvement of the programs, diversification into other comorbidities, and developing alternative strategies such as a variety of modules that can be mixed and matched for each individual patient. Determining the optimal benefits of ICBT will require robust, large-scale studies with long-term follow-up. These studies would also help address whether repeated ICBT exposure at fixed intervals is necessary for optimal effects.

## 8. Conclusions

ICBT has enormous potential. ICBT offers a revolutionary approach for treating patients who suffer from anxiety or anxiety comorbid conditions while minimizing the limited resources and time of health-care professionals. There is robust evidence about its efficacy for patients with anxiety disorders. Governments are increasingly finding this treatment modality attractive in terms of both cost and efficacy. Overall, it is clinically efficient, consistent with the way patients interact with health-care providers, and empowers patients. In the Internet Age, with patients accessing health resources online, from physician ratings to medical information, this type of psychological intervention is an ideal direction for the future.

## References

1 - Baxter AJ, Scott KM, Vos T, Whiteford HA. Global prevalence of anxiety disorders: a systematic review and meta-regression. Psychol Med. 2013;43(5):897-910.
2 - Pedersen CB, Mors O, Bertelsen A, Waltoft BL, Agerbo E, McGrath JJ, et al. A comprehensive nationwide study of the incidence rate and lifetime risk for treated mental disorders. JAMA Psychiat. 2014;71(5):573-81.
3 - Organization WH. Mental Health Atlas: 2011. Geneva: Department of Mental Health and Substance Abuse, World Health Organization; 2011. 82 p.
4 - Katzman MA, Bleau P, Blier P, Chokka P, Kjernisted K, Van Ameringen M, et al. Canadian clinical practice guidelines for the management of anxiety, posttraumatic stress and obsessive-compulsive disorders. BMC Psychiat. 2014;14 Suppl 1:S1.
5 - Kessler D, Lewis G, Kaur S, Wiles N, King M, Weich S, et al. Therapist-delivered Internet psychotherapy for depression in primary care: a randomised controlled trial. Lancet. 2009;374(9690):628-34.
6 - Williams AD, Andrews G. The effectiveness of Internet cognitive behavioural therapy (iCBT) for depression in primary care: a quality assurance study. PloS one. 2013;8(2):e57447.
7 - Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cog Behav Ther. 2009;38(4):196-205.
8 - Australia and New Zealand Health Scanning Network. Horizon Scanning Technology Prioritising Summary. Internet delivered cognitive behavioural therapy for patients with depression In: Commonwealth of Australia Department of Health and Ageing, editor. 2009.
9 - Online Therapy User Update. March 2013. In: University of Regina, editor. 2013.
10 - Reger MA, Gahm GA. A meta-analysis of the effects of Internet- and computer-based cognitive-behavioral treatments for anxiety. J Clin Psychol. 2009;65(1):53-75.
11 - Hedman E, Ljotsson B, Ruck C, Bergstrom J, Andersson G, Kaldo V, et al. Effectiveness of internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta psychiatrica Scandinavica. 2013;128(6):457-67.
12 - Andersson G, Titov N. Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiat: official journal of the World Psychiatric Association. 2014;13(1):4-11.
13 - Hedman E. Therapist guided internet delivered cognitive behavioural therapy. BMJ. 2014;348:g1977.
14 - Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiat: official journal of the World Psychiatric Association. 2014;13(3):288-95.
15 - Hedman E, Furmark T, Carlbring P, Ljotsson B, Ruck C, Lindefors N, et al. A 5-Year follow-up of internet-based cognitive behavior therapy for social anxiety disorder. J Med Internet Res. 2011;13(2):e39.
16 - Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, Andersson G. Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cog Behav Ther. 2009;38(2):66-82.
17 - Spek V, Cuijpers P, Nyklicek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med. 2007;37(3):319-28.
18 - Buhrman M, Syk M, Burvall O, Hartig T, Gordh T, Andersson G. Individualized guided internet-delivered cognitive behaviour therapy for chronic pain patients with comorbid depression and anxiety: a randomized controlled trial. The clinical journal of pain. 2015; 31:504-16.
19 - Dear BF, Titov N, Perry KN, Johnston L, Wootton BM, Terides MD, et al. The Pain Course: a randomised controlled trial of a clinician-guided Internet-delivered cognitive behaviour therapy program for managing chronic pain and emotional well-being. Pain. 2013;154(6):942-50.
20 - Beatty LJ, Koczwara B, Wade TD. ‘Cancer Coping Online’: A pilot trial of a self-guided CBT internet intervention for cancer-related distress. Electr J Appl Psych. 2011;7:17-25.
21 - Compton SN, March JS, Brent D, Albano AMt, Weersing R, Curry J. Cognitive-behavioral psychotherapy for anxiety and depressive disorders in children and adolescents: an evidence-based medicine review. J Am Acad Child Adol Psychiat. 2004;43(8):930-59.
22 - Calear AL, Christensen H. Review of internet-based prevention and treatment programs for anxiety and depression in children and adolescents. Med J Aus. 2010;192(11 Suppl):S12-4.
23 - Calear AL, Christensen H, Mackinnon A, Griffiths KM. Adherence to the MoodGYM program: outcomes and predictors for an adolescent school-based population. J Affect Dis. 2013;147(1-3):338-44.
24 - Khanna MS, Kendall PC. Computer-assisted cognitive behavioral therapy for child anxiety: results of a randomized clinical trial. J Consult Clin Psychol. 2010;78(5):737-45.
25 - Newby JM, Mewton L, Williams AD, Andrews G. Effectiveness of transdiagnostic Internet cognitive behavioural treatment for mixed anxiety and depression in primary care. J Affect Dis. 2014;165:45-52.
26 - Mewton L, Sachdev PS, Andrews G. A naturalistic study of the acceptability and effectiveness of internet-delivered cognitive behavioural therapy for psychiatric disorders in older Australians. PloS one. 2013;8(8):e71825.
27 - Organization WH. Integrating the response to mental health disorders and other chronic diseases in health care systems. Geneva: World Health Organization; 2014. 50 p.
28 - Department of Health and Ageing. E‐Mental Health Strategy for Australia. In: Department of Health and Ageing, editor.: Commonwealth of Australia,. 2012. p. 1-22.
29 - Williams AD, O'Moore K, Mason E, Andrews G. The effectiveness of internet cognitive behaviour therapy (iCBT) for social anxiety disorder across two routine practice pathways. Internet Intervent. 2014;1(4):225-9.
30 - Newby JM, Mackenzie A, Williams AD, McIntyre K, Watts S, Wong N, et al. Internet cognitive behavioural therapy for mixed anxiety and depression: a randomized controlled trial and evidence of effectiveness in primary care. Psychol Med. 2013;43(12):2635-48.
31 - e-Mental Health Alliance. e-Mental Health Services in Australia 2014: Current and Future. 2014. p. 28.
32 - Hayward L, MacGregor AD, Peck DF, Wilkes P. The feasibility and effectiveness of computer-guided CBT (FearFighter) in a rural area. Behav Cog Psychother. 2007;35(04):409-19.
33 - Moritz S, Schroder J, Meyer B, Hauschildt M. The more it is needed, the less it is wanted: attitudes toward face-to-face intervention among depressed patients undergoing online treatment. Depr Anx. 2013;30(2):157-67.
34 - Cuijpers P, Donker T, Johansson R, Mohr DC, van Straten A, Andersson G. Self-guided psychological treatment for depressive symptoms: a meta-analysis. PloS one. 2011;6(6):e21274.
35 - Andrews G, Davies M, Titov N. Effectiveness randomized controlled trial of face to face versus Internet cognitive behaviour therapy for social phobia. Aus N Zea J Psychiat. 2011;45(4):337-40.
36 - Robinson E, Titov N, Andrews G, McIntyre K, Schwencke G, Solley K. Internet treatment for generalized anxiety disorder: a randomized controlled trial comparing clinician vs. technician assistance. PloS one. 2010;5(6):e10942.
37 - Paxling B, Lundgren S, Norman A, Almlov J, Carlbring P, Cuijpers P, et al. Therapist behaviours in internet-delivered cognitive behaviour therapy: analyses of e-mail correspondence in the treatment of generalized anxiety disorder. Behav Cog Psychother. 2013;41(3):280-9.
38 - Hedman E, Ljotsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Exp Rev Pharmacoecon Outcomes Res. 2012;12(6):745-64.
39 - Hedman E, Andersson E, Ljotsson B, Andersson G, Ruck C, Lindefors N. Cost-effectiveness of Internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: results from a randomized controlled trial. Behav Res Ther. 2011;49(11):729-36.
40 - Lintvedt OK, Griffiths KM, Eisemann M, Waterloo K. Evaluating the translation process of an Internet-based self-help intervention for prevention of depression: a cost-effectiveness analysis. J Med Internet Res. 2013;15(1):e18.
41 - Mewton L, Smith J, Rossouw P, Andrews G. Current perspectives on Internet-delivered cognitive behavioral therapy for adults with anxiety and related disorders. Psychol Res Behav Manag. 2014;7:37-46.
42 - Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res. 2009;11(2):e13.
43 - Twomey C, O'Reilly G, Byrne M, Bury M, White A, Kissane S, et al. A randomized controlled trial of the computerized CBT programme, MoodGYM, for public mental health service users waiting for interventions. Brit J Clin Psychol/the British Psychological Society. 2014;53(4):433-50.
44 - van Ballegooijen W, Riper H, Klein B, Ebert DD, Kramer J, Meulenbeek P, et al. An Internet-based guided self-help intervention for panic symptoms: randomized controlled trial. J Med Internet Res. 2013;15(7):e154.
45 - Titov N, Andrews G, Choi I, Schwencke G, Mahoney A. Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia. Aus N Zea J Psychiat. 2008;42(12):1030-40.
46 - Berger T, Caspar F, Richardson R, Kneubuhler B, Sutter D, Andersson G. Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help. Behav Res Ther. 2011;49(3):158-69.
47 - Klein B, Austin D, Pier C, Kiropoulos L, Shandley K, Mitchell J, et al. Internet-based treatment for panic disorder: does frequency of therapist contact make a difference? Cog Behav Ther. 2009;38(2):100-13.
48 - Meehl PE. Clinical versus Statistical Prediction; a Theoretical Analysis and a Review of the Evidence. Minneapolis: University of Minnesota Press; 1954, p. 149.
49 - Mason EC, Andrews G. The use of automated assessments in internet-based CBT: The computer will be with you shortly. Internet Intervent. 2014;1(4):216-24.
50 - World Development Indicators 2014: The World Bank; 2014. 136 p.
51 - Carlbring P, Andersson G. Internet and psychological treatment. How well can they be combined? Comp Hum Behav. 2006;22(3):545-53.
52 - Andersson G, Carlbring P, Berger T, Almlov J, Cuijpers P. What makes Internet therapy work? Cognitive behaviour therapy. 2009;38 Suppl 1:55-60.
53 - Marks IM, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D. Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med. 2004;34(1):9-17.
54 - Gega L, Marks I, Mataix-Cols D. Computer-aided CBT self-help for anxiety and depressive disorders: experience of a London clinic and future directions. J Clin Psychol. 2004;60(2):147-57.
55 - Schneider AJ, Mataix-Cols D, Marks IM, Bachofen M. Internet-guided self-help with or without exposure therapy for phobic and panic disorders. Psychother Psychosom. 2005;74(3):154-64.
56 - McCrone P, Marks IM, Mataix-Cols D, Kenwright M, McDonough M. Computer-aided self-exposure therapy for phobia/panic disorder: a pilot economic evaluation. Cog Behav Ther. 2009;38(2):91-9.
57 - MacGregor AD, Hayward L, Peck DF, Wilkes P. Empirically grounded clinical interventions clients' and referrers' perceptions of computer-guided CBT (FearFighter). Behavi Cog Psychother. 2009;37(1):1-9.
58 - Watts S, Mackenzie A, Thomas C, Griskaitis A, Mewton L, Williams A, et al. CBT for depression: a pilot RCT comparing mobile phone vs. computer. BMC Psychiat. 2013;13:49.
59 - Beatty L, Koczwara B, Wade T, editors. Cancer Coping Online: Findings and lessons learned from a phase II RCT of an eHealth program for reducing cancer-distress. Clinical Oncological Society of Australia Annual Scientific Meeting; 2013; Adelaide, Australia: Asia-Pacific J Clin Oncol.
60 - Sunderland M, Wong N, Hilvert-Bruce Z, Andrews G. Investigating trajectories of change in psychological distress amongst patients with depression and generalised anxiety disorder treated with internet cognitive behavioural therapy. Behav Res Ther. 2012;50(6):374-80.
61 - Cavanagh K, Seccombe N, Lidbetter N. The implementation of computerized cognitive behavioural therapies in a service user-led, third sector self help clinic. Behav Cog Psychother. 2011;39(4):427-42.
62 - Marks IM, Cuijpers P, Cavanagh K, van Straten A, Gega L, Andersson G. Meta-analysis of computer-aided psychotherapy: problems and partial solutions. Cog Behav Ther. 2009;38(2):83-90.
63 - Schneider J, Sarrami Foroushani P, Grime P, Thornicroft G. Acceptability of online self-help to people with depression: users' views of MoodGYM versus informational websites. J Med Internet Res. 2014;16(3):e90.
64 - Stinson JN, Jibb LA, Nguyen C, Nathan PC, Maloney AM, Dupuis LL, et al. Development and testing of a multidimensional iPhone pain assessment application for adolescents with cancer. J Med Internet Res. 2013;15(3):e51.
65 - Greist JH, Marks IM, Baer L, Kobak KA, Wenzel KW, Hirsch MJ, et al. Behavior therapy for obsessive-compulsive disorder guided by a computer or by a clinician compared with relaxation as a control. J Clin Psychiat. 2002;63(2):138-45.
66 - Nakagawa A, Marks IM, Park JM, Bachofen M, Baer L, Dottl SL, et al. Self-treatment of obsessive-compulsive disorder guided by manual and computer-conducted telephone interview. J Telemed Telecare. 2000;6(1):22-6.
67 - Kuhn E, Greene C, Hoffman J, Nguyen T, Wald L, Schmidt J, et al. Preliminary evaluation of PTSD Coach, a smartphone app for post-traumatic stress symptoms. Milit Medicine. 2014;179(1):12-8.
68 - Mansson KN, Skagius Ruiz E, Gervind E, Dahlin M, Andersson G. Development and initial evaluation of an Internet-based support system for face-to-face cognitive behavior therapy: a proof of concept study. J Med Internet Res. 2013;15(12):e280.
69 - Kooistra LC, Wiersma JE, Ruwaard J, van Oppen P, Smit F, Lokkerbol J, et al. Blended vs. face-to-face cognitive behavioural treatment for major depression in specialized mental health care: study protocol of a randomized controlled cost-effectiveness trial. BMC Psychiat. 2014;14(1):290.