A Plea for Mixed Methods Research in the Field of Counseling

Abstract

We need more research in counseling if we want to strengthen counseling as a profession and if we want to implement counseling in mental health systems. Thus, the research should be multiple dimensional. This chapter is a plea for mixed-methods research (MMR) designs in the field of counseling. Even if MMR is very elaborate, it is worth doing. By way of example, I would like to briefly outline three of my projects, using MMR. The first one is a mixed methods research study on the video-based counseling method Marte Meo. The second project is one concerning genograms. Genograms are an integral part of therapy and counseling. The third MMR project is an elaborate research project which we carry out on behalf of the “Deutsche Gesellschaft für Beratung”, the German National Association for Counseling (Member of the European Association for Counseling, EAC, and the International Association for Counseling, IAC) to develop a German qualifications framework for Counseling—in the context of the European Qualifications Framework (EQF). Finally, I refer to Guetterman et al. who provide some empirical evidence for researchers who wish to take full advantage of mixed methods to address pressing clinical and public health issues.

1. Introduction

Mixed methods research (MMR) is defined as the collection, analysis, and integration of both quantitative data (e.g., RCT outcome) and qualitative data (e.g., observations, semi-structured interviews) to provide a more comprehensive understanding of a research problem than might be obtained through quantitative or qualitative research alone [1, 2]. Relevant strategies for the use of mixed methods in health services research include adding qualitative interviews to follow up on the outcomes of intervention trials, gathering both quantitative and qualitative data to assess patient reactions to a program implemented in a community health setting, or using qualitative data to describe or explain the mechanism of a study correlating behavioral and social factors to specific health [3]. We want to find out if this is important in the field of counseling.

2. What is mixed methods research (MMR)?

I want to start this chapter with the “Classification”, the “five purposes for mixing in mixed-methods research”:

1. Triangulation seeks convergence, corroboration, and correspondence of results in different ways.

2. Complementarity seeks elaboration, enhancement, illustration, and clarification of the results from one method with the results from the other method.

3. Development seeks to use the results from one method to help develop or inform the other method, where interpretation includes sampling and implementation, as well as measurement decisions.

4. Initiation seeks the discovery of paradox and contradiction, new perspectives of frameworks, and the recasting of questions or results from one method with questions or results from the other method.

5. Expansion seeks to extend the breadth and range of inquiry by using different methods for different inquiry components [4, 5].

Next to the five purposes MMR has five essential characteristics: (1) the collection and analysis of both quantitative and qualitative data, (2) the use of rigorous procedures in conducting quantitative and qualitative research, (3) the integration of the findings, (4) the use of mixed method designs and (5) the use of a conceptual framework [6]. By “Integration”, we mean integrating quantitative and qualitative research through our research teams, philosophies, research process, and research methods.

A mixed methods research project provides more insight than qualitative or quantitative data alone by greater mining data depth. The different perspectives from linking enable the databases to “talk” to each other. We can compare the two database results and follow up quantitative results with qualitative data collection.

2.1 Core designs

There are three different core designs: convergent design, explanatory sequential design, and exploratory sequential design. A plan’s importance is: identifying a theoretical framework, writing a mixed methods question, and writing a mixed method study aim, composing the study using a writing structure that matches the design, developing a joint display for integration, identifying the methodological/validity issues in design, drawing a diagram of configuration, identifying the type of mixed methods design and creating a title for the project.

The convergent design qualitative interviews, analysis, quantitative survey, and analysis stand alone in the first phase. Then they are merged and an interpretation follows. You should choose a convergent design when your mixed method project intends to compare results, develop broader products, validate data, and build cases. It can be helpful when you need rapid data collection. It is also beneficial when you have equal emphasis on both quantitative and qualitative data (Figure 1).

Figure 1.

The convergent design [6].

The explanatory sequential design has three phases. In the first phase, the quantitative survey and analysis of the qualitative interviews and comments will be explained in the second phase. In the third, an interpretation follows (Figure 2).

Figure 2.

The explanatory sequential design [6].

The explanatory sequential design starts with collecting quantitative data through a cross-sectional web-based survey, which delivers numeric data. It follows an analysis of the data through data screening, providing descriptive statistics and factor loadings. In the case selection, an interview protocol will be developed, participant dorm will be selected and interviewed.

The collection of qualitative data happens through documents and telephone interviews. It makes up the text and image data. Lastly, the qualitative data analysis follows a cross-thematic study and delivers a cross-thematic matrix and a visual model of multiple case analyses.

You should choose an explanatory sequential design when your mixed method project intends to explain surprising, contradictory, outlier results or results that do not match theory or form groups/cases for further analysis. Other reasons could be when you have time to conduct your study in phases or emphasize starting a project from a quantitative perspective.

The third core design is the exploratory sequential design.

As illustrated in Figure 3 this design has three phases. In the first phase, interviews, observations, and other qualitative methods are conducted and analyses are made. The analysis of qualitative data leads to the development of a quantitative device.

Figure 3.

Exploratory sequential design [6].

The second phase is the quantitative phase, which includes an instrument design or intervention design. Then follows a quantitative test of an instrument or intervention in phase 3, which leads to an interpretation. “We can first explore qualitatively, and then test out the ideas quantitatively” [6]. Afterward, quantitative data will be collected and analyzed and an interpretation follows.

An exploratory sequential design should be chosen when your mixed methods project intends to build and test an intervention, instrument, survey, app or website, or new variables. Other reasons could be when you emphasize starting your project qualitatively or when you have time to collect in phases over time.

Lastly, a short comment about complex designs. Typically, complex applications are used when researchers have multiple research phases, multiyear research projects, large funded projects, multiple researchers, or the inclusion of mixed methods core designs within different phases of research [7].

3. Three examples of mixed methods research

3.1 Mixed methods research in our Marte Meo project

Marte Meo¹ is a video-based counseling method founded by Maria Aarts in the Netherlands and is now in worldwide use [8]. Marte Meo has been adopted and put into practice by a large and diverse network of trained and certified counselors worldwide.

With the help of a model of beneficial interaction behavior, Marte Meo aims to support personal development. In this respect, it stands in tradition with the humanistic approach. It was founded with the aim of reducing symptomatology. However, we found that in practice, more Marte Meo counselors aim for personal growth. The focus is on relationships that exhibit “complementarity”. This is mostly given in a dyad relationship, where one person is responsible, supports, cares educates, etc. (e.g., parents, educators, teachers, careers), and another person needs this support (e.g., infant, child, adolescent, sick, disabled, dementia sufferer [8, 9].

In our research, we choose an integrated exploratory sequential design [6], which seems to be best suited to our purposes. It enabled us to discover in detail how a selected group of experts and parents applied for the Marte Meo program, and we then tested out the ideas culled from that process quantitatively with a large convenience sample.

The exploratory sequential design of the Marte Meo project has five phases as illustrated in Figure 4. A systematic literature review is carried out in the first phase, which builds into qualitative interviews and analysis in the second phase. The third phase builds an analysis of videotaped observations. A fourth quantitative phase with an online survey follows and in the last step, the fifth phase follows an interpretation.

Figure 4.

Exploratory sequential Design of the Marte Meo Project.

The staged qualitative research consisted of designing, conducting, and analyzing semi-structured interviews with parents and Marte Meo Counselors and then using that analysis to inform the design, the conduct, and analysis of videotaped observations of everyday situations for example in day-care centers to examine the process and effects of Marte Meo interaction elements on children.

Combining these qualitative analyses then became the basis for developing an online questionnaire that could enable us to collect quantitative data on the current use of Marte Meo in practice by experts. As a result of this design, four phases of analysis will be carried out: after the two qualitative phases, after the quantitative phase, and during the integration phase, which will connect the data strands and expand the initial qualitative exploratory results.

With the aim of obtaining more meaningful results on the application of Marte Meo in counseling and therapy, it would be desirable to collect an international and generally more heterogeneous sample that includes various groups of people (clients, other affected persons, or experts in other methods, etc.).

In response to the question “What further development does the method need?”, the experts’ statements strongly pointed to the desire for a scientific foundation for the Marte Meo method. The respondents hoped that increased scientific research on the effectiveness of the method in various fields of application would lead to greater acceptance and consequently to the method being financed by public bodies. The fitting of the effect factors according to Grawe [10] to the basic principles of Marte Meo suggests further follow-up studies are needed to make statements about the effectiveness of Marte Meo. For example, it would be interesting to correlate the experts’ self-declarations of the benefits of Marte Meo with objective behavioral data of the clients and to secure them by means of inference statistics. A concrete criterion for this could be the increased rate of observable Marte Meo elements (beneficial interaction) applied visibly and audibly in video recordings over several counseling sessions [11]. Marte Meo seems to fit different approaches. Possible moderators for the effectiveness of Marte Meo could therefore be the experience, knowledge, or skills of the expert. More than one-fifth of respondents stated that they had children themselves. This factor, too, could be a moderator for the effect of the Marte Meo consultancy services used. Overall, although Marte Meo is often used in combination with other methods in a wide variety of contexts, there seems to be little research on the effect of combined use. Here, it would be interesting to get context-specific information about which combinations of methods are effective for which concerns and contexts.

At the time of research, no study could be found that describes the current status of the nationwide application of Marte Meo in practice, counseling and therapy. The results indicate a high degree of diversity in the use of Marte Meo with a high overall satisfaction among experts. Almost one-third of Marte Meo usage takes place in everyday teaching, followed by counseling, exchange with colleagues, teaching, supervision, and therapy. Marte Meo is often combined with the systemic approach, among other things, because it is flexible enough and allows the experts a lot of leeway in their approach, depending on their individual personalities.

The results of the work show that Marte Meo is perceived as beneficial by its implementers. The experts reported more joy and success in their work. In particular, in the pedagogical context, the daily, resource-oriented “Marte Meo view”, which has been sharpened by the training, seems to be essential, as it allows for an awareness of the needs of the interaction partners and the beneficial interaction elements. Moreover, for some respondents, the use of Marte Meo seems not to be limited to professional practice but is expressed in a general humanistic attitude towards interpersonal relationships of all kinds.

Regarding various application contexts and concrete advisory procedures, well-founded insights into the benefits and effectiveness of Marte Meo could be found in the future, thus ensuring increasing quality assurance or control and the institutional establishment of the method. In order to consider the diversity in the application of Marte Meo, future research can make use of the results of the qualitative and quantitative studies and derive specific questions. It was discussed that a potential benefit of Marte Meo could be based on the fact that central premises or principles of Marte Meo can be applied to the four impact factors according to Grawe [10]. Thus, in order to advance the application of Marte Meo in the future, scientific studies on this point seem promising.

There are still questions about how competently Marte Meo is performed in practice; that is, how much is supervision prescribed so that there is more reason to infer fidelity to the method than with most other interventions. If fidelity is not something that the Marte Meo process itself effectively ensures (especially when it is integrated into other methods that are part of the systemic approach), then implementation studies should be built into outcome/impact studies, so that one can distinguish whether objective results vary depending on the level of fidelity of different practitioners. Implementation studies utilizing the Consolidated Framework for Implementation Research (CFIR) might be highly worthwhile.

In our reflections, our decision to utilize an exploratory sequential design was a useful one. In particular, the enhancements of including the systematic literature research (phase 1) and two different qualitative methods (interviews (phase 2) and observations (phase 3) seemed to be a good choice—and might be a recommendation for other similar research projects.

3.2 Mixed methods research in our genograms project

The second project is one concerning genograms. Genograms are an integral part of therapy and counseling (Figure 5).

Figure 5.

Example genogram created with the InGeno app.

The study investigated the meaning and functions of genograms in the professional practice of counselors.

The article of Rohr et al. [12] presents partial results of the research project InGeno, in which—on the basis of the “actual” use of genograms to be researched—a user-friendly software for genogram creation is being developed. 108 counselors participated in the quantitative online questionnaire study. The data were analyzed descriptively and summarized together with the qualitative results of Rohr’s [13] preliminary study. The results show: Genograms are of central importance for those counselors who use them in their counseling practice: they are an integral part of their counseling work, are used in a variety of counseling situations together with the clients, and are further used and processed throughout the counseling process. In this context, genogram work fulfills a variety of functions, such as gaining information about the clients, recognizing transgenerational patterns and relationship dynamics, strengthening the identity of the clients, and uncovering their resources. The advantages and disadvantages of standardization and creativity are discussed.

Overall, the present study confirmed and extended the results of Rohr [13] to a large extent.

“Genograms are visualizations of the bio-psycho-social situation of the family and enable clients to recognize patterns of behavior—and thus to get to know themselves better” [13]. They consist of objective data (analogous to the family tree) and subjective meanings. “The task in genogram work in the context of counseling and therapy is to work out the structures of the social matrix (the social field).” Further, Hildenbrand [14], a pioneer in the use of genograms in German-speaking countries, writes: “Instead, I separate between the given and the given-up in human action [15] and find the actor in the distance between the two: The latter becomes an actor by making the given into the given up, i.e., by shaping his or her life”.

The article does not describe normatively and ideally how genograms ‘should’ be created. There is no single case study described, as this would not be helpful for our research questions. Our questions are discussed based on an elaborate research design. Based on the results we describe in this article descriptively: How do experienced counselors and therapists actually work with genograms, what advantages and disadvantages, limits, and possibilities (modes of action) do they find?

The questionnaire used here was created, among other things, based on the evaluation of twenty qualitative expert interviews. It is part of the interdisciplinary research project “InGeno”. A research team from the Department of counseling Research at the University of Cologne is developing a software (app) to create genograms with Computer Science Professor Dr. Mario Winter, B.Sc. Sven Kullack from the Cologne University of Applied Sciences (Figure 6) [13, 15].

Figure 6.

Exploratory sequential design of the Genogram Project.

Based on a systematic literature review [16] our project provides a comprehensive overview of genograms’ current research literature. In addition to a detailed account of developed extensions of the standard genogram for specific target groups and counseling settings, research findings on the utility of genograms in training and supervision and the need for discussing psychometric testing of genograms. The presented systematic literature review method aims to invite researchers to “underpin” their future counseling and therapy research with this approach. In this case, it will inspire counselors and therapists to test different extensions of genogram work depending on the target group and setting in practice.

The systematic literature search done on 05/09/2018 returned 348 hits. Thirty publications from other sources supplemented these. After removing duplicates, we checked the remaining 277 publications for thematic fit based on the titles or abstracts. Here, 112 publications were excluded as they did not fit in terms of content or were not available in English, German, or Spanish. The remaining 165 publications were checked for their suitability in the full text. In this step, a further sixty publications were excluded because they did not contribute to the genograms’ role in counseling practice.

The date of publication was not an exclusion criterion; also included were publications before 1990. A total of 105 publications were included in the systematic literature review and were analyzed concerning their empirical content.

Against the background of the quantitative results presented in this article, the systematic literature research [12], and the preceding qualitative interviews with experts the question of the significance and function of genograms in counseling practice can be answered as follows: For those counselors who use genograms in their counseling practice, they are of central importance and an integral part of the counseling process. They are very likely to be used in a large number of counseling situations, whereby most of the time this is done together with the clients. The genogram work is not a one-time activity, but the genogram is used and processed again and again during the counseling process. This leads to the conclusion that for many counselors the genogram fulfills the function of a common thread that runs through the counseling process and can be referred to again and again. The willingness to supplement central and well-known basic functions with further elements that make sense for individual counseling practice is great so that a great heterogeneity in the presentation of the genograms is to be expected. The goals of genogram work are to gain information about the clients, to recognize transgenerational patterns and relationship dynamics, to strengthen the clients’ identity—or their own appreciative understanding—and to uncover their resources. According to this, the genogram fulfills a variety of important functions in the counseling process—as well as within training and supervision: almost all respondents experienced the creation of their own genogram as very or mostly helpful.

The counseling and therapeutic work with genograms are very diverse. This is evident from both the systematic literature review (Phase 1) presented in the previous article [12] as well as from the results presented in this article. This is followed by open, fundamental questions: Are we even talking about the same “thing” when we use the term genogram—or genogram work? Bruno Hildenbrand [14] propagates to call “genogram work” only what aims at case reconstruction based on objective data. This could be called an extreme pole of answers to the question: “How are genograms used?”

For us, genograms are semi-standardized “visualizations of the family’s bio-psycho-social situation and enable clients* to recognize patterns of behavior—and thus to get to know themselves better” [13]; “semi-standardized” because it became clear in the study presented that the use of the symbols proposed by McGoldrick et al. [17] varied significantly in practice. Despite the advantages of standardization, e.g., the majority use of one and the same software, we believe that the focus should be on the common process of genogram use—and not on the genogram as a means of pure information retrieval.

Maybe the question of standardization or creativity is not relevant for individual counselors and therapists, but undoubtedly for the “scientific community”: If professional communication is to be done or if genograms are integrated into therapy applications, standardization is helpful for transparent communication. And if this communication will be digital in the near future (keyword “digital file”), a digitalization of genograms (beyond photos of paper-and-pencil drawings) will be necessary. And yet working with genograms is always an idiographic procedure, i.e., case-oriented, not standard, and developing or progressing from hour to hour. Especially “relationship lines” with double or jagged lines are not “set in stone” from a constructivist-systemic understanding, should not be the basis for “hard diagnostics” (said Tom Levold in the second phase of our project, the qualitative data collection). They are perspectives, circular, they may be experienced quite differently after a few weeks and serve only as hypotheses.

3.3 European qualifications framework (EQF) for lifelong learning—a current MMR project in Germany

In conclusion, this chapter presents an outlook on an elaborate research project which I have started on behalf of the DGfB, the German National Association for Counseling (member of EAC and IAC). Together with Marc Weinhardt (Universität Trier), Cornelia Maier-Gutheil (Evangelische Hochschule Darmstadt), Tim Stanik (Hochschule der Bundesagentur für Arbeit, Schwerin) and Marc Höcker (Universität zu Köln und Universität Mainz) I work on the development of a German qualifications framework for Counseling—in the context of the European Qualifications Framework (EQF).

It is a particular challenge due to the specific conditions in non-formal learning. It requires a well-considered, staged, and explorative approach because it is impossible to fall back on already established procedures for the allocation and reference competencies of the DQR (The German Qualifications Framework) from other fields. The DQR is a mixture of Mixed Methods Program Evaluation Design and Mixed Methods Participatory Design.

In a first step, a multi method approach triangulates a systematic literature review with a quantitative expert survey.

Here, the competencies and competence facets defined in the academic discourse and the competencies and competence facets depicted in counseling curricula are recorded to secure and further explore these in a Delphi study with one hundred experts’ participation. The experts were selected in close consultation with the client to adequately reflect the counseling landscape’s diversity and consider a cross-school perspective.

The Delphi study (100 experts will fill in the questionnaire) aims to check and supplement the literature review results and weighting concerning the Qualifications Framework for Counseling. It is to ensure that a methodically supported and intersubjectively comprehensible consensus is found within the member associations.

At the same time, we will analyze procedures and instruments for competence assessment, examinations, and certification procedures of the DGfB member associations with regard to their outcome and competence orientation in order to systematize their adaptation possibilities for the project context. Evaluating both partial studies’ results (qualification framework for counseling and synopsis of competence assessment procedures). In the course of qualitative group discussions with representatives of the DGfB member associations, proposals for certification criteria and practices will be derived and, in the course of the project, acceptance for the project results will be created in the DGfB member associations. The entire application’s orientation follows a research concept oriented towards impact factors. It is thus natively connectable to competence-oriented discourses and aims at theoretical and empirical modeling of successful counseling actions.

This project is a so-called complex design—an intersection of core designs with complex applications (see Figure 7).

Figure 7.

Process model embedding mixed methods into complex designs [6].

They are used when researchers have multiple research phases, multiyear research projects, large funded projects, multiple researchers, and inclusion of mixed methods core designs within different phases of research [6]. In this case, it is a mix of a “Mixed Methods Program Evaluation Design” (see Figure 8) and a “Mixed Methods Participatory Design” which is used when you want to involve stakeholders or participants in your design, when you want to bring about change and when you understand participatory approaches [6].

Figure 8.

Mixed methods program evaluation design [6].

4. Conclusion

In general, this article can be understood as a plea for mixed methods research. We agree with Teddlie and Tashakkori [18]: “We believe that divergent thought will always be a part of MMR (…), but that it is now time for greater convergence on some basic characteristics and principles” and with Symonds and Gorard [19]: “Death of mixed methods?: Or the rebirth of research as a craft”. Considering the limitations of the quantitative and qualitative paradigms and current definitions of mixed methods, we advocate the development of a research community where ‘all methods have a role, and a key place in the full research cycle from the generation of ideas to the rigorous testing of theories for amelioration’ and do not believe in “oppositional components of paradigms” [20].

This text is a “plea for mixed methods research in the field of counseling” by explaining three empirical examples. Timothy C. Guetterman did a meta-analysis together with my mentor Charles Deutsch from the Harvard School of Public Health and other colleagues [3]. Their goal was to understand how reviewers evaluate mixed methods research by analyzing reviewer comments for grant applications that were submitted primarily to the National Institutes of Health. They asked Mixed Methods Research Training Program (MMRTP) health sciences researchers and consultants to send them summary comments on their mixed methods grant applications and received 40 summary comments on funded (40%) and unfunded (60%) mixed methods grant applications [3]. They conducted a document analysis with a coding rubric based on NIH Best Practices for Mixed Methods Research in the Health Sciences and allowed inductive codes to emerge. Reviewers positively evaluated mixed methods applications that demonstrated coherence between goals and research design elements, detailed methods, plans for integrating mixed methods, and use of theoretical models. Reviewers identified weaknesses in mixed methods applications that lacked methodological detail or rationale, had a high participant load, and did not delineate investor roles. Successful mixed methods applications convey assumptions behind the methods chosen to achieve specific goals and clearly describe the procedures to be followed. Investigators planning to use mixed methods should remember that reviewers are looking for both points of view [3].

Mixed methods approaches are well suited to achieving the goals of health and implementation research. Nonetheless, applicants should be careful to explain the proposed methods based on underlying assumptions so that referees trained in the former methods from disciplines such as epidemiology and statistics will be able to understand the connection between the specific goals and the mixed methods. The reviewers pay attention to details about the samples, the plans for data collection and analysis, and the data integration procedures. Applicants should anticipate and dispel the concerns of the evaluators about possible disadvantages of mixed methods in terms of participants, time and resource expenditure, and generalizability of results [3]. The study of Guetterman et al. provides some empirical evidence for researchers keen to take full advantage of mixed methods to address pressing clinical and health care issues [3]. Therefore, it fits perfectly into this “plea for mixed methods research in the field of counseling”.