MOOCs and Problem-Based Learning: A Happy Marriage?

Abstract

Massive Open Online Courses (MOOCs) target a large number of participants, typically without entry requirements and for free. Inherently, the amount of teacher support is limited and interaction is often limited. This exploratory study aims to investigate whether characteristics of PBL can be implemented in a MOOC, thus applying a student-centred instructional design. The results of two runs of the MOOC Problem-Based Learning: Principles and design. Students at the centre! show that it is possible to design a MOOC based on student-centred learning theories. Participants discussed authentic problem cases in groups and followed a similar structure as in regular Problem-Based Learning (PBL): a brainstorm phase in which they collaboratively generated their own learning questions, a self-study phase in which they individually searched resources to study, and a reporting phase in which they collaboratively discussed what they had found. In the absence of tutor guidance and feedback, participants learned with and from each other. Participants were positive about the design, the learning materials, and the learning process in the PBL. Successful groups managed to collect insights and genuinely co-construct knowledge. However, the absence of a tutor makes means that this design is not comparable to traditional forms of PBL and is not successful for all MOOC participants.

1. Introduction

1.1 Research topic

Massive Open Online Courses (MOOCs) target a large number of participants, typically without entry requirements and for free [1]. MOOCs have been proposed as a way to make higher education accessible to a wide range of learners, independent of time, place, and financial resources of the participants. However, inherently, the amount of teacher support in MOOCs is limited and interaction is often limited [2]. At first sight, this seems at odds with current learning theories favoring a more student-centred approach [3]. In face-to-face higher education, this has led to the adoption of small group learning formats, such as Problem-Based learning (PBL) [4] or Team-Based Learning [5]. This raises questions about the quality of learning that is possible in MOOCs. To what extent can principles of student-centred learning be implemented in a MOOC?

1.2 Literature review

Historically, MOOCs emerged from the Open Educational Resources (OER) movement. The term MOOC was first used in 2008 by David Cormier in relation to the course ‘Connectivism and connective knowledge’ [6]. Although the first MOOCs specifically aimed at co-construction, the vast majority of MOOCs have had a more traditional teacher-directed design [7, 8]. The dropout in MOOCs is often high, typically more than 90% [9, 10]. Completion rates may not be a good way to assess learning in a MOOC, because not all participants may have entered the course with the intention to finish it [11, 12]. Regardless, MOOCs have been criticized for lack of sound instructional design [10]. Collaborative working formats are not used, participants work on individual assignments, and interaction is limited to discussion boards and peer review [2]. Participants complain about the lack of interaction with instructors and poor feedback from peers [9], whereas more interaction is a predictor of completion [13]. A recent literature study showed that also for teachers involved in MOOCs the incompatibility of expectations is one of the main challenges, i.e., they feel challenged to enact their teacher role well in the context of a MOOC [14].

From a learning theory point of view, the complaints and large dropouts are not surprising. Student-centred learning theories stress that learning is a constructive, contextual, collaborative, and self-directed process [3]. PBL is a well-known approach to student-centred learning [4], aiming at co-construction of knowledge by students centred around authentic problems. Typically, PBL groups meet face-to-face in the presence of a tutor and follow a stepwise approach, which includes a collective brainstorm or pre-discussion, followed by individual self-study, and a collective reporting phase or post discussion regarding their findings [15].

Online PBL has been implemented successfully [16]. Research has shown that synchronous online PBL group meetings with tutors can be similar to face-to-face PBL, when students and tutors are motivated and prepared, and adequate technical support is available [17, 18]. These conditions are not always met, and there are also examples of online PBL with less favorable results during the pandemic [19] and general concerns about online education potentially increasing systemic inequities: at home, some students will have better study conditions and more control over their own time than others [20]. PBL using asynchronous online tools, for example, using only asynchronous discussion boards, appears to be generally less successful [16], although it does seem that adding online interactions on a discussion board in between face-to-face meetings can be a valuable addition [21].

In PBL, the role of the tutor is important. The tutor guides the learning process, promotes in-depth discussions and facilitates collaborative knowledge construction [22]. Experiences with tutorless PBL have been mixed. One study found that experienced PBL learners could manage tutorless meetings using ICT support, such as mapping software and communication tools enabling to reach out to a tutor, when necessary. Students enjoyed the increased autonomy, experienced more team cohesion and rarely sought online tutor support [23]. In another study, three conditions were compared: traditional PBL meetings with students and the tutor in one room, synchronous online meetings with students and a tutor, and students meeting in a room with the tutor working as a consultant, giving comments asynchronously and digitally [18]. The results show that the presence of the tutor was influential, regardless of whether the meetings were onsite or online. Students reported less (self-perceived) learning and overall effectiveness and slightly lower motivation when the tutor only gave comments asynchronically. In this study, the researchers did not find a relation with students’ prior PBL experience. In a third study, researchers found no differences in exam scores between tutored and untutored groups, but students were more content with the learning experience in tutored groups and there was more variation in exam scores for the untutored groups. Furthermore, the authors stress that there were other forms of support available to all students, such as formative evaluation and feedback [24].

1.3 Motivation for this study

In summary, many MOOCs employ a traditional teacher-centred instructional design, which is in disagreement with current insights from learning theory, and this likely contributes to suboptimal learning and large drop-out. PBL is a student-centred approach that has been implemented successfully in many domains and contexts. PBL focuses on small-group learning with tutor guidance. There is evidence that online synchronous PBL can be used successfully. In MOOCs, however, there are no tutors and synchronous meetings are often difficult when participants live in different time zones and/or are working professionals. There is limited evidence that PBL does not always require the presence of a tutor, although most studies into tutorless PBL did offer on-demand advice from a tutor at a distance or other supports provided by teachers.

It is clear that PBL in its classical form cannot be implemented in a MOOC, since the massive character prevents designers from implementing much teacher support, but there is clear need to develop cooperative learning activities in online education [20]. Moreover, the authors of this study were driven by growth motives and interested in exploring innovations in online teaching and learning [1, 25, 26]. Therefore, this exploratory study aims to investigate whether characteristics of PBL can be implemented in a MOOC and, thus, contribute to a more student-centred instructional design, while maintaining the massive and open education character of MOOCs. We designed a MOOC about PBL (as content) that also followed the learning format of PBL as far as possible.

2. Design of a ‘PBL’ MOOC

The MOOC Problem-Based Learning: Principles and design. Students at the centre! was designed following PBL principles of learning as a constructive, contextual, collaborative, and self-directed process. Learning is contextual because it is centred around authentic problem cases of teachers, instructional designers and educational leaders trying to implement PBL in their own curriculum. Table 1 gives the text of the first problem. Later ones focused, for example, on the role of the tutor, problem and curriculum design, and assessment in PBL curricula.

“I traveled to Maastricht, the Netherlands, to see whether their „problem-based learning” approach in education, would be suitable for our educational program. I stayed a week at Maastricht University (...).; I learnt that students work in small groups guided by a staff member, called tutor. They discuss peculiar texts, which they call “problems”. Students are required to discuss these problems in small groups. Subsequently, they spend considerable time in the library or at home, searching for and studying appropriate learning resources. Thereafter, they meet again as a group and discuss the problem. Students have to do a lot on their own and they need to interact with each other a lot.; In short, PBL offers a very different learning environment with different building blocks: small groups, problems and tutors. This approach is used in this university across a variety of disciplines, including medicine, health sciences, psychology, business and science, with commonalities but also differences. They claim that it prepares students well for the demands of our current society.; I do not know, however, whether this approach is suitable for our own educational program. And I wonder how it works and why. I need to know more…”

Table 1.

Text of the first problem.

Constructive and collaborative learning are stimulated by making participants study these authentic problems in groups. Participants in the MOOC are asked to form groups in the first week. The first group assignment asks them to discuss task and role division and work strategies with the help of a team Charter [27]. Subsequently, the groups work on four authentic problem task. They are asked to hand in their work and peer review the work of others, but there is no formal assessment.

The instructions (Table 2) stimulate participants to follow PBL-style learning activities. They are instructed to start with a group brainstorming about the problem and to generate study questions for the group. The study questions are meant to guide individual self-study using resources provided in the MOOC (recorded short lectures, literature, examples, and links) and found elsewhere. In the discussion phase, participants are instructed to share and discuss what they have learned from their self-study. Self-regulated learning is stimulated by allowing participants freedom in making groups, as well as the freedom to choose their own task and team approach. The instructions in Table 2 force participants to collaborate and provide structure in the form of a ‘collaboration script’, while allowing freedom in, for example, role division and communication tools [25]. Arguably, the absence of a tutor to guide the group puts a larger demand on the participants’ self-regulated learning skills.

Brainstorm phase	1. Discuss the problem description given above in your team. Brainstorm about what you know already, and what you do not yet understand about [Problem name].; 2. Generate questions that need further study.; 3. When you have done this, ask one group member to submit a document with your group’s list of study questions using the ‘Get Started’ button below.
Self-study phase	4. Study the materials individually. Watch the web lecture and other videos. Consult the resources below. There is a long list of open and licensed resources in the bibliography.; 5. Look for other relevant materials in your own collection or on the internet, e.g., searching with [suggested key words]. Try to find resources about PBL that fit your disciplinary background.; 6. Share your resources and experiences with your team (in the team space) and with all participants (on Discussions [Title discussion board])
Discussion phase	1. Report to each other which resources you have used, and what you have learned from them.; 2. Discuss which of your study questions you can answer and which (old or new) questions cannot be fully answered.; Optional: Share your experiences with other groups (on Discussions - [Title discussion board]); 3. Make a brief summary of the group discussion and the lessons learned. Be creative with the format of your summary. It can be a document, but it can also be a concept map, a word cloud, etc.; 4. Ask one group member to upload your summary, using the ‘Get Started’ button below.

Table 2.

Instructions given to groups with each problem case.

The MOOC was implemented in NovoEd (https://novoed.com), a platform that explicitly supports small group work. Each group was given a private group space with chat facilities, file exchange, and facilities to schedule meetings. A more detailed description of the design of the PBL MOOC and how it relates to PBL principles can be found in [28].

3. Research methods

3.1 Study design

This study can be characterized as design-based research aimed at understanding the complex world of educational practice [29, 30]. In this study, quantitative and qualitative data are used to evaluate a MOOC designed in accordance with PBL principles. The results contribute to an improved design for the MOOC, as well as the theory and practice of PBL and online collaborative learning.

3.2 Setting

The MOOC Problem-Based Learning: Principles and design. Students at the centre! was piloted in the spring of 2015 and subsequently executed twice in the autumn of 2015 and 2017. The MOOC was free and there were no entrance requirements. The target audience was described as ‘teachers, course designers and managers in education in all domains interested in Problem-Based Learning’. The two runs used a similar course design with small variations. The workload for participants was estimated at 4–8 hours a week. There was no formal assessment, but participants who completed the course were offered a ‘Certificate of Participation’.

3.3 Participants

Participants subscribed themselves to the course. Those who filled in the profile page and joined a group were considered active participants. A larger group of participants joined and presumably looked at study resources, but never participated in the PBL group activities (see Table 3). Data from the profile pages show that there is a more or less equal gender division, a majority of participants over 30 years old, and most teachers involved in education professionally. Their background in PBL varied: most participants had no or some experience with PBL; a small minority had extensive PBL experience. About two-thirds of the participants that filled in their profiles had never taken part in a MOOC before. In both runs, participants came from all over the world (see Figure 1).

	Participants	Active participants	Groups
Run 1	2799	794	109
Run 2	849	301	37

Table 3.

Participants in the two runs.

Figure 1.

Participants voluntarily pinned themselves on a Google map (left run 1; right run 2).

3.4 Data collection and analysis

For this study, the following data were used:

• Learner profiles and participation data logged by the platform;

• Evaluation surveys filled in by participants;

• Assignments handed in by participants;

• Verbal interactions on the discussion boards on the MOOC platform;

• Group interviews with facilitators half-way and at the end of the MOOC; and

• Communication and collaboration in the group space of 21 teams (only Run 1).

For the log data, descriptive statistics were generated by the platform. For the questionnaire data, descriptive statistics were computed using SPSS version 24. Discussion fora and assignments were inspected, but not systematically analyzed. Some quotes and assignments will be reported as illustrations. Group interviews with facilitators were summarized and sent back to the participants for member check. During the first run, the interaction in 21 groups was studied in detail to uncover how virtual teams collaborate in online collaborative learning tasks. Detailed results of this study are reported elsewhere [31]. Here, we will only refer to conclusions that relate to how these groups followed the typical phases of PBL (see Table 2).

3.5 Procedure

The MOOC was advertised through the platform website, communication channels of Maastricht University (UM) and the School of Health Professions Education (SHE), existing mailing lists (e.g., for alumni), and at some conferences (with flyers and presentations). In the first week, participants formed groups and they filled in and discuss a team charter. The course platform provided a shared workplace, but the groups had full freedom in deciding how to work together and which communication tools to use. They worked on four consecutive PBL problems. After handing in their results, they were asked to peer review the results handed in by other groups. The course facilitators kept a general overview, answered general and technical questions, and provided general comments or tips in announcements and during short Google Hangouts sessions.

Participants who completed the course received a Certificate of Participation, which could be downloaded from the platform. Questionnaires were sent mid-term and at the end of the MOOC through the MOOC platform (automatically forwarded to the participants’ e-mail addresses. All assignments and the content of the discussion forum were downloaded after the end of the MOOC.

Participation in the MOOC was free and participants could withdraw at any time without explanation. Participants subscribing to the MOOC were asked for permission to use their anonymized data for research purposes upon entering the platform. Participants whose assignments are shown in publications were asked for permission explicitly by e-mail when anonymity could not be guaranteed.

4. Results

4.1 Completion rates

Table 4 shows the completion rates for both runs. Completion rates for active participants who had filled in their profile page and joined a group were considerably higher (28.4% in run 1; 34.2% in run 2).

	Participants	Active participants	Finished the course	Completion rate
Run 1	2799	794	264	9.4%
Run 2	849	v301	103	12.1%

Table 4.

Completion rates.

Group completion rates were 44% in Run 1 and 51.4% in Run 2, but all groups lost members along the way. Data from questionnaires and profile pages show that groups often shared a common interest (e.g., worked in a certain domain, e.g., an international group of teachers in English, or a group of medical educators) or were based in the same region.

4.2 Student satisfaction

Table 5 provides data from the questionnaire filled out by participants at the end of the course. The responses show that active participants were very positive about the PBL MOOC, and felt that they learned a lot. Team functioning and the short sessions organized by the facilitators scored slightly lower. Comments on the discussion boards confirm that active participants valued the PBL MOOC.

Item	Run 1; Mean on; Scale 1–5 (SD)	Run 2; Mean on; Scale 1–5 (SD)
Overall, I am satisfied with the way this MOOC was organized	4.1 (0.8)	4.3 (0.8)
I have learned a lot during this MOOC	4.3 (0.8)	4.3 (0.7)
The content of this MOOC linked well with the level of my prior knowledge	4.2 (0.8)	4.1 (0.9)
This MOOC encouraged me to formulate my own specific learning objectives that required further study	4.3 (0.8)	4.3 (0.8)
This MOOC encouraged me to study the learning resources that were offered	4.1 (0.8)	4.4 (0.8)
I see how I can apply what I learned in this MOOC to ‘real life’ (in practice)	4.1 (0.8)	4.4 (0.8)
The activities I performed in this MOOC are relevant to my professional goals or field of practice	4.3 (0.8)	4.4 (0.7)
This MOOC encouraged me to collaborate with my fellow students	4.2 (0.9)	4.1 (1.0)
The collaboration with other students in this MOOC has been productive for my learning	4.2 (0.9)	4.1 (1.0)
This team functioned well	3.9 (1.0)	3.9 (1.1)
In this team, I shared all relevant information and ideas I had	4.4 (0.8)	4.2 (0.9)
If something was unclear, we asked each other questions	4.3 (0.9)	4.1 (1.0)
Team members elaborated on each other’s information and ideas	4.1 (1.0)	4.0 (1.0)
The sessions (e.g., Google Hangouts) organized by the instructors were useful	3.7 (1.1)	3.8 (1.0)
The platform of this MOOC (NovoEd) worked well	4.0 (1.0)	4.2 (1.0)
Using this online learning system supported me in my learning	4.1 (0.9)	4.2 (1.0)
I felt comfortable conversing through the online medium	4.0 (1.0)	4.0 (1.0)
I felt comfortable interacting with other course participants	4.2 (0.9)	4.1 (0.9)

Table 5.

Questionnaire results at the end of the course (n = 169 for Run 1; n = 104 for Run2).

4.3 The PBL process in group learning

Contributions to the discussion boards and inspection of the assignments show that participating groups followed a PBL-like process in three phases: brainstorming, formulating learning questions, reporting and discussing results. On the discussion boards, it was clear that some participants saw the PBL-based group work as a large benefit:

“All the best, and the MOOC has been an outstanding success for me, I have to say. I have always been a sceptic about MOOCs, and have never really committed to one before, but the attraction of working in small groups in a structured way (i.e. PBL!) meant that I had to change my status late in October, from ‘audit’ to ‘participant’ […]it has made for a memorable experience. We have enough active members still, even at this late stage now, which is a very good sign I think.”

Others, however, struggled to find a good group: “I am having a problem with my group. I was part of Group 7, but all of them left the course.” And: “I have [been] unable to join any team in the last three days, hence unable to make any progress.”

Inspection of assignments by the facilitators showed a large variety in quality. Some were very superficial or even almost empty. Others were more elaborate but mostly a list of individual contributions (identified by the name of the contributor) without an in-depth discussion of results. Some groups, however, clearly constructed new knowledge, synthesizing their main findings in their own words and/or applying it to their own context. The problem designers commented that the quality of the best assignments went far beyond their expectations. Groups submitted assignments in different formats, mostly in text and/or concept maps (see Figure 2), sometimes using more creative formats (see Figure 3).

Figure 2.

Example of group product summarizing what the group had learned in a concept map (screenshot used with permission).

Figure 3.

Examples of creative group products for the last assignment: A handbook summarizing lessons learned as ‘The future of PBL’, comparing PBL aspects to artworks, and an animation (screenshots used with permission).

Detailed analysis of the interaction in the group spaces of 21 groups in Run 1 confirmed this but also showed that groups collaborated in very different ways [31]. Most groups opted for asynchronous interaction and this may have limited the amount of discussion. A minority of groups engaged in a deeper level of discussion where input was combined and synthesized, with new insights developed collaboratively. We saw the importance of having team members explicitly discuss team processes and task division, and set clear expectations and timelines. Successful teams showed team adaptability in a positive atmosphere and were able to plan reactively, which helped them to deal with unexpected events.

5. Discussion

The results of two runs of the MOOC Problem-Based Learning: Principles and design. Students at the centre! showed that it is possible to design a MOOC based on student-centred learning theories. Participants discussed authentic problem cases in groups and followed a similar structure as in regular PBL: a brainstorm phase in which they collaboratively generated their own learning questions, a self-study phase in which they individually searched resources to study, and a reporting phase in which they collaboratively discussed what they had found. In the absence of tutor guidance and feedback, participants learned with and from each other.

A part of the assignments shows truly creative and innovative thinking and co-creation of knowledge. However, the envisioned kind of learning was not realized in all groups, and the absence of a tutor is likely to have been an important factor in this. Tutors are an important part of PBL [22, 32]. Another reason might be that many participants did not have much experience in MOOC participation and may have had limited digital literacy. Co-creation requires intensive brainstorming and discussion, which is not well-supported by basic asynchronous tools such as chat and file exchange that were provided by the platform.

The evaluation results show that participants who completed the MOOC were very satisfied with the design, the learning materials, and the learning process in the PBL MOOC. Individual completion rates were comparable to other MOOCs or slightly better: 9.4% in Run1; 12.1% in Run 2. cf. [10]. There was a large dropout, which may have been caused by participants discovering that they do not have enough time or that the course is not what they expected. Participants may have subscribed just to look around, without the intention of completing the course [11, 12]. Higher initial commitment and motivation, especially intrinsic motivation, might be related to performance, perceived quality, and satisfaction [33, 34], but working professionals may be more driven by their current learning needs and knowledge gaps than by completion or certification [35].

Completion rates of active participants were much higher (28.4% in Run 1; 34.2% in Run 2), possibly because those who subscribed to groups were more motivated from the start, but it is also possible that they felt a commitment to their group to continue. There are indications that some participants could not find a suitable, active group to work with. They may lack self-regulated learning skills that are important in MOOC settings [36] and/or need more support to develop skills needed for virtual group work [31]. One option would be to stimulate and support regrouping more actively. It is not clear, however, whether participants would like to regroup halfway through.

Groups form around common interests, and participants taking action to get into a group are likely to be more motivated. Once in a group commitment may have been enforced by social cohesion and social equalization, which is also typical of online group work, may have helped to share information on PBL in culturally diverse groups cf. [37]. On the other hand, some of the dropouts and poor group performance may be explained by social comparison. Exposure to exemplary peer performances might cause participants to quit a MOOC when they perceive that they cannot attain their peers’ high levels of performance. Insufficient teamwork skills and digital literacy skills [31, 37, 38] may add to this. MOOCs are bringing together thousands of learners from diverse backgrounds across the globe. Research on inclusive learning cultures in MOOCs has only recently gained attention [39, 40].

6. Strengths and limitations

The MOOC Problem-Based Learning: Principles and design. Students at the centre! is designed for a specific target group of people with a professional or personal interest in education in general, and PBL in particular. This group may have been more motivated to follow a PBL approach in their group work, so more research is needed to explore whether similar student-centred MOOC designs would be possible for other target audiences in other domains.

Another limitation of this study is that our data do not give much insight into the quality of the interactions within groups and the reasons for the more in-depth discussions in some groups vs. the superficial discussions in other groups. Synchronous PBL group discussions or specific tools to support brainstorming or knowledge synthesis could be tried to improve the PBL group discussions [31]. Another option would be to stimulate more explicit discussion of the group process [21, 25]. Future research could explore if and how different styles of leadership and role division might enable group participants to take over (part of) the role of a tutor.

7. Conclusion

In conclusion, this study shows that innovative student-centred instructional designs for MOOCs are feasible. The design of the MOOC ‘Problem-Based Learning: Principles and design. Students at the centre!’ applies PBL learning principles of stimulating constructive, contextual, collaborative, and self-directed learning. Successful groups managed to collect insights from active group members, despite dwindling numbers of active participants and a large number of participants who were new to PBL, MOOCs, and virtual group work. However, the absence of a tutor makes means that this design is not comparable to traditional forms of PBL and is not successful for all MOOC participants.

Acknowledgments

The authors would like to thank the other facilitators in the two MOOC runs (Maryam Asoodar, Geraldine Clarebout, Amber Dailey, Jimmy Frèrejean, and Lianne Loosveld) as well as all other contributors to the MOOC projects and the UM EdLab for supporting the MOOC project: Paul Adriaans, Denis Ancion, Ellen Bastiaens, Lex Borghans, Anique de Bruin, Thomas Cleij, Wilfred van Dellen, Diana Dolmans, Jeroen Donkers, Odin Essers, Corrie Eurlings, Bart Golsteyn, Willem de Grave, Angelique van den Heuvel, Harm Hospers, Nynke de Jong, Chris Keurentjes, Menno Knetsch, Geert Konijnendijk, Mariëtte van Loon, Heidi Maurer, Lars Mennen, Gaby Odekerken, Annabel Reker, Catherine de Rijdt, Caroline Roulaux, Susan Stead, and Margje van der Wiel.

Conflict of interest

The authors declare no conflict of interest.