MOOC as a Way of Dissemination, Training and Learning of Telecommunication Engineering

Abstract

In this chapter, the use of massive open online courses (MOOCs) for the dissemination, training capabilities and learning of telecommunication engineering is described taking as example the successful MOOC ‘Ultra- Dense Networks for 5G and its Evolution’ developed under the European innovative training network (ITN) TeamUp5G. MOOCs are usually understood as a way of teaching or learning for massive potential students. Indeed, this is the main goal of any MOOC. However, we also propose its use for training and dissemination. The ITN TeamUp5G is a training network for 15 PhD students of seven different institutions (universities and companies) where the students make research on different interconnected topics for the common goal of Ultra dense networks for 5G. At the same time they researched, they prepared a MOOC to disseminate their most recent advances and their challenges. For the MOOC, they needed to collect their thoughts, organizse their knowledge and establish a common vision of the whole system. The cooperative work, the cross-related meetings and, the preparation of all the materials for the MOOC were very interesting and useful in their training process. The whole experience of designing and creating the MOOC is described in detail along with the challenges and lessons learned.

1. Introduction

Massive open online courses (MOOCs) are widely seen by students as the perfect option for acquiring new skills, learning new concepts, developing their career and including new knowledge in their curricula in an easy and practical way. In fact, this is the common view for most of the people. There are MOOCs in almost every discipline [1]. In this book chapter, we are going to focus on a MOOC devoted to engineering, although some of the main concepts, applications and lesson learned could be straightforwardly applied to other disciplines.

Sometimes students access MOOCs because they want to learn specific contents or skills. Some others, they do for other reasons such as the completion of some knowledge they already own, but they are willing to improve or reinforce, curiosity on some specific topics or only for pleasure or culture. In all these cases, the MOOC is used by external participants and must be designed for that. Videos, questionnaires, activities must be carefully planned for obtaining three main goals, namely the transmission of knowledge (educational), variety of activities to guarantee the learning process and attractive productions for the engagement of students. It is worth saying that MOOCs are most of the times followed voluntarily by students, so the engagement is very important. Any wrong decision on video or activities can seriously affect the participation of the people and thus void the learning process and effort [2].

Several universities and institutions have created and shared their own experience on virtual and remote content creation through MOOCs development over the years. Authors from [3] submitted experimental findings from the Virtual Instrument Systems in Reality laboratory. The authors of [4] compared the results of several courses on signal processing and digital communication they had created over the years. In [5], a study about MOOCs’ effectiveness in improving undergraduate students’ performance in a normal Digital Signal Processing (DSP) class was conducted. There is also a discussion in [6] on the advantages and disadvantages of MOOC courses for microelectronics. Recently, in [7], an exhaustive analysis of MOOCs on the higher education in Saudi Arabia is presented showing an important impact on the University system and the outcomes. A more recent research has shown that they are eight factors that affect the outcomes and effectiveness of a MOOC directly, namely behaviour intention, learning engagement, students’ motivation, perceptions, satisfaction, performance, self-regulation and social networks across the world [8].

Along with the MOOCs, some universities also provide small private online courses (SPOCs), associated to specific classes. Those SPOCs complement the in-person courses or subjects with videos, activities and interactive materials that can only be accessed by the students on these courses. They can also be seen as blended courses where some contents are shown in an SPOC and others in class [9] and proof of excellent outcomes exists [10, 11]. This is also used in companies for training. It is important to highlight here that an SPOC is different from a self-paced open online course (SPOOC), which is similar to a MOOC although the itinerary, times and schedule are adapted to the student’s needs. It is very common for a MOOC to be converted into an SPOOC after one or two editions: all the materials and activities are there from the beginning and the students decide when access to them. As a drawback, SPOOCs do not have the possibility of supervision and guidance by the professors and the encouragement and dynamism of the course are limited. The forum is still open, but it is static, and it is not usually supervised by a moderator.

From the point of view of MOOCs devoted to telecommunication engineering, there are more than 100 official MOOCs/SPOOCs closely related to its topics such as digital signal processing, information theory, time and frequency analysis, internet of things (IoT), wireless communications, modulations, signal and systems, programming and many others. Those MOOCs are mainly hosted by relevant universities such as EPFL, Curtin University and Universidad Carlos III de Madrid (UC3M) or technological research centres such as MIT, Georgia Tech, Tokyo Tech, among others. The number of discussions in the literature is large, but to the best of our knowledge, there was no MOOC focusing on the fifth generation of mobile communications (5G) and its advancement, which led to the creation of the MOOC addressed in this book chapter.

Although the ultimate goal of a MOOC is the learning contribution for the students, a recent study on the literature showed that they are also used for research purposes on how to improve traditional education and learning (see references in [12] and references therein).

On the other hand, MOOCs can also be used as a tool for training PhD students in the following sense. Nowadays, the world is changing rapidly and so the learning methods. New doctors in engineering need to acquire not only research capabilities and a deep knowledge on the state of the art in their fields (in this case, ultra-dense networks for 5G), but also skills and abilities for the new technological, visual and online era. According to the last European policies in open research under the Horizon 2020 and Horizon Europe [13], research must be oriented to citizens, open, close to real problems and fair in order to advance in the different fields and to improve citizens’ life and society. Thus, researchers must accommodate this new scenario and way of researching. In this sense, the dissemination of results, the collaboration and the way of making visible their work for the society move to a more visual, open and online. In this new scenario, MOOCs appear as a way of dissemination and training. However, the conception of a MOOC, its design and production are not an easy task, so new PhD students need to learn how to do it properly. For these reasons, a MOOC can be seen as another training for the PhD and a new way of dissemination according to the new rules. In this book chapter, our experience on designing and creating a MOOC is shared.

The project ‘New RAN TEchniques for 5G UltrA-dense Mobile networks’ (TeamUp5G) [14] is a prestigious Marie Skłodowska-Curie action Innovative Training Networks (MSCA ITN) in the frame of the European Commission’s Horizon 2020 framework [15], with grant-agreement number 813391. The team is investigating the evolution of the 5G wireless communications and has been preparing an extensive MOOC under the scope of ‘Ultra-Dense Networks for 5G and Its Evolution’. The goal is sharing the recent research advances and the knowledge about the main technological innovations and new 5G mobile networks applications.

From the best knowledge of the authors, there is no other experience like this in training PhD students. Motivated by MOOCs’ role in the scope of higher education while providing a positive impact on student’s performance, a well-designed, structured and open comprehensive accessible online course has been prepared by the TeamUp5G team. As an outcome of this join effort, this book chapter provides the detailed steps and procedures about the methodology adopted and experienced during the preparation of the MOOC, highlighting the acquired experience, challenges, potential opportunities, training capabilities and main advantages.

2. MOOC structure and goals

The rapid technological development over the past few decades has led to a continuous flow of new discoveries and innovations in science and technology. Meanwhile, new patents and scientific advancements are constantly being made, bringing new challenges for integrating them into our everyday lives. It is important that qualified professionals are able to learn and use new technologies effectively in order to promote social progress. After taking a closer look at the scope of the necessary road map for the target technology and its evolution affecting the telecommunications industry, we have identified gaps that can be filled by our MOOC. Our team has noticed a gap between the latest innovative evolutionary technologies and the way current students see the future of networks, especially 5G and beyond technologies. Bridging this gap is beneficial for students, professionals and researchers as they can keep their knowledge up to date and understand the latest technologies in communications and networking. Source materials providing information about the topics of our MOOC can be grouped into two classes: general dissemination documents and research publications. General dissemination documents (e.g. materials provided at university undergraduate and postgraduate levels and online resources from the internet) usually do not summarise the target technologies in a well-developed plan. At the same time, research publications commonly focus on a specific topic, requiring substantial background knowledge. Therefore, there is a concrete possibility that university students and young researchers would build their expertise on 5G and beyond technologies by sourcing information from both classes following a disorganised approach. It is easy to predict where similar behaviour could lead: a chaotic knowledge of the subjects and/or a general feeling of dissatisfaction.

2.1 Objectives

The gaps mentioned in the previous section are observable enough to require particular care to address them. Since the key feature of our MOOC was the simplicity of explanation, the structured strategy depicted in Figure 1 was employed. The aim is to transfer this knowledge by covering both the technical principles and fundamentals of each enabling technology, as well as a higher-level view of 5G and beyond mobile networks. It is critical to take into account the need for an innovative strategy for such knowledge exchange to achieve successful results. Thus, the present MOOC package effectively covers both the high-level vision and the technical perspective of the ‘Ultra-dense Networks for 5G and its Evolution’.

Figure 1.

Showcasing of the objectives (created from free open licence CC).

The MOOC covers a wide range of topics, mainly related to 5G and its evolution, its system requirements and new transmission technologies. It addresses aspects of interference management and energy efficiency, low power networks, packet and multi-band scheduling, data sensing, spectrum sharing, carrier aggregation, full-duplex communication, beamforming, use cases and prototypes (such as augmented and virtual reality), security/privacy, unmanned aerial vehicles (UAVs) and simultaneous radar and communications (RADCOM), complemented by a vision of the future ahead (e.g. 6G and terahertz communications).

Overall, through our produced MOOC, we target the transfer of knowledge in a comprehensive and transparent technical language (dissemination). After developing a good understanding of the vision for the audience, we progressively share the technical perspective of each key technology. Such process would enable the audience to get enough readiness to understand the latest technological developments. Eventually, it will broaden their horizons about the topics discussed which may lead to enhancements of their knowledge and future contributions to both academia and industry.

2.2 Targeted audience

We designed the MOOC to be simple, understandable and intelligible, targeting an audience with diverse background knowledge, requirements and obligations. In this sense, the target audience is made of professionals and students who are related or simply interested in the research and development of 5G New Radio networks and their evolution. The core concepts are consistently expressed for beginners and students, with the aim of making them easier to understand while building strong foundations. On the other hand, individuals with experience in the telecommunications industry may get familiar with the most recent objectives and cutting-edge research fields that the European Union and related corporations are willing to fund, investigate and develop. Finally, this MOOC can be useful for educators who wish to teach their students the principles and basic ideas of 5G networks.

2.3 Content formatting

The ‘Ultra-dense Networks For 5G And Its Evolution MOOC’ [16] was prepared by an international team made of 14 early-stage researchers (ESRs) under the supervision of highly qualified professors from different backgrounds and disciplines. The course is divided into six modules, each of them containing five different items that cover a wide range of concepts and technologies related to 5G networks and further additional wireless technologies. Learning, assessment and motivational materials have been developed for each item, as shown in Figure 2. In particular, the main learning materials consist of video-recorded presentations and textual extensions. The video-recorded presentations were entirely filmed at the UC3M facilities due to the COVID-19 pandemic and the differences in terms of equipment available to the various partners. This could have caused an audio-visual mismatch which could have impaired the overall experience. By keeping both the environment and the recording equipment constant throughout the MOOC, we were able to achieve consistent results. Based on the input given by the experienced staff of the UC3M, the researchers prepared detailed scripts to be shown on a teleprompter during the recordings, as an aid for the presenters. The scripts also included notes on engagement and motivation, to keep the audience in a state of higher mental concentration. A total of about 2 hours of material was produced for each module, split among 10 minutes of video content, 50 minutes of written information to support the video material and 1 hour of questions and forum discussion.

Figure 2.

Block diagram of the content formatting to summarise the contents.

In addition to supporting the presenters, the scripts were also used to produce text transcriptions on the edX platform. Furthermore, a separate auxiliary textual extension of them was prepared. The goal of the textual extensions was to provide the participants with additional information that could not be included in the videos due to time restrictions. Moreover, they can also be used as in-class materials in the event that the MOOC is used as an aid in higher education institutions.

Finally, in order to verify the knowledge gained by the participants, an assessment process was implemented. In this context, the participants have to answer true/false, multiple-choice, drag and drop and input-type questions. Moreover, after the finalisation of each module, the students are encouraged to further participate in a discussion forum. In this forum, both students and teachers can interact for learning engagement purposes. Such a round-table allows the participants to exchange ideas and concerns about the current and possible 5G features, with the seniors passing on knowledge to the younger ones. Apart from the pure learning purposes, the discussion forum activities may concretely lead to envision novel 5G features and use cases.

All these activities can also be seen as a training process for the ESR: looking for information, organising knowledge, summarising, presenting in an easy and visual way to the audience and preparing extra material for further research.

3. Resources

3.1 Team and project

TeamUp5G is a multi-partner research training network whose beneficiaries come from academic and non-academic sectors to form a structured, international, intersectoral and interdisciplinary research and training environment. The PhD students and young researchers are spread in different European countries. It aims to optimise the existing 5G in various domains in terms of throughput, energy and spectral efficiency. Some challenges are the demand for increasing data rates and users served per km² and the energy efficiency of the entire system. The goal of the educational training network (ETN) is to propose metrics and develop energy-aware algorithms and protocols to enhance small cells in ultra-dense deployments, making use of massive antenna solutions (mMIMO), millimetre wave (mmWave) bands and Visible Light Communications (VLC), in relevant scenarios, through a combination of analytical work, simulation and prototyping. The details and information regarding our ESRs, their works and the hosting institutions can be found in [14].

3.2. University facilities and prior experience

The technical team of UC3M and some of the involved supervisors had prior experience creating and organising MOOCs [17]. UC3M provided around 35 different MOOCs both in English and Spanish in the edX and MiriadaX platforms. For instance, the course on mobile communications from the Signal Theory and Communications department at UC3M is published in edX [18]. This MOOC is open to the public and targets an audience who have no previous knowledge on mobile communications. UC3M experience guided the journey of this MOOC and helped the team to overcome the challenges.

As stated before, the MOOC was fully recorded at UC3M, utilising the in-campus Audio/Video (AV) facilities. UC3M has three recording studios, to allow university staff and students to generate teaching materials for various purposes, such as MOOCs or teaching innovation projects. The rooms are provided with all the recording facilities such as HD cameras, a system for mixing and compositing images in HD, special background lighting for generating a virtual background and a teleprompter, as shown in Figure 3. Concerning the prior experiences in MOOC production, UC3M has experienced staff for editing, mixing and processing videos. UC3M also provides support for the creative process such as covers, course images and original creation of materials, such as animations or even small interactive materials.

Figure 3.

Recording room facilities available at UC3M Leganés campus.

3.3 ETN contributions and resources

The MOOC ‘Ultra-Dense Networks for 5G and its Evolution’ results from a great teamwork, supervision and constant guidance. In its production, 14 ESRs and nine supervisors have participated. From the 14 ESRs, two acted as both producers and supervisors of the MOOC, as it happened with three of the supervisors of the TeamUp5G project. The other 12 ESRs and the other six supervisors acted only as producers and as supervisors of the MOOC, respectively. Each producer was responsible for the content of the MOOC relevant to one’s research area. The contributions by the supervisors were invaluable in coordinating the teams, reviewing and providing continuous insights on improving the content. In Section 4, the MOOC’s structure and contents are discussed in detail. To ensure high-quality videos and synchronisation, UC3M took the responsibility of recording and coordinating the MOOC. Some producers could not travel to the UC3M premises amid the COVID-19 pandemic. For this reason, some of the producers residing in Madrid recorded most of the videos. However, leveraging on the research stays at institutions in Madrid and project meetings, several ESRs could record their videos in their visits.

4. Production of the MOOC

In order to give a picture on the timing for the production of the MOOC, this section includes information about the timeline of the main tasks (with details of the months and duration of each task), the creation of the material (with details of the involved actors and processes followed for its creation), the copyright compliance (performed by the MOOC authors and the supporting audiovisual team at UC3M), the contribution from the non-academic sector and the quality assurance (performed by the MOOC authors and the supporting audiovisual team at UC3M). Figure 4 shows an overview of the timeline, involved tasks, copyright and quality processes of the production of the MOOC.

Figure 4.

Overview of the production timeline, involved tasks and copyright and quality steps of the MOOC production.

4.1 Main tasks and timeline

The kick-off meeting was in early March 2021, when the MOOC structure was defined. The two major goals were to begin the video production phase in late July 2021 and to finish the entire MOOC in January 2022, in order to begin the lessons at the end of February 2022. Six different modules were identified, each one divided into five items, spanning from introductory topics to more advanced technical ones. To structure the overall work, a table of contents for each item was proposed in April 2021. Based on this defined structure, the production of the presentations and scripts of all modules was carried out during May and June 2021. A common template was used to maintain a homogeneous look and feel throughout the entire MOOC, always taking into account the requirement that the MOOC was going to be provided utilising the edX platform. Therefore, bearing this in mind, we focused on having as less text as possible in the videos, in order to keep an adequate level of attention from the viewers. Also, a great number of illustrations (both images and schematics) were used to take advantage of visual learning, since many of the videos were expected to be visualised utilising small screens such as those present in smartphones and tablets. In the end, this phase has proven to be the most challenging one, both in terms of research and time. Since the maximum duration for each video was set to 7 minutes, the use of written scripts became essential to ensure that we would comply with this limitation. This fact also forced the authors to be as efficient as possible in the content formatting to try to add as much clear information as possible in the little time that was available.

The videos were recorded during June, July and September 2021, supported by the previously created presentations and scripts. Among the parties involved, only the UC3M had the required facilities for multimedia production (i.e. filming and video production) and the best way to have a centralised quality control was to record all the videos at the UC3M premises. This was done using a small subset of the authors that were in charge of the recording process. This subset was made of both instructors and speakers. The filming process took about 3 months in total.

Apart from the video, a textual extension as extra studying material was provided, in which more information than that available in the videos was offered as optional. This allowed the authors to better explain some concepts that might not be clear only from the videos. The starting points for the textual extensions were the previously written scripts utilised for the video recording process. As stated before, some particularly complex topics were further extended to provide a more complete picture. In order to provide an homogeneous result, a common template was used for all the textual extensions. The textual extensions were created in October and November 2021.

The evaluation questions were created in parallel to the creation of the textual extensions, during the months of October and November 2021. Two different evaluation phases were defined: a test related to each item and a more general test for the entire module. The item-wise test contained six questions and a starting point topic (including references) to be used for general discussion purposes. The module-wise test featured 10 questions about any item included in the module. Both the item and module tests featured different test modalities, such as true/false, multiple-choice, drag and drop and numerical response, to avoid them becoming too uniform and tedious.

All these three previously described steps (video recording, textual extensions and evaluation) were supported by an active involvement of the UC3M audiovisual team. This interaction ensured that a high-quality product was developed, always taking into account the particularities of the edX platform, as we will further detail in a following subsection. This quality assurance process was performed in December 2021 and served to correct any issues and to ensure a proper quality of the contents and resources produced.

Besides, a forum discussion was proposed in each item to motivate the active participation of all the students of the MOOC, which is something that the edX platform regards as useful. The audiovisual team at UC3M took care of the upload process of the contents, adapting them to the specific format required by edX. Every time that the content of a full week was uploaded to the platform, the beta testing process was carried out for each of the content in that week. This beta testing was the last step, and it was performed in January and February 2022. Figure 4 shows the general timeline of the MOOC, where one can see the steps of copyright check and quality check in different parts of the production, where there was an active involvement of the authors and the UC3M audiovisual team.

4.2 Organisation and creation of the study material

As previously stated, the content of the MOOC was divided into six modules, each with five items:

• Module 1 – ‘Ultra-dense networks and small cells’ introduces to the audience the concept of ultra-dense network, the 5G paradigm, the new scenarios foreseen as well as innovative applications. Besides, it introduces the emerging technologies for 5G that are key to its development and will be explained in the course.

• Module 2 – ‘New transmission technologies’ focuses on the physical layer transmission technologies such as massive MIMO, beam forming and full-duplex technologies, as well as VLC. These technologies are key to physically support the transmission and reception of signals to ensure the required 5G performance.

• Module 3 – ‘ Interference management and energy efficiency’ presents scheduling mechanisms for the use of the wireless resources, the cell-free paradigm, which has implications in both physical and MAC layers, and approaches for energy efficiency, which is becoming increasingly critical in a world where energy is a scarce resource.

• Module 4 – ‘Spectrum sharing and carrier aggregation’ introduces the fundamentals of Carrier Aggregation (CA), the coexistence of small cells and Low Power Wide Area Networks and architectures for spectrum sharing.

• Module 5 – ‘Use cases and prototyping’ presents some testbeds and their importance to validate the theoretical developments in real scenarios, the privacy issue in communications, which is critical given the sharing of more data, and some insights about augmented and virtual reality applications (AR/VR) and immersive media rendering.

• Module 6 – ‘The future ahead’ introduces the emerging technologies that are getting an increased interest for the future 6G generation, such as RADCOM, THz communications, extreme massive MIMO, etc. This module also provides with an early discussion about what the authors think 6G will be, and the experience of the TeamUp5G ETN is also summarised.

The content for each of the items was created by the ESRs and supervisors within the TeamUp5G ETN following an iterative process of planning, development and checking. Besides, it is important to highlight that TeamUp5G members were spread all over Europe, which together with the pandemic situation originated by COVID-19, highly limited the in-person planning and brainstorming events for the MOOC development. This meant that almost all the content creation process was carried out online, mainly with email exchanges and teleconference meetings, which led to a more strict and tight planning of the meetings and deadlines to ensure proper adjustment to the overall timeline.

After defining the MOOC structure and recording capabilities (i.e. facilities and human resources), the specific content of each item was discussed between the members of each module, targeting coherence and avoiding content overlap between items. This discussion was a nice experience that allowed ESRs and supervisors to share knowledge and find ideas for networking. The next step was the writing of the main ideas for the script of the video and the initial structure of the items. Each item had been assigned a set of creators (both ESRs with their respective supervisors) and a set of checkers (other ESRs and supervisors). This avoided the typical problem of creator’s blindness, in which the creators of something do not see the faults in their creation. The first step of initial content creation was done by the ESRs and reviewed by their supervisors for each of the modules, and feedback was given to the researchers in charge of the items. The second step was done by the other set of ESRs and supervisors to validate the content from the point of view of a not so expert audience. In this second step, the audiovisual team of the UC3M was involved in order to check for image quality/copyright issues and unclear descriptions. The initial iteration of this process identified several issues such as the heterogeneity of the slideshows (e.g. design, animations, fonts and number of slides) and the use of images with poor quality or subject to copyright. Many of the original images were conceived to be used in a conventional classroom during teaching activities, so they did not fulfil the required quality for a MOOC. Consequently, in a second iteration, a slideshow/video template and specific strict guidelines were provided for the content creation, which ensured homogeneity between items. When the slideshows and scripts were ready, the video recording process started, which led to the production team to provide specific guidelines for recording, but induced changes in the already approved slideshows. Some of the main issues found were the use of a large amount of text in the slideshows. Replacing it with illustrations was challenging because of the copyright compliance, explained in the next section. The videos were ensured to not exceed the 7-minute timing constraint and achieved the required presentation quality.

4.3 Copyright compliance

Any useful lecture requires well-designed illustrations to provide useful and complementary visual information to the explained topic. Public and massive lectures as those in a MOOC not only require the quality and suitability of the selected illustrations to be high, but also to ensure that all of them, with no exception, are copyright compliant. The selection process is more complex, as the content creators not only need to find or produce high-quality illustrations but ensure only the ones with appropriate licencing are selected. We mainly used the following sources: commercial or licence-free online repositories, proprietary academic or industrial resources and original illustrations by the MOOC contributors. This process led to some delays and repetitions since in the initial steps, for simplicity, the authors tend to utilise the resources that are already available from previous works. Unfortunately, since these works are either published papers or lectures’ materials, they cannot be used since they would result in copyright non-compliance.

Even though all the processes in the MOOC creation were performed cautiously to ensure copyright compliance given the aforementioned requirements, all the illustrations and images that were included in the developed resources (presentations and textual extensions mainly) were double checked by the UC3M audiovisual team which had access to special-purpose copyright validation tools and was able to validate each resource’s licence individually. This process resulted, once again, in an iterative production between the creators (ESRs and supervisors) and the validators (UC3M audiovisual team).

4.4 Contribution from the non-academic sector

The TeamUp5G consortium involves multiple non-academic partners (mainly private companies) which contributed with their expertise to give a more practical approach to the MOOC:

• Nokia Bell Labs: the team from Madrid is focused on the study of the most relevant use cases for 5G and beyond ecosystems. Their research is focused on immersive media offloading and industry 4.0. With their expertise, they have produced or revised the lectures related to the description and analysis of 5G use cases. Therefore, they actively participated in the development of Modules 1, 5 and 6.

• PDMFC: a Portuguese company with the goal of providing solutions in areas such as digital transformation, big data, cloud or security. The contributing team of PDMFC has actively contributed also to the Modules 1, 5 and 6. More concretely, they have developed the items related to network security and how it can be improved with the use of machine learning techniques.

• IS-Wireless: a Polish company that targets software-defined 4G and 5G deployments, with a strong presence in the Open RAN community. Its knowledge has been gathered in the items related to cell-free communications, while they have also supported the development of Modules 1, 2, 3 and 6.

4.5 Quality assurance

A successful MOOC requires a high-quality content with updated and relevant topics, which have to be adequately explained. They also have to be up to date and have to give a professional appearance. For this reason, we have followed a multi-layer quality assurance approach. The first quality check came from the authors themselves: we strongly encouraged all the authors to make a huge effort to produce high-quality content with the goal of reducing the overhead from successive quality checks. Most of the authors were PhD students, and the research also served them as a learning process to produce content for their PhD papers and theses.

Consequently, the second checking layer consisted in their direct supervisors, which had a crucial role in the development of the MOOC given their extensive experience in the production of learning materials and teaching. To add an additional layer for quality checking, we used a peer-to-peer approach, in which the authors and contributors had to check other contributors’ work. As stated before, this step ensured that the creator’s blindness problem was avoided, which occurs when a creator of a certain piece of art or content cannot see the deficiencies in his or her own creation. In every production step, each author had to review at least two contributions from other authors, which brought a proper balance between the effort performed by the reviewers and a good diversity of opinions.

Lastly, the UC3M audiovisual team, given their extensive expertise in editing, reviewing and uploading MOOC contents, was in charge of performing an appearance and sound quality assurance to check the videos and textual extensions were clearly presented so that the audience would not have any trouble in understanding the content.

We believe this process has helped us streamlining the production of the MOOC while ensuring high-quality standards. Finally, all the content was checked by the production team, who was in charge of evaluating the quality from the audiovisual point of view. Each of the mentioned layers involved several iterations: feedback was given, and new versions were produced. Quality assurance requires the available time and effort of all participants, and in this MOOC we have committed ourselves to both of them. The quality assurance process has been beneficial for the ESRs in their training as researcher guaranteeing in their future research fairness and quality policies.

5. Experience

After the MOOC was uploaded to edX, we carried out a beta-testing process, in order to find possible deficiencies. The aim was to guarantee the desired quality in contents and learning activities before launching the MOOC. A total of 2 weeks was allocated to this process. The work was divided among the beta-testers, with at least three beta testers (two ESRs and one supervisor) per module, to ensure enough people to review each module. After feedback information was provided, any identified issues were corrected.

5.1 Publicising

The MOOC has been broadcast in two editions, specifically in February and September 2022. In both editions, a strong publicising campaign started 3 months before the broadcast. The advertising was mainly conducted using the social networks of TeamUp5G. Moreover, the particular networks of every researcher of the project were also used to advertise the MOOC. Specialised mailing lists, such as that of the IEEE Communications Society, were used to announce the broadcast date. The MOOC was advertised in each outreach event where the TeamUp5G members participated. To illustrate the effort on dissemination, after the first edition, the experience gained in the creation and production of the MOOC was shared in the 31st Annual Conference of the European Association for Education in Electrical and Information Engineering (EAEEIE) [19] held at the end of June 2022. Other researchers that participated in the conference were interested in the work done and offered themselves to publicise the MOOC to their students at their respective universities. As an example, the second edition has been also announced by colleagues at the Technical University in Prague in the Techpedia repository [20].

5.2 Enrolment

The enrolment started 3 months before the broadcast in parallel with the communication campaign. Figure 5 illustrates the evolution of the enrolment process for the first edition of the MOOC. A total of 144 students was enrolled at the start date of the first broadcast. After 6 weeks of the course broadcast, on April 2022, 232 students were enrolled. Yet, a verified certificate for the completion of the course was requested by 12 students. These latter students had to take an extra exam in order to obtain their certificates. The remaining participants are considered as listeners although they can benefit from all the course materials and can take the tests of each module. It must be pointed out that it is common for massive online courses to have a high percentage of students that attend as simple listeners for the sake of acquiring knowledge.

Figure 5.

Daily enrolment in the first edition of the MOOC.

Since at the end of the broadcast all materials remain available, more students continued to enrol until the middle of June. The only limitation to the students that were enrolled towards the end of the broadcast period is that they were no longer eligible for an official course certificate. Finally, a total of 290 students was enrolled on 12th June 2022.

The broadcast of the second edition of the MOOC started on 13th September 2022, and currently continues. At the start of the second broadcast, 102 students were enrolled. This number has increased to 125 students a month later. It should be noted that the enrolment of this edition exhibits a similar behaviour than the first one.

Figure 6 shows the education level of the enrolled students of the first edition. It can be noted that $35\%$ holds a university bachelor’s degree while $54\%$ has a higher university degree which includes students who have finalised the master or PhD degrees. Inside this percentage of $54\%$, the most representative figure, with $40\%$ of the total, corresponds with students who have completed a master’s degree and are probably pursuing their PhD now. This figure shows that the audience of the course requires a high level of academic education. In the case of the second edition, a similar behaviour is found, where $45\%$ holds a master or PhD degree. These results match with the target audience foreseen during the preparation of the MOOC.

Figure 6.

Level of education of the enrolled students in the first edition of the MOOC.

On the students’ region of origin, the first edition of the MOOC reached a wide geographic distribution. More than 65 countries had at least one student enrolled in the course. In particular, Spain is the country with the highest number of students enrolled with a $15.6\%$ of the total. Figure 7 shows the geographic distribution divided by continents. We noted that the most representative continent is Europe with $40\%$, where the countries with more participants are Spain, Portugal ($5.7\%$), United Kingdom ($3.9\%$), Germany ($3.9\%$) and France ($2.8\%$). In Asia, the country with the highest number of enrolments is India, with $8.5\%$ of the total, followed by Pakistan ($3.9\%$). With respect to America, the United States represents the country with the highest participation, with $8.5\%$ of the total (same as India), followed by Brazil and Canada ($3.2\%$ each).

Figure 7.

Geographic distribution of the enrolled students in the first edition of the MOOC.

5.3 Interactive discussion

To ensure extra support for the learning activities, a discussion forum where students could share their concerns was set up. Two ESRs were actively involved in the forum to answer questions and ensure that no inappropriate messages were posted. An active participation among students was achieved and supported by the two above-mentioned ESRs, with positive feedback. We observed that two main types of questions were posted. First, administrative questions mainly related to how a valid course certificate could be obtained. Also, two students expressed their concerns with a specific video. Their main comments were associated with the fact that the content amount was high, and it was difficult for them to understand the explanations. Therefore, this video has been corrected for the second edition of the MOOC in accordance with the student’s comments. The second type of question is associated with more technical issues. Students expressed interest in learning more about MIMO systems, which are one of the enabling technologies for 5G. Besides, systems operating on millimetre waves aroused a lot of interest. The two ERSs in charge cleared all the doubts and also suggested additional bibliographic material where the students could expand their knowledge.

6. Challenges and Leassons learned

The primary challenge for MOOC is avoiding the lack of continuous engagement, exemplified by cases of high dropout rate [21]. To understand why the dropout happens, it is vital to understand who is using the MOOC and their purpose. Do the ones who are signing up do really they realise what is their objective? Or do they have doubts? The research specifies that most students join MOOCs for four important reasons: address personal challenges, gain qualifications, develop/advance their knowledge in the specific area of interest and curiosity about MOOCs. Data from [21, 22], which correspond to the Harvard and MIT’s first 16 edX MOOCs, show that the dropout rate in the first week happens to be $50\%$. Among them, most of the students are male with median 24 years old. The main factor about dropping is either external pressures, such as life events, or internal factors. Additionally, the dropping contribution factor is directly dependent on the quality of MOOCs, complex content from the level of the students and lack of direct personal support to the students.

MOOC planning is also a crucial task. From the beginning, it is challenging to have a well-defined structure for the content (e.g. main technological innovations and new 5G mobile networks applications that arise from the TeamUp5G MSCA ITN/ETN), identification of the key learning outcomes (e.g. understanding, designing and optimising the 5G heterogeneous small cell and cell-free ultra-dense networks, whilst covering several topics mainly related to 5G phase 2) and underlying available resources for its production (e.g. in campus Audio/Video facilities of UC3M).

The deadlines for the production, detailed reviewing process and acceptance of the content, with the expected quality-check procedures, should be appropriately scheduled. Active cooperation among all authors of the MOOC is essential and periodic monitoring ensures that there are no communication issues or doubts on the producers.

One of the main problems we had was, sometimes, miscommunication between the coordinators and the content producers. Some producers felt that the indications from the coordinators were not clear or changed as the MOOC progressed. This caused that the content initially produced was not of the expected quality, which caused delays in several tasks. In addition, sometimes the performed monitoring was not incisive enough which also caused the delay in the delivery of the content.

We have learned that, in general, although many of the MOOC producers were not initially aware of the high amount of work required to create high-quality content that meets the expectations of a well-prepared audience, they all agree that it has been a rewarding learning experience.

Working with a very diverse team across Europe also brought challenges to the production organisers, due to several reasons. It was necessary to coordinate a team of more than 20 people and the involved people came from a very diverse set of backgrounds. One of the main problems found by the organisers was a weak commitment from some producers. While some of them did not meet the deadlines, other ones did not read the indications given by the organisers, resulting in weird contributions firstly submitted by some producers, whose quality was not sufficient (and resulted in additional iterations to ensure sufficient quality).

The lessons learned are basically two-fold: the communication between the producers and the organisers should be improved to avoid possible mishaps in the production process, and the commitment from producers should be ensured from the beginning, until the end, to avoid possible inefficiencies in the production.

To meet their own individual expectations, the MOOC creators must focus on high-quality course contents, excelling in the presentation style, while timely delivering the lectures, with appropriate course content assessment. The student’s satisfaction can then be achieved and encouraged for further enrolment in other courses, after completing the current course.

Finally, the COVID-19 was also a challenge because it imposed some restrictions to mobility and recording that constrained the initial planning. However, with dynamic planning and some flexibility, we managed to record every video and material properly.

7. Conclusions

The production of a MOOC involves a great amount of work, time and effort. The most complex task was not only the production of the content itself, but also the coordination of the producers and supervisors. More than 20 people from five different countries have been involved in the production of this MOOC and all the work has been carried out online. Therefore, although our project is composed of great professionals, there were some coordination and miscommunication problems between the supervisors and the content producers causing some delays. Besides, the resources to guarantee the recording quality were available at the UC3M premises in Madrid. Hence, some items were not recorded by the authors but by producers residing in Madrid. All these coordination issues implied that efforts had to be doubled to achieve a high-quality outcome.

Another important conclusion is, as drawn in this chapter, that MOOC can be used as an innovative training method providing tools and resources for the PhD students very valuable and useful in their future careers. In this sense, this experience has been quite positive. Besides, the coordination among the different parts and actors in the research in international groups or projects has been improved thanks to the production of the MOOC, thus this is another advantage of using MOOC as additional tool.

Regarding the dissemination of the research and TeamUp5G activities, the MOOC has been also very valuable. Along with the publications in journal or specialised conferences, the dissemination provided by the MOOC to a more general although technical people is important making visible the project, the institutions and the topic and solutions.

To conclude, the MOOC on ‘Ultra-dense Networks for 5G and its Evolution’ has been presented. We have addressed the objectives, the resources that were available, the production of the MOOC itself and its broadcast experience. Although there have been some challenges during content creation and recording, lessons have been learned and important conclusions have been drawn to improve future MOOC recordings, such as a correct content design, appropriate time scheduling, organisation of the tasks and people responsible for them or strict quality control, among others.

Acknowledgments

We would like to acknowledge the audiovisual team at UC3M in charge of the edition and the support team in charge of copyright checking and uploading the content to the edX platform. We would also like to acknowledge the work of all the people involved in the development of this MOOC who do not explicitly appear as authors of this manuscript.

This work has received funding from the European Union Horizon 2020 research and innovative programme under the Marie skłodowska-Curie ETN TeamUp5G, grant agreement No. 813391.

Conflict of interest

The authors declare no conflict of interest.

Abbreviations

DSP	Digital Signal Processing
ESR	Early-Stage Researcher
ETN	Educational and Training Network
IoT	Internet of Things
ITN	Innovative Training Network
MSCA	Marie Skłodowska-Curie Action
MOOC	Massive Open Online Course
RADCOM	simultaneous RADar and COMmmunications
SPOOC	Self-Paced Open Online Course
SPOC	Small Private Online Course
UAV	Unmanned Aerial Vehicles
UC3M	Universidad Carlos III de Madrid
VLC	Visible Light Communications