Innovative Tools to Assess a Large Number of Students in the Open Distance and e-Learning MOOCs

Abstract

The literature on students registered in the Open Distance and e-Learning (ODeL) institutions suggests many obstacles related to their summative-driven assessments, which give insufficient time for study, difficulties in access and use of innovative assessment tools, ineffective feedback, and lack of feedback of study materials. These challenges lead students to learn just enough to get grades without understanding the topics or acquiring knowledge and skills. On the other hand, massive Open Online Courses (MOOCs) give students, who have to fulfil multiple roles and are affected by the barriers of distance, cost and time, an opportunity to pursue their studies online. This chapter employed humanistic learning theory (HLT) to present a variety of digital teaching and learning tools that enable assessment suitable for a large number of students in the ODeL MOOCs. Humanistic learning theory emphasises a shift towards considering students, their characteristics, and their influence on learning. In addressing the gap created by assessments that were not focused on the specific human capabilities, including creativity, personal growth, and choice, this chapter first presents principles of HLT linking them with the form of assessments in MOOCs. Secondly, the ways to assess a large number of students in ODeL MOOCs are outlined. Lastly, various digital tools that can assess a large number of students are discussed, considering students as sources of authority.

1. Introduction

Millions of higher education students are taking distance education courses from different institutions around the world. Qayyum and Zawacki-Richter (2019) [1] confirmed that 23 million students enrolled in distance education courses from institutions in the twelve countries. This increase in enrolment calls for a paradigm shift to meet the needs of assessments. It should be noted that assessment is no longer used for only grading and certification; instead, it has linked with the learning and skill development of the students (Chaudhary & Dey, 2013) [2]. This chapter employs the Humanistic Learning Theory (HLT) to explore the students’ capabilities to set their learning standards and evaluate their work. HLT, often called Humanism, necessitates that the teaching and learning experience help students develop positive relations with their peers (Hare, 2019) [3]. Humanism is a philosophical and ethical stance that emphasises the value and agency of human beings, individually and collectively (Slakmon & Schwarz, 2018) [4]. Furthermore, HLT describes learning in terms of personal growth and the full development of each human’s potential not on just an intellectual level but also on an emotional, psychological, creative, social, physical, and even spiritual level (Gould and Roffey-Barentsen) [5].

The humanistic learning theory was developed by Abraham Maslow, Carl Rogers, and James F. T. Bugental in the early 1900s in response to the typical educational theories at the time, which were behaviourism and psychoanalysis. Maslow [6] emphasised that HLT encourages innovation and creativity while purporting that every student is responsible for their learning and the learning of those around them. Against this background, this chapter highlights the link between HLT and Open Distance and e-Learning (ODeL) assessment, where students need to engage in self and peer assessment activities. However, the following need to be considered:

• Due to the physical distance between the students and teachers in the ODeL, feedback is vital to generate a spirit of consciousness and motivation

• Teachers’ examination of students’ performance at every stage of their study and progress successfully to attain the course objectives

• monitoring the effectiveness of academic programmes and adopting appropriate strategies to accomplish institutional objectives.

• Development of a positive attitude towards the institutional system

2. The humanistic assessments in ODeL MOOCs

The meaning of assessment has been consistent throughout the years. Nevertheless, the implementation of assessments varies depending on the teaching and learning environments. This chapter focuses on assessment as legitimate activities within the HLT, which emphasises growth, subjectivity, agency and student centredness (Friedman & MacDonald) [7]. Having humanistic assessments implies incorporating humanistic strategies and interactive activities in ODeL MOOC assessments, as highlighted by Davis, Chang, and McGlothlin [8], who further attest that humanistic assessments help demonstrate the utility of practical humanistic knowledge and skills. It should be noted that is HLT, as confirmed by Johnson [9], is concerned with personal growth and includes attention to students’ affective dimensions such as self-concept, values, and emotions. Against this background, it is safe to conclude that humanistic assessments should include self-assessments for personal growth and self-concept. They should also include peer assessments because [9] maintain that students do not take away from the other; instead, they both serve to enhance the other. While implementing the self and peer assessments in the ODeL MOOCs, one should consider many student enrolments. The following section put forward the principles of HTL linked with the forms of assessments in ODeL MOOCs.

3. Principles of HLT linked with forms of assessments in MOOCs

Several essential principles are involved in the HLT that the author found to connect with assessment forms in ODeL MOOCs. First, assessment in MOOCs does not necessarily have to be about course completion. Instead, students can be assessed on time-on-task, student-course component interaction, and a certification of the specific skills and knowledge gained from a MOOC (Chauhan, 2014) Table 1 [10]. presents the summary of the link between the principles of HLT and forms of assessments in ODeL MOOCs.

Principle of HLT	Form of assessment in MOOCs
Student choice	Personal learning networks
Fostering engagement	mobile learning on MobiMOOC
The importance of self-evaluation	Automated assessments
A safe learning environment	Adaptive assessments

Table 1.

The link between the principles of HLT and forms of assessments in MOOCs.

Table 1 shows how the principles of HLT connects with the form of assessment described by [6]. The link is described in detail below.

3.1 Student choice

According to Maslow [6], student choice is central to the HLT. Given the fact that humanistic learning is student-centred, students are encouraged to take control over their learning. Students can control their learning by using various online teaching, learning and assessment tools to develop their learning networks. The online tools will help students to make choices that can range from daily activities to future goals also to find motivation and engagement in their learning,

3.2 Fostering engagement

The HLT relies on teachers to fosters engagement, encouraging them to find things they are passionate about so they are excited about learning to inspire students to become self-motivated to learn. When students are self-motivated to learn, it will be easier for them to use mobile technologies to access course content and assessment activities and knowledge creation and sharing within the network of their peers. Learning in a MOOC offers students various online media and interactive tools for student participation and engagement.

3.3 The importance of self-evaluation

Maslow [6] For most humanistic teachers, grades do not matter. Self-evaluation is the most meaningful way to evaluate how learning is going. Grading students encourages students to work for the grade instead of doing things based on their satisfaction and excitement of learning. Routine testing and rote memorisation do not lead to meaningful learning in HLT and thus are not encouraged by humanistic teachers. Instead, humanistic teachers help students perform self-evaluations so they can see how students feel about their progress.

3.4 A safe learning environment

Because humanistic learning focuses on the entire student, humanistic teachers understand that they need to create a safe environment to have as many students need to be met as possible. The adaptive assessments cater for diverse learners because they address different difficulty levels. Based on each assessment item’s response, most adaptive assessments decrease or increase the difficulty level to match learner ability and potential.

There are some critics on the quality of assessment in MOOCs; for example, MOOCs automated grading tools for straightforward testing, such as multiple-choice, true/false, and short problem sets. Linking the principles of HLT with forms of assessment in this chapter aims to address some of these criticisms. The following section presents the current ways to assess a large number of MOOCs before focusing on the innovative tools to assess a large number of students in ODeL MOOCs.

4. Ways to assess a large number of students in ODeL MOOCs

Given the large numbers of students in MOOCs, the following assessments, according to Admiraal, Huisman and Pilli [11], are implemented:

• Self-assessment

• Peer assessment

• Summative assessment

4.1 Self-assessment

Some studies declare that the use of self-assessment in MOOCs is underestimated to some extent; Ventista [12] argue that it is the most suitable assessment method to correspond to the needs of these self-regulated students and a potential solution to the high attrition rates and the patriotic grading bias during peer-assessment. Nevertheless, Admiraal, Huisman and van den Ven [13] corroborate the declaration of undervalued self-assessment in MOOCs because their study suggested a bias of self-assessments that led them to conclude that self-assessments might not be an excellent way to assess students’ performance in MOOCs. As highlighted in the section of HLT principles, humanistic teachers believe that grades are irrelevant and that only self-assessment is meaningful.

4.2 Peer-assessment

Perr assessment is a form of assessment where students receive marks from their peers (Habib & Sanzgiri) [14].

Furthermore, they mark their peers in returnVentista [12] notes that peer assessment does not appear to be implemented in the ideal conditions in the case of MOOCs. According to Comer and White, [15] peer assessment can be beneficial for the students when they reflect on and evaluate the work of their peers but could not be used as a summative indicator of students’ achievement. Much of the current literature on MOOCs pays particular attention to limitations in giving students feedback in MOOCs due to the large enrolment. Piech, Huang, Chen, Do, Ng and Koller [16] recommend using peer assessment, which they refer to as a form of assessment historically used for logistical, pedagogical, metacognitive, and affective benefits. In MOOCs, peer assessment is viewed as a promising solution that can scale the grading of complex assignments for many students (Sadler & Goo [17]. Some students view peer assessment as a motivating element due to helpful feedback and fair grades (Luo & Robinson) [18].

In terms of HLT principles presented in the previous section, peer assessment fosters students’ engagement. Taken together, the note by [12] and recommendation by [15], it seems possible that peer assessment can develop more expert-like evaluative judgement (critique, analyse, provide feedback). This chapter also notes that with peer assessment, students can demonstrate expertise through the creation of non-automatically gradable materials (e.g., video presentations, essays, reports, reflections, designs).

4.3 Summative assessment

The term summative assessment has come to be used to refer to assessments of learning, which record students’ cumulative progress. Xiong and Sueng [19] differentiate between formative and summative assessments in MOOCs by emphasising that the stakes involved in the summative assessment are usually higher than those in a formative assessment because the former leads to course grade assignment. In general, summative assessment is defined as evaluating what students have achieved after a period of study relative to the learning aims and in accordance with a national qualification framework. Within MOOCs, summative assessments can employ either one or multiple assessment types. Combining multiple assessment types can help reduce the time and cost of marking per student and provide more chances for students to obtain helpful and meaningful feedback.

In MOOCs, summative assessment is viewed as cost-effective because it reduces the cost of marking per student and offers opportunities for instant feedback depending on the tasks. MCQ tests allow automatic evaluation of group and individual performance. The online media and interactive tools enable the humanistic approach in the assessment activities. In addition, some Learning Management Systems (LMS) used to host the MOOCs offer various assessment tools that enable integration of teaching, learning and assessments. The assessment tools are discussed in the section below.

5. What are innovative tools to assess a large number of students in ODeL MOOCs?

Most MOOCs offer automated grading tools for straightforward testing, such as multiple-choice, true/false, and short problem sets. However, when assessments wade into more complex territory--such as student essays--the grading solutions take on the controversy. In this section, some tools that can be used to conduct the assessments presented in the previous section. Table 2 summarises the link between the assessment tool, types of assessment and HLT principles.

Assessment tool	Type of assessment	HLT principle	Grading
Quiz	Formative and summative	Student choice; The importance of self-evaluation; Fostering engagement; mobile learning on MobiMOOC	Teacher and computer
Discussion Forum	Formative	Fostering engagement; A safe learning environment	Teacher and peer
Blog	Formative	Fostering engagement; A safe learning environment; Personal learning networks	Teacher and peer
e-Portfolio	Formative and summative	Fostering engagement	Teacher and peer

Table 2.

Summary of innovative tools to assess a large number of students in ODeL MOOCs.

Table 2 illustrates how and when the innovative tool to assess a large number of students in ODeL MOOCs can be used. This chapter assessments for a large number of students in ODeL institutions should not be a nightmare but should encourage innovation, creativity and responsibility. As shown in Table 2 that Blogs and discussion forums are not yet used as a summative assessment because summative assessments are almost always formally graded and often heavily weighted. The innovative assessment tools discussed below can be used either for formative or summative assessments. With the formative assessments, teachers can assign them either self or peer assessment, particularly for many students.

5.1 Quizzes

Quizzes in Moodle are used to evaluate student understanding of the material. Chauhan and Goel [20] regard quizzes as some of the primary elements of MOOCs for evaluating the students’ knowledge. In addition, Gamage, Ayres, Behrend and Smith [21] attest that quizzes can be used to improve student engagement addition, Gamage et al. [21] attest that quizzes can be used to assess students’ competencies during the various stages of a study period through automated marking and easily extractable statistics as well as improve student engagement. Chauhan and Goel [20] established that quizzes are used for two reasons; first is for evaluating students’ performance, second for practice purpose to provide instant feedback to the students for self-evaluation, without worrying about the effect of their score on final score outcome. With a large ODeL MOOC class, the teacher can set up a computer-graded quiz with feedback for each question or only correct and incorrect question feedback. The students will receive feedback either immediately after each question or after submitting it for grading. That will depend on the teacher’s settings. The same quiz can be used as a self-assessment activity where the students can use the feedback provided to perform personal, unguided reflection on performance to generate an individually derived summary of one’s level of knowledge, skill, and understanding in a particular area (Andrade) [22].

5.2 Discussion forums

MOOCs have focused on social interactions between students due to the physical distance and large enrolments and, most importantly, improving learning outcomes [21]. However, some researchers do not associate discussion forum with assessment. Lan, Spencer, Chen, Brinton and Chiang [23] posit that discussion forums are tools to facilitate social learning in MOOCs. Similarly, Onah, Sinclair, and Boyatt [24] view discussion forums as a primary means of interaction among students and teachers in MOOCs. The study conducted by [24] using data from a specific MOOC run by the University of Warwick revealed low discussion forums and inadequate peer support. Another concern about discussion forums in MOOCs raised by Lan et al. [23] is being structured instead ofo being generic. According to Brinton, Chiang, Jain, Lam, Liu, and Wong [25], discussion forums are mostly centred around course content, assignments, and course logistics. This chapter argues that discussion forums can also be used as an innovative assessment tool and peer or teacher graded. For a teacher to enter a grade and feedback for each student in a discussion forum assessment for a large number of students in ODeL MOOC can be a monumental task. In keeping with their significance to MOOCs, where students are scattered across the globe, such as in ODeL institutions, teachers should create peer-graded discussion forums to empower the students to test out new concepts, galvanise ideas and reinforce new thinking (Sharif & Magril) [26].

5.3 Blog

Much literature has been published on Blogs deeming them as communication and learning tools in MOOCs. Mak, Williams and Mackness [27] reflect on several definitions of a Blog from various sources and maintain that blogs are associated with creating personal space for personal learning, quiet reflection and developing personal relationships with bloggers and others. Depending on the LSM used for the MOOC, blogging for assessment can be effective to a certain extend. For example, in the ODeL environment, it might be ideal to use a blog for peer and self formative assessments. The blogging activity does not only encourage students to engage but also enhances their digital literacy skills.

It should be noted that the blogs that can be used for assessments are those embedded in the LSM; however, they should still allow self-motivated bloggers to freely and easily post ideas, individual experiences, and opinions. The teacher can ask the students to create a blog or respond to a blog that they created to be graded through peer or teacher grading. Some of the LMSs hosting the MOOCs incorporated a peer review system in their learning platform that guides students using grading rubrics to evaluate and provide feedback for each other’s work. The teacher can set up a blog activity and ask each student to grade at least three other students or more blog posts. In this activity, students can be allowed to evaluate their blog post and allocate self-grading scores.

5.4 E-portfolio

There is some evidence to suggest that e-portfolios in ODeL MOOCs are used mainly for summative assessments. Nevertheless, Cheng [28] validates that e-Portfolio-based formative assessment can record students’ progress to offer teachers and students information about how students’ proficiency improves and enhance students’ autonomy in learning. In line with the first statement, Downes (2013) [29] explains e-portfolio in a MOOC can serve as a resource that a student has to present as proof of his or her learning. A portfolio can be graded with a peer or teacher grading. In the LMS like Moodle, a teacher can use a workshop tool for students to submit their portfolios to distribute among peers for assessment based on a specific grading scale or rubric.

5.5 Game-based assessment

There are millions of learning games in the online learning environment. Although game-based assessment, according to Smith [30], is mainly Game-Based assessments, or GBAs, use gaming technology to help assist employer decisions during their recruitment processes. They form part of the puzzle as to how suitable a candidate is for the role and company.

The most commonly used game-based assessment tool in ODeL MOOCs is Kahoot. It is a game-based approach for learning and assessment. Students can even create their own “kahoots” to share with classmates, creating an interactive experience. In addition, the game-based assessments can be integrated with the quizzes.

6. Conclusions

This chapter put forward the principles of HLT in related forms of assessments in ODeL MOOCs. The section touched on the criticism made by some researcher about the quality of assessments in MOOCs. In concluding this chapter, it addresses those criticisms by associating them with the reliability and validity of assessments. Luo, Robinson and Park [31] affirm that the joint efforts of multiple student graders can produce fairly consistent grading results using MOOCs’ peer review systems. Their study of investigating the reliability and validity of peer grading found high levels of agreement between student-assigned scores and teacher-assigned scores measured by the correlation coefficients, which support the validity of peer grading in the MOOC context.

The ways of assessing a large number of students in ODeL MOOCs presented in this chapter are not new to the general teaching and learning environment. However, in this chapter, self, peer and summative assessments were linked with HLT principles and how they can be used in ODeL MOOCs. Lastly, the chapter depicted innovative tools to assess a large number of students in ODeL MOOCs. Some examples are given on how a large number of students can use the tools.

Since humanistic teachers are passionate about helping students meet as many of their needs as possible, using the innovative tools mentioned in this chapter may assist the students to adapt to learning and measure their performance.

Conflict of interest

I declare no conflict of interest.