Perspective Chapter: Perspectives on the Emergency Remote Assessment during the COVID-19 Pandemic

Abstract

The unprecedented health crisis triggered by the COVID-19 pandemic put on hold traditional educational practices. Emergency remote teaching was adopted as a response, with various degrees of success and satisfaction around the globe. This perspective chapter focuses on the remote emergency assessment as a measure to ensure the completion of the educational cycle for students caught in the 2020 crisis, after debates around the relevance and soundness of such activities both for students, and for society at large. Some voices enthusiastically champion the technological innovation and point to the benefits brought by the computer-assisted assessment, while others warn against the ‘one-size-fit-all’ approach and insist that the emergency measures need a careful examination and, although lessons can be taken away from the situation, the traditional patterns should be kept in place.

1. Introduction

In the modern world, universities are living communities, immersed in the socio-economic and political environment of their residence, sharing the ideas, concerns, and major objectives of society at large. As Ernest Boyer [1] so adequately defined it, the university community, comprised of professors, students, administrative and support staff, is educationally purposeful, open, just, disciplined, caring and celebrative. The university community is educationally purposeful, in the sense that the community members work together to strengthen teaching and learning; it is open because freedom of expression is protected and affirmed; it strives to be just, in the sense that it believes in the sacredness of the person and champions diversity; it is a disciplined community, with individuals accepting their responsibilities and working towards the common good; the community is caring, in the sense that the well-being of each member is supported and service to others is encouraged; finally, it is a celebrative community, traditionally re-enacting rituals of passage that transform learners into professionals.

Universities face the same opportunities and threats as all the other types of communities and need to project plans ensuring their resilience during and after unexpected disruptions. They are not immune to natural or man-made disasters, the numerous examples in the 21st century – to limit the discussion only to these – offering ample food for thought. Hurricane Katrina in Louisiana (USA) in 2005, shooting on campus at Virginia Tech in 2007, earthquakes in Taiwan, 2019, extreme weather events in Australia, led to the adoption of guidebooks, emergency plans and training packages preparing the higher education system to manage the crisis and ensure the transition to the “business-as-usual” model. Region by region and country by country universities adopted the types of measures and documents suiting the specifics of their situation. Australian universities, for instance, constantly renew their internal regulations according to the nationally developed Emergency Planning Handbook [2], which emphasizes extreme weather events (2020). American universities base their crisis and emergency plans on the Department of Education’s Guide of 2013 [3], preparing both for natural, and man-made disasters. Taiwan incorporated disaster literacy in school programs, to facilitate disaster prevention and recovery from the frequent earthquakes in the region. Resilience plan for earthquakes and tsunamis are also adopted in Japan, Indonesia, India, etc. [4]. Inland countries, such as Romania, are less exposed to extreme weather events. Apart from drills related to firefighting or earthquakes, no significant preparation was made in Romania, for example, to face potential threats posed to education since the most major crises in the past century did not affect this sector to the extent of disruption. Higher education institutions, therefore, treated crisis-related topics lightly. Emergency topics remain relevant on the global scale, with UNESCO, as the United Nations lead for education, unfolding ample processes to ensure crisis preparedness and the response of countries and regions facing armed conflict, refugee crisis, natural hazards, and other types of risks [5].

Despite the theoretical, practical and actional preparedness, reflected in the above-mentioned guidelines, handbooks, policy recommendations referring to disasters (natural, man-made or otherwise), crisis, emergencies, or other significant events, the magnitude and depth of the global health crisis triggered by the COVID-19 pandemic took the educational system by surprise. On March 11, 2020, the World Health Organization publicly declared COVID-19 a pandemic [6]. Social distancing, face masks wearing in public places and stay-at-home measures were recommended, to “flatten the curve” and slow the spread of COVID-19. Reportedly, the pandemic affected 94% of the world’s student population, educational institutions struggling to diminish the impact of the crisis [7]. Buildings were untouched, the population was not displaced, but traditional educational face-to-face interactions were put on hold for an indefinite period. The solution resided in adopting emergency remote formats. The continuation of educational processes could not prevent regression in the recruitment and retention rates of students, many of the dropouts not having the intention to return to school even after “business as usual” is resumed. Hence, the importance of reflecting upon education during and in the aftermath of the crisis.

2. Emergency remote education

Emergency remote education (ERE) is not an innovation brought by searching for responses to the COVID-19 crisis. It received special attention with the creation of the Inter-agency Network for Education in Emergencies (INEE) [8], during a strategy session on Education in Emergencies held at the 2000 World Education Forum in Dakar. Since its’ creation, INEE developed the concept of education in emergencies, understood as “quality learning opportunities for all ages (…). Education in emergencies provides physical, psychosocial, and cognitive protection that can sustain and save lives. Common situations of crisis in which education in emergencies is essential include conflicts, situations of violence, forced displacement, disasters, and public health emergencies”. INEE develops standards, training packages and even publishes a Journal on Education in Emergencies (since 2015) which deals with academic research, but also with field notes addressing topics relating to emergency education response in natural disasters, conflict/fragile states and complex emergencies, resilience, transitions from emergency to recovery/post-conflict to development, forced migration and education, etc. The fact that emergency remote education does not automatically mean switching in-person education to online learning deserves emphasis, despite many researchers considering the technological intervention as automatic and online learning as the hallmark option in times of crisis [7, 9, 10]. In low-tech environments, emergency remote education can resort to low-tech solutions, but with higher education institutions, the focus of this chapter, technological affordances are presupposed as being present in the form of computers, telephones, some sort of digital devices, accompanied by ICT skills, shared by academia and students.

Higher education systems relied, at the peak of the COVID-19 crisis, on the pedagogical acquisitions of online learning experiences, even though for many stakeholders in the educational process online learning, e-learning or distance education represented entirely different cohorts from the in-campus education. One of the most referred to articles in the current literature on the educational response to the COVID-19 crisis is Charles Hodge’s The Difference Between Emergency Remote Teaching and Online Learning [11]. Apart from the insights into the specificity of emergency remote education and its numerous challenges, Hodges rightfully highlights the fact that the learning outcomes should be carefully assessed, the pre-crisis models being almost impossible to apply. In his words, “a common misconception is that comparing a face-to-face course with an online version of the course constitutes a useful evaluation” [11]. Hodges’s list of open questions to evaluating the outcomes of the emergency remote education is quite impressive, starting with normative and administrative aspects, going through technological affordances, and culminating in the shared, informed consensus between educational stakeholders on the way in which evaluation is to be performed and incorporated in the institutional memory and practice, in the aftermath of the crisis. At this point, the emergency remote assessment becomes a crucial concept [12, 13], especially because first-response studies on education during the COVID-19 crisis showed that “assessment was deprioritized in initial planning” and that “teachers saw assessment expectations as unstable or unfair during a crisis” [14]. Scenarios concerning the postponement or even cancelation of final assessment were under debate and the greatest fear was that society might be reluctant in accepting the results of assessment activities (especially for graduation exams) carried out during the unfamiliar global crisis triggered by the COVID-19 pandemic [15].

2.1 Assessment and emergency remote assessment

Assessment is a crucial element of the educational process, representing the segment during which professors evaluate, measure, and document the academic readiness, learning progress, skill acquisition, or educational needs of students after a teaching-learning sequence. It is often considered as a pivotal element for modernizing education systems, affecting how teaching and learning take place, but also as a tool for certification, enabling potential employers to understand the academic achievements of an individual [16]. Educational literature emphasizes the need to incorporate assessment provisions in the curriculum design and to ensure coherence and synergy between curriculum, pedagogy and learning outcomes [17]. In UNESCO’s Education strategy 2014–2021 assessment is treated as an important part of action towards “more just, inclusive, and equitable learning societies” [18]. However, the crisis of 2020–2021 brought new meanings to the assessment-related concerns.

The traditional pen-and-pencil exam was replaced by new tools, and frenzic searches for reliable and acceptable forms of assessment were enacted. Studies from the field show mixed reactions: to some practitioners, COVID-19 was the modernizing driver, that accelerated the incorporation of technological tools and facilitated the qualitative leap forward [19]. Others looked more at the “dark side” of technologically mediated assessment activities, which leave ample room for cheating and academic dishonesty on the part of the students [20]. A third path, however, is more fruitful in terms of enhancing the reliability and acceptability of assessment activities: the path opened by David Boud in his conceptualization of sustainable assessment, launched in 2000 and constantly developed with new resources available for in higher education, presented on the website—www.assessmentfutures.com [21].

Due to social isolation measures and the remote delivery of educational content, assessment activities were organized also in a remote fashion. Emergency remote assessment could not take place without the mediation of technology. Therefore, depending on the taught subject-matter, on the technological affordances, the teacher’s own capacity to innovate, students’ consent to the proposed activity, the emergency remote assessment took the form of:

• self-assessment [22, 23]

• peer-assessment [22]

• take-home exams [24]

• portfolios [23, 25]

• e-assessment activities (exams), human-led or computer-based [26].

All the above have been already carefully documented by Boud and Soler [21] as paths towards sustainable assessment implementation, but the disruption caused by the pandemic accelerated these possibilities as viable alternatives to the traditional assessment activities and enrich the community of practice with new case studies, reflections, and recommendations regarding the modernization of assessment. The calls to experiment and change in assessment practices, long claimed for [21, 27] could not be ignored.

Evidence from the abundant literature on teaching amid the COVID-19 crisis points to the fact that teachers felt overwhelmed by the added responsibilities to the workload assumed before the outbreak of the pandemic, while students felt anxiety, anger, detachment, and a loss of purpose in their educational path. Universities succeeded at uneven rates to provide normative and instructional support mechanisms to ease the shift from in-campus education to emergency remote teaching and learning [11]. The whole higher education ecosystem was challenged, and the psychological factors had to be considered alongside pedagogical solutions, technical affordances and the commitment of teachers and students to keep educational processes functional.

The pandemic situation forced a change in the environment for assessment activities. Instead of face-to-face exams (be those pen-and-pencil, oral, or computer-assisted, but with the physical presence of instructors in the exam room), a virtual environment had to be created, affecting the very medium of the exam conditions. It was terra incognita for both participants in the exam, professors, and students alike. Some of the elements in this new setting did not allow for familiar mechanisms of control. So, professors tried to cope with the novelty by negotiating new rules of engagement with the students, adapting assessment techniques and types to the situation.

Overwhelmed by the amount of data and the task of offering rapid feedback to students, some professors experimented with diversifying the identity of the assessor. Self-assessment and peer-assessment were introduced to students in a more vigorous fashion, to deal with the task of mastering the assessment flow. While this is a fruitful solution, if applied correctly, it cannot spread over all types of exams. Students can estimate their success in exams, by having access to the grading system, with details regarding the measured skills and/or knowledge in a given exam, but for graduation exams, for instance, the process is entrusted to an examination committee.

Another element of the assessment that professors may have control over is the timing of the examination interaction. Many examples of the published case studies indicate that the dominant choice was a synchronous examination, with instructors doing their best to replicate the familiar exam atmosphere, while ensuring that they share the same medium with students. Access to technology, a fairly good Internet connection and the skill to ensure the flow of the assessment are preconditions for success. An ample debate in pedagogical literature goes towards the manner of organizing the technology-assisted assessment, with a focus on the technological affordances and tools, but also with an eye to the security, fairness, and accuracy of the process. To diminish the concerns about academic ethics a variety of solutions was proposed, among which open book exams seem by far the most interesting, and with the highest degree of implications. Open book exams may revolutionize the assessment type, since students no longer need to rely on memory, but focus, instead, on demonstrating the ability to creatively apply the knowledge to the task at hand. They have to prove concept understanding and skill acquisition, instead of a good memorizing capacity, but challenges to this type of exam are numerous. Students may only reproduce what they find in the resources, without memorizing even the basic information compulsory as guidance towards their future profession. Professors need to be trained in designing and assessing open-book exams and by far these are not simple tasks.

The type of work submitted for assessment is of consequence to the way the activity is organized. Faced with the necessity to ensure access to education and non-discriminatory conditions for students with limited access to technology and/or the Internet, professors embraced to a larger extent asynchronous exam, in the form of take-home exams or portfolio-based assessment. These are not merely terminological choices but shifts in the assessment type. Take-home exams are typically unsupervised and open book. Logistically they are simple to organize, the instructor sending the task to students via any medium at hand (e-mail, discussion group, WhatsApp, phone call, etc.). The content of the take-home assessment, however, needs to be meaningful and appropriate for this type of examination. Among major risks, signaled by students, are the work overload (students complained that it increased during the pandemic education), the duration of the exam, and even the complexity of the task.

Emergency remote assessment (ERA) can best work if Boyer’s six principles are applied:

• ERA needs to be educationally purposeful. While the co-creation of new assessment formats had limited timespans, professors and students needed to negotiate the assessment activities for them to have meaning and to contribute to strengthening teaching and learning. Despite sometimes untested methods of evaluation used to complete the teaching and learning process, an assessment still preserved its’s most salient feature: that of accounting for the learning outcomes during a stage or after a course.

• ERA needs to be open, in the sense that professors should have the freedom to choose those forms of assessment that are fit for the content of the course and the pedagogical experience of the instructor. While universities may recommend assessment strategies and need to provide the normative frames, especially for end-term, admission and graduation exams, some degrees of freedom should remain with the teachers developing assessment activities and providing their professional expertise.

• ERA needs to be just, in the sense that no student should be discriminated because of their lack of technological affordances or skills. The university support needs to go towards ensuring that the learners have the information, means and possibilities to continue their learning voyage throughout the storm of crises and that their accomplishments are properly acknowledged.

• ERA relies on discipline, with individuals accepting their responsibilities and working towards the common good. Clear rules, adapted to the situation and brought to the attention of all participants in the educational process, embraced by the academic community, spread over the assessment activities as well.

• ERA needs to be caring, in the sense that the well-being of each member is supported and where service to others is encouraged. This feature ensures that teachers continue to feel valued as experts in their field of work, while students enjoy the respect and find support in reducing assessment-related stress.

• Finally, ERA needs to be celebrative. Since the foundation of universities, academic life is marked by reiterative rituals [28]. Assessment activities, especially exams are a significant part of academic rituals [12]. Studies show that during the emergency remote assessment activities students missed most, exam-related rituals [12]. Although, some of these, such as verbal interaction with peers before and after the exam could easily be carried into the virtual environment, most students remained encapsulated in their constructed shell that felt like artificial, alienating, although familiar and somewhat supportive, since most of the students were confined to home-staying with their families.

In terms of the elements that need special attention for unfolding the emergency remote assessment, findings in the literature (case studies, surveys, and pedagogical reflections on the matter) show that the following four dimensions need to be considered:

• Technological solutions that facilitate access to the educational process and ensure the support for assessment activities.

• Appropriate pedagogical approaches, that consider the affordances of the newly created learning environment (virtual or otherwise).

• Supporting measures, in the form of norms, regulations and procedures about the exceptionality of the emergency situation and allowing for the accommodation with the adaptive tools and procedures that account for the assessment activities.

• Support structures and provisions ensure the well-being of teachers and learners as well, preventing burnout, technology fatigue, alienation, or disconnectedness.

For the first issue, ERA can benefit largely from the acquisitions of e-learning practices and computer-based assessment [7, 9, 29]. Students’ voices, however, warn that while ERA represented a breakthrough in educational practices and they welcome the comfort brought by the use of computers for assessment purposes, the human factor still needs to be present in the appraisal of the results [12]. Students liked that they could use computers or other digital devices during exams and those professors tried to offer feedback on time. However, they also showed that their satisfaction rate depends on the relationship negotiated with the instructor and that they do not fully trust technology to provide an accurate assessment of the learning outcomes, a reaction anticipated also in earlier research [30]. Change in assessment is often unwelcomed [30], but the COVID-19 crisis made it compulsory. Efforts were undertaken to overcome resistance in the face of new forms and in presenting assessment as more than a mechanism for gatekeeping and achievement certification.

In terms of pedagogical approaches, we share the belief that Torrey Trust and Jeromie Whalen voiced in proposing that teachers be trained in emergency remote education as part of their set of professional skills and as a precondition to ensuring the resilience of the educational system in times of future crisis [31, 32]. These authors advance the idea that exploring the difference between emergency, blended teaching, and online teaching “may help scholars and teacher educators identify professional learning topics that can improve teachers’ feelings of preparedness for teaching in any situation moving forward”, in all the instances about the educational process [32]. Taking one step further, we believe that the experiences of remote teaching and remote assessment should not be forgotten and the return to the “new normal” education needs to preserve the lessons of 2020–2021 in the institutional memory and practices, such as it is recommended by crisis management literature [33]. In reflecting upon the accumulated experiences, teachers and institutions contribute to building resilience, understood as the ability of colleges and universities to prevent, protect against, mitigate, respond to, and recover from the crisis, be it triggered by natural disasters or by man-made disruptive activities. By incorporating the lessons of education during the COVID-19 pandemic into institutional memory and practices, appropriate plans can begin to be formulated on how to steer universities towards being resilient.

Supporting measures were rapidly developed during the pandemic, a synthetic document drafted by the Organization for Economic Co-operation and Development grouping the available resources along with three categories: Curriculum Resources, Professional Development Resources, and tools [34]. These resources were developed by a variety of actors: educational systems, represented by ministries or alliances of universities, by private initiatives, by educational non-governmental organizations, etc. At the micro-level, each institution adopted its’ own set of measures to provide support to teachers and/or students, as shown by the numerous cases in the research literature on emergency remote education during the COVID-19 crisis.

Finally, the support measures and provisions ensuring the well-being of teachers and learners ensure the possibility of “building back better” the educational system in the aftermath of the pandemic, as listed among the nine ideas that shape the future of education in the post-pandemic world [35].

3. Conclusions

This intellectual journey into the pedagogical, managerial, and political experiences accumulated during the COVID-19 pandemic necessitating the adoption of emergency remote assessment (as part of the emergency remote education) stresses the necessity of researchers, policymakers, and educational practitioners to join forces and find solutions to the unexpected challenges brought by crisis situations. Beyond the immediate ad hoc measures aimed at ensuring learning continuity (often referred to as sudden online education), higher education institutions need to process the whole string of experiences [36] and extract long-term strategic implications for their return to in-campus learning and their resilience plans.

Throughout its history, the university assessed itself as a learning, adaptive and evolving organization. It follows naturally that the university will use the pandemic lessons in post-event analyses, increasing the knowledge of the organization, in building anticipation of and preparation for future (disruptive) events. The coping phase, manifested at the outbreak of the pandemic in the form of adopting emergency educational solutions was replaced by the adaptation phase, during which acquisitions from other experiences were called upon (mainly from e-learning and technology-assisted education) and, finally, by the transformational phase, during which universities once again prove themselves to be drivers of innovation. Educational systems aim at building sustainable, resilient universities as part of the development of academic life, “continuously influenced by sociocultural, technological, and environmental changes” [37]. In the process, assessment practices need to evolve towards sustainable models, ensuring that appropriate pedagogical resources are activated, that students understand, accept and extract meaning from the assessment activities, that society acknowledges and recognizes the result and those technological affordances are put to use in the process. With each case study, a note from the field, review, or reflection the community of assessment practices enlarges and gains momentum.

Assessment activities, although occurring after a cycle of teaching-learning string, are planned at the initial phase, and incorporated in the curriculum. The specifics of the emergency situations cancel some of the features, if not the entire projection of assessment and new models need to be put in place. The future, resilient university should make room in the curriculum design for sustainable assessment forms, that are less influenced by external factors and allow for a smooth shift from in-campus activities to remote education. Such a sustainable model needs to be not only student-centered, but also co-created with students, to ensure that they understand, accept, and comply with the new provisions in their academic journey. Assessment should gain more visibility in institutional debates and policies, and despite all adverse events it cannot be canceled without risking that the progress of students through the curriculum eludes appraisal instruments. Innovative models of assessment should be encouraged, such as self-assessment, peer assessment, take-home exams, open-book exams, enhancing the responsibility of the learners towards learning, their understanding of the process and their preparation for transitioning from school to work.