Improvement of Student Attention Monitoring Supported by Precision Sensing in Learning Management Systems

Abstract

A Learning Management Systems (LMS) can benefit from the inclusion Computer-Mediated-Communications (CMC) software for delivering materials. Incorporating CMC tools in virtual classrooms or implementing educational blogs, can be very effective in e-learning platforms. In such student-centered interaction scenarios, it is important to monitor and manage student attention in a precise way to enhance student performance. Sensing with precision through 6G/7G technology allows to include electronic and software devices to produce such monitoring. This chapter contextualizes and describes an abstraction application scenario of sensing and monitoring student attention with high precision in Learning Management System with new communication systems. In that context, technology (e.g. sensors), is used to perform automatic attention monitoring, helping to manage students in e-Learning. Additionally, the document presents a possible scenario which supports intelligent services to the monitoring of student attention during e-learning activities in the context of Smart HEI (Higher Education Institutes).

1. Introduction

The Global Higher Education community is nowadays facing new educational challenges due to the Coronavirus pandemic. There is an opportunity for this community to implement a new strategy at the university level [1]. Migrating from traditional or blended learning to a fully virtual and online deliverable strategy is, therefore, crucial to ensure quality education. This transition occurs gradually. Linked to this issue are several questions related to the lack of “home office” infrastructure; skillsets needed for professional design, and online/virtual education options.

Since World Health Organization declared a Coronavirus (COVID-19) pandemic in January 2020, new challenges appeared in the Higher Education ecosystem.

Top ten most affected countries, reported on March 2020, were: China, Italy, the United States, Spain, Germany, Iran, France, South Korea, Switzerland, and the United Kingdom [2]. That context raised the opportunity in on-line and virtual education, to meliorate e-learning, and infrastructures.

The knowledge and skills required can empower workers for future challenges of new jobs that are appearing along with technological advances. Nowadays, it is important to create solutions, simultaneously at operational, services, and technological levels at Higher Education Institutions (HEIs) that can help students develop those competencies.

To date, in e-learning, there is not sufficient research that investigates intelligent technological artifacts designed to enhance student attention through the monitoring process.

Additionally, there is a lack of research in technological integration frameworks that propose design strategies for those artifacts that includes sensitive aspects in education, such as emotions or attention.

E-Learning allows academic institutions to deliver the learning content electronically, both in mobile and online environments. This content might commonly be delivered through Learning Management Systems (LMS). One might say that the tendency of LMS is to become complex when compared with the earlier versions. Nowadays, LMS deals with complex representations of the relationship between resources, teachers, and students, and these systems have become much more customized. These LMS platforms can be implemented by a Service-Oriented Architectures (SOA), a conceptualization that supports the development of web applications. Specifically, in SOA the service provider manages services designed and its implementation. Services are published in the registry, and then will be available for service requests, find the service specifications and the correspondent service provider.

A branch of approaches to e-learning systems focuses on the ability to sense a situation, interface, as well as interact and communicate effectively with the environment. These smart-systems can incorporate sensors and actuators, interacting with other systems, and be incorporated into platforms. It is important to notice that those technologies might be intelligently and methodologically and introduced them into the learning context effectively.

In digital environments, it is important to monitor and manage student emotions and attention. In this work, the term “attention” might be seen as an integration of different aspects or perspectives of attention, aggregating cleverly these aspects. Thus, focused attention sustained attention, selective attention, alternating attention, and divided attention are considered different types of attention and can be monitored depending on the task to be performed. Attention, thus, might be managed in the educational virtual settings, which in this study is done with the support of NeuroIS, a relatively recent branch of information system which allows one to establish a close and fast correspondence between the variables of a problem specification and those of the solution space [3]. Behind that correspondence are the devices that allow one to monitor student attention which is well framed and delineated in the NeuroIS approach. These devices can be used in more complex systems.

Attention-aware systems manage attention using sensory mechanisms, both detecting student focus and making predictions, which allows one to offer customized learning. Several ways have been used for attention detection in the e-learning field, for instance, eye tracking, video, electroencephalogram webcam, electrocardiogram. These mechanisms, for instance, webcam or electroencephalogram, can have an accuracy rate of up to 90%.

The hereby-presented research work encloses the following research question: How to enhance student’s attention in an e-learning environment?

Concerning the proposed research question, aforementioned, the authors argue by the hypothesis that if it is possible to sense student’s attention based on bio-signals, the e-learning environment can be adapted for each student profile. The definition of an attention-aware system under the paradigm of IoT could be an available solution.

2. Learning management system

In technological learning, several buzzwords can be found. Most of them are complementary, among them are the terms: e-learning, m-learning, d-learning, and b-learning. Conceptually, e-Learning can is according to Hope et al. “the learning supported by digital electronic tools and media” [4]. Mobile learning (m-learning), is considered a sub-set of e-learning and refers to the portable electronic devices which aim to share content information [5]. Harriman [6] identifies different types of e-learning, among them, are online learning, distance learning, blended learning, and m-learning.

According to Pant & Pant [7], E-learning is “the use of computer network technology through the Internet to deliver the information to individuals”. It is a macro-concept that includes both mobile and online environments. At that level, e-Learning is directly related to the concept of Learning Management Systems (LMS), i.e. a web software application used to plan, implement and assess learning processes [8], which technologically supports an educational or learning environment.

Traditional versions of LMS described information learning in a simplified way, we are unable to describe the complex relationship between resources, teachers, students. Recently several more sophisticated LMS architectures have been proposed in literature considering both features of creating and distribution of content; and features that monitor the level of training or training.

Evale [9] proposed architecture to enhance existing LMS through the integration of educational data mining and recommendation systems. In the methodology used to develop the system, the authors considered two different models: the Fayyad knowledge discovery in databases (KDD) process model for data mining; and evolutionary prototyping specifically to develop the system. In a study entitled “A Personalized Learning Recommendation System Architecture for Learning Management”, it is proposed an effective personal learning recommendation system to support students via LMS, to enhancing the learning experience. The architecture, based on Moodle LMS, is composed of three main components, specifically: ‘learning material data source’, ‘seeking student information’, and ‘generation’. The recommendation employs a hybrid filtering technique based on educational metadata and educationally influenced filtering decisions.

In LMS platforms, the material or content can be adapted and change according to the learner’s needs, in a personalized way [10]. It allows increasing learner interest, comprehension, and success [11]. Students’ performance, has also been recently evaluated automatically in LMS using a learning analytic tool based on some input variables: total login frequency in LMS; time spent in the system; the number of downloads; interactions with peers; the number of performed exercises; and the number of forum posts [12]. The same study, performed with two courses in Moodle, with a total of 171 students, reveals that peer interaction, forum posts, and exercises have a significant impact on student’s performance. With increase in popularity of social network tools, such as Twitter similar tools have appeared on LMS.

3. Attention-aware systems

During learning, activity maintains sustained attention important to achieve successful learning. However, it is a challenge to evaluate when students maintain their attention in learning tasks. To maintain student performance in e-learning environments, have been developed attention-aware systems (AAS) with models that consider student’s attention states. AAS systems are “capable of adapting to and supporting human attentional processes especially in situations of multi-tasking, frequent interactions with other users, and highly dynamic environments” [13].

According to D’Mello [14], the attention-aware learning technologies, in which one or more types of attention are modeled, are focused on attention. Accordantly, they should not be confused with similar systems that monitor different but related states (e.g. stress, affect, etc.). The automated attention-aware systems in e-learning settings have the advantage of estimate and respond in real-time without interrupting the learner. Typically, attention-aware intelligent systems can both access the current user focus, and make predictions concerning attention shifts. In the attention management field, the goal is on capturing the user’s attentional focus, which can be built to offer personalized instruction dynamically supporting learning.

3.1 Traditional sensory-based mechanisms for attention detection in e-learning

This section is dedicated to the most recent sensory-based mechanisms concerning the attention-aware topic in e-learning. In e-learning have been used different sensory-based mechanisms for attention detection. D’Mello [15] refers to emergent technologies, in artificial intelligence in education, those related to eye-tracking and EEG devices. Eye-tracking is probably the most direct method supported by decades of scientific evidence concerning the eye-mind link [16] paradigm. While Brain-Computer Interfaces, such as those based on EEG, may complement or replace Eye tracking in the future. According to the same author, other indicators, such as physiology or gestures are undifferentiated signals that encode other information in addition to attention.

Typically, where someone is looking at is strongly associated with what him/she is paying attention to and think about [17]. Eye-tracking is the process of identifying where someone is looking with eye tracker equipment. Current research on multimedia learning has been used eye-tracking technology to study cognitive processes [18]. It allows to measure characteristics of eye movements; usually, there are two main types of measurements: fixations and saccades. The former reflects the attention process, while the latter reflects the change in the focus of visual attention [19]. Eye-tracking is considered one of the most direct and non-invasive ways of study attentional focus.

In a study entitled “Towards Automatic Real-Time Estimation of Observed Learner’s Attention using Psychophysiological and Affective Signals: The Touch-Typing Study Case” [20] an experimental study is presented, in which attention is estimated in real-time for the touch-typing task. Results revealed that multiple linear regression models were successful to discriminate between low and high levels of attention. The proposed model is based on real-time sensory data from eye and gaze movement, pupil dilatation, and affective valences of valence and arousal. It is important to notice that this method does not take advantage of saccades and fixations typical used features of eye-tracking.

Electroencephalogram (EEG), already referred to as a “window on the mind” [21] is a physiological measurement used to examine the relationship between mental and bodily processes, in this study related to attention. EEG records the electrical activity of the brain in a non-invasive way at the scalp surface, which is a result of the summed potential currents across membranes of cells. Electrodes placed at the scalp, capture the signal most of the brain regions which are near the surface. Those signals are a) the Event-Related Potentials (ERPs) b) event-related changes in EEG activity in specific frequency bands.

In a study [22], an AAS was developed to identify low and high attention of students based on a genetic algorithm for EEG feature selection, followed by the application of the Support Vector Machines (SVM) classifier. Li et al. proposed an EEG-based approach for attention recognition using k-Nearest-Neighbor Classifier (KNN) achieving an accuracy of 51.9% and 63.0% for 5-class and 3-class of attention respectively. Despite these classification rates are not high, the authors suggest use EEG along with other techniques such as pressure sensor, camera, eye tracking to have a higher accuracy rate. In a study entitled “Classification of EEG-Based Attention for Brain-Computer Interface” [23] the authors considered 4 levels of attention to be classified into different classes by an Artificial Neural Network (ANN) classifier. The accuracy, in that classification, was on average 63.5%.

Liu et al. [24] proposed a system to detect learning attention using a webcam composed of three layers: 1) image processing for face and eyes detection; 2) eyebrow region detection; 3) classifier. The system, which used SVM for classification achieved an accuracy varying between 89–93%. In a study entitled “Attention Decrease Detection Based on Video Analysis in E-Learning” [25], it is presented a scenario for analyzing individual learning attention level based on the video. It was analyzed using the OpenFace tool [26], specifically: head posture estimation, gaze focus estimation, eye movement estimation (closure and blink); mouth opening and yaw estimation; facial expression recognition. Result achieved an accuracy of 92%. Liang et al. [27] proposed a new technique to recognize human attention state using cardiac pulse from noncontact and automatic and webcam-based measurement. This approach has six different phases: 1) recording images; 2) converting images to RGB (red, green, blue) format; 3) Independent Component Analysis (ICA); 4) calculating human cardiac pulse signals using Fast Fourier Transform Algorithms (FFT); 5) featuring extraction; 6) Classification task with the algorithms: SVM, Naïve Bayes, and Gene expresser programming (GEP) based. Results revealed an accuracy of 81.82%ar in attention detection.

Artifice et al. [28], propose a methodology based on Heart Rate Variability that allows detection attention. The authors argue by hypothesis, that if we define a methodology, the authors can conduct an analysis of attention based on biosignals, then the process to determine better concentration conditions for a person can be facilitated. HRV, i.e., “the amount of heart rate fluctuations between the mean heart rate” [29], have been used to detect ECG data patterns. That variability has been studied in different target populations [30, 31]. In the field of attention, it has been proven a correlation between ECG and electroencephalogram (EEG) devices [32]. The proposed methodology for attention detection is composed of the following phases 1) pre-processing, which is dedicate to noise removal, and detection of correspondent artifacts; 2) feature extraction, refers to the extraction of HRV features, both in frequency and time domain, for further analysis; and 3) data analytics, which aims to inspect data to detect useful information that supports decision-making. A study [33], proposes an attention estimation system with modified smart glasses with inner camera for eye movement detection and, and inertial measurement for head pose position, and machine learning algorithms. Inertial measurement unit allow to acquire three-dimensional orientations, acceleration, and angular velocities. Eye tracking uses Hough transform for central point is the iris, and regions of interest allows to derive the left and right eye corners. Head pose is captured initial data from which are generated. Features, captured from eye images and perceived from IMU data are processed separately for further feature selection procedure through Sequential Floating Forward Selection (SFFS) and computed using Genetic Algorithm (GA) Support Vector Machine (SVM), in which GA optimize parameters of SVM. The system achieves an accuracy of 93.1%.

Sensory-based mechanisms for detection of user’s attention in e-learning previously mentioned are synthetized concerning goals, techniques, methods, and algorithms employed, and achieved accuracy and presented in the next table (Table 1).

Mechanism	Studies of user attention detection mechanisms in e-Learning.
	Goals	Techniques, Methods, and Algorithms	Accuracy	Ref.
Eye tracking, camera, video	Attention modeling, distinguish between low and high attention levels	Multiple linear regression model		[20]
Electroencephalogram (EEG)	Develop a neural attention-aware system (AAS) based on raw EEG signals.	Feature extraction Feature selection using genetic algorithm Support Vector Machine (SVM) classifier	90.39%	[22]
	user attention recognition	Pre-processing Feature extraction Classification: K-Nearest-Neighbor (KNN) Naïve Bayes	Higher for KNN: 51.9% 5-class attention 63.9% - 3-class attention	[34]
	Attention classification	Pre-processing Feature extraction Artificial Neural Network (ANN) Classifier	63.5%	[23]
Webcam	detect learning attention	3-layered system: image processing (face and eyes detection); eyebrow region detection classifier- SVM	89–93%	[24]
Webcam	Attention level based on the video.	OpenFace Software, capable of: head posture estimation – based on Conditional Local Neural Fields (CLFN) gaze focus estimation – using CLFN. eye movement estimation (closure and blink) – using CLFN mouth opening and yaw estimation facial expression recognition. Uses Conditional Local Neural Fields (CLFN) and Automatic detection of Facial Action Unit based on appearance and geometry features.	92%	[25, 26, 35]
webcam	Attention classification with a new approach; Methodology for recovering cardiac pulse rate from video recorders of the human face	Camera Recording Convert images to RGB (red, green, blue) format Independent Component Analysis (ICA) Calculate human cardiac pulse signals using Fast Fourier Transform Algorithm (FFT) Feature extraction Classification: SVM Naïve Bayes Gene expresser programming (GEP) based	Higher for SVM: 81.82%	[27]
Electrocardiogram (ECG)	Methodology for attention detection based on Heart Rate Variability (HRV)	ECG recording Preprocessing: Filtering + Artifact detection Feature Extraction: Artifict HRV parameters		[28]
Glasses; (with inner camera for eye movement detection and inertial measurement for head position)	Classification with support vector Machine	Smart Glasses Eye-movement detection Capturing eye images Capturing eye images Finding iris positions Generated features Head pose estimation Perceiving IMU data Generating features Performing normalization Features selections (SFFS) GA-SVM Attention assessment	91.3%	[33]

Table 1.

Sensory-based mechanisms for detection of user’s attention in e-learning.

Considering the current literature in the field, one can say that learner attention in e-learning environments can be estimated based on feature estimation methods acquired from devices as those previously mentioned (e.g. EEG and eye-tracker). Afterward, those features are used in machine learning models of attention enclosed in attention-aware systems.

However, such approaches do not have the appealing characteristics of newer generations of wireless network devices. The inclusion of those devices can disrupt traditional design principles, and thus revolutionize the interaction with the environment in an educational context.

4. Sensing

Internet of Things (IoT) architectures provide means to interconnect people, devices, and to deal with different wireless networks, which regarding its interoperability facilitate the use of smart applications [36]. The progress of mobile wireless communication has allowed to improve sensing systems. One might say that those sensing systems has been continuously adjusted to concepts of speed, technology, frequency, data capacity, framework. A promised field are future generation 6G/7G wireless network regarding its advanced characteristics, expectations. The sixth-generation wireless network enables sensing solutions with “fine range, Doppler and angular resolutions, as well as localizations to cm-level degree of accuracy” [37]. 7G is identical to 6G regarding global coverage, additionally defining satellite functions for mobile communications [38]. On one hand, “new materials, device types, reconfigurable surfaces will allow the network operations to reshape and control the electromagnetic response of the environment”. On the other hand, according to the same source, machine learning, and artificial intelligence will allow us to address the major challenges in communication systems. 6G might simultaneously provide ubiquitous communication and provide high accuracy localization and high resolution sensing services. Hight frequency bands allow fine resolution in different dimensions (range, angle, doppler). It allows both active and passive sensing. The former, active sensors emit the sounding waveforms and process echoes concerning the image doppler and angle information. While the latter, transmit natural reflection of surfaces and arrays of pictures, that represents the image. Sensing applications may exploit a vast wider channel with a bandwidth above 100 GHz [39].

Future networks, allows the combination of several materials and technologies in order to create smart innovative contexts. Intelligent Reflective Surface (IRS) [40] technology encloses an array of units, that occur modifications in the incident signal [41]. Those changes may occur in terms of phase, amplitude, frequency, or polarization. In a broad sense, IRS configures the wireless environment to facilitate transmissions between sender and the receiver [42].

Beam scanning technology, it is possible to generate images of the physical spaces, implementing systematic monitoring of the received signals using steering algorithms. Thus, we can create conditions for future “wireless reality sensing” in the university context [43]. Additionally, might be used miniaturized radars for gesture detection, smartphones, monitoring systems with bio-signals. Sensing and location might guide communication sharing mapping information between devices [37].

To date there is a scarcity of studies focused on attention, emersed on those smart environments. Would be important to add new knowledge, studying attention in innovative smart environments created with aforementioned technologies. Specifically, including precision sensing devices and considering the future wireless network generation applied to the study of attention in e-learning. There are promising devices, that regarding their characteristics might be used to study student attention. Traditional devices identified in Table 1 entitled “Sensory-based mechanisms for detection of user’s attention in e-Learning” function as a basis to identify new devices to student attention. Thus, analogous devices might be used in new wireless network generation scenarios. For instance: biosensors, webcam, electroencephalogram, Augmented Reality / Virtual Reality glasses that have recently been used to study attention. Biosensors, highly compact and wearables, have the potential to be used to provide continuous real-time physiological information through contactless measurements. One of the main advantages of such devices is the permeability to adapt to a variety of technological contexts, and its usage within the expansion of wireless communication networks. Next it is described one of such scenarios.

5. Precision sensing attention monitoring of student in e-learning (e-PSAM)

Higher Education Institutes (HEIs) can benefit from using a mix of pedagogical services, including those provided through IoT platforms, such those presented in this chapter.

In that context, it is proposed the “Precision Sensing Attention Monitoring of Student in e-learning” (e-PSAM) scenario, which is the students’ real-time sensing and monitoring, with emphasis on attention by using technology (sensors, sound, cameras). E-PSAM applies to control engineering devices that are used in order to optimize these processes.

Precision Sensing Attention Monitoring of Student in e-learning (e-PSAM) is the application of Information and Communication Technologies (ICT) in real-time to monitoring student attention. Technology, for instance, sensors (e.g. bio-signal sensors) might be used to continuously monitor the student attention and their behavior during an e-learning task. This allows for helping both students and teachers by supervising and managing their activities. Engineering is used to optimize learning management processes. The focus is on attention monitoring and management. The goal of the e-PSAM scenario is to improve students’ attention and performance, through monitoring and analysis in the e-learning environment, considering relevant parameters that have an impact on learning and health during the pandemic. e-PSAM management relates these sensing features to provide solutions to monitor, collect and evaluate processes. Figure 1 illustrates the e-PSAM scenario included in the Smart HEI (the use of smart technology in Higher Education Institutes). Inside the Smart Universities field, which” involves a conceptual modernization of all the educational processes” [44]; it is integrated the proposed approach which encloses future 6G and 7G wireless networks, IoT platforms, and related technologies, as those mentioned in previous section. On the top left it is represented the core of the scenario: a student performing an e-learning task in an smart and monitoring environment which encloses radar and intelligent reflective surfaces. The student might use Augmented Reality / Virtual Reality glasses, and another bio-sensors devices that are instruments used to monitor student attention while performing e-learning activities.

Figure 1.

Precision sensing attention monitoring of student in e-learning in the context of smart HEI.

The smart sensing application that supports attention monitoring, at the normal flow, collects and stores variables corresponding to attention measurement. The monitoring processes that will have an impact on student performance are measured electronically. The IoT platform, which supports the system, should be prepared to host the collected data from sensing. Processed information from sensing is sent back to the IoT platform and is made available to all monitoring.

The HEIs are equipped with electronic sensors supported by a new wireless communications provider. It is triggered when the e-PSAM monitoring function is activated. The HEIs stay in monitoring mode until that function is not deactivated, as can be seen in Figure 2.

Figure 2.

High-level illustration of IoT data streams and corresponding communication networks.

The aforementioned scenario, seen as integrated in a network of HEIs, might be supported by a System-of-Systems (SoS) dedicated to management of HEI data (Figure 3). The represented SoS is divided in two main components: one dedicated to the creation environments, and another related to the data analytics which is focused on the management, monitoring, and analytics functionalities.

Figure 3.

System-of-systems, IoT based data-centric HEI management.

The system is able to support different data structures, and organized in different schemas, in order to create a knowledge and formalization. Generally speaking, the system supports data volume, velocity and heterogeneity. The implementation would require a cloud platform, envisaging 6G and 7G wireless network and technology.

It would be very useful to study end-to-end performance analysis of the system through simulation in order to derive metrics.

In a broad perspective, the aforementioned “Precision Sensing Attention Monitoring of Student in eLearning” scenario might be seen as a possible technological solution to monitor student’ attention on a global 6G and 7G wireless network technological environment of HEI Management and Benchmarking trend. Thus, contributing to the comparisons of various HEIs processes and performance metrics, giving the possibility to the system to learn and support high-level decisions. In such a case, at the top level of decision, the HEI managers can control the HEIs comparing the results, accepting or not the data analysis results, comparing their results with other HEIs. Figure 4 illustrates three possible scenarios: “Precision sensing”, use case, after equipping the HEIs and learning environment with electronic sensors (step 1), the manager can monitor the HEIs (step 2), through benchmarking with other HEIs (step3).

Figure 4.

HEIs management.

The solution might benefit different stakeholders in the chain: universities, students, centers of excellence, teachers; and IoT Devices manufacturers, communication network suppliers, and IoT platforms providers. HEIs managers profit from data analytics management services since they might take decisions based on HEIs they are responsible for and benchmarking. IoT platform with acts at the level of platform collecting data from the university. Additionally, it is appropriated to monitor health conditions of the HEI population through sense which seems to be crucial in the pandemic period.

6. Conclusions and future work

Taking what was said into consideration, in the Smart Universities context, endowed with high technology, such as next generation wireless networks and connected materials, is presented e-PSAM scenario, i.e. “Precision Sensing Attention Monitoring of Student in eLearning”. Smart sensing monitoring, with focus on student attention might be monitored and managed in e-Learning context with devices, such as electrocardiogram, smart glasses, electroencephalogram.

The ideal technological environment to accomplish such achievements might be supported by IoT platforms with special emphasis on data centric management. Presented ideas will start to be experimented, implemented, tested, and validated, in the context of SHYFTE project – Build skills 4.0 through University and Enterprise Collaboration. Additionally, the project aims to implement a Center of Excellence network; bringing together academy and industry in a symbiotic relation, in order to manage productivity and human labour.

Acknowledgments

The SHYFTEproject referred by “598649-EPP-1-2018-1-FR-EPPKA2-CBHE-JP” has been funded with support fromthe EuropeanCommission. This publication reflects the views onlyof the authors, and the Commission cannot be held responsibleor any use which may be made of the information contained therein.