Elementals, A Chemistry Inclusive Serious Game

Abstract

Promoting equity and social and educational inclusion of disabled people is a priority of European Education. Disabled students often feel challenged to communicate or access information. However, with the emergence of new technologies, disabled students are offered more chances to have the same learning opportunities as others. Our current work contributes to the field of Educational Serious Game. Elementals, a chemistry-inclusive serious game, aims to assist all students in the learning process. It was designed to be inclusive for players with visual, hearing, and mobility impairments. Its purpose is to help students with chemistry and to be a complementary solution to common learning methods to enhance education. In our chapter, we will provide information about this developed game that educates, informs, entertains, and motivates chemistry students in their learning experience through a card game in which they learn the Table of Periodic Elements and use them to form molecules.

1. Introduction

Nowadays, education continues to be a challenge for deaf students, as they try to advance in their academic activities and future professional careers [1]. Back in 1965, the learning process for individuals with disabilities was challenging. However, the learning process for deaf individuals remains inherently demanding, mainly due to communication problems, emphasizing the critical need for new approaches that can help overcome barriers to teaching and learning [2]. The aim of new approaches is not only to enhance the classroom experience for these students but also to facilitate their ongoing educational journey in self-learning environments. In this context, the development of inclusive and educational tools is of utmost importance.

One such tool is “Elementals,” an innovative serious game that has been developed to address the numerous challenges faced by deaf students and promote their inclusion in educational settings. Recognizing the difficulties encountered by students in learning chemistry, Elementals offers a complementary solution to traditional teaching methods, mitigating the reluctance often associated with the intensive reading of textbooks. By employing engaging gameplay elements and a collectible card system, Elementals actively educates and informs students, fostering a deep understanding of such complex subject matter, through entertainment.

Besides being an educational tool that helps students to learn the Table of Periodic Elements, the game also serves as a catalyst for raising awareness about the learning challenges experienced by students with disabilities [1]. With an inclusive design approach, the game addresses the needs of individuals with color blindness, mobility impairments, and hearing impairments, ensuring accessibility for all students. This is made possible through the implementation of technologies such as augmented reality and georeferencing [3]. Georeferencing, as highlighted in Hackeloeer’s publication from 2014, enables precise association of geographic coordinates with spatial data. By leveraging georeferencing techniques, the game seamlessly integrates virtual elements into real-world environments, creating an immersive and inclusive gaming experience.

By addressing the challenges specific to chemistry education, the game opens doors to enhanced educational experiences and fosters equal opportunities for all students. Through ongoing research and innovation, the researchers are committed to empowering students with diverse abilities and paving the way for a more inclusive education system [1].

Following this line of thought, the chapter is organized into different sections and subsections. Firstly, the researchers will provide a summarized background/framework of their work, followed by a methodology section, where the concept and objectives of this work are outlined. In addition to these, the chapter also includes a Game Development section that focuses on the detailed process of developing Elementals. The researchers discuss the various stages of game development, including design, programming, and implementation, highlighting the key features and mechanics incorporated into the game to ensure its effectiveness as an educational tool.

Finally, the researchers included a Game Evaluation and Quality Scenario Control. In this section, the researchers delve into the evaluation and quality control aspects of the game. They discuss the evaluation methodology employed to assess the effectiveness and impact of the game. The researchers explain the use of the Quality Evaluation Framework (QEF) as a tool to measure the educational value, accessibility, and inclusivity of the game. The section presents the findings of the evaluation process, including feedback from users and the overall performance of the game in meeting its intended objectives.

By including these sections, the chapter provides a comprehensive overview of the framework of the game development, the game development process itself, and the evaluation and quality control measures undertaken to ensure the effectiveness and inclusivity of Elementals.

2. Framework: the use of serious games in education

Access to equal opportunities and social inclusion for disabled individuals is a major concern in modern society and a key issue within European education. In response to this, significant efforts are being made to enhance education through the implementation of educational and social inclusion initiatives. The concept of inclusion, in this context, aims on ensuring that all students, regardless of their abilities, can actively participate and thrive in mainstream educational settings. Achieving this goal requires comprehensive changes in pedagogies within the education system [4].

Addressing these challenges requires the incorporation of new technologies to drive positive transformations in educational settings. These technologies can introduce valuable opportunities for social inclusion, as part of a holistic approach. It becomes crucial not only to improve the education system and the overall educational settings but also to promote the utilization of new technologies to assist students in their studies and self-learning environments.

In this regard, the development of Educational Serious Games has emerged as a highly promising and innovative approach to enhancing education and inclusion. These serious games are specifically designed with the primary goal of education in mind, setting them apart from conventional entertainment-focused games. They are carefully crafted to serve an educational purpose, aiming to develop knowledge, skills, or specific learning outcomes. Throughout the last two decades, there has been a significant emphasis on developing educational games that not only serve the primary purpose of enhancing knowledge but also aim to be engaging and entertaining. These games have been designed with the intention of captivating players and fostering motivation, thereby promoting active participation [5, 6, 7].

An essential aspect of serious games is their ability to align with their educational purpose, guiding players toward meaningful learning experiences. They offer distinct and effective mechanisms for learning and assessment, which set them apart from traditional instructional methods [2].

The relationship between game development, computer science, and disabled people’s education is a powerful and transformative one. Through the convergence of these fields, innovative solutions can be created to address the unique challenges faced by individuals with disabilities, ultimately fostering inclusivity and equal access to education. Let us explore this relationship in more detail:

1. The Role of Game Development: Game development plays a pivotal role in shaping educational experiences, especially for students with disabilities. By leveraging interactive and immersive gameplay, game developers can create engaging environments that capture the attention and interest of learners. Games provide a safe and enjoyable space for individuals to explore complex concepts, experiment with different approaches, and reinforce their understanding through active participation. The integration of educational content within games can transform traditional learning into a dynamic and accessible experience.

2. The Intersection with Computer Science: Computer science serves as the backbone of game development, providing the technical foundation for creating interactive experiences. In the context of education and disability empowerment, computer science brings forth powerful tools and technologies that can be harnessed to enhance accessibility and inclusivity in games. From adaptive control systems and assistive technologies to data analysis and machine learning algorithms, computer science empowers game developers to create tailored solutions that accommodate diverse learning needs and overcome barriers for disabled individuals.

3. Empowering Disabled People through Education: Education is a fundamental right, and it is crucial to ensure that individuals with disabilities have equal access to quality educational opportunities. Game development, driven by computer science advancements, can significantly contribute to empowering disabled people in education. By incorporating accessibility features and adaptive technologies, games can provide tailored experiences that cater to specific disabilities, allowing individuals to actively engage and participate in the learning process. This inclusion not only facilitates knowledge acquisition but also fosters confidence, self-expression, and social interaction.

The relationship between game development, computer science, and disabled people’s education goes beyond just creating accessible games. It opens doors to new possibilities, empowering individuals to overcome limitations, develop critical skills, and thrive in various educational domains. Through the use of adaptive technologies, such as brain-computer interfaces or augmented reality, games can provide unique learning experiences that bridge gaps and promote independence for disabled learners. Moreover, the collaboration between game developers, computer scientists, and educators can lead to continuous innovation, driving the evolution of educational approaches that prioritize inclusivity and personalized learning.

By embracing educational serious games and leveraging their potential, we can advance educational and social inclusion by providing interactive and effective learning experiences for all students. These games serve as a bridge between education and entertainment, offering a powerful tool for engagement and knowledge acquisition [6, 7]. Through their thoughtful design and purposeful implementation, serious games contribute to creating an inclusive and enriching educational environment for both disabled and non-disabled students alike.

3. Methodology

The Elementals game is designed to enhance the learning process for chemistry students, providing them with valuable support that complements common teaching and learning methods. Its overarching purpose is to be educational and inclusive. With a strong focus on accessibility and social inclusion, the Elementals game strives to ensure that students and players with disabilities can actively engage with the content and participate in the learning experience. The Elementals game not only enriches the educational journey of junior and high-school chemistry students but also emphasizes the importance of equal opportunities for all learners.

Through its carefully crafted methodology and project outcomes, the Elementals game makes a significant contribution to the advancement of inclusiveness in education. Integrating accessible features in response to the needs of different learners creates an environment that fosters engagement, comprehension, and knowledge acquisition.

3.1 Concept

The Elementals project sets out to create an immersive and educational serious game that revolves around the concept of collecting cards. The game will offer multiplayer support, enabling players to engage and compete with one another, and it will leverage cutting-edge technologies such as augmented reality and geolocation.

The success of the game will be measured by its impact on academic performance, as it aims to improve learning outcomes through the application of gamification principles and a less intimidating approach to the study of chemistry. The mechanisms and features developed for the game will be carefully crafted to ensure inclusivity, allowing students with disabilities to effortlessly engage with the game.

At the core of the game’s objective is the collection of molecule cards, with players striving to increase their ranking by accumulating a diverse range of cards. To obtain these molecule cards, players must first gather atom cards that are dispersed across real-world map locations. Leveraging their chemistry knowledge and skills, players strategically select and combine atom cards to construct unique and valuable molecule cards, which are then added to their ever-expanding collection.

To assess the presence of any disabilities, the game begins with a puzzle-based evaluation. This assessment includes challenges designed to determine color blindness, featuring squares of varying sizes and colors, as well as options for players to select their preferred communication mode, such as written or sign language, to identify any hearing impairments.

The game takes players on an exploration journey starting from their current geographic location, searching for atom cards scattered throughout the real-world map [3]. The level of engagement and success in finding hidden atoms depends on the players’ determination and their familiarity with gameplay mechanics. Once close to an atom card, players can activate the augmented reality feature to view the atom directly within their surroundings and add it to their card collection. Additionally, players can strategically combine sets of atom cards to create unique and powerful molecule cards, earning valuable score points that contribute to their leaderboard ranking. This leaderboard serves as a representation of each player’s progress and learning achievements.

To provide ongoing motivation and rewards, the game presents daily objectives to players based on their ranking. By completing these objectives, players unlock rewards and advance their user level. As players ascend to higher levels, they gain access to crafting more complex molecules that were previously beyond their reach. This progressive system not only enhances the learning process but also encourages continuous skill development and deepens their understanding of chemistry.

The Elementals project strives to revolutionize the educational gaming landscape by offering an inclusive and captivating experience that seamlessly integrates learning and entertainment.

3.2 Objectives

Elementals is a groundbreaking game that has been designed to revolutionize education and ensure that every student has access to equal opportunities. It seeks to achieve two main objectives: developing an educational component focused on the field of chemistry and fostering inclusivity for players with disabilities, including color blindness, and mobility and hearing impairments. By incorporating innovative features and adaptations, Elementals empowers individuals with disabilities to fully participate in the game. Thus, the game aims to incorporate the following specific objectives:

1. Educational component: The game offers a unique approach to learning chemistry by complementing traditional schoolbooks with a card-collecting system. This engaging gameplay mechanic encourages students to actively explore and acquire knowledge about chemical elements and compounds. By collecting and organizing cards, players gain a deeper understanding of the subject matter and develop essential skills.

2. Inclusivity for players with disabilities: One of the key aspects of Elementals is its commitment to allowing players to configure the game according to their specific needs. Importantly, this configuration is entirely optional and not compulsory, ensuring that players can choose to customize the game to their preferences. This empowers individuals to create an experience that aligns with their unique requirements and learning styles.

3. Inclusion of color-blind players: To promote the inclusion of color-blind players, Elementals incorporates feedback mechanisms that go beyond traditional color-based cues. Instead, the game utilizes well-defined and easily understandable symbols that provide clear and unambiguous information. By relying on these symbols, color-blind players can fully comprehend and interact with the game’s elements without facing any disadvantages.

4. Inclusion of deaf players: Elementals embraces the inclusion of deaf players by providing feedback that caters to their specific needs. This involves the use of sign language to express simple messages, which are then translated into sequences of sign language characters within the game. By incorporating sign language, the game ensures effective communication and enables deaf players to fully engage with the gameplay experience.

5. Inclusion of players with mobility impairments: Elementals is dedicated to supporting the inclusion of players with mobility impairments. It achieves this by integrating an EPOC device, which enables alternative input methods. These methods may include mimicking actions, utilizing shortcuts, or employing pushbuttons to interact with the game. By providing full user accessibility, the game ensures that players with mobility limitations can navigate through the game world and participate in all aspects of gameplay.

Overall, Elementals represents a groundbreaking approach to education and inclusivity. By focusing on chemistry and incorporating features specifically tailored to individuals with disabilities, the game breaks down barriers and promotes equal opportunities for all players. Elementals empowers students to actively learn, raises awareness of learning challenges, and fosters a sense of inclusion and accessibility within the gaming community.

4. Game development

The game architecture of Elementals comprises six primary components, each playing a crucial role in delivering a cohesive and immersive gameplay experience. These components work together seamlessly, leveraging cutting-edge technologies to achieve the game’s objectives and provide an engaging learning environment for players.

4.1 Game architecture

The Elementals game encompasses six primary components (Figure 1) that perform together to create a cohesive and immersive experience. These components are carefully designed to fulfill specific functionalities and contribute to the overall game objectives. These components are as follows:

1. Mobile Application Component: The Mobile Application Component serves as the user-facing interface of the game. It enables users to interact with the game and access various features and functionalities. This component establishes communication with the backend component to exchange data and trigger actions. The backend component provides a set of services that the mobile application can consume.

2. Backend Component: The Backend Component consists of a Node server hosted on the Azure platform. It acts as an intermediary between the Mobile Application component and the other game components. This component offers a well-defined interface, exposing a range of services that the mobile application can utilize. Through this backend component, the mobile application can access and manipulate data, perform calculations, and interact with the wider system.

3. Database Component: The Database Component, implemented using MongoDB, plays a vital role in storing and retrieving essential information required by the game. The Backend Component communicates with the Database Component to fetch the necessary data. MongoDB, a flexible and scalable NoSQL database, provides efficient storage and retrieval mechanisms, ensuring the game’s smooth functioning.

4. MapBox Component: The MapBox Component configures and generates the world map, incorporating real coordinates and serving as a foundational layer for realism and immersion. This component enhances the gameplay experience by providing a dynamic and interactive environment where players can explore, search for atoms, and encounter other players. The MapBox component contributes to the game’s spatial awareness and location-based features.

5. Emotive EPOC (Brain-Computer Interfaces) Component: The Emotive EPOC Component serves as an interface specifically designed to cater to individuals who may have physical movement limitations. It allows users to interact with the application using inclusive inputs based on brain-computer interfaces. This component ensures that the game remains accessible to a wider range of users, promoting inclusivity and equal participation.

6. Augmented Reality Framework Component: The Augmented Reality (AR) Framework is a crucial component that brings an immersive experience to the game. It enables players to collect atoms in a visually captivating and interactive manner. By leveraging AR technology, this component overlays virtual elements onto the real world, enhancing the gameplay and fostering a deeper sense of engagement.

Figure 1.

System component diagram.

Each of these components within the Elementals game serves a distinct purpose and contributes to its overall functionality and user experience. The seamless integration and collaboration among these components create a cohesive and compelling environment for players to learn and enjoy the game.

4.2 Special technical requirement

For this game, the researchers applied a range of technologies and tools to meet the technical requirements. These include:

1. Unity 3D: The researchers utilized the Unity 3D engine as the foundation for developing the game. Unity 3D offers a comprehensive set of tools and features for creating interactive and immersive experiences.

2. Visual Studio Code: This code editor was employed by the researchers to write and edit the game’s source code. Visual Studio Code provides a lightweight and versatile environment with support for various programming languages.

3. Visual Studio Community Edition 2019: The researchers used this integrated development environment (IDE) for building and debugging the game. Visual Studio Community Edition offers a rich set of features for software development.

4. Map Box—Georeferencing Services: Map Box’s georeferencing services were incorporated by the researchers to enable location-based functionalities within the game. These services provide access to accurate and up-to-date mapping data.

5. Photon: Photon was used for implementing multiplayer networking capabilities in the game. This framework facilitates real-time communication and synchronization between multiple users.

6. Blender for 3D modeling: The researchers employed Blender, a powerful open-source software, for creating and editing 3D models used in the game. Blender offers a wide range of tools and functionalities for modeling, texturing, and animating objects.

7. REST Services for server communication: REST (Representational State Transfer) services were utilized by the researchers for server communication. RESTful Application Programming Interfaces (APIs) enable efficient and standardized communication between the game’s client-side and server-side components.

8. Server and Database: The researchers set up a server infrastructure to handle data storage and processing requirements. Additionally, a database system was employed to store and manage game-related data efficiently.

9. Microsoft Paint 3D: Microsoft Paint 3D was used by the researchers as a supplementary tool for creating or modifying 3D assets. This software allows for basic 3D modeling and texture editing.

10. InkScape: InkScape, a vector graphics editor, was employed by the researchers for creating and manipulating scalable 2D graphics. It provides a comprehensive set of tools for designing and editing vector-based artwork.

11. EPOC Unity Plugin: The EPOC Unity Plugin was integrated into the game by the researchers to interface with the EPOC Emotiv headset. This plugin allows for capturing and processing data from the EPOC Emotiv headset, enabling brain-computer interface functionality.

12. Emotiv App—EPOC Emulator: The Emotiv App was employed as an emulator for the EPOC Emotiv headset. This software simulates the functionality of the physical headset, enabling development and testing without the actual hardware.

13. Vuforia—Augmented Reality SDK: Vuforia, an Augmented Reality Software Development Kit (SDK), was selected to implement augmented reality features within the game. It provides tools and APIs for marker-based and markerless AR experiences.

By incorporating these technologies and tools, the researchers ensured that the game was equipped with the necessary features and functionalities to deliver a dynamic, inclusive, accessible, and engaging user experience.

4.3 Game engine

The researchers selected Unity 3D as the game engine for this game, taking into consideration several key factors. Firstly, Unity 3D was chosen because it is designed to be multiplatform, meaning that the game can be developed and deployed on various operating systems and devices. This multiplatform capability ensures a wider reach and accessibility for players.

Additionally, Unity 3D offers an integrated development environment (IDE), which provides a seamless and efficient workflow for game development. The IDE includes a comprehensive set of tools and features that streamline the development process, allowing the researchers to focus on creating engaging gameplay experiences. Furthermore, Unity 3D comes with an extensive asset library, providing a rich collection of pre-built game objects, resources, and visual effects. This asset library significantly enhances the development speed and efficiency, as it offers a wide variety of ready-to-use components.

Unity 3D has gained immense popularity within the gaming community in recent years. It has become one of the most widely adopted game engines due to its versatility and ease of use. Developers appreciate its flexibility and adaptability, enabling them to create games for different genres and platforms with relative ease. Moreover, Unity 3D has an active and vibrant community that actively contributes to its growth and provides valuable resources, tutorials, and support.

One of the significant advantages of Unity 3D is that it is an open-source engine, which means that it is freely available for developers to use and modify. This open-source nature encourages collaboration, innovation, and the sharing of knowledge among developers. Additionally, the free-to-use model makes Unity 3D accessible to developers of all backgrounds, irrespective of their budgetary constraints.

Considering these factors, the researchers employed Unity 3D as the game engine for this game, leveraging its multiplatform capabilities, integrated development environment, extensive asset library, widespread popularity, versatility, user-friendly interface, and open-source nature to meet the requirements of the game.

4.4 Rendering

Blender’s 3D modeling engine played a crucial role in the development and design of various components within the game. Specifically, the researchers utilized Blender to create and design the Atom models, Molecule models, Cards, and other objects featured in the game.

Blender, a powerful and versatile software, provided the researchers with a comprehensive set of tools and functionalities to bring these objects to life in a three-dimensional space. The Atom models, representing the building blocks of matter, were meticulously crafted using Blender’s modeling tools, allowing for precise detailing and accurate representation.

Similarly, the Molecule models, which showcase the complex structures and interactions of chemical compounds, were developed using Blender’s advanced features. The researchers leveraged Blender’s capabilities to accurately depict the intricate bonds, atoms, and molecular arrangements, ensuring a visually compelling and scientifically accurate representation.

In addition to the Atom and Molecule models, Blender was instrumental in designing other objects essential to the game’s gameplay and visual esthetics. Cards, for instance, were crafted using Blender’s 3D modeling engine to create distinct and visually appealing playing cards with intricate designs, textures, and animations.

Blender’s extensive capabilities in 3D modeling, texturing, and animation enabled the researchers to unleash their creativity and bring their vision to life. The software’s intuitive interface and the vast array of tools facilitated the creation of high-quality and realistic objects, enhancing the overall immersion and visual fidelity of the game.

By leveraging Blender’s powerful 3D modeling engine, the researchers were able to develop visually captivating and engaging components such as Atom models, Molecule models, Cards, and various other objects.

4.5 Gameplay objects

The gameplay objects in this game were divided into two distinct major components, namely menus for navigation and a game canvas where the actual card collection levels are played. The separation of menus and the game canvas was carefully implemented, following the guidelines provided in Unity’s documentation. This approach aimed to create a structured pattern that efficiently isolates different layers of the game, facilitating organization and maintenance.

To achieve this separation, the development process involved creating the menus and game canvas’ as separate entities. The menus component focused on designing and implementing the user interface elements responsible for navigation, such as main menus, options menus, and level selection screens. These menus were developed following the recommended Unity documentation guidelines, ensuring consistency and usability.

On the other hand, the game canvas component encompassed the actual gameplay levels where players engage in card collection. The game canvas was implemented as a distinct area within the game, dedicated to providing an immersive and interactive experience. The development of the game canvas followed Unity’s recommended practices, taking advantage of the available tools and features to create engaging and challenging levels.

To further streamline the development process, the game’s development components were organized into a structure. This structure encompassed the following elements:

1. Game actions: This component involved defining the various actions and interactions that players could perform within the game. Examples include card selection, shuffling, and scoring mechanisms. The game actions component served as the foundation for implementing the gameplay mechanics.

2. Game action scripts: These scripts were responsible for implementing the specific functionalities associated with each game action. By utilizing scripts, the researchers could define the behavior and logic behind the gameplay mechanics, ensuring smooth and accurate execution.

3. Graphic Resources: This component encompassed the visual assets used in the game, including textures, sprites, animations, and other graphical elements. The graphic resources component played a vital role in creating visually appealing and engaging gameplay experiences.

4. Graphic Assets: This component involved the creation and management of graphic assets, such as card designs, backgrounds, and visual effects. The researchers carefully crafted these assets to enhance the esthetics of the game and provide an immersive visual experience.

5. Game Engine controllers: The game engine controllers served as the bridge between the various development components and the underlying Unity game engine. These controllers were responsible for managing the flow of the game, handling input events, managing game states, and ensuring proper integration between different elements of the game.

By organizing the development components into this structure, the researchers ensured a systematic and efficient approach to the game’s implementation.

4.6 Game data storage and tracking

The gameplay data, specifically the scores and ranking levels of each player, were stored and managed in a dedicated Servers Database deployed on the Microsoft Azure platform. This cloud-based database solution provided a reliable and scalable infrastructure for securely storing and retrieving game-related information.

To facilitate seamless communication between the game user and the server database, a REST (Representational State Transfer) API was developed. This API utilized HTTP methods to enable the game client to interact with the server and access the scoring information. Two primary HTTP methods were employed for this purpose:

1. POST scores: The game client utilized the REST API’s POST method to send player scores to the server. When a player completed a level or achieved a significant milestone, the game client would initiate a POST request to the server, transmitting the score data. This data was then stored securely in the server’s database, ensuring the persistence of each player’s score.

2. GET scores: The REST API’s GET method was utilized by the game client to retrieve scores from the server. When required, the game client would initiate a GET request to the server, which would respond with the requested score information. This allowed the game client to fetch and display scores, rankings, or any other relevant gameplay data for the players.

By implementing the REST API with these HTTP methods, the game user could seamlessly communicate with the server database, securely transmitting and retrieving the scoring information. This integration enabled players to persistently store their scores, track their progress, and compare their rankings with other players.

The choice to deploy the Servers Database in Microsoft Azure provided several benefits. Azure’s cloud infrastructure ensured high availability, scalability, and reliability, allowing for efficient storage and retrieval of gameplay data. Additionally, Microsoft Azure’s robust security measures protected the integrity and confidentiality of the stored information, ensuring that player scores and ranking data remained secure.

4.7 Development lifecycle

The game was structured with distinct development lifecycle phases, which are outlined in Table 1 for easy reference and understanding.

Phase	Process
Alpha	Alpha tests assure the first verification to confirm if the game’s overall goals are met and that the game is effectively evolving into the predefined scope. These tests are performed by the project team developers.
Beta	Beta tests guarantee the first user experience of the gameplay, the feedback in terms of usability, playability, and the learning experience. These tests are by randomly selected users.
RC	Release candidate tests focus mainly on performance and heavy load tests. Once these tests are passed with success, assuring a real-life scenario load environment, the release candidate is cleared to be deployed or streamed to market.
RTM	The Release to the Market version of the game is the ultimate revised version in terms of bug fixing, with the highest degree of quality assurance. From this point on only bug fixing from end-user input is performed.

Table 1.

Game development lifecycle.

5. Game evaluation and quality scenario control

The evaluation of the project is conducted in accordance with recognized quality standards, specifically ISO 9126, which provides guidelines for assessing software quality, and SCORM, a set of standards for developing learning content. These standards ensure that the evaluation process follows established principles and best practices in the field.

To facilitate the evaluation process and ensure its objectivity and reliability, a quantitative evaluation framework called the Quantitative Evaluation Framework (QEF) is utilized. The QEF offers a systematic and structured approach to measuring the quality of the system in a quantitative manner. It operates at three levels: dimensions, factors, and requirements. Each dimension encompasses a collection of factors, while each factor consists of a set of specific requirements [8, 9, 10].

This work extends the foundational research conducted by Escudeiro et al. from 2008 to 2013 on QEF. By building upon their contributions, the researchers have further developed, adapted, and updated the framework to meet the evolving needs of evaluating educational serious games, demonstrating its ongoing significance in evaluating educational serious games [11, 12].

By employing QEF, the evaluation process becomes systematic and standardized. It enables the researchers to assess the quality of the system and to measurably evaluate outcomes, providing valuable insights into the system’s performance.

5.1 QEF: dimensions, factors, and requirements

The quality scenario of the Elementals game is evaluated using QEF, which follows the standards set by ISO 9126 for quality assessment. The framework considers three key dimensions: Functionality, Adaptability, and Usability [8, 9, 10].

The Functionality dimension, which is vital for assessing the overall effectiveness of the game, consists of seven specific factors: gameplay, trading, collection, user interaction, features, account, and game engine. Each of these factors contributes to the overall functionality and performance of the game. Evaluating gameplay ensures that the game mechanics and interactions provide an engaging and enjoyable experience. Trading and collection assess the game’s ability to facilitate the exchange and acquisition of cards. User interaction focuses on how players engage with the game interface and controls. Features encompass additional functionalities and enhancements that enrich the gameplay experience. Account evaluates the management and security of user accounts, while game engine examines the technical aspects and performance of the game’s underlying system.

The Adaptability dimension assesses the game’s ability to adapt to new conditions and requirements. This dimension consists of factors such as gameplay, versatility, accessibility, and maintenance. Evaluating gameplay adaptability involves determining whether the game can effectively adjust its mechanics and challenges to accommodate different player levels and preferences. Versatility assesses the game’s flexibility in providing diverse gameplay options and modes. Accessibility examines how well the game caters to players with disabilities, ensuring they can fully participate and enjoy the experience. Maintenance evaluates the game’s ability to receive updates, bug fixes, and improvements to ensure its continued functionality and relevance.

Usability, another crucial dimension, focuses on how well the game can be used by players to achieve their goals with effectiveness, efficiency, and satisfaction, as outlined by ISO 9241 standards. It incorporates factors such as menu navigation, map navigation, content quality, and integrity. Evaluating menu navigation involves assessing the intuitiveness and ease of navigating through the game’s menus and settings. Map navigation examines the clarity and ease of navigating the game’s virtual environment. Content quality ensures that the educational content, including cards and information, is accurate, informative, and engaging. Integrity focuses on the consistency and reliability of game data and ensures that players can trust the information provided by the game.

To evaluate each requirement within the dimensions, the QEF framework utilizes a discretized approach with predefined values. These values vary depending on the specific requirement. Some requirements are assessed using a binary approach, where they are either fulfilled (100%) or not fulfilled (0%). Others are evaluated using a Likert scale ranging from 1 to 5, allowing for a more nuanced assessment. Additionally, certain requirements follow a three-level threshold (0%, 50%, and 100%) or a five-level threshold (0%, 25%, 50%, 75%, and 100%) structure, which provides a graded evaluation based on specific criteria [8, 9, 10].

To ensure clarity and consistency during the evaluation process, all levels and thresholds for each requirement are clearly defined.

5.2 Evaluation methodology

The evaluation methodology employed by the project team involved assessing the product under development at various stages during the game’s development process. The QEF provided a clear and quantitative perspective on the product’s quality at any given stage.

This section outlines the approach to testing and data collection for the game Elementals, with a specific focus on its target population, including individuals with color blindness, and mobility and hearing impairments.

5.2.1 Technical testing

Technical testing is essential for ensuring product quality and goes beyond end-user testing. It provides insights into the system’s capabilities, limitations, and performance issues, allowing necessary fixes to be made. The game specifications, including mandatory requirements and recommended specifications, are analyzed using the QEF framework to assess their completion and alignment with the project plan.

Performance testing involves evaluating the game on different devices with varying operating systems to assess compatibility and measure performance metrics. The goal is to identify any potential frame rate drops during gameplay.

5.2.2 Play testing

Play testing is a crucial process in game development where designers thoroughly test the game to identify and address bugs and design flaws before its release. Different types of playtest can be conducted. In this case, a playtest without external participants was chosen.

During the playtest, the researchers assess the game’s usability by measuring how quickly users become familiar with the interface and navigate through the game. Feedback on game balance, mechanics (e.g., card picking, available options), and necessary improvements will be collected. The accuracy of the Dynamic Difficulty Adjustment (DDA) system will also be evaluated to ensure that the game’s difficulty matches players’ skills, regardless of disabilities. Analyzing the game’s flow from early development stages helps streamline the process of fixing game mechanics.

5.2.3 Focus testing

Focus testing involves testing the product with a focus group to gain insights into people’s feelings and opinions. In this type of testing, the group examines mockups and color schemes without actively using the product, understanding that their responses may change once they interact with the actual game.

The primary objective of focus testing is to evaluate the game’s inclusivity for players with color blindness, hearing impairments, and mobility disabilities. The focus tests conducted include different testers:

• Color-blind: Testing the color schema with color-blind users to ensure that this impairment does not impact gameplay.

• Hearing-impaired: Testing the correctness and appropriateness of sign language used in the game for individuals with hearing impairments. It is essential to verify that sign language does not impact gameplay and is easily understandable.

• Movement impaired: Testing the user interface (UI) to assess its ease of use for players using the EPOC device or individuals with movement impairments. This test aims to verify if players with movement impairments can play the game without their disabilities significantly affecting gameplay.

5.2.4 Usability testing

Usability testing involves observing how individuals interact with the game and adapt to it if needed. Unlike focus testing, this method allows for the assignment of specific tasks and the collection of quantitative data.

During the initial interaction with the game, the researchers analyze players’ responses to the interface, available options, gameplay, and the ease and efficiency of navigation. This testing phase aims to gather objective data on players’ interactions with the game.

5.2.5 Data collection

The evaluation process relied on questionnaires and interviews as the primary methods for data collection. The choice of method depended on the limitations of the testers, with interviews being preferred for individuals with movement impairments and hearing impairments.

Questionnaires maintained consistency across different formats, whether written or verbal, and covered aspects such as user experience, graphics, gameplay, color schemes, and sign language adaptation.

5.3 Evaluation results

5.3.1 Alpha test analysis report

This section presents the findings of the analysis conducted on the Alpha Tests, providing an in-depth examination of the results to identify possible areas for improvement and assess the current state of the game.

Due to the previous Covid-19 restrictions, the researchers opted for an online questionnaire using the Microsoft Forms platform, facilitating both data collection and analysis. The Alpha Test included participation from 11 members of the development team, all of whom had no hearing impairments. However, it is worth noting that one question remained unanswered and unanalyzed as a result.

Overall, the evaluations for the questions were positive, highlighting aspects such as the ease of starting the game, text quality, game stability, interface design, avatar location, and ranking presentation. Despite the favorable feedback, there is still room for improvement, and certain considerations should be considered to achieve a more comprehensive assessment.

Specific improvements are related to the user interface, aiming to enhance navigation within the application. Conducting additional tests to identify potential errors and evaluate the game’s performance is advisable. Furthermore, conducting a thorough review of the game’s complete content, including text messages, color utilization, and other elements, is highly recommended. Lastly, assessing the effectiveness of the DDA as implemented in the game is crucial.

5.3.2 Beta tests analysis report

This section presents the findings of the analysis conducted on the Beta Tests, providing an in-depth examination of the results to identify possible areas for improvement and assess the current state of the game. Additionally, an evaluation based on the System Usability Scale (SUS) will be provided.

Similar to the alpha test, and to overcome Covid-19 restrictions, the researchers conducted an online questionnaire using the Microsoft Forms platform, facilitating both data collection and analysis. Once the questionnaire was distributed to the participants, they were instructed to play the game and respond to a predetermined set of questions. To ensure clear and consistent game testing, a comprehensive user guide was provided.

The Beta Test involved the participation of 11 subjects, all of whom had no hearing impairments. However, it should be noted that one question remained unanswered and unanalyzed due to this circumstance.

The evaluations for the questions were positive, with the notable aspects such as the ease of game initiation, text quality, game stability, interface design, avatar location, and ranking presentation receiving praise.

By employing the SUS analysis methodology, it can be concluded that the game has reached a usable and stable phase. Moreover, it is evident that all test subjects thoroughly enjoyed the game experience.

6. Conclusions

Education plays a crucial role in shaping the future of students, but it can often present unique challenges for those with disabilities. As such, the researchers aim to address these challenges by developing innovative educational approaches that ensure equity for all students. The researchers’ primary focus revolves around the development of Educational Serious Games, which offer an inclusive and effective approach to delivering education. Specifically, their work contributes to this field through the creation of Elementals, an inclusive and educational game.

Elementals are designed to actively engage students and serve as a complementary solution to the traditional teaching and learning process. By harnessing the power of technology, the game promotes social inclusion and accessibility in educational settings, providing students with access to educational information and enhancing self-learning environments. The game particularly addresses the learning challenges in chemistry, with the potential to extend its scope to other subjects in engineering and electrical engineering education.

In future work, Elementals has the potential to expand its reach to encompass a broader range of subjects within the field of engineering and electrical engineering education. Through continuous development and improvement of the game, the researchers strive to enhance students’ educational experiences and foster their learning across multiple disciplines.

Acknowledgments

This work is being developed at the research group GILT – Games, Interaction and Learning Technologies from the Polytechnic Institute of Porto, with the support of FEDER – Fundo Europeu de Desenvolvimento Regional through the Sistema de Incentivos à Investigação e Desenvolvimento Tecnológico – I&D Empresas, under the frame of Portugal 2020, project number POCI-01-0247-FEDER-069949, and the Erasmus+ Programme, European Education and Culture Executive Agency, through project InSign - Advancing inclusive education through International Sign, project number 2019-1-DE01-KA203-004964.

Conflict of interest

The authors declare no conflict of interest.