Open access peer-reviewed chapter

Study of the Assessment Criteria on e-Learning Websites

Written By

Kuei-Chih K.C. Chuang and Mei Chuan Tsai

Submitted: 23 March 2015 Reviewed: 16 June 2015 Published: 21 October 2015

DOI: 10.5772/61062

From the Edited Volume

E-Learning - Instructional Design, Organizational Strategy and Management

Edited by Boyka Gradinarova

Chapter metrics overview

1,412 Chapter Downloads

View Full Metrics

Abstract

This study aimed at exploring and discussing cognizance of teachers and students toward construction of e-learning websites. It was evaluated and it developed five assessment indexes required as “Assessment Guidelines on e-Learning Websites.” These five assessment indexes were “teaching material and the structure,” “layout design,” “interface design,” “interaction design,” and “establishment of system configuration.” The development of survey questionnaires was based on the above five assessment indexes as well. In this study, several goals are achieved, for example, teaching resources could be augmented, quality of Web-based instruction could be improved, learners’ time and efforts in Web-based learning could be saved, and effects of Web-based teaching and learning could be highlighted. In the end, a digital Taiwan can become possible when the Web-based instructions follow the assessment guidelines and prevail over the aggressive competition.

Keywords

  • Assessment guidelines of an instructional website
  • Internet
  • Instructional websites
  • Internet server end
  • Assessment guidelines

1. Introduction

The twenty-first century has been now called the Internet Age which is greatly affected by the various computerized multimedia technologies, such as the Internet. With the wide range of interactive computer and Web technologies, it tends to produce more effective methods of transferring skills and knowledge than the traditional lecture-style approaches. For example, the professors can integrate notes, graphics, diagrams, full-motion video segments, audio segments, and hyper link texts and materials into a comprehensive website as a cognitive and motivational tool to facilitate teaching and learning. In addition, the students can acquire and construct knowledge and skills by accessing their learning performance and take part in discussing with instant feedbacks [1].

In the way of online learning theory and application, teachers and students can get advantages from flexible teaching and learning, and make much progress step by step. Teachers easily get the feedback by students’ on-line questions and reply the answers to students as soon as possible. Besides, setting up the instructional websites from teachers by professional knowledge and skills makes effective responses for the demands by students. Eventually, teachers and students get the win-win advantages for each other from such a valid and cognitive system. However, educational curriculum designers and faculty face great challenges and potentials by new teaching experiences. More and more researches have focused on teaching and learning characteristics, learning styles, and interactions between teachers and students about Web-based instructional setting. Nowadays, it has become the greatest concern for all the teachers on what the criteria should be and how to design and evaluate an effective and interactive Web-based instructional setting. We have seen in the cases presented that clearly articulated assessment strategies are vital to the effective design of online courses and programs. The peer assessment case demonstrates a program-level solution to the need to provide a tool that assesses professional skills in group-level and individual-level performance within an online context. McCracken, Cho, Sharif, Wilson, and Miller [2] made a conclusion that “it presented that clearly articulated assessment strategies are vital to the effective design of online courses and programs. The peer assessment case demonstrates a program-level solution to the need to provide a tool that assesses professional skills in group-level and individual-level performance within an online context.” Therefore, the more requirement for e-learning environments grows in higher education, the more needs for website application of learning and educational assessment strategy theory to design, develop, and deliver in e-learning environments.

Graff [3] pointed out that “evaluation feedback from participants indicated that each online task was rated positively.” And Smith [4] made a comment that “it is suggested that as instructors make the transition from traditional to blended/online instruction, they consider jettisoning the traditional essay requirement and replace it with some form of ‘assignment essay/peer review’ system such as the one described. Contemporary Learning Management Systems facilitate peer review and peer assessment approaches in ways that were not available in traditional offline education.” Somehow, doing the assessment study should consider that there are numerous limitations on how to learn performance that is evaluated by assessment theories and skills. The patterns as online testing show are usually in the form of multiple choice questions without any essay type of learning assessment. Most of the reasons for offering multiple choice tasks in e-learning are for the sake of ease of implementation and ease of managing learners’ replies.

The study purposes were to establish a set of criteria for assessing instructional websites by using instructors’ and students’ perception of current experiences in instructional websites specifically for undergraduate- and graduate- level courses in order to meet the following objectives: collecting and understanding the settings and operating rules of the present instructional websites, analyzing and developing the updated policies of the instructional websites, exploring the users’ and designers’ perceptions, and providing tips for building an effective instructional website. Web evaluation criteria include ideas for incorporating Web evaluation into the curricula that promote information literacy. In particular, we intended to explore five-section criteria related to the website evaluation. The five criteria were (1) website material development, (2) website graphic design, (3) website interface, (4) website communication interaction, and (5) website system.

The study measures included collecting and reviewing reference papers related to instructional websites, analyzing and inducting the websites’ advantages and disadvantages, interviewing experts, and analyzing evaluation of e-learning website. During the study period, the research targets covered 33 instructors who established their own instructional websites at the first semester of the academic year 2013 at National Yunlin University of Science and Technology, 35 teachers who participated in the teacher workshop of research methods for teachers of science and technology in south Taiwan, and 240 college students of the National Yunlin University of Science and Technology. From the participants’ responses of the survey questionnaires, the most essential features and characteristics required for a quality instructional website were concluded as the assessment guidelines of an instructional website.

The expert group had revised the assessment guidelines of instructional websites twice, and the effectiveness of the guidelines was high. From the analysis of data collected, it was found that the items of indexes for the assessment guidelines of instructional websites were recognized by the research targets including navigation group, teacher group, and student group. Furthermore, these items were consistent with the hypothesis of the study and results of the analysis of pilot test data conducted by the navigation group. As a result, the guidelines which resulted from the five assessment indexes displayed much high value. The research results were also regarded as database where teachers of all disciplines could refer to whenever they would develop their own instructional websites.

Advertisement

2. Methods

2.1. Participants

The samples of this study consisted of three groups. In the first group called control group for pilot test as well, there were 20 pre-teaching teachers who still were students enrolled in the Teacher Education Program at the National Yunlin University of Science and Technology (NYUST). In the second group, there were 33 professors who had built instructional websites for their courses at NYUST since academic year 2012, and for the other group, there were 45 teachers who attended “The Research Methods Conference of Southern Institute and College” held by NYUST in 2012. In the third group, there were 240 current NYUST students enrolled in the College of Engineering, College of Management, College of Design, and College of Humanities and Science.

2.2. Instrument and return rate

A descriptive survey method was used to collect data. After a close review of studies [5, 6, 7, 8] and particular website guidelines [9, 10, 11], a closed-ended questionnaire, called Criteria of Instructional Website, was designed to gather data related to participants’ perceptions of experiences on instructional websites. The questions related to each of the five criteria are listed in Table 1 including six variables, gender, professional field, years of hands-on experience of Internet surfing, educational background, years of teaching (for faculty only), and purposes of using the Internet, also set up for analyzing vocational education faculty’s and students’ perceptions of instructional websites. The overall response rate for this study was 77.6 %.

Item of criteria Sample question
1. Website material development The material should be genuinely related to the Web name that tells
2. Website graphic design The instructional website design should contain multimedia effects, such as sounds, 3-D pictures
3. Website interface The web should offer an exact time for downloading each file
4. Website communication interaction (interactivity) The instructional website should offer some kinds of sections for motivating students to participate in discussion
5. Website system The instructional website should offer a diversity of learning assessment

Table 1.

Sample questionnaire

Advertisement

3. Results

3.1. Control group (n = 20)

The number of pre-teaching teachers of the control group was 20 (n = 20). Among the participants, 75 % were females, 55 % with master degree, 30 % were studying in College of Management, and 70 % with experiences of Internet surfing for less than four years. And they were all sharing similar purposes of Internet surfing, such as e-mails, chatting on-line, gathering information, on-line games, and on-line learning. In addition, the results indicated that the participants rated the hypothesized five criteria as important elements with an average of more than 3.4 of the five-point Likert scale on the same variables mentioned above.

Figure 1.

distribution of gender of pre-teaching teachers

Among pre-teaching teachers who attended the seminar hosted by the National Yunlin University of Science and Technology Teachers’ Center in 2012, the distribution percentages were as follows: male teachers with 25 % and female teachers with 75 %.

Figure 2.

distribution of pre-teaching teachers who studied in college

Among pre-teaching teachers who attended the seminar hosted by the National Yunlin University of Science and Technology Teachers’ Center in 2012, the distribution percentages were as follows: College of Management with 30 %, College of Design with 25 %, College of Engineering with 20 %, and College of Humanities and Applied Sciences with 25 %.

Figure 3.

distribution of pre-teaching teachers

Among pre-teaching teachers who attended the seminar hosted by the Center of Teacher Education in the National Yunlin University of Science and Technology Teachers’ Center in 2012, the percentages were as follows: master degree with 55 %, four-year bachelor’s degree with 20 %, two-year bachelor’s degree with 20 %, and doctoral degree with 5 %.

Figure 4.

distribution of time in using Internet for pre-teaching teachers

Among pre-teaching teachers who attended the seminar hosted by the Center of Teacher Education in the National Yunlin University of Science and Technology Teachers’ Center in 2012, the percentages were as follows: never used with 0 %, 1 year and less with 30 %, 2-4 years with 40 %, 5-7 years with 30 %, and more than 7 years with 0 %.

Figure 5.

Distribution of the purposes of using Internet for pre-teaching teachers

Among pre-teaching teachers who attended the seminar hosted by the Center of Teacher Education in the National Yunlin University of Science and Technology Teachers’ Center in 2012, the percentages were as follows: receiving and sending e-mails with 25 %, BBS or online chatting with 20 %, browsing data with 25 %, playing online games with 25 %, and online learning with 5 %

3.2. Teachers group: (n = 78)

Two thirds of the participants were males teaching in the College of Management with 20.8 %, College of Art and Design with 20.8 %, College of Humanities with 20.8 %, and College of Engineering with 37.5 %. One fourth of them had less than 15 years of teaching experiences. Interestingly, a teacher who had more than 10 years of teaching experiences indicated that the main purpose of Internet surfing was only for e-mail sending/receiving rather than instructional purposes. Compared with others, most of the teachers indicated that they used Internet for class activities with 83.3 %, and information collection with 77.5 %. More importantly, there was not significant difference between the control group and the teacher group for the perception of the instructional website based on the hypothesized five criteria at the probability of 0.05. Nevertheless, different college faculties shared slightly different opinions about the importance of item 1 of the criteria of website material development (e.g. the material should be compatible with the title which it belonged to) as shown in Table 2. Also, different college faculties shared slightly different perceptions on the items of the appropriate location of frame design in the criteria of “website interface” based on the Schaffer test.

College N N Alpha =0.05
1 2
Management 29 3.52
Engineering 3 4.00 4.00
Art and Design 21 4.24 4.24
Humanities 6 4.83
0.465 0.335

Table 2.

Website material development, item #1 Schaffer test

Figure 6.

Distribution of gender of YUNST teachers who have built web pages as teaching aids

Among YUNST teachers who have built Web pages as teaching aids, the percentages were as follows: male with 87 % and female with 13 %.

Figure 7.

Distribution of YUNST teachers who have built Web pages as teaching aids

Among the college of YUNST teachers who had built Web pages as teaching aids, the percentages were as follows: College of Management with 21 %, College of Design with 21 %, College of Humanities and Applied Sciences 21 % and College of Engineering with 37 %.

Figure 8.

Distribution of seniority of YUNST teachers who have built Web pages as teaching aids

Among seniority of YUNST teachers who have built Web pages as teaching aids, the percentages were as follows: 5 years and less with 25 %, 6-10 years with 38 %, 11-15 years with 25 %, 16-20 years with 8 %, and 21 years and above with 4 %.

Figure 9.

Distribution diagram of time YUNST teachers who have built Web pages as teaching aids used the Internet

Among the time YUNST teachers who have built Web pages as teaching aids used the Internet, never used was 0 %, one year and less 46 %, 2-4 years 50 %, 5-7 years 0 %, 7-10 years 0 %, and more than 10 years 4 %.

Figure 10.

Distribution program of the purpose of YUNST teachers who have built Web pages as teaching aids to use the Internet

Among the purpose of YUNST teachers who have built Web pages as teaching aids to use the Internet, the percentages were as follows: receiving and sending e-mails with 25 %, browsing data with 25 %, BBS or online chatting with 20 %, playing online games with 25 %, and online learning with 5 %.

3.3. NYUST students group (n = 240)

About half of the respondents were male or female. Most of them studied in the College of Management (31.8 %) and College of Engineering (31.3 %) with a college degree (47.0 %), and two- to four-year Internet application experience (55.6 %). The results also revealed that the major purposes of surfing the Internet were for e-mail (69.7 %) and information collection (76.1 %). Based on the results of t-test analysis, there was no significant difference between the control group and the student group for the perceptions of the instructional website based on the hypothesized five criteria at the probability of 0.05. However, different college students have slightly different perceptions on the items of the appropriate updated relevant information given on the side of the site in the criteria of “website material development” based on the Schaffer test.

Third, NYUST students (n = 240), about half of the respondents were male/female. Most of them studied in the College of Management (31.8 %) and Engineering (31.3 %) with a college degree (47.0 %) and two- to four-year Internet application experience (55.6 %). The results also revealed that the major purposes of surfing the Internet were for email (69.7 %) and information collection (76.1 %). Based on the results of t-test analysis, there was no significant difference between the control group and the student group for the perceptions of the instructional website based on the hypothesized five criteria at the probability of 0.05. However, different college students have slightly different perceptions on the items of the appropriate updated relevant information given on the side of the site in the criteria of “website material development” based on the Schaffer test.

Figure 11.

Distribution of gender of YUNST students

Among YUNST students, the percentages were as follows: male with 46 %, and female with 54 %.

Figure 12.

Distribution of college students at YUNST

Among students at YUNST, the percentages were as follows: College of Management with 32 %, College of Design with 14 %, College of Engineering with 23 %, and College of Humanities and Applied Sciences with 31 %.

Figure 13.

Distribution of students’ degree at YUNST

Among the degrees of YUNST students, the percentages were as follows: master’s with 25 %, four-year bachelor’s with 47 %, two-year bachelor’s with 21 %, and doctoral with 7 %.

Figure 14.

Distribution of years in using the Internet of YUNST students

Among the years YUNST students have used the Internet, the percentages were as follows: never used with 3 %, less than 1 year with 5 %, 2-4 years with 53 %, 5-7 years with 30 %, and 7-10 years with 6 % and more than 10 year with 4 %.

Figure 15.

Distribution of purpose of YUNST students in using the Internet

Among the purpose of YUNST students in using the Internet, the percentages were as follows: receiving and sending e-mails with 43 %, BBS or online chatting with 7 %, browsing data with 45 %, playing online games with 2 %, and online learning with 3 %.

Advertisement

4. Conclusion

The central concern of this study was to develop efficient ideas of instructional websites criteria for teachers and instructional Web designers to use and build excellent instructional Web environment. The results revealed that hypothesized criteria were perceived as a highly significant contribution. Indeed, the results could help Web designers construct highly effective and interactive instructional Web environment and bring up education in Taiwan into a higher-level digital learning situation. Based on the results, the study provided some recommendations for further instructional Web establishment and further studies as follows:

  1. Instructors should be flexible in constructing their instructional websites specifically to the target students’ educational backgrounds.

  2. A comparative study for perception of long-term teaching experienced teacher versus new or short-term teaching experienced teacher on criteria of instructional website was suggested for further study.

  3. Researches must go on to expand to other schools, faculty and students to deepen scopes and verify the findings.

Advertisement

Acknowledgments

The authors would like to thank the National Science Council and National Yunlin University of Science and Technology for the grant support.

The study has much appreciated the enthusiastic participants including three instructors who established their own instructional websites in the first semester of the academic year 2013 at the National Yunlin University of Science and Technology, 35 teachers who participated in the teacher workshop on research methods for teachers of science and technology in south Taiwan, and 240 college students of the National Yunlin University of Science and Technology.

References

  1. 1. Kuo, C.H. & Chang, T.S. English On-line: Designing a Web-based EST Course. Proceedings of 2003 International Conference on English Teaching and Learning in the Republic of China. Providence University Taichung (2003).
  2. 2. McCracken, J., Cho, S., Sharif, A., Wilson, B., & Miller, J. “Principled Assessment Strategy Design for Online Courses and Programs.” Electronic Journal of E-learning, 10.1, (2012):107-119.
  3. 3. Graff, Martin. “Cognitive style and attitudes towards using online learning and assessment methods.” Electronic Journal of e-learning 1.1, (2003): 21-28.
  4. 4. Smith, Martin. “Can Online Peer Review Assignments Replace Essays in Third Year University Courses? And If So, What Are the Challenges?.” Electronic Journal of e-Learning 10.1, (2012): 147-158.
  5. 5. Martinez, M. Successful Web Learning Environments: New Design Guidelines. ERIC Document ED 446745 (2000).
  6. 6. Ministry of Education. Education in the Republic of China. Bureau of Statistics, MOE: Taipei. (1996)
  7. 7. Ministry of Education. (2001). Education in the Republic of China. Taipei: MOE.
  8. 8. Pan, A. C. (1998). Optimize the Web for Better Instruction. ERIC Document 421098.
  9. 9. Technology for all Americans Projects. Technology for All Americans: A Rationale and Structure for the study of technology. Blacksburg: VA(1997)
  10. 10. Summary, R. & Summary, L. (1998). The Effectiveness of the World Wide Web as an Instructional Tool. ERIC Document 431393.
  11. 11. Wang, J. & Chuang, K.C. (2003). A Study of Instructional Website Guidelines. World Conference on E-Learn in Corp., Govt., Health, & Higher Ed., Vol. 2002, Issue 1, pp.999-1005.

Written By

Kuei-Chih K.C. Chuang and Mei Chuan Tsai

Submitted: 23 March 2015 Reviewed: 16 June 2015 Published: 21 October 2015