Student evaluation is the process of determining the performance levels of individual students in relation to educational learning objectives. A high quality evaluation system certifies, supports, and improves individual achievement and ensures that all students receive a fair evaluation in order not to constrain students' present and future prospects. Thus, the system should regularly be reviewed and improved to ensure that it is suitable, fair, impartial and beneficial to all students. It is also desirable that the system is transparent and automation measures should be embedded in the evaluation. Fuzzy reasoning has proven beneficial to infer scores of students (e.g. Saleh & Kim, 2009). However, in order to improve the reliability and robustness of the system, Gaussian membership functions (MFs) are proposed as an alternative to the traditional triangular MFs.
Since its introduction in 1965 by Lotfi Zadeh (1965), the fuzzy set theory has been widely used in solving problems in various fields, and recently in educational evaluation. Biswas (1995) presented two methods for evaluating students’ answer scripts using fuzzy sets and a matching function; a fuzzy evaluation method and a generalized fuzzy evaluation method. Chen and Lee (1999) presented two methods for applying fuzzy sets to overcome the problem of rewarding two different fuzzy marks the same total score which could result from Biswas’ method (1995). Echauz and Vachtsevanos (1995) proposed a fuzzy logic system for translating traditional scores into letter-grades. Law (1996) built a fuzzy structure model for an educational grading system with its algorithm aimed at aggregating different test scores in order to produce a single score for an individual student. He also proposed a method to build the MFs of several linguistic values with different weights. Wilson, Karr and Freeman (1998) presented an automatic grading system based on fuzzy rules and genetic algorithms. Ma and Zhou (2000) proposed a fuzzy set approach to assess the outcomes of Student-centered learning using the evaluation of their peers and lecturer. Wang and Chen (2008) presented a method for evaluating students’ answer scripts using fuzzy numbers associated with degrees of confidence of the evaluator. From the previous studies, it can be found that fuzzy numbers, fuzzy sets, fuzzy rules, and fuzzy logic systems are and have been used for various educational grading systems.
Evaluation strategies based on fuzzy sets require a careful consideration of the factors included in the evaluation. Weon and Kim (2001) pointed out that the system for students’ achievement evaluation should consider three important factors of the questions which the students answer: the difficulty, the importance, and the complexity. Singleton functions were used to describe the factors of each question reflecting the effect of the three factors individually, but not collectively. Bai and Chen (2008b) stressed that the difficulty factor is a very subjective parameter and may cause an argument about fairness in the evaluation.
The automatic construction of the grade MFs of fuzzy rules for evaluating student’s learning achievement has been attempted (Bai & Chen, 2008a). Also, Bai and Chen (2008b) proposed a method for applying fuzzy MFs and fuzzy rules for the same purpose. To solve the subjectivity of the difficulty factor embedded in the method of Weon and Kim (2001), Bai and Chen (2008b) acquired the difficulty parameter as a function of accuracy of the student’s answer script and time used for each question. However, their method still has the subjectivity problem, since the resulting scores and rankings are heavily dependent on the values of several weights which are assessed by the subjective knowledge of domain experts.
Saleh and Kim (2009) proposed a three node fuzzy logic approach based on Mamdani’s fuzzy inference engine and the centre-of-gravity (COG) defuzzification technique as an alternative to Bai and Chen’s method (2008b). The transparency and objective nature of the fuzzy system makes their method easy to understand and enables teachers to explain the results of the evaluation to sceptic students. The method involved conventional triangular MFs of fixed parameters which could result in different results when changed. In this chapter, the Gaussian MFs are proposed as an alternative and a sensitivity study is conducted to get the appropriate values of their parameters for a more robust evaluation system.
The chapter will be organized as follows: In Section 2, a review of the three nodes fuzzy evaluation method based on triangular MFs is introduced. In Section 3, Gaussian MFs are proposed for a more robust evaluation system. A comparison of the two methods is presented in Section 4. Conclusions are drawn in Section 5.
2. A review of the three nodes fuzzy evaluation system
The method proposed by Bai and Chen (2008b) has several empirical weights which are determined subjectively by the domain expert. Quite different ranks can be obtained depending on these weight values. By using this method, the examiners could not easily verify how new ranks are acquired and could not persuade sceptical students. Naturally, there is no method to determine the optimum values of these weights. Also, the weighted arithmetic mean formula used to compute the outputs do not satisfy the concept of fuzzy set. To reduce the degree of subjectivity in this method and provide a method based on the theory of fuzzy set, Saleh and Kim (2009) proposed a system applying the most commonly used Mamdani’s fuzzy inference mechanism (Mamdani, 1974) and center of gravity (COG) defuzzification technique. In this way, the system is represented as a block diagram of fuzzy logic nodes as shown in Fig. 1. The model of Bai and Chen (2008b) can be considered as a simple specific case of the block diagram by replacing each node with a weighted arithmetic mean formula. The system consists of three nodes; the difficulty node, the cost node and the adjustment node. Each node of the system behaves like a fuzzy logic controller (FLC in Fig. 1) with two scalable inputs and one output, as in Fig. 2. Each FLC maps a two-to-one fuzzy relation by inference through a given rule base.
The inputs to the system in the left hand side of Fig. 1 are given by examination results and domain experts. Assume that there are n students to answer m questions. From examination results, we get an accuracy rate matrix, A, of dimension m x n, which is the student’s scores in each question divided by the maximum score assigned to this question
A = [a ij ], m x n,
where denotes the accuracy rate of student j on question i. We also get a time rate matrix, T, of dimension m x n, which is the time used by a student to solve a question divided by the maximum time allowed to solve this question
T = [t ij ], m x n,
where denotes the time rate of student j on question i. We are also given a grade vector, G, of dimension m x 1
G = [g i ], m x 1,
where denotes the assigned maximum score of question i satisfying
Based on the accuracy rate matrix, A, and the grade vector G, we obtain the total score vector of dimension n x 1,
S = ATG = [s j ], n x 1,
where s j is the total score of student j. The “classical” ranks of students are then obtained by sorting the element values of S in descending order.
Example. Assume that 10 students laid to an exam of 5 questions and the accuracy rate matrix, the time rate matrix, and the grade vector are given as follows (Bai & Chen, 2008b, Saleh & Kim, 2009):
Here, GT denotes the transpose of G. Total score for each individual student is then obtained by formula (1) as
and thus the “classical” ranks of students is become
S 9> S 1> S 2> S 8> S 4 = S 5 = S 10> S 6> S 7> S 3,
where S a > S b means that the score of student a is higher than score of student b and S a = S b means that their scores are equal.
From the domain expert, we get the importance matrix, P, of dimension m x l which describes the degree of importance of each question in the fuzzy domain.
P = [p ik ], m x l,
where denotes the degree of importance of question i belonging to the importance level k. Here, five levels of importance (l = 5) are represented by five fuzzy sets; k = 1 representing the linguistic term “low”, k = 2 representing “more or less low”, k = 3 representing “medium”, k = 4 representing “more or less high” and k = 5 representing “high”. The MFs are shown in Fig. 3. Once crisp values are given as a vector for the importance of questions by a domain expert, the values of fuzzy importance matrix or p ik ’s are obtained by the fuzzification.
It is noted that the same five fuzzy sets, shown in Fig. 3, are applied to represent the accuracy, the time rate, the difficulty, the complexity, and the adjustment of questions in the fuzzy domain.
We are also given the complexity matrix of dimension m x l, which is an important factor indicating the ability of students to give correct answers of complex questions
Co = [co ik ], m x 1,
where denotes the degree of complexity of question i belonging to the complexity level k.
Example. For the above example we get the following, in fuzzy domain, by the domain expert:
As a primary step, the inputs from the examination results are averaged and fuzzified based on the defined levels (fuzzy sets) in Fig. 3. The average accuracy rate vector of dimension m x 1 is obtained as
where denotes the average accuracy rate of question i which is obtained by
and the average time rate vector of the same dimension m x 1 is obtained as
where denotes the average time rate of question i which is obtained by
Next, by fuzzification, we obtain the fuzzy accuracy rate matrix of dimension m x l,
FA = [fa ik ], m x l,
where fa ik denotes the membership value of the average accuracy rate of question i belonging to level k, and the fuzzy time rate matrix of dimension m x l,
FT = [ft ik ], m x l,
where ft ik denotes the membership value of the average time rate of question i belonging to level k, respectively.
Example. For the above by formula (2) and formula (3), respectively, we get
Based on the fuzzy MFs in Fig. 3 we obtain the fuzzy accuracy rate matrix and the fuzzy time rate matrix as:
In the first node, both inputs are given by examination results, whereas in the later nodes, one input is the output of its previous node and the other is given by a domain expert. The output of each node can be in the form of a crisp value (defuzzified) or in the form of linguistic variables (MFs). Each node has two scale factors (SFs in Fig. 1). We can adjust the effects of inputs by varying the values of scale factors. Here, we let both scaling factors have the same value of unity assuming equal influence of each input on the output. Each fuzzy node proceeds in three steps:
Step 1 (Fuzzification). In this step, if given in crisp sets, inputs are converted into membership values in the fuzzy sets shown in Fig. 3. Triangular MF is the commonly used due to its simplicity and easy computation.
Step 2 (Inference). Here, inference is performed based on the given rule base, in the form of IF–THEN rules. We use Mamdani’s max-min inference mechanism which is the most commonly used inference mechanism to produce fuzzy sets for defuzzification (1974). In Mamdani’s max-min mechanism, implication is modelled by means of minimum operator, and the resulting output MFs are combined using maximum operator.
The inference mechanism can be written into the form:
where is the output of inference (i.e. the fire-strength) of question i in fuzzy set k. A matrix of dimension m x l is then obtained as
Step 3 (Defuzzification). In this step, fuzzy output values are converted into a single crisp value or final decision. Here, the COG method is applied. The crisp value of question i is obtained by
where integrals are taken over the entire range of the output where µ(x) is obtained from step 2. By using the COG method, a computable and reliable crisp value can be obtained. The proposed system has been implemented using the Fuzzy Logic Toolbox™ 2.2.7 by MathWorks (http://www.mathworks.com/products/fuzzylogic).
In order to obtain the adjustment vector, each of the three nodes follows the above scheme. The difficulty node has two inputs comprising the accuracy rate and the time rate, and one output displaying the difficulty. The cost node has two inputs comprising difficulty and complexity, and one output displaying the cost. Likewise, the adjustment node has two inputs comprising cost and importance, and one output displaying the resulting adjustment.
The adjustment vector, W, is then used to obtain the adjusted grade vector of dimension m x 1,
where is the adjusted grade of question i,
Then, the obtained value is scaled to its total grade (i.e. 100) by using the formula
Finally, we obtain the adjusted total scores of students by
New and modified ranks of students are then obtained by sorting element values of in descending order.
Example. By referring to Fig. 1, the average values of A and T in the difficulty node are computed using formula (2) and formula (3) and then fuzzified, in step 1, to obtain the fuzzy matrices FA and FT, respectively. Next, as an example for step 2, the output for question 1 in level 4 (fuzzy set “more or less high”) is computed based on the rule base given in Table 1a. The computation uses the Mamdani’s fuzzy interference mechanism and is obtained by formula (7) as the following:
By the same procedure, we obtain the inference output, the fire-strength of the difficulty matrix as:
In Step 3, we use COG to compute the crisp value of the difficulty of question 1 by formula (8) as 0.576, as illustrated in Fig. 4. Likewise, we compute the crisp values of other questions as
At the next cost node, in Step 1, the crisp values of D obtained at the previous node are fuzzified to obtain the fuzzy difficulty matrix as:
In Step 2, based on the rule base given in Table 1b, we obtain the inference output (i.e. the fire-strength) of the cost matrix as a function of the fuzzy difficulty and fuzzy complexity matrices by formula (7) as:
In Step 3, the crisp values of cost are obtained as
At the final adjustment node, in Step 1, the crisp values of C obtained at the previous node are fuzzified to obtain the fuzzy cost matrix as
In Step 2, based on the rule base given in Table 1c, we obtain the inference output (i.e. the fire-strength) of the adjustment matrix as a function of the fuzzy cost and fuzzy importance matrices by formula (7) as:
Finally, we get the fuzzified adjustment matrix as:
Now, the adjusted grades are obtained using formula (9) as
After scaling to the total score of the manuscript (i.e., 100) using formula (10), we have:
The new total scores of the students are then obtained using formula (11) as:
With the new ranks being
S 9> S 1> S 2> S 4> S 6> S 10> S 8> S 7> S 5> S 3.
3. Three nodes system based on Gaussian membership functions
The three nodes fuzzy evaluation system described in Section 2 is based on triangular MF’s which are the simplest MF’s formed using straight lines. Triangular MFs are defined by three parameters and there is no way to acquire its optimum values. It was noted that when theses parameters are changed slightly, different ranking orders are obtained which could impair the system’s reliability. In order to avoid losing reliability and having a robust system, the system should be able to give the same ranking orders without changing students’ scores and for various values of these parameters. As an alternative approach, Gaussian MF’s are proposed. Gaussian MF’s are suitable for problems which require continuously differentiable curves and therefore smooth transitions, whereas the triangular do not posses these abilities. Gaussian MF’s are defined by two parameters which is one parameter less than that of the triangular MF’s. The tuning of a reduced number of parameters will result in a reduced Degree Of Freedom (DOF) and hence a more robust system. Gaussian MF’s is defined as
Where c i is the center (i.e., mean) of the i th fuzzy set and σ i is the width (i.e., standard deviation) of the i th fuzzy set, which have by nature, infinite support (i.e., every control point influences the whole calculations of the output) (Lohani et al., 2007). Therefore, for Gaussian MF’s with wide widths it is possible to obtain a membership degree to each fuzzy set greater than 0 and hence every rule in the rule base fires. Consequently, the relationship between input and output can be described accurate enough. Here, the centres of the five Gaussian MF’s are chosen to be the same as that of the triangular MFs shown in Fig. 3 (i.e. [0.1 0.3 0.5 0.7 0.9]). Gaussian MF’s of the five levels for = 0.1 are shown in Fig. 6. From Fig. 6, it is obvious that Gaussian MF’s provide more continuous transition from one interval to another and hence provides smoother control surface from the fuzzy rules. The surface view of the rule base in Table 1a and b for Gaussian MFs of = 0.1 are shown in Fig. 7.
Example. The width of the Gaussian MF’s, , is varied between 0.05 and 12 in steps of 0.05 and the three nodes fuzzy system based on Gaussian MF’s is applied to the same example introduced in Section 2. The new total scores of the students are then obtained using formula (11) as shown in Table 2.
The mean of the new scores of the 10 students is computed for each value of the membership width, , and is shown in Fig. 8. From the figure, it is obvious that the mean of the new scores is equal to the mean of the classical scores obtained using formula (1) for membership width of 4.0 and higher. Although the new scores, rounded to two digits, are equal, the system is still able to give the correct ranking order of the students with equal total scores.
4. Comparison of the methods
The ranking order has been obtained when the three nodes fuzzy system based on triangular MF’s are applied to students with equal total scores. When the three nodes fuzzy system based on triangular membership functions is applied to all students, new scores and hence new rankings are obtained. When the three nodes system based on Gaussian membership of width of 4.0 is applied to all students, the resultant new total scores of students rounded to two digits are equal to that of the classical scores but with new ranking orders. Ranking orders and scores of students are shown in Table 3.
From Table 3 it is seen that the same ranking order has been obtained when the three nodes fuzzy system based on triangular MF’s is applied to only students with total scores and the three nodes fuzzy system based on Gaussian MF’s applied to all students. The varying of the parameters of the triangular MF’s results in different scores and different ranking orders while the same scores and the same ranking orders are obtained for Gaussian MF’s of various widths.
In this chapter, we proposed Gaussian MF’s to represent the fuzzy sets (i.e., levels) representing the importance, the complexity and the difficulty of the questions given to students. Results show that using Gaussian MFs with a width value (i.e., standard deviation) of 0.4 and higher provide a more reliable evaluation system which is able to provide new ranking orders without changing students’ total scores. Gaussian MFs provide smooth transition between levels and provides a way to fire the maximum number of rules in the rule base and hence a more accurate representation of the input-output relationship is achieved. Gaussian MF’s also provides a system with less degree of freedom and hence more robustness. The proposed three nodes fuzzy evaluation system based on Gaussian MF’s provides a new ranking order while keeping the scores of students unchanged. The system is implemented by using the Fuzzy Logic Toolbox™ 2.2.7 by MathWorks®.