Fostering the Diagnostic Competence of Teachers with Multimedia Training – A Promising Approach?

Multimedia as a learning tool offers new opportunities in supporting learning processes. At the same time, it also plays a role in teacher education through e-learning platforms, which provide online resources for learning or computeror web-based training, which facilitate the acquisition of specialized knowledge. Soft skills (e.g., communicative or diagnostic skills) can also be trained in a multimedia-based learning environment. In this chapter, fostering the diagnostic competence of pre-service teachers through a multimedia-based learning environment is discussed.


Introduction
Multimedia as a learning tool offers new opportunities in supporting learning processes.At the same time, it also plays a role in teacher education through e-learning platforms, which provide online resources for learning or computer-or web-based training, which facilitate the acquisition of specialized knowledge.Soft skills (e.g., communicative or diagnostic skills) can also be trained in a multimedia-based learning environment.In this chapter, fostering the diagnostic competence of pre-service teachers through a multimedia-based learning environment is discussed.
Diagnostic competence is one of the Main competencies that a teacher should have to be able to deal with the current and future challenges of teaching at a school (Bruehwiler et al., 2004;Schrader, 1997).The concept of "diagnostic competence in teaching situations" is a broad one, which brings together different aspects of teaching competence (Barth, 2010).Some of the components of "diagnostic competence" include automated behavior.Automated behavior needs a different instructional approach compared to learning new things (Henninger & Weingandt, 2003).Multimedia-based learning environments can be effective tools in developing the ability to reflect on and change partly automated behavior, such as communication skills.Different studies examining the effects of a multimedia training tool called CaiManOnline© have shown that this can be done (Barth, Hauck, Hörmann & Henninger, 2007;Henninger & Hörmann, 2007;Henninger & Mandl, 2000;Jaschniok, Barth, Amann & Henninger, 2008).This kind of virtual teacher training focuses on the receptive part of communication, i.e., understanding and interpreting utterances, in which video sequences of teaching behavior are used in a multimedia-based learning environment.
Recognizing the need for teachers, who are especially competent in diagnosing learning prerequisites in class, we first want to present a conceptualization of diagnostic competence in teaching situations that focuses on the trainable aspects.Next, a study is presented, which explains the starting point for a multimedia training approach.Following are recommendations on how to create multimedia applications in the field of automated behavior.

The importance of diagnostic competencies for teachers
One of the responsibilities of teachers is to be able to diagnose an extremely wide variety of learning prerequisites on an ongoing basis and determine the opportunities available to their students.This diagnosis is a precondition for adaptive teaching competency, which is defined as "the teacher's ability to adjust instruction to the individual learning processes of pupils in such a way as to create favourable conditions for each student's learning for understanding" (Bruehwiler & Vogt, 2007).
Making a competent diagnosis in teaching situations means that a teacher has to be able to recognize how students indicate their learning requirements within the scope of social interaction.The teacher also has to gather additional specific information relevant to this diagnosis or be able to recall this information in order to utilize it (Barth, 2010).
This means that the teacher has to use different sources of information and knowledge to be able to make a diagnosis, such as: 1. Situation-dependent information: This information is gathered from observable aspects of the teaching situation, like the social interaction in the classroom or the individual student's interaction with the lesson content.2. Person or class-specific information: This is the information about an individual student or a class that is available to the teacher.3. Professional and experiential knowledge: Both knowledge types take place at the same time since it is assumed that both are integrated in the individual knowledge structures of the teacher.These individual knowledge structures can be described as subjective theories (Groeben et al., 1988;Steinke, 1998;Wahl, 1997).
To foster diagnostic teacher competencies, it is crucial to know which information or knowledge sources play the most important role during the process of diagnosing students' learning prerequisites.If the teachers or pre-service teachers mainly use prior knowledge (e.g., about a special student or class), then the usefulness of a multimedia learning environment, in which video sequences of a classroom situation are used would be very limited.If teachers refer primarily to situation-dependent information, then such a learning environment could have great potential in fostering diagnostic teacher competencies.
In addressing the question of whether multimedia learning environments are able to foster the ability to make a proper diagnosis of learning prerequisites in teaching situations, two different things need to be considered: 1.The theoretical concept of a competent diagnosis in teaching situations 2. The actual diagnosis of learning prerequisites in classroom situations made by the preservice teachers

Analysis of the theoretical concept of competent diagnosis in teaching situations
The term competent diagnosis in teaching situations refers to the diagnosis of situative learning requirements of students during classroom teaching sessions.Three of the most important learning requirements or key learning prerequisites are motivation, emotion and comprehension.Except for the first dimension, all the other dimensions refer to the teacher's behavior in the classroom situation and all imply interpretation and meaning-making processes.This is the first indication that a learning environment, which enables pre-service teachers to reflect on existing interpretation patterns, generate new ones and consciously deal with them, like the CaiManOnline© learning tool, can foster diagnostic competence in classroom situations.
Typical CaiManOnline© Scenario: The multimedia training tool, CaiManOnline©1 (Henninger & Mandl, 2003), considers instructional standards deduced from the cognitive-apprenticeship theory (Collins et al., 1989).Pre-Service-teachers or other learners have to analyze video-based communicative situations with CaiManOnline©.While working on this task, the students are assisted by several elements, which support reflection.The students have to verbalize and explain their analysis of the communication situation presented.To foster reflection processes, expert solutions pertaining to the tasks recommended are available.In addition, depending on the specific implementation, online coaches can provide feedback to the students.Feedback by a coach is regarded as a critical factor, which could either motivate or hinder learning (Hattie & Timperley, 2007).
CaiManOnline© is usually integrated into a training as a virtual self-learning phase of a blended learning communication training to foster receptive communication behavior.

Analysis of the actual diagnosis of learning prerequisites in classroom situations made by pre-service teachers
The DIAL study (empirical clarification of the basis for the diagnosis of situative learning requirements of students during classroom sessions) conducted at the University of Education Weingarten examines the kind of information and knowledge that pre-service teachers use to make a diagnosis of comprehension as a situative learning requirement.
Comprehension was chosen as a learning prerequisite because it is one of the most important in learning situations.

Research questions
In order to create an adequate learning environment, we need specific answers related to what is being learned in order to decide which video-based material should be included or which specific instructional design should be used to foster reflection of diagnostic behavior.
Thus, the main research questions of the DIAL project were: What kind of class-specific information (information which is independent of the specific situation) do teachers use to make a diagnosis (e.g., basic information about the class)?What kind of situation-dependent information (verbal, nonverbal and paraverbal signs of the students) do teachers use to make a diagnosis?What kind of professional knowledge and experiential knowledge do teachers use to make a diagnosis?How important are the different categories of information and knowledge for the diagnosis?

Design of the DIAL study
To answer these research questions, this study used a quasi-experimental design.A total of 36 pre-service teachers (29 female, 7 male) were asked to watch videotaped classroom sessions.Each participant was presented with a school lesson in both a school subject that was part of his/her course of studies and with a school subject that was not familiar to him/her.In total, we examined three school subjects: German, geography und chemistry.Furthermore, the information about the school class (information vs. no information) and modes of classroom interaction 2 (individual work/work in pairs vs. teacher interaction with the whole class) was varied.

Data collection
An online questionnaire was used for the data collection.After the participants had watched the videotaped classroom session, they were asked to make a diagnosis of the degree of comprehension of different students and to explain what information or knowledge led to this diagnosis.The written responses of the participants regarding the source of their information and knowledge were the textual basis of a qualitative content analysis.The setting is presented in Fig. 1. 2 With regard to data privacy, we could not disclose real information about the individual pupils to the teachers.Thus, we created an information situation that was comparable to when a teacher meets a new class for the first time: The teacher knows a little bit about the students from information obtained from colleagues or former teachers, but does not have specific information about the individuals.In our scenario, the teachers know something about the level of proficiency of the class, the variance in performance between the pupils in the class or whether there are "difficult" pupils in the class.

www.intechopen.com
Fostering the Diagnostic Competence of Teachers with Multimedia Training -A Promising Approach?

53
The aim of this qualitative analysis was to determine the categories of the different sources of information and knowledge that were relevant to the diagnoses made by the teachers.In addition, quantitative methods were used to find out how often specific categories were used under the different experimental conditions.

Results of the DIAL study for multimedia learning
In the first result, the categories of the different sources of information and knowledge identified, which the participants used to make a diagnosis, are presented in Table 1.To develop a set of categories, a qualitative content analysis (Mayring, 2008) was used.The following categories were found by analyzing 9 of the 36 data sets and matching them to the previously mentioned sources of information and knowledge.Afterwards, the 36 data sets were categorized by two raters according to the following categories: Situation-dependent information -observable Body language (e.g., viewing direction of the student, facial expression, posture/gesture) Interaction between the people (e.g., existing or missing interaction between different students, interaction between students and teacher) Actions of the people (e.g., verbal expressions of the students, reactions to external stimuli, behavior of their classmates) Participation in class Observation of how a student works (e.g., positive or negative assessment of how student works, writing behavior)

Situation-dependent informationnon-observable
Interpretation of student's appearance (e.g., positive appearance, negative appearance) Interpretation of how a student works (e.g., effective way of working, ineffective way of working) Stereotypes of students activated by watching the videotape (e.g., stereotypes about personality traits, skills, etc.) Class-specific information This category was used by only one participant

Professional and experiential knowledge
Knowledge with respect to teaching, methods and contents of instruction Knowledge with respect to the general behavior of a student Knowledge with respect to determining if the answers of the students were adequate or inadequate Table 1.Categories of information and knowledge sources One remarkable result was that the participants did not refer to the class-specific information provided.Only one of the 20 participants who received class-specific information made a connection between his judgement and this information.We limited the theoretically based category, "person or class-specific information" to "class-specific information" because only "class-specific" information was available to the participants.
Based on these qualitative results, the first research question can be answered: What kind of class-specific information (information which is independent of the specific situation) do teachers use to make a diagnosis (e.g., basic information about the class)?
The qualitative analysis shows that only one participant explicitly used the situationindependent information that was made available prior to the viewing to make a judgment concerning the degree of comprehension of a student.However, it should be noted that only anonymized descriptions of the class were given to the participants due to data privacy concerns.The participant who used the class-information, made just the one statement following the lesson plan [which included a description of the class], which was that "the class is accustomed to working in pairs and group work."On the basis of this statement alone, a differentiated analysis of the class-specific information was not possible.Nevertheless, this category was kept in the data analysis due to the fact that this category is based on theoretical foundations.
What kind of situation-dependent information (verbal, nonverbal and paraverbal signs of the students) do teachers use to make a diagnosis?
To answer this question, we created two categories: one includes specific observable signs and the other is based on inferences about the situational activities.
The "observable" category: This category includes statements made by the participants about the body language of the students (e.g., "He often turns around"), their facial expressions (e.g., "puzzled look") or the posture or gesture of a student (e.g., "she is sealing herself off from the other students with her hand").This category also includes statements concerning the interaction between the people (e.g., "the boy is discussing the subject matter with the student next to him") or the actions of the people (e.g., "I heard him saying: 'Eh?What a bullshit'").Statements concerning class participation (e.g., "she doesn't participate in class;" "he is participating actively in class") or an observation of how they work (e.g., "the student quickly begins to complete the text") are also included in this category.
The "non-observable" category: This category includes statements concerning the student's (positive vs. negative) appearance (e.g., "no interest to complete the task;" "it seems the he is preoccupied with something else"), an interpretation of how the student works (effective vs. ineffective) (e.g., "he is distracted whilst working;" "the student is following the lesson attentively") or statements based on stereotypes of the students that were triggered by watching the videotape (e.g., "I think he is a clever tot;" "I assume he is one of the good students in class").
What kind of professional and experiential knowledge do teachers use in making a diagnosis?With regard to the data, we created three sub-dimensions of these knowledge types: The first covered knowledge about teaching, methods and content of instruction (e.g., the work order was quite clear), knowledge about a student's general behavior (e.g., "I suppose she couldn't complete the worksheet because she wasn't able to deal with the content") and knowledge about whether the students gave adequate answers (e.g., "right example;" "wrong answer").
In the next step, all the data of the 36 participants of the study were matched to these categories by two raters.Cohen's kappa coefficient was used to estimate interraterreliability (Cohen's kappa = .67)and showed that there was good agreement according to Altmann (1990).If only the main categories are considered, Cohen's kappa would be even higher (.73).This could be an indication that it is difficult for raters to distinguish between Based on these diagrams, we can answer the question, "How important are the different category types of information and knowledge for the diagnosis?"The data shows that the overwhelming majority of the participants used situation-specific information, as well as professional and experiential knowledge to make a diagnosis.Just one person explicitly used the class-specific information provided.
To determine if the participants used some of the categories of information and knowledge sources (situation-dependent information -observable, situation-dependent informationnon-observable, class-specific information, professional and experiential knowledge) more frequently than others, the data was aggregated.Therefore, the diagnoses of the participants for all four scenes were put together in a way that summarizes the explanatory statements.If a person mentioned the category "professional and experiential knowledge" in three of the four scenes presented, the value 3 was assigned to this person.If a person mentioned this category in only one scene, the value 1 was assigned to this person.In this manner, an ordinal variable was created.With this variable, it was possible to use Friedman's test to examine if a category ranked significantly higher or lower than the others.In fact, the results showed that to be the case (2 (3, N=36) = 89.45,p=.000).57

Mean
Thereafter, pairwise comparisons were used to identify which categories were used more often.The Bonferroni adjustment was used to calculate the levels of significance (α = 0.01/6 = 0.002; α = 0.05/6 = 0.008).Statistically, all the differences were highly significant except for the difference between the situation-dependent information -non-observable and professional experiential knowledge categories.The results are presented in tables 2 and 3.According to these results, the category "situation-dependent information -observable" was used most frequently, followed by the category "situation-dependent information -non observable."The difference between the latter category and the category "professional and experiential knowledge" was not statistically significant.As expected, the category "classspecific information was used the least.

Pairwise comparisons
Further analyses refer to the influence of the different experimental conditions.To check whether the information about the class, which was given to some of the participants really had no influence on the statements made by them, a Chi² test for each of the six classroom scenes presented (German, geography and chemistry -each school subject varied between a situation of individual work/working in pairs or teacher interaction with the whole class) was conducted.The results of the Chi² test (see Appendix A) showed that there was no difference in any of the scenes presented.This means that a participant's decision to use or not use a category did not depend on whether the participant received information about the class or not.
However, it should be noted that this test was not based on a random sample of the target population, and that each of the two variables can only take one of two values (classinformation vs. no class-information; category used vs. category not used).For the Chi² test approximation to be valid, the expected frequency should be at least 5, which was actually not the case in every single Chi² test.
These results not only show that the participants who received "class-specific information" did not use the category "class-specific information" more often than participants without this specific information (it was not possible for the participants without class-specific information to use this category), but also that there was no statistically significant difference between both groups of participants in the use of the other categories.
Furthermore, a test was done to see whether a difference between the subjects presented could be determined.Therefore, the aggregate data (sum of all four scenes) was used again.
The Wilcoxon test was used to compare how often the different categories were used.The result was that there was no statistically significant difference between the school subjects presented (German, geography, chemistry) in terms of how often the diverse categories are used (situation-dependent information -observable, situation-dependent informationnon-observable, class-specific information, professional and experiential knowledge).The results of the different tests are presented in Appendix B.
To compare the classroom scenes showing individual work/work in pairs or an interaction of the teacher with the whole class, the Wilcoxon test was also used.The results showed that there were no statistically significant differences between the different situation types with respect to the categories "situation-dependent information -observable," "situationdependent information -non-observable" and "class specific" (see Appendix C).This was also not expected because these categories were used by nearly all (category: situationdependent information) or almost none (category: class-specific) of the participants.A significant difference resulted, however, with the category "professional and experiential knowledge" (Z= -2.162; p=.031).In situations where a teacher interacts with the whole class, this category was used more frequently compared to the situations in which the students worked individually or in pairs.This is actually not surprising because in situations where a teacher interacts with the whole class, the participants can hear what the students say and determine whether their statements are correct or incorrect.In situations where the students work individually or in pairs, the participation of the students cannot, by its very nature, be heard or assessed.
More detailed qualitative and quantitative analyses of the DIAL study are still in progress.

Discussion
In this section, the results of the DIAL study are summarized.The Dial study showed that pre-service teachers mainly used the situation-specific information available to make a diagnosis of a student's comprehension.In doing so, the participants of the DIAL study usually referred to observable information (i.e., their perceptions), but also to nonwww.intechopen.com Fostering the Diagnostic Competence of Teachers with Multimedia Training -A Promising Approach?59 observable information.This means that the participant interpreted the student's behavior.
To outsiders, such as researchers and possibly the pre-service teacher as well, it was not possible to know which perception led to which interpretation.Class-specific information, which was given to only some of the participants, was obviously not used to make a diagnosis.
This result (if confirmed by additional studies) could be an indication that previous knowledge plays less of a role in the process of diagnosing the learning prerequisites of students.
Some of the critical aspects of the DIAL study need to be addressed.Surely one of the most critical aspects is the fact that for reasons of data privacy only anonymized class information was given to the participants instead of situation-independent true information about the different students filmed.In a real classroom situation, it is certainly the exception rather than the rule that a teacher only has previous knowledge about the class instead of previous knowledge about the different students.In the DIAL study, it was therefore not possible to verify the impact of previous knowledge about specific students on the teacher's diagnosis with respect to the learning prerequisites, like comprehension in "real-life" classroom situations.
A further critical aspect of the study is that individual work or work in pairs could not be presented in a realistic way to the participants compared with situations, in which the teacher interacts with the whole class, which were more realistic.This is because of the fact that in a real classroom situation a teacher is able to move about the classroom freely and observe how tasks are processed and solved by the students.Unfortunately, the presentation of a videotaped classroom situation does not provide any interaction.This means that the participant is not able to actively participate in the situation.Therefore, the realistic presentation, especially of the situation with individual work or work in pairs, has to be viewed critically, as well as the transferability of the results concerning these special situations.

Conclusions
How can the DIAL study help with respect to developing recommendations for multimedia applications?With regard to a soft skill, like making a diagnosis concerning a student's comprehension, we can draw different conclusions from the findings.
The DIAL study showed that previous knowledge (in this case about the class) plays less of a role compared to the teacher's diagnosis of the student's comprehension.This indicates that a multimedia application could be a worthwhile approach in fostering the teacher competency described.If previous knowledge about the class or individual students does not seem to have a strong influence on the pre-service teacher diagnoses, a learning environment which focuses mainly on situative cues, could be a useful way to foster the ability to make a competent diagnosis in teaching situations.Such a tool, which includes video sequences of classroom sessions and enables students to reflect on existing interpretation patterns and deal with them consciously, could be a promising approach in helping to develop the skills to be able to make a competent diagnosis in teaching situations.
With reference to the DIAL results, a multimedia learning environment could especially support pre-service teachers to become aware of the basis of their own diagnostic behavior.This is possible if a pre-service teacher gets a chance to de-automate his/her automated diagnostic behavior in everyday life (cf.de-automation approach of speech receptive behavior, Henninger & Mandl, 2003).One way to accomplish this is to summon someone to explain the basis of her/his own diagnoses and study how they make the diagnosis.Such a practice was used in the DIAL study, by asking the participants to write down the specific information or knowledge that led to their diagnosis.A similar procedure is also implemented in CaiManOnline© training tool.
The results of the DIAL study show that it is not easy for pre-service teachers to describe the observations that lead to their diagnoses.Often, they described their interpretations instead, which were not linked to a specific perception.To support pre-service teachers become aware of the perceptions underlying their interpretations, training in a multimedia learning environment could certainly be a viable approach.Additionally, usage of one's own professional an experiential knowledge can be made clear and transparent to the pre-service teachers as learners in a multimedia learning environment, which includes videotaped classroom situations.This can also be done by asking the participants to write down the specific knowledge that led to their diagnosis.In this way, the learners can be supported in developing their professional competence and to be aware of this competence.
All in all, our data suggests that it is useful to create a multimedia-based learning scenario that supports pre-service teachers develop diagnostic competencies.The study shows that teachers could benefit from a reflection of their behavior, thus an instructional design should focus on this aspect.
Therefore, a multimedia learning environment should fulfill the following requirements:  It should provide a realistic simulation of a classroom situation.This is easy to implement by using a videotaped classroom situation that shows a teacher interacting with the whole class.However, it is barely possible to present a realistic situation of an individual work or a work in pairs.Therefore, the classroom situations presented should be selected carefully.


Providing a realistic representation implies that a learner does not have to work with classroom scenarios that do not have a context, but rather with scenarios, in which a context is integrated.This can be implemented by giving information about the subject of the lesson and about the scene in the videotaped lesson.In the DIAL study this was realized with a detailed draft for the teaching unit, which was given to the participants.This information could also support pre-service teachers in using their content knowledge, pedagogical content knowledge and generic pedagogical knowledge, which are all part of a teacher's professional knowledge (Shulman, 1986) that enables a teacher to make a hypothesis about a student's comprehension and test it through specific observation, which is one of the dimensions of diagnostic competence in classroom situations (Barth, 2010).


The diagnosis of learning prerequisites is assumed to be a strongly automated behavior.
To change automated behavior, it is necessary to de-automate and become aware of the steps taken and to reflect on them (Antos, 1992;Henninger & Mandl, 2003;Lentiev, 1981;Schooler, Ohlsson & Brooks, 1993).Afterwards, the new behavior can be rewww.intechopen.com Fostering the Diagnostic Competence of Teachers with Multimedia Training -A Promising Approach?61 automated.Therefore, a multimedia learning environment should support a learner in de-automating and reflecting on his/her own diagnostic behavior.This can be realized by asking the pre-service teacher to explain the basis of his/her diagnoses, e.g., in a written text.This not only gives insight into the teacher's diagnoses, such as in the DIAL study, but also gives the teacher insight in understanding why he/she is making a specific diagnosis.This also means that the dimension "reflecting on one's subjectivity" of the diagnostic competence in classroom situations can be developed in a multimedia learning environment (Barth, 2010).


To become aware of one's own interpretation process, it could also be helpful to the pre-service teachers to have the opportunity to analyze the diagnoses and the basis of the diagnoses of other people, such as expert teachers, or to get an overview of the different sources that can be used to make a diagnosis, like the ones identified in the DIAL study.
The findings of this chapter can be summarized as follows: The potential for fostering the diagnostic competence of teachers with multimedia training exists, even if there are some limitations, like the difficulty in creating a realistic classroom situation in a simulation based on videotaped classroom situations.Although it is not possible to support all dimensions of a competent diagnosis, some of them can be fostered.
This does not mean that practical training, which is an essential part of a teacher's education in Germany, can be replaced because real-life classroom situations cannot be perfectly simulated.However, multimedia teacher training that supports the acquisition of some teaching competencies, especially if they are related to behavioral factors and reflection is necessary to change this behavior pattern, can be helpful.

Appendix A
Chi² tests: Information about the class given vs. not given (N = 36)

Fostering
identified five different dimensions of diagnostic competence in classroom situations: Designing the lesson to serve the diagnosis of situative learning requirements  Competent perception of the situation (which includes the ability to structure the situation cognitively, the ability to change the focus of attention and the willingness and ability to adopt other perspectives)  Competent hypothesis generation and hypothesis testing  Reflecting on one's subjectivity  Competent receptive and productive communicative behavior