adAstra: A Rubrics' Set for Quality eLearning Design

We gather the lessons learned in a rubrics' set that we call adAstra, borrowing the name from the famous saying “per aspera ad astra” (i.e. through hardships to the stars), from Lucius Annaeus Seneca, philosopher and politician that lived in the ancient Rome around the beginning of the Christian era. adAstra would like to be a guide in designing, developing, delivering and evaluating the academic online courses.


Community of Inquiry
During our studies we asked ourselves which strategies the teacher have to use to promote deep and meaningful online learning. About this topic we found particularly interesting the idea of Community of Inquiry by Garrison, Anderson & Archer (Garrison et al., 2000), that later on benefited from many other authors contributions. 1 These authors started from the study of the impact of written interaction in the eLearning forum, building up a model of a "deep and meaningful learning" environment, the Community of Inquiry Model, that includes: Teaching, Social and Cognitive Presence.
1. The Teaching Presence is the teacher attention to the eLearning design and development, to obtain deep and significant learning. Including what the teacher do before, during and after the module's delivery to determine the outcomes and facilitate the Cognitive and the Social Presence ); 2. The Social Presence is the student possibility to project him/herself socially and affectively in the computer mediated communication, so that he/she is perceived like real person (Rourke et al., 2001); 3. The Cognitive Presence is the possibility for the student to build and consolidate his/her knowledge through the reflection, the meaningful discussion, the critical thoughts and the "practical inquiry" .
The goal is to help the teacher to improve its online courses, understanding which characteristics can facilitate the meaningful debate and the critical thinking development.

adAstra: A tool for the design phase
Garrison and his colleagues, beside giving a description of each one of the "presences", provides an indicators' set that allow to evaluate the three presences during the forum analysis. The tools' set, as far as I understand, is created to be applied on the forum transcription and it is useful during and after the end of an online course.
This approach is also found in most papers of the scientific literature in which the eLearning quality is analyzed ex-post, when the course is over. Indeed the monitoring phase is often seen as the final collection and analysis of the students and teachers impressions about the course. The benefits of this kind of evaluation are for the future editions of the course, but is limited for the course under construction.
In our adAstra approach we choose to scaffold the teachers and the professionals in the design phase. We are strongly convinced that the evaluation is an activity that have to be conducted before, during and after the delivery of an online course.
Therefore we decided to investigate what the designer, the teacher and the eTutor have to do to create an online course that bring to a meaningful learning:  before the course: during the design of the learning environment, the textual and multimedia materials, the assignments, the assessment methods and the feedback.
but also:  during the course, monitoring the students' and teacher's social and cognitive activities;  after the course, for an efficient monitoring of the process, of the project and of the course's effectiveness .
The main goals of our adAstra tools are: 1. to help designing a quality course avoiding rough mistakes for the inexperienced, and slips for professionals; 2. to maintain high the interest for quality during all the course stages; 3. to keep track of the problems encountered by the designers and to store the more effective solutions found, constantly improving adAstra by the inclusion of the experiences gained from different students, teachers, technologies, media and methods. This drill & practice philosophy toward mastering a quality design reminded us the saying "per aspera ad astra", or briefly "adAstra", that we adopted in our research team, initially joking, and to which we afterward became affectionate.
The rubrics that we have up to now validated are about: 1. analysis 2. design and development 3. teachers/eTutors/instructional designers feedback 4. students feedback The idea is to build up a guide that lists the "good" online course characteristics, founded on the experience accumulated by our group in years and years of eLearning design for our Faculty and successively validated during our research work.

TIA-instruction and learning technologies
The "laboratory" that allowed us to design and test the adAstra solutions is the course TIA-Tecnologie dell'Istruzione e dell'Apprendimento (i.e. Instruction and Learning Technologies), that here following we would like to present. Tecnologie dell'Istruzione e dell'Apprendimento (thereafter TIA) is an optional course of the Cognitive Faculty, Trento University, Italy, delivered in the academic years from 2003/2004 to 2008/2009 and today replaced by a mandatory course, with the same educational strategies, but with larger contents and more students. The optional course's number of students was from 15 to 25, and formed 3/5 goups of people, followed by one teacher and 3 part-time eTutors The TIA goal was to allow the students to experiment, in eleven weeks, the distributed learning community dynamics, the collaborative learning, the online communication and the eTutoring principles. The educational strategies used to reach this goals are: collaborative design and creation of a multimedia ipertext; role play; online collaboration; asyncronous forum discussions; self-evaluation and peer assessment. The course was based on socio-constructivistic learning paradigm (Brown et al, 1989, Lave & Wenger 1991, Rogoff 1990, Wenger, 1998. In the first three year the course was based on a proprietary LMS-Learning Management System (screen shot in Fig. 1).  The home page presents: 1. the information environment, with all the general information about the course, the bulletin board for the most urgent announcement, the environment for the group registration; 2. the communication environment for the syncronous and asyncronous communication among the teacher/ eTutors/designer and the students during the whole course; 3. the content environment for multimedia and text contents publishing by the teacher and the eTutors; 4. the group environment that supply the students with all the tools and the resources for the collaboration: i. the forum for the discussion among the students about the contents and about the group organization; the "caffè" for the informal socialization among the students; the teacher and eTutors comments forum, about the weekly students assignments; ii. the wiki, where the group publishes the assignments; iii. the chat for the synchronous group discussions; 5. the student environment, a metacognitive tool to draw up a weekly personal selfevaluation by the student, that receive a timely feedback by the teacher/eTutors;. 6. the agenda to share with the students and groups the deadlines.

TIA teaching strategies
During the various TIA editions many methodological and design choices remained unchanged.
Based on the collaborative and constructivistic strategies we promoted the responsible engagement and the continuous partecipation asking the students to autonomously form small groups (from 4 to 6 people) to work together on the teacher assignments. This approach is important to support the motivation and to facilitate the participants involvement.
The scripted collaboration, a role play collaborative strategy, invite the students to hold by rotation one from the six roles established by the teacher and let them, every week of the course, to have personalized tasks.
The weekly roles are: 1. chairman: sets the agenda, defines, at the beginning of the course, the roles for all the weeks, sends at the module beginning the message to the group, manages the face-toface meeting and possible conflicts; 2. editor: publishes the final version of the group paper/multimedia no later than the 12 p.m. of Sunday; 3. moderator: moderates and enlivens the forum communication, organizes and creates the new threads for a better forum organization; 4. researcher: every week researches and publishes, making them available to the group, two meaningful url (site or papers) about the topic under study; 5. editor: publishes, no later than the 12 p.m. of Sunday, the weekly diary, i.e. five row about the group week; 6. spy: lurks in the other groups and takes back in its own all the strategic information about the other students work in progress.
The responsibility towards the group, but also towards himself and the teacher, guarantees the continuous participation, clearing the drop out risk. In six edition of the course, with circa 110 students, we didn't have any drop-out. 2 Initially many students complains of the pressing pace. Nevertheless to be part of a group that have to work hard, but also warrants, in case of difficulties, a psychological support, a concrete help and an often warm social environment, lets the majority of the students say that they are satisfied of the experience. And many of them express their regrets when the course close down.
Another way to guarantee the participation is to ask the students to send, no later than the Sunday at 12 p.m. a personal diary, a message of 3-5 row about how well went off the week.
To this message the teacher or the eTutor gives always a timely feedback, that gives to the student the feeling that there are a recurrent moment of communication with the teacher, and that she/he can asks if needed. Moreover gives to the student the impression that his/her presence in the course and the contributions he/she gives are appreciated. In the traditional course the teacher interact with the student only when he have to judge him/her, and only in the case of oral examination, that are more and more infrequent, due to the students increasing number. In the eLearning course the teacher assure a continuous help in the knowledge construction.
Another strategy that we use is to facilitate the student in the initial choice to participate to the online course.
The critical moment is at the course beginning, when the student have to enter in the learning environment, understand how it work, what he has to do, how he has to interact with the others. Everything is new. Indeed online courses based on the collaboration, on the interaction with the teacher, on the metacognitive self-evaluation are not so common today, at least in Italy. And moreover the students don't understand if eLearning is "advantageous", compared to the traditional course, from the point of view of the time and energy that have to be dedicated. We can imagine they ask themselves: "If I can pass the exam like non attendee student, why I have to complicate my life attending a demanding online course?
To overcame this situation and avoid the initial drop-out the first fourteen hours of TIA lessons are face-to-face and we present: 1. goals, contents, prerequisites, technologies, activities organization, assessment criteria, educational agreement, teachers, eTutor, designer, reference, agenda, etc. 2. the course environment: how to enter, how it is organized, which tools are available, how to reach the teacher and to communicate with the other students.
Moreover a socialization activity is promoted. We ask the students to register to a group when they enter the first time in the course site. Every group coin a name and create a selfpresentation video. We experienced that this activity, during which the students have to call upon both in the personal plane and in the competences to be shared, is an excellent way to learn to interact, to know each other, to present him/herself and its own potentials and limits to the team mates.
6. adASTRA: The rubrics' set as "lessons learned" history TIA educational and organizational strategies have been improved during the academic years, thanks to the constant attention to our pedagogy best practices, the colleagues advices, the design suggestions and tools found in the scientific literature and in the web and, above all, thanks to our research activities. We registered everything we learned in a set of criteria, that we selected and organized in the rubrics' set (Fig. 3).
The rubrics' set in the present release is a tool for monitoring ex-ante, in itinere ed ex-post, the design, development and delivery of academic eLearning courses based on socioconstructivistic paradigm.
We use the "rubric" term because our tool is not only a check list of criteria to be verified. It includes the basic elements that have to be present in the course design and also gives suggestions to create a good constructivistic course.

The rubrics' set validation and the research methodologies
We wanted to be sure that both the contents and the form of our tool were valid (complete, correct, consistent, easy-to-use, effective, efficient) and for that we submitted the rubrics' set to a multiple check: 1. we asked an audit to the designers of the Trento University eLearning Division; 2. we offered the rubrics' set to the students of an eLearning design course we organized in 2010, and asked them to validate it; 3. we offered the rubrics' set to the eTutors of the Firb (i.e. Investement Fund for Basic Research) project whose title is "Net@ccessible: teaching/learning together and for all in a lifelong project". Net@ccessible is a project funded by the Italian Ministry of Public Education, University and Scientific Research to design and create an accessible online learning environment.
We had mainly a qualitative research approach.
We conducted semi-structured interviews with 20 eLearning designers and 5 eTutors. Some of them were new to the eLearning design and some very experienced people.
We collected eLearning designers needs and requirements and positive or negative remarks about the rubrics' set and we used them to improve our tool.
We discovered that adAstra is very useful for the eLearning design apprentices but it is also considered quite good, at least like pro memory, by the experts.

TIA and the "adASTRA" rubrics
In the following paragraphs we will look through the TIA course using the rubrics like a guide.

Needs and resources analysis rubric
I would like to start this paragraph with a simple example of how the rubric use would have allowed us to bypass a design mistake. Last year during IATI (Artificial Intelligence and Informatics Technologies, the course that replaced TIA from the academic year 2009/2010) in spite of all the experience made in the six previous years, we had to ask to two visually handicapped person to attend the course externally. Indeed nor the course site nor the educational materials were designed to be suitable for the software/hardware equipment (like text reader or enlangers) that allow visually handicapped person to use the PC and internet. A better initial analysis, and the use of an accessibility rubric (that we immediately created after this experience), would have allowed us to design an accessible course.
In the analysis rubric we consider three kind of elements:  the needs that the course would like to respond to;  the resources that are available;  some design choices (es: educational technologies) that could be anticipated already in this phase and that is good to know to plan the necessary resources.
We can refer to TIA like an example. The course have been created to let the students know the dynamics of a learning community, the online communication and the eTutoring principles. Essentially these are the needs we settled with TIA, because we think this topics are more and more important for learning. Due to the course content and to the fact that we designed a constructivistic learning environment, we had to plan the presence of an extended teachers staff (designer, eTutor, technical help desk, teacher: who?, how many?) and to decide which Learning Management System to use.
It was also necessary to create a user profile. We collected the data about the students age, the previous education, motivational factors, working situation (part-time worker, non worker, etc.), ICT-Information and Communication Technologies familiarity, english knowledge, collaboration and working in group ability, possible disabilities.
The designer had to collect many of this data also about the teacher and the eTutors.
If they are novice in the use of eLearning it will be necessary to foresee an individual support so that they can interact effectively with the students and among them. We had to plan in a different way the interaction between the teacher and the eTutor if the latter had competencies about the eLearning facilitation but not about the course content.
At the analysis' end we will have available these type of information:  goals, needs and objectives, educational contents;  course length, credits number, budget, technologies;  available educational materials (textual and multimedia), copyright, prerequisites;  users, teacher, eTutors, staff characteristics;  methods of quality monitoring;  some general idea about teaching strategies and assessment methods.
All these information will ease the following phases and, at the same time, will produce the design of a course feasible with the available resources and suitable for the specific users.

Design and development rubrics
The course design and development are intimately connected and developed together.
For making the design and development rubrics more useful we divided the elements to be planned and verified in seven criteria: For all this areas we prepared a series of questions that help the teacher to remember the course more important elements and the possible choices. It happen that a parameter is not suitable for the course we are developing and in this case we do not consider it.
For every parameter we analyze the characteristics that, following the rubrics, we considered for the TIA course.
1. Organization: users characteristic; prerequisites (basic knowledge about computer, internet and english), frequency of student access to the site to follow all the online course activities; frequency of teacher and eTutors access to the forum; maximum answer time delay to the students questions, so that he/she doesn 't feel neglected; face-to-face and online meeting agenda, qualitative monitoring by questionnaires and interviews. 2. Educational materials: they included a bibliography, a webliography, the teacher slides. The rubrics' set helps to monitor the coherence with the objectives but also the completeness, the contents updating and its adequacy to the users characteristics and to the credits number, accessibility, copyright. 3. Pedagogy: we choose a collaborative constructivistic teaching strategy, following the Jonassen thoughts (Jonassen, 2007), and also the Wenger community of practice (1998). The students formed a small group of 4-6 persons, so that the communication was more effective. The teacher proposed a scripted collaboration, i.e. the interpretation of some roles to facilitate the involvement of all the people. We foresee a weekly self-evaluation by the participants, commented by the teacher feedback. To make this design architecture effective was very important to explain to the students the teacher's pedagogical approach, what he will expect from them, and in which time frame. 4. Assessment: we decided to assess the students according to their participation (access, forum messages, etc.); to the group's assignments quality; to the result of a final written examination. Whichever assessment method is used is really important that it is coherent with the adopted teaching strategies and that it is explicitly declared since the course beginning. 5. Communication: our course was based above all on the forums (news, student/teacher; student/eTutor, helpdesk, caffè, group's forum). We used the email for solving possible access problems to the course site. Recently we began to use skype. And, if necessary, we used also the telephone. 6. Technologies: we integrated Moodle with some web 2.0 tools. We employed Delicious for social bookmarking; Skype for audioconferences; MediaWiki for collaborative writing/multimedia publishing. Another software that was extremely useful was CMap, to build collaboratively conceptual maps. Everything that could be helpful for the course was used. Paying attention not to confuse the students asking them to use too many different software environments. We gave all the necessary information about the software in classroom, and published them on the course site. 7. Accessibility: the course has to be as ergonomic as possible, allowing teachers, students and eTutors to navigate without any problems, using meaningful labels, essential links, clear information, etc... Environments and educational materials have to consider all the accessibility rules. Publishing the educational materials, the textual and multimedia assignments, and using the communication environment have to be very easy and "natural". After we discovered our course was not accessible for visually handicapped students we decide to create an accessibility rubric, that explain how to design and implement a site for student with disabilities, following the World Wide Web Consortium and the WCAG-Web Content Accessibility Guidelines (World Wide Web Consortium W3C, 2008).

Teacher's and students' feedback rubrics
The rubrics about the teacher's and students' feedback bring to mind the classical idea of monitoring through the participants (teachers, eTutors and students) impressions, opinions, and comments.
The students feedback have to give information about their satisfaction regarding some elements:  educational and organizational aspects: if there is something not clear is better if it surfaces as soon as possible, otherwise the participation will decline;  technological and communication aspects: no problems have to be found for what is concerned the access, the use, the participation;  the course in general: the positive and productive atmosphere and the adequacy of the workload are two fundamental elements. The problems concerning other aspects influence the former: if for example a student write a message many time without receiving a timely answer his feeling concerning the course will get worse (and that could cause dissatisfaction also in the group mates). Every problem detected or directly stated by the students have to be confronted and solved as soon as possible. If it is impossible to find an acceptable solution, it will be necessary to give a satisfactory explanation. In every case, it is mandatory not to let the answer in abeyance.
The problems' and suggestions' collection during the course allow for immediate in itinere adjustments.
The students' feedback is collected in many ways and in different moments.
 in the middle of the course -or more often-at the end of the course, through the two purposely rubrics;  all through the course, by means of the teacher's, eTutors' and technical helpdesk's forum The teacher's and eTutors' self-evaluation satisfy two requirements:  allow the designer to verify that there are no obstacles concerning the organization, the technologies, the communication. Indeed whether the teacher has a problem in publishing the educational material or the eTutor is not able to monitor the forum, the course will have for sure some problems. In fact delays and shortcomings from the teaching staff, if not explained at the right time and exhaustively, may give to the students the impression to be neglected.  allow the teacher and the eTutors to verify if they are doing what was established in the design phase regarding their tasks and the time to perform them (communication environments monitoring, assignments comments, textual and multimedia material publishing, etc. ) maintaining the often pressing pace of an online course.

Conclusion
Many interviews and focus group with teachers, experts or novices in designing eLearning module, testify that our adAstra rubrics' set is a valid help for creating effective socioconstructivistic eLearning courses.
Nevertheless we know that we are still far from creating a complete and validated tool useful to scaffold the teachers and the instructional designers in creating whichever eLearning course, with different teaching strategies, different multimedia environments, different students characteristics. And that this goal will be completed in many years, through the work of many researchers, teachers and Ph.D. students.
There are many topics we would like to consider in our studies, to be tested preliminary on the field and to be included in a comprehensive rubrics' set. We list the two more urgent here following: 1. we would like to consider how the affectivity influence the eLearning participation and the eLearning effectiveness, considering the theories recently presented in scientific literature by many authors (Campbell, 2006;Clore, & Palmer, 2009, Damasio, 1994Salovey & Mayer 1990, Kort et al., 2001). 2. we would like to widen our rubrics to include different teaching models. adAstra is now a good scaffold for designing socio-constructivistic courses. Although it can offer useful hints also for self-learning environments or transmissive courses, we need more researches to have a complete set of theoretically founded tools to scaffold the design of eLearning courses based on various pedagogical frameworks and to monitor their quality.
Next year, in the 2012, will be celebrated the Alan Mathison Turing year. Turing is an English mathematician, logician -that lived in England from 1912 to 1954 and was treated in an "appalling" way for being homosexual-widely considered to be the father of computer science and artificial intelligence. I would like, to honor him, closing with the words he used to conclude his most famous paper: Computing machinery and intelligence, published in October 1950 by the Mind review (Turing, 1950) We can only see a short distance ahead, but we can see plenty there that needs to be done

Acknowledgement
This paper was developed within the FIRB project 2007 RBNE07WBKL_004, entitled Net@ccessible: teaching -learning for one and all in a lifelong plan, funded by Miur (Italian Ministry of Public Education, University and Scientific Research) and by the Universities involved.