Assisted On-Job Training

Adaptive E-learning was proposed to be suitable for students with unique profiles, particular interests, and from different domains of knowledge, so profiles may consider specific goals of the students, as well as different preferences, knowledge level, learning style, rendering psychological profile, and more. Another approach to be taken into account today is the self-directed learning. Unlike the adaptive E-learning, the Self-directed learning is related to independence or autonomy in learning; it is a logical link for readiness for E-learning, where students pace their classes according to their own needs.This book provides information on the On-Job Training and Interactive Teaching for E-learning and is divided into four sections. The first section covers motivations to be considered for E-learning while the second section presents challenges concerning E-learning in areas like Engineering, Medical education and Biological Studies. New approaches to E-learning are introduced in the third section, and the last section describes the implementation of E-learning Environments. following:


Introduction
When dealing with critical systems that users must comprehend to take full advantage of their functionalities, deployment and support teams must take special care with users' training. Even when the development process takes in consideration testers' feedback, there is always the need for more users' hands-on training.
Of course, this is easier said than done. Users tend to be overloaded with regular tasks and finding time to assist a regular formation course is difficult, if not impossible in most cases.
This chapter presents the work being done in Cape Verde relating this subject. We have been developing an information system, named SIPP, to be used in the Courts of law in Cape Verde. SIPP stands for Sistema de Informação do Processo Penal (Criminal Proceedings Information System). SIPP is a joint initiative, sponsored by the Ministry of Justice of Cape Verde and developed from scratch by the University of Aveiro and University of Cape Verde. The SIPP has been in testing phase since June 2010 and will be put in production phase on October 2011.
Being a project developed from scratch, there were some barriers (cultural, practical and technical) that we had to overcome to succeed in the project.
One of our main concerns was to help users apprehend how the system works; so, along with online and printed material, we have developed a teaching system that simulates the different actors and automatically checks if the right information was collected from a given textual document, giving feedback to the users about the task's outcome.
To do so, we've deployed a training platform with the same web interface, but different data. On this platform, users could rest at ease on regards of the correctness of the information inserted. Moreover, this was a controlled scenario, so we could monitor and act accordingly when the actions made by the user were distinct from the expected ones. This chapter explains briefly the overall information system and explains in depth the workflow architecture used to deploy the assisted learning process.
From the preliminary results, there are a few key aspects worth mentioning:


Being a country divided in 10 islands, regular user's training, using traditional classroom approaches is very expensive (mainly due to the country's geography) and bothersome, mainly due to the shear amount of regular court work left behind during training stage; course materials, work deliveries and discussion forums are amongst the most used features.
In (Thibault, 2011) authors compare both e-learning platforms. Two of the most widely used Learning Management Systems are almost identical in terms of functionality (roughly 95%). The author states that with added plugins and extensions, it is possible to obtain 100% coverage of site's functionality.

On-job training -Final thoughts
Considering our scenario: teach users how to use a specific application, besides the approach of (Borchardt & Grap, 2010), no other approach seemed to benefit the teaching of SIPP's functionalities. Therefore, we started developing a model where users could not only practice, but also learn to use the system. This will be discussed in later sections.

SIPP overview
SIPP is a web-based information system, developed to support the digital workflow of law case files in the Courts of Cape Verde. With this system, the different may interact with the case files.
SIPP's users are the Court's administrative staff, Judges, District Attorneys, Lawyers, the Court's Presidents and Members of the Supreme Councils.
SIPP is an N-tier system, composed, in fact by two distinct applications: a web-based application and a service-oriented workflow engine. Figure 1 outlines SIPP's architecture.

Fig. 1. SIPP Overview
Each user will access the application and expects to be able to execute the same tasks as it would on Court. This means that access to sensitive information within the case files must be handled with care: each user cannot access more data that it would on the paper case files.

Database and business logic
The database layer consists of a high availability SQL Server cluster. This cluster holds database records and stored procedures for better query performance on complex queries.
The business logic consists of a set of classes and transformations for simpler queries and for regular CRUD (Create, Retrieve, Update and Delete) operations. Enhanced retrieve operations (information of related objects, complex cross table queries, etc.) is also handled on the business logic layer. All database handling is processed by this layer. This enables a one point entry for audit and log tracking of operations. Also, security enforcement mechanisms are in place at this layer, to prevent erroneous or misbehaved application or user access.

Workflow service
The workflow service is deployed on an application cluster, enabling high availability of the service. The service is built upon Microsoft.Net Workflow Foundation (Zhu, 2010). Service communication is built upon Windows Communication Framework (Resnick, Crane, & Bowen, 2008).
To orchestrate and manage the information flow from the handling of case files, we decided to use a workflow engine. This seemed like a natural step towards the dematerialization of the case files. Workflow engines are capable of handling long running processes. In this particular case, a single process may stretch for several years, depending on its complexity and number of appeals. One other feature that we intended to explore using workflow engines was the dynamic update of a given workflow. When referring to laws and systems that must adhere to the strict regulation of laws, if significant changes in the law are made, then systems must be updated. However, the updating method must consider the specificities of running processes and how these should be treated: in some cases, the new laws will only apply to new case files, in other cases the new laws will apply to all new case files and to some (or all) running process. The use of a workflow service enables us to handle these specificities and more.
SIPP's workflow service is based on several smaller services and workflows, as illustrated in Figure 2. Most of the workflows in place are internal to the service, meaning that they are only available for communication with other workflows. The main entry point of communication with the overall workflow service is the Versioning Service.
Versioning Service -This workflow handles document edition. Documents are edited online, using the web application. The versioning service handles any kind of data, from simple document edition to N-step forms. At any given time, users may save their work and resume it when needed. If required, they may continue editing the document on any given saved version (actual or older).

Request and Decision
-This workflow handles the basic process of submitting the request (by any party in the case file) and decision of the request (usually by the judge of the case). Judges may also issue decisions with no pending request. This workflow is used on both cases. Workflow actions are triggered by the reception of completed documents from the versioning service. This workflow mimics most of the traditional (manual) information process. However, given the inability for computer systems to accurately understand textual documents and automatically extract meaningful information from it, we have added an extra step (when comparing with the traditional paper version) to the workflow: after the decision document is received, the administrative staff must extract the meaningful information from the document, "explaining" to the application what to do with the decision received earlier. Despite seeming as an additional and cumbersome task, it is vital for the correct functioning of the entire system. The top-level workflows (Habeas Corpus, Regular Process and Appeals) are triggered based on the information extracted by the administrative staff. The upside of this extra step is the ability to latter extract a series of reports and statistical information from the entire system, at no extra cost. Request and decision workflow consists mainly of three events: request received, decision received and explanation received. Each event is triggered by a different actor.

Fig. 2. Detailed view of the workflow service
Habeas Corpus Service -This workflow handles the entire process related with Habeas Corpus case files. It receives events triggered from the request and decision workflow and from the Appeals workflow.
Regular Process Service -This workflow handles the regular court case files, regardless of the process' complexity. Regular Court process case files are divided into four categories, depending on the nature of the crimes and the Judge's decision. It receives events triggered from the request and decision workflow and from the Appeals workflow.
Appeals Service -This workflow handles the case files on Higher Court instances. It receives events triggered from the request and decision workflow, from the Habeas Corpus and from the regular process workflows.

Web application
User-wise, the web application is the single point of interaction with the system. For security reasons, the communication between the workflow service and the web application is restricted to just that; users may not directly communicate with the workflows.
The web application enables users to access the system and work on their tasks and portfolio. In this scenario, portfolio refers to the active case files of a given user.
Workflow usage was kept to the minimum, meaning that only effective workflow-like tasks were to be deployed as workflow services. The remaining information access is done by reaching directly to the business logic layer. When triggered, the workflow layer processes information and permanently stores it in the database, through the business logic layer. Therefore, it is possible to assess all information by using the business logic layer. Besides giving a faster access to information, it lessens the load on the workflow engine, making it available to other requests.
1. Figure 3 presents the look and feel of the web application, with 8 areas marked: 2. User's location: enables the user to rapidly identify the currently working area; 3. Main menu: this area controls the top level sections available to each user. Depending on the user's roles, the available options may differ. 4. User's control panel: the area holds the user's identification and session termination options. It also enables the access to the user's control panel, where he may edit personal information. 5. Portfolio case file holder: this section lists the user related court file process in progress. 6. Court file process information: the information about any given process is organized in tabs, for easier reading. 7. Case file tasks: lists all pending tasks related to a given process. 8. Pending actions: lists all on-going user tasks related to a given process. These are organized by vertical tabs. On-going tasks are usually related to versioning documents and decision processes. 9. System and page loading information. As mentioned, the process of detailing or dissecting the information within a decision was called a "Decision explanation". In this stage, the user (usually an administrative Court staff member) must extract meaningful information to feed the system. This means that for each type of considered meaningful information there is an explanation form associated. Figure 4 presents the explaining form of a final sentence on a case file. In this case, the accused was found guilty as charged and condemned.
In Figure 4 this information as already submitted by the user. The next step is to fill out the information related to the Crimes (on which was the user found guilty on which was he found not guilty) (1), the main sentence (2) with information on days in prison, fine amount, etc., accessory sentences (3) as inhibition to drive, and finally, information regarding civil damages (4).

Demographics and regional concerns
Cape Verde is an archipelago of 10 islands that cover around 4,000 square kilometres of ground. Across these islands, there are 16 regular Courts and 3 higher instance Courts, where this application must be available, and with which the District Attorneys, Judges and Court staff must be at ease. These 16 regular Courts are unevenly divided, with most of the islands with just one Court, consisting of less than 10 users per Court.
Communications wise, the government sponsors a public network, available for all the country's public services, Courts included. Figure 5 depicts such network. All communications converge to the main island, Santiago. Most of the national online services are based on Santiago and, more importantly, it is through Santiago that all International network traffic flows. Inter-island communication links are usually saturated at about 95%, during working hours. Despite this obstacle, regular internet communication is feasible from almost every public office, thanks to strong cache mechanisms and quality of service rules imposed by the network administrators.
Along with the heavily packed inter-island network, one other technical difficulty usually arises: power outage. Given its archipelago nature, each island must be able to produce its own electricity. From time to time, due to generator problems, excessive demand and other reasons, an island region is left in the dark. With no electricity there is no communications and therefore, no web services. This holds true not only for the physical location (Court, lawyer's office, etc.), but for power outages along the network routing to the datacentre. This means that, depending on the outage location, a region or the entire country may unable to access any given web service.
These conditions mean that any given online system deployed in these conditions, to be considered "environmentally full proof", must be able to support off-site processing. In this case, the option was to go back paper trail and later proceed with the digitization of the record produced during the inaccessibility period. Adding to these tough conditions, the familiarity of the court's staff with computer systems is below the desirable level. Most of them have, in fact, a computer standing in their desks. And most do use them for work tasks (and received training to do it). However, the regular usage usually involves repetitive tasks, consisting on text edition and printing. Given the overload of case files per Court, staff usually does not have the on-job time (or at ease) to pursue on practicing, discovering and learning additional features.

Assisted on-job training using SIPP
Considering the concerns that were raised on the previous section, we had to devise a way to overcome the staff related problems and to mitigate the impact of the infrastructural problems. As mentioned, the infrastructure problems were mitigated by enabling users to post-process documents.
The staff related problems, however, posed a new set of challenges. How could we guarantee a satisfactory level of basic computer skills to all Court staff, Judges and District Attorneys? How could we teach them to use a new application, with minimal downtime of the Court's service? Considering the different social stratus amongst the Court staff, how could we ensure a teaching service where each person would feel comfortable to reach out and express his difficulties, without being judged by others?
To ensure a satisfactory level of computer skills, the University of Cape Verde was put in charge of an ambitious program of teaching all Courts' administrative staff on basic computer skills. These sessions were conducted in several islands, giving the opportunity to the staff to refresh or learn basic computer skills, as scanning, text processing, spread sheet processing and web browsing.

Human assisted on-job training
Following this learning program, users apprehended most of the skills required to the next level: SIPP specific training. Considering the responsibilities of each person and the available features for each role, we were required to perform different learning sessions, covering the different features assigned to the different roles.
When the first prototype was deployed, still on early testing phase, we started the training of the users and the validation of the prototype concept. After each round of testing and training, we delivered to the development team a set of results and opinions of the users about the system. This way, we could try to improve the perceived usability of the system to the next round. The first rounds, until the interface concept was considered stabilized by the users, were done with the same participants. This way they could better understand the impact of their opinions in the system. As mentioned, in most islands there is just on Court of law. In each Court, there are at least one Judge, one District Attorney and four administrative staff elements working every day. This meant that we would either move all personnel to common facilities during the SIPP training, or we would take the teaching team in a visit to all Courts. Due to the country's geography and considering the shear costs of transporting all the Court's personnel to a common location, the choice of having an itinerant teaching team prevailed. This team would perform one-to-one on-job sessions, with no need to close the Court to the public, since this would be done based on pre-set scheduling, adjusting regular tasks with this additional task. This on-job, one-to-one teaching method, proved to be extremely effective. Trainees were at ease to pose any question they felt like, for more basic that it would seem to them. This could only be achieved in such personalized and private environment as the user's office or desk. This way, we could assess the effects of the basic skill learning program: based on the ease revealed during SIPP training and based on their technical questions.
SIPP's human assisted on-job training consisted of two separated phases: an overall user related explanation of features (done by the teacher) and a hands-on experience, where the teacher would observe and ask for the fulfilment of a set of given tasks. On every session, this was the stage where most of the time was spent. During the first iterations, teachers often had to assist users and walk them through the required task. The session was considered as terminated when users were able to execute a fair amount of tasks without the teacher's intervention. During the entire process, teachers would take notes referring to the user's interaction. These notes were then reported to the development team, for user interface refining.
Even before the first SIPP on-job training session, a new set of questions were raised. How will users keep their learnt skills? How often can we make this kind of intervention? The answer to the last question was easy: if possible, never again. Just one visit per Court was planned. Despite being cheaper than closing the Court for two or three days and moving everyone to different places, with the intrinsic costs on travel, accommodation and meals, this approach still has the cost of having the teaching teams on the field. To answer the first question, we deployed a testing web application, similar to the online system, where users could exercise what they had learnt and continue exploring the application. This testing application will still be available when SIPP enters production phase.
As always, this led us to a new question: How to assess and validate the users' training without human supervision?

Computer assisted on-job training
To answer the question raised in the last section, we developed a computer assisted on-job training agent, an inspector, that could overcome the lack of human supervising and, at the same time, reassure users about the outcome of their practice. This inspector is bounded to the main workflows: Habeas Corpus Process, Regular Process and Appeals. To validate the information state of a given electronic case file, we partially adapted our unit test framework, so that we could validate on the fly the information being inserted (resulting of the decision's explanation) with the expected information.
Every time a decision is explained and an event on these workflows is triggered, this agent inspects the outcome of the explanation, and compares it to the expected outcome. In the event of mismatch, the agent triggers a new decision for that process, explaining to the user his mistake, and what to do to correct it. When the outcome of the explanation matches the expected outcome, the agent signals the main workflow to proceed the information processing. Figure 6 shows the overhead processing in the workflow engine to support the computer assisted training. It represents a triggered event, the reception of the explained decision of www.intechopen.com setting a judgement day for the case file (1). Upon reception, we check whether the process is a training case file or a regular case file (2). Being a training case file, we make a second check, to compare the information received with the expected information (in this case, this would be the time, date and place for the judgement, along with the defendant lawyer (if necessary) (3). If everything is as expected, or if this process is not a training case file, then the inspector allows the workflow to continue (4). Otherwise, it will not allow the workflow to continue and will issue a new request/decision, stating what was wrong and how to correct it (5).

Case file generator
The final piece for this puzzle was a case file generator, where we could define training session parameters, like the number of case files to open, the case file state (according to the main workflows), the initial process information and a set of requests and/or decisions (depending on the user's role) for users to process. We developed a model-based random text generator that would open case files, submit requests and decisions and, most importantly, record the expected outcome of the request or decision.
Pending user's action, the inspector agent compares both data and decides whether to allow the workflow's execution or not.
This case file generator is used during all training phases (teacher assisted and computer assisted). This way, users were already familiarized with the generator's way of producing textual requests and decisions when they started using the computer assisted on-job training application.
The case file generator was inspired on our internal testing mechanisms. Besides single unitary tests, we also need to test the workflow as a hole. However, it is impossible to test the same case file with all the options available at the workflow. To handle this challenge, we developed a testing tool that simulates the web application, communicating directly with the Business Layer and with the Workflow Service.
The test tool would issue new case files, inquire on the available state options to trigger and selectively choose one event to trigger. Testing would continue until the case file was terminated. The test tool was preloaded with a set of rules and conditions to guarantee that we had covered nearly all options within the workflows. These rules and conditions could be in the form of probabilistic influence (when in state X give an N% probability to event Z) or the form of closed pathways (when in state X, proceed with event Z).
The test tool also served another important aspect: assessing the overall system performance. Since the testing tool, workflow service and database were detached modules, these could be executed from different computers. This way, we could test the limitations of our approach and proceed accordingly, early on the development phase.

Case file planner
Along with the case file generator, we developed the case file planner. After the case file generator publishes a new process, for every process state change triggered, the case file planner inspects the current process execution phase and acts accordingly, impersonating the remaining parties of case file. It is up to the case file planner to devise the actual traversed workflow for any given process.
As with the case file generator, the planner was also inspired by our internal testing mechanisms.
More than simply generating random case files using pre-defined situations and models, we needed users to experiment the full length of the process, according to their responsibilities. This way, users could help us assess the validity of each workflow and propose modifications, if required.
A simple path for a case file would be, for example: 1) when the judgement day is marked, issue a new witness request; and, 2) on the judgement day, issue the final judgement act, with the final decision about the accused party. As with the testing tool, probabilistic pathways could also be defined, providing a wider range of training options, with the same amount of supervision preparation.
Each request/decision issued will then be validated by the inspector agent previously presented.
The case file planner is an iterative agent that prepares tasks according with user's expertise on the system. To do that, it needs to understand the user's evolution so, as in a computer game, it ranks users' achievements. This way, it may plan more complex case file paths to more experienced users and more simple tasks to users starting their training or struggling with the system. The user interaction is recorded jointly by the planner and the enforcer, allowing the supervising team to personally compare the expected outcome with the actual outcome produced by the user. This additional debugging feature

Conclusion
When we started this process, users were apprehensive about what to expect as an outcome. This was the third attempt that the government had made to implement such a system and users who had participated on previous attempts were reluctant to start yet another project. The project's initial risk was in fact high. The fear factor associated to a new system combined with the perception of a possible "Big Brother", working for the Ministry of Justice and against the Justice personnel, wer e m o r e t h a n e n o u g h t o p u t u s i n a n uncomfortable starting position.
We managed to get around the user's initial misbeliefs by presenting them the advantages (and disadvantages) of such system. These presentations have shown us that people were eager to use an electronic system to support their work, as long as the infrastructural and technological problems were solved and the security and privacy concerns could be verified at all times.
The development process included elements of each perceived role in the Court. This benefited the overall result, since these members were the first to see the developments, to comment and propose new directions on what they felt was incorrect. More importantly, they would comment with their colleagues on the developments of the system and on the possibilities that the system already possessed. This information dissemination generated a positive expectation wave along the users. This was indeed observed when the general testing and training phase started.
Throughout the training process, we realized that some users were still struggling with basic computer skills. To help them overcome this situation, a new computer course was proposed, more specific and focused on the user's difficulties.
We produced printed and electronic versions of the system's manual, adapted to each user's roles. Along with this help, we pursued on a one-on-one on-job teaching methodology. With this approach, users were more collaborative than on initial group sessions. Moreover, they would feel comfortable enough to pose questions relating both SIPP and general computer subjects.
In this chapter we presented SIPP and explained in detail the workflow architecture used to deploy the on-job assisted learning process. The on-job assisted training was performed on two separated phases: one-on-one human assisted training and computer assisted training. The first phase helped users understand the insights and working mechanisms required by the SIPP. The second phase helped users gain confidence about their autonomy and ability to use the system.
To give users the proper feedback, even on the second training phase, we developed an inspector agent. This agent had the responsibility to assess user's submitted information and instruct the main workflows on the outcome of such information. In the event of erroneous information submitted, the agent would also issue a new request/decision, impersonating the requestor/judge of the case file. With such feedback, users were reinsured that the training would produce the desired results.
The case file planner enabled us to understand where some unforeseen problems were. When several users struggle to perform a planned task or work plan, the planner informs the management team to further investigate on the roots of the problem. It may be an interface deficiency, a programming bug or a modelling fault.
The training application inherited most of the features we had in place internally for testing purposes. This way, the training application was relatively cheap, since it consisted of refactoring the test framework and deploying a series of agents, for inspecting and for producing test information.
Trainees found that this training application was helpful to understand the several available options faster and without worries of destroying real life work due to incorrect system operation. The project managers, however, found that most of the trainees (Court staff) lacked on sufficient computer usage knowledge. This was an important assessment that triggered an initial course on computer usage for all Court staff, prior to launching this application.
The initial goal for this project was met: deploy an on-job assisted learning application, with the same look and feel of the "real" application, where users could try all the application's features without worries of doing things wrong.
After the initial training, we monitored user's activity in the system, and realized that users were in fact using and testing the system on their own. Some users were just exploring the SIPP's features, while other were in fact doing their homework, working on the case files generated for training purposes. Despite being on-job training, some users were actually accessing the system from their homes, since the system is available online.
However, the amount of users actively using the system after the initial training started to decay around two weeks after the one-on-one training. Despite being a relatively normal situation, since users cannot neglect their regular tasks, it is far from the desired usage patterns. With low usage patterns, users tend to forget, at least partially what they had already learnt about the system. We expect that with the paced adoption of the system on all Courts during the next months, users will find time to refresh their SIPP's knowledge.