Content analysis and occurring themes.
Evaluation of clinical skills is a demanding and complex process and is dependent on many complex factors, such as teaching and learning approaches, simulated learning, and psychometrically validated assessment tools. Therefore, it is imperative that adequate strategies and methods are employed to evaluate the success of a nursing care activity. One such strategy in the field of nursing care is the application of objective structured clinical examination (OSCE) of a nursing activity. The purpose of this article is to highlight the importance of evaluating nursing activities in a simulated clinical environment with OSCE to determine synchronicity of the teacher and student assessment. A cross-sectional study was carried out, in which we compared the evaluation of nursing activity by the teacher and the 51 students. Summative content analysis was used to analyze open-ended questions about possible improvement of performed nursing activity. The data showed a large discrepancy (81.9%) in evaluating nursing activity between the teacher and the student. The synchronicity between the teacher and student assessment modality occurred only in 18%. Students were mostly less successful in their assessment of competence with knowledge about carrying out interventions (36.5%), preparing for interventions (24.3%), and infection control (14.4%). Clinical skills acquisition remains an essential element of a student nurse’s development, as competence in nursing skills is essential to patient safety. Simulation is viewed as an increasingly popular approach to the teaching and assessing of clinical skills. The process of evaluating nursing activity demands the usage of objective instruments that require objectivity, fairness, impartiality, and comprehension. The use of OSCE is one such method of promoting reliable and valid assessments in nursing skills.
- nursing care activities
Nursing is a discipline based on clinical practice–based interventions  with the purpose of nursing development and adjusting to needs of contemporary society . The study of nursing care must prepare students to carry out and accept professional responsibility in a changing clinical environment [3–5]. Only in a clinical environment does a nurse's knowledge become most evident . Because of the importance of clinical training, appropriate strategies for clinical assessment must be sought, with the purpose of specifying a success rate of clinical training . Such strategies are important for evaluating the quality of teaching and learning processes , identifying student's weaknesses, increasing their motivation for competence achievement, and helping teachers to assess their work [9–12]. Knowledge evaluation is a time intensive and complicated process, leaning mostly on teacher's subjective judgment. In addition, students identify they are not satisfied with current methods and results of assessment [7, 13, 14].
Simulated learning using objective structured clinical examination (OSCE) has emerged as an alternative teaching method to assist students in acquiring clinical skills competency because it attempts to replicate a real-life situation in a simulated environment [15, 16]. There is a current trend in nursing education to use OSCE to assess and examine clinical competence [16, 17] within this simulated setting. The OSCE was originally designed for the medical profession  but has been modified and applied in nursing education, providing means to assess competence in a simulated environment without posing a risk to patient safety [16, 19]. A number of studies using various research designs including qualitative [20, 21], quantitative , and mixed methods [23, 24] have all reported positive results from using OSCE as a teaching methodology .
Objective structured clinical examination is a versatile multipurpose assessment tool that can be used to evaluate the nursing care that students deliver to patients in a clinical setting . It is used in a planned and structured manner with a clear emphasis on objectivity . Additionally, as an assessment tool, it uses a task analysis of general clinical skills  specific to the student’s program and year of study and as stated previously is usually carried out in a simulated environments [28, 29]. Contrary to traditional assessment modalities, OSCE evaluates areas that are most critical to nursing competence, for example, blood pressure, pulse, respiration, communication skills, and various other nursing interventions  in a simulated work environment . The OSCE has for a long time been recognized as a formal assessment method in medical education  and is now more frequently used as an assessment method in nursing [32, 33]. It is argued that despite the rhetoric of a student-centered approach, education remains wedded to conventional teaching and learning approaches, which fail to engage with the individual and unwittingly silence the individual’s voice [34, 35]. Therefore, it is imperative that more contemporary student-centered approaches to teaching and learning are incorporated in nursing assessment.
The OSCE assessment tool was introduced to the University of Maribor Faculty of Health Sciences to examine performance of various clinical skills in a simulated clinical environment. The course unit of study was “Nursing Care in academic year 2015/2016” (first year students of nursing care). Standardized assessment checklists were evaluated together with the teacher's assessment of clinical training success. Students were encouraged to reflect on various nursing case scenarios in order to recognize mistakes and ensure competent practice. Simulation as a teaching method allows for multiple learning objectives to be taught in a realistic clinical environment without causing harm to patients and can provide ‘microworlds’ whereby important interactions between patients, doctors, nurses, and other health professionals can be highlighted, illustrated, explained, and replayed [16, 36].
Emphasis was placed on recognizing the mistakes made by both students and teacher when carrying out various nursing care skills and on the aspects of practice they did. The following research questions were posed:
Is there consensus between student and teacher evaluations when assessing nursing practice using an OSCE?
Does a student assessed with OSCE method recognize mistakes made when carrying out various nursing care skills?
Can the student critically evaluate the performed nursing care activity in concordance with teacher’s evaluations?
To answer the research questions, a cross sectional observational research study was used. Nursing students performed care activities within an OSCE situation and then assessed their own performance. The nursing care was also independently assessed by a teacher. In this study, we compared these assessments. To explore this, we conducted an analysis of open-ended question about possible improvements of nursing care activity using summative content analysis.
2.2. Setting and participants
An initial available sample of 51 participants was recruited from a cohort of first-year B.Sc. general nursing students using an available sampling approach. Available sampling involves taking a sample of what one would call typical, normal, or average for a particular phenomenon under study . All students were recruited through purposive sampling. The student successfully completed training in the clinical skills laboratory for nursing care of an adult patient and also successfully passed mid-term assessment. Participants were all first year nursing students who had completed the same clinical skills training course in the 1st degree study program of Nursing Care. The majority of participants (46.90%) were female. The response rate was 56% (n = 52). This means that more than half of the students completed the OSCE and wrote a reflective piece. Assessment was carried out by higher education teachers of nursing care in January 2016. Students had to meet the following inclusion criteria: (1) active participation at laboratory clinical skills in a simulated clinical environment (150 hours), (2) student's assessment based on OSCE assessment paper, (3) student's reflective writing after completing the nursing care activity, and (4) written self-evaluation.
2.3. Ethical considerations
The study complied with ethical procedures of research, the Declaration of Helsinki, provisions of the Oviedo Convention and Code of Ethics for nurses and medical technicians from Slovenia , which was adhered here throughout the process. Research participants' anonymity was assured by numerical coding of their written self-evaluation sheet, so that the participants could not be identified from the information. Data collection took place in classroom settings, and participation in the research was completely voluntary. The research team was the only people who had access to the data, and the data were stored on a computer that was password protected.
2.4. Data collection
Students' practical work at the course unit Nursing Care was assessed with objective structured clinical examination checklist OSCE, which consisted of 42 clinical skills. The checklist was composed by teachers, which perform training for students in the clinical skills laboratory. Each checklist of simulated interventions consisted of a series of performance-based observations and rated students’ performance as completed accurately; inaccurately or not completed. Students were acquainted with the method of evaluation for their clinical skills competence, and they also had the opportunity to view the assessment checklist items. All OSCE assessors had worked previously with the students in the clinical skills laboratory. An independent evaluator (a teacher who didn't participate in the training process) assessed the students during the performance of the nursing care activity by observing and filling out the assessment checklists. Using OSCE, 42 nursing care activities were evaluated. After performing the nursing intervention, the students had approximately 10 min to evaluate their nursing care activity using an unstructured method of reflection with the purpose of analyzing mistakes and endeavoring to improve future practice of such skills and prevent student from being awarded a fail grade. To assist the critical reflection process, students were posed a question: What could you have improved during carried out intervention or would you improve something?
2.5. Data analysis
The first and last author transcribed the students’ reflective writing (after 1 month of the evaluation process). The transcripts were then analyzed and coded by two authors (ZF & KČT) using descriptive statistics and summative content analysis  for analysis of the open-ended question. This approach contains a quantitative element, because the results show in the proportions of how many times some words are repeated or a response depending on the particular subject of research. Summative content analysis includes counting and comparing keywords and content with the interpretation of the meaning of these . In the first step, the reflective sheets were read carefully, trying to obtain a sense of the entire contents [39, 40] and then to derive topics. We highlight the exact word from the text, which covered the main idea or concept. The incidence of identified keyword and concepts were then calculated and presented in percentages. This method was chosen as there was a large quantity of data  and using this approach enabled the opportunity to quantify words or content and subsequent data interpretation.
Answers to the open question were categorized in six themes (see Table 1) and organized after occurrence—from most frequent to less frequent repetition.
|Theme||Number of items||Percentage (%)||Meaning|
|Knowledge needed for carried out interventions||159||36.5||Theoretical and practical knowledge|
|Preparation for intervention||106||24.3||Assuring privacy, condition evaluation, preparation of the room and accessories|
|Infections treatment||63||14.4||Disinfection of material, accessories and hands, preventives, recycling|
|Safety assurance||51||11.6||Placement of safety protectors and buzzer by the bed, comparison of identity and doctor's requirements, marking of material|
|Communication skills||43||9.8||Explanation of purpose, goal and course of care intervention, obtaining of an agreement, passing future instructions|
|Treatment after intervention||15||3.4||Fixing up the patient, arranging of material and of the room, documenting|
The analysis of clinical skills' evaluation (performing a nursing care activity) showed that students were least successful at themes of “Knowledge needed for carried out interventions (159 [36.5%])” that contained theoretical and practical knowledge needed for the successful performance of an intervention. Themes of category “Treatment after intervention” were assessed as less successful from student's and teacher's assessment viewpoint (15 [3.4%]).
Student's and teacher's assessments were in harmony in 18.1%, regarding all items of the assessment papers. In 81.9%, the assessments were not in harmony (Table 2).
Most discrepancies at evaluation were noted in items of “Knowledge needed for carried out interventions,” where discrepancy between the student's and teacher's assessment was 37%. Least discrepancies were noted at items of “Treatment after interventions,” 3.5% (Table 3).
|Theme||Total number ofitems||Percentage (%)||Student's grade N (%)||Teacher's grade|
|Preparation for intervention||88||24.3||24||19.7||64||26.6|
|Knowledge needed for carried out interventions||134||37.0||41||33.6||93||38.7|
|Treatment after intervention||13||3.5||5||4.0||8||3.3|
For assessment of a nursing care activity, we sought approaches that help teachers objectively evaluate student's knowledge. One of such instrument for evaluation of theoretical knowledge and performance of nursing care activity is the OSCE . In the research carried out on 51 first-year nursing care students, we wanted to find out differences in assessment of a nursing care activity between a student and a teacher in a simulated environment. The data identified a big discrepancy (81.9%) in the evaluation of nursing care. Harmony between the teacher and student was present only in 18% of the data. On completion of the activity, students had to write their reflection about performance and answer questions on possible improvements. In this process, they showed a lack of criticism and in-depth performance insight. This supported other literature  as the most critical assessment areas: communication skills and ability to manage patient's inappropriate behavior. In our research, we found out that a student was least successful in the knowledge area of intervention performance, preparation for intervention, and infection management. Results indicate that students lacked self-criticism and lack of theoretical knowledge. The reference  states that objective methods for assessment are an appropriate alternative to traditional knowledge assessment forms. These methods are even more effective because of instant feedback of students and teachers . As in previous studies [44, 45], the methods of objective assessment requires a lot of time, many teachers, and financial investment, although this should be balanced with greater satisfaction to students and teachers.
4.1. Limitations of this study
Our research has some limitations that require careful consideration. Because of a small sample, generalization is less reliable. Students were not assessed using exactly the same interventions. They chose interventions randomly, and they were not the same level of difficulty. Audio and video recording as a feedback method on their own performance for recognizing mistakes was not assessed. A structured reflective model was not used as students wrote their reflection based on questions posed by the teacher. Finally, the assessment was performed by one teacher; therefore, adequate tests of objectivity cannot be assured.
5. Implications for nursing education
OSCE is an assessment technique in which students perform nursing interventions under a variety of simulated conditions. Through the assessment of students, performance on the same skill and of the same place, the equity, and the objectivity to the students are enhanced. The introduction of a student self-assessment checklist contributed to a reflection of their own performance of each step of the skill before evaluation with OSCE and prompt identification of student’s strengths and weaknesses during exercise of skills performance. Using the OSCE as an assessment method that is based on real scenarios increased the confidence of student.
There has been increased attention over the past decades on the use of objective structured clinical examination systems as a training and assessment tool. OSCE mainly assesses basic knowledge and technical performance of an activity. It is harder to assess the ability of critical thinking, interacting communication skills, experiencing, student's expectations and skills, in order to treat a patient like a whole. The use of new assessment systems is limited and one-sided; therefore, a richer combination of assessment systems is required. In pursuing clinical competence in nursing students' clinical skills, nursing teachers have to endeavor to find new ways to assess and examine these essential skills that are necessary for patient safety and care. Future studies should include a larger sample of students, and the use of a reflective model to capture student perception of learning is recommended.