Characteristics of studies included in the literature review.
Introduction: In Slovene nursing higher education, there is a lack of empirical evidence to support the choice of tolls for assessment of clinical skills and competencies. This literature review aims to critically discuss identified methods of clinical nursing skills assessment and competencies currently used in nursing higher education in other countries.
- clinical skill
- clinical competence
- nursing competencies
Nursing students have to develop clinical knowledge, skills, and attitudes for professional practice, and nursing educators have to assess and evaluate students’ core skills readiness for clinical practice , and the assessment should be a real indicator of knowledge . Assessment in clinical practice can either be formative or summative , with the formative often used to discuss and analyze students’ performance  and the summative examining practical performance in the clinical or simulation environment . Both methods should ensure that the criteria for assessment reference the intended learning outcomes .
Three approaches to assessment of assessing nursing students’ nursing competencies were identified from the literature and include observation methods [1, 7], self-perception methods  and methods combining both approaches . Of these methods, observation of student performance and the use of skills checklists appear to be the most common [10, 11]. This can be done either by direct observation in the clinical environment [7, 12] or by observing the student in the clinical skills laboratory (CLS) using scenarios and clinical skills checklists to measure performance . Other multimethod approaches are used and include clinical portfolio evaluation , along with critical incident reports, case-based assessment, peer assessment , and reflection . Reflection is important because nurses need to think critically and reflection develops responsibility in clinical practice .
The last decade has seen the emergence of new measurement tools being developed and tested for validity and reliability . These include the objective structured clinical examinations (OSCEs)  that have numerous advantages over other observation tools , such as the development of student’s self-confidence , the grounding of more expressive learning , and the assessment of not only psychomotor skills but also allows for the assessment of knowledge and attitudes . The OSCE, however, is not the only assessment tool used in nursing education. There are numbers of different scales for assessing student’s competencies and psychometric properties [21–23]. This literature reviews, therefore, set out to identify and critically analyze current methods of clinical nursing skills assessment and competencies used in nursing higher education in other countries with regard to developing a comprehensive and effective method for assessing clinical competency in Slovene nursing higher education.
The aim of this literature review is to identify methods of clinical nursing skills assessment and competencies currently used in nursing higher education in other countries.
2.1. Eligibility criteria
Studies were included if they met the following inclusion criteria: empirical research primarily focused on methods of clinical nursing skills and competencies assessment and their reliability and validity, full-text available articles published in peer-reviewed journals and written in English, published between 2010 and 2016. Exclusion criteria were systematic review articles, assessment of clinical nursing skills in vocational training, assessment of special clinical nursing skills, editorial and commentary pieces, and all other literature not meeting the inclusion criteria.
2.2. Search strategy and study identification
Three electronic databases were searched for relevant literature: Medline, CINAHL, and PubMed. Key word combinations that were used included competency, competence, clinical competency, clinical competencies, clinical skill, clinical competence, professional competence, competency based education, assessment, measuring, measurement, test, scale, standards, validity, reliability, generalizability, and nursing student. Literature published within the last 5 years was searched due to the contemporary interest in clinical skills and competencies assessment in nursing.
2.3. Study selection and extraction
Identified references were merged with reference software EndNote, and duplicates were removed. The titles and abstracts of the identified results were then assessed for eligibility criteria by two of the authors (DV, ML). Studies not relevant to this review were removed. After retrieval of the full text, two of the authors (DV, ML) independently screened the studies and made decisions concerning final inclusion of the studies. A further two reviewers were then consulted (NMR, MS). Disagreements were solved by discussion. Data were extracted by predefined criteria, which included source, country, objectives, methods, and main findings.
2.4. Assessment of study quality
The Mixed Methods Appraisal Tool (MMAT) was used for assessing their quality. The tool is useful for appraising quantitative, qualitative, and mixed methods studies . Methodological quality criteria are scored on a nominal scale. The tool includes two screening questions and four criteria for qualitative studies, quantitative randomized controlled trials, quantitative nonrandomized studies, quantitative descriptive studies, and three criteria for mixed methods. The score is based on the number of criteria met divided by four (from one criteria met—25% to all criteria met—100%) . Each study was checked for quality by one author (NMR) and then rechecked by two other authors (ML, MS). Disagreements were solved by discussion until consensus was reached.
2.5. Data synthesis
A convergent qualitative synthesis design was selected and results from the identified studies were transformed into qualitative findings , using a narrative synthesis as described by Harrison et al.  and Dixon-Woods et al. . This approach was selected as studies were heterogeneous.
3.1. Study selection and its characteristics
The search revealed a total of 160 records. Figure 1 provides a flow diagram of the literature selection process.
The flow diagram (Figure 1) shows that after removing duplicates, 129 records were screened by title and abstract for their relevance, leading to the exclusion of 77 records. The remaining 52 full texts were assessed for eligibility. Critical reading of the full text led to 12 studies being retained for inclusion in the review.
3.2. Methodological quality of studies
The selected studies were conducted in Australia, Sweden, Iran, Canada, Ireland, Spain, Pakistan, and Taiwan. The studies have utilized different study designs, and a number of different methods were identified including a variety of assessment tools, OSCE, complex approaches, and others. There are presented selected studies objectives, design, main findings, and the MMAT score in Table 1.
|Source and country||Objectives||Design||Main findings||MMAT|
|Athlin et al. |
|To describe the development and evaluation of a model for a National Clinical Final Examination in the bachelor nursing education.||Collaborative project between four universities and adjunctive healthcare areas supplying clinical placements using the Delphi technique and literature review followed by evaluation.|
73 students included in theoretical test and 68 students included in bedside test.
|Hengameh et al. |
|To compare the effect of applying direct observation procedural skills and routine evaluation method on clinical skills of nursing students.||Randomized clinical trial.|
Nursing students included.
|Hsu and Hsieh |
|To develop a competency inventory to measure learning outcomes of baccalaureate nursing students and to test its psychometric properties.||Cross-sectional survey.|
599 nursing students included.
|Iglesias-Parra et al. |
|To develop an evaluation system of clinical competencies for the practicum of nursing students based on the Nursing Interventions Classification (NIC).||Psychometric validation study: the first two phases addressed definition and content validation, and the third phase consisted of a cross-sectional study for analyzing reliability.|
The population of undergraduate nursing students and clinical tutors.
|Imanipour and Jalili |
|To develop a comprehensive assessment system for nursing students in their critical care rotation based on a programmatic approach.||Development in three phases followed by assessment: determination of the educational objectives based on the nursing curriculum; identification of a list of appropriate assessment methods, selection; determination of a content validity.|
38 bachelor nursing students included.
|Khan et al. |
|To identify nursing students’ perceptions about the effectiveness of utilized teaching and learning strategies of clinical education, in improving students’ knowledge, skills, and attitudes.||A descriptive cross-sectional study design using both qualitative and quantitative approaches.|
74 nursing students included.
|Levett-Jones et al. |
|To describe the design, implementation, and evaluation of the SOAP, a model used to assess third-year undergraduate nursing students’ clinical competence.||Evaluation of Structured Observation and Assessment of Practice (SOAP) using quantitative and qualitative approach.|
1031 nursing students included.
|Meskell et al. |
|To explore electronic objective structured clinical examination (OSCE) delivery and evaluate the benefits of using an electronic OSCE management system.|
To explore assessors’ perceptions of and attitudes to the computer-based package.
|A study was conducted using electronic software in the management of a four station OSCE assessment with a cohort of first-year undergraduate nursing students delivered over two consecutive years.|
The quantitative descriptive survey methodology was used to obtain the views of the assessors on the process and outcome of using the software.
203 undergraduate students included.
|Nilsson et al. |
|To develop and validate a new tool intended for measuring self-reported professional competence among both nursing students prior to graduation and among practicing nurses.||Construction of a new scale and evaluation of its psychometric properties.|
1086 newly graduated nurse students.
|Ossenberg et al. |
|To advance the assessment properties of a new instrument, the ANSAT, and investigate the acceptability of this instrument for the evaluation of the professional competence of nursing students.||Mixed method approach to collect evidence of validity supporting the instrument.|
23 clinical assessors included.
|Ulfvarson and Oxelmark |
|To develop of a new criterion-based reference tool to assess nursing knowledge and competence in clinical practice, Assessment of Clinical Education (AClEd)||Development of an instrument using the social constructivist process followed by an assessment|
Focus group of 5 students and 80 nurses from clinical settings.
|Walsh et al. |
|To test the psychometric properties of the Novice Objective Structured Clinical Evaluation Tool.||An instrument-testing design.|
565 nursing students included.
Table 1 provides a detailed description of the individual studies included in the review. There are five columns in the table. The first column provides details of the source and origin of the study and are presented in alphabetical order. The second and third columns list the key objectives and the research design. The main findings are presented in the fourth column, and the final column lists the MMAT score.
The quality of studies identified was mixed (Table 1). Two of twelve studies were judged with a low quality score (25%) with the main reasons for the low quality score being the use of a nonrepresentative sample and uncontrolled testing. Four studies were judged with high quality (75%). Three studies were evaluated as moderate quality (50%), and three studies as very high quality (100%).
The studies identified in Table 1 were heterogonous that is why they were transformed into qualitative findings using a narrative synthesis . The results were grouped into four assemblages according to the thematic approach: assessment tools, objective structured clinical examination (OSCE), complex assessment approaches, and other approaches.
3.3. Assessment tools
Hsu and Hsieh  developed an instrument known as the Competency Inventory of Nursing Students (CINSs) for measuring nursing students’ competencies and testing psychometric qualities of baccalaureate nursing students in Taiwan. They used a cross-sectional survey including 599 nursing students. This inventory assesses eight categories that cover ethics and accountability, general clinical skills, lifelong learning, biomedical science, caring, critical thinking, communication, and team working. Ulfvarson and Oxelmark  used the social constructivist process to develop a tool for assessing knowledge, and clinical practice contains four domains: nursing, documentation, caring, and skills and manual handling. The tool was tested and found to be valid to measure nursing skills not only of the nursing student’s ability to perform a task but also the quality of nursing care. This Assessment of Clinical Education (AClEd) tool evaluated learning outcomes during clinical practice. MMAT score for this study was very low, only 25%. The reliability of the assessment tool was not detected. Nilsson et al.  developed a Nurse Professional Competence (NPC) scale for measuring self-reported professional competence that covers eight factors: nursing care, value-based nursing care, medical/technical care, teaching/learning and support, documentation and information technology, legislation in nursing and safety planning, leadership in the development of nursing care, education, and supervision of staff/students. They developed a new scale and evaluated its psychometric properties on a large sample of newly graduated nurse students (n = 1086) from 11 educational institutions in Sweden. This tool can be used to estimate the outcomes of nursing education programs. It can assess nurses’ competence in relation to the needs of healthcare organizations, and it can help identify self-reported capabilities and assist in modifying introduction programs for newly employed nurses . Face validity was evaluated by asking students to critically review the item and their understanding of the item within the questionnaire. The data quality was described by mean score, and the construct validity and reliability were described with orthogonal rotation . We recorded the MMAT score for Nilsson et al.’s  study very high (100%). Ossenberg et al.  based their Australian Nursing Standards Assessment Tool (ANSAT) on the National Competency Standards for the Registered Nurse in Australia, covering professional practice, critical thinking and analysis, provision and coordination of care, and collaborative and therapeutic practice. The validation and acceptability of ANSAT was conducted in a pilot study on 23 clinical assessors, interviews, and with the posttest survey. The recorded MMAT score of study was 25%. More psychometric testing is needed to address current deficits . Iglesias-Parra et al.  developed an evaluation system of clinical competencies for the practicum of nursing students based on the Nursing Interventions Classification (NIC). They have prepared a list of 73 NIC interventions that were associated with each of the 26 competencies in nine domains. They took a psychometric validation study in two phases and the cross-sectional study on the population of undergraduate nursing students and clinical tutors. It was found that the competency system, structured on the NIC assessment tool, is a reliable method for assessing and evaluating nursing interventions. Reliability and construct validity were tested by the clinical mentors on 107 students. The survey was conducted with the Delphi technique. The MMAT score was very high (100%). The assessment tool represents a multidimensional approach in formative and combined assessing .
3.4. Objective structured clinical examination
Meskell et al.  and Walsh et al.  both examined OSCE. Meskell et al.  evaluated the benefits of using an electronic OSCE assessment system in undergraduate students (n = 203). The electronic software facilitated the storage and analysis of results, thus offering significant time savings. Walsh et al.  were focused on the development of a Novice OSCE that included the following competencies: safety, asepsis, knowledge, organization, and caring. An instrument-testing design on a sample of nursing students (n = 565) was used. The MMAT score of both papers was 50%. Some psychometric analysis, reliability, and stability of OSCE tool should be done. OSCE is shown as a formative assessing tool, and it is argued that students should also be assessed in critical thinking and problem-solving skills in addition to clinical skills performance [1, 35].
3.5. Complex assessment approaches
Three studies focused on more complex approaches. Athlin et al.  developed a model of a National Clinical Final Examination (NCFE). Their model integrates knowledge from theoretical and practical studies and includes knowledge, skills, capacity of critical thinking, problem-solving, ethical reasoning, independence, and readiness to act. They prepared a two-part examination. This included a written theoretical test with problem-solving characteristics and a bedside test performing nursing care by using observation. Their model was used to assess theoretical and practical knowledge. They found that the model was highly appreciated, and its relevance, usability, and validity were considered as “quite good” for the assessment of nursing students’ clinical competence at the final stage of their education. This study recorded a high MMAT score (75%). There is a need to evaluate the model on extensive students’ groups because the study was completed using a relatively small sample in theoretical test (n = 73) and a bedside test (n = 68). The model for evaluation of theoretical and practical knowledge used a holistic approach with opportunities for feedback and reflection for students. Imanipour and Jalili  developed an assessment system including multiple methods. They used a combination of oral examination and direct observation of a procedural skill. The cognitive knowledge was evaluated by oral exam, and clinical skills were evaluated by direct observation using a global rating scale. The exam includes some generic procedures and two specific procedures. Clinical work sampling was used to evaluate undergraduate bachelor of nursing students’ (n = 38) professional behavior. They found that the students and instructors were very satisfied with a comprehensive clinical performance assessment system. Levett-Jones et al.  describe the design, implementation, and evaluation of the Structured Observation and Assessment of Practice (SOAP) model used to assess the third-year undergraduate nursing students’ (n = 1031) clinical competences. While significant enhancements have been identified in students’ overall performance, the SOAP approach has discovered an insufficiency in the learning outcomes of some students.
3.6. Other approaches
Khan et al.  evaluated nursing students’ perceptions about the effectiveness of utilized teaching and learning strategies of clinical education in improving students’ knowledge, skills, and attitudes: demonstration, reflection, and problem-based learning, and concept map. They used both qualitative and quantitative methods in a descriptive cross-sectional study of 74 nursing students to identify nursing students’ perceptions about the efficacy of the applied teaching and learning strategies used in clinical education. Problem-based learning and the use of concept maps were perceived to be effective teaching and learning strategies. Hengameh et al.  compared the routine evaluation method (a subjective judgment of an instructor about general skills of the student during their clinical course, hence the scoring) with direct observation of procedural skills (DOPS) (clinical activities evaluated based on direct observation using the checklists). They found that applying direct observation of procedural skills (DOPS) significantly enhanced clinical skills and students’ scores in clinical procedures.
The aim of this chapter was to review the literature and critically discuss in relation to identified methods of clinical nursing skills assessment and competencies currently used in nursing higher education. Multidimensional approaches in nursing assessment should be based on a number of differing assessments methods . It should be the combination of knowledge, critical thinking, caring and communication [1, 7, 30], problem-solving, and reflection . Holistic assessment was found to encourage students to be more person-centered , rather than purely task-oriented . The literature review identified a wide variety of tools and assessment methods, each with their own advantages and disadvantages. Some were evaluated by nursing students, others by nurses and clinical experts. The studies reviewed were completed in different countries from differing nursing education curriculum and this, along with the range of sample size and approaches used, has proved difficult to make any direct comparison. Nurse educators have a responsibility to ensure that graduates are well prepared for the demands and challenges they will encounter in practice . There is a current imperative to implement a modern and appropriate method of clinical evaluation in nursing education [9, 29]. The current trend requires moving from a generic, technical approach to a more holistic model of clinical assessment, which supports the nurturing and development of competent nursing professionals . The OSCE is a practical test [17, 38] in a simulation area, where the student shows the skills  and technical performance . It is also a well-established method to assess clinical skills , using a checklist  to assess all students with the same set of criteria in order to determine the level of competency achieved in their performance [17, 39]. It provides a level of objectivity in how competency is assessed . The review identified a number of benefits from using OSCE including the achievement of deeper meaningful learning , deeper consequential learning , and an increase in students’ confidence in practice . The OSCE was also identified as a means to facilitate the assessment of psychomotor skills, as well as knowledge and attitudes . As an assessment method, the OSCE helps in the identification of strengths and weaknesses and can focus more on the student getting constructive feedback with or without the consequence of a subsequent examination . In addition to the previous advantages already outlined, Ulfvarson and Oxelmark  found that the OSCE can also be used for examining learning outcomes especially those comprising practical skills, such as medical techniques and interpretation of results. It has been recognized as a reliable and valid method to assess clinical skills competency [16, 39–41], and Carraccio and Englander  have suggested that the OSCE becomes a key standard for assessing clinical competence. Some criticisms of the OSCE have, however, been identified.
The lack of authenticity due to students not being observed in a real clinical context was identified by Levett-Jones et al. , and they further criticized how the OSCE focused on the measurement of technical skills rather than the whole caring situation including the use examination of empathy and interpersonal relationships. The OSCE, however, should be used in conjunction with other evaluation methods [36, 43]. Evaluation methods should be coherent with curriculum and learning out comes. The holistic evaluation methods motivate nursing students’ learning, stimulates critical reflective thinking, and make their readiness for professional practice more preferable. Good assessment tools should also be valid and reliable .
4.1. Implications for nursing education
Assessment of clinical nursing skills requires collaboration between clinical partners and academia to enhance the clinical experiences of students, the professional development of preceptors or mentors, and the clinical credibility of academics . The findings from the literature review represent a first opportunity to prepare our own assessment tools, according to the cultural and clinical environment, material and economic conditions, national nursing standards, capabilities and purposes of nursing care in Slovenia. There is now an opportunity for all educational institutions with the nursing study programs in the country to prepare assessment tool with cooperation of students, educational experts, and clinical nursing experts.
The findings from the literature review must be considered with respect to the limitations of the studies reviewed and the methods used. Some relevant work may have been omitted due to the inclusion of material only in the English language. The methodological quality of included studies varied from very low [12, 22] to very high [8, 29]. The validity and reliability of the different approaches used were not always discussed, and therefore, our conclusions should be drawn with caution. The MMAT is considered as an efficient tool, although its reliability could be further improved as it appraises only the methodological quality of included studies and not the quality of their reporting [45, 46]. Narrative summary is considered as a more informal approach and can, therefore, be subject to criticism, because of its lack of transparency .
Despite the heterogeneity of designs and methodology, the findings from the literature review present an overview of current clinical skills assessment tools in practice and in the simulation environment. The assessment of nursing students should include a variety of methods and procedures. It should include the assessment of knowledge, clinical skills, and critical problem-solving in nursing care. There is a need for further research to develop a holistic clinical assessment tool with a reasonable level of validity and reliability, and it must be tested before being applied to the nursing curriculum.