Health Technology Management

1. Introduction

The term “health technology management” includes all those activities and actions necessary to carry out a safe, appropriate and economic use of technology in health organizations. The progression of technological innovation has changed many aspects of medical instrument management and related processes in order to achieve the main aims of a health technology manager, including acquisition, maintenance planning and the replacement of the medical devices in a specific environment where the continuity of services, privacy, reliability and safety are indispensable.

The most important changes concern the integration of different technologies into the same device, the increasing presence of software (stand alone or not), the subsequent rise in technological complexity as well as new device problems, such as usability, systems alarm and software validation.

For standard medical devices of typical electronic or mechanic equipment the presence of official regulations and guidelines provide decision makers with a valid and reliable support for a correct management.

For medical systems the lack of norms represents a reduction of quality and safety in health care: testing and maintenance planning are carried out according to the particular need in specific health contexts without common strategies or methods fundamental for benchmarking. Therefore, even if some general guidelines are being developed, medical software management still represents a critical point for health organizations.

In addition to standard operations related to management of technology, innovative and appropriate approaches have to be developed in order to maintain high quality and safety levels for healthcare technology. Studies aiming to define the critical points in managing complex systems and medical software have been carried out.

Technology management also has to consider the technology in terms of maintenance planning, technical assistance agreements, training courses, quality controls and replacement. All these aspects have to be provided for by specific analyses as an effective response to clinical processes’ needs (failures and failure modes analysis, Health Technology Assessment, usability).

2. Technology Management and Safety

2.1. Safety in Health Structures

2.1.1. General Aspects

The use of technology in medical practice implies the existence of a wide range of hazards, given by the critical nature of processes in the clinical environment in which these devices are generally used, that significantly increases the level of risk.

The hazards present in such activities are such that it is not possible to provide absolute protection to virtually nullify the risk because this would reduce the clinical efficacy, making the device inappropriate for the purpose to which it was designed. Therefore, in the face of this unacceptable reduction of functionality a series of measures are adopted that ensure a degree of protection that is appropriate for both the needs of the patient and of the operator; both clinical and safety.

The management of a medical device must be guided by criteria that ensure its safe, appropriate and economical use. The main steps for achieving this goal include a well planned purchase, appropriate preventive and corrective maintenance of equipment especially performance testing and calibration, and evaluation. It is evident that this perspective involves not only the entire life of the device within the health structures, by monitoring the perfect working status, but also in the acquisition phase through evaluating the clinical appropriateness. Indeed, from these considerations it is not possible to determine the potentially harmful aspects of the use of a device simply by observing the intrinsic characteristics, but it is necessary to evaluate the whole process in which the device is used. The risk analysis process is therefore a management tool through which it is feasible to define a set of critical parameters on the use of equipment through the analysis of processes and the risks related to them.

2.1.2. Risk Analysis Methods

The prevention of accidents cannot focus only on users’ training and performance; it must also involve the entire design of the system. Reason (1990) distinguished between active failures, which have immediate consequences, from latent errors, which remain ‘silent’ in the system until a certain triggering event causes them to manifest themselves, resulting in more or less serious consequences. The human operator is part of the incident, but the real cause (root cause) has to be found in bad managerial and organizational decisions. Up to now most efforts to reduce errors have focused on identifying the active errors, those clerical errors made by medical and nursing staff.

Recently, it has come to light that errors of an organizational origin, that is, latent errors, also play a determining role. The safety of the patient therefore, comes from the ability to design and manage processes capable, on the one hand, to curb the effects of errors that occur (protection) and on the other, to reduce the likelihood that such errors occur (prevention).

Two types of analysis can be used:

• Reactive analysis;

• Proactive analysis.

The reactive analysis provides a post-incident study and it is aimed at identifying the causes that have allowed its occurrence.

The most used analysis approaches in hospitals are as follows:

- Incident reporting:

Incident reporting is defined as a means of collecting records of events in a structured way

in order to provide a basis for analysis, preparation of strategies and actions for correction and improvement to prevent future accidents (some examples are given by ASRS. ‘Aviation Safety Reporting System’ created by NASA, the AISM ‘Australian Incident Monitoring System’ introduced in health care 1n 1996 and the NPSA ‘National Patient Safety Agency’ by local health organization in United Kingdom).

-Reviews:

This procedure concerns identifying errors in medicine through the analysis of clinical records in order to estimate and to compare the expected results. This method is usually composed of the following steps:

1. Identification of population and reference sample selection;

2. Patient selection according to clinical and statistical criteria;

3. Any report of an adverse event analysis;

4. Identification of possible adverse events with the aim of prevention.

- The Root Causes Analysis ‘RCA:’

In the RCA it is essential that the intervention is focused on the cause, not on the problem. The RCA focuses first on the system and processes and then on personal performance. There are different techniques for conducting an RCA such as “herringbone diagram,” “5 Whys” and the “process map.”

The Proactive Analysis aims to identify and eliminate the criticalities of the system before the accident occurs by analyzing the process activities in order to identify the critical points with the aim of improving the safety of the whole system.

Proactive Analysis

Some of the most significant proactive analysis methods used are as follows:

- Quality Function Deployment 'QFD:'

QFD has been derived from the latest evolutions in the approach to quality management. The focus therefore, in light of this, has been moved from the finished product (quality by inspection), to the production process (by quality control) in order to reach the final design (quality by design).

Gradually focus on that route is operational, design features that act primarily (in reviews). The concept of QFD can be thus formulated: it is a technique that transforms the needs of customers into quality characteristics, which are incorporated into the project and brought forward as a priority (deployment) in the process and then therefore also in the product, whose result depends on the network of these reports. This path planning can reduce the need for any subsequent changes in the product and processes, reducing time and cost control. The different phases of QFD are:

• Identification of customer/users’ needs

• Product or process evaluation through attributes and characteristics analysis

• State of art comparison;;

• Design definition;

• Identification of relations between the customer needs and the technology and technical features of the design;

• Benchmarking.

- Decision Tree:

This is a graphical tool used for the formulation and evaluation of alternative choices in the decision making process in order to solve complex business problems. It focuses on the

starting condition (context, means, resources), the possible alternatives (every feasibility is studied in probabilistic terms) and the results which these choices lead to.

Graphically, the decision tree consists of a diagram where the various alternative options are inserted in rectangular boxes with all the possible results written beside them.

- Failure Mode and Effect Analysis:

The Failure Mode and Effect (critically) Analysis ‘FME(C)A’ is a technique for prevention, that along with functional analysis and problem solving, is usually recognized as one of three basic fundamental techniques for systems improvement. It takes into consideration all the possible system inconvenience (errors, faults, defects) through a brainstorming process or other techniques. For each inconvenience is defined the level of criticality that depends on three critical factors: the occurrence of the event, the severity and the detection, that is the degree to which the inconvenience can be detected. These three critical factors are previously measured separately and then calculated as one only Index. By applying a criterion of criticality, it is possible to determine which events have the priority for intervention through preventative actions in order to minimize the occurrence of the event, protection protocols in order to reduce severity and control systems with the aim to increase the detection. Finally, this technique is codified in a specific European Technical Standard: EN 60812: 2006.

- Fault Tree Analysis:

The Fault Tree Analysis ‘FTA’ is a deductive method of analysis because it starts from a ‘general’ analysis according to the failure type, and then arrives to detect faults in the specific components. This method is codified in the Standard IEC 1025:1990.

2.2. Case Study: Analysis of Training and Technical Documentation in Technology Management in Health Care.

The training of users is important for safety aspects in technology management in healthcare. A more correct and safer use of medical devices is strictly correlated with the specific clinical activity, respect of procedures and legal standards and norms.

Technical documentation analysis is of equally high importance. It represents one of the most important tools in controlling and evaluating the quality of documentation provided by the manufacturer, the correct use of the devices and if training has been performed, with the purpose of maintaining performance and safety at the appropriate levels for the whole life cycle of the device.

2.3. Methods

The survey method was divided into two phases: firstly the training needs analysis, and secondly the development of a course modeled on the previous training needs:

Each phase is divided into the following steps:

PHASE 1:

1. personnel interviews through a check list distribution;

2. data collection and organization in a predefined and common form;

3. analysis of results ;

PHASE 2:

1. designing of a training course based on the critical points identified during the previous phase;

2. development of the course and related events;

3. investigation of the critical points and reports emerged during the course;

4. analysis of data collected in the preceding step and observation of critical points emerged.

The first point was to identify the basic elements to consider in order to establish objective

criteria to use for quality assessment of the technical documentation.

As a general consideration, a user guide has to be clear and comprehensive, easy and fast to use for being consulted when and where necessary.

In support of these topics the technical standards underlines how the language should be clear, direct and unambiguous (standard UNI 10893, "Technical Product Documentation - Instructions for use - Articulation and display order of content). Table I summarizes the aspects to be considered and possible technical suggestions for improvement.

Evaluation Parameter	Technical Suggestion
Age, availability and identification of technical documentation.	The instructions for use must be produced in a durable, with a clear identification of the editorial year and placed in a location known and accessible to all, depending on the conditions of use.
Ease of reference and traceability of information when it is necessary.	A brief summary of the arguments a nd easy use of the index.
Identification of users and the manufacturer.	Definition of the purpose of the manual, identification of the product and the original language.
Graphic and editorial skills assessment.	Good and appropriate graphic, structure and format in order to stimulate reading.
Good content organizations by appropriate separation of topics and effectiveness of the terminology.	Appropriate design (outline of the arguments) and use of short, uniform and synthetic titles.
Level of readability and completeness of the text	Use short sentences and concise, correct punctuation, use of verbs and sentences indefinitely in active form. Elementary syntax with little use of subordinate .
Ease of comprehension of the instructions	Adoption of the principles of communication " see - think – use” with great use of illustrations accompanied by brief explanations of the actions to be taken.
Detection of specific product and/or service	Refer unambiguously to the specific product

Table 1.

Evaluation Indicators and important aspects to consider for a well designed technical documentation.

In addition, the following comments can represent one reference scheme for the evaluation of the efficacy and comprehension of information:

• Ease of consultation and traceability of information when it is needed (short summary of the arguments in the analysis and use).

• Effective terminology and contents organization;

• Easy to read and completeness of the texts (syntax appropriate to the specific of reader).

• Ease of understanding.

Further, once these criteria are added, criteria of quality of technical documentation from the standard UNI 10893 and UNI 10653 ( "Technical Documentation. Quality of design and product) the complete list for the instruction manual can be defined as:

1. Title page / General instructions / Introduction;

2. How to install and / or linkage with other equipment and / or with the plant;

3. Technical data;

4. Operational use / Instructions for the operator;

5. Preventive and corrective maintenance;

6. User Tasks;

7. Cleaning and sanitation;

8. Diagnostics (analysis of failures and alarms according to a model: Event - Case - Remedy – Residual risk);

Finally, in addition to the above list, it would be necessary to add any specific information that the law requires for certain types of product. In order to carry out an investigation on the matters discussed, a check list was distributed to the personnel with the aim to explore all of the most critical aspects regarding the use of medical equipment.

2.4. Results

The check lists were distributed to previously selected departments, where a designated contact person for each department suggested the equipment to be included in this survey. The criteria for equipment selection were based basically on the intrinsic criticality inside the clinical process by defining as critical the importance of the specific equipment in the clinical activities carried out in the specific department. In addition, the person designated to fill in the check list was also instructed to consider the experience of other users who habitually used the selected equipment.

The data obtained belongs to 47 devices in 5 departments, pertaining to the areas of laboratory medicine and general medicine. Firstly, the results show that the percentage of users that consider themselves to have a good knowledge on device functionality who answer "yes, good" is less than 50% (Figure 1a) Figure 1b shows the results related to knowledge of the safety aspects of the device(alarms, operational risks, other risks)and the response is positive with a percentage of 40%, while the remaining 60% indicates having a partial or low safety aspects knowledge with a significant 30% of low level answers ("just", "very little", " nothing").

Figure 1.

a)Level of knowledge expressed from users on medical device functional aspects; B)Level of knowledge expressed from users on medical device safety aspects.

Further it is significant to analyze the percentage of equipment for which training was given, approximately 40% of equipment tested is used without any course, see Figure 2.

Figure 2.

Medical devices for which users have received training.

By crossing this with the figure 3b trend it is possible to see many situations in which the lack of a training course is related to a significant difficulty in use, by showing that the most users’ votes is “5” and over. Figure 3a confirms and summarizes this trend by showing the percentage of equipment (with and without previous training) considered difficult (voting more than 5).

Figure 3.

a) Medical device complexity to use according to user interview ; b)Complexity of use for medical device used in hospital without any previous training course.

Another important result regards the presence of the user guide. Figure 4a shows how most of the devices are accompanied by the manual(79%). Further, figure 4b shows that devices still exist with the user guide in English (40%).

Figure 4.

a)Number of medical devices with instructions guide; b)Medical devices with user-guide in italian.

Finally, the users’ perception of the quality of the manual is reported in Figure 5 and it shows significant fractions equal to or even below "sufficiency (vote<6).”

Figure 5.

Users opinion on user-guide quality: vote 1 bad…vote 10 good.

2.5. Discussion And Further Developments

The collected data has underlined the following topics:

1. Knowledge of the equipment functionality, especially for those aspects related to safety is low.

2. Technical documentation and other instruments identified by the legislature to improve this knowledge are not perceived by operators as fully adequate to carry out their aim, because they are judged to be insufficient in quality or not complete in contents.

3. Importance of training courses based on the critical aspects of equipment safety especially concerning the safe use, documentation and training.

Finally, further developments regard the check list distribution in more critical areas such as operating rooms or intensive care units, where the first data indicates a problematic situation, for example with the anaesthetic machines, that present good reviews on training but poor reviews concerning the technical documentation. This lack of knowledge on certain aspects is responsible for bottlenecks, repetitions and delays in clinical activities.

3. Health Technology Assessment

3.1. Hospital Based Health Technology Assessment

The term ‘HTA’ Health Technology Assessment, signifies the multi-disciplinary evaluation of properties, effects and/or impacts of health care technology in order to improve efficacy and effectiveness in health care services. An HTA process starts with the analysis of clinical needs and evaluates the patient outcomes of different technologies, by using a multi-disciplinary approach in order to consider, in addition to the clinical efficacy, the technical, usability and economic aspects related to the medical devices.

When HTA is performed on a hospital scale the process is defined as Hospital Based HTA.

It is substantially different from standard HTA processes, usually carried out on a regional or national scale, that include both diffusion and innovation of all technologies, especially high technology, and it is mainly oriented to only consider clinical and economic aspects such as clinical efficacy and hospitalization times.

In hospitals, the HTA process principally involves medium and low technologies and its main aim is usually only to compete with other structures in technological innovation. This leads hospitals to undergo HTA processes to assess the effects of this gain in equipment

complexity and not to plan the technology replacement according to real needs and clinical improvements.

Furthermore, the increasing levels of management expenditure in spite of limited economic resources requires more attention to acquisition contracts, especially the technical aspects such as maintenance, safety controls and personnel training in order to ensure an appropriate and safe use of the device.

Further, the hospital scale strongly characterizes technology depending on its destination of use. The presence of patient or not and experience and preparation of technical personnel represent important aspects to consider in HB-HTA processes, as well as analysing the device insertion impact in specific healthcare processes.

Finally, the increasing presence of medical softwares in hospitals represent a new challenge for HTA processes, especially in hospitals where some critical departments, such as Intensive Care, are strongly dependent on medical software responsible for monitoring the central station.

3.2. Case Study: an experience of HB-HTA at a Complex Hospital

According to different health structures’ needs, the use of a general HTA methodology is essential in order to define the most appropriate and objective Key Performance Indicators for an appropriate evaluation of safety, efficacy and costs of medical equipment (Cohen et Al. 1995).

The methodology has been tested at the University Florence Hospital Careggi (AOU Careggi) and applied to the Clinical Engineering database including all the maintenance data of 15.000 medical devices.

This study has developed a set of indicators, with the aim to provide clinical, safety and economic information to be considered in technology assessment in hospitals.

As shown in figure 6, the result has been the development of a multidisciplinary HTA dashboard “S.U.R.E.,” organized in four sections: Safety, Usage, Reliability and Economics.

Figure 6.

The Multidisciplinary HTA-Dashboard SURE.

For the safety section in particular, the technical KPI have been previously validated to ensure their usefulness in the dashboard.

For instance, “Technology Level” and/or “Destination of Use” have been demonstrated to be strongly related with the safety and economic fields of “SURE.”

Figure 7 shows a failure analysis by using the Global Failure Rate (GFR) and the Age Failure Rate (AFR) applied to medical devices classified according to Technology Level: High-Tech, Medium-Tech, Low-Tech and Limited-Tech devices.

Figure 7.

Age Failure Rate and Global Failure Rate analyses applied to technology level.

Both the analyses demonstrate that the number of failures is proportional to technology level during all medical device life. The AFR analysis further shows that training aspects are fundamental in obtaining a low AFR. After two years users are able to correctly use the device. Indeed, a good HTA process has to heavily evaluate both the quality of training courses and the full risk maintenance, especially for High technology.

Figure 8.

Age Failure Rate and Global Failure Rate analyses applied to destination of use.

The analysis demonstrates as “Destination of Use” is strongly related to failures in hospitals. The analysis demonstrates that “Destination of Use” is strongly related to failures in hospitals and consequently, to safety. Both analyses show that number of failures is strongly related to the presence of patients (protocol depends on patient clinical situation), defining as the most critical ones the Therapeutic (e.g. surgery rooms) and Clinical Support areas (e.g. in-patient rooms).

Further, the AFR analysis demonstrates that the different skills of users characterize “Destination of Use”, suggesting that different learning periods (Laboratory is a very “Technical area” with well educated technicians working there) should be considered in a complete HTA process.

Future development aims to create a simplified SURE in order to carry out continuously a sustainable methodology and an easy tool for supporting decision makers in HB HTA.

3.3. Case Study: Medical Software HTA

The recent European Directive 2007/47/EC added “clinical software” to the Medical Device classification (MD). As a consequence, interest in medical software increased in health care. A critical point in the life cycle process of clinical software is the definition of “performance”, which is not usually applied to healthcare (so not based on specific safety aspects), and focuses on equipment performance, usually connected to other needs (for instance quality in imaging or electrical and mechanical safety rules).

This study aims to explore and to evaluate these aspects related to HTA process for clinical software through the application of a HTA methodology for a software failure analysis.

The goal is to locate Key Performance Indicators ‘KPI’ that could be included from traditional MD HTA and from software engineering.

Four main factors were selected as key points of an HTA process by answering these questions: “Does this technology work?”, “Who can take advantage of it?”, “What are the costs?” and “What comes out from the comparison with the alternatives?”

Indicators from a previous KPI used for traditional HTA were evaluated according to the following criteria: “which ones are suitable for both HTAs”, and if applicable, “how can I adapt their meaning to the software (what is its usability for SW?)”, “is the insertion of different indicators necessary (different standard or systems integration)” or “can the priority scale be modified (maintenance, formation, quality documentation)”.

Three groups of KPI (Key Performance Indicators) are provided for classifying software HTA indicators.

The first includes indicators suitable for both SW and MD processes, the second including parameters needing conceptual adaptations from SW engineering to medical application before being used in SW-HTA (e.g. usability is replaced by interface complexity) and the last group that comprises of indicators not applicable to software HTA.

The proposed methodology allowed indicators for medical software HTA to be evaluated, by taking into consideration the evaluation methods typical of medical device approaches and the measuring parameters related to the informatic and software fields, see Table 2.

First Group	Second Group	Third Group
Appropriateness	Usability	Privacy
Complexity	Interface	Interoperability (HL/IHE,..)
Training Supported	Safety	Reliability
Documentation Provided	Personalization Ability	Consistency
...	...	...

Table 2.

Proposal of KPI for Software HTA.

4. Health Technology Maintenance

A failure is an event responsible for any change in the performance of the device making it unusable or unsafe for the patient and the user, in the use for which it was intended. According to actual standards, maintenance is defined as "a combination of all the technical, administrative and management activities planned during the life cycle of an entity, to keep it or return it to a state where it can perform the required function.“ As mentioned in the definition above, two important aspects are fundamental for all maintenance activities: the specific actions to carry out and the time necessary in order to correct the failure. Generally two main maintenance systems are normally carried out: corrective and preventive maintenance. A balanced compromise between the two is the aim for health technology managers.

4.1. Corrective Maintenance

Corrective maintenance is intended to be applied in case of a failure with the aim to repair the device. Evidence based maintenance systems (Wang et Al., 2006) depend on failure analysis in order to design efficient maintenance plans. Corrective actions and time of interventions are obtained by taking into consideration details concerning the failure and the device involved, such as the failure type and/or the technology complexity.

Indeed, health technology managers have to optimize the integration of external interventions assistance and internal technical service to guarantee an efficient and cost-effective maintenance system.

For instance, internal technicians training has to respond to the real needs evaluated from a failure analysis and then be integrated with external assistance provided from the suppliers.

Effective maintenance must also consider rapidity in problem resolution, especially in healthcare where the services continuity is essential.

In some clinical areas where technology is vital for patient life such as intensive care units or operating theatres, the continuity of the service coincides with the continuity of technological functionality. Here the priority is to minimize the downtime of the device and time of intervention, which is essential for planning suppliers’ assistance services, internal organization and hospital procedures.

4.2. Preventive Maintenance

Preventive maintenance includes care or services by specific personnel with the aim to maintain working equipment and/or extend its life, by providing for systematic inspection, detection, and correction of incipient failures either before they occur or before they develop into major defects.

In particular, these services aim to improve equipment longevity, detection of future need for major service or replacement, maintenance of proper performance, prevention of downtime due to mal-performance or failure and reduction of repair costs.

Two preventive maintenance systems are possible: first level and second level.

First level preventive maintenance is usually carried out by users in terms of visual controls and in some cases simple medical tests on the devices. These controls and/or tests have to be described in the user guide provided by the suppliers.

Second level preventive maintenance is performed by qualified personnel and consists of visual checks, safety tests, control of real efficacy and performance of first level preventive

maintenance and the execution of maintenance programs provided from the manufacturer.

The most important variables for an effective preventive maintenance are the frequency and the specific tests to be carried.

According to international standards, the actions must be provided from the manufacturers, even if some specific actions could be suggested from health technology managers as a result of failure analysis.

The failure rate trend (figure 9a) is a good measure for planning the frequency of the interventions. The trend analysis is useful for two main reasons: firstly, it is possible to improve the maintenance system (changes on frequency and type of interventions) according to the relation between the failure trend analysis and the levels to be reached; secondly, the failure analysis allows for control and adaptation of the maintenance system according to the lifecycle of the device. As shown in Figure 9a, the typical trend is composed by three main sections: high failure rate in early age due to defective parts and small practice with the equipment, followed by a constant trend representing the contribution of “casual” failures that have to be minimized by the use of proper preventive maintenance, and finally, high rate of failure due to age of the device such as the device wear and the difficulty to find the proper components and suitable material for the replacement.

Indeed, the preventive maintenance should therefore take into account these aspects and be adapted according to the lifecycle of the device. Figure 9b shows the situation at AOU Careggi. By analyzing the Age Failure Rate 'AFR' (calculated as N of Failures / Number of Devices / Age of the devices) it is possible to refer the actual “lifecycle moment” to the central sector of the trend of figure 9a. For future management, all the preventive measures described above should be reinforced in order to minimize the expected increase of the failure rate.

Figure 9.

Bathtub function/Failure Rate analysis comparison.

4.3. Case study: Experience of Failure Analysis regarding Medical Devices in Hospitals

This study represents a failure analysis carried out on a complex university hospital by using two main indicators: the FRC and the NTI.

The Failure Rate Category (FRC) index is an indicator that takes into account the Failure Rate for a specific category of failure “N of Failures/N of Devices/Failure Type.” Six categories of failure have been considered and classified by analyzing all technician reports present in the Database: “Software (FRC-SW),” “Electronic/Electric (FRC-ELE),” “Mechanic (FRC-MEC),” ”Accessories (FRC-ACC),” “False Alarm (FRC-FA)” and “Unclassifiable (FRC-UN).”

The NTI indicator takes into consideration the “Number of Technical Interventions” by considering whether the technicians come from the internal Clinical Engineering department, from an external private company or from supplier’s assistance.

By analyzing fig. 10, it is possible to see a characterization between Failure type and technology. FRC-ELE and FRC-SW are typical for High-tech. FRC-MEC is equally distributed over all classes. It is interesting to observe that Limited-Tech does not present any FRC-False Alarm. It is also important to note that FRC-UN characterizes mostly High- and Medium- Tech. The FRC analysis on destination of use classification is reported in Figure 11. FRC-SW characterizes Therapeutic destination of use. Activity Support does not have FRC-FA; further, FRC-FA is low for Diagnostic and Lab areas. It is also interesting to note that FRC-ACC is low for Lab.

Figure 10.

FRC analysis applied to ‘technological complexity’ classification.

Figure 11.

FRC analysis applied to ‘destination of use’ classification.

NTI application on destination of use is reported in Fig. 12. Diagnostic and Lab areas are the exceptions to the general trend of NTI [destination of use], that normally has the highest values for all uses of internal NTI. Both Diagnostic and Lab present higher values for external technical interventions than internal ones. Lab and Diagnostic are characterized by High-Tech medical devices.

Figure 13 shows the NTI analysis according to failure type and to technician provenence (internal or external).

It is interesting to observe that Unclassified Failures represent the only typology with more NTI-EXT than internal ones (no clear reports for external interventions have been common during the analysis). Finally, False Alarms result to be much higher in NTI-INT than in EXT because the hospital protocols provide internal personnel as the first call of intervention.

Figure 12.

NTI analysis applied to ‘destination of use‘ classification and to technician.

The presence of FRC-FA for both Medium- and Low-Tech suggests that a request for user training with lower technological devices should be considered in the acquisition phase.

Another essential aspect during the acquisition phase is asking external companies’

technicians to leave a formal, pre-prepared report of their work after every maintenance in order to create a much more efficient control. This would also help technology managers to better control important hospital areas such as Lab and Diagnostic that present high NTI-EXT and a high concentration of High-Tech equipment.

Figure 13.

NTI analysis applied to technician.

4.4. Case Study: Experience of Failure Analysis with regards to Medical Softwares in Hospitals

515 software failure reports have been analyzed. They have been classified into seven categories according to the type and to the cause of the failure. Further, all “medical softwares” have been classified according to their “Destination of Use” and according to software installation: ‘Stand Alone (SA)-’ or “Hospital Network (NET)-’ softwares. Figure 14 shows the software presence at the hospital analyzed.

Figure 14.

Percentage of different softwares present at the hospital.

Reports are divides into six areas of interest, considering the "Type of failure”, each addressed by an acronym:

• Non-Software - NS,

• Non-Medical Software - NMS,

• Corrective maintenance - MC,

• Preventive Maintenance - MP,

• False alarm - FA,

• Wrong/bad Use - BU.

Furthermore, in order toalso analyze management aspects and health process, all types of software have been classified according to the "Intended Use" in a hospital ('Life Support', 'Diagnostic', 'Therapy', 'Laboratory', 'Support Activities') and in relation to the type of

software to the type of installation ('Stand alone' or 'Hospital Network').

After the definition of these classes, the last step included index, "FR," defined as the number of faults normalized to the total number of softwares, equation 1.

$$\text{FR }=\text{ Failure }\left(\text{N}°\right)÷\text{ Software }\left(\text{N}\right)$$ E1

Further, evidence based relationships exist between different Destination of Uses and Software Type, see Figure 15: “NET-Software”is distributed in all hospital areas whilst the “SA-software”‘is present only in Life Support and Therapeutic areas.

Figure 15.

A)‘SA- Software‘ hospital distribution. b)‘ NET-Software‘ hospital distribution.

Figure 15 shows interesting correlations between software types and failure categories. Net-Softwares present only one third of total failures specific for softwares such as MC,’ ‘MP’ and ‘NMS’, see Figure 16a. Further, ‘false Alarm -33%-’ is the highest failure type for these kind of softwares. Figure 16b shows the situation regarding the Stand Alone softwares. Also for this category only 33% of total failures is related to software faults while the category ‘NS’ represents the highest.

Figure 16.

Failure type analysis applied to software installation: (A)Stand Alone ; (b)NET-Connected.

Finally, by considering the most critical area, ‘Life Support’, it is clear that all failure types are present here, and only 50% are related to medical software failures, see figure 17.

Figure 17.

Failure Type analysis applied to ‘Life Support‘ Areas.

By better analyzing the FA-Failures, it is interesting to note how the ‘Life Support’ area represents 25% of total failures. If the presence of patients in the ‘Therapeutic’ area can justify the highest number (41%) and high technology in imaging can help to explain the percentage of 34% in ‘Diagnostic’, the signifying percentage in life support results to be very high and the object of further analysis, See figure 18.

This approach allows a quantitative evaluation of safety and reliability for “medical software” in hospitals. A multi-dimensional approach has been considered by evaluating software applications according to different user skills and clinical areas needs, and it has shown that personnel training could reduce one third of total software failures. The approach is crucial in terms of multi-dimensional analysis, because it supports the analysis and evaluation of the software linking process characteristics and clinical areas of hospitals with different intrinsic technical characteristics of the software.

Figure 18.

Number of False Alarms according to destination of use in hospital.

5. Future Challenges

5.1. Medical Software Management

Two main groups of Medical Device Softwares "MDS" are present in healthcare: stand alone or isolated MDS, and connected MDS to HIS (Hospital Informatic System) or to other MDS. The management of MDS is basically structured in three main areas: procedural, technical and legal issues.

At the moment, all these areas have no well defined and sure standards or methods to follow.

5.1.1. Procedural aspects

Despite the presence of some international standards, there is a strong lack of detailed procedures, especially for medical devices, including basic procedures for risk assessment, performance control and test certificate tasks. Finally, procedures for recalling in the case of adverse events are not yet fully developed.

For connected MDS especially, the management is more complicated. The HIS/other MDS connection introduces new potential risks to consider, such as when and which update processes are necessary to maintain the proper functionality ie. how much does it depend on the actual connection modes given by actual software versions.

MDS management points also regard organizational problems: the necessity to clearly define the roles for MDS management is essential. For instance in hospitals, when MDS are inserted in the HIS, two different departments are interested in their management; Clinical Engineering and Information Technology. This redundancy is largely responsible for ineffective services in problem solving and decision making rapidity for MDS, and for patients subsequently.

5.1.2. Technical aspects:

Further critical points introduced by the MDS regard the technical aspects that concern the data transmission, its security and integrity.

Robustness, protection tools and user friendly interface are some of the most important technical aspects to consider during the evaluation of MDS.

With regards to connected MDS, most of these difficulties arise from data compatibility with the HIS system and amongst different MDS: the standardization process for data transmission interfacing (DICOM, HL7) would allow to assemble different softwares from different companies with a subsequent simplification of maintenance and acquisition processes.

Finally, inter-operability and integration of MDS in HIS must be considered in order to facilitate use for medical personnel, therefore improving clinical efficacy and efficiency.

5.1.3. Legal aspects:

Finally, data protection for both privacy and safe storage of clinical information is extremely important in health care. The standard protection measures such as user authentication modes, data security protections, back-up and data rescue-recovery systems are not practical in healthcare where, for example, in case as of emergency, these systems must not delay the clinical activities. The aim of the health technology manager is to guarantee fast access in specific cases (emergency and rescue) without reducing the security and data protection levels. The hybrid environment composed by IT systems and medical devices where connected MDS are inserted make the situation even more complicated.

Table 3 summarizes the Medical Software issues and challenges previous cited.

Management	Needs
Procedural	Test certificate Performance evaluation Adverse events calling Role definition for MDS and HIS management Updating procedures
Technical	Data safety and integrity Correct interfacing with other MDS or HIS IT system compatibility
Legal	Privacy Data protection

Table 3.

Main aspects to consider for software management in health care.

5.2. Electro Medical Systems Management

5.2.1. EMS (Electro Medical Systems)

The increasing use of computers, especially in diagnosis and in therapeutic treatment monitoring, has led to the synergic use of several medical devices, creating electro medical systems with the aim to ensure additional functionality in applications without decreasing performance of individual equipment or increasing risk.

Some examples of EMS are present in radio-diagnostic equipment, video endoscopes, ultrasonic equipment with personal computers, computerized tomography, magnetic resonance imaging.

Safety aspects in EMS concern the characteristics of each individual equipment in addition to the characteristics of the entire system.

Further, EMS can be spatially distributed in the clinical area by having parts both inside and outside the “patient area”, each of which need different safety protections:

within the patient area safety level is required to be the same for medical equipment;

outside the patient area safety level is required to be the same for non-medical equipment.

In addition to typical protocols, a new vision is necessary for the EMS life cycle management such as, for instance, the "EMS acceptance testing." This should be performed only once the different parts of the system are connected according to the specifications and the documentation provided from the manufacturer or from the responsible organization of the system.

It is important to identify equipment classed as non-medical, for which the organization must verify the compatibility with other system components (electrical or not).

Finally, regarding the periodic electrical safety inspections, the concept of "separation" is a critical aspect, where, for instance, an high value for the leakage current (due to functional connections between different parts) should be taken under control by applying the safety standard concerning the device separations.

5.2.2. PEMS (Programmable Electrical Medical Systems)

Often, EMS include a significant software component in order to program some parts for system flexibility improvement.

The software results to be critical for the functionality and performance of the PEMS, so that

it is responsible for the proper functioning, stability and reliability of the system.

Since it is not possible to apply to software the concept of "technical parameter measures" for the proper functioning of the other equipment parts, it is necessary to move the risk analysis to the design phase of the PEMS, especially concerning reliability regarding the development of software.

Indeed, in order to facilitate the design of a PEMS, its structural definition is fundamental in order to allow an easy identification and implementation of all those requirements concerning the safety.

The technical documentation of a PEMS describes this structure and thus represents the functional connections between each component and the whole PEMS. The structure description should provide:

• the scheme of the PEMS by identifying all the components, especially those implemented in each programmed component, including software components;

• the functions of each programmable part and its components (including those relating to safety);

• the interfaces between the components of the software;

• the interfaces between the software systems.

Finally, the type of PEMS structure is important: Appropriate structure (following the functionality) allows the recognition of functional connections that leads to a simple risk detection, through an easy identification of those hazards related to both connections and software.

5.2.3. EMS Conformity Inspections

From the previous brief description of EMS it has been possible to extract some fundamental aspects underlining the importance of verifying conformity to, for example, the CE (European Community) or the FDA (USA). In both cases it is essential to analyze the software component by evaluating the effective importance for the proper functioning of the system.

All these aspects concerning particular attention to software have been included in the recent European Directive 2007/47/EC that, having included software in the medical device category, requires that software itself must also conform to all the essential safety requirements necessary for medical devices to be put on the market.

5.2.4. Complexity of a system.

In general, EMS, with the aim of increasing functionality, present a high complexity that requires careful assessment of essential safety requirements. At that regard, following is reported the list with the general categories of hazards associated to medical device / system suggested form the Appendix D of the Standard EN ISO 14971:

1. Energy-related;

2. Biological;

3. Environmental;

4. Incorrect energy outlet and substances;

5. Improper and incorrect use;

6. Unusable interface;

7. Functional failures;

A further aspect to consider is that technological progress tends to transfer functionalities of the device as much as possible to software for two main reasons:

1. Firstly due to economic aspects; the objective is the elimination of “hard elements" whose features can be reproduced by the software;

2. Secondly for the versatility of the system; by delegating the intelligence of the entire system to software, it becomes much simpler for the manufacturer to update and renew the entire system through just the addition of functionality in the software command.

Due to improvements in innovation and efficacy, a correct software risk assessment, including the individuation of risks and the verification of conformity to the essential safety requirements, is becoming ever more complicated.

Generally, because of the difficulty in implementing a comprehensive and effective risk management, software is limited to a purely support /advisory function, where the final decision-making is usually left to the user., who has to choose the parameters suggested from the software.

5.2.5. CE Marking and FDA Certification.

Regarding CE marking or FDA certification, another critical aspect is the different approach between the designing of medical devices and software.

A software programmer would be required to operate so that during the development phases it is possible to control the risk management aspects, since it is difficult to act on safety after the final development.

There is a lack of specific technical standards present for PEMS; the EN 60601-1 on the safety and performance of medical devices in addition to the EN ISO 14971 on the application of risk management to medical devices, are not enough for the development of an appropriate procedure for the design and the safety use of PEMS.

In order to overcome these limitations it is therefore necessary to integrate these standards with the requirements present in software standards such as the EN 62304 regarding the processes for the lifecycle of a software.