An mHealth Technology for Chronic Wound Management

Abstract

Increasingly, mobile consumer electronic devices are able to make meaningful applications in healthcare, and this chapter discusses the development of a mHealth app called SmartWoundCare, designed to document and assess chronic wounds on smartphones and tablets. Pressure ulcers (bedsores) were selected as the application area for SmartWoundCare due to their pervasiveness in healthcare and their associated impacts on patients’ quality of life and mortality, and electronic documentation is considered as an important intervention in pressure ulcer prevention and treatment. The chapter reviews the design of SmartWoundCare on Android and iOS platforms. Its benefits over paper‐based charting include automatically generated wound histories in graph and text formats, alerts and notifications for user‐set conditions, wound image galleries, and positioning for telehealth consultation by transmitting wound data across sites. The mobile app was implemented in a user trial in a long‐term care facility in Winnipeg, Canada, and the user trial illuminated that a key benefit of SmartWoundCare was the ability to take wound photographs. This feature had benefits for patients as well as caregivers. Consequently, algorithms were developed to analyse wound images for size and colour to provide additional indicators of wound progression.

1. Introduction and background

Increasingly, mobile consumer electronic devices are able to make meaningful applications in mobile health or mHealth, defined as the delivery of healthcare and healthcare support through mobile devices. For example, there are apps that allow users to track diet and fitness, health condition monitoring (e.g. diabetes [1]; arthritis [2]), and using mobile devices to replace paper records and share information between multiple healthcare providers [3].

This chapter overviews the development of a mHealth app called SmartWoundCare, designed to document and assess chronic wounds on Android and iOS smartphones and tablets. The chapter reviews the design of SmartWoundCare, the results of a user trial in a long‐term care facility in Winnipeg, Canada, and the subsequent development of algorithms to provide automated analysis of wound images for wound size and colour.

The initial application area is pressure ulcers, which is also known as bedsores. However, the app is easily applicable to other wounds as well, such as venous leg ulcers, diabetic foot ulcers, and surgical wounds.

1.1. Pressure ulcers as the application area

Decubitus ulcers are more commonly referred to as pressure ulcers or bedsores. They are injuries to the skin, or skin lesions which may extend to underlying tissues. Pressure ulcers typically occur over bony areas of the body as a result of skin pressure and friction when an individual sits or lies in one position for a long time. As such, pressure ulcers often occur in the elderly population and people who may be relatively immobile due to other illness or injury. Bedsores are preventable, but easily aggravated with heat and humidity at the wound site once they are present. Bedsores are also regrettably common, with the incidence of pressure ulcers reported to be as high has 30% in non‐acute care settings, with an average incidence rate of 25% over all types of healthcare facilities [4, 5].

Pressure ulcers have numerous negative impacts on patients, both in immediate comfort and well‐being and in long‐term quality of life. When they develop after a patient is admitted to hospital for other conditions, they can lengthen the patient's overall stay and complicate their overall healing. There are also numerous quality of life impacts reported including the psycho‐emotional impacts of chronic pain and the negative impacts of social isolation when patients’ movements are significantly impaired. A pressure ulcer starts as a seemingly minor skin wound and obscures its significant risk. Pressure ulcers are noted to be the second leading iatrogenic cause of death. From an institutional perspective, pressure ulcers treatment is also costly to the healthcare system [6–11].

There are many standard patient treatments used to prevent pressure ulcers in patients who are known to be at risk. These include regularly turning patients, optimizing diet and nutrition, caring for skin before pressure ulcers occur, and using pressure mattresses, pillows, and other supports to relieve pressure [12]. However, studies have also identified that due to the chronic and often long‐term duration of pressure ulcers, significant information about the wound over time can become obscured when documentation is not standardized, when risk assessments are not integral to the regular wound assessment protocol, or when assessments are incomplete or lack detail. In part, standardized forms – designed to capture all possible types of pressure ulcers – often become too unwieldy for healthcare workers with heavy patient loads to use effectively [13–15].

In many other areas of healthcare delivery, electronic health (eHealth) is being examined for its promise to increase the overall efficiency of a healthcare system and to improve patient outcomes. As eHealth grows in scope and maturity, its potential includes improvements and enhancements to patient safety, health outcomes, financial efficiencies, and communication between multiple healthcare providers.

When considering the health burden of pressure ulcers, the area of electronic medical records (EMR) within eHealth is of particular interest. In the research literature, EMRs are reported to have positive impacts on the quality of care and to reduce the reliance or use of care [16].

Several studies examined the impact of EMRs relative to chronic wounds specifically. In one, an EMR system simplified wound evaluation and treatment. In this case, the impact is highly dependent on a standardized protocol for taking pictures of the wound [17]. In another study, the financial benefits of home telehealth in treating bedsores were examined. The findings indicated low‐cost technologies did lead to cost savings, whereas high‐cost technologies did not have that benefit. The study also determined that home telehealth could decrease the prevalence of advanced stage pressure ulcers [18]. However, not all EMR systems for wound care are effective. Other research identified that common problems with wound EMRs included redundancy or the opposite situation where the platform was not flexible or detailed enough to consider all potential types of wounds. Other issues included the lack of standard vocabulary, and custom‐built EMRs which were not transferable to include or integrate with other medical records or across facilities [19].

While EMRs and other forms of electronic documentation are not a panacea, there is emerging evidence that when properly designed, they can potentially lead to better communication, better patient information and wound charting, and ultimately improved patient care and health outcomes. The work outlined in this chapter follows this anticipation that better compliance in documenting wound care, higher consistency in how a wound is documented, and the added intelligence provided by the app relative to alerts and information presentation can influence health outcomes.

2. Mobile apps for wound management

To date, many eHealth technologies have been and are being developed; however, they are not well‐catalogued. Relative to wound care, MediSense, WoundRounds, and How2Trak offer web‐based and/or mobile interfaces for wound management. In 2013, WoundMAP Pump, Ulcercare, and Wound Mender entered the stage of wound care apps in various stages of development [20].

2.1. SmartWoundCare system design

SmartWoundCare is similarly a mobile app for Android and iOS devices, developed in a computer engineering research lab at the University of Manitoba, Canada. SmartWoundCare was designed to replace the paper chart used in the Winnipeg Regional Health Authority (WRHA) for pressure ulcer management. The WRHA is a publicly funded system which includes both services and facilities. It serves over 700,000 people and supports referral services to another 500,000 people outside of its boundaries in hospitals, personal care homes, as well as a home care program. Over 28,000 people are employed by the WRHA in over 200 facilities.

As its core functionality, SmartWoundCare allows nurses and other healthcare providers to replicate the information that would be entered on a paper chart. A user can create a new patient record, view an existing patient's record, enter new wounds, and assess existing wounds using the Pressure Ulcer Scale for Healing (PUSH tool) [21], Braden Scale [22], and the Bates‐Jensen tool [23]. Several configurations were considered, in that one device could be associated with a given patient, and each nurse or other healthcare provider who cares for that patient would enter information on that patient's unique device. However, the model chosen was to associate the device with an individual nurse or other healthcare provider, who would use the device with all of their patients on that shift, and then transfer the device to the healthcare provider on the next shift.

As with all software, some general design objectives were established. These included keeping the user interface as simple as possible, using colours and other cues to focus the user's attention on important information, minimizing the steps needed to complete tasks, aligning the flow of information with emerging standard expectations from users (“look and feel”), and using the user's input to guide them to the applicable areas (and conversely, using the user's input to skip over areas not relevant for the particular patient or the particular wound). In light of the small screen size of a smartphone or tablet device, free‐form comments in data entry are discouraged by design. Entering data from pre‐set menu options is designed to reduce errors and to enable better comparisons between assessments, even when completed by different people. In a large‐scale rollout within a facility or a healthcare region, attention would also need to be given to battery life of the device, protocols for infection control, and the EMR as part of the legal medical record.

Beyond the duplication of paper‐based charting, SmartWoundCare was designed for several intended benefits:

1. Data sharing between multiple healthcare providers:the potential to seek consultation between multiple healthcare providers, including wound clinicians, physicians, allied health professionals (e.g. occupational therapy), and other specialists as needed. This potential reduces the need to transport the patient between facilities, saving the patient considerable discomfort and stress, and saving cost in the overall wound treatment. Just as significantly, the timeliness of interventions and changes in the direction of care can be improved. Information sharing (i.e. a telehealth framework) can occur within a given facility, within the same community, or between major centres and remote communities where remote communities do not have specialized health services. In Canada, with a small population living in a large geographical area, this is of particular relevance.

2. Data organization and interpretation:

   • Alerts: When logging into SmartWoundCare, the user will see a list of alerts, including wounds that are due for re‐assessment and wounds that are deteriorating. The specific parameters for the alerts (days between assessments, criteria used to determine deterioration) can be set by the user.

   • Because users have individual preferences on how they best understand data, SmartWoundCare presents wound histories in three formats: text, graph, and photographs. Text histories allow a user to scroll through a summary of the main wound parameters from one assessment to the next. Graph histories plot an overall wound score (e.g. generated from the PUSH tool) against time. Using the smartphone or tablet devices’ built‐in cameras, users can also add wound photographs to the record, and scroll through the images in a chronological gallery for each individual wound.

By design, the benefit of SmartWoundCare is its potential as an EMR, either on a stand‐alone basis or integrated into a wider EMR system within a facility or region. As such, privacy of data is a non‐negotiable concern. In its current form, SmartWoundCare requires each user to set up a unique user ID and password to facilitate a secure login and the login is restricted to that device. When envisioning a fully networked application within a facility or wider region, SmartWoundCare access rights would be confirmed by a secure connection to a server storing all information. Connections would be via cellular or Wi‐Fi, relying on all standard Internet security protocols. In that case, all login IDs and passwords would be managed centrally by a server‐side application rather than a device‐based login. An additional benefit of a central server, which could be facility‐specific or shared between several facilities, is the potential for additional data analysis in a Big Data framework. For example, when large datasets are available centrally in standard formats, they can be examined for anomalies, trends, and correlations that ultimately feed into the body of knowledge for pressure ulcer treatment.

Selected screenshots of SmartWoundCare (iOS version) are shown in Figures 1–5.

Figure 1.

Patient list upon login (iOS).

Figure 2.

Wound locations and status (iOS).

Figure 3.

Assessment data entry screen (iOS).

Figure 4.

Single wound summary in list and graph format.

Figure 5.

Chronological wound image gallery (iOS).

2.2. User trial – SmartWoundCare on Android

SmartWoundCare in a prototype Android version was subject to a small‐scale user trial. Voluntary participants were nurses in a personal care home in Winnipeg, Canada, and they used the mobile app with their patients. The objective was to obtain nurses’ impressions on the app's design, its functionality, and how it performed as a part of their daily clinical experiences in treating patients’ wounds. Investigating patients’ experiences and patients’ health outcomes with the app was beyond the scope of the user trial.

The user trial took place in Riverview Health Centre (RHC) in Winnipeg, Canada. Riverview Health Centre provides rehabilitation, palliative, and long‐term care. The facilities consist of hospital and personal care home units with almost 400 beds overall, as well as community programs and outpatient services. Riverview specializes in geriatric rehabilitation, brain injury, and stroke rehabilitation, palliative care, and complex long‐term care.

All nurses at RHC were invited to participate in the user trial. Approximately 12 nurses expressed interest, and after timelines and the scope of the nurses’ participation were established, eight nurses (three men and five women) remained willing to participate. Their participation was entirely voluntary and was not financially compensated. The nurses all had regular duties caring for patients with pressure ulcers or other wounds, and they were full‐time employees of RHC. The participants had a range of experience, ranging from less than 10 years nursing experience to over 20 years in a personal care home settings specifically, and ranged from 30 to 60 years in age.

Participants were also asked to judge themselves on their comfort with technology. Four participants judged themselves to be “very tech‐savvy” while the other four judged themselves to be “comfortable with common features of phones and tablets”. Participants’ confidence with smartphone/tablet interfaces and with touch screens was self‐assessed at 4.57/5.00 (range=4.0–5.0; SD=0.53) and 4.71/5.00 (range=4.0–5.0; SD=0.49), respectively.

To preserve anonymity, the characteristics of participants were intentionally not cross‐referenced with one another.

The nurses received a new Nexus 4 smartphone (four nurses) or a new Nexus 7 tablet (four nurses) with SmartWoundCare loaded and a training manual for the wound care app. They were given a 90‐minute training and demonstration of the app. After this training session, the nurses took the mobile devices home and familiarized themselves with SmartWoundCare further before beginning the user trial.

The nurses used SmartWoundCare (Android version) during their nursing shifts. SmartWoundCare was only used for patients who had pressure ulcers and who had consented to participate in the user trial. Given the patient population, patient consent was provided either directly or through a designate such as a family member. Participants used SmartWoundCare for at least seven shifts. At times, vacation schedules interrupted data collection over consecutive shifts. In most cases, participants were able to use SmartWoundCare for a longer period (more than seven shifts), enhancing the depth and scope of their feedback. All data collection was completed within two‐and‐a‐half months of the start of the user trial.

Using SmartWoundCare in nursing practice was an additional workload over the participants’ regular nursing duties, because it did not replace but rather it duplicated the paper chart that forms the patient's official medical record.

Once the nurses had been using SmartWoundCare for approximately 3 weeks, the nurses completed an anonymous on‐line survey. This data collection instrument was timed to gain participants’ immediate opinions and experiences of SmartWoundCare's functionality and design. The survey was administered via Surveymonkey and included open‐ and closed‐ended questions on SmartWoundCare features, content, look and feel, usability, navigation between screens, assessment of its intended advantages over paper‐based charting, as well as overall qualitative impressions of how well SmartWoundCare fits into nursing practice. An important part of the survey was for participants to assess the commensurability of the wound data entered into SmartWoundCare relative to data entered on paper‐based forms (scope and format), as this forms the basis of the integrity of the app.

Six weeks later and after an initial analysis of the survey results, a focus group session was held with the participants and the researchers. The focus group was used to probe into the survey results. In that way, the findings of the user trial include both the immediate and the long‐term impressions of the app's features and intended benefits, both of which are valuable to assess functionality. The research design complied with qualitative research norms, in which data and interpretations of data are validated by using triangulation and member checks.

The findings were then used to identify the key design issues for ongoing development of both the Android and a subsequent iOS version of SmartWoundCare.

2.2.1. Findings

The objectives of the survey and the focus group were to obtain feedback on the design and functionality of the app and to investigate the nurses’ experiences in using the app. The main numerical findings discussed in this section are summarized in Table 1.

Survey parameter All parameters are ranked on a Likert‐type scalefrom 1.0 (low) to 5.0 (high)	Mean; score	Range	Standard; deviation
How well‐matched is the scope and depth of the software; application to the Braden Scale tool?	4.60	4.0–5.0	0.55
How well‐matched is the scope and depth of the software; application to the PUSH tool?	4.57	4.0–5.0	0.53
Ease of entering a new patient record	4.57	4.0–5.0	0.53
Ease of finding my existing patient’s / resident’s wound; record	4.71	4.0–5.0	0.49
Ease of adding a new wound to the patient’s record	4.50	3.0–5.0	0.84
Ease of assessing a new wound for the first time	4.57	3.0–5.0	0.79
Ease of assessing an existing wound that had been previously; assessed	4.29	2.0–5.0	1.11
Screens were presented in an expected and logical order	4.17	3.0–5.0	0.75
Text history: this presentation is easy to understand	4.50	4.0–5.0	0.55
Text history: this presentation is helpful in understanding; wound progression	4.50	4.0–5.0	0.55
Text history: this presentation adds to my understanding of; the history of the patient's/resident's wounds and wound care, compared to not; having this text‐based history available	4.50	4.0–5.0	0.55
Graph history: this presentation is easy to understand	3.67	2.0–5.0	1.03
Graph history: this presentation is helpful in understanding wound; progression	3.83	3.0–5.0	0.75
Graph history: this presentation adds to my understanding of the; history of the patient's/resident's wounds and wound care, compared to not; having this graph‐based history available	3.67	2.0–5.0	1.03

Table 1.

Numerical findings of a user trial on the android version of SmartWoundCare.

In general, findings over the user trial indicated that SmartWoundCare was easily learned and used in the participants’ nursing duties, and that it was well‐matched to the PUSH and Braden Scale tools. The benefit of the smartphone was that it was easily carried in the pocket of a uniform; however, a drawback was that the text size was difficult to read. On the other hand, tablet devices were more difficult to carry and store but had the advantage of readability.

The user trial used an Android version of the SmartWoundCare prototype, and as a custom‐built software application, it did not always conform to users’ expectations of the look and feel of software and how one navigates through software. Areas that caused some initial confusion included cross‐navigation between different parts of the app, and confirming saves and deletions of data. Subsequent development on the Android version and later the iOS version of SmartWoundCare was a marked shift to the expected “look and feel” of mobile apps, as opposed to a custom interface.

As an important part of validating the robustness of SmartWoundCare for its intended application, nurses confirmed a strong commensurability in content and data entry between SmartWoundCare and paper versions of the PUSH and Braden Scale tools. Participants reported that the intuitive guidance accurately reflected the fields necessary for a given patient and their wound condition.

However, SmartWoundCare was developed to do more than duplicate a paper chart, and the user trial also investigated the nurses’ perceptions of the added intelligence in the app. Although the user trial took place over a relatively short period of time, the nurses indicated that they appreciated and recognized the potential of the wound histories. The text histories were met with slightly better perception than the graph histories (Table 1), although not to an extent of statistical significance (p= 0.05).

A suggestion for additional features in SmartWoundCare is centred on developing a glossary of specialized terms. This was identified as a useful feature even for experienced wound care nurses.

Another feature of SmartWoundCare over and above paper charts are the alerts that display to the user upon login. These alerts received mixed reviews by the users, with the primary complaint being that the alerts needed a more prominent place within the app rather than their location within a menu with five other menu options. In the subsequent iOS version, alerts follow a more standard format for iOS mobile apps.

2.2.2. Wound images as the key benefit

The strongest finding of the user trial was the value and benefit of wound images (photographs) in SmartWoundCare. Through both the survey and the focus group, nurses identified numerous benefits for the nurse at the bedside, for the patient and their family, and for the physician and allied health professionals. Nurses appreciated the ability to photograph the wound and the associated ability to show the wounds to the patient on the device.

There are several benefits of wound images. At times, wounds are located on body parts that a patient cannot directly observe, such as buttocks, heels, or the soles of feet. The wound photo allowed them to see the wound and get a sense of its size and severity. Often, this led to a better understanding for patients and their families regarding the importance of wound hygiene and treatments.

Another reported benefit is the time saved with each wound assessment, which could add up to significant time during a shift. It can take up to 20 minutes to undress, treat, and re‐dress a wound. If another healthcare provider (e.g. physician, physical therapist, wound clinician) asks to see the wound, the dressings need to be removed and the wound redressed after consultation. As a first option, the nurses could show the wound photograph to others in the healthcare team, and then a judgement was made as to whether the wound needed to be undressed or whether the photograph met the needed information within the healthcare team. A further advantage is when the healthcare team is consulting on a wound, the additional information that the wound photograph provided in comparison to solely having a verbal or written description of the wound.

Overall, the ability to add a wound history from photographs to the patient record was recognized for its potential to reduce the number of dressing changes and thus promote healing. The finding also supported SmartWoundCare's potential impact in telehealth.

The findings of the user trial also corresponded to other research findings related to the value of wound photograph, which is contingent on the quality of camera equipment, photomicrography (the art of photographing small objects in large scale), the orientation of the camera lens relative to the wound, flash settings relative to consistent lighting, and duplicate photographs [17]. Two separate studies examined measurements of wounds taken in traditional ways compared to measurements taken from photographs. In those studies, the wounds were venous leg ulcers and diabetic foot ulcers, respectively [24, 25]. The conventional technique to measure wounds is to lay a transparent film over the wound, to trace the wound margin on the film, and then to lay the film over graph paper and count the number of squares. When comparing this technique to measurements derived from digital images, the latter method resulted in improved accuracy, lower inter‐observer variations, and improved ease of use. Because the film physically touches the patient's wound and can cause irritation, the digital photograph also had the advantage of being a non‐contact method. Another study explored the potential of telehealth, specifically videoconferencing, compared to in‐person assessment for pressure ulcer assessment. Both procedures led to very similar assessment of the stage of the wound. However, the telehealth approach led to an overestimate of wound size and volume when compared to in‐person assessment [26].

3. Algorithms for wound image analysis: wound size and colour

Given the key finding of the user trial of the significant value of wound photographs, further work focussed on developing algorithms that would add intelligence to SmartWoundCare relative to image analysis.

The objective of the image analysis work was to develop algorithms to determine the size of the wound in both relative and absolute terms, and to analyse the colour breakdown of a wound, all from an image of the wound taken by a smartphone or tablet camera. Further, this objective was to be carried out without any peripheral or ancillary devices. Such devices, as seen in related literature, might include templates or positioning boxes by which the user would help the patient to position themselves and the wound, or it may include ultrasonic transducers and additional lenses for the mobile device. Carrying out the image analysis independent of any ancillary devices contrasts work by other researchers which, for example, control the lighting and wound position with an image capture box when performing image analysis of diabetic foot ulcers [27].

The application represented a general objective applicable to other fields, in that the work was intended to produce non‐contact measurements of irregularly‐shaped images taken with a smartphone or tablet camera, where the target range for error is <10% for images taken from distances of up to 30 cm. Relying only on the internal smartphone sensors to generate high‐accuracy measurements brings novelty to the work and specifically to the field of wound management.

Each new smartphone and tablet that comes to market generally has a higher‐resolution camera than the previous version of the device, and these progressions are often evident in short to medium timeframes of 6–18 months. Nonetheless, consumers are still hesitant to rely on on‐board cameras for any application that requires high precision and accuracy. In prior work, the state of image analysis from photographs was reviewed [28]. At first instance, several mobile apps were identified which claim to measure objects and distances in the 0.5–20 m range [29, 30], as well as ultrasonic‐transducer that ranges for measurements in the 1–6 cm range [31], and infrared distance measurements in the 4–30 cm range [32]. Depth‐of‐field cameras were also considered [33–35]. That early research also explored one method for determining distance from the camera to the wound and two algorithms to determine the size of the wound. Although both methods are promising, the specifications for error were not met [28].

It is foreseeable that smartphones with dual‐lens camera will enter the market within a timeframe of 6–24 months [36]. This development would create new and significant potential for high‐resolution images and subsequent analysis for accurate and precise characterization. The analysis techniques would build on the existing work in other fields, such as stereoscopic cameras in manufacturing. Google's Project ARA, a collaborative effort to develop modular smartphone hardware may also provide a future framework by which to include dual‐lens cameras in mobile devices.

3.1. Overview

Three components of the image analysis work are outlined in the following sections. In the first component referred to as Mask Image, the objective is to obtain the relative dimensions of an object in the image (in this case, a wound), in which the size determination is relative to the previous image of the same object. The second component, referred to as Camera Calibration, reconstructs an image taken on an angle and references it back to a two‐dimensional (2D) plane, in this way facilitating a measurement of the absolute or actual size of the object in the image. The third algorithm determines the range of colours present in an image. The algorithm separates the image into three component colours by extracting components from the red‐green‐blue (RGB) format of the image, and by doing so, makes possible an inference of the wound stage.

The software framework (Figure 6) in a high level abstraction consists of modules including acquisition of the wound image, pre‐processing of the wound image, segmentation of the wound image, recognition of the wound type, and classification of the wound. In reference to the three major components of the analysis indicated previously, the Mask Image component lies within the image acquisition module. Grabcut (a segmentation method [37]) and the Camera Calibration component both lie within the segmentation module, and the colour analysis component lies within both the segmentation and the wound recognition modules.

Figure 6.

Basic application model.

Although the wound photographs are taken with the cameras built into a mobile device (smartphone or tablet as per Table 2) or a webcam, all of the processing takes place on a computer. Computation times are generally in the order of seconds. Further work to have the processing take place on the mobile device itself is ongoing, and comes with the usual challenges of carrying out computation‐ and memory‐intensive processes on mobile devices.

Processing the photograph on a computer allows for both static and dynamic environments. In this case, a static environment denotes an environment where both the camera setup relative to the wound position is fixed (e.g. known, constant distance and angle, often with the use of staging devices) and the light source is stable. A dynamic environment refers to a mobile camera (i.e. smartphone or tablet) and/or the wound in a natural position at varying distances and angles to the camera and in varying lighting conditions.

With a series of photographs taken in a static environment, the Camera Calibration component, which corrects for angle by reconstructing an image in three‐dimensional (3D) space back to a 2D plane, only needs to be done once and the correction can be applied to the entire series of photographs. In a dynamic environment where distance and angle between the wound and the camera vary with each photograph, the Camera Calibration component needs to be done for each image.

Table 2 summarizes the hardware and software specification applied in this work.

Nexus 4 (LG‐E960)	MacBook Pro
Krait Quad‐core 1.5 GHz	Processor 2.6 GHz Intel Core i7
Display resolution 1280 × 768	Memory 8 GB 1600 MHz DDR3
Camera resolution 8MP (3264 × 2448)	Graphics Intel Iris Pro 1024 MB
High Performance Adreno 320 GPU	Software OS X 10.9.5 (13F34)
Bluetooth 3.0 BLE
Wi‐Fi 802.11 a/b/g/n	Software
	Android 4.2 (Jelly Bean)
Samsung Galaxy S4	Android NDK r9d
ARM Cortex‐A15 Quad‐core 1.9 GHz processor	OpenCV 2.4.9 Android SDK
Display resolution 1080 × 1920	Python 2.7.10
13+ megapixel camera	Numpy
Bluetooth 4.0	Matplotlib
802.11 a/b/g/n	OpenCV 3.0.0; Matlab

Table 2.

Hardware and software specifications.

3.2. Mask image for relative size

The first two components of the image analysis work, Mask Image and Camera Calibration, are used to determine the relative size and the absolute size of a wound, respectively, from the wound photograph. Figure 7 expands the first two modules of basic software framework in Figure 6, specifically the image acquisition module and the image pre‐processing module. The Mask Image component is situated within these modules.

Figure 7.

Image acquisition and pre‐processing flowchart.

Wounds are generally three‐dimensional, with volume below the skin surface. Wounds can also exhibit undermining, which refers to a wound that is larger at its base (below the skin) than the opening at the surface of the skin suggests, creating a cavity below the surface of the skin. Tunnelling refers to wounds, similar to undermining, which have channels (rather than cavities) below the skin surface.

As noted earlier, conventional methods to measure wound dimensions and/or area often use contact methods, in which adhesive strips or transparent films are laid around or on the wound, respectively, and wound edges are noted on the strips or films. The strips or films are then read directly for size or overlaid on to graph paper or rulers for measurement. The depth is generally measured with a cotton‐tipped applicator to the deepest part of the wound.

Two approaches in the literature to automatically determine the size of a wound include grid capture and scanner capture. Grid capture is a hybrid of conventional contact methods and digital image analysis. In this case, a transparent film with a marked grid is placed on the wound and the wound perimeter is traced on to the film. The film with the tracing on a known grid is then the basis from which the dimensions and area of the wound can be calculated with a software application [38]. This approach has the advantage of basing the calculation on a real tracing of the wound perimeter and a known grid, thus capturing the near‐real orientation of the wound. However, the disadvantage remains the potential for discomfort to the patient when the film rests on the wound.

In another approach denoted as scanner capture, a box with two internal mirrors is constructed as a template. The box has openings for a mobile device and an LED light source. In the scanner capture approach developed by others, a box with two mirrors inside is placed at 45 degrees relative to the horizontal, with openings for a smartphone and an LED light source [27]. The patient rests their foot in the box, and in this way, the setup maintains a constant distance between camera and wound and constant lighting conditions. While the computation remains intensive, the advantage of this method is that these two conditions serve to simplify the image processing requirements. The disadvantage of this method is the reliance on ancillary staging devices, and the setup will be impractical for certain areas of the body.

In this work, the objective of the Mask Image component is to obtain the comparative dimensions of an object in the image relative to a previous image of the same wound. An initial photograph is taken, from which a transparent digital ‘mask’ of the wound is created. The user then overlays or aligns this digital mask to the wound for the subsequent assessment and photograph (Figures 8 and 9). While most of the perimeter is expected to align between the mask image and the wound in its current state, one can reasonably anticipate that if the wound is either healing or deteriorating, portions of the perimeter between the digital mask and the wound in its current state will differ. The algorithm compares the digital mask to the current wound image, recognizing and aligning wound perimeter, and estimating the relative size difference. From this size difference, either healing, deterioration, or no change is inferred. The result is given as a percentage change in the area of the most current image relative to the previous digital mask image.

The mask image or mask overlay essentially serves to provide a point of reference when aligning the wound for the current assessment with its previous condition. As such, the point of reference does not necessarily need to be the transparent mask overlay. A medical tattoo could also act as a point of reference. In this case, it would be either a temporary or permanent skin marker or pattern (e.g. three dots) close to the wound. This marker or pattern would be used each to create a digital overlay which would provide the point of reference when aligning the camera for all subsequent photographs.

Figure 8.

Creating a mask image from the wound.

Figure 9.

Overlay of mask image to new wound.

The Mask Image component of the work provides the relative size of the wound from one assessment to the next. Users can choose to create one digital mask and compare all subsequent photographs to the initial digital mask; alternately, users can create a new digital mask at each wound assessment so that wound size comparison is always to the most recent assessment. A combination of the two methods is also possible. The advantage of the method is the absence of direct contact with the wound, thus preventing patient discomfort. Another advantage is that no additional devices to the camera or to the patient (e.g. props) are required. The error inherent in the approach is largely determined by the user's dexterity in aligning the digital mask over the current wound. A limitation of the method is that wound depth is not considered in the calculation. A further limitation of the method is that the outcome is a relative size of the wound rather than an absolute size. When an absolute size of the wound is desired, the Camera Calibration component is implemented.

3.3. Camera calibration for absolute size

Figure 10 shows the Camera Calibration component within the basic software framework outlined in Figure 6.

Figure 10.

Size estimation with segmentation flowchart.

Grabcut, a segmentation method used to differentiate an object (in this case, a wound) in the foreground from its background (in this case, the surrounding skin or body part), is applied in this module. Grabcut accomplishes this by using colour information to compare side by side pixels and also by using edge or contrast information to identify an object in an image. Further, Grabcut uses progressive iteration and runs the process multiple times to optimize the results. The result is a segmented image (the foreground object, in this case, a wound). This segmented image is then used in the Camera Calibration component as well as the third component of colour analysis. While other segmentation algorithms are available, Grabcut is considered an efficient algorithm and has the benefit of minimal user interaction [37], which was a requirement in this work. An example of Grabcut applied to wound photographs is found at https://youtu.be/Iyvochswrws.

The purpose of the Camera Calibration component is to take an image photographed on an angle and reconstruct or reference it back to a two‐dimensional plane. Essentially, the Camera Calibration module computationally achieves one of the objectives of the scanner capture box [27] in terms of aligning the wound to known and fixed positions relative to the camera. The Camera Calibration component uses a known pattern with 13 or more fixed reference points, and applies the Tsai2D algorithm [39, 40] to obtain a reconstructed image of the wound. Since the distance between the points are known from the calibration model, the view angle can be calculated and the image can be reconstructed on a 2D plane. From here, the size of the wound can be calculated. Like the Mask Image component, the Camera Calibration component also does not identify depth or volume of wounds. This is a known limitation, given that surface size and area alone are an incomplete descriptor of wounds.

A chessboard pattern was chosen as the pattern. This was found to be effective for photographs taken in static and dynamic conditions. Similar to the conventional approach of placing an adhesive ruler near the wound to measure size, the chessboard pattern is placed close to the wound and then photographed. The inherent assumption is that the wound and the pattern are in the same two‐dimensional plane. Given that the chessboard pattern is known and fixed, the planar orientation of the pattern in the photograph can be calculated and then the image corrected accordingly. This approach has been shown to be effective in calculating the dimensions of a soccer field, in which a top (plan) view of the field was reconstructed from images taken on an angle, using Camera Calibration [41]. In this work, the chessboard pattern is used for calibration to obtain the extrinsic matrix of the wound. The extrinsic matrix provides information on the camera location and the view direction, allowing for translation and rotation to the two‐dimensional plane.

Figure 11 demonstrates the Camera Calibration sequence at a high level. The red lines denote the objects which were detected, i.e. the dark squares. The algorithm finds the centre of each square and applies the Tsai 2D algorithm to process the coordinates. The blue lines show the scanning sequence. The green lines are the re‐projected lines from the model points to the real world coordinates, as an indication of the success of the Camera Calibration algorithm. If the green lines were curved or otherwise irregular, this would indicate that the projection back to a two‐dimensional plane was not successful.

Figure 12 shows the Camera Calibration component applied to a wound. The wound was photographed at an angle and then re‐projected on a two‐dimensional plane at 90 degrees to the viewer.

Figure 11.

Original and re‐projected planes.

Figure 12.

Wound image before (left) and after (right) reconstruction.

While the Mask Image component results in a relative size of the wound and the Camera Calibration component results in a corrected orientation and an absolute size of the wound, taken together, they allow for more accurate calculations. When applied to a Canadian dollar coin (26.5 mm diameter with eleven edges), the actual size was determined with an error of <1%.

A demonstration of the Camera Calibration module is available at https://youtu.be/OiJk3nMymSE.

3.4. Colour analysis

The third algorithm focuses on colour analysis of the wound. It determines the range of colours present in an image, separating the image into three component colours by extracting components from the red‐green‐blue (RGB) format of the image and presenting them in a histogram. These data can then be fed into an expert system to infer the stage of the wound. Figure 13 shows the Colour Analysis component within the software framework outlined in Figure 6.

Figure 13.

Colour analysis flowchart.

Pressure ulcers will be assessed as one of six stages (stage I through IV, Suspected Deep Tissue Injury, and Unstageable) [42]. Because the current work is unable to calculate the depth of the wound, the last two categories (both of which are wounds with some depth below the skin surface) have been combined as Unstageable. In addition to wound depth, other factors that determine the stage of a wound include skin condition (intact or broken), tissue loss, the colour of the skin, tissue, and wound bed, and the presence and nature of discharge.

To analyse the colour of a wound, the algorithm uses an RGB format of the image and determines the presence of the three component colours. Each component colour has a defined range, although the user can adjust that range or calibrate the range for variable lighting conditions.

While segmentation is not mandatory, the results of the colour analysis component are much more accurate if done on a segmented image, as this allows the algorithm to disregard the background (Figure 14 images taken from http://reference.medscape.com/features/slideshow/pressure‐ulcers).

Users can also consider hue, saturation, value (HSV) and red‐yellow‐black (RYB) formats for colour analysis. Hue, saturation, value (HSV) format responds to lighting, and as such, it may be a good option when one wants to tune the colour more specifically. RYB (red‐yellow‐black) has a fitting relationship to wound stages, and RGB results can be converted to RYB. The approximate ratios of red, yellow, and black correlated to wound stages are shown in Figure 15. Wound stages I and II rely only on red, but are differentiated on the intensity of the red in the image. The subsequent wound stages are differentiated on the proportions of each of the three colours in the image. The error inherent in this method depends to some extent on the definitions of colours set by the user. A recommendation is to associate this component with a machine learning component, once a large enough data set is collected. In this way, colour parameters can be more precisely defined.

Finally, expert systems can be developed to determine wound stages from the RGB and/or RYB data. This again relies on collecting a sufficiently large data set. Alternatively, support vector machine (VSM) or another machine learning algorithm can be applied to determine the stages of a wound. In the current work, the framework for an expert system is in place. The next step is to collect and populate the expert system with training data.

An example of the colour analysis on wound photographs can be viewed at https://youtu.be/Iyvochswrws.

Figure 14.

Histogram results before and after segmentation.

Figure 15.

RYB output correlated to wound stage.

4. Conclusion

SmartWoundCare as a mobile wound management prototype demonstrates the wide relevance of mHealth for applications within healthcare facilities and their integration with larger EMR and eHealth systems, as well as the application of telehealth to connect underserved communities. Community health and home‐based care is an equally important and in some way a more urgent implementation. For example, nurses of the Winnipeg Regional Health Authority alone carry out 450,000 wound visits per year in its Home Care program in clients’ homes. Particularly in home‐based care, the integration of SmartWoundCare with a suite of mHealth tools is a natural extension. A logical partner app for SmartWoundCare is diabetes monitoring, as well as novel pre‐emptive applications such as an early warning system for injury or damage to diabetic feet due to neuropathy [43].

SmartWoundCare and other mHealth applications also illuminate opportunities in Big Data, in which a community of users generate data – in this case, a wound database – from which relevant trends in wound diagnosis and healing can be extracted and form part of the body of knowledge in wound care.