The aim of this chapter is to describe the development and evaluation of an online multilingual information resource focused on medication management, targeting people living with dementia and their family caregivers. Maintaining effective medication management is important to allow ongoing quality of life within the community setting and avoiding medication-related preventable hospitalisations for the person living with dementia. Family caregivers are likely to assume the role of medication management on behalf of the person in their care as dementia progresses. Little training or information is available to family caregivers to assist them with this role. A pilot online information resource was developed and evaluated. Responding to the evaluation, this resource was improved, and a more extensive evaluation process was undertaken. The development and evaluation process are outlined with a view to guiding the development of similar resources, especially those targeting linguistically diverse family caregivers and those with dementia. This is especially important given that many older adults will migrate during their lifetime, often to a country where they are not familiar with the language or health services. Extra support is needed to assist older immigrants who are themselves at risk or are caring for someone with dementia.
- medication management
- family caregiver
- ethnic minority
- computer-based education
- health literacy
Effective medication management enables people living with dementia to avoid hospitalisations related to preventable medication errors and prolongs their ability to remain living within a community setting. This chapter describes the development and evaluation of a multilingual online information resource on medication management for family caregivers of people living with dementia (PLWD). The resource aims to improve medication management to enable PLWD to remain living at home, in their community for as long as possible. This resource is unique in that it targets not only caregivers within the general Australian community but also Italian and Macedonian immigrant caregivers who find themselves caring for someone with dementia in their host country, Australia. Qualitative research identified caregiver concerns regarding medication management and findings from a literature review were used to inform were used to inform the development of the information resource. The principle of universal access was incorporated into the website design, and the precautionary principle was applied in regard to health literacy when planning the content of the resource. The initial site evaluated, changes were made, and a refined site developed and further evaluated in light of our two guiding principles and the reported usefulness of the site to caregivers. Lessons learned from the development and evaluations of the information resource to assist caregivers and PLWD with medication management tasks are explained. The chapter aims to outline a toolkit of resources that can be used by other projects to develop and evaluate similar information resources.
Many older adults are not ageing in their place of origin as mobility increases within and between countries. Increases in net migration in countries with high rates of immigration, such as Australia, will see their growing older population become more ethnically diverse over time . In Australia, linguistically diverse caregivers comprise 33% of caregivers of PLWD living in a community setting . Health literacy rates are known to be generally lower in immigrant populations, and they continue to have increased difficulties when accessing health services and information . For example, findings from a qualitative Australian study of 25 non-English-speaking community living adults from the former Yugoslavia suggest that language barriers, different expectations about medicines and the healthcare system resulted in members of this community being more likely to experience medication management problems . Based on this example and anecdotal evidence from other linguistically diverse groups, it is likely that this population group of family caregivers may require additional support to safely and effectively manage medications for their older relatives who are living with dementia.
Regardless of ethnicity, people living with a dementia often need to manage complex medication regimes. They may be prescribed one or more cognitive enhancing medications for the management of their dementia. This is especially the case for those who live in developed countries where early diagnosis and treatment are available . These cognitive enhancing medications can be useful in managing some symptoms of early-stage dementia and delay institutional placement . PLWD may also experience other age-related chronic co-morbidities such as diabetes, cardiovascular diseases, arthritis and osteoporosis . As a result, it is common for older adults with dementia to also have a number of co-existing illnesses that may be partly or completely managed by the use of medication. Ongoing safe and effective medication use for the management of all co-morbidities is important for PLWD to avoid medication-related adverse events, prevent hospitalisation or placement into permanent institutionalised aged care and to maintain or improve their quality of life [8, 9].
Recent studies have documented the impact of dementia on a person’s ability to safely and effectively manage medications [10–12]. The cognitive, social and physical losses associated with dementia compound to make ongoing independent medication management more difficult for PLWD [13, 14]. Cognitive losses may affect memory, attention, language, communication and decision making; all capabilities which are necessary to maintain autonomy regarding safe and effective medication management . Cotrell et al’s.  study of 47 caregiving/care recipient dyads indicated that 85% of PLWD relinquish some or all medication management tasks to their family caregivers over the course of their disease, compared to only 30% of older adults receiving caregiver assistance for other chronic illnesses. Similar results were found in a larger study by Thorpe et al.  of 566 dyads which found the majority of family caregivers will assume a medication management role throughout the progression of a dementia with 54% noting involvement at any stage of dementia and rates exceeding 90% in the later stages of the disease.
Specific medication management tasks carried out by a caregiver of a PLWD may include maintaining continuous supplies of medications, assisting with administration, making clinical judgements and communicating with healthcare providers and care recipients . To be effective in their medication management role, caregivers need to not only be informed about what is prescribed but also be capable of asking relevant questions of healthcare professionals, be able to monitor medication adherence, be aware of and watch out for medication-related side effects. They also need to be able to grant or refuse consent for medication use or discontinuation if the PLWD is no longer capable of making this decision .
Medication management issues, including compliance concerns for PLWD may be resolved by involving a family caregiver in the medication management process where possible. This has been shown to be a somewhat effective approach in reducing the number of identified medication misadventures, especially if support from health professionals is given to caregivers in addition to the PLWD . Ongoing access to information and support in their medication management role is necessary for caregivers in order to adjust to the changes which accompany the progression of disease in the PLWD . However, findings from a number of studies indicate that there is a lack of information, training and support for family caregivers in their medication management role [15, 19, 20]. This lack of information was confirmed by our own search for paper based on online information material which found that no specific Australian resource was available that targeted caregivers of PLWD in their medication management role.
Given the importance of ongoing safe and effective medication management for PLWD, we sought to fill the knowledge gap we had identified. Funding was obtained to develop an information resource for family caregivers of people living with dementia from linguistically diverse backgrounds. This group of caregivers were targeted as being the most in need of an information resource as evidenced by the findings of an earlier Australian report on the experiences of managing medications at home of 12 older adults and 10 family caregivers from linguistically diverse backgrounds . The report highlighted that this group of Australian caregivers faced additional language and literacy barriers, requiring information to be available in their own language but also in a variety of formats to increase accessibility for those with low literacy . In targeting this group which may have the lowest literacy and as a result the lowest health literacy levels, we aimed to produce a resource that was useful and accessible to everyone.
The following sections outline the development and evaluation of this online pilot resource.
3. Online resource development
3.1. Phase 1 Pilot online resource
We aimed to develop a useful, easy to use, accessible, up-to-date and trustworthy resource that met the needs of our target group of caregivers and was also available for all caregivers and PLWD. In order to do this we first conducted a review of the literature and a qualitative research project involving participants from our target audience which included the generation of older European immigrants who arrived in Australia following the Second World War. This group is now entering old age and are at high risk of developing dementia. These immigrants were mainly from southern European countries and as such the most common languages spoken among our target group included Macedonian, Italian, Greek Spanish, Maltese, Polish, Dutch and Hungarian . Their education levels are generally low in both their original language and in English, especially among the women .
3.2. Literature review
To build our own knowledge of this topic and investigate other related research, a review of the relevant literature from January 2000 to April 2013 was conducted. This revealed a lack of research investigating the medication management experience of family caregivers, especially those caring for PLWD . Furthermore, previous research of general medication management for older adults included linguistically diverse family caregivers; however, sample sizes were generally small or results for linguistically diverse groups had not been analysed separately . We viewed this as a significant gap in the literature given that linguistically diverse caregivers make up a significant proportion of the family caregiving population in many countries, that have high immigration rates, like Australia.
Despite the limited availability of previous research, the main findings of the review concluded that family caregivers received little or no training, information or support as they performed medication management tasks . The role is complex, and this complexity increases as medication regimens become more complicated. For instance, family caregivers are often responsible for the supply and scheduling of multiple medications, the administration of medications via different formulations (e.g. oral, rectal, transdermal patches, inhalers, injection, creams and drops), the calculation of dosages, the monitoring of side effects and communication with multiple healthcare providers.
3.3. Qualitative study
A qualitative study, including focus group discussions and one-on-one interviews, was conducted in order to understand the everyday medication management experiences of linguistically diverse family caregivers of PLWD living in the Illawarra region, a multicultural urban area south of Sydney, Australia, with a population of 450,000. One quarter of people resident in the area are born overseas and 14.5% report speaking a language other than English at home .
The research team worked in partnership with the local multicultural health services in order to make contact with linguistically diverse family caregivers of PLWD living in the Illawarra. Local community workers involved with the area’s carer support groups noted that family caregivers experienced isolation, having little time available outside of their caregiving role, low English proficiency and limited transport options. It was also noted that dementia was viewed with considerable stigma by many linguistically diverse communities and family caregivers may not identify as caregivers, as the roles they take on are seen as a normal part of family responsibility . This knowledge and the established relationships formed by the multicultural workers were invaluable to the research team.
A total of 29 participants from five different cultural groups participated in the three focus groups and seven semi-structure interviews which were conducted between July and October 2012 (see Tables 1 and 2). The majority of the focus group participants were spouse caregivers, while most of the individual interviews were conducted with adult child caregivers. The majority of the participants were female, with only three male caregivers included in the study. The dominance of female participants reflects the gendered nature of informal caregiving and is comparable with samples in other studies of family caregivers of older adults and people with dementia [11, 27, 28]. Further details of the methodology of this study and the major findings have been published elsewhere .
|Nationality||Number of caregivers||Gender||Carer status|
|Italian||n = 6||All female||Spouse caregivers|
|Macedonian||n = 11||8 female/3 male||Spouse caregivers|
|Portuguese||n = 5||All female||4 spouse caregivers/1 adult child caregiver|
|Nationality||Number of caregivers||Gender||Carer status|
|Italian||n = 2||Female||Adult child caregivers|
|Greek||n = 2||Female||1 adult child/1 spouse caregiver|
|Dutch||n = 1||Female||Spouse caregiver|
|Croatian||n = 1||Female||Adult child caregiver|
|Macedonian||n = 1||Female||Adult child caregiver|
Overall, the following key themes were identified from the qualitative data:
Managing medication is a source of stress for family caregivers.
Medication management may be a point of familial conflict that needs to be carefully controlled.
Family support of this caregiving role is important.
Family caregivers believe that they would benefit from more sources of external information and support regarding medication management.
Quotes to support these four key themes are provided below:
Managing medication is a source of stress for family caregivers.
Caregivers reported stress resulting from their medication management role as they managed complex medication regimes. Sometimes these difficulties arose as an outcome of the progression of dementia:
”He would say “No, No! ‘That’s not the tablet I’m supposed to have now’ or ‘I had it before!’ or ‘that’s the one I’m supposed to have later in the night” so I just ignored this for a while and we would start all over again”…”.Sometimes I gave him the tablet and he would take a sip of water and then next minute I found it, that tablet, on the floor. He spat it out!”’ (Greek spouse caregiver interview)
Caregivers adopted various strategies to help them manage this stress. The most common was the use of blister packaged medications prepared weekly by a pharmacist.
This strategy was not always failsafe:
“My friend, his wife she has a Webster (blister) pack and he was not there…. she took the whole pack at once and had to go to hospital”. (Macedonian Focus Group)
“And there were occasions when he was taking hers. He took her medication (pack) instead of his!” (Greek adult child caregiver interview):
Medication management may be a point of familial conflict that needs to be carefully controlled.
Caregivers lost trust in their relative’s ability to manage their medications autonomously when they observed medication errors being made. This often became a source of conflict as the PLWD hoped to maintain autonomy while the caregiver feared that medication errors could result in problems.
We were finding too much on the floor, not only from hers but also from his. Oh it was just horrendous, it was just horrendous!….. “And when we got the Webster (blister) packs he was so angry with us. He said ‘What do you think?! Do you think I can’t look after my medications?!’” (Greek adult child caregiver interview)
Crushing medications was the most common strategy adopted by caregivers in order to avoid further conflict around medication administration.
So we started hiding the medication in food, so we crushed it for example, put it in some yoghurt if it was breakfast time”. (Italian adult child caregiver interview)
Family support of this caregiving role is important.
Many caregivers in our study noted that they shared medication management tasks such as picking up prescriptions or taking the PLWD to the doctor to have medication prescriptions written. The burden of medication administration was also shared with other family members:
“Yes, and the girls (daughters) are here all the time, in and out and sometimes they would encourage him to take this and that”. (Greek spouse caregiver interview)
Family caregivers believe that they would benefit from more sources of external information and support regarding medication management.
Caregivers noted a general lack of information and support available to them in their medication management role even though they spoke of contact with community pharmacists, general practitioners (GP), geriatricians and family caregiver support groups. Caregivers noted that pharmacists were more accessible sources of information than doctors.
“You know we would go and ask the pharmacist when we were picking up the Webster (blister) pack, you know most of the time it was the pharmacist not the GP”. (Macedonian adult child caregiver interview)
Some participants used internet sources for information to assist with their medication management role.
“Two tablets were interfering with each other and it was my daughter who saw it in the computer and she told the doctor”. (Portuguese Focus Group)
Overall, the caregivers suggested that they lacked information about common medication-related side effects such as constipation, resulting from the use of pain medication, as well as concerns relating to the use of multiple medications.
“Mum’s on 12 or 10 or something and it’s just such a chemistry set inside a body and how it reacts with Mum is so different to how it might react to someone else and so how can they get it right? You know that’s what I’m wondering and especially at Mum’s stage where the dementia is; what we should really be worrying about, just to make her life more comfortable from now….. Does she really have to worry?…. If you stopped something like cholesterol medication or thyroid medication she’ll drop dead tomorrow, I don’t know? I don’t want that. But all the other peripheral medications are they really necessary, you know are they really necessary?” (Italian adult child caregiver interview)
Spouse caregivers in particular faced additional challenges in managing medications resulting from language barriers and a lack of access to appropriate services.
“I go to a chemist where there is a Portuguese girl. We understand each other. It’s the other ones I can’t understand a word. I like to go in when she is working but she only works one day a week”. (Portuguese Focus Group)
3.4. Resource development
Once we had gathered the information based on our aforementioned literature review and qualitative study, we used our findings to inform the content of a pilot version of the resource to enable initial evaluation. Clearly a comprehensive information resource was needed to address the many medication management issues and/or questions caregivers faced. We wanted to design something that would be accessible to all, even for those with limited literacy. We also wanted to provide reliable information that could be kept up to date and could be re-visited as the needs of the caregiver(s) changed. For this reason, the research team decided to produce an online resource.
An online resource has the advantage of being available at all times of the day and can be bookmarked and revisited if information is forgotten or caregiving requirements change . The internet allows both written and oral versions of the information to be presented, addressing potential literacy issues. It also enables different language versions to be added to the resource, overcoming language barriers. We acknowledged that some of the older spouse caregivers might not be able to access this resource as many older adults are not familiar with technology use . However, we hoped that the resource would be accessed by the adult children of spouse caregivers or could be used as a resource during family caregiver support group meetings.
We were mindful of the fact that the health information we provided online needed to be both understandable and reliable. This is especially true when considering older people living with a dementia and their caregivers; as it is highly probable that they are less likely to have the time or ability to evaluate online health information. We sought to ensure the reliability of the content by basing the content on current research findings. Caregivers were referred to links for other reliable sites if they wanted further information.
We knew that in developing our resource we needed to address literacy and health literacy concerns. Approximately 44% of Australians have problems in understanding health information . Poor health literacy is particularly prevalent amongst older adults especially those with limited education, low incomes and from linguistically diverse populations [33, 34]. In order to cater for those with low health literacy, we drew on current health literacy research [35–37] and guidelines for web design suited to older adult users [38, 39] to guide the content and design of the resource.
3.4.2. Addressing literacy concerns
Based on evidence that in order to improve health literacy, information needs to be written at a grade 8 or below level , health literacy guidelines [35, 37] were followed to improve the readability of our online resource. We used plain language, avoiding overly technical jargon wherever possible. Sentence structures were simplified, an active voice used and dot point lists used where appropriate. Simple visuals were included if they would aid understanding allowing for white space on the page to avoid clutter. Furthermore, an audio version of each page (both in English and Italian) was made to cater for those who might have a stronger command of the oral language, as opposed to written text available in both languages.
3.4.3. Addressing website accessibility
Working on this assumption that many of our site visitors would be older people and/or their adult children, we approached the web design process aiming to make the website universally accessible. Our pilot web design was informed by the SilverWeb guidelines , and the checklist for the development of senior friendly websites available at https://www.nlm.nih.gov/pubs/staffpubs/od/ocpl/agingchecklist.html . A larger font size and a sans-serif font were used in the written text, while avoiding yellow, green and blue colours to accommodate declining visual acuity. Visual content was kept to a minimum, which meant that the resource included only text relevant images. Navigation buttons were large, and each page had prompts to click forward or return to previous pages. Menu tabs were organised to make it easy to locate relevant information and the need for scrolling on pages was avoided where possible.
3.4.4. Content of the online Phase 1 resource
The key messages of the site included recommendations to:
Undertake regular medication checks and reviews with a pharmacist and/or doctor familiar with the PLWD.
Use reputable sources of information about medication—general practitioner, pharmacist, or nurse.
Actively find information about medications taken by the person with dementia, document these and keep an up-to-date list.
The site also described practical strategies to help the person with dementia to take the right medicines at the right time. It was hoped that these strategies would also reduce familial conflict and stress experienced by the caregivers in their medication management role. The main recommendation was to use blister packaged medications; regularly checking for their ongoing safe usage, and as suggested by Zedler et al.  to help improve medication adherence. Blister packs also offer benefits for the PLWD and their caregiver by allowing them to see which medications need to be taken when, and whether or not they have been taken appropriately. Given that many caregivers reported crushing medications in our qualitative study, a page was included to alert caregivers that not all medications are safe to crush. The online resource advises PLWD and/or their caregivers to ask their pharmacist or doctor before crushing any medications or to enquire about alternate formulations (e.g. liquid or transdermal patches) that would avoid the need to crush medications.
Finally, the online resource also provided additional information regarding other reputable websites, support groups, translation services and organisations which could help facilitate safe and effective medication management for PLWD.
3.5. Evaluation of Phase 1 pilot resource
3.5.1. Survey evaluation of the pilot resource
We aimed to evaluate the pilot website considering our two guiding design principles: the accessibility of the site and the accessibility of the content, especially for our target audience. An adapted version of the ‘Quality checklist for reviewing health information’  was used to initially assess the resource. This checklist asks the user to review the content, usefulness, appeal, cultural relevance, readability, presentation, accessibility, credibility and need for improvement of the information resource. The checklist, only available in English, was completed by nine users of the pilot resource all of whom gave positive responses. The survey respondents also provided useful suggestions about additional information regarding specific medications, the need for other formats such as a DVD version of the resource, and translation of the resource in to other languages.
3.5.2. Support group evaluations of the pilot resource
Evaluation of the pilot site was also undertaken with caregivers from English and Italian support groups, who also suggested a DVD version of the resource, as evident in the following participant quote: “Not sure how well used the online resource will be for the older Italian carers who I suspect may not be well versed in using this sort of media however I think having things online is the way to go in the future…..even using a DVD version would be hard for them”. (Italian adult child caregiver)
“Not sure how well used the online resource will be for the older Italian carers who I suspect may not be well versed in using this sort of media however I think having things online is the way to go in the future…..even using a DVD version would be hard for them”. (Italian adult child caregiver)
Support group participants were positive about the resource, suggesting that the information on the site was well received, comprehensive and useful. They suggested including support group contact details on the site and were not in favour of ‘pop-outs’ for some of the visual components on the site.
4. Phase 2 resource refinement and current evaluation underway
Background: The original research group was expanded to include researchers with backgrounds in nursing, public health, education and technology. Further funding was obtained which enabled the translation of the resources into a third community language; Macedonian, and the migration of the site to WordPress (https://wordpress.com/). This change to hosting platform was made to facilitate future editing and updating through a what-you-see-is-what-you-get (WYSIWYG) content management system. The results of the initial pilot study were used to inform the second iteration of the website, which is currently available at: (www.dementiameds.com) (Figure 1).
While the Managing Medicines site was designed to enable maximum accessibility for users, by utilising a larger font size and providing written information in an audio format, there remained the need to evaluate the site. This was completed using a mixture of methodologies, which incorporated readability, usability and server log data analysis.
4.1. Usability testing
Website users inherently rely on their previous internet experiences to inform their online behaviour. Nielsen found that experienced users are faster at doing things on websites that they use often; they are, more confident at clicking and less afraid that they will break something, better at searching, using the mouse and scrolling. As such, usability testing was undertaken to ensure that the Managing Medicines for People with Dementia website met the needs of the end-users. Previous research has shown that usability testing with five participants can identify 85% of problems .
A convenience sample of ten participants (age range: 26–79 years of age; mean age 51.8 years) completed the usability testing. Participants were all family caregivers of PLWD, spoke English, had used the internet and were willing to be recorded during the usability testing process. Participants were also asked to complete a short demographic survey, which included questions on age, gender, language spoken at home, country of birth, computer and internet use. Participants were subsequently asked to think-aloud as they completed five set tasks. This allowed the researchers insight into their thought process, personal opinions and reactions to the website. The tasks were designed to mimic the actions that website visitors might have for example, “Please change the language settings from English to Italian”, and goals e.g. “Please find the phone number for xxx”.
The software program Morae Recorder (TechSmith, Okemos, Michigan) was used to conduct the usability testing. Morae records participant’s interactions with the website in the form of visual, audio and mouse movements.
A number of small issues were identified through the usability testing. Most participants encountered difficulties when asked to find specific information. Participants suggested that a dedicated page be added to the site with contact details for support services. Currently, this information is embedded within the website. It was also observed that it would have been useful to have incorporated a short practice into the usability testing, to allow participants to become familiar with how Morae operates.
It should be noted that numerous variables affected task completion time, and the successful completion of tasks. Factors included the telling of personal stories, the presence of children during testing, and phones ringing. This supports the observations from another usability testing study with older adults.
Participants who reported over 10 hours internet usage per week were more likely to be successful in completing the tasks. It is likely that this difference was observed due to their increased familiarity with website navigation and increased internet skill level.
4.1.1. Method critique
While the use of dedicated usability software facilitated recording and analysis of sessions, it would be possible to undertake basic usability testing using traditional audio-visual recording methods and researcher notes. Ideally, usability testing should be undertaken by an expert. However, the use of the dedicated software enabled two research assistants to undertake the usability testing. Another possible method of testing would be a Cognitive Walk Through . This involves an independent health researcher working their way through the website checking that the content is clear, uses plain English, and that the website had consistent formatting with images related to the text.
4.2. User research
In order to gain feedback from a larger number of site users, a site evaluation survey was compiled and made available using SurveyMonkey. Visitors to the site were asked to complete this online evaluation survey, through a message at the bottom of the left-hand side navigation bar. This message was visible on all site pages. The survey contained 26 items and was available for a four-month period in 2015. It included demographic questions as well as two previously validated tools: a modified version of the Perceived Health Web Site Usability Questionnaire for Older Adults (PHWSUQ)  and Chew’s three screening questions for health literacy . The evaluation was promoted through personal contacts and via articles in relevant state and national newsletters.
4.2.1. Method critique
While the use of SurveyMonkey allowed a large response rate, it was flawed in that the survey was only available in English, despite the site being promoted to people from Italian and Macedonian backgrounds. While this decision was made due to financial constraints, it is a significant limitation to the evaluation of the site. Further, the majority of respondents were healthcare workers and not the target audience of family caregivers of people living with a dementia. It is possible that healthcare workers are more comfortable using the internet and completing evaluation surveys. One of the strengths of this method was the ability to incorporate previously validated tools into the survey.
4.3. Web analytics
Google Analytics (https://www.google.com.au/analytics/) was used to track and report website traffic. This program can be easily configured to monitor a specific website and runs behind the scenes, collecting data and information about site visitors. Data on the number of unique site visits, visit duration and country of origin of the visitor were collected. Notably, many site visitors were from outside of Australia.
4.3.1. Method critique
One of the main benefits of Google Analytics is that it provides insight in to site users’ behaviour and location, and is free to use.
Most health information and educational material on dementia are presented at high reading levels [33, 34, 46]. A recent review of online health information found that sites with information on dementia were the hardest to read when compared to 11 other health conditions .
The readability levels of the Managing Medicines for People with Dementia website were evaluated using six readability tools:
Flesch-Kincaid, readability statistics found in Microsoft Word, based on US school grade levels
Gunning Fog Index readability formula (http://gunning-fog-index.com/)
Simple Measure of Gobbledygook (SMOG - http://www.learningandwork.org.uk/misc/SMOG-calculator/smogcalc.php?redirectedfrom=niace)
Dale-Chall readability formula (http://www.readabilityformulas.com/free-dale-chall-test.php)
Italian Read-IT DyLan Text Tool v2.1.9 (http://www.ilc.cnr.it/dylanlab/apps/texttools/?tt_user=guest), and
Italian Readability Analyzer (http://labs.translated.net/text-readability/).
For each test, the text from the website was copied and pasted into the tool. We were unable to identify any tools that evaluated readability in Macedonian. Results of the readability tests indicated that the written content was somewhat difficult to understand. This result may be reflective of some of the medical terms used, for example “Alzheimer’s” and may be improved by rewording the written content of each page to reduce the number of long sentences. It is recognised that reducing both sentence length and word count can improve upon the overall quality of the website . However, it is worth noting that the information presented on the site is quite complex, which is why caregivers experience so many difficulties in managing the medications of their loved ones. Thus, it may not always be possible to avoid complex sentence structures or word use in medication management resources.
4.4.1. Method critique
Readability testing proved to be both quick and inexpensive to undertake as it did not require any specialist software. As with other studies, readability testing allowed the identification of content problems not found through usability testing alone . The different readability tools produced different readability scores, which is reflective of the different criteria assessed and different formulae used to calculate the scores. Although the readability levels were found to be relatively high, it is worth noting that the information is also available on the website as an audio version in all three languages. This makes the information more accessible to people with stronger oral literacy skills.
A further Readability Analysis using VocabProfile Compleat (VPC) (http://www.lextutor.ca/vp/comp/)  was conducted as part of our site evaluation, to gain a better understanding of the readability for immigrants whose second language might be English. This tool is useful to analyse the text according to the most frequently used word families in the English language. VPC categorised all the words throughout the Managing Medicines for People with Dementia website into levels of the most common words used in English with 1000 words in each level: first (K-1 words), second (K-2), third (K-3.) thousand. The frequency scores show which words an immigrant is most likely to be exposed to. Those in the lower thousands would likely be known whereas those in the higher thousands, less so. The results of this analysis revealed that the majority of words fell within the first one thousand level (79.32%) and second thousand level (10.61%) categories overall, suggesting that the word choice throughout the site is accessible to English as a second language speakers.
This frequency information is important to consider because the greater the number of words that fall outside the first two categories (K 1-2), the greater the potential difficulty that second language speakers of English would likely experience in understanding the website, especially if they have limited proficiency in English. Ideally, websites imparting information to second language speakers should try to utilize words that make up at least 95% of the readers’ receptive vocabulary. This figure was determined based on Hsueh-Chao and Nation’s  description of optimal conditions for extensive reading within an English as a Second Language curriculum being 95–98% of known words and Nation’s  argument that 98–99% is ideal.
4.5. Quality of information on health websites
Almost anyone can create a website and author content that can be accessed by the general public. It has been estimated that 30 million new websites are created every day, with the vast majority relating to health conditions . As the number of people with easy access to the internet continues to rise, it is important that the information that they access be reliable, particularly in relation to health conditions.
A number of organisations have developed codes of ethics and evaluation instruments to determine the reliability and creditability of information on the internet. A third-party approach is often used to evaluate online health information, so that users can determine the quality of information, through displaying either a label, seal or logo on their website . A number of such accreditation systems are available for medical and health websites. Credibility, relevance and accuracy are especially important for health websites where information can have a direct impact on the health and well-being of the site user.
Methods: Three website rating tools were used to assess the Managing Medicines for People with Dementia website for accessibility, quality and reliability.
Suitability Assessment of Materials (SAM http://aspiruslibrary.org/literacy/sam.pdf);
Health on the Net Checklist (HONcode - http://www.hon.ch/HONcode/Patients/);
Health-Related Website Evaluation Form .
The Suitability Assessment of Materials (SAM) instrument contains six sections: content, literacy demand, graphics, layout and type, learning stimulation and motivation and cultural appropriateness. The SAM rates factors affecting the difficulty of readability and the comprehension relative to understanding the meaning. For each factor, the materials were categorised and rated as not suitable (0), adequate (1) and superior (2) based on objective criteria.
The Health-Related Web Site Evaluation Form consists of 36 questions, covering: content, accuracy, author, currency, audience, navigation, external links and structure. The rating scale criteria were defined as poor (<75%), adequate (75–89%) and excellent (>90%).
The Health on the Net Foundation (HONcode) is a self-regulating non-government organisation with a set of principles adopted to standardise the creditability and reliability of health and medical information on the internet . It was developed to provide a code of ethics for medical and health-related information on the internet, for three target audiences, the general public, health professionals and web publishers. It is suggested that if health information websites adhere to the HONcode, it will go a long way to ensuring that the health information provided to the public is both of good quality and reliable. The HONcode certification has eight principles: authority, complementary, confidentiality, attribution, justification, transparency, financial disclosure and advertising, which are assessed through 12 items.
4.5.1. Method critique
All of the instruments were found to be quick and easy to administer, and showed that the website provided reliable information and allowed the identification of some minor problems in the website design.
4.6. Future recommendations
It is important to plan to undertake assessment for usability, readability and accessibility levels when designing a website. Often this is an iterative, multimodal process. The Phase 2 evaluation highlighted the importance of involving end-users in the design of a health-information website. It showed that a combination of qualitative and quantitative methods can be effectively utilised to identify design and content problems. Further, the methodologies used were often simple to undertake and relatively cheap to conduct.
The results of the evaluation process will be used to inform the final design changes to the website. The site will subsequently be made available to national bodies within Australia, such as Alzheimer’s Australia, for promotion. Further funding will be sought in order to translate information into other community languages.
Good management of the individual needs of the person with dementia may mean that they can live in the community setting for as long as possible. This includes attention to the safe and effective management of medications. Acknowledgement, information and support for caregivers in their medication management role are important, both for the ongoing care of the PLWD and to help prevent or reduce caregiver stress. Medication management support and information are especially needed by linguistically diverse caregivers.
The provision of accessible and reliable online information was found to be useful for caregivers of PLWD, including those from linguistically diverse backgrounds. It is important that resources are developed, evaluated and refined to ensure their content is accessible to a wide range of audience, including those with lower literacy levels. Following the precautionary principle, if the needs of low literacy groups are met, then a resource will be accessible to the widest audience. However, developing an online dementia-specific information resource to meet low literacy levels is problematic and requires attention to word choice and sentence length to improve readability. Uniquely, this resource incorporated additional languages, other than English, and oral versions of the content. This helped to address the specific literacy needs of the resource's target audience.
The research and development team recommend that further funding and research are urgently needed to meet the needs of those who are ageing in a host country, especially those with dementia. This is especially important given that many older adults will migrate during their lifetime, often to a country where they are not familiar with the language or health services.
The research team wishes to thank the Multicultural Health Service of the Illawarra and Shoalhaven Local Health District for their recognition of the importance of this study and the initial funding of the project.
Two summer scholarships were provided by the Illawarra Health and Medical Research Institute via the Bridgewater Family Trust to develop the initial pilot resource.
A Living Well Living Longer Global Challenges Grant was also received from the University of Wollongong to redesign the website and complete the evaluation stage.
We thank the Multicultural Health Workers who have assisted with the project giving helpful advice and making contact with study participants.
Finally, this work could not have been completed without the stories shared by the family caregivers of people living with dementia. We thank each one for their willingness to give of their time and talk about their experiences in their caring role.
The authors would like to acknowledge the work of Holly Gleeson, Rhett Thomas-Lee and Catherine Kostovski who undertook the: readability assessment, quality of information on health websites analysis and usability testing as part of their final year, undergraduate population health placement at the University of Wollongong.