Development and Assessment of Scales in the Area of Psychiatry and Mental Health during the COVID-19 Pandemic

Abstract

Nowadays, mental health problems and psychiatric disorders have a high prevalence and are caused by co-factors. They can relapse and be exacerbated by internal and external factors such as stressful life events, poor coping skills, and COVID-19. The early detection of specific signs and symptoms is complicated. Frontliner clinical nurses must assess patient signs and symptoms as soon as possible. For this process, they require a quick and early detection measurement tool that precedes the interview, physical examination, and laboratory tests. A scale with good psychometric properties will help nurses screen and identify individuals as high-risk or non-high-risk, the severity of their symptoms (mild, moderate, or severe), and provide efficient nursing care.

1. Introduction

Measurement is essential for healthcare providers to understand the population’s health status and trends over time and to measure the effectiveness of interventions to improve it. COVID-19 has spread worldwide at an unprecedented rate and scale. People are experiencing its various psychological effects, ranging from severe symptoms to stressful responses, such as depression, anxiety, suicidal ideation, concern about infection, uncertainty and helplessness from the prolonged pandemic, and loneliness from quarantine and social isolation [1, 2, 3]. These psychological problems persist without being identified or treated and can lead to more serious psychological diseases [4, 5, 6]. The current pandemic has caused people to become exhausted in their daily lives. Therefore, it is essential to understand the psychological problems experienced by the general population and cope with them during the pandemic to protect them from psychological illness.

A scale in the area of psychiatry and mental health has been developed using valid and reliable measures. Various indicators can measure aspects of health; that is, instruments that summarize the data related to a unique phenomenon. Excellent quality instruments should measure what is hypothetical, provide the same result if measured by different people in similar circumstances, and measure and reflect changes only in the situation concerned. Sound mental health and psychiatric scales should reflect an aspect of a chosen target, staging of the problem, and cost–benefit. It measures the state of mental health and related needs. It should inform its users whether the set targets are being achieved.

To choose the right scale, knowledge of the process of scale development and assessment of the scale in psychometric properties is essential. This chapter aims to provide suggestions for developing and choosing the scale.

2. Assessment of mental health and psychiatric needs and needs index models

The need for care, perceived need for care, demand for care, and use of care are four concepts. Mental and psychiatric problems are linked to various physical, psychological, social, and economic needs. Three groups of these needs have been assigned by WHO, which are associated with impairment, disabilities, and handicaps. The needs for mental health can be determined at either the individual or population level, and needs are estimated at the population level using four methods: (a) the survey method, (b) analysis of utilization data, (c) analysis of socioeconomic factors, and (d) a combination of techniques. The need for intervention due to mental health problems is not satisfactory. To bridge this gap, there is a need for a new scale to assess mental health needs and psychiatric problems during the COVID-19 pandemic.

3. Scale in the field of mental health and psychiatry during the COVID-19 pandemic

There are some new measures to assess psychological problems during the COVID-19 outbreak in the database during 2020–2022 as follows:

1. Stress

   • COVID-19 student stress scale [7]

2. Fear

   • The fear of COVID19 scale [8]

   • The fear of COVID-19 scale [9]

3. Phobia

   • The COVID-19 phobia scale [10]

4. Anxiety [11]

   • The coronavirus anxiety scale [12]

5. Depressive

   • COVID-19 depression scale for healthcare workers [13]

6. Posttraumatic stress disorders

   • Posttraumatic stress disorder questionnaire [14]

Scale	Sample	Administration	Psychometric properties
Stress
COVID-19 Student Stress Scale [7]	514 Italian university students	The Likert scale consists of seven items. The scale consists of three factors (relationships and academic life, isolation, and fear of contagion).	The scale of fit values were found to be good (v2 /df = 0.56; CFI = 0.95; TLI = 0.95; RMSEA = 0.06), and; cronbach’s alpha coefficient = .71.
Fear
The fear of COVID19 scale [8]	717 Iranian participants	This 7-item scale uses a five-point Likert-type; Scoring ranges from “strongly disagree (1) to “strongly agree (5).” A high score shows a high level of fear of the COVID-19 outbreak. The scores on this scale range between 7 and 35. This scale has no reverse-scored items.	The corrected item-total correlation range from 0.47 to 0.56, and factor loadings range from 0.66 to 0.74.; The properties evaluated using classical test theory and the Rasch model were satisfactory on the seven-item scale.; Internal consistency (α = .82) and test–retest reliability (ICC = .72).; Concurrent validity was measured by the Hospital Anxiety and Depression Scale (with depression, r = 0.425, anxiety, r = 0.511) and the Perceived Vulnerability to Disease Scale with perceived infectability (r = 0.483) germ aversion (r = 0.459).
The fear of COVID-19 scale; [9]	1304 participants from 75 cities in Turkey. Of all the participants, 917 (70.3%) were females, and 387 (29.7%) were males whose ages ranged from 18 to 64 years (M = 29.47, SD = 10.54)	A unidimensional seven-item, five-point Likert scale, ranging from 1 to 5 (1-strongly disagree, 5-strongly agree). There are no reverse-scored items. Scores range between 7 and 35. A high score indicates a high level of fear of the COVID pandemic.	CFA analyses showed that fit indices were all within the acceptable limit [χ2 (13, N = 1304) = 299.47, p < .05; SRMR = .061; GFI = .936; NFI = .912; IFI = .915; CFI = .915]. The factor loadings of the Fear of COVID-19 Scale ranged from .484 to .723.
Phobia
The COVID-19 phobia scale [10]	1250; participants (765 women, 61.2%) with a mean age of 37.53 years; (SD = 16.94, range = 17–89 years).	A self-report scale to measure levels of coronavirus (COVID-19) phobia. This scale was rated on a five-point scale from 1 to 5 (strongly disagree to strongly agree. The scores on the scale can range between 20 and 100, and a higher score indicates a greater phobia in the respective subscales and total scale.	Kaiser measure of sampling adequacy for EFA was 0.926; Bartlett test of sphericity was significant; χ2 (df = 190) = 14,396.195, p < .001. χ2 (df = 125) = 446.93, χ2/df = 3.57, p < .001, GFI = 0.97, AGFI = 96, NFI = 0.98, IFI = 0.98, TLI = 0.98, CFI = 0.98, and RMSEA = 0.03 [90% confidence interval = 0.03 and 0.03].
Anxiety
The coronavirus anxiety scale; [11]	Coronavirus Anxiety Scale: A brief mental health screener for COVID-19	A pool of 20 candidate items was created based on; the psychology of fear and anxiety literature; [15, 16, 17, 18]. Each item was written to capture a; a unique manifestation of this particular form of anxiety in 4 dimensions: cognitive (i.e., repetitive thinking, worry, processing biases, dreaming, planning) behavioral (i.e., dysfunctional activities, avoidance, compulsive behaviors) emotional (i.e., fear; anxiety; anger) physiological (i.e., sleep disturbances, somatic distress, tonic immobility) dimensions of coronavirus anxiety. All item was rated on a five-point scale to assess the symptom frequency, ranging from “not at all (0)” to “nearly every day(4)” over the preceding 2 weeks.; This measurement arrangement is based on the DSM-5’s cross-cutting symptom quantity, the adult self-rated version (APA, 2013, pp. 734), consistent with the American Psychiatric Association’s system of determining psychiatric symptoms over time and response to treatment.	Confirmatory factor analyses demonstrated that the CAS measures a reliable (α = 0.92), unidimensional construct with a structure that was shown to be invariant across gender, race, and age. Construct validity was demonstrated with correlations between CAS scores and demographics, coronavirus diagnosis, history of anxiety, coronavirus fear, functional impairment, alcohol/drug coping, religious coping, hopelessness, suicidal ideation, and social attitudes. The CAS also demonstrated solid discrimination ability for functional impairment (AUC =0.88), while the original cut score of ≥9 (76% sensitivity and 90% specificity) showed the most robust diagnostic effectiveness among scores.
Depress
COVID-19 Depression Scale for healthcare workers; [13]	320 Health-care workers (HCWs), including; physicians of various medical specialties, dental specialists, and nurses.	Self-report questionnaires of the 10-item COVID-19 depression scale are composed of a two-component structure identified as; work-related anxiety psychological distress All the items were formulated as a; 5-point Likert-type scale for response options which; ranged from 1 = I strongly agree, to 5 = I strongly disagree	The mean CDS-HW score of the study participants was observed to be 23.67 ± 2.82, and the scale demonstrated good internal consistency reliability (Cronbach’s alpha: 0.741)
Post-traumatic stress disorders
Post-Traumatic Stress Disorder; Questionnaire; [14]	Of 2286 respondents, there were 1706 women (74% of the sample). The mean age of the participants; was 29.61 (SD = 11.42), and the age ranged between 18 and 74 years.	A self-report questionnaire (COVID-19-PTSD), consisting of 19 items, was developed starting from the PTSD Check List for DSM-5 (PCL-5) questionnaire and was administered to analyze its psychometric properties.; The item is a five-point Likert scale ranging from “not at all (0)” to extremely (4).”; The questionnaire asks the respondents to “Referring to the COVID-19 pandemic and the social distancing events implemented to contain it, specify how you feel for each of the following dimensions.”	The confirmatory factor analysis showed that a seven-factor model (Intrusion, Avoidance, Negative effect, Anhedonia, Dysphoric arousal, Anxious arousal, and Externalizing behavior) fits the data.; Significant correlations were found among COVID-19-PTSD scores, general distress, and sleep disturbance.

4. Implication

The developer should focus on the development process to properly develop the new scale, as given in Table 1.

Process	Implication
Step 1: Identification of the dimensions of the scale	This step includes exclusive deductive methods using a literature review and inductive methods by conducting interviews. Additionally, combined deductive and inductive methods are used for a new scale [19]. Qualitative research: A deductive and inductive approach to scale development, full literature review, subjective analysis, assessment of content validity, and recruitment of a more significant number of interviewers should be a concern. Theoretical analysis: This step involves underpinning the scale theoretically some Ref. to be here to refine experts’ items or used opinions. Some developers use target population opinions, and only one expert or population judge in the area of deep specialization [19, 20, 21, 22, 23]. The fundamental theories that support the scale development process are classical test theory (CTT)—known as classical psychometry, and item response theory (IRT)—known as modern psychometry [24]. Content Validation: The panel of experts comprising 3–5 people with experience in the concerned field are asked to sort behaviors into groupings and indicate which items are applicable. This item sort is distributed in Microsoft Word format, editable by those not collocated with the researcher. This helps solidify the classifications made by the researcher and provides content validity for the survey.
Step 2: Item pool generation	Creating questionnaire draft Gathering of initial items
Step 3: Determination of the measurement scale	Likert rating scale Yes or no
Step 4: Expert review of the initial item pool	Validity of the questionnaire Content validity: At least three professionals
Step 5: Revision and inclusion of items	Pilot study Missing data: Includes numbers that have been grouped, aggregated, rounded, censored, or truncated, resulting in partial loss of information.
Step 6: Administration of the items to a development sample; Survey for validity and reliability	Methodological: The cross-sectional methodology is a limitation of scale development. The longitudinal approach in scale development facilitates a better acceptance of the analyzed variables and assesses the predictive validity. Self-reporting methodology is a limitation of new scale development. The self-reporting nature of quantitative studies raises the possibility of participant bias, social desirability, demand characteristics, and response sets which affect the validity of the findings. Incorporating objective or independent measures to supplement the subjective evaluation of the variables studied in developing the new scale and improving the interpretation of findings should be of concern. Web-based surveys are limitations of a new measurement created. The coverage bias is bias due to sampled individuals not having—or choosing not to access—the Internet) and nonresponsive bias due to the informants of a study differing from those who did not respond in terms of demographic or attitudinal variables) (Kim et al., 2011). In-person surveys or survey interviews are suggested as these methods reduce problems related to concerns about confidentiality and the potential for coverage and nonresponsive bias [25]. Sample characteristics: Homogeneous and convenient samples are the limitations of generalization. Small sample size: The new measurement should have a larger sample size (minimum ratio of 10:1) to increase the credibility of the results and thus obtain a more exact outcome in the psychometric analysis. Using separate samples is recommended for EFA and CFA [26]. However, some researchers use the same sample due to the rarity of samples, vulnerability groups, illegal/criminals (i.e., methamphetamine use/drug use/violence), the social stigma of patient and family caregivers, and high stress and anxiety to get involved in the research from their history of drug use and criminal [27]. Difficulty controlling all variables: Knowing the target construct in detail during the item generation and allowing all possible and vital variables to be investigated and forbidden. Hypothesizing and testing potential variables that could be controlled during the scale development process is the way to help control variables.
Step 7: Evaluation of the items	Psychometric analysis: The robust demonstration of the construct validity and reliability. Inadequate choice of the instruments or variables to be correlated with the study variable.; Validity Exploratory factor analysis (EFA): Analyzing a certain amount of subjectivity by identifying and labeling factors. The sample statistical results for EFA-- KMO, Bartlett test of sphericity. Confirmatory factor analysis (CFA): Analyzing and assigning items to factors, testing the hypothesized structure of the data, and statistically comparing alternative models. Examples of results from CFA are CFI, GFI, and RMSEA. Most studies opt to combine EFA and CFA to analyze the new instruments. Convergent validity is used after EFA and CFA Discriminant validity, predictive/nomological validity, and criterion validity are also used to test the construct validity Reliability: The most common technique is internal consistency, and test–retest reliability is the second technique. Item-total correlation/inter-item reliability, Split-half reliability, Inter-judge reliability
Step 8: Optimization of the tool length; Final questionnaire selection	The brevity of the scale: The short scales can critically conciliate the instrument’s reliability [28]. Many scale items tend to be more reliable, with higher alpha values [19]. Deciding to remove items from the scale may decrease Cronbach’s alpha. An alpha value between 0.8 and 0.9 is considered ideal. Item: Items are easy to answer, and reverse-scored items require good scale development.

Table 1.

Process of scale development.

For this section, the core concerns in choosing a good measurement are listed in Table 2.

Core concern	Implication
Content representativeness and relevance,	The scale should use a representative sample from a more significant performance domain and support the intended use of the assessment.
Thinking skills and processes (Substantive evidence)	The thinking processes and skills used to complete the instrument and it is significant to building knowledge in research and innovation that should be checked successfully before deciding to use the scale [12].
Internal structure evidence evaluates the relationship among the assessment tasks or parts of the instrument.	The internal structure evidence that evaluates the scale relationship among the assessment tasks or parts of the instrument should be checked.
External structure	The external structure should be composed of: How well do the assessment results correlate with other variables or criteria. Within this category are predictive validity evidence (the extent to which the individual’s future can be predicted by prior performance on an assessment instrument) and concurrent validity evidence (the extent to which an individual’s current status on a criterion can be estimated from current performance on an assessment instrument).
Reliability over time, assessors, and content domain (Reliability evidence)	The psychometric properties should be sound [19, 22].
Cost, efficiency, practicality, and instructional features (practicality evidence)	The impediments to the proper use of the assessment, such as complexity, training requirements, and cost, should be examined.
Manual instructions	The instructions should contain the details as below: Manualized instructions for raters Instructions about determining the application methods of the new scale. It defined the development of operational strategies that will enable the application of the measurement and the format in which it will be obtainable, determining how the participant’s response will be given for each item and how the respondent should respond to each item [24]. They define how the scale scores would be analyzed. The manual should be short and without confusion to the subjects, contain one or more examples of how the items should be answered, and ensure that the subject is not tense while answering the question.
Stressful life events	Recently, a new scale developed to evaluate psychological troubles based on stress responses considers the COVID-19 outbreak as an external stressor. Stress response refers to a set of affective, cognitive, somatic, and behavioral manifestations within the range of functional integrity. Stressors impact the increase or decrease in one’s adaptive capacity. The composition may vary depending on which emotional, cognitive, physical, and behavioral factors of stress responses are focused on. Therefore, choosing and measuring stress responses to COVID-19 should be a scope of the definition of stress.

Table 2.

Core concerns in choosing the measurement.

5. Conclusion

This chapter presents the type of measurement in psychiatry and mental health, the steps of scale development, assessment of mental health and psychiatric needs, needs index models, points of concern, and implications. Various concepts and methodological strategies have been identified and discussed, along with suggestions for choosing appropriate scales and scale development. We believe this chapter makes essential contributions to the literature, mainly because it provides a comprehensive set of recommendations to increase the quality of future practices in the scale development process and selection.