ABSTRACT
Background
The COVID-19 pandemic has impacted the global health system, with different repercussions on the mental health of health professionals. This study introduces the process of developing and validating a survey to assess factors associated with the quality of life of hospital pharmacists during the COVID-19 pandemic.
Materials and Methods
This is a descriptive study involving the phases of survey development and internal validation, performed by expert pharmacists who rated the proposed items for relevance and clarity. The assessment was conducted in February and March 2022.
Results
Responses were evaluated by the Item-level Content Validity Index (I-CVI Desirable outcome ≥ 0.78) and the scale-level Content Validity Index (total CVI; Desirable outcome ≥ 0.9). The survey was designed with 23 questions and presented three sections: 1-Socio-demographic profile; 2-Work characteristics; and 3-Perceptions related to COVID-19, with the objective of composing a final instrument in which Section 4-Instrument for assessing Professional Quality of Life will be added. The CVI analyses of all items proved to be acceptable (I-CVI > 0.78, total CVI-clarity = 0.97, total CVI-relevance = 0.99). Thirty suggestions were received, and 16 questions had some wording and/or options changes.
Conclusion
The survey showed desirable content validation results and was considered ‘valid’ for investigating factors associated with the quality of life of pharmacists in the context of the COVID-19 pandemic.
INTRODUCTION
The COVID-19 pandemic has impacted the global health system, with different repercussions on the mental health of health professionals.1 The deterioration of quality of life has been considered a crucial threat to the mental health of healthcare providers and the advancement of studies on this health condition can contribute to the development of strategies to promote mental health in this professional category.2
One way to observe professional and emotional involvement in healthcare providers is the Professional Quality of Life assessment, which evaluates the quality that a professional feel in relation to their work.3 This assessment includes compassion satisfaction, characterized by feelings of satisfaction toward care work, and compassion fatigue, which refers to the negative aspects of care work, a condition of physical and emotional exhaustion as a result of the empathic cost of dealing with another’s suffering2 and includes the concepts of burnout, defined as physical and psychological exhaustion that arises as a reaction to chronic stress at work4 and secondary traumatic stress, characterized by fear and work-related trauma. Professional Quality of Life is established from the balance between compassion satisfaction and compassion fatigue, so that positive feelings prevail over negative ones.3
Prior to the COVID-19 pandemic, burnout rates among healthcare providers were already reported to be higher than in the general working population. Among pharmacists, studies indicated that burnout rates were over 50% and largely driven by exhaustion.5–7 In a study conducted with American hospital clinical pharmacists, Jones, et al. found a high burnout rate due to emotional exhaustion, with time spent on non-clinical activities and feeling that contributions are undervalued, for example, as predictors.7
Another study, conducted with pharmacists during the first weeks of the COVID-19 pandemic, revealed that nearly half of pharmacists in the U.S. healthcare system were experiencing burnout; moreover, pharmacists had a moderate or high chance of secondary traumatic stress, but the level of compassion satisfaction was also high. This research highlights that the development of burnout and secondary traumatic stress can lead to work-related consequences and suggests that further studies are essential to better understand the long-term effects of the COVID-19 pandemic on pharmacists’ well-being.8
In Brazil, on the other hand, a significant impact was observed during the pandemic period in several aspects. The hospital pharmacist was one of the professional categories affected in facing this tragedy and there were no Brazilian studies focused on the assessment of their professional quality of life and associated factors during the pandemic. Thus, national and international studies have begun to assess issues related to the mental health of healthcare providers in this context.8–12 However, although they use validated instruments to assess the investigated conditions, the associated factors to be analyzed are defined by the researchers without prior validation. That way, the purpose of the present work is to introduce the process of development and validation of a survey to assess the factors associated with the professional quality of life of pharmacists working in Brazilian hospitals during the COVID-19 pandemic.
MATERIALS AND METHODS
This is a descriptive study involving the development phases of a specific survey, from its structuring to its validation.
Developing the survey
For the development of the survey in order to assess the factors associated with the professional quality of life of hospital pharmacists during the COVID-19 pandemic in Brazil, a literature review was initially conducted.7,8 The survey was designed in three sections: Section 1- Socio-demographic profile; Section 2- Work characteristics; Section 3- Perceptions related to COVID-19, totaling 23 questions, with the objective of composing a final instrument, in which Section 4- Instruments for the evaluation of Professional Quality of Life (ProQOL)2. will be added. The survey was prepared in a Word® document and then structured as an electronic form using Google Docs®.
Internal Validation and sampling strategy
The sample of pharmacists targeted by the study was composed of a non-probabilistic strategy, based on literature.13 For internal validation, two pharmacists from every Brazilian region were selected (total: 10 evaluators)13 with proven experience in the area of hospital pharmacy and selected according to the following criteria: 1- Active search for names on the website of the Brazilian Society of Hospital Pharmacy (members of the regional boards, lecturers at congresses and/or preparation of technical material);14 2- Analysis of their entry on the Brazilian national resumé database,15 with practice and specialization or residency in the area of hospital pharmacy or related areas as a pre-requisite; 3-Contact with the professional and evaluation of availability to contribute as a evaluator of the research.
The experts were invited to participate in the research via e-mail or phone contact and, during this first contact, the purpose of the research, the role of the evaluator, the time required to answer the survey (estimated in 40 to 50 min), and the deadline date were explained. The survey was sent via e-mail to the experts who agreed to participate in the survey (n=6) in two versions: version 1 – “Hospital Pharmacist: How is your professional quality of life two years after the beginning of the COVID-19 pandemic in Brazil?” and version 2 – “Evaluator Validation”, along with the instructions for completing the survey.
Initially, the experts were instructed to answer the Free and Informed Consent Form and the four sections of the survey “Hospital Pharmacist: How is your professional quality of life two years after the beginning of the COVID-19 pandemic in Brazil?”, simulating the completion of the survey as a participating member of the research and marking the beginning and end times of completion on a stopwatch. Next, the experts were asked to answer the “Evaluator validation” survey, in which they were asked about the clarity and relevance of each question in sections one to three. In each item, there was a space to record the evaluators’ suggestions concerning the question at hand, as well as a space at the end of the survey for general suggestions. The evaluators were asked to maintain research confidentiality during the validation process.
In the instructions for completing the survey, it was explained that the relevance criterion considered the importance and adequacy of the question to achieve the proposed objectives (relevance scale: 1-irrelevant, 2-somewhat relevant, 3-relevant, and 4-very relevant), and whether all necessary dimensions of the purpose were included. The clarity criterion assessed the editing of the items, in order to verify if the concept expected to be measured is fully understandable and adequately expressed (clarity scale: 1-not clear, 2-somewhat clear, 3-clear, and 4-very clear).13
The evaluation was conducted individually and independently by each expert during the period of one month (February 25, 2022 to March 24, 2022), followed by a moment of interaction between the researcher and the evaluator to clear up any doubts and validate the adjustments, if necessary.
The experts’ answers were evaluated via the Content Validity Index of each item (I-CVI), which measures the proportion or percentage of evaluators who agree on certain aspects of the instrument and its items for each question (formula used: I-CVI = Number of 3 or 4 answers / Total number of answers). According to Polit and Beck, questions with CVI lower than 0.78 must be reviewed and adjusted in an analysis with 6 evaluators.16 We also evaluated the Scale Content Validity Index (total CVI), calculated as the average of the I-CVI (adding and dividing by the number of items), which must be equal to or greater than 0.9.17
Data Analysis
The data from the validation process were organized in a computerized Excel® spreadsheet to determine the o Content Validity Index for each item as well as for the scale.
Ethical Considerations
The study was approved by the Research Ethics Committee, Federal University of Ceará, on 19 November 2021, with CAAE (Certificate of Presentation for Ethical Consideration) number: 52286121.0.0000.5054. The participants signed the Terms of Free and Informed Consent before entering the study.
RESULTS
The survey was designed with 23 questions, including two open questions (Questions 2 and 9), and 21 multiple choice questions, introducing three sections: Section 1 – Socio-demographic profile, with seven questions related to status, age, gender, marital status, children, practice of physical activity, and sleep; Section 2 – Work characteristics, with eight questions related to the type of institution they work in, salary, functions performed, workload, years of professional experience, additional training, type and amount of employment ties; Section 3 – Perceptions related to COVID-19, with eight questions referring to the period of the COVID-19 pandemic, including the number of hours worked, suspension of the work contract, change of activities performed, salary, remote work, situations experienced, appreciation, desire to work in a hospital.
After the designing of the survey, the internal validation was performed by six experts, with one representative each from the North, Northeast, Midwest, and South regions, and two representatives from the Southeast region of Brazil. The experts had different lengths of professional experience (4 to 17 years of experience) and reported an average time of 7 min to answer the survey (ranging from 5 to 9 min).
When analyzing relevance, only one item was rated “somewhat relevant” (question 4). In the clarity analysis, four items were rated “somewhat clear” by one evaluator each (questions 4, 6, 7, 17) (Figures 1 and 2).
Figure 1:
Judges’ Responses in The Survey Relevance Analysis.
Figure 2:
Judges’ Responses in The Survey Clarity Analysis.
The evaluation of the survey was conducted in only one cycle, because, although some items were evaluated as “somewhat relevant” or “somewhat clear”, the CVI analyses of clarity and relevance of all items showed to be acceptable (I-CVI > 0.78, ranging from 0.83 to 1.0), with total CVI = 0.97 for clarity and total CVI = 0.99 for relevance, and is, therefore, above 0.90, as recommended (Tables 1 and 2).
| Questions | Judge 1 | Judge 2 | Judge 3 | Judge 4 | Judge 5 | Judge 6 | Agreement Value | I-CVI | Interpretation |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 4 | 4 | 3 | 4 | 4 | 3 | 6 | 1.00 | Acceptable |
| 2 | 4 | 4 | 3 | 4 | 4 | 3 | 6 | 1.00 | Acceptable |
| 3 | 4 | 4 | 3 | 4 | 4 | 3 | 6 | 1.00 | Acceptable |
| 4 | 4 | 2 | 3 | 3 | 4 | 3 | 5 | 0.83 | Acceptable |
| 5 | 4 | 3 | 4 | 3 | 4 | 3 | 6 | 1.00 | Acceptable |
| 6 | 4 | 4 | 4 | 3 | 4 | 3 | 6 | 1.00 | Acceptable |
| 7 | 4 | 4 | 4 | 3 | 4 | 4 | 6 | 1.00 | Acceptable |
| 8 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 9 | 4 | 4 | 3 | 4 | 4 | 4 | 6 | 1,00 | Acceptable |
| 10 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 11 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 12 | 4 | 4 | 4 | 3 | 4 | 4 | 6 | 1.00 | Acceptable |
| 13 | 3 | 4 | 3 | 3 | 4 | 3 | 6 | 1.00 | Acceptable |
| 14 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 15 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 16 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 17 | 4 | 3 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 18 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 19 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 20 | 4 | 3 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 21 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 22 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 23 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| Total CVI | 0.99 | Acceptable |
| Questions | Judge 1 | Judge 2 | Judge 3 | Judge 4 | Judge 5 | Judge 6 | Agreement Value | I-CVI | Interpretation |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 4 | 4 | 3 | 4 | 4 | 3 | 6 | 1.00 | Acceptable |
| 2 | 4 | 4 | 3 | 4 | 4 | 3 | 6 | 1.00 | Acceptable |
| 3 | 4 | 4 | 3 | 4 | 4 | 3 | 6 | 1.00 | Acceptable |
| 4 | 4 | 2 | 3 | 3 | 4 | 3 | 5 | 0.83 | Acceptable |
| 5 | 4 | 4 | 4 | 3 | 4 | 3 | 6 | 1.00 | Acceptable |
| 6 | 4 | 4 | 4 | 2 | 4 | 3 | 5 | 0.83 | Acceptable |
| 7 | 3 | 4 | 4 | 2 | 4 | 3 | 5 | 0.83 | Acceptable |
| 8 | 4 | 4 | 4 | 4 | 4 | 3 | 6 | 1.00 | Acceptable |
| 9 | 3 | 4 | 3 | 4 | 4 | 3 | 6 | 1.00 | Acceptable |
| 10 | 4 | 3 | 4 | 3 | 4 | 3 | 6 | 1.00 | Acceptable |
| 11 | 4 | 3 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 12 | 3 | 4 | 4 | 3 | 4 | 4 | 6 | 1.00 | Acceptable |
| 13 | 4 | 3 | 3 | 3 | 4 | 4 | 6 | 1.00 | Acceptable |
| 14 | 4 | 3 | 4 | 4 | 4 | 3 | 6 | 1.00 | Acceptable |
| 15 | 4 | 4 | 4 | 4 | 4 | 3 | 6 | 1.00 | Acceptable |
| 16 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 17 | 4 | 2 | 4 | 4 | 4 | 3 | 5 | 0.83 | Acceptable |
| 18 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 19 | 4 | 3 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 20 | 4 | 4 | 4 | 3 | 4 | 4 | 6 | 1.00 | Acceptable |
| 21 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 22 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| 23 | 4 | 4 | 4 | 4 | 4 | 4 | 6 | 1.00 | Acceptable |
| Total CVI | 0.97 | Acceptable |
During the validation process, 30 suggestions were received, 36.7% of which (n=11) were for adjustments in the wording of the questions; 30.0% (n=9) were for adjustments in the options (inclusion, exclusion, change in the text, permitting the choice of more than one option); 13.3% (n=4) were for adjustments in the wording and options; 10.0% (n=3) were for inclusion of new questions; 6.7% (n=2) were suggestions for the validation process; 3.3% (n=1) were for change in the order of the questions. From the suggestions received, 16 questions had some changes, with eight changes in the wording of the question; five changes in the statement and options; three changes only in the options.
The final survey (Appendix 1), with 23 questions plus section 4, was sent to Brazilian hospital pharmacists from April 1, 2022 to July 9, 2022 (100 days of data collection), 236 valid responses were obtained from representatives from all states of Brazil.
DISCUSSION
In order for the results obtained in a research study to be reliable, the use of adequate and accurate data collection instruments is necessary. A large number of surveys and scales are currently being applied in healthcare, making it necessary for researchers to watch out for the quality of the data collection instruments used.13
In order to achieve a good quality in our instrument, the development of the survey had an initial stage in which a literature search was conducted to establish the questions and items of interest.18 After, the validation stage was performed by experts.19 The experts selected for the validation of our survey were chosen so as to obtain representativeness from all regions of Brazil and from different periods of professional practice; thus, we considered obtaining different views on the suggested questions, which contributed to the development of a final instrument capable of assessing different factors associated with the professional quality of life of pharmacists in the context of the pandemic of COVID-19.
When analyzing relevance and clarity, it was possible to obtain the desired result in the very first cycle, which suggests that the survey was sufficiently ‘clear’ and ‘relevant’ to be used in research, with no need to conduct a second cycle.20
Despite this result, some questions underwent changes as suggested by the experts. These changes were passed on to the evaluators, however, there was no need to perform a second cycle of analysis because the changes did not generate substantial changes in the items. Thus, as observed in other studies,20,21 the suggestions received were crucial for the improvement of the survey, since they increased the clarity of the questions, contributing for the instrument to assess, effectively, what was intended.
Our study had some limitations, such as the use only of the content validity index as a psychometric measure and the possibility of not having all the variables that could be related to the professional quality of life of hospital pharmacists identified, thus exhausting the theme. Yet, the validated survey was used in a national survey and allowed the identification of factors associated with the professional quality of life of hospital pharmacists, suggesting the groups that need more attention (data not shown).
The designing process of this instrument with the experts brought more reliability for the analysis of socio-demographic and work variables that may be associated with the professional quality of life of hospital pharmacists. The final instrument, divided into three sections (Sociodemographic profile, Work characteristics, and Perceptions related to COVID-19), can be used as a reference in future studies with hospital pharmacists, and can be used in its entirety for studies in the context of the COVID-19 pandemic or only the socio-demographic profile and work characteristics sections for professional quality of life studies in other scenarios.
CONCLUSION
The instrument developed aims to be applied to groups of hospital pharmacists in order to assess factors associated with the professional quality of life of these healthcare providers when faced with the pandemic of COVID-19. The survey presented results for content validation in desirable standards, and, therefore, the content is considered “valid” for investigation of associated factors and their interfaces in this context.
References
- Restauri N, Sheridan AD. Burnout and posttraumatic stress disorder in the coronavirus disease 2019 (COVID-19) pandemic: intersection, impact, and interventions. J Am Coll Radiol. 2020;17(7):921-6. [PubMed] | [CrossRef] | [Google Scholar]
- Lago KC, Codo W. Fadiga por compaixão: evidências de validade factorial e consistência interna do ProQol-BR. Estud Psicol (Natal). 2013;18(2):213-21. [CrossRef] | [Google Scholar]
- Stamm BH. The Concise ProQOL Manual. 2010 [Feb 15 2022].
- Maslach SC. Comprendiendo el burnout. Rev Cien Trab. 2009;11(32):37-43. [CrossRef] | [Google Scholar]
- Ball AM, Schultheis J, Lee HJ, Bush PW. Evidence of burnout in critical care pharmacists. Am J Health Syst Pharm. 2020;77(10):790-6. [PubMed] | [CrossRef] | [Google Scholar]
- Durham ME, Bush PW, Ball AM. Evidence of burnout in health-system pharmacists. Am J Health Syst Pharm. 2018;75(23-S4):S93-100. [PubMed] | [CrossRef] | [Google Scholar]
- Jones GM, Roe NA, Louden L, Tubbs CR. Factors associated with burnout among US hospital clinical pharmacy practitioners: results of a nationwide pilot survey. Hosp Pharm. 2017;52(11):742-51. [PubMed] | [CrossRef] | [Google Scholar]
- Jones AM, Clark JS, Mohammad RA. Burnout and secondary traumatic stress in health-system pharmacists during the COVID-19 pandemic. Am J Health Syst Pharm. 2021;78(9):818-24. [PubMed] | [CrossRef] | [Google Scholar]
- Turcu-Stiolica A, Bogdan M, Subtirelu MS, Meca AD, Taerel AE, Iaru I, et al. Influence of COVID-19 on health-related quality of life and the perception of being vaccinated to prevent COVID-19: an approach for community pharmacists from Romania and Bulgaria. J Clin Med. 2021;10(4):864 [PubMed] | [CrossRef] | [Google Scholar]
- Elbay RY, Kurtulmuş A, Arpacıoğlu S, Karadere E. Depression, anxiety, stress levels of physicians and associated factors in Covid-19 pandemics. Psychiatry Res. 2020;290:113130 [PubMed] | [CrossRef] | [Google Scholar]
- Lai J, Ma S, Wang Y, Cai Z, Hu J, Wei N, et al. Factors associated with mental health outcomes among health care workers exposed to coronavirus disease 2019. JAMA Netw Open. 2020;3(3):e203976 [PubMed] | [CrossRef] | [Google Scholar]
- Rossi R, Socci V, Pacitti F, Di Lorenzo G, Di Marco A, Siracusano A, et al. Mental health outcomes among frontline and second line health care workers during the coronavirus disease 2019 (COVID-19) pandemic in Italy. JAMA Netw Open. 2020;3(5):e2010185 [PubMed] | [CrossRef] | [Google Scholar]
- Alexandre NMC, Coluci MZO. Content validity in the processes and adaptation of measuring instruments. Cien Saúde Colet.. 2011;16(7):3061-8. [PubMed] | [CrossRef] | [Google Scholar]
- Members of the Regionals of the Brazilian Society of Hospital Pharmacy [internet]. [Feb 15 2022].
- Search CNPQ. Curriculum lattes [internet]. [Feb 15 2022].
- Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? critique and recommendations. Res Nurs Health. 2006;29(5):489-97. [PubMed] | [CrossRef] | [Google Scholar]
- Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382-5. [PubMed] | [CrossRef] | [Google Scholar]
- DeVon HA, Block ME, Moyle-Wright P, Ernst DM, Hayden SJ, Lazzara DJ, et al. A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh. 2007;39(2):155-64. [PubMed] | [CrossRef] | [Google Scholar]
- Grant JS, Davis LL. Selection and use of content experts for instrument development. Res Nurs Health. 1997;20(3):269-74. [PubMed] | [CrossRef] | [Google Scholar]
- Pedreira RBS, Rocha SV, Santos CAD, Vasconcelos LRC, Reis MC. Content validity of the Geriatric Health Assessment Instrument. Einstein (São Paulo). 2016;14(2):158-77. [PubMed] | [CrossRef] | [Google Scholar]
- Rabelo Néri EDR, Woods DJ, França Fonteles MMF. Assessment of knowledge, skills and attitudes in the use of information technology to support hospital pharmacist’s clinical practice: development and validation of a questionnaire. J Young Pharm. 2018;10(4):439-43. [CrossRef] | [Google Scholar]
