Development, Validation, and Psychometric Analysis of a Patient Information Leaflet for Heart Failure with Reduced Ejection Fraction

Heart Failure with reduced Ejection Fraction (HFrEF) is a complex clinical syndrome marked by the left ventricle’s structural or functional limitations, leading to diminished heart pumping ability (left ventricular ejection fraction <40%) characterized by increased mortality rates and frequent rehospitalization. Commonly associated risk factors include smoking, obesity, Diabetes Mellitus (DM), Hypertension (HTN), and a sedentary lifestyle.¹ Educating HFrEF patients about disease management and providing clear guidance on lifestyle adjustments is pivotal to enhancing their well-being and quality of life.² According to the 2022 American Heart Association/American College of Cardiology (AHA/ACC) recommendation, HFrEF patients should adhere strictly to pharmacological/non-pharmacological treatments for better clinical outcomes. Non-pharmacological treatment includes personalized counselling during discharge and follow-up visits, ultimately reflecting on medication adherence and quality of life.³ Therefore, self-care practices like fluid management, recognizing warning signs, and dietary restrictions are crucial aspects covered in the developed Patient Information Leaflet (PIL) for HFrEF patients.

The communication between a physician and a patient is the foundation of all therapeutic interventions.⁴ Today, patient-centered care emphasizes empowering individuals by providing comprehensive information and encouraging them to participate in decision-making.⁵ However, disparities in communication can lead to misunderstandings and uncertainties between healthcare providers and patients. This is especially true when considering literacy levels among populations.⁶ Research has demonstrated that incorporating PIL as supplementary aids during doctor-patient consultations significantly enhances patients’ understanding compared to consultations conducted without PIL.^7,8 Despite India’s 77% literacy rate, health literacy, especially reading skills, remains low.⁹ Therefore, designing a service system that is simple to comprehend in local languages and can be conveniently accessed and utilized at home with minimal need for professional assistance is of utmost importance.¹⁰

PIL must be crafted with precision to communicate vital information effectively.¹¹ Despite their advantages, PILs across diverse medical fields are often released without prior evaluation, exhibiting significant quality variations.¹² Ideally, PILs should undergo meticulous design and psychometric evaluation in alignment with established scientific guidelines.¹² Utilizing the user testing method can facilitate this objective.¹³ This approach involves testing PIL with a select group of targeted HFrEF patients before broader distribution. It evaluates how well patients can comprehend and navigate the content, ensuring its accessibility and clarity.¹⁴ To bridge this gap, the present study was designed to develop, validate, and evaluate the psychometric properties of the PIL for HFrEF patients in English and local (Kannada and Malayalam) languages. The PIL underwent rigorous scrutiny regarding its design, reliability, readability, and compliance with scientific standards. Its efficacy in imparting knowledge was assessed across three different languages. During the critical post-discharge period known as the ‘vulnerable phase’, individuals with HFrEF necessitate specialized attention, focusing on empowering them through self-care and education. There is a dearth of validated PILs for HFrEF patients, and the results of this study demonstrate its significance.¹⁵

The study was carried out for six months, from October 2022 to May 2023, at a hospital in south Karnataka, with approval from the Central Ethics Committee (Ref. No. NU/CEC/2022/319).

A panel of eight experts, consisting of two cardiologists, two academic pharmacists, two clinical pharmacists, one heart failure-trained nurse, and one nutritionist, collaborated to create the PIL content. Experts from various fields were selected to make PIL more user-friendly and address caregivers’ needs. The content was compiled by consulting primary, secondary, and tertiary sources. Tertiary sources were national and international guidelines for heart failure and textbooks such as “Clinical Pharmacy and Therapeutics and Pharmacotherapy: A Pathophysiological Approach, Textbook of Therapeutics: Drug and Disease Management, and Harrison’s Principles of Internal Medicine”. Secondary sources were various directories such as Micromedex, Medscape, Up-To-Date, and WebMD. Primary sources were reviews and research papers on patient education and pharmaceutical care for HFrEF patients. The information was organized systematically and divided thematically. It included all the essential information for HFrEF patients, such as the definition of heart failure, risk factors, signs and symptoms, tips for improving medication adherence, lifestyle modifications, and warning signs.

The PIL was validated by a panel of eight experts using the Lawshe method. Each item necessitated their acknowledgement by responding to a specific question provided. i.e., “Is the skill or knowledge measured by this item ‘essential,’ ‘useful, but not essential,’ or ‘not necessary’ to the performance of the construct?” The modification was done based on expert opinion. Lawshe’s method says if over half of experts consider an item essential, it has content validity. Furthermore, as more experts reach a consensus on an item’s importance, its content validity is reinforced. Given this presumption, the content validity ratio can be calculated using the following equation.

In the above equation, a value of ‘ne’ indicates that a panellist has displayed an item as “essential,” while a value of ‘N’ indicates how many panellists participated in the assessment. The formula provides results ranging from +1 to -1, where positive values signify that most panellists considered the item essential. The mean CVR of each item can measure overall content validity. According to Lawshe, the required mean CVR depends on the number of experts in the panel. The mean CVR should be above 0.75 if the panel has eight members.¹⁶ Several changes were made to the content in response to the expert’s suggestions.

Following the validation of the PILs’ content, the readability of the PILs was assessed using the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL) formulas. According to the FRE and FKGL formulas, readability depends on polysyllabic language and sentence length. A score ranging from 60 to 70 is generally acceptable. The readability of a document can be assessed using the following scale: very easy (90-100), easy (80-89), fairly easy (70-79), standard (60-69), fairly difficult (50-59), difficult (30-49), and very confusing (0-29).¹⁷ The PIL has been modified to improve its FRE/FKGL scores.

Using Baker Able Leaflet Design (BALD), the layout and design of validated English leaflets were evaluated. Design elements such as layout, line spacing, font, word choice, and images were considered. The maximum score is 32. Several modifications have been made to the document, including font, alignment, pictures, and paper quality changes.¹⁸

Following the preparation of the English PIL with clear readability, layout, and design, a proficient translator, well-versed in both languages and clinical terminologies, translated them into Kannada and Malayalam. A back-translation of the Kannada and Malayalam PILs into English was performed to ensure consistency. A language professional team reviewed the original and reverse-translated PILs to verify consistency. Subsequent evaluation revealed minimal differences between the original and back-translated versions. The translated PILs in Kannada and Malayalam also underwent layout and design assessment based on BALD criteria.

The formulas (FRE and FKGL) available for assessing the readability of English PIL cannot be used to validate PILs in local languages.¹⁹ In this context, the user-testing method was employed by issuing a reliable UTQ, which is the best-known method for evaluating readability.¹³

The sample size was determined by the change in the patient’s knowledge between the pre-and post-test.

In the above equation, power (β): 0.20; Significance level (a): 0.05; minimum significant difference (d): 3; average of standard deviations (s): 10.15. To ensure reliable results, the formula determines that a minimum of 60 participants is needed to plan the present study. Each subject was enrolled in the study according to their ability to read and write Kannada, English, and Malayalam. A subset of 30 patients from the sample population was used to assess the reliability of the User Testing Questionnaire (UTQ).

Ten multiple-choice questions (UTQ) were developed using the original English PIL version (source language). A team of experts validated the UTQ using the Lawshe method; then, language specialists translated it into Kannada and Malayalam. Before conducting user testing within the patient population, assessing the UTQ’s reliability was essential. This study opted for the test-retest methodology, which is widely recognized for reliability testing.²⁰ Test-retest reliability aims to gauge individuals’ responses to the UTQ consistency across repeated administrations. By administering the same UTQ twice to the same individuals, the Intraclass Correlation Coefficient (ICC) was calculated to evaluate test-retest reliability. An essential consideration in estimating test-retest reliability is determining the optimal time gap between questionnaire administrations. This duration should be sufficient for memory effects to diminish but not so long as to influence changes that may affect the reliability estimate. The UTQ reliability was examined in a subpopulation of 30 randomly selected HFrEF patients. To assess reliability, they were issued the Kannada, English, and Malayalam versions of the UTQ twice, spaced a week apart.

The PIL was tested on 60 HFrEF patients who visited the outpatient department of the hospital during the study period [English (n)=15, Malayalam (n)=15, Kannada (n)=30]. Patients aged 18 to 75 with a primary education background were enrolled in the outpatient cardiology department after informed consent. Data on socio-demographic characteristics such as age, sex, social history, comorbidities, education, and economic status were collected and scrutinized using the Kuppuswamy socio-economic scale.²¹ As part of a study, a pre-validated UTQ was given to 60 HFrEF patients to evaluate their baseline knowledge. The patients were then given a pre-validated PIL in their preferred language, English, Kannada, or Malayalam, and asked to read it for 20 min. After reading the PIL, the same UTQ was again administered to the same patients to evaluate their understanding, and a post-test score was calculated. The response was evaluated based on the following formula.

A questionnaire containing four questions, with scores ranging from 5 to 1, was administered to obtain patients’ opinions regarding the P-PIL’s design and layout. Patients have been provided with a PIL tailored to their preferred language. “Good” is determined by a score of 20-14 for legibility and content of PILs; “Average” is determined by a score of 14-9; and “Poor” is assessed by a score of less than 9.¹⁹

Test-retest reliability was assessed using ICC value. An ICC value of 0.9 is considered highly reliable, 0.7 to 0.89 is deemed moderately reliable, and 0.7 is considered low. Data were summarized using descriptive statistics. Pre- and post-user testing scores were calculated by paired student t-tests with a p-value of less than 0.05 considered statistically significant. Data were digitalized and analyzed using Jamovi software and Excel.

A panel of eight experts pinpointed key topics encompassing the definition, risk factors, symptoms, medication adherence tips, lifestyle adjustments, and warning signs related to HFrEF. However, heart failure classification, pharmacological treatment, and device implantation were deliberately excluded. The pertinent subjects were more thoroughly detailed in the PIL to provide comprehensive information, utilizing high-contrast text fields accompanied by graphic illustrations. While assessing the content validation of UTQ, there was a divergence in opinion among the experts, with one rating an item as “necessary but not essential.” As a result, the mean content validity ratio for UTQ was 0.97.

In contrast, the PIL obtained an ideal score of 1, indicating complete consensus among the expert panel regarding its content. Following recommendations from the expert committee, the PIL underwent revisions focusing on enhancing content related to diet and exercise. The final version of the PIL was meticulously crafted by integrating suggestions from the panel of experts. Further details regarding these adjustments are outlined in supplementary files 1 and 2.

Figure 1 represents the development process of patient information leaflet for HFrEF patients.

Figure 1:
Development, validation and psychometric analysis of PIL for heart failure patients with reduced ejection fraction.

The readability of English PIL was determined using FRE/ FKGL scores. The leaflet was revised three times to achieve an optimum FRE/FKGL score, which improved after each revision. The final English version of this leaflet scored 76.9 on the FRE test and four on the FKGL test. Scores of over 60 on the FRE test and under eight on the FKGL test indicate ease of reading and understanding. Table 1 displays the results.

The PIL design was meticulously crafted according to the latest guidelines for patient education tools. Vector graphics were employed to ensure adaptability to various formats. Specifically, the layout incorporated a colour-coded system (red, yellow, green) for warning signs, denoting the severity of symptoms. Within the PIL, there were 15 schematic illustrations presented in diverse colours alongside six black-and-white pictograms. Those visual aids were strategically positioned adjacent to the particular textual information, enhancing comprehension and clarity. A score of 29 was achieved by the English leaflet and 25 by the Malayalam and Kannada leaflets according to the BALD criteria (Table 2).

The user testing study involved 60 participants, predominantly male (72%), with an average age of 59.2±9.8 years within the patient cohort. Among the participants, 45% completed primary school, while 30% attended middle school. A majority, 55%, were identified as belonging to a low socio-economic status. Notably, 56.6% of patients had comorbidities like diabetes mellitus and hypertension, and 48% had smoking histories. Detailed demographic information about the enrolled patients is presented (Table 3). Before the user testing phase, a reliability test involving 30 patients, representing a subset of the overall population, was conducted.

The UTQ reliability was evaluated across 30 patients, with 10 participants in each language group (English, Malayalam and Kannada). The obtained ICC values were commendable: 0.95 for English, 0.94 for Kannada, and 0.91 for Malayalam. The mean ICC value across the languages was 0.93, indicating a high-reliability level (Table 4).

Among the 60 PIL users, 30 (50%) used the Kannada PIL, followed by 15 (25%) Malayalam PIL users and 15 (25%) English PIL users. The user testing scores for English PIL users improved from 38.33±9.63 to 86.6±10.7, for Kannada PIL users from 35.5±10.71 to 84±10.57, and for Malayalam PIL users from 36.6±11.46 to 85.27±9.7 with p-value <0.001. Notably, the mean scores significantly improved, increasing from 36.81±10.6 to 88.29±10.11 (<0.001). Table 5 and Figure 2 provide a detailed summary of these findings.

Figure 2:
User testing scores for English, Kannada, and Malayalam leaflet users. The blue colour indicates pre-test scores and the green colour indicates post-test scores.

The consensus among all participants indicated that the English PIL was excellently designed and written, followed by 86% agreement for the Kannada version and 94% agreement for the Malayalam PIL. Overall, 92% of patients perceived the PIL as well-designed and well-written. A detailed user opinion score for each PIL is provided in Table 6.

Clinical studies show that patients retain limited information from doctor consultations. Recalling information may be affected by psychological factors.²² Providing written information alongside verbal information enhances communication.^7,8 The PIL was arranged on two pages in this study. Thus, the reading time will be reduced to 10 min. This PIL is a source of figurative material and a guide for the doctor to address all essential topics during the consultation. The patient can acquire information about their disease and revise it at any time without consulting with a physician through this cost-effective health communication method.²³ The heavy patient load in India makes it difficult for cardiologists to devote adequate time to educating patients regarding their treatment.²⁴ Despite the widespread use of written education materials in Indian hospitals, there is a dearth of validated PILs for HFrEF patient care.²⁵ Since the disease itself is complex, this could be possible. This study is first-of-its-kind in India, preparing, validating, and evaluating systematically the psychometric properties of PIL for patients with HFrEF.

In this study, experts from various fields validated a PIL and UTQ. The panel of experts was selected based on their qualifications, experience, training, research interests, publications, and presentations. As per the Lawshe method, the CVR score should range between -1 and +1, and an expert committee of eight members must have a CVR of at least 0.75 to ensure validity.¹⁶ In this study, PIL and UTQ’s content validity was high, with a CVR score of 1 and 0.97, respectively. Once relevant topics for the PIL have been elaborated and formulated, it is imperative to create an appropriate format that captures the target audience’s interest. Key facts are graphically represented in a pictogram to make it easier to understand. Other studies have shown that pictograms improve information recall.²⁶ The BALD score for the English leaflet was 29, which is lower than the score obtained by Vooradi et al. but higher than the scores achieved in similar studies conducted by Adepu and Swamy (2012) and Mateti et al.^19,27,28 Shrestha et al. obtained comparable results for the BALD scores of the Malayalam and Kannada leaflet versions, with the Kannada version scoring 26 and the Malayalam version scoring 27.²⁹

Readability is a measure of how easily text can be read;¹⁷ Typically, improving readability results in better memory retention of the text and shorter reading times. Text that is difficult to read usually has an index between 30 and 50, while moderately complex text has an index between 50 and 70. An index of readability greater than 70 generally aims for comparable PILs.¹⁹ The original text had a readability index of 53.6, which we improved to 76.9 by rewording and using more straightforward language. The results of the present study show that the readability of the leaflets using FRE/FKGL formulae was better than the previous studies conducted by Vinay et al. and Nandakumar et al.^30,31 Previous studies assessed the PIL readability using Western formulas such as FRE, FKGL, and Simplified Measure of Gobbledygook index readability. In the current research, user testing was also conducted to evaluate PIL readability. User testing is an accurate method of assessing PIL readability in any language. The formulas used to evaluate readability can only be applied to the English version of the PIL. It was also unknown whether Western-developed formulae were valid and reliable for Indian languages.^13,19

Male heart failure patients constituted the majority of the survey population, consistent with previous research suggesting a 70:30 male-female ratio.³² The mean age of the HFrEF patients in the study was 58.8±10.02. Over 75% of the patients had diabetes mellitus, hypertension, or both, in addition to heart failure. According to European Society of Cardiology guidelines, hypertension is a primary risk factor for HF, and patients with both hypertension and diabetes are more susceptible to heart disease than those with diabetes or hypertension alone.³³ The present study reveals that most patients had primary school education. A similar mean age, male-female ratio, and educational background to Vinay et al. were observed among HFrEF patients in this study in which the majority of patients (72.27%) were males with a mean age of 58.06± 8.35 and had either completed high school or primary school education.³⁰

Based on the ICC, the most commonly used statistical method to assess test-retest reliability, the developed UTQ is reliable in all three languages. Based on the results of an assimilation study conducted to evaluate a PIL for haemodialysis patients, good reliability scores of 0.93 and 0.98 were obtained for Kannada and English UTQ, respectively.¹⁹ The study’s results confirmed that the reliability of the UTQ developed to test the readability of HFrEF PIL. According to the results of user testing, the average readability score has significantly increased from 36.81±10.6 to 88.29±10.11. This outcome is compatible with previous studies conducted by Vooradi et al., which showed an improvement from 44.80±20.17 to 87.99±12.97 in diabetes patients and 63.19±15.77 to 90.13±11.10 in hypertension patients.²⁷ Additionally, Mansoor et al. (2003) found that incorporating pictograms in PILs can enhance information recall, while Gibbs et al. (1990) revealed that verbal advice combined with a PIL could improve knowledge of medication adherence.^34,35

Opinions about PIL were evaluated using a pre-validated user opinion questionnaire that received encouraging participant feedback. Users generally rated PIL content, design, and layout positively (92%). The study’s results were better than those reported by Shrestha et al., who found 75% good and 25% average ratings.²⁹ An earlier study by Vooradi et al. revealed that only 57.5% of PIL users rated their PIL as good and 40% as moderate.²⁷ It is uncommon to find articles with over 90% of users agreeing that the PIL design and layout are satisfactory.

Certain constraints need to be considered when considering our study’s outcomes. Firstly, we could not determine the long-term retention of the information provided in the patient information leaflets by the participants. This limitation hinders our understanding of how well the acquired knowledge persists over time. Additionally, our study only focused on a single centre’s target audience, which limits our ability to generalize the findings on knowledge retention.

PILs serve as crucial educational tools across various medical fields, yet their quality often varies due to a lack of prior evaluation. However, the developed PIL adheres closely to PIL design guidelines. We discovered a marked improvement in patients’ comprehension of their medical condition through user testing. Notably, PIL ensures the information’s clarity, accuracy, and appropriateness tailored to the targeted population. This study underscores the pivotal role of pre-evaluation in PIL implementation. Hence, the developed PIL is a reliable resource for educating larger cohorts.

The authors express gratitude to all the patients who participated in the study. we want to thank the Department of Cardiology at K S Hegde charitable hospital, the Department of Pharmacy Practice at NGSM Institute of Pharmaceutical Sciences, and NITTE (Deemed to be University)for their support in fascilitating the research.