Abstract

Post-intensive care syndrome (PICS) is frequently undertreated because of the complexity of its three domains and the need for different assessment tools. There is a need for clinical tools that can assess all PICS domains simultaneously and within a short period of time. This study aimed to determine the psychometric properties of the Turkish versions of the Post-Intensive Care Syndrome Questionnaire (PICSQ-T) and Healthy Aging Brain Care Monitor Self-Report (HABC-M-T).

This methodological study included 157 intensive care unit patients. The data were collected via telephone two weeks after patient discharge. Data collection tools included the Patient Characteristic Form, the PICSQ-T, the HABC-M-T, and the standardized external scales Pfeiffer’s Short Portable Mental Status Questionnaire (cognitive domain), the Barthel Index (physical domain), and the Hospital Anxiety and Depression Scale (mental domain).

The content validity indices of both scales were greater than 0.80 at the item and scale levels. According to confirmatory factor analysis, the 18-item PICSQ-T and 27-item HABC-M-T had good fit indices, and the factor loadings of the items of these scales were above 0.30. The scales showed a significant correlation with the standard scales corresponding to the three domains of PICS. Cronbach’s alpha values were 0.94 for the PICSQ-T and 0.96 for the HABC-M-T. Test-retest analysis results were 0.84 for the PICSQ-T and 0.89 for the HABC-M-T.

The results show that the PICSQ-T and HABC-M-T, which are highly valid and reliable, may be easily used to screen for PICS.

INTRODUCTION

Intensive care medicine has advanced dramatically over the last quarter of a century through technological innovations, the development of organ support systems, and the standardization and refinement of training programs.^{1, 2} These developments have led to significant improvements in mortality. However, such promising improvements in quality of care have also created a population of intensive care patients who are likely to face a variety of challenges that can last for years after hospital discharge.^{2, 3}

The challenges faced by the survivors of intensive care units (ICUs), particularly in the physical, cognitive, and mental domains, are termed post-intensive care syndrome (PICS).⁴ Over the past 20 years, studies have shown that critical illnesses can have widespread and devastating long-term consequences, which can severely affect the patients themselves as well as their family members.⁵ In the United States, 5.7 million patients are admitted to ICUs annually, of whom approximately 4.8 million survive.⁶ It is estimated that up to 80% of critically ill patients discharged from the hospital will have symptoms of PICS, and although PICS may improve over time, more than half of them will continue to experience symptoms for a year.⁷ In an observational cohort study of ICU-treated survivors, a significant proportion of the patients were found to have newly acquired cognitive impairment, depression, and/or inability to perform activities of daily life in the three-month (64%) or twelve-month (56%) period after their discharge.⁸

To detect and treat the consequences of critical illnesses experienced by patients, healthcare professionals providing intensive care services must assess patients after discharge. Valid and reliable scales are required for this purpose. There are two scales in the literature that currently assess PICS: The Post-Intensive Care Syndrome Questionnaire (PICSQ) and the Healthy Aging Brain Care Monitor Self-Report version (HABC-M).^{9, 10} Adaptation studies on these scales are being conducted in many countries and languages (Chinese, Japanese, Spanish, French) but their adaptation into Turkish has not yet been performed. This study aimed to adapt the PICQ and HABC-M scales to Turkish and test their psychometric properties.

MATERIALS AND METHODS

Design, Setting, and Sample

This cross-sectional and methodological study was conducted between April 2022 and April 2023 in four ICUs of a hospital in the fourth largest city of Türkiye. The inclusion criteria were (a) age 18 years or older, (b) staying in ICU for 48 hours or more, (c) being on mechanical ventilation (MV) for 48 hours or more, (d) having consciousness on the day of transfer, and (e) being discharged from the ICU after more than 2 weeks.^10-13 Patients with problems in reading, writing, or comprehension in Turkish, patients with neurological diseases that may affect cognition, patients with a history of dementia or cognitive impairment, patients who were readmitted to the ICU within the study period, and patients who were referred from an external ICU were excluded from the study.

In the validity and reliability studies, the sample size was based on the number of items in the measuring instruments, with a minimum of 5 times that number.¹⁴ Since the PICSQ comprises 18 items and the HABC-M comprises 27 items, it was planned to reach at least 135 patients, considering the scale with the highest number of items.

Data Collection Procedures and Instruments

The data collection was performed by two researchers who identified potential ICU patients who met the inclusion criteria. During the discharge of these patients, the patients and their relatives were given detailed information about the study and were invited to participate in the study. Informed consent and contact information were obtained from patients or their relatives who agreed to participate. The participants were informed that they would be contacted via telephone two weeks after their discharge¹⁰ because the hospital did not have an outpatient clinic for post-intensive care follow-up. Participants were handed copies of the study’s data collection tools so that they would be familiar with these tools and could follow during the interview if they wished when they were called by phone (except for the patients included in the test-retest phase). Two weeks later, in line with the contact information received, the patient or his/her relatives were contacted by telephone, and the items for each data collection tool were read one by one, and their answers were recorded by the researchers. On average, the interviews lasted approximately 18-25 minutes per participant.

The data collection tools included the Patient Characteristic Form, the PICSQ, and the HABC-M, as well as the standardized external scales selected to represent the three domains of PICS to assess the concurrent validity of the scales: Pfeiffer’s Short Portable Mental Status Questionnaire (SPMSQ) (for the cognitive domain), the 10-item Barthel Index (BI) (for the physical domain), and the Hospital Anxiety and Depression Scale (HADS) (for the mental domain). For the three domain-specific standardized scales, we considered the most commonly used instruments in studies reporting the use of PICS in the literature.^12-14

Patient Characteristic Form: This form was prepared by the investigators and included questions regarding the socio-demographic characteristics (age, gender, etc.) and health status of the patients (comorbidities, APACHE-II score, duration of MV monitoring, presence of delirium with CAM-ICU, duration of ICU stay, etc.).^{9, 10, 12, 13}

PICSQ: This 18-item self-report scale was developed by Jeong and Kang⁹ and consists of three subscales (six items each) covering the cognitive, physical, and mental domains of PICS. Items are rated using a four-point Likert scale (0= “never,” 1= “sometimes,” 2= “most often,” 3= “always”) and the total score ranges from 0 to 54. Higher scores indicate more severe PICS.⁹The validity and reliability of the scale were tested through exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and internal consistency. The three-factor structured PICSQ demonstrated high internal consistency reliability, with a Cronbach’s alpha of 0.93 for the overall scale and 0.84-0.90 for the sub-scales.⁹

HABC-M: Although this scale was originally developed for patients with dementia, its content similar to the PICS has led to its validation for assessing PICS in ICU patients.¹⁰ The scale consists of a total of 27 items that examine the cognitive, functional, and behavioral/mood domains. The cognitive subscale consists of 6 questions related to memory, orientation, and judgment; the functional subscale consists of 11 questions related to activities of daily life; and the behavioral subscale consists of 10 questions related to depression, psychotic symptoms, and anxiety symptoms.¹⁰ Each item was graded according to the patient’s perceived frequency of the symptom within the last 2 weeks: 0= Not at all (0-1 day), 1= Several days (2-6 days), 2= More than half the days (7-11 days), 3= Almost daily (12-14 days). The maximum total score across the scale was 81, and higher scores were associated with higher symptom severity.¹⁰Psychometric properties of the scale were tested concurrently, in known group validity; and internal consistency reliability. The total scale and all subscales demonstrated good to excellent internal consistency, with Cronbach’s values ranging from 0.83 to 0.92.¹⁰

Standard scales in the three domains of PICS: The SPMSQ is a brief, 10-item cognitive screening tool with a score range of 0-10. Scoring is based on the number of incorrect responses: 0-2 incorrect responses indicate normal cognitive function and 3-4 incorrect responses indicate mild, 5-7 incorrect responses indicate moderate, ≥8 incorrect responses indicate high cognitive impairment.¹⁵ The 10-item BI is commonly used in functional disability to measure an individual’s performance in activities of daily living. The total score ranged from 0 to 100, where low scores indicate low functional status.¹⁶ The HADS was developed by Zigmond and Snaith and consists of 14 items: seven for anxiety symptoms and seven for depression symptoms. The maximum score for each subscale is 21; the cut-off is ≥8; and higher scores indicate higher levels of anxiety or depression.¹⁷

Translation of Scales

The five recommended steps were followed for the translation and cross-cultural adaptation procedures of the scales:¹⁸

1) Translation from English to Turkish: The scales were translated from English to Turkish by two independent sworn translators (T1 and T2) who are native Turkish speakers and have good knowledge of both languages.

2) Synthesis of the translations: The two first translators synthesized the translations of the scales and summarized them into a single version (T1.2).

3) Back translation: Two native and bilingual translators (BT1 and BT2) who were not familiar with the original version of the scales independently translated the T1.2 version of the scales into English.

4) Expert Committee: An expert committee of T1, BT1, two specialist physicians with at least five years of experience and research studies in the field of intensive care, one physiotherapist, one psychiatrist, one neurologist, and two nursing faculty members who have research experience in the field of intensive care and scale development/adaptation evaluated the accepted versions of the scales in terms of semantic, idiomatic, experiential, and conceptual equivalence and created the preliminary final versions of the scales in Turkish.

5) Pilot testing of the pre-final version: Finally, two groups of five (a total of 10) ICU survivors were excluded from the overall sample and invited to participate in a pilot test (face validity) of the PICSQ and HABC-M. They assessed the clarity and understanding of the items of the PICSQ and HABC-M on a four-point Likert scale (1= not clear/understandable, 2= somewhat clear/understandable, 3= quite clear/understandable, and 4= highly clear/understandable). Since all items had at least a 90% level of clarity and understandability (a score of three or higher), the final Turkish versions of the scales were created without additional revision.

A second panel of 10 experts was formed and asked to assess the content validity of the scales using item-level content validity index (I-CVI) and scale-level content validity index (S-CVI/Ave) within two weeks. Accordingly, experts rated each item on a 4-point Likert-type CVI scale according to its clearness, relevance, and important an item was.¹⁹ Panel members were also asked to provide additional comments or feedback on the sections they considered necessary. The I-CVI was calculated as the number of experts scoring 3 or 4 divided by the total number of experts (minimum acceptability ≥0.78); the S-CVI/Ave was calculated as the sum of the I-CVIs divided by the total number of items (minimum acceptability ≥0.80).¹⁹

Ethical Considerations

The research was conducted in accordance with the Declaration of Helsinki. Permissions were obtained from the Ethics Committee for Clinical Research at Bursa City Hospital (approval number: 2022-5/1, date: 06.04.2022) and the hospital. Prior to data collection, each patient or their relatives was informed about the study, and written consent was obtained. Participants were informed that they had the right to withdraw from the study at any time without any impact on their treatment or services. For the adaptation of the PICSQ, under the guidance of Dr. Jiyeon Kang, a request for permission to use the questionnaire was made via https://www.thepersoncenteredcare.org/picsq-1, and the request was approved. Permission to adapt HABC-M was obtained from Dr. Malaz A. Boustani via e-mail. Permission was also obtained via e-mail from the authors who performed the Turkish validation of the standard scales used for the PICS.

Statistical Analysis

The data were analyzed using the SPSS (Statistical Package for Social Sciences for Windows version 28.0) and AMOS (Analysis of Moment Structures version 28) software packages. Results from each scale were analyzed independently. The normal distribution was assessed using skewness/kurtosis and the Shapiro-Wilk test. Descriptive statistics were used to summarize the participants’ characteristics and outcome variables. CFA was used to test the construct validity of the scales, and the existing structure was examined with the following acceptance criteria for goodness-of-fit indices: chi-square/degree of freedom (c²/df) <5, goodness-of-fit index (GFI) >0.85, comparative fit index (CFI) >0.90, root mean square error of approximation (RMSEA) <0.08, Tucker-Lewis Index (TLI) ≥0.90, and Standardized Root Mean Residual (SRMR) ≤0.06.^{20, 21} Item factor loadings were expected to be ≥0.30.²¹ For concurrent validity, the relationship between each domain level of the PICS and the validated instruments was assessed via correlation analysis. The interpretation of correlation coefficients was as follows: negligible relationship (<0.20); low correlation, (0.20-0.40); moderate correlation (0.40-0.70); high correlation (0.70-0.90); and very high correlation (0.90-1.00).²² Cronbach’s alpha coefficient and item-total correlation (ITC) were used to measure internal consistency reliability. Cronbach’s alpha results were assessed as follows: α ≥0.9 is excellent, 0.9> α ≥0.8 is good, 0.8> α ≥0.7 is acceptable, 0.7> α ≥0.6 is questionable, 0.6> α ≥0.5 is poor, 0.5> α is unacceptable.²³ For the ITC, a value greater than 0.30 indicated that the item was correlated with the overall scale.²³ The intraclass correlation coefficient (ICC) was used to determine test-retest reliability (stability), and the acceptable value was ≥0.70.²⁴ The significance level was accepted at p<0.05.

RESULTS

Characteristics of Patients

Of the 332 patients reached during the study, 175 were excluded from the study for the following reasons: 36 patients died during the study period, 85 patients did not meet the inclusion criteria, 24 patients wanted to drop out of the study or did not respond to follow-up phone calls, and 20 patients were excluded because they were involved in the test-retest (10 patients for each scale) and face validity (5 patients for each scale) phases of the study. Therefore, the overall sample consisted of 157 ICU patients.

The mean age of the patients was 62.13±11.47 years; 66.2% were male, and more than half (65.0%) had at least a high school education. Comorbidities were present in 73.9% of patients, most of whom were medical patients (62.4%), and 22.3% developed delirium. Participants had a median APACHE II score of 16 [interquartile range (IQR): 13-20], median number of MV days of 5 (IQR: 2-11), and median ICU stay of 8 days (IQR: 2-15) (Table 1).

Validity

Content Validity

According to the expert panel, the items of both scales were highly acceptable in terms of clarity, relevance, and importance. I-CVIs were within the range of 0.80-1.00 for both PICSQ-T and HABC-M-T; S-CVIs were within 0.91-0.95 for PICSQ-T and 0.93-0.96 for HABC-M-T.

Construct Validity

CFA with maximum likelihood was performed to test whether the factor structures of the adapted scales were compatible with those of the original scales (Figure 1). As a result, the fit indices of PICSQ-T and HABC-M-T were found to be χ²/df=2,017 and 2.043, respectively; GFI=0.863 and 0.876, respectively; CFI=0.930 and 0.945, respectively; RMSEA=0.071 and 0.062, respectively; TLI=0.915 and 0.923, respectively; and SRMR=0.053 and 0.049, respectively (Table 2). As a result of CFA, the factor loadings of the PICSQ-T items ranged from 0.54 to 0.88, and for the HABC-M-T items, they ranged from 0.64 to 0.82, and the critical ratio and p-values of the items were significant (p<0.05) (Table 2).

Concurrent Validity

As shown in Table 3, all three domains in the PICSQ-T and HABC-M-T scales showed a close and significant correlation with the corresponding standard scales (p<0.05).

Reliability

Internal Reliability

Cronbach’s alpha value was between 0.852 and 0.903 for the subdimensions of the PICSQ-T and 0.945 for the total scale; it was between 0.865 and 0.933 for the subdimensions of the HABC-M-T and 0.963 for the total scale (Table 4). ITC values were between 0.530 and 0.806 for the PICSQ-T and between 0.517 and 0.824 for the HABC-M-T.

Test-Retest Reliability

In the test and retest measurements, ICCs were between 0.842 and 0.875 for the subscales of the PICSQ-T and 0.840 overall and between 0.712 and 0.877 for the subscales of the HABC-M-T and 0.892 overall (Table 4).

DISCUSSION

Despite its high prevalence and numerous adverse patient outcomes, PICS remains underrecognized, undertreated, and ignored. This is also true for Turkey, and to our knowledge, no study has been conducted on PICS in ICU patients in Turkey. One of the most important reasons for this situation is that the syndrome affects all three domains; there is a complex relationship between these domains, and different assessment tools should be used for each domain.^{3, 7} A systematic review of 18 studies investigating the characteristics of existing instruments used to measure PICS in adults reported that there were 41 different instruments in the studies, with two or more instruments used in each study.²⁵ It is clear that there is a need for clinical tools that evaluate all PICS domains together and within a short period of time. The HABC-M and PICSQ are two tools available to identify PICS in current studies.^{9, 10} In this study, we aimed to adapt both scales to Turkish and examine their psychometric properties. Our results showed that the Turkish versions of the scales were highly appropriate and acceptable properties.

CFA was performed to validate the subfactors of the scales. In the validation study of the Chinese version of the HABC-M, EFA was utilized, and factor loadings of the 19-item scale were reported to be >0.45.²⁶ In the study of the Spanish version of the HABC-M, the three-factor model structure of the scale was examined with CFA, and it was reported that the model showed a good to excellent fit with RMSE=0.073, CFI=0.99, and TLI=0.98, similar to our study.²⁷ In the original study on the development of the PICSQ, the CFA analysis for the three-factor and 18-item structure resulted in a χ²/df of 3.08, CFI of 0.90, RMSEA of 0.090, TLI of 0.90, and SRMR of 0.06.⁹ In light of these results, the model fit indices of the PICSQ-T in our study seem to be higher than those in the original study. This suggests that the translated items are better adapted to the Turkish language and culture and potentially better represent the underlying constructs.

The PICS domains of both scales showed moderate to high correlation with standardized scales, with the highest correlation found for the physical/functional domain and the lowest for the cognitive domain. This may have been due to the SPMSQ, which allowed us to assess the cognitive domain based on the data collection method used in the study. In studies reporting on PICS, the scales used to assess the cognitive domain are varied and include the MMSE, MoCa, and SPMSQ.^12-14 The SPMSQ has been reported to have high specificity but low sensitivity in identifying cognitive impairment, whereas the MoCA shows higher agreement and sensitivity than the other tests.²⁸

In the analysis of the internal consistency of the scales, Cronbach’s alpha values for the PICSQ-T and HABC-M-T were 0.94 (0.85-0.90 across subscales) and 0.96 (0.86-0.93 across subscales), respectively. In addition, the ITC values were >0.30 for both scales (lowest 0.525 for PICSQ-T and lowest 0.517 for HABC-M-T); therefore, no items were removed from the scales. In the original study for the development of the PICSQ, Cronbach’s alpha was reported to be 0.93 for the overall scale and 0.84-0.90 for the subscales, which are very similar to those in our study.⁹ In the first study for the validation of the HABC-M for PICS, its internal consistency was found to be 0.92 (0.83-0.84 for sub-dimensions).¹⁰ In the studies adapting the HABC-M to other cultures, the internal consistency was reported as 0.92 (0.82-0.92) for the Chinese version,²⁶ 0.94 (0.87-0.90) for the Spanish version,²⁷ 0.80-0.91 for sub-dimensions of the Japanese version,²⁹ and 0.79 for the overall scale in French.³⁰ These results indicate that the HABC-M-T has higher internal consistency for both the overall scale and its subdimensions than other cross-cultural adaptations. This result reflects the impact of careful language and cultural adaptation, sample homogeneity, cultural sensitivity, and high-quality translation that we used throughout the study.

To assess the stability of the scales, a test-retest was performed with a group of 10 patients for each scale at a 2-week interval. The ICC values were acceptable (>0.70) for both scales at the subscale level and overall. In the original study for the PICSQ, a test-retest was performed with a 1-week interval, and the ICC values for each factor were within the range of 0.82-0.88 (p<0.001) and 0.90 (p<0.001) for total scores.¹⁵ In the French version of the HABC-M, ICC values were reported to be higher than those reported here (0.98-0.99).³⁰ However, that study was conducted with patients in the post-intensive care follow-up clinic, and all patients were instructed to answer the questionnaire for the second time a day after the first administration. The high test-retest results in that study may have been due to the very short interval between the two tests.

Although various tools have been applied to assess the symptoms of PICS, ICU nurses and physicians may need additional training to use these tools, and the assessment process may be time-consuming. Our study showed that the Turkish versions of the HABC-M and PICSQ have significant potential as standardized, user-friendly clinical tools that enable the screening and assessment of PICS symptoms across the three domains. Therefore, we believe that PICS measurement tools can be used by ICU nurses and physicians to assess all domains quickly and easily without resorting to different assessment tools for each domain when screening for PICS. They can also be used in a wide variety of healthcare settings (e.g., primary care and outpatient care) and applied in patient follow-up via phone calls or the internet, in addition to face-to-face examinations. Rapid screening of ICU survivors for cognitive, physical, and mental impairments may help identify where they need additional support and treatment and referral them to appropriate subspecialty care. In addition, we believe that the introduction and validation of these two PICS-related scales in the Turkish language will lead to the emergence and acceleration of epidemiological studies on PICS in Turkey. Although the patients in our study were in the early period after their discharge, their mean scores on the scales indicated severe PICS (41.20±9.85 points for PICSQ-T and 60.10±14.59 points for HABC-M-T), which is remarkable and illustrative for future studies to urgently address the cognitive, physical, and mental problems of intensive care survivors in Türkiye.

Study Limitations

This study has some limitations. First, since the study was conducted on patients within 2 weeks of their discharge from the ICU, the results reflected the observations of early ICU survivors. Second, the results cannot be generalized because the study was conducted at a single center, and the sample size was 157. Third, we excluded patients with diseases that may affect cognition because self-report scales are not suitable for these patients. Fourth, we used the Pfeiffer test, which has a lower performance than the MMSE or MoCA in assessing cognitive impairment, because the data were collected via phone calls. Finally, the disadvantages of the phone survey include the inability to make an objective face-to-face assessment while collecting data about the patient; the interview may take more than 15 minutes due to the number of questionnaires; calls may be interrupted; and the interview may have to be postponed as individuals are sometimes unavailable to answer the call.

CONCLUSION

Our results showed that the Turkish versions of the HABC-M and PICSQ had high validity and reliability and could be easily used to screen for PICS. Routine adoption of these tools to screen for PICS will enable healthcare professionals to recognize PICS symptoms in patients and ensure timely referral to appropriate subspecialties and post-ICU follow-up clinics, if available.

MAIN POINTS

• This study showed that the Turkish versions of the PICSQ and HABC-M had high validity and reliability and could be easily used to screen for PICS.

• These PICS assessment tools can be used by ICU nurses and other healthcare professionals to quickly assess all domains together, without having to resort to different assessment tools for each domain when screening for PICS.

• By using these scales, ICU survivors can be rapidly screened by healthcare professionals for PICS to identify which area of PICS they need additional support and treatments and can thus be referred to appropriate subspecialty care.

Acknowledgments: The authors thank the ICU patients and their relatives who agreed to participate in this study.