Issue: 2020 > February > original article

The APOP screener and clinical outcomes in older hospitalised internal medicine patients

L.C. Blomaard, J.A. Lucke, J. de Gelder, S. Anten, J. Alsma, S.C.E. Schuit, J. Gussekloo, B. de Groot, S.P. Mooijaart
AbstractFull textPDF

Full text


Frailty, geriatric emergency medicine, internal medicine, older people, risk stratification

Older patients acutely hospitalised for complaints within the remit of internal medicine are at high risk of adverse health outcomes, with 25-35% showing functional decline during hospitalisation,1,2 which rises to 23-43% at three months, together with 10-20% mortality rates three months after acute admission.3-5 Patients with high risks of adverse outcomes require adaptations of care and extra attention to prevent further decline.6 Risk stratification during the initial stages of an acute care episode is therefore an important first step in targeting interventions and improving outcomes for individual older patients.7-9 However, the identification of patients at highest risk is challenging and therefore rarely used in practice. The Acutely Presenting Older Patient (APOP) screener is a validated instrument to predict risk for functional decline and mortality within three months for the total population of older patients presenting to the Emergency Department (ED).10,11 After arrival in the ED, patients can be screened for their individual risk of adverse outcomes in less than two minutes using the APOP screener, and APOP screening has already been implemented in routine ED care in several Dutch hospitals. However, how predicted risk for adverse outcomes based on APOP screening relates to various clinical outcomes in older patients who are acutely hospitalised for internal medicine needs to be further defined. For example, if the APOP screener can predict a long hospital length of stay (LOS) and 12-month adverse outcomes in this patient group, it could also be used to guide treatment decisions and care planning from a very early stage onwards during hospital admission. Therefore, the aim of the present study was to investigate the association between predicted risk of adverse outcomes, as assessed by the APOP screener, and clinical outcomes during hospitalisation and at 3 and 12-month follow-ups in acutely hospitalised older internal medicine patients. This information could be a first step in exploring whether routine APOP-based risk stratification can predict individual prognoses useful in tailoring clinical approaches in this vulnerable patient group.


Study design and setting
This paper describes a secondary analysis of the Acutely Presenting Older Patient (APOP) study, a prospective multicentre study which was performed in four Dutch hospitals. A detailed description has been published elsewhere.10 Briefly, consecutive older patients visiting the ED of the participating hospitals were included from September to November 2014 at Leiden University Medical Centre (LUMC); from March to June 2015 at Alrijne hospital; from May to July 2016 at Haaglanden Medical Centre (HMC, location Bronovo); and from July 2016 to January 2017 at Erasmus University Medical Center (Erasmus MC). Patients were included 24 hours a day at the LUMC; seven days a week (from 10 a.m.-10 p.m.) at Alrijne; six days a week (from 10 a.m.-10 p.m.) at HMC Bronovo; and four days a week (from 10 a.m.-10 p.m.) at Erasmus MC.

Study participants
In the APOP study, all consecutive patients aged 70 years or older visiting the ED were included. Patients who were triaged ‘red’ according to the Manchester Triage System (MTS),12 patients with an unstable medical condition, patients with an impaired mental status without a proxy to provide informed consent, patients with a language barrier and patients who refused to participate were excluded. For the purposes of the present study, we included all acutely hospitalised patients allocated to the specialism internal medicine, and with an APOP screening result at baseline. The participating hospitals had no separate geriatric departments. We excluded patients who were transferred from the ED for hospitalisation elsewhere. The Medical Ethics Committees of the four hospitals approved the study and written informed consent was obtained from all patients.

For the present study, we defined the following outcomes at hospitalisation: hospital LOS in days, in-hospital mortality, and discharge destination. Adverse outcomes assessed were functional decline and mortality, 3 months and 12 months after acute hospitalisation. The 3-month adverse outcome was met if a patient had died or showed functional decline at the 3-month follow-up compared to baseline functioning. The 12-month adverse outcome was met if a patient had died or showed functional decline at the 12-month follow-up compared to baseline functioning. Functional decline was defined as at least one-point increase in the Katz Index of Activities of Daily Living (ADL) score or new institutionalisation (higher level of assisted living).13 Patients with a maximum Katz ADL score at baseline, institutionalisation at baseline, or patients who were lost to follow-up were considered as having no functional decline.

Data collection

Patient characteristics
Three domains were assessed at baseline in the ED: demographics, disease severity, and geriatric measurements. Demographics consisted of age, sex, living arrangements, and level of education. Disease severity consisted of characteristics related to the ED visit, including arrival by ambulance, triage urgency according to MTS, chief complaint, and a fall-related ED visit. Geriatric measurements consisted of the number of different medications as stated by the patient (≥ 5 medications meaning polypharmacy), use of a walking device, Katz ADL questionnaire (functional status two weeks before the ED visit),13 the Six-item Cognitive Impairment Test (6-CIT),14 and a history of diagnosed dementia reported by the patient or a proxy.

The APOP screening result
The APOP screening instrument was developed and validated to identify older patients at risk for the composite outcome of mortality and/or functional decline within three months.11 The screener comprises seven predictors which are collected at baseline in the ED: age, sex, arrival by ambulance, need of regular help, need for help with bathing and showering, hospitalisation in the past six months and impaired cognition (defined as having dementia or an incorrect answer on at least one out of two 6-CIT questions [‘what year is it now?’ and/or ‘say the months in reverse order’] or no data on cognition). For the purposes of the present study, we retrospectively calculated the APOP screening results for all acutely hospitalised patients allocated to internal medicine, meaning that the medical staff, at the time, were unaware of the screening results during admission. Validation and threshold testing of APOP screening has been described previously.11 The threshold for a ‘high risk’ APOP screening result is a predicted risk ≥ 45% on the composite outcome of mortality and/or functional decline within three months. The final APOP screening model is calibrated to identify the approximately 20% of patients with a predicted risk ≥ 45%. Previously, we compared the APOP screener with the Identification of Seniors At Risk - Hospitalised Patients (ISAR-HP), another frequently used screening tool in the Netherlands, and found that the APOP screener demonstrated better predicting performance for this composite outcome.15

Follow-up data
The outcomes at hospitalisation including hospital LOS, in-hospital mortality, and discharge destination were collected from the electronic health records of the participating hospitals. Hospital LOS was measured by subtracting the date of admission to the hospital ward after the ED visit from the hospital discharge date. The discharge destination was compared with the patient’s former place of residence before hospital admission. We divided discharge destination into two groups: discharge to the former place of residence (either living at home or in a nursing home) or new institutionalisation at discharge. To obtain follow-up data on functional decline, patients were contacted by telephone 3 and 12 months after acute hospitalisation. In cases of no response after three attempts, the general practitioner was contacted to verify phone number and living arrangements. Finally, a letter was sent requesting a written response from those patients who could not be contacted. Data on mortality was obtained from municipal records. Patients who had not died and could not be reached at follow-up were considered as having no functional decline.

Sample size estimation
The required sample size to determine differences in 12-month mortality was calculated for the present study. Taking a difference of 20% in the mortality rate as relevant, 93 patients per group were needed to detect a difference between ‘APOP high risk’ and ‘APOP low risk’ patients with 80% power and a 5% significance level.

Data analyses
Continuous data are presented as means (standard deviation: SD) if normally distributed, and as medians (interquartile range: IQR) if skewed. Categorical data are presented as numbers (n, %). Differences in patient characteristics and outcomes between the APOP ‘high risk’ and ‘low risk’ patients were assessed using the independent samples t-test for normally-distributed data, the Mann-Whitney U test for skewed data, and the χ² test for categorical data. For categorical data, we present outcomes with 95% confidence intervals (95% CI). Differences in risks for adverse outcomes at 3 and 12 months between the APOP ‘high risk’ and ‘low risk’ patients were calculated using relative risk (RR; 95% CI). Survival was calculated by using Kaplan Meier survival curves for the population stratified by APOP screening result. We also conducted sensitivity analyses which led to the exclusion of patients with a maximum Katz ADL score at baseline, institutionalisation at baseline, and those lost to follow-up. A p-value < 0.05 was considered as statistically significant. Statistical analyses were performed using IBM SPSS Statistics version 23.

The APOP study included 2629 individual ED patients aged 70 years and older from four hospitals, of whom, 1157 (44.0%) patients were admitted to various hospital wards of the participating hospitals. A subset of 323 (27.9%) of the 1157 patients were acutely hospitalised and allocated to internal medicine. After excluding four patients due to an incomplete APOP screening result, a total of 319 patients could be included in the present study (figure 1).

Patient characteristics
Table 1 presents the patient characteristics of the study population in total and stratified per APOP screening result. In the total study population of 319 patients, the median age was 80 years (IQR 74-85), 152 (47.6%) patients were male, and 202 (63.3%) patients arrived at the ED by ambulance. Of the total study population, 29.5% (n = 94) were identified as ‘high risk’ by the APOP screener. These ‘high risk’ patients, when compared with ‘low-risk’ patients, were older (median 84 years vs. median 78 years, respectively; p < 0.001) and less likely to live independently (75.5% vs. 97.3%, respectively; p < 0.001). ‘High risk’ patients were also more likely to have had a fall-related visit (20.2% ‘high risk’ vs. 4.0% ‘low risk’; p < 0.001) and had more geriatric-related impairments, including greater use of a walking device (89.2% vs. 41.8%, respectively; p < 0.001), a higher Katz ADL score (median 3 vs. median 0, respectively; p < 0.001) and a higher 6-CIT score (median 14 vs. median 4, respectively; p < 0.001).

Outcomes at hospitalisation
The median hospital LOS for the entire study population was four days (IQR 1-8) (table 2). When stratified by APOP risk group, the ‘high risk’ group had a median hospital LOS that was two days longer than the ‘low risk’ patient group (5 (IQR 3-10) vs. 3 (IQR 1-7) days, respectively; p = 0.006). In total, 21 (6.6%) patients died during hospitalisation, with numbers similar in both groups (p = 0.381). Following hospital admission, the discharge destination was significantly different between ‘high risk’ and ‘low risk’ patients, with ‘high risk’ patients more often newly institutionalised to a nursing home compared to ‘low risk’ patients (11.6% (6.4-20.1) vs. 3.3% (1.6-6.7), respectively; p < 0.001).

Outcomes at three months
At three months, 134 (42.0%) patients had an adverse outcome, including 67 (21.0%) who had died and 67 (21.0%) who experienced functional decline compared to their level of functioning two weeks before hospitalisation. Outcomes stratified per APOP screening result are shown in figure 2. Of the 94 ‘high risk’ patients, 27 (28.7%) patients had died and an additional 29 (30.9%) patients showed functional decline within three months. Of the 225 ‘low risk’ patients, 40 (17.8%) patients had died and an additional 38 (16.9%) patients had functional decline. ‘High risk’ patients showed an adverse outcome (deceased or functional decline) more often compared to ‘low risk’ patients (59.6% (49.5-68.9) vs. 34.7% (28.8-41.1), respectively; p < 0.001). ‘High risk’ patients showed a 1.7-fold higher relative risk (95%CI 1.3-2.2) for an adverse outcome at three months compared to ‘low risk’ patients.

Outcomes at 12 months
At 12 months, a total of 167 (52.4%) patients had an adverse outcome, of whom 109 (34.2%) had died and 58 (18.2%) experienced functional decline compared to their level of functioning two weeks before hospitalisation. Of the 94 ‘high risk’ patients, 46 (48.9%) patients had died and an additional 17 (18.1%) patients showed functional decline within 12 months. Of the 225 ‘low risk’ patients, 63 (28.0%) had died and an additional 41 (18.2%) patients had functional decline. More ‘high risk’ patients had an adverse outcome compared to ‘low risk’ patients (67.0% (57.0-75.7) vs. 46.2% (39.8-52.7), respectively; p = 0.001). ‘High risk’ patients also showed a 1.5-fold higher relative risk (95%CI 1.2-1.8) for an adverse outcome at 12 months compared to ‘low risk’ patients. Supplementary figure 1 shows survival plots for 12-month mortality stratified per APOP screening result. Significantly more ‘high risk’ patients died within 12 months compared to ‘low risk’ patients (48.9% vs. 28.0%, respectively; p < 0.001).

We found similar differences between APOP ‘high risk’ and ‘low risk’ patients in the sensitivity analyses of outcomes at 3 and 12 months, from which we first excluded those patients who were lost to follow-up for the outcome functional decline and patients who by definition could not show a decline in function (supplementary table 1).


’High risk’ acutely hospitalised older patients with indications related to internal medicine had a longer hospital LOS and were more often discharged to a nursing home compared to ‘low risk’ patients. One year after admission, two-thirds of this patient group was deceased or showed a decline in function, showing an overall 1.5-fold higher risk compared to ‘low risk’ patients.

In the present study, the APOP screener was used as a risk stratification instrument to identify risk of adverse outcomes in older patients. APOP ‘high risk’ patients could be considered ‘frail’, although no consensus on the definition of frailty exists. The present study shows how the APOP screener can be used to operationalise the concept of frailty in the ED, by showing the implications of the screener for acutely hospitalised older internal medicine patients.

Over the short term, APOP ‘high risk’ patients had a 2-day longer median hospital LOS and ~4 times higher risk for new institutionalisation to a nursing home, compared to ‘low risk’ patients. These results are aligned with existing literature, in which frailty was found to be a good predictor of various short-term adverse outcomes such as hospital length of stay, in-hospital mortality, and institutionalisation.6,16,17 A recent review concerning acutely admitted general medicine patients reported that frailty was predictive of LOS in 57% of studies and of institutionalisation in 100% of studies.6 Using frailty/ risk-stratification tools at the beginning of an acute care episode may therefore have additional value because it facilitates the identification of those internal medicine patients who will be hospitalised for a longer period and are likely to be subsequently discharged to a new living environment.
At three months, around one-third of ‘high risk’ hospitalised internal medicine patients had died and almost half of the survivors exhibited functional decline. These proportions are very comparable to previous Dutch studies in this patient group.5,18 More importantly, we showed that early risk stratification at admission can also predict long-term adverse outcomes at one year. Despite the fact that the APOP screener was originally designed to predict outcomes at three months, we found that higher risks for mortality or functional decline were still statistically significant at one year; our results align with another Dutch study by Buurman et al., which also reported a significant association between one-year mortality and various geriatric conditions.19

The present study has a number of implications for clinical practice. The routine use of the APOP screener upon arrival in the ED can help to identify vulnerable patients at the very beginning of an acute episode. This risk stratification could allow better targeted assessment (i.e., comprehensive geriatric assessment) in patients who need it most and could avoid unnecessary assessment of severely frail/high-risk patients. If risk stratification is not used, care providers may be unaware of differences in frailty amongst older patients, leading to a risk of generalisation of treatment advice. On the one hand, generalisation might lead to overtreatment of frail older patients. This is especially problematic as frail patients are often underrepresented in clinical studies and thus the impact of treatment is often unclear or not focused on the outcomes of interest for these patients.20,21 On the other hand, there is also a risk of undertreatment of frail older patients. Some of the effects of hospitalisation, such as immobility resulting in functional decline, might be preventable by initiating assessments immediately during hospital admission.22 Despite the fact that it is unclear why the ‘high risk’ patients in our study had a longer LOS, the extra 2 days of hospitalisation could be used as a window of opportunity. In some hospitals, these patients could be admitted to specific geriatric departments, but if this is not possible, an internist ought to be aware of opportunities to improve patient outcomes. Perhaps the most important opportunity would be first, to use a comprehensive geriatric assessment, which has known positive effects on prevention of institutionalisation, death, and deterioration in older patients.23,24 Second, the use of advance care planning would help to establish goals and preferences for future care.25 And finally, safe transitions between care settings should be ensured, for example, by the use of transitional care.26 In addition, it is also worth considering that the interventions described above could be of benefit to patients screened as ’low risk’. An important clinical impact of the use of frailty/ risk-stratification tools is increased awareness of the risk of poor outcomes, which in turn, may help clinicians tailor approaches to the individual patient. The specific details of how clinicians can do this to improve outcomes or to prevent further decline should be addressed in future research. 

Our study has several strengths. First, an unselected group of acutely hospitalised older internal medicine patients was included from four separate Dutch hospitals. Second, although the APOP screener is not technically a frailty screening instrument, it is validated to identify adverse outcomes. As it can be used directly after patient arrival in the ED and requires only two minutes to complete, it is clearly suitable for large-scale use in clinical practice. Our study also has several limitations. First, we did not have reliable data on the medical reason or diagnosis at hospitalisation, which may have influenced the risk of adverse outcomes. However, a novel aspect of the present study was the risk stratification of patients at the very beginning of an acute care episode to predict outcomes even before the final diagnosis was clear. Second, for the present study we used the development and validation cohort of the APOP study and calculated the APOP screener retrospectively. Nevertheless, we consider the degree of selection or information bias due to the retrospective design to be minimal due to the prospective follow-up design of the study and the inclusion of all consecutive older ED patients. A retrospective design could also be considered an advantage, as clinicians were unaware of the screening results and it therefore could not have influenced course and clinic. In view of the ongoing implementation of the APOP screener in several Dutch hospitals, it would be of value to repeat these analyses in different populations in the future.

In conclusion, the APOP screener identifies acutely hospitalised internal medicine patients at high risk of short and long-term poor outcomes. Early risk stratification at admission could aid in individualising treatment decisions and therefore facilitate optimised outcomes for acutely hospitalised older patients with internal medicine-related indications.


Prior presentations
This paper formed the basis of an oral presentation given at the Dutch Internist Days (Internistendagen), Maastricht, April 24th, 2019.

The authors acknowledge the contribution of G.J. Blauw to the collaboration of the Haaglanden Medical Centre (location Bronovo) as a participating centre in the APOP study.

Conflict of interest statement
The authors declare that they have no conflicts of interest.

The Institute for Evidence-Based Medicine in Old Age (IEMO) is supported by the Dutch Ministry of Health, Welfare and Sport and by the Netherlands Organisation for Health Research and Development (ZonMw project number 62700.3001 and 62700.4001).


  1. Covinsky KE, Palmer RM, Fortinsky RH, et al. Loss of independence in activities of daily living in older adults hospitalized with medical illnesses: increased vulnerability with age. J Am Geriatr Soc. 2003;51:451-8.
  2. Inouye SK, Wagner DR, Acampora D, et al. A predictive index for functional decline in hospitalized elderly medical patients. J Gen Intern Med. 1993;8:645-52.
  3. Boyd CM, Landefeld CS, Counsell SR, et al. Recovery of activities of daily living in older adults after hospitalization for acute medical illness. J Am Geriatr Soc. 2008;56:2171-9.
  4. Buurman BM, Hoogerduijn JG, van Gemert EA, et al. Clinical characteristics and outcomes of hospitalized older patients with distinct risk profiles for functional decline: a prospective cohort study. PLoS One. 2012;7:e29621.
  5. van Seben R, Reichardt LA, Aarden JJ, et al. The Course of Geriatric Syndromes in Acutely Hospitalized Older Adults: The Hospital-ADL Study. J Am Med Dir Assoc. 2019;20:152-8.
  6. Theou O, Squires E, Mallery K, et al. What do we know about frailty in the acute care setting? A scoping review. BMC Geriatr. 2018;18:139.
  7. Carpenter CR, Shelton E, Fowler S, et al. Risk factors and screening instruments to predict adverse outcomes for undifferentiated older emergency department patients: a systematic review and meta-analysis. Acad Emerg Med. 2015;22:1-21.
  8. Jorgensen R, Brabrand M. Screening of the frail patient in the emergency department: A systematic review. Eur J Intern Med. 2017;45:71-3.
  9. Hughes JM, Freiermuth CE, Shepherd-Banigan M, et al. Emergency Department Interventions for Older Adults: A Systematic Review. J Am Geriatr Soc. 2019.
  10. de Gelder J, Lucke JA, de Groot B, et al. Predicting adverse health outcomes in older emergency department patients: the APOP study. Neth J Med. 2016;74:342-52.
  11. de Gelder J, Lucke JA, Blomaard LC, et al. Optimization of the APOP screener to predict functional decline or mortality in older emergency department patients: Cross-validation in four prospective cohorts. Exp Gerontol. 2018;110:253-9.
  12. Mackway-Jones K. Manchester Triage Group: Emergency Triage. London: BMJ Publishing Group; 1997.
  13. Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illness in the aged: The index of ADL: a standardized measure of biological and psychosocial function. Jama. 1963;185:914-9.
  14. Katzman R, Brown T, Fuld P, et al. Validation of a short OrientationMemory-Concentration Test of cognitive impairment. Am J Psychiatry. 1983;140:734-9.
  15. de Gelder J, Haenen E, Lucke JA, et al. Optimising the ISAR-HP to screen efficiently for functional decline in older patients. Neth J Med. 2017;75:379-85.
  16. Basic D, Shanley C. Frailty in an older inpatient population: using the clinical frailty scale to predict patient outcomes. J Aging Health. 2015;27:670-85.
  17. Evans SJ, Sayers M, Mitnitski A, Rockwood K. The risk of adverse outcomes in hospitalized older patients in relation to a frailty index based on a comprehensive geriatric assessment. Age Ageing. 2014;43:127-32.
  18. Buurman BM, van Munster BC, Korevaar JC, et al. Prognostication in acutely admitted older patients by nurses and physicians. J Gen Intern Med. 2008;23:1883-9.
  19. Buurman BM, Hoogerduijn JG, de Haan RJ, et al. Geriatric conditions in acutely hospitalized older patients: prevalence and one-year survival and functional decline. PLoS One. 2011;6:e26951.
  20. van Deudekom FJ, Postmus I, van der Ham DJ, et al. External validity of randomized controlled trials in older adults, a systematic review. PLoS One. 2017;12:e0174053.
  21. Meyer GS, Nelson EC, Pryor DB, et al. More quality measures versus measuring what matters: a call for balance and parsimony. BMJ Qual Saf. 2012;21:964-8.
  22. Admi H, Shadmi E, Baruch H, Zisberg A. From research to reality: minimizing the effects of hospitalization on older adults. Rambam Maimonides Med J. 2015;6:e0017.
  23. Ellis G, Whitehead MA, O’Neill D, Langhorne P, Robinson D. Comprehensive geriatric assessment for older adults admitted to hospital. Cochrane Database Syst Rev. 2011:Cd006211.
  24. Parker SG, McLeod A, McCue P, et al. New horizons in comprehensive geriatric assessment. Age Ageing. 2017;46:713-21.
  25. Piers RD, van Eechoud IJ, Van Camp S, et al. Advance Care Planning in terminally ill and frail older persons. Patient Educ Couns. 2013;90:323-9.
  26. Buurman BM, Parlevliet JL, Allore HG, et al. Comprehensive Geriatric Assessment and Transitional Care in Acutely Hospitalized Patients: The Transitional Care Bridge Randomized Clinical Trial. JAMA Intern Med. 2016;176:302-9.