ADM Prediction (Made Before Running Models)
Predicted winner: Domain-led or simple models. Published geriatric risk factors provide a strong starting point for functional-decline prediction. The test is whether learned nonlinear structure adds enough held-out discrimination to justify greater complexity.
Expected: Domain captures the essential signal. ML may improve by 2–4 AUC points. Actual: Domain AUC 0.739, learned-model AUC 0.777. The gain matches the predicted range, but the stronger-baseline test shows logistic regression performs as well as the tree ensemble. The result supports simple supervised learning, not the domain score alone.
Domain knowledge — age, disease count, and activity level — already predicts functional decline well (AUC 0.739). The ML model adds 3.8 AUC points (0.777), and the hybrid is close to the ML model (0.781). The point-estimate gain is modest; no paired significance test for this exact contrast is reported here. Where do those extra points come from?
Walking difficulty is the #1 feature — a functional marker that traditional domain models often overlook. Self-rated health, depression, and hospital stays capture the subjective and episodic signals that age + disease count miss.
Before deploying a complex ML pipeline, we should ask: does logistic regression — the simplest supervised model — capture the same signal?
Logistic regression actually outperforms GradientBoosting on this problem (AUC 0.777 vs 0.773). When the signal is largely linear (age + disease count → decline), simpler models can match or beat complex ones.
With a 57% base rate, accurate probability estimates are critical for triage decisions — distinguishing “likely to decline” from “certain to decline” determines intervention urgency.
The domain score (ECE — Expected Calibration Error — 0.208) is less well calibrated in these folds than the learned model (ECE 0.025) and hybrid (0.022).
Functional decline is defined as losing the ability to perform at least one Activity of Daily Living (ADL) — bathing, dressing, eating, transferring, or toileting — within 4 years.
The domain model uses age, number of chronic diseases, and vigorous activity frequency — the three strongest epidemiological predictors.
The ML model uses gradient-boosted trees trained on all available features including walking difficulty, self-rated health, depression scores, BMI, and hospitalization history.
The hybrid model uses the domain risk score as a feature alongside the full feature set, allowing it to build on rather than replace domain knowledge.
ADM Insight
This is the closest call in the study — and the most instructive. Published frailty criteria (age, disease count, activity level) already predict functional decline well (AUC 0.739). The ML adds 3.8 AUC points. Adding the domain score as a feature to the ML (the hybrid) doesn’t move AUC further — the learned feature set already contains that signal. The stronger-baseline comparison is the key fidelity result: logistic regression matches or slightly exceeds GradientBoosting. The analysis therefore supports adding the measured features, but not the more complex tree ensemble. Whether the 3.8-point AUC difference over the three-variable score changes a real decision remains untested. Sometimes the right fidelity is the simplest model that captures the essential signal. A hospital system would need a prospective utility and cost analysis before deciding whether the incremental discrimination changes care.
High decline rate: 57% of the 50+ cohort experienced functional decline over 30 years. This high base rate makes prediction easier and AUC differences smaller than in rare-event prediction.
ADL definition sensitivity: Functional decline is defined as losing ≥1 ADL (bathing, dressing, eating, transferring, toileting). Different thresholds (≥2 ADLs, IADL-based — Instrumental Activities of Daily Living) might show different domain-vs-ML gaps.
4-year prediction window: Results are specific to 4-year decline prediction. The study does not establish whether shorter or longer windows would change the model ranking.
No cost-effectiveness analysis: The 3.8-point AUC difference is presented without a decision threshold, paired utility analysis, or implementation-cost context.
Self-reported outcomes: ADL limitations in HRS (Health and Retirement Study) are self-reported, subject to reporting bias, cultural differences, and mood effects.