Adopting Drift-Aware Machine Learning for Real-Time Fraud Detection in Financial Institutions: A Simulation-Based Pre-Deployment Evaluation

Abstract

Financial fraud models degrade as fraud tactics, customer behaviour and fraud prevalence change, and confirmed labels arrive only after investigation. Financial institutions considering drift-aware models must decide before deployment whether such a policy will maintain detection, how it will affect alert workload, and how often it will change the model under governance — but pre-deployment evidence over realistic time horizons is limited. Objective. To provide a pre-deployment evaluation, for financial institutions, of a drift-aware monitoring layer implementing the temporal-drift framework of Zuberi et al. [1], and to derive a governed adoption roadmap. Methods. A 52-week stream of 520,000 transactions with sudden (card testing), gradual (behavioural) and recurring (account takeover and seasonal) drift and a two-week label delay was simulated to represent conditions a financial institution faces. A gradient-boosting scoring model was maintained by five strategies: static, periodic (four-weekly), sliding-window (weekly), performance-triggered and drift-aware monitoring. The drift-aware layer applied PSI, Kolmogorov–Smirnov and Jensen–Shannon tests to SHAP-selected features, a prevalence-shift z-test and F2 threshold tuning, plus a diagnosis step separating real from virtual drift. Strategies were evaluated prequentially over 5 replications. Results. The static model fell to an average precision of 0.289. Drift-aware monitoring reached 0.643 ± 0.012 and a recall of 0.653 at a 1% alert budget, 98.9% of the weekly sliding window (0.650), while retraining 4.6 instead of 47 times (90% fewer). It outperformed performance-triggered (0.628) and periodic (0.611) retraining. The prevalence test triggered timely retraining in four of five replications, and the diagnosis step suppressed retraining on harmless seasonal drift. Distributional tests, however, did not detect a card-testing campaign confined to the fraud minority. Scoring latency remained at most 11 µs per transaction. Conclusions. For financial institutions, the results support adopting drift-aware retraining in which supervised and prevalence monitoring decide when to retrain and unsupervised drift tests are used for diagnosis. Because the evidence is simulation-based, adoption should proceed through an offline backtest on the institution’s own data, shadow mode and a pilot portfolio before production; a roadmap with readiness requirements, rollout phases and key risk indicators is provided.