Cross-Institution Generalizability of Financial Risk Models: Validation, Reliability, and Operational Governance

Abstract

Financial risk models can perform well within the institution that trained them yet fail when transferred to another, because institutions differ in customer composition, product mix, data definitions, label-confirmation processes, and operating policy, and a model's apparent accuracy can rest on regularities specific to the environment it was built in rather than on any property that travels across environments. This article develops a cross-institution validation framework for drift-resilient early-warning models used in SME loan distress and transaction fraud, building on the multi-institution validation architecture that Rashid and Sattiraju propose and on the broader concept-drift literature's treatment of non-stationary learning problems. The framework separates four questions that are often collapsed into a single reported accuracy figure: whether a model is stable over time within the institution that trained it; whether its ranking quality transfers to an institution it never trained on; whether a small amount of local, target-site data can correct its probability calibration without retraining; and whether a statistically transferable model is actually deployable given an institution's workflow, privacy constraints, and investigation capacity. It specifies evaluation designs for each question, including leave-one-institution-out and pairwise transfer testing, and proposes criteria, beyond detection speed alone, for judging unsupervised drift detectors: false-alert burden, computational cost, robustness to changing sample sizes, and demonstrated association with subsequent, outcome-confirmed performance deterioration. The framework is then applied to 375,261 public SBA-guaranteed small-business loans, with eight industry-sector portfolios standing in for institutions because the public file identifies neither lender nor approval date. Ranking transferred almost intact (leave-one-site-out AUC 0.68 to 0.79), but calibration did not: across 56 site pairs the median absolute error in the predicted charge-off rate was 0.032 and the worst was 0.117. About 1,000 local outcomes were enough for recalibration. Unsupervised detectors at default thresholds missed most shifts; re-tuned to a 5 percent false-alarm rate, D3 detected almost all of them, but flagged harmful and harmless shifts alike, so detection did not identify which transfers needed action. Because sector portfolios are not institutions and the data carry no dates, these results test cross-segment transfer but not temporal stability. Its central argument is that cross-institution generalizability is not a binary property a model either has or lacks, but a bounded set of claims that must be supported by explicit, site-level evidence across time, institutions, and operational conditions before transfer is assumed.