Model Drift Monitoring and Recalibration in Automated Underwriting Systems: A Five-Year Empirical Study

Abstract

Online lenders in the U.S. Midwest have increasingly relied on automated underwriting pipelines, yet model performance often degrades quietly as applicant populations and verification practices change. A five-year dataset (2019–2024) from a regional installment-loan platform covering 1.06 million applications across Illinois, Indiana, and Michigan was examined, including 71,400 charge-off events and monthly policy updates. Clear drift appeared in employer verification features and “thin-file” borrower segments after mid-2021, coinciding with changes in income documentation and credit bureau refresh frequency. A rolling-window recalibration policy was applied using a 9-month training window with quarterly threshold adjustment under fixed approval-rate constraints. Out-of-time AUC improved from 0.703 to 0.781, and the false-decline rate for borrowers with stable repayment histories dropped by 10.8% relative. Model monitoring alarms based on population stability index (PSI) reduced the average drift-detection delay from 11 weeks to 4 weeks. Practical underwriting stability turned out to be more sensitive to data pipeline changes than to classifier choice.