QueryShift: Detecting Performance Regressions from Workload Evolution in Analytical Databases

Abstract

Database performance regressions are frequently caused not by system failures but by gradual changes in query structure, data distribution, and execution plans. This paper introduces QueryShift, a workload-aware method for detecting emerging performance regressions in analytical database systems. QueryShift constructs compact representations from query templates, execution plans, cardinality estimates, operator statistics, and historical latency distributions, and identifies deviations relative to previously stable workload periods. We evaluate the approach on 11.3 million query executions collected over 12 weeks from six analytical database clusters. The dataset contains 1,426 verified regression episodes caused by plan changes, cardinality estimation errors, resource contention, and workload shifts. QueryShift detects 91.8% of regression episodes with a 6.2% false-positive rate, outperforming latency-threshold monitoring, which achieves 73.5% recall at a comparable false-positive rate. The median detection time is 14.6 minutes after regression onset, compared with 38.2 minutes for the strongest baseline. In 68.7% of detected cases, the highest-ranked abnormal execution-plan operator corresponds to the operator identified during subsequent diagnosis. The findings show that workload evolution provides useful signals for identifying database performance problems before they become persistent service-level violations.