Adaptive Scheduling for Evolving Workloads in Shared High-Performance Computing Centers

Abstract

Shared high-performance computing (HPC) centers support diverse research workloads with varying computational and memory demands. Fixed scheduling policies often fail to accommodate shifting usage patterns. This study analyzes job submission records from a multi-disciplinary computing center serving more than 3,000 active users. Over two years, memory-intensive workloads increased by nearly 60%, leading to frequent queue backlogs and resource fragmentation. An adaptive scheduling strategy incorporating workload classification and historical usage profiles was deployed. Queue wait times decreased for most job categories, although peak-period congestion remained difficult to eliminate. Effective scheduling requires continuous workload monitoring rather than static policy design.