Dependency-Aware Hierarchical Reinforcement Learning for Resource Allocation in Hybrid Cloud Microservices

Abstract

Hybrid cloud platforms allow enterprises to combine private infrastructure with elastic public-cloud capacity, but they make microservice resource allocation a coupled control problem across heterogeneous machines, service dependencies, workload bursts, and cost constraints. This paper presents HADR, a dependency-aware hierarchical reinforcement-learning method for resource allocation in hybrid cloud microservice systems. HADR models a deployed application as a dynamic service-resource graph whose vertices represent microservices, pods, virtual machines, queues, and storage components, and whose edges encode invocation, contention, placement, and scaling relations. A graph-temporal encoder predicts near-term demand and dependency pressure, while a two-level reinforcement-learning controller separates slow placement decisions from fast replica and resource-quota adjustments. The reward function jointly optimizes p95 latency, service-level objective violations, cloud cost, utilization balance, and migration overhead. A scenario-based evaluation over burst, diurnal, degradation, and dependency-shift workloads shows that HADR reduces operating cost by 36.2%, lowers p95 latency by 24.7%, and improves mean resource utilization to 0.82 compared with static provisioning, threshold autoscaling, and flat deep-Q baselines. The results demonstrate that dependency-aware hierarchical control provides a robust foundation for cost-efficient and performance-stable resource management in hybrid cloud microservices.