Author(s)

Kelvin Wong¹, Mei-Ling Chan², Victor Leung^2*

Affiliation(s)

¹Department of Computer Science, City University of Hong Kong, Hong Kong region 999077, China.

²Department of Information Systems, City University of Hong Kong, Hong Kong region 999077, China.

Corresponding Author

Victor Leung

ABSTRACT

Modern advertising systems face increasing complexity in API workflow management due to interconnected service dependencies, dynamic resource requirements, and stringent energy efficiency constraints. Traditional workflow orchestration approaches struggle to optimize complex API execution sequences while maintaining energy consumption within operational limits. The heterogeneous nature of advertising workflows, including real-time bidding pipelines, content personalization processes, and analytics aggregation tasks, requires sophisticated control mechanisms that can adapt to varying performance requirements and energy availability.

This study proposes a Reinforcement Learning (RL)-based architecture for hierarchical control of API workflows in energy-constrained advertising systems. The framework employs a multi-tier control structure where high-level workflow coordinators manage execution strategies while low-level API controllers optimize individual service performance within energy budgets. Deep Deterministic Policy Gradient (DDPG) and Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithms enable adaptive workflow control policies that balance execution efficiency with energy consumption across distributed advertising infrastructure.

Experimental evaluation using enterprise advertising system traces demonstrates that the proposed architecture achieves 41% improvement in workflow completion rates while reducing energy consumption by 37% compared to traditional orchestration methods. The hierarchical approach successfully manages complex workflow dependencies and energy constraints, resulting in 33% better resource utilization efficiency and 29% reduction in workflow execution latency.

KEYWORDS

Reinforcement learning; API workflow management; Hierarchical control; Energy-constrained systems; Deep deterministic policy gradient; Twin delayed DDPG; Advertising systems; Workflow orchestration

CITE THIS PAPER

Kelvin Wong, Mei-Ling Chan, Victor Leung. A reinforcement learning-based architecture for hierarchical control of API workflows in energy-constrained Ad systems. Innovation and Technology Studies. 2025, 2(2): 1-8. DOI: https://doi.org/10.61784/its3016.

REFERENCES

[1] Anitha K, Anitha A, Preetha S, et al. Seamless data flow: Constructing end-to-end data pipelines for real-time marketing analytics. In Data Engineering for Data-driven Marketing, 2025: 73-90.

[2] Xing S, Wang Y. Proactive data placement in heterogeneous storage systems via predictive multi-objective reinforcement learning. IEEE Access, 2025.

[3] Maddukuri N. Workflow optimization for mobile computing. International Journal of Science and Research Archive, 2025, 14(01): 340-346.

[4] Dakic V. Machine learning and dynamic parameters-based system architecture for automatic workload scheduling in heterogenous high-performance computing. Doctoral dissertation, University of Zagreb, Faculty of Electrical Engineering and Computing, Department of Control and Computer Engineering, 2025.

[5] Wang Z. Analysis of the shared energy storage business model for building clusters in commercial pedestrian blocks. Economic Management and Practice, 2025, 3(3): 1-21. DOI: 10.61784/emp2001.

[6] Menaka M, Kumar K S. Workflow scheduling in cloud environment—Challenges, tools, limitations & methodologies: A review. Measurement: Sensors, 2022, 24: 100436.

[7] Sheta S V. A comprehensive analysis of real-time data processing architectures for high-throughput applications, 2022.

[8] Choudhary A, Govil M C, Singh G, et al. Energy-aware scientific workflow scheduling in cloud environment. Cluster Computing, 2022, 25(6): 3845-3874.

[9] Ramachandran K K. Optimizing IT performance: A comprehensive analysis of resource efficiency. International Journal of Marketing and Human Resource Management (IJMHRM), 2023, 14(3): 12-29.

[10] Peddinti S R, Pandey B K, Tanikonda A, et al. Optimizing microservice orchestration using reinforcement learning for enhanced system efficiency. Distributed Learning and Broad Applications in Scientific Research| Annual, 2021, 7.

[11] Reddy P V, Reddy K G. An energy efficient RL based workflow scheduling in cloud computing. Expert Systems with Applications, 2023, 234: 121038.

[12] Ji E, Wang Y, Xing S, et al. Hierarchical reinforcement learning for energy-efficient API traffic optimization in large-scale advertising systems. IEEE Access, 2025.

[13] Verma R, Rane D, Jaswal S. Service-oriented computing paradigms: Opportunities and challenges. Soft Computing Principles and Integration for Real-Time Service-Oriented Computing, 2024: 154-173.

[14] Barika M, Garg S, Zomaya A Y, et al. Orchestrating big data analysis workflows in the cloud: Research challenges, survey, and future directions. ACM Computing Surveys (CSUR), 2019, 52(5): 1-41.

[15] Rajesh S C, Goel L. Architecting distributed systems for real-time data processing in multi-cloud environments. International Journal of Emerging Technology and Innovative Research, 2025, 12: b623-b640.

[16] Kocot B, Czarnul P, Proficz J. Energy-aware scheduling for high-performance computing systems: A survey. Energies, 2023, 16(2): 890.

[17] Hathwar D K, Bharadwaj S R, Basha S M. Power-aware virtualization: Dynamic voltage frequency scaling insights and communication-aware request stacking. In Computational Intelligence for Green Cloud Computing and Digital Waste Management, 2024: 84-108. IGI Global Scientific Publishing.

[18] Lu Y, Chen C, Mei Y. Evaluation of the vertical synergy of science and technology financial policies from a structural-functional perspective: Based on the experience of Guangdong Province and cities. Economic Management and Practice, 2025, 3(3): 22-34. DOI: 10.61784/emp2002.

[19] Chen Y, Chan W H, Su E L M, et al. Multi-objective optimization for smart cities: A systematic review of algorithms, challenges, and future directions. PeerJ Computer Science, 2025, 11: e3042.

[20] Nguyen T T, Nguyen N D, Nahavandi S. Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications. IEEE Transactions on Cybernetics, 2020, 50(9): 3826-3839.

[21] Qadeer A, Lee M J. Deep-deterministic policy gradient based multi-resource allocation in edge-cloud system: A distributed approach. IEEE Access, 2023, 11: 20381-20398.

[22] Pateria S, Subagdja B, Tan AH, et al. Hierarchical reinforcement learning: A comprehensive survey. ACM Computing Surveys (CSUR), 2021, 54(5): 1-35.

[23] Adepoju AH, Eweje A, Collins A, et al. Automated offer creation pipelines: An innovative approach to improving publishing timelines in digital media platforms. International Journal of Multidisciplinary Research and Growth Evaluation, 2024, 5(6): 1475-1489.

[24] Buyya R, Ilager S, Arroba P. Energy-efficiency and sustainability in new generation cloud computing: a vision and directions for integrated management of data centre resources and workloads. Software: Practice and Experience, 2024, 54(1): 24-38.

[25] LuoLe Z, ZuChang Z, XiaoMin L, et al. The dual effects of a country’s overseas patent network layout on its export: scale-up or quality improvement. Social Science and Management, 2025, 2(2): 12-29. DOI: 10.61784/ssm3046.

[26] Verma P, Maurya AK, Yadav RS. A survey on energy-efficient workflow scheduling algorithms in cloud computing. Software: Practice and Experience, 2024, 54(5): 637-682.

[27] Usman M, Ferlin S, Brunstrom A, et al. A survey on observability of distributed edge & container-based microservices. IEEE Access, 2022, 10: 86904-86919.

[28] Cao J, Zheng W, Ge Y, et al. DriftShield: Autonomous fraud detection via actor-critic reinforcement learning with dynamic feature reweighting. IEEE Open Journal of the Computer Society, 2025.

[29] Araújo G, Barbosa V, Lima LN, et al. Energy consumption in microservices architectures: a systematic literature review. IEEE Access, 2024, 12: 186710-186729.

[30] Mai N, Cao W. Personalized learning and adaptive systems: AI-driven educational innovation and student outcome enhancement. International Journal of Education and Humanities, 2025.

[31] Wang M, Zhang X, Yang Y, et al. Explainable machine learning in risk management: balancing accuracy and interpretability. Journal of Financial Risk Management, 2025, 14(3): 185-198.

[32] Cao W, Mai N, Liu W. Adaptive knowledge assessment via symmetric hierarchical Bayesian neural networks with graph symmetry-aware concept dependencies. Symmetry, 2025.