Zhengkai Ding^1,2, Qiming Fu^1,2,*, Jianping Chen^2,3,4,*, You Lu^1,2, Hongjie Wu¹, Nengwei Fang⁴, Bin Xing⁴

CMES-Computer Modeling in Engineering & Sciences, Vol.136, No.3, pp. 2759-2785, 2023, DOI:10.32604/cmes.2023.026091

Abstract The optimization of multi-zone residential heating, ventilation, and air conditioning (HVAC) control is not an easy task due to its complex dynamic thermal model and the uncertainty of occupant-driven cooling loads. Deep reinforcement learning (DRL) methods have recently been proposed to address the HVAC control problem. However, the application of single-agent DRL for multi-zone residential HVAC control may lead to non-convergence or slow convergence. In this paper, we propose MAQMC (Multi-Agent deep Q-network for multi-zone residential HVAC Control) to address this challenge with the goal of minimizing energy consumption while maintaining occupants’ thermal comfort. MAQMC is divided into MAQMC2 (MAQMC… More >

R. Gayathri^1,*, K. Sheela Sobana Rani²

Computer Systems Science and Engineering, Vol.46, No.1, pp. 43-56, 2023, DOI:10.32604/csse.2023.032491

Abstract Classification of speech signals is a vital part of speech signal processing systems. With the advent of speech coding and synthesis, the classification of the speech signal is made accurate and faster. Conventional methods are considered inaccurate due to the uncertainty and diversity of speech signals in the case of real speech signal classification. In this paper, we use efficient speech signal classification using a series of neural network classifiers with reinforcement learning operations. Prior classification of speech signals, the study extracts the essential features from the speech signal using Cepstral Analysis. The features are extracted by converting the speech… More >

Zulqar Nain¹, B. Shahana², Shehzad Ashraf Chaudhry³, P. Viswanathan⁴, M.S. Mekala¹, Sung Won Kim^1,*

CMC-Computers, Materials & Continua, Vol.74, No.3, pp. 5705-5721, 2023, DOI:10.32604/cmc.2023.033194

Abstract Fog computing brings computational services near the network edge to meet the latency constraints of cyber-physical System (CPS) applications. Edge devices enable limited computational capacity and energy availability that hamper end user performance. We designed a novel performance measurement index to gauge a device’s resource capacity. This examination addresses the offloading mechanism issues, where the end user (EU) offloads a part of its workload to a nearby edge server (ES). Sometimes, the ES further offloads the workload to another ES or cloud server to achieve reliable performance because of limited resources (such as storage and computation). The manuscript aims to… More >

Xiaojuan Wang¹, Zikui Lu^1,*, Siyuan Sun², Jingyue Wang¹, Luona Song³, Merveille Nicolas⁴

CMC-Computers, Materials & Continua, Vol.74, No.1, pp. 2055-2071, 2023, DOI:10.32604/cmc.2023.031750

Abstract With the development of the Industrial Internet of Things (IIoT), end devices (EDs) are equipped with more functions to capture information. Therefore, a large amount of data is generated at the edge of the network and needs to be processed. However, no matter whether these computing tasks are offloaded to traditional central clusters or mobile edge computing (MEC) devices, the data is short of security and may be changed during transmission. In view of this challenge, this paper proposes a trusted task offloading optimization scheme that can offer low latency and high bandwidth services for IIoT with data security. Blockchain… More >

Chao Zhu¹, Lei Wang¹, Dai Pan¹, Zifei Wang², Tao Wang², Licheng Wang^2,*, Chengjin Ye³

CMES-Computer Modeling in Engineering & Sciences, Vol.134, No.1, pp. 599-609, 2023, DOI:10.32604/cmes.2022.021052

Abstract In this paper, a model free volt/var control (VVC) algorithm is developed by using deep reinforcement learning (DRL). We transform the VVC problem of distribution networks into the network framework of PPO algorithm, in order to avoid directly solving a large-scale nonlinear optimization problem. We select photovoltaic inverters as agents to adjust system voltage in a distribution network, taking the reactive power output of inverters as action variables. An appropriate reward function is designed to guide the interaction between photovoltaic inverters and the distribution network environment. OPENDSS is used to output system node voltage and network loss. This method realizes… More >

Eman K. Elsayed¹, Asmaa K. Elsayed^2,*, Kamal A. Eldahshan³

CMC-Computers, Materials & Continua, Vol.73, No.3, pp. 5103-5120, 2022, DOI:10.32604/cmc.2022.030803

Abstract Industry 4.0 production environments and smart manufacturing systems integrate both the physical and decision-making aspects of manufacturing operations into autonomous and decentralized systems. One of the key aspects of these systems is a production planning, specifically, Scheduling operations on the machines. To cope with this problem, this paper proposed a Deep Reinforcement Learning with an Actor-Critic algorithm (DRLAC). We model the Job-Shop Scheduling Problem (JSSP) as a Markov Decision Process (MDP), represent the state of a JSSP as simple Graph Isomorphism Networks (GIN) to extract nodes features during scheduling, and derive the policy of optimal scheduling which guides the included… More >

Yongjiang Zhao, Jae Hung Yoo, Chang Gyoon Lim^*

CMC-Computers, Materials & Continua, Vol.73, No.3, pp. 5671-5686, 2022, DOI:10.32604/cmc.2022.027443

Abstract With the rapid economic growth and improved living standards, electricity has become an indispensable energy source in our lives. Therefore, the stability of the grid power supply and the conservation of electricity is critical. The following are some of the problems facing now: 1) During the peak power consumption period, it will pose a threat to the power grid. Enhancing and improving the power distribution infrastructure requires high maintenance costs. 2) The user's electricity schedule is unreasonable due to personal behavior, which will cause a waste of electricity. Controlling load as a vital part of incentive demand response (DR) can… More >

Christy James Jose^1,*, M. S. Rajasree²

Computer Systems Science and Engineering, Vol.44, No.2, pp. 1357-1372, 2023, DOI:10.32604/csse.2023.025672

Abstract The predominant method for smart phone accessing is confined to methods directing the authentication by means of Point-of-Entry that heavily depend on physiological biometrics like, fingerprint or face. Implicit continuous authentication initiating to be loftier to conventional authentication mechanisms by continuously confirming users’ identities on continuing basis and mark the instant at which an illegitimate hacker grasps dominance of the session. However, divergent issues remain unaddressed. This research aims to investigate the power of Deep Reinforcement Learning technique to implicit continuous authentication for mobile devices using a method called, Gaussian Weighted Cauchy Kriging-based Continuous Czekanowski’s (GWCK-CC). First, a Gaussian Weighted… More >

A. Sivaranjani^1,*, B. Vinod²

Intelligent Automation & Soft Computing, Vol.35, No.1, pp. 1135-1150, 2023, DOI:10.32604/iasc.2023.028126

Abstract Autonomous navigation of mobile robots is a challenging task that requires them to travel from their initial position to their destination without collision in an environment. Reinforcement Learning methods enable a state action function in mobile robots suited to their environment. During trial-and-error interaction with its surroundings, it helps a robot to find an ideal behavior on its own. The Deep Q Network (DQN) algorithm is used in TurtleBot 3 (TB3) to achieve the goal by successfully avoiding the obstacles. But it requires a large number of training iterations. This research mainly focuses on a mobility robot’s best path prediction… More >

Sun Yang-Yang, Yao Jun-Ping^*, Li Xiao-Jun, Fan Shou-Xiang, Wang Zi-Wei

Journal on Artificial Intelligence, Vol.4, No.1, pp. 27-35, 2022, DOI:10.32604/jai.2022.027839

Abstract Deep deterministic policy gradient (DDPG) has been proved to be effective in optimizing particle swarm optimization (PSO), but whether DDPG can optimize multi-objective discrete particle swarm optimization (MODPSO) remains to be determined. The present work aims to probe into this topic. Experiments showed that the DDPG can not only quickly improve the convergence speed of MODPSO, but also overcome the problem of local optimal solution that MODPSO may suffer. The research findings are of great significance for the theoretical research and application of MODPSO. More >