Scalarized Q Multi-Objective Reinforcement Learning for Area Coverage Control and Light Control Implementation

Akkachai Phuphanin, Wipawee Hattagam


Coverage control is crucial for the deployment of wireless sensor networks (WSNs). However, most coverage control schemes are based on single objective optimization such as coverage area only, which do not consider other contradicting objectives such as energy consumption, the number of working nodes, wasteful overlapping areas. This paper proposes on a Multi-Objective Optimization (MOO) coverage control called Scalarized Q Multi-Objective Reinforcement Learning (SQMORL). The two objectives are to achieve the maximize area coverage and to minimize the overlapping area to reduce energy consumption. Performance evaluation is conducted for both simulation and multi-agent lighting control testbed experiments. Simulation results show that SQMORL can obtain more efficient area coverage with fewer working nodes than other existing schemes.  The hardware testbed results show that SQMORL algorithm can find the optimal policy with good accuracy from the repeated runs.

Full Text:



H. Zhang and J. C. Hou, “Maintaining Sensing Coverage and Connectivity in Large Sensor Networks,” International Journal of Ad Hoc & Sensor Wireless Networks, Vol. 1, pp.89-124. 2015.

A. A. Kumaar, G. Kiran and T. S. B. Sudarshan, “Intelligent Lighting System Using Wireless Sensor Networks,” International Journal of Ad hoc, Sensor & Ubiquitous Computing, Vol. 1, No. 4, pp. 17-27, 2010.

T. P. Huynh, Y. K. Tan and K. J. Tseng, Energy-Aware, “Wireless Sensor Network with Ambient Intelligence for Smart LED Lighting System Control,” Annual Conference on IEEE Industrial Electronics Society, 2011.

R. Mohamaddoust, A. T. Haghighat, M. J. M. Sharif and N. Capanni, “A Novel Design of an Automatic Lighting Control System for a Wireless Sensor Network with Increased Sensor Lifetime and Reduced Sensor Numbers,” Journal of Sensors, Vol. 11, pp.8933-8952, 2015.

P. Meng-Shiuan, Y. Lun-Wu, C. Yen-Ann, L. Yu-Hsuan and T. Yu-Chee, “A WSN-Based Intelligent Light Control System Considering User Activities and Profiles,” IEEE Sensor Journal, Vol. 8, No. 10, pp.1710-1721, 2008.

M. Okada, H. Aida and H. Ichikawa, “Design and Implementation of an Energy-Efficient Lighting System Driven by Wireless Sensor Networks,” International Conference on Mobile Computing and Ubiquitous Networking, 2015.

M. Iqbal, M. Naeem, A. Anpalagan, N. N. Qadri and M. Imran, “Multi-objective optimization in sensor networks: Optimization classification, applications and solution approaches,” Journal of Computer Networks, Vol. 99, pp.134-161, 2016.

T. Ratnasingham and K. Thiagalingam, “Optimization-Based Dynamic Sensor Management for Distributed Multi-target Tracking,” IEEE Transaction on Systems, Man, and Cybernetics, Vol. 39, No. 5, pp.534-546, 2009.

M. Iqbal, M. Naeem, A. Anpalagan, A. Ahmed and M. Azam, “Wireless Sensor Network Optimization: Multi-Objective Paradigm,” Journal of Sensors, Vol. 15, No. 7, pp.17572-17620, 2015.

Z. Fei, B. Li, S. Yang, C. Xing, H. Chen and L. Hanzo, “A Survey of Multi-Objective Optimization in Wireless Sensor Networks: Metrics, Algorithms, and Open Problems,” IEEE Communications Surveys & Tutorials, Vol. 19, pp.550-586, 2016.

V. Singhiv, A. Krause, C. Guestrin, Jr. Jame and H. Matthews, “Intelligent Light Control using Sensor Networks,” Proceedings of the 3rd International Conference on Embedded Networked Sensor Systems, pp.218-229, 2005.

A. C. C. Carlos, G. T. Pulido and M. S. Lechuga, “Handling Multiple Objectives with Particle Swarm Optimization,” IEEE Transactions on Evolutionary Computation, Vol. 8, No. 3, pp.256-279, 2014.

J. Jia, J. Chen, G. Chang and Z. Tan, “Energy efficient coverage control in wireless sensor networks based on multi-objective genetic algorithm,” Journal of Computers and Mathematics with Application, Vol. 57, Issue 11-12, pp.1756-1766, 2009.

J. Barbancho, C. Leon, F. J. Molina and A. Barbancho, “Using artificial intelligence in routing schemes for wireless networks,” Journal of Computer Communications, Vol. 30, Issue 14-15, pp.2802-2811, 2007.

Z. Tafa, “Artificial Neural Networks in WSNs Design: Mobility Prediction for Barrier Coverage,” IEEE International Symposium on Signal Processing and Information Technology, 2016.

M. Rovcanin, E. D. Poorter, D. V. D. Akker, I. Moerman, P. Demeester and C. Blondia, “Experimental validation of a reinforcement learning based approach for a service-wise optimization of heterogeneous wireless sensor networks,” Journal of Wireless Networks, Vol. 21, No. 3, pp.931-948, 2015.

A. Phuphanin and W. Usaha, “A Multi-Agent Scheme for Energy-Efficient Coverage Control in Wireless Sensor Networks,” Proceedings of International Conference on Information Technology and Science, 2016.

S. M. Jameii, K. Faez and M. Dehghan, “Multi-objective Optimization for Topology and Coverage Control in Wireless Sensor Networks,” International Journal of Distributed Sensor Networks, Vol. 11, Issue 2, 2015.

K. V. Moffaert, M. M. Drugan and A. Nowe, “Scalarized Multi-Objective Reinforcement Learning: Novel Design Techniques,” IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, 2013.


  • There are currently no refbacks.

E-Journal © ECTI Asscoiation, Thailand, Contact Us.