Personalized Adjustment Method of Intelligent Lamp Based on Deep Reinforcement Learning

doi:10.3778/j.issn.1002-8331.2009-0505

Abstract

Abstract: This paper proposes a deep reinforcement learning-based method for adjusting personalized smart lamp brightness. It considers the influence of both natural light and the user’s position on his/her actual visual brightness and sets the light intensity dynamically to meet the user’s personalized habits. After each automatic adjustment, according to whether the user takes further changes manually, positive or negative feedback will be collected to train the reinforcement learning model gradually fits the user’s usage habits. The experiment implements three algorithms of DQN, DDQN, and A3C, respectively, and presents the comparative analysis on the data set generated on the DIALux environment. The hardware and software implementation of the prototype system is also introduced.

Key words: deep reinforcement learning, personalized control, deep Q-network（DQN）, double DQN（DDQN）, asynchronous advantage actor-critic（A3C）

摘要： 提出一种基于深度强化学习的智能灯亮度个性化调节方法，综合考虑自然光亮度及用户位置对用户实际感受亮度的影响，动态计算并设置灯光亮度，以满足用户个性化使用习惯。在每次完成灯光亮度自动调节后，根据用户是否再次进行手动调节设定正、负反馈，训练强化学习模型逐渐拟合用户使用习惯。实验分别实现了DQN、DDQN和A3C三种算法，在基于DIALux环境产生的数据集上进行对比分析，并给出原型系统的软硬件实现。

关键词: 深度强化学习, 个性化控制, DQN算法, DDQN算法, A3C算法

DENG Xin, NA Jun, ZHANG Handuo, WANG Yulin, ZHANG Bin. Personalized Adjustment Method of Intelligent Lamp Based on Deep Reinforcement Learning[J]. Computer Engineering and Applications, 2022, 58(6): 264-270.

邓心, 那俊, 张瀚铎, 王昱林, 张斌. 基于深度强化学习的智能灯个性化调节方法[J]. 计算机工程与应用, 2022, 58(6): 264-270.

References

[1] FRIEDEMANN M，FLOERKEMEIER C.From the Internet of computers to the Internet of things[C]//From Active Data Management to Event-Based Systems and More.Berlin，Heidelberg：Springer，2010：242-259.
[2] AL HORR Y，ARIF M，KAUSHIK A，et al.Occupant productivity and office indoor environment quality：a review of the literature[J].Building and Environment，2016，105（8）：369-389.
[3] TANG S，KALAVALLY V，NG K Y，et al.Development of a prototype smart home intelligent lighting control architecture using sensors onboard a mobile computing system[J].Energy & Buildings，2017，138：368-376.
[4] PANDHARIPANDE A，NEWSHAM G R.Lighting controls：evolution and revolution[J].Lighting Research & Technology，2018，50（1）：115-128.
[5] KANDASAMY N K，KARUNAGARAN G，SPANOS C，et al.Smart lighting system using ANN-IMC for personalized lighting control and daylight harvesting[J].Building and Environment，2018，139（7）：170-180.
[6] CHRAIBI S，LASHINA T，SHRUBSOLE P，et al.Satisfying light conditions：a field study on perception of consensus light in Dutch open office environments[J].Building and Environment，2016，105：116-127.
[7] NAGY Z，YONG F Y，SCHLUETER A.Occupant centered lighting control：a user study on balancing comfort，acceptance，and energy consumption[J].Energy and Buildings，2016，126（8）：310-322.
[8] NAGY Z，YONG F Y，FREI M，et al.Occupant centered lighting control for comfort and energy efficient building operation[J].Energy and Buildings，2015，94（5）：100-108.
[9] WEN Y J，AGOGINO A M.Personalized dynamic design of networked lighting for energy-efficiency in open-plan offices[J].Energy and Buildings，2011，43（8）：1919-1924.
[10] CHENG Z，ZHAO Q，WANG F，et al.Satisfaction based Q-learning for integrated lighting and blind control[J].Energy and Buildings，2016，127（9）：43-55.
[11] AGNE P T，NERIJUS M，AUDRONE J，et al.The usage of artificial neural networks for intelligent lighting control based on resident’s behavioural pattern[J].Elektronika ir Elektrotechnika，2015，21（2）：72-79.
[12] SUTTON R S，BARTO A G.Introduction to reinforcement learning[M].Cambridge：MIT Press，1998.
[13] WATKINS C J C H，DAYAN P.Q-learning[J].Machine Learning，1992，8（3/4）：279-292.
[14] SILVER D，HUANG A，MADDISON C J，et al.Mastering the game of Go with deep neural networks and tree search[J].Nature，2016，529（7587）：484.
[15] MNIH V，KAVUKCUOGLU K，SILVER D，et al.Playing atari with deep reinforcement learning[J].arXiv：1312.5602，
2013.
[16] MNIH V，KAVUKCUOGLU K，SILVER D，et al.Human-level control through deep reinforcement learning[J].Nature，2015，518（7540）：529.
[17] VAN HASSELT H，GUEZ A，SILVER D.Deep reinforcement learning with double q-learning[J].arXiv：1509.06461，2015.
[18] MNIH V，BADIA A P，MIRZA M，et al.Asynchronous methodsfor deepreinforcement learning[C]//International Conference on Machine Learning，2016：1928-1937.
[19] LI X，YANG W，ZHANG Z.A unified framework for regularized reinforcement learning[J].arXiv：1903.00725，2019.