Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
A Review of Reinforcement Learning Applications to Control of Heating, Ventilation and Air Conditioning Systems
Department of Electrical Engineering and Automation, School of Electrical Engineering, Aalto University, FI-00076 Espoo, Finland.ORCID iD: 0000-0002-0402-315X
Department of Electrical Engineering and Automation, School of Electrical Engineering, Aalto University, FI-00076 Espoo, Finland.ORCID iD: 0000-0002-8135-0859
Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering, Computer Science. Department of Electrical Engineering and Automation, School of Electrical Engineering, Aalto University, FI-00076 Espoo, Finland; International Research Laboratory of Computer Technologies, ITMO University, 197101 St. Petersburg, Russia.ORCID iD: 0000-0002-9315-9920
2022 (English)In: Energies, E-ISSN 1996-1073, Vol. 15, no 10, article id 3526Article, review/survey (Refereed) Published
Abstract [en]

Reinforcement learning has emerged as a potentially disruptive technology for control and optimization of HVAC systems. A reinforcement learning agent takes actions, which can be direct HVAC actuator commands or setpoints for control loops in building automation systems. The actions are taken to optimize one or more targets, such as indoor air quality, energy consumption and energy cost. The agent receives feedback from the HVAC systems to quantify how well these targets have been achieved. The feedback is captured by a reward function designed by the developer of the reinforcement learning agent. A few reviews have focused on the reward aspect of reinforcement learning applications for HVAC. However, there is a lack of reviews that assess how the actions of the reinforcement learning agent have been formulated, and how this impacts the possibilities to achieve various optimization targets in single zone or multi-zone buildings. The aim of this review is to identify the action formulations in the literature and to assess how the choice of formulation impacts the level of abstraction at which the HVAC systems are considered. Our methodology involves a search string in the Web of Science database and a list of selection criteria applied to each article in the search results. For each selected article, a three-tier categorization of the selected articles has been performed. Firstly, the applicability of the approach to buildings with one or more zones is considered. Secondly, the articles are categorized by the type of action taken by the agent, such as a binary, discrete or continuous action. Thirdly, the articles are categorized by the aspects of the indoor environment being controlled, namely temperature, humidity or air quality. The main result of the review is this three-tier categorization that reveals the community’s emphasis on specific HVAC applications, as well as the readiness to interface the reinforcement learning solutions to HVAC systems. The article concludes with a discussion of trends in the field as well as challenges that require further research. 

Place, publisher, year, edition, pages
MDPI, 2022. Vol. 15, no 10, article id 3526
Keywords [en]
reinforcement learning, machine learning, heating, ventilation, air conditioning, building energy simulator, indoor environment, artificial intelligence, thermal comfort
National Category
Information Systems Other Electrical Engineering, Electronic Engineering, Information Engineering Control Engineering
Research subject
Dependable Communication and Computation Systems
Identifiers
URN: urn:nbn:se:ltu:diva-91637DOI: 10.3390/en15103526ISI: 000803393000001Scopus ID: 2-s2.0-85130584683OAI: oai:DiVA.org:ltu-91637DiVA, id: diva2:1672706
Note

Validerad;2022;Nivå 2;2022-06-20 (joosat);

Funder: Business Finland (7439/31/2018)

Available from: 2022-06-20 Created: 2022-06-20 Last updated: 2023-08-28Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Vyatkin, Valeriy

Search in DiVA

By author/editor
Sierla, SeppoIhasalo, HeikkiVyatkin, Valeriy
By organisation
Computer Science
In the same journal
Energies
Information SystemsOther Electrical Engineering, Electronic Engineering, Information EngineeringControl Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 99 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf