Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Applications of Unsupervised Deep Learning for Analysing Time-Varying Power Quality Big Data
Luleå University of Technology, Department of Engineering Sciences and Mathematics, Energy Science.ORCID iD: 0000-0001-5845-5620
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Continuous power quality monitoring allows grid stakeholders to obtain information about the performance of the network and costumer facilities. Moreover, the analysis of continuous monitoring allows researchers to obtain knowledge on power quality phenomena. Power quality measurements result in a large amount of data. Power quality data can be classified as big data, not only for its volume, but also for the other complexities: velocity, variety, and veracity. Manual analysis of power quality data is possible but time-consuming. Moreover, data reports based on standardized indexes and classical statistical techniques might hide important information of the time-varying behaviour in power quality measurements. Artificial intelligence plays a role in providing automatic tools for proper analytics of big data.  A subset of artificial intelligence called machine learning has enabled computers to learn without explicit programming. Driven by the huge improvements in computer processing, a subset of machine learning based on multiple layers of artificial neural networks has been developed to tackle increasingly complex problems. The so-called deep learning applications teach themselves to perform a specific task by automatically extracting essential features from the raw data. Despite the possibilities of automatic feature extraction, most applications of deep learning to power quality are still the same as expert systems or earliest machine learning tools. Moreover, most of the applications are based on synthetic generated data and supervised techniques. In this context, the main motivation of this thesis is providing a new tool based on unsupervised deep learning to handle analytics of time-varying power quality big data. 

The unsupervised deep learning method proposed in this thesis combines a deep autoencoder with clustering for extracting patterns and anomalies in power quality big data. The deep autoencoder maps the original data to a compressed format that contains the principal features of the data. Automatic results are provided by the deep learning, and inferences can be obtained without requiring prior knowledge of deep learning. The outputs from unsupervised deep learning can serve as a guide for further data analysis, highlighting important time steps within large power quality datasets. By following these indications from the deep learning results, experts gain valuable insights into power quality phenomena, which can be referred to as "learning from deep learning". The interpretation of the deep learning results in this thesis allowed to making proper inferences for patterns and anomalies. For power quality measurements synchronised with 24-h, the results allowed making inferences concerning daily variations, seasonality, and the origins of power quality disturbances. For power quality measurements non-synchronised with 24-h, the results could be interpreted visually through the distribution of the patterns in a physical variable, such as the dynamic operating conditions of an electrical railway power system.

An important contribution of this thesis concerns the physical interpretation of the phenomena is related to the anomalies in harmonics caused by geomagnetically induced currents. An interesting finding by applying the deep anomaly detection to measurements in the Swedish transmission grid is the damping of the anomalies caused by geomagnetically induced currents in the winter due to the heating load. This thesis also demonstrated that the signatures for anomalies in harmonic measurements in a Swedish transmission location are similar to the ones found in a low-latitude transmission location at the South Atlantic Anomaly due to geomagnetically induced currents. Moreover, by cross-checking the anomalies at the South Atlantic Anomaly with protection trips with undetermined causes, this thesis demonstrated that anomaly harmonics due to geomagnetically induced currents can cause protection mal trips.

This thesis demonstrates that unsupervised deep learning can serve as an additional tool for compressing time-varying power quality big data into a more interpretable form. Despite the application of an unsupervised method, power quality experts remain significant in power quality studies. The main conclusion of this thesis is that unsupervised deep learning enhances the understanding of power quality experts and provides a complementary approach for analysing and extracting insights from time-varying power quality big data.

Place, publisher, year, edition, pages
Luleå: Luleå University of Technology, 2023.
Series
Doctoral thesis / Luleå University of Technology 1 jan 1997 → …, ISSN 1402-1544
Keywords [en]
power quality, power system harmonics, electric power distribution, interharmonics, big data analytics, pattern analysis, unsupervised learning, deep learning, artificial intelligence, geomagnetically induced currents
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Electric Power Engineering
Identifiers
URN: urn:nbn:se:ltu:diva-100718ISBN: 978-91-8048-353-7 (print)ISBN: 978-91-8048-354-4 (electronic)OAI: oai:DiVA.org:ltu-100718DiVA, id: diva2:1791302
Public defence
2023-10-04, Hörsal A, Luleå tekniska universitet, Skellefteå, 09:00 (English)
Opponent
Supervisors
Available from: 2023-08-25 Created: 2023-08-24 Last updated: 2023-09-13Bibliographically approved
List of papers
1. Deep learning for power quality
Open this publication in new window or tab >>Deep learning for power quality
2023 (English)In: Electric power systems research, ISSN 0378-7796, E-ISSN 1873-2046, Vol. 214, article id 108887Article, review/survey (Refereed) Published
Abstract [en]

This paper aims to introduce deep learning to the power quality community by reviewing the latest applications and discussing the open challenges of this technology. Publications covering deep learning to power quality are stratified in terms of application, type of data, and learning technique. This work shows that the majority of the deep learning applications to power quality are based on unrealistic synthetic data and supervised learning without proper labelling. Some applications with deep learning have already been solved by previous machine learning methods or expert systems. The main barriers to implementing deep learning to power quality are related to lack of novelty, low transparency of the deep learning methods, and lack of benchmark databases. This work also discusses that even with automatic feature extraction by deep learning methods, power quality expert knowledge is still needed to implement and analyse the results. The main research gaps identified in this work are related to the applications of semi-supervised learning, explainable deep learning and hybrid approaches combining deep learning with expert systems. Suggestions for overcoming the present limitations are: providing a stronger collaboration among the grid stakeholders and academy to keep track of power quality events; proper labelling and enlarging of datasets in deep learning methods; explaining the end-to-end decision making of deep learning methods; providing open-access databases for comparison purposes.

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Artificial intelligence, Data analysis, Deep learning, Machine learning, Power quality
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering Computer Sciences
Research subject
Electric Power Engineering
Identifiers
urn:nbn:se:ltu:diva-93726 (URN)10.1016/j.epsr.2022.108887 (DOI)001024999300001 ()2-s2.0-85139840780 (Scopus ID)
Funder
Swedish Energy Agency
Note

Validerad;2022;Nivå 2;2022-10-26 (hanlid)

Available from: 2022-10-26 Created: 2022-10-26 Last updated: 2024-03-07Bibliographically approved
2. Deep Feature Clustering for Seeking Patterns in Daily Harmonic Variations
Open this publication in new window or tab >>Deep Feature Clustering for Seeking Patterns in Daily Harmonic Variations
2021 (English)In: IEEE Transactions on Instrumentation and Measurement, ISSN 0018-9456, E-ISSN 1557-9662, Vol. 70, article id 2501110Article in journal (Refereed) Published
Abstract [en]

This article proposes a novel scheme for analyzing power system measurement data. The main question that we seek answers in this study is on “whether one can find some important patterns that are hidden in the large data of power system measurements such as variational data.” The proposed scheme uses an unsupervised deep feature learning approach by first employing a deep autoencoder (DAE) followed by feature clustering. An analysis is performed by examining the patterns of clusters and reconstructing the representative data sequence for the clustering centers. The scheme is illustrated by applying it to the daily variations of harmonic voltage distortion in a low-voltage network. The main contributions of the article include: 1) providing a new unsupervised deep feature learning approach for seeking possible underlying patterns of power system variation measurements and 2) proposing an effective empirical analysis approach for understanding the measurements through examining the underlying feature clusters and the associated reconstructed data by DAE.

Place, publisher, year, edition, pages
IEEE, 2021
Keywords
Autoencoder, clustering, deep learning, pattern analysis, power quality, power system harmonics, unsupervised learning
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Electric Power Engineering
Identifiers
urn:nbn:se:ltu:diva-80564 (URN)10.1109/TIM.2020.3016408 (DOI)000591842200029 ()2-s2.0-85096659899 (Scopus ID)
Funder
Swedish Energy Agency
Note

Validerad;2021;Nivå 2;2021-01-04 (alebob)

Available from: 2020-08-26 Created: 2020-08-26 Last updated: 2023-08-24Bibliographically approved
3. Visualizing The Results From Unsupervised Deep Learning For The Analysis Of Power-Quality Data
Open this publication in new window or tab >>Visualizing The Results From Unsupervised Deep Learning For The Analysis Of Power-Quality Data
2021 (English)In: Cired 2021 - The 26Th International Conference And Exhibition On Electricity Distribution, 2021, p. 653-657, article id 0030Conference paper, Published paper (Refereed)
Abstract [en]

This paper presents a visualisation method, based on deep learning, to assist power engineers in the analysis of large amounts of power-quality data. The method assists in extracting and understanding daily, weekly and seasonal variations in harmonic voltage. Measurements from 10 kV and 0.4 kV in a Swedish distribution network are applied to the deep learning method to obtain daily harmonic patterns and their distribution over the week and the year. The results are presented in graphs that allow interpretation of the results without having to understand the mathematical details of the method. The inferences given by the results demonstrate that the method can become a new tool that compresses power quality big data in a form that is easier to interpret.

Keywords
Power-Quality, Power-Quality Monitoring, Power-System Harmonics, Machine Learning, Deep Learning
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Electric Power Engineering
Identifiers
urn:nbn:se:ltu:diva-92473 (URN)10.1049/icp.2021.1771 (DOI)
Conference
26th International Conference and Exhibition on Electricity Distribution (CIRED 2021), Online, September 20-23, 2021
Funder
Swedish Energy Agency
Note

ISBN för värdpublikation: 978-1-83953-591-8 (elektroniskt)

Available from: 2022-08-15 Created: 2022-08-15 Last updated: 2023-08-24Bibliographically approved
4. Unsupervised deep learning and analysis of harmonic variation patterns using big data from multiple locations
Open this publication in new window or tab >>Unsupervised deep learning and analysis of harmonic variation patterns using big data from multiple locations
2021 (English)In: Electric power systems research, ISSN 0378-7796, E-ISSN 1873-2046, Vol. 194, article id 107042Article in journal (Refereed) Published
Abstract [en]

This paper addresses the issue of automatically seeking and identifying daily, weekly and seasonal patterns in harmonic voltage from measurement data at multiple locations. We propose a novel framework that employs deep autoencoder (DAE) followed by k-mean clustering. The DAE is used for extracting principal features from time series of harmonic voltages. A new strategy is used for training the encoder in DAE from data at one selected location that is effective for subsequent feature extraction from data at multiple locations. To analyze the patterns, several empirical analysis approaches are applied on the clustered principal features, including the distribution of daily patterns over the week and the year, representative waveform sequences of individual classes, and feature maps for visualizing high-dimensional feature space through low-dimensional embedding. The proposed scheme has been tested on a dataset containing harmonic measurements at 10 low-voltage locations in Sweden for the whole year of 2017. Results show distinct principal patterns for most harmonics that can be related to the use of equipment causing harmonic distortion. This information can assist network operators in finding the origin of harmonic distortion and deciding about mitigation actions. The proposed scheme is the first to provide a useful analysis tool and insight for finding and analyzing underlying patterns from harmonic variation data at multiple locations.

Place, publisher, year, edition, pages
Elsevier, 2021
Keywords
Electric Power distribution, Power quality, Power system harmonics, Variation data, Big data analytics, Pattern analysis, Unsupervised deep learning, Autoencoder, Clustering
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Electric Power Engineering
Identifiers
urn:nbn:se:ltu:diva-82711 (URN)10.1016/j.epsr.2021.107042 (DOI)000632386100011 ()2-s2.0-85100014673 (Scopus ID)
Funder
Swedish Energy Agency
Note

Validerad;2021;Nivå 2;2021-01-29 (alebob)

Available from: 2021-01-29 Created: 2021-01-29 Last updated: 2023-08-24Bibliographically approved
5. Deep Learning Method With Manual Post-Processing for Identification of Spectral Patterns of Waveform Distortion in PV Installations
Open this publication in new window or tab >>Deep Learning Method With Manual Post-Processing for Identification of Spectral Patterns of Waveform Distortion in PV Installations
2021 (English)In: IEEE Transactions on Smart Grid, ISSN 1949-3053, E-ISSN 1949-3061, Vol. 12, no 6, p. 5444-5456Article in journal (Refereed) Published
Abstract [en]

This paper proposes a deep learning (DL) method for the identification of spectral patterns of timevarying waveform distortion in photovoltaic (PV) installations. The PQ big data with information on harmonic and/or interharmonics in PV installations is handled by a deep autoencoder followed by feature clustering. Measurements of voltage and current from four distinct PV installations are used to illustrate the method. This paper shows that the DL method can be used as a starting point for further data analysis. The main contributions of the paper include: (a) providing a novel DL method for finding patterns in spectra; (b) guiding the manual post-processing based on the patterns found by the DL method; and (c) obtaining information about the emission from four PV installations.

Place, publisher, year, edition, pages
IEEE, 2021
Keywords
power quality, power system harmonics, electric power distribution, interharmonics, pattern analysis, unsupervised learning, deep learning, solar power
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Electric Power Engineering
Identifiers
urn:nbn:se:ltu:diva-83447 (URN)10.1109/TSG.2021.3107908 (DOI)000709090100078 ()2-s2.0-85114608265 (Scopus ID)
Funder
Swedish Energy Agency, 245 110
Note

Validerad;2021;Nivå 2;2021-11-08 (johcin)

Available from: 2021-03-30 Created: 2021-03-30 Last updated: 2023-09-05Bibliographically approved
6. Analytics of Waveform Distortion Variations in Railway Pantograph Measurements by Deep Learning
Open this publication in new window or tab >>Analytics of Waveform Distortion Variations in Railway Pantograph Measurements by Deep Learning
2022 (English)In: IEEE Transactions on Instrumentation and Measurement, ISSN 0018-9456, E-ISSN 1557-9662, Vol. 71, article id 2516211Article in journal (Refereed) Published
Abstract [en]

Waveform distortion in general represent a widespread problem in electrified transports due to interference, service disruption, increased losses and ageing of components. Given the multitude of moving sources and the extremely variable operating conditions, short time records must be considered for analysis, and this increases in turn its complexity, from which the need for effective automated processing, as offered by a deep learning (DL) approach. This paper proposes an application of unsupervised DL to measurements of railway pantograph quantities to identify waveform distortion patterns. Data consists of pantograph current from a Swiss 15 kV 16.7 Hz railway system. Three DL input types are considered: waveforms, harmonic spectra, and supraharmonic spectra. The applied DL method applied is the deep autoencoder (DAE) followed by feature clustering, using techniques to define a suitable number of clusters. Short-term distortion is evaluated over sub-10 min intervals of harmonic and supraharmonic spectra down to sub-second intervals. Results are explained among others by connecting the distribution of the clusters (determined by self-supervised method) to the dynamic operating conditions of the rolling stock. Resulting DAE performance are superior in terms of accuracy and comprehensiveness of spectral components compared to a more traditional principal component analysis (PCA) that was chosen as reference for comparison.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2022
Keywords
Autoencoder, clustering, deep learning (DL), pattern analysis, power quality (PQ), power system harmonics, unsupervised learning
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Electric Power Engineering
Identifiers
urn:nbn:se:ltu:diva-92467 (URN)10.1109/tim.2022.3197801 (DOI)000844142300008 ()2-s2.0-85136013724 (Scopus ID)
Funder
Swedish Transport AdministrationSwedish Energy Agency
Note

Validerad;2022;Nivå 2;2022-09-26 (hanlid)

Available from: 2022-08-15 Created: 2022-08-15 Last updated: 2023-09-05Bibliographically approved
7. Deep Anomaly Detection of Voltage Waveform Distortion in the Transmission Grid due to Geomagnetically Induced Currents
Open this publication in new window or tab >>Deep Anomaly Detection of Voltage Waveform Distortion in the Transmission Grid due to Geomagnetically Induced Currents
2024 (English)In: IEEE Transactions on Instrumentation and Measurements, ISSN 0018-9456, Vol. 73, article id 2509212Article in journal (Refereed) Published
Place, publisher, year, edition, pages
IEEE, 2024
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Electric Power Engineering
Identifiers
urn:nbn:se:ltu:diva-100715 (URN)10.1109/TIM.2024.3366271 (DOI)2-s2.0-85185388685 (Scopus ID)
Note

Validerad;2024;Nivå 2;2024-04-09 (joosat);

Available from: 2023-08-24 Created: 2023-08-24 Last updated: 2024-04-09Bibliographically approved
8. Deep Learning for Power Quality with Special Reference to Unsupervised Learning
Open this publication in new window or tab >>Deep Learning for Power Quality with Special Reference to Unsupervised Learning
2023 (English)In: 27th International Conference on Electricity Distribution (CIRED 2023), IEEE, 2023, p. 935-939, article id 10417Conference paper, Published paper (Refereed)
Place, publisher, year, edition, pages
IEEE, 2023
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering Computer Sciences
Research subject
Electric Power Engineering
Identifiers
urn:nbn:se:ltu:diva-100717 (URN)10.1049/icp.2023.0593 (DOI)2-s2.0-85181538517 (Scopus ID)
Conference
27th International Conference and Exhibition on Electricity Distribution (CIRED 2023), Rome, Italy, June 12-15, 2023
Note

ISBN for host publication: 978-1-83953-855-1;

Available from: 2023-08-24 Created: 2023-08-24 Last updated: 2024-03-11Bibliographically approved
9. Harmonic Anomalies Due to Geomagnetically Induced Currents as a Potential Cause of Protection Mal-Trips at the South Atlantic Anomaly Area
Open this publication in new window or tab >>Harmonic Anomalies Due to Geomagnetically Induced Currents as a Potential Cause of Protection Mal-Trips at the South Atlantic Anomaly Area
Show others...
2024 (English)In: IEEE Transactions on Power Delivery, ISSN 0885-8977, E-ISSN 1937-4208, Vol. 39, no 2, p. 1124-1136Article in journal (Refereed) Published
Place, publisher, year, edition, pages
IEEE, 2024
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Electric Power Engineering
Identifiers
urn:nbn:se:ltu:diva-100716 (URN)10.1109/TPWRD.2024.3353487 (DOI)2-s2.0-85182951668 (Scopus ID)
Note

Validerad;2024;Nivå 2;2024-04-03 (joosat);

Funder: Swedish Energy Agency and Rönnbäret Foundation (24549);

This article has previously appeared as a manuscript in a thesis.

Available from: 2023-08-24 Created: 2023-08-24 Last updated: 2024-04-03Bibliographically approved

Open Access in DiVA

fulltext(21377 kB)422 downloads
File information
File name FULLTEXT01.pdfFile size 21377 kBChecksum SHA-512
b014101297be2ca77dd5f5408088308a33e4e878ecd71a8785c75b5109a5dd201d1c7b6579f8b2c6a06994b9c6a0efd86577095f002060b7f2a06db35fabfc7d
Type fulltextMimetype application/pdf

Authority records

de Oliveira, Roger Alves

Search in DiVA

By author/editor
de Oliveira, Roger Alves
By organisation
Energy Science
Other Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 423 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 2134 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf