Today’s research on Quality of Experience (QoE) mainly addresses multimedia services, where the end-users' subjective perception is the prime factor of determining the QoE. With the introduction of the Internet of Things (IoT), there is a need for new ways of evaluating the QoE. Emerging IoT services, such as remotely-controlled operations, autonomous vehicles (AVs), and energy management, are more complex, creating additional quality requirements emerging from the machine-to-machine (M2M) communication and autonomous processes. One challenge, as an extension of the legacy QoE concept, is understanding the perception of end-users QoE in the context of IoT services. For instance, within the current state of the art in QoE it is not clear how intelligent machines can impact end-users' QoE, but also how end-users can alter or affect an intelligent machine. Another challenge is the quality evaluation of the M2M and systems that enable the machines to run by themselves. Consider a self-driving vehicle, where multiple autonomous decisions are simultaneously made as a result of predictive models that reason on the vehicle's generated data. An evaluation of the predictive models is inevitable due to abundance of the potential sources of failures. A quality degradation of the IoT hardware, the software enabling autonomous decision, and the M2M communication can raise life-threatening concerns, directly impacting the end-users' QoE. In this thesis, we argue for a paradigm shift in the QoE area that understands the relationships between humans and intelligent machines, as well as within the machines. Our contributions are as follows: first, we introduce the term Quality of IoT-experience (QoIoT) to extend the conventional QoE approaches in covering IoT services. Within QoIoT, we consider a quality evaluation from the perspective of the end-users, as well as from intelligent machines. The end-user's perception is captured by following the conventional QoE approaches, while regarding intelligent machine we propose the usage of objective metrics to describe their experiences and performance. As our second contribution, we propose a novel QoIoT architecture that consists of a layered methodology in order to determine the overall QoIoT. The QoIoT architecture, firstly, models the data-sources of an IoT service, classified within four layers: physical, network, application, and virtual. Secondly, the architecture proposes three layers for measuring the QoIoT by considering Quality of Data (QoD), Quality of Network (QoN), and Quality of Context (QoC), with QoC being the prime layer in measuring the objective performance metrics. Finally, the third contribution considers a case-study of cellular IoT, involving autonomous mining vehicles, which we utilize to achieve a preliminary results that validate the proposed QoIoT architecture.

Quality of Experience (QoE) is a research domain that measures the ”degree of de-light or annoyance of the users of an application or service”. Today’s research on QoE mainly addresses multimedia services, where the user’s subjective perception is the prime factor of determining the QoE. However, with the proliferation of the Internet of Things (IoT) in industrial services, this thesis argues for extending the conventional QoE defi-nition, architecture, and evaluation methods. Emerging IoT services, such as industrial transportation and manufacturing, are more complex, with different quality requirements than multimedia services. For instance, evaluating the machine-to-machine (M2M) com-munication that enables autonomous operations. Some IoT services closely engage with human lifestyle and privacy; hence, delivering and measuring quality is of utmost impor-tance. Consider a self-driving vehicle, making multiple real-time decisions as a result of automated predictive models. Therein, measuring QoE faces two key challenges. The first challenge is about measuring the QoE of an autonomous service. Traditional QoE is typically measured through subjective tests - an approach that doe not apply to intelli-gent machines. The second challenge is about understanding and measuring the impact of intelligent machines and their autonomous decisions. Quality degradation caused by the machines and M2M communication can affect the business, environment, and even raise life-threatening concerns.This thesis argues for a paradigm shift within the area of QoE in the direction of understanding the relationships between humans and intelligent machines, as well as among the machines. Thus, the main outcome of the thesis is the introduction of Quality of IoT experience (QoIoT), which extends the traditional QoE definition in covering IoT services. Within QoIoT, we consider a quality evaluation from the perspective of users, as well as from intelligent machines. The user’s perception is captured by following the traditional QoE models, while intelligent machines are evaluated throughput objective metrics describing their experiences and performance.As a result of the extended QoE definition, this thesis presents a QoIoT architecture consisting of a methodology and measurable parameters in emerging IoT services. The QoIoT architecture models low-level objective metrics from four layers of the IoT service: physical, network, application, and virtual. Then, the architecture argues for automated approaches in determining the quality of network (QoN) and quality of data (QoD) based on machine learning methods. In the next step of the architecture, we map the measured low-level objective metrics into high-level contextual metrics as part of a quality of context (QoC) layer. Finally, we aggregate the QoC metrics by considering the stakeholder’s requirements to form a unified metric measuring the QoIoT. The last part of the thesis validates the QoIoT architecture in a commercial industrial IoT service, involving autonomous mining vehicles. First, we devise experiments and propose methods for measuring the objective metrics within the QoN and QoD layers. Key results within the QoN layer include estimations of latency of IoT vehicular data, with the highest achieved of R2 accuracy of 90%, and cellular link throughput per vehicle, with R2 accuracy of 92%. Regarding QoD, we detect anomalous behavior within the data generated by the vehicles, including network and sensory data. The results show on average 0.90 F1 score when detecting network anomalies while driving the vehicle in the mining, rural, highway, and city environment, whereas 0.92 F1 anomaly detection score on average regarding speed, positioning, and heading directions in the same environments. Based on the mining stakeholder requirements, we then combine the QoN and QoD data with utility theory to calculate QoC metrics - driver’s perception, reliability, and safety of the vehicle’s operations; Finally, we aggregate the contextual metrics to measure the productivity of the vehicle, which is the ultimate proposed QoIoT metric for the mining use-case.