Change search
Link to record
Permanent link

Direct link
Publications (7 of 7) Show all publications
Kim, J. C., Saguna, S. & Åhlund, C. (2024). The Effects of Augmented Reality Companion on User Engagement in Energy Management Mobile App. Applied Sciences, 14(7), Article ID 2621.
Open this publication in new window or tab >>The Effects of Augmented Reality Companion on User Engagement in Energy Management Mobile App
2024 (English)In: Applied Sciences, E-ISSN 2076-3417, Vol. 14, no 7, article id 2621Article in journal (Refereed) Published
Abstract [en]

As the impact of global warming on climate change becomes noticeable, the importance of energy efficiency for reducing greenhouse gas emissions grows immense. To this end, a platform, solution, and mobile apps are developed as part of the European Union’s Horizon 2020 research and innovation program to support energy optimization in residences. However, to ensure long-term energy optimization, it is crucial to keep users engaged with the apps. Since augmented reality (AR) and a virtual animal companion positively influenced user engagement, we designed an AR companion that represented the user’s residence states; thereby making the user aware of indoor information. We conducted user evaluations to determine the effect of the AR companion on user engagement and perceived usability in the context of energy management. We identified that the user interface (UI) with AR (ARUI) barely affected user engagement and perceived usability compared to the traditional UI without AR (TUI); however, we found that the ARUI positively affected one of the user engagement aspects. Our results show AR companion integration’s potential benefits and effects on energy management mobile apps. Furthermore, our findings provide insights into UI design elements for developers considering multiple interaction modalities with AR.

Keywords
user interface, user evaluation, user engagement, perceived usability, augmented reality, Internet of Things, energy management
National Category
Human Computer Interaction
Research subject
Pervasive Mobile Computing
Identifiers
urn:nbn:se:ltu:diva-104850 (URN)10.3390/app14072671 (DOI)
Funder
EU, Horizon 2020, 893079
Note

Validerad;2024;Nivå 2;2024-03-22 (signyg);

Full text license: CC BY

Available from: 2024-03-22 Created: 2024-03-22 Last updated: 2024-03-22Bibliographically approved
Kim, J. C., Saguna, S. & Åhlund, C. (2023). Acceptability of a Health Care App With 3 User Interfaces for Older Adults and Their Caregivers: Design and Evaluation Study. JMIR Human Factors, 10, Article ID e42145.
Open this publication in new window or tab >>Acceptability of a Health Care App With 3 User Interfaces for Older Adults and Their Caregivers: Design and Evaluation Study
2023 (English)In: JMIR Human Factors, E-ISSN 2292-9495, Vol. 10, article id e42145Article in journal (Refereed) Published
Abstract [en]

Background: The older population needs solutions for independent living and reducing the burden on caregivers while maintaining the quality and dignity of life.

Objective: The aim of this study was to design, develop, and evaluate an older adult health care app that supports trained caregivers (ie, formal caregivers) and relatives (ie, informal caregivers). We aimed to identify the factors that affect user acceptance of interfaces depending on the user’s role.

Methods: We designed and developed an app with 3 user interfaces that enable remote sensing of an older adult’s daily activities and behaviors. We conducted user evaluations (N=25) with older adults and their formal and informal caregivers to obtain an overall impression of the health care monitoring app in terms of user experience and usability. In our design study, the participants had firsthand experience with our app, followed by a questionnaire and individual interview to express their opinions on the app. Through the interview, we also identified their views on each user interface and interaction modality to identify the relationship between the user’s role and their acceptance of a particular interface. The questionnaire answers were statistically analyzed, and we coded the interview answers based on keywords related to a participant’s experience, for example, ease of use and usefulness.

Results: We obtained overall positive results in the user evaluation of our app regarding key aspects such as efficiency, perspicuity, dependability, stimulation, and novelty, with an average between 1.74 (SD 1.02) and 2.18 (SD 0.93) on a scale of −3.0 to 3.0. The overall impression of our app was favorable, and we identified that “simple” and “intuitive” were the main factors affecting older adults’ and caregivers’ preference for the user interface and interaction modality. We also identified a positive user acceptance of the use of augmented reality by 91% (10/11) of the older adults to share information with their formal and informal caregivers.

Conclusions: To address the need for a study to evaluate the user experience and user acceptance by older adults as well as both formal and informal caregivers regarding the user interfaces with multimodal interaction in the context of health monitoring, we designed, developed, and conducted user evaluations with the target user groups. Our results through this design study show important implications for designing future health monitoring apps with multiple interaction modalities and intuitive user interfaces in the older adult health care domain.

Place, publisher, year, edition, pages
JMIR Publications, 2023
Keywords
Internet of Things, health monitoring, older adults, augmented reality, user experience, independent living, design study, mobile phone
National Category
Human Computer Interaction
Research subject
Pervasive Mobile Computing
Identifiers
urn:nbn:se:ltu:diva-95822 (URN)10.2196/42145 (DOI)001017203700025 ()36884275 (PubMedID)2-s2.0-85149873927 (Scopus ID)
Note

Validerad;2023;Nivå 2;2023-08-10 (joosat);

Funder: Swedish Governmental Agency for Innovation Systems (grant 2017-02807)

Licens fulltext: CC BY License

Available from: 2023-03-09 Created: 2023-03-09 Last updated: 2023-09-05Bibliographically approved
Paananen, S., Kim, J. C., Kirjavainen, E., Kalving, M., Mitra, K. & Häkkilä, J. (2023). Augmenting Indigenous Sámi Exhibition - Interactive Digital Heritage in Museum Context. In: José Abdelnour Nocera, Marta Kristín Lárusdóttir, Helen Petrie, Antonio Piccinno, Marco Winckler (Ed.), Human-Computer Interaction – INTERACT 2023: . Paper presented at 19th IFIP TC13 International Conference, York, UK, August 28 – September 1, 2023 (pp. 597-617). Springer, Part II
Open this publication in new window or tab >>Augmenting Indigenous Sámi Exhibition - Interactive Digital Heritage in Museum Context
Show others...
2023 (English)In: Human-Computer Interaction – INTERACT 2023 / [ed] José Abdelnour Nocera, Marta Kristín Lárusdóttir, Helen Petrie, Antonio Piccinno, Marco Winckler, Springer, 2023, Vol. Part II, p. 597-617Conference paper, Published paper (Refereed)
Abstract [en]

Museums and cultural heritage institutions have an important role in presenting accurate information and sharing cultural knowledge, and new technologies are increasingly implemented. For the best results, the appropriateness of a specific technology must be evaluated for each context. Research has shown the need for participatory methods and local knowledge in Indigenous design contexts. We describe a case study where an Indigenous Sámi museum exhibition was augmented with interactive technology through multidisciplinary co-design work with museum experts, designers, and developers. The traditional clothing of the Sámi people was digitized by filming, and information related to it was presented as a touchscreen installation in a renewed exhibition. User tests including interactive tasks and interviews (n = 7) and a questionnaire (n = 27) were completed on-site. The installation was rated interesting and easy to use, while some users struggled to find all the features. Our study shows that a technically relatively simple digital installation can be easy-to-use but interesting for a standard museum visitor. Additionally, the work demonstrates how to build successful collaborations that highlight Indigenous cultural heritage. We discuss the implications of using technology to promote cultural heritage and identities.

Place, publisher, year, edition, pages
Springer, 2023
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 14143
Keywords
Interactive systems, User studies, Digital cultural heritage, Museum exhibitions, Indigenous HCI
National Category
Human Computer Interaction Cultural Studies
Research subject
Pervasive Mobile Computing
Identifiers
urn:nbn:se:ltu:diva-101078 (URN)10.1007/978-3-031-42283-6_32 (DOI)978-3-031-42282-9 (ISBN)978-3-031-42283-6 (ISBN)
Conference
19th IFIP TC13 International Conference, York, UK, August 28 – September 1, 2023
Available from: 2023-08-29 Created: 2023-08-29 Last updated: 2023-09-05Bibliographically approved
Kim, J. C., Saguna, S., Åhlund, C. & Mitra, K. (2021). Augmented Reality-Assisted Healthcare System for Caregivers in Smart Regions. In: 2021 IEEE International Smart Cities Conference (ISC2): . Paper presented at IEEE International Smart Cities Conference (ISC2 2021), Manchester, United Kingdom, September 7-10, 2021. IEEE
Open this publication in new window or tab >>Augmented Reality-Assisted Healthcare System for Caregivers in Smart Regions
2021 (English)In: 2021 IEEE International Smart Cities Conference (ISC2), IEEE, 2021Conference paper, Published paper (Refereed)
Abstract [en]

The rise in the aging population worldwide is already negatively impacting healthcare systems due to the lack of resources. It is envisioned that the development of novel Internet of Things (IoT)-enabled smart city healthcare systems may not only alleviate the stress on the current healthcare systems but may significantly improve the overall quality of life of the elderly. As more elderly homes are fitted with IoT, and intelligent healthcare becomes the norm, there is a need to develop innovative augmented reality (AR) based applications and services that make it easier for caregivers to interact with such systems and assist the elderly on a daily basis. This paper proposes, develops, and validates an AR and IoT-enabled healthcare system to be used by caregivers. The proposed system is based on a smart city IoT middleware platform and provides a standardized, intuitive and non-intrusive way to deliver elderly person's information to caregivers. We present our prototype, and our experimental results show the efficiency of our system in IoT object detection and relevant information retrieval tasks. The average execution time, including object detection, communicating with a server, and rendering the results in the application, takes on average between 767ms and 1,283ms.

Place, publisher, year, edition, pages
IEEE, 2021
Keywords
Internet of Things, augmented reality, healthcare, human computer interaction, smart city
National Category
Computer Sciences
Research subject
Pervasive Mobile Computing
Identifiers
urn:nbn:se:ltu:diva-87577 (URN)10.1109/ISC253183.2021.9562927 (DOI)000853860800060 ()2-s2.0-85118102826 (Scopus ID)
Conference
IEEE International Smart Cities Conference (ISC2 2021), Manchester, United Kingdom, September 7-10, 2021
Funder
Vinnova, 2017-02807
Note

ISBN för värdpublikation:978-1-6654-4919-9;978-1-6654-4920-5 

Available from: 2021-10-21 Created: 2021-10-21 Last updated: 2023-09-05Bibliographically approved
Kim, J. C., Laine, T. H. & Åhlund, C. (2021). Multimodal Interaction Systems Based on Internet of Things and Augmented Reality: A Systematic Literature Review. Applied Sciences, 11(4), Article ID 1738.
Open this publication in new window or tab >>Multimodal Interaction Systems Based on Internet of Things and Augmented Reality: A Systematic Literature Review
2021 (English)In: Applied Sciences, E-ISSN 2076-3417, Vol. 11, no 4, article id 1738Article, review/survey (Refereed) Published
Abstract [en]

Technology developments have expanded the diversity of interaction modalities that can be used by an agent (either a human or machine) to interact with a computer system. This expansion has created the need for more natural and user-friendly interfaces in order to achieve effective user experience and usability. More than one modality can be provided to an agent for interaction with a system to accomplish this goal, which is referred to as a multimodal interaction (MI) system. The Internet of Things (IoT) and augmented reality (AR) are popular technologies that allow interaction systems to combine the real-world context of the agent and immersive AR content. However, although MI systems have been extensively studied, there are only several studies that reviewed MI systems that used IoT and AR. Therefore, this paper presents an in-depth review of studies that proposed various MI systems utilizing IoT and AR. A total of 23 studies were identified and analyzed through a rigorous systematic literature review protocol. The results of our analysis of MI system architectures, the relationship between system components, input/output interaction modalities, and open research challenges are presented and discussed to summarize the findings and identify future research and development avenues for researchers and MI developers.

Place, publisher, year, edition, pages
MDPI, 2021
Keywords
multimodal interaction, interaction modalities, mixed/augmented reality, internet of things, systematic literature review, multimodal UI
National Category
Human Computer Interaction
Research subject
Pervasive Mobile Computing
Identifiers
urn:nbn:se:ltu:diva-83646 (URN)10.3390/app11041738 (DOI)000632121600001 ()2-s2.0-85101409553 (Scopus ID)
Funder
European Regional Development Fund (ERDF), 20201434, NYPS 20204318
Note

Validerad;2021;Nivå 2;2021-04-14 (alebob);

Finansiär: Ajou University (S-2020-G0001-00478)

Available from: 2021-04-14 Created: 2021-04-14 Last updated: 2023-09-05Bibliographically approved
Kim, J. C. (2020). Multimodal Interaction with Internet of Things and Augmented Reality: Foundations, Systems and Challenges. Luleå: Luleå University of Technology
Open this publication in new window or tab >>Multimodal Interaction with Internet of Things and Augmented Reality: Foundations, Systems and Challenges
2020 (English)Report (Other academic)
Abstract [en]

The development of technology has enabled diverse modalities that can be used by humans or machines to interact with computer systems. In particular, the Internet of Things (IoT) and Augmented Reality (AR) are explored in this report due to the new modalities offered by these two innovations which could be used to build multimodal interaction systems. Researchers have utilized multiple modalities in interaction systems for providing better usability. However, the employment of multiple modalities introduces some challenges that need to be considered in the development of multimodal interaction systems to achieve high usability. In order to identify a number of remaining challenges in the research area of multimodal interaction systems with IoT and AR, we analyzed a body of literature on multimodal interaction systems from the perspectives of system architecture, input and output modalities, data processing methodology and use cases. The identified challenges are regarding of (i) multidisciplinary knowledge, (ii) reusability, scalability and security of multimodal interaction system architecture, (iii) usability of multimodal interaction interface, (iv) adaptivity of multimodal interface design, (v) limitation of current technology, and (vi) advent of new modalities. We are expecting that the findings of this report and future research can be used to nurture the multimodal interaction system research area, which is still in its infancy.

Place, publisher, year, edition, pages
Luleå: Luleå University of Technology, 2020. p. 33
Series
Technical report / Luleå University of Technology, ISSN 1402-1536
National Category
Media and Communication Technology
Research subject
Pervasive Mobile Computing
Identifiers
urn:nbn:se:ltu:diva-78116 (URN)978-91-7790-562-2 (ISBN)
Available from: 2020-03-20 Created: 2020-03-20 Last updated: 2023-09-05Bibliographically approved
Kim, J. C., Lindberg, R. S. N., Laine, T. H., Faarinen, E.-C., De Troyer, O. & Nygren, E. (2019). Multidisciplinary Development Process of a Story-based Mobile Augmented Reality Game for Learning Math. In: František Jakab (Ed.), Proceedings on the 17th International Conference on Emerging eLearning Technologies and Applications: PROCEEDINGS. Paper presented at 17th International Conference on Emerging eLearning Technologies and Applications, November 21-22, 2019, Starý Smokovec, Slovakia (pp. 372-377). IEEE
Open this publication in new window or tab >>Multidisciplinary Development Process of a Story-based Mobile Augmented Reality Game for Learning Math
Show others...
2019 (English)In: Proceedings on the 17th International Conference on Emerging eLearning Technologies and Applications: PROCEEDINGS / [ed] František Jakab, IEEE, 2019, p. 372-377Conference paper, Oral presentation with published abstract (Refereed)
Abstract [en]

Despite the high number of educational games released, only a few games have a strong story that is more than an excuse for players’ actions. Furthermore, even fewer story-based games utilise the affordances of augmented reality (AR) to concretise abstract concepts while engaging players.Based on our literature review, we were inspired to merge AR into a story-based educational mobile game for teaching fractions to elementary school students. The game Tales & Fractions was created through a two-phase multidisciplinary development process. In order to successfully integrate AR into a story-based educational game, we employed an adapted version of the Scrum agile software development method implemented by a multidisciplinary team of experts from computer science, pedagogy, design and arts. During the development process, we faced many issues that other story-based AR game developers could meet. We summarised the encountered issues with our solutions which could be useful for developers to avoid common pitfalls and to enrich the user engagement.

Place, publisher, year, edition, pages
IEEE, 2019
Series
International Conference on Emerging eLearning Technologies and Applications (ICETA)
Keywords
Multidisciplinary, Storytelling, Educational game, Augmented Reality, Game-based learning, Mathematics
National Category
Media and Communication Technology Didactics
Research subject
Pervasive Mobile Computing; Mathematics Education
Identifiers
urn:nbn:se:ltu:diva-78156 (URN)10.1109/ICETA48886.2019.9039962 (DOI)2-s2.0-85082988012 (Scopus ID)
Conference
17th International Conference on Emerging eLearning Technologies and Applications, November 21-22, 2019, Starý Smokovec, Slovakia
Note

ISBN för värdpublikation: 978-1-7281-4967-7, 978-1-7281-4968-4

Available from: 2020-03-24 Created: 2020-03-24 Last updated: 2023-09-05Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-1536-5753

Search in DiVA

Show all publications