Usability Study of a Virtual Walking System in The Metaverse For Managing Frailty

Frailty is an age-associated biological syndrome characterized by a decrease in the biological functional reserve and resistance to stressors that results from alterations in multiple physiological systems. This condition predisposes individuals to increased vulnerability to minor stressors, ultimately leading to adverse outcomes such as disability, loss of independence and hospitalization [1-4]. According to Fried et al. [5], the prevalence of frailty in individuals aged >65 years is 6.9%, with an increasing trend observed with advancing age. In addition, several recent studies examining the prevalence of frailty in middle-aged cohorts (age 35-64 years) have reported prevalence ranges from 2.0% to 11.0% [6-9]. These findings suggest that frailty can also occur in middle age. Frailty has been identified as a significant risk factor for falls, disability, hospitalization and mortality. The diagnostic criteria for frailty encompass five domains: Unintentional weight loss, self-reported exhaustion, slow walking speed, weak grip strength and low physical activity levels [5]. To be classified as frail, an individual must exhibit positive characteristics in three or more of these domains [5]. Frailty is considered a state with a high risk of requiring care, but one that is also potentially reversible with appropriate intervention [10,11]. Indeed, a meta-analysis focusing on older adults with an average follow-up of 3.9 years reported that the state of frailty worsened in 29.1% of cases, improved in 13.7% and remained unchanged in 56.5% [12]. Measures to prevent and improve frailty include regular physical activity, strengthening of social support networks, adequate nutrition and provision of medical care, as needed [13]. Among these, exercise intervention suggests the need for appropriate exercise guidance and preventive measures before frailty becomes severe, especially in the early stages.

As a means of managing frailty, we have developed a virtual walking system in the metaverse that utilizes olfactory and auditory stimuli to motivate physical activity and enhance walking ability. A meta-analysis by Lee et al. [14] reported that interactive Virtual Reality (VR) training programs significantly improve walking speed and balance in older people with frailty. Olfactory and auditory cues in a VR environment tend to enhance high levels of immersion [15,16]. The system we designed was aimed at increasing walking distance by enhancing immersion through diffusing the fragrance of a video and/or virtual flowers and by generating sounds such as bird songs and babbling streams when approached by users in the virtual environment. In the pursuit of the practical application of this system, efforts have been focused on ensuring its functionality, improving its operational quality and enhancing its stability. However, to validate the system specifications and meet user requirements seamlessly, a usability survey of end users is necessary during the development phase.

Usability is defined as the extent to which a product can be used by specified users to achieve specified goals with efficiency [17]. That is, usability refers to the ease of use and effectiveness of a system, particularly from the perspective of the end user. In the context of our virtual walking system in the metaverse, usability encompasses how easily and effectively users can navigate and interact with the system to achieve their goals, such as increasing walking distance and improving physical abilities. Conducting a usability study is essential for several reasons. First, it allows us to gather feedback directly from the target audience, which helps us understand their preferences, needs and challenges in using the system. This feedback is crucial for refining the system’s design and functionality to meet user requirements and expectations. Second, it helps us identify any usability issues or barriers that users may encounter while using the system. By addressing these issues early in the development phase, we can prevent potential usability problems from affecting the system’s overall effectiveness and user satisfaction. Furthermore, using a standardized tool such as the System Usability Scale (SUS) to assess user satisfaction in a specified context enables us to measure the system’s usability quantitatively and compare it with industry benchmarks [18-20]. Given this background, the present study aimed to evaluate the usability of a virtual walking system in the metaverse for frailty management in end users aged  40 years.

This usability study, utilizing questionnaires, was approved by the Medical Ethics Committee of the National Center for Geriatrics and Gerontology (No. 1765) and conformed to the provisions of the Declaration of Helsinki (as revised in Brazil, 2013). All participants were informed about the study through an explanatory document approved by the committee and their consent to participate in this study was obtained through their responses to the questionnaire.

Participants

This study included community-dwelling residents aged 40 years who observed and experienced a demonstration of the virtual walking system in the metaverse.

Instruments

Virtual park: Figure 1 shows the configuration of the metaverse. Each Personal Computer (PC) was equipped with a set of headsets and an olfactory display (Aroma Shooter2; Aromajoin, Kyoto, Japan), which were linked via a universal serial bus. Wearable devices in the form of waist and ankle sensors (KAT locoS; KATVR JAPAN, Tokyo, Japan) established a connection with the PC through a Bluetooth receiver. Through these sensors, users guided avatars in the metaverse, communicating with the receiver via Bluetooth. The PC established a network link with a server housing a virtual park, depicted in (Figure 2). From the server, the PC retrieved the virtual park to navigate predetermined pathways, offering various routes. At branching points, as depicted in (Figure 3), users selected either left or right to determine their path (further elucidation on route selection mechanisms are presented in subsequent sections of this paper).

Figure 1:System configuration of the metaverse.

Figure 2:Map of the virtual park.

Figure 3:Example of branching.

Wearable sensors: The wearable sensor system for walking support comprised sensors worn at the waist and ankles, as depicted in (Figure 4). By wearing ankle sensors, users could walk through the virtual park by performing a forward (stepping) motion. The waist sensor (Figure 4) enabled users to adjust their orientation in the virtual park. At branching points along the virtual park’s pathways, users could determine their direction by manipulating the waist sensor to select either left or right. Furthermore, Computer-Generated (CG) zones were spatially represented in the virtual park (Figure 5), providing users with the ability to enter and move through them freely.

Olfactory display: In the virtual park, areas designated as olfactory zones featured virtual flowers (referred to as “scent spaces”, defined by Huang et al. [21] as spherical regions centered around the flower corolla, conceptualized as a spectrum where fragrance becomes detectable, as outlined in [22]). As users navigated through these scent spaces, the olfactory display shown in Figure 4 cyclically activated and deactivated the release of flower scents.

Walking support: In the pursuit of enriching immersion in the metaverse, movement speed in the virtual park dynamically adjusted according to the walking pace. The metaverse delineated into video spaces (depicted in Figure 3) and CG spaces (illustrated in Figure 5). In the video spaces, users were confined to forward and backward motion to promote a concentrated walking experience. These spaces incorporated video recordings captured by a 360 camera, which provided users with panoramic views while traversing the space.

Figure 4:Waist and ankle sensors and olfactory display.

Figure 5:Example of Computer-Generated (CG) buildings: a) a building; b) an avatar in a building.

In the CG spaces, users enjoyed unrestricted movement in all directions (i.e., forward, backward, left and right). Furthermore, in these CG spaces, users manifested as avatars (Figure 5b), which facilitated interpersonal interaction and dialogue through headphones. Encouraging users to progress toward CG spaces for social interaction was considered to foster motivation for walking. In the metaverse, blooming flowers emitted fragrances from the olfactory display upon a user’s approach. In video spaces, both video and CG flowers bloomed (as depicted in Figure 6 and Figure 7), whereas only CG flowers bloomed in CG spaces (as depicted in Figure 8). Considering the well-documented enhancement of cerebral blood flow in response to olfactory stimuli, this feature is poised to enhance the effectiveness of rehabilitation efforts. In addition, ambient sounds such as bird songs and flowing streams emanated from the headphones during movement. Leveraging olfactory and auditory cues to heighten presence during walking was anticipated to extend the duration of the walking sessions. The selection of emission locations for fragrances and auditory stimuli could be randomized from predefined sites.

Outcome measurements

The participants were instructed to complete the SUS and a customized questionnaire based on the Technology Acceptance Model (TAM) immediately after interacting with the virtual walking system. This intervention was conducted in three locations in Aichi Prefecture: Obu, Toyota and Toyoake. To start, the participants walked through the virtual park’s video space, as illustrated in Figure 6, for approximately 6 minutes to experience the scent of cherry blossoms and the sound of bird songs. Subsequently, they explored the CG space depicted in Figure 8 for about 4 minutes to experience the scent of lavender and engage in conversations with other users (see Figure 2 for the locations). Next, they completed the questionaries. For simplicity, an operator used a mouse to select each location instead of the participants, enabling them to move to the selected location, as shown in Figure 3. According to the findings of a systematic review investigating trends in usability study methods for digital health technologies, the SUS has been identified as the most commonly used standardized questionnaire for assessing the usability of digital health technologies among older adults [20]. In addition, the review reported that several customized questionnaires have been developed based on the TAM [20].

Figure 6:Example of video spaces with cherry blossoms.

Figure 7:Example of Computer-Generated (CG) flowers (jasmine) in the video space.

Figure 8:System configuration of the metaverse.

The SUS is a brief 10-item Likert-type questionnaire, with each item offering five response options on a scale from “1: fully disagree” to “5: fully agree” and includes questions that yield findings pertinent to understanding usability [18]. The odd-numbered items convey a positive tone, whereas the even-numbered items convey a negative tone [18]. The SUS generates an overall score ranging from 0 to 100 in 2.5-point increments [18]. The SUS has demonstrated high reliability (approximately 0.90) and its validity and sensitivity in discerning usability disparities, interface types, task accomplishment, usability and business success indicators have been established [23]. In addition, the global average score for the overall SUS is 68.0±12.5 [23]. The grading scale is as follows: F, SUS score <60; D, 60≤ score <70; C, 70≤ score <80; B, 80≤ score <90 and A, score ≥90 [24]. This grading scale is structured in a way that considers a SUS score of 68 as the center of the range for the “C” grade [25].

The TAM, initially formulated by Davis [25] in 1989, is a widely recognized theoretical framework in the field of technology adoption and user acceptance that aims to clarify the factors that influence an individual’s decision to accept and use a new technology or information system. Its core components are its perceived usefulness and perceived ease of use, which directly influence a user’s intention to use a technology, subsequently affecting their actual usage behavior [25]. The TAM has served as a foundation for developing questionnaires that assess user perceptions of new technologies, specifically measuring perceived usefulness and ease of use, which help predict technology adoption. We prepared a customized questionnaire based on the TAM (Table 1). Among the question items, Q1-Q12, Q14 and Q15 were structured as a choice between “Yes” or “No,” whereas Q13 and Q16-Q19 were openended, allowing respondents to provide free-form answers.

Table 1:Customized questionnaire based on the Technology Acceptance Model (TAM).

Q1–Q12, Q14 and Q15 were presented in a binary format, whereas Q13 and Q16-Q19 were open-ended.

Statistical analysis

Age and gender distribution were analyzed using percentages. The total SUS score and the average scores for each subitem, which were revised, were analyzed using descriptive statistics, including means and Standard Deviations (SDs). The customized questionnaire items Q1-Q12, Q14 and Q15 were answered in a binary format and thus, the number of participants and response ratios were analyzed. In addition, Q13 and Q16-Q19 of the openended survey questions were qualitatively analyzed using the Grounded Theory Approach with MAXQDA 2022 Standard (VERBI GmbH, Berlin, Germany). Participants with missing responses in the SUS or customized questionnaire were excluded from the analysis.

All 42 participants completed all items of the SUS and customized questionnaires. Among the participants, 19 were male (45.2%) and 23 were female (54.8%). Five participants were in their 40s (11.9%), 12 in their 50s (28.6%), 11 in their 60s (26.2%), 10 in their 70s (23.8%) and four in their 80s or above (9.5%). The mean SUS score was 69.3±15.3 (range, 30-100), with the corrected mean scores for each subitem shown in Figure 9. The results of the customized questionnaire items Q1-Q12, Q14 and Q15 are presented in Table 2. The results of a qualitative analysis of open-ended questions Q13 and Q16-Q19 revealed the following. The participants’ responses to Q13, “Who would you like to see in the virtual walking system in the metaverse?”, were as follows: 14 answered “parents”; 4 each answered “myself”, “grandparents” and “older adults”; 2 answered “frail individuals; and 1 each answered “friends”, “husband and wife”, “grandchild” and “child”.

Figure 9:Corrected mean scores for each subitem of the System Usability Scale.

Q1: I think that I would like to use this system frequently; Q2: I found the system unnecessarily complex; Q3: I think that the system was easy to use; Q4: I thought that I would need the support of a technical person to be able to use this system; Q5: I found that the various functions in this system were well integrated; Q6: I thought there was too much inconsistency in this system; Q7: I would imagine that most people would learn to use this system very quickly; Q8: I found the system very cumbersome to use; Q9: I felt very confident using the system; and Q10: I needed to learn a lot of things before I could get going with this system.
SUS: System Usability Scale.

Table 2:Results of the customized questionnaire items Q1-12, Q14 and Q15.

For Q16, “Are there any other places you would like to visit through the virtual walking system in the metaverse besides the park?”, the participants’ answers were as follows: 15 answered with natural locations such as mountains and seas; 10 with urban locations such as department stores and shopping malls; 8 with historical/cultural locations such as tourist destinations, castles and world heritage sites; 2 with recreational locations such as golf courses and gyms; 1 with train station platforms; and 8 with unspecified locations such as overseas locations, places never visited before and rare places. Regarding Q17, “What scent do you think would be good to include in the virtual walking system in the metaverse?”, the participants’ responses were as follows: 20 answered with food scents; 11 with other floral scents such as rose, chrysanthemum and citrus; 3 with wood or forest scents; and 1 with a sea scent.

For Q18, “What sound do you think would be good to include in the virtual walking system in the metaverse?”, the participants’ responses were as follows: 11 answered with nature sounds such as animal sounds, babbling brooks, rustling leaves and waves; 7 with environmental sounds such as busy street sounds, including cars and trains; 6 with musical sounds such as healing, rhythmic and quiet music; and 3 with other sounds such as footsteps or people’s voices. For Q19, “What other avatars do you think would be good to have?”, the participants’ responses were as follows: 15 answered with animals, including pets such as dogs and cats; 5 with children; 4 with a free setting; 2 each with a familiar person, shop clerk and moving object; and 1 each with foreigner, celebrity, myself and anime character.

The present results provide valuable insights into the usability of our newly developed virtual walking system in the metaverse for managing frailty in individuals aged 40 years. The mean SUS score of 69.3 indicated moderate usability, exceeding the global average score of 68.0±12.5 [23]. Morizio et al. [26] evaluated the usability of immersive VR during robot-assisted gait training using the SUS on healthy participants before testing the device in patients after stroke, resulting in a score of 86.1±7.5. Moon et al. [27] assessed the ease of use of the Virtual Reality Comprehensive Balance Assessment and Training in healthy young adults using the SUS, resulting in a score of 79.7±9.9. These findings suggest that while the virtual walking system shows promise, there is room for improvement to enhance user experiences and satisfaction. Analysis of the mean scores for each subitem revealed specific areas for improvement. Specifically, subitems such as Q1 “I think that I would like to use this system frequently” and Q4 “I thought that I would need the support of a technical person to be able to use this system” received relatively low scores, indicating negative opinions toward the system and suggesting room for improvement. Conversely, subitems such as Q2 “I found the system unnecessarily complex” and Q8 “I found the system very cumbersome to use” received relatively high scores, suggesting positive opinions and indicating that specific aspects of the system may be particularly user-friendly. Based on the results of the SUS, the following specific recommendations could be made. To address the desire for more frequent use (Q1), incorporating features that encourage regular engagement such as personalized goal setting, progress-tracking and rewards for achievements could motivative users to use the system more frequently. To address Q4, concerns related to the need for technical support could be mitigated by providing more comprehensive user guides, tutorials and troubleshooting resources, thereby increasing user confidence and reducing barriers to use. Furthermore, providing targeted training sessions or workshops to familiarize users with the system’s features and functionalities could improve user confidence and reduce the perceived need for technical support. Implementing these recommendations could help enhance the usability of and user experiences with the virtual walking system in the metaverse, ultimately improving its effectiveness in managing frailty.

The results of the customized questionnaire items provide valuable insights into the participants’ experiences and perceptions of various aspects of the virtual walking system in the metaverse. Overall, the majority of participants reported positive responses to the usability of the system. For example, a high percentage of the participants found the ankle sensor (Q2) and torso sensor (Q3) easy to operate, indicating a user-friendly interface for controlling the system’s functionalities. In addition, most participants did not report discomfort in selecting paths using the torso sensor (Q5) or in the relationship between the flowers on the screen and their scent (Q8), suggesting that these aspects of the system were well designed and did not impede user experiences. However, some areas for improvement were identified. For instance, a notable proportion of the participants expressed discomfort with the timing of the scent release (Q9) and the output of sounds (Q10), indicating that these aspects of the system may need to be adjusted to align better with user expectations and preferences. In addition, while the majority of the participants believed that the screen changed smoothly (Q6) and that it would be good for the screen refresh rate to change depending on the walking speed (Q11), some participants did not share these views, suggesting that further optimization may be needed in these areas.

The other key finding of this study was the diverse preferences of participants regarding the use of the virtual walking system. The participants expressed a strong preference for natural environments such as mountains and seas, as well as urban locations such as department stores and shopping malls. This suggests that the system’s ability to simulate a wide range of environments is crucial for engaging users and motivating physical activity. In addition, the participants’ interest in historical/cultural locations highlights the potential for the system to offer educational and cultural experiences to enhance its overall appeal and utility. The inclusion of olfactory and auditory stimuli in the virtual walking system was well received by the participants. The use of fragrances such as food and floral scents, along with sounds such as animal noises and music, added a multisensory dimension to the experience, thereby enhancing immersion and enjoyment. These findings align with previous research suggesting that multisensory stimulation can improve engagement and motivation in physical activity programs for older adults with frailty [14].

The usability of the virtual walking system was further supported by the positive responses to the walking support features, including the dynamic adjustment of movement speed and the ability to interact with CG spaces and avatars. These features may not only enhance immersion, but also promote social interaction and engagement, which are important factors in maintaining motivation and adherence to physical activity programs. Despite these positive findings, several limitations of the study should be noted. First, the sample size was relatively small and the participants were recruited from a single prefecture in Japan, limiting the generalizability of the results. In addition, this study focused on usability and user preferences; further research is needed to assess the system’s effectiveness in improving physical activity levels and health outcomes in individuals with frailty.

This study evaluated the usability of a virtual walking system in the metaverse for managing frailty in individuals aged  40 years. The system showed moderate usability, with room for improvement to enhance user experiences and satisfaction. The participants reported positive responses overall, particularly regarding the ease of operating sensors and the immersive nature of the system. Areas for improvement included the timing of scent release and the output of sounds. The participants expressed diverse preferences for virtual environments, highlighting the importance of offering a variety of settings to engage users. The inclusion of olfactory and auditory stimuli was well received, enhancing immersion and enjoyment. The system’s walking support features, such as dynamic movement speed adjustment and interaction with CG spaces and avatars, were also positively received. However, further research is needed to assess the system’s effectiveness in improving physical activity levels and health outcomes in individuals with frailty.

We wish to express our appreciation to all the study participants. This research was based on data collected for a project funded by “Knowledge Hub Aichi”, Priority Research Project IV from the Aichi Prefectural Government in September 2022.

The authors declare no conflicts of interest.

1. Campbell AJ, Buchner DM (1997) Unstable disability and the fluctuations of frailty. Age Ageing 26(4): 315-318.
2. Rockwood K, Mitnitski A (2007) Frailty in relation to the accumulation of deficits. J Gerontol A Biol Sci Med Sci 62(7): 722-727.
3. Rodríguez-Mañas L, Féart C, Mann G, Viña J, Chatterji S, et al. (2013) Searching for an operational definition of frailty: A delphi method based consensus statement: The frailty operative definition-consensus conference project. J Gerontol A Biol Sci Med Sci 68: 62-67.
4. Walston J, Fried LP (1999) Frailty and the older man. Med Clin North Am 83(5): 1173-1194.
5. Fried LP, Tangen CM, Walston J, Newman AB, Hirsch C, et al. (2001) Frailty in older adults: Evidence for a phenotype. J Gerontol A Biol Sci Med Sci 56(3): M146-M156.
6. Brunner EJ, Shipley MJ, Ahmadi AS, Valencia HC, Abell JG et al. (2018) Midlife contributors to socioeconomic differences in frailty during later life: A prospective cohort study. Lancet Public Health 3(7): e313-e322.
7. Daniel Eduardo DCL (2021) The use of bayesian network models to identify factors related to frailty phenotype and health outcomes in middle-aged and older persons. Arch Gerontol Geriatr 92: 104212.
8. Griffin FR, Mode NA, Ejiogu N, Zonderman AB, Evans MK, et al. (2018) Frailty in a racially and socioeconomically diverse sample of middle-aged Americans in Baltimore. PLoS One 13(4): e0195637.
9. Hanlon P, Nicholl BI, Jani BD, Lee D, McQueenie R, et al. (2018) Frailty and pre-frailty in middle-aged and older adults and its association with multimorbidity and mortality: A prospective analysis of 493 737 UK biobank participants. Lancet Public Health 3(7): e323-e332.
10. Kuzuya M (2017) 9 Impact of sarcopenia and frailty on preventive care for older people. The Journal of the Japanese Society of Internal Medicine 106(3): 557-561
11. Yoshimura Y, Wakabayashi H, Yamada M, Kim H, Harada A, et al. (2017) Interventions for treating sarcopenia: A systematic review and meta-analysis of randomized controlled studies. J Am Med Dir Assoc 18(6): 553.e1-553.e16.
12. Kojima G, Taniguchi Y, Iliffe S, Jivraj S, Walters K, et al. (2019) Transitions between frailty states among community-dwelling older people: A systematic review and meta-analysis. Ageing Res Rev 50: 81-88.
13. Apóstolo J, Cooke R, Bobrowicz CE, Santana S, Marcucci M, et al. (2018) Effectiveness of interventions to prevent pre-frailty and frailty progression in older adults: A systematic review. JBI Database System Rev Implement Rep 16(1): 140-232.
14. Lee YH, Lin CH, Wu WR, Chiu HY, Huang HC, et al. (2023) Virtual reality exercise programs ameliorate frailty and fall risks in older adults: A meta-analysis. J Am Geriatr Soc 71(9): 2946-2955.
15. Cowan K, Ketron S, Kostyk A, Kristofferson K (2023) Can you smell the (virtual) roses? the influence of olfactory cues in virtual reality on immersion and positive brand responses. Journal of Retailing 99(3): 385-399.
16. Kaplan RR, Cockerham D, Ferdig R (2023) The impact of sound and immersive experience on learners using high-immersion virtual reality and tablet: A mixed-method study. SSRN Electronic Journal, p. 27.
17. (2018) Standardization IOF International Standard ISO 9241-11.
18. Brooke J (1995) SUS: A quick and dirty usability scale. Usability Evaluation in Industry, (1^st edn), CRC press, Florida, USA, pp. 189-194.
19. Brooke J (2013) SUS: A retrospective. Journal of Usability Studies 8(2): 29-40.
20. Takano E, Maruyama H, Takahashi T, Mori K, Nishiyori K, et al. (2023) User experience of older people while using digital health technologies: A systematic review. Applied Sciences 13(23): 12815.
21. Huang P, Ishibashi Y, Fukushima N, Sugawara S (2012) Quality of experience assessment of dynamic output timing control of fragrance in networked virtual environment (in Japanese). The Journal of the Institute of Image Information and Television Engineers 66(12): J495-J499.
22. Huang P, Ishibashi Y (2023) Enhancement of dynamic output timing control of fragrance in metaverse. IEEE/IEIE The 8^th International Conference on Consumer Electronics - Asia (ICCE-Asia), Special Session on QoE/QoS for Cyber-Physical Systems, South Korea.
23. Bangor A, Kortum PT, Miller JT (2008) An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction 24(6): 574-594.
24. Bangor A, Kortum P, Miller J (2009) Determining what individual SUS scores mean: Adding an adjective rating scale. J Usability Studies 4(3): 114-123.
25. Davis F (1989) Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly 13(3): 319-339.
26. Morizio C, Compagnat M, Boujut A, Labbani IO, Billot M, et al. (2022) Immersive virtual reality during robot-assisted gait training: Validation of a new device in stroke rehabilitation. Medicina (Kaunas) 58(12): 1805.
27. Moon S, Huang CK, Sadeghi M, Akinwuntan AE, Devos H et al. (2021) Proof-of-concept of the virtual reality comprehensive balance assessment and training for sensory organization of dynamic postural control. Front Bioeng Biotechnol 9: 678006.