Exploring Embodiment and Body Perception with an Augmented Reality Mirror

This project is already completed.

In the last decades, eating and weight disorders have become an enormous problem for our society (WHO, 2016). In obesity therapy, the research project ViTraS (Virtual Reality Therapy by Stimulation of Modulated Body Perception) investigates approaches to support the treatment of eating and body image disorders with the help of virtual reality (VR) and augmented reality (AR) (Döllinger et al., 2019). The fundamental idea of this research project involves a virtual mirror in which the patient sees a personalized avatar that embodies himself and whose weight can be changed. This setting can reveal a patient’s mental body image and help visualise a distorted body image. Their distorted self-perception could be improved by the described application to regain a real image of themselves and further increase the motivation for the therapy. Furthermore, it is conceivable to show the patient their weight loss and successes during therapy with the application. In addition, from the beginning of the therapy, the patient can deal more intensively with their own desired weight and visualizes their own goal (weight). The project is conducted within ViTraS and builds on the work of Wolf et al. (2020). They developed an interactive virtual mirror embodiment application, as described above, to investigate the differences between an augmented reality video see-through head-mounted-display (HMD) and a virtual reality HMD on presence and embodiment. In this environment, the subject embodies a non-personalized, standardized virtual avatar with unchanging weight.

Goal

The work extends the developed interactive virtual mirror embodiment application by the possibility to use the application with an augmented reality optical see-through HMD. Through this extension, first comparative insights between an augmented reality optical see-through (OST), an augmented reality video see-through (VST), and a VR embodiment systems can be gained with regard to presence and embodiment. For this comparison the data from Wolf et al. (2020) are used and compared with newly collected data of the AR optical see-through HMD. To create a valid comparison between the newly collected data and the study from 2020, the experimental setup, execution, and analysis were based on the procedure used by them. In addition, the project aims to provide more information based on bodyweight perception in AR.

An AR application has the advantage over a VR application that the isolated virtual experience is brought back into reality. This provides several benefits for later use in a therapeutic context. Most importantly, reality is not completely shut-out and comparisons of the virtual with the real body are still possible. For example, an exposure would be conceivable, when the patient can look at his virtual self in a virtual mirror and by looking to the side, they simultaneously see theirselves in a real mirror. In this way, differences between the real self and the perceived self could be worked out even better. Futhermore, it allows more interaction between therapist and patient, as patients are not completely isolated in the virtual world. While the patient in the VR application is alone with their virtual avatar in a virtual therapy room during treatment, the patient in the AR application sees all three their virtual avatar in a virtual mirror, their therapist, and their own body in the real world. This allows the therapist to interact with the patient during the treatment (e.g., support, motivate). On the other hand, a VR application would be feasible in which the therapist, also embodied by a virtual avatar, becomes part of the application. This would make location-independent therapy possible. Nevertheless, with AR systems the patient can remain in the environment they are familiar with. Since they do not have to immerse themselves in a virtual environment, the therapies can be integrated into the patient’s daily routine and would enable regular and minimally intrusive sessions, according to Eckhoff, Cassinelli and Sandor (2019). The fact that the normal daily routine is not interrupted could lead to a higher acceptance of such treatments. However, the advantage of AR to interact with the real world could be accompanied by a lower sense of presence and embodiment in the virtual world. Along Milgram’s reality-virtuality continuum a disadvantage of an AR application is a lower immersion compared to the VR application, which in turn can have a negative effect on presence and embodiment (Milgram, 1995). Wolf et al. (2020) investigated how the type of medium (VR vs. AR) affects the sense of embodiment, presence, and body weight perception. It is assumed that a higher immersion increases the sense of embodiment and the presence and thus enables a more precise perception of the body dimensions of a self-embodied avatar. They found out that the inﬂuence of the used system conﬁguration was rather small with a descriptive stronger underestimation of the avatar weight when using VST and no inﬂuencing effect of presence and embodiment. Thus, an AR application does not deliver significantly worse results in terms of presence and embodiment despite less immersion. From the current situation, it seems very useful to develop both an AR, and a VR application to support obesity therapy.

Augmented reality optical see-through HMD (OST) vs. Augmented reality video see-through HMD (VST)

In addition to comparing AR and VR, it is useful to compare the AR variants. When using OST the direct view “ensures that visual and proprioception information is synchronized” as described by Rolland and Fuchs (2000). But the usually narrow fields of view of OST HMDs can break the continuity of presence or ownership illusions as parts of the virtual elements might get cropped by the optical system. In addition, as noted by Gilbers (2017), virtual content is partially transparent and sometimes causes depth perception problems that can affect the embodiment experience. Due to still very limited research in the AR field, a comparison of OST, VST (and VR), tailored to our scenario, is useful to gain more insights about embodiment in the AR context and to gain first comparative insights between all three systems. Therefore, the addition of an OST system to the already existing VR and VST systems in the application could be useful and is the focus of my HCI project.

Concept

In the application, the real world is augmented by a virtual mirror, in which the user sees a virtual avatar that they embody. The movements of the avatar are synchronized with the real movements of the participant. As an OST HMD, the HoloLens 2 was choosen. To map the user’s body movements on the virtual avatar, we integrated a markerless tracking system to enable recognition and processing of all the user’s body parts while also providing the correct orientation for the recognized body parts. We used the Captury motion capture system which allows markerless tracking via cameras. In our case, it works with eight cameras placed on the ceiling in our lab room that capture and record the user’s body and movements. The biggest difference from the comparative paper by Wolf et al. (2020) is the nature of body tracking. They used a marker-based tracking system. The reason why the tracking system was changed was on the one hand the already existing hand tracking through the HoloLens 2, which made the controllers obsolete. On the other hand, markerless tracking requires less preparation time and offers no risk of positioned trackers slipping during exposure. The virtual environment was developed using Unity 2020.3.11f1 and consists of a virtual version of the laboratory room used for the study, augmented by a virtual mirror. The subject embodies a standardized virtual avatar within this environment. To induce the feeling of embodiment, the participants’ movements are continuously captured and used to animate their avatar in real-time. The only visible virtual object within the presented AR environment is the virtual mirror in front of the subject, which allows the participant to view their virtual representation from an allocentric perspective: to see themselve in the mirror. At the same time, the participant can observe the real environment and their real body from an egocentric perspective through the HoloLens 2. The lab was remodeled exactly according the real lab, because virtual mirror cannot show the real lab as background. The generated virtual environment is used to render the background of the virtual mirror reﬂection behind the avatar.

Following the completed implementation, a study was conducted. The study explores the influence of embodiment on body weight perception of a virtual human in three different systems while keeping the degree of personalization constant. The aim of this study is to collect data on presence and embodiment with an OST HMD within the described context and to compare it with the data of a VST HMD and VR HMD from Wolf et al. (2020). In addition, the study will provide further information based on body weight perception in AR. Furthermore, a qualitative survey on the perception of the new application will evaluate the used technology, as it is largely used for the first time in a study.

Partners and Cooperation

This is a subproject of the ViTraS project within the Embodiment Lab. Our partners in this project are the following:

Hochschule für Technik und Wirtschaft Berlin
Technische Universität Dortmund
Hochschule für Gesundheit
Technische Universität München
The Captury GmbH
brainboost GmbH
Neomento Spin-off (NEO)
Centrum für Biomedizinische Bild- und Informationsverarbeitung
Bundesministerium für Bildung und Forschung

References

N. Döllinger, C. Wienrich, E. Wolf, M. Botsch, and M. E. Latoschik. Vitras - virtual reality therapy by stimulation of modulated body image - project outline. Mensch und Computer 2019 - Workshopband, pp. 606–611. Gesellschaft für Informatik e.V., 2019.

D. Eckhoff, A. Cassinelli, and C. Sandor. Exploring perceptual and cognitive effects of extreme augmented reality experiences. 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2019.

C. Gilbers. The sense of embodiment in augmented reality: A third hand illusion. Master’s thesis, Utrecht University, Utrecht, 2017.

P. Milgram, H. Takemura, A. Utsumi, and F. Kishino. Augmented reality: a class of displays on the reality-virtuality continuum. Telemanipulator and Telepresence Technologies, vol. 2351, pp. 282–293. International Society for Optics and Photonics, 1995. doi: 10.1117/12.197321

J. P. Rolland and H. Fuchs. Optical versus video see-through headmounted displays in medical visualization. Presence, 9(3):287–309, 2000. doi: 10.1162/105474600566808

WHO. Obesity and overweight, 2016. Accessed at https://www.who.int/news-room/fact-sheets/detail/obesity-and-overweight.

E. Wolf, N. Döllinger, D. Mal, C. Wienrich, M. Botsch, and M. E. Latoschik. Body weight perception of females using photorealistic avatars in virtual and augmented reality. 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2020.

Contact Persons at the University Würzburg

Erik Wolf (Primary Contact Person)
Chair of Human-Computer Interaction, University of Würzburg
erik.wolf@uni-wuerzburg.de