Survey to go: Designing a notification generator for experience sampling studies

This project is already assigned.

Motivation and Goals

The Experience Sampling Method (ESM) is used in different fields of research. It aims to assess peoples’ behaviours, feelings and thoughts in their daily lives and is mostly based on self assessment and reporting [9]. There are many ways to conduct studies using ESM from having participants write a paper diary daily to collecting smartphone usage data etc. For this Bachelor’s thesis I want to develop a platform that supports ESM studies by enabling researchers to remind participants of surveys during their daily lives through their own mobile phones. We chose that approach, because even though there are many different (free) online survey platforms that enable researchers to conduct online studies that can be long term and on a in-situ basis, the reminders to fill out questionnaires are mostly just sent via eMail. Combined with a known tendency for participants of online studies to reply less to questionnaires over time [8], we aim to develop a smartphone application, that sends notifications to participants. We hope to decrease drop-out rates for online surveys by letting the participants use their own phones, thus enabling easier access for reminders to fill out questionnaires. Secondly, we aim to make this tool as accessible and usable for researchers as possible, by letting them use a survey framework of their choice that they are already used to (e.g. formr, limesurvey, etc.) to design the (temporal) structure of the experiment and only change the way participants receive reminders and the actual link to the task.

Experience Sampling and mobile phones

Combining ESM and smartphones is not a novel idea. There are a few solutions designed specifically for designing and conducting ESM research [4]. Applications like iForm, SERENA [1] or LifeData provide tools to create own surveys [4]. Programs like Survey Signal or Diario deliver links to online surveys via SMS [4]. The main barriers when looking for ESM survey tools are that some are rather costly (up to 1000 USD per device [4]) , others lack in the flexibility and breadth of range when designing studies [11], while some are developed either for iOS or Android but not both [4].

App notifications and nudging

Notifications are an essential trigger for the usage of applications [7]. Research also showed, that not sending notifications can be distressing for app user as it can be perceived as a sign of connectivity malfunction [7]. As such, notifications play a vital role in nudging, which is defined as “any aspect of the choice architecture, that alters people’s behaviour in a predictable way without forbidding any option […]” [3]. The aim in this case is, to create an application that nudges research participants towards the behaviour of reliably filling out surveys. The challenge on this part will be to design the application to prevent the nudging effect decreasing drastically over time. This can be achieved by designing the notifications and application as unintrusive as possible while also sustaining reminders [3] and will be part of the usability evaluation.

Planned Methods and Requirements

The requirements for this project will be split into two parts: the webserver to be used by researchers and the mobile phone application for participants.

The webserver should be designed for supporting user administration and different profiles. Researchers should be able to determine the time, frequency and quantity of questionnaires per participant and manage the weblinks to those.

The mobile app should work under both, Android and iOS. It should support enrolling in different studies with a registration code. After registration, the participants should receive Push-Notifications at times set by the researcher. Those notifications should be customizable and their contents, along with the current link to the survey should be visible in the app. Van Berkel et al [2] advise against the stacking of questionnaires and notifications, which should also be solved by a form of expiration date for the links.

Since research data can be personal and sensible, all user data will be kept safely on the university’s servers or the clients’ devices.

To implement those requirements, I plan to use the Meteor framework [10] and React library [5] for the server and database, as well as the browser interface for the researchers. The mobile app will be implemented using the React Native framework [6].

The empirical part of the thesis will focus on testing the functionality of both the webserver and the mobile app as well as checking the usability (effectiveness, efficiency and level of satisfaction) for both programs. For an initial overview of the usability a heuristic evaluation will be done. After that, an user study will be conducted in the style of a classical usability test. We will test effectiveness and functionality by creating exemplary tasks and monitoring the completion of those tasks. After each task, the participants should also fill out a mental work load questionnaire to evaluate efficiency. A short post-interview should evaluate the level of satisfaction.

References

[1] P. Baudisch, M. Beaudouin-Lafon, W. Mackay, A. for Computing Machinery,S. G. . U.S.), and A. D. Library.CHI2013 Changing perspectives : extendedabstracts : the 31st Annual CHI Conference on Human Factors in ComputingSystems : 27 April - 2 May, 2013, Paris, France.

[2] N. V. Berkel, D. Ferreira, and V. Kostakos. The experience sampling methodon mobile devices.ACM Computing Surveys, 50, 12 2017

[3] A. Caraban, E. Karapanos, D. Gonçalves, and P. Campos. 23 ways to nudge:A review of technology-mediated nudging in human-computer interaction. Association for Computing Machinery, 5 2019.

[4] T. Conner. Experience sampling and ecological momentary assessment withmobile phones, 2015.

[5] I. Facebook. React, 2020.

[6] I. Facebook. React native, 2020.

[7] D. Ferreira, J. Goncalves, V. Kostakos, L. Barkhuus, and A. K. Dey. Con-textual experience sampling of mobile application micro-usage. pages 91–100.Association for Computing Machinery, Inc, 9 2014.

[8] E. W. Knightly, A. D. Library., and A. SIGMOBILE.Proceedings of the 5thinternational conference on Mobile systems, applications and services.ACM,2007.

[9] R. Larson and M. Csikszentmihalyi. The experience sampling method, 2014.

[10] Meteor. Meteor, 2020.

[11] D. J. Rough and A. Quigley. End-user development of experience samplingsmartphone apps -recommendations and requirements.Proceedings of the ACMon Interactive, Mobile, Wearable and Ubiquitous Technologies, 4, 6 2020.

Contact Persons at the University Würzburg

Marc Erich Latoschik (Primary Contact Person)
Human-Computer Interaction, Universität Würzburg
marc.latoschik@uni-wuerzburg.de

Jan-Philipp Stauffert
Human-Computer Interaction, Universität Würzburg
jan-philipp.stauffert@uni-wuerzburg.de