Use of Proximity Sensors to Inform Just-in-time Adaptive Interventions

Mobile technologies have the capacity to track factors that are important to health like being physically active and provide customized interventions exactly when and where it would be most beneficial for each person. While this possibility exists in theory, in practice it is very hard to know when the “right” time is for delivering these sorts of messages.

Technologies such as indoor location sensors that can track where a person is inside buildings provide better opportunities to identity those exact moments when a person might both benefit from and be receptive to messages to help them achieve their personal goals.

We (Dr Eric Hekler, Aaron Coleman from Fitabase, and I) received a $25,000 Agile project grant from the Health Data Exploration Project to explore how proximity sensors (iBeacon) could be used to inform such “just-in-time” interventions (JITAI), and also to develop an open-source iOS tool that others could use to utilize proximity sensors for their own purpose.

Aaron led the software development, while I led the other parts of the design and research process along with Eric.

Location as a contextual factor in driving interventions

A user’s precise location is an important factor that could drive their opportunity and receptivity towards a certain intervention. For example, if we could approximate indoor location, opportunity and receptivity, we could theoretically keep track of a person sitting and watching TV in her living room and nudge them to take a break from sitting or to take a walk at an opportune moment (such as during a commercial break, or between episodes/shows).

Proximity sensing technologies such as the iBeacon are sensors that work on Bluetooth Low Energy and enable a smartphone or other device to perform assigned actions when in close proximity to them. They can be placed at any location of choice (e.g., on the desk, in the kitchen, in the car). They allow mobile phones to detect the user’s precise location and context when in proximity to them (and when the location is known).  These proximity sensors can also be useful for tracking proximity to other things that are important to a person. For example, proximity sensors could be placed in a person’s car to know when he or she is in the car or could be attached to a close friend’s phone to enable easy detection when the two friends are in close proximity to one another.

Applied to mHealth, this contextual technology provides unique opportunities for establishing decision rules for identifying potential desirable and undesirable locations and moments when the mHealth app should interact with the user.

Motivation

At the Designing Health Lab, we strive to design interventions that are able to effectively estimate when a person is receptive and interruptible. We immediately jumped on the opportunity and decided to design a study to explore if these sensors (We used Estimote Inc.) can be used as an effective contextual signal for indoor location sensing and to detect “opportune” moments when mHealth apps can interrupt a user.

The focus was on documenting the challenges that come up and identifying opportunities for incorporating proximity sensors in a JITAI.

This was a relatively short project and we did not go through the design process in a way that I would ideally like. But this was the process we followed:

Semi-structured interviews

We conducted semi-structured interviews with a convenience sample of smartphone users ( (N = 12; 3 females, 9 males, age (M + SD) = 25.83 ± 1.58). Participants were iPhone (iOS; n= 8), Android  (n= 3) and Windows (n= 1) users, either graduate students (n= 4), or professionals (n= 8).

Results from the interviews were organized based on key themes to:

1. Understand peoples’ perceptions of proximity sensors, interest and ideas on how these tools might be used to support more personalized interventions
2. Gain insights on useful locations to take advantage of proximity sensors for individualized behavior change

Key Insights:

Perceptions related to proximity sensors

1. Privacy concerns: A third person knowing their precise location seemed “creepy” – need to address security and privacy concerns when using this technology
2. Lots of individual differences in perceptions related to receptive locations, cellphone usage and interruptibility – narrow down and focus on concrete use-case to do some rapid prototyping

Ideas on use of proximity sensors

1. Prompt to workout when close to gym shoes
2. Healthy eating suggestions while in the kitchen (although this one could be biased because this was the example used by the interviewer, eeek)
3. Prompts to sit less at work
4. “Shut off” notifications in certain locations such as car, while sleeping, when too busy at work.

People also seemed to like the idea of being able to ‘shut off’ notifications at particular places (e.g., in the car, while sleeping, when too busy at work, etc), thus establishing some interest in using proximity sensors as a strategy to define moments when they were NOT receptive to interacting with their phone.

Another critical insight was that – many (5 out of 12, to be precise) said that they try to keep their phones on silent, away/inverted on their desks in order to avoid distractions from notifications while working. Few also said that they use their laptops and computers instead of phones to check the most important notifications while at work.

Implications for designing “just-in-time” interventions

When looked at overall, even with only 12 individuals interviewed, it became apparent that there were as a great deal of individual differences in when someone thought they would be receptive to interacting with a technology. For example, few participants indicated that they would be interested in using them only at the workplace and not in their home, while others said the exact opposite. Based on this, it is highly likely that some initial data will need to be gathered (e.g., preferences about being interrupted that are connected with different locations or times of day, places where person spends a significant portion of time, which device is used for receiving notifications at what time) to fully interpret data that can be provided from proximity sensors.

Development of a proximity sensor tool

Based on the interviews, brainstorming sessions, and resources we had available, the team chose to focus on a concrete use-case of a just in time intervention to reduce sitting and increase walking in an office environment.  

Two tools:

1. Prox: Open source location sensing app
2. ProxMessage: app to deliver JIT suggestions to sit less and walk more (more in the next section about feasibility study to test this concept out)

What we did NOT target

As this was an “agile” project, the team explicitly identified important potential issues with proximity sensors for just in time interventions that they deemed were secondary to their goal of developing an open source tool.  These included acknowledgement that, for the current design:

1. Individuals would be required to carry their phones at all time within the office complex (a behavior this is not common with all individuals)
2. While possible to create (and, indeed, we had an initial plan that would have taken advantage of the beacons, described below), we did not have resources to track when a person was and was not carrying their phones
3. We recognized that other data sources (e.g., information about busyness from a digital calendar) would complement our understanding of moments of receptivity, for the purpose of this agile project, we remained focused only on proximity sensors
4. We recognized that for true just in time interventions, the use of notifications via a smartwatch is very likely more appropriate but, based on budgetary limitations, it was not feasible to buy smartwatches for participants nor develop the custom software needed to provide these notifications on the smart watches.

We saw all of these as other important areas to study but that were ultimately separate components to our primary target of developing an open source tool to help individuals gather proximity sensor data for whatever use they so choose, with ours being for just in time interventions.

Prox

We utilized an agile software development process with repeated rapid iterative development between Fitabase and the Designing Health Lab that was then extended to external uses.  Examples of variations in visual prototypes for the proximity sensor tool can be seen below:

Goals:

1. Gain understanding of beacon functionality
2. Simple
3. Free
4. No server set-up
5. De-identified data
6. Open source
7. Background mode w/ no participant responsibilities

Technical lessons learned from the prototyping phase

There were a variety of lessons we learned about proximity sensors, particularly the beacons from Estimote, during our development. These included:

1. Estimote offers two types of proximity sensors, large Beacons and “stickers” which are smaller and more compact (about the size of a quarter both in width and depth). At the time of development, it was found that these two sensor types actually functioned with different code, thus not enabling a single software tool to be developed that could support them both (though this is being rectified). Our original design idea for knowing when a person was wearing their phone was to attach a sticker to each person’s Fitbit Charge HR (a tracker that we provided to the participants in the subsequent feasibility testing described next). Unfortunately, the API for using the stickers were different from the original beacons and thus this option was no longer feasible.
2. Beyond this, the larger beacons have fairly large ranges (though not necessarily what was reported).  In terms of implications of this, this created problems with doing very fine-grained tracking within locations.  For example, in an ~800 sq. ft apartment, one beacon would effectively cover the entire apartment, thus making it difficult to determine where inside the apartment (e.g., living room or kitchen) a person may be.

This is what the beacon set-up process looked like within Prox:

• 

• 

• 

• 

Beacons can be configured one-at-a-time or can be imported from another previously set up configuration.

Prox app features:

1. Naming individual beacons
2. Defining custom categories which will be labeled in exports
3. Works with any Estimote beacons out of the box.

We then set-up and connected a few Estimote beacons in our building at Arizona State University to test the Prox prototype with 2-4 users, to get a sense of whether it was sensing locations appropriately, how close/far enough two beacons need to be to avoid multiple beacons from getting sensed, etc.

Prox-of-fun (no idea why we called it that, but we did)

After we had a prototype for Prox (app only doing the beacon sensing and collecting data about locations when a recognized beacon was in range), we sought to do an initial exploration of feasibility of the proximity sensor tool in a real-world context.  

To do this, t designed a subsequent simplified app for delivering just in time prompts based on the information from the proximity app. As this was only a small feasibility study, only 4 individuals were recruited. This was a simple app that was developed to deliver just-in-time prompts to sit less and walk more based on location sensing done by the Prox app. More about the “how did it determine when and how” in a bit.

4-week field test 

Aim: Feasibility testing of the app in a real-world context, uncovering challenges and design opportunities

Ideally, and now looking back, a 4-week test was too long too soon. However, I feel like with the pressure of having to complete this project within the given time-frame and having a “study” completed, this is what we did. If I were to redesign this study now, I would do much shorter deployments with observations and contextual inquiries before a long deployment (where we did not interrupt/observe). 

Participants: We recruited four employees working at a single location on ASU campus (it was the HR department) that was interested in this concept and would allow us to place beacons all over for four weeks. The Prox app is only on iOS for now, so they needed to be iPhone users.

Prior to the study set-up session, all participants gave us a list of important locations they frequented on a typical day (home, work and others).

Study set-up:

1. Each participant was provided a Fitbit Charge HR tracker to be used at all times for all four weeks.
2. Based on the list provided, final locations were chosen based on distances between listed locations, ending with roughly 10 beacons per participant.
3. I met with each participant at their workplace to set them up in the study and explain the protocol.
4. We installed both apps on their phone, tagged all the beacons in the Prox app and placed the beacons at the locations they had listed in the form. One beacon was placed at each of their desks.
5. We decided to tag some of the common locations such as kitchen, bathroom and front desk on all participants’ phones, irrespective of whether they chose it or not.

Pre-identified issues for this field test

• No knowledge of times when participant’s phone isn’t with them
• Beacon range is high, hence unable to do very fine-grained tracking
• Did not use any other data sources to understand receptivity

Considering those issues, we asked them to:

• Carry their phone with them at all times
• Keep Bluetooth switched ON
• Wear Fitbit all day for four weeks.

So Phase A (Weeks 1 and 2) was what we call our baseline phase, which is something we almost always for a long time (at least 2 weeks) do in our studies for physical activity, because of Hawthorne effect (peoples’ behavior getting affected as a result of being observed) and also to get a sense of their usual levels of activity. We asked all participants to maintain their usual levels of activity.

Phase B – was the intervention phase, where participants began receiving prompts in the form of push notifications.

JIT prompts: The idea was to prompt participants to take a short walk either when they had been sitting for too long (desk, triggered when detected around it for ~60 minutes), or when they were already on the move and close to the frequented locations where the beacons were placed.

Examples of notifications:

• “Need to be more productive? Walk 100 steps!”
• “Sitting too long? Try walking 100 steps!”
• “Walk a 100 steps more while you’re on the move already!”

We used a micro-randomization design for this intervention phase – the desk-based prompts were programmed with a 50% probability, while other prompts were delivered 30% of the times. This was done to avoid too many notifications and also so that we would be able to assess differences in PA when the participants received the prompt vs not. Users were prompted a maximum of 10 times a day.

Participants also received a daily evening survey asking them to provide insights and comments about the walking prompts provided that day.

Insights from 4-week field test

Based on the daily surveys, and interviews with all 4 participants, looking at collected data, and team discussions, we got the following insights:

Lessons learned about the technology itself

1. The app is very easy to use, and tagging a beacon does not take more than 1 or 2 minutes.
2. iBeacon technology requires permission to access user’s location data, along with requiring Bluetooth to be switched on at all times.
3. Participants did not mind the beacon being placed visibly on their desk (aesthetically not a problem).

Issues with the sensors

1. High range, so not easy to tease out locations if they’re close to each other.
2. Require location too
3. Seems like it would be best used in combination with other sensors. Then they might be helpful.
4. Need to have phone with you at all times.

Lessons learned from users about just in time interventions

• Logistical issues:
  • Carrying the phone around at all times was not easy. Participants reported that they tend to forget to carry their phones especially if it’s just for a short break (such as going to the restroom)…this was a known limitation. Smartwatches could potentially solve this issue.
  • Ideally, the prompts received should be in the moment when the participant is on the move. The prompts were delivered with a lag, so often participants were already at their desk when they received the prompt
  • If the user’s phone is on silent, notifications are going to go unnoticed. For these “just-in-time” notifications to be effective and noticed, the phone needs to at least be on vibration mode.
  • On the iPhone, the notifications are set to “banner” by default. These notifications are visible only for a few seconds and later sit in the Notification Center. If the user did not notice these prompts as soon as delivered, many went unnoticed.
  • Notifications were at times received within few minutes of one another, which was another bug we weren’t able to fix in time.
• Some breaks are different than others
  • On most occasions, participants reported not noticing the prompts when they actually received them. For example, one of them informed us they on a few occasions, she was already back at her desk when she received it…I think this made us realize that some breaks are different than others (long/short, for your own task vs. going to meet someone).
• Participants preferred desk notifications over others
  • They found the notifications sent at the desk to be more noticeable and helpful. This might be due to the fact that those were sent in a more stable environment (if person was in close proximity to beacon for ~60 minutes).
  • One participant said that she did not understand what the sensors were doing and what their connection with the prompts till the end of the study. Did not find the “on the move” prompts helpful.
• Other
  • Three of the participants who were interviewed after study said that they found the Fitbit more helpful in increasing their walking… this is a big issue. If the Fitbit itself is novel for them, adding another layer of tech makes it difficult to assess if proximity sensor-based prompts added anything.
  • Participants did not find sensors obtrusive.

Future Opportunities

If this technology is to be used in JITAI, then some future opportunities for exploration include:

1. Using them in combination with smartwatches
2. Triangulation to accurately predict location
3. Using them in combination with important pieces of contextual information about the user – such as cellphone-related habits at the workplace, social context (alone or with someone?), busyness (calendar?), posture (not just if they’re at desk, but at desk and sitting), proximity to phone, etc. — This might be a useful signal not just for driving interventions, but even for understanding dynamics of health related factors like inter-individual dynamics, understanding key locations, people the user is with when most or least active, understanding routines, etc.

I think the meta question I landed on was: Are so many notifications even necessary? Or might one or two sent at the perfect moment be more potent? 

I also think that on a personal level, this was the project that made me realize that I made the mistake of letting the technolody drive my work instead of users needs and motivations.

Lesson learnt. Sometimes you can read something in 15 textbooks but only lived experience helps you actually understand what it means.

I don’t know if Aaron is still up for offering any support since its been a while, but if interested,

Contact him for a build: fitabase.com/Prox

Code: github.com/Fitabase/Prox