I designed the largest addition Tidepool made to the open-source Loop app: onboarding and in-app training that enabled new users to train at their own pace on their own devices while still ensuring safety in all critical tasks.

Challenge:

Onboarding: from 0 to 1

By building on an open source project with no in-app onboarding or training, we knew that one of the greatest opportunities to add to the DIY app was in enabling people to onboard new users safely and quickly. Further, we needed the onboarding to be complete prior to our Human Factors testing because it was our primary means of supporting new users and would be a key area of inquiry in our FDA 510k submission. It also needed to integrate third-party partner device training, setup and pairing, including their previously validated training content modified only where our UX differed.

The Pump-Start Paradigm

The current state of insulin pump onboarding is almost exclusively through in-person trainings usually over 2-3 hours with a manufacturer-paid pump trainer. This in-person, high-touch approach is both costly to device makers and burdensome to many people with diabetes and caregivers because it involves taking time off from work, and travel and doesn’t often adapt well to different learning styles. That said, it’s hard to replace the safety benefits that come from in-person observation that the user is using the interface correctly. When that interface moves to software, the human factors validation becomes that much more important.

Controller Software:
an opportunity for interoperability

The opportunity for Tidepool Loop exclusively playing the role of the controller software (and not the CGM or Insulin Pump), is that we had an opportunity to define a new onboarding experience for a digital therapeutic that, while coordinating the activity of connected devices, was in fact only exclusively software. Our goals were to use this onboarding to:

• increase access to more people with diabetes by minimizing time cost
• provide digital delivery of training in-context, on the device they would be using
• be flexible and modular enough to enable the addition of new pumps and CGM’s as needed without breaking what came before

…all while ensuring that it trained them to safely use the system for delivering insulin, a hormone that, if dosed incorrectly can be fatal within minutes.

For regulatory geeks: Tidepool Loop is designed to  function within the FDA's interoperable automated insulin delivery paradigm as the Controller software, designed to interface with other interoperable, "Alternate Control Enabled" insulin pumps and interoperable "iCGM's." 

Alain Silk, FDA Branch Chief, explaining the agency’s vision for interoperability with a clothes metaphor in 2019.

Approach:

Phase 1: Mapping and Sketching

Identify Stakeholders and SME’s

We began by identifying and consulting with key subject matter experts:

• Open-source app users and
• Community training leaders
• Endocrinologists and Clinical Advisors
• Diabetes educators

Which Hat Are You Wearing?

It also became clear that many internal team members also had assumptions and ideas about what this onboarding would cover as many of them also live with type 1 diabetes and were open-source users. At first, it was difficult to determine whether folks were sharing input from their professional roles as developers and QA engineers or from their personal roles as diabetics or caregivers. I started modeling and requesting that, whenever it was ambiguous, we explicitly state which hat we were wearing when championing a given perspective or feature. This is a key user-centered design value I regularly champion: holding a clear boundary between our own experiences and that of our users. This is all the more critical when there is significant overlap.

From these early conversations, I drafted the formal Product Requirements Document for this feature set and guided it through several rounds of revision with the executive team, product, and engineering.

Information Architecture of Content

We developed an outline of the content and began mapping out the information architecture of what sequencing made the most sense. Our target was a setup time of half an hour or less if done sequentially.

It became clear that our onboarding needed to be significantly longer than most consumer-facing app onboardings (1-3 minutes) but significantly shorter than the current industry standard for insulin pumps (~3 hours).

So we started organizing the content in chapters that could serve as checkpoints to enable users to complete parts of it and return with their progress saved. Further, we wanted to enable new or potential users to learn about the app in detail before doing the work to get a prescription from their doctor. By moving the content lock to after account setup and detailed training, we preserved an important marketing funnel whereby the content itself could continue to make the value props of our complex product explicit and we could reach out to potential users who started to onboard but never completed prescription.

This clarified the need for back-end infrastructure to manage user prescriptions and gate access by them. At this point, I did some quick mapping of where this might fit with existing infrastructure and what new components might be needed. This map enabled me to have a clear shared understanding with our VP of Engineering and enabled the product and engineering teams to anticipate what features we would be needing and get them on the roadmap.

Wireframes and Storyboards

Along with my colleague, Paul Forgione, we then began wireframing out the onboarding experience. We wanted a simple to grasp, visual description of the system and how it fits into daily life so I started storyboarding illustrations for some of those components.

Live Stakeholder Review

Our VP of product asked me to present the onboarding work thus far at our next in-person onside retreat. After walking through our process, feedback thus far and goals, We printed the 30-minute onboarding out and hung it up gallery-style inviting questions and feedback from the whole team. This not only resulted in lots of great input but generated a lot of clarity among the rest of the organization about where we were in the process and how their departments and teams could fit into the future we were designing for Tidepool Loop.

Phase 2: Refining, De-Risking, Branding & Testing

Instructions for Use

While we always envisioned the in-app onboarding as the primary source of safety training for new users, we were simultaneously developing a more comprehensive set of instructions for use. Our initial draft was based on and expanded from the content from the onboarding. But the process of writing instructions for each feature forced us to make and document clear decisions on consistency in language, labels, and even metaphors. In collaboration with the Marketing team’s lead writer, we began to create an authoritative glossary. From this, we were then able to document each use of keywords and terms we expected to be impacted by future updates so that we could track their occurrence in the app and gauge the lift of seemingly small terminology or label changes.

Use Related Risk Analysis

Even though Tidepool Loop was still under development, we had a unique opportunity to learn from known risks in the open-source app. DIY Loop, as we called it internally, had been in use for several years and had a thriving online community in a Facebook group devoted to troubleshooting and support. In collaboration with the product team, I designed and ran a study with two major inputs:

1. A survey of the Looped User Group requesting reports of known use errors or risks.
2. A sample of one month of posts taken from the Loop users’ Facebook group

We requested permission from the leadership of the group before conducting any research. They granted permission provided we shared the results of our findings. They, like many other disability communities, have learned the all-too-common pattern of designers and researchers extracting value and expertise from their community without sharing any of the learnings or insights back. We wanted to honor their work by sharing the knowledge we built from it.

We wanted to honor their work by sharing the knowledge we built from it back to the community.

Collecting Facebook post data from a Facebook group without violating the terms of use of Facebook by using any scripting or automation is a tedious and non-trivially technical task. Facebook is not designed to load that many posts in a single view without crashing and it tended to crash mid-scroll. I literally had to set up the computer running my browser in a particular way to enable more ram allocation to Chrome and use a custom search query.

The number of posts in one month was on the scale of 10’s of thousands. I saved a total of about 600 that were potential use-related risks. I then carefully read through each, tagging and categorizing them as I went, developing a schema of level of risk and root cause in perception, cognition or action. After applying the same analysis to the survey results we ended up with approximately 117 use-related risks.

This research served as the foundation for an interdisciplinary risk-management team made up of clinical advisors, product, engineering and QA documenting the risk assessment for each and how we planned to mitigate each risk.

Many of the design revisions I worked on and highlighted in my Tidepool Loop Case Study came from the most critical risks. Knowing the real user stories behind these made a huge impact on being able to design mitigations for them quickly and effectively.

Branding

The product and marketing teams were simultaneously working on developing the brand of Tidepool Loop. While the design team had mostly been focused on the UX of the experience up until this point I did contribute to the decision on which agency to hire, their orientation to the existing products and brand, and to several key decisions for the visual design. We landed on a simple cartoony visual style lifted from Japanese subway signs. I lobbied that we should name our character Skippy. But I was unsuccessful beyond Design and Product. I was however pleased to see most of the layouts and visuals from my storyboards intact in the final illustrations. You can see them all in DIY Loop Onboarding.

Human Factors Testing

Finally, after another pass through the onboarding consistent brand colors, illustrations, and layout, we were ready to test whether our onboarding was sufficient to enable anyone with diabetes, regardless of prior use of CGM, pump, or iOS to use Tidepool Loop safely.

While our human factors testing process was dramatically impacted by the pandemic (see Remote Usability Testing), our study design was reliant on the in-app training as the primary source of training for all critical tasks identified by our risk management team.

We had study participants complete the in-app training as designed, wait 24 hours, and then complete critical tasks on an iPhone SE gen 2.

Results:

To our delight and relief, there were no misses on any high-risk tasks. And even when the study showed that users had cognitively misunderstood an aspect of what the app was showing or doing, their misunderstanding did not result in any unsafe actions on critical tasks.

Later, during our submission, the FDA reviewers specifically called out our HF testing methods and results as excellent.

Finally, and most impactfully for our ongoing work, the agency proposed no changes or refinements to our onboarding and in-app training.

One Sheet

The Larger Story

Continuous Glucose Monitors (CGM’s) are among the most powerful new technologies improving the quality of life and clinical outcomes for a growing number of people with diabetes.

And yet there are many situations like being in class, on a date, driving in a car, engaged in a meeting doing sports, when loud, repetitive alarms and disruptive screen-watching just don’t work. 

Ironically it is often at times when physical or cognitive performance matter most, CGM users have to choose between staying aware of their blood glucose and risking the distraction or embarrassment of a loud alarm calling attention to them among their classmates, colleagues, and friends.  Though some customization is available, the current paradigm often leaves CGM users chasing highs and lows and suffering alarm fatigue.

In our survey of over 100 cgm users we found that not one of them had not suffered some negative impacts on their relationships and social life due to CGM alerts, alarms and notifications.

But what if we shifted this vital data stream from audio and visual alerts to the user’s sense of touch? Subtle, ongoing tactile input from your wrist.

Almost imperceptible when you’re steady and in range, but slowly ramping up in intensity and differentiation as you move towards the boundaries of your target numbers.

This would allow you to stay on top of of your blood glucose without even having to look up from a task, or break eye contact with your date, or the road!

This new paradigm of ambient glucose awareness through your sense of touch would enable you to shift your focus back —to what you don’t want to miss.

The research suggests that in as little as 4 days, your brain will, unconsciously learn to recognize the haptic patterns associated with different CGM states and map those to a conscious, sensory awareness of blood glucose. And this happens without conscious effort or additional cognitive load. 

After a few days, the patterns will stop feeling like vibration and start feeling like a new sense as your brain begins to sense blood glucose changes through the CGM.  Importantly, this would work just as well for parents and caregivers. The sensory augmentation transplants the data stream from the person with diabetes through the CGM, and into the brain of the person wearing our device.

We started sketching prototypes for both haptic hardware and designing haptic patterns that communicate the most common movements of blood glucose.

We’ve also been in conversations about partnerships with companies developing hardware for a dedicated haptic device that would serve as the most robust incarnation of bgAWARE.

But hardware takes time. So we are actively developing bgAWARE as an application for the latest Fitbit smartwatches and for the Apple Watch. 

While more limited than the dedicated hardware device we are designing, this will allow us to assess the community interest in a new modality for CGM with existing wearable hardware before we have to the investment needed to get custom hardware to market.

And while some companies are more open with their data than others, we see this as a potentially transformative new interaction model for any CGM pushing an ongoing datastream, including Dexcom, Freestyle Libre with transmitter, and a number of new players about to hit the market promising less-invasive CGM’s.

Epilogue

This pitch won the Disrupt Diabetes Design Challenge, hosted by Stanford Med School and judged by Stanford, Medtronic, IDEO and more.

DiabetesMine 2018 Demo – Full Video

We demo’ed it at DiabetesMine!

Challenge:

Design a mobile app that tests the effectiveness of character growth interventions on teens
–but keep it fun enough so that they will want to use it.

The first challenge in this project was to make use of the rich body of positive psychology research evidence suggesting the value of certain actions and practices for emotional health. But often those interventions, while well researched and peer-reviewed, aren’t presented in ways that make them easy to use for the young people who most need them.

A deeper challenge that quickly emerged after engaging the research is that the process of building patience, self-control etc. require the spending of self-regulatory capacity —something we all have a finite amount of in a given day. The research suggests that you can grow your capacity for self-regulation but, just like building muscle, the only way to do so it through exercising it.

Given that the exercise of self-control is an inherently high friction experience, how might we design experiences that mitigate that friction and encourage users to keep coming back?

My Role(s)

I served as:

• user researcher

• lead designer of UX and UI

• product owner & project manager

I also contributed to branding and identity in designing both the logo and the app name to ensure easy search-ability on both web and app stores.

PROCESS

Literature Review + User Surveys

While reading through a literature review prepared by my collaborators, Dr. Sarah Schnitker and Dr. Benjamin Houltberg, I prepared a simple survey to be sent out to students in schools who had expressed interest in participating in the research study.  While immersing myself in the positive psychology research literature, looking for aspects of interventions we could translate into games, I wanted to start filling in some demographic data on our high school teen users.

Some kinds of interventions were already on the minds of our researchers so we went ahead and added them with a 1-5 scale from least to most interested to see if any experiences would show up as intriguing.

App Research

One of the critical pieces of data from the user survey was the list of apps they were already using on a daily basis.  We wanted the interface to immediately feel familiar and intuitive.  By starting with clear examples of apps they already live in, we avoided any unnecessary novelty that might impede easy use.

We borrowed the simple, bottom nav from Instagram and the bright, playful colors of Snapchat along with its whole-screen swiping gesture as a secondary navigation.

One aspect we did not borrow was the deliberate use of social reward to create addictive patterns in our users.

By now it has been widely reported that a small group of behavioral scientists at Stanford have had an outsized influence on the development of some of the most used most addictive apps.  By applying insights from neureo and social science they have mapped a kind of set of “best practices” for creating addictive experiences.  We resolved early not to follow these dark patterns in our design.  We were aiming to create experiences that were intrinsically valuable and rewarding without using manipulation to do it.

User Interviews

Armed with this preliminary work we had some clarifying ideas of which games and interactions we thought were promising.  We took that list and started visiting schools.  The research team had already cultivated positive relationships with some teachers at a variety of local public and private high schools so we started visiting technology classes and lunch hours –really anytime the teachers could invite kids to their class and give me the floor.

I began by asking them to open their phones and go the settings screens that showed what version of iOS or Android they were running.  I then walked around snapping photos to share with our developer.  Next, I described the app’s goals generally and started asking for a show of hands on which features seemed interesting.  As people responded I began to engage them one on one to talk more about why that feature or game sounded interesting.

The Importance of Language

One feature I was particularly interested in was one that came from the scrolling card view. I imagined a stack of conflict resolution strategies that any of these kids could pull out and use to help figure out better ways of resolving persistent conflicts.

“How many of you get into conflicts regularly?  Show of hands.”

*crickets*

“No one?  No one has conflict with their parents?  Teachers? Friends?

A few girls shyly put up their hands.

“I can’t believe this.  You guys must be some of the most peace-loving teenagers ever.”

“Do you mean, fights?” One young man asked.

“Yes fights.  How many of you get into fights?”

*smiles* *all hands go up*

“How many of you would be interested in an app that had a stack of cards, each with a new strategy to try to solve a conflict?”

The room erupted with interest and narratives about the persistent recurring conflicts they have with parents, teachers, friends and significant others, often stemming from misunderstanding.  This feature which had been an afterthought to the rest of the interventions was the aspect they were most interested in trying.

This is an excellent example of the power of user-interviews and the importance of paying attention to hints and clues in the responses of real people who are users or represent their interests.

Pencil and Paper

I like to begin the mock-up process with pencil and paper at 1:1 scale.  I prefer detailed and accurate text in mock-ups as the process of thinking through text and labels often reveals issues and problems in information architecture.

 Meet, Discuss, Approve

The pen and paper mock-ups allow the key stakeholders to have something visual and narrative to respond to.  It helps to build the mental model of the app in the minds of the team and surface assumptions *before* a pixel or line of code have been committed.  This level of detail is also what enabled my developer partner to begin laying the groundwork for the app.

High Fidelity Mock-Ups

I prefer the vector drawing tool, Sketch, for high fidelity mock-ups.  Once I had completed each screen and some keyframes for animations I could then share the sketch file with my developer sharing actual colors, typography, CSS and layouts down to the pixel.

Programming Begins

The pencil and paper mock ups enabled development to begin simultaneously to the high fidelity mock-ups.  We kept in regular contact as questions emerged and shared visuals back and forth which enabled me to confirm that the details and atmosphere of my design vision were coming through in the finished product.  Because of our use of a hybrid native and web architecture, I was able to load views of the app in the browser during development, long before we were able to compile the app into iOS or Android.

A Word on Naming

Up until this point we had used the working title “Thrive App.” But as we prepared to deploy to the app stores we needed a title with a cleaner search and some brand identity distinct from the organization overseeing its development.  Before presenting the question to our stakeholders I did some quick and dirty user research by listing our top five name choices in a direct message on instagram to a small group of teens in our target audience.

The response was near-unanimous: CharacterMe was their favorite.  It also happened to have the least overlap with any search on google search, iOS and Android app stores.

I’m a big believer in some research being better than none.  And the research you can execute in the time-frame you need to help validate a decision with the target audience is more valuable than a more rigorous or higher sample size that comes after you need it.  Jake Knapp in his book outlining Google Ventures’ user testing process agrees and underlines the diminishing returns on insights once you go beyond five user testers.

User Testing, Pilot Study

We knew that the iOS app store had a reputation for rejecting apps with little notice and feedback, we targeted the iOS app for deployment first.  To our great surprise our app was approved on the first attempt after one week.

Next we deployed to the Google Play store.  Once the two apps were available for download we went ahead with a pilot study with one school as a beta test to discover any remaining bugs and to work out the onboarding process.

Research Phase

Study Groups + User Interviews

Over the course of two years the research team conducted study groups at a wide variety of schools in southern California representing a mix of public, private and socio-economic status.  Each study consisted of two weeks of daily use prompted by the app.

After each study concluded I met with as many of the students in person, as a group, on-site at their schools to interview them about their experience.

In general, they found the app intuitive and helpful.  70% of students polled said they would continue using it after the study concluded and that they would recommend it to friends.  They listed the prompt to consider and record their emotional states and the conflict resolution strategies to be the most valuable activities in the app.  The timer based activities were among the least favored.

Future of the Project

Presentations & Publications

My collaborators, Drs. Sarah Schnitker and Benjamin Houltberg continue to analyze the data captured in the research phase.  We hope to publish the findings of the research study, as co-authors, later next year.

Further Development

Since beginning the project we have been invited by two separate funding bodies to write proposals for further development of CharacterMe for a public audience.  Since the future of the app is less about research and more about the technology development and design, I took the role of primary on one of the grant proposals, wrote the project plan, budget and full narrative of the proposal.

The Thrive Center is currently hearing proposals for funding partnerships.

This project at the intersection of social science research and building technology that aims to help us be more human is exactly where my professional interests lie.

Building with the tools of technology enable us to deliver and test all kinds of new interventions that might prove beneficial to all sorts of people.  The tools of social science add scientific rigor to test and assess whether what we’ve built is actually doing what we hope or having another outcome.  The power of this iterative process toward building new knowledge and new experiences has only just begun to remake our world.  With a bit more rigor and attention, we can steer that remaking in ways that support and reinforce our best human impulses.