Case Study

Predictive Diagnosis

Scalable, interactive algorithms improve speed-to-answers

The Basics

Pathways is an interactive decision tree that helps clinicians find answers to common medical situations.

Users answer numerous “Yes or No” questions until ultimately selecting a treatment option.

User Problems

Traffic data showed that new customers were trying out Pathways, but there was little repeat usage.
User interviews revealed that many clinicians using Pathways (which requires an additional subscription) took longer than using the base UpToDate product.
Because of the yes/no format, users often answer up to 15-20 questions to get an answer.
Recorded user session data shows that Pathways takes about 3 minutes to use. UpToDate takes about 1-3 minutes.
Repeat users said they liked using Pathways to reinforce their decisions, or to find specific dosing information for a particular treatment.

Traffic data showed that new customers were trying out Pathways, but there was little repeat usage.
User interviews revealed that many clinicians using Pathways (which requires an additional subscription) took longer than using the base UpToDate product.
Because of the yes/no format, users often answer up to 15-20 questions to get an answer.
Recorded user session data shows that Pathways takes about 3 minutes to use. UpToDate takes about 1-3 minutes.
Repeat users said they liked using Pathways to reinforce their decisions, or to find specific dosing information for a particular treatment.

Hypothesis

Most users coming to a particular Pathway were likely trying to answer one of a few common questions. If the team could solve the 90% use-cases in a much faster experience, repeat usage would increase.

Project Priorities

Record user sessions to see how customers are interacting with Pathways
Find areas to increase speed-to-answer and reduce the number of clicks needed

UX Team Members

UX Designer (me)
UX Researcher
Editorial Staff (practicing clinicians)
Product Manager

Early Designs & User Reactions

Early iterations featured different widgets to display the three most common treatment options.

Further Exploration

To prove scalability, the editorial team (practicing clinicians) authored what they thought were the three most common outcomes for different Pathways.
They found that 41% (29-70) of Pathways could be quickly edited to have these common outcomes — with more added over time.
An early, low-fidelity design was created for the purpose of gaining quick user feedback.

Research

Using Pendo, the UX researcher build the first trial version of what was called “Rapid Answers” for the UTI Pathway.

Using FullStory, the team was able to watch user sessions to add context to how users were answering the Pendo form.

Data started to show early signs of validation to the hypothesis, as a small sample size (34 users) responded to the Pendo campaign.

88% of users answered that they were coming to the Pathway for a specific reason.
94% of those users came to the Pathway for one of the reasons the editorial team listed.

Prototypes for Feedback

With the momentum gained from the research experiment, further iterations and a simple clickable prototype were born.

The team decided to expand the reach of this feature, and built a prototype within UpToDate itself.

Findings

Using the search term “hypertension,” this one specific sample showed that an answer could be found in under 40 seconds.

Using UpToDate took our editors over 3 minutes to find the same information.

Commitment to Testing Other Design Iterations

The team launched an experiment through Optimizely to test further iterations.

Rapid Answers for an algorithm-only variation

Rapid Answers in the right-side text panel

Current Status

While the experiments showed that users found their answers faster using the Rapid Answers feature, the status of this project was pending based on the direction the Pathways product when I left the company. Having had conversations with former colleagues, this project led the way for more prescriptive and "smart" work throughout the entire UpToDate platform.