Loading...
The Usability Blog
A Practical Guide to User Experience Insights

Usability Testing Healthcare.gov

It’s a topic that has dominated the headlines everywhere – all the way from trusted news sources like MSNBC, CNN, and Fox News to comedy programs like Weekend Update on Saturday Night Live – there are issues with the recently launched Healthcare.gov website. During the first month it was open for enrollment, 106,185 customers signed up for insurance, and about a quarter of those did so via the marketplace (healthcare.gov) falling short of the original goal of 500,000. While much of the early focus has been on the site being down, perhaps even more important are the critical usability issues with the site. In a recent speech, the president himself admitted to the need for user testing. Rather than simply lamenting the problems with the site or, to paraphrase another great president, asking what our country could do for us – we at Usability Sciences decided to focus on what we could do for our country. In this case, it’s what we do best – usability testing – and we decided to put Healthcare.gov under the microscope to understand how target market users interact with the site, determine if it meets their needs and identify any usability issues. We recently ran a usability study with 6 users. While we found a host of usability issues – some obvious and others less so – we’re going to stick to our most critical findings here. Critical issues with Healthcare.gov include the following:

  1. Users were unclear how to begin, most wanted to start by viewing plan prices and details, but struggled to figure out how.
  2. The critical path was hidden in text-heavy pages and too many steps were required to access available plans.
  3. The health care plans shown lacked key details needed to make a decision, such as deductible, co-pay and prescription medication costs.
  4. The strict username and password requirements were difficult for users to understand.
  5. Error messages were unhelpful and frustrating.

1. Users Unclear Where to Begin

Our testing found that most users want to begin their interaction with the site by browsing available healthcare plans. Users wanted the opportunity to see what the site and the plans were all about before making any commitment to apply for a plan. The expectation was that the user could enter a few small details about themselves, such as age, location and income, and be shown a variety of plan options, similar to the way TheHealthSherpa.com works. (Ed. note: Health Sherpa was created by a team of three developers who noticed this gap in the user experience on Healthcare.gov. They pulled the code from Healthcare.gov, fixed it up, and released it to the public, free of charge.) Unfortunately, that was far from the reality.

Unsure-where-to-begin-1024x566

 

The homepage is set to default to the ‘Learn’ tab, with a second tab option of ‘Get Insurance’ located to the right. The ‘Learn’ tab, is overloaded with places to click, and icons to interpret, leaving most users unsure where to start. This is a clear violation of Jakob Nielsen’s 10 Heuristics for User Interface Design (aesthetic and minimalist design). The result is that the call to action button for ‘See Plans Now’ is so far down the page, it is completely below the fold at lower screen resolutions, and even at higher resolutions, the button is small, lacks distinction and was often difficult for users to locate. The call to action buttons users noticed first were ‘Apply Online’ and “Apply by Phone’, however as previously noted, users wanted to see plans before determining if applying was right for them. No users wanted to apply before seeing plans. Additionally, applying, either by phone or online, are both options that are better suited to the ‘Get Insurance’ tab, as opposed to the ‘Learning’ tab.

2. Streamline Steps to Access Plans

Once located, clicking the ‘See Plans Now’ action button took the user to a text-heavy page with no clear indication of where the plans might be viewed. The call-to-action buttons most prominently displayed on the page are ‘Getting Lower Costs on Coverage’ and ‘Health Insurance Marketplace’.

Critical-Links-1024x566

Users were required to read (and as research shows, users do not read) a text-heavy page to locate a hyperlink, buried within the text, that reads ‘plans and sample prices’. Once they located the link and clicked through they were met with a blue box of text (which had very important information about how the estimates work), and a single question to answer.

Most users did not read the text in the blue box (on this page, or the following pages on which it appeared), and were confused as to why the page only contained one question. 6 questions were asked over 4 pages, all containing the blue text box, followed by another 3 pages of text, and finally a selection of plans to look over.

3. Plan Pages Lacking Details

However, once users finally navigated to the health care plan pages, there wasn’t really anything to go over. The only information provided was the name of the plan and the estimated monthly cost. Nothing else.

lacking-details

 

Users expected to be able to click through or hover over the plan names and have more information made available. Users stated unequivocally that this was not enough information to proceed with purchase.

4. Very Strict Username and Password Requirements

The username and password requirements are extensive, garbled and complicated. According to the directions on the site, user names must be “6-74 characters long, and must contain a lower case or capital letter, a number or one of these symbols _@/-“. Passwords must “contain 8-20 characters, and there must be at least 1 uppercase letter, 1 lowercase letter, and 1 number” and “cannot contain your username or any of these characters = ? <> ( ) ’ ” / \ &”. It requires far too much thought to come up with an acceptable username and password, and many users had to attempt multiple times, greatly reducing the chance they will be able to recall their username and password. Allowing users to sign up using an email address, which is a unique identifier, would alleviate user error and give users something easy and familiar to remember on their next visit. Additionally, relax the requirements for password creation, and in the event that it is not possible to allow users to use an email address as their username, make sure the requirements for password and username match.

5. Unhelpful Error Messaging

errors

There are six error messages total on the page to let the applicant know that ‘mbruce’ is not going to work, but none of them mention WHY. With the strict username and password requirements, it is important to let the user know exactly in what way their choice has failed to meet the requirements. The error messages should contain language identifying the issues and facilitating correction. As is, the error messages simply frustrated users.

The issues plaguing the site may seem overwhelming, but it isn’t uncommon for a website to go through several (if not infinite) iterations, testing between, and getting better and better for the user each time. The government could have avoided many of the issues currently plaguing Healthcare.gov by simply conducting user experience testing on a small group, uncovering issues before the site went live, ultimately avoiding quite a bit of embarrassment. User testing, it does a website good.

-Authored by Usability Analysts Matt Bruce, Erin Johnson and Priscilla Lim

BECOME A PAID TEST PARTICIPANT

Sign up to become a Paid Test Participant.

Sign UP Now

We have revised our Privacy Policy
as of August 19, 2014

CLIENT TESTIMONIALS

“From beginning to end, everyone I interacted with from Usability Sciences was professional and thorough. I was impressed with the testing technology, the methodology and especially the team that led the project. This is one of the most impactful pieces of research I have ever delivered to my team. Thank you!”

Kevin King
Senior Director of Digital Media, A&E Television Networks

“USC managed tight timelines and a client team that was tough to wrangle, But more importantly, the quality of the work was exemplary. It's work I would hold up as "the way we should do things" and share as a case study across the organization.”

Group Product Director
Digital Marketing, Pharmaceutical Company