So much has been made of the role of the Internet in the 2008 presidential election. In particular, the Obama campaign made some impressive claims about the success of BarackObama.com in fund-raising and organization of supporters. Even on the most superficial level, the two candidate sites have a lot in common in terms of structure and content. But, to our eyes, there are many differences in terms of the design approaches taken by the two organizations. We wondered how ordinary voters might see these differences, so we conducted a comparative usability study of the two sites.
As the 2008 Presidential election approaches, much is being made of the increasing importance of the Internet as a communication channel between the candidates and their supporters, donors, and undecided voters. The prevailing wisdom at the moment (and the focus of much of the press coverage on this topic) is that the Obama campaign is doing a better job of leveraging the Internet to reach large numbers of people who might be interested in donating money to the campaign and participating in grassroots campaign activities. The Obama website is, at least, attracting more press attention. In just the past month, media mentions of “barackobama.com” outnumber “johnmccain.com” 431 to 80. But does this prevailing wisdom actual reflect the real-world impressions of potential voters?
When we at Catalyst Group tested the two candidate sites against each other, one of the surprising results was that most people found the McCain site easier to use than the Obama site once they had spent time on both. This was the case, even though virtually all the test participants initially responded much more positively to the Obama site’s design. Does this result suggest that visual flair and ease-of-use are not totally compatible? Does achieving some laudable goals (such as better usability or navigation) sometimes work against or “crowdout” other important branding or design goals? It’s a common debate in the web design field – essentially whether aesthetics trump function or vice versa. The question has particular poignancy in the case of political websites that need to be both informative regarding a candidate and his / her positions and reflective of the candidate’s “brand.”
The results of our research indicated that aesthetics play an important role in users’ evaluation of a candidate’s site, but that ease-of-use may ultimately carry the day as the quality that people value most.
As user-centered design and usability practitioners, we at Catalyst are trained to look beyond the surface of a site’s design and ask whether the design actually satisfies the core user and business objectives that it is intended to address. Given the apparent consensus (at least in the media) that the Obama site was far superior to the McCain site, we wanted to find out for ourselves whether one site was actually more successful from a design perspective than the other. So, we arranged a comparative Usability and Eye Tracking study to learn more about how voters would respond to the two sites. We conducted the testing at our Usability and Eye Tracking Lab in New York, using the same proven research methodology that we use for all of our client engagements.
We recruited 15 “undecided” voters with a mixture of demographic characteristics and an even split of Republican / Democrat voting history. Test Participants were interviewed, one at a time, according to a pre-determined script. However, spontaneous reactions and diversions from the core tasks were encouraged in order to capture a wide range of user feedback. The script focused on a small set of core site tasks on both sites (the order of presentation of the sites was alternated in order to avoid any order bias). The site tasks were:
- Initial Homepage and overall site design impressions
- Locate the Candidates’ biographical information
- Find the details of the Candidates’ tax plans
- Make a campaign donation
- Get the latest campaign news
- Sign up for future campaign news via email
- Locate an event where the candidate will be appearing in-person
- Volunteer / get involved with the campaign
Finally, we captured and analyzed participant Eye Tracking data during participants’ initial explorations of the two candidate homepages.
Note: The following images indicate the key areas and functions on both sites that are referenced in this report.
Barack Obama’s site made a much better first impression than John McCain’s site, mainly due to the Obama site’s visual design (particularly the homepage). Participants also immediately felt that the Obama site was more “youthful” and “modern” than McCain’s site, which was deemed too “boxy,” dark and crowded. Eye Tracking data further showed that users were confused by the visual appearance and layout of the McCain homepage. By contrast, Eye Tracking on the Obama homepage demonstrated that the test participants were likely to quickly focus on the same high-priority functions, such as the Issues menu, the main content “Flipper” and the Make a Difference functions.
However, once test participants moved past their initial reactions to the two sites, they began to “warm up” to the McCain site – particularly the more straightforward navigation and clear labeling. When asked to accomplish a variety of core site tasks, most users found the tasks much easier to complete on the McCain site.
In the end, a clear majority of users preferred McCain’s site to Obama’s. Of the 15 users tested, 11 (5 Democrats, 6 Republicans) favored John McCain’s site while 4 favored Barack Obama’s (2 Democrats, 2 Republicans).
At first glance, several users (equally Democratic and Republican voters) responded favorably to Barack Obama’s Homepage, especially its visual design. The light blue “warm” colors, abundant use of videos, animation, and interactive modules (especially the ‘Social Networking’ functions) gave users the impression that Obama’s site was more “youthful” and “modern” than John McCain’s. In fact, initial impressions of the McCain site were generally less positive overall than Obama’s. Users said that they were not particularly impressed by the visual presentation of John McCain’s Homepage, which seemed more “antiquated” due to its dark colors and “boxed off” areas. Interestingly, though, these same users considered the McCain site to have a more “presidential” appearance – as if they perceived an inconsistency between a more polished, youthful presentation and one that is appropriately serious and presidential.
The Eye Tracking “Heat Maps” of the two homepages reveal two very different patterns of attention. These graphics aggregate the relative attention of all the tested users. Areas in red indicate portions of the page that received the highest visual attention relative to other page areas.
This is an excerpt from an August 2008 report, download the full version here.
- New York, New York
- User-centered research & design firm,dedicated to helping companies connect their business objectives to the needs and expectations of their customers