Results from a usability study of Moxie Employee Spaces with 10 Community Managers.
In most organizations Microsoft SharePoint is installed by IT organizations as a collaboration and document management tool, however adoption by business users has been low. Usability Resources Inc. conducted an independent study sponsored by Moxie Software to identify the key elements that drive adoption of collaboration technologies, and to learn the importance of a user-centric vs. document-centric design in a social computing platform.
Usability Resources Inc. recruited ten Community Managers from medium-sized to very large organizations, all using customized versions of SharePoint’s collaborative features. We focused on Community Managers because they handle training and customer support and work closely with teams to launch and sustain employee-led online communities.
We looked for Community Managers who had been involved in selecting a social computing platform for their organization, and who were responsible for collecting and reporting ROI metrics to senior executives.
The study was designed to give each of the Community Managers–none of whom had prior experience with Moxie Spaces–a solid grounding in its key features. The Community Managers watched a 25-minute training video on Moxie Employee Spaces and then performed a series of tasks that replicated the initial user experience with the product. Using an out-of-the box version of Moxie Employee Spaces, participants created a login account, used the wizard to complete their rich profile, explored their home page and default account, joined and explored groups, participated in ideations, viewed other’s rich profiles, and searched and browsed for content as well as for individuals with particular expertise.
Adding collaborative features to SharePoint is costly
The participants said that SharePoint is reliable, integrates well with their IT infrastructure, and aligns with their company’s workflow; they also noted that it carries many hidden costs and requires IT intervention. These costs include the need for IT intervention in creating and maintaining communities, as well as the necessity of costly customizations to make SharePoint collaborative and understandable to nontechnical audiences.
SharePoint primarily searches documents, vs. people, conversations and other social objects that hold a vast amount of expertise and information. SharePoint’s hierarchical structure, which one person described as “cold and mechanical,” discourages learning about the people you work with and how they are interrelated. Because SharePoint is document-centric (vs. people centric) you cannot easily leverage people and groups to find information.
If one does not know exactly where to look and the precise keywords to use it can be hard to find information. Here are are some quotes from our participants that elaborate on this point:
"Our users tend to be older and do not have the skills or patience to deal with SharePoint’s many hurdles. No one wants to touch it, which has affected our bottom line." Maggie, Professional Society
"Sharing ideas is an itegral part of what scientists do. We had to put in a separate ideation platform because SharePoint was lacking one" Brooke, Biotechnology Engineering
Adding Moxie Employee Spaces social layer is a cost effective way to optimize SharePoint data
On the other hand, participants found Moxie Software Employee Spaces powerful and intuitive. They thought of it as a social layer on top of SharePoint that would greatly enhance collaboration and makes it easy to find information. After using the product for just a few minutes they were able to perform complex operations such as:
- finding people with particular expertise
- creating and joining groups
- locating specific content
- participating in brainstorming sessions
- finding others with a common interest
- and learning about their office location.
Participants stated that navigation was simple, and noted that the one-click access to most operations would make it easy to get involved. Participants were especially impressed by the ease with which they were able to find information and how Spaces integrates social elements with a traditional content search.
“Spaces environment offers robust employee engagement that is beautifully integrated with document management." Dale, HR Manager, Higher Education
“This product's value is immediate. I know if I don't see content I can search groups, then people, or pose a question directly to a community of experts." Brice, Senior Manager, Healthcare
"The profile makes it easy to search for employees with particular skill sets which would transform our internal recruiting process." Alex, HR Manager, Temporary Staffing
Participants rank the features that drive adoption of an employee online community
Group functions are seen as most critical
We asked participants to rate on a scale from 1 to 5 the impact that particular features have on employee adoption of a collaborative community. The ability to post group messages came out on top, followed closely by the ability to set up communities of interest; the ability to comment, like, and share; having access to detailed personal profiles, and having built-in community administration (self service). We then asked the Community Managers to rank SharePoint’s and Moxie Employee Spaces’ ease of use for each feature. The following chart summarizes the results.
The platform’s ability to foster enhanced work relationships is important
After finishing the usability study, the participants noted that the human factor design of Moxie Employee Spaces, which was developed in collaboration with IDEO, “makes people human,” which leads to an enhanced team feeling. People mentioned how Space’s design “makes it personal to them,” allowing the user to feel a part of a community with others who share common interests.
Employee Spaces was seen as a different type of tool–the next step beyond a document management system that makes an organization’s information assets more accessible by leveraging people and relationships. Space’s uniqueness theme came across from the results of the Product Reaction Card survey where participants were asked to quickly select 5 attributes from among 118 choices that are balanced between positive and negative. The attributes most often chosen for SharePoint were “integrated,” “businesslike,” “collaborative,” and “efficient.” Those chosen most often for Employee Spaces were “engaging,” “useful,” “easy,” and “creative.”
Participants gave Employee Spaces superior usability ratings
We asked each participant to complete the System Usability Scale for both SharePoint and Moxie Employee Spaces. The System Usability Scale consists of ten questions on key aspects of usability. The survey produces a score between 0 and 100. Scores of less than 60 indicate poor usability. When evaluating SharePoint, four out of ten participants’ SUS scores fell below 60. However, when they evaluated Employee Spaces scores from all ten participants were above 75.
Participant reaction to Moxie Spaces’ product design
Good design increases employee adoption while lowering training costs
Participants saw SharePoint as a tool for managing projects and documents vs. a platform for self-managed communities. SharePoint has a power user interface that takes less technically-inclined employees months to learn. Even though using SharePoint is mandatory for their jobs, many people find it intimidating and don’t enjoy using it.
On the other hand, navigation within Moxie Employee Spaces is easy and intuitive because the information is well integrated. Participants immediately saw the platform’s value and felt it was usable “right out of the gate.” It was easy to find people and follow links to learn about related and recommended content. Users thought having a lot of related information on a page enhanced usability because the content is clearly separated, thus reducing the number of mouse clicks.
Participants also remarked how Employee Space’s design incorporates concepts from well-known products such as LinkedIn and FaceBook, which minimizes training while enhancing adoption and productivity. Spaces leverages social media concepts such as Follow, Like, Activity Feeds, Profile, and Share that are part of the common vernacular.
Rich Profiles help people learn about each other so they can work together more effectively
Our participants saw the Spaces Rich Profile as its most compelling feature. They were impressed to see that by viewing a co-worker’s profile they could learn about that person’s biography, current postings and status, history, expertise, projects, network, social media contributions, and much more. Several remarked how difficult it can be to get people who do not know each other to collaborate on projects, and saw the Rich Profile feature as an easy way for co-workers to get a clearer understanding of the people on their team. This would be especially valuable for global and geographically dispersed teams who never have the chance to meet each other face to face.
Wizard encourages users to complete their profile
For a number of reasons, participants mentioned that employees in the communities they manage tend not fill out their profiles. They assumed people don’t see the value or may be uncomfortable giving personal information in a work situation. The net result is that the community is used primarily for project work and document management rather than as a platform for employees to come together to share ideas, resources, and expertise.
When an employee uses Moxie Employee Spaces for the first time, their first order of business is to complete the profile wizard, which asks them to upload a picture and give basic information about their expertise and their back-ground. As the employee joins groups, works on projects, follows other people, and contributes to ideations, Wikis, blogs, discussions, and so forth this activity is noted on their profile. Over time the system constructs a well-rounded profile that gives a sense of who that person is in the organization that requires very little effort to construct. This profile can be searched and browsed and becomes a primary mechanism for learning about people and how they fit into the work environment.
Social search and browse reduces information overload and makes it easy to find content and expertise
To find information in a typical document-centric collaborative platform, an individual usually needs to know the right keywords and have a sense of the repository’s structure. They do not have the ability to leverage people, groups, or relationships to find what they need. Document management systems such as SharePoint are often geared more towards technical audiences than toward the business user, and each group has its own terminology and “mental model” around the search topic. For this reason browsing and searching can be time-consuming, tedious, and hit or miss; and there is no chance to learn about the people in the organization who are experts on a given topic through this process.
Our participants were impressed by how Moxie Employee Spaces’s socially-oriented design efficiently leads them to the right people, groups, and information. Users can filter content through tags, likes, trending topics, recommendations, communities of interest, and individuals. Any search–whether on a keyword or for best practices or expertise--searches through the document repository as well as through user-generated objects such as discussions, ideation sessions, blogs, wikis, and rich profiles. Participants noted how Moxie Employee Spaces’s design would help them more quickly locate the best content and minimize information overload by focusing on content that has been favorably reviewed by acknowledged experts.
Ideation platform encourages free exchange of ideas and is a quick and easy way to poll all employees
Being able to run a quick poll with employees or to vet ideas and brainstorm with teammates can be challenging for organizations. Several of our participants noted how hard it is to gather simple feedback on ideas - to for example launch surveys in their companies. They often resort to using survey tools such as WebMonkey or asking employees to e-mail responses to an individual.
While they felt brainstorming and ideation is often more successful in a face-to-face meeting, they also noted its limitations in their company’s environment. Valuable input is lost when employees feel uncomfortable disagreeing with their superiors and colleagues. They might refrain from contributing an offbeat but brilliant idea because they fear looking silly or need more time to mull it over before “going public” with it.
Moxie Employee Spaces offers blogs, wikis, discussion forums, chats, and several other methods that allow employees easy ways to contribute ideas, evaluate them, and make decisions. One feature that participants were able to test drive at length in our study was the “IdeaStorm,” The IdeaStorm is an online bulletin board for presenting ideas to which colleagues can comment or give a “thumbs up” by pressing “Like”. Along with other socially-generated content, IdeaStorm submissions appear in Space’s search results, revealing ideas that have already been considered as well as individuals and groups with an interest in and passion for those topics. Employees can share an idea through an IdeaStorm or monitor its progress to gauge colleague’s reactions.
While reviewing IdeaStorms our participants were impressed by how easy it is to contribute ideas and comment on others’ ideas. Participants who also had HR duties noted how they would use IdeaStorm as a platform for quickly launching company-wide surveys. Most felt the IdeaStorm feature provides a simple but effective forum for employees to weigh in on team decision-making. They also saw it as a great brainstorming tool for virtual and geographically distributed teams, as well as a feature that would drive adoption. It is simple to use and helps employees to easily engage with colleagues around developing and refining ideas that move projects forward.
Many saw Moxie Spaces’ emphasis on location as providing a bridge between the virtual and face-to- face work environment. Having a location community which employees can reference to find directions, read reviews of restaurants and local businesses, read employee profiles and gather information on initiatives at the local office would encourage people to use the platform. The Location page would be greatly useful when visiting field offices or to learn about newly acquired offices during a merger and acquisition.
Participants especially liked the calendar widget, which can be used on a Location page as well as within any project or group. Within a Location page, it would help employees keep track of important events and deadlines that pertain to their home office.
Most of our participants were responsible for communities whose members live all over the world. They liked how Moxie Employee Spaces integrates expertise with location, which would facilitate collaboration and sharing among their field offices.
This study involved 10 seasoned Community Managers who work on highly customized SharePoint communities. Using an out-of-the-box version of Moxie Employee Spaces they were able to create a profile, orient themselves to their new company and office location, join and contribute to groups, find people with specific expertise to help them in their work, participate in ideation sessions, and search for content.
All participants were impressed with Moxie Employee Spaces and saw the value it would bring to their SharePoint installation by driving up collaboration and knowledge sharing. Spaces intuitive design makes it easy for employees find experts and connect with people to share knowledge and collaborate, and to find information by searching through people, groups and content. Most felt this capability would transform their organizations.
Kay Corry Aubrey, Qualitative Researcher and author of this study Kay Corry Aubrey is the owner of Usability Resources, which specializes in user-centered research and design. Kay has over 20 years of experience in applying qualitative research methods and usability testing to technology-oriented products and collaborative software. She has led user research and usability and design efforts for dozens clients including AT&T, Affinnova, Constant Contact, Monster Worldwide, the Massachusetts Medical Society, the Mayo Clinic, and iRobot.
Kay has taught at Northeastern University and Bentley University’s Center for Human Factors and Information Design. She is the managing editor of the QRCA VIEWS magazine, a market research journal that is read by over 5,000 qualitative research consultants and buyers. Kay has an MSW from Boston University’s School of Social Work, an MS in information systems from Northeastern University’s Graduate School of Engineering, and a BA from McGill University. She is a RIVA-certified Master Moderator and enjoys doing research with both groups and individuals.
Usability Resources Inc
- Bedford, Massachusetts
- We help our clients create products and services that are easy to learn and use through expert user experience research and usability testing.