Testing your WordPress site’s usability doesn’t need to be time-consuming or expensive. You can easily do it yourself if you have the right mindset.
The Right Mindset for Testing Your WordPress Site Usability
Be open. Be humble. Observe, but don’t dominate your tester. Accept what they say and observe what they do without argument. Think of yourself as a fly on the wall of a user in their own home or office trying to use your site. Be ready to have all of your assumptions challenged.
Finding Your Testers
Your testers should reflect your users. If you have an ecommerce site that sells women’s jewelry, don’t choose elderly men as your testers. You’ll want women (of course) in 2 or 3 age groups. A website about tourist activities for children should have testers who are parents of young children.
I discourage using relatives as testers because the resultant testing will get too personal. Try to use actual customers or strangers. Send a call out on Twitter or Facebook.
How Many Testers Do You Need?
Steve Krug, in his SxSW talk “Rocket Surgery Made Easy” claims you need no more than three. Shane Pearlman, in Smashing Magazine’s article “Help Us Help WordPress” agrees. He found that by the third person his group was identifying where the tester would have trouble. Shane followed Steve Krug’s suggestions and found that his team spent 6 hours total testing their Events Calendar plugin broken up as follows:
- Set-up – 1 hour
- Testing (3 testers) – 3.25 hours
- Notes – 45 minutes
- Review – 1 hour
Define Your Scope and Purpose
Usability.gov has some useful information about defining the scope and purpose of your test.
Scope asks “what are you testing?” Are you testing the navigation? Content? Advertising? Search functions? A combination of these?
Purpose narrows down your “concerns, questions or goals” for the test. This can be quite broad, such as “Can users navigate to important information from the home page?” or specific: “Can users easily find the price of a certain product”?
Choose Your Device(s)
Will your testers be using desktop or laptop computers? Smartphones? Tablets? What screen sizes? What browser(s) do you want to test?
I suggest doing separate tests for desktop/laptop computers and mobile devices. If your site is responsive (and it should be) users will see different versions of the site and it will affect the conclusions that are drawn from the testing.
Write Your Scenarios
WordPress site usability testing consists of giving each tester 8-12 tasks to complete. Each task is a scenario, and is a brief description of a problem or issue that a typical users of your site may have.
You want to write your scenario in general terms, perhaps with some background information or context which gives the tester a chance to role play. Some scenarios I might use for Beyond-Paper.com might be:
You are the owner of a WordPress website and are looking for information about WordPress, as well as help making changes to your site. Please complete the following:
- Sign up for Beyond Paper’s newsletter
- Find an article about security issues in WordPress
- Find examples of Beyond Paper’s previous work
- Find out what services Beyond Paper offers
- Find out if Beyond Paper accepts payment online.
- Make contact with Beyond Paper
Read NNGroup’s article about creating task scenarios for more information.
Identify the Data You Want to Collect
Successful Task Completion: Each scenario requires the participant to obtain specific data that would be used in a typical task. The scenario is successfully completed when the participant indicates they have found the answer or completed the task goal. In some cases, you may want give participants multiple-choice questions. Remember to include the questions and answers in the test plan and provide them to note-takers and observers.
Critical Errors: Critical errors are deviations at completion from the targets of the scenario. For example, reporting the wrong data value due to the participant’s workflow. Essentially the participant will not be able to finish the task. Participant may or may not be aware that the task goal is incorrect or incomplete.
Non-Critical Errors: Non-critical errors are errors that are recovered by the participant and do not result in the participant’s ability to successfully complete the task. These errors result in the task being completed less efficiently. For example, exploratory behaviors such as opening the wrong navigation menu item or using a control incorrectly are non-critical errors.
Error-Free Rate: Error-free rate is the percentage of test participants who complete the task without any errors (critical or non-critical errors).
Time On Task: The amount of time it takes the participant to complete the task.
Subjective Measures: These evaluations are self-reported participant ratings for satisfaction, ease of use, ease of finding information, etc where participants rate the measure on a 5 to 7-point Likert scale.
Likes, Dislikes and Recommendations: Participants provide what they liked most about the site, what they liked least about the site, and recommendations for improving the site.
Applying What You’ve Learned
It can be frustrating and humbling to find out that the fancy navigation system you worked so hard on is confusing to users, or that the testers couldn’t see your daily special even though it seems perfectly obvious to you. Remember how familiar you are with your own site – you already know everything about it and how to find the information. You aren’t your site’s user! What you think is obvious doesn’t count if your website’s users aren’t finding the information they need to user your product, hire you or otherwise stay on your site.
On the other hand, if 2 out of 3 of your testers passed everything, but the third had issues, evaluate if changes need to be made. Perhaps a few wording or formatting changes are needed instead of an overhaul.
So, make the changes suggested by the tests and run the tests again, with different test subjects.
If you used usability testing for your site, I’d love to hear your experience. Interested in organizing a WordPress site usability test? Contact me to help you set it up.