What Can Be Tested?

> Websites that are live on the Internet.
> Intranets and internal applications: you will need to get us an office or meeting room at your location where we can test your employees.
> Software products, whether shrink-wrapped or for internal use.
> Consumer electronics, new devices, cellphones: anything that has a user interface should be tested. For hardware products, it is important to run an "out-of-the-box" study of the initial user experience and installation.
> Prototypes of all of the above: we can test the design as soon as it is prototyped. No need to waste engineering resources implementing something that people can't use. Better to find the usability problems in your design as soon as possible.
> Paper prototypes, mock-ups, or wireframes of early designs, alternative homepage comps, etc. 

Variations on the Method

We recommend that you conduct usability testing at multiple stages of your projects, but even a single test will significantly increase the quality of your user experience.

Qualitative studies are the most traditional form of user testing: aimed at producing insights into user behavior to discover what works and doesn't work in the design. The goal is to tell you how to change the design to serve your users better. These tests do not generate large amounts of statistics. Qualitative studies of websites and intranets typically cost around $22,000, with prices up to $40,000 for more complex cases. Very small and targeted studies, such as testing a few homepage comps, may be $20,000.

Quantitative studies are much more expensive: to generate statistically significant numbers that compare two different websites or two versions of the same system will typically cost around $70,000. We do not recommend this, except for organizations that are highly metrics-driven. It is nice to have proof of how much a redesign improved a product or how much better/worse you are than the competition. However, it is more important to know what to improve, and cheap qualitative studies are much better at answering that question.

Competitive studies are one of the most valuable methods. We recommend qualitative testing of your own design compared with two or three of the most important competitors. Such studies typically cost around $45,000 and provide the best basis for a redesign project because you learn from other designs as well as your own. No reason to repeat others' mistakes.

International studies are important if you have customers in other countries (or overseas employees, in the case of an intranet). Costs are often around $100,000-$150,000, but vary widely depending on the number of countries. If you are located outside the United States, you are in luck: we can test American users for you at the cheaper price of a standard test and it will count as an international study of your design.

Schedule

Subsequent rounds of testing within the same project can be accomplished faster. It is recommended to schedule two or three rounds of testing as you revise and improve the design through multiple iterations of prototypes.

Competitive tests, international studies, and quantitative measurements take longer (1-2 months).

Availability of our staff is highly variable, so it is recommended to book studies with as much advance notice as possible. 

Confidentiality

As a client you gain the added benefit of being benchmarked against this vast amount of additional data, but your own data is never published (except for a few cases where a client has agreed to publish limited information about a project for PR reasons).

Deliverable

The report addresses three levels of recommendations:
> Specific usability problems in the user interface are identified together with recommended changes that will improve usability. We often find around 50 things to change in even a small design.
> Task support and communications analysis of how well your design communicates and supports users' top goals.
> Future usability directions for the design and our recommended strategy for achieving significant advances in the user experience.

Frequently asked question: Can I see a sample report?

Answer: No. Each test report is highly confidential for the client in question because it exposes strategic information about the behavior of their customers (or employees, in the case of an intranet study). However, we have many usability reports available for download that describe findings from projects that we have funded ourselves. These reports are representative of the deliverable from competitive testing because they describe tests of a range of websites instead of tests of a single design. 

How to Order

Please contact Nielsen Norman Group at info@nngroup.com if you would like more information or are interested in this service.