Usability Testing and Design Inspection Methods

Usability testing evaluates a site by collecting data from people as they use the product. A group of people are invited to attain a session where they are asked to perform various tasks while moderator makes notes about problem areas, other difficulties encountered. Various techniques are available for UIT (Usability testing). Lets have a look at each one of them & their significance.

1. Think aloud protocol: This method of UIT, involve participants thinking aloud as they are performing a set of specified tasks. Users are asked to say whatever they are looking at, thinking, doing, and feeling, as they go about their task. This enables observers to see first-hand the process of task completion. Observers at such a test are asked to objectively take notes of everything that users say, without attempting to interpret their actions and words. Test sessions are often audio and video taped so that moderator can go back and refer to what participants did, and how they reacted. The purpose of this method is to make explicit what is implicitly present in subjects who are able to perform a specific task.

Application Stage: Design, coding, testing and release of application

2. Talk aloud protocol: This involves participants only describing their action but not giving explanations. This method is thought to be more objective in that participants merely report how they go about completing a task rather than interpreting or justifying their actions.

Application Stage: Design, coding, testing and release of application

3. Remote testing: In this method, a tracking software is installed on users computer. Moderator can view users activity by logging into the software. There are various Remote Monitoring softwares available in the market.

Application Stage: Design, coding, testing and release of application

Think aloud protocol

4. Focus groups: A focus group is a form of qualitative research in which a group of people are asked about their attitude towards a product, concept or idea. Questions are asked in an interactive group setting where participants are free to talk with other group members (i.e. participants) to share their thoughts, feelings, attitudes and ideas on the given subject.

Focus groups are most often used as an input to design. They generally produce non-statistical data and are a good means of getting information about a domain (e.g. what peoples’ tasks involve). It’s necessary to have an experienced moderator and analyst for a focus group to be effective.

Application Stage: Before creating prototype, Conceptual stage, Testing and release of application

Focus Groups

5. Card Sorting: This is a method for suggesting intuitive structures/categories. A participant is presented with an unsorted pack of index cards. Each card has a statement written on it that relates to a page of the site. The participant is asked to sort these cards into groups and then to name these groups. The results of multiple individual sorts are then combined and analysed statistically. Usually this is used as an input to Informtion design. It’s an excellent way of suggesting good categories for a site’s content and deriving its information architecture. Card sorting can be used generate statistical data.

Read “Definative Guide to Card Sorting” written by Dona Spencer for detailed description on card sorting.

Application Stage: Before creating prototype, IA level

Steps in UCD

These are some of the basic techniques of a typical of a UCD process for designing Web applications.

1. Defining Project Vision:

This is where the company defines the product vision, target market, goals, inspirations, best practices, challenges, constrains etc. For example, Google as a company may have defined their vision as follows:

  1. Focus on people, their lives, work and dreams
  2. Every millisecond counts.
  3. Simplicity is powerful.
  4. Engage beginners and attract experts.
  5. Dare to innovate.
  6. Design for the world.
  7. Plan for today’s and tomorrow’s business.
  8. Delight the eye without distracting the mind.
  9. Be worthy of people’s trust.
  10. Add a human touch.

2. User analysis:

This is the step of information gathering and analysis.

Step 1 – Groups

Target users are divided into primary & secondary groups (aka Focus groups). Primary groups division parameters can be very basic, for example a social networking site can simply divide their users into primary groups:

  • Group A – Users who already have an account on any other social networking sites.
  • Group B – Users who have never used a social networking site before, but who access internet.

Now Group A can be further divided into various secondary groups depending on the product vision and business requirement. So for example, Group A can be further divided as follows:

  • A1 – Gender Male
  • A2 – Gender Female
  • A3 – Career based (IT, teacher, engineer, BPO employee etc)
  • A4 – Education based
  • A5 – Business networking user
  • A6 – Family networking user
  • A7 – Friends networking user

Step 2 – Categories

This is the step where business defines the user categories. This can be as simple as techies and non techies. Categories can also be defined by the age groups division of the target group.

Step 3 – User matrix

The User Group Matrix table below includes colour coding that is with the text in order to simplify the analysis.

  • Range – Usage patterns can be defined as Low, medium, high and everything in between like Low to High, Medium to High etc.
  • Parameters for Analysis – These are factors defined by the business against which the analysis table is drawn.
User matrix

User matrix

Step 4 – Creating Personas

Personas are fictitious characters created to represent the different user types within a targeted demographic that might use a site or product. Personas are useful in considering the goals, desires, and limitations of the target group.

Step 5 – Analysis & Implications

Methods of analysis like field study, contextual analysis etc. reveals various aspects that can heavily influence the product design. For example, if the finding is that many uses do not visit the site regularly, then the implications in design can be as follows:

1. In making specific design decisions, easy-to-use for first-time and infrequent users is higher priority than efficient-to-use for frequent users
2. Exploring the possible exception of areas targeted for working group members
3. News feed in the email/ on site can maximise usage

Scenarios & Tasks analysis

“Scenarios are a concrete description of user activity while performing a specific task, a description sufficiently detailed so that design implications can be inferred and reasoned about” – J M Carroll

Scenarios are about the user tasks n expectations. They help business understand the user decision making, decision influences, motivation.

Scenarios provide the ‘How’ perspective in understanding the following:

· Verifying the tasks and possible tasks combination’s

· Task influencers

· Task relationships based on users mental model

· User expectations from system functions.

Step 1 – Creating task list

This is primarily a data gathering activity. Purpose of the activity is to find out what users are looking for in the new product. It also gathers information about the user tasks, how they currently perform it and how to they wish to perform it in future. There are various techniques of gathering this data.

1. Real time – This is done by simply visiting the user at his workplace and observing his usage pattern to understand the interaction with the system or product at present. For example, if a company wants to launch a new portable media player with the target market being urban girls between the age of 16 to 24, then the best place for gathering real time data will be a junior college campus. Observer understands the way they interact with the system currently. Observer can also ask specific questions about the product they are using with its pros and cons.

2. Discussion support center – A room with several computers loaded with the group meeting software is used for this kind of activity. Focus groups are invited here. The facilitator explains the task and the group performs the task. Meeting software gathers this data and returns it for analysis.

3. Low tech methods – In this method, index cards or sticky notes are used. User arranges the cards in order of priority regarding to the functionality or feature usage parameter.

4. Internet surveys – Online survey sites use the forms for collecting this kind of data from the potential users.

At the end of the exercise, a list of task is created. This list tells the observer about how people would like to use the product & what purpose they may use it for. Once this list is prepared, a task user matrix is formed for further analysis.

Step 2 – Task user matrix

This matrix helps in analyzing, which users do which tasks. This can help business decide on interface design and planning documentation for products aka FRS (functional requirement specification)

For this kind of matrix, all the task gets listed in one column and the other columns shows the frequency of usage.

Step 3 – Task analysis

Method of Task analysis is demonstrated in the excel file attached here:


These tasks are then further divided into four major categories: High, Medium, Low, Future

Each user task is analyzed using this technique. While in the early stages a more generic approach is taken in defining the scenarios as it’s the conceptual stage of product. The purpose here is to identify major tasks & deriving the overall product functionality.

Step 4 – Creating Task flows based on functionality

Task flow is created in order to show the way in which tasks are done step by step. This includes logic & decision the user might take while doing the task. A flow-chart showing the branching & recursion of task.

Wire-frames are useful in:

1. Identifying mismatch with scenarios – Evaluating how well the current task flow supports the scenarios helps business identify aspects in the current flow that match & aspects that do not match.

2. Predicting errors – Inspection of possible errors such as steps out of order, errors of omission, errors of commission & misinterpretation.

3. Optimizing the task flow – Improving efficiency of the system with fewer clicks, lesser branching.