Usability Walkthroughs

Usability walkthrough (aka inspections) is the name for a set of methods where an evaluator inspects a user interface. Walkthrough can generally be used early in the development process by evaluating prototypes or specifications for the system that can’t be tested on users. Usability inspection methods are generally considered to be cheaper to implement than testing on users. Please remember that Usability inspections (walk through) is entirely different that UAT (User acceptance testing)

Usability inspection methods includes:

1. Cognitive Walk through

The Cognitive walkthrough method is a usability inspection method used to identify usability issues in a piece of software or web site, focusing on how easy it is for new users to accomplish tasks with the system. Its a low cost method, used early in the design phases, before coding has even begun.


Defining input to the walkthrough

User – Use data gathered from persona analysis

Tasks – Use data gathered from tasks analysis

Sequence – Each tasks should be given a description of how user is expected to view the task before understanding the interface. There must also be a description of the sequence of actions that should accomplish the task with the current definition of the interface.

Interface – The definition of the interface must describe the prompts preceding every action required to accomplish the tasks being analyzed, as well as the reaction of the interface to each of these actions. If the interface has been implemented, all information is available from the implementation. Earlier in the development process, the evaluation can be performed with a paper description of the interface. For a paper description, the level of detail in defining the interface will depend on the expertise that the anticipated users have with existing systems.

Walking Through the Actions

The analysis phase consists of examining each action in the solution path and attempting to tell a credible story as to why the expected users would choose that action. Credible stories are based on assumptions about the user’s background knowledge and goals, and on an understanding of the problem-solving process that enables a user to guess the correct action.

As the walkthrough proceeds, the evaluators ask the following four questions:

Will the users try to achieve the right effect? For example, their task is to print a document, but the first thing they have to do is select a printer. Will they know that they should select a printer?

Will the user notice that the correct action is available? This relates to the visibility and understandability of actions in the interface.

Will the user associate the correct action with the effect to be achieved? Users often use the “label-following” strategy, which leads them to select an action if the label for that action matches the task description.

If the correct action is performed, will the user see that progress is being made toward solution of the task? This is to check the system feedback after the user executes the action.

The evaluator(s) will try to construct a success story for each step in the task case(s). General conditions where a success story can be told is given next in “common features of success”. When a success story cannot be told, construct a failure story, providing the criterion (one or more of the four questions above) and the reason why the user may fail

2. Heuristic Evaluation
The main goal of heuristic evaluations is to identify any problems associated with the design of user interfaces. Usability consultant Jakob Nielsen developed this method on the basis of several years of experience in teaching and consulting about usability engineering.

Quite often, usability problems that are discovered are categorized—often on a numeric scale—according to their estimated impact on user performance or acceptance. Often the heuristic evaluation is conducted in the context of use cases (typical user tasks), to provide feedback to the developers on the extent to which the interface is likely to be compatible with the intended users’ needs and preferences.

Principles of Heuristic evaluation

Visibility of system: The system should keep users informed about what is going on, through appropriate feedback within reasonable time.
It means keeping the user informed about his statistical data. This is a process of letting a user know about his activity stats, his no. of friends, his page ranking and all the general reports user might be interested in.

Match between system and the real world:
The system should speak the users language, with words, phrases and concepts familiar to the user, rather then system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

User control and freedom:
Users often choose system functions by mistake and will need a clearly marked ‘emergency exit’ to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

Consistency and standards:
User should not have to wonder whether different words, situations or actions mean the same thing. Follow the pattern.

Error prevention:
Even better then good error message is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present user with a confirmation option before they commit to the action.

Recognition rather then recall:
Minimize the user’s memory load by making objects, actions and options visible. The user should not have to remember information from one part of dialogue to another. Instructions for use of the system should be visible or easily retrievable.

Flexibility an efficiency of use:
Accelerators — unseen by novice user-may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

Aesthetic and minimalist design:
Dialogues should not contain information which is irrelevant or rarely needed. Every extra unite of information in a dialogue competes with the relevant unites of information and diminishes relative visibility.

Help users recognize, diagnose and recover from errors:
Error messages should be expressed in plain language (no codes) precisely indicates the problem and constructively suggests a solution.

Help and documentation:
Even though it is better if system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user’s task, list concrete steps to be carried out and not be too large.

3. Pluralistic walk though:

At the design stage, when paper prototype is available, a group of users, developers, and huaman factors engineers meet together to step through a set of tasks, discussing and evaluating the usability of a system. Group walkthroughs have the advantage of providing a diverse range of skills and perspectives to bear on usability problems. As with any inspection, the more people looking for problems, the higher the probablility of finding problems. Also, the interaction between the team during the walkthrough helps to resolve usability issues faster.

A walkthrough team must be assembled prior to the pluralistic walkthrough. Three types of participants are included in the walkthrough: representative users, product developers and human factors (usability) engineers/professionals. Users should be representative of the target audience, and are considered the primary participants in the usability evaluation. Product developers answer questions about design suggest solutions to interface problems users have encountered. Human factors professionals usually serve as the facilitators and are also there to provide feedback on the design as well as recommend design improvements. The role of the facilitator is to guide users through tasks and facilitate collaboration between users and developers. It is best to avoid having a product developer assume the role of facilitator, as they can get defensive to criticism of their product.

Paper prototype of the interface are to be used for the walkthrough. Hard-copy panels of screens, dialog boxes, menus, etc. will be presented in the same order in which they would appear online for each task.

For each step, All participants are presented with the interface design in the form of a screen panel and asked to write down separately the action they want to take. The participants need to write their actions in as much detail as possible.

After all participants have written the actions they would take for the task, a discussion begins, in which the users speak first. Only when the users’ comments are exhausted do the usability experts and the product developers offer their opinions.

After the discussion, the coordinator will tell the participants what actions they are supposed to take according to the user interface design and present the new screen panel after the actions. Thus the walkthrough moves to the next step.

4. Feature Inspection:

This inspeciton technique focuses on the feature set of a product. The inspectors are usually given use cases with the end result to be obtained from the use of the product. Each feature is analyzed for its availability, understandability, and other aspects of usability. For example, a common user scenario for the use of a text input section to create a blog post. The features that would be used include entering text, formatting text, spell-checking, saving the text to a file, and uploading images. Each set of features used to produce the required output (a letter) is analyzed for its availability, understandability, and general usefulness.


Usability Testing and Design Inspection Methods

Usability testing evaluates a site by collecting data from people as they use the product. A group of people are invited to attain a session where they are asked to perform various tasks while moderator makes notes about problem areas, other difficulties encountered. Various techniques are available for UIT (Usability testing). Lets have a look at each one of them & their significance.

1. Think aloud protocol: This method of UIT, involve participants thinking aloud as they are performing a set of specified tasks. Users are asked to say whatever they are looking at, thinking, doing, and feeling, as they go about their task. This enables observers to see first-hand the process of task completion. Observers at such a test are asked to objectively take notes of everything that users say, without attempting to interpret their actions and words. Test sessions are often audio and video taped so that moderator can go back and refer to what participants did, and how they reacted. The purpose of this method is to make explicit what is implicitly present in subjects who are able to perform a specific task.

Application Stage: Design, coding, testing and release of application

2. Talk aloud protocol: This involves participants only describing their action but not giving explanations. This method is thought to be more objective in that participants merely report how they go about completing a task rather than interpreting or justifying their actions.

Application Stage: Design, coding, testing and release of application

3. Remote testing: In this method, a tracking software is installed on users computer. Moderator can view users activity by logging into the software. There are various Remote Monitoring softwares available in the market.

Application Stage: Design, coding, testing and release of application

Think aloud protocol

4. Focus groups: A focus group is a form of qualitative research in which a group of people are asked about their attitude towards a product, concept or idea. Questions are asked in an interactive group setting where participants are free to talk with other group members (i.e. participants) to share their thoughts, feelings, attitudes and ideas on the given subject.

Focus groups are most often used as an input to design. They generally produce non-statistical data and are a good means of getting information about a domain (e.g. what peoples’ tasks involve). It’s necessary to have an experienced moderator and analyst for a focus group to be effective.

Application Stage: Before creating prototype, Conceptual stage, Testing and release of application

Focus Groups

5. Card Sorting: This is a method for suggesting intuitive structures/categories. A participant is presented with an unsorted pack of index cards. Each card has a statement written on it that relates to a page of the site. The participant is asked to sort these cards into groups and then to name these groups. The results of multiple individual sorts are then combined and analysed statistically. Usually this is used as an input to Informtion design. It’s an excellent way of suggesting good categories for a site’s content and deriving its information architecture. Card sorting can be used generate statistical data.

Read “Definative Guide to Card Sorting” written by Dona Spencer for detailed description on card sorting.

Application Stage: Before creating prototype, IA level