Before the second reading seminar

Chapter 13 is about evaluation framework and includes practical and ethical aspects of the research process and evaluation process.

The main goal of any design process is to develop a product that meets the user's requirements.

DECIDE (determine, explore, choose, identify, decide, evaluate) is an example of an evaluation framework.

User study methods like 'think out loud', 'into the wild', lab studies and 'wizard of Oz' are useful tools to understand how the user interacts with the product. When designing, it is important to determine the goals - who is the user and why is he/she interacting with the product like this?

A study should never take more than 20-30 minutes per person. It is important that the participant does not get tired or uncomfortable because that will affect the results. We separate between 'into the wild' and lab studies. Lab studies are good for factual, quantitative data, but the data might not be very realistic because the user and the product are taken out of the intended context. 'Into the wild' studies are a bit more like ethnographical studies. The user is usually more comfortable in his natural environment and is it easier to obtain qualitative data.

Another thing that is important to consider are practical and ethical aspects of user studies. Practical matters are e.g. cost, Schedule and time management. Ethical aspects are about preserving the participant's anonymity by leaving out sensitive information, being honest with the participant and letting him know what is going to happen beforehand and making sure that the participant never is uncomfortable.

Five keywords to consider when  evaluating, analysing, interpreting and presenting the data are

Reliability

Validity

Ecological validity

Biases

Scope

 

Chapter 15 is about inspection methods, evaluating data and two predictive methods: GOMS and Fitt's law.

Sometimes users are not easily accessible, or involving them is too expensive or too time consuming. In such circumstances other people, called experts, can provide feedback. Experts can also compliment user studies.

Heuristic evaluation is a usability inspection method where experts guided by a set of ruls evaluate whether UI-elements correspond to tried and tested principles. These rules include visibility, error prevention, simple design and aesthetics and recognition. Heuristic evaluation has three stages: a briefing session, the evaluation period and the debriefing session.

Walkthroughs are an alternative approach to heuristic evaluation. A walkthrough usually don't have a user. Then, the developer or expert take the product through a task and note usability errors. Pluralistic walkthroughs include users, developers and usability experts. An example is the cognitive walkthrough where the user's problem-solving process is simulated to see if the user's actions can lead to the next action. If he can perform a task as intended by the developer.

Analytics is a method for quantifying the user traffic through a system. User activity is logged so that the data can be analysed to understand what parts of a web site is being used and when. This can for intance be used for advertising and investigating mapping.

GOMS is an acronym for goals, operators, methods and selection rules. GOMS is an attempt to model the cognitive processes of a user that interacts with the product. It takes in consideration the state the user wants to achieve, the actions that needs to be performed, learned procedures and selection rules.

Fitt's law is a mathematical model that predicts the time it takes to reach a target with a pointing device. It is based on the size and of the object and the distance to it. This helps designers orient and give shape to buttons on for instance a website. The bigger the target, the easier it is to locate it. 

Question for the reading seminar: I have not quite understood what the what characterizes the 'experts' in heuristic evaluation. Are they a part of the developer team or ate they independent people with expertise in interaction design?