Before Reading Seminar #2

For this seminar we read the chapters 13 and 15. Both of which contained theory about evaluating systems generally. Frameworks and methods are described.

In chapter 13 the main key point was the description of the DECIDE framework, listing six different steps when planning to do an evaluation study. In relation to our project the framework could be very suitable when starting to evaluating our first mockup.

As this will be the first experience of doing an evaluation study, the DECIDE framework is a great  model for us to start with. Similar issues from the previous chapters read in the previous seminar are brought up in these chapters, but in a context where the issues are put in practice. Though ethical issues aren’t as current seeing that we don’t use the private data and does not therefor need to be as much considered. However, other issues concerning the evaluation is higly important and the DECIDE framework is useful in that way whereas it gives a great overview of the many key points.

Chapter 15 included methods for an evaluation study, mainly the methods Heuristic Evaluation and Walkthroughs.

A Heuristic Evaluation is a method using a set of usability principles, the heuristics, to guide the inspectors doing the evaluations. The method Walkthroughs can be divided into two parts; cognitive and pluralistic. This method involve walking the inspectors through a task with the evaluated product.

The primary is that these methods don’t have to include the users, or more importantly, it doesn’t. Instead the methods are tools for understanding and getting the knowledge about the user’s behaviour and consequently finding the problem issues, fitting requirements or whether the functionality of a system serves the user or not. All by the use of so called experts & specialists that works as evaluators whose task is to ”examine the interface of an interactive product, often role-playing typical users, and suggest problems users would likely have when interacting with it(...)”.

An important note is that the evaluators can be several and the more evaluators, more accurate results will appear when finding the user’s issues.

Using methods that also doesn’t require involving users, fewer practical and ethical issues have to be considered.

I find the approach of the evaluation needs to be clearly set since a Heuristic evaluation compared to using Walkthroughs are good for different evaluations. A Heuristic evaluation is suitable when evaluating a complete system whereas a Walkthrough is suitable for evaluating a small part of a system.

These methods desribed should not be the primary methods, but instead it’s a good compliment to user testing when analyzing a system or a product.

My question for the 2nd reading seminar: As our group of a small amount of people that will maybe the most knowledgable of our system, what determine who should be the evaluators(experts) in our case scenario?