ISO-9241 standard defines Satisfaction with the product as “comfort and relevance of application”.
In contrast to Efficacy and Productivity, where we observe and evaluate the user’s actions, Satisfaction aims at subjective thoughts of the user.
There are two widespread methods of Satisfaction measurement.
Method 1: Satisfaction is measured through a formalized questionnaires featuring satisfaction scales.
Each questionnaire contains a number of statements reflecting subjective user’s opinion on interaction with the product. For example, “I like the way interface looks” or “I believe the product to be too complicated”.
Each scale features between 3 and 11 answers. Here is the example of a 5-point scale:
Weight – numeric value of the satisfaction coefficient, which depends on the answer.
Weight for a 5-point scale can be estimated differently, e.g. from -2 to +2, from 1 to 5 or from 0 to 4.
The questions and statements concerning the product might sound both positive and negative.
A positive question with weight driven by answers:
Example: “I am satisfied with the speed of this product”
A negative question with weight driven by answers:
Example: “I often made mistake while doing tasks”
R – the number of respondents
Q+ – the number of positive questions featuring in the questionnaire
Q- – the number of negative questions
pij+ – weight for the answer to a positive question (0-4) for i-scenario and j-respondent
pij- – weight for the answer to a negative question (4-0) for i-scenario and j-respondent
the average satisfaction (S) is estimated through the following formula:
Statistical error of Satisfaction:
Total satisfaction with the product:
Satisfaction questionnaires applied for our tests:
- SUS (System Usability Scale) suggested by the Digital Equipment Corporation © 1986
(10 questions, a 5-point scale)
- QUIS (Questionnaire for User Interaction Satisfaction) suggested by the Human-Computer Interaction Lab (HCIL) at University of Maryland at College Park © 1990
(24 questions, a 10+1 point scale)
- SUMI (Software Usability Measurement Inventory) suggested by the Human Factors Research Group © 1993
(50 questions, a 3-point scale)
- CSUQ (Computer System Usability Questionnaire) suggested by IBM © 1995
(19 questions, a 7+1 point scale)
Besides, we apply ISO-20282 standard ("Easy of operation of everyday products: Test method for walk-up-and-use products"), which expands ISO-9241 standard – applicable for products aimed at day-to-day use. This standard presupposes application of extra Smiley Scale © Kunin, 1955.
The Smiley Scale method. The respondent is to choose one smiley in five to reflect his/her satisfaction with a certain action:
Weight coefficients range from -2 to +2.
Method 2. The Word Method enables to uncover subjective associations generated by the product.
One of the Word lists versions comprises 118 adjectives with positive and negative connotation. © 2002 Microsoft Corporation. All rights reserved.
Each user is to highlight words reflecting his/her interaction with the product. The words feature in alphabetical order and negative ones are mixed with the positive.
The aggregated results enable to draw a diagram reflecting the general impression of users confronted with the product. Fr convenience purposes positive associations are highlighted in green and negative – in red. The Associations diagram is aligned in accordance with the number of respondents.
Associations diagram for 11 respondents:
R – number of respondents
A+j – number of positive words-associations provided by j-respondent
A-j – number of negative words-associations provided by j-respondent
The total Associate Satisfaction is estimated through the following formula:
Users and Goals | Effectiveness | Efficiency | Satisfaction | Methods