How to use usability testing?

Usability Testing: Which confidence interval should I specifically use for average time on task results?

• I am trying to account for the invariable variability (ha!) across my time on task data and want to know which confidence interval measure method is best. I am no expert statistician!

• Answer:

  If you report mean it's advised to report the geometric mean, instead of the arithmetic mean. This is because there tend to be people who take a lot longer than the mean, but no people who are that same amount of time quicker. E.g. If the mean for a task is 40 seconds, there might be some people who takes 100 seconds (+60), but there can't be people who take -20 seconds (-60) to complete the task. This means the distribution is not equally spread on both sides of arithmetic mean and reporting the arithmetic mean will not be a good representation of the time on task. If you want read more about this, I suggest looking at http://www.measuringusability.com/average-times.php from Jeff Sauro. Now your question was about intervals, instead of means. First intervals already give a better respresentation than just the arithmetric mean. There's no standard for what kind of measure to use for the spread of your results, but intervals in general are okay. Normally I would go for the full range, 5-95 interval or 25-75 interval. If your tset group was more uniform than you expect the real group of users expect to be, you could use a bigger interval. If your test group was really diverse and you expect the best and worst users were overrepresented, you might want to go for a smaller interval. If you really don't know, you would probably always do okay with a 25-75 percentile interval.