A few years back while using YouTube, the user interface
that I was seeing would change in between videos and visits. Eventually, the
new interface fully replaced the old, but it happened over a long enough period
that it made me wonder at the time if this was something Google was doing in an
attempt to test some aspect of their new interface. Although I have no proof
this was what was actually going on, and Google does not seem to publish
anything on the subject, it did leave me wondering exactly what might be learned
from testing an old and new interface side by side.
Questions like whether users who were using the new
interface, measured possibly through logged in users or even just through IP
addresses or cookies, would stay on their site longer, would watch more or
fewer videos, would have to spend more time on a particular video's page
playing with the new UI, or whether the new UI was able to direct them to
videos generating more advertising revenue for the site might all be answered
this way. Groups for experiments could come from similar geographic areas, and
if they are using logged in users they may even be able to further divide into
groups based on gender, age, and other information which a social networking
account can provide. Although not all of these questions relate directly to
people finding information, even looking only at the questions that are would
be interesting.
This may not seem entirely like an experiment, but it does
have the main components. If they keep groups with similar geography or other
factors they have controlled variables, while the user interface they see would be
an independent variable which can be measured. Knight talks about the problems
with quasi-experiments, where they cannot prove conclusively that the variable
they are testing is what caused different results between the groups in their
experiment (Knight, 2002), but considering how much information Google can
gather, they may be able to control enough of the variables to get large enough
samples sizes to account for this. Even though this may not be what they were
doing with the slow switch over to the new UI, it was something that brought up
some questions I found interesting at the time.
Sources:
Knight, P. (2002). Small-Scale Research. 1 Oliver’s Yard, 55 City Road, London England EC1Y 1SP United Kingdom: SAGE Publications Ltd. Retrieved from http://srmo.sagepub.com/view/small-scale-research/SAGE.xml
Reading this, it seemed to me as a very legitimate experiment, but as you mentioned more likely to be funded by Google themselves, perhaps conducted by their own in-house researchers, and as a result that data would be privatized in the interests of Google, if they have the intention to covet the best UI know-hows to themselves.
ReplyDeleteOn a related note, if you had the interest (or anyone else), this experiment could be turned into a more publicly conducted research study (or at the very least the results could be published) to understand accessibility of UIs and how to create a standardized format to apply for government service websites.
And yet my suggestion might not come into actual fruition as a fund-able study, because there are general textbooks and heuristic evaluations…as well as competing issues such as allowing a designer’s creative freedom to roam free without specific set ‘standards’ (to go off on another tangent: to which then a possible research question might be “how restrictive should UI guidelines be?”)