Tuesday, September 14, 2010


"People," continued the taxi driver driving me to the airport in Dublin, "do the strangest things with chocolate." He paused, reflectively, before adding, "I mean in private."

When I didn't immediately respond, he hurried to explain himself. "You know, a Bounty bar?" I did. "I pick the chocolate off of the outside and then eat the inside separately. Do you do that?" As politely as I could I explained that I didn't like Bounty bars. "What do you do then?" he asked. The best thing that I could come up with was Oreo cookies, that I twist them open and eat out the middle, "A lot of people do that," I added. This puzzled him, until he brightened, "Oh, I heard about this biscuit in Australia and you bite off two of the corners and you drink your tea right through the biscuit. It has some sort of a cream filling that just melts as you drink. It's supposed to be just lovely." He thought for a moment. "It's Tam Tam or Yam Yam or something like that it's called."

I tried to imagine the Tam Tam or the Yam Yam and what it might look like. I was in Perth for about four days after SIGIR 2008, but didn't remember any cookies like that. "Do you suppose," I asked him, "that people just take the biscuit out of the package and look at it and think, 'oh, I should break off two of the corners and drink my tea through it' or was there one person who invented it and then it quickly spread as an idea throughout Australia?"

His response surprised me: he laughed! Then, "It's like the comedian," he pronounced. And then he filled me in: there is a comedian one-liner about watching a chicken lay an egg. "Hey, I think I could eat that!" was the punch line.

And so, I end up discussing with a Dublin taxi driver, the principle of affordance, the ability of an object to be acted on in its environment, and, in a larger definition, communicate its use via its appearance.

In multimedia information retrieval, I am obsessed with affordance in this latter sense. At the first glance or very quickly during interaction, the system should implicitly communicate to the user what it does, what the user can do with what it does and the extent to which it can be trusted to reliably do what it does in all cases.

A few years ago, I believed that a speech retrieval system should not show transcripts to users because users are disturbed by errors. Now we are all a lot further. People are used to reading relatively unedited or unconventional text in text messages, blogs (!) and comments. Now, the level of error can signal to the user that the text has been created by a speech recognizer and how well that speech recognizer can be trusted to capture the spoken content of the audio signal.

But his is negative affordance, the message what can't this system do. It is quite possible that negative affordance is much more challenging to communicate to the user since the space of possible non-uses is not intuitively constrained.

And with biscuits, of course, comes the problem of distributed affordance. What works well once does not continue working well with repeated applications. The package of biscuits should tell you, individually, we are delicious, but if you eat the whole package you won't feel nice and full, but instead you will have an unhappy stomach. Even it was written explicitly on the package, I imagine I would mostly ignore that message.

This is Part III (final part!) of the "Irish Chocolate Discussion", reflections on the conversation I had with a Dublin taxi driver and how that relates to finding things and search systems in general.