How to use the word "subjective" in multimedia content analysis

Multimedia content analysis is devoted to the automatic processing of video, image, audio, and text content with the purpose of describing it, or otherwise associating it with information that will make it findable, and also useful, to users. Previously, I have urged multimedia content analysis researchers to avoid the word “subjective” and instead formulate their insights in terms of inter-annotator agreement with respect to the data that they are using and the protocol that they give to the annotators who are providing the target labels. Since we don’t seem to be inclined to stop using the word “subjective” soon, it makes sense to formulate some guidelines on how to use it "safely".

Best practice for the use of the word “subjective”: When the word "subjective" it is used, it should be first defined.

The word "subjective" has different definitions. It’s not particularly productive to fix any one way of using it as “the only right way”. Instead, when using the word "subjective" you should simply declare which definition you are using, and you will avoid a lot of unproductive confusion. You do not want to risk that you use “subjective” in one sense, and your reader/listener interprets it in another sense.

,

We can gain further understanding of why it is important to "define well before use" by examining the dictionary entry for “subjective” provided by Merriam-Webster. Here, you can see the many meanings that “subjective” can take on. I haven’t observed any issues caused by definitions 1 or 2. Multimedia content analysis research is generally not interested in these definitions. Where we get into trouble is with 3-5, so I will focus on these.

Let’s start with definition 4c: “arising out of or identified by means of one's perception of one's own states and processes” This definition of subjective is related to the conceptualization of a situation as being exclusively determined by the point of view of the “subject”, i.e., the person who is undergoing the experience of perceiving something.

Such a conceptualization, in the case of certain situations, is standard, and when we communicate with each other, we don’t even think about the fact that we assume it.  Let’s take a closer look at how this conceptualization works. When we use language, we rely on an unspoken agreement that certain phenomena (for example, the emotion that music evokes in a person) are subjective. Specifically, the agreement means that the way in which we understand the world gives all listeners the power to determine what they feel when listening to music (i.e., induced emotion) for themselves.

Simply stated: if someone says, “This music makes me so happy”, it is nonsensical for me to assert, “No, it doesn’t”. I might say this to tease someone, but it is clear that I am not using language in a standard way. An emotion felt while listening to music can only be asserted by the subject, and I, who am not in the subject’s mind, do not have the power to originate a meaningful statement on the matter. It is not a trivial point: Without this shared understanding, the convention/assumption of subjectivity behind "This music makes me happy", the function of language would break down and we would have failed to communicate.

Here’s where things can go wrong for a researcher working in the area of multimedia content analysis. Imagine you are collecting multimedia content labels from a group of annotators who are judging the content, and you at the end of experiment, and declare, “The results show that the phenomenon we are studying is subjective”. Readers who are using definition 4c of subjective will find this conclusion invalid. The reason is that under this definition, “subjective” is something that is established ahead of time by convention: it cannot be determined experimentally. (Full disclosure: for me this is the preferred definition of "subjective", because it is the most literal interpretation. The word "subjective" contains the word "subject". I also prefer it since it ensures the sanctity of the private world of the individual, and the right of the individual to an independent voice.)

Moving to 4b: “arising from conditions within the brain or sense organs and not directly caused by external stimuli” This definition is not so interesting for multimedia researchers: we study multimedia content, which is an external stimuli. 

Now, we go on to definition 4a: (1): “peculiar to a particular individual:  personal” This definition of subjective is related to the idea that each individual has their own unique view. (Merrian-Webster's Definition 1 of "peculiar" is "characteristic of only one person, group, or thing") Under this definition, something is "subjective" it means that everyone disagrees with everyone else. This definition is also not so interesting for multimedia researchers: if everyone has their own completely different interpretation, then we are lost: we cannot hope to build algorithms that generalize over the different meanings that find in multimedia. Until the field of multimedia starts working extensively on systems used only by a single person, this definition of subjective is probably not one that will be used often.

Note that the field of recommender systems strives to develop personalized algorithms, and users evaluation methodologies that assess whether personal predictions are successful. However, even recommender systems rely on the fact that people are similar to each other. In a world populated exclusively with utterly unique individuals, collaborative filtering algorithms will necessarily fail.

More helpful is definition 4a (2): “modified or affected by personal views, experience, or background” This definition of "subjective" is often implicitly assumed in multimedia content analysis. People’s interpretations are affected by what they know, the opinions they hold, and the life experience that they have had. These factors can lead to there being a multitude of different interpretations that apply to certain multimedia content. However, in contrast to the situation above with definition 4a (1), we are not assuming that everyone has their own “peculiar” interpretations. It makes sense for us to try to create systems that generalize or predict meaning, only in the case that we are not dealing with exclusively unique interpretations.

We can see 4a (2) as closely related to 3b: “relating to or being experience or knowledge as conditioned by personal mental characteristics or states”

With both of these definitions, 4a (2) and 3b, we can reasonably have hope that we can find islands of consistency in the perceptions of users of multimedia (and in the labels of our annotators). Within these islands we can make stable inferences that will be useful to users.

Let’s check again if, under these definitions, you can make a statement in your paper, “The results show that the phenomenon we are studying is subjective”. This time you can. But in order to do so, you need to have an experiment that shows that the background of the users is what is causing your classifier not to give you stable predictions. Otherwise, it might be the case that your classifier just has not been well designed or trained.

You also need to provide evidence that the protocol that your annotators are using to make judgements is not unduly steering people to diverse interpretations. Your protocol should put people reasonably on the same page, and then ask them for judgements at all times being careful not to ask "leading" questions, cf. [1, 2]. For some research work, you might not be using a protocol. Many tasks involve "found" labels such as tags. In this case, you need to state the assumptions that you are making concerning the original labeling context, including the reasons for which the labels were assigned.

With any definition of subjective, it is important to strictly avoid arguing along these lines: “This phenomenon is subjective, and therefore it is not important and we should not be studying it.” 

Scientifically, there is no a priori reason to prioritize the “objective” over the “subjective” if we use definitions 4a (2) and 3b.  It is true that we tend to study phenomena with high inner-annotator agreement since these are easier to get a handle on. However, at the same time we remain aware that this tendency steers us dangerously close to the famous story of Nasreddin Hodja who looks for his ring outside, since it is too dark inside where he lost it. In short, define “subjective”, but never use it as an excuse for failure or avoidance.

To drive that particular point home: The message is "Keep up your guard". Your problem should arise from the needs of users. Practically, speaking the problem you choose will be influenced by your ability to access the resources needed to study it, including carrying out a well designed, conclusive experiment. It will not, however, be influenced by your personal decision that something is "subjective".

  

Next, we turn to definition 3a: “characteristic of or belonging to reality as perceived rather than as independent of mind.” Using this definition is dangerous. It forces you to take a position on the difference between effects that are real, and effects that are imagined. As scientists, we determine this difference experimentally. We do not presume it. Unless we are undertaking experiments directed a making this difference, it makes sense to steer clear of this definition.

Finally, definition 5: “lacking in reality or substance” The same comment applies as in the case of definition 3a. We cannot a priori say whether patterns that can be found in multimedia content lack reality or substance.  If we don’t find evidence for the reality of some phenomenon in our data, it simply means that there is no evidence for its reality in our data. Lack of observation does not disprove existence. We must guard ourselves against jumping to conclusions. Again, this is a definition to be avoided, unless you are actually directly investigating the nature of reality.

As researchers in the area of multimedia content analysis, we must carefully keep ourselves from creating our own realities: the reality we assume must be the reality (possible multiple realities) of the users that we serve—all of them. The fact that we do not necessarily understand this reality fully, or have the type of information or data that would capture it in its complexity, richness, and continuous, rapid evolution, is a challenge that we face. This challenge is inherent to the types of algorithms and technologies that we design and develop.

Other posts in this blog on subjectivity:

http://ngrams.blogspot.com/2011/08/subjectivity-vs-objectivity-in.html
http://ngrams.blogspot.com/2011/11/affect-and-concepts-for-multimedia.html
http://ngrams.blogspot.com/2016/02/mediaeval-2015-insights-from-last-years.html

[1] Larson, M., Melenhorst, M., Menéndez, M. and Peng Xu. Using Crowdsourcing to Capture Complexity in Human Interpretations of Multimedia Content. In: Ionescu, B. et al. Fusion in Computer Vision – Understanding Complex Visual Content, Springer, pp. 229-269, 2014.

[2] M. Riegler, V. R. Gaddam, M. Larson, R. Eg, P. Halvorsen and C. Griwodz, "Crowdsourcing as self-fulfilling prophecy: Influence of discarding workers in subjective assessment tasks," 2016 14th International Workshop on Content-Based Multimedia Indexing (CBMI), Bucharest, 2016, pp. 1-6.

Choosing movies to watch on an airplane? Compensate for context

For some people, an airplane is the perfect place to catch up on their movies-to-watch list. For these people there is no difference between sitting on a plane and sitting on the couch in their living room.

If you are one of these people, you are lucky.

If not, then you make want to take a few moments to think about what kind of a movie you should be watching on an airplane.

These are our two main insights on how you should make this choice:

• Watch a movie that uses a lot of closeups (or relatively little visual detail), is well lit, and moves relatively slowly so that you can enjoy it on a small screen.
• Watch something that is going to engage you. Remember that the environment and the disruptions on an airplane might affect your ability to focus, and, in this way disrupt your ability to experience an empathetic relationship with the characters. In other words, unless the plot and characters really draw you in movie might not "work" in the way it is intended.

For an accessible introduction to how movies manipulate your brain see the Wired series on Cinema Science article:

http://www.wired.com/2014/08/how-movies-manipulate-your-brain
At the perceptual level your brain needs to be able exercise its ability to "stitch things together to make sense". It's plausible that this "stitching" also has to be able to take place at an emotional level. Certain kinds of distractors can be expected to simply get in the way of that happening as effectively as it is meant to.

When we began to study what kinds of movies that people watch on planes we used these two insights as a point of departure. We started with these insights after having made some informal observations about the nature of distractors on an airplane, which are illustrated by this video.

At the end of the video, we formulate the following initial list of distractors, which impact what you might want to watch on an airplane.

1. Engine noise
2. Announcements
3. Turbulence
4. Small screen 
5. Glare on screen
6. Inflight service
7. Fellow travelers
8. Kid next to you

In short, when choosing a movies to view on the airplane, you should pick a movie that can "compensate for context", meaning that you can enjoy it despite the distractors inherent in the situation aboard an airplane.

We are looking to expand this list of distractors as our research moves forward.

Our ultimate goal (still a long way off) is to build a recommender system that can automatically "watch" movies for you ahead of time. The system would be able to suggest movies that you would like, but above and beyond that the suggested movies would be prescreened to be suitable for watching on an airplane. Such a system would help you to quickly decide what to turn on at 30,000 feet, without worrying that half way through you will realize that it might not have been a good choice.

Since we started this research, I have been paying more an more attention to the experiences that I have with movies on a plane. Here are two.

• On a recent domestic flight: The woman next to me started to watch Penny Dreadful. She turned it off about ten minutes in. I then also tried to watch it. I really like the show, but it's meant to be dark, gory, and mysterious. These three qualities turned into poorly visible, disconcerting, and confusing at 10,000 feet. This is what I am trying to capture in the video above.
• On a recent Transatlantic flight: The man sitting next to me turned on his monitor, and turned on The Color Purple as if it were on his watch list. It's probably on most people's movies-to-watch list, so this seems like a safe choice. However, I was trying to review a paper, and was subject to over two hours of unavoidable glimpses of violence on a screen a few feet from my own. It's the emotional impact of these scenes that make it a great movie. By the same token, you might not want to be watching it on a plane, especially if you are not going to experience the entire emotional arc. (The movie should be watched with full focus, from beginning to end.)

Until now, the work on context-aware movie recommender systems that I have encountered has recommended movies for situations that are part of what is considered to be "normal" daily life, e.g.,  watching movies during the week vs. on the weekend, watching movies with your kids vs. with your spouse. We need more recommender system work that will allow us to get to movies that are suitable for less ideal, more unpleasant, perhaps less frequent situations.  Why waste a good movie by watching it in the wrong context? And why suffer anymore than necessary while on an airplane?

The work is being carried out within the context of the MediaEval Benchmarking Initiative for Multimedia Evaluation, see Context of Experience 2015. It owes a lot to the CrowdRec project, which pushes us to understand how we can make recommenders better by asking people explicitly to contribute their input.

Tagging Love and Affection: Part II

Perhaps even the more interesting thing about the wedding in terms of modern media was the interaction between the professional photographers and the wedding guests who were taking pictures. It seemed like these were two completely different activities in terms of the results that they were aiming to produce. The photographers created an amazing album of storybook moments -- and the guests -- well, speaking for myself at least -- took pictures of people as people that they knew.

The professional photos actually included shots of members of the bridal party and guests photographing the bride and groom and each other. There was a particular dramatic one of the best man from the back taking a picture of the groom and you can see the groom twice: once over the shoulder of the best man in the display of his cell phone and once sitting as the main subject of the image.

There is another one in which one of the bridesmaids is taking a picture of the newly weds. It's like this act of greeting, an 'I was there with you in your moment of bliss and I was so so happy for you.' Taking a picture is like smiling, waving or winking at someone -- except that it is asynchronous, delayed in time. From the past, a shout out, "Congratulations!"

I was struck how the professional photographers were able to use the act of photo-taking as a way to depict the love and affection between friends and family members. Its not just the places that we tag with our photos, as I've discussed in a previous blog post, but its people, too.

And this is where the difference between the professional photographers and the guests really became apparent. I was using the little camera of my mom, so my pictures of the wedding are not qualitatively speaking very good. It would probably take some improvement of my photographic skills and not just a better camera to get high quality pictures.

But there was something that struck us about them. I naturally looked for the people in our family who we see in frequently and took pictures of people talking that only get to see each other once every several years -- if at all. I took pictures of people holding the youngest baby in our extended family -- that capture the moment that generations within the extended family meet for the first time. I took pictures that showed siblings engrossed in conversations with each other -- showing how the intensity of how we speak and how we listen. The photographers didn't know us and although the wedding pictures were beautiful, aesthetically not to be surpassed, we really love to look at the personal pictures, because they are somehow more "us".

Maybe the "real" pictures are the pictures that we take that mark love and affection. They encode our personalities, our common past and our hope for the future.

The implications are quite large for the field of multimedia retrieval, as revealed by the following line of reasoning: Life is finite. We only live so long and can support close relationships with so many people. If Dunbar is right it is a very limited number indeed. If we take photos at particular moments, such as moments of expression of affection that I am describing here, then the number of total pictures that we take is also limited. If we keep on insisting that our multimedia retrieval algorithms must be able to handle millions and millions of photos, then we run the danger of missing out on developing important techniques. Multimedia algorithms developed for relatively small numbers of photos can afford to be computationally more complex. If we ignore the "small set" problem, we run the danger of not developing the best possible algorithms to personal multimedia retrieval challenges.

As final comment, I can add that during the wedding I was already challenged by an image retrieval problem. I took maybe 200 photos. I wanted to show a special photo of my mom -- taken a few minutes back -- to the cousins I was sitting with at the dinner table. It took me so long to flip through the index to find those photos on that small screen. Very disruptive for dinner conversation.

I had the idea that I was the one that should have been wearing the GSR sensor. I am sure that my affective peak was physiologically measurable when I took the picture of my mom, saw it on my camera display for the first time and realized I had gotten a once-in-a-lifetime shot. If my camera display could take me right to peak pictures, it could be a much more functional device: transcending a capture to also support storytelling as well.

On second thought skip the GSR. I'm sure I jumped up and down. The accelerometer on a mobile phone could have picked that up. If all else fails, make a photo taking app that encourages me to shake the thing when I notice I like a picture. Better stop blogging and start implementing.

MediaEval 2010 Workshop Report

We were delighted that Bill Bowles attended the MediaEval 2010 workshop and that he made us our own MediaEval video trailer, in which he tells the story of MediaEval from his own point of view. The MediaEval 2010 Affect Task was devoted to analyzing Bill's travelogue video from his Travel Project and ranking it by how boring viewers reported it to be. As a filmmaker, another rational reaction would be "Who are these people, what did they do to my video? I don't want to get anywhere near them!" But instead, he came, participated and told us about ourselves using the very same medium we devote so much effort to studying.



I was amazed at how quickly this video accumulated views, it quickly outstripped any video I've ever posted to the Internet. However, if video is not your thing and you want the text version of what happend here is the text of a workshop report written for a project newsletter.

MediaEval 2010 Workshop Report

The MediaEval 2010 workshop was held on Sunday, October 24, 2010 in Pisa, Italy at Santa Croce in Fossabanda. MediaEval is a benchmarking initiative for multimedia retrieval, focusing on speech, language and contextual aspects of multimedia (geographical and social context) and their combination with visual features. Its central sponsor is the PetaMedia Network of Excellence. In total, four tasks were run during MediaEval 2010. To approach the tasks, participants could make use of spoken, visual, and audio content as well as accompanying metadata. Two “Tagging Tasks’ (a version for professional content and one for Internet video) required participants to automatically predict the tags that humans assign to video content. An ‘Affect Task’ involved automatic prediction of viewer-reported boredom for Travelogue video. Finally, a ‘Placing Task’ required participants to automatically predict the geo-coordinates of Flickr video. The Placing Task was co-organized by PetaMedia and Glocal. It was also given special mention in the talk of Gerald Friedland entitled “Multimodal Location Estimation” in the “Brave New Ideas” session at ACM Multimedia 2010.

During the MediaEval 2010 workshop, researchers presented and discussed the algorithms developed and the results achieved on the MediaEval 2010 tasks. The workshop drew 29 participants from 3 continents. More information about the 2010 results including participants’ short working notes papers, are available at: http://www.multimediaeval.org/mediaeval2010
Currently, MediaEval 2010 participants are working towards a special session at the 2010 ACM International Conference on Multimedia Retrieval (ICMR 2010), which will be dedicated to presenting extended results on MediaEval 2010 tasks.

Mediaeval 2011 will be organized again with sponsorship from PetaMedia and in collaboration with other projects from the Media Search Cluster. The task offering in 2011 will be decided on the basis of participants' interest, assessed, as last year, via a survey. At this time, we anticipate that we will run a Tagging Task and a Placing Task as well as a couple innovative other, new tasks as dictated by popularity. If you are interested in participating in MediaEval 2011 or if your project would like to organize a task, please contact Martha Larson m.a.larson@tudelft.nl Additional information on MediaEval 2011 is available on the website: http://www.multimediaeval.org

SIGIR 2010 Crowdsourcing for Search Evaluation Workshop

We used Amazon Mechanical Turk (MTurk) to gather annotations for the video corpus to be used in the Affect Task at the MediaEval 2010 benchmark evaluation. The task involves automatically identifying videos that viewers report to be particularly boring. We wrote the corpus development up and submitted it to the Crowdsourcing for Search Evaluation Workshop at SIGIR 2010. Initially we wondered a bit if the paper was appropriate for the workshop, since we were working on affect and not directly on search. But we were glad that we took the risk and went for it. The paper was accepted and the workshop was great -- right on target with our interests.

Soleymani, M. and Larson, M. Crowdsourcing for Affective Annotation of Video: Development of a Viewer-reported Boredom Corpus. In Proceedings of the SIGIR 2010 Workshop on Crowdsourcing for Search Evaluation.

We also received the Runner Up for the Most Innovative Paper Award, which was sponsored by Microsoft Bing. Thank you! We are already considering how to get the most bang for our Bing bucks. Probably it will flow directly back into MTurk for our next crowdsourcing project.

Emotive speech and navigation systems

During a recent family weekend, my best friend, my mother and I found ourselves in a car using my aunt's navigation system to guide us to our destination. We quickly developed a love-hate relationship with the device -- our feelings of annoyance generally outweighing our gratefulness of having been guided efficiently to our destination.

My mom then circulated an article from CNN entitled Why GPS voices are so condescending. And my aunt mailed back, 'Hey, isn't this what you work on?'

The answer to that question is, yes, well, not quite. I go the other direction. Instead of automatically producing emotive speech, I start with a recording of emotive speech and automatically analyze how it was produced. We just got a paper accepted in a session entitled "Paralanguage" at Interspeech 2010 :

Jochems, B., Larson, M., Ordelman, R., Poppe, R. and Thruong, K. Towards Affective State Modeling in Narrative and Conversational Settings. Proceedings of Interspeech 2010 (to appear).

The CNN article also falls into the category of paralanguage. Paralanguage is basically the things that we do with speech that modifies the factual content or conventional meaning of what we are saying. In this case, it's adding emotive nuance.

The designers of navigation systems are stuck with the following impasse: If a navigation system is good, it will always be right. Socially, there is a tabu against always being right. An always-right system will always be perceived as condescending, be its voice ever so loving and sweet. That's simply the way that social behavior works -- we count on each other to act responsibly, but not to pretend that we're perfect. The implication is that we, as humans, will never truly adopt the metaphor of "it's just a person telling me where to drive" for a navigation system if that system's understood purpose is to deliver infallibility.

In my opinion, what the designers of navigational systems should do is to use the voice of someone who enjoys special social status and as such, "gets away" with being always right. For example, theoretical physicist Stephen Hawking. His smarts are generally acknowledged to transcend the smarts of the rest of us mere mortals. Interestingly, he also speaks using a computer voice because he has neuro-muscular distrophy. It wouldn't take a whole lot of memory space on your little navigation device in order to produce a believable rendition of his speech.

The issue also has a huge safety aspect (which is also raised in the CNN article). If the navigational system uses emotive speech in a very convincing manner, it is smooth sailing. However, what if something goes wrong? To the driver, it will be like a thunder-bolt out of a blue sky. Everything was going fine, and all of a sudden the device turned and lashed-out with an emotively inappropriate direction. Possibly, this would happen at a critical driving point. The driver shouldn't be so comfortable with the device as to completely exclude the possibility that it goes way off the mark.

Basically, a car navigation system presents us with another instance of the Paradox of 'Simplicity'. It takes a lot of very complicated innards to make a device that drivers perceive simply as a human telling us how to get there. The paradox comes in when that device does something wrong and all of a sudden the human is stuck both solving the immediate driving issue and also compensating for the apparently inexplicable (those complicated innards!) failure of the system. In this case, for example, a beautifully real rendition of a plaintive tone pleading "Turn back! Turn back!" when actually we find ourselves stuck in the express lane in heavy traffic.

The theoretical physicist persona would help to lessen the impact of such errors. Sorry, Stephen Hawking, but theoretical physicists can get away with being socially inappropriate once in a while without throwing us into a state of shock -- we assume that they are simply busy on a higher plane and don't mean to really insult or confuse us.

However, instead of talking to Stephen Hawking about a deal to have him donate his authority to make navigational systems safer, navigational system companies (according to CNN) are looking into fitting the systems with the driver's own voices. It sounds cool, until you think about some of the implications.

First, there are probably people who don't react well to their own voices. Perhaps I could accept my own voice reminding myself of the route to somewhere I've been before, but my own voice directing me to somewhere I have never been, for example, Makuhari, Japan (where Interspeech 2010 will take place in September) is absolutely implausible. I know I can't trust myself on that one.

Second, drivers need to be encouraged not to turn off their human intelligence when driving with a navigational system. The system doesn't tell you, "Stop here, the light is red". Listening to your own voice is probably not the right way to ensure that you are actively applying the underlying rules and your own common sense to driving.

Third, it's not uncommon to rent a car borrow someone else's car or navigational system on a single-case basis. For example, my aunt lent us her device for one trip. Wouldn't we like our devices more if they were one-size-fits all? Just as Walter Cronkite provided widespread satisfaction as the voice of the evening news, what's wrong with generally-acceptable central voice for all navigation systems?

Fourth, it's not only the driver would needs to listen to the navigation system. With several people in the car, navigation often involves pooling knowledge of the route and negotiating consensus. If the driver's voice is talking on the navigational system, the passengers are shut out of the process. For maximally safe driving, you don't want a "back seat driver", but a co-pilot who is engaged in the process is very helpful.

Fifth, it is not clear that the navigation system companies are the ones that should be making the decision about how navigation system personae can be made more acceptable to drivers. If they can convince individual drivers that they need to have a personalized voice for their system will open up an incredible new opportunity for profit for navigation device companies. On top of the system and the route information, they will also be able to sell you your own personna.

Additionally, a universal "Stephen Hawking" solution, which I am arguing may actually be safer, would make it impossible for navigation system companies to distinguish themselves from each other on the basis of the differential appeal of their navigation personna and is simply not in companies best business interest.

My suggestion is simply to learn to love the condescending dead-pan delivery of your current navigation system -- demanding anything different may prompt the designers of navigation systems to make the situation a whole lot worse.

Don't we do this already? How often have you ever been directed somewhere by a fellow human issuing emotively inappropriate directions? You've reminded yourself to take some deep breaths, stay concentrated on the road and gotten there in the end. We shouldn't demand from our automatic devices more than what we get from our fellow human beings.

P.S. Whoa, this claims to be a blog on the topic of search, what does this have to do with search? OK. You've caught me. Sometimes I just write things here because I know that I can find them again.

Diep Blue

I am sitting in the train from Brussels to Rotterdam. I have work to do, but I am also too tired to do it well. I need to entertain myself instead. What to do? Hmm. Hey, write a blog entry. But I have the strangest problem. I don't have internet connectivity. I can't write. But I could write a post and upload it later.

I still have the same feeling: I just can't write.

Can't write? Well, first, I have my blog drafts stored on blogger.com. I currently have one half finished and I'd like to finish it off. But I can't get to my draft and if I could, I couldn't finish it, since I'd like to put in some links and check some data sources and facts...that requires connectivity. Slowly it strikes me: I carry an inner conviction that I can't write without access to a search engine. Is it true? I can't create without search? I'm moved to prove that this is not true.

Pause of several minutes.

We pass over the Hollands Diep by Dordrecht. Is that really what it's called? I've just always thought that this water here was what people were talking about when they say Hollands Diep.

Well, no way to look that up now. Instead I gaze out on it, the first few seconds wishing that I was making a video of the view from the bridge. It's the fading light. The sky and the water are nearly the same color and the struts of the bridge (hmm, shall I call them struts or girders--can't check that either...) pass by in regular intervals, with a comforting rhythm. I realize the bridge will be over, so I give my thought flow a little nudge towards simply enjoying the view and not wishing I were capturing it.

The conductor comes by and checks my ticket...ok that was another couple minutes.

Now were stopped in Dordrecht...here comes the coffee guy...in Dordrecht still...Hey, at least I can add a nice Dordrecht image to this entry.

The first several don't come out well. Then the station is gone, but there are trees. Rows of trees. That's iconic for the Netherlands, at least in my mind. Click, click, I capture a couple images. Trees and telephone lines. Whereby the telephone lines are strangely crisp against the blurry trees. Color looks interesting grey/blue. That's nice...I can fiddle with it more later.

I've listened to quite a few talks lately on classifying multimedia according to affect. These talks are mostly discussion the emotional impact that an image or a video has on the viewer as opposed to discussion of the affect being experienced by humans depicted in the image or video.

I look at my trees and telephone line shot again and wonder what the emotional impact on the viewer would be. I guess the color would be sadness: the Dutch word verdriet comes to my mind. Of course, I can't go and look up what the affect people say about this color. I think it's a cool color. I also remember one talk in which there was a slide that wanted to convince me that people consider this sort of color masculine.

I remember that slide exactly because it caused me a bit of mental distress. If I had a group of subjects doing user test and I asked them to arrange pictures on a masculine to feminine scale -- what exactly would they be doing? Accessing their emotional experience? Or, and I consider this more plausible, "cheating" by tapping into past baby congratulations card selection experience? Look around the train, where do we see blue? The button that closes the doors. Blue. The button that opens the doors. Yellow. The labels get removed or the malicious switch them, but the regular train riders know. It's convention and not emotion.

I think that when I will look at this picture the future, it will remind me of tiredness. Of how exhausted I was when I took it. Of my failure to capture the moment in the way I thought it should be captured. And how befuddled my brain was both by the tiredness and also by the attempt to write -- to develop a coherent point -- without access to search.

But here we are now. Rotterdam.