Tagging Love and Affection: Part II

Perhaps even the more interesting thing about the wedding in terms of modern media was the interaction between the professional photographers and the wedding guests who were taking pictures. It seemed like these were two completely different activities in terms of the results that they were aiming to produce. The photographers created an amazing album of storybook moments -- and the guests -- well, speaking for myself at least -- took pictures of people as people that they knew.

The professional photos actually included shots of members of the bridal party and guests photographing the bride and groom and each other. There was a particular dramatic one of the best man from the back taking a picture of the groom and you can see the groom twice: once over the shoulder of the best man in the display of his cell phone and once sitting as the main subject of the image.

There is another one in which one of the bridesmaids is taking a picture of the newly weds. It's like this act of greeting, an 'I was there with you in your moment of bliss and I was so so happy for you.' Taking a picture is like smiling, waving or winking at someone -- except that it is asynchronous, delayed in time. From the past, a shout out, "Congratulations!"

I was struck how the professional photographers were able to use the act of photo-taking as a way to depict the love and affection between friends and family members. Its not just the places that we tag with our photos, as I've discussed in a previous blog post, but its people, too.

And this is where the difference between the professional photographers and the guests really became apparent. I was using the little camera of my mom, so my pictures of the wedding are not qualitatively speaking very good. It would probably take some improvement of my photographic skills and not just a better camera to get high quality pictures.

But there was something that struck us about them. I naturally looked for the people in our family who we see in frequently and took pictures of people talking that only get to see each other once every several years -- if at all. I took pictures of people holding the youngest baby in our extended family -- that capture the moment that generations within the extended family meet for the first time. I took pictures that showed siblings engrossed in conversations with each other -- showing how the intensity of how we speak and how we listen. The photographers didn't know us and although the wedding pictures were beautiful, aesthetically not to be surpassed, we really love to look at the personal pictures, because they are somehow more "us".

Maybe the "real" pictures are the pictures that we take that mark love and affection. They encode our personalities, our common past and our hope for the future.

The implications are quite large for the field of multimedia retrieval, as revealed by the following line of reasoning: Life is finite. We only live so long and can support close relationships with so many people. If Dunbar is right it is a very limited number indeed. If we take photos at particular moments, such as moments of expression of affection that I am describing here, then the number of total pictures that we take is also limited. If we keep on insisting that our multimedia retrieval algorithms must be able to handle millions and millions of photos, then we run the danger of missing out on developing important techniques. Multimedia algorithms developed for relatively small numbers of photos can afford to be computationally more complex. If we ignore the "small set" problem, we run the danger of not developing the best possible algorithms to personal multimedia retrieval challenges.

As final comment, I can add that during the wedding I was already challenged by an image retrieval problem. I took maybe 200 photos. I wanted to show a special photo of my mom -- taken a few minutes back -- to the cousins I was sitting with at the dinner table. It took me so long to flip through the index to find those photos on that small screen. Very disruptive for dinner conversation.

I had the idea that I was the one that should have been wearing the GSR sensor. I am sure that my affective peak was physiologically measurable when I took the picture of my mom, saw it on my camera display for the first time and realized I had gotten a once-in-a-lifetime shot. If my camera display could take me right to peak pictures, it could be a much more functional device: transcending a capture to also support storytelling as well.

On second thought skip the GSR. I'm sure I jumped up and down. The accelerometer on a mobile phone could have picked that up. If all else fails, make a photo taking app that encourages me to shake the thing when I notice I like a picture. Better stop blogging and start implementing.

Tagging Love and Affection: Part I

Oh, my baby cousin is all grown up and got married. The wedding was a fairy tale: the kind that they show in the last scene of a good movie where you then sit through the entire credits in hopes that your eyes are reasonably dried up by the time you walk out into the bright public space beyond the theater.

The picture shows the bride's foot. Maybe one would expect a glass slipper, but this is a galvanic skin response sensor that recorded throughout the ceremony so that the bride and groom can later transcribe a mutual emotional trajectory. The sensor records skin conductance, which is related to moisture, and the signal must have been off the charts -- it was a wonderful wedding and but it was also a beautiful sunny day. Sunny in the intense sense, where you gain an practical understanding of why the British royal family feels that weddings require large-brimmed hats.

The nuptial couple must have been perspiring at least lightly and I suspect their respective sensors registered one continuous high throughout the proceedings. And indeed: I wish them this for their married lives that their joint signal continuously reflects a wonderful life experience. Or, if it's not a continuous high, may they at least be gifted with the ability to recognize any large dips as outliers and ignore them.

Lasting love is a complex phenomenon and I suspect that the most reliable expressions of it are more under our conscious control. In the next post I relate it to ... picture taking!