Passing the time waiting for Wikipedia by musing on SOPA and multimedia information retrieval

It's like the SOPA blackout blacked out my brain or something. I am tired and cranky and feel like I basically spent my day just waiting for Wikipedia to come back on line. I guess I was actually pretty productive, but I did spend a lot of time looking at Twitter. And in fact was looking when a tweet on darkening Flickr photos came through from Mor Naaman and led me to his darkened photo in the screenshot above.

Yes, of course I am a stop SOPA person. If its piracy today, it's free speech under attack tomorrow... my thoughts naturally flow along those lines of logic. If online piracy is indeed a problem, there's gotta be a more creative, effective solution that one concocted by Congress.

But then it occurred to me that maybe SOPA actually would have positive implications for the effectiveness of multimedia retrieval. I mean, hey, if SOPA is going to force website owners to police user-contributed material anyhow, then we should put those website owners to work enforcing complete and consistent metadata.

For every video, every image, SOPA wants to know who it belongs to. Well, we could also require a complete description of its visual content. Stop Online Messy Metadata! Let's propose a SOMMA.

Let's require each multimedia items to have a minimum number of tags, and have those tags to be drawn a set of concepts organized in a formal ontology. The US Federal Government could actually start publishing one -- updating it each year to cut down on those concepts known to not contribute constructively to retrieval. (We could start with LSCOM.)

Let's require people posting videos to make sure that their speech transcripts have been corrected are are flawless (...avoid using words not in the vocabulary of the speech recognizer) and that their songs are published with links to pages containing the lyrics (...avoid including references to obscure Russian novels).

And then to control that overwhelming multimedia flood, we can set limits. Individuals should only publish a small quota of items, so that these can all be kept in a central index and found by a topical look up -- Dewey Decimal Classification style.

Formulating an information need is easy: just choose one from a well organized standardized list, defined to cover everything you could possible want to see in an image.

Basically, if they put their minds to it the US Federal Government could really improve multimedia retrieval.

Well, OK. I'm a stop SOPA person because I like my job and I like my messy world and the messy metadata that goes with it.

In its subtle way, Flickr is telling us that its actually really difficult as an Internet user to be pro-SOPA. The caption on the blacked out photo above is:

A Flickr user has darkened this photo for up to 24 hours as a symbolic gesture to raise awareness about the potential impact of two pieces of pending U.S. anti-piracy legislation - SOPA (The Stop Online Piracy Act) and PIPA (The Protect IP Act).

While the inclusion of this blackout screen is merely a symbolic gesture -- not an actual allegation that this photo violates any copyright -- a user wanted you to take note of the harmful impact these bills could have on innovation online. For more information see http://blog.flickr.net

And then at the bottom there's a link labeled:

Show the photo anyway. I do not wish to participate. Do not darken my photo.

OK. Wait a minute, if I click "Do not darken my photo" is that then literally or symbolically? Flickr is subtly showing us how natural the extension is from our daily use habits to the decision to support the principle-based movement to stop SOPA and preserve the basic freedom of Internet flow.

Look at how this works: Symbolically, what the darkened photo is, is a big message screaming "Do not darken my photo!!" to the US Federal Government. So aren't I really by clicking this link demonstrating that I am against a specific darkened photo, which is nearly the same thing as being against darkened photos in general? And from there also against legislation that might blot out some of that rich multimedia primordial soup that we take in, digest and put out as conversations, comments, more multimedia, the fabric of our online lives?

Hey, SOPA, leave my Internet alone. I do not wish to participate. I want my brain lit back up.

Where's Wikipedia?

The ACM Multimedia Grand Challenge is a high-adrenaline event where researchers from the Multimedia community compete against each other to develop the best solutions to problems posed by industry. For example, Google formulated two challenges, Video Genre Classification and Personal Diaries, in this year's competition.

Today in Tokyo at Interspeech 2010, I stopped to chat with last year's Grand Challenge winner, who is competing once again this year. I was struck anew by the realization that in the pressure-cooker of the Grand Challenge, creativity, raw intelligence, technical competence, competitive drive and off-beat thinking gives rise to lines of attack that might never have emerged in a traditional R&D setting. Such solutions stand to benefit us all.

But is it really only industry who should be formulating the challenges for such competitions? Where, for example, is Wikipedia? If there is any major player in the Internet information arena that deserves a crowd-sourced solution from the research community, it is Wikipedia, the knowledge resource homegrown by collaborative effort.

Wikipedia does truly inspire the research community. Very recently I've witnessed up close how fired up scientists get about Wikipedia. The Tribler team, who sit on the ninth floor of our building, have been sinking unbelievable time and effort into the development of the Swarmplayer V2.0. Their dedication is inspiring and their incredible belief in the power of a distributed solution for videos on Wikipedia is infective.

Datasets from Wikipedia have been used by multiple benchmarking initiatives such ImageCLEF and INEX as well as in MediaEval, the benchmark I co-ordinate. We certainly enjoyed coming up withour own Wikipedia-related task. However, it would be great to hear directly from the Wikimedia Foundation, in the form of a Grand Challenge, what problems they see on the horizon in the next 2-5 years for which the research community could be helpful in generating solutions. The Challenge takes the form of a simple textual description of the problem and researchers do the rest, presenting the solution in form of a system or system demo and a paper describing it.

There's a lot out there of course that I don't know about. For example, just read this post on the ECML PKDD 2010 Data Challenge: Measuring Web Data Quality. But I've never seen a clear Challenge originating from the Wikipedia community and published for the research community.

One aspect that researchers need to think seriously about, however, is the form in which solutions for Wikipedia or developed using Wikipedia data are published. ACM Multimedia Proceedings are not an open access publication. It's a contradiction to carry out research on a free knowledge resource and publish results under conventional copyright. Peer-reviewed open access journals such as the Journal of Digital Information should be preferred when publishing results obtained using Creative Commons licensed data.

Maybe that's actually one Challenge that the Wikimedia Foundation actually has to offer the research community: challenging us to breaking the habit of creating solutions in a rush of creative joy and technical muscle, and then publishing them where they cannot be accessed by everyone.