I Don't Specifically Wanna be AnarCHI, but it sure does strike a chord...

If I am following ACM CHI 2014 on Twitter, and read a tweet with a picture of a document entitled "Never mind the Bullocks: I Wanna Be AnarCHI: A Manifesto for Punk CHI", I feel compelled to find it, read it and remark.

Except you can't really "read" this paper in the standard sense because it consists of a title, abstract, guitar chords, and lyrics.

It's not a paper, but rather a recipe that gets you to a performance. (You, yes, that's you, who consider yourself to be the "reader".) You have to interact with the thing to make it whole. To interact, you need to follow "instructions" for playing music, formulated using the standard conventions for how guitar music is written.

Which invites the following line of thought: If this paper can be an interactive set of instructions that can be carried out by people familiar with certain standard conventions, then, how far do we get if we view other papers as similar animals?

Unpacking that question: On the one hand, you can say about any paper, that you don't really get a complete entity until you interact with it: read it, interpret it, cite it, extend it. On the other hand, you can say about any paper, if you don't master the system of conventions (I don't know how to play the guitar), you are stuck and do not move beyond square one.

So it's a paper like any other paper, after all? Um, well. I continue with some additional remarks.

The lyrics in the paper refer to the "subtle kind of pressures that go unseen". If this were a film, the mention of the "unseen" would trigger us to expect that other forms of interpretation might lead to more insight. We could call it a foreshadowing or a hint that the message of this paper cannot be directly perceived, but must be witnessed through participation.

If the paper includes the participation, what does this participation tell us? Specifically, what does it tell us about ourselves, the "readers" of "papers"? My remarks regard the first Commentary on the paper.

This Commentary includes the statement, "It is not clear what the authors are trying to accomplish." My remark: This suggests the existence of an assumption among readers (i.e., readers like ourselves) that a paper should represent an attempt at a well-formulated achievement.

The Commentary also includes the statement, "...we are forced to question whether this paper should really be seen as anarchist". My remark: This suggests the existence of an assumption that a paper should not open up possibilities to discover contradictions between its literally expressed message, and its larger implications.

Then, the Commentary includes the statement, "The authors do draw important attention to the government and corporate funding of HCI research...However, we wonder how much of the authors' own research is funded by such." My remark: This suggests the existence of an assumption that the authors of a paper must not advocate actions differing from, or going beyond, those in which they are currently engaged. 

Finally, it includes the question "Would such mainstream acceptance paradoxically undermine the movement's very purpose?" The "movement" is here the purported AnarCHI "movement". My remark: This suggests the existence of an assumption that ideas should not come into being with the anticipation that they will ultimately destroy themselves, but rather, should come into being in order to establish permanence.

Yes, looking at this list, it does look like a set of conventions. And because the authors of this "paper" do not respect these conventions they do not get beyond square one with this Commentary.

However let's ask this: Don't we value exactly the opposite of these assumptions? Don't many of us believe that there needs to be flexibility in research for exploration (not all goals should be well-formulated), for writing papers in which the readers might discover contradictions (see things that the original authors don't), for papers that inspire future scientists that they can be better than us, and for papers that present ideas aimed at moving forward the field as a whole, rather than establishing their contribution as part of an immutable canon.

In the form of these assumptions, the subtle pressures that go unseen have stepped out of the darkness and into the spotlight. They are not inherently bad pressures. Conventions allow us to communicate, just like they allow a guitarist to interpret written music to reproduce the intentions of the compose.

However, they are there. And naturally, they are brightly illuminated when someone invokes punk along with a cool CHI play on words like AnarCHI.

Heck yeah, they are there. And like the gravity that keeps us glued to the surface of the planet, we need, as human beings, heroic acts of will, insight, and technological development in order to, physically, be able to fly.

The Commentary refers to this paper as "exceedingly clever". But maybe it's not clever at all. It could just be considered a standard method to transcend the unspoken assumptions of a mature community, and to realign them with the underlying values privately cherished by that same community, i.e., the values that we would like to believe that we hold and can act on.

The Commentary implies that the paper subverts the mainstream. But it doesn't subvert the mainstream. It reveals the tension that exists inside everyone of us. We struggle to keep our intuitions, investigation, and ambitions free of its destructive load of expecting science to progress in neat, self-consistent packages, self-contained packages with long lifespans in the larger community. It helps us because it encourages us with the reminder that we are not alone.

The Commentary admits the possibility "...we just don't get it". With respect to the duck video, I would be with you.

I've never attended CHI. Maybe I also need to learn how to play the guitar.

Multimedia Of the People, By the People needs multimedia technology For the People

Flick: krazydad

Yesterday, our Multimedia Computing Group had a strategy day at the Delft Arts Center. These are the days that we take time—that strictly time speaking we cannot really spare from deadlines and projects—and step back and look at the larger picture of our group and of multimedia research. The location of the Arts Center is green and the Dutch April obliged us with sun, ideal to take time to contemplate and discuss the bigger questions.

My larger picture is this: I am interested in the digital reflexes of human thought, creativity, and communication. These reflexes lead to the generation of multimedia and interaction data. The natural group with interest in this data is the people that it represents, i.e., the people whose efforts and activities caused it to come into being. I create algorithms and technology that support people in getting the most out of “their” data.

Of course their is a lot of multimedia data out there, also including satellite images, medical images,  and surveillance video. These are, of course, also forms of multimedia. However, typically, the people who generate these data (or the people who create the systems that generate these data), are not themselves represented in the data. 

In my view, multimedia “Of the People, By the People” (i.e., data arising from the creativity and activity of a large number of general-population users) should be distinguished from special purpose multimedia. This means that we need a concept of multimedia systems "For the People” and that these should be the focus of special development effort. 

There are conceptual and algorithmic reasons why multimedia systems "For the People” should be developed as a separate class. One important point is that multimedia of the people and by the people is also generated for multiple purposes, giving rise to complexity not encountered in other systems. For example, a satellite image wouldn’t be expected to be able to fulfill to radically different goals, such as “education” and “entertainment”, whereas such multi-facetedness is quite common for a video on YouTube. We tackled such complexity in a recent publication, “Using Crowdsourcing to Capture Complexity in Human Interpretations of Multimedia Content”, but I will not discuss it further here.

Here, instead, I would like to focus on the ethical aspects of why multimedia “Of the People, By the People” should be the subject of dedicated research devoted to creating multimedia systems for users. My motivation was a conversation during yesterday's lunch that came scarily close to arriving at the question, “Do we really have time for ethics in our research anyway?” Given that it required a significant amount of sacrifice to make time for the strategy day, I was really not ready to come to the conclusion that, whoops, no, after all, no time for ethics. I tried to make the point that ethics does not necessarily require time, but rather it involves making explicit choices that bear our research forward in one direction as opposed to another. The difference between the directions may be subtle, but the implications are huge.

Differentiating multimedia “Of the People, By the People” from other types of multimedia is a way at looking at what we do that opens to the door to consideration of the ethical aspects of our research. In particular, if we target multimedia systems "For the People", our research makes a contribution to supporting the dialogue on an important issue of the digital age: the right of individuals and communities to benefit from the data produced as a result of their own creativity and activity. I am not asserting that creating multimedia technologies "For the People" is in itself an act that one can consider “ethical” Rather, that choosing to do so, and making this choice explicit, we support people working in other disciplines in defining, and finding answers, to the tough ethical questions.

Social multimedia sharing on the internet has led to the creation of incredible resources. YouTube and Flickr contain an enormous amount of material that teaches, records aspects of human life, or represents the product of creative talent. In order that multimedia resources are able to reach their full potential of serving the good of individuals and society as a whole, it is important that technical tools are developed that will allow these resources to be searched and browsed. 

A somewhat trivial way of understanding (or "feeling") the way in which the people who created these resources also have a right to them is this, let's try a simple exercise. Imagine Google “turned off” YouTube. Would you feel that something had been taken away from you? Would schools, communities, social relationships between people be damaged? The answer is clearly “yes”. I am not going to far as to make the argument that YouTube videos should be considered public goods. There are complicated intellectual property issues at play there. The point here is simple: “turning off” YouTube would have larger implications than of “just” a company making a business decision for business reasons.

Let’s look at a less trivial example: Imagine that Google does not turn off YouTube, but rather starts to spend less effort on maintaining it (i.e., that there are streaming problems, or that we can’t find what we are looking for.) That’s a slippery slope going down from what YouTube is today, to the point of “turning it off”. Google's position on that slope again has consequences for ability of people to access content that they themselves have created. In other words, that position goes beyond mere business decisions.

Finally, let's turn to where we are today: Google makes certain decisions about the types of multimedia search technologies to develop and not to develop for YouTube. These decisions have consequences. One decision in which these consequences are perhaps most clearly evident is that YouTube does not provide time-code level search functionality to videos. This means that, for example, a student studying data fitting, can’t directly search for short segments of lecture videos explaining least squares, but instead, must do a significant number of video-level searches. 

Of course it would be a strange, if not bad, business decision to make the search function of YouTube capable of precisely addressing specific questions. Users that come, get their question answered with one query, and leave again, are not going to hang around and click the ads that make such a critical contribution to revenue. 

The point is that decisions whether or not to develop certain technologies (and I can assure you that time-code level search is non-trivial) have implications for the ability of the community who created and contributed the content also to be able to use it. 

In order to drive this point home, let me give one more example, based on Wikipedia. There is wide-spread agreement that Wikipedia is peer-produced resource that is important for society world wide. In the case of Wikipedia, the importance of access technology in guaranteeing people’s ability to make use of the resource is clear. Imagine that all of Wikipedia were dumped into an enormous text file without structure and without a search function. Would it be useful to the larger community? Scarcely. The “right” to use Wikipedia is guaranteed not only by the Creative Commons license under which its content is created, but also by the structure of the Wikipedia site and the search functionality that it offers. 

As systems get more complicated, and involve processing of pixels and sound samples, we cannot guarantee that appropriate search and access technologies will be developed without significant effort. The “ethical” decision to make as a multimedia researcher in academia is to focus efforts on those research topics that lead to multimedia systems “For the People” that industry is not necessarily incentivized to address.

Note that I am not asserting that companies like Google, are not working to making access to user-contributed collections of multimedia better. Quite to the contrary. Flickr, for example, just announced integration of object recognition http://techcrunch.com/2014/04/17/yahoo-acquisitions-power-flickrs-new-object-recognition-search-editing-and-video-capture/).

However, in order to remain profitable, companies must pay careful attention to their bottom line. For this reason, it is important that actors outside of industry (such as universities) remain actively engaged in developing technologies that help people get the most of multimedia data that they themselves have created. Such actors experience no conflict of interest, and care thus free to stand squarely on the side of the people who created the data when conceiving and developing new multimedia algorithms.

How much time do we need to consider ethics in our research? Not that much. We greatly benefit from a few odd moments of discussion and reflection. Yesterday, I welcomed the chance to view the big picture, and to step back and ask if I, as a multimedia researcher, am truly serving the interests of the people whose effort and action gave rise to the data that I study in the first place.

The Importance of Information Friction: Safer Societies and Subtle Satisfactions

Wet leaves abound on the ground these days in Europe, and when I walk the streets they lead me to think about the critical role that friction plays in our worlds. I own a special pair of dance shoes that give just the right amount of friction to let me glide across the floor. More would prevent me from dancing. But less, oh, less could be disastrous---movement from one spot to another would be uncontrolled and dancing would be impossible.

If you stop to think about it, it really is a similar phenomenon that makes the stories that we tell interesting. Imagine that there were no friction and that the punchline of a joke would always simply slip out of our mouths with the first line. Its not a nasty as taking a tumble during the tango, but if we collectively lost the ability to tell jokes and stories, it would not be making the world a better place.

For us to entertain each other in this way, some kind of an "information friction" is necessary, i.e., a force that holds back everyone knowing everything all the time.

This observation seems a bit of a trivial, given that concerns about eroding "information friction" are more frequently related to "big issues" involving, e.g., involving safety in society.

For example, people worry about private information being leaked that can do them serious damage. This is a legitimate concern. We are thankful when the dangers of sharing are treated constructively by the news media, e.g., ConsumerWatch: Social Media Users May Be Revealing Too Much About Location « CBS San Francisco

Here, the benefit of information friction is clear. On Twitter information about people's location travels as far and as fast as light. Unlike light, it's not gone once the source stops emitting, but rather hangs around and reveals patterns capable of haunting or even hurting, should they be traced and exploited by people with evil intent. We still want to share with others around us, but this sharing should have some natural limits, just like the physical forces that keep us from flying around without control in the real world.

We share the vision of a world in which governments can exploit patterns in real-time personal data to prevent of mitigate the effects of epidemics and catastrophes:  http://www.theguardian.com/media-network/media-network-blog/2013/sep/05/combating-epidemics-big-mobile-data
But another "big issue" is that we don't want to trust every government with all the data in the world. We would appreciate a bit of information friction if it could keep our data a bit closer to home and in hands that we feel we can count on because we know them well.

Here, I would like to explore another, less dramatic aspect to information friction: I would like to highlight the critical role that friction plays in the fun, loveliness and delight of life. In addition to jokes, as I described above, think also about movies. We see online communities carefully creating information friction: adding spoiler alerts so that viewers don't come to know the end of the movie until the time is perfectly right for it.

Browsers offer private browsing, which seems to me the only way that it is possible to buy anyone a gift these days---anyone (family member, roommate, colleague), that might happen to glance at the screen of your computer that is. Currently, my browser is happily showering me with recommendations for flights to the destinations that I have recently investigated using my search engine. Some may see benefit in this added inconvenience for people who are trying to deceive their loved ones. But the romantics among us regret the passing of the days in which we could book a surprise getaway trip for two without having to remember to adjust the browser settings.

We need a certain amount of information friction to make it possible to give gifts to each other. To surprise each other with unexpected kindnesses. To perform magic tricks, pass on secret recipes to the next generation, to have fun with puzzles and riddles and generally create a sense of joy and wonder.

So yes, limits to the flow of information will keep us safer from the dangers of misuse or misinterpretation of private data by evil others. But it will also keep the underlying exchanges that bind us together in relationships, social groups and societies alive and well. We need to be able to tell each other stories and to give each other little presents. For these exchanges to work, information friction must be balanced to admit flow, but also to restrict it in just the right way.

It's not a simple balance to achieve. In the real world, so much is simply handed us by physics. Friction simply exists and doesn't have to be created for a specific purpose. The digital world, on the other hand, provides a serious challenge: we can't count on social gravity giving us a constant acceleration.

I'll end by noting the reason that I think it is so important to relate the need for the right amount of information friction not only to "safer societies" but also to the smaller things like jokes and surprises, what I call "subtle satisfactions" in the title of this post. It is fatiguing to self-control our level of social sharing with alarmist thoughts preventing the revelation of too much information leading to burglary or kidnapping. Rather, our brains prefer focusing on the enjoyable and comforting parts of life. We can exercise a more gentle rein by considering the positive: Contributions to protecting individuals' privacy online also uphold a world where your loved one can give you the most wonderful gift of your life on your birthday without having to execute technical gymnastics.

Rather than only working towards an world in which bad things are impossible, we should realize that we are also working towards a world in which the good things stay possible. Achieving and maintaining information friction balance is a lot of work, so I do not doubt that we can use both motivators.

Power behind MediaEval: Reflecting on 2013

Power behind MediaEval 2013

The MediaEval 2013 Workshop was held in the Barri Gòtic, the Gothic Quarter, of Barcleona on 18-19 October, just before ACM Multimedia 2013 (held in Barcelona). The workshop was attended by 100 participants, who presented and discussed the results that they had a achieved on 11 tasks, 6 main tasks, and on 5 "Brave New Tasks" that ran for the first time this year.

The workshop produced a working notes proceedings containing 96 short working papers, and has been published by CEUR-WS.org.

The purpose of MediaEval is to offer tasks to the multimedia community that  support research progress on multimedia challenges that have a social or a human aspect. The tasks are autonomous and each run by a different team of task organizers. My role within MediaEval is to guide the process of running tasks, which involves providing feedback to task organizers and sending out the cues that keep the tasks ruining smoothly and on time. Today I did a quick count that revealed that during the 2013 season, I wrote 1529 personal emails to people that contained the keyword "MediaEval" in them.

What makes MediaEval work, however, cannot be expressed in numbers. Rather, it is the dedication and intensive effort of a large group of people, who propose and organize tasks and carry out the logistics that make the workshop come together. My motivation to continue MediaEval year after year stems largely from an underlying sense of awe at what these people do: both at the work that I am aware of and also at the many things that they do behind the scenes that make largely invisible. These people are the power behind MediaEval. Here I represent them with the picture above, which are the power plugs arranged by Xavi Anguera from Telefonica with the assistive effort of Bart Thomee from Yahoo! Research. The process involved a combination of precision car driving and applied electrical engineering.

In the airplane back from Barcelona yesterday, I finished processing the responses that we received from the participant surveys (collected during the workshop), input from the organizers meeting (held on Sunday after the workshop), and feedback that people gave me verbally during ACM Multimedia (last week). These points are summarized below.

Thus endeth MediaEval 2013, but at the same time beginneth the season of MediaEval 2014. Hope to have you aboard.

Community Feedback from the MediaEval 2013 Multimedia Benchmarking Season + Workshop

The most important feedback point this year was the new structure of the workshop, which was very well received. This year the workshop was faster paced and we introduced poster sessions. We were happy that people liked the short talks and that the poster sessions were considered to be useful and productive. There is a clear trend to preferring there to be more discussion time at the workshop, both in the presentation sessions and in the poster sessions. An idea for the future is to separate passive poster time (posters are hanging and people can look at them but the presenter need not be present) from active poster time (presenter is standing at the poster).

The number one most frequent request was for MediaEval to provide more detailed information. This request was made with respect to a range of areas: descriptions of the tasks should always strive to be maximally explicit; descriptions of the evaluation methods should be detailed and available in a timely manner; task overview talks at the workshop should contain examples and descriptions that allow a general audience (i.e., people who did not participate in the task) to understand the task easily.

Other suggestions were to increase consistency check and continue to promote industry involvement. Finally, requests for more time for preparation of presentations and to explicitly invite (and support) groups to make demos with the posters.

The organizers meeting on Sunday was the source of additional feedback. Task organization requires a huge amount of time and dedication from task organization teams and it is important that this is distributed as evenly as possible across the year and across people. In general, tasks would benefit from additional practical guidance on organization. This includes task management and evaluation methodologies. Since MediaEval is a decentralized system, the source of this guidance must be people with past experience with task organization and communication between tasks. Here, the bi-weekly telcos for organizers are an important tool.

In the coming year, the awards and sponsorship committee can expect an expanded role. The outreach to early-career researchers and to researchers located outside of Europe (in the form of travel grants) is seen by the organizers to be not merely a "nice-to-have", but rather a central part of MediaEval's mission. There is solid consensus about the usefulness of the MediaEval Distinctive Mentions (MDMs). MDMs are peer-to-peer certificates awarded by task organizers to each other or to the participants of their tasks. The MDMs  allow the community to send public messages between members of the community, and especially to point out participant submissions that are highly innovative or have particularly high potential (although they may not have been top scorers according to the official evaluation metric). It is important to make clear that the MediaEval Distinctive Mention is not an "award", since the process by which they are chosen is intentionally kept very informal. In the coming year, we will be investigating the issue of whether MediaEval should introduce a five-year impact award, that would be more formal in nature. The peer-to-peer MDMs will be maintained, although and effort will be made to make them increasingly transparent.

In general we were satisfied with the process used to produce the proceedings. Having groups do an online check of their metadata was helpful. If future years also involve proceedings with 50+ papers, we will need to further streamline the schedule for submission---with the ultimate goal of having the proceedings online at the moment that the workshop opens.

CrowdMM 2013: Crowdsourcing in Mutlimedia Emerged or Emerging?

CrowdMM 2013, the 2nd International ACM Workshop on Crowdsourcing for Multimedia, was held in conjunction with ACM Multimedia 2013 on 22 October 2013. This workshop is the second edition of the CrowdMM series, which I have previously written about here. This year it was organized by Wei-Ta Chu (National Chung Cheng University in Taiwan) and Kuan-Ta Chen (Academia Sinica in Taiwan) and myself, with critical support from Tobias Hossfeld (University of Wuerzburg) and Wei Tsang Ooi (NUS). The workshop received support from two projects funded by the European Union, Qualinet and CUbRIK.

During the workshop, we had an interesting panel discussion with the topic "Crowdsourcing in Mutlimedia Emerged or Emerging?" The members of the panel were Daniel Gatica-Perez from Idiap (who keynoted the workshop with a talk entitled, "When the Crowd Watches the Crowd: Understanding Impressions in Online Conversational Video"), Tobias Hossfeld (who organized this year's Crowdsourcing for Multimedia Ideas Competition) and Mohammad Soleymani from Imperial College London (together we presented a tutorial on Crowdsourcing for Multimedia Research the day before). The image above was taken of the whiteboard where I attempted to accumulate the main points raised by the audience and the panel members during the discussion. The purpose of this post is to give a summary of these points.

At the end of the panel, the panel together with the audience decided that crowdsourcing for multimedia has not yet reached its full potential, and therefore should be considered "emerging" rather than already "emerged". This conclusion was interesting in light of the fact that the panel discussion revealed many areas in which crowdsourcing represents an extension of previously existing practices, or stands to benefit from established techniques or theoretical frameworks. These factors are arguments that can be marshaled in support of the "emerged" perspective. However, in the end the arguments for "emerging" had the clear upper hand.

Because the ultimate conclusion was "emerging", i.e., that the field is still experiencing development, I decided to summarize the panel discussion not as a series of statements, but rather as a list of questions. Please note that this summary is from my personal perspective and may not exactly represent what was said by the panelists and the audience during the panel. Any discrepancies, I hope, rather than being bothersome, will provide seeds for future discussion.

Summary of the CrowdMM 2013 Panel Discussion
"Crowdsourcing for Multimedia: Emerged or Emerging"

Understanding: Any larger sense of purpose that can be shared in the crowdsourcing ecosystem could be valuable to increase motivation and thereby quality. What else can we do to fight worker alienation? Why don't taskaskers ask the crowdworkers who they are? And vice versa?

Best Practices: There is no magic recipe for crowdsourcing for multimedia. Couldn't the research community be doing more to share task design, code and data? Would that help? Factors that contribute to the success of crowdsourcing are watertight task design (test, test, test, test, test, test the task design before running a large scale experiment), detailed examples or training sessions, inclusion of verification questions, and making workings aware of the larger meaning of their work. Do tasks have to be fun? Certainly, they should run smoothly so that crowdworkers can hit a state of "blissful productivity".

History: Many of the issues of crowdsourcing are the same ones encountered when we carry out experiments in the lab. Do we make full use of the carry over? Crowdsourcing experiments can be validated by corresponding experiments that have been carried out in a lab environment. Do we do this often enough?

Markets: Many of the issues of crowdsourcing are related to economics, and in particular to the way that the laws of supply and demand operate in an economic market. Have we made use of theories and practices from this area?

Diversity: Why is crowdsourcing mostly used for image labeling by the community? What about other applications such as system design and test? What about techniques that combine human and conventional computing in online systems?

Reproducibility: Shouldn't reproducibility be the ultimate goal of crowdsourcing? Are we making the problem to simple in the cases that we struggle with reproducibility? Understanding the input of the crowd as being influenced by multiple dimensions can help us to better design crowdsourcing experiments that are highly replicable.

Reliability: Have we made use of reliability theory? How about test/retest reliability used in psychology?

Uncertainty: Are we dealing with noisier data? Or has crowdsourcing actually allowed us to move to more realistic data? Human consensus is the upper limit of what we can derive from crowdsourcing, but does human consensus not in turn depend on how well the task has been described to crowdworkers? What don't we do more to exploit the whole framework of probability theory?

Gamification: Will it solve the issues of how to effectively and appropriately incentivize the crowd? Should the research community be the ones to push forward gamification (Does any of us realize how many people and resources it takes to make a really successful commercial game?)

Design: Aren't we forgetting about a lot of work that has been done in interaction design?

Education: Can we combine crowdsourcing systems with systems that help people learn skills that are useful in real life? In this way, crowdworkers receive more from the system in exchange for their crowdwork, in addition to just money.

Cats: Labeling cats is not necessarily an easy task. Is a stuffed animal a cat? How about a kitten?

Towards Responsible and Sustainable Crowdsourcing

Humans are the ultimate intelligent systems. Units of human work can be used to address the problems studied in the fields of pattern recognition and artificial intelligence. After years of research to crack certain tough problems, mere utterance of the phrase "human cycle" makes it seem like someone turned on a light in the room. Suddenly, we feel we are no longer feeling our way forward in darkness as we develop solutions. Instead, a bright world of new possibilities has been opened.

The excitement that crowdsourcing has generated in computer science is related to the fact that large crowdsourcing platforms make it possible to apply abstraction to human input to the system. It is not necessary to consider who exactly provides the input, or how they "compute" it, rather the human processor can be treated as a black box. The magic comes when it is possible to make a "call the the crowd" and be sure that there will be a crowdworker there to return a value in response to that call.

However, crowdsourcing raises a whole new array of issues. At the same time that we excitedly pursue the potential of "Artifical artificial intelligence" (as it's called by MTurk), it is necessary to also remember "Human human computation".

I am not an ethicist, and my first foray into crowdsourcing ethics was relatively recent and necessarily superficial. In fact, I started by typing the word "ethics" into my favorite mainstream search engine and picking a definition to study that seemed to me to be authoritative. However, I am convinced that the community of crowdworkers and taskaskers together form an ecosystem and that the main threat to this ecosystem is that we treat it irresponsibly.

In other words, we should not throw out everything that we have learned over centuries of human civilization about creating healthy and happy societies, stable economies and safe and fulfilled individuals in our quest to create new systems. Ultimately, these systems must serve humanity as a whole, and not disproportionately or detrimentally lean on the portion of the population that serves as crowdworkers.

Because of this conviction, I have put together a set of slides about responsible crowdsourcing that serve as notes on the ethical aspects of crowdsourcing. At a recent Dagstuhl seminar entitled, "Crowdsourcing: From Theory to Practice and Long-Term Perspectives" I used the slides to make a presentation intended to serve as a basis for opening a discussion on the ethical issues of crowdsourcing.

The hopeful part of this undertaking is that it revealed many solutions to address ethical aspects of crowdsourcing. Some of them pose challenges that are just as exciting as the ones that motivated us to turn to crowdsourcing in the first place.

Please see the references in the slides, and also these links:

http://crowdwork-ethics.wtf.tw
http://www.slideshare.net/mattlease/lease-statisticsethics

What is multimedia?

These days a mirrored ceiling is an unambiguous call to take a photo.

Yesterday evening after the conclusion of a very successful First Workshop on Speech, Language, and Audio in Multimedia (SLAM 2013) in Marseille, participants naturally drifted to various scenic spots for debriefing, including the Vieux Port. There, over a glass of pastis (predictable, given the locale) the conversation naturally moved to the question, "What is multimedia?"

One obvious answer to the question is the Wikipedia definition of multimedia, "Multimedia is media and content that uses a combination of different content forms". The classic example, is of course, video, which has a visual modality and also an audio modality. Other examples include social images (for example, images on Flickr have a visual modality, but also have tags and geo-tags) and podcasts (which have an audio modality and also a textual modality included in their RSS feeds).

One can argue about that answer, for example, by pointing out that some people define multimedia as being any non-text media. For example, an image, like the one above. The image was taken at the new events pavilion in the Vieux Port in Marseille. The events pavilion is basically a set of columns with a plane laid on top of them, the bottom of the plane is shiny, so that when you stand under it you are looking up at a ceiling, which is a large mirror.

In my view, an image in isolation cannot be taken to be multimedia since it includes a single medium, name pixels. It become multimedia in conjunction with this blog post, which adds an additional medium, namely text.

Another dimension for the debate on multimedia is whether it must necessarily involve human communication. The combination of this image and this blogpost were created by me with the intent to communicate a message to a certain audience, i.e., the readership of my blog (which, as I have previously mentioned, is largely a few fellow researchers in conjunction with future instantiations of myself).

Researchers who share my view of multimedia, insist on the point that multimedia must contain a message. It must come into being as an act of communication and also be consumed in a process that involves the interpretation of meaning. This definition excludes a set of geo-tagged surveillance videos as being multimedia, although they would involve two different media, namely video content and geo-coordinates.

Note that when you require multimedia to contain a message created with explicit human intent, you enter a bit of a slippery slope. If a human being had set up the surveillance cameras with the specific intent of creating a body of information that would give fellow humans information on current street conditions, then we are back to multimedia.

The slipperiness of the definition of multimedia reveals to us something very important about our field. In order to know whether or not something is multimedia, it is not sufficient to examine the multimedia data, rather it is also necessary to look at the production and consumption chain. Multimodal data in some contexts remains just data, but if an act of encoding and decoding meaning is involved, then the same multimodal data must be considered to be multimedia.

A report on SLAM 2013 has appeared in SIGMM Records. The SLAM 2014 website went online immediately after SLAM 2013 and the anticipation is already building.