Why should recommender system researchers care about platform policy?

In this post, I reflect on why recommender system researchers should care about platform policy. These reflections are based on a talk I gave last week at the Workshop on Online Misinformation- and Harm-Aware Recommender Systems (OHARS 2020) at ACM RecSys 2020, which was entitled, "Moderation Meets Recommendation: Perspectives on the Role of Policies in Harm-Aware Recommender Ecosystems."

Every online platform has a policy that specifies what is and what is not allowed on the platform. Platform users are informed of the policy via platform guidelines. All major platforms have guidelines, e.g., Facebook Community Standards, Twitter Rules and Policies, Instagram Community Guidelines. Amazon's guidelines are sprawling and a bit more difficult to locate, but can be found at pages like Amazon Product Guidelines and Amazon restricted products.

Policy is important because it is the language in which the platform and users communicate about what constitutes harm and needs to be kept off the platform. Communicating via policy, which is expressed in everyday language, ensures that everyone can contribute to the discussion of what is and is not appropriate. Communication via technical language or computer code would exclude people from the discussion. The language of policy is what offers the possibility (which should be used more often) for us to reach consensus on what is appropriate. It also acts as a measuring stick to make specific judgements in specific cases, which is necessary in order to enforce that consensus completely and consistently.

Policy is closer to recommender system research that we realize

On the front lines of enforcing platform policy are platform moderators. Moderation is human adjudication of content on the basis of policy. Moderators keep inappropriate content off the platform. (Read more about moderation in Sarah T. Roberts' Behind the Screen and Tarleton Gillespie's Custodians of the Internet.)

Historically, there has been a separation between moderators and the online platforms that they patrol. Moderators are often contractors, rather than regular employees. It is easy to develop the habit of placing both responsibility for policy enforcement and the blame for enforcement failure outside of the platform (which would also make it distant to the recommender algorithms). An example of such distancing occurred this summer, when Facebook failed to remove a post that encouraged people with guns to come to Kenosha in the wake of the shooting of Jacob Blake. The Washington Post reported that Zuckerberg said: "The contractors, the reviewers who the initial complaints were funneled to, didn’t, basically, didn’t pick this up." He refers to "the contractors", implicitly holding moderators at arm's length from Facebook. It is important that we as recommender system researchers resist absorbing this historic separation between "them" and "us".

Recommender system researchers, as computer scientists, live by the wisdom of GIGO (Garbage In Garbage Out). In order to produce harm-free lists of recommended items, we need an underlying item collection that does not contain harmful items. This is achieved via policy, and the help of moderators enforcing policy.

Second, recommender systems are systems. Recommender system research understands them as not only as systems, but as ecosystems, encompassing both human and machine components. When we think of the human component of recommender systems we generally think of users. However, moderators are also a part of the larger ecosystems, and we should include them and their important work in our research.

Connecting recommendation and moderation opens new directions for research

Currently, most of the interest in moderation has been around how to combine human judgement and machine learning in order to quickly, and at large scale, decide what needs to be removed from the platform. At the end of the talk at the workshop, I introduced a case study of a system that can translate the nuanced judgments of moderators into automatic classifiers. I discussed the potential of these classifiers for helping platforms to keep up with the fast change of content and quickly evolving policy. The work has not yet been published, but is current still under preparation (hope to be able to add a reference here at some later point).

However, not all policy enforcement involves removal. Some examples of how platform policy interacts with ranking are mentioned in the recent Wired article YouTube's Plot to Silence Conspiracy Theories. It is worth noting, that even if downranking can be largely automated it is important to keep human eyes in the loop to ensure that the algorithms are having their intended effects. We should strive to understand how this collaboration can be designed to be most effective.

Finally, I will mention that together with Manel Slokom, I have previous proposed the concept of hypotargeting for recommender systems (hyporec), a recommender system algorithm that produces a constrained number of recommended lists (or groups, sets, sequences). Such an algorithm would make it easier to enforce platform policy not only for individual items, but also for associations between items (which are created when the recommender produce a group, list or stream of recommendations).

In order to understand the argument for hypotargeting consider the following observation: There is a difference between a situation in which I view one conspiracy book online as an individual book, and a situation in which I view one book online and am immediately offered a discount to purchase of set of three books promoting the same conspiracy.

The difference lies in the impact that the recommender has on the user. Associations of items can be easily interpreted as "a trail of crumbs" leading the user to assume more broader supporting evidence for an idea than is actually justified. If the recommender produced a constrained number of sets, it would be easier to review them manually, and to make the subtle judgement of whether it is appropriate to be incentivizing purchase of these items.

Ultimately these ideas open new possibilities for policy as well: the e-commerce site should be transparent not only about which items they remove, but also about the items they prevent from occurring together in lists, groups, or streams.

There are no silver-bullet solutions to the problem of harm caused by recommender systems. However, it does seem like there is a great deal of potential in researching algorithms that can be steered by humans in order to enforce policy.