Letter to the Algorithm

By Timothy M. O'Brien
January 5, 2009 | Comments: 2

Consider this my cryptic response to the recent buzz over a change.org entry in which Jen Nedeau calls out Digg for being sexist. My take: is Reddit sexist? No, algorithms can't be sexist, but communities can be, and the communities attracted to Digg and Reddit are certainly male-dominated and coarse. Reddit and Digg appear to be sexist to Nedeau because the algorithms are so primitive.

Collaborative Filters Don't Have Opinions

Reddit and Digg are not sexist inasmuch as a laptop or an iphone is sexist (and I'm sure someone could argue that point). The "opinions" they record are an aggregate of thousands, and the conversations they record are as far from reasoned discourse as you could get. While I disagree that Digg is sexist, I do think that the experience of Nedeau points to some cracks in the foundation of collaborative filtering. I'd translate "Digg is sexist" into:

"Because the current iteration of collaborative filtering tools lack a mature mechanism for preference clustering, they fail to scale beyond the dominant preference cluster."

In other words, sites like Reddit and Digg are currently a giant blender, overlaying the opinions of thousands of users into a single opinion seldom customized for the individual. Digg appears to be sexist for Nedeau because it hasn't developed an idea of her preference cluster. Digg doesn't "know" her "metapragmatics". One day it will, and it will be able to perfectly isolate the Neanderthals from the rest of us.

The future is one of segregated preference clusters, more on that later on this week...

You can't criticize an algorithm... For those of you struggling with the whole "Reddit as News Source" theme, here's a hypothetical letter to the algorithm...


You might also be interested in:

2 Comments

I'm pretty sure I zeroed in on the comments being sexist, rather than the algorithm - but to each their own interpretation. Make sure to check out the summary post as well:

http://womensrights.change.org/blog/view/digg_community_responds_to_changeorg_cr_efforts

@jen, in a decade we'll all look back on Digg and Reddit as the initial experiments in collaborative filtering. In many ways, you are right "Digg is Sexist". An algorithm that only accounts for the aggregated preferences of a "winner-takes-all" cluster will produce content that has the biases of that community. Subtracting the single issue of gender from the discussion, I do believe that this "single-cluster" approach to collaborative filter drags us all toward an unsatisfying average coarseness.

In a decade, maybe we will have developed the necessary algorithms to create dynamic clusters in a way that is feasible (read economic). Until then, we'll have to settle for the collective sentiment of average users.

News Topics

Recommended for You

Got a Question?