June 2008

In an easily recognizable, but nonetheless idiotic, ploy to sell magazines, Wired's editor-in-chief Chris Anderson has published a short article called The End of Theory: The Data Deluge Makes the Scientific Method Obsolete. In it he claims that the mere availability of data on a huge scale means that theories and models are unnecessary. As long as we have statistics that can pick trends, correlations out of the madness, we don't need the scientific method anymore.

I'll let this excellent rebuttal by John Timmer at Ars Technica do most of the work in explaining why Anderson's argument is so flawed it should never had been printed. (Ah hah! More evidence against Andrew Keen's argument for the return of the old-school editor! Anderson's crap would never have passed muster on Wikipedia.)

For me, the most important rebuttal is about falsification and repair. Without theories that we can test, how can we know we're wrong? What will being wrong look like? Without reasoned explanations for why things happen, how will we know what to do when things break? The reason that scientists are so wary of correlations is because they offer no explanatory power – they're misleading as often as not. If we act on them, completely ignorant of the underlying mechanism, we don't learn anything at all. Anderson's most staggeringly ignorant move is to suggest that theories and models are somehow unnecessary simply because they're often wrong. Wha? I guess the benefit of never having a model or a theory is that if you make no assumptions or predictions no one can ever disprove you.

I'd be willing to dismiss Anderson entirely if, presented differently, he wouldn't have otherwise tackled an interesting topic. As Timmer says, certainly the availability of massive data is changing the way we do science. But the end of theory? C'mon, Chris, that's ridiculous, and a transparent attempt to appeal to the data-heads that read Wired. This point of view is SO common, at least around San Francisco. I'm amazed that otherwise smart people would adopt such an ignorant, arrogant point of view. Fighting this kind of thinking is depressing. It reminds me of something Anthony Bourdain said about his most hated chef nemesis, Rachel Ray. (I noted this in a previous post.)

Complain all you want. It’s like railing against the pounding surf. She only grows stronger and more powerful. Her ear-shattering tones louder and louder. We KNOW she can’t cook… She’s a friendly, familiar face who appears regularly on our screens to tell us that “Even your dumb, lazy ass can cook this!” Wallowing in your own crapulence on your Cheeto-littered couch you watch her and think, “Hell…I could do that. I ain’t gonna…but I could–if I wanted! Now where’s my damn jug a Diet Pepsi?

A lazy, soulless, superficial, inexplicably popular idea. It's days are numbered, though. I predict that inside of 5 years, Google is going to hit the wall on its data-center driven problem solving. They'll call for more cowbell, find there's none to be had, and return to the land of the living where the rest of us live.

Yahoo!'s new reputation design patterns got me thinking – what makes a reputation? When I browsed through the 9 design patterns lumped under the title of 'reputation', my first thought was that these are interesting and valuable, but they are not reputation elements.

But then, step back. A reputation system is a substitute for personal experience. It provides you with the information you need to make a determination about someone (something?) else without having had to go to all the trouble of getting to know them. Traditionally, that determination has been about interpersonal trust. eBay's reputation systems is the best example. I don't know anyone who's selling blidgets on eBay, so I don't know how likely they are to cheat me. eBay has found a formal way to represent the likelihood that I'm dealing with a seller who will meet my expectations.

So, the reputation design patterns aren't like that. They're not about trust, at least not directly. But, they are signifiers that help me know someone better. Just like my score on eBay represents something about my behavior, so do achievement badges, rankings, etc. They encapsulate information about the type or volume of my participation in different ways. And this information, in turn, may help me figure out something more about trust. Certainly, in an indirect way at least, these things act as elements of a reputation because they substitute for my personal experience with someone else's contributions over time. If I'd been there to see what they did myself, I wouldn't need the badge or the level information.

Still, if we take this view, is my age reputation information? My address? After all, that information saves you the trouble of having to be around to count the years, or having to travel to my town to check where i live. (I'm going off the deep end now!)

More importantly, why does any of this matter? Who cares whether it counts as a reputation. Well, there's a bit of truth in that – maybe reputation is in the eye of the beholder, at least in practice. But only sort of. When it comes to design patterns, I think the important thing is to realize what badges or points are good for. So, certainly my badges and levels help others figure out how to assess my contributions when they don't know any better. But they also work as incentives that make me feel valued, like my contributions count, like I'm making progress, and like people think of me as an expert. They give me a goal or a quota to shoot for, or a status marker that tempers my insecurity.

I think Yahoo! gets this. But the patterns kind of mix up the part that's for you (the reputation) and the part that's for me (the incentive). If the point is to understand your users deeply, design incentive mechanisms with them in mind, then breaking those two apart is essential, and there's a lot of good work to be done there. Anyway, as I said, I'm somewhat conflicted. Comments welcome (as always)!

Through it's Developer Network, Yahoo! has just released a nice set of design patterns for reputation systems. I have some issues with some of the language and patterns, but overall, I think they put together a really great typology.

Yahoo! Reputation Design Patterns

My biggest beef is with the most 'meta' pattern that they call the 'Competitive Spectrum'. I understand the desire to simplify, but in my view, these 5 things are not really on the same spectrum at all. I think the 'combative' type is off in a corner of its own – a corner that really doesn't exist much on the web. As for the other four, I can't make out what the axis is that they vary on – overall level of competition doesn't make sense to me. Yahoo! seems to realize the confusion themselves, as they include a variety of caveats in their description of the spectrum.

Competitive Spectrum

I really agree with Bryce Glass (one of the patterns' creators), who points out that these patterns are pretty ubiquitous now, and so simply pointing them out isn't enough. It's how they're used – or more specifically how intelligently they're used – that will make them powerful. I think Yahoo! still has some work to do to provide best practices for implementing these patterns intelligently. Obviously, given my interests, I'd like to see them look at some of the underlying social psychological processes, and use them to make some informed recommendations. Also, I think designers really need an accessible way to understand the 'corruption effect of extrinsic motivation' (or, as economists call it, 'crowding out'). I would argue that in many contexts when incentives like the ones Yahoo! lays out don't work as expected, the corruption effect is a big reason why. But, all in all, its a great start. (Note that my opinion is in no way influenced by the fact that I'm an intern at Yahoo! this summer… heh)

(Thanks to Ben for the tip!)

You've run a fine campaign. You've shown that a vigorous and competitive process is an improvement over the usual method of ordaining whichever candidate happens to win the first few primaries. You've made huge leaps for women and politics, and shown that sexism, though still rampant, doesn't have to be a barrier. So, all in all, I think you've done well.

Hillary Clinton

But now it's time to go. Throughout this race you've proven yourself willing and able to do and to say whatever it takes to move forward. You've reinvented yourself so many times that it's hard to know who you are anymore. I understand, that's politics. But the point is that we don't want that politics anymore, and more than that we don't need it. We have an alternative, in Barak Obama, who has told the same substantive story, who has made the same case, who has fought on the issues since the beginning of his campaign. More importantly, by every algebra but yours, he has won more states, more pledged delegates, more popular votes than you have. He's won. So it's time for you to step aside.

Yesterday's DNC Rules Committee meeting was really your last chance to make any gains at all, I understand. (Though, as you know, even if all of the delegates had been seated at full strength, you would still be far behind in pledged delegates.) Thankfully, your attempts to influence the committee failed.

In my view the correct political decision but the wrong moral decision was made. Michigan and Florida need to have a voice, and we need them in the general election, so this was the right choice to that end. However, no reasonable person (without something to gain by it) could claim that the elections in those states were fair or unbiased. It's not the voters' fault, it's the Rules Committee's fault for invoking a bad punishment to begin with. But we can't turn back time. As it is, you've come out ahead! You've gained delegates from unfairly contested elections – can't argue with that!

So, now to the purpose of this letter. (The purpose other than filling up my blog.) Let's see what happens in the last three primaries today and on Tuesday. Then, please, let that be that. You'll have lost. Find a way to bow out gracefully, accept the accolades that are rightfully yours, and throw yourself full-tilt into the fight to get Obama into the White House. Please don't take it to the Credentials Committee, please don't let this drag on. Remember the serenity prayer, and go back to the Senate with new purpose.

All the best.