Sun 6 Dec 2009
Well, yesterday was the day, and in less than 9 hours DARPA crowned a winner: The MIT Red Balloon Challenge Team. I spent a good part of the day stuck to Twitter watching the contest develop and trying to read between the lines (mostly unsuccessfully). So after all the build up and a day of searching, what have we learned from the DARPA Network Challenge?
- DARPA Made it Easy Take a look at the map of the balloon locations. Notice anything? Huge swathes of the country with no balloons? Yup.
Remember when I flippantly dismissed the Twitter-only strategy by pointing to the NY Times infographic showing how few Tweets there are in so much of the country? Well, apparently DARPA knew that too, and decided to make the challenge very, very easy. The balloons were in major parks in major cities, almost entirely on the Atlantic and Pacific seaboards. I mean, for peets sake they put a balloon in Union Square, San Francisco!
- How did MIT win? Well, first let me tell you how they did not win. MIT's victory had absolutely nothing to do with the ridiculous reverse pyramid scheme they were using to hand out money to tipsters. I am willing to stake a fortune on that one. Here's the graphic they use to try and explain it:
I'm hoping the folks over at MIT aren't patting themselves on the back for their brilliant use of monetary incentives.
So how did MIT win? Well, you can find the answer to that question on Slashdot, Techcrunch, CNet News, the Washington Post, and… Get the picture? Shocking. Well known educational institution gets huge amount of publicity, attracts more tipsters, wins scavenger hunt.
- Where was the Innovation? Unclear. I didn't see any interesting innovation in incentives. I didn't see many creative uses of technology. Most teams set up websites with simple web forms for submitting tips. Army of Eyes had an iPhone app. for that same purpose, and they did quite well I think. But it still boiled down to a simple form for submitting tips. I mean, the FBI has been doing the same thing with a phone number and a few operators for years. This is a nice reminder that, although the internet and social media provides some fascinating new contexts for interaction, what we do with it in social interaction, organizing is fundamentally the same as before.
MIT's team says they were using some algorithm to verify balloon tips. I'm not sure what that means. I know that they were keeping track of all DARPA-related posts during the challenge. They may have been looking for posts such as these, which I found earlier on in the day…
Of course, none of those actually referred to a balloon in the challenge (as best as I can tell), but looking for those types of messages was more the sort of strategy I was expecting.
- Fin I can't help but feel that the whole thing was a bit of a disappointment. The way DARPA decided to place the balloons meant that teams could win without any secret sauce. I was excited to see what people would do to find the balloons on a random stretch of I80 in Nebraska or whatever.
I'm hoping that this is all part of DARPA's strategy, and the next thing they'll do give people a chance to really organize, make the challenge really hard. I think they've got the right idea that there's innovation to be done in this space. But Twitter is such a blip on the radar. To really tackle nationwide emergencies, and to effectively harness the power of networked media and the internet we need to learn to integrate new technologies with old organizational tools. We need to look at lessons from MoveOn and Obama – people who arguably did networked organizing, combined the power of new and old media better than anyone ever has – and we should see how we can use those lessons for a more directed goal.
I'd love to work on that problem. DARPA – bring it!



Some articles may have been "too late" in the game. E.g. I only found out about this today from Google news (Dec 06, BBC article — and I am in USA) and I check online news every 15 minutes. The fact is, very small % of population would have seen those particular stories. And if I saw some red balloon today, I probably wouldn't know to go looking for a story. But there is a grain of truth there — it helped them, sure. Now, different teams had different incentives. Had MIT done some risk-averse scheme like $10K draw (10redbaloons team), it may not have won. They clearly had the incentive system to create a network of people who would "hope" to become affiliates through their large friend network. Media attention + incentives for sure. (If I had seen a red balloon but not known about the news story but if one of my far-removed acquaintances saw the article about MIT, he'd tell me with higher probability had he been an affiliate!)
As far as obvious locations, I completely agree.
When were these "samples" posted? Strong hint at looking in metro parks (streetlight & trees).
https://networkchallenge.darpa.mil/Gallery.aspx
Well, MIT's team was specifically linked to in a raft of popular media articles. So even though it's only a small % of people who'd see those articles, that's a lot more people than paid attention to any of the other anonymous teams out there. Add that to the name-brand recognition and you've got one huge, but entirely traditional, publicity advantage for the MIT team.
We'll never know for sure if MIT's zany incentive scheme helped them out. I know many (especially at MIT) will claim that their incentive scheme "clearly" had some positive effect or another. But I think they could have won with just about any scheme.
I must agree with you Judd. Without the clear advantage of the university's name and the mystique associated with it, I do not believe the MIT team would have won. Their payout system wasn't much more than a pyramid scheme, and while I respect their winning the contest, I'm not sure they are as deserving of all the accolades they seem to be receiving. There were many other teams who performed quite well, yet no one seems to be mentioning anyone but MIT. There was even a team that was managed by one guy alone who got 9 balloons. If you ask me, that is much more impressive when compared to all the resources and media attention MIT had on their side. In the end, was anything at all really learned by all this? I'm not so sure.
Still, the $ to Data ratio was Excellent for DARPA.
$40k (sneezing money for these guys) + Some Rubber + A few guys to hang out near the balloons + a website
= loads of social network data + fantastic DARPA news coverage
= Win
Ben, you're so right. DARPA p0wns us again!