Scott Page's Keynote at XP2010

If you're wondering why I'm not live-blogging, it's because I have an awful battery in my computer - it conks off in less than 30 minutes and is at 30% health. Not too great. What I'm doing however, is mindmapping notes from each talk I find interesting at XP2010. I'll keep translating them into a short synopsis like this so you can perhaps get a gist of what I got out of the session.

Scott Page

Scott is quite a renowned academic having written a great book on complex, adaptive systems and perhaps an even more insightful text on 'The Difference - How the Power of Diversity creates better groups, firms, schools and societies'. He's a collegiate professor at the University of Michigan and an external faculty at the Santa Fe institute.

The Keynote - Leveraging Diversity in Parallel: Perspective, Heuristics, and Oracles

In his talk, Scott helped the audience see the two sides of the diversity debate:

• Two heads are better than one.
• Too many cooks spoil the broth.

Which one is true? Turns out that there's no free lunch - the answer is case-by-case and context is key here. So Scott proceeded to break down the problem into a few different parts. Over the next few minutes, Scott made some brilliant points that I can only summarise here:

• You can't sum up people by using things like IQ tests - it's like putting measuring tape around people's heads and saying one person's head is bigger than the other. People are more multi-dimensional than that.
• The most common solution isn't always the best solution. In the 'Sum to Fifteen' card game, you can use an unconventional method to win, by looking at it as if it were tic-tac-toe. This is a different perspective and the varying perspectives that people bring are part of their toolbox.
• In an experiment of 20 best people in a certain arena and a group of 20 diverse people, the diverse group almost always wins. "A diverse group will almost always outperform an alpha group."
• 'Problem solving is ideas having sex together.' People have different heuristics or 'rules of thumb' to solve problems. The combination of these heuristics usually generates newer ideas as well. For example the parts for a combustion engine existed years before we even thought of the damn thing.
• Diversity is the combination of different perspectives and heuristics. The conditions for diversity to work better than homogeneity is that:
1. the diverse groups has really smart people who can tell what their heuristic is (and have a good reason for it);
2. the problem they're solving is complex.
• "Diverse groups work better if: the problem is hard, they are all smart and bring different heuristics to the table"
• Scott started to to explain the concept of Prediction markets which I've first heard of from Andrew Mcafee. I've heard several great things about these and according to Scott the prediction markets are almost always more accurate than than expert statistics.
• Scott explained to us how Best Buy uses a prediction market for store managers to tell how much inventory they need to have and usually this seems to work quite well. "Individual, diverse predictions lead to collectively accurate decisions."
  
• Scott recommends that we read the book called Expert Political Judgement, which touches upon how diverse foxes outsmart the one-track hedgehogs.
• I must confess I've never heard of this, but Scott brought out the story of the Netflix prize. You can read the full story here and it's amazing how combined perspectives and heuristics can together beat an algorithm that's so comprehensive.
• The reason why the diverse crowd is wiser is mathematical
• crowd error = average error - diversity
• So it's useful to remember that what the crowd lacks in expertise, they make up in diversity.
• "If you can't press a button to evaluate something, you want to have two heads." (or more) When you have no Oracle to give you a divination, the crowd is your oracle - you need diversity.
  
• Group think is bad when everyone is thinking the same way. You need to keep the diversity of the group high to nullify your average error.
• Pair programming is a great way to get multiple perspectives to a problem, though it's important that both people are thinking about things in different manners.

To me, this talk is intriguing. We talk about training, we talk about education and cultural indoctrination. In fact that's one of the conversations that's going on in my professional circle at work too. But should we really be doing this -- do we really want an army of people drinking the kool aid and saying the same things, thinking similarly? I ask this question in the light of the great empirical evidence that Scott put out.

One of the big problems with training is that it teaches you one way of doing things. On the other hand I think training and education needs to educate people more about the purpose of doing things. If smart people with diverse perspectives and heuristics approach the problem they'll more likely come up with a better solution than what we consider the push-button oracle. I really want to chew on this some more - because I feel I'm encountering more and more non-diverse group think each day.