Why diversity is the secret to solving complex problems

October 31, 2022

U-M Professor Scott Page explains why diverse groups consistently outperform teams of like-minded experts.

A portrait of U-M Professor Scott Page
Credit: Narissa Escanlar/U-M College of Literature, Science, and the Arts

The effort to make our workplaces and institutions more diverse is typically framed as a moral issue or an opportunity to make society more just. Given the many aspects of our society that advantage certain groups of people and disadvantage others, this is a worthwhile pursuit — and a fair way of framing the challenge. But what if it wasn’t the only way of establishing a value proposition for diversity and inclusion? For years, Scott Page, the John Seely Brown distinguished university professor of complexity, social science and management at UM-Ann Arbor, has been making a numbers case for diversity. His basic thesis: If you want innovation, or to solve complex 21st-century problems like income inequality or climate change, then groups of experts, at least as we’re accustomed to thinking about them, are going to have a hard time competing with talented teams of people with relevant diverse perspectives. 

Page recently laid out his argument in an entertaining lecture in UM-Dearborn’s Thought Leaders series, which brings experts to campus to talk about big ideas relating to our strategic plan priorities. A key point in Page’s case is distinguishing between simple and complex tasks. For example, he says if you’re looking to maximize output for your logging business in northern Michigan in the 1820s, then a straightforward measure of how many trees a person can cut down in a day may be a good hiring metric. By assembling a team of the most productive loggers and aggregating their effort, you reap the most profit — simple as that. But Page says group dynamics work differently for more complex tasks. For example, say you want to make forecasts about the economy, and you ask 40 really good economists to make predictions and then average them. (This, by the way, is a common method for producing economic forecasts in the U.S. and European Union.) Page says when you crunch the numbers, a couple interesting phenomena emerge. One, the group’s average prediction is better than any of the predictions of the individual economists. Even more interesting, the group’s prediction actually has less error than the average error of the individual members, and the size of the extra benefit from this group average actually corresponds to how different their predictions were. In other words, diversity yields a “bonus.”

So why is this the case? “Once you have something that’s really high-dimensional, by definition, people are going to go about it in different ways, and when they go about it in different ways, you get this benefit,” Page said. Stated a little differently, when something is very complex, it’s hard to figure out, which means no one is going to get it exactly right. So “you want people getting it wrong in different ways” so you’re accounting for a greater degree of the complexity. Needless to say, this is a different approach than many institutions take to solving problems. Typically, Page says, our inclination is to assemble a team of the best experts on a particular topic, as measured by an accepted set of credentials. But in doing so, we’re missing an opportunity to reap a diversity bonus. “For example, when I go to the New York Fed, they’ll have 60 people with PhDs in economics and no sociologists and no psychologists,” Page said. “They’re all trained to see the world in the same way, through the exact same categories, the exact same models.” Page is quick to point out, however, that diversity doesn’t mean random difference. If you’re trying to solve a complex physics problem, the solution isn’t to “bring Tony Hawk in to CERN.” The people on your team have to have knowledge or skills that are germane to the task. But if you’re trying to come up with policy solutions to, say, income inequality or inflation, it’s going to help to have economists working alongside sociologists or psychologists, because they’ll all approach the problem a little differently and the group’s solution will capture more complexity.

In addition, Page says we shouldn’t assume that traditional metrics, like what academic discipline a person got their doctorate in, are the only ways to measure or predict this advantageous “cognitive diversity” of the group. The amalgam of someone's life experiences is also very important in what they bring to the table, which is why diversity of identity can also matter. People of different races, genders, social classes, national origins, etc. will inherently have had different life experiences, which inform how they see the world and thus their approach to problems. When this kind of identity diversity contributes to more cognitive diversity, Page says it can boost diversity bonuses.

Page also notes that environmental conditions must be favorable to maximize this benefit you get from diverse teams. Most importantly, institutions have to create environments where people feel trusted and validated, so there’s no holding back when they’re working as a group. Under these conditions, Page says you can often reap even more benefits through “synergy” — moments when ideas combine in unexpected ways to create especially great solutions. Viewed this way, creating an inclusive environment where everyone has a seat at the table becomes an “amazing opportunity” to create solutions for today’s complex problems — in addition to being the right thing to do.

###

Want to learn more about Scott Page’s work on diversity and innovation? Check out his book "The Diversity Bonus: How Great Teams Pay Off in the Knowledge Economy." Also, look out for the next installment in our Thought Leaders series on Nov 10 : “How Technology Developers and Social Scientists Can Work Together to Combat Bias in the Metaverse” with the University of Pennsylvania’s Desmond Patton. Register for the event.