New and interesting mistakes

I never cease to be surprised by how smart people can be very stupid. To be more precise, people can be very clever in certain areas and extremely dumb in others. Why do we compartmentalise intelligence?

You see people who are clearly bright – e.g. succeeding in highly competitive and difficult professions – who at the same time believe the most naive conspiracy theories. Or otherwise thoughtful individuals who, when it comes to politics, can only repeat the ideology they uncritically received from their parties or tribes1.

I consider a particular case of this paradox the many smart people who are also deeply unhappy. The proved lack of correlation between intelligence and happiness is more puzzling than it may seem at first: after all, “humans are intelligent to the extent that our actions can be expected to achieve our objectives” (Stuart Russell). If the most intelligent individuals don’t seem to achieve the supreme objective (happiness) better than others, then… in what sense are they more intelligent?

I’ve recently come across a brilliant passage that shed some light on this paradox. In his book “Rationality: from AI to Zombie“, Eliezer Yudkowsky writes:

“When I think about how my younger self very carefully followed the rules of Traditional Rationality in the course of getting the answer wrong, it sheds light on the question of why people who call themselves “rationalists” do not rule the world. You need one whole hell of a lot of rationality before it does anything but lead you into new and interesting mistakes.”

Yudkowsky is really onto something here. So many smart people are smart only in the sense that they can fool themselves in highly sophisticated ways. After all, they are inclined to the same biases and prejudices of less smart people, but because they can make a more convincing case for their errors they will stick with them deeper and for longer.

Sometimes – in some narrow field where there are strong, external incentives to reach the truth, e.g. in their professions or studies – these people are “nudged” into using their sharper cognitive tools to be correct more often than the average person. But in most other areas there is no such incentive: when you spend your night watching the umpteenth video confirming your suspicion that the world is controlled by a zionist conspiracy there is no external penalty for indulging in confirmation bias (on the contrary, finding evidence that confirms your belief is quite satisfying), at least not in the same way as there is a penalty for building a bridge that will collapse2,3.

Maybe the paradox simply rests on a misunderstanding: the implicit assumption that the objective of human intelligence is to reach the truth. If we instead accept that our brain is first of all a tool for survival, and that self-deception used to be often instrumental for survival in our ancestral environment (e.g. by making us stick closer to our own tribe), it is no surprise that intelligent people are wrong as often as anyone else.

I have used an analogy with Artificial Intelligence (AI) before on this blog, but here is another one: current AI systems are very smart in some narrow fields externally designed by human researchers (e.g. the game of Go), but they are genuinely stupid when they try to generalise their skills to other tasks. In the case of AI, this is an objective limit of the algorithms (and this limit is probably the main technical obstacle on the path to human-level artificial intelligence), whereas humans could in principle be rational across the board if they really wanted and had the right incentives.

Unlocking the true potential of our intelligence would greatly benefit all of us. A benefit to humanity that is no inferior to the (much more celebrated) opportunities that true human-level AI could bring about.

[1] I would be tempted to describe these people as affected of “cognitive dissonance”, but that would not be accurate because the contradiction in their minds is not (necessarily) about beliefs: it is about their selective use of rationality. “Epistemic dissonance” is more appropriate.

[2] This example neatly explains why there are some many smart engineers believing stupid conspiracy theories.

[3] I believe another reason has to do with sheer complexity: after all, truly understanding the causes of complex social dynamics is way more complicated than understanding how to solve equations.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: