The Dunning-Kruger Effect Is Probably Not Real

“For an effect of human psychology to be real, it cannot be rigorously replicated using random noise. If the human brain was predisposed to choose heads when a coin is flipped, you could compare this to random predictions (heads or tails) made by a computer and see the bias. A human would call more heads than the computer would because the computer is making random bets whereas the human is biased toward heads. With the Dunning-Kruger effect, this is not the case. Random data actually mimics the effect really well…

…Measuring someone’s perception of anything, including their own skills, is fraught with difficulties. How well I think I did on my test today could change if the whole thing was done tomorrow, when my mood might differ and my self-confidence may waver. This measurement of self-assessment is thus, to a degree, unreliable. This unreliability–sometimes massive, sometimes not–means that any true psychological effect that does exist will be measured as smaller in the context of an experiment. This is called attenuation due to unreliability. ‘Scores of books, articles, and chapters highlight the problem with measurement error and attenuated effects,’ Patrick McKnight wrote to me. In his simulation with random measurements, the so-called Dunning-Kruger effect actually becomes more visible as the measurement error increases. ‘We have no instance in the history of scientific discovery,’ he continued, ‘where a finding improves by increasing measurement error. None.'”

—Jonathan Jarry, “The Dunning-Kruger Effect Is Probably Not Real.” McGill: Office for Science and Society. December 17, 2020.

[Question] Is Stupidity Expanding? Some Hypotheses.

“To be explained: It feels to me that in recent years, people have gotten stupider, or that stupid has gotten bigger, or that the parts of people that were always stupid have gotten louder, or something like that.

I’ve come up with a suite of hypotheses to explain this (with a little help from my friends). I thought I’d throw them out here to see which ones the wise crowd here think are most likely. Bonus points if you come up with some new ones. Gold stars if you can rule some out based on existing data or can propose tests by which they might be rendered more or less plausible.

-David Gross, “[Question] Is Stupidity Expanding? Some Hypotheses.” October 15, 2020.

George Carlin kind of nails it for me: stupid, full of shit and fuckin’ nuts. While the Venn diagram has overlap, you really cannot think about this issue without the other two.

Prima facie evidence? See hypotheses in Section A, Hypothesis 11:

“There is no truth, only power. What I’ve been interpreting as truth and rationality has been my own attempt to align my thinking with the political clique that was in power when I was being educated. What I’m interpreting as rising stupidity has been the collapse in power and status of that clique and the political obsolescence of the variety of ‘truth’ and ‘rationality’ I internalized as a child. Those pomo philosophers were right all along.”

Or Section B, Hypothesis 10:

“Stupid choices used to reliably have undesirable results; now there is more of a disconnect where people are shielded from the results of their stupid choices, or even rewarded for them (man lights himself on fire in an easily-forseeable misadventure, becomes YouTube legend). So people may be appearing stupid not as a result of being stupid but as the result of a perverse cost-benefit analysis. People are no dumber than they used to be, but for [reasons] it has become advantageous to display stupidity and so smart people sometimes mimic idiocy so as to reap such advantages. The smarter they are, the quicker they caught on to this and the better mimics they are, so this makes it look as though the smart people are being replaced by morons, when really it’s more a matter of camouflage.”

Both are clearly in the full of shit category. Much of crazy is indistinguishable from stupid. Section B, Hypothesis 8, for instance:

“Back in the day, when a person had a stupid idea, they would be reluctant to put it forward as their own. Rather, they would wait to see if someone else would voice the idea so they could just agree with it. This used to be relatively rare, but now you just have to google “[my stupid idea]” to find that someone or other has said it first, and then you’re off to the races.

Replace stupid with crazy in that sentence, and it is every bit as valid.

Cults: Dissociation, Group Psychology, and Cognitive Dissonance

“”How does cult psychology work? How is it possible to persuade human adults to enter a weird cognitive landscapewith no basis in reality? To enter a fantasy realm so profound that they’ll willingly die for whomever has been selected as the local Messiah?”

–Matthew J Sharps Ph.D, “Cults and Cognition: Programming the True Believer.” Psychology Today. October 2, 2020.

Partial answer: Through dissociation group psychology, and cognitive dissonance.

“…cognitive dissonance (e.g ., Festinger et al. 1956), which manifests itself in the tendency to overvalue anything in which we’ve invested too much—money, time, emotional energy, whatever. Cognitive dissonance essentially means that the more you’ve paid, the better you like. Whether it makes any sense or not.


Dunning-Kruger Effect = Satisfaction

“…people with the biggest gap between their abilities and their view of [themselves] say they have the highest levels of satisfaction with their life, career and relationships. “People who report being more adjusted are those who have a combination of relatively lower true abilities and actual higher views of themselves,” says Stéphane Côté, a social psychologistat the Rotman School of Management at the University of Toronto and an author of the paper.”

—Lydia Denworth, “New Insights into Self-Insight: More Might Not Be Better.” Scientific American. August 27, 2019.

Simpson’s Paradox

“Simpson’s paradox (or Simpson’s reversal, Yule–Simpson effect, amalgamation paradox, or reversal paradox), is a phenomenon in probability and statistics, in which a trend appears in several different groups of data but disappears or reverses when these groups are combined.

s.v. Simpson’s Paradox, Wikipedia.

An example using arithmetic from the Stanford Encyclopedia of Philosophy:

1/5 < 2/8

6/8 < 4/5

7/13 > 6/13

Rearranging Our Minds

Open Question: Should we make an effort to change our minds in some fundamental way? And if so, how?

There are a number of stories of people suffering a traumatic brain injury that results in the brain being rearranged in a way that gives them a new ability. Generally, this involves some skill with art, understanding music, improved memory or doing calculations in math. Although, a few also involve different kinds of experience, such as synesthesia.

It’s not limited to injuries. There is also the question of psychedelics. Scott Alexander makes this point in an article in his blog Slate Star Codex:

“The third possibility is the one that really intrigues me. A 2011 study found that a single dose of psilocybin could permanently increase the personality dimension of Openness To Experience. I’m emphasizing that because personality is otherwise pretty stable after adulthood; nothing should be able to do this. But magic mushrooms apparently have this effect, and not subtly either; participants who had a mystical experience on psilocybin had Openness increase up to half a standard deviation compared to placebo, and the change was stable sixteen months later. This is really scary. I mean, I like Openness To Experience, but something that can produce large, permanent personality changes is so far beyond anything else we have in psychiatry that it’s kind of terrifying.”

Scott Alexander, “Why Were Early Psychedelicists So Weird?” Slate Star Codex. April 28, 2016.

Anyone that has been around people that have taken a lot of LSD know that they are different. Often, they are different in ways that make it more difficult to function in society, not easier. But, the opposite can also be true.

There was also a lot of discussion a few years ago about how people in Silicon Valley were microdosing LSD in an effort to boost their creativity. Clearly, in this case, psychedelics were being used to improve performance in a particular context and probably without full consideration of the effects beyond creativity.

There has also been research done in using electrical impulses to change mental states in people. The U.S. military, for example, is using electrical brain stimulation to enhance skills. Of course, there has been a dark side to this as well, as any discussion of Electroconvulsive Therapy will invariably bring up.

Meditation is also said to have effects on our mental states. A meta-analysis into meditation research by the medical community described it as follows:

“Results indicate that meditation leads to activation in brain areas involved in processing self-relevant information, self-regulation, focused problem-solving, adaptive behavior, and interoception. Results also show that meditation practice induces functional and structural brain modifications in expert meditators, especially in areas involved in self-referential processes such as self-awareness and self-regulation. These results demonstrate that a biological substrate underlies the positive pervasive effect of meditation practice and suggest that meditation techniques could be adopted in clinical populations and to prevent disease.”

M. Boccia, L. Piccardi, P. Guariglia. “The meditative mind: a comprehensive meta-analysis of MRI studies.” Biomed. Res. Int. 2015:419808. 10.1155/2015/419808

It seems like meditation is a good idea and has many positive aspects, but it also fundamentally changes the biology and the functioning of our brains. Should we be doing it?

You could probably make arguments that music, creating art, exercise and many other activities have dramatic and important effects on the mind and likely change it on a biological level. But, should we be striving to reorganize our minds to achieve some goal or mental state? And what techniques should we be using and why? This strikes me as a fundamental unanswered question about human life that warrants investigation.

Reference: Might be useful to consult Michael Pollan’s “How to Change Your Mind: What the New Science of Psychedelics Teaches Us About Consciousness, Dying, Addiction, Depression, and Transcendence” to get a sense of how psychedelics are currently being used.

You Can’t Tell People Anything

“We all spend a lot of our time talking to bosses or investors or marketing people or press or friends or other developers. I’m totally convinced that a new idea or a new plan or a new technique is never really understood when you just explain it. People will often think they understand, and they’ll say they understand, but then their actions show that it just ain’t so…

…What’s going on is that without some kind of direct experience to use as a touchstone, people don’t have the context that gives them a place in their minds to put the things you are telling them. The things you say often don’t stick, and the few things that do stick are often distorted. Also, most people aren’t very good at visualizing hypotheticals, at imagining what something they haven’t experienced might be like, or even what something they have experienced might be like if it were somewhat different.”

—Chip Morningstar. You Can’t Tell People Anything. Habitat April 22, 2004.

A variant of “show, don’t tell.” If it is not based in experience, then imagination has to do the heavy lifting to make meaning, and most of the time, people’s imagination isn’t up for the task.