“For an effect of human psychology to be real, it cannot be rigorously replicated using random noise. If the human brain was predisposed to choose heads when a coin is flipped, you could compare this to random predictions (heads or tails) made by a computer and see the bias. A human would call more heads than the computer would because the computer is making random bets whereas the human is biased toward heads. With the Dunning-Kruger effect, this is not the case. Random data actually mimics the effect really well…
…Measuring someone’s perception of anything, including their own skills, is fraught with difficulties. How well I think I did on my test today could change if the whole thing was done tomorrow, when my mood might differ and my self-confidence may waver. This measurement of self-assessment is thus, to a degree, unreliable. This unreliability–sometimes massive, sometimes not–means that any true psychological effect that does exist will be measured as smaller in the context of an experiment. This is called attenuation due to unreliability. ‘Scores of books, articles, and chapters highlight the problem with measurement error and attenuated effects,’ Patrick McKnight wrote to me. In his simulation with random measurements, the so-called Dunning-Kruger effect actually becomes more visible as the measurement error increases. ‘We have no instance in the history of scientific discovery,’ he continued, ‘where a finding improves by increasing measurement error. None.'”
—Jonathan Jarry, “The Dunning-Kruger Effect Is Probably Not Real.” McGill: Office for Science and Society. December 17, 2020.
cafebedouin.org 