“Two years ago I was at an event in Boston and I happened to sit at a dinner table across from a guy widely recognized to be one of the most brilliant people in the world. We talked mostly about AI but at one point the conversation turned to hiring, and he told me that for 2 decades he has tracked the performance of everyone who worked for him. Based on that performance tracking, he had stopped hiring from Harvard and Stanford. He said that historically, his best employees came from Harvard, Stanford, and MIT, but that starting 5 or 6 years prior, the people coming out of Harvard and Stanford started to really slip in their performance. I asked why, and he said that he didn’t know, but had a hypothesis. He said “I believe that ivy league college admissions has become so competitive that it rewards people who are good at the admissions process, not people who are good.”
This is a form of Goodhart’s Law, which says “when a measure becomes a target, it ceases to be a good measure.” You can see where this is going, and that there may be a similar law for machine learning. We are in a phase of AI where we are using data sets that were created for other purposes, not with AI in mind. What happens when you know all your data is going to be fed into an AI? Does it change the data you create?-Rob May, “Goodhart’s Law and AI Data Sets.” Inside AI. June 3, 2019.