Calling Bullshit

“Our learning objectives are straightforward. After taking the course, you should be able to:

* Remain vigilant for bullshit contaminating your information diet.

* Recognize said bullshit whenever and wherever you encounter it.

* Figure out for yourself precisely why a particular bit of bullshit is bullshit.

* Provide a statistician or fellow scientist with a technical explanation of why a claim is bullshit.

* Provide your crystals-and-homeopathy aunt or casually racist uncle with an accessible and persuasive explanation of why a claim is bullshit.

We will be astonished if these skills do not turn out to be among the most useful and most broadly applicable of those that you acquire during the course of your college education.”

Calling Bullshit Syllabus

College Return-on-Investment (ROI)

Open Question: Is a college education worth the expense, including tuition, opportunity costs, debt obligation, etc.?

“Using data from the expanded College Scorecard, this report ranks 4,500 colleges and universities by return on investment. A First Try at ROI: Ranking 4,500 Colleges finds that bachelor’s degrees from private colleges, on average, have higher ROI than degrees from public colleges 40 years after enrollment. Community colleges and many certificate programs have the highest returns in the short term, 10 years after enrollment, though returns from bachelor’s degrees eventually overtake those of most two-year credentials.”

A First Try at ROI

Goodhart’s Law & The Ivy League

“Two years ago I was at an event in Boston and I happened to sit at a dinner table across from a guy widely recognized to be one of the most brilliant people in the world.  We talked mostly about AI but at one point the conversation turned to hiring, and he told me that for 2 decades he has tracked the performance of everyone who worked for him. Based on that performance tracking, he had stopped hiring from Harvard and Stanford. He said that historically, his best employees came from Harvard, Stanford, and MIT, but that starting 5 or 6 years prior, the people coming out of Harvard and Stanford started to really slip in their performance. I asked why, and he said that he didn’t know, but had a hypothesis. He said “I believe that ivy league college admissions has become so competitive that it rewards people who are good at the admissions process, not people who are good.”

This is a form of Goodhart’s Law, which says “when a measure becomes a target, it ceases to be a good measure.” You can see where this is going, and that there may be a similar law for machine learning.  We are in a phase of AI where we are using data sets that were created for other purposes, not with AI in mind. What happens when you know all your data is going to be fed into an AI?  Does it change the data you create?

-Rob May, “Goodhart’s Law and AI Data Sets.” Inside AI. June 3, 2019.

The Case Against Education | RadioWest

“Bryan Caplan says our higher education system is a waste of time and money. Caplan is a Princeton-educated, tenured professor of economics at George Mason University. He argues though that while a degree has become indispensable for competing in the job market, college isn’t actually teaching applicable skills or even teaching people how to learn. And worse yet? Many graduates are deep in debt and still not getting a great job.”

Listen to the Interview.