Goodhart’s Law & The Ivy League

“Two years ago I was at an event in Boston and I happened to sit at a dinner table across from a guy widely recognized to be one of the most brilliant people in the world.  We talked mostly about AI but at one point the conversation turned to hiring, and he told me that for 2 decades he has tracked the performance of everyone who worked for him. Based on that performance tracking, he had stopped hiring from Harvard and Stanford. He said that historically, his best employees came from Harvard, Stanford, and MIT, but that starting 5 or 6 years prior, the people coming out of Harvard and Stanford started to really slip in their performance. I asked why, and he said that he didn’t know, but had a hypothesis. He said “I believe that ivy league college admissions has become so competitive that it rewards people who are good at the admissions process, not people who are good.”

This is a form of Goodhart’s Law, which says “when a measure becomes a target, it ceases to be a good measure.” You can see where this is going, and that there may be a similar law for machine learning.  We are in a phase of AI where we are using data sets that were created for other purposes, not with AI in mind. What happens when you know all your data is going to be fed into an AI?  Does it change the data you create?

-Rob May, “Goodhart’s Law and AI Data Sets.” Inside AI. June 3, 2019.

Loving Day

Loving v. Virginia is a landmark civil rights decision of the United States Supreme Court which struck down all state laws banning interracial marriage on June 12, 1967.

Topic has four films on The Loving Generation that “tells the story of a generation of Americans born to one black parent and one white parent. Their narratives provide a fascinating and unique window into the borderland between ‘blackness’ and ‘whiteness’, and, in some cases, explode fixed ideas about race and identity.”

Tech Veganism

“One place where the veganism metaphor breaks down is that, although nearly anyone can be a vegan, tech veganism is mostly practiced by those who are expert enough or privileged enough to learn the elaborate workarounds to avoid the GAFAMs of the world. Setting up an Ubuntu laptop, a LineageOS phone, a Fastmail account, and wiring it all together so that you actually get calendar notifications is no easy feat. You will probably have to get your hands dirty on the command line.

I find that there’s a bit of a ‘let them eat cake’ attitude among tech vegan boosters, because they often discount the sheer difficulty of all this stuff. (‘Let them use Linux’ could be a fitting refrain.) After all, they figured it out, so why can’t you? What, doesn’t everyone have a computer science degree and six years experience as a sysadmin?”

-Nolan Lawson, “Tech Veganism.” nolanlawson.com. May 31, 2019.

Reposting my comment here:

The point that a lot of people miss is that there are two tech revolutions going on.

On one end, there’s the application, end-user technology that is designed to fit some specific need and be easy to use. Ideally, it’s just tapping buttons on a screen.

These tend to follow direct physical analogs of specific, single purpose tools: Search/Keep replaces the file cabinet, Maps replaces maps, YouTube is mobile television, Play is mobile board games, News replaces the newspaper, Gmail is mail, Contacts replaces the Rolodex, Drive replaces the suitcase, calendar replaces the Day Planner, Translate replaces the foreign language phrase book, Photos replaces the photo album, etc.

They do one thing. They do it well. They are useful. And they are all being combined in one device. But, they aren’t the standard by which every tool needs to be judged.

Some tools are general tools that can be applied to a wide variety of problems. Any idiot can pick up an arc welder or write a Python script. But, it takes time to learn how to use these general purpose tools well. They’re never going to be easy in the way that using email is easy. And there is no need to use Python or an arc welder on your phone.

Should TeX be compared to Word? It’s apples and oranges. How much does that Venn diagram overlap, regardless of definition?

Emacs is as different from a word processor as a word processor is from the legal pad. There’s inherent capability that doesn’t reduce down to the level of a pull-down menu or button. As soon as you make ease of use the defining feature, you narrow down capability to what a pull down menu can handle.

Most people aren’t solving problems that require training A.I.s, collaboratively writing programs with a tool like Git, making CGI movies with Renderman, etc. So, they have no need to learn these tools, and these tools do not need to simplified to suit them. They are what they are. OpenBSD is about security, not being “user-friendly” to the novice user. If you want “user-friendly,” use what everyone else uses.

“Tech vegans,” as you describe them, have different needs and different values. Some day, it would be nice if a LineageOS device were available at your telephone carrier’s store, Google didn’t mess with ad blocking extensions in their browser to make more money, and so forth. But, the incentives are what they are. Opting out of the default is hard by design. That’s not only a technology problem. It’s also an economic one.

Trump’s Tuxedo: ‘It’s wrong in every way!’

“And Patrick Murphy, the head cutter at tailors Davies and Son, who are based in Savile Row, London, told MailOnline that ‘everything you can imagine’ was wrong with Trump’s tuxedo.

‘Its totally disproportionate to his height and girth,’ he said.

‘The waistcoat is far too long, it should not show underneath his jacket.”

—Conner Boyd and Harry Howard, “Trump’s look is not my fault, says White House tailor as he insists he doesn’t know where President got his poorly fitted state banquet attire from – while Savile Row expert reveals: ‘It’s wrong in every way!’The Daily Mail. June 8, 2019.

My apologies for pointing toward the cancer that is The Daily Mail website, but I think ‘wrong in every way’ pretty much sums up the suit and the man. Keep Making America Great Again, buddy!

What I Learned Trying To Secure Congressional Campaigns

“Attempts to work with the DNC and DCCC. The national party was so unhelpful that in the end I had to treat them as part of the threat model. Particularly vexing was their addiction to sending email attachments.

To cite one small example: on August 22, the DNC had a phishing scare, where they mistook a vulnerability assessment for an actual attack. The next day, DCCC Executive Director Dan Sena sent an email to all campaings with the subject line “Reminder About Cybersecurity”. That email included three attachments, including a file evocatively titled “2—20170712—Falcon.docx”.

I can’t think of a more efficient way to compromise every campaign in the country than blasting security alerts with dodgy attachments from the DCCC email account.

The DCCC sent out attachments constantly. It drove me nuts. And I was never able to get a meeting with anyone there to slug it out.”

—Maciej Ceglowski, “What I Learned Trying To Secure Congressional Campaigns.” IdleWords.com. May 27, 2019.

Explains much of what is wrong with politics in the United States and the Democratic Party in particular.