Let’s not call Goldman Sachs evil (yet, and for this potential infraction)

im-117999

Goldman Sachs is the company everyone loves to hate. I’m among them. They perpetuated the financial crisis by creating financial instruments no one understood; they bet against their own customers; they benefited enormously from a government bailout; they will continue to push every line. Watch The Big Short for more on this.

Closer to home, they (along with other investment bankers) extract large amounts of money for little value add when it comes to IPOs. Money that should rightly go to the founders, employees or the company gets skimmed off and put into the hands of their preferred clients. VC extraordinaire Bill Gurley has written extensively on this.

The point of the last two paragraphs is to let you know that I’m by no means a shill for Goldman. I consider them the ExxonMobil, Monsanto, Koch Industries of the financial world.

BUT, we need to have some intellectual honesty here. We have one data point from one blogger and the Internet blew up. This was followed by another data point from Steve Wozniak. Then we have baseless speculation around it.

I have consulted for the largest players in credit cards and payments. I can’t tell you why it happened. (Yet.)

I can tell you what didn’t happen. Goldman doesn’t have an algorithm that says “IF male, SET credit to 4X female.”

If anyone at a large company made a decision based on a few data points, they’d be (rightly) fired.

Credit decisions are made by complex algorithms. Unless you’re applying for a mortgage, no one is looking at your application individually. (There are some edge cases, but not relevant to this discussion.) A bunch of data about you is crunched based on historical risk models. Out of that comes whether you are approved, what your credit line is and what your interest rate is. Then there’s the bank’s own risk profile.

We don’t know what algorithms are going to spit out. Because they’re based on historical models and trained with existing data, it’s likely that they will reflect gender and racial bias. Amazon designed an AI algorithm to make hiring less biased against females. It turned out it did the opposite, based on the assumptions that went into it.

These algorithms are regularly updated based on new assumptions. FICO 9, which is being rolled out, is much better than the widely used FICO 8. Banks and credit card customers get data from various credit agencies. (You actually have 3 credit scores!) Although FICO algorithms are widely used, credit card issuers and banks often use additional data from other sources.

For many years, Google thought that you needed a computer science degree from a top-tier university to be a successful product manager. Google analyzed the data and found that neither were useful signals and have theoretically abandoned them.

There’s a way we can predict algorithms: use test data and see how they respond. It doesn’t happen often enough, but I don’t know if Goldman did this.

Google Photos had a massive fail a few years ago, when it tagged some African Americans as gorillas. Clearly they didn’t do enough testing.

But I do know that the Internet loves a data-free pile on. Something that fits the narrative will drive clicks. This reinforces the common view people have of Goldman (the same view I have). The only potentially worse co-brand partner for Apple in this would have been Wells Fargo.

I’ve been on the other side of an Internet pile on. A company I worked for put out highly misleading information about me on Twitter. Because what they said fit a common and sensationalist narrative, it took off. No one waited to see what actually happened.

We don’t even know if there is a problem. It’s possible that the Goldman algorithm gives higher credit limits to women than to men. We don’t know. It’s possible that it gives roughly the same credit limits. We don’t know. It’s possible that it gives men higher credit limits than women, which is what is alleged. We don’t know.

Perhaps the biggest problem with the claims so far: American Express, Chase and Citi don’t even ask for gender on their applications. (I already have an Apple Card, so I can’t verify whether this is true for the Goldman card.)

So I’m going to wait until the facts are in to say whether Goldman is evil. For this one thing.

 

 

 

About Rakesh Agrawal

Rakesh Agrawal is Senior Director of product at Amazon (Audible). Previously, he launched local and mobile products for Microsoft and AOL. He tweets at @rakeshlobster.
This entry was posted in evil, Uncategorized. Bookmark the permalink.