I actually don't know if we have to give you the final "credit score". I'm more on the technical infrastructure side. For our project though, it's my impression that we aren't aiming to provide the number to the customer (It's a ranking from 1-10, binned arbitrarily based on internal rules, so practically meaningless to any individual). The credit score can be manually overwritten by individual advisors, and may be adjusted to fit sector wide government mandated risk levels, so its use beyond internal calculations is limited.

We will rather try to provide explanations for specific decisions, so if we deny you a loan we want to be able to tell you what pieces of hard data we are basing that decision on. Try to make it actionable, instead of the opaque numbers that lead to a feeling of "the computer says no." It's still an open question of if we will be able to provide that insight.

Legally I don't know if the credit score, or any of the components (LGD and PD) count as "personal data". If it does, we are obliged to provide it. Basically no customers make those kinds of requests so there's not really any internal discussions around those questions. As far as my bank is concerned, I'm sure we'd just tell you if you asked. We're bad at secrets.

> I believe they should be able to come to this conclusion however they like, except based on the obvious factors like culture, gender (or lack/fluidity of), etc.

Really? How do you figure? I genuinely cannot tell how you are reconciling these two points of view. While should a private business be able to do business however it wants, but somehow be restricted when it comes to this list of 3-4 factors? Can you elaborate on that?

More specifically, how do you reconcile this opinion with the fact that many algorithmic models do perpetuate existing discriminations against these groups that you listed, despite these things supposedly not being parameters of the algorithms?

(Note: I am not condoning discrimination. In fact, I personally don't think that private business should be able to do whatever they want, and that they should be much more restricted than they are today in many, many different respects.)

What about immediate proxies for those things? I can determine your culture, gender, etc based on things like your shopping characteristics. Maybe i just deny loans to people who don't ever buy sunscreen? Or people who prefer menthol cigarettes? Or people who donate to the wrong charities?

Saying that private businesses can just do whatever they want is basically incompatible with Western society as we know it. It only works at small scale precisely because little shops don't have access to data about their customers and they are legally barred from using the obvious clues like "wears a turban" or "has short hair and cargo pants"

> - They have a history of not paying off their debts.

If only. In practice it's more like "They don't have a history of paying off their debts", which crucially means if you don't have a history of being in debt then you have no history of paying it off, and therefore you're considered high risk. Thus you get otherwise-nonsencial behaviours like taking out a loan only for the reason of paying it off.

You'd think so, but banks aren't "all or nothing" businesses. It's not like you can either get a loan for any amount you want, or you can't borrow at all. There's a bunch of intricacies that make is much more interesting.

Say you lend $200 to a guy who also has a house worth $100, you're the only creditor. In the event of a default, you can take that $100 house. That means you only lose $100, not he $200 you're actually out. This is called the Loss Given Default, or LGD. That number encapsulates the you first point, and half of point two.

The other half of point two, and the entirety of point three is covered in what we call the Probability of Default, or PD. The chance that a given debtor is going to default.

I hope you can see how these two number interact, especially when you then also take into account the upside of giving the loan. Providing loans might entice a large counterparty to do more business with you, or it might provide you with access to a new network. It might make sense to make a risky loan if the downside is very small, or conversely it might not make sense to make a pretty safe loan, if the downside is huge. In practice you can multiply these two numbers together to get an "expected cost" of proving the loan.

Now when someone comes and asks you why you aren't going to lend him $300, you then have to be able to trace all that data back to the source.