How is it different from previous forms of social injustice? Why is this a new kind of systematic prejudice and discrimination? - Firstly, the "old" way was entrenched in law and custom, whereas now we look at entrenchment in automated evaluation and selection practices. Second, before it was easy to identify individuals and groups engaging in customary social prejudice; but now behind algorithms and web interfaces, the human perpetrators of discriminatory practices may find it hard to identify themselves as prejudicial, let alone be confronted by society.

Of course there is a history of advocacy for responsible coding practice, e.g. a recent example. But for the most part these come across as overbearing warnings like "Is your smartphone listening to you?". Still, one can make a case that "justice" were trumped by economic interest from this article: Secret Algorithms Are Deciding Criminal Trials and We’re Not Even Allowed to Test Their Accuracy.

Hiring and firing people are certainly not an easy job, and there is an awful lot of it going on. Unsurprisingly algorithmic selection of people is both widespread and entrenched.

Question: Is there any work done in a formal context, in ethics or law, to meet (possible) resurgent systemic biases?

Some further reading: -



  • 1
    In short, yes, we are...but there are so many aspects in this I'm not sure where to begin from. Maybe i'll try putting some of the aspects in an answer. Good topic, anyway. – Overmind Apr 12 '19 at 9:37
  • You would need the sustained attention of Subjects now to get control of the inertia of the object world. It would take a catastrophe of some kind to bring the Subject back into focus. – Gordon Apr 14 '19 at 3:12
  • 1
    Isn't "cold calculating code" potentially better at avoiding prejudice and discrimination than inscrutable "looking in the eye"? This reminds me GW looking into Putin's eyes, he would have been better off looking at cold numbers. An algorithm can, of course, be biased, and this is a concern, but algorithmic biases are written into a documented code, which is easier to track than what's behind the looking. It is true that people feel more comfortable when decisions are made by other people, but it is doubtful that this comfort stems from something very rational. – Conifold Apr 14 '19 at 9:27
  • 1
    No it was not by design I don't think. Going back to the proper use of "alienation" this is Hegel again originally, not Marx. What the worker or creator of the object gives it is part of himself. This is to alienate like under English property law. Once we create this object world we cannot really control it. It has something of its own Subject about it. Like Frankenstein's monster. – Gordon Apr 14 '19 at 18:31
  • 1
    I agree with your concerns, and laws/procedures for weeding out algorithmic biases should be developed, just as those for weeding out human biases. But human racism/sexism can be subtle as well, and often detectable only through disparate impact. Also, it does not all go one way, consider the computer generated redistricting maps vs the gerry-mandered ones routinely produced by the legislatures. So I do not see why controlling algorithmic biases should be a greater challenge overall than controlling human biases, and we should face a "new wave" compared to the one already here. – Conifold Apr 14 '19 at 21:38

Ancient Athens used an algorithm to select members of the jury. They used lots. The key principle here being simplicity and transparency.

This is something that most algorithms are not. They are not transparent and nor are they simple. One of the recent key provisions of EU Law is force companies to be more transparent about their algorithms.

Another possibility are ethics boards; for example, Deep Mind, the machine learning company when it was set up had an ethics board.

It also makes sense to me that independent auditing mechanisms should be set up that have some bite.

  • The only way to audit code is to see the source. And companies don't like to reveal their source code because the code represents proprietary trade secrets. You can't call up Microsoft or Apple or Amazon and ask to see their source code. Well you can ask. They'll say no. Are you proposing that the government should authorize "independent" auditing boards to be allowed to see the source code of private corporations? Now personally I'd like to see everything open-sourced. It's not going to happen though. – user4894 Apr 17 '19 at 13:02
  • 1
    @user4894: I don’t follow your argument. Financial accounts are often information that companies want to keep to themselves, nevertheless, if they are public companies, then they are forced by law to go through annual financial audits. Apple is publically traded and donut goes through such audits. The principle here is the public interest. Algorithms are so pervasively used that one might say that they are already in the public domain whether the company is publically traded or not. – Mozibur Ullah Apr 17 '19 at 13:15
  • You don't follow my argument? You could disagree with it but I don't see how you don't follow it. You can't see Apple's source code. Surely you know that. It's doubtful that future legislation would open-source all code. – user4894 Apr 17 '19 at 13:20
  • @user4894: When someone says they follow an argument there is usually an implicit acknowledgement that they also agree with it. I’m not. That’s why I said I don’t follow you. I’m pointing out that algorithmic audits are possible. I’m not sure why you’re pushing open-source as a solution to algorithmic auditing. Open source is not going to reveal patterns of use or abuse. – Mozibur Ullah Apr 17 '19 at 13:26
  • Can you just explain how you are going to get the government to order private companies to reveal their source code? I don't see that happening. If you don't think open-source reveals the code, then we disagree on the most basic terms. How else are you going to audit the code without seeing the source? – user4894 Apr 17 '19 at 13:28

Not the answer you're looking for? Browse other questions tagged or ask your own question.