Would it be ethical to use AI to determine a user’s gender from the content they upload, without them knowing?

Question

I originally tried to find an answer on /UX but was told that this would get a better response here.

I’ve been thinking about this for a while, and am unsure how the average user would feel about this “ethically ambiguous intrusion”.

My reasoning for why it is ethical: Given a photo of a person, a human can visually determine the individual’s gender automatically, without any conscious effort involved; it is an innate, evolutionary pattern recog ability (ie: you immediately know whether you’re looking at a female or male (majority of the time)). It’s such a basic assumption that the individual in the photo doesn’t need to hear your assumption of their gender, as it is obvious (again, in the majority of cases).

Now in theory, if I use an AI algorithm to analyze a user’s public photos and determine their owner’s gender without the user’s knowledge, nothing has changed. The algorithm is going off the same publicly available information as any human, and making an assumption of the profile owner’s gender, just like any human would.

Is my reasoning just, or is this a risky approach?

Answer 1

This question goes back at least to 2002. The Wall Street Journal wrote an article titled,

If TiVo Thinks You Are Gay, Here's How to Set It Straight

The article is behind a strong paywall but its value does not lie in its content. Rather, the point is that as long as 17 years ago, it was clear that companies that provide digital services can and do use your behavior to make judgments about your personal life.

I found a copy of the full abstract of this article here.

https://boards.fool.com/my-tivo-thinks-im-gay-18221251.aspx

Basil Iwanyk is not a neo-Nazi. Lukas Karlsson isn't a shadowy stalker. David S. Cohen is not Korean.

But all of them live with a machine that seems intent on giving them such labels. It's their TiVo, the digital videorecorder that records some programs it just assumes its owner will like, based on shows the viewer has chosen to record. A phone call the machine makes to TiVo, Inc., in San Jose, Calif., once a day provides key information. As these men learned, when TiVo thinks it has you pegged, there's just one way to change its "mind": outfox it.

Mr. Iwanyk, 32 years old, first suspected that his TiVo thought he was gay, since it inexplicably kept recording programs with gay themes. A film studio executive in Los Angeles and the self-described "straightest guy on earth," he tried to tame TiVo's gay fixation by recording war movies and other "guy stuff."

"The problem was, I overcompensated," he says. "It started giving me documentaries on Joseph Goebbels and Adolf Eichmann. It stopped thinking I was gay and decided I was a crazy guy reminiscing about the Third Reich."

Another reference to the same issue is from 2016:

https://journals.sagepub.com/doi/abs/10.1177/1527476416644978?journalCode=tvna

This is a paywalled academic article. Here's the abstract.

In 2002, during Silicon Valley’s recovery after the dot-com crash and the recent push for sexual equality in the United States and across the globe, various media began pondering the question of what to do if TiVo “thinks you are gay.” Here, I analyze a King of Queens (1998–2007) episode and a The Mind of the Married Man (2001–2012) episode that center on this question and how they illustrate a sudden breakdown in sexual norms and identities even as they served to make TiVo’s personal video recorders (PVRs) and recommendation systems more attractive to the urban, liberal, and largely heterosexual viewer that TiVo desired. These narratives became deeply connected to TiVo’s identity in ways that made the PVR appear simultaneously transgressive and conventional—the birth of a new algorithmic culture and the furtherance of the television industry as status quo.

The "birth of a new algorithmic culture" indeed.

I should mention in case anyone doesn't know, TiVo was the first company to market digital video recorders (DVRs). DVRs allow you to program them to record your favorite television shows. I'm not even sure if they're popular anymore since you can just download whatever you want these days.

I can't tell you what the philosophers say about how things should be. In practice this train has long left the station. Every single one of your clicks goes into a database to be datamined and monetized. What's called "AI" these days is just very sophisticated datamining. This is a fact of the world we live in.

Here's another similar story I ran across.

How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did

https://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/#497d98ea6668

Ethical? At this point what difference does it make? Our entire economy is based on monetizing clicks. As Socrates might say if he came back and got a job as Google's in-house philosopher: The unmonetized life is not worth living. Yes, Google has an in-house philosopher.

https://venturebeat.com/2011/05/14/damon-horowitz-moral-operating-system/

Answer 2

Places to look for ethical issues regarding the collection of identifying data, including gender, on individuals would be computer and information ethics and internet research ethics.

Regarding privacy Elizabeth A. Buchanan and Michael Zimmer write:

The protection of privacy and confidentiality is typically achieved through a combination of research tactics and practices, including engaging in data collection under controlled or anonymous environments, the scrubbing of data to remove personally identifiable information (PII), or the use of access restrictions and related data security methods.

PII includes:

...date of birth, place of birth, mother’s maiden name, gender, sexual orientation, and other distinguishing features and biometrics information, such as height, weight, physical appearance, fingerprints, DNA and retinal scans... [my emphasis]

However, scrubbing datasets of PII does not guarantee anonymity.

In various cases, researchers (and sometimes even amateurs) have been able to re-identify individuals by analyzing and comparing such datasets, using data-fields as benign as one’s zip code (Sweeny 2002), random Web search queries (Barbaro & Zeller 2006), or movie ratings (Narayanan & Shmatikov 2008) as the vital key for reidentification of a presumed anonymous user.

Scrubbing data of PII is not enough. Data also has to be protected from becoming publicly available.

The question asks about the ethics of collecting gender information from people "without them knowing". Sometimes individuals provide this information without knowing the consequences to their privacy in doing so. Did the user intend this information to be private? However,

...it is difficult to understand with any certainty what a user’s intention was when posting an item onto a social media platform (Acquisti & Gross 2006). It could be been meant to be visible to only a small circle of friends, but the user failed to completely understand how to adjust the privacy settings accordingly. Or, the information might have previously been restricted to only certain friends, but a change in the technical platform suddenly made the data more visible to all.

Information collected without consent may involve ethical privacy issues. Making it publicly available may also have legal consequences. One should make sure one's research ethics board approves the collection and access to such data.

---

Buchanan, Elizabeth A. and Zimmer, Michael, "Internet Research Ethics", The Stanford Encyclopedia of Philosophy (Winter 2018 Edition), Edward N. Zalta (ed.), URL = https://plato.stanford.edu/archives/win2018/entries/ethics-internet-research/.

Answer 3

Now in theory, if I use an AI algorithm [...], nothing has changed.

A few things have changed.

The most obvious is the scale at which you are now able to do it. The ethical implications of storing and processing a hundred million people's personal data are not the same as those of storing and processing a hundred people's personal data; the potential harm from a data breach is a million times greater, for instance. Meanwhile, if you are processing only a hundred people's data it is feasible for you to respect privacy rights such as the right of the data subject to correct mistakes, or request erasure of their own data; it is unlikely that you have the capacity to provide those services to a hundred million data subjects.

Another is that there is no longer a human in the loop who can feel uncomfortable about the task and refuse to perform it. For example, the photo or the context it occurs in could hold evidence that the photo was never meant to be public. A human performing the task could discard that photo; an algorithm designed only to recognise gender would not.

Answer 4

Apply Golden rule

Unfortunately, ethics is not a cookbook with instant and easy solutions for every dilemma. Philosophers could discus about various ethical topics at nausea, but that seldom translates into real life . However there are some universally accepted ethical rules, and one of them is Golden rule.

Therefore, to apply this rule, simply ask yourself would you like to be profiled using the pictures you uploaded on internet ? And if you don't mind that how deep should this profiling go ? Note that picture carries a lot of information, not only about your gender, but also about your race, age, health, even IQ (determined by size and shape of the head) . Not to mention what your choice of clothes could imply . Make a list what is and what is not acceptable for you, draw a line between public and private and try not to cross it.