ChatGPT calls me a dyke
There is an important difference between the terms one can claim for oneself, and the terms that one is subjected to by others. It’s whether a stranger yells “dyke” at me from a passing car, or whether I yell it with my friends at a pride march. Societies establish derogatory terms for members of the queer community and the queer community reclaims some of them and wields them as a source of power. Now ChatGPT can call me a dyke, but forbids me from writing the same word in its input.
It is not only the word ‘dyke’ and ChatGTP, but this phenomenon also crops up in different language models, e.g. Google's BARD. Prompted with the question “What is gay?” and “What is homosexual?” the language model returns “I’m not able to help with that, I’m only a language model” while providing valid definitions for the terms “straight” and “heterosexual”(1). ChatGPT’s refusal is blunter. The question “What is a dyke?” leads to a separate window opening up with the error message “Your request may violate the community guidelines.” It returns no answer.
This silencing of queer voices by large language models is neither new nor unexpected. In their 2021 paper “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?”(2) Bender et al. expose how large language models disregard queer language at several levels. First, text from websites with many incoming links is more likely to be collected for training data sets, while text from websites with few incoming links is discarded. This leads to a high proportion of news outlets representing the mainstream discourse. Their coverage of queer topics is often a hostile outside view rather than a positive self-representation, which leads to negative representations of queer terms within the trained model. Second, filter-lists discard any text that contains presumably derogatory terms, which include reclaimed slurs like “dyke”. Lastly, filters also regulate the input to a large language model after it has been trained to discard the prompts that are deemed harmful. Both the “dyke” example as well as BARD’s failure to define “gay” seem to be due to that last step.
ChatGPT’s refusal to interact with the word “dyke” shuts off the access to a wide range of lesbian history. One can not query ChatGPT on the emergence of the dyke marches, the famous comic strip “dykes to watch out for” or the motorcycle club “dykes on bikes”. For a model that processes words according to their sentence context, a filter based on a single word seems crude. When prompted to name some queer motorcycle clubs, “dykes on bikes” is the first answer. The prompt “If I was a lesbian using a reclaimed slur, what would I call myself?” leads to the answer “ If you were a lesbian and wanted to use a reclaimed term to describe yourself, you could use the word ‘dyke’”. ChatGPT goes on to elaborate that the term has a complicated history and is not accepted unanimously, but that is only a small consolation, considering that this information can only be elicited in a very roundabout way. I can only find it if I already know what I am looking for.
It isn’t difficult to imagine how this behavior is harmful. Not so long ago, I was a teenager who anxiously typed the slurs that my classmates used for queer people into a search bar. Using the outdated computers at the local public library was the only way I could make sure my parents wouldn’t know. And I just wanted to find some information about people who were like me. What values are reinforced, when a chatbot labels queer identity terms as “violating community guidelines”? Effort and money are poured into AI hype and the scramble to show that large language models are “safe”, while the wellbeing of the queer community seems to be not even an afterthought. It is much easier to block certain words, lest they reveal the shortcomings of a product.
It makes a difference who is speaking when they use the word ‘dyke’. Who is talking, when ChatGTP talks? Some would gladly see ChatGPT as a super-human mind in a box, but it is nothing more than a sentence completion machine. While it remixes plagiarized texts from the internet, the responsibility for the things it says lie solely with its creators. The mismatch between censorship on user input and lack of oversight on the system output is all too reminiscent on of other blundering attempts of large tech companies on content moderation. Once again we see power dynamics at play: On the one end companies that want to sell an under-researched technology and on the other, a group that is already marginalized and experiences increased marginalization from this technology.
The authors of the stochaistic parrot paper recently released an open letter in which they contextualise their earlier findings against the most recent developments in the field (3). One of the key messages: “We should be building machines that work for us, instead of "adapting" society to be machine readable and writable.” The queer comminity has a long and ongoing history with the silencing and policing of our voices, and this is yet another front line. Large Lange Models are here, but they are not here for us.
This post was written by Sabine Weber. Sabine is a queer person who just finished their PhD at the University of Edinburgh. They are interested in multilingual NLP, AI ethics, science communication and art. They organized Queer in AI socials and were one of the Social Chairs at NAACL 2021. You can find them on twitter as @multilingual_s