Beware of the Binary
When I first joined a queer activist group at my university as an undergrad, I suggested solving the lack of gender neutral bathrooms by removing the ‘mens’ and ’womens’ signs from bathroom doors. More experienced activists had to explain to me that removing the labels would not magically create queer-inclusive spaces and that taking a screwdriver to bathroom signs would more likely harm trans and non-binary students than help them. While I managed to learn something from the embarrassment, it is this kind of short-sighted actionism that pervades most gender-bias research in NLP.
“Bias research sort of exploded in recent years and gender has become seen as the easy case”, says Hannah Devinney. Hannah is a fourth year PhD student at the University of Umeå, where they research the intersection of gender studies and natural language processing. “There is a preponderance of gender data out there. Governments tend to track it; social media sites track it, too. […] We are culturally concerned with gender equality, and people have gotten into gender-bias research without thinking. And that's the part that's frustrating because if you actually stopped and thought about it, it's a much more difficult question than it seems on the surface.”
While gender data is easy to come by, the assumptions under which it is created are often both wrong and harmful to the queer community. Gender is generally assumed to be binary (there are only two groups, men and women), immutable (people don’t change their gender) and externally observable (one can deduce gender from pictures, names, pronouns, chromosomes and so on). It’s easy to come up with dozens of counterexamples to these assumptions: After all, non-binary, trans and intersex people do exist. A widely accepted conceptualisation of gender that includes all of these groups is gender performativity: Gender is constructed by the actions we take, for example by the way we behave in different situations, the words we choose to describe ourselves and the way we position ourselves in relation to existing gender categories. Gender can change depending on both cultural and situational context. But this view is mostly ignored in the gender-bias research community. Gender data sets strip individual gender performance from its situational context, showing only the result of two distinct categories. In their paper Hannah and their collaborators show that 93% of gender-bias papers in NLP treat gender as binary and immutable, all the while claiming to produce gender-fair systems.
On our Skype call I ask Hannah how they keep from becoming frustrated. I felt angry just reading about how a majority of gender-bias research in NLP perpetuates harms that I and my trans and non-binary friends face every day. My gender is already deemed ‘too complicated’ for my work place, my family and a good portion of my cisgender friends, and it is infuriating that it should be also ignored by a research field that cares about fairness and values complexity. “I‘ve read 176 papers and there are a lot of curse words in the spreadsheets”, says Hannah “But doing the time split actually helped, because it was an encouraging difference”. In their study Hannah compared papers written in 2020 to papers written in 2021 and found that while the overall level is low, the number of papers with a more nuanced view of gender increased, with 9 papers actively including multiple genders.
Additionally, in their doctoral program Hannah has found a good support network. “Most of us are doing interdisciplinary research and most of us are in this model where we're co-funded by our home departments and the Graduate School for Gender Research. So there's a lot of solidarity.” And the way in which gender is formalised in the research discourse is just one part of their PhD thesis. They also look at quantitative support methods for qualitative analysis and they are adapting a Swedish part of speech tagger for gender-neutral pronouns. “There is this tagline of ‘AI for all’ and I don’t think that this is my idealised way forward”, they say “I'm not particularly worried if what I make doesn't work well for a straight majority, because I'm not building it for them. They have enough stuff.”
Looking forward Hannah wishes for more AI systems being built with queer people front and centre. But they fear that this will become more difficult with the increased prominence of large language models. “Large language models are eroding the trust between the people who are producing the data and the people who are training models on that data”, Hannah says. Because art was used without the artists’ consent to train models, creators recently started moving their work to locked sites. “But I think that some of the coolest research has been done out of ‘Archive of Our Own’ and other online creative spaces and it was done by asking people for permission.”, says Hannah “With that permission we can build something that is written by and for and about queer and trans people. This is the trajectory that we should take when we're talking about inclusive technologies.”
In a short term view of the future, Hannah is off to a two month break without internet, working with 9 to 14 year old kids in a youth camp. “My cabin mates have been doing this cool thing where they come up with a new pronoun for their non-binary friend every day and announce it before breakfast. And then that's just the one they use all day. It's so cute.”, Hannah says. While the internet is awash with people ascribing superhuman intelligence to large language models I keep hoping for systems as eloquent and inclusive as those kids in the woods.
You can find more of Hannah’s research and writing here
This post was written by Sabine Weber. Sabine is a queer person who just finished their PhD at the University of Edinburgh. They are interested in multilingual NLP, AI ethics, science communication and art. They organized Queer in AI socials and were one of the Social Chairs at NAACL 2021. You can find them on twitter as @multilingual_s