Examining Gender Bias in Languages with Grammatical Gender : Literature Review

Ashima Suvarna & Hritik Bansal

Department of Computer Science

University of California, Los Angeles

Introduction

In linguistics, grammatical gender system is a specific form of noun class system, where nouns are assigned with gender categories that are often not related to their real-world qualities (eg: पानी (water) in Hindi is masculine). In such noun class systems, the corresponding dependent articles, adjectives, and verbs must also agree in gender with the noun ( e.g. in Spanish: la buena enfermera: the good female nurse, el buen enfermero: the good male nurse).

To further understand the manifestation of grammatical gender across languages globally, we can classify language groups broadly into three categories with respect to gender :

• Grammatical gender languages (e.g., French, Spanish, Czech, German, Hindi) are languages in which personal (i.e., human) nouns (French l’enseignant, l’enseignante “the teacher”, le fils, la fille “the son,” “the daughter”), as well as inanimate nouns (Spanish la mesa n.f. “the table,” el despacho n.m. “the desk”), are classified for gender. These nouns control agreement of various other lexical categories such as determiners, adjectives, or pronouns.

• Natural gender languages (e.g., English) do not assign any gender to the inanimate nouns. Most personal nouns are not specified for sex or gender identity.

• Genderless languages (e.g., Turkish, Finnish, Odiya, Bengali) are languages where most nouns, as well as pronouns, are generally unspecified for gender. The structure of these languages, therefore, does not enforce the use of gender-marked forms, even though this information can be conveyed by lexical means, such as in the Turkish, market for “man or male” or kız for “girl.”

In this work blog, we review the techniques and concepts outlined in the 2019 EMNLP paper - Examining Gender Bias in Languages with Grammatical Gender. The paper focuses on Spanish and French as grammatical gender languages which belong to the family of Romance languages and proposes techniques to identify and mitigate gender bias for them. The novelty of this blog lies in its brevity and extension of preliminary experiments on a language, Hindi, that was not covered in the paper.

Contribution: Analysis on Hindi

We extend the experiments to Hindi to further this study on a family of Indo-Aryan languages. Hindi is the second largest spoken language in the world with various unique features like grammatical gender, lack of gender-neutral pronouns, and the flexible subject-object-verb syntax as opposed to the fixed subject-verb-object syntax common in English, Spanish, or French. Moreover, limited research has been dedicated towards exploring and mitigating gender bias in Hindi.

(Subject)(Verb)(Object)

The quick brown fox jumps over the lazy dog. (English)

तेज, भूरी लोमडी आलसी कुत्ते के उपर कूद गई | (Hindi)

El rápido zorro marrón salta sobre el perro perezoso. (Spanish)

Le renard brun rapide saute par-dessus le chien paresseux. (French)

Gender Bias in Languages

Existing studies on gender bias almost exclusively focus on English word embeddings which is a natural gender language [1-4]. This work focuses on Monolingual FastText Wikipedia embeddings that are publicly available for community use.

The mitigation techniques used to mitigate bias in English cannot be directly applied to grammatical gender languages where all nouns (animate and inanimate) are assigned a gender class and control the agreement of adjectives, determiners, or pronouns with gender. Most analyses on gender bias in embeddings assume languages (eg: English ) to have only the semantic gender direction. However, when analyzing languages like Spanish and Hindi, the grammatical gender direction is equally important and is explored at length in the paper.

In order to analyze bias in the embeddings of gendered languages, the authors define the idea of two gender directions.

• Semantic Gender Direction: This gender direction in a language is defined by a set of gender definition words (eg: महिला (woman), पुरुष (man), अध्यापिका (female teacher)). This direction would carry the meaning of gender in non-genderless languages.

• Grammatical Gender Direction: This gender direction is specifically defined for languages with grammatical gender. It is calculated by a set of most common masculine and feminine nouns (e.g: पानी (water), table, chair).

Semantic and Grammatical Gender Direction in Word embeddings

Semantic Gender

The authors collect a set of gender definition pairs (eg “mujer” (woman) and “hombre” (man) in Spanish). Then, the gender direction is derived by calculating the major component of principal component analysis (PCA) (Jolliffe, 2011) over the differences between male- and female-definition word vectors. The paper assumes that the major component of the PCA captures the meaning of gender in French and Hindi just like English.

Grammatical Gender

To identify the grammatical gender direction, assuming that the language has only two gender classes, we collect a set of masculine and feminine nouns for Spanish and Hindi. Since most nouns (e.g., water) do not have a paired word in the other gender class, we do not have pairs of words to represent different grammatical genders. Therefore, instead of applying PCA, the authors learn the grammatical gender direction by Linear Discriminant Analysis (LDA). In our implementation of the paper, we use SVM Linear Classifier to learn the grammatical gender direction as it has been shown the grammatical gender direction emerging out of LDA and SVM methods are highly correlated.

Semantic gender directions are made orthogonal to the grammatical gender directions by projecting out the grammatical gender direction from the gender direction computed by PCA. Mathematically:

Gender Direction Plots

In the following experiments, Hindi definition pairs were translated from the english definition pairs include in the paper.

Semantic Gender Direction - Spanish and Hindi

We visualize the gender-definition pairs along the semantic gender direction in Spanish and Hindi in Fig 1 and 2. We note two key observations here :

(a) Masculine and Feminine Gendered words are clustered on either side of the origin for Spanish and Hindi. However, there exists a few feminine Hindi words on the masculine side for Hindi.

(b) Gender definitional pairs in both Hindi and Spanish are not equidistant from the origin, that is, feminine words in Spanish are farther along the negative semantic gender axis than the masculine words are along the positive semantic gender axis. On the contrary, feminine words in Hindi are closer to the origin than masculine words.

Note on all Figures:

Masculine words are represented in blue color whereas feminine words are represented in red color.

Some words in Hindi are spelled incorrectly due to limitation of the matplotlib Hindi font display and do not affect the embedding values.

Grammatical Gender Direction - Spanish and Hindi

We plot the monolingual word embeddings of Spanish and Hindi language along the grammatical gender direction below. We make the following key observations:

(a) Masculine and Feminine Gendered words are clustered on either side of the origin for Spanish and Hindi. However, there exists a few feminine Spanish as well as Hindi words on the masculine side.

(b) The grammatical gender component of a Spanish word changes significantly when translated to Hindi. For instance, the Spanish word for mother (madre) and Hindi word for the same (मां) have very different strengths along the grammatical gender direction.

Here we plot the monolingual word embeddings along the semantic and grammatical gender direction together. We make the following key observations:

(1) Most inanimate nouns (e.g., water, bowl) lie near the origin point of the semantic gender axis for both languages.

(2) The feminine words are farther on the feminine side on the semantic gender direction compared to masculine words. This indicates the presence of gender bias in the word embeddings of Spanish and Hindi.

Figure 5 : Projections of selected words in Spanish on grammatical and semantic directions.

Figure 6 : Projections of selected words in Hindi on grammatical and semantic directions.

What do we desire from an unbiased monolingual word embedding of language with grammatical gender?

Inanimate Nouns

Since, inanimate nouns in languages with grammatical gender do not emulate the gender characteristics assigned arbitrarily to them, in unbiased monolingual embeddings, these nouns should have no component in the semantic gender direction with some component in the grammatical gender direction.

Animate Nouns

Animate nouns such as occupation pairs (eg : abogado (lawyer_male), अध्यापक (teacher_male)) and gender-definition words (eg : man, महिला) in unbiased monolingual embeddings should be equidistant from a chosen anchor point along the semantic gender direction (origin in monolingual settings).

Mitigation Objectives - Monolingual Settings

In this survey, we focus on post hoc processing word embeddings of inanimate nouns only -- does not require any training.

Post hoc processing of word embeddings of Inanimate nouns

Short Note on Bilingual Word Embeddings

In this survey, we focused on monolingual embeddings and explored the gender directions and mitigation for monolingual settings. In order to extend this survey to bilingual embeddings, the authors of the paper focus on bilingual word embeddings that align a grammatical gender language like Spanish or Hindi with a language without grammatical gender like English. The definitions of the two gender directions are similar: they construct the grammatical gender direction using the same sets of words in the gendered language since the other language does not mark grammatical gender. They combine the gender-definition words for both languages and remove the grammatical gender component to get the semantic gender direction similarly.

Mitigation techniques in bilingual settings can use English word vectors to de-bias word embeddings in the grammatical gender languages using three techniques proposed in the paper :

• Mitigating Before Alignment (De-Align)

  • Gender bias can be observed when aligning gendered languages with language without grammatical gender, the authors propose using English to facilitate mitigating bias. Specifically, they first apply the “hard-debasing” approach[1] to English embedding and then align gendered language with the bias-reduced English.

• Shifting Along Semantic Gender Direction (Shift)

  • Similar to the approach proposed in monolingual setting where we shift the gender biased words along the semantic gender direction, in the bilingual settings the authors propose using English words as the anchor point to compute the 'shift'.

• Hybrid Method (Hybrid)

  • This approach combines the above-mentioned approaches. In particular, they first mitigate English embeddings, align English embeddings with the embeddings of gendered languages, and then shift words in languages with grammatical gender along the semantic gender direction. (Anchor points can be chosen as the origin of the semantic gender axis or the English words.

References

1. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems, pages 4349–4357.

2. Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and KaiWei Chang. 2018b. Learning gender-neutral word embeddings. In Empirical Methods in Natural Language Processing, pages 4847–4853.

3. Sunipa Dev and Jeff Phillips. 2019. Attenuating bias in word vectors. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 879– 887.

4. Kawin Ethayarajh, David Duvenaud, and Graeme Hirst. 2019. Understanding undesirable word embedding associations. In Proceedings of the 57th Conference of the Association for Computational Linguistics, pages 1696–1705.

Link to Notebook :

You can find the reproduced results of the topics covered in the blog on https://colab.research.google.com/drive/1ZuEgNP8DuIBTOyl6bZzfd-j4KQBm3x0d?usp=sharing