Divided by discipline? A systematic literature review on the quantification of online sexism and misogyny using a semi-automated approach

Observing the pros and cons of all the citation databases, it was decided to use the Web of Science API to collect data based on the individual areas of discipline - SS and CS, as the primary data source. Since many of the relevant computational papers were seen to be published in ArXiv within the given period, those papers were also considered as part of the data collection. It was done to ensure that we get full coverage of both published and unpublished works (pre-prints), relevant to the study of sexism and misogyny during the 11 years. As discussed before, we also included the publications that were informed through external sources. While the Web of Science was taken as the main source for published works, ArXiv was taken as a source for unpublished works. We then combine the selected search results for the next section 3.3, before removing the duplicates.

Topic modeling⁴⁴4“Topic modeling is a machine learning technique that automatically analyzes text data to determine cluster words for a set of documents. It leverages ‘unsupervised’ machine learning to analyze and identify clusters or groups of similar words within a body of text” (Pykes, , 2023). was used with the pre-processed data containing the abstracts and titles from both disciplines, to generate clusters of topics based on the documents (i.e., the collection of studies containing abstracts and topics). Among all the topic modeling techniques experimented with, BERTopic (Grootendorst, , 2022) proved to be the best choice for the task. It is because BERTopic “leverages transformers and c-TF-IDF to create dense clusters allowing for easily interpretable topics whilst keeping important words in the topic descriptions”(Grootendorst, , 2022), hence enhancing the topic recognition ability by the model.⁵⁵5More information on the techniques used in this methodology is explained in the supplementary section: J.1

We applied the BERTopic algorithm to the collections of CS and SS papers separately to capture the topics of research between these two disciplines, and to check the differences in the themes of sexism between them. Among all the experimentation conducted - including setting different ranges of parameters to get the best representative models from there, we further employed fine-tuning of the model to improve on that, by using multiple representations from the model. For our work, we used these different representations from keywords and phrases to summarize and custom labels. The Figures 3.3 and 3.4 indicate the topics recognized by the model. Using the aforementioned parameters, we used the BERTopic model to groups documents into topic clusters, identified by their keywords and keyphrases. It uses clustering to define topics and hence does not assign more than one topic to each document. In the figures, each point corresponds to each document in their respective disciplines. BERTopic uses HDBSCAN by default for clustering, and it does not force all the data points to be a part of any of the recognized clusters or topics. Simultaneously, BERTopic uses UMAP to perform dimensionality reduction. We then used further customization of the UMAP by setting the parameter ‘n_components’ to 2, to ‘pre-reduce’ embeddings for visually depicting our model results in the two figures. For those topics that do not form a part of any groups (also termed as “outliers”), the points are marked in grey in the figures. The colored points in both the figures indicate topics, and each color represents a unique topic for the sets of documents, which have further been marked correspondingly with labels of the same color boxes. The algorithm itself exhibits strong local clustering to group similar topic categories together, to which we also controlled the balance between the local and the final structure to efficiently distinguish between each topic.

It uses a light hue of the same colors encircling each topic to indicate the cluster belonging to the respective topic. In the figure, different colors indicate

While some points in the same cluster may look further away than the points from another cluster, it is due to its projection in 2D-dimensional space which we did for better visualization; hence the points within the same clusters are closer in a multi-dimensional space.

In the figures, the highlighted topics depict the most relevant searches for our study. We see that while SS studies have the most diversity in their topical approach as compared to CS, they still lack research on the automated detection or identification task of online sexism and misogyny (which is the aim of this research). Online sexism and misogyny form a part of online hate speech, we use the topic in CS that specifically looks into the quantification of the same, rather than analyzing the gender bias in various forms. To quantify the hate speech, we would require research works focusing on the detection tasks. Hence, at this stage, the topic we chose out of the two disciplines and the various categories is the ‘Hate Speech Detection in Social Media’. In the following step, we further exhibit the relevance of our chosen topic from this step by performing the keyword co-occurrence network analysis.

To validate if the topics captured from the automated selection of topics from each discipline in the previous section were representative of the corresponding documents, we navigated the disciplines and each topic, alongside their respective keywords. To obtain the most frequent keywords in the set of documents, we use KeyBERT⁶⁶6KeyBERT is a minimal and easy-to-use keyword extraction technique that leverages BERT embeddings to create keywords and key phrases that are most similar to a document.(Grootendorst, , 2020) to extract embeddings with a BERT model to get a document-level representation from our abstracts and titles. From each document, we used KeyBERT to identify key phrases that would provide with a more accurate summary of the documents, rather than simple keywords. KeyBERT works by creating an embedding of document texts, from which BERT key phrase embeddings of a pre-defined word n-gram range length of 1-2 words⁷⁷7Word n-gram range lets users decide the length of sequence of consecutive words that should be extracted from a given text. were created. Consequently, cosine similarities between the document and their respective keyphrase embeddings are calculated to extract the top 10 keyphrases that best describe that document. These selected keyphrases per document are then compared against the whole set of documents. We chose to look into the 100 most common keywords in the documents taken at both discipline level (CS and SS), as well as the topic level (each topic based on the topics we generated in Section 3.3.1). This was done to check the relevance of the keywords, and consequently the set of documents that would best represent our research objective of performing a literature review on the quantification of online sexism and misogyny.

On comparing the keywords present in Computer Science (see Figure I.1) and Social Science (see Figure I.2), we found that the contents of the papers (from their title and abstract) focused on different kinds of sexism and misogyny - both indicating similarity in topics, but contribution at different capacities. Social science focuses on multiple aspects of sexism and misogyny and has been studied using both qualitative and quantitative methodologies. While computer science mostly focuses on task types (such as prediction, detection, identification, etc.) and setting analytical goals. On further analysis, we selected the most relevant topic among all of the highlighted topics from both disciplines and performed a keyword search on each of them. Figure 3.5 corresponds to the topic ‘Hate Speech Detection in Social Media’, and it showed the most promising result of containing the necessary keywords needed for this study.

The figure consists of the 100 most frequent keyphrases in the topic. The size of each circle (node) indicates the weight of that particular keyphrase in the set of documents. The color of the nodes can be anything that is recognized by Matplotlib in colormap specified and could be randomly generated. The edge width of the edges joining the nodes indicates the number of associations between the two connected nodes - the more the number of connections, the thicker the edge width. The co-occurrence network for all the highlighted topics from the previous step showed us the most used keywords pertaining to the said topics. By studying the generated keywords, we could capture the general objectives of the respective papers under the same topic, as stated from their abstracts. We then selected the most relevant topic for our work, which focuses on computational approaches in detecting online sexism and misogyny (a part of the online hate speech discourse). Consequently, we decided to perform the full-text literature review on the articles which fell under the topic of ‘Hate Speech Detection in Social Media’.

On completing this step, we included a few more of the studies that had been identified through citation tracking and did a full-text screening of all the collected texts.

Following the selection of the citation database, the automatic filtering of papers using BERTopic, and the validation via topic keywords, we identified the necessary characteristics 3.1 of the data to emphasize the findings which eventually led us to select the research articles based on the selection criteria: A.2, by performing a manual screening. Eligible articles were divided into two categories at this stage. The first category is the data acquired from the automated stage, while the second category is the records identified from citation tracking.

Finally, we thoroughly reviewed all publications related to the quantification of sexism and misogyny in online social platforms to determine their focus and methodologies, by reading their full text.

A total of 96 full-text articles were analyzed qualitatively, as shown in Figure 3.1. We assessed them based on four criteria, namely:

1. (i)

   Irrelevant study focus - Whether they are focused on studying the propagation of sexism and misogyny (irrespective of whether they indicated such in the abstract). Many of the works focused on hate speech, but because we wanted to only review sexism and misogyny, they were eliminated.

2. (ii)

   Irrelevant study designs - Whether the intended outcome of the research was not about performing a computation analysis on the detection or identification of online sexism and misogyny.

3. (iii)

   Studies not quantifying sexism/misogyny - Whether the paper focused on a review of studies in the relevant topic; or contained a summary of the author’s thoughts from multiple papers, such as opinion pieces around the same topic.

4. (iv)

   Sources could not be traced back - in case the paper’s paraphrased contents with citations were not reflective of the summary the original authors indicated in their study.

45 out of the 96 research articles qualified from this step, from surpassing these exclusion criteria and as a result, were included in the final qualitative analysis.

The full text of all the papers was reviewed qualitatively and information about each was added to a summary table covering the following points:

• •

  Forms of hate speech studied. It is because hate speech could encompass a lot of things, including sexism and misogyny.

• •

  Definitions of sexism or misogyny (or both) used in the study.

• •

  Language(/s) of the data used for the study.

• •

  Data selection criteria. This could depend on the original data collection method, such as - using keywords, hashtags, public profiles by monitoring user’s online activity, users identified as sexist/misogynist, tags of sexism, specific phrases; or even based on a particular timeline of interest.

• •

  Datasets used and their types (external, API-generated, etc.)

• •

  Dataset modifications, if done. This could be in the form of data augmentation, counterfactual examples, document expansion by adding semantically similar words, transliterating multilingual dataset to uniform language, and many more.

• •

  Broadcast media or social media platform which is of interest for the study.

• •

  Annotators used in the study, and their tasks. If each or group of annotators had different tasks, that was also recorded.

• •

  Pertaining to the previous point, the Kappa values that are statistical measures used to measure inter-rater reliability, are also noted.

• •

  Research bias addressed or acknowledged in the study. If acknowledged, it is posted as a limitation in the paper.

• •

  Pre-processing or post-processing done on the data.

• •

  Performance metric used.

• •

  Embedding type used, since this could range from word-level to node-based.

• •

  Classification or clustering type, and the respective models.

• •

  Syntactic, linguistic, and semantic/lexical features.

• •

  Prompt topics and intersectionality (if present).

Recently several SLR tools have incorporated semi-automation using Artificial Intelligence techniques, for supporting the screening and extraction (pre-screening) phases (Bolanos et al., , 2024), like we did in our research. Of such tools, only a few use topic modeling for their work. Such as, RobotAnalyst⁸⁸8https://www.nactem.ac.uk/robotanalyst/ and SWIFT-Review⁹⁹9https://www.sciome.com/swift-review/ uses Latent Dirichlet Allocation (LDA) that assigns a topic to a paper based on the most recurrent terms shared by other papers using a generative probabilistic model, while Iris.ai¹⁰¹⁰10https://iris.ai/ clusters the papers according to a two-level taxonomy of global topics and specific topics (Bolanos et al., , 2024). The former two tools depend on the term frequencies while the later perform Named Entity Recognition (NER) and allow users to to customize entity extraction by letting them define their own set of categories beforehand. Even with its advantage of the superior language capabilities to produce one of the most advanced techniques in language topic modeling today (Briggs, , 2023), BERTopic has remained unexplored for the same task. In our work, we use that potential alongside the promising result of Large Language Models (LLMs) in their information extraction capabilities, to cluster our topics before validating the results with network analysis and selecting the topic(/s) more suited for our work. This proved to be particularly useful to us in the screening and qualitative assessment phase as empirical analysis of the topics generated to their corresponding papers showed that the approach accurately clustered similar papers together.

We provide an author collaboration network in Figure 4.1, where the name of the researchers are nodes, their size and color indicating the number of relevant manuscripts they authored or co-authored. The connections between the authors are indicative of co-authorship on manuscripts, and their weighted edges imply the frequency of co-authorship.

Table 4.1 gives a summary of all the general characteristics we found in the full-text reviews of the selected studies. It provides a summary of the most used categories of each field (design/methodology) that the documents in our literature review have used. The other categories which were not featured in the table were mostly used by only one document. Each document could have multiple categories under the same field of design or methodology. For example, one document could be researching datasets of multiple languages in different platforms of interest and using multiple models, with different levels of classification at different stages. The categories that are uniquely present in a document are marked with an asterisk(^∗) beside it, while the fields with their entire list of categories in the table are marked with an obelisk(^†). In here, the asterisk(^∗) symbol would not just indicate that the feature itself is unique to the field, but also all the documents should add up to the total number of literature listed.

As per the Table 4.1, we do see the frequency of each source of online data and the machine learning models used in the 45 manuscripts we reviewed. The type of classification of sexism and misogyny used in the said studies are otherwise unknown and how they link between the sources of online data and the computational methodologies is an important source of information to indicate the multi-level connections between the variables, and consequently its impact on the quantification of sexism and misogyny. Figure 4.2 show the connection between nodes in each level of information (source of online data, classification type and model used), while the connections are the links between each level with their weights indicating the frequency of connections between each node type at different levels. This is a many-to-many mapping between the three levels, and show the flow of information between each of them. The colors of each node at different levels represent the unique relation between the linked nodes at each level (e.g., Twitter [level 1] $=>$ Misogyny (5 categories) [level 2] is different from Twitter [level 1] $=>$ Misogyny (binary) [level 2], and Reddit [level 1] $=>$ Misogyny (binary) [level 2] is different from Reddit [level 1] $=>$ Abuse/Aggression [level 2]). The abbreviations for the models in level 3 are found in Supplementary Section H. Overall, the Figure 4.2 gives a clear evidence that Twitter was the most explored online data source to investigate different forms of sexism and misogyny.

Most computational works on sexism and misogyny took the automated identification problem as a binary classification task, i.e., deciding if the text in question is sexist/misogynistic or not. For defining the terms, researchers used different (non-standardized) forms for their work because of computational benefits, such as model performances. Though they served to be effective in some instances, they also presented with limitations of their own. For instance, Grosz and Conde-Cespedes, (2020) define sexism as the prejudicial and discriminatory nature of sexist behavior pervading in the social context, especially for women. Using theoretical concepts, along with the typology of abuse presented in earlier research, Guest et al., (2021) define misogyny as content “directed abuse at women or a closely-related gendered group (e.g., feminists).” Whereas Lynn et al., (2019) define misogyny as a hate crime, a result of a “cultural attitude of hatred for females because they are female”, and presented with two bidirectional-DL models (Bi-LSTM and Bi-GRU) with dropout layers, which performed well in sensitivity and accuracy even with a slightly imbalanced dataset. Having a multi-agent classification approach to a model can further enhance its performance when built using sentence embedding techniques and TF-IDF, enriched with misogyny lexicons (Attanasio and Pastor, , 2020). Plaza-del Arco et al., (2021) proposed Multi-Task Learning (MTL) system with hard parameter sharing approach (sharing the hidden layers between all tasks, while keeping several task-specific output layers) using BERT-based models for utilizing the transferred knowledge from multiple other (but related to sexism identification) tasks like polarity and emotion classification and offensive language detection classification helped in the identification, both in binary and multiple categories. Though emotion was not helpful in categorizing, MTL shows promising generalization to the original task. Frenda et al., (2019) exploit stylistic, semantic and topic information about misogynistic speech to identity misogyny and classify it to different categories. For gathering linguistic features, they propose an approach based on stylistic features captured by means of character n-grams, on sentiment information and on a set of lexicons built by examining the misogynistic tweets from training data provided by the organizers. Each text was represented by a vector composed of all specific topic features(set of lexicons), pondered with Information Gain, and character n-grams, weighted with TF-IDF measure. This set of features were experimented employing a Support Vector Machine (SVM) algorithm and an ensemble technique, reaching promising results. Canós, (2018) worked on the same data and task, experimenting SVM alongside TF-IDF with a one-vs-one and one-vs-rest classifier approach, where the later proved better for English, presumably because of larger vocabulary. On the other hand, Nozza et al., (2019) defined several templates to create a balanced synthetic dataset for their proposed DL model- Universal Sentence Encoder (USE), which further debiased their model features to be less-sensitive to identity terms and yet obtain a better categorization.

When it comes to evaluating performance, classification models with traditional computational approaches seem to fair relatively similar or better (in some cases) at automated sexism/misogyny identification tasks. Indurthi et al., (2019)’s work shows how different set of pretrained embeddings trained from different state-of-the-art architectures and methods when used with simple machine learning (ML) classifiers like SVM and XGBoost perform very well in binary classification tasks. Kohli et al., (2021) used two kinds of methods: first using an ensemble approach comprising of XGBoost, LightGBM and Naïve Bayes; and second employing BERT-based architecture. Both the models performed well on binary identification task, but differently on different languages and aggression label analysis, one of which was gendered, due to the overlapping context in all. Overall, SVM is seen to be the best-performing conventional classifier (hence taken as a baseline for some of the works), and a lot of papers have used it in their work as a standalone classifier, or as an ensemble voting classifier (Frenda et al., , 2018; Nascimento et al., 2022a, )- alongside other classifiers like Gradient Boosting and Random Forest (RF). Regression models like Logistic Regression have also been used by a lot of studies, especially for binary tasks. While Decision Tree (Plaza-Del-Arco et al., , 2020), and RF (Singh et al., , 2021) has also been used, they do not show much success among the conventional ones whereas most DL models use Fully Connected (FC) layers for classification (Bashar et al., , 2019).

Though almost all the computational methods employ classification techniques, it is not the only way. But it is favorable, for good reasons. Clustering techniques are mostly useful for content analysis and to study discourse, to help identify implicit themes/topics from the data which may be (unintentionally) omitted during manual inspection, and reassignment into its overarching categories for better interpretation, even though may sometimes provide superficial results (Siddiqi et al., , 2018). Karami et al., (2019) employs unsupervised text-mining approaches like LDA topic modeling. Utilizing the themes they found, they performed qualitative thematic analysis before finally moving to a theoretical thematic analysis to group the previously identified topics into four categories of sexism. Melville et al., (2019) also uses LDA for grouping 7 topics, and alongside clustering based on Louvain algorithm (Blondel et al., , 2008) for grouping 20 topics. They define sexism based on themes and sites associated with the experience of sexism from Everyday Sexism (Bates, , 2015) and journalism. From these studies, it is evident that clustering is more useful for content analysis, rather than for the detection/identification tasks.

This subsection is intended to shed light on the different computational strategies that have been used to quantify sexism and misogyny, while also segregating the strategies based on some differentiators like the small dataset size and the dataset languages used.

Following from the previous subsection where we introduced our argument that sexism/misogyny is not a binary task, in this section, we expand on that point by providing social science theories and scales to explain the need to not computationally limit the classification to the binary output. To support that, alongside including the theories and scales, we also analyze how some studies have aided their work with these theories in any capacities (i.e., the extent of adaptation - using one or more categories of the scaling) and implemented them at any stage of their research. We distinguished each subdivision into two parts: the concept and the applications, to help us differentiate between the concepts themselves and on how they are implemented in studies.