OSPC: Artificial VLM Features for Hateful Meme Detection

Hatefulness in memes can be classified into several categories. According to a definition from the challenge website, these include: Hate, Offensive, Propaganda, Harassment/Cyberbullying, Violence, Self-inflicted harm, and Exploitation (Sharma et al., 2022). Each category requires a nuanced understanding and specific detection strategies. However, defining a meme as hateful is complicated by the inherent subjectivity and cultural sensitivity of each category, making the establishment of a universal threshold for meme classification challenging.

In the context of Singaporean meme analysis, the TotalDefMeme dataset (Prakash et al., 2023), another resource cited on the challenge website, illustrates the complexity of this task. The dataset was annotated by three annotators per meme, revealing significant discrepancies in their interpretations, despite all annotators being from the same cultural background.

Given these challenges, our approach shifts focus from trying to replicate the exact sensitivity levels defined by the OSPC. Instead, we utilize foundational models trained on a vast scale to encompass a broad range of contextual and cultural nuances. These models are better suited to capturing a general consensus on harmfulness than any single annotator, particularly in a zero-shot classification scenario where no specific dataset is provided for training.

Beyond merely processing text, the model must comprehend its meaning and the intentions behind it. The phrase “ya ya papaya” might sound like a reference to a delicious fruit to an average reader, but in Singapore, it connotes an arrogant or proud person. This highlights a critical challenge: the model must grasp the cultural context of multiple languages. To achieve comprehensive understanding, a model trained across all four national languages of Singapore is essential, as merely relying on a selectively curated dataset would be insufficient.

Whilst such LLMs like AISG’s SEA-LION (Lowphansirikul et al., 2021) exist, these models lack integration with visual understanding capabilities. For instance, the phrase “I love Singapore” can convey drastically different meanings depending on the associated imagery — a burning trashcan versus a plate of delicious chili-crab.

Hence, it is crucial to employ a Vision-Language Model (VLM) trained on a broad spectrum of Singaporean national languages and equipped with visual adaptors. The best candidate for our requirements was the NousHermes2Yi model from NousResearch, which boasts 34 billion parameters³³3https://huggingface.co/NousResearch/Nous-Hermes-2-Yi-34B, and has been recently adapted for visual tasks in Llava-NeXT (Liu et al., 2024). Through heuristic testing, this model demonstrated robust understanding capabilities in English, decent performance in Chinese, and basic proficiency in Tamil and Malay. It also excelled in identifying cultural nuances and harmfulness, proving its efficiency.

However, to fully harness its capabilities, support for understanding Tamil and Malay was necessary. A baseline model without translation achieved an AUROC of 0.6127 on the OSPC test dataset; with Tamil translation, this score increased to 0.6522 AUROC. Interestingly, adding languages the model was already proficient in, like English or Chinese, resulted in a decreased AUROC to 0.6292, when captured via Optical Character Recognition (OCR) using PaddleOCR (Du et al., 2020), which demonstrated excellent Chinese and Tamil text recognition capabilities. In fact to further speed-up processing, we make use of an OCR trick, that is to only consider characters which contain Chinese or Tamil symbols. This very efficiently filters out any English and allows the VLM model to refer to the relevant passages, without needing a language-based dictionary. In contrast, the inclusion of Malaysian text, which uses the Latin alphabet, proved problematic as the OCR would inadvertently capture English text as well, leading to poorer performance.

After processing the text, we turn to visual processing, which is integral to our understanding of imagery, as supported by recent research (Liu et al., 2024). Llava’s innovation to increase the input dimensions of images and allocate more tokens for encoding visual information significantly enhanced model performance. Using OpenAI’s⁴⁴4https://openai.com/ CLIP technology, they managed to encode images up to 672x672 pixels from an original 336x336, marking a substantial improvement. Local testing on the MultiOFF dataset (Suryawanshi et al., 2020a) with the adjusted resolution, resulted in a 6% improvement in AUROC, showcasing a direct correlation between enhanced image resolution and better meme understanding.

You can find the full architecture of the proposed solution in Figure 1.

One of the most critical aspects of the challenge was to engineer a solution that is efficiently deployable within limited performance constraints, notably a maximum evaluation time of five seconds per sample on a Nvidia V100 with 16GB of RAM, as stipulated in the challenge guidelines.

To achieve efficient deployment, we leveraged the ongoing developments in the open-source llama.cpp project, which has introduced advanced quantization methods such as K-quants⁵⁵5https://github.com/ggerganov/llama.cpp/pull/1684. These methods have enabled the reduction of our 32 Billion parameter model down to a manageable size of 12.8GB. Moreover, to optimize the quantization process for our specific needs, we employed the importance matrix technique⁶⁶6https://github.com/ggerganov/llama.cpp/blob/master/examples/imatrix/README.md, critical for ensuring that the quantization process preserves the most vital aspects of the model, thereby preventing catastrophic degradation in performance.

The importance matrix was created by compiling an extensive dictionary of Singlish terms, reflecting the colloquial language frequently found in memes, additionally it also contained news sources, hateful texts, research papers and definitions of harmful memes (please refer to the code for further details). We then applied this matrix to a larger, Q4(bit) K_M quantized model on a rented 24GB GPU, using custom text tailored to the challenge’s requirements. This resulted in a downsized Q2(bit) K quantized model that was employed in our final submission. For the visual CLIP encoder, the precision was not critically affected by slight quantization, so a pre-quantized 6-bit model was used.

This setup achieved an AUROC of 0.69 on the OSPC test dataset. Despite the efficient configuration, the model’s behavior remained somewhat erratic due to the diversity of sampling methods employed: LLMs generally generate several token options based on the context of previous tokens and select among the top contenders using mechanisms such as top-p or top-k sampling.

To ensure precision, we implemented a fixed prompt designed to elicit a specific type of response from the model, beginning with:

    "<|im_start|>user\n[img-1]"+text+prob+"<|im_end|>\n
    <|im_start|>assistant\n0."

Here, text and prob encapsulate the OCR-detected text and a request to respond probabilistically, respectively. The model’s responses were constrained to single-digit outputs via a specific grammar, thereby strictly limiting it to numeric answers between 0 and 9.

This technique then allowed us to interpret the model’s output probabilities for the candidate tokens(in other words, the probabilities associated with each possible digit) as a sort of artificial last-layer, specializing the model via text-based inputs. By aggregating these probabilities: $\frac{\sum_{i=0}^{9}p_{i}\cdot i}{9\cdot\sum_{i=0}^{9}p_{i}}$ we achieved an AUROC of 0.72 on the MultiOFF dataset and 0.76 on the OSPC test set, the latter being our best submission. Further local testing with a Fully-Connected Network (FCN) using these probabilities as input demonstrated significant potential, enhancing performance up to 0.78 AUROC on the MultiOFF test set and showing especially promising results on the unseen Tamil Troll Memes dataset (Suryawanshi et al., 2020b), improving from a baseline AUROC of 0.43 to 0.64. Unfortunately, due to submission constraints, these results could not be officially submitted, therefore we leave this as future work.