Through the Eyes of the Beholder: Biometric and Demographic Conditioning for Multimodal Sexism Detection

Overview

Motivation & Contributions

Our Approach

The system is a five-stage pipeline. Each meme is enriched with clean text and a structured visual description, expanded through cross-lingual translation, and fed to a deep multimodal network whose physiological branch is warm-started by a pretrained autoencoder and whose embedding space is shaped by an auxiliary contrastive objective. At inference, the network is soft-voted with a feature-based SVM.

Human Conditioning via FiLM

Demographic embeddings are averaged across a meme's annotators and concatenated with the 32-D physiological embedding to form a 56-D human-context vector. Two linear layers map it to per-feature scale and shift parameters that modulate the fused representation as h ⊙ (1+γ) + β. Both projections are zero-initialized, so the model begins as an unconditioned classifier and progressively learns how much to let annotator context reshape its predictions.

Data Analysis

The EXIST 2026 dataset comprises 3,984 memes viewed by 16 subjects, recording eye movements, heart rate, and EEG for each annotator. The text is almost perfectly balanced between English (n = 2,005) and Spanish (n = 1,979), so the network cannot falsely tie one language to the presence of sexism. Boxplot analysis shows that sexist content triggers significantly higher reaction times, fixation counts, and heart-rate variability, which points to evidence that sexist memes generate greater cognitive friction.

Demographic & Physiological Significance

Table 1. Highlighted rows are the four physiological features retained for the neural branch (significant at α = 0.05). EEG bandpower was not individually significant and is exploited only by the classical SVM component. Eye-tracking metrics show extreme multicollinearity (Fixations and Saccades with ρ ≈ 1.0), so the 4-D sensor vector likely carries less independent information than its dimension suggests.

Text & Image Statistics

Table 2. Descriptive statistics over N = 3,984 records. Text is short and right-skewed (75% of memes hold 26 words or fewer). The aspect-ratio distribution peaks at a 1:1 square, characteristic of meme formats, with brightness centered in the mid-tones.

Results

Official EXIST 2026 results for the best submitted run per language split. The system is most competitive on Subtask 2.2 (source intention) and clears the organizer majority-class baseline by a wide margin on Subtasks 2.1 and 2.2 (e.g. 0.4861 vs. 0.2947 hard ICM-Norm on 2.1, and 0.3612 vs. 0.1369 on 2.2). Subtask 2.3 is the hard case: the soft score (0.1568) beats the majority baseline of 0, but the hard score collapses onto it.

Table 3. Official EXIST 2026 results, best submitted run per split.

Our Findings

Ablation Studies

Augmentation & Ensemble

Four configurations on Subtask 2.1 isolate the contributions of cross-lingual augmentation and the SVM ensemble. The pure deep model on the non-augmented corpus gives the best All-split scores on both validation and test.

Table 4. Subtask 2.1 hard evaluation. Aug. marks cross-lingual translation augmentation during training; Ens. marks the SVM soft-vote at inference. Augmentation lifts Spanish but degrades English; the ensemble helps in the noisier augmented regime but adds variance otherwise. The augmented-without-ensemble run was not submitted.

Architectural Components (5 seeds, Holm-Bonferroni)

Each component was disabled independently and retrained over five seeds. We applied paired Welch t-tests against the baseline across a family of 63 comparisons and corrected with the Holm-Bonferroni step-down procedure at α = 0.05. No ablation survives the correction; the closest case (removing the visual description on Subtask 2.3 soft, uncorrected p = 0.001) fails the Holm threshold (≈ 0.00079).

Table 5. Validation hard-evaluation ICM-Norm, mean ± std. over five seeds, non-augmented deep model only. The no-image variant shows dramatically higher variance, suggesting the image stream provides training stability even when its mean contribution is small. The 2.3 head collapses to a near-identical value across multi-task configurations and is broken only by single-task training on 2.3.

Configuration

Limitations

Contact & Acknowledgements

Citation (Placeholder until the official proceedings entry is available)

@InProceedings{Mocanu_2026_EXIST_CLEF,
    author    = {Mocanu, Ana-Maria Luisa and Mocanu, Sebastian and Truică, Ciprian-Octavian and Apostol, Elena-Simona},
    title     = {Through the Eyes of the Beholder: Biometric and Demographic Conditioning for Multimodal Sexism Detection},
    booktitle = {Conference and Labs of the Evaluation Forum (CLEF), EXIST 2026 Lab, Task 2},
    month     = {September},
    year      = {2026}
}