Narrlangen @ SemEval2025 Task 10 (Subtask 2)

Comparing (mostly) simple multilingual approaches to narrative classification

This repository accompanies our contribution to SemEval 2025 Task 10, SubTask 2: Multlingual characterisation and extraction of narratives from online news.

Our task solvers comprise

• simple (bag-of-words) machine learning baselines,
• prompt engineering of LLMs (only for English) (PromptEng),
• a zero-shot approach based on sentence similarity,
• direct classification of fine-grained labels using SetFit,
• fine-tuning encoder models on fine-grained labels (FGM),
• and hierarchical classification, and narrative_attention_model using encoder models with two different classification heads.

The manually crafted narrative descriptions – about climate change and the War in Ukraine – are solely used in the sentence similarity approach.

For more details, refer to our upcoming publication:

@inproceedings{BlombachETC-SemEval2025,
   title = {Narrlangen at SemEval-2025 Task 10: Comparing (mostly) simple multilingual approaches to narrative classification},
   author = {Blombach, Andreas and Doan Dang, Bao Minh and Evert, Stephanie and Fuchs, Tamara and Heinrich, Philipp and Kalashnikova, Olena and Unjum, Naveed},
   booktitle = {Proceedings of the 19th International Workshop on Semantic Evaluation},
   series = {SemEval 2025},
   year = {2025},
   address = {Vienna, Austria},
   month = {July},
   pages = {}, 
   doi= {} 
}