The First Comprehensive Study of Stance Detection Modeling for the Sorani Kurdish Language

Abstract

Stance detection has become a fundamental task in natural language processing (NLP), yet it remains under-explored for low-resource languages such as Sorani Kurdish. Building on the previously released Bochun dataset, the present work focuses exclusively on the comprehensive evaluation of stance detection models and provides the first benchmarking of classical machine learning, deep learning (DL), and transformers. Eight models are implemented: Support vector machine (SVM), logistic regression, random forest, extreme gradient boosting, convolutional neural network, bidirectional long short-term memory, and two frozen transformer encoders, Central Kurdish Bidirectional Encoder Representations from Transformers (BERT) and XLM-RoBERTa-base (XLM-R), each combined with a logistic-regression classification head. All models are trained and evaluated under four protocols (80/20 and 70/30 stratified splits and 5-fold and 10-fold cross-validation) with five random seeds. Class-imbalance handling is investigated through a dedicated ablation comparing no weighting, model-appropriate weighting, and Synthetic Minority Oversampling Technique oversampling, and pairwise statistical comparisons are conducted using Wilcoxon signed-rank tests complemented by Cohen’s d effect sizes. With the chosen feature representations and dataset size, the SVM trained on term frequency–inverse document frequency features delivers the strongest performance (accuracy and weighted F1 of 74% on the 80/20 split and 72% under 10-fold cross-validation), outperforming the DL and frozen-transformer baselines. The findings provide a foundational reference point for future research on Kurdish NLP in general and stance detection specifically.