Debiasing Neural Retrieval via In-batch Balancing Regularization

People frequently interact with information retrieval (IR) systems, however, IR models exhibit biases and discrimination towards various demographics. The in-processing fair ranking methods provides a trade-offs between accuracy and fairness through adding a fairness-related regularization term in the loss function. However, there haven ' t been intuitive objective functions that depend on the click probability and user engagement to directly optimize towards this.In this work, we propose the textbf I n- textbf B atch textbf B alancing textbf R egularization (IBBR) to mitigate the ranking disparity among subgroups. In particular, we develop a differentiable textbf normed Pairwise Ranking Fairness (nPRF) and leverage the T-statistics on top of nPRF over subgroups as a regularization to improve fairness. Empirical results with the BERT-based neural rankers on the MS MARCO Passage Retrieval dataset with the human-annotated non-gendered queries benchmark cite rekabsaz2020neural show that our ibbr method with nPRF achieves significantly less bias with minimal degradation in ranking performance compared with the baseline.