Decision-centric fairness: Evaluation and optimization for resource allocation problems
_{Simon De Vos, Jente Van Belle, Andres Algaba, Wouter Verbeke, Sam Verboven}

This paper proposes a decision-centric fairness approach for binary classification models used in resource allocation. It argues that fairness constraints should be applied only within the decision-making region, where predicted scores translate into real-world actions, rather than across the entire score distribution, thereby achieving fairer outcomes with minimal loss in predictive utility.

The main contributions of this paper are as follows:

1. We introduce and formalize the concept of decision-centric fairness for resource allocation optimization.
2. We propose a decision-centric fairness approach to optimize classification models used in resource allocation.
3. We introduce a decision-centric predictive performance metric for classification models.
4. We empirically compare our proposed decision-centric fairness methodology to a global fairness approach on multiple (semi-synthetic) datasets, identifying scenarios where — from a decision-centric evaluation perspective — focusing on fairness only where it truly matters outperforms imposing fairness everywhere.

A preprint is available on ArXiv

Methodology

Optimization of decision-centric fairness: The evolution of density distributions for different regularization strengths. Solid lines represent full prediction densities, while dotted lines represent top-n selected predictions. The decision threshold $\tau$ is indicated as a vertical line.

λ = 0.0

λ = 0.3

λ = 0.6

The evolution of density distributions for different values of λ.

Installing

We have provided a requirements.txt file:

pip install -r requirements.txt

Please use the above in a newly created virtual environment to avoid clashing dependencies.

Instructions

• Specify project configurations in projectconfig.json
• Specify experiment configurations in experimentconfig.json
• In main.ipynb:
  • Set the project directory to your custom folder. E.g., DIR = r'C:\Users\...\...\...'
  • Run main.ipynb.
  • Results can be logged in the specified WandB project (more info on WandB's website). Configurations can be specified in \DCF\config\projectconfig.json
• Run results.ipynb to plot pareto curves similar to those in the paper. The results are imported from your configured WandB project

Repository Structure

This repository is organized as follows:

|- config/
    |- experimentconfig.json    
    |- projectconfig.json    
|- data/
    |- adult/             
    |- TelecomKaggle/       
|- figures/
    |- 01_intro/             
    |- 03_methodology/       
    |- 04_experiment/             
    |- 05_results/
|- notebooks/ 
    |- main.ipynb
    |- results.ipynb            
|- src/
    |- data.py
    |- loss.py
    |- metrics.py
    |- model.py
    |- training.py
    |- utils.py
|- requirements.txt