Fair performance metric elicitation
G Hiranandani, H Narasimhan… - Advances in Neural …, 2020 - proceedings.neurips.cc
Advances in Neural Information Processing Systems, 2020•proceedings.neurips.cc
What is a fair performance metric? We consider the choice of fairness metrics through the
lens of metric elicitation--a principled framework for selecting performance metrics that best
reflect implicit preferences. The use of metric elicitation enables a practitioner to tune the
performance and fairness metrics to the task, context, and population at hand. Specifically,
we propose a novel strategy to elicit group-fair performance metrics for multiclass
classification problems with multiple sensitive groups that also includes selecting the trade …
lens of metric elicitation--a principled framework for selecting performance metrics that best
reflect implicit preferences. The use of metric elicitation enables a practitioner to tune the
performance and fairness metrics to the task, context, and population at hand. Specifically,
we propose a novel strategy to elicit group-fair performance metrics for multiclass
classification problems with multiple sensitive groups that also includes selecting the trade …
Abstract
What is a fair performance metric? We consider the choice of fairness metrics through the lens of metric elicitation--a principled framework for selecting performance metrics that best reflect implicit preferences. The use of metric elicitation enables a practitioner to tune the performance and fairness metrics to the task, context, and population at hand. Specifically, we propose a novel strategy to elicit group-fair performance metrics for multiclass classification problems with multiple sensitive groups that also includes selecting the trade-off between predictive performance and fairness violation. The proposed elicitation strategy requires only relative preference feedback and is robust to both finite sample and feedback noise.
proceedings.neurips.cc