Skip to content

Commit 043f9e2

Browse files
committed
feat: add notebooks
1 parent 157c213 commit 043f9e2

23 files changed

+72613
-0
lines changed

tools/AIF360/AIF360_tutorial_bias_advertising.ipynb

Lines changed: 1807 additions & 0 deletions
Large diffs are not rendered by default.

tools/AIF360/README.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# AIF360
2+
3+
## Tool description
4+
5+
AIF360 focuses exclusively on bias detection and mitigation.
6+
7+
It was developed by IBM and it was donated to the Linux Foundation AI & Data Foundation.
8+
9+
AIF360 has three main components. Firstly, it has estimators, which are Machine Learning models or libraries that are used to mitigate bias in some ways, they can apply, before training, in training or after training.
10+
11+
It contains metrics as well that can help us assess the model predictions, these are divided into two sub groups. Group metrics, that applies to subpopulations within the dataset or individual metrics which assess that similar individuals are treated similarly.
12+
13+
Lastly it has detectors, which identify subgroups that are favoured or discriminated against by our model.
14+
15+
Our experience with AIF360 was very positive, the documentation was broad and clear, it was easy to use and it is relatively lightweight. It is true that its purpose is very specific but also it is worth to say that there are not many tools that assess bias and discrimination and it is nice to have one as mature and as backed as this one.
16+
17+
## Notebooks
18+
19+
### AIF360_tutorial_bias_advertising.ipynb
20+
21+
In this example, we adapted one of the example notebooks that AIF360 had in its repo: [tutorial_bias_advertising.ipynb](https://github.com/Trusted-AI/AIF360/blob/main/examples/tutorial_bias_advertising.ipynb).
22+
23+
This tutorial shows how bias in advertising data can be discovered, measured and mitigated using AIF360.
24+
25+
## Tooling
26+
27+
- **Algorithms**
28+
- **Preprocessing:** Techniques applied before model training to mitigate bias in data.
29+
- **DisparateImpactRemover:** Edits feature values to enhance group fairness while preserving rank-ordering within groups.
30+
- **Learning Fair Representations (LFR):** Finds a latent representation that encodes data well but obfuscates information about protected attributes.
31+
- **Optimized Preprocessing:** Learns a probabilistic transformation that edits features and labels with constraints on group fairness, individual distortion, and data fidelity.
32+
- **Reweighing:** Assigns weights to examples in each (group, label) combination to ensure fairness before classification.
33+
- **Inprocessing:** Methods that incorporate fairness constraints into the model training process.
34+
- **Adversarial Debiasing:** Learns a classifier to maximize prediction accuracy while reducing an adversary's ability to determine the protected attribute from predictions.
35+
- **ARTClassifier:** Wraps an instance of an `art.classifiers.Classifier` to extend Transformer.
36+
- **GerryFairClassifier:** An algorithm for learning classifiers that are fair with respect to rich subgroups.
37+
- **MetaFairClassifier:** A meta-algorithm that takes the fairness metric as input and returns a classifier optimized with respect to it.
38+
- **Prejudice Remover:** Adds a discrimination-aware regularization term to the learning objective to mitigate bias.
39+
- **Exponentiated Gradient Reduction:** An approach for fair classification that reduces the problem to a sequence of cost-sensitive classification problems.
40+
- **Grid Search Reduction:** Performs a grid search over classifier parameters to find models that balance fairness and accuracy.
41+
- **Postprocessing:** Techniques applied after model training to adjust predictions for fairness.
42+
- **Calibrated Equalized Odds Postprocessing:** Optimizes over calibrated classifier score outputs to find probabilities with which to change output labels with an equalized odds objective.
43+
- **Equalized Odds Postprocessing:** Solves a linear program to find probabilities with which to change output labels to optimize equalized odds.
44+
- **Reject Option Classification:** Gives favorable outcomes to unprivileged groups and unfavorable outcomes to privileged groups in a confidence band around the decision boundary with the highest uncertainty.
45+
- **Deterministic Reranking:** A collection of algorithms for constructing fair ranked candidate lists.
46+
47+
- **Metrics**
48+
- **Dataset Metrics:** Assess fairness at the dataset level.
49+
- **AI Fairness 360**
50+
- **DatasetMetric:** Computes metrics based on a single `StructuredDataset`.
51+
- **BinaryLabelDatasetMetric:** Computes metrics based on a single `BinaryLabelDataset`.
52+
- **Classification Metrics:** Evaluate fairness in classification tasks.
53+
- **AI Fairness 360**
54+
- **ClassificationMetric:** Computes metrics based on two `BinaryLabelDatasets`, typically the original and predicted datasets.
55+
- **MDSSClassificationMetric:** Utilizes bias subset scanning to identify bias in predictive models using subset scanning techniques.
56+
- **Sample Distortion Metrics:** Measure distortions introduced during preprocessing.
57+
- **SampleDistortionMetric:** Computes metrics based on two `StructuredDatasets` to assess the impact of preprocessing transformations.
58+
- **Utility Functions:** Provide helper functions for metric computations.
59+
- **compute_boolean_conditioning_vector:** Computes a boolean conditioning vector based on specified conditions.
60+
- **compute_num_instances:** Calculates the number of instances conditioned on protected attributes.
61+
- **compute_num_pos_neg:** Determines the number of positive or negative instances, optionally conditioned on protected attributes.
62+
- **compute_num_TF_PN:** Computes the number of true/false positives/negatives, optionally conditioned on protected attributes.
63+
- **compute_num_gen_TF_PN:** Calculates the number of generalized true/false positives/negatives with optional conditioning on protected attributes.
64+
- **compute_distance:** Computes element-wise distances between two sets of vectors.

0 commit comments

Comments
 (0)