My highlights from ACM FAT* 2020

I’m back from FAT* 2020 (which will from now on be known as FAccT, and held next year in Toronto). This is a good time to round up a few of the presentations I found interesting. Partial summaries and random thoughts below (in no particular order), as well as a shameless plug for the paper I presented.

1. Awareness in Practice: Tensions in Access to Sensitive Attribute Data for Antidiscrimination

A. Rieke; M. Bogen; S. Ahmed

One of the cornerstones of fair machine learning is the understanding that we need to know individuals’ “protected attributes” (membership in disadvantaged groups) in order to both evaluate existing unfairness and to intervene to correct it. While this is something we know to be “theoretically optimal”, there are significant challenges in actually operationalizing it in practice, because this means that all this data must be collected and stored. In this work the authors conducted a comprehensive examination of the state of affairs and common practices regarding the collection of sensitive attributes in three different domains: credit, employment and healthcare, which I find very valuable.

2. What’s Sex Got to Do With Fair Machine Learning?

L. Hu, I. Kohler-Hausmann

The Berkeley admission data (1973) is the textbook example for what statisticians call “Simpson’s paradox”. In his book Pearl resolves this paradox: he argues that to understand what’s really going on in the data - is there discrimination or not - one needs to draw a causal graph (DAG). In her talk, Issa delivered a precise and clear rebuke of the claim that this comes near to “solving” this issue. The argument is that there are several ontological assumptions implicit in this DAG. In the short talk she focused on the third, which is that the relationship between the different causal pathways are actually modular. Modularity means that each cause-effect pair (in this case, Sex → Admission and Sex → Department) represents distinct causal mechanisms that exist in the world seperately and can be manipulated seperately. Without this assumption, one cannot proceed to reason about the direct and indirect effects. I can’t fully recreate her argument (and the paper isn’t online yet), but I found she made the point that any social category (race, gender) will necessarily violate this assumption extremely clearly.

3. The Disparate Equilibria of Algorithmic Decision Making when Individuals Invest Rationally

L. Liu; A. Wilson; N. Haghtalab; A. Kalai; C. Borgs; J. Chayes

So much of the fairness literature has focused on understanding the “one-shot” setup. This has produced a lot of useful insights, but is obviously very far from modeling the real world. Several recent papers are making progress in this direction and attempt to understand long-term dynamics. In this paper the authors consider a stylized model with the following dynamics: (i) a group responds to a classifier by investing towards a positive outcome (ii) the decision rule is updated, and analyze the resulting equilibrium behaviour. When the groups are identical, all is well: there is a single equilibrium and this equilibrium is “optimal”. The main finding is that things are significantly more difficult when there is heterogeneity between the groups and when the decision-maker can’t reach a near-zero error classifier: in this case there are two equilibria in a way that might lead to unfairness (a vague statement as I didn’t catch all the details).

4. Value-laden Disciplinary Shifts in Machine Learning

R. Dotan; S. Milli

This paper looks at recent trends in machine learning from a philosophy of science perspective. The main argument seems to be a rebuttal against the common perspective that the transition to deep learning (which is usually “placed” at 2012, with the success of AlexNet in the ImageNet challenge) is due to objective progress. Rather than an improvement in some global, objective notion of accuracy, they claim this transition is partly because there was a shift in the notion of accuracy, to one that puts a greater focus on evaluation in compute-rich and data-rich environments. In this sense, they argue, a transition from one model type to another also encodes certain values, such as a lesser concern for environmental issues.

5. The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons

S. Barocas; A. Selbst; M. Raghavan

This paper discusses the hidden assumptions and built-in normative tensions that underlie “feature highlighting explanations”. The latter is a term the authors use to (roughly) describe explanations that are supposed to educate the decision subject; they do so by pointing to specific features that are “important” for the individual’s prediction. This is a generalization of concepts common in machine learning (counterfactual explanations, Wachter et al) as well as law (“principal reasons” approach from US credit laws) and one that has been getting significant attention in the “explainability” community lately, for reasons that the authors discuss, but mainly as a “model agnostic” approach for providing individuals with more autonomy in the face of algorithmic predictions. The main argument is this paper is that there are a couple of (so far undiscussed?) assumptions that underlie the promised utility of this approach. Most of these are quite intuitive. For example, one assumption is that people can actually “implement” proposed changes; that is, that there is always a clear mapping between actions in the real world to required changes in feature space (and that people know this mapping). Another assumption can be thought of as some notion of independence between tasks: that is, that implementing a proposed change (that will improve my probability of receiving a loan, say) will not have adverse effects on other things that are important to that individual. There is also an interesting discussion of some inherent tensions that this type of explanations give rise to. In particular, these explanations are “selective” by design - they are intended to provide some autonomy without revealing the entire model, which is thought of as proprietary. Since we expect there might be many reasonable explanations that are all reasonable, this immediately gives rise to some form of paternalism, which may be undesirable by itself but especially when it has the potential to be abused.

6. Roles for Computing in Social Change

R. Abebe; S. Barocas; J. Kleinberg; K. Levy; M. Raghavan; D. Robinson

As a computer scientist working on social aspects (fairness etc), I am very much aware of the critical view of computer science as focused on “solutionism”. The criticism is roughly that by focusing on the problematic features of the status quo (unfairness) we necessarily treat the status quo as fixed. This shifts the focus away from addressing the deeper, fundamental issues of injustice and inequality and towards technical solutions. While I completely agree with this concern, I often feel that this criticism is too one-sided, and does not sufficiently recognize the benefits that the CS perspective has to offer. I have some thoughts of my own, but I was really looking forward to hearing the distilled thoughts of much smarter, more informed and more experienced researchers. This paper breaks down the positive roles that computing research can serve into four aspects: (1) “diagnostic”: aid understanding and measuring of social problems (e.g., the now famous bias in word embeddings paper, auditing of face recognition tools, etc); (2) “formalizer”: shaping how social problems are defined; (3) “rebuttal”: sketching out the boundaries of what is possible and what isn’t; (4) “synecdoche”: shedding a new light on long-standing social problems, in a way that might rejuvenate the public interest (Karen gave as an example Virginia Eubanks’ book “automating inequality”, that does precisely this to poverty). I liked that in her talk Karen emphasized that each of these four positive roles should be taken with a grain of salt. E.g., for (1), we should be careful that the diagnostic measures don’t end up becoming the target (e.g., we would remove the observed bias in word embeddings and think our job is done) and for (3) we should be careful - the fact that something is technically possible doesn’t at all imply that it should be done. Looking forward to reading the paper!

7. Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination

N. Kallus; X. Mao; A. Zhou

This is a great followup to the “roles for computing” paper because it delivers precisely on the third role, which is to sketch the boundaries of what can and can’t be achieved. The starting point for this paper is the fact that in many cases where we’re interested in assessing algorithmic fairness, this might be a difficult task since membership in protected groups is not directly observed in the data. This hasn’t been an impediment in practice: usually what practitioners do is to infer membership in protected groups from “proxies” (sometimes with the help of additional data about proxies and true group membership) and then use these estimates to assess cross-groups disparity measures. For example, BSIG is a popular heuristic that estimate the probability of race from geolocation and last name. But is this a good approach? Can we “trust” the disparity estimates that it produces? This is the question this paper is trying to answer. The formal model is that there are two datasets: a “main” dataset in which we observe outcomes (Y), features (Z), but not group membership (A), used to learn predictions (Y_hat), and an “auxiliary” dataset in which we observe A and Z (but not the outcomes). For simplicity (and to abstract issues coming from sampling), the paper assumes we know both of these distributions exactly. The quantities we are interested in, however, are a function of the joint distribution over all these variables, which is in general not identifiable from only these two distributions. This means that using a proxy method such as the one described above is essentially a point estimate of the unknown disparity, which highlights that what we want is actually set estimates: a collection of all the values of the disparity that are compatible with the observed datasets. At first glance this might not seem very useful: if we can’t “pin down” the value, why would this possibly very large set help? This is precisely related to the third role of computing research discussed above: The “gist” of the paper is that these sets can be computed, in a way that allows for interpreting their size as a quantitative measure of the amount of uncertainty present that is a result of not observing the protected attribute directly. This is a useful “red flag”: it lets us know when we should be careful about making conclusions. Since estimating the disparities is often the first step before future interventions, it’s obviously important to get this stage right!

8. Recommendations and User Agency: The Reachability of Collaboratively-Filtered Information

S. Dean; S. Rich; B. Recht

As an avid Netflix user I am fully aware of the significant impact different recommendation engines have on our lives. Are these systems “fair”? This paper takes a perspective to this question which I really like: user recourse, or what the authors call “reachability”. If I understood correctly, this is loosely defined as the fraction of all possible outcomes that could potentially be recommended to the user if they change some constant fraction of their “choices” (e.g. you re-evaluate the scores you gave to some k movies). I found this interesting because it seems to suggest a way to audit these systems, and the answer seems to say a lot about the extent a user actually has agency over the system’s behaviour: e.g., is Netflix incredibly good at predicting my preferences, or has it simply placed me in some “bucket” with other (supposedly) like-minded users but in a way that really confines me to that bucket from now on?

9. Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy

K. Yang, K. Qinami, L. Fei-Fei, J. Deng, O. Russakovsky

This presentation was quite a detailed description of the efforts conducted by the authors to improve upon the problematic ImageNet subtree of “People”. This is a great effort (though long overdue…) and I encourage anyone interested to check out the paper for the full details. I left this talk with the feeling that there are so many things that are inherently wrong with dataset and the way by which it was collected, that perhaps the best thing we should be doing at this point in time is not to try to “patch up” the problems in ImageNet, but build an entirely new dataset from scratch, with a thoughtful, transparent, community-led effort. Obviously this is a massive undertaking, but this seems important: while ImageNet served its’ initial purpose very well (provide a challenging benchmark that accurately ranks different models and inspires progress in computer vision), it doesn’t seem at all appropriate for its’ modern purpose: the “DNA” of every single image recognition model out there.

10. Preference-Informed Fairness

M. Kim; A. Korolova; G. Rothblum; G. Yona

I’ll conclude with a shameless plug for the paper I presented. We consider a really well-studied setting: we have a bunch of individual, a bunch of outcomes, and we want to map individuals to outcomes in a way that provides strong fairness guarantees. Individual fairness (Dwork et al, 2012) is a natural candidate, but the notion of “treat similar people similarly” doesn’t make sense when individuals differ in their preferences over the outcome space. Envy-freeness is another classical notion and a natural candidate once we start talking about individuals’ preferences, but it doesn’t take into account the metric (qualifications), so it too can be overly restrictive. For example, when everyone has the same preferences (e.g. prefer getting a loan to not getting it), envy-freeness means everyone should get the loan with the same rate, which is obviously not sensical and doesn’t align with any intuitive notion of fairness in this case. We suggest a new fairness definition, “preference informed fairness”. Without going into the details of the definition (check out the paper!), I argued that in many cases incorporating individuals’ preferences into our fairness notions is desirable: for example, it might not make sense to ask for “parity” in ad delivery (e.g. between men and women) if this involves some people being worse off: for example if they are actually not interested in seeing this ad. While this argument doesn’t go through in every domain, I think this highlights that before incorporating fairness constraints in real-world, complex systems, we should really verify that they are “working as intended”.

Other presentations I found super-interesting, but don’t have enough time to write about: Fairness Is Not Static: Deeper Understanding of Long Term Fairness via Agents and Environments, Studying Up: Reorienting the study of algorithmic fairness around issues of power, POTs: Protective Optimization Technologies; see the full list of accepted papers.