Abstract:

Informally, a “spurious correlation” is the dependence of a model on some aspect of the input data that an analyst thinks shouldn't matter. In machine learning, these have a know-it-when-you-see-it character; e.g., changing the gender of a sentence’s subject changes a sentiment predictor’s output. To check for spurious correlations, we can “stress test” models by perturbing irrelevant parts of input data and seeing if model predictions change. In this paper, we study stress testing using the tools of causal inference. We introduce counterfactual invariance as a formalization of the requirement that changing irrelevant parts of the input shouldn't change model predictions. We connect counterfactual invariance to out-of-domain model performance, and provide practical schemes for learning (approximately) counterfactual invariant predictors (without access to counterfactual examples). It turns out that both the means and implications of counterfactual invariance depend fundamentally on the true underlying causal structure of the data — in particular, whether the label causes the features or the features cause the label. Distinct causal structures require distinct regularization schemes to induce counterfactual invariance. Similarly, counterfactual invariance implies different domain shift guarantees depending on the underlying causal structure. This theory is supported by empirical results on text classification.

ZOOM INVITATION

Topic: Colloquia -- Department of Statistics and Actuarial Science, The University of Iowa
Time: Feb 3, 2022 03:15 PM Central Time (US and Canada)

Join Zoom Meeting
https://uiowa.zoom.us/j/98928693758

Meeting ID: 989 2869 3758
One tap mobile
+13126266799,,98928693758# US (Chicago)
+16468769923,,98928693758# US (New York)

Dial by your location
        +1 312 626 6799 US (Chicago)
        +1 646 876 9923 US (New York)
        +1 301 715 8592 US (Washington DC)
        +1 346 248 7799 US (Houston)
        +1 669 900 6833 US (San Jose)
        +1 253 215 8782 US (Tacoma)
Meeting ID: 989 2869 3758
Find your local number: https://uiowa.zoom.us/u/adodl1V2PF

Join by SIP
98928693758@zoomcrc.com

Join by H.323
162.255.37.11 (US West)
162.255.36.11 (US East)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia Sydney)
103.122.167.55 (Australia Melbourne)
64.211.144.160 (Brazil)
69.174.57.160 (Canada Toronto)
65.39.152.160 (Canada Vancouver)
207.226.132.110 (Japan Tokyo)
149.137.24.110 (Japan Osaka)
Meeting ID: 989 2869 3758