### Will Fithian - Adaptive Sequential Model Selection

179
views
0
12 months ago by
Community: WHOA-PSI 2017

0
12 months ago by
Can you give a reference for the counterexample?
It is in the paper at: https://arxiv.org/abs/1512.02565 . Look at "Proposition 1" on page 11.
written 12 months ago by Will Fithian
0
12 months ago by
The paper related to the talk: https://arxiv.org/abs/1512.02565
0
12 months ago by
Can you summarize your interpretation of the final 2 slides (after `Thank you!') which compare selected and saturated models. And what is your view about the appropriateness of the saturated model as a default choice?
The final two slides show a stylized example illustrating when, and why, the (selected model) max-t test is much more powerful than the saturated model test. You can see a full description of the example in Section 5.3 of the paper; I don't think I can improve on that description here.

Saturated model inference is often a good default choice when we want to construct intervals for coefficients after applying the LASSO. It is more robust to some forms of model misspecification because it still controls a meaningful definition of the Type I error rate even when the linear model in $E$ is possibly missing important signal variables.

In goodness-of-fit testing, however, we are testing whether the linear model given by $E_k$ is misspecified. For small $k$, we fully expect that the model may be still missing important signal variables; and when it is, then we want to reject. So we don't want to control "Type I error" in any sense; we want to reject and with a very small p-value.
written 12 months ago by Will Fithian