I learned nothing that changes my mind, let’s go out!Scenario 2 — You show me data: “Scientists just discovered a new kind of sea slug.
” What should I do?.I learned nothing that changes my mind, I’m staying in.
Notice that the sea slug factoid leads to a failure to reject the null, not an acceptance of the null.
It might still be raining, I just don’t know that from the data, so I keep doing what I was going to do anyway.
Analyze the evidence and decide!.(Surprising data)Scenario 1 — You show me data: “It’s raining.
” What should I do?.Not my default action.
I’ll stay in.
Scenario 2 — You show me data: “Spencer Krug is on stage in an hour.
” What should I do?.Not my default action.
I’ll get off the couch for Spencer any day.
These are easy facts to interpret — they make my null hypothesis look ridiculous (in fact, the p-value is 0) so they force me to take the alternative action in each case.
What if we’d gotten the same data but we flipped which scenario (world) it shows up in?Analyze the evidence and decide!.(Boring data)Scenario 1 — You show me data: “Spencer Krug is on stage in an hour.
” What should I do?.I learned nothing that changes my mind, let’s go out!Scenario 2 —You show me data: “It’s raining.
” What should I do?.I learned nothing that changes my mind, I’m staying in.
If you’re not careful, you might make the mistake of thinking that the evidence has anything to do with the decision.
Actually, this evidence doesn’t change my mind any more than the sea slug factoid would… even though I’d rather go to a good show.
I would have made the same decision in Scenario 1, regardless of the music.
Sure, seeing an awesome show makes me feel better about going out, but I would have done that anyway.
If you’ve absorbed this, you’re ready to add some nuance — dive into “Statistical inference in one sentence” for a deeper example.
.. More details