I’ve never taken graduate statistics, but I’m generally not impressed with the state of statistics or statistics education. Just because almost every university teaches almost every student this (and most published papers work this way) doesn’t mean it’s good. Intuitive, sure. Wise, not so much.

> Let's look at two questions: (1) Does X have an effect? (2) How large is that effect? If your prior puts a non-zero probability that the answer to the first question is "No", then priors for the second question will have non-zero at point 0, even though the probability of any other point may be zero.

Even ignoring whether it makes sense to have a nonzero prior for X having no effect, it’s generally not useful to learn the answer. An arbitrarily small effect is, for practical purposes, indistinguishable from no effect — an experiment that is intended to be useful should say something about the size of an effect. If a pill helps depression enough to be worth taking the pill, that’s one thing. If it helps in the sense that you could dose literally everyone in the world and one person would feel very slightly better for a day, that’s not helpful. Similarly, the existence of an effect says nothing about the sign of the effect.

So I think hypotheses being tested should be useful. If you want to determine whether something is useful, at least set a threshold for usefulness and test that. Or come up with a quantitative measure. The Bayesian-vs-frequentist debate is IMO somewhat secondary to this except insofar as it seems less common to make worthless but mathematically correct Bayesian tests because thinking about priors at all requires some acknowledgment of whether a prior is remotely plausible.

Also, how exactly can you make a well defined experiment that can confirm the null hypothesis without putting something resembling a prior on the non-null hypothesis if the non-null hypothesis contains distributions that are arbitrarily close to null? I’m sure it’s doable, but it seems likely to be pretty messy if you dig in.

(There are exceptions. For example, the existence of a neutrino mass is very interesting irrespective of what that mass is. But even then, physics results like this generally put bounds on a value that is hypothesized to be zero instead of merely testing for zeroness, because every experiment has finite power to detect small effects, and the readers of an outcome of an experiment should care more about the detection limits than, say, the number of dollars the experiment cost.)