“Technically” here could imply “not corrected for multiple tests”. But the typical qualifier I use is “nominally significant” when I don’t apply an correction for multiple tests.
Or “not using a one-way t test”.
The most appropriate null hypothesis in this lovely study is “does theanine REDUCE anxiety”, not “does theanine change anxiety either up or down”.
What impressed me most is the suggestion for an improved experimental design to remove his temporal drift by using 100 pre-loaded envelopes and only decoding the results at the end.
> The most appropriate null hypothesis in this lovely study is “does theanine REDUCE anxiety”, not “does theanine change anxiety either up or down”.
I disagree with this. You have a prior belief that theanine might reduce anxiety; if you wanted to you could codify that subjective belief and perform some variety of Bayesian hypothesis test [1] and compute a Bayes factor. The main reason that one-sided tests are advocated for is power; that is often the same as having a prior belief in disguise. Why not quantify it?
However, scientifically, if the data conclusively show that "theanine increases anxiety" that is a meaningful, non-artefactual result: it is hugely important to be sensitive to the answer 'you are wrong' and may well ironically spur development in a direction to help understand what is going on. I personally think that one sided tests are best avoided except in the case where it is physically impossible to have an effect in the other direction. Examples of this are rare, but they do occasionally exist.
Sure. I can see your point, but the most reasonable posterior probability of the null given the biohacker community’s belief is one-tailed. This also gives more power to reject the null.
Or “not using a one-way t test”.
The most appropriate null hypothesis in this lovely study is “does theanine REDUCE anxiety”, not “does theanine change anxiety either up or down”.
What impressed me most is the suggestion for an improved experimental design to remove his temporal drift by using 100 pre-loaded envelopes and only decoding the results at the end.