After performing a study, you can correctly conclude there is an effect or not, but you can also incorrectly conclude there is an effect (a false positive, alpha, or Type 1 error) or incorrectly conclude there is no effect (a false negative, beta, or Type 2 error). The goal of collecting data is to provide evidence for or against a hypothesis.

I’m happy to announce my first R package ‘TOSTER’ for equivalence tests (but don’t worry, there is an old-fashioned spreadsheet as well). In an earlier blog post I talked about equivalence tests. Sometimes you perform a study where you might expect the effect is zero or very small.

One widely recommended approach to increase power is using a within subject design. Indeed, you need fewer participants to detect a mean difference between two conditions in a within-subjects design (in a dependent t -test) than in a between-subjects design (in an independent t -test). The reason is straightforward, but not always explained, and even less often expressed in the easy equation below.

I’m really excited to be able to announce my “Improving Your Statistical Inferences” Coursera course. It’s a free massive open online course (MOOC) consisting of 22 videos, 10 assignments, 7 weekly exams, and a final exam. All course materials are freely available, and you can start whenever you want. In this course, I try to teach all the stuff I wish I had learned when I was a student.

I think it was somewhere in the end of 2012 when my co-authors and I received an e-mail from Greg Francis pointing out that a study we published on the relationship between physical weight and importance was ‘too good to be true’. This was a stressful event. We were extremely uncertain about what this meant, but we realized it couldn’t be good. For me, it was the first article I had ever published. What did we do wrong?

You might have seen the ‘Dance of the p -values’ video by Geoff Cumming (if not, watch it here). Because p -values and the default Bayes factors (Rouder, Speckman, Sun, Morey, & Iverson, 2009) are both calculated directly from t -values and sample sizes, we might expect there is also a Dance of the Bayes factors. And indeed, there is. Bayes factors can vary widely over identical studies, just due to random variation.

The Times Higher Education reports on two new initiatives in the Netherlands to bolster scientific standards. Here, I want to talk about one of these initiatives I was involved in: A fund for replication research. The board of the Dutch science funder NWO still has to officially approve the final call for proposals, but the draft text is basically done.

See the follow up post where I introduce my R package and spreadsheet TOSTER to perform TOST equivalence tests, and link to a practical primer on this topic. When you find p > 0.05, you did not observe surprising data, assuming there is no true effect.

After my previous post about one-sided tests, some people wondered about two-sided F -tests.

TL;DR: Don’t like one-sided tests? Distribute your alpha level unequally (i.e., 0.04 vs 0.01) across two tails to still benefit from an increase in power. My two unequal tails in a 0.04/0.01 ratio (picture by my wife). This is a follow-up to my previous post, where I explained how you can easily become 20% more efficient when you aim for 80% power, by using a one-sided test.

I've created an easy to use R script that will import your data, and performs and writes up a state-of-the-art dependent or independent t-test.