Psychology journals should require, as a condition for publication, that data supporting the results in the paper are accessible in an appropriate public archive. I hope that in the near future, the ‘should’ in the previous sentence will disappear, and that data sharing has become a requirement. Many journals already have requirements to share data, but often not in a public database.

I like p-curve analyses. They are a great tool to evaluate whether sets of studies have evidential value or not. In a recent paper, Simonsohn, Nelson, & Simmons (2014) show how p-curve analyses can correct for publication bias using only significant results. That’s pretty cool. However, trim-and-fill is notoriously bad at unbiased effect size estimation.

I think it is important people report confidence intervals to provide an indication of the uncertainty in the point estimates they report. However, I am not too enthusiastic about the current practice to report 95% confidence intervals. I think there are good reasons to consider alternatives, such as reporting 99.9% confidence intervals instead.

Clintin Davis-Stober recently pointed me to his blog about the v -statistic. The v -statistic (nothing Greek, just the letter v) is a measure of how accurately your data is estimating the corresponding parameter values in the population.

Excellent news this morning: The negotiations between Dutch universities and Elsevier about new contracts for access to the scientific literature have broken down (report in Dutch by VSNU and De Volkskrant). This means universities are finally taking a stand: We want scientific knowledge to be freely available as open access to anyone who wants to read it, and we believe publishers are making more than enough money as it is. Moving towards fully

Yesterday Mike McCullough posted an interesting question on Twitter. He had collected some data, observed a p = 0.026 for his hypothesis, but he wasn't happy. Being aware that higher p -values do not always provide strong support for H1, he wanted to set a new goal and collect more data.

Chris Fraley and Simine Vizare (F&V) published a very interesting paper in PlosOne where they propose to evaluate journals based on the sample size and statistical power of the studies.

[a slightly adapted version of this blog post is now in press at QJEP: see https://osf.io/ycag9/ for the manuscript and R scripts] In this blog, I'll explain how p-hacking will not lead to a peculiar prevalence of p-values just below .05 (e.g., in the 0.045-0.05 range) in the literature at large, but will instead lead to a difficult to identify increase in the Type 1 error rate across the 0.00-0.05 range.

I'd like to gratefully acknowledge the extremely helpful comments and suggestions by Anton Kühberger and his co-authors on an earlier draft of this blog post, who patiently answered my questions and shared additional analyses. I'd also like to thank Marcel van Assen, Christina Bergmann, JP de Ruiter, and Uri Simonsohn for comments and suggestions.

This Thursday I’ll be giving a workshop on good research practices in Leuven, Belgium. The other guest speaker at the workshop is Eric-Jan Wagenmakers, so I thought I’d finally dive in to the relationship between Bayes Factors and p -values to be prepared to talk in the same workshop as such an expert on Bayesian statistics and methodology.

I have no idea how many people take the effort to reproduce a meta-analysis in their spare time. What I do know, based on my personal experiences of the last week, is that A) it’s too much work to reproduce a meta-analysis, primarily due to low reporting standards, and B) we need to raise the bar when doing meta-analyses.