Researchers increasingly use the Open Science Framework (OSF) to share files, such as data and code underlying scientific publications, or presentations and materials for scientific workshops. The OSF is an amazing service that has contributed immensely to a changed research culture where psychologists share data, code, and materials. We are very grateful it exists.   But it is not always the most user-friendly.

Are meta-scientists ignoring philosophy of science (PoS)? Are they re-inventing the wheel? A recent panel at the Metascience conference engaged with this question, and the first sentence of the abstract states “Critics argue that metascience merely reinvents the wheel of other academic fields.” It’s a topic I have been thinking about for a while, so I will share my thoughts on this question.

code{white-space: pre-wrap;} span.smallcaps{font-variant: small-caps;} div.columns{display: flex; gap: min(4vw, 1.5em);} div.column{flex: auto; overflow-x: auto;} div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;} ul.task-list{list-style: none;} ul.task-list li input[type="checkbox"] { width: 0.8em; margin: 0 0.8em 0.2em -1em; vertical-align: middle; } pre > code.sourceCode { white-space: pre; position: relative; } pre >

When the scientific discipline started to emerge, Bacon (1620) remarked: “And it is nothing strange if a thing not held in honour does not prosper”. His idea was that we need to give honour to scientists who do good work, otherwise no one would want to become a scientist.

The Journal of Personality and Social Psychology is one of the main outlets for social and personality psychologists. It publishes around 110 empirical articles a year (and a small number of other types of articles) and is considered a prestigious outlet. It is also often criticized. That’s to be expected.

What is the goal of data collection? This is a simple question, and as researchers we collect data all the time. But the answer to this question is not straightforward. It depends on the question that you are asking of your data. There are different questions you can ask from your data, and therefore, you can have different goals when collecting data. Here, I want to focus on collecting data to test scientific theories.

This blog post is based on a pre-print by Coles, Tiokhin, Scheel, Isager, and Lakens “The Costs and Benefits of Replications”, submitted to Behavioral Brain Sciences as a commentary on “Making Replication Mainstream”. In a summary of recent discussions about the role of direct replications in psychological science, Zwaan, Etz, Lucas, and Donnellan (2017) argue that replications should be more mainstream.

A p -value is the probability of the observed, or more extreme, data, under the assumption that the null-hypothesis is true. The goal of this blog post is to understand what this means, and perhaps more importantly, what this doesn’t mean. People often misunderstand p -values, but with a little help and some dedicated effort, we should be able explain these misconceptions.

If I ever make a follow up to my current MOOC, I will call it ‘Improving Your Statistical Questions’. The more I learn about how people use statistics, the more I believe the main problem is not how people interpret the numbers they get from statistical tests. The real issue is which statistical questions researchers ask from their data. Our statistics education turns a blind eye to training people how to ask a good question.

Science-Wise False Discovery Rate Does Not Explain the Prevalence of Bad Science h1 { font-size: 34px; } h1.title { font-size: 38px; } h2 { font-size: 30px; } h3 { font-size: 24px; } h4 { font-size: 18px; } h5 { font-size: 16px; } h6 { font-size: 12px; } .table th:not([align]) { text-align: left; } .main-container { max-width: 940px; margin-left: auto; margin-right: auto; } code { color: inherit;

The replication crisis is over. Sure, not everyone has gotten the memo (either about it having started, or about it having ended) but the majority of scientists agree that there were (slightly) too many findings from the past that cannot easily be replicated. The underlying reasons are clear: publication bias, flexibility in the data analysis, low power, and not enough rewards for replication studies.