Ciencias SocialesInglésSubstack

Critical Metascience

Critical Metascience
Critical metascience takes a step back to question some common assumptions, approaches, problems, and solutions in metascience.
Página de inicioFeed RSS
language
Replication CrisisPublication BiasP-hackingHARKingQuestionable Research PracticesCiencias SocialesInglés
Publicado
Autor Mark Rubin

Abstract Research on questionable research practices (QRPs) includes a growing body of work that questions whether they are as problematic as commonly assumed. This article provides a brief and selective review that considers some of this work. In particular, the review highlights work that questions the prevalence and impact of QRPs, including p -hacking, HARKing, and publication bias.

Replication CrisisOpen Science MovementMetascienceAcademic WorkloadScience ReformCiencias SocialesInglés
Publicado
Autores Sheena Bartscherer, Sven Ulpts

Background Sparked by highly publicised cases of scientific misconduct as well as the identification of issues surrounding questionable research practices and irreproducibility of research findings, a scandalization process ensued (Penders, 2024). This scandalization initially started around narratives of a crisis concerning the research (processes) in some psychological and biomedical (sub)disciplines.

P-hackingError StatisticsPreregistrationPopperCritical RationalismCiencias SocialesInglés
Publicado
Autor Mark Rubin

Preregistration Distinguishes Between Exploratory and Confirmatory Research? Previous justifications for preregistration have focused on the distinction between “exploratory” and “confirmatory” research. However, as I discuss in this recent presentation, this distinction faces unresolved questions. For example, the distinction does not appear to have a formal definition in either statistical theory or the philosophy of science.

Replication CrisisPsychologyStatistical InferenceReproducibilityCiencias SocialesInglés
Publicado
Autor Mark Rubin

An often overlooked source of the “replication crisis” is the tendency to treat the replication study as a definitive verdict while ignoring the statistical uncertainty inherent in both the original and replication studies. This simplistic view fosters misleading dichotomies and erodes public trust in science.

Replication CrisisMetascienceP-hackingPredictionHARKingCiencias SocialesInglés
Publicado

A conference called “The promises and pitfalls of preregistration” was hosted by the Royal Society in London from 4th-5th March 2024. Here, I discuss the presentations by Chris Donkin and Stephan Lewandowsky, both of which consider some of the potential “pitfalls” of preregistration.

MetascienceWorkload ModelsMetaresearchAcademic WorkloadScience ReformCiencias SocialesInglés
Publicado
Autor Mark Rubin

The Centre for Open Science’s symposium on “Critical Perspectives on the Metascience Reform Movement” took place on 7 th March 2024. It was organised by Sven Ulpts, and it includes presentations by Bart Penders, Tom Hostler, Stephan Guttinger, Sarahanne Field, Nicole Nelson, and Berna Devezer.

Multiple TestingMultiple ComparisonsMultiplicityPer Family Error RateFamilywise Error RateCiencias SocialesInglés
Publicado
Autor Mark Rubin

During multiple testing, researchers often adjust their alpha level to control the familywise error rate for a statistical inference about a joint union alternative hypothesis (e.g., “ H1 or H2 ”). However, in some cases, they do not make this inference.

Replication CrisisSignificance TestingMultiple TestingMultiple ComparisonsMetascienceCiencias SocialesInglés
Publicado
Autor Mark Rubin

The inflation of Type I error rates is thought to be one of the causes of the replication crisis. Questionable research practices such as p -hacking are thought to inflate Type I error rates above their nominal level, leading to unexpectedly high levels of false positives in the literature and, consequently, unexpectedly low replication rates. In this article, I offer an alternative view.

Replication CrisisMetascienceP-hackingPredictionHARKingCiencias SocialesInglés
Publicado
Autor Mark Rubin

In an article published last week in Synthese, philosopher of science Pekka Syrjänen asked “does a theory become better confirmed if it fits data that was not used in its construction versus if it was specifically designed to fit the data?” The first approach is called prediction, and the second approach is called accommodation . The debate over the epistemic advantages of prediction and accommodation has been bubbling away for

Replication CrisisMetascienceMetaresearchTheory TestingPopperCiencias SocialesInglés
Publicado
Autor Mark Rubin

Abstract Popper’s (1983, 2002) philosophy of science has enjoyed something of a renaissance in the wake of the replication crisis, offering a philosophical basis for the ensuing science reform movement. However, adherence to Popper’s approach may also be at least partly responsible for the sense of “crisis” that has developed following multiple unexpected replication failures.

MetascienceScience, Technology, And SocietyOpen ScienceAcademic WorkloadScience ReformCiencias SocialesInglés
Publicado
Autor Tom Hostler

An Intellectual Vocation In his book “The Soul of a University” (2018), Chris Brink describes the story of G.H. Hardy, a Cambridge Mathematician whose principled stance on his academic research was that it had no practical use whatsoever: “No discovery of mine” Hardy proudly wrote, “has made, or is likely to make, directly or indirectly, for good or ill, the least difference to the amenity of the world”