Gordon Feld, Dr. rer. nat.

Arbeitsvorhaben

Towards Reproducibility: Adapting Past and Future Projects in Sleep and Memory to a Transparent and Sustainable Model of Science

In the biomedical and social sciences, published findings have recently been found to be far less robust than many scientists had thus far believed. A systematic attempt to replicate several key studies in psychology estimates that fewer than half of the effects reported in the literature are true effects. This reproducibility crisis has cast doubt on psychology as a science. Too much flexibility in the way data are handled and reported has been identified as one of the key causes of this low reproducibility. The open science movement attempts to induce new trust by making methods and data more openly available, which restricts such flexibility and makes it more traceable. At the Wissenschaftskolleg, I will pursue three goals to transform my research pipeline according to these principles. First, I will prepare all of the data that I have used in published manuscripts for their publication online. These data have been collected from adult participants after receiving their informed consent and include bio-physiological information (such as blood analyses and sleep EEG). At the time of collection, these participants did not consent to an online publication of their data, therefore, I will evaluate to which extent their anonymous data can be shared online by conferring with the appropriate ethics committees. Second, for my next study I will prepare a detailed record of the methods and submit it to a journal before any data is collected, also documenting any statistical analyses I will run. This registered reports format is being adopted by more and more research outlets and guarantees publication irrespective of the achieved results. Third, I will perform a meta-science survey to assess the prevalence of flexible methods and the adoption of open science principles in the fields of biological psychology and neuroscience. This will allow an informed discussion about the incentive structure that is needed to improve psychology as a science.

Recommended Reading

Feld, G. B. and J. Born (2017). "Sculpting memory during sleep: concurrent consolidation and forgetting." Curr Opin Neurobiol 44: 20-27.
Feld, G. B., L. Besedovsky, K. Kaida, T. F. Münte, and J. Born (2014). "Dopamine D2-like receptor activation wipes out preferential consolidation of high over low reward memories during human sleep." J Cogn Neurosci 26, 10: 2310-2320.
Feld, G. B., T. Lange, S. Gais, and J. Born (2013). "Sleep-dependent declarative memory consolidation - unaffected after blocking NMDA or AMPA receptors but enhanced by NMDA coagonist D-cycloserine." Neuropsychopharmacology 38, 13: 2688-2697.

Kolloquium, 27.09.2018

The Crisis Is Dead, Long Live the Crisis - How Much Trust Can We Put in Psychological Findings?

Since its inception in the 19th century, modern experimental psychology has been repeatedly described as being in crisis, and without surprise it has recently entered a new cycle. The latest crisis was triggered by several cases of blatant scientific fraud, as well as by a failure to replicate some of its most famous findings. I will give a brief introduction to how modern psychology evaluates evidence for its theories, then point out some loopholes in this approach, and finally end with some suggested solutions.

Psychology as an empirical science is heavily influenced by Popper's falsificationism. Ideally, this means that, after theories are generated, their predictions are tested empirically and survive only until they are falsified. In reality, most psychologists evaluate theories using null-hypothesis-significance-testing (NHST), which means that a statistical hypothesis (e.g., there is a difference between two groups regarding variable A) is tested by attempting to falsify its null hypothesis (e.g., that there is no difference between two groups regarding variable A). If the null is falsified, trust in the theory increases; if it is not falsified, trust decreases. This type of statistical inference relies on a series of assumptions that can easily lead to cheating - even without intent.

Diederik Stape, the most famous fraudster in modern psychological science, has retracted more than 50 publications to date - including from the prestigious journal Science. In his autobiography "Ontsporing" ("Derailed"), he describes the feeling of publishing shiny papers in prestigious journals as akin to addiction, but also emphasizes the incentive structure of science that encouraged his deeds. He reveals how a lack of sufficient checks and balances made him ever more brazen. Although singular in its extremity, his case demonstrates that the emphasis of the scientific community on sexy results and the reliance of career advancement on high-impact publications have led to a misalignment of incentives and scientific rigor.

Reacting to this shake-up, a team of psychologists attempted to replicate 100 findings from three reputed psychology journals. In these experiments, only 40% of the prior findings could be replicated. This replication failure has been attributed mainly to the small sample sizes in psychological experiments, paired with the huge flexibility that researchers have when analyzing their data (researcher degrees of freedom). Additionally, the scientific publishing system penalizes null findings (findings where the null was not falsified) and thus motivates creative analysis strategies to produce a positive and thus publishable finding.

Several solutions have been proposed to solve this crisis. For example, preregistering studies will allow others to control how many degrees of freedom went into finding a given result, whereas open data and open materials will allow others to reproduce the analyses and replicate the studies. All in all, psychology has started to develop the tools to overcome this new crisis and stands strengthened.

Publikationen aus der Fellowbibliothek

Feld, Gordon (London, 2020)

Building the bridge : outlining steps toward an applied sleep-and-memory research program

Feld, Gordon (Cold Spring Habor, 2019)

The gap junction blocker mefloquine impairs sleep-dependent declarative memory consolidation in humans