2022/2023

Rose O'Dea, PhD

Biological Sciences

Universität Melbourne

from September 2022 to March 2023
Born in 1992 in Canberra
Studied Biological Sciences at the Australian National University and at the University of New South Wales

Fellowship

College for Life Sciences

Arbeitsvorhaben

Appraising Evidence and Generality in Ecology

Confronted by a “reproducibility crisis” in the life sciences, researchers have turned to systematic reviews and replication projects to find and amplify reliable evidence. The ideal systematic review evaluates the quality of the primary evidence by considering, for each study, whether internal biases were minimised and whether inferences from that sample transfer or generalise to a different target population. Replication projects ideally repeat studies using the same methods to evaluate whether the original inference can be reliably reproduced, then iteratively vary aspects of the study design to define the limits of generality. Therefore, both systematic reviews and replication projects prompt researchers to define and estimate generality.
I think ecology has a generality problem. The conditions of a single study are often impossible to closely replicate. In those cases, replication projects could be entirely based on “conceptual” replications (testing the same phenomena in a different way), but these can only be interpretable if theory is strong enough to define the conditions under which we’d expect phenomena to emerge. Unlike replication projects, systematic reviews and meta-analyses are common in ecology, but they typically combine very diverse types of evidence, and there are no methods for assessing their quality and generality on a common scale. Doing quality assessments badly could be worse that not doing them at all; we risk overweighting carefully controlled laboratory studies, while losing sight of the wild and messy world that we’re trying to understand.
At the Wissenschaftskolleg I want to begin answering three questions to increase the utility of ecological research. First, how often do ecologists estimate or define the generality of their claims? Second, how prevalent are “conceptual replications” (studies of the same phenomena conducted in a different way or a different setting) in ecology? Third, can we estimate the generality of existing research, and if so, how?

Recommended Reading

O’Dea, Rose E., Malgorzata Lagisz, Michael D. Jennions, Julia Koricheva, Daniel W. A. Noble, Timothy H. Parker, Jessica Gurevitch, et al. (2021). “Preferred Reporting Items for Systematic Reviews and Meta-Analyses in Ecology and Evolutionary Biology: A PRISMA Extension.” Biological Reviews 96: 1695–1722. https://doi.org/10.1111/brv.12721.
O’Dea, Rose E., Timothy H. Parker, Yung En Chee, Antica Culina, Szymon M. Drobniak, David H. Duncan, Fiona Fidler, et al. (2021). “Towards Open, Reliable, and Transparent Ecology and Evolutionary Biology.” BMC Biology 19: 68. https://doi.org/10.1186/s12915-021-01006-3.
O’Dea, Rose E., Daniel W. Noble, and Shinichi Nakagawa (2022). “Unifying Individual Differences in Personality, Predictability, and Plasticity: A Practical Guide.” Methods in Ecology and Evolution 13: 278–293. https://doi.org/10.1111/2041-210X.13755.