Teaching Good Research Practice

I attended a webinar on how to teach students to document empirical research by Richard Ball and Norm Medeiros from Havorford College and hosted by the Interuniversity Consortium for Political and Social Research (ICPSR). This idea aims to counter current norms, policies and practices in teaching empirical research by having students submit all their statistical analyses with their final project. This should include all the necessary documentation to allow a third-party to replicate all statistical results, what Ball and Medeiros call “a soup-to-nuts approach”. This approach in turn enhances professional norms and practices through a trickle-up effect, students actually understand what they are doing, and students know they are being held accountable. The webinar used an example from an economics course, but it is easy to imagine the potential for social work education and research.

The slides are available on their YouTube channel. It’s worth checking out and rethinking how we can use this in our classrooms and research.

Replication called for in psychology ‘priming’ studies

Daniel Kanemann calls for ‘priming’ studies in psychology to be replicated in NATURE.

I find it interesting that one of Kanemann’s motivations is to protect graduate students on the job market.

h/t @deankarlan

06 July update:

I see that the funder 3ie now has a replication program to fund replications of past programs/studies. HERE.

They have an informative bibliography and FAQ there.

And here is Stevenson and Wolfers review of replication in economics literature

For a scholar, replication offers an unappealing bet. Heads, you discover that the findings of an original study are largely correct, and no journal will publish your paper because there’s no interest in learning that something is still true. Tails, you find a serious flaw, but your results still probably won’t be published and you’ve earned enemies who may try to land some reputational punches against you.

 

Big blunders in published research

In the wake of the Reinhart Rogoff errors, Catherine Rampell at the NYT provides an overview of recent research blunders in the field of economics.
HERE

As hard as we try no research is perfect. Reflecting on these errors helps put the mistakes I’ve made in perspective. These errors also help make the case for replication and data sharing – a topic we’ve discussed frequently at the CRCFBB.

Blog authors are solely responsible for the content of the blogs listed in the directory. Neither the content of these blogs, nor the links to other web sites, are screened, approved, reviewed or endorsed by McGill University. The text and other material on these blogs are the opinion of the specific author and are not statements of advice, opinion, or information of McGill.