講演会(社会科学研究所)
Cecilia Mo氏講演会(社会科学研究所)
Observational Open Science: An Application to the Literature on Irrelevant Events and Voting Behavior
日時:2019年3月26日(火)11:00~12:30 (10:30開場)

場所:東京大学本郷キャンパス 赤門総合研究棟 549室
キャンパス内の地図はこちら

講師:Cecilia Mo (Department of Political Science, UC Berkeley)
講師についてはこちら

要旨:Replication and transparency have become increasingly important in bolstering the credibility of political science research, yet open science tools are typically designed for experiments. For observational studies, which make up the bulk of political science, current practice suffers from an important pathology: just as researchers can often ``p-hack’’ their way to initial findings, it is often possible to ``null hack’’ a finding away through specification and case search. We propose an observational open science (OOS) framework that consists of (1) leveraging the passage of time to add new out-of-sample cases, (2) independent collection of all data, (3) pre-registration of alternative specifications, (4) multiple simultaneous replications, and (5) collaboration between original authors and skeptics. We apply the approach to three highly cited studies in the literature on ``irrelevant’’ events and voting behavior. On some dimensions, the studies all replicated quite well. On other dimensions, they replicated quite poorly. Had we sought to debunk any of these three studies in a conventional approach to replication with ex post specification search, we could have easily done so. However, given that we conducted pre-registered analyses, we have a more accurate sense of the full and complicated picture. We conclude with suggestions for future refinements to our approach.