Thank you to everyone for your amazing submissions! We are thrilled to announce the winners of this year’s Outstanding Contribution to Communication Science Award. We had top notch submissions and the committee (Emily Falk, Richard Huskey, in consultation with the rest of the CSaB officers) decided that two complementary efforts really exemplify values we seek to promote in CSaB. Both of these papers represent the work of large teams– Miller and colleagues promote a new combination of methods to increase reproducibility and generalizability in communication science and biology. McEwan and colleagues organized an entire special issue that motivated people to actually conduct replication studies in communication science. We applaud both of their team efforts, and encourage you to check out not only the target papers but also the replies in Psychological Inquiry and the special issue in Communication Studies.
Causal Inference in Generalizable Environments: Systematic Representative Design
Lynn C. Millera,f, Sonia Jawaid Shaikha, David C. Jeonga, Liyuan Wanga, Traci K. Gilligb, Carlos G. Godoya, Paul R. Applebya, Charisse L. Corsbie-Massayc, Stacy Marsellad, John L. Christensene, and Stephen J. Readf
https://doi.org/10.1080/1047840X.2019.1693866
aAnnenberg School for Communication and Journalism, University of Southern California, Los Angeles, California; bMurrow College of Communication, Washington State University, Pullman, Washington; cS. I. Newhouse School of Public Communications, Syracuse University, Syracuse, New York; dInstitute of Neuroscience & Psychology, Glasgow University, Glasgow, United Kingdom, and Khoury College of Computer Sciences, Northeastern University, Boston, Massachusetts; eDepartment of Communication, University of Connecticut, Storrs, Connecticut; fPsychology Department, University of Southern California, Los Angeles, California
SUMMARY: Causal inference and generalizability both matter. Historically, systematic or classic experimental designs -with roots in Wundt(1902)– emphasize causal inference. Representative designs, with roots in Brunswik (1943, 1955a, 1955b) — often used in conjunction with observational and correlational research — focus on generalizability. As Brewer and Crano (2014, p. 19) note, generalizability or replicability, could be designed into an experimental operationalization by using representative sampling of the targets to which researchers wish to generalize. Representative sampling (e.g., of persons, stimuli, contexts, and their interactions) in the design phase of developing one’s experimental and control conditions would simultaneously “build in” a new type of generalizability — generalizability to everyday life (GEL). Situations that are not, by design, representative in the first place, may demonstrate external validity (e.g., cause-effect relationship in one experiment found in a second with a different population) but not GEL: The extent of the problem (i.e., that our findings may not have GEL) is unknown.
Although most psychologists have considered it not feasible within experiments to optimize representativeness (Brewer and Crano, 2014) of the “organism-in-situation” (Cronbach, 1957) to which we wish to generalize, we suggest that it is and offer a transformative synthesis – Systematic Representative Design (SRD) – concurrently enhancing both causal inference and “built-in” generalizability enabled by leveraging today’s intelligent agent, virtual environments, and other technologies. In SRD, a “default control group” (DCG) can be created in a virtual environment by representatively sampling from real-world narrative sequences, situations, and behavioral options leading up to the behaviors of interest (BOI) for the target populations of interest (POI). Experimental groups can be built with systematic manipulations onto the DCG base. And, we propose checks (virtual validity coefficients between everyday life situation-responses and similar virtual situation-responses given the same individuals) on our ability to create representative worlds that we have used in four of our studies yielding promising coefficients (e.g., r=.70). Applying systematic design features (e.g., random assignment to DCG versus experimental groups) in SRD affords valid causal inferences. In our article, after explicating the proposed SRD synthesis, we delineate how the approach concurrently advances generalizability and robustness, cause-effect inference and precision science, a computationally- enabled cumulative communication science supporting both “bigger theory” (as physicists, for example, might construe it) and concrete implementations grappling with tough questions (e.g., what is context?) and affording rapidly-scalable interventions for addressing real-world communication problems.
On Replication in Communication Science.
Bree McEwan, Christopher J. Carpenter & David Westerman
https://doi.org/10.1080/10510974.2018.1464938
aCollege of Communication at DePaul University; bDepartment of Communication at Western Illinois University; cDepartment of Communication at North Dakota State University
SUMMARY: Replications are an important part of the research process because they allow for greater confidence in the findings of communication research. However, engaging in replications is often undervalued, replication studies can be difficult to publish, and thus it is difficult for individual scholars to devote their resources toward replication. The paper that we submitted for this award outlines the importance of replications in communication as a key component of the scientific method but also notes that so often as field we fail to conduct research that supports this key aspect. We note that “without replication, we are no longer doing good science; we are simply playing with numbers.”
With this essay we join our voices to other prominent scholars who over the years have called for increased replications in the social sciences (e.g. Benoit & Holbert, 2011; Bostrom & Donohew,1992; Hunter, 2001; Nosek & Laken, 2014). However, this essay comprised one part of a larger project – a special issue of Communication Studies that provided an outlet for nine replication projects examining communication findings across disciplinary contexts. The special issue used a quasi-registered reports format where paper proposals were solicited and designs were reviewed before any data was collected. Accepted proposals were assured publication regardless of the findings of the paper. Thus, our project connects the existing infrastructure of our science to possibilities for future improvements. We are hopeful that the success of this project will encourage others to look for ways to further the quality of communication science in similarly achievable and scalable ways.