Spencer C. Evans , Christina M. Amaro, Robyn Herbert, Jennifer B. Blossom, Michael C. Roberts Published: February 14, 2018 https://doi.org/10.137

Abstract

If a doctoral dissertation represents an original investigation that makes a contribution to one’s field, then dissertation research could, and arguably should, be disseminated into the scientific literature. However, the extent and nature of dissertation publication remains largely unknown within psychology. The present study investigated the peer-reviewed publication outcomes of psychology dissertation research in the United States. Additionally, we examined publication lag, scientific impact, and variations across subfields. To investigate these questions, we first drew a stratified random cohort sample of 910 psychology Ph.D. dissertations from ProQuest Dissertations & Theses. Next, we conducted comprehensive literature searches for peer-reviewed journal articles derived from these dissertations published 0–7 years thereafter. Published dissertation articles were coded for their bibliographic details, citation rates, and journal impact metrics. Results showed that only one-quarter (25.6% [95% CI: 23.0, 28.4]) of dissertations were ultimately published in peer-reviewed journals, with significant variations across subfields (range: 10.1 to 59.4%). Rates of dissertation publication were lower in professional/applied subfields (e.g., clinical, counseling) compared to research/academic subfields (e.g., experimental, cognitive). When dissertations were published, however, they often appeared in influential journals (e.g., Thomson Reuters Impact Factor M = 2.84 [2.45, 3.23], 5-year Impact Factor M = 3.49 [3.07, 3.90]) and were cited numerous times (Web of Science citations per year M = 3.65 [2.88, 4.42]). Publication typically occurred within 2–3 years after the dissertation year. Overall, these results indicate that the large majority of Ph.D. dissertation research in psychology does not get disseminated into the peer-reviewed literature. The non-publication of dissertation research appears to be a systemic problem affecting both research and training in psychology. Efforts to improve the quality and “publishability” of doctoral dissertation research could benefit psychological science on multiple fronts.

Introduction

The doctoral dissertation—a defining component of the Doctor of Philosophy (Ph.D.) degree—is an original research study that meets the scientific, professional, and ethical standards of its discipline and advances a body of knowledge [1]. From this definition it follows that most dissertations could, and arguably should, be published in the peer-reviewed scientific literature [1–2]. For example, research participants typically volunteer their time and effort for the purposes of generating new knowledge of potential benefit; therefore, to breach this contract by not attempting to disseminate one’s findings is to violate the ethical standards of psychology [3] and human subjects research [2,4]. The nonpublication of dissertation research can also be detrimental to the advancement of scientific knowledge in other ways. Researchers may unwittingly and unnecessarily duplicate efforts from doctoral research when conducting empirical studies, or draw biased conclusions in meta-analytic and systematic reviews that often deliberately exclude dissertations. Many dissertations go unpublished due to nonsignificant and complicated results, exacerbating the “file drawer” problem [5–6]. Indeed, unpublished dissertations are rarely if ever cited [7–8].

The problem of dissertation non-publication is of critical importance in psychology. Some evidence [9] suggests that unpublished dissertations can play a key role in alleviating file drawer bias and reproducibility concerns in psychological science [10]. More broadly, the field of psychology—given its unique strengths, breadth, and diversity—poses a useful case study for examining dissertation nonpublication in the social, behavioral, and health sciences. Like other scientific disciplines, many Ph.D. graduates in psychology may be motivated to revise and submit their dissertations for publication for the usual reasons offered by academic and research careers. However, other new psychologists might not pursue this goal for a variety of reasons. Those in professional and applied subfields may commit most or all of their working time to non-research activities (e.g., professional practice, clinical training) and have little incentive to seek publication. Even those in more research-oriented subfields increasingly take non-research positions (e.g., industry, consultation, teaching, policy work) or other career paths which do not incentivize publications. Negative graduate school experiences, alternative career pursuits, and personal or family matters can all be additional factors that may decrease the likelihood of publication. Moreover, it is typically a challenging and time-consuming task to revise a lengthy document for submission as one or more journal articles. Still, all individuals holding a Ph.D. in psychology have (in theory) produced an original research study of scientific value, which should (again, in theory) be shared with the scientific community. Thus, for scientific, ethical, and training reasons, it is important to understand the frequency and quality of dissertation publication in psychology.

There is an abundance of literature relevant to this topic, including student or faculty perspectives (e.g., [11–13]) and studies of general research productivity during doctoral training and early career periods (e.g., [14–19]). However, evidence specifically regarding dissertation publication is remarkably sparse and inconsistent [8,20–24]. This literature is limited by non-representative samples, biased response patterns, and disciplinary scopes that are either too narrow or too broad to offer insights that are useful and generalizable for psychological science. For example, in the only psychology-specific study to our knowledge, Porter and Wolfle [23] mailed surveys to a random sample of individuals who earned their psychology doctorates. Of 128 respondents, 59% reported that their dissertation research had led to at least one published article. Unfortunately, this study [23] and others (e.g., [8]) are now over 40 years old, offering little relevance to the present state of training and research in psychology. A much more recent and rigorous example comes from the field of social work. Using a literature searching methodology and a random sample of 593 doctoral dissertations in social work, Maynard et al [22] found that 28.8% had led to peer-reviewed publications. However, this estimate likely does not generalize to psychology and its myriad subfields. Thus, there is a need for more comprehensive, rigorous, and recent data to better understand dissertation publication in psychology.

Accordingly, the present study investigated the extent and nature of dissertation publication in psychology, specifically examining the following questions: (a) How many dissertations in psychology are eventually published in peer-reviewed journals? (b) How long does it take from dissertation approval to article publication? (c) What is the scientific impact of published dissertations (PDs)? and (d) Are there differences across subfields of psychology? Based on the literature and our own observations, we hypothesized that (a) a majority of dissertations in psychology would go unpublished; (b) dissertation publication would occur primarily during the first few years after Ph.D. approval, diminishing thereafter; (c) PDs would show evidence of at least moderate scientific influence via citation rates and journal metrics; and (d) professional/applied subfields (clinical, counseling, school/educational, industrial-organizational, behavioral) would yield fewer PDs than research-oriented subfields (social/personality, experimental, cognitive, neuroscience, developmental, quantitative).

Materials and methods

Sample

The dataset of psychology dissertations was obtained directly from ProQuest UMI’s Dissertations and Theses Database (PQDT), which is characterized as “the world’s most comprehensive collection of dissertations and theses. . . [including] full text for most of the dissertations added since 1997. . . . More than 70,000 new full text dissertations and theses are added to the database each year through dissertations publishing partnerships with 700 leading academic institutions worldwide” [25]. While international coverage varies across countries, PQDT’s repository is estimated to include approximately 97% of all U.S. doctoral dissertations [26], across all disciplines, institutions, and training models.

Upon request, PQDT provided a database of all dissertations indexed with the term “psychology” in the subject field during the year 2007. This resulted in a total population of 6,580 dissertations, which were then screened and sampled according to pre-defined criteria. The number of dissertations included at each stage in the sampling process is summarized in a PRISMA-style [27] flow diagram for the overall sample in Fig 1, and broken down by subfield in Table 1. Dissertations were excluded if written in a language other than English, for any degree other than Ph.D. (e.g., Psy.D., Ed.D.), or in any country other than U.S. The remaining dissertations were recoded for subfields based on the subject term classification in PQDT, with a few modifications (e.g., combining “neuroscience” and “biological psychology”). This left a remaining sample of 3,866 relevant dissertations, representing our population. This figure is approximately in line with the U.S. National Science Foundation’s Survey of Earned Doctorates [28] estimate that 3,276 research doctorates in psychology were granted during the year 2007, suggesting that PQDT could be slightly broader or more comprehensive in scope.

Fig 1. Flow diagram reflecting the numbers of dissertations included at each stage of the sampling process.

Note. PQDT = ProQuest Dissertations and Theses. ^a Categories of excluded dissertations are mutually exclusive, summing to 100%. ^b PQDT exclusion criteria were applied sequentially in the order presented; thus, the number associated with each exclusion criterion reflects how many were excluded from the sample that remained after the previous criterion was applied. Adapted from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram [27].