skip to main content
Language:
Search Limited to: Search Limited to: Resource type Show Results with: Show Results with: Search type Index

Characterizing the Google Books Corpus: Strong Limits to Inferences of Socio-Cultural and Linguistic Evolution

PloS one, 2015-10, Vol.10 (10), p.e0137041-e0137041 [Peer Reviewed Journal]

COPYRIGHT 2015 Public Library of Science ;COPYRIGHT 2015 Public Library of Science ;2015 Pechenick et al. This is an open access article distributed under the terms of the Creative Commons Attribution License: http://creativecommons.org/licenses/by/4.0/ (the “License”), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. ;2015 Pechenick et al 2015 Pechenick et al ;ISSN: 1932-6203 ;EISSN: 1932-6203 ;DOI: 10.1371/journal.pone.0137041 ;PMID: 26445406

Full text available

Citations Cited by
  • Title:
    Characterizing the Google Books Corpus: Strong Limits to Inferences of Socio-Cultural and Linguistic Evolution
  • Author: Pechenick, Eitan Adam ; Danforth, Christopher M ; Dodds, Peter Sheridan
  • Barrat, Alain
  • Subjects: Culture ; Datasets ; Digitization ; Divergence ; Evolution ; Humans ; Information services ; Information services industry ; Information theory ; Language ; Libraries ; Library collections ; Linguistic analysis (Linguistics) ; Linguistics ; Linguistics - trends ; Mathematics ; Metadata ; Reading ; Science ; Services ; Social networks ; Texts
  • Is Part Of: PloS one, 2015-10, Vol.10 (10), p.e0137041-e0137041
  • Description: It is tempting to treat frequency trends from the Google Books data sets as indicators of the "true" popularity of various words and phrases. Doing so allows us to draw quantitatively strong conclusions about the evolution of cultural perception of a given topic, such as time or gender. However, the Google Books corpus suffers from a number of limitations which make it an obscure mask of cultural popularity. A primary issue is that the corpus is in effect a library, containing one of each book. A single, prolific author is thereby able to noticeably insert new phrases into the Google Books lexicon, whether the author is widely read or not. With this understood, the Google Books corpus remains an important data set to be considered more lexicon-like than text-like. Here, we show that a distinct problematic feature arises from the inclusion of scientific texts, which have become an increasingly substantive portion of the corpus throughout the 1900 s. The result is a surge of phrases typical to academic articles but less common in general, such as references to time in the form of citations. We use information theoretic methods to highlight these dynamics by examining and comparing major contributions via a divergence measure of English data sets between decades in the period 1800-2000. We find that only the English Fiction data set from the second version of the corpus is not heavily affected by professional texts. Overall, our findings call into question the vast majority of existing claims drawn from the Google Books corpus, and point to the need to fully characterize the dynamics of the corpus before using these data sets to draw broad conclusions about cultural and linguistic evolution.
  • Publisher: United States: Public Library of Science
  • Language: English
  • Identifier: ISSN: 1932-6203
    EISSN: 1932-6203
    DOI: 10.1371/journal.pone.0137041
    PMID: 26445406
  • Source: GFMER Free Medical Journals
    MEDLINE
    PubMed Central
    Public Library of Science
    ProQuest Central
    DOAJ Directory of Open Access Journals
Back to results list
INSPIRE LIBRARY - TON DUC THANG UNIVERSITY (84-028) 37 755 057 Feedback
19 Nguyen Huu Tho St. Dist.7, HCM thuvien@tdtu.edu.vn

Copyright © 2017 INSPIRE LIBRARY - TON DUC THANG UNIVERSITY

Searching Remote Databases, Please Wait