A fully data-driven method to identify (correlated) changes in diachronic corpora
- In this paper, a method for measuring synchronic corpus (dis-)similarity put forward by Kilgarriff (2001) is adapted and extended to identify trends and correlated changes in diachronic text data, using the Corpus of Historical American English (Davies 2010a) and the Google Ngram Corpora (Michel et al. 2010a). This paper shows that this fully data-driven method, which extracts word types that have undergone the most pronounced change in frequency in a given period of time, is computationally very cheap and that it allows interpretations of diachronic trends that are both intuitively plausible and motivated from the perspective of information theory. Furthermore, it demonstrates that the method is able to identify correlated linguistic changes and diachronic shifts that can be linked to historical events. Finally, it can help to improve diachronic POS tagging and complement existing NLP approaches. This indicates that the approach can facilitate an improved understanding of diachronic processes in language change.
Author: | Alexander KoplenigORCiDGND |
---|---|
URN: | urn:nbn:de:bsz:mh39-42569 |
DOI: | https://doi.org/10.1080/09296174.2017.1311447 |
Place of publication: | Mannheim |
Document Type: | Working Paper |
Language: | English |
Year of first Publication: | 2015 |
Date of Publication (online): | 2015/10/13 |
Publicationstate: | Preprint |
Tag: | BNC; COHA; Google Books Ngram corpora; diachronic corpora; text mining; time series analysis |
GND Keyword: | Englisch; Korpus <Linguistik>; Sprachgeschichte; Text Mining; Wortschatz |
Edition: | Version 1, 08-2015 |
Page Number: | 46 |
Note: | Der Artikel wurde vom "Journal of Quantitative Linguistics" zur Veröffentlichung angenommen und ist am 26. April 2017 online erschienen: http://dx.doi.org/10.1080/09296174.2017.1311447. This is a preprint of an accepted article published by Taylor & Francis in "Journal of quantitative linguistics" on 26 April 2017, available online: http://dx.doi.org/10.1080/09296174.2017.1311447. |
DDC classes: | 400 Sprache / 410 Linguistik |
Open Access?: | ja |
Licence (German): | Creative Commons - Namensnennung-Nicht kommerziell-Keine Bearbeitung 3.0 Deutschland |