TY  - JOUR
U1  - Wissenschaftlicher Artikel
A1  - Wolfer, Sascha
A1  - Koplenig, Alexander
A1  - Kupietz, Marc
A1  - Müller-Spitzer, Carolin
T1  - Introducing DeReKoGram: A novel frequency dataset with lemma and part-of-speech information for German
JF  - Data
N2  - We introduce DeReKoGram, a novel frequency dataset containing lemma and part-of-speech (POS) information for 1-, 2-, and 3-grams from the German Reference Corpus. The dataset contains information based on a corpus of 43.2 billion tokens and is divided into 16 parts based on 16 corpus folds. We describe how the dataset was created and structured. By evaluating the distribution over the 16 folds, we show that it is possible to work with a subset of the folds in many use cases (e.g., to save computational resources). In a case study, we investigate the growth of vocabulary (as well as the number of hapax legomena) as an increasing number of folds are included in the analysis. We cross-combine this with the various cleaning stages of the dataset. We also give some guidance in the form of Python, R, and Stata markdown scripts on how to work with the resource.
KW  - language
KW  - n-grams
KW  - corpus frequency
KW  - dataset
KW  - German
KW  - vocabulary growth
KW  - Wortschatz
KW  - Korpus <Linguistik>
KW  - Deutsch
KW  - Lemma
KW  - Sprachdaten
KW  - Datenanalyse
Y1  - 2023
UN  - https://nbn-resolving.org/urn:nbn:de:bsz:mh39-122939
U6  - https://doi.org/10.3390/data8110170
DO  - https://doi.org/10.3390/data8110170
N1  - Gefördert durch den Zeitschriftenfonds der Leibniz-Gemeinschaft
VL  - 8
IS  - 11
SP  - 10
S1  - 10
PB  - MDPI
CY  - Basel
ER  -