TY  - JOUR
U1  - Zeitschriftenartikel, wissenschaftlich - begutachtet (reviewed)
A1  - Rehbein, Ines
A1  - Ruppenhofer, Josef
A1  - Sporleder, Caroline
T1  - Is it worth the effort? Assessing the benefits of partial automatic pre-labeling for frame-semantic annotation
JF  - Language Resources and Evaluation
N2  - Corpora with high-quality linguistic annotations are an essential component in many NLP applications and a valuable resource for linguistic research. For obtaining these annotations, a large amount of manual effort is needed, making the creation of these resources time-consuming and costly. One attempt to speed up the annotation process is to use supervised machine-learning systems to automatically assign (possibly erroneous) labels to the data and ask human annotators to correct them where necessary. However, it is not clear to what extent these automatic pre-annotations are successful in reducing human annotation effort, and what impact they have on the quality of the resulting resource. In this article, we present the results of an experiment in which we assess the usefulness of partial semi-automatic annotation for frame labeling. We investigate the impact of automatic pre-annotation of differing quality on annotation time, consistency and accuracy. While we found no conclusive evidence that it can speed up human annotation, we found that automatic pre-annotation does increase its overall quality.
KW  - Linguistic annotation
KW  - Semantic role labelling
KW  - Frame semantics
KW  - Semi-automatic annotation
Y1  - 2012
UN  - https://nbn-resolving.org/urn:nbn:de:bsz:mh39-49750
SN  - 1574-020X
SS  - 1574-020X
U6  - https://doi.org/10.1007/s10579-011-9170-z
DO  - https://doi.org/10.1007/s10579-011-9170-z
VL  - 46
IS  - 1
SP  - 1
EP  - 23
PB  - Springer
CY  - Heidelberg/New York
ER  -