Volltext-Downloads (blau) und Frontdoor-Views (grau)

Is it worth the effort? Assessing the benefits of partial automatic pre-labeling for frame-semantic annotation

  • Corpora with high-quality linguistic annotations are an essential component in many NLP applications and a valuable resource for linguistic research. For obtaining these annotations, a large amount of manual effort is needed, making the creation of these resources time-consuming and costly. One attempt to speed up the annotation process is to use supervised machine-learning systems to automatically assign (possibly erroneous) labels to the data and ask human annotators to correct them where necessary. However, it is not clear to what extent these automatic pre-annotations are successful in reducing human annotation effort, and what impact they have on the quality of the resulting resource. In this article, we present the results of an experiment in which we assess the usefulness of partial semi-automatic annotation for frame labeling. We investigate the impact of automatic pre-annotation of differing quality on annotation time, consistency and accuracy. While we found no conclusive evidence that it can speed up human annotation, we found that automatic pre-annotation does increase its overall quality.

Download full text files

Export metadata

Additional Services

Share in Twitter Search Google Scholar

Statistics

frontdoor_oas
Metadaten
Author:Ines Rehbein, Josef RuppenhoferGND, Caroline Sporleder
URN:urn:nbn:de:bsz:mh39-49750
DOI:https://doi.org/10.1007/s10579-011-9170-z
ISSN:1574-020X
Parent Title (English):Language Resources and Evaluation
Publisher:Springer
Place of publication:Heidelberg/New York
Document Type:Article
Language:English
Year of first Publication:2012
Date of Publication (online):2016/06/15
Publicationstate:Veröffentlichungsversion
Reviewstate:(Verlags)-Lektorat
Tag:Frame semantics; Linguistic annotation; Semantic role labelling; Semi-automatic annotation
Volume:46
Issue:1
First Page:1
Last Page:23
Dewey Decimal Classification:400 Sprache / 410 Linguistik
Open Access?:ja
BDSL-Classification:Textwissenschaft
Leibniz-Classification:Sprache, Linguistik
Linguistics-Classification:Computerlinguistik
Licence (German):Es gilt das UrhG