Design and application of a Gold Standard for morphological analysis: SMOR as an example of morphological evaluation
- This paper describes general requirements for evaluating and documenting NLP tools with a focus on morphological analysers and the design of a Gold Standard. It is argued that any evaluation must be measurable and documentation thereof must be made accessible for any user of the tool. The documentation must be of a kind that it enables the user to compare different tools offering the same service, hence the descriptions must contain measurable values. A Gold Standard presents a vital part of any measurable evaluation process, therefore, the corpus-based design of a Gold Standard, its creation and problems that occur are reported upon here. Our project concentrates on SMOR, a morphological analyser for German that is to be offered as a web-service. We not only utilize this analyser for designing the Gold Standard, but also evaluate the tool itself at the same time. Note that the project is ongoing, therefore, we cannot present final results.
Download full text files
Additional Services
Statistics
| Author: | Gertrud FaaßORCiD, Ulrich HeidORCiDGND, Helmut SchmidORCiD |
|---|---|
| URN: | urn:nbn:de:bsz:mh39-118764 |
| URL: | http://www.lrec-conf.org/proceedings/lrec2010/pdf/409_Paper.pdf |
| ISBN: | 2-9517408-6-7 |
| Parent Title (English): | LREC 2010, Seventh International Conference on Language Resources and Evaluation |
| Publisher: | European Language Resources Association |
| Place of publication: | Luxemburg |
| Editor: | Nicoletta Calzolari, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Mike Rosner, Daniel Tapias |
| Document Type: | Conference Proceeding |
| Language: | English |
| Year of first Publication: | 2010 |
| Date of Publication (online): | 2023/05/31 |
| Publishing Institution: | Leibniz-Institut für Deutsche Sprache (IDS) |
| Publicationstate: | Veröffentlichungsversion |
| Reviewstate: | Peer-Review |
| Tag: | Evaluation methodologies; Morphology; Usability; user satisfication |
| GND Keyword: | Deutsch; Korpus <Linguistik>; Morphologie <Linguistik>; Sprachanalyse; Web Services |
| First Page: | 803 |
| Last Page: | 810 |
| DDC classes: | 400 Sprache / 400 Sprache, Linguistik |
| Open Access?: | ja |
| Linguistics-Classification: | Computerlinguistik |
| Licence (German): |  Urheberrechtlich geschützt |