Refine
Document Type
- Working Paper (3)
- Part of a Book (1)
- Conference Proceeding (1)
- Part of Periodical (1)
Has Fulltext
- yes (6)
Is part of the Bibliography
- yes (6) (remove)
Keywords
Publicationstate
Reviewstate
- Peer-Review (2)
- (Verlags)-Lektorat (1)
Beyond Citations: Corpus-based Methods for Detecting the Impact of Research Outcomes on Society
(2020)
This paper proposes, implements and evaluates a novel, corpus-based approach for identifying categories indicative of the impact of research via a deductive (top-down, from theory to data) and an inductive (bottom-up, from data to theory) approach. The resulting categorization schemes differ in substance. Research outcomes are typically assessed by using bibliometric methods, such as citation counts and patterns, or alternative metrics, such as references to research in the media. Shortcomings with these methods are their inability to identify impact of research beyond academia (bibliometrics) and considering text-based impact indicators beyond those that capture attention (altmetrics). We address these limitations by leveraging a mixed-methods approach for eliciting impact categories from experts, project personnel (deductive) and texts (inductive). Using these categories, we label a corpus of project reports per category schema, and apply supervised machine learning to infer these categories from project reports. The classification results show that we can predict deductively and inductively derived impact categories with 76.39% and 78.81% accuracy (F1-score), respectively. Our approach can complement solutions from bibliometrics and scientometrics for assessing the impact of research and studying the scope and types of advancements transferred from academia to society.
Mit diesem Papier wird die neue Online-Reihe IDSopen des Leibniz-Instituts für Deutsche Sprache konzeptuell aufgelegt. Die Reihe bietet Autor/-innen und Rezipient/-innen aus allen Bereichen der Linguistik eine moderne und offene Plattform für digitales Publizieren. Mit IDSopen steht eine zeitgemäße Publikationsumgebung zur Verfügung, die schwerpunktmäßig Arbeiten veröffentlicht, die auf Ressourcen des IDS beruhen und deren Verwendungsmöglichkeiten in besonderem Maße zeigen. Gleichzeitig zeichnet sich IDSopen durch eine Öffnung für unkonventionelle Publikationsformen und -formate aus. Transparente Begutachtungsprozesse gehören dabei genauso zum Profil der Reihe wie ein offener Erscheinungsturnus und das Ansprechen unterschiedlicher Zielgruppen. IDSopen verfolgt entlang der Leitlinien des IDS und der Leibniz-Gemeinschaft (vgl. LeibnizOpen) das Open-Access-Prinzip und veröffentlicht ausschließlich digital, ohne gedruckte Form (Online-only). Diese Maßnahmen haben das Ziel, kurze Veröffentlichungszeiten für Manuskripte zu ermöglichen, einen unbeschränkten und kostenlosen Zugang zu qualitäts-geprüfter wissenschaftlicher Information rund um die IDS-Ressourcen im Internet zu bieten und liquide Publikationsprozesse zu unterstützen.
The actual or anticipated impact of research projects can be documented in scientific publications and project reports. While project reports are available at varying level of accessibility, they might be rarely used or shared outside of academia. Moreover, a connection between outcomes of actual research project and potential secondary use might not be explicated in a project report. This paper outlines two methods for classifying and extracting the impact of publicly funded research projects. The first method is concerned with identifying impact categories and assigning these categories to research projects and their reports by extension by using subject matter experts; not considering the content of research reports. This process resulted in a classification schema that we describe in this paper. With the second method which is still work in progress, impact categories are extracted from the actual text data.