Papers in Innovation Studies,
Lund University, CIRCLE - Centre for Innovation Research

Abstract: The emergence of large language models and the exponential growth of digitized text data have revolutionized research methodologies across a broad range of social sciences. News articles are an important source of digitized text data in this context. News data is crucial for the social sciences as it provides real-time insights into public discourse and societal trends, helping to understand various social phenomena and dynamics. However, most research involving news data is conducted at the national level, as geographically more granular news data is often unavailable. In this paper, we address this gap by providing insights into how news articles can be geolocated and how the texts can then be further analyzed. More specifically, we collect data from the CommonCrawl News dataset and clean the text data for further analysis. We then use a named-entity recognition model for geocoding, linking news articles to geographic locations. Finally, we transform the news articles into text embeddings using SBERT, enabling semantic searches within the news data corpus. In the paper, we apply this process to all German news articles and make the German location data, as well as the embeddings, available for download. As a result, we compile a dataset containing text embeddings for about 50 million German news articles, of which about 70% include geographic locations. The process can be replicated for news data from other countries, as we provide all code and workflows.

Keywords: News data; Natural language processing; Geography

JEL-codes: C45; O33; R12

Language: English

18 pages, September 25, 2026

Note: We acknowledge financial support by the German Federal Ministry of Education and Research (BMBF 031B1281).

Full text files

24091  Full text

Download statistics

• Lund University, CIRCLE - Centre for Innovation Research
• Home page for this series

Questions (including download problems) about the papers in this series should be directed to Torben Schubert ()
Report other problems with accessing this service to Sune Karlsson ().

RePEc:hhs:lucirc:2025_004This page generated on 2026-09-25 17:08:57.