Distributed Web 2.0 Crawling for Ontology Evolution
Andreas Juffinger, Thomas Neidhart,Michael Granitzer, Roman Kern,Albert Weichselbraun, Gerhard Wohlgenannt,Arno Scharl
Journal of Digital Information Management, Vol. 7, No. 2, 2009.
DOI: 
Link: http://www.dirf.org/jdim/v7n2a7.asp
Abstract:
The World Wide Web as a social network reflects changes of interest in certain domains. It has been shown that free online content available through blogs, wikis, news media and online forums is a valuable source of information to identify trends in certain domains. Utilizing this data, one can construct ontologies that describe this information and provide a semantically correct overview of a domain. Tracked over time this also enables a user to identify trends and hypes. The decentralised structure of the Internet, the huge amount of data and upcoming Web2.0 technologies pose several challenges to a crawling system for ontology learning, evolution and trend analysis. This paper presents a distributed crawling system with browser integration for Web2.0. The proposed crawler is a high performanceWeb data retrieval system aimed to gather browser-equivalent textual Web content and prepare it for ontology learning.