A Survey of Focused Web Crawling Approaches
Sameendra Samarawickrama, Lakshman Jayaratne
Journal of Information Organization, Vol. 2, No. 1, 2012.
DOI: 
Link: http://www.dline.info/jio/fulltext/v1n1/1.pdf
Abstract:
In the last few years the volume and size of web has been growing exponentially making it difficult to search the related pieces of information. To ease such problem, we have various general as well as focused crawlers. These two types of crawlers have their own features and limitations. In this paper we provide an extensive survey of the available general and focused crawlers. Besides we support our work with experimental results. Additionally we provide classification of the crawlers based on the parameter, the text analysis. Based on this survey we also project the near future developments in crawler research.