Abstract Unsupervised learning plays an important role in the knowlede exploration discovery. The basic task of unsupervised learning is to find latent variablesor relationships in a given dataset wihout any assumed regularities or patterns. In this paper we apply two advanced models, clustering analysis and hierarchial analysis to accomplish unsupervised learning. K-Means clustering presents its strength in large scale clustering. The... Unsupervised learning plays an important role in the knowlede exploration discovery. The basic task of unsupervised learning is to find latent variablesor relationships in a given dataset wihout any assumed regularities or patterns. In this paper we apply two advanced models, clustering analysis and hierarchial analysis to accomplish unsupervised learning. K-Means clustering presents its strength in large scale clustering. The original data can be preprocessed and the potential variables are targeted. Correlations among these variables are explored in the subsequent sets by Local Global Hierarchial Analysis (LGHA) assisted by three main steps. In the first step, we use a structural approach to find qualititative patterns from the given variables. Then, the second step applies a quantitative based algorithm to find quantitative patterns from those variables. The and last step generated global hybrid patterns by combining the local patterns obtained from the first two steps based on a certain criterion. Both of the K-Means and Local Global Hierarchial Analysis (LGHA) models are applied in experiments with real world longitutional medical datasets. Read More Read Less

Abstract Many efforts have been made for compression of web graphs. Most of these methods are suitable for search engines and are centered around encoding links and URLs efficiently. The purpose is to handle a large set of web pages in the main memory against any web based search. The authors of the present paper are interested in studying web graph... Many efforts have been made for compression of web graphs. Most of these methods are suitable for search engines and are centered around encoding links and URLs efficiently. The purpose is to handle a large set of web pages in the main memory against any web based search. The authors of the present paper are interested in studying web graph as a social network and to develop a data model for it. So, suitable compression techniques, which are space efficient for disk based storage, are required. This paper has provided a two level compression technique. In the first level, the structural properties of a graph are studied and strongly connected components are fused to reduce the original graph to a DAG. Paths on this DAG are then stored efficiently using a Path Normalization technique. Space complexity expressions indicate the efficiency of the method. Relevant operators required for accessing the original graph through the compressed representations have also been discussed. Important earlier works have been referred to indicate the requirements of the present approach. Read More Read Less

Abstract In this paper we present the Infocious Web search engine [23], which currently indexes more than 2 billion pages collected from the Web. The main goal of Infocious is to enhance the way that people find relevant information on the Web by resolving ambiguities present in natural language text. Towards this goal, Infocious performs linguistic analysis to the content of... In this paper we present the Infocious Web search engine [23], which currently indexes more than 2 billion pages collected from the Web. The main goal of Infocious is to enhance the way that people find relevant information on the Web by resolving ambiguities present in natural language text. Towards this goal, Infocious performs linguistic analysis to the content of the Web pages prior to indexing and exploits the output of this analysis when ranking and presenting the results to the users. Our hope is that this additional step of linguistic processing provides Infocious with two main advantages. First, Infocious tries to gain a deeper understanding of the content of Web pages and to match the users’ queries with the indexed documents better, improving the relevancy of the returned results. Second, based on its linguistic processing, Infocious tries to organize and present the results to the users in a structured and more intuitive way. In this paper we present the linguistic processing technologies that we investigated and/or incorporated into the Infocious search engine, and we discuss the main challenges in applying these technologies to Web documents. We also present the various components in the architecture of Infocious, and how each one of these components benefits from the added linguistic processing. Finally, we present preliminary results from our experimental study that evaluates the effectiveness of the described linguistic analysis. Read More Read Less

Abstract The rank aggregation problem has been studied extensively in recent years with a focus on how to combine several different rankers to obtain a consensus aggregate ranker. We study the rank aggregation problem from a different perspective: how the individual input rankers impact the performance of the aggregate ranker. We develop a general statistical framework based on a model of... The rank aggregation problem has been studied extensively in recent years with a focus on how to combine several different rankers to obtain a consensus aggregate ranker. We study the rank aggregation problem from a different perspective: how the individual input rankers impact the performance of the aggregate ranker. We develop a general statistical framework based on a model of how the individual rankers depend on the ground truth ranker. Within this framework, one can generate synthetic data sets and study the performance of different aggregation methods. The individual rankers, which are the inputs to the rank aggregation algorithm, are statistical perturbations of the ground truth ranker. With rigorous experimental evaluation, we study how noise level and the misinformation of the rankers affect the performance of the aggregate ranker. We introduce and study a novel Kendall-tau rank aggregator and a simple aggregator called PrOpt, which we compare to some other well known rank aggregation algorithms such as average, median, CombMNZ and Markov chain aggregators. Our results show that the relative performance of aggregators varies considerably depending on how the input rankers relate to the ground truth. Read More Read Less

Abstract The amount of information available on the Web and in Digital Libraries is increasing over time. In this context, the role of user modeling and personalized information access is becoming crucial: Users need a personalized support in sifting through large amounts of retrieved information according to their interests. Information filtering and retrieval systems relying on this idea adapt their behavior... The amount of information available on the Web and in Digital Libraries is increasing over time. In this context, the role of user modeling and personalized information access is becoming crucial: Users need a personalized support in sifting through large amounts of retrieved information according to their interests. Information filtering and retrieval systems relying on this idea adapt their behavior to individual users by learning their preferences during the interaction in order to construct a profile of the user that can be later exploited in the search process. We propose a novel technique to learn user profiles which exploits word sense disambiguation based on the WordNet lexical database, in an attempt to produce semantic user profiles that might discover topics semantically closer to the user interests. Semantic profiles are used in the definition of a retrieval model that turns the traditional document-query search paradigm into a novel document-query-profile paradigm. As an example of this paradigm, we present an extension of the vector space model in which profiles are used to modify the ranking of search results obtained in response to a query, hopefully putting personally relevant items on the top of the result list. Experimental results in a movie retrieval scenario indicate that the proposed model to personalize Web search is effective. Read More Read Less

Abstract Web Site Summarization is the process of automatically generating a concise and informative summary for a given Web site. It has gained more and more attention in recent years as effective summarization could lead to enhanced Web information retrieval systems such as searching for Web sites. Extraction-based approaches to Web site summarization rely on the extraction of the most significant... Web Site Summarization is the process of automatically generating a concise and informative summary for a given Web site. It has gained more and more attention in recent years as effective summarization could lead to enhanced Web information retrieval systems such as searching for Web sites. Extraction-based approaches to Web site summarization rely on the extraction of the most significant sentences from the target Web site based on the density of a list of key phrases that best describe the entire Web site. In this work, we benchmark five alternative key phrase extraction methods, TFIDF, KEA, Keyword, Keyterm, and Mixture, in an automatic Web site summarization framework we previously developed. We investigate the performance of these underlying methods via a formal user study and demonstrate that Keyterm is the best choice for key phrase extraction while Mixture should be used to obtain key sentences. We also discuss why one method performs better than another and what could be done to further improve the summarization system. Read More Read Less