Abstract This article evaluates the performance of two techniques for query reformulation in a system for information retrieval, namely, the concept based and the pseudo relevance feedback reformulation. The experiments performed on a corpus of Arabic text have allowed us to compare the contribution of these two reformulation techniques in improving the performance of an information retrieval system for Arabic texts.... This article evaluates the performance of two techniques for query reformulation in a system for information retrieval, namely, the concept based and the pseudo relevance feedback reformulation. The experiments performed on a corpus of Arabic text have allowed us to compare the contribution of these two reformulation techniques in improving the performance of an information retrieval system for Arabic texts. Read More Read Less

Abstract Non Standard Word (NSW) identification and its interpretation is a major challenge in information retrieval system. Real text not only contains ordinary words and names but also contains non-standard “words” including numbers, abbreviations, dates, months, currency, amounts and different types of numbers. In reality, we can not find NSWs in a dictionary, nor can one find their pronunciation by an... Non Standard Word (NSW) identification and its interpretation is a major challenge in information retrieval system. Real text not only contains ordinary words and names but also contains non-standard “words” including numbers, abbreviations, dates, months, currency, amounts and different types of numbers. In reality, we can not find NSWs in a dictionary, nor can one find their pronunciation by an application of ordinary “letter-to-sound” rules. Non standard words have greater inclination towards ambiguity in terms of their interpretation and pronunciation than ordinary words. So it is very much required to identify and interpret the NSW properly while we are going for further analysis of text. In real text, NSW are represented in diversified formats. It could be represented by digits, words or combination of both. There are different ways to identify and interpret the NSW such as n-gram language model, decision tree, supervised and unsupervised techniques, weighted finite-state transducers and pattern identification using regular expression. In this paper we presented a novel work to identify and interpret the non standard words in Bengali news corpus. We introduced the coupling of the normal text normalization using the three components of analysis, viz. (i) generation of optimized regular expression for different semiotic classes, (ii) consideration of optimized context window size, and (iii) employ the concept of context identification array (CIA). Read More Read Less

Abstract Name matching between multiple natural languages is an important step in cross-enterprise integration applications and data mining. It is difficult to decide whether or not two syntactic values (names) from two heterogeneous data sources are alternative designation of the same semantic entity (person), this process becomes more difficult with Arabic language due to several factors including spelling and pronunciation variation,... Name matching between multiple natural languages is an important step in cross-enterprise integration applications and data mining. It is difficult to decide whether or not two syntactic values (names) from two heterogeneous data sources are alternative designation of the same semantic entity (person), this process becomes more difficult with Arabic language due to several factors including spelling and pronunciation variation, dialects and special vowel and consonants distinction and other linguistic characteristics. This paper proposes a new framework for name matching between Arabic language and other languages. The framework uses a dictionary based on a new proposed version of the Soundex algorithm to encapsulate the recognition of special features of Arabic names. The framework proposes new proximity matching algorithm to suit the high importance of order sensitivity in Arabic name matching. New performance evaluation metrics are proposed as well. The framework is implemented and verified empirically in several case studies demonstrating substantial improvements compared to other well-known techniques found in literature. Read More Read Less