Abstract With the rapid development of the Internet, the advantages of Web text in scalability and diversity play an important role in English research and teaching. This paper presents a N-Grams based English search engine for words in context, which incorporate information retrieval, part-of-speech, named entity recognition, word similarity and other natural language processing technology with web based text data. The engine... With the rapid development of the Internet, the advantages of Web text in scalability and diversity play an important role in English research and teaching. This paper presents a N-Grams based English search engine for words in context, which incorporate information retrieval, part-of-speech, named entity recognition, word similarity and other natural language processing technology with web based text data. The engine retrieves N-Grams in response to a given query, which might contain keywords, wildcards, wild parts of speech (PoS), name entity information, synonyms, and other regular expression operators. On the side of word similarity measures, this paper proposes a method of automatically extracting similar words in corpus, which use a deep learning based tool, Word2Vec. We can train word representations to measure the words similarity and test on WBST (WordNet-based Synonymy Test); our result was comparable with the result in Freitag et al. (2005) [2]. Then, we can automatically extract the similar words and embedded them in our search engine. Read More Read Less

Abstract Kernel Methods are known for their robustness in handling large feature space and are widely used as an alternative to external feature extraction based methods in tasks such as classification and regression. This work follows the approach of using different string kernels such as n-gram kernels and gappy-n-gram kernels on text classification. It studies how kernel concatenation and feature combination... Kernel Methods are known for their robustness in handling large feature space and are widely used as an alternative to external feature extraction based methods in tasks such as classification and regression. This work follows the approach of using different string kernels such as n-gram kernels and gappy-n-gram kernels on text classification. It studies how kernel concatenation and feature combination affects the classification accuracy of the system. It also explores how the kernel combination algorithms work on the system. The kernels are implemented as rational kernels, which satisfies the Mercer’s Theorem ensuring the kernel matrices to be positive definite symmetric. The rational kernels are computed with a general algorithm of composition of weighted transducers which help in dealing with variable length sequences. These kernels are then used with SVM formulating efficient classifier for text categorization. Both one-stage and two stage algorithms are applied for kernel combination which were successful in achieving better system performance compared to that given by individual kernels. Read More Read Less

Abstract This paper describes the initial steps in the project of building a prototype of a free/open-source rule-based machine translation system that translates from English to Kazakh. The goal of this article is to examine the grammatical and lexical problems, which we often face while translating English texts, and during the translation process there is no detailed statement of grammatical or lexical phenomenon... This paper describes the initial steps in the project of building a prototype of a free/open-source rule-based machine translation system that translates from English to Kazakh. The goal of this article is to examine the grammatical and lexical problems, which we often face while translating English texts, and during the translation process there is no detailed statement of grammatical or lexical phenomenon available. Apertium platform translates many different languages and it uses Hidden Markov Models. This article shows description of machine translation which translates English to Kazakh. Read More Read Less

Abstract In this paper we proposed a framework for integrating text mining with E-Governance. We suggested that the users of electronic governance can use the text terms to describe their interest which can be processed for clustering and term extraction. The words thus expressed by users are tracked and subjected to processing wherein it is possible to generate content. We have provided the... In this paper we proposed a framework for integrating text mining with E-Governance. We suggested that the users of electronic governance can use the text terms to describe their interest which can be processed for clustering and term extraction. The words thus expressed by users are tracked and subjected to processing wherein it is possible to generate content. We have provided the framework and tested it in a few web sites. We have used the clustering and pre-processing for the content management. The results are encouraging and it is possible to extent such exercises for other text minging processes. Read More Read Less