Parsimonious translation models for information retrieval
SCIE
SCOPUS
- Title
- Parsimonious translation models for information retrieval
- Authors
- Na, SH; Kang, IS; Lee, JH
- Date Issued
- 2007-01
- Publisher
- PERGAMON-ELSEVIER SCIENCE LTD
- Abstract
- In the KL divergence framework, the extended language modeling approach has a critical problem of estimating a query model, which is the probabilistic model that encodes the user's information need. For query expansion in initial retrieval, the translation model had been proposed to involve term co-occurrence statistics. However, the translation model was difficult to apply, because the term co-occurrence statistics must be constructed in the offline time. Especially in a large collection, constructing such a large matrix of term co-occurrences statistics prohibitively increases time and space complexity. In addition, reliable retrieval performance cannot be guaranteed because the translation model may comprise noisy non-topical terms in documents. To resolve these problems, this paper investigates an effective method to construct co-occurrence statistics and eliminate noisy terms by employing a parsimonious translation model. The parsimonious translation model is a compact version of a translation model that can reduce the number of terms containing non-zero probabilities by eliminating non-topical terms in documents. Through experimentation on seven different test collections, we show that the query model estimated from the parsimonious translation model significantly outperforms not only the baseline language modeling, but also the non-parsimonious models. (c) 2006 Elsevier Ltd. All rights reserved.
- Keywords
- information retrieval; language model; parsimonious translation model; query expansion
- URI
- https://oasis.postech.ac.kr/handle/2014.oak/23752
- DOI
- 10.1016/j.ipm.2006.04.005
- ISSN
- 0306-4573
- Article Type
- Article
- Citation
- INFORMATION PROCESSING & MANAGEMENT, vol. 43, no. 1, page. 121 - 145, 2007-01
- Files in This Item:
- There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.