I am creating a search engine, with the corpus consisting of websites crawled through a webcrawler (Apache Nutch). I need the query searches to be both fast and relevant. So far, I have been trying to develop a Latent Semantic Indexing system, but I recently read an article that told me that LSA had not shown to be useful to them. Basically, LSA uses a k-rank approximation of the singular value decomposition of a matrix whose dimensions are the documents and a weighted term frequency (IDF, TF-IDF, Binary, and Raw).
Does anyone know a much better data-mining algorithm that I could use? Through your own personal experience? I only have about 2 weeks left for this project, and I need to make sure I can end it right.