Archive of RubyForge sup-talk mailing list
 help / color / mirror / Atom feed
* [sup-talk] [PATCH] xapian: drop excessively long terms
@ 2009-08-01 22:56 Rich Lane
  2009-08-05 13:55 ` William Morgan
  0 siblings, 1 reply; 9+ messages in thread
From: Rich Lane @ 2009-08-01 22:56 UTC (permalink / raw)


---
 lib/sup/xapian_index.rb |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/lib/sup/xapian_index.rb b/lib/sup/xapian_index.rb
index 6358a20..5a5dfc1 100644
--- a/lib/sup/xapian_index.rb
+++ b/lib/sup/xapian_index.rb
@@ -304,6 +304,8 @@ class XapianIndex < BaseIndex
 
   DATE_VALUENO = 0
 
+  MAX_TERM_LENGTH = 245
+
   # Xapian can very efficiently sort in ascending docid order. Sup always wants
   # to sort by descending date, so this method maps between them. In order to
   # handle multiple messages per second, we use a logistic curve centered
@@ -428,7 +430,7 @@ class XapianIndex < BaseIndex
 
     @term_generator.document = doc
     text.each { |text,prefix| @term_generator.index_text text, 1, prefix }
-    terms.each { |term| doc.add_term term }
+    terms.each { |term| doc.add_term term if term.length <= MAX_TERM_LENGTH }
     doc.add_value DATE_VALUENO, date_value
     doc.data = m.id
 
-- 
1.6.0.4



^ permalink raw reply	[flat|nested] 9+ messages in thread
end of thread, other threads:[~2009-08-05 18:06 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-08-01 22:56 [sup-talk] [PATCH] xapian: drop excessively long terms Rich Lane
2009-08-05 13:55 ` William Morgan
2009-08-05 14:54   ` Edward Z. Yang
2009-08-05 15:36   ` Rich Lane
2009-08-05 15:48     ` William Morgan
2009-08-05 17:29     ` Nicolas Pouillard
2009-08-05 17:48       ` Rich Lane
2009-08-05 17:58         ` Nicolas Pouillard
2009-08-05 18:06           ` Rich Lane

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox