* [sup-talk] Too many clauses @ 2007-12-03 19:52 Dennis Griffith 2007-12-09 19:44 ` William Morgan 0 siblings, 1 reply; 4+ messages in thread From: Dennis Griffith @ 2007-12-03 19:52 UTC (permalink / raw) I started getting this crash upon start after updating to 0.3 from 0.1. -- Dennis Griffith -------------- next part -------------- --- Ferret::StateError from thread: load threads for thread-index-mode State Error occured at <except.c>:93 in xraise Error occured in q_boolean.c:1595 - bq_add_query_nr Two many clauses. The max clause limit is set to <1024> but your query has <1024> clauses. You can try increasing :max_clause_count for the BooleanQuery or using a different type of query. /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/index.rb:270:in `add_query' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/index.rb:270:in `each_message_in_thread_for' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/index.rb:268:in `each' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/index.rb:268:in `each_message_in_thread_for' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/thread.rb:321:in `load_thread_for_message' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/thread.rb:313:in `load_n_threads' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/index.rb:217:in `each_id_by_date' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/index.rb:217:in `each' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/index.rb:217:in `each_id_by_date' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/thread.rb:308:in `load_n_threads' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/modes/thread-index-mode.rb:415:in `__unprotected_load_n_threads' (eval):12:in `load_n_threads' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/modes/thread-index-mode.rb:404:in `load_n_threads_background' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup.rb:69:in `reporting_thread' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup.rb:67:in `initialize' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup.rb:67:in `new' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup.rb:67:in `reporting_thread' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/modes/thread-index-mode.rb:403:in `load_n_threads_background' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/modes/thread-index-mode.rb:454:in `__unprotected_load_threads' (eval):12:in `load_threads' /usr/lib/ruby/gems/1.8/gems/sup-0.3/bin/sup:185 /usr/bin/sup:16:in `load' /usr/bin/sup:16 --- SystemExit from thread: main State Error occured at <except.c>:93 in xraise Error occured in q_boolean.c:1595 - bq_add_query_nr Two many clauses. The max clause limit is set to <1024> but your query has <1024> clauses. You can try increasing :max_clause_count for the BooleanQuery or using a different type of query. /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup.rb:73:in `select' /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/buffer.rb:31:in `nonblocking_getch' /usr/lib/ruby/gems/1.8/gems/sup-0.3/bin/sup:198 /usr/bin/sup:16:in `load' /usr/bin/sup:16 ^ permalink raw reply [flat|nested] 4+ messages in thread
* [sup-talk] Too many clauses 2007-12-03 19:52 [sup-talk] Too many clauses Dennis Griffith @ 2007-12-09 19:44 ` William Morgan 2007-12-10 1:31 ` Dennis Griffith 0 siblings, 1 reply; 4+ messages in thread From: William Morgan @ 2007-12-09 19:44 UTC (permalink / raw) Excerpts from Dennis Griffith's message of Mon Dec 03 11:52:38 -0800 2007: > I started getting this crash upon start after updating to 0.3 from 0.1. > --- Ferret::StateError from thread: load threads for thread-index-mode > State Error occured at <except.c>:93 in xraise > Error occured in q_boolean.c:1595 - bq_add_query_nr > Two many clauses. The max clause limit is set to <1024> but your query has <1024> clauses. You can try increasing :max_clause_count for the BooleanQuery or using a different type of query. > > /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/index.rb:270:in `add_query' At first, I thought, "Great! An error message that contains its own solution!" But the max_clause_count thing seems to have disappeared in recent versions of Ferret. Can you try this patch, please? --- lib/sup/index.rb (revision 748) +++ lib/sup/index.rb (working copy) @@ -236,6 +236,7 @@ ## true, stops loading any thread if a message with a :killed flag ## is found. SAME_SUBJECT_DATE_LIMIT = 7 + MAX_CLAUSES = 1000 def each_message_in_thread_for m, opts={} #Redwood::log "Building thread for #{m.id}: #{m.subj}" messages = {} @@ -264,13 +265,16 @@ until pending.empty? || (opts[:limit] && messages.size >= opts[:limit]) q = Ferret::Search::BooleanQuery.new true + # this disappeared in newer ferrets... wtf. + # q.max_clause_count = 2048 - pending.each do |id| + lim = [MAX_CLAUSES / 2, pending.length].min + pending[0 ... lim].each do |id| searched[id] = true q.add_query Ferret::Search::TermQuery.new(:message_id, id), :should q.add_query Ferret::Search::TermQuery.new(:refs, id), :should end - pending = [] + pending = pending[lim .. -1] q = build_query :qobj => q Thanks, -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 4+ messages in thread
* [sup-talk] Too many clauses 2007-12-09 19:44 ` William Morgan @ 2007-12-10 1:31 ` Dennis Griffith 2007-12-10 2:54 ` William Morgan 0 siblings, 1 reply; 4+ messages in thread From: Dennis Griffith @ 2007-12-10 1:31 UTC (permalink / raw) Excerpts from William Morgan's message of Sun Dec 09 13:44:43 -0600 2007: > Excerpts from Dennis Griffith's message of Mon Dec 03 11:52:38 -0800 2007: > > I started getting this crash upon start after updating to 0.3 from 0.1. > > --- Ferret::StateError from thread: load threads for thread-index-mode > > State Error occured at <except.c>:93 in xraise > > Error occured in q_boolean.c:1595 - bq_add_query_nr > > Two many clauses. The max clause limit is set to <1024> but your query has <1024> clauses. You can try increasing :max_clause_count for the BooleanQuery or using a different type of query. > > > > /usr/lib/ruby/gems/1.8/gems/sup-0.3/lib/sup/index.rb:270:in `add_query' > > At first, I thought, "Great! An error message that contains its own > solution!" But the max_clause_count thing seems to have disappeared in > recent versions of Ferret. I thought that too and tried to fix it. However not knowing hardly any ruby proved a bit of a hinderence. > Can you try this patch, please? I could, although I wouldn't be able to tell you if it works anymore. After installing 0.3 again, in preparation for trying the patch, I found I no longer hit the problem. I haven't updated ferret during this time, so perhaps my message count has fallen low enough not to trigger the problem, if I hit this again I will try the patch and let you know. -- Dennis Griffith ^ permalink raw reply [flat|nested] 4+ messages in thread
* [sup-talk] Too many clauses 2007-12-10 1:31 ` Dennis Griffith @ 2007-12-10 2:54 ` William Morgan 0 siblings, 0 replies; 4+ messages in thread From: William Morgan @ 2007-12-10 2:54 UTC (permalink / raw) Excerpts from Dennis Griffith's message of Sun Dec 09 17:31:23 -0800 2007: > I could, although I wouldn't be able to tell you if it works anymore. > After installing 0.3 again, in preparation for trying the patch, I > found I no longer hit the problem. I haven't updated ferret during > this time, so perhaps my message count has fallen low enough not to > trigger the problem, if I hit this again I will try the patch and let > you know. Weird. I think this problem surfaces with very large threads, so it shouldn't have anything to do with raw message count. But all's well that ends well. I've committed the patch to SVN anyways because it's probably the right thing to do. (And is a better solution than just increasing the Ferret thingy.) -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2007-12-10 2:54 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2007-12-03 19:52 [sup-talk] Too many clauses Dennis Griffith 2007-12-09 19:44 ` William Morgan 2007-12-10 1:31 ` Dennis Griffith 2007-12-10 2:54 ` William Morgan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox