Subject: [sup-talk] More sup errors...

From: [email-masked] (Nicolas Pouillard)
Date: Mon, 19 Nov 2007 14:53:33 +0100

Sorry to came up with some new backtraces...

I've saved some more mails and get this kind of errors:

The problem was: ':20492 is out of range [0..20456] for IndexReader#[]' (error type ArgumentError)
A backtrace follows:
/opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:411:in `[]': :20492 is out of range [0..20456] for IndexReader#[] (ArgumentError)
	from /opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:411:in `[]'
	from /opt/local/lib/ruby/1.8/monitor.rb:238:in `synchronize'
	from /opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:403:in `[]'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/index.rb:304:in `build_message'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/index.rb:286:in `each_message_in_thread_for'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/thread.rb:319:in `call'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/thread.rb:319:in `load_thread_for_message'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/index.rb:297:in `each_message_in_thread_for'
	 ... 36 levels...
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/util.rb:485:in `method_missing'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/bin/sup:196
	from /opt/local/bin/sup:16:in `load'
	from /opt/local/bin/sup:16

And then this kind of errors:

The problem was: 'State Error occured at <except.c>:93 in xraise
Error occured in index.c:4142 - sr_get_lazy_doc
	Document 9761 has already been deleted

' (error type Ferret::StateError)
A backtrace follows:
/opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:411:in `[]': State Error occured at <except.c>:93 in xraise (Ferret::StateError)
Error occured in index.c:4142 - sr_get_lazy_doc
	Document 9761 has already been deleted

	from /opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:411:in `[]'
	from /opt/local/lib/ruby/1.8/monitor.rb:238:in `synchronize'
	from /opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:403:in `[]'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/index.rb:217:in `each_id_by_date'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/index.rb:217:in `each'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/index.rb:217:in `each_id_by_date'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/thread.rb:304:in `load_n_threads'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/modes/thread-index-mode.rb:413:in `__unprotected_load_n_threads'
	 ... 17 levels...
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/lib/sup/buffer.rb:203:in `handle_input'
	from /opt/local/lib/ruby/gems/1.8/gems/sup-0.2/bin/sup:210
	from /opt/local/bin/sup:16:in `load'
	from /opt/local/bin/sup:16

Best regards,
-- 
Nicolas Pouillard aka Ertai

---

From: [email-masked] (William Morgan)
Date: Sat, 24 Nov 2007 09:18:06 -0800

Excerpts from nicolas.pouillard's message of Mon Nov 19 05:53:33 -0800 2007:
> /opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:411:in
> `[]': :20492 is out of range [0..20456] for IndexReader#[] (ArgumentError)
[snip]
> /opt/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:411:in
> `[]': State Error occured at <except.c>:93 in xraise (Ferret::StateError)

These errors (and the previous errors you reported about the
non-matching message ids) suggest that Ferret is very confused---it's
returning document ids (each message is stored as a Ferret "document")
from a search that don't correspond to the right messages, or that
correspond to documents that have already been deleted, or that are
simply bigger than the max document id in the index.

I suspect a bug in Ferret has somehow broken your index. Here's what I
suggest.

0. Make a backup of ~/.sup in case something terrible happens.
1. Upgrade your Ferret gem to 11.5 (the latest).
2. sup-sync --all --optimize. (Might take a while.)

If step #2 fails with the same Ferret errors, try:

3. sup-dump | bzip2 > /tmp/state.bz2
4. rm -rf ~/.sup/ferret # you did make a backup, right?
5. sup-sync --all --restore <(bzcat /tmp/state.bz2) # assumes bash

But hopefully step #2 will work. And if we're lucky step #1 will fix
whatever Ferret bug caused this.

-- 
William <wmorgan-sup at masanjin.net>

---

From: [email-masked] (Nicolas Pouillard)
Date: Sat, 24 Nov 2007 20:00:52 +0100

> I suspect a bug in Ferret has somehow broken your index. Here's what I

Thanks a lot.

I no longer have these errors, so I keep the recipe when errors will come.