* [sup-talk] DatabaseCorruptError
@ 2011-05-06 22:06 Andrew Pimlott
2011-05-11 0:36 ` William Morgan
0 siblings, 1 reply; 3+ messages in thread
From: Andrew Pimlott @ 2011-05-06 22:06 UTC (permalink / raw)
To: sup-talk
Sup had been running for a while, and I just came back to my terminal to
find it dead:
--- IOError from thread: periodic poll
DatabaseError: Error reading block 4279173482: got end of file
./lib/sup/index.rb:271:in `_dangerous_allterms_begin'
./lib/sup/index.rb:271:in `each_prefixed_term'
./lib/sup/index.rb:284:in `each_source_info'
./lib/sup/mbox.rb:147:in `each'
./lib/sup/mbox.rb:147:in `map'
./lib/sup/mbox.rb:147:in `last_indexed_message'
./lib/sup/util.rb:204:in `call'
./lib/sup/util.rb:204:in `benchmark'
/usr/lib/ruby/1.8/benchmark.rb:293:in `measure'
./lib/sup/util.rb:204:in `benchmark'
./lib/sup/mbox.rb:147:in `last_indexed_message'
./lib/sup/mbox.rb:152:in `first_new_message'
./lib/sup/mbox.rb:124:in `poll'
./lib/sup/poll.rb:154:in `poll_from'
./lib/sup/poll.rb:113:in `do_poll'
./lib/sup/poll.rb:103:in `each'
./lib/sup/poll.rb:103:in `do_poll'
./lib/sup/poll.rb:102:in `synchronize'
./lib/sup/poll.rb:102:in `do_poll'
./lib/sup/util.rb:609:in `send'
./lib/sup/util.rb:609:in `method_missing'
./lib/sup/modes/poll-mode.rb:15:in `poll'
./lib/sup/poll.rb:49:in `poll_with_sources'
./lib/sup/poll.rb:68:in `poll'
./lib/sup/poll.rb:86:in `start'
./lib/sup.rb:78:in `reporting_thread'
./lib/sup.rb:76:in `initialize'
./lib/sup.rb:76:in `new'
./lib/sup.rb:76:in `reporting_thread'
./lib/sup/poll.rb:83:in `start'
./lib/sup/util.rb:609:in `send'
./lib/sup/util.rb:609:in `method_missing'
bin/sup:222
I tried to start it again, and got
--- IOError from thread: poll after loading inbox
DatabaseCorruptError: Db block overwritten - are there multiple writers?
/usr/lib/ruby/1.8/xapian.rb:259:in `_dangerous_postlist_begin'
/usr/lib/ruby/1.8/xapian.rb:259:in `postlist'
./lib/sup/index.rb:530:in `term_docids'
./lib/sup/index.rb:717:in `index_message_threading'
./lib/sup/index.rb:633:in `sync_message'
./lib/sup/util.rb:609:in `send'
./lib/sup/util.rb:609:in `method_missing'
./lib/sup/poll.rb:168:in `poll_from'
./lib/sup/mbox.rb:127:in `poll'
./lib/sup/poll.rb:154:in `poll_from'
./lib/sup/poll.rb:113:in `do_poll'
./lib/sup/poll.rb:103:in `each'
./lib/sup/poll.rb:103:in `do_poll'
./lib/sup/poll.rb:102:in `synchronize'
./lib/sup/poll.rb:102:in `do_poll'
./lib/sup/util.rb:609:in `send'
./lib/sup/util.rb:609:in `method_missing'
./lib/sup/modes/poll-mode.rb:15:in `poll'
./lib/sup/poll.rb:49:in `poll_with_sources'
./lib/sup/poll.rb:68:in `poll'
./lib/sup/util.rb:609:in `send'
./lib/sup/util.rb:609:in `method_missing'
bin/sup:212
./lib/sup.rb:78:in `reporting_thread'
./lib/sup.rb:76:in `initialize'
./lib/sup.rb:76:in `new'
./lib/sup.rb:76:in `reporting_thread'
bin/sup:212
./lib/sup/modes/thread-index-mode.rb:684:in `call'
./lib/sup/modes/thread-index-mode.rb:684:in `__unprotected_load_threads'
./lib/sup/modes/thread-index-mode.rb:625:in `call'
./lib/sup/modes/thread-index-mode.rb:625:in `load_n_threads_background'
./lib/sup.rb:78:in `reporting_thread'
./lib/sup.rb:76:in `initialize'
./lib/sup.rb:76:in `new'
./lib/sup.rb:76:in `reporting_thread'
./lib/sup/modes/thread-index-mode.rb:623:in `load_n_threads_background'
./lib/sup/modes/thread-index-mode.rb:694:in `__unprotected_load_threads'
(eval):12:in `load_threads'
bin/sup:212
My xapian dir looks like:
total 334396
-rw------- 1 andrew andrew 0 May 6 14:43 flintlock
-rw-rw-r-- 1 andrew andrew 12 May 31 2010 iamflint
-rw-rw-r-- 1 andrew andrew 103014400 May 6 13:09 position.DB
-rw-rw-r-- 1 andrew andrew 1594 May 6 11:22 position.baseA
-rw-rw-r-- 1 andrew andrew 161333248 May 6 13:29 postlist.DB
-rw-rw-r-- 1 andrew andrew 2475 May 6 11:22 postlist.baseA
-rw-rw-r-- 1 andrew andrew 20807680 May 6 13:04 record.DB
-rw-rw-r-- 1 andrew andrew 339 May 6 11:22 record.baseA
-rw-rw-r-- 1 andrew andrew 50692096 May 6 13:29 termlist.DB
-rw-rw-r-- 1 andrew andrew 795 May 6 11:22 termlist.baseA
-rw-rw-r-- 1 andrew andrew 6184960 May 6 13:04 value.DB
-rw-rw-r-- 1 andrew andrew 115 May 6 11:22 value.baseA
I'm running an old git pull from next, commit 1bafcf6 dated 2010-10-12.
Anything I should do to try to diagnose and recover this? Or just load
from sup-dump?
Andrew
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [sup-talk] DatabaseCorruptError
2011-05-06 22:06 [sup-talk] DatabaseCorruptError Andrew Pimlott
@ 2011-05-11 0:36 ` William Morgan
2011-05-11 6:14 ` Andrew Pimlott
0 siblings, 1 reply; 3+ messages in thread
From: William Morgan @ 2011-05-11 0:36 UTC (permalink / raw)
To: sup-talk
Reformatted excerpts from Andrew Pimlott's message of 2011-05-06:
> Sup had been running for a while, and I just came back to my terminal to
> find it dead:
>
> --- IOError from thread: periodic poll
> DatabaseError: Error reading block 4279173482: got end of file
> I'm running an old git pull from next, commit 1bafcf6 dated 2010-10-12.
Yipes. Are you out of disk space, or anything like that?
> Anything I should do to try to diagnose and recover this? Or just
> load from sup-dump?
Not knowing very much about Xapian, I suggest sup-dump. Not a great
solution, I know. Wait for heliotrope to solve your problems (or at
least be something I understand and can fix).
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [sup-talk] DatabaseCorruptError
2011-05-11 0:36 ` William Morgan
@ 2011-05-11 6:14 ` Andrew Pimlott
0 siblings, 0 replies; 3+ messages in thread
From: Andrew Pimlott @ 2011-05-11 6:14 UTC (permalink / raw)
To: sup-talk
Excerpts from William Morgan's message of Tue May 10 17:36:54 -0700 2011:
> Reformatted excerpts from Andrew Pimlott's message of 2011-05-06:
> > Sup had been running for a while, and I just came back to my terminal to
> > find it dead:
> >
> > --- IOError from thread: periodic poll
> > DatabaseError: Error reading block 4279173482: got end of file
> > I'm running an old git pull from next, commit 1bafcf6 dated 2010-10-12.
>
> Yipes. Are you out of disk space, or anything like that?
I wasn't out of space--unless possibly Xapian caused some transient
spike in disk use that was freed when it died. The system does run
without a ton of free space though, maybe 100 MB for a Xapian database
that is ~300 MB total.
I didn't get any disk errors either. This is on a VPS that could
conceivably be hiding some errors.
> > Anything I should do to try to diagnose and recover this? Or just
> > load from sup-dump?
>
> Not knowing very much about Xapian, I suggest sup-dump. Not a great
> solution, I know.
It turned out that sup-dump still worked. Probably some index was
corrupt but the data could still be walked. So I was able to restore
without losing any data. (Well, it did use most of the 160 MB of memory
on the VPS.)
> Wait for heliotrope to solve your problems (or at
> least be something I understand and can fix).
Dum de dum dum...
Andrew
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2011-05-11 6:37 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-05-06 22:06 [sup-talk] DatabaseCorruptError Andrew Pimlott
2011-05-11 0:36 ` William Morgan
2011-05-11 6:14 ` Andrew Pimlott
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox