* [sup-talk] Crash, bad index, and ensuing misery
@ 2009-10-03 20:04 Guillaume Quintard
2009-10-03 20:17 ` Rich Lane
2009-10-03 22:25 ` Rich Lane
0 siblings, 2 replies; 8+ messages in thread
From: Guillaume Quintard @ 2009-10-03 20:04 UTC (permalink / raw)
To: Sup-talk@rubyforge.org
Hi, I crash my computer with sup running, and it seems the index
didn't really like. sup-sync -a didn't help, what can I do?
--- RuntimeError from thread: load threads for thread-index-mode
DocNotFoundError: Document 2461178145 not found.
./lib/sup/xapian_index.rb:132:in `document'
./lib/sup/xapian_index.rb:132:in `each_message_in_thread_for'
./lib/sup/xapian_index.rb:132:in `map'
./lib/sup/xapian_index.rb:132:in `each_message_in_thread_for'
./lib/sup/thread.rb:341:in `load_thread_for_message'
./lib/sup/thread.rb:333:in `load_n_threads'
./lib/sup/xapian_index.rb:117:in `each_id_by_date'
./lib/sup/xapian_index.rb:110:in `each_id'
./lib/sup/xapian_index.rb:110:in `each'
./lib/sup/xapian_index.rb:110:in `each_id'
./lib/sup/xapian_index.rb:117:in `each_id_by_date'
./lib/sup/thread.rb:328:in `load_n_threads'
./lib/sup/modes/thread-index-mode.rb:625:in `__unprotected_load_n_threads'
(eval):12:in `load_n_threads'
./lib/sup/modes/thread-index-mode.rb:609:in `load_n_threads_background'
./lib/sup.rb:77:in `reporting_thread'
./lib/sup.rb:75:in `initialize'
./lib/sup.rb:75:in `new'
./lib/sup.rb:75:in `reporting_thread'
./lib/sup/modes/thread-index-mode.rb:608:in `load_n_threads_background'
./lib/sup/modes/thread-index-mode.rb:679:in `__unprotected_load_threads'
(eval):12:in `load_threads'
bin/sup:197
--
Guillaume
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [sup-talk] Crash, bad index, and ensuing misery
2009-10-03 20:04 [sup-talk] Crash, bad index, and ensuing misery Guillaume Quintard
@ 2009-10-03 20:17 ` Rich Lane
2009-10-03 21:06 ` Guillaume Quintard
2009-10-03 22:25 ` Rich Lane
1 sibling, 1 reply; 8+ messages in thread
From: Rich Lane @ 2009-10-03 20:17 UTC (permalink / raw)
To: Guillaume Quintard; +Cc: Sup-talk@rubyforge.org
Excerpts from Guillaume Quintard's message of Sat Oct 03 16:04:14 -0400 2009:
> Hi, I crash my computer with sup running, and it seems the index
> didn't really like. sup-sync -a didn't help, what can I do?
I'd hoped this kind of corruption wouldn't be possible with newer
xapian-index versions. What sup commit are you on? What version of
Xapian are you using? Which Xapian backend, Flint or Chert?
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [sup-talk] Crash, bad index, and ensuing misery
2009-10-03 20:17 ` Rich Lane
@ 2009-10-03 21:06 ` Guillaume Quintard
2009-10-03 21:39 ` Rich Lane
2009-10-06 12:40 ` Olly Betts
0 siblings, 2 replies; 8+ messages in thread
From: Guillaume Quintard @ 2009-10-03 21:06 UTC (permalink / raw)
To: Sup-talk@rubyforge.org
On Sat, Oct 3, 2009 at 10:17 PM, Rich Lane <rlane@club.cc.cmu.edu> wrote:
> I'd hoped this kind of corruption wouldn't be possible with newer
> xapian-index versions. What sup commit are you on? What version of
> Xapian are you using? Which Xapian backend, Flint or Chert?
>
ubuntu 9.10
libxapian-ruby1.8 1.0.14-1
I'd say flink since I often have to remove flintlock after sup
commiting suicide.
and git status says
# On branch next
# Your branch is ahead of 'origin/next' by 6 commits.
(ahead?)
--
Guillaume
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [sup-talk] Crash, bad index, and ensuing misery
2009-10-03 21:06 ` Guillaume Quintard
@ 2009-10-03 21:39 ` Rich Lane
2009-10-03 21:54 ` Guillaume Quintard
2009-10-06 12:40 ` Olly Betts
1 sibling, 1 reply; 8+ messages in thread
From: Rich Lane @ 2009-10-03 21:39 UTC (permalink / raw)
To: Guillaume Quintard; +Cc: Sup-talk@rubyforge.org
Excerpts from Guillaume Quintard's message of Sat Oct 03 17:06:36 -0400 2009:
> On Sat, Oct 3, 2009 at 10:17 PM, Rich Lane <rlane@club.cc.cmu.edu> wrote:
> > I'd hoped this kind of corruption wouldn't be possible with newer
> > xapian-index versions. What sup commit are you on? What version of
> > Xapian are you using? Which Xapian backend, Flint or Chert?
> >
>
> ubuntu 9.10
> libxapian-ruby1.8 1.0.14-1
> I'd say flink since I often have to remove flintlock after sup
> commiting suicide.
>
> and git status says
> # On branch next
> # Your branch is ahead of 'origin/next' by 6 commits.
>
> (ahead?)
>
Please post the output of "git log --pretty=oneline 'origin/next^'.."
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [sup-talk] Crash, bad index, and ensuing misery
2009-10-03 21:39 ` Rich Lane
@ 2009-10-03 21:54 ` Guillaume Quintard
0 siblings, 0 replies; 8+ messages in thread
From: Guillaume Quintard @ 2009-10-03 21:54 UTC (permalink / raw)
To: Sup-talk@rubyforge.org
On Sat, Oct 3, 2009 at 11:39 PM, Rich Lane <rlane@club.cc.cmu.edu> wrote:
> Please post the output of "git log --pretty=oneline 'origin/next^'.."
>
here you go
80789d873ac555bb1979f7734e95f104d848007e Merge branch 'next' of
git://gitorious.org/sup/mainline into next
0eee0973223b625b66e30c196ccd45d2f7bf358b Merge branch 'master' into next
93b5552730c10e2a352bd33f5ee98800bcd8679e more release-script updates
d1eabf9cb21940b933464ab3efc25df3c1a5f7e1 Merge branch 'next' of
git://gitorious.org/sup/mainline into next
df96980d0573cf91742a8f9ae5e8a49366d338b8 Merge branch 'next' of
git://gitorious.org/sup/mainline into next
a6dcb02dda7dd8bb1742890d460b1b0abfc28454 Merge branch 'next' of
git://gitorious.org/sup/mainline into next
823148627f554fbf5a7fb13379567276eeee14c7 Merge branch 'next' of
git://gitorious.org/sup/mainline into next
40ecc407a8aacf16d7f9f104501311cfd1fb2eb7 Revert "Merge branch
'after-add-message-hook' into next"
--
Guillaume
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [sup-talk] Crash, bad index, and ensuing misery
2009-10-03 21:06 ` Guillaume Quintard
2009-10-03 21:39 ` Rich Lane
@ 2009-10-06 12:40 ` Olly Betts
2009-10-06 13:10 ` Guillaume Quintard
1 sibling, 1 reply; 8+ messages in thread
From: Olly Betts @ 2009-10-06 12:40 UTC (permalink / raw)
To: sup-talk
Guillaume Quintard writes:
> On Sat, Oct 3, 2009 at 10:17 PM, Rich Lane <rlane <at> club.cc.cmu.edu> wrote:
> > I'd hoped this kind of corruption wouldn't be possible with newer
> > xapian-index versions. What sup commit are you on? What version of
> > Xapian are you using? Which Xapian backend, Flint or Chert?
>
> ubuntu 9.10
> libxapian-ruby1.8 1.0.14-1
> I'd say flink since I often have to remove flintlock after sup
> commiting suicide.
NEVER EVER remove the flintlock file. It's not the presence of the file which
determines the lock, but rather whether there's an fcntl() lock on it, so
removing it smashes any lock which is currently held, but leaves the process
which held it thinking it still has exclusive write access to the database,
which will likely quickly lead to a corrupt database, especially if you're
doing so because Xapian says the database is already locked. If Xapian says
that, then there really is a process which still has the database open for
writing.
My guess is that removing the flintlock file is the cause of the corruption
you're seeing. Can you reproduce it on a database where you haven't removed
this file?
Also, both chert and flint use the same locking approach with a file called
flintlock, so that doesn't discriminate between them. The easy way to tell
which you have is whether there's a file called "iamflint" or "iamchert" in
the database directory.
Cheers,
Olly
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [sup-talk] Crash, bad index, and ensuing misery
2009-10-06 12:40 ` Olly Betts
@ 2009-10-06 13:10 ` Guillaume Quintard
0 siblings, 0 replies; 8+ messages in thread
From: Guillaume Quintard @ 2009-10-06 13:10 UTC (permalink / raw)
To: Olly Betts; +Cc: sup-talk
On Tue, Oct 6, 2009 at 2:40 PM, Olly Betts <olly@survex.com> wrote:
> NEVER EVER remove the flintlock file.
Ooops, didn't know, won't do it again.
> My guess is that removing the flintlock file is the cause of the corruption
> you're seeing. Can you reproduce it on a database where you haven't removed
> this file?
Unfortunately, no
> which you have is whether there's a file called "iamflint" or "iamchert" in
Flint then
--
Guillaume
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [sup-talk] Crash, bad index, and ensuing misery
2009-10-03 20:04 [sup-talk] Crash, bad index, and ensuing misery Guillaume Quintard
2009-10-03 20:17 ` Rich Lane
@ 2009-10-03 22:25 ` Rich Lane
1 sibling, 0 replies; 8+ messages in thread
From: Rich Lane @ 2009-10-03 22:25 UTC (permalink / raw)
To: Guillaume Quintard; +Cc: Sup-talk@rubyforge.org
Excerpts from Guillaume Quintard's message of Sat Oct 03 16:04:14 -0400 2009:
> --- RuntimeError from thread: load threads for thread-index-mode
> DocNotFoundError: Document 2461178145 not found.
> ./lib/sup/xapian_index.rb:132:in `document'
The relevant line:
docs = term_docids(mkterm(:thread, thread_id)).map { |x| @xapian.document x }
Basically expands to:
@xapian.postlist(term).map { |x| @xapian.document x.docid }
So, Xapian is giving us docids it can't turn into documents. At this
point I think you should just do a sup-dump (which might still work),
move the old xapian directory out of the way, and re-sync. You can try
running xapian-check on the corrupted version to see if it finds
anything interesting.
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2009-10-06 13:15 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-10-03 20:04 [sup-talk] Crash, bad index, and ensuing misery Guillaume Quintard
2009-10-03 20:17 ` Rich Lane
2009-10-03 21:06 ` Guillaume Quintard
2009-10-03 21:39 ` Rich Lane
2009-10-03 21:54 ` Guillaume Quintard
2009-10-06 12:40 ` Olly Betts
2009-10-06 13:10 ` Guillaume Quintard
2009-10-03 22:25 ` Rich Lane
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox