* [sup-talk] Fwd: Re: Crash while in thread-view-mode
@ 2009-10-07 22:42 Ben Gamari
2009-10-11 20:28 ` William Morgan
0 siblings, 1 reply; 4+ messages in thread
From: Ben Gamari @ 2009-10-07 22:42 UTC (permalink / raw)
To: sup-talk
Excerpts from Guillaume Quintard's message of Wed Oct 07 04:48:56 -0400 2009:
> On Wed, Oct 7, 2009 at 8:44 AM, Rich Lane <rlane@club.cc.cmu.edu> wrote:
> > You could try running sup with the -n flag to disable threading. The
> > major downside is that you have to hit P to poll manually.
>
> That "solves" the problem for me.
>
After going through the process of rebuilding my index (again), things
seem to be more stable (for now). I understand that designing software around a
contingency like this might not be the best practice, but the frequency
with which I've needed to rebuild really does make me think that ruby
isn't the best language for the indexer.
This is easily the fifth time I've needed to rebuild and each time it
has taken over 30 minutes for 1.5 GB of mail. That's substantially less than
1MB/second for what should be an I/O bound operation. Ouch. It'll be
really interesting to see how Carl's work comes out.
- Ben
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [sup-talk] Fwd: Re: Crash while in thread-view-mode
2009-10-07 22:42 [sup-talk] Fwd: Re: Crash while in thread-view-mode Ben Gamari
@ 2009-10-11 20:28 ` William Morgan
2009-10-11 21:04 ` Ben Gamari
2009-10-13 20:18 ` Ben Gamari
0 siblings, 2 replies; 4+ messages in thread
From: William Morgan @ 2009-10-11 20:28 UTC (permalink / raw)
To: sup-talk
Reformatted excerpts from Ben Gamari's message of 2009-10-07:
> I understand that designing software around a contingency like this
> might not be the best practice, but the frequency with which I've
> needed to rebuild really does make me think that ruby isn't the best
> language for the indexer.
The indexer isn't in Ruby, it's written in C++ in the case of Xapian and
C in the case of Ferret.
> This is easily the fifth time I've needed to rebuild and each time it
> has taken over 30 minutes for 1.5 GB of mail. That's substantially
> less than 1MB/second for what should be an I/O bound operation. Ouch.
I think this isn't the indexer's fault so much as the mbox parsing,
which is Ruby.
I'm sorry you've had to rebuild the index so many times. The Xapian side
of things is very new, and I think you've had a run of bad luck. But I
am personally not motivated to improve index time performance, because
that's not a common event. At least, it shouldn't be.
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [sup-talk] Fwd: Re: Crash while in thread-view-mode
2009-10-11 20:28 ` William Morgan
@ 2009-10-11 21:04 ` Ben Gamari
2009-10-13 20:18 ` Ben Gamari
1 sibling, 0 replies; 4+ messages in thread
From: Ben Gamari @ 2009-10-11 21:04 UTC (permalink / raw)
To: sup-talk
Excerpts from William Morgan's message of Sun Oct 11 16:28:48 -0400 2009:
> Reformatted excerpts from Ben Gamari's message of 2009-10-07:
> > I understand that designing software around a contingency like this
> > might not be the best practice, but the frequency with which I've
> > needed to rebuild really does make me think that ruby isn't the best
> > language for the indexer.
>
> The indexer isn't in Ruby, it's written in C++ in the case of Xapian and
> C in the case of Ferret.
Sorry, I was referring to the mail indexer (i.e. message,
mbox/maildir parser), not the backend indexing engine (e.g. Xapian).
Should have been more specific.
>
> > This is easily the fifth time I've needed to rebuild and each time it
> > has taken over 30 minutes for 1.5 GB of mail. That's substantially
> > less than 1MB/second for what should be an I/O bound operation. Ouch.
>
> I think this isn't the indexer's fault so much as the mbox parsing,
> which is Ruby.
Exactly. This is where I think C++ is probably appropriate.
>
> I'm sorry you've had to rebuild the index so many times. The Xapian side
> of things is very new, and I think you've had a run of bad luck. But I
> am personally not motivated to improve index time performance, because
> that's not a common event. At least, it shouldn't be.
Completely understandable. I really don't have a right to complain. It
does work a large majority of the time, after all. Just figured I'd let
you know of problems as they happen. Thanks for the awesome client.
- Ben
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [sup-talk] Fwd: Re: Crash while in thread-view-mode
2009-10-11 20:28 ` William Morgan
2009-10-11 21:04 ` Ben Gamari
@ 2009-10-13 20:18 ` Ben Gamari
1 sibling, 0 replies; 4+ messages in thread
From: Ben Gamari @ 2009-10-13 20:18 UTC (permalink / raw)
To: sup-talk
Excerpts from William Morgan's message of Sun Oct 11 16:28:48 -0400 2009:
> I'm sorry you've had to rebuild the index so many times. The Xapian side
> of things is very new, and I think you've had a run of bad luck. But I
> am personally not motivated to improve index time performance, because
> that's not a common event. At least, it shouldn't be.
Unfortunately, it seems that the crashes have returned (In fact, they
returned only hours after I rebuilt). Interestingly, it is once again my
LKML label that is the culprit.
On the note of crashing, is it really necessary for the "wrong id called
on nil" exception to be fatal? Couldn't the client at least make some
attempt at recovering (skipping the problematic thread while catching
and logging the exception)? It seems like these issues could be rather
tricky to sort out (I've put an hour or so into tracking down the source
of the nils to no avail), and with a dynamically typed language like
Ruby, could occur at any time. Considering this, it seems like it might
be a good idea to handle these failures a bit more gracefully. Would
this be problematic?
Regardless, it might be a good idea to have a hierarchy of exception
classes instead of just throwing Strings. It seems that throwing
specialized classes would make the above substantially easier.
Thanks,
- Ben
--- RuntimeError from thread: load threads for thread-index-mode
wrong id called on nil
/opt/exp/sup/lib/sup.rb:17:in `id'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:225:in `update'
/usr/lib64/ruby/1.8/rubygems/custom_require.rb:31:in `sort_by'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:225:in `each'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:225:in `sort_by'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:225:in `update'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:223:in `synchronize'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:223:in `update'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:628:in `__unprotected_load_n_threads'
/opt/exp/sup/lib/sup/thread.rb:334:in `load_n_threads'
/opt/exp/sup/lib/sup/xapian_index.rb:117:in `each_id_by_date'
/opt/exp/sup/lib/sup/xapian_index.rb:110:in `each_id'
/opt/exp/sup/lib/sup/xapian_index.rb:110:in `each'
/opt/exp/sup/lib/sup/xapian_index.rb:110:in `each_id'
/opt/exp/sup/lib/sup/xapian_index.rb:117:in `each_id_by_date'
/opt/exp/sup/lib/sup/thread.rb:328:in `load_n_threads'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:625:in `__unprotected_load_n_threads'
(eval):12:in `load_n_threads'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:609:in `load_n_threads_background'
/opt/exp/sup/lib/sup.rb:77:in `reporting_thread'
/opt/exp/sup/lib/sup.rb:75:in `initialize'
/opt/exp/sup/lib/sup.rb:75:in `new'
/opt/exp/sup/lib/sup.rb:75:in `reporting_thread'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:608:in `load_n_threads_background'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:679:in `__unprotected_load_threads'
(eval):12:in `load_threads'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:81:in `initialize'
/opt/exp/sup/lib/sup/modes/line-cursor-mode.rb:22:in `call'
/opt/exp/sup/lib/sup/modes/line-cursor-mode.rb:22:in `initialize'
/opt/exp/sup/lib/sup/modes/line-cursor-mode.rb:22:in `each'
/opt/exp/sup/lib/sup/modes/line-cursor-mode.rb:22:in `initialize'
/opt/exp/sup/lib/sup/modes/line-cursor-mode.rb:19:in `new'
/opt/exp/sup/lib/sup/modes/line-cursor-mode.rb:19:in `initialize'
/opt/exp/sup/lib/sup/modes/thread-index-mode.rb:54:in `initialize'
/opt/exp/sup/lib/sup/modes/label-search-results-mode.rb:9:in `initialize'
/opt/exp/sup/lib/sup/modes/label-search-results-mode.rb:32:in `new'
/opt/exp/sup/lib/sup/modes/label-search-results-mode.rb:32:in `spawn_nicely'
/opt/exp/sup/lib/sup/buffer.rb:350:in `spawn_unless_exists'
/opt/exp/sup/lib/sup/util.rb:520:in `send'
/opt/exp/sup/lib/sup/util.rb:520:in `method_missing'
/opt/exp/sup/lib/sup/modes/label-search-results-mode.rb:32:in `spawn_nicely'
/opt/exp/sup/lib/sup/modes/label-list-mode.rb:134:in `select_label'
/opt/exp/sup/lib/sup/mode.rb:51:in `send'
/opt/exp/sup/lib/sup/mode.rb:51:in `handle_input'
/opt/exp/sup/lib/sup/buffer.rb:267:in `handle_input'
/usr/bin/sup:245
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2009-10-13 20:27 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-10-07 22:42 [sup-talk] Fwd: Re: Crash while in thread-view-mode Ben Gamari
2009-10-11 20:28 ` William Morgan
2009-10-11 21:04 ` Ben Gamari
2009-10-13 20:18 ` Ben Gamari
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox