Subject: Re: [sup-talk] Slow opening of threads?

From: Philipp Überbacher <[email-masked]>
Date: Fri, 22 Oct 2010 13:16:13 +0200

Excerpts from Shadowfirebird's message of 2010-10-22 11:23:42 +0200:
> Is it me, or is Sup sometimes very slow at opening a thread?  I mean: a couple of minutes to open a thread with two or three messages and no attachments, about one time in three?
> 
> It *could* be because my maildir is on the other side of an NFS mount.  Or, not.   
> 
> I'm just wondering if anyone else is experiencing it.

That sounds weird. In my experience it's only slow when flushing index,
which means everything is frozen for a couple of seconds. Opening
threads is fast enough here. Threads with 50-100 messages take a couple
of seconds though, but it's acceptable.

_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Moritz Wilhelmy <[email-masked]>
Date: Fri, 22 Oct 2010 15:09:00 +0200

Excerpts from Shadowfirebird's message of Fri Oct 22 13:48:45 +0200 2010:
Dear all,
 
> I've got 343,000 messages.  Maybe that's it...

I heard rumours that notmuch is supposed to be significantly faster than sup
with huge amounts of mail, while also being tag-based.  If you don't mind using
emacs, which is - I think - the only usable interface to it, you could give it
a try (sorry for promoting different mailclients here :)

I like sup, because I love the UI (and because I am not bound to emacs)
I love the thread and signature folding. Keep up the good work!

Best regards,

Moritz

---

From: Shadowfirebird <[email-masked]>
Date: Fri, 22 Oct 2010 12:48:45 +0100

Excerpts from Amadeusz Żołnowski's message of Fri Oct 22 11:40:22 +0100 2010:
> I've just opened thread of 18 messages in less than second.  My maildir
> holds above 7000 messages and my machine is quite antique.

-- 
Some performers on television appear to be horrible people, but when
you finally get to know them in person, they turn out to be even worse.
		-- Avery
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Mark Alexander <[email-masked]>
Date: Fri, 22 Oct 2010 12:28:01 -0400

Excerpts from Shadowfirebird's message of Fri Oct 22 09:03:15 -0400 2010:
> Interesting that you should mention that, because I have an
> encrypted $HOME (bad choice at Ubuntu install time -- ::shrug:: ).  I
> suppose it might apply the other way around, too. 

That could do it.  It would mean your .sup directory (and the Xapian
index) would be on ecryptfs, which could be slow.

In my case, my home directory is not encrypted.  I only put
work-related stuff (e.g., email) on an encrypted directory
($HOME/Private) that I set up initially with ecryptfs-setup-private.
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Amadeusz Żołnowski <[email-masked]>
Date: Fri, 22 Oct 2010 12:40:22 +0200

Excerpts from Shadowfirebird's message of Fri Oct 22 11:23:42 +0200 2010:
> Is it me, or is Sup sometimes very slow at opening a thread?  I mean:
> a couple of minutes to open a thread with two or three messages and no
> attachments, about one time in three?
> 

Amadeusz Żołnowski

PGP key fpr: C700 CEDE 0C18 212E 49DA  4653 F013 4531 E1DB FAB5
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Shadowfirebird <[email-masked]>
Date: Sat, 23 Oct 2010 11:42:11 +0100

> >> drop_caches
> 
> >> Writing to this will cause the kernel to drop clean caches, dentries
> >> and inodes from memory, causing that memory to become free.
> 
> >> To free pagecache:
> >>         echo 1 > /proc/sys/vm/drop_caches
> >> To free dentries and inodes:
> >>         echo 2 > /proc/sys/vm/drop_caches
> >> To free pagecache, dentries and inodes:
> >>         echo 3 > /proc/sys/vm/drop_caches
> 
> >> As this is a non-destructive operation and dirty objects are not
> >> freeable, the user should run `sync' first.
> 
> Sascha
> 
> [1] http://www.kernel.org/doc/Documentation/sysctl/vm.txt

Thank you!  

-- 
Know what I hate most?  Rhetorical questions.
		-- Henry N. Camp
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Mark Alexander <[email-masked]>
Date: Fri, 22 Oct 2010 08:38:16 -0400

> It *could* be because my maildir is on the other side of an NFS mount.  Or, not.   

I have that experience with one of my maildirs that is in an ecryptfs
directory.  It's not quite a couple of minutes, but sometimes up to 30
seconds opening a very long thread.  I've seen slowdowns on ecryptfs
doing other things like compiles.

So my guess is that the slowdown you're seeing is due to NFS.  In my
experience NFS is a performance hog and should be avoided at all
costs.  If at all possible, run sup on a single machine with your
maildirs (and your .sup directory) on a local disk.  If you need to
access your mail from other machines, log into your sup-running
machine via ssh.

Also (and this is perhaps the most important thing), run sup inside a
screen session so you won't lose any of your work if the ssh
connection goes down.  Screen is useful even if you only access your
mail from a single machine, because it will keep your session alive if
Gnome/KDE crashes or you have to log out and back in for any other
reason.
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Shadowfirebird <[email-masked]>
Date: Fri, 22 Oct 2010 14:03:15 +0100

> doing other things like compiles.

Interesting that you should mention that, because I have an encrypted $HOME (bad choice at Ubuntu install time -- ::shrug:: ).  I suppose it might apply the other way around, too.

> access your mail from other machines, log into your sup-running

I think I'll plan to run something like offlineIMAP (or just rsync) to sync my maildir to a local folder when I rework my machine in the near future.  Also, I'm moving the whole house to gigabit ethernet, so maybe that will help.

> connection goes down.

Not a problem here; there's no ssh involved.  The maildir is on the server downstairs, and I've got my home directory there mounted here via NFS.

-- 
If you will practice being fictional for a while, you will understand that
fictional characters are sometimes more real than people with bodies and
heartbeats.
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Shadowfirebird <[email-masked]>
Date: Fri, 22 Oct 2010 10:23:42 +0100

-- 
"One day I woke up and discovered that I was in love with tripe."
		-- Tom Anderson
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Moritz Wilhelmy <[email-masked]>
Date: Sat, 23 Oct 2010 01:32:46 +0200

Excerpts from Shadowfirebird's message of Fri Oct 22 20:02:39 +0200 2010:
> Oh gods.  I'm emacs-phobic.

I meant to say, you could try if notmuch performs any better. You don't have to
use the emacs extension, it also works as plain command line command. 
If the performance is not better, it's probably your NFS.
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Ben Walton <[email-masked]>
Date: Fri, 22 Oct 2010 08:21:26 -0400

> of seconds though, but it's acceptable.

I periodically see slow thread loading and I'm also doing Maildir over
NFS.  It's not frequent enough that I've bothered to look into it, but
it does happen from time to time.  I've seen it on threads of various
sizes.

HTH.
-Ben

--
Ben Walton
Systems Programmer - CHASS
University of Toronto
C:416.407.5610 | W:416.978.4302

---

From: Sascha Silbe <[email-masked]>
Date: Fri, 22 Oct 2010 19:19:39 +0200

> Is it me, or is Sup sometimes very slow at opening a thread?  I mean: a couple of minutes to open a thread with two or three messages and no attachments, about one time in three?

My experience is that 
a) thread opening time is proportional to cumulated message size and
b) time for collapsing/expanding a message is proportional to the number
   of (expanded) lines in the buffer.

In both cases there's a large factor involved.

Are you getting different timings for the same thread (remember to flush
all OS level caches to check this) or are some threads always slow?
Does it happen in conjunction with polling? I noticed that sup doesn't
always display "Flushing index..." right away (maybe it doesn't flush
the output buffer when displaying the message?).

FWIW, notmuch is not much faster (SCNR) on my systems. Sup is
alternating between IO bound and CPU bound; notmuch kind of eliminates
the CPU bound part, but because it doesn't support mbox the IO bound
part increases significantly. I imagine that notmuch would benefit
greatly from an SSD.

Sascha

--
http://sascha.silbe.org/
http://www.infra-silbe.de/
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Gaute Hope <[email-masked]>
Date: Fri, 22 Oct 2010 13:04:34 +0200

> It *could* be because my maildir is on the other side of an NFS mount.  Or, not.   

I'd test that :) I have no problems - might be that simple seek
operations are really expansive over NFS.

- gaute

---

From: Shadowfirebird <[email-masked]>
Date: Fri, 22 Oct 2010 20:56:07 +0100

> all OS level caches to check this) or are some threads always slow?

It seems pretty random, but how do I flush my caches?

-- 
No one can guarantee the actions of another.
		-- Spock, "Day of the Dove", stardate unknown
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Sascha Silbe <[email-masked]>
Date: Sat, 23 Oct 2010 10:28:42 +0200

Sascha

---

From: Shadowfirebird <[email-masked]>
Date: Fri, 22 Oct 2010 19:02:39 +0100

But I can live with Sup taking an age to open threads every so often, at least until I reorganise away from NFS and/or it gets fixed.  

I love Sup for Ruby-ness as much as anything.  The other day I had a corrupt index and I fixed it myself!  </smug>

-- 
Advice from an old carpenter: measure twice, saw once.
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk