Subject: [sup-talk] Some comments on sup-0.6

From: [email-masked] (Bart Schaefer)
Date: Sat, 23 Aug 2008 16:15:12 -0700

"The goal of Sup is to become the email client of choice for nerds everywhere."

I encountered sup a few days ago via a link on Justin Mason's blog,
and it sounded interesting enough to try it out.  There are quite a
few things about it that I like, but there are a few things about it
that currently are just plain show-stoppers.  None of them is
irreparable, but IMO taken together they almost disqualify it from
being called an "email client" at this time.

First the good stuff:  I entirely approve of the choices of ideas to
borrow from gmail, and the addition of being able to mark and coalesce
threads is something I wish gmail would borrow back.  The key bindings
weren't too hard to figure out (and probably would have been easier if
I'd used mutt more in the past).  The expanding/collapsing views of
threads and quoted messages work nicely most of the time.  I could
immediately see how sup could help me clean up the mess some of my
mail stores have become, to find and conceal old stuff without
completely losing track of it and prioritize important stuff.  Great
work.

(At the same time, that last is what seems to be lacking in
follow-through.  See below.)

One suggestion for an improvement on the gmail interface:  Consider
introducing a hierarchy syntax in labels.   E.g., if I've got a label
for "restaurants" I might create a label "restaurants/lunch" that
means "first search for anything tagged restaraurants and then narrow
by searching for anything also tagged lunch".  In any UI that shows a
list of the labels, collapse the heirarchy so that I don't have to see
both "restaurants" and "restaurants/lunch" at the same time.  This
creates an interface familiar to users who are accustomed to having
folders arranged hierarchically, without changing the search model
that ultimately locates the referenced messages.

(Is it possible to search for messages based on which source they came from?)

Next some nits or minor problems, in no particular order ...

The default color scheme is pretty terrible.  (I did mention there'd be "nits".)

When I widened my terminal window to try to see more of the preview
text on the thread screen, sup didn't pick up on the size change.

There's either no on-screen help for how one builds up a search
expression, or it's simply too hard to find that help, and in
particular it wasn't obvious how one searches for all messages having
a certain label (I was expecting maybe something like gmail's
label:Xyz syntax).

It should be possible to create search that matches a label without
finding other messages that happen to contain the same word (maybe it
is possible and I just haven't figure out how yet).

I happened to encounter a case of new mail arriving while I was using
sup, wherein a colleague had replied to a message by editing the
Subject such that his entire reply was there and the message body was
empty.  Sup collapsed this into the thread and hid his new subject
behind the original subject.  I had to bail out and switch over to
alpine to figure out why he'd sent a seemingly empty message.

OK, now the show stoppers.

I have a LOT of IMAP folders (conservatively, hundreds), many of them
in a shared hierarchy not under my control; sup's interface would be
the perfect way to get a handle on these, but it would take me way too
long to add them one by one as separate URL sources.  At the moment
sup might be described as phenomenal cosmic power in an itty-bitty
living space.

If you're going to have any hope of becoming the choice of nerds
everywhere, you've got address the "play well with others" problem, at
the very least with the IMAP INBOX, if not with other mailboxes.
Forcing me to exit from sup and do a complete re-index whenever a
message disappears is just not going to fly, particularly for a
protocol that provides so many tools for keeping in sync.  I know,
this doesn't fit with the "never delete anything, just stop looking at
it" model that sup wants to present, but in the real world even nerds
can't store things in their inbox forever, and for it to really be THE
email client of choice, I want to keep sup running for weeks at a time
and never have a reason to leave the interface (except maybe to escape
to a web browser or to drop into my editor when composing mail).

By the same token, sup needs to be able to push deletion marks back up
to the IMAP server and purge the mailbox.  Even if you violate the
IMAP usage model by implementing delete as some kind of move-to-trash
operation, so that sup can keep indexing the trash, there has to be a
way for the user to manage the size of his inbox without switching to
some other means of accessing the server.  And to really take
advantage of sup's ability to search and classify to clean up my messy
mail store, I need a way to permanently throw away all the spam and
duplicate messages that have accumulated in the odd procmail-created
corners of my folder space.  It's even fine if the only way to do this
is through IMAP or the like; sup does not need to incorporate the
mechanics of local mailbox management.

That's about it for impressions from a few hours of experimentation.
I think sup shows great promise and I've joined the mailing list to
keep track of its progress.  Congratulations on what you've got so
far, and I look forward to seeing more.

---

From: [email-masked] (Luis Villa)
Date: Sat, 23 Aug 2008 19:20:22 -0400

Possibly of interest, Bart:
http://all-thing.net/2008/06/rethinking-sup.html
http://all-thing.net/2008/07/rethinking-sup-part-ii.html

Luis (who isn't actually using sup, for many of the reasons you
listed, but remains a fascinated lurker)

On Sat, Aug 23, 2008 at 7:15 PM, Bart Schaefer
<barton.schaefer at gmail.com> wrote:
> _______________________________________________
> sup-talk mailing list
> sup-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/sup-talk
>

---

From: [email-masked] (Bart Schaefer)
Date: Sat, 23 Aug 2008 16:57:37 -0700

> http://all-thing.net/2008/07/rethinking-sup-part-ii.html

That is of interest, thanks.  Doesn't actually invalidate very much of
what I said, though. :-)

Those blog posts are from a couple of months ago.  I could go comment
there, or I could comment here.  Any preference?

---

From: [email-masked] (William Morgan)
Date: Sun, 24 Aug 2008 17:01:51 -0700

> there, or I could comment here.  Any preference?

Here's probably better for anything substantial. Non-threaded blog
comments are about as crappy a discussion medium as I can imagine.
-- 
William <wmorgan-sup at masanjin.net>

---

From: [email-masked] (William Morgan)
Date: Sun, 24 Aug 2008 17:30:30 -0700

Hi Barton,

Thanks for all the comments! Many of the issues you bring up are real.
My current focus is on the backend to Sup 2.0, but in general I'm happy
to accept patches and merge requests to fix these issues.

Detailed comments inlined:

Reformatted excerpts from barton.schaefer's message of 2008-08-23:
> by searching for anything also tagged lunch".

Personally I don't like the idea of adding structure to labels, because
this proposal (and others before it) are oriented at people with a lot
of labels, and I suspect that too many labels is a symptom that you're
doing something wrong.

That said, I know I'm crazy, and I would accept a patch that made this
an (optional!) feature. One easy way might be to make it solely a UI
tweak to LabelSearchMode, where any labels with a "/" in them are fit
into a pre-collapsed hierarchy instead of displayed in a big list.

> (Is it possible to search for messages based on which source they came

Technically yes, but there's not a good interface on top of it. Search
for "source_id:X", where X is the numeric source id of the source, which
you have look up in sources.yaml.

> The default color scheme is pretty terrible.  (I did mention there'd

I like it, but maybe I'm just accustomed to it. It's configurable, and
I'd accept patches for the default, if, say, more than one other person
felt was better.

> text on the thread screen, sup didn't pick up on the size change.

I could never get SIGWINCH to work due to a weird interaction between
curses and Ruby threads. If you can, I'd be happy to accept a patch.
In the meantime, you can press ^L to redraw the screen, and Sup should
notice the change.

> a certain label (I was expecting maybe something like gmail's

The "label:asdf" syntax should work as you expect.

I agree that better search expression help would be nice. Patches
accepted. :)

> alpine to figure out why he'd sent a seemingly empty message.

I think that if you had expanded the header (with 'h') while in
thread-view-mode, you would've seen the full subject.

> long to add them one by one as separate URL sources. 

Agreed. This has been on the todo list for a while.

> the very least with the IMAP INBOX, if not with other mailboxes.

Actually, I'm going to be taking a very different tack in Sup 2.0, and
not deal with IMAP at all. Sup will maintain its own document store, and
it's going to be up to the user to insert your mail into it.

I know that's probably not the answer you want. But I have no desire to
spend the requisite years of my life dealing with with hokey IMAP
servers, Ruby's hokey IMAP libraries, and everything in between. I've
taken a few steps down that path, but that's been more than enough to
recognize it as the path to madness.

---

From: [email-masked] (Bart Schaefer)
Date: Mon, 25 Aug 2008 09:39:14 -0700

On Sun, Aug 24, 2008 at 5:30 PM, William Morgan
> Hi Barton,

"Bart" is fine.  I only display it the "long way" to stop people from
calling me "Bartholomew."

> to accept patches and merge requests to fix these issues.

Unfortunately I'm not much of a ruby programmer yet.  Practically
anything else for longer than I want to think about, but not ruby,
though I'm looking at it.

> of labels, and I suspect that too many labels is a symptom that you're

It's an organizational thing.  Some people think better in hierarchy.
And occasionally one has as a message that relates closely to a
particular subject but doesn't mention any of the usual terms one
would search for, so it's handy to have a way to force a connection.

> into a pre-collapsed hierarchy instead of displayed in a big list.

That's essentially what I was suggesting.

> I agree that better search expression help would be nice. Patches

Does anyone other than you really know what the help should say?

> it's going to be up to the user to insert your mail into it.

After reading your blog posts this was the one thing I really wanted
to comment on.

I think this is a mistake.  By all means separate the document index
from the viewer, but I'd advise against attempting to be the warehouse
for the data.  The most important thing needed to play nicely is to
gracefully handle the case of data going missing from the source;
sure, you can do that by always copying the data up front, but that's
a blunt instrument that destroys several of the other advantages of
the current sup.

My suggestion:  Separate the indexing and querying of the index from
the sources.  Make it the responsibility of the viewer component to
fetch the full documents from the sources and to fail gracefully when
a query returns a document that's no longer available.  Give the
viewer a hook so that with the appropriate plug-in, certain indexing
updates (like spam-tagging) can be pushed back to the source at the
same time.  (It's up to the plug-in to decide how.)

If you have that, you can still use your own document source into
which you dump everything.

> servers, Ruby's hokey IMAP libraries, and everything in between.

I'm not suggesting (or even encouraging) that sup become a
full-fledged IMAP client.

---

From: [email-masked] (Marcus Williams)
Date: Mon, 25 Aug 2008 21:55:40 +0100

> a certain label (I was expecting maybe something like gmail's

Off the top of my head, some search expressions that are possible in
sup:

From/to/subject searches:

from:email at domain.tld
to:email at domain.tld
subject:*wildcards*

... wildcards are useful in from/to/cc searches as well "from:will*"

Date searches:

on:(16th feb)
before:yesterday
after:(last friday)
during:(last week)

(note that because search results are returned as threads you can get
messages outside of your date bounds because the thread containing the
searched message may have messages outside the bounds if that makes
sense).

Attachments:

has:attachment  (attachment is a label)
filetype:xls
filename:thisisafilename.txt

(Use brackets to specify a filename with spaces)

Labels:

is:read
is:spam
is:deleted
has:somelabel
label:someotherlabel

Phrases are treated as searches on the body text. Combinations work
and are treated as conjunctions/ands as far as I'm aware.

Hope that helps,

Marcus