Subject: [sup-talk] What's your sup workflow (and a spew of ideas)

From: [email-masked] (Carl Worth)
Date: Tue, 18 Aug 2009 14:02:05 -0700

I'm now a few days into using sup, and it's clearly helped me to be
more productive at processing mail, (inbox is down to 0 from *lots* of
messages), so thanks!

However, I still don't feel like I'm experiencing the zen of sup
yet. So I'm interested in hearing from others:

    What is your typical workflow with sup? That is:

    What different modes of task do you find yourself doing with sup?
    Precisely what keystrokes do you use for those tasks?
    Where, if anywhere, does sup get in your way rather than help?

What follows is my (rather long) answer to those questions, and more
or less a todo list of things I'd like to help improve in sup.

My desired workflow
===================
I think there's a basic two-level approach I take to processing new
email. Fortunately, these two modes correspond well with sup's
inbox-mode and thread-view-mode. Here's what I'd like to do:

Identify "unimportant" threads (no need to read)
------------------------------------------------
This is a quick scan of the thread titles in my inbox. The goal here
is to identify as quickly as possible threads that I will never need
to read at all, (so that they can be immediately removed from my
inbox). Ideally, I will make a decision in one or two seconds, and
issue a single character command to indicate the decision.

Dealing with remaining "important" threads
------------------------------------------
After having eliminated as many threads as possible during the scan of
the inbox, I now switch to a mode where I'm actually looking at mail
messages. The goal here is to identify whether I need to do anything,
and how much time that will take. There are three possible outcomes:

    * No action needed

    * Quick action, (less than two minutes)

    * Longer action required (to assign to some project)

(Any similarity to a method proposed by David Allen in Getting Things
Done is not accidental, and he deserves credit as such.)   

Regardless of the three results, I will be quickly archiving the
thread so that it's gone from my inbox. That will be either immediate
archiving, archiving after a quick action (such as a reply), or
archiving after tagging the thread with a label for the appropriate
project.

How sup could support my workflow better
========================================
So sup obviously fits in quite well with the above model---which is
why I immediately fell in love with it. So first of all I have to
again say thank you. And I apologize that I will go into more detail
on the things I'd like to change in sup as opposed to the things that
already work so well. This isn't a criticism of the tool, just an
expression of my desire to improve it.

I'll phrase the list below as feature requests. Some of these things
may already exist in sup, and I just missed them. In such cases, I'll
be glad to hear pointers to the things I missed. Otherwise, I plan to
work on developing patches for these, and I'll be glad for any help.

I'll give each feature a name in [square-brackets] for easy reference in followup messages.

inbox-mode
-----------
* [black-on-white-color-scheme]

  We've talked in another thread about adding support for a
  black-text-on-white-background color scheme to sup-config. Before I
  can write a patch for that, does somebody have an example
  black-on-white color scheme that works well? I've been trying to
  come up with one but I've been unable to find how to change some
  colors, specifically:

  + [change-color-of-thread-selector]

    The selector for the current thread in thread-mode appears as
    black-foreground-on-cyan right now. I haven't been able to find
    the setting to make that a more easy-to-read background color, and
    this is the only line in the view that isn't bold when unread,
    (which is not ideal since it's the only line I'm trying to read
    and process).

  + [change-color-of-tag-markers]

    The tag markers are yellow, which is hard to see on white. I also
    couldn't find the setting to change this.

* [import-read-messages-as-archived-from-initial-sup-config-based-sync]
  
  When I very first started using sup, (running sup-config which setup
  sources and ran sup-sync) I was surprised to find my inbox with
  thousands of unread messages in it. Now that I'm using sup more I
  love that sup treats inbox and unread as separate labels, (far too
  many email programs fail to separate those notions). But for that
  initial import, I think it would have been much more useful to have
  only given the inbox tag to unread messages.

* [sort-priority-labels-before-date]
 
  The default sorting for the inbox is reverse-chronological which is
  a reasonable-enough default, (and works fine for me when I keep my
  inbox small). But when I get behind and my inbox gets large, (say
  after a few days away from email), I do have some tags that I would
  like to be sorted to the top, regardless of date. How about support
  for a configurable list of "priority" tags that would provide a
  primary sort before the date-based sort?

* [save-all-state-changes-immediately]

  The 'a' and 'd' commands do almost exactly what I expect, (by
  immediately making the current thread disappear and advancing the
  selector to the next thread). That's perfect for fast
  processing. One thing missing here is that they don't actually save
  this state, (requiring me to hit '$' at some point). Perhaps that
  safety feature was more important before undo was implemented, but
  now that it exists, would immediate state saving make sense?

  The two bad side effects I've experienced due to lack of immediate
  saving are: 1. Seeing confusingly inconsistent results after
  processing a bunch of my inbox and then doing a search based on
  label:unread or label:inbox (Why am I seeing all these messages
  again?). 2. Losing a bunch of processing state due to crashes. (The
  crashes have been due to the mbox-processing bug which has since
  been fixed, but being immune to state loss for any future crashes
  would be beneficial.)

search-results-mode
-------------------
* [provide-commands-to-refine-the-current-search]

  I mentioned above wanting to sort priority labels before
  date. Similarly, when processing a very large email I sometimes want
  to process related messages together, (to reduce mental context
  switches). So I want variants of both "\, F: Search all messages"
  and "L: list labels" search commands that refine the current search
  rather than doing a new global search. That is, the new commands
  would simply append new search terms to the current search rather
  than starting a new search.

* [allow-for-inbox-mode-magic-elsewhere]

  Another way to reword this one would be to "eliminate any magic of
  inbox mode". My question is, "What makes inbox-mode different than
  search-results-mode and how can I get the behavior of inbox-mode in
  my searches?".

  Without commands to refine the current search as described above,
  I've been trying to achieve results like "inbox refined to
  label:foo" with a search as follows:

	label:inbox -label:killed label:foo

  This gives me the set of threads that I want, but now my standard
  processing commands no longer work. For example, the 'a' key still
  removes the inbox label, but it doesn't make the tag then disappear
  immediately.

  One approach to fix this would be that when adding the commands to
  refine the inbox search, the new search results would also be in
  inbox-mode with all necessary magic.

  A more general fix would be to process the current thread after any
  changes, (such as the addition or removal of a label), and removing
  it from the current search results if it no longer meets those
  criteria.

* [fix-handling-of-kill-thread-outside-inbox]

  The handling of the "&: kill thread" command when applied outside of
  the inbox, (such as in search-results-mode), is currently very
  confusing. What it currently seems to do is to add the killed label
  and make the thread disappear from the current view. But then,
  running "@: Refresh view" will make it reappear, (whereupon one can
  make it disappear again with &, ad infinitum). The removal of the
  thread from the current view seems to be magic associated with the
  kill-thread command, (leading to inconsistent behavior). I would
  again suggest to instead implement the general fix described above,
  (always reprocessing a thread after label changes to see if it still
  meets the current search). That way, trying to kill a thread without
  -label:killed in the current search would simply add the "killed"
  label and the user would learn to add the necessary search terms,
  (or to do searches based on refining the searches of existing views
  to inherit this term).

thread-view-mode
----------------
* [override-arrow-keys-to-jump-to-next-actionable-line]

  In thread-view-mode the up and down arrows currently advance a
  selector by one line at a time, (inherited from line-cursor-mode),
  all the way into message bodies. As far as I can tell, this is not a
  useful behavior. I propose that the down arrow key should instead
  advance the selector to the next line which would cause a context
  change for the various operations that can be performed based on the
  selector, (or scroll to the next page if there are no such lines on
  the current or next page).

* [be-less-overzealous-in-moving-text-to-the-left-column]

  I tend to use sup in a very wide terminal, (200+ columns). And I'm
  glad to see that the inbox-mode takes advantage of this by showing a
  preview of the message body (that's a nice touch!) where many
  console-based clients just cut things off at 80 columns.

  However, in thread-view-mode, sup always pushes the current message
  all the way over to the left-hand column. This does mean that the
  "current" message is visible, but I have a tendency to read much
  more than the current message, (even before advancing), and
  subsequent messages are often scrolled such that the first several
  columns of text are scrolled off to the left. Meanwhile, I have
  acres of terminal real-estate to the right that sup isn't using.

  I would propose that messages only be scrolled to the left if
  necessary to avoid the 80th column of the message body not being
  visible, (assuming the terminal is at least 80 columns wide).

compose-mode
------------
* [repeated-postponing-shouldnt-make-additional-drafts]

  I find that while composing a message I often want to go look at
  some other messages for reference. This is currently quite awkward
  with sup. It would be easier if sup could be run multiple times. It
  would also be easier if sup could somehow fire off an editor as an
  external process, while still allowing for sup to be operated.

  As it stands, instead, I have to save and quit from my editor,
  postpone the message within sup, and then later come back to edit it
  again, and find where I was in the editor. It's an expensive context
  switch with a lot of keystrokes and lost state, (such as where point
  was inside emacs, my kill ring, etc.). I'm currently finding myself
  sometimes just drafting messages in emacs sessions unrelated to sup,
  and then doing "M-x insert-file" once sup has launched emacs for
  me. (This does relate a bit to the point I made in a previous thread
  where I would love to find a way to keep all the mail-composition
  tasks very separate from sup, but not lose things like replied-to
  bits.)

  Anyway, the current misfeature I'm hitting is that if I do postpone
  a draft, then return to edit it more, then postpone it again,
  etc. eventually when I send the message I'll have several
  partially-composed drafts in various states that I need to go and
  manually clean up. It seems like most email programs avoid this
  problem by removing the draft as soon as its selected for editing
  again, and I propose that sup do the same.

Anyway, I've probably run into several other little things that I
didn't capture in this email, but I'll hopefully remember them
later. If anyone has feedback on any of the above, (and actually read
this far!), then I'll appreciate getting it.

Otherwise, like I said, I hope to start trying to implement some of
these ideas, and when I do, I'll of course come back with separate new
threads for each.

-Carl

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20090818/43f15a25/attachment.bin>

---

From: [email-masked] (Carl Worth)
Date: Tue, 18 Aug 2009 14:46:41 -0700

Excerpts from Carl Worth's message of Tue Aug 18 14:02:05 -0700 2009:
> later.

And of course, here are a few:

inbox-mode
----------
 * [dont-perturb-selected-thread-when-new-mail-arrives]

   I think I just saw the following race condition:

	1. I have the top-most thread selected and I identify it as spam
	2. I start moving to press the d key to delete it
	3. Before I get their, new mail arrives and becomes selected
	4. I delete mail I really care about

   I propose that new mail arriving should not cause the selector to
   move from the thread currently selected.

label-list-mode
---------------
 * [allow-for-limiting-to-interesting-labels]

   In the workflow I described earlier, I process all mail mercilessly
   and get it archived and out of my inbox as quickly as possible. But
   some of those threads I identified as needing some significant time
   to deal with, so I labelled them to assign them to a particular
   "project".

   So, later, I'd like to look at my list of projects, choose one, and
   then choose the first task to be done.

   The label-list-mode is *almost* the perfect thing for this. The
   only problem is that it displays several labels, (with *lots* of
   messages each) that are not interesting to me at all in this
   sense. For example, killed and unread, for example, as well as
   several other labels that I use in ways other than my "project"
   labels.

   What I'd like here is a feature much like 'u' which toggles to
   display of only labels with unread messages, but insteads toggles
   to display only the labels that I've somehow marked as my "project"
   labels. (I'm not sure exactly how to name this feature for general
   use. The "project" sense is something personal to my usage. Maybe,
   "todo" or something?)

inbox-mode/search-result-mode
-----------------------------
* [display-number-of-unread-messages]

  In my ruthless scan while processing new messages, I want to be able
  to make split-second, yet accurate, decisions on whether I need to
  read messages, (and if so, how much time it will take). The current
  thread display shows a total number of messages within a thread, but
  that leads to me making an inaccurate estimate of how "expensive" it
  would be to read a thread. I'd much rather see the number of unread
  messages in the thread. Perhaps best would be to diplay both numbers
  such as: (1/64) with the 1 in bold to indicate that a thread with 64
  total messages has 1 unread message.

-Carl
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20090818/5f65c5b3/attachment.bin>

---

From: [email-masked] (Andrei Thorp)
Date: Tue, 18 Aug 2009 18:07:28 -0400

>   primary sort before the date-based sort?

Yeah, I'd really really like this also. This has been discussed, and in
general, it'd be lovely to be able to sort by hook or something. I think
grouping by labels first and then by time would be a good default.

>   been fixed, but being immune to state loss for any future crashes

I assume here that part of the problem is the time it takes to write
changes. If I read all my morning e-mails and then press $, it usually
takes a few seconds for this to finish. If it's just one e-mail, there
is still usually a noticeable-ish delay.

However, perhaps Xapian will improve this speed also, and if this is the
case, I'd love an "autosave" option at least in the config.
>   again, and I propose that sup do the same.

I think I agree there, and vaguely recall someone attempting to
implement being able to switch buffers even when an external editor is
in effect.

Anyway, impressive post there. You seem to really abuse your e-mail
client and know what you're on about. Good stuff.
-- 
Andrei Thorp, Developer: Xandros Corp. (http://www.xandros.com)

---

From: [email-masked] (Carl Worth)
Date: Tue, 18 Aug 2009 15:16:08 -0700

> implement being able to switch buffers even when an external editor is

That would be nice. In the meantime, I have realized that I don't need
to actually postpone my messages to do what I want---I can just exit
the editor and then switch buffers from compose-mode/reply-mode. So
that at least avoids the multiple-drafts issue, (but still leaves the
I-lose-my-state-when-I-quit-the-editor issue).

> client and know what you're on about. Good stuff.

Thanks. I have had a lot of ideas cooking for years about what my
dream email-system would look like. I think the impressive bit is that
of the half-dozen programs I've used in the last decade, sup is the
first one to get me to write up a post like that. (All other programs
were so far away from what I wanted as to make it infeasible to fix
them incrementally.)

-Carl
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20090818/6e240f41/attachment.bin>

---

From: [email-masked] (Rich Lane)
Date: Tue, 18 Aug 2009 19:02:01 -0400

provide-commands-to-refine-the-current-search:

'|' exists for search-results-mode (and it should be implemented for
label-results-mode if it doesn't exist there).

sort-priority-labels-before-date:

This isn't entirely easy for large numbers of messages. We've optimized
the index (especially Xapian) for reverse chronological order. You could
either do the sorting entirely in the UI, but just on the subset of
results currently loaded, or you could create a new mode that merges and
prioritizes the results of multiple queries.

allow-for-limiting-to-interesting-labels:

I have a hack for this. I reopen Redwood::Mode in the startup hook, then
add a keybinding to spawn a SearchResultsMode for a set of interesting
labels. Same for starred messages. A better solution would be good.

save-all-state-changes-immediately,
allow-for-inbox-mode-magic-elsewhere,
fix-handling-of-kill-thread-outside-inbox:

These are related - saving label changes immediately means we can use
the index to decide which threads are relevant. The new index api
methods unblock this, and now that they're in next I plan to implement
this soon.

be-less-overzealous-in-moving-text-to-the-left-column,
dont-perturb-selected-thread-when-new-mail-arrives:

+1 to these.

---

From: [email-masked] (Carl Worth)
Date: Tue, 18 Aug 2009 18:58:55 -0700

> label-results-mode if it doesn't exist there).

Ah, thanks for pointing this out. I totally missed it somehow.

This doesn't exist for inbox-mode, which is where I think I would like
it the most. And conceptually, the inbox is just a search, right? It
seems to me that inbox-mode should be at most a very slight tweak of
search-results-mode. I'll go digging in the code to see how far away
that would be.

For what it's worth, | doesn't exist for search-results mode.

And also, the shortcuts to refine the search by adding a label would
be nice too, (so one wouldn't need to type "label:"). So now I at
least know what already exists.

> prioritizes the results of multiple queries.

I think the former would be fine. The UI encourages only a small number of
results to be loaded already, so just sorting there should work find I
thin.

> labels. Same for starred messages. A better solution would be good.

While waiting for a better solution, would you mind sharing your code
for this? I'm quite new to ruby, so even if what you described should
be trivial for me to replicate, it's not yet. :-)

Maybe a page on the wiki?

> methods unblock this, and now that they're in next I plan to implement

Excellent. It did seem that things were moving this direction, and I'm
glad to hear that they are.

> dont-perturb-selected-thread-when-new-mail-arrives:

Great. And those don't sound too complicated, so maybe I'll try
cutting my teeth on those first.

Meanwhile, I thought of another race condition. If new mail arrives
for the current thread when in thread-view-mode, does it get added to
the view? It might seem nice for it to show up, but it might also lead
to it getting accidentally archived away if I "knew" that there were
only 4 messages, say, when I entered the thread view, then I hit ".a"
and archived away 5 messages.

It seems a silly thing, but it's the kind of thing that makes me start
distrusting ".a" and instead using "x", double-checking, then "a"
which is obviously less efficient. (Oh, and there's another place
where the current selector needs to not be perturbed. After I hit "x"
from thread-view-mode I find that a different thread can be selected
than the one I was just viewing if new mail arrived.)

-Carl
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20090818/ab65d341/attachment.bin>

---

From: [email-masked] (Rich Lane)
Date: Tue, 18 Aug 2009 23:02:57 -0400

> Maybe a page on the wiki?

Sure, I added an example to the end of the Hooks page.

> than the one I was just viewing if new mail arrived.)

I'm glad to know there are other people annoyed by UI race conditions :).
Even after a quick look at the code I'm not sure what ThreadViewMode
will do when a thread is added to. It might actually archive/read/etc
new messages but not display them.

I'd like a keybinding to reload/redisplay the thread and a status bar
note if there are new messages. Any label changes should only affect
messages that have been displayed. What do you think?

---

From: [email-masked] (Andrei Thorp)
Date: Wed, 19 Aug 2009 10:03:49 -0400

> messages that have been displayed. What do you think?

Ooh, I like it. I guess gmail does this.
-- 
Andrei Thorp, Developer: Xandros Corp. (http://www.xandros.com)

---

From: [email-masked] (Carl Worth)
Date: Wed, 19 Aug 2009 16:21:08 -0700

> Sure, I added an example to the end of the Hooks page.

Excellent. That looks very helpful.

And looking closer at the hooks wiki page, I see that it shows me how
to label incoming mail with ruby code, (which I've been wanting to
switch to instead of using maildrop to deliver mail to different
maildirs, and sup adding labels based on the source).

The hooks page also shows me how to display the number of unread
messages in a thread like I wanted, (though I'll still propose that
for sup itself rather than relying on a user hook for this).

So thanks for the encouragement to take a closer look at this page.

> new messages but not display them.

That was definitely my fear.

> messages that have been displayed. What do you think?

Only affecting messages that have been displayed is the critical
piece, yes. Adding a status bar notice would be a nice addition.

-Carl
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20090819/62013666/attachment.bin>