Subject: [sup-talk] Two questions

From: [email-masked] (Reid Thompson)
Date: Fri, 17 Apr 2009 15:41:12 -0400

>From the readme:
Note that Sup never changes the contents of any mailboxes; it only
  indexes in to them. So it shouldn't ever corrupt your mail. The flip
  side is that if you change a mailbox (e.g. delete messages, or, in
  the case of mbox files, read an unread message) then Sup will be
  unable to load messages from that source and will ask you to run
  sup-sync --changed.

So, in order to actually manage my email, I have to utilize a different email client to delete unwanted mail - essentially doing the same work twice, or am I missing something?

Could sup-sync --changed be incorporated into sup, and triggered automatically by sup itself?

---

From: [email-masked] (William Morgan)
Date: Tue, 21 Apr 2009 06:06:24 -0700

Reformatted excerpts from Reid Thompson's message of 2009-04-17:
> So, in order to actually manage my email, I have to utilize a
> different email client to delete unwanted mail - essentially doing the
> same work twice, or am I missing something?

For the two special cases of "physically" removing messages marked as
deleted or spam, you can use sup-sync-back, which is a batch operation
operating on an entire mail source at a time. Currently it only applies
to mbox sources and can't be run while Sup is also running.

> automatically by sup itself?

You could certainly do that via Sup's hook mechanism, but it's a very
slow operation (it has to scan over the entire mailbox).
-- 
William <wmorgan-sup at masanjin.net>

---

From: [email-masked] (Reid Thompson)
Date: Tue, 21 Apr 2009 09:13:54 -0400

> slow operation (it has to scan over the entire mailbox).

OK -- so to 'manage' an imap store, i'd need to setup a mechanism to
fetch all the email to a local store, deleting from the imap store, and
then do something like schedule an overnight run of sup-sync-back,

---

From: [email-masked] (William Morgan)
Date: Tue, 21 Apr 2009 07:26:20 -0700

Reformatted excerpts from Reid Thompson's message of 2009-04-21:
> fetch all the email to a local store, deleting from the imap store,
> and then do something like schedule an overnight run of sup-sync-back,

There are a couple issues at play. First, if you're serious about Sup
with IMAP, many people have gone the route of mirroring their IMAP
folders locally using something like offlineimap. The Ruby IMAP
libraries, and possibly IMAP itself, is otherwise too slow for how Sup
likes to treat its mailstores.

But that's not strictly necessary.

Second, if you're serious about deleting email from your IMAP server (as
opposed to just letting it stay there forever, since storage is cheap
and the too-fleeting moments of your precious mortality are not), you'll
need to periodically run sup-sync-back with the appropriate flags.

This all stems from design decisions around Sup's target environment,
which is that you have too much email to scan every message except in an
offline manner. I have 229k emails in my inbox. Sup's the only client
I know of that can scale to that. (Besides GMail of course.)
-- 
William <wmorgan-sup at masanjin.net>

---

From: [email-masked] (Reid Thompson)
Date: Tue, 21 Apr 2009 12:15:59 -0400

> I know of that can scale to that. (Besides GMail of course.)

for 'fun', i setup a fastmail.fm account, to which i'm using rss2email
to push a number of feeds to.  I setup offlineimap to pull these feeds
down for sup, so that I could 'manage' the imap store via sup.

Why is it expecting mbox format?

rthompso at raker ~/.sup $ sup-sync-back -d  maildir:~/.fastmaildotfm/INBOX
[Tue Apr 21 12:10:07 -0400 2009] using character set encoding "UTF-8"
[Tue Apr 21 12:10:08 -0400 2009] crypto: detected gpg binary
in /usr/bin/gpg
[Tue Apr 21 12:10:08 -0400 2009] locking /home/rthompso/.sup/lock...
[Tue Apr 21 12:10:08 -0400 2009] loading index...
[Tue Apr 21 12:10:08 -0400 2009] loaded index of 920 messages
Error: maildir:~/.fastmaildotfm/INBOX is not an mbox source.
[Tue Apr 21 12:10:08 -0400 2009] unlocking /home/rthompso/.sup/lock...

offlineimap is populating ~/.fastmaildotfm/INBOX

rthompso at raker ~/.sup $ cat sources.yaml
--- 
....
- !masanjin.net,2006-10-01/Redwood/Maildir 
  uri: maildir:~/.fastmaildotfm/INBOX
  cur_offset: 12403255620003971
  usual: true
  archived: false
  id: 4
  labels: 
  - rssmail
  mtimes: 
    cur: 2009-04-21 11:47:05 -04:00
    new: 2009-04-21 11:47:08 -04:00
- !masanjin.net,2006-10-01/Redwood/SentLoader 
  cur_offset: 19809
- !masanjin.net,2006-10-01/Redwood/DraftLoader 
  cur_offset: 0

rthompso at raker ~ $ find .fastmaildotfm/
.fastmaildotfm/
.fastmaildotfm/INBOX.Trash
.fastmaildotfm/INBOX.Trash/cur
.fastmaildotfm/INBOX.Trash/new
.fastmaildotfm/INBOX.Trash/tmp
.fastmaildotfm/INBOX.Sent Items
.fastmaildotfm/INBOX.Sent Items/cur
.fastmaildotfm/INBOX.Sent Items/new
.fastmaildotfm/INBOX.Sent Items/tmp
.fastmaildotfm/INBOX
.fastmaildotfm/INBOX/cur
.fastmaildotfm/INBOX/cur/1240328825_0.15391.raker,U=1,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,S
.fastmaildotfm/INBOX/new
.fastmaildotfm/INBOX/new/1240328827_7.15391.raker,U=29,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328825_5.15391.raker,U=6,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328827_6.15391.raker,U=28,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_0.15391.raker,U=8,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328827_5.15391.raker,U=27,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_9.15391.raker,U=17,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328827_4.15391.raker,U=26,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_8.15391.raker,U=16,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328825_6.15391.raker,U=7,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328827_3.15391.raker,U=25,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_7.15391.raker,U=15,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328827_2.15391.raker,U=24,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_1.15391.raker,U=9,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_6.15391.raker,U=14,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328827_1.15391.raker,U=23,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_13.15391.raker,U=21,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_5.15391.raker,U=13,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328825_1.15391.raker,U=2,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328827_0.15391.raker,U=22,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_12.15391.raker,U=20,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_4.15391.raker,U=12,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_3.15391.raker,U=11,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_2.15391.raker,U=10,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328825_2.15391.raker,U=3,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_11.15391.raker,U=19,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328828_0.15391.raker,U=32,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328826_10.15391.raker,U=18,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328825_3.15391.raker,U=4,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328827_9.15391.raker,U=31,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328825_4.15391.raker,U=5,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/new/1240328827_8.15391.raker,U=30,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
.fastmaildotfm/INBOX/tmp
.fastmaildotfm/INBOX.Drafts
.fastmaildotfm/INBOX.Drafts/cur
.fastmaildotfm/INBOX.Drafts/new
.fastmaildotfm/INBOX.Drafts/tmp

---

From: [email-masked] (William Morgan)
Date: Wed, 22 Apr 2009 08:12:40 -0700

> Why is it expecting mbox format?

Sorry, I gave you kind of a crappy answer above. sup-sync-back currently
only works on mbox files, though not for any reason other than no one's
gotten around to extending it yet. Deleting files from a Maildir is
significantly easier than removing them from an mbox, so the bar's not
very high on this one.
-- 
William <wmorgan-sup at masanjin.net>