From eg@gaute.vetsj.com Tue Nov 5 07:51:42 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Tue, 05 Nov 2013 08:51:42 +0100 Subject: [sup-devel] sup 0.15? Message-ID: <1383637745-sup-2758@qwerzila> hi there, any objections to releasing sup 0.15 (I figured it deserves a whole new version number!) with maildir-syncback as it is now? Is it ready to be released into the wild? (maildir-syncback is now in 'develop' - go ahead and test if you haven't) Cheers, Gaute From eg@gaute.vetsj.com Tue Nov 5 08:10:54 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Tue, 05 Nov 2013 09:10:54 +0100 Subject: [sup-devel] some easy issues for those who want to get involved Message-ID: <1383638882-sup-7342@qwerzila> hi there, there are a few easy issues in the issue-tracker [0] for those who want to get involved in developing sup. Please check them out: - #174: add a keybinding for opening a decoded attachment [1] - #175: expunge script for maildir-syncback [2] rewards: - a better sup - you get to be added to the contributors team for sup (on github) (yey!) Cheers, Gaute [0] https://github.com/sup-heliotrope/sup/issues?state=open [1] https://github.com/sup-heliotrope/sup/issues/174 [2] https://github.com/sup-heliotrope/sup/issues/175 From matthieu.rakotojaona@gmail.com Tue Nov 5 19:36:06 2013 From: matthieu.rakotojaona@gmail.com (Matthieu Rakotojaona) Date: Tue, 05 Nov 2013 20:36:06 +0100 Subject: [sup-devel] sup 0.15? In-Reply-To: <1383637745-sup-2758@qwerzila> References: <1383637745-sup-2758@qwerzila> Message-ID: <1383638661-sup-4880@kpad> Hello, Excerpts from Gaute Hope's message of 2013-11-05 08:51:42 +0100: > any objections to releasing sup 0.15 No objections for me (but I haven't been using the maildir-syncback feature much, so that's not really much...) -- Matthieu Rakotojaona -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From eg@gaute.vetsj.com Thu Nov 7 15:04:36 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Thu, 07 Nov 2013 16:04:36 +0100 Subject: [sup-devel] Release 0.15.0 Message-ID: <1383836355-sup-568@qwerzila> Hi all, Sup 0.15.0 is out, get it at rubygems: $ gem install sup (detailed installation instructions in the wiki [1]) News, please at least skim through: * Maildir Syncback has now been merged into main sup! This is a long-time waiting feature initially developed by Damien Leone, then picked up by Edward Z. Yang who continued development. Additionally several others have been contributing. Eventually, recently, Eric Weikl has picked up this branch, modernized it to current sup, maintained it and gotten it ready for release. Main authors: Damien Leone Edward Z. Yang Eric Weikl Not all of the features initially proposed have been included. This is to maintain compatibility with more operating systems and wait with the more daring features to make sure sup is stable-ish. This is a big change since sup now can modify your mail (!), please back up your mail and your configuration before using the maildir syncback feature. For instructions on how to migrate an existing maildir source or how to set up a new one, refer to the wiki: https://github.com/sup-heliotrope/sup/wiki/Using-sup-with-other-clients It is possible to both disable maildir syncback globally (default: disabled) and per-source (default: enabled). * Sup on Ruby 2.0.0 now works - but beware, this has not been very throughly tested. Patches are welcome. * We are now using our own rmail-sup gem with fixes for Ruby 2.0.0 and various warnings fixed. * sup-sync-back has been renamed to sup-sync-back-mbox to conform with the other sync-back scripts. * You can now save attachments to directories without specifying the full filename (default filename is used). * Various encoding fixes and minor bug fixes Important: If you are upgrading from a Sup version prior to 0.14.0 please read the migration instructions in the wiki [0]. Please report any issues you encounter to our GitHub issue tracker [2]. Thanks to all who have contributed to sup either with code [3] or testing! Cheers, Gaute [0] https://github.com/sup-heliotrope/sup/wiki/Migration-0.13-to-0.14 [1] https://github.com/sup-heliotrope/sup/wiki [2] https://github.com/sup-heliotrope/sup/issues [3] https://github.com/sup-heliotrope/sup/blob/release-0.15.0/CONTRIBUTORS -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From sdothum@gmail.com Sun Nov 10 15:49:47 2013 From: sdothum@gmail.com (Steven Hum) Date: Sun, 10 Nov 2013 10:49:47 -0500 Subject: [sup-devel] sup 0.15? In-Reply-To: <1384097900-sup-2821@luna> References: <1383637745-sup-2758@qwerzila> <1384097900-sup-2821@luna> Message-ID: <1384098458-sup-7304@luna> Edit: regarding the rmail comment.. I noticed the sup-rmail-1.0.1 gem which I presume sup-0.15.0 installed. Steven Excerpts from Steven Hum's message of 2013-11-10 10:42:00 -0500: > > Hello Gaute, > > I did a gem install of 0.15.0 on a Debian Sid system and got this error: > > > --- RuntimeError from thread: load threads for thread-index-mode > invalid source 13 > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:237:in `block in build_message' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:235:in `map' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:235:in `build_message' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:161:in `block (2 levels) in each_id_by_date' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/thread.rb:338:in `call' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/thread.rb:338:in `block in load_n_threads' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:161:in `block in each_id_by_date' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:284:in `block in each_id' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:284:in `each' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:284:in `each_id' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:161:in `each_id_by_date' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/thread.rb:334:in `load_n_threads' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/modes/thread_index_mode.rb:674:in `load_n_threads' > (eval):12:in `load_n_threads' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/modes/thread_index_mode.rb:658:in `block in load_n_threads_background' > /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup.rb:87:in `block in reporting_thread' > > > Gem version 0.14.1.1 works fine on Debian Sid (rmail did not appear to > be installed with 0.15.0 but that doesn't seem to be related to the > above error). > > Regards, > Steven > > Excerpts from Gaute Hope's message of 2013-11-05 02:51:42 -0500: > > hi there, > > > > any objections to releasing sup 0.15 (I figured it deserves a whole new > > version number!) with maildir-syncback as it is now? Is it ready to be > > released into the wild? > > > > (maildir-syncback is now in 'develop' - go ahead and test if you > > haven't) > > > > Cheers, Gaute > > > -- "Truth or die." Steven Hum 5 - 28 Gilmour St Ottawa, ON K2P 0N3 email sdothum at gmail.com tel 613.237.9058 From eg@gaute.vetsj.com Sun Nov 10 19:45:13 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Sun, 10 Nov 2013 20:45:13 +0100 Subject: [sup-devel] sup 0.15? In-Reply-To: <1384097900-sup-2821@luna> References: <1383637745-sup-2758@qwerzila> <1384097900-sup-2821@luna> Message-ID: <1384112448-sup-9012@qwerzila> Excerpts from Steven Hum's message of 2013-11-10 16:42:00 +0100: > I did a gem install of 0.15.0 on a Debian Sid system and got this error: > > > --- RuntimeError from thread: load threads for thread-index-mode > invalid source 13 Hi Steven, have you changed your sources.yaml in ~/.sup somehow? It seems like sup can't find source number 13. Sup 0.14.1.1 worked? If you upgraded from before 0.14 you need to follow these instructions: https://github.com/sup-heliotrope/sup/wiki/Migration-0.13-to-0.14 Btw; I haven't seen anything about sup-rmail-1.0.1, is that a debian package? - gaute From sdothum@gmail.com Sun Nov 10 15:42:00 2013 From: sdothum@gmail.com (Steven Hum) Date: Sun, 10 Nov 2013 10:42:00 -0500 Subject: [sup-devel] sup 0.15? In-Reply-To: <1383637745-sup-2758@qwerzila> References: <1383637745-sup-2758@qwerzila> Message-ID: <1384097900-sup-2821@luna> Hello Gaute, I did a gem install of 0.15.0 on a Debian Sid system and got this error: --- RuntimeError from thread: load threads for thread-index-mode invalid source 13 /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:237:in `block in build_message' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:235:in `map' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:235:in `build_message' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:161:in `block (2 levels) in each_id_by_date' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/thread.rb:338:in `call' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/thread.rb:338:in `block in load_n_threads' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:161:in `block in each_id_by_date' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:284:in `block in each_id' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:284:in `each' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:284:in `each_id' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/index.rb:161:in `each_id_by_date' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/thread.rb:334:in `load_n_threads' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/modes/thread_index_mode.rb:674:in `load_n_threads' (eval):12:in `load_n_threads' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup/modes/thread_index_mode.rb:658:in `block in load_n_threads_background' /var/lib/gems/1.9.1/gems/sup-0.15.0/lib/sup.rb:87:in `block in reporting_thread' Gem version 0.14.1.1 works fine on Debian Sid (rmail did not appear to be installed with 0.15.0 but that doesn't seem to be related to the above error). Regards, Steven Excerpts from Gaute Hope's message of 2013-11-05 02:51:42 -0500: > hi there, > > any objections to releasing sup 0.15 (I figured it deserves a whole new > version number!) with maildir-syncback as it is now? Is it ready to be > released into the wild? > > (maildir-syncback is now in 'develop' - go ahead and test if you > haven't) > > Cheers, Gaute > -- "Truth or die." Steven Hum 5 - 28 Gilmour St Ottawa, ON K2P 0N3 email sdothum at gmail.com tel 613.237.9058 From sdothum@gmail.com Mon Nov 11 13:18:45 2013 From: sdothum@gmail.com (Steven Hum) Date: Mon, 11 Nov 2013 08:18:45 -0500 Subject: [sup-devel] sup 0.15? In-Reply-To: <1384112448-sup-9012@qwerzila> References: <1383637745-sup-2758@qwerzila> <1384097900-sup-2821@luna> <1384112448-sup-9012@qwerzila> Message-ID: <1384175088-sup-8928@luna> Hello Gaute, Thanks for pointing me in the right direction. My sources.yaml file was unchanged from sup 0.14.x but only had one source defined in it. I was using a startup.rb hook to scan and define my offlineimap sources and that is where the problem seems to have originated for sup 0.15.0.. (due to a possible change to the SourceManager.add_source?) By eliminating my startup hook and manually defining all my sources, I was able to reinitialize the xapian mail database and all seems to be functioning now. Guess I will have to poke around to find out why my startup hook appears to be causing problems. Regards, Steven Excerpts from Gaute Hope's message of 2013-11-10 14:45:13 -0500: > Excerpts from Steven Hum's message of 2013-11-10 16:42:00 +0100: > > I did a gem install of 0.15.0 on a Debian Sid system and got this error: > > > > > > --- RuntimeError from thread: load threads for thread-index-mode > > invalid source 13 > > Hi Steven, > > have you changed your sources.yaml in ~/.sup somehow? It seems like sup > can't find source number 13. > > Sup 0.14.1.1 worked? If you upgraded from before 0.14 you need to follow > these instructions: > > https://github.com/sup-heliotrope/sup/wiki/Migration-0.13-to-0.14 > > Btw; I haven't seen anything about sup-rmail-1.0.1, is that a debian > package? > > - gaute -- "Truth or die." Steven Hum 5 - 28 Gilmour St Ottawa, ON K2P 0N3 email sdothum at gmail.com tel 613.237.9058 From eg@gaute.vetsj.com Mon Nov 11 13:31:57 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Mon, 11 Nov 2013 14:31:57 +0100 Subject: [sup-devel] sup 0.15? In-Reply-To: <1384175088-sup-8928@luna> References: <1383637745-sup-2758@qwerzila> <1384097900-sup-2821@luna> <1384112448-sup-9012@qwerzila> <1384175088-sup-8928@luna> Message-ID: <1384176621-sup-7237@qwerzila> Excerpts from Steven Hum's message of 2013-11-11 14:18:45 +0100: > > Hello Gaute, > > Thanks for pointing me in the right direction. My sources.yaml file was > unchanged from sup 0.14.x but only had one source defined in it. I was > using a startup.rb hook to scan and define my offlineimap sources and > that is where the problem seems to have originated for sup 0.15.0.. > (due to a possible change to the SourceManager.add_source?) > > By eliminating my startup hook and manually defining all my sources, > I was able to reinitialize the xapian mail database and all seems to be > functioning now. > > Guess I will have to poke around to find out why my startup hook > appears to be causing problems. Yes, weird. Perhaps some of the maildir-syncback startup checking has messed up the startup hook? (@ericweikl: poke.) Great that you figured it out. Regards, Gaute From eric.weikl@gmx.net Tue Nov 12 19:42:50 2013 From: eric.weikl@gmx.net (Eric Weikl) Date: Tue, 12 Nov 2013 20:42:50 +0100 Subject: [sup-devel] sup 0.15? In-Reply-To: <1384175088-sup-8928@luna> References: <1383637745-sup-2758@qwerzila> <1384097900-sup-2821@luna> <1384112448-sup-9012@qwerzila> <1384175088-sup-8928@luna> Message-ID: <1384285065-sup-2892@alderaan> Hi Steven, On 11/11/2013 14:18:45, Steven Hum wrote: > Thanks for pointing me in the right direction. My sources.yaml file was > unchanged from sup 0.14.x but only had one source defined in it. I was > using a startup.rb hook to scan and define my offlineimap sources and > that is where the problem seems to have originated for sup 0.15.0.. > (due to a possible change to the SourceManager.add_source?) > > By eliminating my startup hook and manually defining all my sources, I > was able to reinitialize the xapian mail database and all seems to be > functioning now. > > Guess I will have to poke around to find out why my startup hook > appears to be causing problems. Hmm... I'm not aware of any changes that could cause that behavior. Could you post your hook? I'd like to make sure we didn't change any expected behavior. Cheers, Eric From sdothum@gmail.com Wed Nov 13 00:47:55 2013 From: sdothum@gmail.com (Steven Hum) Date: Tue, 12 Nov 2013 19:47:55 -0500 Subject: [sup-devel] sup 0.15? In-Reply-To: <1384285065-sup-2892@alderaan> References: <1383637745-sup-2758@qwerzila> <1384097900-sup-2821@luna> <1384112448-sup-9012@qwerzila> <1384175088-sup-8928@luna> <1384285065-sup-2892@alderaan> Message-ID: <1384303088-sup-6652@luna> Hello Eric, Attached is my startup.rb file. Essentially, it scans my offlineimap folder directory for the mail folders (and labels them with their folder names -- I use imapfilter to pre-process my incoming mail and move my mail to the appropriate folders). If I initialize (rm) the xapian database, all mail is initially archived. Hope this helps. For now, I have simply created an explicit sources.yaml file (one to archive all to rebuild the mail database from scratch, one with selected archiving to simulate my startup hook) -- so sup behaves as before. Regards, Steven Excerpts from Eric Weikl's message of 2013-11-12 14:42:50 -0500: > Hi Steven, > > On 11/11/2013 14:18:45, Steven Hum wrote: > > Thanks for pointing me in the right direction. My sources.yaml file was > > unchanged from sup 0.14.x but only had one source defined in it. I was > > using a startup.rb hook to scan and define my offlineimap sources and > > that is where the problem seems to have originated for sup 0.15.0.. > > (due to a possible change to the SourceManager.add_source?) > > > > By eliminating my startup hook and manually defining all my sources, I > > was able to reinitialize the xapian mail database and all seems to be > > functioning now. > > > > Guess I will have to poke around to find out why my startup hook > > appears to be causing problems. > > Hmm... I'm not aware of any changes that could cause that behavior. > Could you post your hook? I'd like to make sure we didn't change any > expected behavior. > > Cheers, > Eric -- "Truth or die." Steven Hum 5 - 28 Gilmour St Ottawa, ON K2P 0N3 email sdothum at gmail.com tel 613.237.9058 -------------- next part -------------- A non-text attachment was scrubbed... Name: startup.rb Type: application/x-ruby Size: 1597 bytes Desc: not available URL: From eg@gaute.vetsj.com Mon Nov 25 15:24:58 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Mon, 25 Nov 2013 16:24:58 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) Message-ID: <1385392376-sup-8491@qwerzila> Hi fellow suppers, for a while I have been using a 'maildir-root' source that I created. It is based on the maildir-syncback code, but goes further: you only specify a root folder with maildirs to the source, then each sub folder (maildirsub) is mapped to a label. Some of the labels are special: like deleted, inbox and archive (the special-label to maildirsub mapping can be specified in the source configuration). This means that a sup label corresponds to maildir inside the maildir-root and the behavior works like this: - (remote to local) if a message is found in a maildirsub that label is added to it. - (remote to local) if a message is removed from a maildirsub that label is removed (unless it has just been moved). - (local to remote) if you add a label the message is written to that maildirsub (of all the maildir-roots the message belongs to) - (local to remote) if you remove a label from a message the message is deleted from that maildirsub. if it is deleted from all maildirsubs it is copied to the archive maildirsub. - (local to remote) if you archive a message it is removed from the inbox maildirsub, if it doesn't belong to any other labels/maildirsubs it is copied to archive. - (local to remote / remote to local) the same flag features as maildir-syncback (done on all copies in all maildirsubs) - (local to remote) if you add a label to a message and the message belongs to a maildir-root where no corresponding maildirsub exists a maildir will be created for it. I have been using it for 4-5 months now and it is getting fairly stable. It works fairly well on gmail (label sync), but I am also using it on regular IMAP servers (here it might, in the future, be useful to always keep a copy in archive). If you are interested in trying this please have a look at: https://github.com/gauteh/sup/blob/maildir-root/lib/sup/maildirroot.rb or my branch: https://github.com/gauteh/sup/tree/maildir-root, but please be careful. I am using this with offlineimap and utf-7-imap to utf-8 conversion. It is probably necessary to do a full sync every time to make sure things stay in sync. Remember that it is technically possible to still do changes to the maildir while offlineimap is syncing, I haven't had any problems with this (except read flag), but it is potentially a little careless. Cheers, Gaute -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From steven@schmeiser.org Mon Nov 25 18:23:24 2013 From: steven@schmeiser.org (Steven Schmeiser) Date: Mon, 25 Nov 2013 13:23:24 -0500 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385392376-sup-8491@qwerzila> References: <1385392376-sup-8491@qwerzila> Message-ID: <1385403694-sup-5692@indy.local> Hi, I'm very interested in trying out this branch. I don't know the sup internals very well, so I'm having a hard time figuring this out on my own, but what config options need to be changed/set to use maildir-root? I'm currently using marildir-sync. Thanks for all your work on sup! Steve > Hi fellow suppers, > > for a while I have been using a 'maildir-root' source that I created. > It is based on the maildir-syncback code, but goes further: you only > specify a root folder with maildirs to the source, then each sub folder > (maildirsub) is mapped to a label. Some of the labels are special: like > deleted, inbox and archive (the special-label to maildirsub mapping can > be specified in the source configuration). > > This means that a sup label corresponds to maildir inside the > maildir-root and the behavior works like this: > > - (remote to local) if a message is found in a maildirsub that label is added to it. > - (remote to local) if a message is removed from a maildirsub that > label is removed (unless it has just been moved). > - (local to remote) if you add a label the message is written to that > maildirsub (of all the maildir-roots the message belongs to) > - (local to remote) if you remove a label from a message the message is deleted > from that maildirsub. if it is deleted from all maildirsubs it is > copied to the archive maildirsub. > - (local to remote) if you archive a message it is removed from the > inbox maildirsub, if it doesn't belong to any other > labels/maildirsubs it is copied to archive. > - (local to remote / remote to local) the same flag features as > maildir-syncback (done on all copies in all maildirsubs) > - (local to remote) if you add a label to a message and the message > belongs to a maildir-root where no corresponding maildirsub exists a > maildir will be created for it. > > I have been using it for 4-5 months now and it is getting fairly stable. > It works fairly well on gmail (label sync), but I am also using it on > regular IMAP servers (here it might, in the future, be useful to always keep a copy in > archive). > > If you are interested in trying this please have a look at: > https://github.com/gauteh/sup/blob/maildir-root/lib/sup/maildirroot.rb > > or my branch: https://github.com/gauteh/sup/tree/maildir-root, but please > be careful. > > I am using this with offlineimap and utf-7-imap to utf-8 conversion. It > is probably necessary to do a full sync every time to make sure things > stay in sync. Remember that it is technically possible to still do > changes to the maildir while offlineimap is syncing, I haven't had any > problems with this (except read flag), but it is potentially a little > careless. > > > Cheers, Gaute From matthieu.rakotojaona@gmail.com Mon Nov 25 21:08:48 2013 From: matthieu.rakotojaona@gmail.com (Matthieu Rakotojaona) Date: Mon, 25 Nov 2013 22:08:48 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385392376-sup-8491@qwerzila> References: <1385392376-sup-8491@qwerzila> Message-ID: <1385413528-sup-6595@kpad> Wow, that's amazing stuff. Would you care to share your .offlineimaprc ? I myself use the gmail source (https://github.com/rakoo/sup/tree/gmail_source), but it's not very smart at syncing old labels, and it's mostly targeted towards gmail, so I'd like to switch to something else (plus having a local tree of maildirs is always helpful) -- Matthieu Rakotojaona -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From eg@gaute.vetsj.com Mon Nov 25 21:51:09 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Mon, 25 Nov 2013 22:51:09 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385392376-sup-8491@qwerzila> References: <1385392376-sup-8491@qwerzila> Message-ID: <1385415206-sup-5054@qwerzila> Hi, I have attached my relevant configs (and hopefully managed to anonymize them enough). As you can see I use the 'utf7.py' module (placed in ~/.offlineimap/) to decode/encode between imap4-utf-7 and utf-8 on system (if your filesystem uses a different encoding you might have issues). Please don't use the configs verbatim, but rather as inspiration: I might have cut out important parts. Some of this is in the wiki. Some of the folders are translated (bi-directional of course) in my offlineimaprc, but if you check out my sources.yaml you will see that I have specified which folders are related to special (deleted, starred, inbox, etc) for some of the sources. My translation setup in offlineimaprc for my first GMail account corresponds to the default values the maildirroot source suggests. The only relevant change in config.yaml is to set sync_back_to_maildir: true. The way I set up sup with maildirroot is: $ sup-config fill in, don't add any sources. If you have a working config.yaml, you can use it. add sources: $ sup-add maildirroot:///home/gaute/.mail/imap_account I edit ~/.sup/sources.yaml and enable: - confirm_enable_experimental: true - sync_back: true - maildir_creation_allowed: true and tweak the folder names if necessary. Then if your index is empty (you _cannot_ merge an existing maildir source by using the same source id) - I run sup-sync: $ sup-sync -v Note that sup-sync does not support the :updated flag (when a message has moved on disk), for that you have to run regular Sup and do a poll. sup-sync scans through and adds labels; you can also just start up sup and see what happens, but that takes a longer time: you probably do not need to enable the stuff in sources.yaml before after sup-sync if you are nervous (I would be!). Some annoying issues: - deleting a maildir is hard: * if you delete it remotely it will be recreated by offlineimap (i think) * you have to first remove all emails remotely, sync sup, to make sure labels are removed. * if you remove all messages from a label the maildir will be empty, to delete the maildir: sync, delete the maildir locally + delete it remotely; then re-sync. this is the behaviour of offlineimap, but I think it is hard to do it differently. this means that if you accidentally add a bogus label, it is a bit of a pain to get rid of the maildir. - if a message is deleted from the inbox remotely (archived) it is not removed from the inbox view (or any thread index view it is visible in) before sup is restarted. To test maildirroot: checkout my branch; make the gem: $ rake gem and install it: $ gem install pkg/sup-999.gem. If develop has changed you can try to merge it in first. Again; please make a backup copy of your .mail so that you can restore it if it turns out maildirroot has been eating or multiplying emails :) I probably don't have the time to difficult bugs, but I would be happy to accept patches and I could also put this branch on official Sup if it turns out to be worth something. Cheers, Gaute -------------- next part -------------- A non-text attachment was scrubbed... Name: offlineimaprc Type: application/octet-stream Size: 3998 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: utf7.py Type: application/x-python Size: 2213 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sources.yaml Type: text/x-yaml Size: 1560 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: config.yaml Type: text/x-yaml Size: 603 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From eg@gaute.vetsj.com Mon Nov 25 22:28:32 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Mon, 25 Nov 2013 23:28:32 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385415206-sup-5054@qwerzila> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> Message-ID: <1385418447-sup-2385@qwerzila> By the way: sub dirs show up with dots between them so maildir: dev/sup becomes label dev.sup (can't remember if this is standard offlineimap behavior or something I did). - gaute -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From steven@schmeiser.org Tue Nov 26 14:43:49 2013 From: steven@schmeiser.org (Steven Schmeiser) Date: Tue, 26 Nov 2013 09:43:49 -0500 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385415206-sup-5054@qwerzila> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> Message-ID: <1385476504-sup-9378@indy.local> I've installed the branch and am trying it out. So far it looks good, but there have been a few oddities... I'm running with a standard IMAP account (not Gmail). The root looks like [steve at indy ~]$ ls -l .mail total 0 drwx------ 5 steve staff 170 Nov 19 13:44 Archive drwx------ 5 steve staff 170 Nov 22 12:31 INBOX drwx------ 5 steve staff 170 Nov 19 13:44 drafts drwx------ 5 steve staff 170 Nov 19 13:44 econpapers drwx------ 5 steve staff 170 Nov 19 13:44 sent drwx------ 5 steve staff 170 Nov 19 13:44 spam drwx------ 5 steve staff 170 Nov 25 19:25 trash and my sources are configured as follows [steve at indy ~]$ more .sup/sources.yaml --- - !supmua.org,2006-10-01/Redwood/MaildirRoot uri: maildirroot:///Users/steve/.mail usual: true archived: false id: 1 labels: [] sync_back: true confirm_enable_experimental: true maildir_creation_allowed: true inbox_folder: INBOX sent_folder: sent drafts_folder: drafts spam_folder: spam trash_folder: trash archive_folder: Archive - !supmua.org,2006-10-01/Redwood/SentLoader{} I had around 40k messages in the Archive folder and ~20 in the INBOX. After sup-sync, around 10k of the messages in Archive were moved into sup's inbox with labels 'INBOX' and 'Inbox'. They were also (incorrectly) marked as unread. I'll keep experimenting. Thanks for your effort -- I think this is a great new feature and would really like to see it merged into the mainline someday. Steve > Hi, > > I have attached my relevant configs (and hopefully managed to anonymize > them enough). As you can see I use the 'utf7.py' module (placed in > ~/.offlineimap/) to decode/encode between imap4-utf-7 and utf-8 on > system (if your filesystem uses a different encoding you might have > issues). Please don't use the configs verbatim, but rather as > inspiration: I might have cut out important parts. Some of this is > in the wiki. > > Some of the folders are translated (bi-directional of course) in my > offlineimaprc, but if you check out my sources.yaml you will see that I > have specified which folders are related to special (deleted, starred, > inbox, etc) for some of the sources. > > My translation setup in offlineimaprc for my first GMail account > corresponds to the default values the maildirroot source suggests. The > only relevant change in config.yaml is to set sync_back_to_maildir: > true. > > The way I set up sup with maildirroot is: > $ sup-config > > fill in, don't add any sources. If you have a working config.yaml, you > can use it. > > add sources: > $ sup-add maildirroot:///home/gaute/.mail/imap_account > > I edit ~/.sup/sources.yaml and enable: > - confirm_enable_experimental: true > - sync_back: true > - maildir_creation_allowed: true > > and tweak the folder names if necessary. > > Then if your index is empty (you _cannot_ merge an existing maildir > source by using the same source id) - I run sup-sync: > $ sup-sync -v > > Note that sup-sync does not support the :updated flag (when a message > has moved on disk), for that you have to run regular Sup and do a poll. > sup-sync scans through and adds labels; you can also just start up sup > and see what happens, but that takes a longer time: you probably do not > need to enable the stuff in sources.yaml before after sup-sync if you > are nervous (I would be!). > > Some annoying issues: > - deleting a maildir is hard: > * if you delete it remotely it will be recreated by offlineimap (i > think) > * you have to first remove all emails remotely, sync sup, to make sure > labels are removed. > * if you remove all messages from a label the maildir will be empty, > to delete the maildir: sync, delete the maildir locally + delete it > remotely; then re-sync. > > this is the behaviour of offlineimap, but I think it is hard to do > it differently. this means that if you accidentally add a bogus > label, it is a bit of a pain to get rid of the maildir. > > > - if a message is deleted from the inbox remotely (archived) it is not > removed from the inbox view (or any thread index view it is visible > in) before sup is restarted. > > To test maildirroot: > checkout my branch; make the gem: $ rake gem and install it: $ gem > install pkg/sup-999.gem. If develop has changed you can try to merge it in > first. > > Again; please make a backup copy of your .mail so that you can restore > it if it turns out maildirroot has been eating or multiplying emails :) > > I probably don't have the time to difficult bugs, but I would be happy > to accept patches and I could also put this branch on official Sup if it > turns out to be worth something. > > > Cheers, Gaute From eg@gaute.vetsj.com Tue Nov 26 15:14:25 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Tue, 26 Nov 2013 16:14:25 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385476504-sup-9378@indy.local> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> Message-ID: <1385478548-sup-9110@qwerzila> Excerpts from Steven Schmeiser's message of 2013-11-26 15:43:49 +0100: > I've installed the branch and am trying it out. So far it looks good, but there have been a few oddities... > > I'm running with a standard IMAP account (not Gmail). The root looks like > > [steve at indy ~]$ ls -l .mail > total 0 > drwx------ 5 steve staff 170 Nov 19 13:44 Archive > drwx------ 5 steve staff 170 Nov 22 12:31 INBOX > drwx------ 5 steve staff 170 Nov 19 13:44 drafts > drwx------ 5 steve staff 170 Nov 19 13:44 econpapers > drwx------ 5 steve staff 170 Nov 19 13:44 sent > drwx------ 5 steve staff 170 Nov 19 13:44 spam > drwx------ 5 steve staff 170 Nov 25 19:25 trash > > and my sources are configured as follows > > [steve at indy ~]$ more .sup/sources.yaml > --- > - !supmua.org,2006-10-01/Redwood/MaildirRoot > uri: maildirroot:///Users/steve/.mail > usual: true > archived: false > id: 1 > labels: [] > sync_back: true > confirm_enable_experimental: true > maildir_creation_allowed: true > inbox_folder: INBOX > sent_folder: sent > drafts_folder: drafts > spam_folder: spam > trash_folder: trash > archive_folder: Archive > - !supmua.org,2006-10-01/Redwood/SentLoader{} > > I had around 40k messages in the Archive folder and ~20 in the INBOX. After sup-sync, around 10k of the messages in Archive were moved into sup's inbox with labels 'INBOX' and 'Inbox'. They were also (incorrectly) marked as unread. > > I'll keep experimenting. Thanks for your effort -- I think this is a great new feature and would really like to see it merged into the mainline someday. Ok, there are likely to be a few more of the oddities - you are brave :) Please set the environment variable: SUP_LOG_LEVEL=debug and try to start sup, it would be interesting to see how the maildirsubs are initialized (the startup section from ~/.sup/log). Were any of the messages in Archive also somewhere else (duplicates with the same id)? Also, if sup discovers a message that is part of a thread it will join them and (possibly) apply all the labels to all the messages in that thread. If any of those are unread it might be affected. The labels 'INBOX' and 'Inbox' are both not the same as the special label 'inbox', so apparently it is not recognized as the special label but a custom label. It could be that you have more luck with starting with a clean index and just starting regular sup (not sup-sync). You can follow the log by doing: tail -f ~/.sup/log. - gaute From steven@schmeiser.org Tue Nov 26 16:42:48 2013 From: steven@schmeiser.org (Steven Schmeiser) Date: Tue, 26 Nov 2013 11:42:48 -0500 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385478548-sup-9110@qwerzila> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> <1385478548-sup-9110@qwerzila> Message-ID: <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> On Nov 26, 2013, at 10:14, Gaute Hope wrote: > Excerpts from Steven Schmeiser's message of 2013-11-26 15:43:49 +0100: >> I've installed the branch and am trying it out. So far it looks good, but there have been a few oddities... >> >> I'm running with a standard IMAP account (not Gmail). The root looks like >> >> [steve at indy ~]$ ls -l .mail >> total 0 >> drwx------ 5 steve staff 170 Nov 19 13:44 Archive >> drwx------ 5 steve staff 170 Nov 22 12:31 INBOX >> drwx------ 5 steve staff 170 Nov 19 13:44 drafts >> drwx------ 5 steve staff 170 Nov 19 13:44 econpapers >> drwx------ 5 steve staff 170 Nov 19 13:44 sent >> drwx------ 5 steve staff 170 Nov 19 13:44 spam >> drwx------ 5 steve staff 170 Nov 25 19:25 trash >> >> and my sources are configured as follows >> >> [steve at indy ~]$ more .sup/sources.yaml >> --- >> - !supmua.org,2006-10-01/Redwood/MaildirRoot >> uri: maildirroot:///Users/steve/.mail >> usual: true >> archived: false >> id: 1 >> labels: [] >> sync_back: true >> confirm_enable_experimental: true >> maildir_creation_allowed: true >> inbox_folder: INBOX >> sent_folder: sent >> drafts_folder: drafts >> spam_folder: spam >> trash_folder: trash >> archive_folder: Archive >> - !supmua.org,2006-10-01/Redwood/SentLoader{} >> >> I had around 40k messages in the Archive folder and ~20 in the INBOX. After sup-sync, around 10k of the messages in Archive were moved into sup's inbox with labels 'INBOX' and 'Inbox'. They were also (incorrectly) marked as unread. >> >> I'll keep experimenting. Thanks for your effort -- I think this is a great new feature and would really like to see it merged into the mainline someday. > > Ok, there are likely to be a few more of the oddities - you are brave :) > > Please set the environment variable: SUP_LOG_LEVEL=debug and try to > start sup, it would be interesting to see how the maildirsubs are > initialized (the startup section from ~/.sup/log). > > Were any of the messages in Archive also somewhere else (duplicates with > the same id)? Also, if sup discovers a message that is part of a thread > it will join them and (possibly) apply all the labels to all the > messages in that thread. If any of those are unread it might be > affected. > > The labels 'INBOX' and 'Inbox' are both not the same as the special > label 'inbox', so apparently it is not recognized as the special label > but a custom label. > > It could be that you have more luck with starting with a clean index and > just starting regular sup (not sup-sync). You can follow the log by > doing: tail -f ~/.sup/log. > > - gaute Startup log follows... [2013-11-26 11:34:20 -0500] using character set encoding "UTF-8" [2013-11-26 11:34:21 -0500] dynamically loading setlocale() from libc.dylib [2013-11-26 11:34:21 -0500] setting locale... [2013-11-26 11:34:21 -0500] locking /Users/steve/.sup/lock... [2013-11-26 11:34:21 -0500] maildirroot:///Users/steve/.mail: setting up maildirroot.. [2013-11-26 11:34:21 -0500] setting up maildir subs.. [2013-11-26 11:34:21 -0500] maildirsub set up, type: archive, label: archive [2013-11-26 11:34:21 -0500] maildirsub set up, type: inbox, label: inbox [2013-11-26 11:34:21 -0500] maildirsub set up, type: sent, label: sent [2013-11-26 11:34:21 -0500] maildirsub set up, type: draft, label: draft [2013-11-26 11:34:21 -0500] maildirsub set up, type: spam, label: spam [2013-11-26 11:34:21 -0500] maildirsub set up, type: deleted, label: deleted [2013-11-26 11:34:21 -0500] setting up generic folders.. [2013-11-26 11:34:21 -0500] maildirsub set up, type: generic, label: econpapers [2013-11-26 11:34:21 -0500] no draft source, auto-adding... [2013-11-26 11:34:21 -0500] starting curses [2013-11-26 11:34:21 -0500] loading user colors from /Users/steve/.sup/colors.yaml [2013-11-26 11:34:21 -0500] initializing log buffer [2013-11-26 11:34:21 -0500] Welcome to Sup! Log level is set to debug. [2013-11-26 11:34:21 -0500] initializing inbox buffer [2013-11-26 11:34:21 -0500] ready for interaction! I'm now not able to run sup-sync or open some of the threads in my inbox without crashing with the following error: /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/bin/sup-sync:180:in `block (2 levels) in ': NotImplementedError (NotImplementedError) from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/poll.rb:292:in `block (2 levels) in poll_from' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/index.rb:298:in `block in each_message' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/index.rb:285:in `block in each_id' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/index.rb:285:in `each' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/index.rb:285:in `each_id' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/index.rb:297:in `each_message' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/util.rb:649:in `method_missing' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/poll.rb:251:in `block in poll_from' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/maildirroot.rb:512:in `block (2 levels) in poll' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/maildirroot.rb:511:in `each' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/maildirroot.rb:511:in `block in poll' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/util.rb:261:in `call' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/util.rb:261:in `block in benchmark' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/1.9.1/benchmark.rb:280:in `measure' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/util.rb:261:in `benchmark' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/maildirroot.rb:474:in `poll' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/poll.rb:201:in `poll_from' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/lib/sup/util.rb:649:in `method_missing' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/bin/sup-sync:125:in `block in ' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/bin/sup-sync:120:in `each' from /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/bin/sup-sync:120:in `' from /usr/local/opt/ruby193/bin/sup-sync:23:in `load' from /usr/local/opt/ruby193/bin/sup-sync:23:in `
' I'll try starting with a clean index. Steve From eg@gaute.vetsj.com Tue Nov 26 16:55:16 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Tue, 26 Nov 2013 17:55:16 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> <1385478548-sup-9110@qwerzila> <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> Message-ID: <1385484573-sup-869@qwerzila> Excerpts from Steven Schmeiser's message of 2013-11-26 17:42:48 +0100: > > On Nov 26, 2013, at 10:14, Gaute Hope wrote: > > > Excerpts from Steven Schmeiser's message of 2013-11-26 15:43:49 +0100: > [2013-11-26 11:34:21 -0500] maildirsub set up, type: deleted, label: deleted > [2013-11-26 11:34:21 -0500] setting up generic folders.. > [2013-11-26 11:34:21 -0500] maildirsub set up, type: generic, label: econpapers > [2013-11-26 11:34:21 -0500] no draft source, auto-adding... > [2013-11-26 11:34:21 -0500] starting curses > [2013-11-26 11:34:21 -0500] loading user colors from /Users/steve/.sup/colors.yaml > [2013-11-26 11:34:21 -0500] initializing log buffer > [2013-11-26 11:34:21 -0500] Welcome to Sup! Log level is set to debug. > [2013-11-26 11:34:21 -0500] initializing inbox buffer > [2013-11-26 11:34:21 -0500] ready for interaction! Looks good: Only econpapers is identified as a generic label, the other folders are mapped to one of the specials. > > I'm now not able to run sup-sync or open some of the threads in my inbox without crashing with the following error: > > /usr/local/Cellar/ruby193/1.9.3-p448/lib/ruby/gems/1.9.1/gems/sup-999/bin/sup-sync:180:in `block (2 levels) in ': NotImplementedError (NotImplementedError) I haven't implemented :updated in sup-sync yet (see first, or possibly second email). To update: start Sup normally, let the Sup poll _finish_ before you try to open the problematic message. This happens now and then, when you label a message (copy to a label dir) it gets a generic name, upon offlinimap sync it is renamed (by offlineimap) and untill a new poll has been run the source is out of sync. Since the poll has to finish to fix the error it is easy to trigger it again when if you start sup again. > I'll try starting with a clean index. Always a good idea :) - gaute From steven@schmeiser.org Wed Nov 27 00:48:20 2013 From: steven@schmeiser.org (Steven Schmeiser) Date: Tue, 26 Nov 2013 19:48:20 -0500 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385484573-sup-869@qwerzila> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> <1385478548-sup-9110@qwerzila> <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> <1385484573-sup-869@qwerzila> Message-ID: <1385513221-sup-4344@indy.local> > Excerpts from Steven Schmeiser's message of 2013-11-26 17:42:48 +0100: > > > > On Nov 26, 2013, at 10:14, Gaute Hope wrote: > > > > > Excerpts from Steven Schmeiser's message of 2013-11-26 15:43:49 +0100: > > [2013-11-26 11:34:21 -0500] maildirsub set up, type: deleted, label: deleted > > [2013-11-26 11:34:21 -0500] setting up generic folders.. > > [2013-11-26 11:34:21 -0500] maildirsub set up, type: generic, label: econpapers > > [2013-11-26 11:34:21 -0500] no draft source, auto-adding... > > [2013-11-26 11:34:21 -0500] starting curses > > [2013-11-26 11:34:21 -0500] loading user colors from /Users/steve/.sup/colors.yaml > > [2013-11-26 11:34:21 -0500] initializing log buffer > > [2013-11-26 11:34:21 -0500] Welcome to Sup! Log level is set to debug. > > [2013-11-26 11:34:21 -0500] initializing inbox buffer > > [2013-11-26 11:34:21 -0500] ready for interaction! > > Looks good: Only econpapers is identified as a generic label, the other > folders are mapped to one of the specials. It turned out to be a before-add-message hook that was causing all the problems. Working great now. Steve From eg@gaute.vetsj.com Wed Nov 27 06:21:41 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Wed, 27 Nov 2013 07:21:41 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385513221-sup-4344@indy.local> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> <1385478548-sup-9110@qwerzila> <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> <1385484573-sup-869@qwerzila> <1385513221-sup-4344@indy.local> Message-ID: <1385533106-sup-5646@qwerzila> Excerpts from Steven Schmeiser's message of 2013-11-27 01:48:20 +0100: > > > Excerpts from Steven Schmeiser's message of 2013-11-26 17:42:48 +0100: > > > > > > On Nov 26, 2013, at 10:14, Gaute Hope wrote: > > > > > > > Excerpts from Steven Schmeiser's message of 2013-11-26 15:43:49 +0100: > > > [2013-11-26 11:34:21 -0500] maildirsub set up, type: deleted, label: deleted > > > [2013-11-26 11:34:21 -0500] setting up generic folders.. > > > [2013-11-26 11:34:21 -0500] maildirsub set up, type: generic, label: econpapers > > > [2013-11-26 11:34:21 -0500] no draft source, auto-adding... > > > [2013-11-26 11:34:21 -0500] starting curses > > > [2013-11-26 11:34:21 -0500] loading user colors from /Users/steve/.sup/colors.yaml > > > [2013-11-26 11:34:21 -0500] initializing log buffer > > > [2013-11-26 11:34:21 -0500] Welcome to Sup! Log level is set to debug. > > > [2013-11-26 11:34:21 -0500] initializing inbox buffer > > > [2013-11-26 11:34:21 -0500] ready for interaction! > > > > Looks good: Only econpapers is identified as a generic label, the other > > folders are mapped to one of the specials. > > It turned out to be a before-add-message hook that was causing all the problems. Working great now. Cool! Let me know how it works in the longer run! I was thinking about changing one thing: currently messages are not copied into 'archive' before it has been _no_ other labels. On GMail you can still find it on the 'All mail' page on the web interface, but for regular IMAP accounts that doesn't work. Should the message always be copied to 'archive' when it is removed from the inbox? - gaute From eg@gaute.vetsj.com Wed Nov 27 06:23:52 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Wed, 27 Nov 2013 07:23:52 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385484573-sup-869@qwerzila> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> <1385478548-sup-9110@qwerzila> <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> <1385484573-sup-869@qwerzila> Message-ID: <1385533325-sup-4841@qwerzila> Excerpts from Gaute Hope's message of 2013-11-26 17:55:16 +0100: > I haven't implemented :updated in sup-sync yet (see first, or possibly > second email). To update: start Sup normally, let the Sup poll _finish_ > before you try to open the problematic message. This happens now and > then, when you label a message (copy to a label dir) it gets a generic > name, upon offlinimap sync it is renamed (by offlineimap) and untill a > new poll has been run the source is out of sync. Since the poll has to > finish to fix the error it is easy to trigger it again when if you start > sup again. This, by the way, is also a problem in current Sup (maildir-syncback) - :updated is not handled in sup-sync. Only in lib/sup/poll.rb. - gaute -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From steven@schmeiser.org Wed Nov 27 12:42:33 2013 From: steven@schmeiser.org (Steven Schmeiser) Date: Wed, 27 Nov 2013 07:42:33 -0500 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385533106-sup-5646@qwerzila> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> <1385478548-sup-9110@qwerzila> <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> <1385484573-sup-869@qwerzila> <1385513221-sup-4344@indy.local> <1385533106-sup-5646@qwerzila> Message-ID: <1385555365-sup-7389@indy.local> > Excerpts from Steven Schmeiser's message of 2013-11-27 01:48:20 +0100: > > > > > Excerpts from Steven Schmeiser's message of 2013-11-26 17:42:48 +0100: > > > > > > > > On Nov 26, 2013, at 10:14, Gaute Hope wrote: > > > > > > > > > Excerpts from Steven Schmeiser's message of 2013-11-26 15:43:49 +0100: > > > > [2013-11-26 11:34:21 -0500] maildirsub set up, type: deleted, label: deleted > > > > [2013-11-26 11:34:21 -0500] setting up generic folders.. > > > > [2013-11-26 11:34:21 -0500] maildirsub set up, type: generic, label: econpapers > > > > [2013-11-26 11:34:21 -0500] no draft source, auto-adding... > > > > [2013-11-26 11:34:21 -0500] starting curses > > > > [2013-11-26 11:34:21 -0500] loading user colors from /Users/steve/.sup/colors.yaml > > > > [2013-11-26 11:34:21 -0500] initializing log buffer > > > > [2013-11-26 11:34:21 -0500] Welcome to Sup! Log level is set to debug. > > > > [2013-11-26 11:34:21 -0500] initializing inbox buffer > > > > [2013-11-26 11:34:21 -0500] ready for interaction! > > > > > > Looks good: Only econpapers is identified as a generic label, the other > > > folders are mapped to one of the specials. > > > > It turned out to be a before-add-message hook that was causing all the problems. Working great now. > > Cool! Let me know how it works in the longer run! > > I was thinking about changing one thing: currently messages are not > copied into 'archive' before it has been _no_ other labels. On GMail you > can still find it on the 'All mail' page on the web interface, but for > regular IMAP accounts that doesn't work. Should the message always be > copied to 'archive' when it is removed from the inbox? > > - gaute I don't have much personal preference between these two options. The second option is closer to Gmail's "All Mail" folder that a lot of people are familiar with, so maybe that is the way to go. Especially if the "copy" is a hard link that doesn't take up disk space. Steve From eg@gaute.vetsj.com Wed Nov 27 13:08:36 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Wed, 27 Nov 2013 14:08:36 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385555365-sup-7389@indy.local> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> <1385478548-sup-9110@qwerzila> <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> <1385484573-sup-869@qwerzila> <1385513221-sup-4344@indy.local> <1385533106-sup-5646@qwerzila> <1385555365-sup-7389@indy.local> Message-ID: <1385557223-sup-3280@qwerzila> Excerpts from Steven Schmeiser's message of 2013-11-27 13:42:33 +0100: > > I was thinking about changing one thing: currently messages are not > > copied into 'archive' before it has been _no_ other labels. On GMail > > you can still find it on the 'All mail' page on the web interface, > > but for regular IMAP accounts that doesn't work. Should the message > > always be copied to 'archive' when it is removed from the inbox? > > > > - gaute > > I don't have much personal preference between these two options. The > second option is closer to Gmail's "All Mail" folder that a lot of > people are familiar with, so maybe that is the way to go. Especially > if the "copy" is a hard link that doesn't take up disk space. I noticed Gmail does it that way in their IMAP interface (archive on no other labels), thats why I did it. I guess it doesn't matter all that much as long as we are all using Sup. Archive becomes a place for the uncategorized. I don't delete an email from archive if it is already in there, so if it at some point has been copied in there - it stays. Copy is hard-link, but I doubt offlineimap is able to identify remote identical emails. So hard-linking is only on the machine that the link happened on. - gaute From eg@gaute.vetsj.com Wed Nov 27 13:24:15 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Wed, 27 Nov 2013 14:24:15 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385557223-sup-3280@qwerzila> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> <1385478548-sup-9110@qwerzila> <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> <1385484573-sup-869@qwerzila> <1385513221-sup-4344@indy.local> <1385533106-sup-5646@qwerzila> <1385555365-sup-7389@indy.local> <1385557223-sup-3280@qwerzila> Message-ID: <1385558459-sup-8094@qwerzila> Excerpts from Gaute Hope's message of 2013-11-27 14:08:36 +0100: > Excerpts from Steven Schmeiser's message of 2013-11-27 13:42:33 +0100: > > > I was thinking about changing one thing: currently messages are not > > > copied into 'archive' before it has been _no_ other labels. On GMail > > > you can still find it on the 'All mail' page on the web interface, > > > but for regular IMAP accounts that doesn't work. Should the message > > > always be copied to 'archive' when it is removed from the inbox? > > > > > > - gaute > > > > I don't have much personal preference between these two options. The > > second option is closer to Gmail's "All Mail" folder that a lot of > > people are familiar with, so maybe that is the way to go. Especially > > if the "copy" is a hard link that doesn't take up disk space. > > I noticed Gmail does it that way in their IMAP interface (archive on no > other labels), thats why I did it. I guess it doesn't matter all that > much as long as we are all using Sup. Archive becomes a place for the > uncategorized. I don't delete an email from archive if it is already in > there, so if it at some point has been copied in there - it stays. > > Copy is hard-link, but I doubt offlineimap is able to identify remote > identical emails. So hard-linking is only on the machine that the > link happened on. Another nice feature is that I now have the possibility to have my filters on the server copy/move the message to a IMAP folder and it will be automatically tagged in Sup as well. Some servers automatically delete stuff in Trash / Deleted Items or email marked as Trash (T), these will also be removed from Sup when they are deleted (might not work when it is in several folders and only marked with T). - gaute From matthieu.rakotojaona@gmail.com Wed Nov 27 18:32:02 2013 From: matthieu.rakotojaona@gmail.com (Matthieu Rakotojaona) Date: Wed, 27 Nov 2013 19:32:02 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385557223-sup-3280@qwerzila> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> <1385478548-sup-9110@qwerzila> <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> <1385484573-sup-869@qwerzila> <1385513221-sup-4344@indy.local> <1385533106-sup-5646@qwerzila> <1385555365-sup-7389@indy.local> <1385557223-sup-3280@qwerzila> Message-ID: <1385576444-sup-3271@kpad> Excerpts from Gaute Hope's message of 2013-11-27 14:08:36 +0100: > I noticed Gmail does it that way in their IMAP interface (archive on no > other labels), thats why I did it. _Any_ email in gmail is also in the All Mail [0], except when it is manually moved to the [Gmail]/Trash folder, in which case it disappears forever (well, at least for you :). I think this is the best solution, because it is the most understandable for all MUAs out there: the fact that "a maildir is a label" is something pretty specific to gmail-pioneered MUAs, so at least when I backup the [Gmail]/All Mail I know I won't have lost any of them (even if I have lost the labels). Regarding hardlinking, I'm afraid of the cross-platform issues. I know some people here use Mac OSX, is it safe to use hardlinking on this platform ? (Oh and by the way I found a script [1] that basically hardlinks every copy of an email, this could be especially useful for gmail-based installations) [0] https://support.google.com/mail/answer/78755?hl=en [1] http://jak-linux.org/projects/hardlink/ -- Matthieu Rakotojaona -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From eg@gaute.vetsj.com Wed Nov 27 21:10:38 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Wed, 27 Nov 2013 22:10:38 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385576444-sup-3271@kpad> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> <1385478548-sup-9110@qwerzila> <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> <1385484573-sup-869@qwerzila> <1385513221-sup-4344@indy.local> <1385533106-sup-5646@qwerzila> <1385555365-sup-7389@indy.local> <1385557223-sup-3280@qwerzila> <1385576444-sup-3271@kpad> Message-ID: <1385584974-sup-8936@qwerzila> Excerpts from Matthieu Rakotojaona's message of 2013-11-27 19:32:02 +0100: > Excerpts from Gaute Hope's message of 2013-11-27 14:08:36 +0100: > > I noticed Gmail does it that way in their IMAP interface (archive on no > > other labels), thats why I did it. > > _Any_ email in gmail is also in the All Mail [0], except when it is > manually moved to the [Gmail]/Trash folder, in which case it disappears > forever (well, at least for you :). I think this is the best solution, > because it is the most understandable for all MUAs out there: the fact > that "a maildir is a label" is something pretty specific to > gmail-pioneered MUAs, so at least when I backup the [Gmail]/All Mail I > know I won't have lost any of them (even if I have lost the labels). Hm ok. I just figured it out by trail-and-error, I probably missed something. I have changed sync_back to always make sure the message is present in the archive. This does not have any effect on existing messages before you do some edit that triggers sync_back. It also currently does not check new messages (since non-gmail sources will not have a copy in archive), we should do a check upon :add in poll.rb#do_poll to the source that checks if existing_sources has @archive, if not add and return to poll with updated locations (no sync_back needed and no index sync needed as poll will do that afterwards). About deletion, current behavior: - apply :deleted label; this means: - the message is moved to the trash folder - the T flag is added to all the labels it is part of this might not work exactly as it should on GMail, but it is the only way to preserve labels (in case of restore). > Regarding hardlinking, I'm afraid of the cross-platform issues. I know > some people here use Mac OSX, is it safe to use hardlinking on this > platform ? I am using the util function I created for maildir-sync (in develop) which falls back to copying the file if hardlinking fails, that function works on Mac OS X. Hardlinking doesn't work for me since I am using encfs to store mail. > (Oh and by the way I found a script [1] that basically hardlinks every > copy of an email, this could be especially useful for gmail-based > installations) > > [0] https://support.google.com/mail/answer/78755?hl=en > [1] http://jak-linux.org/projects/hardlink/ That could be very useful and run as a cronjob now and then. - gaute -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From eg@gaute.vetsj.com Fri Nov 29 09:22:44 2013 From: eg@gaute.vetsj.com (Gaute Hope) Date: Fri, 29 Nov 2013 10:22:44 +0100 Subject: [sup-devel] A maildir root source: syncing labels with maildirs (imap folders) In-Reply-To: <1385584974-sup-8936@qwerzila> References: <1385392376-sup-8491@qwerzila> <1385415206-sup-5054@qwerzila> <1385476504-sup-9378@indy.local> <1385478548-sup-9110@qwerzila> <2AC33855-6B00-42FB-9F3B-D160D8B556D4@schmeiser.org> <1385484573-sup-869@qwerzila> <1385513221-sup-4344@indy.local> <1385533106-sup-5646@qwerzila> <1385555365-sup-7389@indy.local> <1385557223-sup-3280@qwerzila> <1385576444-sup-3271@kpad> <1385584974-sup-8936@qwerzila> Message-ID: <1385716758-sup-298@qwerzila> Excerpts from Gaute Hope's message of 2013-11-27 22:10:38 +0100: > Excerpts from Matthieu Rakotojaona's message of 2013-11-27 19:32:02 +0100: > > Excerpts from Gaute Hope's message of 2013-11-27 14:08:36 +0100: [..] > It also > currently does not check new messages (since non-gmail sources will not > have a copy in archive), we should do a check upon :add in > poll.rb#do_poll to the source that checks if existing_sources has > @archive, if not add and return to poll with updated locations (no > sync_back needed and no index sync needed as poll will do that > afterwards). As of commit db5e75f we check in poll if the message is in archive, otherwise add (archive is the first subdir to be polled so it should always be present). Another issue is messages that get labels added automatically, these are not synced back before the message is opened or somehow otherwise activated. Could possibly do a full sync_back after :add, :delete or :update in poll. - gaute -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: