Sorry the patch I sent has a small problem. Use this patch instead. On Tue, May 17, 2011 at 11:59 PM, Horacio Sanson wrote: > I implemented a new version of the GMail -> Heliotrope sync script and > attach it here in hopes > someone will test it and provide some feedback/comments. This is by no > means ready for general > use so don't use for anything else than experimentation. But since > this script only opens mailboxes > in read only mode (examine) there should not be any problems with you > emails on the server. > > This script is GMail specific and has these features: > >  - Downloads emails from all mailboxes (except All, Trash and Spam) > automatically using >    the XLIST GMail IMAP extension and feeds them to Heliotrope via > REST interface. >  - Remembers the last email downloaded so it does not start from the beginning >    every time. >  - Synchronizes GMail labels using the X-GM-LABELS IMAP extension. >  - Synchronizes GMail flags with Heliotrope state flags. >  - Adds a new mailbox property to messages. This may allow later to implement >    Heliotrope -> GMail synchronization. > > Things to check and do: >  - I am seeing some negative thread_id's in the response. Need to check if >    this is normal or a bug in Heliotrope or my script. >  - GMail inbox is with a capital "I" (e.g. Inbox) while heliotrope > uses a small "i". >    Shall I down case all labels? or make a special treatment for Inbox? >  - Refactor the script into something more modular and elegant. > > To use this script I had to modify heliotrope-server.rb to allow > setting labels and states when > posting new messages (see attached patch). > > regards, > Horacio > > On Tue, May 17, 2011 at 12:02 AM, Horacio Sanson wrote: >> On Mon, May 16, 2011 at 12:01 AM, William Morgan >> wrote: >>> Reformatted excerpts from Horacio Sanson's message of 2011-05-10: >>>> Is there a way to query Heliotrope what is the largest msg_id >>>> currently in the index? >>> >>> Sort of---it's a hack, but if you search for e.g. "a OR -a" you'll get >>> every message in the index, and the first result will be the message >>> with the highest id thanks to Whistlepig's search semantics. >>> >> >> Indeed I have been trying to wrap my head around the IMAP spec and >> still don't get a lot of things. For now I will just keep the max UID >> read from IMAP server somewhere on disk. >> >>>> I am trying to improve the imap-dumper.rb so >>>> it does not download all my emails every time but only the new ones. >>> >>> Sounds great. Unfortunately the Heliotrope message id and the IMAP >>> message id / message uid are completely different things, and >>> maintaining a cross-session mapping of them is impossible for generic >>> IMAP servers, because the uid of every message can change every time you >>> connect to an IMAP server---see the section on IMAP's 'uidvalidity' >>> variable. So you'll have to rescan the inbox every time and rebuild the >>> mapping. Welcome to hell. >>> >> >> When UIDVALIDITY differs I will simply re-scan the whole mailbox and >> feed it to Heliotrope. I trust Heliotrope won't add duplicates. >> >>>> Also while looking at the code I see that messages are stored in the >>>> index using the msg_id as parsed by RMail. There is no further >>>> association with the source or mailbox from where the messages were >>>> downloaded. This I think may cause collisions if we use one Heliotrope >>>> server with more than one email account. Not sure what is the >>>> probability of two messages from two different IMAP servers having the >>>> same msg_id but nothing in the standard rules out that possibility. >>> >>> This is yet another id: the Message-Id header of the email. This is only >>> needed to build up the thread structure and should otherwise be ignored. >> >> I am attaching my first small hack for GMail <-> Heliotrope >> synchronization. For now it only downloads mail from GMail and injects >> them to Heliotrope just as the imap-dumper.rb does. The difference is >> that I keep track of the last message UID and UIDVALIDITY values to >> avoid re-scanning the whole folder every time. >> >> Now I wan't to take advantage of GMail IMAP extensions (e.g. >> X-GM-LABELS, X-GM-THRID) to allow labels/threads synchronization. But >> have some doubts about how to correctly use the Heliotrope REST API. >> For example in the Heliotrope::Index the add_message method allows to >> insert a message and assign it labels, flags and extra parameters at >> the same time. How can I do this with the REST API? The only example I >> see only adds a message body. >> >>    RestClient.post "http://localhost:8042/message", :message => body >> >> Also for what purpose are the ext array used for? Can I use it to add >> an account/mailbox property to each message so I can latter retrieve >> all messages associated to a mailbox/account pair? >> >> regards, >> Horacio >> >>> -- >>> William >>> _______________________________________________ >>> Sup-devel mailing list >>> Sup-devel@rubyforge.org >>> http://rubyforge.org/mailman/listinfo/sup-devel >>> >> >