Subject: Re: [sup-devel] Heliotrope limitations for backward synchronization, , ,

From: Vivien Didelot <[email-masked]>
Date: Mon, 23 May 2011 18:23:58 -0400

Hi Horacio,

FWIW, you may use -- or even just take a look at -- the 'gmail' gem.
It works pretty well for back syncing and is built on RFCs too (if I'm not
mistaken).
For maildir backsyncing, the 'maildir' gem also do the job,
both are using the great 'mail' gem (which I hope, Heliotrope & co will use
soon).

Vivien.

On Mon, May 23, 2011 at 10:42 AM, Horacio Sanson <[email-masked]> wrote:

> Now that I have GMail to Heliotrope initial synchronization (first
> time) and incremental (new messages) synchronization working in my
> script I started working on Heliotrope to GMail synchronization.
> Unfortunately I found some difficulties to achieve this.
>
> I am following rfc4549.txt and the client-to-server synchronization
> says verbatim:
>
>     c) "Client-to-server synchronization": for each IMAP "action" that
>      was pending on the client, do the following:
>
>      1) If the action implies opening a new mailbox (any operation that
>         operates on messages), open the mailbox.  Check its UID
>         validity value (see Section 4.1 for more details) returned in
>         the UIDVALIDITY response code.  If the UIDVALIDITY value
>         returned by the server differs, the client MUST empty the local
>         cache of the mailbox and remove any pending "actions" that
>         refer to UIDs in that mailbox (and consider them failed).  Note
>         that this doesn't affect actions performed on client-generated
>         fake UIDs (see Section 5).
>
>      2) Perform the action.  If the action is to delete a mailbox
>         (DELETE), make sure that the mailbox is closed first (see also
>         Section 3.4.12 of [RFC2683]).
>
>
> Seems simple to do but Heliotrope currently does not store/provide
> enough information to implement this like:
>
>  1)  Account and Mailbox information of messages.
>  2) Heliotrope msg_id to GMail UID map.
>  3) Per mailbox action FIFO queues.
>
>
> Each action performed via Heliotrope (add label, remove label, state
> change, delete) should be stored in some kind of FIFO associated to
> the an account/mailbox pair. For example adding a label to a message
> in my personal account's inbox would add an action like:
>
>   label_add <label> <msg_id>
>
> where label_add is the action, <label> the action parameter and
> <msg_id> the message ID as seen by Heliotrope.  Then during
> synchronization for an account I would iterate over all mailboxes
> associated to the account and for each one I would replay the same
> actions stored in the mailbox FIFO in the GMail server. Once an action
> succeeds on the GMail side I  pop it out of the FIFO and continue with
> the next action. Using the msg_id to UID map I can determine to what
> message on the GMail side the action should be performed on.
>
> Now how to implement these features? One way could be using the extra
> parameters hash in the Heliotrope::Index::message_add method to store
> the messages account, mailbox and UID's of each message when they are
> added to Heliotrope index. This would also require a way to query this
> information.
>
> Another way would be to use special labels to mark mailboxes and
> accounts. And add some new query formats to get all mailboxes for a
> certain account and the FIFO actions of the mailbox.
>
> Finally I am more inclined to keeping a separate Index/Store for each
> account and access them via REST URLs like:
>
>   POST  /<account>                                              #
> Create new account Index/Store files
>   POST   /<account>/<mailbox>                       # Create new
> mailbox in account. Implemented using special labels?
>   GET     /<account>/mailboxes                        # List all
> mailboxes for account
>   GET     /<account>/<mailbox>/action          # Get the next pending
> action on mailbox
>   DELETE /<account>/<mailbox>/action       # Delete next pending
> action on mailbox
>   POST  /<account>/<mailbox>/message     # Add new message to an
> account/mailbox
>   GET    /<account>/<mailbox>/message     # Get all messages from
> account/mailbox
>
> This of course is how I think  IMAP offline synchronization could be
> done. If there are other ways to do this I am open to suggestions.
>
> regards,
> Horacio Sanson
> _______________________________________________
> Sup-devel mailing list
> [email-masked]
> http://rubyforge.org/mailman/listinfo/sup-devel
>

-- 
Vivien Didelot,
vivien.didelot.org
_______________________________________________
Sup-devel mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-devel

---

From: Horacio Sanson <[email-masked]>
Date: Mon, 23 May 2011 23:42:16 +0900

POST  /<account>                                              #

regards,
Horacio Sanson