[sup-devel] Heliotrope limitations for backward synchronization, , ,

From: Horacio Sanson <hsanson@gmail.com>
To: Sup developer discussion <sup-devel@rubyforge.org>
Subject: [sup-devel] Heliotrope limitations for backward synchronization, , ,
Date: Mon, 23 May 2011 23:42:16 +0900	[thread overview]
Message-ID: <BANLkTikfZiyezXe3tF-VVgsEWjZb33EEVg@mail.gmail.com> (raw)

Now that I have GMail to Heliotrope initial synchronization (first
time) and incremental (new messages) synchronization working in my
script I started working on Heliotrope to GMail synchronization.
Unfortunately I found some difficulties to achieve this.

I am following rfc4549.txt and the client-to-server synchronization
says verbatim:

     c) "Client-to-server synchronization": for each IMAP "action" that
      was pending on the client, do the following:

      1) If the action implies opening a new mailbox (any operation that
         operates on messages), open the mailbox.  Check its UID
         validity value (see Section 4.1 for more details) returned in
         the UIDVALIDITY response code.  If the UIDVALIDITY value
         returned by the server differs, the client MUST empty the local
         cache of the mailbox and remove any pending "actions" that
         refer to UIDs in that mailbox (and consider them failed).  Note
         that this doesn't affect actions performed on client-generated
         fake UIDs (see Section 5).

      2) Perform the action.  If the action is to delete a mailbox
         (DELETE), make sure that the mailbox is closed first (see also
         Section 3.4.12 of [RFC2683]).

Seems simple to do but Heliotrope currently does not store/provide
enough information to implement this like:

 1)  Account and Mailbox information of messages.
  2) Heliotrope msg_id to GMail UID map.
  3) Per mailbox action FIFO queues.

Each action performed via Heliotrope (add label, remove label, state
change, delete) should be stored in some kind of FIFO associated to
the an account/mailbox pair. For example adding a label to a message
in my personal account's inbox would add an action like:

   label_add <label> <msg_id>

where label_add is the action, <label> the action parameter and
<msg_id> the message ID as seen by Heliotrope.  Then during
synchronization for an account I would iterate over all mailboxes
associated to the account and for each one I would replay the same
actions stored in the mailbox FIFO in the GMail server. Once an action
succeeds on the GMail side I  pop it out of the FIFO and continue with
the next action. Using the msg_id to UID map I can determine to what
message on the GMail side the action should be performed on.

Now how to implement these features? One way could be using the extra
parameters hash in the Heliotrope::Index::message_add method to store
the messages account, mailbox and UID's of each message when they are
added to Heliotrope index. This would also require a way to query this
information.

Another way would be to use special labels to mark mailboxes and
accounts. And add some new query formats to get all mailboxes for a
certain account and the FIFO actions of the mailbox.

Finally I am more inclined to keeping a separate Index/Store for each
account and access them via REST URLs like:

   POST  /<account>                                              #
Create new account Index/Store files
   POST   /<account>/<mailbox>                       # Create new
mailbox in account. Implemented using special labels?
   GET     /<account>/mailboxes                        # List all
mailboxes for account
   GET     /<account>/<mailbox>/action          # Get the next pending
action on mailbox
   DELETE /<account>/<mailbox>/action       # Delete next pending
action on mailbox
   POST  /<account>/<mailbox>/message     # Add new message to an
account/mailbox
   GET    /<account>/<mailbox>/message     # Get all messages from
account/mailbox

This of course is how I think  IMAP offline synchronization could be
done. If there are other ways to do this I am open to suggestions.

regards,
Horacio Sanson
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel