* [sup-devel] Update in heliotrope !
@ 2012-09-24 22:13 Matthieu Rakotojaona
2012-09-25 19:15 ` Steven Hum
0 siblings, 1 reply; 6+ messages in thread
From: Matthieu Rakotojaona @ 2012-09-24 22:13 UTC (permalink / raw)
To: sup-talk, sup-devel
Hello everyone,
I have updated my version of heliotrope on my imap branch
(https://github.com/rakoo/heliotrope/tree/imap). You just have to
clone it and checkout the branch.
You may now synchronize your mails between heliotrope and a maildir
with offlineimap. I tested most of the situations that can happen
when using it (Adding/removing a label on heliotrope side,
adding/removing a mail on maildir side)
A few notes :
* It is not "multi-write-safe", in the sense that you should stop all
other activity with heliotrope while you sync with offlinimap.
Concurrent access will have unknown outcome.
* To continue on this point, use offlineimap with the `-1` flag, which
will turn off multithreading. It should work without it though,
because AFAIK offlineimap threads do not share mailboxes. But I'm not
sure.
* There are a few hardcoded labels/dir that you should find in your
maildir, such as INBOX/inbox and "All Mail", which has no equivalent
in heliotrope. Fear not, this one is taken care of.
* Syncing with another IMAP server should also work,because it is
transparent to offlineimap (at least from the outside)
Have fun !
--
Matthieu RAKOTOJAONA
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [sup-devel] Update in heliotrope !
2012-09-24 22:13 [sup-devel] Update in heliotrope ! Matthieu Rakotojaona
@ 2012-09-25 19:15 ` Steven Hum
2012-09-25 19:44 ` Matthieu Rakotojaona
0 siblings, 1 reply; 6+ messages in thread
From: Steven Hum @ 2012-09-25 19:15 UTC (permalink / raw)
To: sup-devel
Hello Matthieu,
I am able to initialize the heliotrope mailstore directory easy enough
with
ruby -Ilib bin/heliotrope-server -d <mailstore>
(stuff gets created :-) but when I try to import with
ruby -Ilib bin/heliotrope-import -a <maildir> -d <mailstore>
<maildir> being the dir that contains the cur/new/tmp directories that
contain the actual emails downloaded by offlineimap,
the import process hangs after the "Adding mail..." message and doesn't
load any emails (I just terminate the process with a ctrl-c). What am
I missing? I tried using heliotrope-add with the server running, but it
hung at the same point (not surpisingly).
Secondly, your note "stop all other activity with heliotrope while you
sync with offlinimap". Does that mean heliotrope-server should not be
running if offlineimap is (updating the maildirs)?
Thanks,
Steven
>
Excerpts from Matthieu Rakotojaona's message of 2012-09-24 18:13:57 -0400:
> Hello everyone,
>
> I have updated my version of heliotrope on my imap branch
> (https://github.com/rakoo/heliotrope/tree/imap). You just have to
> clone it and checkout the branch.
> You may now synchronize your mails between heliotrope and a maildir
> with offlineimap. I tested most of the situations that can happen
> when using it (Adding/removing a label on heliotrope side,
> adding/removing a mail on maildir side)
>
> A few notes :
>
> * It is not "multi-write-safe", in the sense that you should stop all
> other activity with heliotrope while you sync with offlinimap.
> Concurrent access will have unknown outcome.
> * To continue on this point, use offlineimap with the `-1` flag, which
> will turn off multithreading. It should work without it though,
> because AFAIK offlineimap threads do not share mailboxes. But I'm not
> sure.
> * There are a few hardcoded labels/dir that you should find in your
> maildir, such as INBOX/inbox and "All Mail", which has no equivalent
> in heliotrope. Fear not, this one is taken care of.
> * Syncing with another IMAP server should also work,because it is
> transparent to offlineimap (at least from the outside)
>
> Have fun !
>
--
"Truth or die."
Steven Hum
5 - 28 Gilmour St
Ottawa, ON K2P 0N3
email sdothum@gmail.com
tel 613.237.9058
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [sup-devel] Update in heliotrope !
2012-09-25 19:15 ` Steven Hum
@ 2012-09-25 19:44 ` Matthieu Rakotojaona
2012-09-25 21:01 ` Steven Hum
0 siblings, 1 reply; 6+ messages in thread
From: Matthieu Rakotojaona @ 2012-09-25 19:44 UTC (permalink / raw)
To: Sup developer discussion
On Tue, Sep 25, 2012 at 9:15 PM, Steven Hum <sdothum@gmail.com> wrote:
> Hello Matthieu,
Hello,
> I am able to initialize the heliotrope mailstore directory easy enough
> with
>
> ruby -Ilib bin/heliotrope-server -d <mailstore>
>
> (stuff gets created :-) but when I try to import with
>
> ruby -Ilib bin/heliotrope-import -a <maildir> -d <mailstore>
Did you even read the README ? =]
It clearly says : "To bulk import mail, use heltrope-import. You must
stop your server first."
The thing is, both heliotrope-server and heliotrope-import will try to
access the same resources, which are not designed for
multi-concurrency. So, you have to shut heliotrope-server down before
using heliotrope-import.
> I tried using heliotrope-add with the server running, but it
> hung at the same point (not surpisingly).
This is more problematic, because it should work. Can you share some
of your logs ?
> Secondly, your note "stop all other activity with heliotrope while you
> sync with offlinimap". Does that mean heliotrope-server should not be
> running if offlineimap is (updating the maildirs)?
No, on the contrary, it means that while heliotrope-server is running,
you should either access it with a traditional http client, or with an
IMAP client, but you should never do both the same time.
But you will need heliotrope-server running for accessing IMAP.
--
Matthieu RAKOTOJAONA
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [sup-devel] Update in heliotrope !
2012-09-25 19:44 ` Matthieu Rakotojaona
@ 2012-09-25 21:01 ` Steven Hum
2012-09-27 15:50 ` Matthieu Rakotojaona
0 siblings, 1 reply; 6+ messages in thread
From: Steven Hum @ 2012-09-25 21:01 UTC (permalink / raw)
To: sup-devel
Yes! :-) (though I did try the import with the server running after the
persistent hanging without.. just in case the documentation wasn't
correct!).
As nothing appeared to be working, I purged the heliotrope mailstore and
performed 2 actions: 1) the initial server startup to initialize the
mailstore and associated files, and 2) an import of a small maildir
(after stopping the server). I presume the logs you are referring to is
the LOG file in the <mailstore>/store dir.
LOG from step 1) (or LOG.old after completing both steps)
2012/09/25-16:42:13.770269 7ff2345bc700 Delete type=3 #1
LOG from attempted import, step 2)
2012/09/25-16:43:12.450537 7f0c45ff8700 Recovering log #3
2012/09/25-16:43:12.450585 7f0c45ff8700 Level-0 table #5: started
2012/09/25-16:43:12.458919 7f0c45ff8700 Level-0 table #5: 344 bytes OK
2012/09/25-16:43:12.460026 7f0c45ff8700 Delete type=0 #3
2012/09/25-16:43:12.460050 7f0c45ff8700 Delete type=3 #2
Oh, the second thing you should know is that I run on debian (test
/ wheezy) and have been using rvm 1.9.3. I know that the debian
environment doesn'd always make it easy for ruby installs at times. In
addition to the gems requirements listed in the README, I had to install
the mail and thin gems to get heliotrope to run.
Regards,
Steven
Excerpts from Matthieu Rakotojaona's message of 2012-09-25 15:44:49 -0400:
> On Tue, Sep 25, 2012 at 9:15 PM, Steven Hum <sdothum@gmail.com> wrote:
> > Hello Matthieu,
>
> Hello,
>
> > I am able to initialize the heliotrope mailstore directory easy enough
> > with
> >
> > ruby -Ilib bin/heliotrope-server -d <mailstore>
> >
> > (stuff gets created :-) but when I try to import with
> >
> > ruby -Ilib bin/heliotrope-import -a <maildir> -d <mailstore>
>
> Did you even read the README ? =]
>
> It clearly says : "To bulk import mail, use heltrope-import. You must
> stop your server first."
>
> The thing is, both heliotrope-server and heliotrope-import will try to
> access the same resources, which are not designed for
> multi-concurrency. So, you have to shut heliotrope-server down before
> using heliotrope-import.
>
> > I tried using heliotrope-add with the server running, but it
> > hung at the same point (not surpisingly).
>
> This is more problematic, because it should work. Can you share some
> of your logs ?
>
> > Secondly, your note "stop all other activity with heliotrope while you
> > sync with offlinimap". Does that mean heliotrope-server should not be
> > running if offlineimap is (updating the maildirs)?
>
> No, on the contrary, it means that while heliotrope-server is running,
> you should either access it with a traditional http client, or with an
> IMAP client, but you should never do both the same time.
> But you will need heliotrope-server running for accessing IMAP.
>
--
"Truth or die."
Steven Hum
5 - 28 Gilmour St
Ottawa, ON K2P 0N3
email sdothum@gmail.com
tel 613.237.9058
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [sup-devel] Update in heliotrope !
2012-09-25 21:01 ` Steven Hum
@ 2012-09-27 15:50 ` Matthieu Rakotojaona
2012-09-27 16:46 ` Steven Hum
0 siblings, 1 reply; 6+ messages in thread
From: Matthieu Rakotojaona @ 2012-09-27 15:50 UTC (permalink / raw)
To: Sup developer discussion
Sorry for the delay.
On Tue, Sep 25, 2012 at 11:01 PM, Steven Hum <sdothum@gmail.com> wrote:
> I presume the logs you are referring to is
> the LOG file in the <mailstore>/store dir.
Hmm no, I would like to see the heliotrope-server logs, or the
heliotrope-import logs. These could help solve the problem.
> Oh, the second thing you should know is that I run on debian (test
> / wheezy) and have been using rvm 1.9.3. I know that the debian
> environment doesn'd always make it easy for ruby installs at times. In
> addition to the gems requirements listed in the README, I had to install
> the mail and thin gems to get heliotrope to run.
Sorry, this is totally undocumented, but yes, you have to install the
mail gem. I decided to part from rmail, since it isn't maintained
anymore. Thin was just there to improve performance and is optional. I
just found the HTTP part stable enough fore my needs, so I changed to
it (And effectively saw some improvements)
--
Matthieu RAKOTOJAONA
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [sup-devel] Update in heliotrope !
2012-09-27 15:50 ` Matthieu Rakotojaona
@ 2012-09-27 16:46 ` Steven Hum
0 siblings, 0 replies; 6+ messages in thread
From: Steven Hum @ 2012-09-27 16:46 UTC (permalink / raw)
To: sup-devel
I couldn't find any physical log files anywhere (at least any text files
with an obvious name...). The only other file in the heliotrope
mailstore directory I found was a 32 byte binary file named "000022.log"
Where should these be located? Looks like there could be something
fundamentally broken on my system if these log files aren't even being
written!
Regards,
Steven
Excerpts from Matthieu Rakotojaona's message of 2012-09-27 11:50:40 -0400:
> Sorry for the delay.
>
> On Tue, Sep 25, 2012 at 11:01 PM, Steven Hum <sdothum@gmail.com> wrote:
> > I presume the logs you are referring to is
> > the LOG file in the <mailstore>/store dir.
>
> Hmm no, I would like to see the heliotrope-server logs, or the
> heliotrope-import logs. These could help solve the problem.
>
> > Oh, the second thing you should know is that I run on debian (test
> > / wheezy) and have been using rvm 1.9.3. I know that the debian
> > environment doesn'd always make it easy for ruby installs at times. In
> > addition to the gems requirements listed in the README, I had to install
> > the mail and thin gems to get heliotrope to run.
>
> Sorry, this is totally undocumented, but yes, you have to install the
> mail gem. I decided to part from rmail, since it isn't maintained
> anymore. Thin was just there to improve performance and is optional. I
> just found the HTTP part stable enough fore my needs, so I changed to
> it (And effectively saw some improvements)
>
--
"Truth or die."
Steven Hum
5 - 28 Gilmour St
Ottawa, ON K2P 0N3
email sdothum@gmail.com
tel 613.237.9058
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-09-27 17:07 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-09-24 22:13 [sup-devel] Update in heliotrope ! Matthieu Rakotojaona
2012-09-25 19:15 ` Steven Hum
2012-09-25 19:44 ` Matthieu Rakotojaona
2012-09-25 21:01 ` Steven Hum
2012-09-27 15:50 ` Matthieu Rakotojaona
2012-09-27 16:46 ` Steven Hum
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox