* [sup-devel] importing your sup email into heliotrope
@ 2011-12-30 20:31 William Morgan
2012-01-08 20:37 ` Michael Stapelberg
2012-01-08 20:50 ` Michael Stapelberg
0 siblings, 2 replies; 4+ messages in thread
From: William Morgan @ 2011-12-30 20:31 UTC (permalink / raw)
To: sup-devel
If you're feeling adventurous, here is how to import your Sup email into
Heliotrope. This might break horribly for you--no guarantees. Bug
reports are welcome.
1. Install heliotrope and its dependencies. There is no gem yet, so
follow the instructions at https://github.com/wmorgan/heliotrope.
2. Dump your sup labels to a file by running sup-dump > <dumpfile>.
3. Import your mail sources into heliotrope. Mbox and maildir sources
should go pretty quick; IMAP, as usual, will be very slow. For each
source you can do this:
$HELIOTROPE_DIR> ruby -Ilib bin/heliotrope-import --dir <mailstore dir> --sup-dumpfile <dumpfile> --mbox <mbox file>
Or replace "--mbox <mbox file>" with the corresponding argument to
your source (use --help to see them all).
4. Add your ~/.sup/drafts.mbox file using the same commands.
5. Finally, reorder the index so that messages appear in order:
$HELIOTROPE_DIR> ruby -Ilib bin/heliotrope-reindex --reorder --dir <mailstore dir>
$HELIOTROPE_DIR> mv <mailstore dir>/index <mailstore dir>/index-old
$HELIOTROPE_DIR> mv <mailstore dir>/index-reindexed <mailstore dir>/index
6. Start the server:
$HELIOTROPE_DIR> ruby -Ilib bin/heliotrope-server --dir <mailstore dir>
Point your web browser to http://localhost:8042 and see if that works!
If it looks good, you can also try and get turnsole working, for a
Sup-like client: https://github.com/wmorgan/turnsole.
Good luck and have fun!
--
William <w@masanjin.net>
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* [sup-devel] importing your sup email into heliotrope
2011-12-30 20:31 [sup-devel] importing your sup email into heliotrope William Morgan
@ 2012-01-08 20:37 ` Michael Stapelberg
2012-01-08 20:50 ` Michael Stapelberg
1 sibling, 0 replies; 4+ messages in thread
From: Michael Stapelberg @ 2012-01-08 20:37 UTC (permalink / raw)
To: sup-devel
Hi William,
Excerpts from William Morgan's message of 2011-12-30 20:31:24 +0000:
> 3. Import your mail sources into heliotrope. Mbox and maildir sources
> should go pretty quick; IMAP, as usual, will be very slow. For each
> source you can do this:
>
> $HELIOTROPE_DIR> ruby -Ilib bin/heliotrope-import --dir <mailstore dir> --sup-dumpfile <dumpfile> --mbox <mbox file>
I think you meant --sup-label-file, or you did not push some commits yet :).
> Or replace "--mbox <mbox file>" with the corresponding argument to
> your source (use --help to see them all).
FYI: I used the following command to get all my sources in one command line:
$ grep uri .sup/sources.yaml | sed 's/^[ \t]*uri: maildir://g' | tr '\n' ' '
The total time for importing was:
; scanned 133872, indexed 122314, skipped 9485 bad and 2073 seen messages in 12484.5s = 10.7 m/s
Which is awfully long. I took a look at the process list and saw lots of
html2text process running. Why are they run at all? Are they also run for
plain-text emails? Most of my emails surely are plain-text, without any need to
convert HTML.
> 5. Finally, reorder the index so that messages appear in order:
>
> $HELIOTROPE_DIR> ruby -Ilib bin/heliotrope-reindex --reorder --dir <mailstore dir>
> $HELIOTROPE_DIR> mv <mailstore dir>/index <mailstore dir>/index-old
> $HELIOTROPE_DIR> mv <mailstore dir>/index-reindexed <mailstore dir>/index
Ugh, this took a lot of time, too:
; reindexed 120451 messages, skipped 0 spam and 1863 deleted in 9509.3s = 12.7 m/s
Also, the numbers don’t add up. This is a bit confusing. Is there maybe no
printing of the *final* status? Or is this intended?
Best regards,
Michael
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* [sup-devel] importing your sup email into heliotrope
2011-12-30 20:31 [sup-devel] importing your sup email into heliotrope William Morgan
2012-01-08 20:37 ` Michael Stapelberg
@ 2012-01-08 20:50 ` Michael Stapelberg
2012-01-09 22:30 ` William Morgan
1 sibling, 1 reply; 4+ messages in thread
From: Michael Stapelberg @ 2012-01-08 20:50 UTC (permalink / raw)
To: sup-devel
Hi William,
(Re-sent message due to missing sendmail hook and thus wrong from address.)
Excerpts from William Morgan's message of 2011-12-30 20:31:24 +0000:
> 3. Import your mail sources into heliotrope. Mbox and maildir sources
> should go pretty quick; IMAP, as usual, will be very slow. For each
> source you can do this:
>
> $HELIOTROPE_DIR> ruby -Ilib bin/heliotrope-import --dir <mailstore dir> --sup-dumpfile <dumpfile> --mbox <mbox file>
I think you meant --sup-label-file, or you did not push some commits yet :).
> Or replace "--mbox <mbox file>" with the corresponding argument to
> your source (use --help to see them all).
FYI: I used the following command to get all my sources in one command line:
$ grep uri .sup/sources.yaml | sed 's/^[ \t]*uri: maildir://g' | tr '\n' ' '
The total time for importing was:
; scanned 133872, indexed 122314, skipped 9485 bad and 2073 seen messages in 12484.5s = 10.7 m/s
Which is awfully long. I took a look at the process list and saw lots of
html2text process running. Why are they run at all? Are they also run for
plain-text emails? Most of my emails surely are plain-text, without any need to
convert HTML.
> 5. Finally, reorder the index so that messages appear in order:
>
> $HELIOTROPE_DIR> ruby -Ilib bin/heliotrope-reindex --reorder --dir <mailstore dir>
> $HELIOTROPE_DIR> mv <mailstore dir>/index <mailstore dir>/index-old
> $HELIOTROPE_DIR> mv <mailstore dir>/index-reindexed <mailstore dir>/index
Ugh, this took a lot of time, too:
; reindexed 120451 messages, skipped 0 spam and 1863 deleted in 9509.3s = 12.7 m/s
Also, the numbers don’t add up. This is a bit confusing. Is there maybe no
printing of the *final* status? Or is this intended?
Best regards,
Michael
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* [sup-devel] importing your sup email into heliotrope
2012-01-08 20:50 ` Michael Stapelberg
@ 2012-01-09 22:30 ` William Morgan
0 siblings, 0 replies; 4+ messages in thread
From: William Morgan @ 2012-01-09 22:30 UTC (permalink / raw)
To: sup-devel
Excerpts from Michael Stapelberg's message of 2012-01-08 12:50:02 -0800:
> I think you meant --sup-label-file, or you did not push some commits yet :).
Yes, you are right. Thanks.
> The total time for importing was:
> ; scanned 133872, indexed 122314, skipped 9485 bad and 2073 seen messages in 12484.5s = 10.7 m/s
>
> Which is awfully long. I took a look at the process list and saw lots of
> html2text process running. Why are they run at all? Are they also run for
> plain-text emails? Most of my emails surely are plain-text, without any need to
> convert HTML.
They should only be run for text/html mime parts that are not part of a
multipart/alternative mime part with a text component (in that case the
text part is just used directly.)
I believe that is the case, but if you can find a counterexample, please file
it as a bug: http://github.com/wmorgan/heliotrope/issues.
> ; reindexed 120451 messages, skipped 0 spam and 1863 deleted in 9509.3s = 12.7 m/s
>
> Also, the numbers don’t add up.
120451 + 1863 = 122314, so they do add up. Perhaps what's confusing is that
import happily preserves your deleted emails, but reindex by default skips
over them?
Glad to hear you're giving it a try. Keep the bug reports coming.
--
William <wmorgan@masanjin.net>
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2012-01-10 0:17 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-12-30 20:31 [sup-devel] importing your sup email into heliotrope William Morgan
2012-01-08 20:37 ` Michael Stapelberg
2012-01-08 20:50 ` Michael Stapelberg
2012-01-09 22:30 ` William Morgan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox