Subject: [sup-devel] importing your sup email into heliotrope

From: Michael Stapelberg <[email-masked]>
Date: Sun, 08 Jan 2012 20:37:25 +0000

Hi William,

Excerpts from William Morgan's message of 2011-12-30 20:31:24 +0000:
> 3. Import your mail sources into heliotrope. Mbox and maildir sources
>    should go pretty quick; IMAP, as usual, will be very slow. For each
>    source you can do this:
> 
>      $HELIOTROPE_DIR> ruby -Ilib bin/heliotrope-import --dir <mailstore dir> --sup-dumpfile <dumpfile> --mbox <mbox file>
I think you meant --sup-label-file, or you did not push some commits yet :).

>    Or replace "--mbox <mbox file>" with the corresponding argument to
>    your source (use --help to see them all).
FYI: I used the following command to get all my sources in one command line:
$ grep uri .sup/sources.yaml | sed 's/^[ \t]*uri: maildir://g' | tr '\n' ' '

The total time for importing was:
; scanned 133872, indexed 122314, skipped 9485 bad and 2073 seen messages in 12484.5s = 10.7 m/s

Which is awfully long. I took a look at the process list and saw lots of
html2text process running. Why are they run at all? Are they also run for
plain-text emails? Most of my emails surely are plain-text, without any need to
convert HTML.

> 5. Finally, reorder the index so that messages appear in order:
> 
>      $HELIOTROPE_DIR> ruby -Ilib bin/heliotrope-reindex --reorder --dir <mailstore dir>
>      $HELIOTROPE_DIR> mv <mailstore dir>/index <mailstore dir>/index-old
>      $HELIOTROPE_DIR> mv <mailstore dir>/index-reindexed <mailstore dir>/index
Ugh, this took a lot of time, too:
; reindexed 120451 messages, skipped 0 spam and 1863 deleted in 9509.3s = 12.7 m/s

Also, the numbers don’t add up. This is a bit confusing. Is there maybe no
printing of the *final* status? Or is this intended?

Best regards,
Michael
_______________________________________________
Sup-devel mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-devel

---

From: William Morgan <[email-masked]>
Date: Fri, 30 Dec 2011 12:31:24 -0800

If you're feeling adventurous, here is how to import your Sup email into
Heliotrope. This might break horribly for you--no guarantees. Bug
reports are welcome.

1. Install heliotrope and its dependencies. There is no gem yet, so
   follow the instructions at https://github.com/wmorgan/heliotrope.

2. Dump your sup labels to a file by running sup-dump > <dumpfile>.

4. Add your ~/.sup/drafts.mbox file using the same commands.

6. Start the server:

     $HELIOTROPE_DIR> ruby -Ilib bin/heliotrope-server --dir <mailstore dir>

Point your web browser to http://localhost:8042 and see if that works!
If it looks good, you can also try and get turnsole working, for a
Sup-like client: https://github.com/wmorgan/turnsole.

Good luck and have fun!
-- 
William <[email-masked]>

---

From: Michael Stapelberg <[email-masked]>
Date: Sun, 08 Jan 2012 20:50:02 +0000

Hi William,

(Re-sent message due to missing sendmail hook and thus wrong from address.)

Best regards,
Michael

---

From: William Morgan <[email-masked]>
Date: Mon, 09 Jan 2012 14:30:42 -0800

> I think you meant --sup-label-file, or you did not push some commits yet :).

Yes, you are right. Thanks.

> plain-text emails? Most of my emails surely are plain-text, without any need to

They should only be run for text/html mime parts that are not part of a
multipart/alternative mime part with a text component (in that case the
text part is just used directly.)

I believe that is the case, but if you can find a counterexample, please file
it as a bug: http://github.com/wmorgan/heliotrope/issues.

> Also, the numbers don’t add up.

120451 + 1863 = 122314, so they do add up. Perhaps what's confusing is that
import happily preserves your deleted emails, but reindex by default skips
over them?

Glad to hear you're giving it a try. Keep the bug reports coming.
-- 
William <[email-masked]>
_______________________________________________
Sup-devel mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-devel