Subject: [sup-talk] various problems/questions.

From: [email-masked] (shivan)
Date: Wed, 19 Dec 2007 12:50:14 +0100

Hi, as the title says, I have some problems/questions, so, here they are.

I find sup pretty slow : if I lanch sup, archive every mail presented
to me (53) and exit, it takes roughly 7 seconds to save the changes,
is this expected ?
I use an mbox source (gmail account via mpop) and there are 26000
mails in index, I'm on Ubuntu 8.04

I thought it could be because most of the mail are in the inbox, so I
tried to archive them, which led me to the problem : how to do it ?
I tried "ruby -Ilib bin/sup-sync --all --archive" which lasted 17
minutes, but didn't seem to do anything.

I also have warning during the sup-sync command :
[Wed Dec 19 12:39:19 +0100 2007] warning: error
(Iconv::InvalidEncoding) decoding message body from unicode-1-1-utf-7:
invalid encoding ("UTF-8//IGNORE", "unicode-1-1-utf-7")
any idea ?

and it seems to be "faking message-id for message from : sup-faked-"
alot, is that normal, or is my source the problem ?

thank you for reading :-)

-- 
shivan

---

From: [email-masked] (William Morgan)
Date: Wed, 19 Dec 2007 18:54:59 -0800

> to me (53) and exit, it takes roughly 7 seconds to save the changes,

Saving state back to disk is slow, mainly because Ferret (the underlying
search engine) doesn't have a modify command; it only has delete and
add. Since the message body is not stored explicitly in the index, this
requires loading the entire message, parsing it, etc. for every single
label change.

> tried to archive them, which led me to the problem : how to do it ?  I
> tried "ruby -Ilib bin/sup-sync --all --archive" which lasted 17

Sup-sync is for syncing the index with a particular source. It's not the
right tool for the job here. Actually there is no right tool for this
job yet, so I will add one. (It is possible to do this currently with
the development console, but there should be a better way.)

> invalid encoding ("UTF-8//IGNORE", "unicode-1-1-utf-7")

That just means you have a message with a weird encoding. No harm done,
we just use it unconverted.

> alot, is that normal, or is my source the problem ?

That means you have some email without (required!) message-id headers.
Typically these are the output of a mail client or mail script that was
written by incompetents.

-- 
William <wmorgan-sup at masanjin.net>

---

From: [email-masked] (William Morgan)
Date: Wed, 19 Dec 2007 18:59:59 -0800

Excerpts from William Morgan's message of Wed Dec 19 18:54:59 -0800 2007:
> Sup-sync is for syncing the index with a particular source. It's not
> the right tool for the job here. Actually there is no right tool for
> this job yet, so I will add one. (It is possible to do this currently
> with the development console, but there should be a better way.)

I've added a bin/sup-tweak-labels script to git, which allows adding to
or removing labels from all messages for a given source. This should be
directly applicable to this type of problem.