Re: [sup-devel] [Heliotrope/Turnsole] How to use IMAP?

From: Michael Stapelberg <michael+sup@stapelberg.de>
To: sup-devel <sup-devel@rubyforge.org>
Subject: Re: [sup-devel] [Heliotrope/Turnsole] How to use IMAP?
Date: Mon, 27 Feb 2012 10:16:43 +0100	[thread overview]
Message-ID: <1330333805-sup-1287@stapelberg.de> (raw)
In-Reply-To: <1330150790-turnsole-3005@terminus-est>

Hi William,

Sorry for clinging to that topic, but it’s important for me to properly
understand it.

Excerpts from William Morgan's message of 2012-02-25 07:35:32 +0100:
> Excerpts from Michael Stapelberg's message of 2012-02-23 03:24:22 -0800:
> > Well, I guess you are correct. So, when somebody sends me a spam email
> > or a malicious email with a faked date, what happens? I think that the
> > code will figure out it needs to re-add a lot of messages. Also, my
> > index will need to be reordered, right?
> 
> You won't need to reorder. Importing is all incremental, at least if you
> use --state-file: it will keep track of the last message imported, and
> successive imports will only pick up new messages.
Apparently, I didn’t properly describe the scenario I’m thinking about:

state-file contains an entry "/my/maildir/2012-02-22-foo-bar", which is an
email that contains 2012-02-22 as date.

Now, a spammer sends me an email with a faked date, let’s say 2001-01-01, let’s
call it "/my/maildir/2001-01-01-spam-mail". The next run of heliotrope-add has
two possibilities:

1) It will completely ignore it. I think this is what will happen, based on
   your description. This is horrible! Emails are absolutely not guaranteed to
   arrive in my system in the order they were sent. Is this the case? Do we
   ignore email because of date problems like this currently?

2) It will pick up this email because the date inside is older. Subsequent runs
   will discover all email from the last 10 years as new, and try to re-add it
   to the index. Lots of unnecessary overhead.

> But you are making me realize that the current maildir implementation is
> not right. I think the way to handle Maildir is to look only at the
> ctime of the files and not the date headers. That will make incremental
> importing possible (just save the filename of the last imported file,
> and look for all files newer than that). It will speed things up
> anyways.
Right, that sounds good.

> And if you're in the funny situation where the ctime is not correlated
> with the Date: headers in your files, which is perfectly possibly by the
> Maildir spec, then you will have to reorder after that initial import.
> But in reality that's a possibility with the other source types
> too---it's just more likely with Maildir.
So, let’s stick to my above example of the spam email and let’s assume that you
changed the code to use ctimes. The spam mail arrives, its ctime is new, it
gets picked up into my index. To my understanding, I now need to reorder. I
have multiple questions:

1) What are the immediate consequences? Where will this email appear in my
   inbox, when my inbox contains 5 emails. Always at the top? Bottom? You
   mentioned I can see the effect when paging. Can you elaborate please?

2) How do I know when I have to reorder?

Best regards,
Michael
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel