From mboxrd@z Thu Jan  1 00:00:00 1970
From: nicolas.pouillard@gmail.com (Nicolas Pouillard)
Date: Fri, 25 Jan 2008 09:50:20 +0100
Subject: [sup-talk] header cache
In-Reply-To: <1201230428-sup-8760@south>
References: <704d6aa20801150454j635ed24p63fd544579c6e148@mail.gmail.com>
	<1200453820-sup-4777@south>
	<1200514969-sup-7541@clarabella.clarabella>
	<1200516410-sup-1963@south>
	<1200519319-sup-915@clarabella.clarabella>
	<1200557516-sup-8444@ausone.local>
	<1200570003-sup-1529@clarabella.clarabella>
	<1200581670-sup-9273@ausone.local> <1200969359-sup-2816@south>
	<1200990925-sup-1529@ausone.local> <1201071069-sup-6394@south>
	<1201079068-sup-438@ausone.inria.fr> <1201230428-sup-8760@south>
Message-ID: <1201250434-sup-4567@ausone.local>

Excerpts from William Morgan's message of Fri Jan 25 04:28:07 +0100 2008:
> [Sending to list. We accidentally fell off.]
> 
> Reformatted excerpts from nicolas.pouillard's message of 2008-01-23:
> > Excerpts from William Morgan's message of Wed Jan 23 08:04:27 +0100 2008:
> > > Reformatted excerpts from nicolas.pouillard's message of 2008-01-22:
> > > > Looking  at  the  code  'scan_mailbox' seems to be called quite
> > > > often (but not more than every 30 seconds).
> > > 
> > > Oops, you're exactly right.
> > > 
> > > > I'm  wondering  if  peeking  a  file  in  a very large directory
> > > > is as fast as seeking to a particular offset in a large file?
> > > 
> > > Actually, we could short-circuit this check trivially by comparing
> > > the directories' mtimes. Then this really shouldn't be a slowdown.
> > 
> > You will still scan it at when receiving new mails...
> 
> That's true. But some kind of scan is unavoidable. We need to know which
> files are new, and one way or another that will involve pulling in the
> whole list of files.
> 
> I think we can do a couple easy things to speed up Maildir dramatically.
> Skip the scan when the mtime is unchanged, maybe increase the delay,
> stop repeating make_id calls for messages we've already processed...
> basically the current implementation is a little naive and I'm not ready
> to give up on Maildir quite yet.

What about having two directories? new and cur for instance :)
You always have to scan new and move them to cur when they are in the index.

> > What about having a custom mail box format for sup?
> > 
> > For  instance  I think the only drawback (in the context of using sup)
> > of mbox is when you often backup you mails. Why not having a bunch of
> > mbox using dates inbox-12-2008.mbox? Or a directory where file names
> > are sup message ids?
> 
> What about mbox makes it difficult to do backups?

Every  hour,  if  my  backup  drive is plugged in, my OS make a new tree of my
entire  disk  by  hard  linking  what's  haven't  change and by coping others.
However  this  behave particularly bad with files that just keep growing every
hour ~400MB at this day.

That's  why  splitting  out  my  big  mail  box in small ones, let's say every
month, would better behave w.r.t. to backups.

> The amount of work for a custom format seems really high. (Look at all
> the design considerations that went into Maildir!) And ultimately your
> MTA has to be able to write that format, or there has to be a conversion
> script, which has to be run every 30 seconds, and has to be fast, and
> then we have the same set of problems we have now but in a different
> place.

Your  MTA  call a MDA like maildrop, and making a specific maildrop in ruby is
pretty easy.

Best regards,

-- 
Nicolas Pouillard aka Ertai
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 186 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/sup-talk/attachments/20080125/cd24b771/attachment.bin