Re: [sup-devel] after second heliotrope import, new messages not appearing

From: Hamish D <dmishd@gmail.com>
To: Sup developer discussion <sup-devel@rubyforge.org>
Subject: Re: [sup-devel] after second heliotrope import, new messages not appearing
Date: Tue, 1 Jan 2013 18:33:09 +0000	[thread overview]
Message-ID: <CAOxvSbcBPGCsL_ZDqMgi6zk2uodnLqrdh89SkzMFg=F8aYdCHA@mail.gmail.com> (raw)
In-Reply-To: <CAMiZLn1Uv-5Csf7O2kxijbwanVsT78W4puy_ypN0vMr=3mA0Yg@mail.gmail.com>

> Your problem may have multiple origins, and I have no idea where you could
> start :
>
> - Did you use the correct directories for messages and index for _both_ the
> import and the server ? (sorry if this sounds stupid)

Always good to check the basics, but yes, I do use the same directory.
I have scripts for the import and the server and both use the -d
argument to ensure they use the same directory.

> - Did the index size in heliotrope change ? (as given by a GET to
> /size.json)

No - it remained the same. (And is about half of the count that sup reports).

> - Did the messages file size change ?

It did on one import, but not the most recent.

> - Are the messages absent in inbox only or from the whole turnsole ?

All searches that I do stop in May.

> - As a last resort, could you send the mbox file so that I can try this at
> home, if your mails aren't too personal ?

It's my work email, with a total of over 100,000 messages. So I'd
prefer not to send it over.

Any logs that might be worth looking at? While doing the import, I do
get various lines along the line of:

    ; scanned 71, indexed 71, skipped 0 bad and 0 seen messages in
22.1s = 3.2 m/s

I do use the state argument (-t) to save reimporting old mail. I have
also tried doing a reindex (as I import from multiple mboxes) but to
no avail.

I could just try another import of the whole lot into a clean directory I guess.

I'm also happy to type stuff into heliotrope-console if that will help
diagnose stuff.

Thanks for your suggestions so far
Hamish

> On Thu, Dec 27, 2012 at 5:19 PM, Hamish D <dmishd@gmail.com> wrote:
>>
>> Hello
>>
>> I've just been trying to carry on from where I left off with migrating
>> my work email into heliotrope (which I last worked on in May).
>>
>> So I've rsynced the mbox files across to where I will run heliotrope
>> and tried running heliotrope-import to put all the new messages in the
>> index. It appeared to work properly, reporting that messages were
>> being added to the index and the index files themselves have had their
>> timestamps updated. But when I start turnsole, I can't find any email
>> after May.
>>
>> Any ideas on where to start looking to work out what is going on?
>>
>> Hamish
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel