Re: [sup-devel] Tokyo Cabinet fails with invalid record header

From: Horacio Sanson <hsanson@gmail.com>
To: Sup developer discussion <sup-devel@rubyforge.org>
Subject: Re: [sup-devel] Tokyo Cabinet fails with invalid record header
Date: Thu, 23 Jun 2011 07:41:19 +0900	[thread overview]
Message-ID: <201106230741.19602.hsanson@gmail.com> (raw)
In-Reply-To: <1308713034-sup-9195@masanjin.net>

I left heliotrope-add running but it did not finished. I got some strange nil 
errors (see below):

I think this error happens when find_msgids is unable to find the message's id. 
Not sure what is the best action in this case but skipping it sounds a good 
option.
###################################################
; requesting messages 141445..141564 from imap server
scanned 130401, indexed 130278, skipped 25 bad and 98 seen messages in 9566.0s 
= 13.6 m/s
; requesting messages 141565..141688 from imap server
scanned 130501, indexed 130378, skipped 25 bad and 98 seen messages in 9572.0s 
= 13.6 m/s
/media/DATA/Apps/heliotrope/lib/heliotrope/message.rb:155:in `digest': can't 
convert nil into String (TypeError)
        from /media/DATA/Apps/heliotrope/lib/heliotrope/message.rb:155:in 
`hexdigest'
        from /media/DATA/Apps/heliotrope/lib/heliotrope/message.rb:155:in 
`munge_msgid'
        from /media/DATA/Apps/heliotrope/lib/heliotrope/message.rb:20:in 
`parse!'
        from bin/heliotrope-add:138:in `<main>'
####################################################

This error happens when @imap.uid_fetch doesn't raise any exception and 
returns nil instead of empty array. Was thinking to check for nil and set it
to an empty array if true but it would likely result in an infinite loop.
####################################################
; loading mail...
; connecting...
; login as horacio.sanson@gmail.com ...
; found 293004 new messages...
; found 293004 messages to scan
; requesting messages 283664..283763 from imap server
scanned 1, indexed 1, skipped 0 bad and 0 seen messages in 7.9s = 0.1 m/s
; requesting messages 283764..283863 from imap server
/media/DATA/Apps/heliotrope/lib/heliotrope/gmail-dumper.rb:171:in 
`next_message': undefined method `empty?' for nil:NilClass (NoMethodError)
        from bin/heliotrope-add:128:in `<main>'
#####################################################

Finally the leveldb branch is not working that well. After reaching 130378 
indexed messages I started the heliotrope server and the web interface only 
showed two messages. 

I will try again without the hooks for Japanese and see how it goes.

On Wednesday 22 June 2011 12:25:16 William Morgan wrote:
> Hi Horacio,
> 
> Reformatted excerpts from William Morgan's message of 2011-06-20:
> > Ok, I have a potential fix for this that I'll try and get out tonight.
> 
> Can you please:
> 
> 1. git clone git://github.com/wmorgan/leveldb-ruby.git
> 2. git checkout fpic
> 3. rake gem
> 4. gem install pkg/leveldb-ruby-0.4.gem
> 
> And tell me if that works?
> 
> If so, I will release that as 0.5.
> 
> Thanks!

-- 
regards,                                                                                                                                                                                                       
Horacio Sanson
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel