Subject: [sup-talk] problems with rubymail on an initial sup-add/sync

From: [email-masked] (Jimmy Tang)
Date: Wed, 19 Sep 2007 13:36:16 +0100

Hi,

I've just installed 0.1 of sup with the current stable version of ruby via gems.

i've just tried adding some of my mbox's and this seems to happen...

Scanning mbox:///u1/trhpc/jtang/Mail/system.rancid...
Scanned 0, added 0, updated 0 messages from mbox:///u1/trhpc/jtang/Mail/system.rancid.
Scanning sup://drafts...
Scanned 0, added 0, updated 0 messages from sup://drafts.
Scanning mbox:///u1/trhpc/jtang/Mail/system.RequestTracker...
## 2300 (61.25%) read; 0:01:00 elapsed; 0:00:37 remaining
[Wed Sep 19 13:32:59 +0100 2007] problem getting messages from mbox:///u1/trhpc/jtang/Mail/system.RequestTracker: For some bizarre reason, RubyMail was unable to parse this message.
[Wed Sep 19 13:32:59 +0100 2007] problem getting messages from mbox:///u1/trhpc/jtang/Mail/system.RequestTracker: For some bizarre reason, RubyMail was unable to parse this message.

is there anyway of turning on more debugging to see whats causing the problem...?

i tried using -v on the commandline as well but it doesnt seem to say anything...

Thanks,
Jimmy

-- 
Jimmy Tang
Trinity Centre for High Performance Computing,
Lloyd Building, Trinity College Dublin, Dublin 2, Ireland.
http://www.tchpc.tcd.ie/ | http://www.tchpc.tcd.ie/~jtang

---

From: [email-masked] (William Morgan)
Date: Wed, 19 Sep 2007 07:41:12 -0700

> [Wed Sep 19 13:32:59 +0100 2007] problem getting messages from mbox:///u1/trhpc/jtang/Mail/system.RequestTracker: For some bizarre reason, RubyMail was unable to parse this message.

Interesting. I've never seen that triggered, to date. Congratulations!
Basically this means I asked RubyMail to parse the message, and RubyMail
gave up. 

> is there anyway of turning on more debugging to see whats causing the

Unfortunately no. The best I can offer is the following patch against
0.1, which will print out the message ids. You can then grep the mbox
for those and see if there's anything weird about the messages. I'm
guessing the MIME is screwed up in such a way that RubyMail chokes.

I'm interested in figuring out why this happens.

--- cut here ---
--- lib/sup/message.rb  (revision 503)
+++ lib/sup/message.rb  (working copy)
@@ -320,7 +320,7 @@
         $1
       end
 
-    m.body && body = m.decode or raise MessageFormatError, "For some bizarre reason, RubyMail was unable to parse this message."
+    m.body && body = m.decode or raise MessageFormatError, "For some bizarre reason, RubyMail was unable to parse this message (#{m.header['message-id']})."
 
     if charset
       begin
--- cut here ---

-- 
William <wmorgan-sup at masanjin.net>

---

From: [email-masked] (Jimmy Tang)
Date: Wed, 19 Sep 2007 23:44:56 +0100

Hi again

replying to my own mail, i eventually figured out why the previous problem was failing (corrupt mbox's)
but now, this is happening after starting up sup for about 5mins as it scans my mboxes

23:32:23 jtang at vual ~ $ sup
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/buffer.rb:32: warning: already initialized constant KEY_ENTER
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/buffer.rb:33: warning: already initialized constant KEY_CANCEL
[Wed Sep 19 23:32:25 +0100 2007] locking /u1/trhpc/jtang/.sup/lock...
[Wed Sep 19 23:32:25 +0100 2007] loading index...
[Wed Sep 19 23:32:25 +0100 2007] loaded index of 65133 messages
[Wed Sep 19 23:32:25 +0100 2007] starting curses
[Wed Sep 19 23:41:20 +0100 2007] stopped cursing
[Wed Sep 19 23:41:20 +0100 2007] oh crap, an exception
[Wed Sep 19 23:41:20 +0100 2007] unlocking /u1/trhpc/jtang/.sup/lock...
----------------------------------------------------------------
I'm very sorry, but it seems that an error occurred in Sup.
Please accept my sincere apologies. If you don't mind, please
send the backtrace below and a brief report of the circumstances
to sup-talk at rubyforge dot orgs so that I might address this
problem. Thank you!

Sincerely,
William
----------------------------------------------------------------

The problem was: 'just added message <200610312309.38117.rj200@  but couldn't find it in a search' (error type RuntimeError)
A backtrace follows:
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/index.rb:187:in `sync_message': just added message <200610312309.38117.rj200@  but couldn't find it in a search (RuntimeError)
        from /u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/util.rb:401:in `send'
        from /u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/util.rb:401:in `method_missing'
        from /u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/poll.rb:122:in `add_messages_from'
        from /u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/source.rb:93:in `each'
        from /u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/util.rb:437:in `send'
        from /u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/util.rb:437:in `__pass'
        from /u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/util.rb:426:in `method_missing'
        from /u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/poll.rb:101:in `add_messages_from'
         ... 26 levels...
        from /u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/modes/thread-index-mode.rb:394:in `load_threads'
        from /u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/bin/sup:173
        from /u1/trhpc/jtang/local/ruby/bin/sup:16:in `load'
        from /u1/trhpc/jtang/local/ruby/bin/sup:16

Thanks,
Jimmy

Excerpts from Jimmy Tang's message of Wed Sep 19 13:36:16 +0100 2007:
> Scanned 0, added 0, updated 0 messages from
> mbox:///u1/trhpc/jtang/Mail/system.rancid.
> Scanning sup://drafts...
> [Wed Sep 19 13:32:59 +0100 2007] problem getting messages from
> mbox:///u1/trhpc/jtang/Mail/system.RequestTracker: For some bizarre reason,
> RubyMail was unable to parse this message.
> [Wed Sep 19 13:32:59 +0100 2007] problem getting messages from
> mbox:///u1/trhpc/jtang/Mail/system.RequestTracker: For some bizarre reason,
> RubyMail was unable to parse this message.
>

---

From: [email-masked] (Jimmy Tang)
Date: Thu, 20 Sep 2007 15:31:09 +0100

Hi William,

> Basically this means I asked RubyMail to parse the message, and RubyMail

not too sure whats wrong with that mail box, i eventually found the
offending mails and *cough* just deleted them *cough* they are just old
archives that I have that i wanted indexed.

> I'm interested in figuring out why this happens.

Thanks for the small patch, if i come across it again, i will save the
email and send it on to you to figure out whats broken. In hindsight I
should have kept some samples before deleting things.

on a slightly off topic on sup, i've eventually got sup working
relatively well with my archives of mail. and i quite like it :), I have
one question about the overall list of the inbox-box mode. Is there any
way of say doing an "all mails" mode? or is there a magic query that will
let me list *all* of my mails much like how gmail has the "all mails"
list where it shows "all mails except spam and trash" ?

Thanks,
Jimmy.

---

From: [email-masked] (William Morgan)
Date: Thu, 20 Sep 2007 08:24:45 -0700

Excerpts from Jimmy Tang's message of Thu Sep 20 07:31:09 -0700 2007:
> offending mails and *cough* just deleted them *cough* they are just
> old archives that I have that i wanted indexed.

That last batch of errors looks like more corrupted email, but I could
be wrong. If it works now, it works.

> Is there any way of say doing an "all mails" mode? or is there a magic
> query that will let me list *all* of my mails much like how gmail has
> the "all mails" list where it shows "all mails except spam and trash"
> ?

Yes, search for "*".