Subject: [sup-talk] Stack overflow in regexp matcher

From: [email-masked] (Mark Drayton)
Date: Mon, 3 Aug 2009 09:16:24 +0100

Hi there

I'm trying to run sup 0.8.1 on Mac OS 10.5.6, Ruby 1.8.6 but sup-sync
is repeatedly dying:

MC-S001930:~ draytm01$ sup-sync -c
[..]
[Mon Aug 03 08:45:18 +0100 2009] Connecting to IMAP server foo:143...
[Mon Aug 03 08:45:18 +0100 2009] Logging in...
[..]
[Mon Aug 03 08:45:19 +0100 2009] fetching IMAP headers 1..1002
[Mon Aug 03 08:45:20 +0100 2009] done fetching IMAP headers
Scanning imap://foo/INBOX...
[..]
## read 440m (about 52%) @ 2.2m/s. 0:03:19 elapsed, about 0:03:03 remaining
[Mon Aug 03 08:48:49 +0100 2009] unlocking /Users/draytm01/.sup/lock...
/Library/Ruby/Gems/1.8/gems/sup-0.8.1/lib/sup/util.rb:204:in `split':
Stack overflow in regexp matcher:
/,\s*(?=(?:[^"]*"[^"]*")*(?![^"]*"))/ (RegexpError)
	from /Library/Ruby/Gems/1.8/gems/sup-0.8.1/lib/sup/util.rb:204:in
`split_on_commas'
	from /Library/Ruby/Gems/1.8/gems/sup-0.8.1/lib/sup/person.rb:110:in
`from_address_list'
	from /Library/Ruby/Gems/1.8/gems/sup-0.8.1/lib/sup/message.rb:104:in
`parse_header'
	from /Library/Ruby/Gems/1.8/gems/sup-0.8.1/lib/sup/message.rb:208:in
`load_from_source!'
	from /Library/Ruby/Gems/1.8/gems/sup-0.8.1/lib/sup/poll.rb:151:in
`add_messages_from'
	from /Library/Ruby/Gems/1.8/gems/sup-0.8.1/lib/sup/imap.rb:187:in `each'
	from /Library/Ruby/Gems/1.8/gems/sup-0.8.1/lib/sup/imap.rb:175:in `upto'
	from /Library/Ruby/Gems/1.8/gems/sup-0.8.1/lib/sup/imap.rb:175:in `each'
	 ... 7 levels...
	from /Library/Ruby/Gems/1.8/gems/sup-0.8.1/bin/sup-sync:135:in `each'
	from /Library/Ruby/Gems/1.8/gems/sup-0.8.1/bin/sup-sync:135
	from /usr/bin/sup-sync:19:in `load'
	from /usr/bin/sup-sync:19

This error seems to have cropped up in other software so I suspect
it's a problem with Ruby, not sup. Anyone able to confirm this?

Cheers,

Mark

---

From: [email-masked] (William Morgan)
Date: Tue, 11 Aug 2009 13:22:30 -0700

Hi Mark,

> /,\s*(?=(?:[^"]*"[^"]*")*(?![^"]*"))/ (RegexpError)

Interesting. Looks like Sup is trying to split a header line into
multiple email addresses and something's causing it to bomb. Are you
able to identify the line that's causing this problem? That would be
helpful to know.

> it's a problem with Ruby, not sup. Anyone able to confirm this?

It's a problem with Ruby's regular expression implementation, I suppose,
but we've never seen it before. The solution is to either tweak the
regexp until it starts working, or implement a custom parsing solution,
neither of which sounds that entertaining...
-- 
William <wmorgan-sup at masanjin.net>

---

From: [email-masked] (Mark Drayton)
Date: Wed, 12 Aug 2009 09:45:01 +0100

Hi William

> able to identify the line that's causing this problem? That would be

Sorry, I should have dug a bit deeper before reporting this. I was
naively assuming it was a known problem (though Google didn't know, so
I guess it's not) and that the answer would be "upgrade foo".

The line it's choking on is 284058 bytes long and contains ~6400
addresses. (gotta love mail from "internal communications"). If I
halve the number of addresses it works, so perhaps this isn't worth
fixing at all. If you do want to tackle it I'm happy to help off-list.

Cheers,

Mark

---

From: [email-masked] (William Morgan)
Date: Sun, 16 Aug 2009 14:43:03 -0700

Reformatted excerpts from Mark Drayton's message of 2009-08-12:
> addresses. (gotta love mail from "internal communications").

Wow!

> If I halve the number of addresses it works, so perhaps this isn't
> worth fixing at all. If you do want to tackle it I'm happy to help
> off-list.

Well... it's clearly a weird case, but it would be nice to do something
besides barf. I suppose we could detect very long lines, emit a warning,
and just parse the first few kb. What do you think?
-- 
William <wmorgan-sup at masanjin.net>