* [sup-devel] sup v2 progress report
@ 2011-03-27 20:41 William Morgan
2011-03-28 4:14 ` [sup-talk] " Matthieu Rakotojaona
` (4 more replies)
0 siblings, 5 replies; 11+ messages in thread
From: William Morgan @ 2011-03-27 20:41 UTC (permalink / raw)
To: sup-talk, sup-devel
Hello all,
I'm happy to report that Sup version 2 is well underway!
Sup version 2 features:
- a client/server model that works over HTTP, allowing
a) development of other clients, e.g. web-based and phone-based
b) simultaneous access from multiple clients
c) IMAP emulation, aka no more lock-in!
- an improved console-based client
Heliotrope, the server component, is close to ready for a version 1 release.
You can find it at https://github.com/wmorgan/heliotrope/.
The client, which I'm calling Turnsole, is coming along swimmingly. I hope to
release a pre-alpha some-stuff-actually-works version within the next few days.
Much of the UI code has been borrowed from Sup, but the internals are quite
different:
- it's event-based, rather than thread-based, which streamlines a lot of the
code and avoids a whole big class of bugs.
- all the index and email threading code is ripped out
- most of the email parsing code is gone
- it handles Ruby 1.9 string encoding stuff correctly, rather than having
random checks scattered around
I think you're going to like it, since:
- the threads are pre-computed on the server side, so it's much, much faster
- you can finally view attachments locally!
Stay tuned for more. We're still a ways off before it's a drop-in replacement,
but I'm excited about how everything is coming together.
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [sup-talk] sup v2 progress report
2011-03-27 20:41 [sup-devel] sup v2 progress report William Morgan
@ 2011-03-28 4:14 ` Matthieu Rakotojaona
2011-03-28 22:57 ` William Morgan
2011-03-28 12:52 ` [sup-devel] " Nicolas Pouillard
` (3 subsequent siblings)
4 siblings, 1 reply; 11+ messages in thread
From: Matthieu Rakotojaona @ 2011-03-28 4:14 UTC (permalink / raw)
To: sup-talk, sup-devel
Thank you very much.
I've been using sup for quite a few months now, and I must say that
the labels-centric point of view really is an improvement over the
other mail management systems.
I've been following sup news very closely, and I'm really excited
about heliotrope/turnsole.
I have tried the new system a little bit, and I have a few things to say :
- you must have 'html2text' (not specified) installed on your system,
as long as you have any html mail. I think this is the case for too
many of us, unfortunately.
- I didn't find rubymail with gem, but found it under the name rmail
- I still had some problem with the encoding stuff. I was using ruby
v1.9.2 (or so I thought), but got the "ArgumentError - invalid byte
sequence in UTF-8" error. Strange thing, the log showed me evidence
that I was using ruby v1.9.1 (problems came from files in
"/usr/lib/ruby/gems/1.9.1"). So I switched back to v1.8.7 from AUR
(I'm using archlinux), and the problem just went away.
In the light of what you wrote, I have two questions :
- Is sup development bound to be stopped, at least when
heliotrope/turnsole will be ready ?
- Is there any mailing-list for this new project, or should we keep
using those related with sup ?
- Do you have any address where I can find and test turnsole ?
Again, thank you very much for your work and your help, and I'm
talking to everyone on these lists.
Regards,
--
Matthieu RAKOTOJAONA
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [sup-devel] sup v2 progress report
2011-03-27 20:41 [sup-devel] sup v2 progress report William Morgan
2011-03-28 4:14 ` [sup-talk] " Matthieu Rakotojaona
@ 2011-03-28 12:52 ` Nicolas Pouillard
2011-03-29 3:11 ` [sup-talk] " William Morgan
2011-03-29 4:00 ` William Morgan
2011-04-13 20:18 ` Alvaro Herrera
` (2 subsequent siblings)
4 siblings, 2 replies; 11+ messages in thread
From: Nicolas Pouillard @ 2011-03-28 12:52 UTC (permalink / raw)
To: William Morgan, sup-talk, sup-devel
On Sun, 27 Mar 2011 20:41:59 +0000, William Morgan <wmorgan-sup@masanjin.net> wrote:
> Hello all,
>
> I'm happy to report that Sup version 2 is well underway!
>
> Sup version 2 features:
> - a client/server model that works over HTTP, allowing
> a) development of other clients, e.g. web-based and phone-based
> b) simultaneous access from multiple clients
> c) IMAP emulation, aka no more lock-in!
> - an improved console-based client
>
> Heliotrope, the server component, is close to ready for a version 1 release.
> You can find it at https://github.com/wmorgan/heliotrope/.
>
> The client, which I'm calling Turnsole, is coming along swimmingly. I hope to
> release a pre-alpha some-stuff-actually-works version within the next few days.
> Much of the UI code has been borrowed from Sup, but the internals are quite
> different:
> - it's event-based, rather than thread-based, which streamlines a lot of the
> code and avoids a whole big class of bugs.
> - all the index and email threading code is ripped out
> - most of the email parsing code is gone
> - it handles Ruby 1.9 string encoding stuff correctly, rather than having
> random checks scattered around
>
> I think you're going to like it, since:
> - the threads are pre-computed on the server side, so it's much, much faster
> - you can finally view attachments locally!
>
> Stay tuned for more. We're still a ways off before it's a drop-in replacement,
> but I'm excited about how everything is coming together.
While mass importing an mbox I got this:
$ ... heliotrope-add ...
; forced to decode html. running html2text on 868b mime part...
end offset is 237534473
.../heliotrope/lib/heliotrope/decoder.rb:109:in `gsub': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
from .../heliotrope/lib/heliotrope/decoder.rb:109:in `decode_rfc2047'
from .../heliotrope/lib/heliotrope/message.rb:208:in `decode_header'
from .../heliotrope/lib/heliotrope/message.rb:30:in `parse!'
from bin/heliotrope-add:105:in `<main>'
Best regards,
--
Nicolas Pouillard
http://nicolaspouillard.fr
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [sup-talk] sup v2 progress report
2011-03-28 4:14 ` [sup-talk] " Matthieu Rakotojaona
@ 2011-03-28 22:57 ` William Morgan
0 siblings, 0 replies; 11+ messages in thread
From: William Morgan @ 2011-03-28 22:57 UTC (permalink / raw)
To: sup-talk, sup-devel
Reformatted excerpts from Matthieu Rakotojaona's message of 2011-03-28:
> I have tried the new system a little bit, and I have a few things to say :
> - you must have 'html2text' (not specified) installed on your system,
> as long as you have any html mail. I think this is the case for too
> many of us, unfortunately.
> - I didn't find rubymail with gem, but found it under the name rmail
Thanks, I will update the README.
> - I still had some problem with the encoding stuff. I was using ruby
> v1.9.2 (or so I thought), but got the "ArgumentError - invalid byte
> sequence in UTF-8" error.
Ok, I'm still working on tracking this down. If you have a backtrace
handy, that would be useful.
> - Is sup development bound to be stopped, at least when
> heliotrope/turnsole will be ready ?
I'm planning on moving my efforts entirely over to heliotrope +
turnsole. Which isn't that big of a deal, really, since I haven't done
much on Sup for years.
> - Is there any mailing-list for this new project, or should we keep
> using those related with sup ?
I'll keep talking about it here until people complain.
> - Do you have any address where I can find and test turnsole ?
Check back in a few days.
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [sup-talk] [sup-devel] sup v2 progress report
2011-03-28 12:52 ` [sup-devel] " Nicolas Pouillard
@ 2011-03-29 3:11 ` William Morgan
2011-03-29 4:00 ` William Morgan
1 sibling, 0 replies; 11+ messages in thread
From: William Morgan @ 2011-03-29 3:11 UTC (permalink / raw)
To: sup-talk, sup-devel
Reformatted excerpts from Nicolas Pouillard's message of 2011-03-28:
> .../heliotrope/lib/heliotrope/decoder.rb:109:in `gsub': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
Hm. I think I know what the problem is. Stay tuned.
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [sup-devel] sup v2 progress report
2011-03-28 12:52 ` [sup-devel] " Nicolas Pouillard
2011-03-29 3:11 ` [sup-talk] " William Morgan
@ 2011-03-29 4:00 ` William Morgan
2011-03-29 21:19 ` [sup-talk] " Nicolas Pouillard
1 sibling, 1 reply; 11+ messages in thread
From: William Morgan @ 2011-03-29 4:00 UTC (permalink / raw)
To: sup-talk, sup-devel
Reformatted excerpts from Nicolas Pouillard's message of 2011-03-28:
> .../heliotrope/lib/heliotrope/decoder.rb:109:in `gsub': incompatible
> character encodings: UTF-8 and ASCII-8BIT
> (Encoding::CompatibilityError)
Can you try with the latest master? If it still doesn't work, are you
able to narrow down the string encodings of from and word?
This 1.9 string encoding stuff is tricky business, especially combined
with the messy world of email. I think I'm getting close.
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [sup-talk] [sup-devel] sup v2 progress report
2011-03-29 4:00 ` William Morgan
@ 2011-03-29 21:19 ` Nicolas Pouillard
2011-03-29 21:31 ` William Morgan
0 siblings, 1 reply; 11+ messages in thread
From: Nicolas Pouillard @ 2011-03-29 21:19 UTC (permalink / raw)
To: William Morgan, sup-talk, sup-devel
On Tue, 29 Mar 2011 04:00:36 +0000, William Morgan <wmorgan-sup@masanjin.net> wrote:
> Reformatted excerpts from Nicolas Pouillard's message of 2011-03-28:
> > .../heliotrope/lib/heliotrope/decoder.rb:109:in `gsub': incompatible
> > character encodings: UTF-8 and ASCII-8BIT
> > (Encoding::CompatibilityError)
>
> Can you try with the latest master? If it still doesn't work, are you
> able to narrow down the string encodings of from and word?
>
> This 1.9 string encoding stuff is tricky business, especially combined
> with the messy world of email. I think I'm getting close.
Nice, it seems to go a lot further. However there seems to be some strange
blocking behavior, like using CPU and producing nothing in hours.
--
Nicolas Pouillard
http://nicolaspouillard.fr
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [sup-talk] [sup-devel] sup v2 progress report
2011-03-29 21:19 ` [sup-talk] " Nicolas Pouillard
@ 2011-03-29 21:31 ` William Morgan
0 siblings, 0 replies; 11+ messages in thread
From: William Morgan @ 2011-03-29 21:31 UTC (permalink / raw)
To: sup-talk, sup-devel
Reformatted excerpts from Nicolas Pouillard's message of 2011-03-29:
> Nice, it seems to go a lot further. However there seems to be some strange
> blocking behavior, like using CPU and producing nothing in hours.
Weird. I wonder if it's triggering some regex worst case like we've seen
before in Sup. Are you able to narrow down what message or text is
causing this? I will add a --verbose option, which might make this
easier.
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [sup-devel] sup v2 progress report
2011-03-27 20:41 [sup-devel] sup v2 progress report William Morgan
2011-03-28 4:14 ` [sup-talk] " Matthieu Rakotojaona
2011-03-28 12:52 ` [sup-devel] " Nicolas Pouillard
@ 2011-04-13 20:18 ` Alvaro Herrera
2011-04-14 7:39 ` [sup-talk] " Ico Doornekamp
2011-04-14 7:51 ` Ico Doornekamp
4 siblings, 0 replies; 11+ messages in thread
From: Alvaro Herrera @ 2011-04-13 20:18 UTC (permalink / raw)
To: Sup developer discussion
Excerpts from William Morgan's message of dom mar 27 17:41:59 -0300 2011:
> Heliotrope, the server component, is close to ready for a version 1 release.
> You can find it at https://github.com/wmorgan/heliotrope/.
Hi,
I find this to be great. Thanks for writing it.
I just tried to follow the instructions on a Debian machine and noticed
that you need to install the package ruby1.9.1-dev in order for the
whistlepig and oklahoma_mixer gems. Probably most Ruby-addicted people
already have it so they don't see that problem, but since I'm not one of
those, it did bit me. I suggest this should be in the README file.
I ran into a problem importing email that has Latin1 headers. The crash
I got was:
/home/alvherre/Code/heliotrope/lib/heliotrope/maildir-walker.rb:44:in `block in get_date_in_file': invalid byte sequence in UTF-8 (ArgumentError)
from /home/alvherre/Code/heliotrope/lib/heliotrope/maildir-walker.rb:42:in `open'
from /home/alvherre/Code/heliotrope/lib/heliotrope/maildir-walker.rb:42:in `get_date_in_file'
from /home/alvherre/Code/heliotrope/lib/heliotrope/maildir-walker.rb:35:in `block in get_files'
from /home/alvherre/Code/heliotrope/lib/heliotrope/maildir-walker.rb:35:in `map'
from /home/alvherre/Code/heliotrope/lib/heliotrope/maildir-walker.rb:35:in `get_files'
from /home/alvherre/Code/heliotrope/lib/heliotrope/maildir-walker.rb:21:in `done?'
from bin/heliotrope-add:98:in `<main>'
I added a debugging rescue to that block and found that the message has
this header:
List-Id: Consultas T<E9>cnicas sobre Linux y Software Libre <l-linux.velug.org.ve>
Note that the <E9> is actually a Latin-1 "é". (The message has a valid Date:
header).
This is obviously a bogus message; you're not supposed to use non-ASCII
chars in the header. But these guys did it anyway. I'm not sure what
would be a good fix for this problem. I guess it involves discarding
the failing line so that it can index the message using the proper date
instead of "0".
I haven't tried Turnsole yet. I checked the WEBrick thingy and it seems
pretty neat.
Thanks!
--
Álvaro Herrera <alvherre@alvh.no-ip.org>
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [sup-talk] [sup-devel] sup v2 progress report
2011-03-27 20:41 [sup-devel] sup v2 progress report William Morgan
` (2 preceding siblings ...)
2011-04-13 20:18 ` Alvaro Herrera
@ 2011-04-14 7:39 ` Ico Doornekamp
2011-04-14 7:51 ` Ico Doornekamp
4 siblings, 0 replies; 11+ messages in thread
From: Ico Doornekamp @ 2011-04-14 7:39 UTC (permalink / raw)
To: William Morgan; +Cc: sup-talk, sup-devel
* On Sun Mar 27 22:41:59 +0200 2011, William Morgan wrote:
> Heliotrope, the server component, is close to ready for a version 1 release.
> You can find it at https://github.com/wmorgan/heliotrope/.
Heliotrope (Actually Time.parse()) crashes on some illegaly formatted dates:
/usr/lib/ruby/1.9.1/time.rb:137:in `apply_offset': undefined method `<' for nil:NilClass (NoMethodError)
from /usr/lib/ruby/1.9.1/time.rb:197:in `make_time'
from /usr/lib/ruby/1.9.1/time.rb:261:in `parse'
from /home/ico/external/heliotrope/lib/heliotrope/message.rb:27:in `parse!'
from bin/heliotrope-add:108:in `<main>'
The date of the message was "Wed, 7 2005 22:55: 1 -0180".
Fixed by adding a NoMethodError catch:
diff --git a/lib/heliotrope/message.rb b/lib/heliotrope/message.rb
index 1682062..d63e411 100644
--- a/lib/heliotrope/message.rb
+++ b/lib/heliotrope/message.rb
@@ -23,7 +23,7 @@ class Message
@from = Person.from_string decode_header(validate_field(:from, @m.header["from"]))
@date = begin
Time.parse(validate_field(:date, @m.header["date"])).to_i
- rescue ArgumentError
+ rescue ArgumentError, NoMethodError
#puts "warning: invalid date field #{@m.header['date']}"
Time.at 0
end
--
:wq
^X^Cy^K^X^C^C^C^C
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [sup-devel] sup v2 progress report
2011-03-27 20:41 [sup-devel] sup v2 progress report William Morgan
` (3 preceding siblings ...)
2011-04-14 7:39 ` [sup-talk] " Ico Doornekamp
@ 2011-04-14 7:51 ` Ico Doornekamp
4 siblings, 0 replies; 11+ messages in thread
From: Ico Doornekamp @ 2011-04-14 7:51 UTC (permalink / raw)
To: William Morgan; +Cc: sup-talk, sup-devel
* On Sun Mar 27 22:41:59 +0200 2011, William Morgan wrote:
> Heliotrope, the server component, is close to ready for a version 1 release.
> You can find it at https://github.com/wmorgan/heliotrope/.
Here's another crash, but I've not been able yet to find out what exactly is
happening. The problem seems to be caused by a mismatch of multipart mime
boundary strings.
/home/ico/external/heliotrope/lib/heliotrope/message.rb:151:in `decode_mime_parts': undefined method `multipart?' for nil:NilClass (NoMethodError)
from /home/ico/external/heliotrope/lib/heliotrope/message.rb:154:in `decode_mime_parts'
from /home/ico/external/heliotrope/lib/heliotrope/message.rb:130:in `mime_parts'
from /home/ico/external/heliotrope/lib/heliotrope/message.rb:116:in `has_attachment?'
from /home/ico/external/heliotrope/lib/heliotrope/index.rb:77:in `add_message'
from bin/heliotrope-add:113:in `<main>'
You have new mail in /home/ico/Maildir
I've stripped the offending mail down to the following minimal mbox which causes the crash:
From foo@bar.com Tue May 9 11:22:38 2006
Date: Tue, 09 May 2006 02:22:18 -0800
From: foo@bar.com
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="one"
Message-Id: <1>
--two
Part
--three
--
:wq
^X^Cy^K^X^C^C^C^C
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2011-04-14 8:08 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-03-27 20:41 [sup-devel] sup v2 progress report William Morgan
2011-03-28 4:14 ` [sup-talk] " Matthieu Rakotojaona
2011-03-28 22:57 ` William Morgan
2011-03-28 12:52 ` [sup-devel] " Nicolas Pouillard
2011-03-29 3:11 ` [sup-talk] " William Morgan
2011-03-29 4:00 ` William Morgan
2011-03-29 21:19 ` [sup-talk] " Nicolas Pouillard
2011-03-29 21:31 ` William Morgan
2011-04-13 20:18 ` Alvaro Herrera
2011-04-14 7:39 ` [sup-talk] " Ico Doornekamp
2011-04-14 7:51 ` Ico Doornekamp
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox