Archive of RubyForge sup-talk mailing list
 help / color / mirror / Atom feed
* [sup-talk] Character encoding when displaying quoted-printable messages
@ 2009-06-17 21:39 Jim Cheetham
  2009-07-01  5:20 ` Jim Cheetham
  0 siblings, 1 reply; 5+ messages in thread
From: Jim Cheetham @ 2009-06-17 21:39 UTC (permalink / raw)


Hi, I've been looking at sup for only a day, as I'm trying to find a
decent email client. I migrated from mutt to gmail, but now it's time
to move away from gmail because their client is not smart enough.

Sup has passed my number 1 feature test; I can easily read messages
while keeping multiple unsent messages available. Mutt does that, but
it's not easy to see what messages you are currently working on.

However, I'm coming across some character decoding issues with
quoted-printable that are making some messages hard to read. Any
advice on how to improve things would be welcome ... In the example
below, a NBSP entity is showing up as the characters "M-BM-"

I'm on Ubuntu 9.04, with the current master from git. Sup is running
in GNOME Terminal 2.26.0, with UTF-8 encoding, although unsurprisingly
switching to ISO-8859-1 makes no difference. $TERM=xterm (or 'screen'.
vt100 and vt220 cause sup to halt at startup).  This is the text that
gets displayed for a sample message sent from Yahoo! mail ...

Visible in sup thread-view-mode:
videos for specific hints regarding the gestures.M-BM-  Do you have a decent
recording device?M-BM-  Have you seen what's available?M-BM-  Do you have
editing software?

Message source:
X-Mailer: YahooMailClassic/5.4.12 YahooMailWebService/0.7.289.15
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="0-1465842634-1245251490=:14771"
...
--0-1465842634-1245251490=:14771
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
...
more videos for specific hints regarding the gestures.=A0 Do you have a dec=
ent video recording device?=A0 Have you seen what's available?=A0 Do you ha=
ve any video editing software?

Displayed in Gmail client:
I was thinking about cleaning up the videos on YouTube, and perhaps
adding more videos for specific hints regarding the gestures.  Do you
have a decent video recording device?  Have you seen what's available?
 Do you have any video editing software?

HTML source when viewing the MIME attached message locally:
I was thinking about cleaning up the videos on YouTube, and perhaps
adding more videos for specific hints regarding the gestures.  Do
you have a decent video recording device?  Have you seen what's
available?&nbsp; Do you have any video editing software?<br><br>[1]


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [sup-talk] Character encoding when displaying quoted-printable messages
  2009-06-17 21:39 [sup-talk] Character encoding when displaying quoted-printable messages Jim Cheetham
@ 2009-07-01  5:20 ` Jim Cheetham
  2009-07-01  8:38   ` [sup-talk] ncurses-ruby-1.2.3 + ncursesw hack breaks search Ingmar Vanhassel
  0 siblings, 1 reply; 5+ messages in thread
From: Jim Cheetham @ 2009-07-01  5:20 UTC (permalink / raw)


On Thu, Jun 18, 2009 at 9:39 AM, Jim Cheetham<jim at gonzul.net> wrote:
> However, I'm coming across some character decoding issues with
> quoted-printable that are making some messages hard to read. Any
> advice on how to improve things would be welcome ... In the example
> below, a NBSP entity is showing up as the characters "M-BM-"

Nicely fixed by using the hack ncursesw branch, by the way.
http://sup.rubyforge.org/wiki/wiki.pl?UTF8

-jim


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [sup-talk]  ncurses-ruby-1.2.3 + ncursesw hack breaks search
  2009-07-01  5:20 ` Jim Cheetham
@ 2009-07-01  8:38   ` Ingmar Vanhassel
  2009-07-01 15:28     ` William Morgan
  2009-07-01 15:42     ` Jörg-Hendrik Bach
  0 siblings, 2 replies; 5+ messages in thread
From: Ingmar Vanhassel @ 2009-07-01  8:38 UTC (permalink / raw)


Excerpts from Jim Cheetham's message of Wed Jul 01 07:20:10 +0200 2009:
> Nicely fixed by using the hack ncursesw branch, by the way.
> http://sup.rubyforge.org/wiki/wiki.pl?UTF8

Has anyone noticed that newer versions of ncurses-ruby plus that
ncursesw hack [1] (to link to ncurses with widechar support) breaks search?

When I search for 'foo', sup launches a search for a bunch of weird
characters, not for 'foo', which obviously returns nothing useful.

I can trigger this with ncurses-ruby 1.2.2, 1.2.3. Version 1.1 seems to
be unaffected.

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=477366#52
-- 
Exherbo KDE, X.org maintainer


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [sup-talk] ncurses-ruby-1.2.3 + ncursesw hack breaks search
  2009-07-01  8:38   ` [sup-talk] ncurses-ruby-1.2.3 + ncursesw hack breaks search Ingmar Vanhassel
@ 2009-07-01 15:28     ` William Morgan
  2009-07-01 15:42     ` Jörg-Hendrik Bach
  1 sibling, 0 replies; 5+ messages in thread
From: William Morgan @ 2009-07-01 15:28 UTC (permalink / raw)


Reformatted excerpts from Ingmar Vanhassel's message of 2009-07-01:
> [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=477366#52

Interesting thread. I disagree with the conclusion that a separate
libncursesw-ruby package should be created. Maybe I'll respond.

The analysis of Sup is wrong. He says: "Looking at the source code of
the mailer I'd say that it is not really suited for UTF-8 encoded
strings yet, as it still assumes that the length of a string in bytes is
equal to the number of characters in the string." This is not really
true. What Sup assumes is that String#length gives the length of the
string in characters. This is obviously false in Ruby 1.8 (without
monkeypatching), but it is true in Ruby 1.9.

What *does* need to happen for Sup to really, actually, correctly
display non-ASCII characters, besides having a ncurses library that's
linked against ncursesw, is to be able to call the 'wcwidth' and
'wcswidth' functions. The current #display_length function is a hack
that's broken for e.g. Chinese characters.

One way to do this would be to use dl/import like we do for setlocale().
I've played around with this but haven't really gotten it to work. If
someone can get any further than the below, please let me know:

  require 'dl/import'
  module LibC
    extend DL.const_defined?(:Importer) ? DL::Importer : DL::Importable
    dlload "libc.so.6"
    extern "void setlocale(int, const char *)"
    extern "int wcwidth(int)"
    extern "int wcswidth(const int*, int)"
  end

That "works", as in doesn't throw any exceptions, but then calling
LibC.wcswidth doesn't return the right thing, presumably because the
Ruby string isn't being converted into an array of ints, or something. I
don't really know how this works.

> Has anyone noticed that newer versions of ncurses-ruby plus that
> ncursesw hack [1] (to link to ncurses with widechar support) breaks
> search?

I don't have these newer versions available on my system yet (Ubuntu).

> When I search for 'foo', sup launches a search for a bunch of weird
> characters, not for 'foo', which obviously returns nothing useful.

Disturbing.
-- 
William <wmorgan-sup at masanjin.net>


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [sup-talk] ncurses-ruby-1.2.3 + ncursesw hack breaks search
  2009-07-01  8:38   ` [sup-talk] ncurses-ruby-1.2.3 + ncursesw hack breaks search Ingmar Vanhassel
  2009-07-01 15:28     ` William Morgan
@ 2009-07-01 15:42     ` Jörg-Hendrik Bach
  1 sibling, 0 replies; 5+ messages in thread
From: Jörg-Hendrik Bach @ 2009-07-01 15:42 UTC (permalink / raw)


Ingmar Vanhassel, Wed Jul 01 10:38:36 +0200 2009:
 
> Has anyone noticed that newer versions of ncurses-ruby plus that
> ncursesw hack [1] (to link to ncurses with widechar support) breaks search?

I can confirm the behaviour.

> When I search for 'foo', sup launches a search for a bunch of weird
> characters, not for 'foo', which obviously returns nothing useful.

True. However, with the following workaround, you can get proper search results:
- search for foo -> garbled search, no proper results
- hit 'x' to destroy the result buffer
- hit '\' to search again.
- hit uparrow to go to previous search. backspace all the garbled characters there.
- repeat the uparrow + remove until it shows your last working search (or until there's no more previous search to go to)
- type foo again, hit enter: works.

Not that this is the type of thing you'd want to do for a simple search. Guess we'll have to wait for ruby 1.9?

cheers,
- J?rg-Hendrik


> 
> I can trigger this with ncurses-ruby 1.2.2, 1.2.3. Version 1.1 seems to
> be unaffected.
> 
> [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=477366#52


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2009-07-01 15:42 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-06-17 21:39 [sup-talk] Character encoding when displaying quoted-printable messages Jim Cheetham
2009-07-01  5:20 ` Jim Cheetham
2009-07-01  8:38   ` [sup-talk] ncurses-ruby-1.2.3 + ncursesw hack breaks search Ingmar Vanhassel
2009-07-01 15:28     ` William Morgan
2009-07-01 15:42     ` Jörg-Hendrik Bach

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox