From mboxrd@z Thu Jan  1 00:00:00 1970
From: tarko@lanparty.ee (Tarko Tikan)
Date: Tue, 09 Jun 2009 13:00:13 +0300
Subject: [sup-talk] display_length issue with special-characters on non-UTF8
	terminal
Message-ID: <1244536704-sup-7027@valgus>

hey,

When String.display_length was introduced in recent update, it broke the length for non-UTF8 strings that contain the special characters. Wrong length results corrupted display (line ends chopped off).

Terminal is iso-8859-15 and it's detected by sup correctly.

I've tracked it down to /./u regexp. Here are some examples:

irb(main):001:0> "asd".scan(/./u)
=> ["a", "s", "d"]
irb(main):002:0> "asd????".scan(/./u)
=> ["a", "s", "d", "\365\374\344\366"]
irb(main):017:0> "asd???".scan(/./u)
=> ["a", "s", "d"]

irb(main):008:0* "asd".scan(/./)
=> ["a", "s", "d"]
irb(main):009:0> "asd????".scan(/./)
=> ["a", "s", "d", "\365", "\374", "\344", "\366"]

Expecting UTF8 gives unexpected results :) Also, old behaviour of String.length gives correct results with these test cases.

-- 
tarko