* [sup-devel] sup 0.14: Encoding::UndefinedConversionError from thread: load threads for thread-index-mode
@ 2013-08-15 16:21 Gregor Hoffleit
2013-08-15 16:39 ` Gaute Hope
0 siblings, 1 reply; 5+ messages in thread
From: Gregor Hoffleit @ 2013-08-15 16:21 UTC (permalink / raw)
To: sup-devel
Sup 0.14 fails for me just after the start with the following exception:
--- Encoding::UndefinedConversionError from thread: load threads for thread-index-mode
"\xE2" from ASCII-8BIT to UTF-8
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/util.rb:259:in `width'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/util.rb:259:in `display_length'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/scroll_mode.rb:226:in `block in draw_line_from_array'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/scroll_mode.rb:224:in `each'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/scroll_mode.rb:224:in `each_with_index'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/scroll_mode.rb:224:in `draw_line_from_array'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/scroll_mode.rb:199:in `draw_line'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/line_cursor_mode.rb:52:in `draw_line'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/scroll_mode.rb:46:in `block in draw'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/scroll_mode.rb:46:in `each'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/scroll_mode.rb:46:in `draw'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/line_cursor_mode.rb:37:in `draw'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/buffer.rb:118:in `draw'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/buffer.rb:102:in `redraw'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/buffer.rb:335:in `draw_screen'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/buffer.rb:766:in `clear'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/util.rb:639:in `method_missing'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/thread_index_mode.rb:667:in `load_n_threads'
(eval):12:in `load_n_threads'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/modes/thread_index_mode.rb:638:in `block in load_n_threads_background'
/var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup.rb:85:in `block in reporting_thread'
I noticed that this exception looks similar to a report from 2011:
* Horacio Sanson's message of Mo Apr 25 04:25:04 +0200 2011:
> Any attempt to add a label with Japanese characters crashes the application.
> Seems this is a common problem to all Ruby 1.9 applications. I can see Rails
> had or has a lot of problems with this:
>
> http://yehudakatz.com/2010/05/05/ruby-1-9-encodings-a-primer-and-the-solution-for-rails/
>
> https://rails.lighthouseapp.com/projects/8994/tickets/4683-ascii-8bit-and-utf-8-in-hell
>
>
> The backtrace I get follows:
>
> /var/lib/gems/1.9.1/gems/console-0.3/lib/console/string.rb:27:in
> `display_width': "\xE3" from ASCII-8BIT to UTF-8
> (Encoding::UndefinedConversionError)
> from /var/lib/gems/1.9.1/gems/console-0.3/lib/console/string.rb:27:in
> `display_width'
> from /home/ryujin/Apps/turnsole/lib/turnsole/textfield.rb:129:in
> `handle_input'
> from /home/ryujin/Apps/turnsole/lib/turnsole/input.rb:108:in `ask'
> from /home/ryujin/Apps/turnsole/lib/turnsole/input.rb:141:in
> `ask_many_with_completions'
> from /home/ryujin/Apps/turnsole/lib/turnsole/input.rb:198:in
> `ask_for_labels'
> from /home/ryujin/Apps/turnsole/lib/turnsole/modes/thread-index-
> mode.rb:585:in `block in edit_labels'
> from /home/ryujin/Apps/turnsole/lib/turnsole/input.rb:42:in `asking'
> from /home/ryujin/Apps/turnsole/lib/turnsole/modes/thread-index-
> mode.rb:584:in `edit_labels'
> from /home/ryujin/Apps/turnsole/lib/turnsole/input.rb:92:in `handle'
> from /home/ryujin/Apps/turnsole/lib/turnsole/ui.rb:73:in `step'
> from bin/turnsole:134:in `<main>'
to which William responded:
> Thanks for the bug report. This is a bit tricky. The problem is
> actually that ncurses is giving me the characters one byte at a time.
> I'll see what I can do to fix this.
Do you need any more information to debug this problem?
I'm using Ruby 1.9.3p194 on Debian 7.1. I installed Sup 0.14 with
"gem1.9.1 install sup", which pulled in the following dependencies:
# gem1.9.1 list --local
*** LOCAL GEMS ***
chronic (0.9.1)
highline (1.6.19)
locale (2.0.8)
lockfile (2.1.0)
mime-types (1.24)
ncursesw-sup (1.3.1.3)
rmail (1.0.0)
sup (0.14.0)
trollop (2.0)
unicode (0.4.4)
xapian-ruby (1.2.15.1)
Regards,
Gregor Hoffleit
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-devel] sup 0.14: Encoding::UndefinedConversionError from thread: load threads for thread-index-mode
2013-08-15 16:21 [sup-devel] sup 0.14: Encoding::UndefinedConversionError from thread: load threads for thread-index-mode Gregor Hoffleit
@ 2013-08-15 16:39 ` Gaute Hope
2013-08-15 16:49 ` Gaute Hope
2013-08-16 15:15 ` Gregor Hoffleit
0 siblings, 2 replies; 5+ messages in thread
From: Gaute Hope @ 2013-08-15 16:39 UTC (permalink / raw)
To: sup-devel
Excerpts from Gregor Hoffleit's message of 2013-08-15 18:21:43 +0200:
> Sup 0.14 fails for me just after the start with the following exception:
>
> --- Encoding::UndefinedConversionError from thread: load threads for thread-index-mode
> "\xE2" from ASCII-8BIT to UTF-8
> /var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/util.rb:259:in `width'
> /var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/util.rb:259:in `display_length'
Hi Gregor,
did you have a lot of messages in your index added by Sup 0.13 or
earlier? Do you get the same error if you move the old xapian folder out
of the way and try to re-index your messages?
It is possible to restore labels using sup-dump and sup-sync --restore.
Otherwise, I used to hackishly fix this error when it occured at some
other point by doing the following fix:
diff --git a/lib/sup/util.rb b/lib/sup/util.rb
index 5cff6fa..4579a38 100644
--- a/lib/sup/util.rb
+++ b/lib/sup/util.rb
@@ -256,7 +256,7 @@ end
class String
def display_length
- @display_length ||= Unicode.width(self, false)
+ @display_length ||= Unicode.width(self.fix_encoding, false)
end
def slice_by_display_length len
The problem is that the string which is being formatted (or sought the
width of) is of some sort of encoding (probably already UTF-8), but Ruby
or NCurses or RMail or whatever has lost the encoding, and although the
bytes are encoded they are treated as binary (ASCII-8BIT). There is
generally no way to figure out what the original encoding was, but we
can guess that it is UTF-8 and fix it.
Regards, Gaute
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-devel] sup 0.14: Encoding::UndefinedConversionError from thread: load threads for thread-index-mode
2013-08-15 16:39 ` Gaute Hope
@ 2013-08-15 16:49 ` Gaute Hope
2013-08-16 13:13 ` Gregor Hoffleit
2013-08-16 15:15 ` Gregor Hoffleit
1 sibling, 1 reply; 5+ messages in thread
From: Gaute Hope @ 2013-08-15 16:49 UTC (permalink / raw)
To: sup-devel
Excerpts from Gaute Hope's message of 2013-08-15 18:39:45 +0200:
> Excerpts from Gregor Hoffleit's message of 2013-08-15 18:21:43 +0200:
> > Sup 0.14 fails for me just after the start with the following exception:
> >
> > --- Encoding::UndefinedConversionError from thread: load threads for thread-index-mode
> > "\xE2" from ASCII-8BIT to UTF-8
> > /var/lib/gems/1.9.1/gems/sup-0.14.0/lib/sup/util.rb:259:in `width'
> diff --git a/lib/sup/util.rb b/lib/sup/util.rb
> index 5cff6fa..4579a38 100644
> --- a/lib/sup/util.rb
> +++ b/lib/sup/util.rb
> @@ -256,7 +256,7 @@ end
>
> class String
> def display_length
> - @display_length ||= Unicode.width(self, false)
> + @display_length ||= Unicode.width(self.fix_encoding, false)
> end
>
> def slice_by_display_length len
Created pull request #128, please test if you have the chance.
https://github.com/sup-heliotrope/sup/pull/128
Regards, Gaute
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-devel] sup 0.14: Encoding::UndefinedConversionError from thread: load threads for thread-index-mode
2013-08-15 16:49 ` Gaute Hope
@ 2013-08-16 13:13 ` Gregor Hoffleit
0 siblings, 0 replies; 5+ messages in thread
From: Gregor Hoffleit @ 2013-08-16 13:13 UTC (permalink / raw)
To: sup-devel
* Gaute Hope <eg@gaute.vetsj.com> [2013-08-15 18:49:55 +0200]
> Created pull request #128, please test if you have the chance.
>
> https://github.com/sup-heliotrope/sup/pull/128
Yep, with this workaround, the crash no longer happens.
Gregor
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-devel] sup 0.14: Encoding::UndefinedConversionError from thread: load threads for thread-index-mode
2013-08-15 16:39 ` Gaute Hope
2013-08-15 16:49 ` Gaute Hope
@ 2013-08-16 15:15 ` Gregor Hoffleit
1 sibling, 0 replies; 5+ messages in thread
From: Gregor Hoffleit @ 2013-08-16 15:15 UTC (permalink / raw)
To: Gaute Hope; +Cc: sup-devel
* Gaute Hope <eg@gaute.vetsj.com> [2013-08-15 18:39:45 +0200]
> did you have a lot of messages in your index added by Sup 0.13 or
> earlier? Do you get the same error if you move the old xapian folder out
> of the way and try to re-index your messages?
>
> It is possible to restore labels using sup-dump and sup-sync --restore.
Thanks for this explanation.
Yes, I was running with an index created with Sup < 0.13, and it's
pretty clear to me now that this was the cause of this crash.
Well, I moved away the old xapian folder and called sup-sync --restore,
but this crashed after ten minutes and 10.000 messages (of 500.000).
I just filled an issue #131 to github about this crash.
Gregor
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2013-08-16 15:15 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-08-15 16:21 [sup-devel] sup 0.14: Encoding::UndefinedConversionError from thread: load threads for thread-index-mode Gregor Hoffleit
2013-08-15 16:39 ` Gaute Hope
2013-08-15 16:49 ` Gaute Hope
2013-08-16 13:13 ` Gregor Hoffleit
2013-08-16 15:15 ` Gregor Hoffleit
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox