From mboxrd@z Thu Jan 1 00:00:00 1970 Received: by 10.42.240.136 with SMTP id la8cs122475icb; Wed, 9 Feb 2011 13:41:47 -0800 (PST) Received: by 10.150.12.2 with SMTP id 2mr2053025ybl.447.1297287706122; Wed, 09 Feb 2011 13:41:46 -0800 (PST) Return-Path: Received: from rubyforge.org (rubyforge.org [205.234.109.19]) by mx.google.com with ESMTP id l3si1534218qcu.114.2011.02.09.13.41.45; Wed, 09 Feb 2011 13:41:46 -0800 (PST) Received-SPF: pass (google.com: domain of sup-devel-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) client-ip=205.234.109.19; Authentication-Results: mx.google.com; spf=pass (google.com: domain of sup-devel-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) smtp.mail=sup-devel-bounces@rubyforge.org Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 825E21858390; Wed, 9 Feb 2011 16:41:45 -0500 (EST) Received: from ping.pong.ch (ping.pong.ch [77.109.141.101]) by rubyforge.org (Postfix) with ESMTP id 6B7CA1858381 for ; Wed, 9 Feb 2011 16:27:15 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by ping.pong.ch (Postfix) with ESMTP id 9ACCC40CB4B4 for ; Wed, 9 Feb 2011 22:27:13 +0100 (CET) Received: from ping.pong.ch ([127.0.0.1]) by localhost (ping.pong.ch [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MbZOeCM6NKUw for ; Wed, 9 Feb 2011 22:27:12 +0100 (CET) Received: from auth sender gaudenz@ping.pong.ch by ping.pong.ch (Postfix) with ESMTPSA id 7D88C40CB4B3 for ; Wed, 9 Feb 2011 22:27:12 +0100 (CET) Received: by meteor.durcheinandertal.local (Postfix, from userid 1000) id 5EF85AD66; Wed, 9 Feb 2011 22:27:13 +0100 (CET) From: Gaudenz Steinlin To: sup-devel Date: Wed, 09 Feb 2011 22:27:13 +0100 Message-Id: <1297286134-sup-7498@meteor.durcheinandertal.local> User-Agent: Sup/git Subject: [sup-devel] Encoding of message snippet in xapian X-BeenThere: sup-devel@rubyforge.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Sup developer discussion List-Id: Sup developer discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: sup-devel-bounces@rubyforge.org Errors-To: sup-devel-bounces@rubyforge.org Hi While debuging the trackeback posted below I discovered that my xapian index contains some of the message snippets in ASCII-8BIT and others in UTF-8. This leads to probelems when building the thread-view for collapsed messages for messages wich have non 7-bit ASCII characters in their snippet. message_patina_lines combines the snippet which comes from the xapian index and is (sometimes) in ASCII-8BIT with other parts of the message which are in UTF-8. I don't know exactly how I ended up with a xapian index which has some strings in ASCII-8BIT and others in UTF-8. I guess it's because I first used sup with Ruby 1.8 and only recently switched to Ruby 1.9.1. What's the expected encoding of things coming from the xapian index? Is there a way to fix the index? Or is a fix in sup needed for this? Or should I just recreate my index from scratch? Running sup with Ruby 1.8 avoids the problem. But I guess it only masks it because 1.8 is not encoding aware. Gaudenz --- Encoding::CompatibilityError from thread: load messages for thread-view-mode incompatible character encodings: UTF-8 and ASCII-8BIT /home/gaudenz/projects/sup/lib/sup/modes/thread-view-mode.rb:792:in `message_patina_lines' /home/gaudenz/projects/sup/lib/sup/modes/thread-view-mode.rb:853:in `chunk_to_lines' /home/gaudenz/projects/sup/lib/sup/modes/thread-view-mode.rb:724:in `block in regen_text' /home/gaudenz/projects/sup/lib/sup/thread.rb:68:in `block in each' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:174:in `each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' /home/gaudenz/projects/sup/lib/sup/thread.rb:67:in `each' /home/gaudenz/projects/sup/lib/sup/modes/thread-view-mode.rb:713:in `regen_text' /home/gaudenz/projects/sup/lib/sup/modes/thread-view-mode.rb:175:in `buffer=' /home/gaudenz/projects/sup/lib/sup/buffer.rb:387:in `spawn' (eval):1:in `spawn' /home/gaudenz/projects/sup/lib/sup/modes/thread-index-mode.rb:120:in `block in select' /home/gaudenz/projects/sup/lib/sup.rb:78:in `block in reporting_thread' -- Ever tried. Ever failed. No matter. Try again. Fail again. Fail better. ~ Samuel Beckett ~ _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel