From mboxrd@z Thu Jan 1 00:00:00 1970 Received: by 10.42.224.197 with SMTP id ip5cs47344icb; Thu, 17 Feb 2011 06:44:34 -0800 (PST) Received: by 10.42.230.9 with SMTP id jk9mr2785383icb.519.1297953873733; Thu, 17 Feb 2011 06:44:33 -0800 (PST) Return-Path: Received: from rubyforge.org (rubyforge.org [205.234.109.19]) by mx.google.com with ESMTP id l30si1299191vcp.31.2011.02.17.06.44.31; Thu, 17 Feb 2011 06:44:32 -0800 (PST) Received-SPF: pass (google.com: domain of sup-devel-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) client-ip=205.234.109.19; Authentication-Results: mx.google.com; spf=pass (google.com: domain of sup-devel-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) smtp.mail=sup-devel-bounces@rubyforge.org Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 9B2A33C8039; Thu, 17 Feb 2011 09:44:31 -0500 (EST) Received: from ping.pong.ch (ping.pong.ch [77.109.141.101]) by rubyforge.org (Postfix) with ESMTP id CEEBB185835A for ; Thu, 17 Feb 2011 09:43:04 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by ping.pong.ch (Postfix) with ESMTP id 0CC0F40CB4B4 for ; Thu, 17 Feb 2011 15:43:02 +0100 (CET) Received: from ping.pong.ch ([127.0.0.1]) by localhost (ping.pong.ch [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eUW6m0-VPb0u for ; Thu, 17 Feb 2011 15:43:01 +0100 (CET) Received: from auth sender gaudenz@ping.pong.ch by ping.pong.ch (Postfix) with ESMTPSA id 02F8040CB4B3 for ; Thu, 17 Feb 2011 15:43:01 +0100 (CET) Received: by meteor.durcheinandertal.local (Postfix, from userid 1000) id 295151936B; Thu, 17 Feb 2011 15:43:02 +0100 (CET) From: Gaudenz Steinlin To: sup-devel In-reply-to: <1297286134-sup-7498@meteor.durcheinandertal.local> References: <1297286134-sup-7498@meteor.durcheinandertal.local> Date: Thu, 17 Feb 2011 15:42:58 +0100 Message-Id: <1297953544-sup-6116@meteor.durcheinandertal.local> User-Agent: Sup/git MIME-Version: 1.0 Subject: Re: [sup-devel] Encoding of message snippet in xapian X-BeenThere: sup-devel@rubyforge.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Sup developer discussion List-Id: Sup developer discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0858427106==" Sender: sup-devel-bounces@rubyforge.org Errors-To: sup-devel-bounces@rubyforge.org --===============0858427106== Content-Transfer-Encoding: 8bit Content-Type: multipart/signed; protocol="application/pgp-signature"; boundary="=-1297953782-123205-27882-2477-1-=" --=-1297953782-123205-27882-2477-1-= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable HI Excerpts from Gaudenz Steinlin's message of 2011-02-09 22:27:13 +0100: > Hi = > = > While debuging the trackeback posted below I discovered that my xapian > index contains some of the message snippets in ASCII-8BIT and others > in UTF-8. This leads to probelems when building the thread-view for > collapsed messages for messages wich have non 7-bit ASCII characters > in their snippet. message_patina_lines combines the snippet which > comes from the xapian index and is (sometimes) in ASCII-8BIT with > other parts of the message which are in UTF-8. > = > I don't know exactly how I ended up with a xapian index which has some > strings in ASCII-8BIT and others in UTF-8. I guess it's because I > first used sup with Ruby 1.8 and only recently switched to Ruby 1.9.1. > What's the expected encoding of things coming from the xapian index? > Is there a way to fix the index? Or is a fix in sup needed for this? > Or should I just recreate my index from scratch? > = > Running sup with Ruby 1.8 avoids the problem. But I guess it only > masks it because 1.8 is not encoding aware. Just for the record. I solved the issue by dumping all labels and recreating the index with ruby 1.9. That's what I did: sup-dump > label-dump.txt rm -r ~/.sup/xapian sup-sync --restore label-dump.txt --all-sources While reimporting all my messages I discovered a few message parsing bugs and some bugs related to GPGME. I created patches for all these problems and will submit these in separate mails. Gaudenz > = > Gaudenz > = > --- Encoding::CompatibilityError from thread: load messages for thread-= view-mode > incompatible character encodings: UTF-8 and ASCII-8BIT > /home/gaudenz/projects/sup/lib/sup/modes/thread-view-mode.rb:792:in `me= ssage_patina_lines' > /home/gaudenz/projects/sup/lib/sup/modes/thread-view-mode.rb:853:in `ch= unk_to_lines' > /home/gaudenz/projects/sup/lib/sup/modes/thread-view-mode.rb:724:in `bl= ock in regen_text' > /home/gaudenz/projects/sup/lib/sup/thread.rb:68:in `block in each' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) i= n each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) i= n each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) i= n each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) i= n each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) i= n each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) i= n each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) i= n each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block (2 levels) i= n each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:174:in `each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with= _stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with= _stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with= _stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with= _stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with= _stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with= _stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with= _stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:176:in `block in each_with= _stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each' > /home/gaudenz/projects/sup/lib/sup/thread.rb:175:in `each_with_stuff' > /home/gaudenz/projects/sup/lib/sup/thread.rb:67:in `each' > /home/gaudenz/projects/sup/lib/sup/modes/thread-view-mode.rb:713:in `re= gen_text' > /home/gaudenz/projects/sup/lib/sup/modes/thread-view-mode.rb:175:in `bu= ffer=3D' > /home/gaudenz/projects/sup/lib/sup/buffer.rb:387:in `spawn' > (eval):1:in `spawn' > /home/gaudenz/projects/sup/lib/sup/modes/thread-index-mode.rb:120:in `b= lock in select' > /home/gaudenz/projects/sup/lib/sup.rb:78:in `block in reporting_thread'= -- = Ever tried. Ever failed. No matter. Try again. Fail again. Fail better. ~ Samuel Beckett ~ --=-1297953782-123205-27882-2477-1-= Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) iQEVAwUBTV0z9k0yN7tZsYcyAQgAYQgAskNs4N+fFQrpI+uCcTCWWrg/xHYlVFjA KtqLBTgAM4c8N5mWjCbpHzPn2FFRk4uCamvRlvCQWcDOw0Xxrtc8r8/K8noDpFec QruhPa5fhPfYy4+mZc9lgPJTDlidaBCAwCZHsCvKQ7WBORPH3aCq38wReNTBgxDj ZO7bWfv05QcTn36x9PJKeDkUlHmrJEmDx0x27XHgvThtrAuCMxUmhlgxXJApMEr2 X7Y7DFMp3pXAXwGZwygvin/kw8Q4LDS0b2zei4hZ3URu+aC7sLISHhfdVQrfEt/J B5ypV8vewVw2tCrOuNTWXFuW9FxwV7FzH+pxAzlZdr5N082HcmKSpg== =4zI3 -----END PGP SIGNATURE----- --=-1297953782-123205-27882-2477-1-=-- --===============0858427106== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel --===============0858427106==--