From mboxrd@z Thu Jan 1 00:00:00 1970 Received: by 10.90.117.16 with SMTP id p16cs272872agc; Sun, 11 Oct 2009 13:28:55 -0700 (PDT) Received: by 10.224.69.231 with SMTP id a39mr4230845qaj.272.1255292934720; Sun, 11 Oct 2009 13:28:54 -0700 (PDT) Return-Path: Received: from rubyforge.org (rubyforge.org [205.234.109.19]) by mx.google.com with ESMTP id 7si3212932qyk.12.2009.10.11.13.28.54; Sun, 11 Oct 2009 13:28:54 -0700 (PDT) Received-SPF: pass (google.com: domain of sup-talk-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) client-ip=205.234.109.19; Authentication-Results: mx.google.com; spf=pass (google.com: domain of sup-talk-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) smtp.mail=sup-talk-bounces@rubyforge.org Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 611493C803C; Sun, 11 Oct 2009 16:28:54 -0400 (EDT) Received: from entry.masanjin.net (masanjin.net [209.20.72.13]) by rubyforge.org (Postfix) with ESMTP id 9AB3F18581F3 for ; Sun, 11 Oct 2009 16:28:48 -0400 (EDT) Received: from w by entry.masanjin.net with local (Exim 4.69) (envelope-from ) id 1Mx528-00049S-AI for sup-talk@rubyforge.org; Sun, 11 Oct 2009 13:28:48 -0700 From: William Morgan To: sup-talk In-reply-to: <1254955312-sup-7085@ben-laptop> References: <1254955312-sup-7085@ben-laptop> Date: Sun, 11 Oct 2009 13:28:48 -0700 Message-Id: <1255292742-sup-3957@masanjin.net> User-Agent: Sup/git Subject: Re: [sup-talk] Fwd: Re: Crash while in thread-view-mode X-BeenThere: sup-talk@rubyforge.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: User & developer discussion of Sup List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: sup-talk-bounces@rubyforge.org Errors-To: sup-talk-bounces@rubyforge.org Reformatted excerpts from Ben Gamari's message of 2009-10-07: > I understand that designing software around a contingency like this > might not be the best practice, but the frequency with which I've > needed to rebuild really does make me think that ruby isn't the best > language for the indexer. The indexer isn't in Ruby, it's written in C++ in the case of Xapian and C in the case of Ferret. > This is easily the fifth time I've needed to rebuild and each time it > has taken over 30 minutes for 1.5 GB of mail. That's substantially > less than 1MB/second for what should be an I/O bound operation. Ouch. I think this isn't the indexer's fault so much as the mbox parsing, which is Ruby. I'm sorry you've had to rebuild the index so many times. The Xapian side of things is very new, and I think you've had a run of bad luck. But I am personally not motivated to improve index time performance, because that's not a common event. At least, it shouldn't be. -- William _______________________________________________ sup-talk mailing list sup-talk@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-talk