From mboxrd@z Thu Jan 1 00:00:00 1970 Received: by 10.90.117.16 with SMTP id p16cs274569agc; Sun, 11 Oct 2009 14:05:45 -0700 (PDT) Received: by 10.224.59.130 with SMTP id l2mr4291465qah.73.1255295144561; Sun, 11 Oct 2009 14:05:44 -0700 (PDT) Return-Path: Received: from rubyforge.org (rubyforge.org [205.234.109.19]) by mx.google.com with ESMTP id 31si3247045qyk.19.2009.10.11.14.05.44; Sun, 11 Oct 2009 14:05:44 -0700 (PDT) Received-SPF: pass (google.com: domain of sup-talk-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) client-ip=205.234.109.19; Authentication-Results: mx.google.com; spf=pass (google.com: domain of sup-talk-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) smtp.mail=sup-talk-bounces@rubyforge.org; dkim=neutral (body hash did not verify) header.i=@gmail.com Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 461D713C896B; Sun, 11 Oct 2009 17:05:44 -0400 (EDT) Received: from mail-qy0-f176.google.com (mail-qy0-f176.google.com [209.85.221.176]) by rubyforge.org (Postfix) with ESMTP id 04EEC18581EF for ; Sun, 11 Oct 2009 17:04:57 -0400 (EDT) Received: by qyk6 with SMTP id 6so7621528qyk.3 for ; Sun, 11 Oct 2009 14:04:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:content-type:subject:from:to :in-reply-to:references:date:message-id:user-agent :content-transfer-encoding; bh=FZ/QqlG7cc7Du8lYL57cBjKy5z0j6zk+hEQQtTKzBPE=; b=aJshydIyyGcr1kew4edrrcCoqEEmnUy/EWzipPNil6lhDbh4mHrYLbSfBrNvBwjrDN zcVb4+668ofrDXdlpTAXx9xa2w8bODTWazHxsecXlkG70sZ/keQ3Z7rLk2Ljt4abHlP2 wf0JaBe64WouRlz0dxe7oGSAfdtG4/bmOywKI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=content-type:subject:from:to:in-reply-to:references:date:message-id :user-agent:content-transfer-encoding; b=gzzGfbvLyj252zmLRiQ8GwStlp9BlGGWjsxw8rblUMoEaSoIbfAPwFmpRsrdLhoHyG tcFOz3xOI1CistAVGf5jHWAe+zqYfUQ1ZxT4ydDIFzrpABAgYsKAqNqtFGbWOxxqAvya J8+D5xVpt70t4jZedZ/IoGRJMK4EGd67VGEAI= Received: by 10.224.57.72 with SMTP id b8mr4298712qah.222.1255295097768; Sun, 11 Oct 2009 14:04:57 -0700 (PDT) Received: from localhost (sclient.imf.org [206.229.236.124]) by mx.google.com with ESMTPS id 7sm142823qwb.22.2009.10.11.14.04.56 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 11 Oct 2009 14:04:56 -0700 (PDT) From: Ben Gamari To: sup-talk In-reply-to: <1255292742-sup-3957@masanjin.net> References: <1254955312-sup-7085@ben-laptop> <1255292742-sup-3957@masanjin.net> Date: Sun, 11 Oct 2009 17:04:53 -0400 Message-Id: <1255294905-sup-8@ben-laptop> User-Agent: Sup/git Subject: Re: [sup-talk] Fwd: Re: Crash while in thread-view-mode X-BeenThere: sup-talk@rubyforge.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: User & developer discussion of Sup List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: sup-talk-bounces@rubyforge.org Errors-To: sup-talk-bounces@rubyforge.org Excerpts from William Morgan's message of Sun Oct 11 16:28:48 -0400 2009: > Reformatted excerpts from Ben Gamari's message of 2009-10-07: > > I understand that designing software around a contingency like this > > might not be the best practice, but the frequency with which I've > > needed to rebuild really does make me think that ruby isn't the best > > language for the indexer. > > The indexer isn't in Ruby, it's written in C++ in the case of Xapian and > C in the case of Ferret. Sorry, I was referring to the mail indexer (i.e. message, mbox/maildir parser), not the backend indexing engine (e.g. Xapian). Should have been more specific. > > > This is easily the fifth time I've needed to rebuild and each time it > > has taken over 30 minutes for 1.5 GB of mail. That's substantially > > less than 1MB/second for what should be an I/O bound operation. Ouch. > > I think this isn't the indexer's fault so much as the mbox parsing, > which is Ruby. Exactly. This is where I think C++ is probably appropriate. > > I'm sorry you've had to rebuild the index so many times. The Xapian side > of things is very new, and I think you've had a run of bad luck. But I > am personally not motivated to improve index time performance, because > that's not a common event. At least, it shouldn't be. Completely understandable. I really don't have a right to complain. It does work a large majority of the time, after all. Just figured I'd let you know of problems as they happen. Thanks for the awesome client. - Ben _______________________________________________ sup-talk mailing list sup-talk@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-talk