From mboxrd@z Thu Jan 1 00:00:00 1970 Received: by 10.213.28.69 with SMTP id l5cs87304ebc; Wed, 3 Feb 2010 19:58:00 -0800 (PST) Received: by 10.224.17.147 with SMTP id s19mr3379784qaa.246.1265255879188; Wed, 03 Feb 2010 19:57:59 -0800 (PST) Return-Path: Received: from rubyforge.org (rubyforge.org [205.234.109.19]) by mx.google.com with ESMTP id 7si19090384qwf.44.2010.02.03.19.57.58; Wed, 03 Feb 2010 19:57:59 -0800 (PST) Received-SPF: pass (google.com: domain of sup-talk-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) client-ip=205.234.109.19; Authentication-Results: mx.google.com; spf=pass (google.com: domain of sup-talk-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) smtp.mail=sup-talk-bounces@rubyforge.org Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 8E8C218582D3; Wed, 3 Feb 2010 22:57:58 -0500 (EST) X-Greylist: delayed 904 seconds by postgrey-1.31 at rubyforge.org; Wed, 03 Feb 2010 22:57:42 EST Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by rubyforge.org (Postfix) with ESMTP id 7CB5618582AC for ; Wed, 3 Feb 2010 22:57:42 -0500 (EST) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Ncsbx-0002mH-7r for sup-talk@rubyforge.org; Thu, 04 Feb 2010 04:42:33 +0100 Received: from ip-118-90-138-203.xdsl.xnet.co.nz ([118.90.138.203]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 04 Feb 2010 04:42:33 +0100 Received: from olly by ip-118-90-138-203.xdsl.xnet.co.nz with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 04 Feb 2010 04:42:33 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: sup-talk@rubyforge.org From: Olly Betts Date: Thu, 4 Feb 2010 03:42:09 +0000 (UTC) Message-ID: References: <1252867449-30734-1-git-send-email-rlane@club.cc.cmu.edu> <1254339542-sup-886@masanjin.net> <1254341186-sup-4357@zyrg.net> <1254404707-sup-9253@masanjin.net> <1254416360-sup-8957@zyrg.net> Mime-Version: 1.0 X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: sea.gmane.org User-Agent: Loom/3.14 (http://gmane.org/) X-Loom-IP: 118.90.138.203 (Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.1.7) Gecko/20100106 Ubuntu/9.10 (karmic) Firefox/3.5.7) Subject: Re: [sup-talk] =?utf-8?q?=5BPATCH=5D_xapian=3A_do_less_work_for_updat?= =?utf-8?q?e=5Fmessage=5Fstate?= X-BeenThere: sup-talk@rubyforge.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: User & developer discussion of Sup List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: sup-talk-bounces@rubyforge.org Errors-To: sup-talk-bounces@rubyforge.org Olly Betts writes: > On 2009-10-01, Rich Lane wrote: > > Excerpts from William Morgan's message of Thu Oct 01 09:46:20 -0400 2009: > >> Reformatted excerpts from Rich Lane's message of 2009-09-30: > >> > They're about 3 times faster on my machine with this patch. An > >> > optimization the Xapian devs have been planning to make (and that this > >> > patch is necessary to take advantage of) should increase performance > >> > much more. > >> > >> Awesome. Out of curiousity, what's the optimization? > > > > replace_document currently deletes all the old postings and inserts new > > ones. It can be optimized to make the minimal set of modifications. > > This is the ticket for it: > > http://trac.xapian.org/ticket/250 This has now been fixed in Xapian SVN trunk, and I've backported the patches to Xapian's 1.0 branch. You can find snapshot tarballs including these changes here: http://oligarchy.co.uk/xapian/branches/1.0/ Xapian's testsuite passes (including the additional test coverage which I also backported), and I looked over each change carefully, but I would be interested to see some real world testing, particularly in the situation which these changes are intended to improve (i.e. speed of adding/removing tag terms from existing documents). I'm not expecting problems, but this is a larger change than we usually backport, so wider testing would be a good thing. If you are using Xapian 1.0's default backend (flint), you'll need this patch (which was merged before sup 0.10) to get full benefit from the changes: http://article.gmane.org/gmane.mail.sup.devel/177 I've uploaded a snapshot of 1.0 with this patch to Debian experimental. It hasn't built for all Debian architectures yet, but is available for at least amd64 and x86, which are probably the most popular two. If you aren't sure how to pull in packages from experimental, see: http://wiki.debian.org/DebianExperimental I've also put it in a Launchpad PPA for all currently supported Ubuntu releases, and it has built for all of them already: https://launchpad.net/~ojwb/+archive/experimental/ Cheers, Olly _______________________________________________ sup-talk mailing list sup-talk@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-talk