From mboxrd@z Thu Jan 1 00:00:00 1970 Received: by 10.90.87.14 with SMTP id k14cs1205072agb; Tue, 29 Dec 2009 17:39:32 -0800 (PST) Received: by 10.224.107.210 with SMTP id c18mr8405415qap.137.1262137172325; Tue, 29 Dec 2009 17:39:32 -0800 (PST) Return-Path: Received: from rubyforge.org (rubyforge.org [205.234.109.19]) by mx.google.com with ESMTP id 5si19712627qyk.39.2009.12.29.17.39.30; Tue, 29 Dec 2009 17:39:30 -0800 (PST) Received-SPF: pass (google.com: domain of sup-devel-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) client-ip=205.234.109.19; Authentication-Results: mx.google.com; spf=pass (google.com: domain of sup-devel-bounces@rubyforge.org designates 205.234.109.19 as permitted sender) smtp.mail=sup-devel-bounces@rubyforge.org Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 561B31D7888B; Tue, 29 Dec 2009 20:39:30 -0500 (EST) Received: from magnesium.club.cc.cmu.edu (MAGNESIUM.CLUB.CC.cmu.edu [128.237.157.15]) by rubyforge.org (Postfix) with ESMTP id 9BA7C1D78876 for ; Tue, 29 Dec 2009 20:39:27 -0500 (EST) Received: (qmail 13066 invoked from network); 30 Dec 2009 01:39:27 -0000 Received: from pion.club.cc.cmu.edu (HELO localhost.localdomain) (128.237.157.88) by magnesium.club.cc.cmu.edu with SMTP; 30 Dec 2009 01:39:27 -0000 From: Rich Lane To: sup-devel@rubyforge.org Date: Tue, 29 Dec 2009 17:38:03 -0800 Message-Id: <1262137085-25928-2-git-send-email-rlane@club.cc.cmu.edu> X-Mailer: git-send-email 1.6.3.3 In-Reply-To: <1262137085-25928-1-git-send-email-rlane@club.cc.cmu.edu> References: <1262137085-25928-1-git-send-email-rlane@club.cc.cmu.edu> Subject: [sup-devel] [PATCH 2/4] async thread indexing X-BeenThere: sup-devel@rubyforge.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Sup developer discussion List-Id: Sup developer discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: sup-devel-bounces@rubyforge.org Errors-To: sup-devel-bounces@rubyforge.org --- bin/sup | 2 ++ lib/sup/index.rb | 26 +++++++++++++++++++++++++- lib/sup/modes/thread-index-mode.rb | 2 +- 3 files changed, 28 insertions(+), 2 deletions(-) diff --git a/bin/sup b/bin/sup index 471e833..f9ed7d5 100755 --- a/bin/sup +++ b/bin/sup @@ -141,6 +141,7 @@ Index.lock_interactively or exit begin Redwood::start Index.load + Index.start_sync_worker unless $opts[:no_threads] $die = false trap("TERM") { |x| $die = true } @@ -329,6 +330,7 @@ ensure HookManager.run "shutdown" + Index.stop_sync_worker Redwood::finish stop_cursing Redwood::Logger.remove_all_sinks! diff --git a/lib/sup/index.rb b/lib/sup/index.rb index 5d8d714..1131ec7 100644 --- a/lib/sup/index.rb +++ b/lib/sup/index.rb @@ -28,6 +28,8 @@ class BaseIndex def initialize dir=BASE_DIR @dir = dir @lock = Lockfile.new lockfile, :retries => 0, :max_age => nil + @sync_worker = nil + @sync_queue = Queue.new end def lockfile; File.join @dir, "lock" end @@ -175,10 +177,32 @@ class BaseIndex def save_thread t t.each_dirty_message do |m| - update_message_state m + if @sync_worker + @sync_queue << m + else + update_message_state m + end m.clear_dirty end end + + def start_sync_worker + @sync_worker = Redwood::reporting_thread('index sync') { run_sync_worker } + end + + def stop_sync_worker + return unless worker = @sync_worker + @sync_worker = nil + @sync_queue << :die + worker.join + end + + def run_sync_worker + while m = @sync_queue.deq + return if m == :die + update_message_state m + end + end end index_name = ENV['SUP_INDEX'] || $config[:index] || DEFAULT_INDEX diff --git a/lib/sup/modes/thread-index-mode.rb b/lib/sup/modes/thread-index-mode.rb index c4a7d38..f6ea27c 100644 --- a/lib/sup/modes/thread-index-mode.rb +++ b/lib/sup/modes/thread-index-mode.rb @@ -477,7 +477,7 @@ EOS BufferManager.say("Saving threads...") do |say_id| dirty_threads.each_with_index do |t, i| BufferManager.say "Saving modified thread #{i + 1} of #{dirty_threads.length}...", say_id - Index.save_thread t + Index.save_thread_async t end end end -- 1.6.3.3 _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel