From mboxrd@z Thu Jan 1 00:00:00 1970 Received: by 10.58.247.3 with SMTP id ya3csp488741vec; Sun, 2 Sep 2012 22:00:45 -0700 (PDT) Received: by 10.60.3.194 with SMTP id e2mr12963288oee.1.1346648444990; Sun, 02 Sep 2012 22:00:44 -0700 (PDT) Return-Path: Received: from rubyforge.org (50-56-192-79.static.cloud-ips.com. [50.56.192.79]) by mx.google.com with ESMTP id qd10si12044410obc.11.2012.09.02.22.00.44; Sun, 02 Sep 2012 22:00:44 -0700 (PDT) Received-SPF: pass (google.com: domain of sup-devel-bounces@rubyforge.org designates 50.56.192.79 as permitted sender) client-ip=50.56.192.79; Authentication-Results: mx.google.com; spf=pass (google.com: domain of sup-devel-bounces@rubyforge.org designates 50.56.192.79 as permitted sender) smtp.mail=sup-devel-bounces@rubyforge.org Received: from localhost.localdomain (localhost [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 9A9612E06A; Mon, 3 Sep 2012 05:00:44 +0000 (UTC) Received: from dmz-mailsec-scanner-7.mit.edu (DMZ-MAILSEC-SCANNER-7.MIT.EDU [18.7.68.36]) by rubyforge.org (Postfix) with ESMTP id CC1A82E05F for ; Mon, 3 Sep 2012 04:59:36 +0000 (UTC) X-AuditID: 12074424-b7f856d000000942-4d-504439374b71 Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) by dmz-mailsec-scanner-7.mit.edu (Symantec Messaging Gateway) with SMTP id 3B.84.02370.73934405; Mon, 3 Sep 2012 00:59:35 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id q834xZbb018817; Mon, 3 Sep 2012 00:59:35 -0400 Received: from localhost ([18.111.73.140]) (authenticated bits=0) (User authenticated as ezyang@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q834xYQk025611; Mon, 3 Sep 2012 00:59:34 -0400 (EDT) From: "Edward Z. Yang" To: sup-devel@rubyforge.org Date: Mon, 3 Sep 2012 00:59:31 -0400 Message-Id: <1346648371-12305-1-git-send-email-ezyang@mit.edu> X-Mailer: git-send-email 1.7.11.3 In-Reply-To: <1345564795-sup-3898@alvh.no-ip.org> References: <1345564795-sup-3898@alvh.no-ip.org> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrIIsWRmVeSWpSXmKPExsUixCmqrWtu6RJg0NRmZXFh8h92i8fT97A7 MHlc+TWR3WPfvonMAUxRXDYpqTmZZalF+nYJXBm/mr6yFWxXq1i5ZQpLA+MFqS5GTg4JAROJ SfemMEPYYhIX7q1n62Lk4hAS2Mco0f50JjuEs55R4s7LO1DOU0aJd1N+soG0sAmoSzw69pQV xBYRkJY4+HYxC4jNLGAncW/OG3YQW1jARmLr19lgNouAqsSPZ5cZQWxeoJo5jzeyQ6xWlPjx fQ3YGZwChhIrt30EmyMkYCDRv/In0wRGvgWMDKsYZVNyq3RzEzNzilOTdYuTE/PyUot0zfVy M0v0UlNKNzGCQobdRWUHY/MhpUOMAhyMSjy8Kz44BwixJpYVV+YeYpTkYFIS5d1t7BIgxJeU n1KZkVicEV9UmpNafIhRgoNZSYT37iWgct6UxMqq1KJ8mJQ0B4uSOO/1lJv+QgLpiSWp2amp BalFMFkZDg4lCd5mC6ChgkWp6akVaZk5JQhpJg5OkOE8QMODQWp4iwsSc4sz0yHypxgVpcR5 o0ESAiCJjNI8uF5YTL9iFAd6RZi3BKSKB5gO4LpfAQ1mAhp8TBDk6uKSRISUVAOjv0WWfIuE p/aR2+tz9oioFJa6//HXS7iU8JqhSNjotFCVc+Z5C0n9rW2Xp3HFO6aU3Qzduig9+5/rhUUR qwyXPG9bs35ayu00M9WbPCnpzOX3n/SzqLTeyxNqE+Z8LFR6KzPWjaN894x/k1ewMlwOPxjJ 8O+Y8P0zbz8+t7ktsmWr47K/Gz8osRRnJBpqMRcVJwIAcHwP2cQCAAA= Subject: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) X-BeenThere: sup-devel@rubyforge.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Sup developer discussion List-Id: Sup developer discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: sup-devel-bounces@rubyforge.org Errors-To: sup-devel-bounces@rubyforge.org From: "Edward Z. Yang" Signed-off-by: Edward Z. Yang --- lib/sup/maildir.rb | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++-- lib/sup/poll.rb | 33 ++++++++++++++++++++++++++------- lib/sup/source.rb | 4 ++++ 3 files changed, 80 insertions(+), 9 deletions(-) diff --git a/lib/sup/maildir.rb b/lib/sup/maildir.rb index 2a91f05..743156d 100644 --- a/lib/sup/maildir.rb +++ b/lib/sup/maildir.rb @@ -1,5 +1,6 @@ require 'uri' require 'set' +require 'inotify' module Redwood @@ -184,6 +185,45 @@ class Maildir < Source nil end + def continuous_poll poll_mutex + i = Inotify.new + watches = {} + @ctimes.each do |d,prev_ctime| + subdir = File.join @dir, d + wd = i.add_watch(subdir, Inotify::CREATE | Inotify::DELETE | Inotify::MOVE) + watches[wd] = d + end + i.each_event do |ev| + poll_mutex.synchronize do + @mutex.synchronize do + begin + ::Thread.current[@dir] = true + id = File.join watches[ev.wd], ev.name + # check if inotify is stale + # since we have @mutex, there is no race (except for + # an external program fucking us over) + next unless File.exists? File.join(@dir, id) + x = Enumerator.new(Index.instance, :each_source_info, self.id, "#{id}").to_a + if ev.mask & Inotify::CREATE or ev.mask & Inotify::MOVE_TO + next unless x.empty? + yield :add, + :info => id, + :labels => @labels + maildir_labels(id) + [:inbox], + :progress => 0 + elsif ev.mask & Inotify::DELETE or ev.mask & Inotify::MOVE_FROM + next unless !x.empty? + yield :delete, + :info => id, + :progress => 0 + end + ensure + ::Thread.current[@dir] = nil + end + end + end + end + end + def labels? id maildir_labels id end @@ -248,7 +288,16 @@ private end def maildir_move_file orig_path, new_source_id, flags - @mutex.synchronize do + if ::Thread.current[@dir] + _maildir_move_file orig_path, new_source_id, flags + else + @mutex.synchronize do + _maildir_move_file orig_path, new_source_id, flags + end + end + end + + def _maildir_move_file orig_path, new_source_id, flags new_base = (flags.include?("S")) ? "cur" : "new" md_base, md_ver, md_flags = maildir_data orig_path @@ -292,7 +341,6 @@ private end [new_source, new_loc] - end end end diff --git a/lib/sup/poll.rb b/lib/sup/poll.rb index dbd351f..51e0afa 100644 --- a/lib/sup/poll.rb +++ b/lib/sup/poll.rb @@ -94,11 +94,27 @@ EOS poll if @last_poll.nil? || (Time.now - @last_poll) >= @delay end end + # XXX dup dup + SourceManager.usual_sources.each do |source| + Redwood::reporting_thread("inotify poll for #{source}") do + source.continuous_poll @mutex do |sym, args| + poll_handler source, sym, args + end + end + end + SourceManager.unusual_sources.each do |source| + Redwood::reporting_thread("inotify poll for #{source}") do + source.continuous_poll @mutex do |sym, args| + poll_handler source, sym, args + end + end + end end def stop @thread.kill if @thread @thread = nil + # handle inotify polls end def do_poll @@ -172,7 +188,16 @@ EOS ## from the index after being yielded. def poll_from source, opts={} begin - source.poll do |sym, args| + source.poll do |sym,args| + poll_handler source, sym, args + end + source.go_idle + rescue SourceError => e + warn "problem getting messages from #{source}: #{e.message}" + end + end + + def poll_handler source, sym, args case sym when :add m = Message.build_from_source source, args[:info] @@ -224,12 +249,6 @@ EOS UpdateManager.relay self, :updated, m end end - end - - source.go_idle - rescue SourceError => e - warn "problem getting messages from #{source}: #{e.message}" - end end def handle_idle_update sender, idle_since; @should_clear_running_totals = false; end diff --git a/lib/sup/source.rb b/lib/sup/source.rb index 06b6e6b..073a10a 100644 --- a/lib/sup/source.rb +++ b/lib/sup/source.rb @@ -102,6 +102,10 @@ class Source unimplemented end + ## Like poll, but never returns (it is continuous, and uses something + ## like inotify. Will always be run in another thread.) + def continuous_poll poll_mutex; [] end + def valid? info true end -- 1.7.11.3 _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel