* [sup-devel] inotify support for Maildir mailboxes @ 2012-08-21 15:17 Edward Z. Yang 2012-08-21 16:00 ` Alvaro Herrera 0 siblings, 1 reply; 15+ messages in thread From: Edward Z. Yang @ 2012-08-21 15:17 UTC (permalink / raw) To: sup-devel I'm planning on adding inotify support for Maildir mailboxes. This would mean we no longer need to 'poll' to find new messages; they show up instantly. Let me know if you're interested and willing to help test. Cheers, Edward _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] inotify support for Maildir mailboxes 2012-08-21 15:17 [sup-devel] inotify support for Maildir mailboxes Edward Z. Yang @ 2012-08-21 16:00 ` Alvaro Herrera 2012-09-03 4:59 ` [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) Edward Z. Yang 0 siblings, 1 reply; 15+ messages in thread From: Alvaro Herrera @ 2012-08-21 16:00 UTC (permalink / raw) To: sup-devel Excerpts from Edward Z. Yang's message of mar ago 21 11:17:32 -0400 2012: > I'm planning on adding inotify support for Maildir mailboxes. > This would mean we no longer need to 'poll' to find new messages; > they show up instantly. Let me know if you're interested and willing > to help test. I definitely am interested and willing to test. -- Álvaro Herrera <alvherre@alvh.no-ip.org> _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-08-21 16:00 ` Alvaro Herrera @ 2012-09-03 4:59 ` Edward Z. Yang 2012-09-03 5:00 ` Edward Z. Yang 0 siblings, 1 reply; 15+ messages in thread From: Edward Z. Yang @ 2012-09-03 4:59 UTC (permalink / raw) To: sup-devel From: "Edward Z. Yang" <ezyang@mit.edu> Signed-off-by: Edward Z. Yang <ezyang@mit.edu> --- lib/sup/maildir.rb | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++-- lib/sup/poll.rb | 33 ++++++++++++++++++++++++++------- lib/sup/source.rb | 4 ++++ 3 files changed, 80 insertions(+), 9 deletions(-) diff --git a/lib/sup/maildir.rb b/lib/sup/maildir.rb index 2a91f05..743156d 100644 --- a/lib/sup/maildir.rb +++ b/lib/sup/maildir.rb @@ -1,5 +1,6 @@ require 'uri' require 'set' +require 'inotify' module Redwood @@ -184,6 +185,45 @@ class Maildir < Source nil end + def continuous_poll poll_mutex + i = Inotify.new + watches = {} + @ctimes.each do |d,prev_ctime| + subdir = File.join @dir, d + wd = i.add_watch(subdir, Inotify::CREATE | Inotify::DELETE | Inotify::MOVE) + watches[wd] = d + end + i.each_event do |ev| + poll_mutex.synchronize do + @mutex.synchronize do + begin + ::Thread.current[@dir] = true + id = File.join watches[ev.wd], ev.name + # check if inotify is stale + # since we have @mutex, there is no race (except for + # an external program fucking us over) + next unless File.exists? File.join(@dir, id) + x = Enumerator.new(Index.instance, :each_source_info, self.id, "#{id}").to_a + if ev.mask & Inotify::CREATE or ev.mask & Inotify::MOVE_TO + next unless x.empty? + yield :add, + :info => id, + :labels => @labels + maildir_labels(id) + [:inbox], + :progress => 0 + elsif ev.mask & Inotify::DELETE or ev.mask & Inotify::MOVE_FROM + next unless !x.empty? + yield :delete, + :info => id, + :progress => 0 + end + ensure + ::Thread.current[@dir] = nil + end + end + end + end + end + def labels? id maildir_labels id end @@ -248,7 +288,16 @@ private end def maildir_move_file orig_path, new_source_id, flags - @mutex.synchronize do + if ::Thread.current[@dir] + _maildir_move_file orig_path, new_source_id, flags + else + @mutex.synchronize do + _maildir_move_file orig_path, new_source_id, flags + end + end + end + + def _maildir_move_file orig_path, new_source_id, flags new_base = (flags.include?("S")) ? "cur" : "new" md_base, md_ver, md_flags = maildir_data orig_path @@ -292,7 +341,6 @@ private end [new_source, new_loc] - end end end diff --git a/lib/sup/poll.rb b/lib/sup/poll.rb index dbd351f..51e0afa 100644 --- a/lib/sup/poll.rb +++ b/lib/sup/poll.rb @@ -94,11 +94,27 @@ EOS poll if @last_poll.nil? || (Time.now - @last_poll) >= @delay end end + # XXX dup dup + SourceManager.usual_sources.each do |source| + Redwood::reporting_thread("inotify poll for #{source}") do + source.continuous_poll @mutex do |sym, args| + poll_handler source, sym, args + end + end + end + SourceManager.unusual_sources.each do |source| + Redwood::reporting_thread("inotify poll for #{source}") do + source.continuous_poll @mutex do |sym, args| + poll_handler source, sym, args + end + end + end end def stop @thread.kill if @thread @thread = nil + # handle inotify polls end def do_poll @@ -172,7 +188,16 @@ EOS ## from the index after being yielded. def poll_from source, opts={} begin - source.poll do |sym, args| + source.poll do |sym,args| + poll_handler source, sym, args + end + source.go_idle + rescue SourceError => e + warn "problem getting messages from #{source}: #{e.message}" + end + end + + def poll_handler source, sym, args case sym when :add m = Message.build_from_source source, args[:info] @@ -224,12 +249,6 @@ EOS UpdateManager.relay self, :updated, m end end - end - - source.go_idle - rescue SourceError => e - warn "problem getting messages from #{source}: #{e.message}" - end end def handle_idle_update sender, idle_since; @should_clear_running_totals = false; end diff --git a/lib/sup/source.rb b/lib/sup/source.rb index 06b6e6b..073a10a 100644 --- a/lib/sup/source.rb +++ b/lib/sup/source.rb @@ -102,6 +102,10 @@ class Source unimplemented end + ## Like poll, but never returns (it is continuous, and uses something + ## like inotify. Will always be run in another thread.) + def continuous_poll poll_mutex; [] end + def valid? info true end -- 1.7.11.3 _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-03 4:59 ` [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) Edward Z. Yang @ 2012-09-03 5:00 ` Edward Z. Yang 2012-09-03 16:02 ` Alvaro Herrera 0 siblings, 1 reply; 15+ messages in thread From: Edward Z. Yang @ 2012-09-03 5:00 UTC (permalink / raw) To: sup-devel, alvherre The locking is a downright crime (where's the STM when you need it), and it's still racy, but it should work OK. Excerpts from Edward Z. Yang's message of Mon Sep 03 00:59:31 -0400 2012: > From: "Edward Z. Yang" <ezyang@mit.edu> > > Signed-off-by: Edward Z. Yang <ezyang@mit.edu> > --- > lib/sup/maildir.rb | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++-- > lib/sup/poll.rb | 33 ++++++++++++++++++++++++++------- > lib/sup/source.rb | 4 ++++ > 3 files changed, 80 insertions(+), 9 deletions(-) > > diff --git a/lib/sup/maildir.rb b/lib/sup/maildir.rb > index 2a91f05..743156d 100644 > --- a/lib/sup/maildir.rb > +++ b/lib/sup/maildir.rb > @@ -1,5 +1,6 @@ > require 'uri' > require 'set' > +require 'inotify' > > module Redwood > > @@ -184,6 +185,45 @@ class Maildir < Source > nil > end > > + def continuous_poll poll_mutex > + i = Inotify.new > + watches = {} > + @ctimes.each do |d,prev_ctime| > + subdir = File.join @dir, d > + wd = i.add_watch(subdir, Inotify::CREATE | Inotify::DELETE | Inotify::MOVE) > + watches[wd] = d > + end > + i.each_event do |ev| > + poll_mutex.synchronize do > + @mutex.synchronize do > + begin > + ::Thread.current[@dir] = true > + id = File.join watches[ev.wd], ev.name > + # check if inotify is stale > + # since we have @mutex, there is no race (except for > + # an external program fucking us over) > + next unless File.exists? File.join(@dir, id) > + x = Enumerator.new(Index.instance, :each_source_info, self.id, "#{id}").to_a > + if ev.mask & Inotify::CREATE or ev.mask & Inotify::MOVE_TO > + next unless x.empty? > + yield :add, > + :info => id, > + :labels => @labels + maildir_labels(id) + [:inbox], > + :progress => 0 > + elsif ev.mask & Inotify::DELETE or ev.mask & Inotify::MOVE_FROM > + next unless !x.empty? > + yield :delete, > + :info => id, > + :progress => 0 > + end > + ensure > + ::Thread.current[@dir] = nil > + end > + end > + end > + end > + end > + > def labels? id > maildir_labels id > end > @@ -248,7 +288,16 @@ private > end > > def maildir_move_file orig_path, new_source_id, flags > - @mutex.synchronize do > + if ::Thread.current[@dir] > + _maildir_move_file orig_path, new_source_id, flags > + else > + @mutex.synchronize do > + _maildir_move_file orig_path, new_source_id, flags > + end > + end > + end > + > + def _maildir_move_file orig_path, new_source_id, flags > new_base = (flags.include?("S")) ? "cur" : "new" > md_base, md_ver, md_flags = maildir_data orig_path > > @@ -292,7 +341,6 @@ private > end > > [new_source, new_loc] > - end > end > end > > diff --git a/lib/sup/poll.rb b/lib/sup/poll.rb > index dbd351f..51e0afa 100644 > --- a/lib/sup/poll.rb > +++ b/lib/sup/poll.rb > @@ -94,11 +94,27 @@ EOS > poll if @last_poll.nil? || (Time.now - @last_poll) >= @delay > end > end > + # XXX dup dup > + SourceManager.usual_sources.each do |source| > + Redwood::reporting_thread("inotify poll for #{source}") do > + source.continuous_poll @mutex do |sym, args| > + poll_handler source, sym, args > + end > + end > + end > + SourceManager.unusual_sources.each do |source| > + Redwood::reporting_thread("inotify poll for #{source}") do > + source.continuous_poll @mutex do |sym, args| > + poll_handler source, sym, args > + end > + end > + end > end > > def stop > @thread.kill if @thread > @thread = nil > + # handle inotify polls > end > > def do_poll > @@ -172,7 +188,16 @@ EOS > ## from the index after being yielded. > def poll_from source, opts={} > begin > - source.poll do |sym, args| > + source.poll do |sym,args| > + poll_handler source, sym, args > + end > + source.go_idle > + rescue SourceError => e > + warn "problem getting messages from #{source}: #{e.message}" > + end > + end > + > + def poll_handler source, sym, args > case sym > when :add > m = Message.build_from_source source, args[:info] > @@ -224,12 +249,6 @@ EOS > UpdateManager.relay self, :updated, m > end > end > - end > - > - source.go_idle > - rescue SourceError => e > - warn "problem getting messages from #{source}: #{e.message}" > - end > end > > def handle_idle_update sender, idle_since; @should_clear_running_totals = false; end > diff --git a/lib/sup/source.rb b/lib/sup/source.rb > index 06b6e6b..073a10a 100644 > --- a/lib/sup/source.rb > +++ b/lib/sup/source.rb > @@ -102,6 +102,10 @@ class Source > unimplemented > end > > + ## Like poll, but never returns (it is continuous, and uses something > + ## like inotify. Will always be run in another thread.) > + def continuous_poll poll_mutex; [] end > + > def valid? info > true > end _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-03 5:00 ` Edward Z. Yang @ 2012-09-03 16:02 ` Alvaro Herrera 2012-09-03 16:07 ` Alvaro Herrera 0 siblings, 1 reply; 15+ messages in thread From: Alvaro Herrera @ 2012-09-03 16:02 UTC (permalink / raw) To: Edward Z. Yang; +Cc: sup-devel Excerpts from Edward Z. Yang's message of lun sep 03 02:00:59 -0300 2012: > The locking is a downright crime (where's the STM when you need it), > and it's still racy, but it should work OK. Hm. I tried this but ran into trouble: I currently run branch "next", and your patch doesn't apply there; so I tried your ~ezyang fork and branch maildir-sync there, but I find that when in that branch (with or without this patch), Sup seems to eat 100% of a CPU core doing clock_gettime() and select() continuously. Not sure what's happening. I assume you don't see that behavior. -- Álvaro Herrera <alvherre@alvh.no-ip.org> _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-03 16:02 ` Alvaro Herrera @ 2012-09-03 16:07 ` Alvaro Herrera 2012-09-03 16:10 ` ezyang 0 siblings, 1 reply; 15+ messages in thread From: Alvaro Herrera @ 2012-09-03 16:07 UTC (permalink / raw) To: Edward Z. Yang; +Cc: sup-devel Excerpts from Alvaro Herrera's message of lun sep 03 13:02:26 -0300 2012: > Excerpts from Edward Z. Yang's message of lun sep 03 02:00:59 -0300 2012: > > The locking is a downright crime (where's the STM when you need it), > > and it's still racy, but it should work OK. > > Hm. I tried this but ran into trouble: I currently run branch "next", > and your patch doesn't apply there; so I tried your ~ezyang fork and > branch maildir-sync there, but I find that when in that branch (with or > without this patch), Sup seems to eat 100% of a CPU core doing > clock_gettime() and select() continuously. Not sure what's happening. > I assume you don't see that behavior. Oh, I see what's going on: it's deleting all my deleted email! :-) Nevermind. I'll just wait for it to finish before trying out your patch. -- Álvaro Herrera <alvherre@alvh.no-ip.org> _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-03 16:07 ` Alvaro Herrera @ 2012-09-03 16:10 ` ezyang 2012-09-03 18:09 ` Edward Z. Yang 0 siblings, 1 reply; 15+ messages in thread From: ezyang @ 2012-09-03 16:10 UTC (permalink / raw) To: Alvaro Herrera; +Cc: sup-devel Quoting Alvaro Herrera <alvherre@alvh.no-ip.org>: > Excerpts from Alvaro Herrera's message of lun sep 03 13:02:26 -0300 2012: >> Excerpts from Edward Z. Yang's message of lun sep 03 02:00:59 -0300 2012: >> > The locking is a downright crime (where's the STM when you need it), >> > and it's still racy, but it should work OK. >> >> Hm. I tried this but ran into trouble: I currently run branch "next", >> and your patch doesn't apply there; so I tried your ~ezyang fork and >> branch maildir-sync there, but I find that when in that branch (with or >> without this patch), Sup seems to eat 100% of a CPU core doing >> clock_gettime() and select() continuously. Not sure what's happening. >> I assume you don't see that behavior. > > Oh, I see what's going on: it's deleting all my deleted email! :-) > Nevermind. I'll just wait for it to finish before trying out your patch. There is a bug in the patch, where it notices changes that Sup makes from inotify, and as a result undoes any flag changes you make. So there needs to be a fix here but I'm not sure what it is yet. Edward _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-03 16:10 ` ezyang @ 2012-09-03 18:09 ` Edward Z. Yang 2012-09-03 18:29 ` Edward Z. Yang 2012-09-03 23:31 ` Edward Z. Yang 0 siblings, 2 replies; 15+ messages in thread From: Edward Z. Yang @ 2012-09-03 18:09 UTC (permalink / raw) To: Alvaro Herrera, sup-devel OK, cracked a fix; you need this extra patch: commit f9ea07f3c4982ab46d8171fdba8eabc3af00c840 Author: Edward Z. Yang <ezyang@mit.edu> Date: Mon Sep 3 14:09:34 2012 -0400 sync_back after writing to index, not before. Signed-off-by: Edward Z. Yang <ezyang@mit.edu> diff --git a/lib/sup/index.rb b/lib/sup/index.rb index 13798d6..4b533a7 100644 --- a/lib/sup/index.rb +++ b/lib/sup/index.rb @@ -657,10 +657,6 @@ EOS end def sync_message m, overwrite - ## TODO: we should not save the message if the sync_back failed - ## since it would overwrite the location field - m.sync_back - doc = synchronize { find_doc(m.id) } existed = doc != nil doc ||= Xapian::Document.new @@ -703,6 +699,10 @@ EOS @xapian.replace_document docid, doc end + # sync_back must be after label update, so that inotify gets + # fresh data from the index + m.sync_back + m.labels.each { |l| LabelManager << l } true end _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-03 18:09 ` Edward Z. Yang @ 2012-09-03 18:29 ` Edward Z. Yang 2012-09-03 18:42 ` Alvaro Herrera 2012-09-03 23:31 ` Edward Z. Yang 1 sibling, 1 reply; 15+ messages in thread From: Edward Z. Yang @ 2012-09-03 18:29 UTC (permalink / raw) To: Alvaro Herrera, sup-devel I think I may have seen Sup peg IO R/W in the way you saw, but when I restarted Sup with debugging, it went away. Do let me know if you see it again; I think there's an infinite loop somewhere. Edward Excerpts from Edward Z. Yang's message of Mon Sep 03 14:09:57 -0400 2012: > OK, cracked a fix; you need this extra patch: > > commit f9ea07f3c4982ab46d8171fdba8eabc3af00c840 > Author: Edward Z. Yang <ezyang@mit.edu> > Date: Mon Sep 3 14:09:34 2012 -0400 > > sync_back after writing to index, not before. > > Signed-off-by: Edward Z. Yang <ezyang@mit.edu> > > diff --git a/lib/sup/index.rb b/lib/sup/index.rb > index 13798d6..4b533a7 100644 > --- a/lib/sup/index.rb > +++ b/lib/sup/index.rb > @@ -657,10 +657,6 @@ EOS > end > > def sync_message m, overwrite > - ## TODO: we should not save the message if the sync_back failed > - ## since it would overwrite the location field > - m.sync_back > - > doc = synchronize { find_doc(m.id) } > existed = doc != nil > doc ||= Xapian::Document.new > @@ -703,6 +699,10 @@ EOS > @xapian.replace_document docid, doc > end > > + # sync_back must be after label update, so that inotify gets > + # fresh data from the index > + m.sync_back > + > m.labels.each { |l| LabelManager << l } > true > end _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-03 18:29 ` Edward Z. Yang @ 2012-09-03 18:42 ` Alvaro Herrera 2012-09-03 18:49 ` Edward Z. Yang 2012-09-03 19:27 ` Edward Z. Yang 0 siblings, 2 replies; 15+ messages in thread From: Alvaro Herrera @ 2012-09-03 18:42 UTC (permalink / raw) To: Edward Z. Yang; +Cc: sup-devel Excerpts from Edward Z. Yang's message of lun sep 03 15:29:43 -0300 2012: > I think I may have seen Sup peg IO R/W in the way you saw, but when I > restarted Sup with debugging, it went away. Do let me know if you see it > again; I think there's an infinite loop somewhere. Sure. For the record, I let it run for a while and after that the poll log had a couple thousand lines saying "Deleting <some-msg-id> <its subject>". After that, so far it behaves normally. I don't know what prompted those particular messages to be deleted; I certainly have lots of messages "is:deleted" yet. Another curious thing with your branch is that the poll log gets these four lines each time a poll takes place: Message at 0 has changed its source location. Updating labels from draft,personal => draft,personal Message at 1 has changed its source location. Updating labels from draft,inbox,personal => draft,inbox,personal Message at 2 has changed its source location. Updating labels from draft,personal => draft,personal Message at 3 has changed its source location. Updating labels from draft,personal => draft,personal -- Álvaro Herrera <alvherre@alvh.no-ip.org> _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-03 18:42 ` Alvaro Herrera @ 2012-09-03 18:49 ` Edward Z. Yang 2012-09-03 19:27 ` Edward Z. Yang 1 sibling, 0 replies; 15+ messages in thread From: Edward Z. Yang @ 2012-09-03 18:49 UTC (permalink / raw) To: Alvaro Herrera; +Cc: sup-devel Excerpts from Alvaro Herrera's message of Mon Sep 03 14:42:55 -0400 2012: > Message at 0 has changed its source location. Updating labels from draft,personal => draft,personal > Message at 1 has changed its source location. Updating labels from draft,inbox,personal => draft,inbox,personal > Message at 2 has changed its source location. Updating labels from draft,personal => draft,personal > Message at 3 has changed its source location. Updating labels from draft,personal => draft,personal That's odd, I guess Enumerator.new(Index.instance, :each_source_info, self.id).to_a returning a null set so we always emit :add? If you could add a debug call on that enumerator and get me the contents of old_ids and new_ids whereabouts draft.rb line 70 that would be great. Edward _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-03 18:42 ` Alvaro Herrera 2012-09-03 18:49 ` Edward Z. Yang @ 2012-09-03 19:27 ` Edward Z. Yang 1 sibling, 0 replies; 15+ messages in thread From: Edward Z. Yang @ 2012-09-03 19:27 UTC (permalink / raw) To: Alvaro Herrera; +Cc: sup-devel Excerpts from Alvaro Herrera's message of Mon Sep 03 14:42:55 -0400 2012: > For the record, I let it run for a while and after that the poll log had > a couple thousand lines saying "Deleting <some-msg-id> <its subject>". > After that, so far it behaves normally. I don't know what prompted > those particular messages to be deleted; I certainly have lots of > messages "is:deleted" yet. Confirmed it's Xapian taking it's sweet time to delete the mail. You might do better with a higher XAPIAN_FLUSH_THRESHOLD value in bin/sup, but the process seems fairly CPU bound. Edward _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-03 18:09 ` Edward Z. Yang 2012-09-03 18:29 ` Edward Z. Yang @ 2012-09-03 23:31 ` Edward Z. Yang 2012-09-04 16:06 ` Edward Z. Yang 1 sibling, 1 reply; 15+ messages in thread From: Edward Z. Yang @ 2012-09-03 23:31 UTC (permalink / raw) To: sup-devel Extra note: the ruby-inotify plugin appears to be braindead on at least Ruby 1.8.7 for Ubuntu Precise, and will spin-loop. If sup is chomping 15% CPU on idle, try this patch on ruby-inotify Reported here: https://github.com/ruby-building-blocks/ruby-inotify/issues/8 diff --git a/lib/inotify/inotify_native.rb b/lib/inotify/inotify_native.rb index 66db2b4..27d1043 100644 --- a/lib/inotify/inotify_native.rb +++ b/lib/inotify/inotify_native.rb @@ -115,7 +115,7 @@ require 'ffi' # each_event() provides an easy way to loop over all events as they occur def each_event loop do - ready = IO.select([@io], nil, nil, nil) + ready = IO.select([@io]) event = self.read_event yield event end Excerpts from Edward Z. Yang's message of Mon Sep 03 14:09:57 -0400 2012: > OK, cracked a fix; you need this extra patch: > > commit f9ea07f3c4982ab46d8171fdba8eabc3af00c840 > Author: Edward Z. Yang <ezyang@mit.edu> > Date: Mon Sep 3 14:09:34 2012 -0400 > > sync_back after writing to index, not before. > > Signed-off-by: Edward Z. Yang <ezyang@mit.edu> > > diff --git a/lib/sup/index.rb b/lib/sup/index.rb > index 13798d6..4b533a7 100644 > --- a/lib/sup/index.rb > +++ b/lib/sup/index.rb > @@ -657,10 +657,6 @@ EOS > end > > def sync_message m, overwrite > - ## TODO: we should not save the message if the sync_back failed > - ## since it would overwrite the location field > - m.sync_back > - > doc = synchronize { find_doc(m.id) } > existed = doc != nil > doc ||= Xapian::Document.new > @@ -703,6 +699,10 @@ EOS > @xapian.replace_document docid, doc > end > > + # sync_back must be after label update, so that inotify gets > + # fresh data from the index > + m.sync_back > + > m.labels.each { |l| LabelManager << l } > true > end _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-03 23:31 ` Edward Z. Yang @ 2012-09-04 16:06 ` Edward Z. Yang 2012-09-04 18:09 ` Alvaro Herrera 0 siblings, 1 reply; 15+ messages in thread From: Edward Z. Yang @ 2012-09-04 16:06 UTC (permalink / raw) To: Alvaro Herrera; +Cc: sup-devel I've pushed a new revision of the branch: http://gitorious.org/~ezyang/sup/ezyang/commits/maildir-sync which solves some bad interactions with sync-back. Edward _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) 2012-09-04 16:06 ` Edward Z. Yang @ 2012-09-04 18:09 ` Alvaro Herrera 0 siblings, 0 replies; 15+ messages in thread From: Alvaro Herrera @ 2012-09-04 18:09 UTC (permalink / raw) To: Edward Z. Yang; +Cc: sup-devel Excerpts from Edward Z. Yang's message of mar sep 04 13:06:07 -0300 2012: > I've pushed a new revision of the branch: http://gitorious.org/~ezyang/sup/ezyang/commits/maildir-sync > which solves some bad interactions with sync-back. Okay, I'm running this now. Having the emails show up instantly works great ... hooray for continuous, permanent distraction ;-) I haven't debugged the other issue yet. I'll let you know. -- Álvaro Herrera <alvherre@alvh.no-ip.org> _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2012-09-04 18:10 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-08-21 15:17 [sup-devel] inotify support for Maildir mailboxes Edward Z. Yang 2012-08-21 16:00 ` Alvaro Herrera 2012-09-03 4:59 ` [sup-devel] [PATCH] Inotify support for Maildirs. (FIRST DRAFT) Edward Z. Yang 2012-09-03 5:00 ` Edward Z. Yang 2012-09-03 16:02 ` Alvaro Herrera 2012-09-03 16:07 ` Alvaro Herrera 2012-09-03 16:10 ` ezyang 2012-09-03 18:09 ` Edward Z. Yang 2012-09-03 18:29 ` Edward Z. Yang 2012-09-03 18:42 ` Alvaro Herrera 2012-09-03 18:49 ` Edward Z. Yang 2012-09-03 19:27 ` Edward Z. Yang 2012-09-03 23:31 ` Edward Z. Yang 2012-09-04 16:06 ` Edward Z. Yang 2012-09-04 18:09 ` Alvaro Herrera
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox