* [sup-devel] [PATCH 1/4] factor saving out of thread/message classes @ 2009-12-30 1:38 Rich Lane 2009-12-30 1:38 ` [sup-devel] [PATCH 2/4] async thread indexing Rich Lane 2009-12-31 20:09 ` [sup-devel] [PATCH 1/4] factor saving out of thread/message classes William Morgan 0 siblings, 2 replies; 5+ messages in thread From: Rich Lane @ 2009-12-30 1:38 UTC (permalink / raw) To: sup-devel --- lib/sup/index.rb | 7 +++++++ lib/sup/message.rb | 5 +---- lib/sup/modes/thread-index-mode.rb | 2 +- lib/sup/thread.rb | 2 +- 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/lib/sup/index.rb b/lib/sup/index.rb index ff03f19..5d8d714 100644 --- a/lib/sup/index.rb +++ b/lib/sup/index.rb @@ -172,6 +172,13 @@ class BaseIndex def parse_query s unimplemented end + + def save_thread t + t.each_dirty_message do |m| + update_message_state m + m.clear_dirty + end + end end index_name = ENV['SUP_INDEX'] || $config[:index] || DEFAULT_INDEX diff --git a/lib/sup/message.rb b/lib/sup/message.rb index f3ac874..76ce330 100644 --- a/lib/sup/message.rb +++ b/lib/sup/message.rb @@ -182,11 +182,8 @@ class Message ## don't tempt me. def sanitize_message_id mid; mid.gsub(/(\s|[^\000-\177])+/, "")[0..254] end - def save_state index - return unless @dirty - index.update_message_state self + def clear_dirty @dirty = false - true end def has_label? t; @labels.member? t; end diff --git a/lib/sup/modes/thread-index-mode.rb b/lib/sup/modes/thread-index-mode.rb index 82f258b..c4a7d38 100644 --- a/lib/sup/modes/thread-index-mode.rb +++ b/lib/sup/modes/thread-index-mode.rb @@ -477,7 +477,7 @@ EOS BufferManager.say("Saving threads...") do |say_id| dirty_threads.each_with_index do |t, i| BufferManager.say "Saving modified thread #{i + 1} of #{dirty_threads.length}...", say_id - t.save_state Index + Index.save_thread t end end end diff --git a/lib/sup/thread.rb b/lib/sup/thread.rb index 2300305..3fdf1f6 100644 --- a/lib/sup/thread.rb +++ b/lib/sup/thread.rb @@ -112,7 +112,7 @@ class Thread def set_labels l; each { |m, *o| m && m.labels = l }; end def has_label? t; any? { |m, *o| m && m.has_label?(t) }; end - def save_state index; each { |m, *o| m && m.save_state(index) }; end + def each_dirty_message; each { |m, *o| m && m.dirty? && yield(m) }; end def direct_participants map { |m, *o| [m.from] + m.to if m }.flatten.compact.uniq -- 1.6.3.3 _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 5+ messages in thread
* [sup-devel] [PATCH 2/4] async thread indexing 2009-12-30 1:38 [sup-devel] [PATCH 1/4] factor saving out of thread/message classes Rich Lane @ 2009-12-30 1:38 ` Rich Lane 2009-12-30 1:38 ` [sup-devel] [PATCH 3/4] immediate " Rich Lane 2009-12-31 20:09 ` [sup-devel] [PATCH 1/4] factor saving out of thread/message classes William Morgan 1 sibling, 1 reply; 5+ messages in thread From: Rich Lane @ 2009-12-30 1:38 UTC (permalink / raw) To: sup-devel --- bin/sup | 2 ++ lib/sup/index.rb | 26 +++++++++++++++++++++++++- lib/sup/modes/thread-index-mode.rb | 2 +- 3 files changed, 28 insertions(+), 2 deletions(-) diff --git a/bin/sup b/bin/sup index 471e833..f9ed7d5 100755 --- a/bin/sup +++ b/bin/sup @@ -141,6 +141,7 @@ Index.lock_interactively or exit begin Redwood::start Index.load + Index.start_sync_worker unless $opts[:no_threads] $die = false trap("TERM") { |x| $die = true } @@ -329,6 +330,7 @@ ensure HookManager.run "shutdown" + Index.stop_sync_worker Redwood::finish stop_cursing Redwood::Logger.remove_all_sinks! diff --git a/lib/sup/index.rb b/lib/sup/index.rb index 5d8d714..1131ec7 100644 --- a/lib/sup/index.rb +++ b/lib/sup/index.rb @@ -28,6 +28,8 @@ class BaseIndex def initialize dir=BASE_DIR @dir = dir @lock = Lockfile.new lockfile, :retries => 0, :max_age => nil + @sync_worker = nil + @sync_queue = Queue.new end def lockfile; File.join @dir, "lock" end @@ -175,10 +177,32 @@ class BaseIndex def save_thread t t.each_dirty_message do |m| - update_message_state m + if @sync_worker + @sync_queue << m + else + update_message_state m + end m.clear_dirty end end + + def start_sync_worker + @sync_worker = Redwood::reporting_thread('index sync') { run_sync_worker } + end + + def stop_sync_worker + return unless worker = @sync_worker + @sync_worker = nil + @sync_queue << :die + worker.join + end + + def run_sync_worker + while m = @sync_queue.deq + return if m == :die + update_message_state m + end + end end index_name = ENV['SUP_INDEX'] || $config[:index] || DEFAULT_INDEX diff --git a/lib/sup/modes/thread-index-mode.rb b/lib/sup/modes/thread-index-mode.rb index c4a7d38..f6ea27c 100644 --- a/lib/sup/modes/thread-index-mode.rb +++ b/lib/sup/modes/thread-index-mode.rb @@ -477,7 +477,7 @@ EOS BufferManager.say("Saving threads...") do |say_id| dirty_threads.each_with_index do |t, i| BufferManager.say "Saving modified thread #{i + 1} of #{dirty_threads.length}...", say_id - Index.save_thread t + Index.save_thread_async t end end end -- 1.6.3.3 _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 5+ messages in thread
* [sup-devel] [PATCH 3/4] immediate thread indexing 2009-12-30 1:38 ` [sup-devel] [PATCH 2/4] async thread indexing Rich Lane @ 2009-12-30 1:38 ` Rich Lane 2009-12-30 1:38 ` [sup-devel] [PATCH 4/4] force the index sync thread to give up the cpu Rich Lane 0 siblings, 1 reply; 5+ messages in thread From: Rich Lane @ 2009-12-30 1:38 UTC (permalink / raw) To: sup-devel --- lib/sup/modes/inbox-mode.rb | 4 +++ lib/sup/modes/thread-index-mode.rb | 40 +++++++++++++---------------------- lib/sup/modes/thread-view-mode.rb | 7 ++++++ 3 files changed, 26 insertions(+), 25 deletions(-) diff --git a/lib/sup/modes/inbox-mode.rb b/lib/sup/modes/inbox-mode.rb index ba095da..9220925 100644 --- a/lib/sup/modes/inbox-mode.rb +++ b/lib/sup/modes/inbox-mode.rb @@ -34,6 +34,7 @@ class InboxMode < ThreadIndexMode cursor_thread.remove_label :inbox hide_thread cursor_thread regen_text + Index.save_thread thread end def multi_archive threads @@ -50,6 +51,7 @@ class InboxMode < ThreadIndexMode hide_thread t end regen_text + threads.each { |t| Index.save_thread t } end def read_and_archive @@ -66,6 +68,7 @@ class InboxMode < ThreadIndexMode cursor_thread.remove_label :inbox hide_thread cursor_thread regen_text + Index.save_thread thread end def multi_read_and_archive threads @@ -86,6 +89,7 @@ class InboxMode < ThreadIndexMode regen_text end + threads.each { |t| Index.save_thread t } end def handle_unarchived_update sender, m diff --git a/lib/sup/modes/thread-index-mode.rb b/lib/sup/modes/thread-index-mode.rb index f6ea27c..5393189 100644 --- a/lib/sup/modes/thread-index-mode.rb +++ b/lib/sup/modes/thread-index-mode.rb @@ -37,7 +37,6 @@ EOS k.add :toggle_spam, "Mark/unmark thread as spam", 'S' k.add :toggle_deleted, "Delete/undelete thread", 'd' k.add :kill, "Kill thread (never to be seen in inbox again)", '&' - k.add :save, "Save changes now", '$' k.add :jump_to_next_new, "Jump to next new thread", :tab k.add :reply, "Reply to latest message in a thread", 'r' k.add :reply_all, "Reply to all participants of the latest message in a thread", 'G' @@ -269,12 +268,14 @@ EOS UndoManager.register "toggling thread starred status", undo update_text_for_line curpos cursor_down + Index.save_thread t end def multi_toggle_starred threads UndoManager.register "toggling #{threads.size.pluralize 'thread'} starred status", threads.map { |t| actually_toggle_starred t } regen_text + threads.each { |t| Index.save_thread t } end ## returns an undo lambda @@ -352,12 +353,14 @@ EOS undo = actually_toggle_archived t UndoManager.register "deleting/undeleting thread #{t.first.id}", undo, lambda { update_text_for_line curpos } update_text_for_line curpos + Index.save_thread t end def multi_toggle_archived threads undos = threads.map { |t| actually_toggle_archived t } UndoManager.register "deleting/undeleting #{threads.size.pluralize 'thread'}", undos, lambda { regen_text } regen_text + threads.each { |t| Index.save_thread t } end def toggle_new @@ -365,11 +368,13 @@ EOS t.toggle_label :unread update_text_for_line curpos cursor_down + Index.save_thread t end def multi_toggle_new threads threads.each { |t| t.toggle_label :unread } regen_text + threads.each { |t| Index.save_thread t } end def multi_toggle_tagged threads @@ -385,6 +390,7 @@ EOS def multi_join_threads threads @ts.join_threads threads or return + threads.each { |t| Index.save_thread t } @tags.drop_all_tags # otherwise we have tag pointers to invalid threads! update end @@ -421,6 +427,7 @@ EOS UndoManager.register "marking/unmarking #{threads.size.pluralize 'thread'} as spam", undos, lambda { regen_text } regen_text + threads.each { |t| Index.save_thread t } end def toggle_deleted @@ -434,6 +441,7 @@ EOS UndoManager.register "deleting/undeleting #{threads.size.pluralize 'thread'}", undos, lambda { regen_text } regen_text + threads.each { |t| Index.save_thread t } end def kill @@ -458,29 +466,7 @@ EOS regen_text BufferManager.flash "#{threads.size.pluralize 'thread'} killed." - end - - def save background=true - if background - Redwood::reporting_thread("saving thread") { actually_save } - else - actually_save - end - end - - def actually_save - @save_thread_mutex.synchronize do - BufferManager.say("Saving contacts...") { ContactManager.instance.save } - dirty_threads = @mutex.synchronize { (@threads + @hidden_threads.keys).select { |t| t.dirty? } } - next if dirty_threads.empty? - - BufferManager.say("Saving threads...") do |say_id| - dirty_threads.each_with_index do |t, i| - BufferManager.say "Saving modified thread #{i + 1} of #{dirty_threads.length}...", say_id - Index.save_thread_async t - end - end - end + threads.each { |t| Index.save_thread t } end def cleanup @@ -492,7 +478,8 @@ EOS sleep 0.1 # TODO: necessary? BufferManager.erase_flash end - save false + dirty_threads = @mutex.synchronize { (@threads + @hidden_threads.keys).select { |t| t.dirty? } } + fail "dirty threads remain" unless dirty_threads.empty? super end @@ -546,6 +533,7 @@ EOS end UpdateManager.relay self, :labeled, thread.first + Index.save_thread thread end def multi_edit_labels threads @@ -582,6 +570,8 @@ EOS end regen_text end + + threads.each { |t| Index.save_thread t } end def reply type_arg=nil diff --git a/lib/sup/modes/thread-view-mode.rb b/lib/sup/modes/thread-view-mode.rb index 55d81b9..99abb04 100644 --- a/lib/sup/modes/thread-view-mode.rb +++ b/lib/sup/modes/thread-view-mode.rb @@ -131,6 +131,7 @@ EOS @layout[earliest].state = :detailed if earliest.has_label?(:unread) || @thread.size == 1 @thread.remove_label :unread + Index.save_thread @thread regen_text end @@ -258,6 +259,7 @@ EOS new_labels.each { |l| LabelManager << l } update UpdateManager.relay self, :labeled, @thread.first + Index.save_thread @thread UndoManager.register "labeling thread" do @thread.labels = old_labels UpdateManager.relay self, :labeled, @thread.first @@ -284,6 +286,7 @@ EOS ## star to the display update UpdateManager.relay self, :single_message_labeled, m + Index.save_thread @thread end ## called when someone presses enter when the cursor is highlighting @@ -478,6 +481,7 @@ EOS dispatch op do @thread.remove_label :inbox UpdateManager.relay self, :archived, @thread.first + Index.save_thread @thread UndoManager.register "archiving 1 thread" do @thread.apply_label :inbox UpdateManager.relay self, :unarchived, @thread.first @@ -489,6 +493,7 @@ EOS dispatch op do @thread.apply_label :spam UpdateManager.relay self, :spammed, @thread.first + Index.save_thread @thread UndoManager.register "marking 1 thread as spam" do @thread.remove_label :spam UpdateManager.relay self, :unspammed, @thread.first @@ -500,6 +505,7 @@ EOS dispatch op do @thread.apply_label :deleted UpdateManager.relay self, :deleted, @thread.first + Index.save_thread @thread UndoManager.register "deleting 1 thread" do @thread.remove_label :deleted UpdateManager.relay self, :undeleted, @thread.first @@ -511,6 +517,7 @@ EOS dispatch op do @thread.apply_label :unread UpdateManager.relay self, :unread, @thread.first + Index.save_thread @thread end end -- 1.6.3.3 _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 5+ messages in thread
* [sup-devel] [PATCH 4/4] force the index sync thread to give up the cpu 2009-12-30 1:38 ` [sup-devel] [PATCH 3/4] immediate " Rich Lane @ 2009-12-30 1:38 ` Rich Lane 0 siblings, 0 replies; 5+ messages in thread From: Rich Lane @ 2009-12-30 1:38 UTC (permalink / raw) To: sup-devel --- lib/sup/index.rb | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/lib/sup/index.rb b/lib/sup/index.rb index 1131ec7..4f491d6 100644 --- a/lib/sup/index.rb +++ b/lib/sup/index.rb @@ -201,6 +201,8 @@ class BaseIndex while m = @sync_queue.deq return if m == :die update_message_state m + # Necessary to keep Xapian calls from lagging the UI too much. + sleep 0.03 end end end -- 1.6.3.3 _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-devel] [PATCH 1/4] factor saving out of thread/message classes 2009-12-30 1:38 [sup-devel] [PATCH 1/4] factor saving out of thread/message classes Rich Lane 2009-12-30 1:38 ` [sup-devel] [PATCH 2/4] async thread indexing Rich Lane @ 2009-12-31 20:09 ` William Morgan 1 sibling, 0 replies; 5+ messages in thread From: William Morgan @ 2009-12-31 20:09 UTC (permalink / raw) To: sup-devel Branch insta-save, merged into next. I can't believe it works! So cool. -- William <wmorgan-sup@masanjin.net> _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2009-12-31 20:09 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2009-12-30 1:38 [sup-devel] [PATCH 1/4] factor saving out of thread/message classes Rich Lane 2009-12-30 1:38 ` [sup-devel] [PATCH 2/4] async thread indexing Rich Lane 2009-12-30 1:38 ` [sup-devel] [PATCH 3/4] immediate " Rich Lane 2009-12-30 1:38 ` [sup-devel] [PATCH 4/4] force the index sync thread to give up the cpu Rich Lane 2009-12-31 20:09 ` [sup-devel] [PATCH 1/4] factor saving out of thread/message classes William Morgan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox