* [sup-talk] [PATCH 1/4] factor saving out of thread/message classes
@ 2009-11-05 5:45 Rich Lane
2009-11-05 5:45 ` [sup-talk] [PATCH 2/4] async thread indexing Rich Lane
0 siblings, 1 reply; 6+ messages in thread
From: Rich Lane @ 2009-11-05 5:45 UTC (permalink / raw)
To: sup-talk
---
lib/sup/index.rb | 7 +++++++
lib/sup/message.rb | 5 +----
lib/sup/modes/thread-index-mode.rb | 2 +-
lib/sup/thread.rb | 2 +-
4 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/lib/sup/index.rb b/lib/sup/index.rb
index ff03f19..5d8d714 100644
--- a/lib/sup/index.rb
+++ b/lib/sup/index.rb
@@ -172,6 +172,13 @@ class BaseIndex
def parse_query s
unimplemented
end
+
+ def save_thread t
+ t.each_dirty_message do |m|
+ update_message_state m
+ m.clear_dirty
+ end
+ end
end
index_name = ENV['SUP_INDEX'] || $config[:index] || DEFAULT_INDEX
diff --git a/lib/sup/message.rb b/lib/sup/message.rb
index a147c42..95ae050 100644
--- a/lib/sup/message.rb
+++ b/lib/sup/message.rb
@@ -182,11 +182,8 @@ class Message
## don't tempt me.
def sanitize_message_id mid; mid.gsub(/(\s|[^\000-\177])+/, "")[0..254] end
- def save_state index
- return unless @dirty
- index.update_message_state self
+ def clear_dirty
@dirty = false
- true
end
def has_label? t; @labels.member? t; end
diff --git a/lib/sup/modes/thread-index-mode.rb b/lib/sup/modes/thread-index-mode.rb
index 12a76f9..370903a 100644
--- a/lib/sup/modes/thread-index-mode.rb
+++ b/lib/sup/modes/thread-index-mode.rb
@@ -477,7 +477,7 @@ EOS
BufferManager.say("Saving threads...") do |say_id|
dirty_threads.each_with_index do |t, i|
BufferManager.say "Saving modified thread #{i + 1} of #{dirty_threads.length}...", say_id
- t.save_state Index
+ Index.save_thread t
end
end
end
diff --git a/lib/sup/thread.rb b/lib/sup/thread.rb
index 2300305..3fdf1f6 100644
--- a/lib/sup/thread.rb
+++ b/lib/sup/thread.rb
@@ -112,7 +112,7 @@ class Thread
def set_labels l; each { |m, *o| m && m.labels = l }; end
def has_label? t; any? { |m, *o| m && m.has_label?(t) }; end
- def save_state index; each { |m, *o| m && m.save_state(index) }; end
+ def each_dirty_message; each { |m, *o| m && m.dirty? && yield(m) }; end
def direct_participants
map { |m, *o| [m.from] + m.to if m }.flatten.compact.uniq
--
1.6.4.2
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 6+ messages in thread
* [sup-talk] [PATCH 2/4] async thread indexing
2009-11-05 5:45 [sup-talk] [PATCH 1/4] factor saving out of thread/message classes Rich Lane
@ 2009-11-05 5:45 ` Rich Lane
2009-11-05 5:45 ` [sup-talk] [PATCH 3/4] immediate " Rich Lane
2009-11-09 6:02 ` [sup-talk] [PATCH 2/4] async thread indexing Eugene Marinelli
0 siblings, 2 replies; 6+ messages in thread
From: Rich Lane @ 2009-11-05 5:45 UTC (permalink / raw)
To: sup-talk
---
bin/sup | 2 ++
lib/sup/index.rb | 26 +++++++++++++++++++++++++-
lib/sup/modes/thread-index-mode.rb | 2 +-
3 files changed, 28 insertions(+), 2 deletions(-)
diff --git a/bin/sup b/bin/sup
index 78c396a..910ff12 100755
--- a/bin/sup
+++ b/bin/sup
@@ -142,6 +142,7 @@ Index.lock_interactively or exit
begin
Redwood::start
Index.load
+ Index.start_sync_worker unless $opts[:no_threads]
$die = false
trap("TERM") { |x| $die = true }
@@ -335,6 +336,7 @@ ensure
HookManager.run "shutdown"
+ Index.stop_sync_worker
Redwood::finish
stop_cursing
Redwood::Logger.remove_all_sinks!
diff --git a/lib/sup/index.rb b/lib/sup/index.rb
index 5d8d714..1131ec7 100644
--- a/lib/sup/index.rb
+++ b/lib/sup/index.rb
@@ -28,6 +28,8 @@ class BaseIndex
def initialize dir=BASE_DIR
@dir = dir
@lock = Lockfile.new lockfile, :retries => 0, :max_age => nil
+ @sync_worker = nil
+ @sync_queue = Queue.new
end
def lockfile; File.join @dir, "lock" end
@@ -175,10 +177,32 @@ class BaseIndex
def save_thread t
t.each_dirty_message do |m|
- update_message_state m
+ if @sync_worker
+ @sync_queue << m
+ else
+ update_message_state m
+ end
m.clear_dirty
end
end
+
+ def start_sync_worker
+ @sync_worker = Redwood::reporting_thread('index sync') { run_sync_worker }
+ end
+
+ def stop_sync_worker
+ return unless worker = @sync_worker
+ @sync_worker = nil
+ @sync_queue << :die
+ worker.join
+ end
+
+ def run_sync_worker
+ while m = @sync_queue.deq
+ return if m == :die
+ update_message_state m
+ end
+ end
end
index_name = ENV['SUP_INDEX'] || $config[:index] || DEFAULT_INDEX
diff --git a/lib/sup/modes/thread-index-mode.rb b/lib/sup/modes/thread-index-mode.rb
index 370903a..d2b1651 100644
--- a/lib/sup/modes/thread-index-mode.rb
+++ b/lib/sup/modes/thread-index-mode.rb
@@ -477,7 +477,7 @@ EOS
BufferManager.say("Saving threads...") do |say_id|
dirty_threads.each_with_index do |t, i|
BufferManager.say "Saving modified thread #{i + 1} of #{dirty_threads.length}...", say_id
- Index.save_thread t
+ Index.save_thread_async t
end
end
end
--
1.6.4.2
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 6+ messages in thread
* [sup-talk] [PATCH 3/4] immediate thread indexing
2009-11-05 5:45 ` [sup-talk] [PATCH 2/4] async thread indexing Rich Lane
@ 2009-11-05 5:45 ` Rich Lane
2009-11-05 5:45 ` [sup-talk] [PATCH 4/4] force the index sync thread to give up the cpu Rich Lane
2009-11-09 6:02 ` [sup-talk] [PATCH 2/4] async thread indexing Eugene Marinelli
1 sibling, 1 reply; 6+ messages in thread
From: Rich Lane @ 2009-11-05 5:45 UTC (permalink / raw)
To: sup-talk
---
lib/sup/modes/inbox-mode.rb | 4 +++
lib/sup/modes/thread-index-mode.rb | 40 +++++++++++++----------------------
lib/sup/modes/thread-view-mode.rb | 7 ++++++
3 files changed, 26 insertions(+), 25 deletions(-)
diff --git a/lib/sup/modes/inbox-mode.rb b/lib/sup/modes/inbox-mode.rb
index 6c2244a..ad3a864 100644
--- a/lib/sup/modes/inbox-mode.rb
+++ b/lib/sup/modes/inbox-mode.rb
@@ -42,6 +42,7 @@ class InboxMode < ThreadIndexMode
cursor_thread.remove_label :inbox
hide_thread cursor_thread
regen_text
+ Index.save_thread thread
end
def multi_archive threads
@@ -58,6 +59,7 @@ class InboxMode < ThreadIndexMode
hide_thread t
end
regen_text
+ threads.each { |t| Index.save_thread t }
end
def read_and_archive
@@ -74,6 +76,7 @@ class InboxMode < ThreadIndexMode
cursor_thread.remove_label :inbox
hide_thread cursor_thread
regen_text
+ Index.save_thread thread
end
def multi_read_and_archive threads
@@ -94,6 +97,7 @@ class InboxMode < ThreadIndexMode
regen_text
end
+ threads.each { |t| Index.save_thread t }
end
def handle_unarchived_update sender, m
diff --git a/lib/sup/modes/thread-index-mode.rb b/lib/sup/modes/thread-index-mode.rb
index d2b1651..bf3f646 100644
--- a/lib/sup/modes/thread-index-mode.rb
+++ b/lib/sup/modes/thread-index-mode.rb
@@ -37,7 +37,6 @@ EOS
k.add :toggle_spam, "Mark/unmark thread as spam", 'S'
k.add :toggle_deleted, "Delete/undelete thread", 'd'
k.add :kill, "Kill thread (never to be seen in inbox again)", '&'
- k.add :save, "Save changes now", '$'
k.add :jump_to_next_new, "Jump to next new thread", :tab
k.add :reply, "Reply to latest message in a thread", 'r'
k.add :reply_all, "Reply to all participants of the latest message in a thread", 'G'
@@ -269,12 +268,14 @@ EOS
UndoManager.register "toggling thread starred status", undo
update_text_for_line curpos
cursor_down
+ Index.save_thread t
end
def multi_toggle_starred threads
UndoManager.register "toggling #{threads.size.pluralize 'thread'} starred status",
threads.map { |t| actually_toggle_starred t }
regen_text
+ threads.each { |t| Index.save_thread t }
end
## returns an undo lambda
@@ -352,12 +353,14 @@ EOS
undo = actually_toggle_archived t
UndoManager.register "deleting/undeleting thread #{t.first.id}", undo, lambda { update_text_for_line curpos }
update_text_for_line curpos
+ Index.save_thread t
end
def multi_toggle_archived threads
undos = threads.map { |t| actually_toggle_archived t }
UndoManager.register "deleting/undeleting #{threads.size.pluralize 'thread'}", undos, lambda { regen_text }
regen_text
+ threads.each { |t| Index.save_thread t }
end
def toggle_new
@@ -365,11 +368,13 @@ EOS
t.toggle_label :unread
update_text_for_line curpos
cursor_down
+ Index.save_thread t
end
def multi_toggle_new threads
threads.each { |t| t.toggle_label :unread }
regen_text
+ threads.each { |t| Index.save_thread t }
end
def multi_toggle_tagged threads
@@ -385,6 +390,7 @@ EOS
def multi_join_threads threads
@ts.join_threads threads or return
+ threads.each { |t| Index.save_thread t }
@tags.drop_all_tags # otherwise we have tag pointers to invalid threads!
update
end
@@ -421,6 +427,7 @@ EOS
UndoManager.register "marking/unmarking #{threads.size.pluralize 'thread'} as spam",
undos, lambda { regen_text }
regen_text
+ threads.each { Index.save_thread async t }
end
def toggle_deleted
@@ -434,6 +441,7 @@ EOS
UndoManager.register "deleting/undeleting #{threads.size.pluralize 'thread'}",
undos, lambda { regen_text }
regen_text
+ threads.each { Index.save_thread async t }
end
def kill
@@ -458,29 +466,7 @@ EOS
regen_text
BufferManager.flash "#{threads.size.pluralize 'thread'} killed."
- end
-
- def save background=true
- if background
- Redwood::reporting_thread("saving thread") { actually_save }
- else
- actually_save
- end
- end
-
- def actually_save
- @save_thread_mutex.synchronize do
- BufferManager.say("Saving contacts...") { ContactManager.instance.save }
- dirty_threads = @mutex.synchronize { (@threads + @hidden_threads.keys).select { |t| t.dirty? } }
- next if dirty_threads.empty?
-
- BufferManager.say("Saving threads...") do |say_id|
- dirty_threads.each_with_index do |t, i|
- BufferManager.say "Saving modified thread #{i + 1} of #{dirty_threads.length}...", say_id
- Index.save_thread_async t
- end
- end
- end
+ threads.each { |t| Index.save_thread async t }
end
def cleanup
@@ -492,7 +478,8 @@ EOS
sleep 0.1 # TODO: necessary?
BufferManager.erase_flash
end
- save false
+ dirty_threads = @mutex.synchronize { (@threads + @hidden_threads.keys).select { |t| t.dirty? } }
+ fail "dirty threads remain" unless dirty_threads.empty?
super
end
@@ -546,6 +533,7 @@ EOS
end
UpdateManager.relay self, :labeled, thread.first
+ Index.save_thread thread
end
def multi_edit_labels threads
@@ -582,6 +570,8 @@ EOS
end
regen_text
end
+
+ threads.each { |t| Index.save_thread t }
end
def reply type_arg=nil
diff --git a/lib/sup/modes/thread-view-mode.rb b/lib/sup/modes/thread-view-mode.rb
index 2da0555..e59112e 100644
--- a/lib/sup/modes/thread-view-mode.rb
+++ b/lib/sup/modes/thread-view-mode.rb
@@ -132,6 +132,7 @@ EOS
@layout[earliest].state = :detailed if earliest.has_label?(:unread) || @thread.size == 1
@thread.remove_label :unread
+ Index.save_thread @thread
regen_text
end
@@ -263,6 +264,7 @@ EOS
new_labels.each { |l| LabelManager << l }
update
UpdateManager.relay self, :labeled, @thread.first
+ Index.save_thread @thread
end
def toggle_starred
@@ -285,6 +287,7 @@ EOS
## star to the display
update
UpdateManager.relay self, :single_message_labeled, m
+ Index.save_thread @thread
end
## called when someone presses enter when the cursor is highlighting
@@ -503,6 +506,7 @@ EOS
dispatch op do
@thread.remove_label :inbox
UpdateManager.relay self, :archived, @thread.first
+ Index.save_thread @thread
end
end
@@ -510,6 +514,7 @@ EOS
dispatch op do
@thread.apply_label :spam
UpdateManager.relay self, :spammed, @thread.first
+ Index.save_thread @thread
end
end
@@ -517,6 +522,7 @@ EOS
dispatch op do
@thread.apply_label :deleted
UpdateManager.relay self, :deleted, @thread.first
+ Index.save_thread @thread
end
end
@@ -524,6 +530,7 @@ EOS
dispatch op do
@thread.apply_label :unread
UpdateManager.relay self, :unread, @thread.first
+ Index.save_thread @thread
end
end
--
1.6.4.2
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 6+ messages in thread
* [sup-talk] [PATCH 4/4] force the index sync thread to give up the cpu
2009-11-05 5:45 ` [sup-talk] [PATCH 3/4] immediate " Rich Lane
@ 2009-11-05 5:45 ` Rich Lane
0 siblings, 0 replies; 6+ messages in thread
From: Rich Lane @ 2009-11-05 5:45 UTC (permalink / raw)
To: sup-talk
---
lib/sup/index.rb | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/lib/sup/index.rb b/lib/sup/index.rb
index 1131ec7..4f491d6 100644
--- a/lib/sup/index.rb
+++ b/lib/sup/index.rb
@@ -201,6 +201,8 @@ class BaseIndex
while m = @sync_queue.deq
return if m == :die
update_message_state m
+ # Necessary to keep Xapian calls from lagging the UI too much.
+ sleep 0.03
end
end
end
--
1.6.4.2
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [sup-talk] [PATCH 2/4] async thread indexing
2009-11-05 5:45 ` [sup-talk] [PATCH 2/4] async thread indexing Rich Lane
2009-11-05 5:45 ` [sup-talk] [PATCH 3/4] immediate " Rich Lane
@ 2009-11-09 6:02 ` Eugene Marinelli
2009-11-09 6:22 ` Rich Lane
1 sibling, 1 reply; 6+ messages in thread
From: Eugene Marinelli @ 2009-11-09 6:02 UTC (permalink / raw)
To: Rich Lane; +Cc: sup-talk
> - Index.save_thread t
> + Index.save_thread_async t
Where is the "save_thread_async" function?
In patch 3/4, this looks like a syntax error:
> - end
> + threads.each { |t| Index.save_thread async t }
> end
- Eugene Marinelli
On Wed, Nov 4, 2009 at 9:45 PM, Rich Lane <rlane@club.cc.cmu.edu> wrote:
> ---
> bin/sup | 2 ++
> lib/sup/index.rb | 26 +++++++++++++++++++++++++-
> lib/sup/modes/thread-index-mode.rb | 2 +-
> 3 files changed, 28 insertions(+), 2 deletions(-)
>
> diff --git a/bin/sup b/bin/sup
> index 78c396a..910ff12 100755
> --- a/bin/sup
> +++ b/bin/sup
> @@ -142,6 +142,7 @@ Index.lock_interactively or exit
> begin
> Redwood::start
> Index.load
> + Index.start_sync_worker unless $opts[:no_threads]
>
> $die = false
> trap("TERM") { |x| $die = true }
> @@ -335,6 +336,7 @@ ensure
>
> HookManager.run "shutdown"
>
> + Index.stop_sync_worker
> Redwood::finish
> stop_cursing
> Redwood::Logger.remove_all_sinks!
> diff --git a/lib/sup/index.rb b/lib/sup/index.rb
> index 5d8d714..1131ec7 100644
> --- a/lib/sup/index.rb
> +++ b/lib/sup/index.rb
> @@ -28,6 +28,8 @@ class BaseIndex
> def initialize dir=BASE_DIR
> @dir = dir
> @lock = Lockfile.new lockfile, :retries => 0, :max_age => nil
> + @sync_worker = nil
> + @sync_queue = Queue.new
> end
>
> def lockfile; File.join @dir, "lock" end
> @@ -175,10 +177,32 @@ class BaseIndex
>
> def save_thread t
> t.each_dirty_message do |m|
> - update_message_state m
> + if @sync_worker
> + @sync_queue << m
> + else
> + update_message_state m
> + end
> m.clear_dirty
> end
> end
> +
> + def start_sync_worker
> + @sync_worker = Redwood::reporting_thread('index sync') { run_sync_worker }
> + end
> +
> + def stop_sync_worker
> + return unless worker = @sync_worker
> + @sync_worker = nil
> + @sync_queue << :die
> + worker.join
> + end
> +
> + def run_sync_worker
> + while m = @sync_queue.deq
> + return if m == :die
> + update_message_state m
> + end
> + end
> end
>
> index_name = ENV['SUP_INDEX'] || $config[:index] || DEFAULT_INDEX
> diff --git a/lib/sup/modes/thread-index-mode.rb b/lib/sup/modes/thread-index-mode.rb
> index 370903a..d2b1651 100644
> --- a/lib/sup/modes/thread-index-mode.rb
> +++ b/lib/sup/modes/thread-index-mode.rb
> @@ -477,7 +477,7 @@ EOS
> BufferManager.say("Saving threads...") do |say_id|
> dirty_threads.each_with_index do |t, i|
> BufferManager.say "Saving modified thread #{i + 1} of #{dirty_threads.length}...", say_id
> - Index.save_thread t
> + Index.save_thread_async t
> end
> end
> end
> --
> 1.6.4.2
>
> _______________________________________________
> sup-talk mailing list
> sup-talk@rubyforge.org
> http://rubyforge.org/mailman/listinfo/sup-talk
>
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [sup-talk] [PATCH 2/4] async thread indexing
2009-11-09 6:02 ` [sup-talk] [PATCH 2/4] async thread indexing Eugene Marinelli
@ 2009-11-09 6:22 ` Rich Lane
0 siblings, 0 replies; 6+ messages in thread
From: Rich Lane @ 2009-11-09 6:22 UTC (permalink / raw)
To: Eugene Marinelli; +Cc: sup-talk
Excerpts from Eugene Marinelli's message of Mon Nov 09 01:02:07 -0500 2009:
> > - Index.save_thread t
> > + Index.save_thread_async t
> Where is the "save_thread_async" function?
>
> In patch 3/4, this looks like a syntax error:
> > - end
> > + threads.each { |t| Index.save_thread async t }
> > end
Nice catch. I'd originally named the method save_thread_async and it
looks like my search-and-replace wasn't complete. I've fixed this on the
immediate-updates branch. Thanks for the review!
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2009-11-09 6:28 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-11-05 5:45 [sup-talk] [PATCH 1/4] factor saving out of thread/message classes Rich Lane
2009-11-05 5:45 ` [sup-talk] [PATCH 2/4] async thread indexing Rich Lane
2009-11-05 5:45 ` [sup-talk] [PATCH 3/4] immediate " Rich Lane
2009-11-05 5:45 ` [sup-talk] [PATCH 4/4] force the index sync thread to give up the cpu Rich Lane
2009-11-09 6:02 ` [sup-talk] [PATCH 2/4] async thread indexing Eugene Marinelli
2009-11-09 6:22 ` Rich Lane
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox