Subject: [sup-talk] [PATCH] Add an after-add-message hook

From: [email-masked] (Kevin Riggle)
Date: Sun, 16 Aug 2009 17:47:09 -0400

From: Kevin Riggle <kevinr at black-opal.mit.edu>

I want to do some unrelated processing on each message I receive, but I
don't want to block the message being added to the index.  This patch
adds a hook which runs /after/ the message is added to the index.
---
 lib/sup/poll.rb |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/lib/sup/poll.rb b/lib/sup/poll.rb
index 8a9d218..e4b7e02 100644
--- a/lib/sup/poll.rb
+++ b/lib/sup/poll.rb
@@ -11,6 +11,12 @@ Variables:
   message: the new message
 EOS
 
+  HookManager.register "after-add-message", <<EOS
+Executes after a message is added to the index.
+Variables:
+  message: the new message
+EOS
+
   HookManager.register "before-poll", <<EOS
 Executes immediately before a poll for new messages commences.
 No variables.
@@ -156,6 +162,7 @@ EOS
         m_ret = yield(m_old, m_new, offset) or next if block_given?
         Index.sync_message m_ret, opts
         UpdateManager.relay self, :added, m_ret unless m_old
+        HookManager.run "after-add-message", :message => m_new
       end
     rescue SourceError => e
       Redwood::log "problem getting messages from #{source}: #{e.message}"
-- 
1.6.0.4

---

From: [email-masked] (William Morgan)
Date: Tue, 18 Aug 2009 10:56:15 -0700

Reformatted excerpts from Kevin Riggle's message of 2009-08-16:
> I want to do some unrelated processing on each message I receive, but
> I don't want to block the message being added to the index.  This
> patch adds a hook which runs /after/ the message is added to the
> index.

Won't this block subsequent emails from being added to the index, when
Sup gets multiple new emails during a poll?
-- 
William <wmorgan-sup at masanjin.net>

---

From: [email-masked] (Kevin Riggle)
Date: Tue, 18 Aug 2009 19:36:15 -0400

> Sup gets multiple new emails during a poll?

I don't /think/ so, unless I misunderstand the way hooks work.  I assumed
the hook would execute and then execution of the loop would resume, and
I don't think I throw anything in the hook which should cause that either.

- Kevin
-- 
Kevin Riggle (kevinr at free-dissociation.com) 
http://free-dissociation.com

---

From: [email-masked] (William Morgan)
Date: Wed, 19 Aug 2009 11:55:03 -0700

Reformatted excerpts from Kevin Riggle's message of 2009-08-18:
> I don't /think/ so, unless I misunderstand the way hooks work.  I
> assumed the hook would execute and then execution of the loop would
> resume, and I don't think I throw anything in the hook which should
> cause that either.

What I'm saying is that this hook will execute *after* each message is
added to the index, so if Sup found several new messages, all except the
first one will wait on this hook to execute before being added to the
index. Or do you mean block in some sense besides "wait for completion"?
-- 
William <wmorgan-sup at masanjin.net>

---

From: [email-masked] (Kevin Riggle)
Date: Thu, 20 Aug 2009 21:57:42 -0400

This time the loop stores messages in an array as they're added to the
index, and passes that array to the hook.

I want to do some unrelated processing on each message I receive, but I
don't want to block the message being added to the index.  This patch
adds a hook which runs /after/ the message is added to the index.
---
 lib/sup/poll.rb |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/lib/sup/poll.rb b/lib/sup/poll.rb
index 8a9d218..fb3aacf 100644
--- a/lib/sup/poll.rb
+++ b/lib/sup/poll.rb
@@ -11,6 +11,12 @@ Variables:
   message: the new message
 EOS
 
+  HookManager.register "after-add-message", <<EOS
+Executes after all messages are added to the index.
+Variables:
+  messages: an array of the new messages added
+EOS
+
   HookManager.register "before-poll", <<EOS
 Executes immediately before a poll for new messages commences.
 No variables.
@@ -138,6 +144,7 @@ EOS
     begin
       return if source.done? || source.has_errors?
 
+      messages = []
       source.each do |offset, default_labels|
         if source.has_errors?
           Redwood::log "error loading messages from #{source}: #{source.error.message}"
@@ -145,6 +152,7 @@ EOS
         end
 
         m_new = Message.build_from_source source, offset
+        messages.push(m_new)
         m_old = Index.build_message m_new.id
 
         m_new.labels += default_labels + (source.archived? ? [] : [:inbox])
@@ -157,6 +165,8 @@ EOS
         Index.sync_message m_ret, opts
         UpdateManager.relay self, :added, m_ret unless m_old
       end
+      HookManager.run "after-add-message", :messages => messages
+
     rescue SourceError => e
       Redwood::log "problem getting messages from #{source}: #{e.message}"
       Redwood::report_broken_sources :force_to_top => true
-- 
1.6.0.4

---

From: [email-masked] (Kevin Riggle)
Date: Thu, 20 Aug 2009 22:07:17 -0400

> index. Or do you mean block in some sense besides "wait for completion"?

Oh, duh.  I take your point -- I've sent a new patch.

- Kevin

---

From: [email-masked] (William Morgan)
Date: Mon, 24 Aug 2009 15:44:58 -0700

> patch adds a hook which runs /after/ the message is added to the

Branch after-add-message-hook, merged into next. Thanks!
-- 
William <wmorgan-sup at masanjin.net>

---

From: [email-masked] (William Morgan)
Date: Wed, 02 Sep 2009 06:53:17 -0700

> Branch after-add-message-hook, merged into next. Thanks!

I've had to unmerge this, because keeping all the messages around led to
memory problems with large mailboxes.

What about just using the before-add hook and either backgrounding the
process (if it's a separate process) or doing the processing in a Thread
(if it's Ruby code)?
-- 
William <wmorgan-sup at masanjin.net>