Archive of RubyForge sup-talk mailing list
 help / color / mirror / Atom feed
* [sup-talk] Increase poll frequency
@ 2008-12-08 20:59 Florian Bender
  2008-12-08 21:29 ` Ben Walton
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Florian Bender @ 2008-12-08 20:59 UTC (permalink / raw)


Hi,
after a quick glance at the documentation, I didn't notice any option to
increase the frequency sup polls a source for new E-Mails. For my GMail Account
I'll like it, if it'd be polled every minute or so.
Any suggestions?


-- 
Florian Bender <fjbender at googlemail.com>
GPG Fingerprint: D0BB BDF9 5889 BC30 49F4 681D 84F7 6AC7 7C39 85F8
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081208/87819a18/attachment.bin>


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [sup-talk] Increase poll frequency
  2008-12-08 20:59 [sup-talk] Increase poll frequency Florian Bender
@ 2008-12-08 21:29 ` Ben Walton
       [not found] ` <8F89701D-3DC1-4BAF-8A42-090D532FBF64@gmail.com>
  2008-12-11 20:20 ` William Morgan
  2 siblings, 0 replies; 9+ messages in thread
From: Ben Walton @ 2008-12-08 21:29 UTC (permalink / raw)


On Mon, Dec 8, 2008 at 3:59 PM, Florian Bender <fjbender at googlemail.com> wrote:
> Any suggestions?

Stop checking your email so much!?! <grin>

[Just kidding, but I couldn't resist.]

Cheers.
-Ben
-- 
---------------------------------------------------------------------------------------------------------------------------
Ben Walton <bdwalton at gmail.com>

For by doubting we come to inquire, and by inquiring we perceive the truth.

-Peter Abelard
---------------------------------------------------------------------------------------------------------------------------


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [sup-talk] Increase poll frequency
       [not found] ` <8F89701D-3DC1-4BAF-8A42-090D532FBF64@gmail.com>
@ 2008-12-09  6:14   ` Florian Bender
  0 siblings, 0 replies; 9+ messages in thread
From: Florian Bender @ 2008-12-09  6:14 UTC (permalink / raw)


Excerpts from fedzor's message of Mo Dez 08 22:59:52 +0100 2008:
> What setup are you using for your gmail account with sup? I've tried  
> it, but it crashes fairly often for me, so it's unusable.
> 
I enabled IMAP in my GMail preferences, then took a look at the configuration data they provide:
http://mail.google.com/support/bin/answer.py?answer=78799
Then added my GMail account as an IMAP source with sup-conf or sup-add if you have configured sup already.
Works like a charm, no crashes so far. Maybe you should report a bug, if the unexpected behaviour continues.

-- 
Florian Bender <fjbender at googlemail.com>
GPG Fingerprint: D0BB BDF9 5889 BC30 49F4 681D 84F7 6AC7 7C39 85F8
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081209/3f4089ca/attachment.bin>


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [sup-talk] Increase poll frequency
  2008-12-08 20:59 [sup-talk] Increase poll frequency Florian Bender
  2008-12-08 21:29 ` Ben Walton
       [not found] ` <8F89701D-3DC1-4BAF-8A42-090D532FBF64@gmail.com>
@ 2008-12-11 20:20 ` William Morgan
  2008-12-11 20:30   ` Florian Bender
  2008-12-11 20:41   ` Marc Hartstein
  2 siblings, 2 replies; 9+ messages in thread
From: William Morgan @ 2008-12-11 20:20 UTC (permalink / raw)


Reformatted excerpts from Florian Bender's message of 2008-12-08:
> after a quick glance at the documentation, I didn't notice any option
> to increase the frequency sup polls a source for new E-Mails. For my
> GMail Account I'll like it, if it'd be polled every minute or so.

There's no way to do this now, but it would be pretty easy to add.
-- 
William <wmorgan-sup at masanjin.net>


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [sup-talk] Increase poll frequency
  2008-12-11 20:20 ` William Morgan
@ 2008-12-11 20:30   ` Florian Bender
  2008-12-11 20:41   ` Marc Hartstein
  1 sibling, 0 replies; 9+ messages in thread
From: Florian Bender @ 2008-12-11 20:30 UTC (permalink / raw)


Excerpts from William Morgan's message of Do Dez 11 21:20:08 +0100 2008:
> There's no way to do this now, but it would be pretty easy to add.
Due to the fact that I'm no Ruby coder, I'd be glad if you could add it to the next release :-)

-- 
Florian Bender <fjbender at googlemail.com>
GPG Fingerprint: D0BB BDF9 5889 BC30 49F4 681D 84F7 6AC7 7C39 85F8
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081211/cd78f994/attachment.bin>


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [sup-talk] Increase poll frequency
  2008-12-11 20:20 ` William Morgan
  2008-12-11 20:30   ` Florian Bender
@ 2008-12-11 20:41   ` Marc Hartstein
  2008-12-12  1:37     ` William Morgan
  1 sibling, 1 reply; 9+ messages in thread
From: Marc Hartstein @ 2008-12-11 20:41 UTC (permalink / raw)


Excerpts from William Morgan's message of Thu Dec 11 15:20:08 -0500 2008:
> Reformatted excerpts from Florian Bender's message of 2008-12-08:
> > after a quick glance at the documentation, I didn't notice any option
> > to increase the frequency sup polls a source for new E-Mails. For my
> > GMail Account I'll like it, if it'd be polled every minute or so.
> 
> There's no way to do this now, but it would be pretty easy to add.

Christopher Warrington sent a patch to this list in the spring which
allows configuration of poll interval on a per-source basis.  This
should do what's being looked for.

I run it locally, mostly so sup doesn't waste time polling static
sources (like the mailing list archive from before I joined this list),
and it seems to work nicely.

Here's the email as I have it saved in my sup-patches/:

From chrisw at rice.edu  Sat Mar 29 06:32:05 2008
From: chrisw@rice.edu (Christopher Warrington)
Date: Sat, 29 Mar 2008 05:32:05 -0500
Subject: [sup-talk] [PATCH] polling is now done per source
In-Reply-To: <1206784680-sup-3052 at chris-tablet>
References: <1206784680-sup-3052 at chris-tablet>
Message-ID: <1206786725-5456-1-git-send-email-chrisw at rice.edu>

Each source has a poll_interval property. This property is used to
determine whether the source should be polled. The user can still for a
poll of all sources.
---
 bin/sup                    |    4 +-
 bin/sup-add                |   12 ++-
 lib/sup.rb                 |    4 +-
 lib/sup/imap.rb            |    6 +-
 lib/sup/maildir.rb         |    6 +-
 lib/sup/mbox/loader.rb     |    4 +-
 lib/sup/modes/poll-mode.rb |    4 +-
 lib/sup/poll.rb            |  211 +++++++++++++++++++++++++++++++++-----------
 lib/sup/source.rb          |    8 +-
 9 files changed, 189 insertions(+), 70 deletions(-)
 mode change 100755 => 100644 lib/sup/buffer.rb

diff --git a/bin/sup b/bin/sup
index 84fd77c..86a2d9f 100644
--- a/bin/sup
+++ b/bin/sup
@@ -197,7 +197,7 @@ begin
     end
   end unless $opts[:no_initial_poll]
   
-  imode.load_threads :num => ibuf.content_height, :when_done => lambda { reporting_thread("poll after loading inbox") { sleep 1; PollManager.poll } unless $opts[:no_threads] || $opts[:no_initial_poll] }
+  imode.load_threads :num => ibuf.content_height, :when_done => lambda { reporting_thread("poll after loading inbox") { sleep 1; PollManager.auto_poll } unless $opts[:no_threads] || $opts[:no_initial_poll] }
 
   if $opts[:compose]
     ComposeMode.spawn_nicely :to_default => $opts[:compose]
@@ -263,7 +263,7 @@ begin
     when :compose
       ComposeMode.spawn_nicely
     when :poll
-      reporting_thread("user-invoked poll") { PollManager.poll }
+      reporting_thread("user-invoked poll") { PollManager.forced_poll }
     when :recall_draft
       case Index.num_results_for :label => :draft
       when 0
diff --git a/bin/sup-add b/bin/sup-add
index 50bbb29..88705e1 100644
--- a/bin/sup-add
+++ b/bin/sup-add
@@ -39,6 +39,7 @@ EOS
   opt :unusual, "Do not automatically poll these sources for new messages."
   opt :labels, "A comma-separated set of labels to apply to all messages from this source", :type => String
   opt :force_new, "Create a new account for this source, even if one already exists."
+  opt :poll_interval, "The interval (in seconds) between new message polls. The default is #{Redwood::DEFAULT_POLL_INTERVAL}.", :type => :int
 end
 
 Trollop::die "require one or more sources" if ARGV.empty?
@@ -84,6 +85,9 @@ index.lock_or_die
 begin
   index.load_sources
 
+  Trollop::die "The poll interval must be a positive integer." if $opts[:poll_interval] <= 0
+  poll_interval = $opts[:poll_interval] || Redwood::DEFAULT_POLL_INTERVAL
+
   ARGV.each do |uri|
     labels = $opts[:labels] ? $opts[:labels].split(/\s*,\s*/).uniq : []
 
@@ -100,14 +104,14 @@ begin
         say "For SSH connections, if you will use public key authentication, you may leave the username and password blank."
         say ""
         username, password = get_login_info uri, index.sources
-        Redwood::MBox::SSHLoader.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, labels
+        Redwood::MBox::SSHLoader.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels
       when "imap", "imaps"
         username, password = get_login_info uri, index.sources
-        Redwood::IMAP.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, labels
+        Redwood::IMAP.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels
       when "maildir"
-        Redwood::Maildir.new uri, nil, !$opts[:unusual], $opts[:archive], nil, labels
+        Redwood::Maildir.new uri, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels
       when "mbox"
-        Redwood::MBox::Loader.new uri, nil, !$opts[:unusual], $opts[:archive], nil, labels
+        Redwood::MBox::Loader.new uri, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels
       when nil
         Trollop::die "Sources must be specified with an URI"
       else
diff --git a/lib/sup.rb b/lib/sup.rb
index 1946f3c..c27a4bc 100644
--- a/lib/sup.rb
+++ b/lib/sup.rb
@@ -50,6 +50,8 @@ module Redwood
   YAML_DOMAIN = "masanjin.net"
   YAML_DATE = "2006-10-01"
 
+  DEFAULT_POLL_INTERVAL = 300
+
 ## record exceptions thrown in threads nicely
   def reporting_thread name
     if $opts[:no_threads]
@@ -72,7 +74,7 @@ module Redwood
   def save_yaml_obj object, fn, safe=false
     if safe
       safe_fn = "#{File.dirname fn}/safe_#{File.basename fn}"
-      mode = File.stat(fn) if File.exists? fn
+      mode = File.stat(fn).mode if File.exists? fn
       File.open(safe_fn, "w", mode) { |f| f.puts object.to_yaml }
       FileUtils.mv safe_fn, fn
     else
diff --git a/lib/sup/buffer.rb b/lib/sup/buffer.rb
old mode 100755
new mode 100644
diff --git a/lib/sup/imap.rb b/lib/sup/imap.rb
index 1d36976..8b58cba 100644
--- a/lib/sup/imap.rb
+++ b/lib/sup/imap.rb
@@ -51,13 +51,13 @@ class IMAP < Source
 
   attr_accessor :username, :password
   yaml_properties :uri, :username, :password, :cur_offset, :usual,
-                  :archived, :id, :labels
+                  :archived, :id, :poll_interval, :labels
 
-  def initialize uri, username, password, last_idate=nil, usual=true, archived=false, id=nil, labels=[]
+  def initialize uri, username, password, last_idate=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[]
     raise ArgumentError, "username and password must be specified" unless username && password
     raise ArgumentError, "not an imap uri" unless uri =~ %r!imaps?://!
 
-    super uri, last_idate, usual, archived, id
+    super uri, last_idate, usual, archived, id, [poll_interval, SCAN_INTERVAL].max
 
     @parsed_uri = URI(uri)
     @username = username
diff --git a/lib/sup/maildir.rb b/lib/sup/maildir.rb
index 584e657..ef71e0b 100644
--- a/lib/sup/maildir.rb
+++ b/lib/sup/maildir.rb
@@ -12,9 +12,9 @@ class Maildir < Source
   SCAN_INTERVAL = 30 # seconds
 
   ## remind me never to use inheritance again.
-  yaml_properties :uri, :cur_offset, :usual, :archived, :id, :labels
-  def initialize uri, last_date=nil, usual=true, archived=false, id=nil, labels=[]
-    super uri, last_date, usual, archived, id
+  yaml_properties :uri, :cur_offset, :usual, :archived, :id, :poll_interval, :labels
+  def initialize uri, last_date=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[]
+    super uri, last_date, usual, archived, id, [poll_interval, SCAN_INTERVAL].max
     uri = URI(Source.expand_filesystem_uri(uri))
 
     raise ArgumentError, "not a maildir URI" unless uri.scheme == "maildir"
diff --git a/lib/sup/mbox/loader.rb b/lib/sup/mbox/loader.rb
index 7fe9129..44317d5 100644
--- a/lib/sup/mbox/loader.rb
+++ b/lib/sup/mbox/loader.rb
@@ -9,7 +9,7 @@ class Loader < Source
   attr_accessor :labels
 
   ## uri_or_fp is horrific. need to refactor.
-  def initialize uri_or_fp, start_offset=nil, usual=true, archived=false, id=nil, labels=[]
+  def initialize uri_or_fp, start_offset=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[]
     @mutex = Mutex.new
     @labels = ((labels || []) - LabelManager::RESERVED_LABELS).uniq.freeze
 
@@ -26,7 +26,7 @@ class Loader < Source
       @path = uri_or_fp.path
     end
 
-    super uri_or_fp, start_offset, usual, archived, id
+    super uri_or_fp, start_offset, usual, archived, id, poll_interval
   end
 
   def file_path; @path end
diff --git a/lib/sup/modes/poll-mode.rb b/lib/sup/modes/poll-mode.rb
index 5849f3e..5521cdc 100644
--- a/lib/sup/modes/poll-mode.rb
+++ b/lib/sup/modes/poll-mode.rb
@@ -10,11 +10,11 @@ class PollMode < LogMode
     self << s + "\n"
   end
 
-  def poll
+  def poll sources
     puts unless @new
     @new = false
     puts "Poll started at #{Time.now}"
-    PollManager.do_poll { |s| puts s }
+    PollManager.do_poll_sources(sources) { |s| puts s }
   end
 end
 
diff --git a/lib/sup/poll.rb b/lib/sup/poll.rb
index d32c893..1ff8014 100644
--- a/lib/sup/poll.rb
+++ b/lib/sup/poll.rb
@@ -12,13 +12,22 @@ Variables:
 EOS
 
   HookManager.register "before-poll", <<EOS
+Executes immediately before any poll for new messages commences.
+Variables:
+                 accts: an array of source URIs that will be polled
+EOS
+
+  HookManager.register "before-acct-poll", <<EOS
 Executes immediately before a poll for new messages commences.
-No variables.
+Variables:
+                  acct: the URI of the account being polled
 EOS
 
-  HookManager.register "after-poll", <<EOS
-Executes immediately after a poll for new messages completes.
+  HookManager.register "after-acct-poll", <<EOS
+Executes immediately after a poll for new messages completes. The statistics are
+only for the account specified.
 Variables:
+                  acct: the URI of the account being polled
                    num: the total number of new messages added in this poll
              num_inbox: the number of new messages added in this poll which
                         appear in the inbox (i.e. were not auto-archived).
@@ -28,14 +37,25 @@ num_inbox_total_unread: the total number of unread messages in the inbox
                         only those messages appearing in the inbox
 EOS
 
-  DELAY = 300
+  HookManager.register "after-poll", <<EOS
+Executes immediately after any poll for new messages completes. The statistics
+are for all sources polled.
+Variables:
+                 accts: an array of source URIs that were polled
+                   num: the total number of new messages added in this poll
+             num_inbox: the number of new messages added in this poll which
+                        appear in the inbox (i.e. were not auto-archived).
+num_inbox_total_unread: the total number of unread messages in the inbox
+         from_and_subj: an array of (from email address, subject) pairs
+   from_and_subj_inbox: an array of (from email address, subject) pairs for
+                        only those messages appearing in the inbox
+EOS
 
   def initialize
     @mutex = Mutex.new
     @thread = nil
-    @last_poll = nil
     @polling = false
-    
+
     self.class.i_am_the_instance self
   end
 
@@ -44,47 +64,113 @@ EOS
     b
   end
 
-  def poll
+  def start
+    ## Periodically calls auto_poll, effectively automatically polling
+    ## in the background.
+    ##
+    ## auto_poll is called more frequently than the smallest poll
+    ## interval. auto_poll is not guaranteed to be called in multiples
+    ## of poll intervals. I.e., don't rely on auto_poll to be called
+    ## every x seconds: when auto_poll is called assume an
+    ## indeterminate amount of time has passed.
+    @min_poll_interval = Index.usual_sources.collect{|s| s.poll_interval}.select{|pi| !pi.nil?}.min || Redwood::DEFAULT_POLL_INTERVAL
+
+    Redwood::log "Background poll interval is #{@min_poll_interval} seconds."
+
+    @thread = Redwood::reporting_thread("periodic poll") do
+      while true
+        Redwood::log "Sleeping for #{@min_poll_interval / 2} seconds."
+
+        sleep @min_poll_interval / 2
+        auto_poll
+      end
+    end
+  end
+
+  def stop
+    @thread.kill if @thread
+    @thread = nil
+  end
+
+  def auto_poll
+    ## This method is called by the thread spawned in start. It
+    ## collects the sources that should be polled
+    ## (source.poll_interval has expired or the source has never been
+    ## polled) and polls them.
+    ##
+    ## Returns an array [# of new messages, # of new messages to
+    ## inbox, new message subject/name array pairs, subject/name array
+    ## pairs loaded to inbox]
+
+    sources_to_poll = []
+    @mutex.synchronize do # William, do we need to synchronize here?
+      begin
+        sources_to_poll = Index.usual_sources.select do |source|
+          begin
+            source.last_poll.nil? || (Time.now - source.last_poll) >= source.poll_interval
+          end
+        end
+      end
+    end
+
+    poll_sources sources_to_poll
+  end
+
+  def forced_poll
+    ## This method is called when the user explicitly requests a poll.
+    ##
+    ## Returns an array [# of new messages, # of new messages to
+    ## inbox, new message subject/name array pairs, subject/name array
+    ## pairs loaded to inbox]
+    poll_sources Index.usual_sources
+  end
+
+  def poll_sources sources
+    ## Polls the given sources. Clients of PollManager should call
+    ## this method, not do_poll_sources or poll_source. (Well, only
+    ## PollMode should call do_poll_sources)
+    ##
+    ## Returns an array [# of new messages, # of new messages to
+    ## inbox, new message subject/name array pairs, subject/name array
+    ## pairs loaded to inbox]
     return if @polling
     @polling = true
-    HookManager.run "before-poll"
+
+    source_uris = sources.map{|s| s.uri}
+
+    HookManager.run "before-acct-poll", :accts => source_uris
 
     BufferManager.flash "Polling for new messages..."
-    num, numi, from_and_subj, from_and_subj_inbox = buffer.mode.poll
+    num, numi, from_subj, from_subj_inbox = buffer.mode.poll sources
     if num > 0
       BufferManager.flash "Loaded #{num.pluralize 'new message'}, #{numi} to inbox." 
     else
       BufferManager.flash "No new messages." 
     end
 
-    HookManager.run "after-poll", :num => num, :num_inbox => numi, :from_and_subj => from_and_subj, :from_and_subj_inbox => from_and_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] }
+    HookManager.run "after-poll", :accts => source_uris, :num => num, :num_inbox => numi, :from_and_subj => from_subj, :from_and_subj_inbox => from_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] }
 
     @polling = false
-    [num, numi]
-  end
-
-  def start
-    @thread = Redwood::reporting_thread("periodic poll") do
-      while true
-        sleep DELAY / 2
-        poll if @last_poll.nil? || (Time.now - @last_poll) >= DELAY
-      end
-    end
+    [num, numi, from_subj, from_subj_inbox]
   end
 
-  def stop
-    @thread.kill if @thread
-    @thread = nil
-  end
+  def do_poll_sources sources, &block
+    ## Polls each source, keeping track of vital statistics (number
+    ## loaded, name/e-mail pairs, &c.) about the poll results.
+    ##
+    ## We need explicit access to the block so that we can pass it to
+    ## poll_source.
+    ##
+    ## Returns an array [total # of new messages, total # of new
+    ## messages to inbox, all new message subject/name array pairs,
+    ## all subject/name array pairs loaded to inbox]
 
-  def do_poll
     total_num = total_numi = 0
-    from_and_subj = []
-    from_and_subj_inbox = []
+    total_from_and_subj = []
+    total_from_and_subj_inbox = []
 
     @mutex.synchronize do
-      Index.usual_sources.each do |source|
-#        yield "source #{source} is done? #{source.done?} (cur_offset #{source.cur_offset} >= #{source.end_offset})"
+      sources.each do |source|
         begin
           yield "Loading from #{source}... " unless source.done? || source.has_errors?
         rescue SourceError => e
@@ -93,32 +179,57 @@ EOS
           next
         end
 
-        num = 0
-        numi = 0
-        add_messages_from source do |m, offset, entry|
-          ## always preserve the labels on disk.
-          m.labels = entry[:label].split(/\s+/).map { |x| x.intern } if entry
-          yield "Found message at #{offset} with labels {#{m.labels * ', '}}"
-          unless entry
-            num += 1
-            from_and_subj << [m.from.longname, m.subj]
-            if m.has_label?(:inbox) && ([:spam, :deleted, :killed] & m.labels).empty?
-              from_and_subj_inbox << [m.from.longname, m.subj]
-              numi += 1 
-            end
-          end
-          m
-        end
-        yield "Found #{num} messages, #{numi} to inbox." unless num == 0
+        HookManager.run "before-acct-poll", :acct => source.uri
+
+        BufferManager.flash "Polling #{source.uri} for new messages..."
+        num, numi, from_and_subj, from_and_subj_inbox = poll_source source, &block
+
         total_num += num
         total_numi += numi
+        total_from_and_subj += from_and_subj
+        total_from_and_subj_inbox += from_and_subj_inbox
+
+        HookManager.run "after-acct-poll", :acct => source.uri, :num => num, :num_inbox => numi, :from_and_subj => from_and_subj, :from_and_subj_inbox => from_and_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] }
       end
+    end
+
+    [total_num, total_numi, total_from_and_subj, total_from_and_subj_inbox]
+  end
 
-      yield "Done polling; loaded #{total_num} new messages total"
-      @last_poll = Time.now
-      @polling = false
+  def poll_source source
+    ## Polls the given source for new messages.
+    ##
+    ## @mutex must be held before calling! See do_poll_sources.
+    ##
+    ## Returns an array [# of new messages, # of new messages to
+    ## inbox, new message subject/name array pairs, subject/name array
+    ## pairs loaded to inbox]
+
+    num = 0
+    numi = 0
+    from_and_subj = []
+    from_and_subj_inbox = []
+
+    add_messages_from source do |m, offset, entry|
+      ## always preserve the labels on disk.
+      m.labels = entry[:label].split(/\s+/).map { |x| x.intern } if entry
+      yield "Found message at #{offset} with labels {#{m.labels * ', '}}"
+      unless entry
+        num += 1
+        from_and_subj << [m.from.longname, m.subj]
+        if m.has_label?(:inbox) && ([:spam, :deleted, :killed] & m.labels).empty?
+          from_and_subj_inbox << [m.from.longname, m.subj]
+          numi += 1
+        end
+      end
+      m
     end
-    [total_num, total_numi, from_and_subj, from_and_subj_inbox]
+
+    yield "For source #{source.uri}, found #{num} messages, #{numi} to inbox." unless num == 0
+
+    source.last_poll = Time.now
+
+    [num, numi, from_and_subj, from_and_subj_inbox]
   end
 
   ## this is the main mechanism for adding new messages to the
diff --git a/lib/sup/source.rb b/lib/sup/source.rb
index 6510aae..fd8d381 100644
--- a/lib/sup/source.rb
+++ b/lib/sup/source.rb
@@ -62,10 +62,10 @@ class Source
   ## dirty? means cur_offset has changed, so the source info needs to
   ## be re-saved to sources.yaml.
   bool_reader :usual, :archived, :dirty
-  attr_reader :uri, :cur_offset
-  attr_accessor :id
+  attr_reader :uri, :cur_offset, :poll_interval
+  attr_accessor :id, :last_poll
 
-  def initialize uri, initial_offset=nil, usual=true, archived=false, id=nil
+  def initialize uri, initial_offset=nil, usual=true, archived=false, id=nil, poll_interval=nil
     raise ArgumentError, "id must be an integer: #{id.inspect}" unless id.is_a? Fixnum if id
 
     @uri = uri
@@ -73,6 +73,8 @@ class Source
     @usual = usual
     @archived = archived
     @id = id
+    @poll_interval = poll_interval || Redwood::DEFAULT_POLL_INTERVAL #seconds
+    @last_poll = nil
     @dirty = false
   end
 
-- 
1.5.4

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081211/d2fdb6d5/attachment.bin>


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [sup-talk] Increase poll frequency
  2008-12-11 20:41   ` Marc Hartstein
@ 2008-12-12  1:37     ` William Morgan
  2008-12-12  1:58       ` Christopher Warrington
  0 siblings, 1 reply; 9+ messages in thread
From: William Morgan @ 2008-12-12  1:37 UTC (permalink / raw)


Reformatted excerpts from marc.hartstein's message of 2008-12-11:
> Christopher Warrington sent a patch to this list in the spring which
> allows configuration of poll interval on a per-source basis.  This
> should do what's being looked for.

From what I remember there were some minor problems with the patch and
we were promised a fixed version, and I think that's why it wasn't
applied to the repo. But it may be sufficient for your purposes right
now!

(And the miracle of git is that, if you apply it, you can pull --rebase
and keep the patch on top even when you get new changes.)
-- 
William <wmorgan-sup at masanjin.net>


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [sup-talk] Increase poll frequency
  2008-12-12  1:37     ` William Morgan
@ 2008-12-12  1:58       ` Christopher Warrington
  2008-12-20  1:03         ` Marc Hartstein
  0 siblings, 1 reply; 9+ messages in thread
From: Christopher Warrington @ 2008-12-12  1:58 UTC (permalink / raw)


"William Morgan" <wmorgan-sup at masanjin.net> @ 2008-12-11 7:37 PM:

>> From what I remember there were some minor problems with the patch and

Something's up somewhere with "From " and ">From " lines...

> From what I remember there were some minor problems with the patch and
> we were promised a fixed version, and I think that's why it wasn't
> applied to the repo. But it may be sufficient for your purposes right
> now!

Anyone remember what those were? I haven't been using Sup lately, so the
deficiencies haven't been bugging me. :-)

-- 
Christopher Warrington <chrisw at rice.edu>
Jones College




^ permalink raw reply	[flat|nested] 9+ messages in thread

* [sup-talk] Increase poll frequency
  2008-12-12  1:58       ` Christopher Warrington
@ 2008-12-20  1:03         ` Marc Hartstein
  0 siblings, 0 replies; 9+ messages in thread
From: Marc Hartstein @ 2008-12-20  1:03 UTC (permalink / raw)


Excerpts from Christopher Warrington's message of Thu Dec 11 20:58:23 -0500 2008:
> "William Morgan" <wmorgan-sup at masanjin.net> @ 2008-12-11 7:37 PM:
> 
> > From what I remember there were some minor problems with the patch and
> > we were promised a fixed version, and I think that's why it wasn't
> > applied to the repo. But it may be sufficient for your purposes right
> > now!
> 
> Anyone remember what those were? I haven't been using Sup lately, so the
> deficiencies haven't been bugging me. :-)

I do now, as it just bit me again:

If you run a version of sup with the patch, it will crash on startup if
any usual source lacks the :poll_interval property.

If you have just installed the patch, this will bite you.

If you have run a non-patched version of sup for any reason, it will
blow away the :poll_interval properties on clean exit (because it
doesn't recognize them, so it doesn't write them to sources.yaml)


Suggest using default value when no value is supplied to solve the first
case, probably emitting a warning to help slightly with the second case.

I have no idea what to do about the second case more generally, except
to perpetuate the patch to 'next' and 'master'.

There may be other issues that I'm unaware of.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081219/f3703503/attachment.bin>


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2008-12-20  1:03 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-12-08 20:59 [sup-talk] Increase poll frequency Florian Bender
2008-12-08 21:29 ` Ben Walton
     [not found] ` <8F89701D-3DC1-4BAF-8A42-090D532FBF64@gmail.com>
2008-12-09  6:14   ` Florian Bender
2008-12-11 20:20 ` William Morgan
2008-12-11 20:30   ` Florian Bender
2008-12-11 20:41   ` Marc Hartstein
2008-12-12  1:37     ` William Morgan
2008-12-12  1:58       ` Christopher Warrington
2008-12-20  1:03         ` Marc Hartstein

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox