* [sup-talk] New User Questions @ 2008-03-28 0:22 Marc Hartstein 2008-03-28 10:33 ` Marcus Williams 0 siblings, 1 reply; 19+ messages in thread From: Marc Hartstein @ 2008-03-28 0:22 UTC (permalink / raw) Sorry if any of this has been discussed elsewhere. I've read the docs on the website, skimmed the wiki, and skimmed the mailing list archives for the past few months, but I could easily have missed something. First, I ran into the same problem discussed in the thread "startup crash" started on 2008-03-09. (I'm also running Gentoo and installed the latest from portage) I poked around the source a bit, and it seems that when AccountManager.initialize is called out of Redwood::start (sup line 113), the code to set up the default configuration (lib/sup.rb, starts at line 166) has *not* been called, so the $config object is empty. Just in case the information helps; I'm not a Ruby person (yet), and I didn't poke around enough to be clear on how it's supposed to be working. 2. sup is pretty cool. I'm particularly intrigued by this tag (sorry, label) concept. 3. Is there an IRC channel? It would be nice to be able to lurk and drop questions if anybody happens to be around. 4. I use maildir. Is there any way sup can mark messages as read (and, perhaps more importantly, no longer new) once I've read them and hit '$'? I can deal with mutt being confused (I know sup doesn't play well with others anyway), but it's really confusing my mail monitor, and that's bugging me. 5. Can sup be set to automatically poll local sources for new messages every n seconds? Mutt does this, and it's nice not having to tell it that I know there are new messages and it should go find them. 6. Mutt can pass text/html attachments through an external program and display the result in its internal pager. Is there a way to get sup to do the equivalent? 7. How does sup choose the email address to use as "From" when replying to an email? It's definitely choosing the wrong thing in my setup. My primary address forwards to gmail, sup is configured with the primary as my :email:, but whenever I hit reply it sets "From" to be my gmail address. I'd like it to be whatever alternate of mine is in To/CC if any, and my primary address otherwise. 8. Is there any way to specify custom keybindings in a configuration file, or would it require hacking the source? 8.1 I'd really like a one-touch reply-all like I had in mutt. 9. Is there a way to default to making all messages PGP-signed? 10. Is there (planned) a compose-hook? I'd like to be able to switch to sign-and-encrypt when I have a public key for every recipient. Thanks for the cool project. I'll probably have more "I miss this feature from mutt" thoughts in the future, at least if I don't go running back. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://rubyforge.org/pipermail/sup-talk/attachments/20080327/ca129127/attachment.bin ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] New User Questions 2008-03-28 0:22 [sup-talk] New User Questions Marc Hartstein @ 2008-03-28 10:33 ` Marcus Williams 2008-03-28 11:24 ` vasudeva ` (2 more replies) 0 siblings, 3 replies; 19+ messages in thread From: Marcus Williams @ 2008-03-28 10:33 UTC (permalink / raw) On 28.3.2008, Marc Hartstein wrote: > 3. Is there an IRC channel? It would be nice to be able to lurk and > drop questions if anybody happens to be around. Someone has suggested #sup on freenode but no takers so far. I'd probably lurk on a channel if we can agree on one :) > 4. I use maildir. Is there any way sup can mark messages as read (and, > perhaps more importantly, no longer new) once I've read them and hit > '$'? I can deal with mutt being confused (I know sup doesn't play well > with others anyway), but it's really confusing my mail monitor, and > that's bugging me. Technically not (although you could probably write a hook to do this). However, you could change the way your monitor works if you're using sup locally. See the before/after poll hooks [1] > 5. Can sup be set to automatically poll local sources for new messages > every n seconds? Mutt does this, and it's nice not having to tell it > that I know there are new messages and it should go find them. It should do already. I dont know offhand what the delay is though. > 6. Mutt can pass text/html attachments through an external program and > display the result in its internal pager. Is there a way to get sup to > do the equivalent? Theres more than one way to do this. Via a hook for auto-demangling (mime-decode.rb) [1] and via the pipe attachment/message keypress '|' in thread view mode. Basically if you want to view an attachment put your cursor line over the attachment line and hit '|'. sup will pipe the attachment to what ever command line you type in here. html2text works well (as does antiword for word docs). If your cursor is in the message somewhere other than an attachment line it will pipe the raw message to the command. > 7. How does sup choose the email address to use as "From" when replying > to an email? It's definitely choosing the wrong thing in my setup. My > primary address forwards to gmail, sup is configured with the primary as > my :email:, but whenever I hit reply it sets "From" to be my gmail > address. I'd like it to be whatever alternate of mine is in To/CC if > any, and my primary address otherwise. The :alternates: setting is probably what you want here. I think you should just have to add your gmail address as an alternate in your default account. At least this is how I think it should work! Basically the alternate emails list tell sup you receive email at this account as these addresses as well, but that you reply using the default :email: setting. You may also be interested in the :regexen: list - this allows you to setup mail extensions (like marcus-sup that I use) and get sup to reply using the extension. > 8. Is there any way to specify custom keybindings in a configuration > file, or would it require hacking the source? Hack the source afaik > 8.1 I'd really like a one-touch reply-all like I had in mutt. I'm about to submit a patch for this. > 9. Is there a way to default to making all messages PGP-signed? I dont think so at the moment (again, it wouldnt be hard to add) > 10. Is there (planned) a compose-hook? I'd like to be able to > switch to sign-and-encrypt when I have a public key for every > recipient. Theres certainly a before-edit hook, which you may be able to do things like this in. I havnt played with the pgp stuff yet so someone else probably knows more here. HTH Marcus [1] http://sup.rubyforge.org/wiki/wiki.pl?Hooks ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] New User Questions 2008-03-28 10:33 ` Marcus Williams @ 2008-03-28 11:24 ` vasudeva 2008-03-28 17:23 ` Marc Hartstein 2008-03-29 16:41 ` [sup-talk] Different Key Bindings (was: New User Questions) Guarded Identity 2 siblings, 0 replies; 19+ messages in thread From: vasudeva @ 2008-03-28 11:24 UTC (permalink / raw) Excerpts from Marcus Williams's message of Fri Mar 28 06:33:45 -0400 2008: > On 28.3.2008, Marc Hartstein wrote: > > 3. Is there an IRC channel? It would be nice to be able to lurk and > > drop questions if anybody happens to be around. > > Someone has suggested #sup on freenode but no takers so far. I'd > probably lurk on a channel if we can agree on one :) Actually, there are a couple of us lurking there now. Come on in. -- linkswarm.com :: Collaborative Insolence vasudeva.linkswarm.com/gallery :: For The Faint of Heart ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] New User Questions 2008-03-28 10:33 ` Marcus Williams 2008-03-28 11:24 ` vasudeva @ 2008-03-28 17:23 ` Marc Hartstein 2008-03-28 19:51 ` Marcus Williams 2008-03-29 7:57 ` [sup-talk] [PATCH] polling is now done per source Christopher Warrington 2008-03-29 16:41 ` [sup-talk] Different Key Bindings (was: New User Questions) Guarded Identity 2 siblings, 2 replies; 19+ messages in thread From: Marc Hartstein @ 2008-03-28 17:23 UTC (permalink / raw) Excerpts from Marcus Williams's message of Fri Mar 28 06:33:45 -0400 2008: > On 28.3.2008, Marc Hartstein wrote: > > > 5. Can sup be set to automatically poll local sources for new messages > > every n seconds? Mutt does this, and it's nice not having to tell it > > that I know there are new messages and it should go find them. > > It should do already. I dont know offhand what the delay is though. So it does, I think it's just less frequent than I'm accustomed to. This should probably go on the list of things which should eventually be configurable. > > 6. Mutt can pass text/html attachments through an external program and > > display the result in its internal pager. Is there a way to get sup to > > do the equivalent? > > Theres more than one way to do this. Via a hook for auto-demangling > (mime-decode.rb) Thanks, that's exactly what I was looking for. And should be nicely extensible if I ever want. > and via the pipe attachment/message keypress '|' in thread view mode. > Basically if you want to view an attachment put your cursor line over > the attachment line and hit '|'. sup will pipe the attachment to what > ever command line you type in here. Probably too much effort unless it can consult mailcap or something and fill in an appropriate default. I wonder how difficult that would be to add.... > > 7. How does sup choose the email address to use as "From" when replying > > to an email? It's definitely choosing the wrong thing in my setup. My > > primary address forwards to gmail, sup is configured with the primary as > > my :email:, but whenever I hit reply it sets "From" to be my gmail > > address. I'd like it to be whatever alternate of mine is in To/CC if > > any, and my primary address otherwise. > > The :alternates: setting is probably what you want here. I think you > should just have to add your gmail address as an alternate in your > default account. At least this is how I think it should work! > Basically the alternate emails list tell sup you receive email at this > account as these addresses as well, but that you reply using the > default :email: setting. Nope. The gmail address is an alternate, the primary is the :email:, it tries to reply with the gmail address. But only sometimes. I still haven't figured out what's causing it. > You may also be interested in the :regexen: list - this allows you to > setup mail extensions (like marcus-sup that I use) and get sup to > reply using the extension. I'll look into this, thanks. > > 8. Is there any way to specify custom keybindings in a configuration > > file, or would it require hacking the source? > > Hack the source afaik Feh, hope that's on the planned features list. > > 9. Is there a way to default to making all messages PGP-signed? > > I dont think so at the moment (again, it wouldnt be hard to add) Should probably be a config setting. > Theres certainly a before-edit hook, which you may be able to do > things like this in. I havnt played with the pgp stuff yet so someone > else probably knows more here. That one's definitely not on the wiki. I'll have to track down the canonical list of available hooks... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://rubyforge.org/pipermail/sup-talk/attachments/20080328/7a51264e/attachment.bin ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] New User Questions 2008-03-28 17:23 ` Marc Hartstein @ 2008-03-28 19:51 ` Marcus Williams 2008-03-29 7:57 ` [sup-talk] [PATCH] polling is now done per source Christopher Warrington 1 sibling, 0 replies; 19+ messages in thread From: Marcus Williams @ 2008-03-28 19:51 UTC (permalink / raw) On 28.3.2008, Marc Hartstein wrote: > > Theres certainly a before-edit hook, which you may be able to do > > things like this in. I havnt played with the pgp stuff yet so someone > > else probably knows more here. > > That one's definitely not on the wiki. I'll have to track down the > canonical list of available hooks... sup --list-hooks :) Marcus ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] [PATCH] polling is now done per source 2008-03-28 17:23 ` Marc Hartstein 2008-03-28 19:51 ` Marcus Williams @ 2008-03-29 7:57 ` Christopher Warrington 2008-03-29 9:59 ` Christopher Warrington 1 sibling, 1 reply; 19+ messages in thread From: Christopher Warrington @ 2008-03-29 7:57 UTC (permalink / raw) Each source has a poll_interval property. This property is used to determine whether the source should be polled. The user can still for a poll of all sources. --- bin/sup | 4 +- bin/sup-add | 12 ++- lib/sup.rb | 4 +- lib/sup/imap.rb | 6 +- lib/sup/maildir.rb | 6 +- lib/sup/mbox/loader.rb | 4 +- lib/sup/modes/poll-mode.rb | 4 +- lib/sup/poll.rb | 211 +++++++++++++++++++++++++++++++++----------- lib/sup/source.rb | 8 +- 9 files changed, 189 insertions(+), 70 deletions(-) mode change 100755 => 100644 lib/sup/buffer.rb diff --git a/bin/sup b/bin/sup index 84fd77c..86a2d9f 100644 --- a/bin/sup +++ b/bin/sup @@ -197,7 +197,7 @@ begin end end unless $opts[:no_initial_poll] - imode.load_threads :num => ibuf.content_height, :when_done => lambda { reporting_thread("poll after loading inbox") { sleep 1; PollManager.poll } unless $opts[:no_threads] || $opts[:no_initial_poll] } + imode.load_threads :num => ibuf.content_height, :when_done => lambda { reporting_thread("poll after loading inbox") { sleep 1; PollManager.auto_poll } unless $opts[:no_threads] || $opts[:no_initial_poll] } if $opts[:compose] ComposeMode.spawn_nicely :to_default => $opts[:compose] @@ -263,7 +263,7 @@ begin when :compose ComposeMode.spawn_nicely when :poll - reporting_thread("user-invoked poll") { PollManager.poll } + reporting_thread("user-invoked poll") { PollManager.forced_poll } when :recall_draft case Index.num_results_for :label => :draft when 0 diff --git a/bin/sup-add b/bin/sup-add index 50bbb29..88705e1 100644 --- a/bin/sup-add +++ b/bin/sup-add @@ -39,6 +39,7 @@ EOS opt :unusual, "Do not automatically poll these sources for new messages." opt :labels, "A comma-separated set of labels to apply to all messages from this source", :type => String opt :force_new, "Create a new account for this source, even if one already exists." + opt :poll_interval, "The interval (in seconds) between new message polls. The default is #{Redwood::DEFAULT_POLL_INTERVAL}.", :type => :int end Trollop::die "require one or more sources" if ARGV.empty? @@ -84,6 +85,9 @@ index.lock_or_die begin index.load_sources + Trollop::die "The poll interval must be a positive integer." if $opts[:poll_interval] <= 0 + poll_interval = $opts[:poll_interval] || Redwood::DEFAULT_POLL_INTERVAL + ARGV.each do |uri| labels = $opts[:labels] ? $opts[:labels].split(/\s*,\s*/).uniq : [] @@ -100,14 +104,14 @@ begin say "For SSH connections, if you will use public key authentication, you may leave the username and password blank." say "" username, password = get_login_info uri, index.sources - Redwood::MBox::SSHLoader.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, labels + Redwood::MBox::SSHLoader.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels when "imap", "imaps" username, password = get_login_info uri, index.sources - Redwood::IMAP.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, labels + Redwood::IMAP.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels when "maildir" - Redwood::Maildir.new uri, nil, !$opts[:unusual], $opts[:archive], nil, labels + Redwood::Maildir.new uri, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels when "mbox" - Redwood::MBox::Loader.new uri, nil, !$opts[:unusual], $opts[:archive], nil, labels + Redwood::MBox::Loader.new uri, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels when nil Trollop::die "Sources must be specified with an URI" else diff --git a/lib/sup.rb b/lib/sup.rb index 1946f3c..c27a4bc 100644 --- a/lib/sup.rb +++ b/lib/sup.rb @@ -50,6 +50,8 @@ module Redwood YAML_DOMAIN = "masanjin.net" YAML_DATE = "2006-10-01" + DEFAULT_POLL_INTERVAL = 300 + ## record exceptions thrown in threads nicely def reporting_thread name if $opts[:no_threads] @@ -72,7 +74,7 @@ module Redwood def save_yaml_obj object, fn, safe=false if safe safe_fn = "#{File.dirname fn}/safe_#{File.basename fn}" - mode = File.stat(fn) if File.exists? fn + mode = File.stat(fn).mode if File.exists? fn File.open(safe_fn, "w", mode) { |f| f.puts object.to_yaml } FileUtils.mv safe_fn, fn else diff --git a/lib/sup/buffer.rb b/lib/sup/buffer.rb old mode 100755 new mode 100644 diff --git a/lib/sup/imap.rb b/lib/sup/imap.rb index 1d36976..8b58cba 100644 --- a/lib/sup/imap.rb +++ b/lib/sup/imap.rb @@ -51,13 +51,13 @@ class IMAP < Source attr_accessor :username, :password yaml_properties :uri, :username, :password, :cur_offset, :usual, - :archived, :id, :labels + :archived, :id, :poll_interval, :labels - def initialize uri, username, password, last_idate=nil, usual=true, archived=false, id=nil, labels=[] + def initialize uri, username, password, last_idate=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[] raise ArgumentError, "username and password must be specified" unless username && password raise ArgumentError, "not an imap uri" unless uri =~ %r!imaps?://! - super uri, last_idate, usual, archived, id + super uri, last_idate, usual, archived, id, [poll_interval, SCAN_INTERVAL].max @parsed_uri = URI(uri) @username = username diff --git a/lib/sup/maildir.rb b/lib/sup/maildir.rb index 584e657..ef71e0b 100644 --- a/lib/sup/maildir.rb +++ b/lib/sup/maildir.rb @@ -12,9 +12,9 @@ class Maildir < Source SCAN_INTERVAL = 30 # seconds ## remind me never to use inheritance again. - yaml_properties :uri, :cur_offset, :usual, :archived, :id, :labels - def initialize uri, last_date=nil, usual=true, archived=false, id=nil, labels=[] - super uri, last_date, usual, archived, id + yaml_properties :uri, :cur_offset, :usual, :archived, :id, :poll_interval, :labels + def initialize uri, last_date=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[] + super uri, last_date, usual, archived, id, [poll_interval, SCAN_INTERVAL].max uri = URI(Source.expand_filesystem_uri(uri)) raise ArgumentError, "not a maildir URI" unless uri.scheme == "maildir" diff --git a/lib/sup/mbox/loader.rb b/lib/sup/mbox/loader.rb index 7fe9129..44317d5 100644 --- a/lib/sup/mbox/loader.rb +++ b/lib/sup/mbox/loader.rb @@ -9,7 +9,7 @@ class Loader < Source attr_accessor :labels ## uri_or_fp is horrific. need to refactor. - def initialize uri_or_fp, start_offset=nil, usual=true, archived=false, id=nil, labels=[] + def initialize uri_or_fp, start_offset=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[] @mutex = Mutex.new @labels = ((labels || []) - LabelManager::RESERVED_LABELS).uniq.freeze @@ -26,7 +26,7 @@ class Loader < Source @path = uri_or_fp.path end - super uri_or_fp, start_offset, usual, archived, id + super uri_or_fp, start_offset, usual, archived, id, poll_interval end def file_path; @path end diff --git a/lib/sup/modes/poll-mode.rb b/lib/sup/modes/poll-mode.rb index 5849f3e..5521cdc 100644 --- a/lib/sup/modes/poll-mode.rb +++ b/lib/sup/modes/poll-mode.rb @@ -10,11 +10,11 @@ class PollMode < LogMode self << s + "\n" end - def poll + def poll sources puts unless @new @new = false puts "Poll started at #{Time.now}" - PollManager.do_poll { |s| puts s } + PollManager.do_poll_sources(sources) { |s| puts s } end end diff --git a/lib/sup/poll.rb b/lib/sup/poll.rb index d32c893..01db8e2 100644 --- a/lib/sup/poll.rb +++ b/lib/sup/poll.rb @@ -12,13 +12,22 @@ Variables: EOS HookManager.register "before-poll", <<EOS +Executes immediately before any poll for new messages commences. +Variables: + accts: an array of source URIs that will be polled +EOS + + HookManager.register "before-acct-poll", <<EOS Executes immediately before a poll for new messages commences. -No variables. +Variables: + acct: the URI of the account being polled EOS - HookManager.register "after-poll", <<EOS -Executes immediately after a poll for new messages completes. + HookManager.register "after-acct-poll", <<EOS +Executes immediately after a poll for new messages completes. The statistics are +only for the account specified. Variables: + acct: the URI of the account being polled num: the total number of new messages added in this poll num_inbox: the number of new messages added in this poll which appear in the inbox (i.e. were not auto-archived). @@ -28,14 +37,25 @@ num_inbox_total_unread: the total number of unread messages in the inbox only those messages appearing in the inbox EOS - DELAY = 300 + HookManager.register "after-poll", <<EOS +Executes immediately after any poll for new messages completes. The statistics +are for all sources polled. +Variables: + accts: an array of source URIs that were polled + num: the total number of new messages added in this poll + num_inbox: the number of new messages added in this poll which + appear in the inbox (i.e. were not auto-archived). +num_inbox_total_unread: the total number of unread messages in the inbox + from_and_subj: an array of (from email address, subject) pairs + from_and_subj_inbox: an array of (from email address, subject) pairs for + only those messages appearing in the inbox +EOS def initialize @mutex = Mutex.new @thread = nil - @last_poll = nil @polling = false - + self.class.i_am_the_instance self end @@ -44,47 +64,113 @@ EOS b end - def poll + def start + ## Periodically calls auto_poll, effectively automatically polling + ## in the background. + ## + ## auto_poll is called more frequently than the smallest poll + ## interval. auto_poll is not guaranteed to be called in multiples + ## of poll intervals. I.e., don't rely on auto_poll to be called + ## every x seconds: when auto_poll is called assume an + ## indeterminate amount of time has passed. + @min_poll_interval = Index.usual_sources.collect{|s| s.poll_interval}.select{|pi| !pi.nil?}.min || Redwood::DEFAULT_POLL_INTERVAL + + Redwood::log "Background poll interval is #{@min_poll_interval} seconds." + + @thread = Redwood::reporting_thread("periodic poll") do + while true + Redwood::log "Sleeping for #{@min_poll_interval / 2} seconds." + + sleep @min_poll_interval / 2 + auto_poll + end + end + end + + def stop + @thread.kill if @thread + @thread = nil + end + + def auto_poll + ## This method is called by the thread spawned in start. It + ## collects the sources that should be polled + ## (source.poll_interval has expired or the source has never been + ## polled) and polls them. + ## + ## Returns an array [# of new messages, # of new messages to + ## inbox, new message subject/name array pairs, subject/name array + ## pairs loaded to inbox] + + sources_to_poll = [] + @mutex.synchronize do # William, do we need to synchronize here? + begin + sources_to_poll = Index.usual_sources.select do |source| + begin + source.last_poll.nil? || (Time.now - source.last_poll) >= source.poll_interval + end + end + end + end + + poll_sources sources_to_poll + end + + def forced_poll + ## This method is called when the user explicitly requests a poll. + ## + ## Returns an array [# of new messages, # of new messages to + ## inbox, new message subject/name array pairs, subject/name array + ## pairs loaded to inbox] + poll_sources Index.usual_sources + end + + def poll_sources sources + ## Polls the given sources. Clients of PollManager should call + ## this method, not do_poll_sources or poll_source. (Well, only + ## PollMode should call do_poll_sources) + ## + ## Returns an array [# of new messages, # of new messages to + ## inbox, new message subject/name array pairs, subject/name array + ## pairs loaded to inbox] return if @polling @polling = true - HookManager.run "before-poll" + + source_uris = sources.map{|s| s.uri} + + HookManager.run "before-acct-poll", :accts => source_uris BufferManager.flash "Polling for new messages..." - num, numi, from_and_subj, from_and_subj_inbox = buffer.mode.poll + num, numi, from_subj, from_subj_inbox = buffer.mode.poll sources if num > 0 BufferManager.flash "Loaded #{num.pluralize 'new message'}, #{numi} to inbox." else BufferManager.flash "No new messages." end - HookManager.run "after-poll", :num => num, :num_inbox => numi, :from_and_subj => from_and_subj, :from_and_subj_inbox => from_and_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] } + HookManager.run "after-poll", :accts => source_uris, :num => num, :num_inbox => numi, :from_and_subj => from_and_subj, :from_and_subj_inbox => from_and_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] } @polling = false - [num, numi] - end - - def start - @thread = Redwood::reporting_thread("periodic poll") do - while true - sleep DELAY / 2 - poll if @last_poll.nil? || (Time.now - @last_poll) >= DELAY - end - end + [num, numi, from_subj, from_subj_inbox] end - def stop - @thread.kill if @thread - @thread = nil - end + def do_poll_sources sources, &block + ## Polls each source, keeping track of vital statistics (number + ## loaded, name/e-mail pairs, &c.) about the poll results. + ## + ## We need explicit access to the block so that we can pass it to + ## poll_source. + ## + ## Returns an array [total # of new messages, total # of new + ## messages to inbox, all new message subject/name array pairs, + ## all subject/name array pairs loaded to inbox] - def do_poll total_num = total_numi = 0 - from_and_subj = [] - from_and_subj_inbox = [] + total_from_and_subj = [] + total_from_and_subj_inbox = [] @mutex.synchronize do - Index.usual_sources.each do |source| -# yield "source #{source} is done? #{source.done?} (cur_offset #{source.cur_offset} >= #{source.end_offset})" + sources.each do |source| begin yield "Loading from #{source}... " unless source.done? || source.has_errors? rescue SourceError => e @@ -93,32 +179,57 @@ EOS next end - num = 0 - numi = 0 - add_messages_from source do |m, offset, entry| - ## always preserve the labels on disk. - m.labels = entry[:label].split(/\s+/).map { |x| x.intern } if entry - yield "Found message at #{offset} with labels {#{m.labels * ', '}}" - unless entry - num += 1 - from_and_subj << [m.from.longname, m.subj] - if m.has_label?(:inbox) && ([:spam, :deleted, :killed] & m.labels).empty? - from_and_subj_inbox << [m.from.longname, m.subj] - numi += 1 - end - end - m - end - yield "Found #{num} messages, #{numi} to inbox." unless num == 0 + HookManager.run "before-acct-poll", :acct => source.uri + + BufferManager.flash "Polling #{source.uri} for new messages..." + num, numi, from_and_subj, from_and_subj_inbox = poll_source source, &block + total_num += num total_numi += numi + total_from_and_subj += from_and_subj + total_from_and_subj_inbox += from_and_subj_inbox + + HookManager.run "after-acct-poll", :acct => source.uri, :num => num, :num_inbox => numi, :from_and_subj => from_and_subj, :from_and_subj_inbox => from_and_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] } end + end + + [total_num, total_numi, total_from_and_subj, total_from_and_subj_inbox] + end - yield "Done polling; loaded #{total_num} new messages total" - @last_poll = Time.now - @polling = false + def poll_source source + ## Polls the given source for new messages. + ## + ## @mutex must be held before calling! See do_poll_sources. + ## + ## Returns an array [# of new messages, # of new messages to + ## inbox, new message subject/name array pairs, subject/name array + ## pairs loaded to inbox] + + num = 0 + numi = 0 + from_and_subj = [] + from_and_subj_inbox = [] + + add_messages_from source do |m, offset, entry| + ## always preserve the labels on disk. + m.labels = entry[:label].split(/\s+/).map { |x| x.intern } if entry + yield "Found message at #{offset} with labels {#{m.labels * ', '}}" + unless entry + num += 1 + from_and_subj << [m.from.longname, m.subj] + if m.has_label?(:inbox) && ([:spam, :deleted, :killed] & m.labels).empty? + from_and_subj_inbox << [m.from.longname, m.subj] + numi += 1 + end + end + m end - [total_num, total_numi, from_and_subj, from_and_subj_inbox] + + yield "For source #{source.uri}, found #{num} messages, #{numi} to inbox." unless num == 0 + + source.last_poll = Time.now + + [num, numi, from_and_subj, from_and_subj_inbox] end ## this is the main mechanism for adding new messages to the diff --git a/lib/sup/source.rb b/lib/sup/source.rb index 6510aae..fd8d381 100644 --- a/lib/sup/source.rb +++ b/lib/sup/source.rb @@ -62,10 +62,10 @@ class Source ## dirty? means cur_offset has changed, so the source info needs to ## be re-saved to sources.yaml. bool_reader :usual, :archived, :dirty - attr_reader :uri, :cur_offset - attr_accessor :id + attr_reader :uri, :cur_offset, :poll_interval + attr_accessor :id, :last_poll - def initialize uri, initial_offset=nil, usual=true, archived=false, id=nil + def initialize uri, initial_offset=nil, usual=true, archived=false, id=nil, poll_interval=nil raise ArgumentError, "id must be an integer: #{id.inspect}" unless id.is_a? Fixnum if id @uri = uri @@ -73,6 +73,8 @@ class Source @usual = usual @archived = archived @id = id + @poll_interval = poll_interval || Redwood::DEFAULT_POLL_INTERVAL #seconds + @last_poll = nil @dirty = false end -- 1.5.4 ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] [PATCH] polling is now done per source 2008-03-29 7:57 ` [sup-talk] [PATCH] polling is now done per source Christopher Warrington @ 2008-03-29 9:59 ` Christopher Warrington 2008-03-29 10:32 ` Christopher Warrington 0 siblings, 1 reply; 19+ messages in thread From: Christopher Warrington @ 2008-03-29 9:59 UTC (permalink / raw) Excerpts from Christopher Warrington's message of Sat Mar 29 02:57:51 -0500 2008: > Each source has a poll_interval property. This property is used to > determine whether the source should be polled. The user can still > for a poll of all sources. There is a bug in this patch. Watch this space for an updated patch. -- Christopher Warrington <chrisw at rice.edu> ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] [PATCH] polling is now done per source 2008-03-29 9:59 ` Christopher Warrington @ 2008-03-29 10:32 ` Christopher Warrington 2008-03-30 3:15 ` Christopher Warrington 2008-04-17 20:58 ` Marc Hartstein 0 siblings, 2 replies; 19+ messages in thread From: Christopher Warrington @ 2008-03-29 10:32 UTC (permalink / raw) Each source has a poll_interval property. This property is used to determine whether the source should be polled. The user can still for a poll of all sources. --- bin/sup | 4 +- bin/sup-add | 12 ++- lib/sup.rb | 4 +- lib/sup/imap.rb | 6 +- lib/sup/maildir.rb | 6 +- lib/sup/mbox/loader.rb | 4 +- lib/sup/modes/poll-mode.rb | 4 +- lib/sup/poll.rb | 211 +++++++++++++++++++++++++++++++++----------- lib/sup/source.rb | 8 +- 9 files changed, 189 insertions(+), 70 deletions(-) mode change 100755 => 100644 lib/sup/buffer.rb diff --git a/bin/sup b/bin/sup index 84fd77c..86a2d9f 100644 --- a/bin/sup +++ b/bin/sup @@ -197,7 +197,7 @@ begin end end unless $opts[:no_initial_poll] - imode.load_threads :num => ibuf.content_height, :when_done => lambda { reporting_thread("poll after loading inbox") { sleep 1; PollManager.poll } unless $opts[:no_threads] || $opts[:no_initial_poll] } + imode.load_threads :num => ibuf.content_height, :when_done => lambda { reporting_thread("poll after loading inbox") { sleep 1; PollManager.auto_poll } unless $opts[:no_threads] || $opts[:no_initial_poll] } if $opts[:compose] ComposeMode.spawn_nicely :to_default => $opts[:compose] @@ -263,7 +263,7 @@ begin when :compose ComposeMode.spawn_nicely when :poll - reporting_thread("user-invoked poll") { PollManager.poll } + reporting_thread("user-invoked poll") { PollManager.forced_poll } when :recall_draft case Index.num_results_for :label => :draft when 0 diff --git a/bin/sup-add b/bin/sup-add index 50bbb29..88705e1 100644 --- a/bin/sup-add +++ b/bin/sup-add @@ -39,6 +39,7 @@ EOS opt :unusual, "Do not automatically poll these sources for new messages." opt :labels, "A comma-separated set of labels to apply to all messages from this source", :type => String opt :force_new, "Create a new account for this source, even if one already exists." + opt :poll_interval, "The interval (in seconds) between new message polls. The default is #{Redwood::DEFAULT_POLL_INTERVAL}.", :type => :int end Trollop::die "require one or more sources" if ARGV.empty? @@ -84,6 +85,9 @@ index.lock_or_die begin index.load_sources + Trollop::die "The poll interval must be a positive integer." if $opts[:poll_interval] <= 0 + poll_interval = $opts[:poll_interval] || Redwood::DEFAULT_POLL_INTERVAL + ARGV.each do |uri| labels = $opts[:labels] ? $opts[:labels].split(/\s*,\s*/).uniq : [] @@ -100,14 +104,14 @@ begin say "For SSH connections, if you will use public key authentication, you may leave the username and password blank." say "" username, password = get_login_info uri, index.sources - Redwood::MBox::SSHLoader.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, labels + Redwood::MBox::SSHLoader.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels when "imap", "imaps" username, password = get_login_info uri, index.sources - Redwood::IMAP.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, labels + Redwood::IMAP.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels when "maildir" - Redwood::Maildir.new uri, nil, !$opts[:unusual], $opts[:archive], nil, labels + Redwood::Maildir.new uri, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels when "mbox" - Redwood::MBox::Loader.new uri, nil, !$opts[:unusual], $opts[:archive], nil, labels + Redwood::MBox::Loader.new uri, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels when nil Trollop::die "Sources must be specified with an URI" else diff --git a/lib/sup.rb b/lib/sup.rb index 1946f3c..c27a4bc 100644 --- a/lib/sup.rb +++ b/lib/sup.rb @@ -50,6 +50,8 @@ module Redwood YAML_DOMAIN = "masanjin.net" YAML_DATE = "2006-10-01" + DEFAULT_POLL_INTERVAL = 300 + ## record exceptions thrown in threads nicely def reporting_thread name if $opts[:no_threads] @@ -72,7 +74,7 @@ module Redwood def save_yaml_obj object, fn, safe=false if safe safe_fn = "#{File.dirname fn}/safe_#{File.basename fn}" - mode = File.stat(fn) if File.exists? fn + mode = File.stat(fn).mode if File.exists? fn File.open(safe_fn, "w", mode) { |f| f.puts object.to_yaml } FileUtils.mv safe_fn, fn else diff --git a/lib/sup/buffer.rb b/lib/sup/buffer.rb old mode 100755 new mode 100644 diff --git a/lib/sup/imap.rb b/lib/sup/imap.rb index 1d36976..8b58cba 100644 --- a/lib/sup/imap.rb +++ b/lib/sup/imap.rb @@ -51,13 +51,13 @@ class IMAP < Source attr_accessor :username, :password yaml_properties :uri, :username, :password, :cur_offset, :usual, - :archived, :id, :labels + :archived, :id, :poll_interval, :labels - def initialize uri, username, password, last_idate=nil, usual=true, archived=false, id=nil, labels=[] + def initialize uri, username, password, last_idate=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[] raise ArgumentError, "username and password must be specified" unless username && password raise ArgumentError, "not an imap uri" unless uri =~ %r!imaps?://! - super uri, last_idate, usual, archived, id + super uri, last_idate, usual, archived, id, [poll_interval, SCAN_INTERVAL].max @parsed_uri = URI(uri) @username = username diff --git a/lib/sup/maildir.rb b/lib/sup/maildir.rb index 584e657..ef71e0b 100644 --- a/lib/sup/maildir.rb +++ b/lib/sup/maildir.rb @@ -12,9 +12,9 @@ class Maildir < Source SCAN_INTERVAL = 30 # seconds ## remind me never to use inheritance again. - yaml_properties :uri, :cur_offset, :usual, :archived, :id, :labels - def initialize uri, last_date=nil, usual=true, archived=false, id=nil, labels=[] - super uri, last_date, usual, archived, id + yaml_properties :uri, :cur_offset, :usual, :archived, :id, :poll_interval, :labels + def initialize uri, last_date=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[] + super uri, last_date, usual, archived, id, [poll_interval, SCAN_INTERVAL].max uri = URI(Source.expand_filesystem_uri(uri)) raise ArgumentError, "not a maildir URI" unless uri.scheme == "maildir" diff --git a/lib/sup/mbox/loader.rb b/lib/sup/mbox/loader.rb index 7fe9129..44317d5 100644 --- a/lib/sup/mbox/loader.rb +++ b/lib/sup/mbox/loader.rb @@ -9,7 +9,7 @@ class Loader < Source attr_accessor :labels ## uri_or_fp is horrific. need to refactor. - def initialize uri_or_fp, start_offset=nil, usual=true, archived=false, id=nil, labels=[] + def initialize uri_or_fp, start_offset=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[] @mutex = Mutex.new @labels = ((labels || []) - LabelManager::RESERVED_LABELS).uniq.freeze @@ -26,7 +26,7 @@ class Loader < Source @path = uri_or_fp.path end - super uri_or_fp, start_offset, usual, archived, id + super uri_or_fp, start_offset, usual, archived, id, poll_interval end def file_path; @path end diff --git a/lib/sup/modes/poll-mode.rb b/lib/sup/modes/poll-mode.rb index 5849f3e..5521cdc 100644 --- a/lib/sup/modes/poll-mode.rb +++ b/lib/sup/modes/poll-mode.rb @@ -10,11 +10,11 @@ class PollMode < LogMode self << s + "\n" end - def poll + def poll sources puts unless @new @new = false puts "Poll started at #{Time.now}" - PollManager.do_poll { |s| puts s } + PollManager.do_poll_sources(sources) { |s| puts s } end end diff --git a/lib/sup/poll.rb b/lib/sup/poll.rb index d32c893..1ff8014 100644 --- a/lib/sup/poll.rb +++ b/lib/sup/poll.rb @@ -12,13 +12,22 @@ Variables: EOS HookManager.register "before-poll", <<EOS +Executes immediately before any poll for new messages commences. +Variables: + accts: an array of source URIs that will be polled +EOS + + HookManager.register "before-acct-poll", <<EOS Executes immediately before a poll for new messages commences. -No variables. +Variables: + acct: the URI of the account being polled EOS - HookManager.register "after-poll", <<EOS -Executes immediately after a poll for new messages completes. + HookManager.register "after-acct-poll", <<EOS +Executes immediately after a poll for new messages completes. The statistics are +only for the account specified. Variables: + acct: the URI of the account being polled num: the total number of new messages added in this poll num_inbox: the number of new messages added in this poll which appear in the inbox (i.e. were not auto-archived). @@ -28,14 +37,25 @@ num_inbox_total_unread: the total number of unread messages in the inbox only those messages appearing in the inbox EOS - DELAY = 300 + HookManager.register "after-poll", <<EOS +Executes immediately after any poll for new messages completes. The statistics +are for all sources polled. +Variables: + accts: an array of source URIs that were polled + num: the total number of new messages added in this poll + num_inbox: the number of new messages added in this poll which + appear in the inbox (i.e. were not auto-archived). +num_inbox_total_unread: the total number of unread messages in the inbox + from_and_subj: an array of (from email address, subject) pairs + from_and_subj_inbox: an array of (from email address, subject) pairs for + only those messages appearing in the inbox +EOS def initialize @mutex = Mutex.new @thread = nil - @last_poll = nil @polling = false - + self.class.i_am_the_instance self end @@ -44,47 +64,113 @@ EOS b end - def poll + def start + ## Periodically calls auto_poll, effectively automatically polling + ## in the background. + ## + ## auto_poll is called more frequently than the smallest poll + ## interval. auto_poll is not guaranteed to be called in multiples + ## of poll intervals. I.e., don't rely on auto_poll to be called + ## every x seconds: when auto_poll is called assume an + ## indeterminate amount of time has passed. + @min_poll_interval = Index.usual_sources.collect{|s| s.poll_interval}.select{|pi| !pi.nil?}.min || Redwood::DEFAULT_POLL_INTERVAL + + Redwood::log "Background poll interval is #{@min_poll_interval} seconds." + + @thread = Redwood::reporting_thread("periodic poll") do + while true + Redwood::log "Sleeping for #{@min_poll_interval / 2} seconds." + + sleep @min_poll_interval / 2 + auto_poll + end + end + end + + def stop + @thread.kill if @thread + @thread = nil + end + + def auto_poll + ## This method is called by the thread spawned in start. It + ## collects the sources that should be polled + ## (source.poll_interval has expired or the source has never been + ## polled) and polls them. + ## + ## Returns an array [# of new messages, # of new messages to + ## inbox, new message subject/name array pairs, subject/name array + ## pairs loaded to inbox] + + sources_to_poll = [] + @mutex.synchronize do # William, do we need to synchronize here? + begin + sources_to_poll = Index.usual_sources.select do |source| + begin + source.last_poll.nil? || (Time.now - source.last_poll) >= source.poll_interval + end + end + end + end + + poll_sources sources_to_poll + end + + def forced_poll + ## This method is called when the user explicitly requests a poll. + ## + ## Returns an array [# of new messages, # of new messages to + ## inbox, new message subject/name array pairs, subject/name array + ## pairs loaded to inbox] + poll_sources Index.usual_sources + end + + def poll_sources sources + ## Polls the given sources. Clients of PollManager should call + ## this method, not do_poll_sources or poll_source. (Well, only + ## PollMode should call do_poll_sources) + ## + ## Returns an array [# of new messages, # of new messages to + ## inbox, new message subject/name array pairs, subject/name array + ## pairs loaded to inbox] return if @polling @polling = true - HookManager.run "before-poll" + + source_uris = sources.map{|s| s.uri} + + HookManager.run "before-acct-poll", :accts => source_uris BufferManager.flash "Polling for new messages..." - num, numi, from_and_subj, from_and_subj_inbox = buffer.mode.poll + num, numi, from_subj, from_subj_inbox = buffer.mode.poll sources if num > 0 BufferManager.flash "Loaded #{num.pluralize 'new message'}, #{numi} to inbox." else BufferManager.flash "No new messages." end - HookManager.run "after-poll", :num => num, :num_inbox => numi, :from_and_subj => from_and_subj, :from_and_subj_inbox => from_and_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] } + HookManager.run "after-poll", :accts => source_uris, :num => num, :num_inbox => numi, :from_and_subj => from_subj, :from_and_subj_inbox => from_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] } @polling = false - [num, numi] - end - - def start - @thread = Redwood::reporting_thread("periodic poll") do - while true - sleep DELAY / 2 - poll if @last_poll.nil? || (Time.now - @last_poll) >= DELAY - end - end + [num, numi, from_subj, from_subj_inbox] end - def stop - @thread.kill if @thread - @thread = nil - end + def do_poll_sources sources, &block + ## Polls each source, keeping track of vital statistics (number + ## loaded, name/e-mail pairs, &c.) about the poll results. + ## + ## We need explicit access to the block so that we can pass it to + ## poll_source. + ## + ## Returns an array [total # of new messages, total # of new + ## messages to inbox, all new message subject/name array pairs, + ## all subject/name array pairs loaded to inbox] - def do_poll total_num = total_numi = 0 - from_and_subj = [] - from_and_subj_inbox = [] + total_from_and_subj = [] + total_from_and_subj_inbox = [] @mutex.synchronize do - Index.usual_sources.each do |source| -# yield "source #{source} is done? #{source.done?} (cur_offset #{source.cur_offset} >= #{source.end_offset})" + sources.each do |source| begin yield "Loading from #{source}... " unless source.done? || source.has_errors? rescue SourceError => e @@ -93,32 +179,57 @@ EOS next end - num = 0 - numi = 0 - add_messages_from source do |m, offset, entry| - ## always preserve the labels on disk. - m.labels = entry[:label].split(/\s+/).map { |x| x.intern } if entry - yield "Found message at #{offset} with labels {#{m.labels * ', '}}" - unless entry - num += 1 - from_and_subj << [m.from.longname, m.subj] - if m.has_label?(:inbox) && ([:spam, :deleted, :killed] & m.labels).empty? - from_and_subj_inbox << [m.from.longname, m.subj] - numi += 1 - end - end - m - end - yield "Found #{num} messages, #{numi} to inbox." unless num == 0 + HookManager.run "before-acct-poll", :acct => source.uri + + BufferManager.flash "Polling #{source.uri} for new messages..." + num, numi, from_and_subj, from_and_subj_inbox = poll_source source, &block + total_num += num total_numi += numi + total_from_and_subj += from_and_subj + total_from_and_subj_inbox += from_and_subj_inbox + + HookManager.run "after-acct-poll", :acct => source.uri, :num => num, :num_inbox => numi, :from_and_subj => from_and_subj, :from_and_subj_inbox => from_and_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] } end + end + + [total_num, total_numi, total_from_and_subj, total_from_and_subj_inbox] + end - yield "Done polling; loaded #{total_num} new messages total" - @last_poll = Time.now - @polling = false + def poll_source source + ## Polls the given source for new messages. + ## + ## @mutex must be held before calling! See do_poll_sources. + ## + ## Returns an array [# of new messages, # of new messages to + ## inbox, new message subject/name array pairs, subject/name array + ## pairs loaded to inbox] + + num = 0 + numi = 0 + from_and_subj = [] + from_and_subj_inbox = [] + + add_messages_from source do |m, offset, entry| + ## always preserve the labels on disk. + m.labels = entry[:label].split(/\s+/).map { |x| x.intern } if entry + yield "Found message at #{offset} with labels {#{m.labels * ', '}}" + unless entry + num += 1 + from_and_subj << [m.from.longname, m.subj] + if m.has_label?(:inbox) && ([:spam, :deleted, :killed] & m.labels).empty? + from_and_subj_inbox << [m.from.longname, m.subj] + numi += 1 + end + end + m end - [total_num, total_numi, from_and_subj, from_and_subj_inbox] + + yield "For source #{source.uri}, found #{num} messages, #{numi} to inbox." unless num == 0 + + source.last_poll = Time.now + + [num, numi, from_and_subj, from_and_subj_inbox] end ## this is the main mechanism for adding new messages to the diff --git a/lib/sup/source.rb b/lib/sup/source.rb index 6510aae..fd8d381 100644 --- a/lib/sup/source.rb +++ b/lib/sup/source.rb @@ -62,10 +62,10 @@ class Source ## dirty? means cur_offset has changed, so the source info needs to ## be re-saved to sources.yaml. bool_reader :usual, :archived, :dirty - attr_reader :uri, :cur_offset - attr_accessor :id + attr_reader :uri, :cur_offset, :poll_interval + attr_accessor :id, :last_poll - def initialize uri, initial_offset=nil, usual=true, archived=false, id=nil + def initialize uri, initial_offset=nil, usual=true, archived=false, id=nil, poll_interval=nil raise ArgumentError, "id must be an integer: #{id.inspect}" unless id.is_a? Fixnum if id @uri = uri @@ -73,6 +73,8 @@ class Source @usual = usual @archived = archived @id = id + @poll_interval = poll_interval || Redwood::DEFAULT_POLL_INTERVAL #seconds + @last_poll = nil @dirty = false end -- 1.5.4 ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] [PATCH] polling is now done per source 2008-03-29 10:32 ` Christopher Warrington @ 2008-03-30 3:15 ` Christopher Warrington 2008-04-17 20:58 ` Marc Hartstein 1 sibling, 0 replies; 19+ messages in thread From: Christopher Warrington @ 2008-03-30 3:15 UTC (permalink / raw) Excerpts from Christopher Warrington's message of Sat Mar 29 05:32:05 -0500 2008: > Each source has a poll_interval property. This property is used to > determine whether the source should be polled. The user can still > for a poll of all sources. This is the fixed patch. I've been running it all day now with Maildir sources and it seems to be working. I'd love some feedback, especially from those who use mbox or IMAP. -- Christopher Warrington <chrisw at rice.edu> ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] [PATCH] polling is now done per source 2008-03-29 10:32 ` Christopher Warrington 2008-03-30 3:15 ` Christopher Warrington @ 2008-04-17 20:58 ` Marc Hartstein 2008-04-17 22:24 ` Christopher Warrington 1 sibling, 1 reply; 19+ messages in thread From: Marc Hartstein @ 2008-04-17 20:58 UTC (permalink / raw) Excerpts from 's message of Sat Mar 29 06:32:05 -0400 2008: > Each source has a poll_interval property. This property is used to > determine whether the source should be polled. The user can still for a > poll of all sources. Thanks for the patch. Seems to work great. Only one problem. When this patch is applied and then sup is run against an old sources.yaml (which doesn't provide the poll_interval property), sup will crash on startup. (Unfortunately, running git master, say to test a patch in development, will clobber sources.yaml and remove this property, so I keep hitting this problem.) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://rubyforge.org/pipermail/sup-talk/attachments/20080417/6519cca4/attachment.bin ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] [PATCH] polling is now done per source 2008-04-17 20:58 ` Marc Hartstein @ 2008-04-17 22:24 ` Christopher Warrington 0 siblings, 0 replies; 19+ messages in thread From: Christopher Warrington @ 2008-04-17 22:24 UTC (permalink / raw) Excerpts from Marc Hartstein's message of Thu Apr 17 15:58:29 -0500 2008: >> Each source has a poll_interval property. This property is used to >> determine whether the source should be polled. The user can still >> for a poll of all sources. > Thanks for the patch. Seems to work great. Only one problem. Well, there is another. The before-poll hook doesn't get the correct arguments and before-acct-poll is run twice. I've fixed this locally and will post a patch to the patch (or a unified path) once I figure out how to do that in git. > When this patch is applied and then sup is run against an old > sources.yaml (which doesn't provide the poll_interval property), sup > will crash on startup. Hmmm... I don't know enough about the yaml infrastructure to fix this. Does anyone else know if there is a way to specify a default value when one if not found in the yaml file. This would allow for graceful upgrading and (in theory) keep the sources.yaml file backward compatible. -- Christopher Warrington <chrisw at rice.edu> ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] Different Key Bindings (was: New User Questions) 2008-03-28 10:33 ` Marcus Williams 2008-03-28 11:24 ` vasudeva 2008-03-28 17:23 ` Marc Hartstein @ 2008-03-29 16:41 ` Guarded Identity 2 siblings, 0 replies; 19+ messages in thread From: Guarded Identity @ 2008-03-29 16:41 UTC (permalink / raw) Excerpts from Marcus Williams marcus-sup-at-bar-coded.net |Sup_Talk|'s message of Fri Mar 28 05:33:45 > > 8. Is there any way to specify custom keybindings in a configuration > > file, or would it require hacking the source? > > Hack the source afaik I know I'd /definitely/ be interested in this kind of thing. For the most part, I'm really happy with current key bindings, but there's a few that bug me. For instance. I wish 'l' and 'L' were reversed. My rationale is that I don't like the unshifted characters to do persistent things (just in-case I hit the wrong key). Maybe it's just me, but I hit 'l' sometimes, briefly believe that Sup is hung, and then realize that I'm editing labels, and then Ctrl-G'd out before accidentally committing the junk labels. -Sukant ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] New user questions... @ 2009-02-26 17:33 Vadim Gutnik 2009-03-22 17:30 ` William Morgan 0 siblings, 1 reply; 19+ messages in thread From: Vadim Gutnik @ 2009-02-26 17:33 UTC (permalink / raw) I built sup on my ubuntu system as described on the wiki; no problems, though I'll likely switch to the packaged version as it becomes available. I tried using mbox format, ran into the From: bug (I think -- out of a couple thousand emails, maybe a dozen forwarded messages were split from their emails; still, this isn't tolerable. Is there a plan to make sup recognize email breaks at least as well as mutt does?), so I'm using mdir format, which doesn't suffer from it. Two specific questions, though: 1) I use getmail to get my POP3SSL mail and put it into mdir format. Usually, when getmail tells me it found a message, I go to my running sup process and press "P" for it to pick up the mail, and sup shows the new message. Sometimes (maybe 5-10% of the time?) sup does not find the new message until I quit and restart it. Known problem? Is there anything I can do to help debug it? 2) At least once or twice I've hit "d" by habit (and mistake). Is there a way to search for "deleted" messages? I could just rebuild the index from scratch and reapply labels if I had to, but ...there must be an easier way, right? Thanks. Vadim ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] New user questions... 2009-02-26 17:33 [sup-talk] New user questions Vadim Gutnik @ 2009-03-22 17:30 ` William Morgan 2009-03-22 17:56 ` John Bent 0 siblings, 1 reply; 19+ messages in thread From: William Morgan @ 2009-03-22 17:30 UTC (permalink / raw) Hi Vadim, Better late than never, right? Reformatted excerpts from Vadim Gutnik's message of 2009-02-26: > Is there a plan to make sup recognize email breaks at least as well as > mutt does?), Yes, absolutely. This is a high-priority bug. > 1) I use getmail to get my POP3SSL mail and put it into mdir format. > Usually, when getmail tells me it found a message, I go to my running > sup process and press "P" for it to pick up the mail, and sup shows > the new message. Sometimes (maybe 5-10% of the time?) sup does not > find the new message until I quit and restart it. Known problem? Interesting. Someone's recently reported a similar problem, also with Maildir, but in their case the message wasn't added to the index at all. > Is there anything I can do to help debug it? If there's any way to reliably reproduce this so that I can see it happening locally, of course, that's the ideal solution. Otherwise, if you notice it happening in the wild, you can also look at the log buffer and the poll buffer and see if there's anything obviously wrong about what Sup reports. > 2) At least once or twice I've hit "d" by habit (and mistake). Is > there a way to search for "deleted" messages? I could > just rebuild the index from scratch and reapply labels if I had to, > but ...there must be an easier way, right? You can search for deleted messages by using "\" (label:deleted), or by using L. -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] New user questions... 2009-03-22 17:30 ` William Morgan @ 2009-03-22 17:56 ` John Bent 2009-03-22 19:22 ` William Morgan 0 siblings, 1 reply; 19+ messages in thread From: John Bent @ 2009-03-22 17:56 UTC (permalink / raw) Excerpts from William Morgan's message of Sun Mar 22 11:30:30 -0600 2009: > Hi Vadim, > > Better late than never, right? > > Reformatted excerpts from Vadim Gutnik's message of 2009-02-26: > > Is there a plan to make sup recognize email breaks at least as well as > > mutt does?), > > Yes, absolutely. This is a high-priority bug. > > > 1) I use getmail to get my POP3SSL mail and put it into mdir format. > > Usually, when getmail tells me it found a message, I go to my running > > sup process and press "P" for it to pick up the mail, and sup shows > > the new message. Sometimes (maybe 5-10% of the time?) sup does not > > find the new message until I quit and restart it. Known problem? > > Interesting. Someone's recently reported a similar problem, also with > Maildir, but in their case the message wasn't added to the index at all. > > > Is there anything I can do to help debug it? > > If there's any way to reliably reproduce this so that I can see it > happening locally, of course, that's the ideal solution. Otherwise, if > you notice it happening in the wild, you can also look at the log buffer > and the poll buffer and see if there's anything obviously wrong about > what Sup reports. > > > 2) At least once or twice I've hit "d" by habit (and mistake). Is > > there a way to search for "deleted" messages? I could > > just rebuild the index from scratch and reapply labels if I had to, > > but ...there must be an easier way, right? > > You can search for deleted messages by using "\" (label:deleted), or by > using L. > Also, you can hit '@' to revert to your last saved point (i.e. last time you hit '$'), and this will undelete it as well as undoing all your other unsaved changes. Also, hasn't someone recently submitted an Undo patch? ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] New user questions... 2009-03-22 17:56 ` John Bent @ 2009-03-22 19:22 ` William Morgan 0 siblings, 0 replies; 19+ messages in thread From: William Morgan @ 2009-03-22 19:22 UTC (permalink / raw) Reformatted excerpts from John Bent's message of 2009-03-22: > Also, you can hit '@' to revert to your last saved point (i.e. last > time you hit '$'), and this will undelete it as well as undoing all > your other unsaved changes. Also, hasn't someone recently submitted > an Undo patch? Oh right. If you check out the "next" branch in git, you'll have undo power too. -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] New User Questions
@ 2008-01-09 3:39 Guarded Identity
2008-01-10 6:34 ` William Morgan
0 siblings, 1 reply; 19+ messages in thread
From: Guarded Identity @ 2008-01-09 3:39 UTC (permalink / raw)
Hello everyone,
First off. William, thanks a lot for all the work. The Mutt home page
has always had this quote: "All mail clients suck. This one [mutt] just
sucks less." I know Sup has room for more features and functionality,
but it my mind, it's no where near sucking, especially with the option
of using sup-console or Ruby scripts using Redwood libraries.
Having just gotten started, I have a slew of questions. I'll try to
organize/enumerate them below to help with in-line responses.
I'm using maildirs as opposed to mboxes. I've always preferred maildirs
over mboxes to avoid dealing with gigantic mail files.
1. Can sent mail only be stored in an mbox, or are other formats
supported?
I haven't gotten around to digging deeply into Sup's source, so I'm not
sure about the details of how E-mails are indexed. I know Sup doesn't
play well with other MUAs. The FAQ discusses mboxes pretty well. I was
hoping for a little more discussion of how modification of maildir mail
by other MUAs affects Sup. For instance, the maildir specification has
mail put in $MAILDIR/new/ until read by a MUA, which should move the
mail to $MAILDIR/cur/ to mark the mail as "seen".
2. Will migration of maildir mail from new/ to cur/ necessitate an
execution of sup-sync?
Also, "read" flags (and others) are stored in the Maildir filename.
3. If the filename of maildir mail changes, is a sup-sync required?
If 2. or 3. are indeed problems, maybe I could front the maildir sources
with an IMAP server. I had done this before, actually, but I noticed
that Sup indexing was slowed down quite a bit (rss2email and mailing
lists generate a lot of new mail). If I do go with IMAP, I guess I
could sequence a sup-sync right after my crontab call to fetchmail to
help with indexing? I just wasn't keen on Sup being locked out
periodically by crontab.
Alternatively, I could move all my Maildirs into mboxes. I know Sup has
a bias towards not deleting mail, but sup-sync-back is there for mboxes.
4. How far away is sup-sync-back support for maildirs or IMAP? Is
the work straight-forward or are there some technical kinks to be
ironed out?
5. Should I consider using mboxes for any reason beyond support by
sup-sync-back?
Also, I would really like to have time-based auto-expiry (excluding
starred or special-labeled items) for some of my mail (primarily mailing
lists and rss feeds). With mboxes, I guess I could do some
"deleted"-labeling with a Ruby script followed by a call to
sup-sync-back.
6. What are my scripting options for mail expiry with Maildirs? The
search to get the message objects is pretty straight forward. How
much further work is it to delete the message from the index and
to get a filename of the message to delete from the maildir?
Also, I kind of miss urlview.
7. Any proposal for easily opening urls from messages?
urlview is already coded up, so I'm all for a mechanism for piping
messages into an external application. But this leads to other
design issues, I think, like how to enable user-level key
bindings.
I have few questions regarding hooks and threading:
8. I'm a little confused about threading interacts with labeling in
Sup. Labels are applied to entire threads, but it seems that
labels are stored on a per-mail basis. Is seems possible for
message within a thread to have different labels, say by using a
before-add-message.rb hook. Is the label set for the thread a
union of all the labels of its constituents?
9. If a new message comes into a thread, is it auto-labeled with the
labels of the thread?
10. I tried to do some auto-archiving with the before-add-message.rb
hook using message.remove_label("inbox"), but it didn't work. Is
this possible? Or is auto-archiving only possible at the source
level?
Finally. . .
11. Is it possible to search for mail that has no labels without saying
"!label:label1 AND !label:label2 AND ..."?
I looked a little in the Ferret documentation, but didn't find a
way. I was trying to do this to get at some mail I archived
accidentally before applying labels.
That's what I've got for now. I hope I didn't overwhelm the list with a
lot of stupid questions.
Thanks for your help,
Sukant
^ permalink raw reply [flat|nested] 19+ messages in thread* [sup-talk] New User Questions 2008-01-09 3:39 [sup-talk] New User Questions Guarded Identity @ 2008-01-10 6:34 ` William Morgan 2008-03-26 5:57 ` Guarded Identity 0 siblings, 1 reply; 19+ messages in thread From: William Morgan @ 2008-01-10 6:34 UTC (permalink / raw) Excerpts from Guarded Identity's message of Tue Jan 08 19:39:18 -0800 2008: > I know Sup has room for more features and functionality, but it my > mind, it's no where near sucking, especially with the option of using > sup-console or Ruby scripts using Redwood libraries. Thanks! Always nice to hear. > 1. Can sent mail only be stored in an mbox, or are other formats > supported? Right now it's mbox only. There's no technical reason it couldn't be stored in a Maildir, it's just that writing to an mbox is just the easiest possible thing in the world. > 2. Will migration of maildir mail from new/ to cur/ necessitate an > execution of sup-sync? Hm. I don't think so. As long as the mtime and size of the file are preserved (which I believe they are during a move) you should be ok. > 3. If the filename of maildir mail changes, is a sup-sync required? No, for the same reason. > If 2. or 3. are indeed problems, maybe I could front the maildir > sources with an IMAP server. I had done this before, actually, but I > noticed that Sup indexing was slowed down quite a bit (rss2email and > mailing lists generate a lot of new mail). If that's true, then it must be due to IMAP transmission costs. I'm certainly willing to believe that pulling a message from IMAP is significantly slower than a disk read. > 4. How far away is sup-sync-back support for maildirs or IMAP? Is the > work straight-forward or are there some technical kinks to be ironed > out? The actual interactions with the sources themselves should be pretty easy, because they both support deleting individual messages directly. Mbox was actually the hard one, because you have to rewrite the entire file to delete messages. But ideally sup-sync-back would be source type agnostic, so there's some work to be done in terms of adding deletability to the source API in such a manner that it handles both the mbox case and the not-mbox case. (Mbox deletions need to happen in one go, whereas not-mbox deletions can happen one at a time.) > 5. Should I consider using mboxes for any reason beyond support by > sup-sync-back? No. Mbox are a terrible, evil format that just happens to be well-supported by Sup. > Also, I would really like to have time-based auto-expiry (excluding > starred or special-labeled items) for some of my mail (primarily > mailing lists and rss feeds). With mboxes, I guess I could do some > "deleted"-labeling with a Ruby script followed by a call to > sup-sync-back. That sounds like the right approach. > 6. What are my scripting options for mail expiry with Maildirs? The > search to get the message objects is pretty straight forward. How > much further work is it to delete the message from the index and to > get a filename of the message to delete from the maildir? Trivial, although the filename/IMAP id require minor API changes to expose those functions. But with a working sup-sync-back that applies to all source types, you won't have to do anything other than inject :deleted labels everywhere and simply call sup-sync-back. > 7. Any proposal for easily opening urls from messages? > > urlview is already coded up, so I'm all for a mechanism for piping > messages into an external application. But this leads to other design > issues, I think, like how to enable user-level key bindings. I'm not familiar with urlview. If it is a console-based interactive program, it should be possible to just spawn it via the hooks. We may have to add a hook, depending on the exact usage case here. > 8. I'm a little confused about threading interacts with labeling in > Sup. Labels are applied to entire threads, but it seems that labels > are stored on a per-mail basis. Correct. But you never really get to play directly with message-level labels in Sup, except for a few little things like starring and unread status, which can be applied per message in thread-view-mode. > Is seems possible for message within a thread to have different > labels, say by using a before-add-message.rb hook. Is the label set > for the thread a union of all the labels of its constituents? Correct. > 9. If a new message comes into a thread, is it auto-labeled with the > labels of the thread? No. > 10. I tried to do some auto-archiving with the before-add-message.rb > hook using message.remove_label("inbox"), but it didn't work. Is this > possible? Or is auto-archiving only possible at the source level? That should work. If you post your hook we can take a crack at debugging. > 11. Is it possible to search for mail that has no labels without > saying "!label:label1 AND !label:label2 AND ..."? > > I looked a little in the Ferret documentation, but didn't find a way. > I was trying to do this to get at some mail I archived accidentally > before applying labels. Interesting question. I don't know of a better way. If there were one it would be in the Ferret documentation. I suppose you could use DeMorgan's law to save yourself a few characters though. :) -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 19+ messages in thread
* [sup-talk] New User Questions 2008-01-10 6:34 ` William Morgan @ 2008-03-26 5:57 ` Guarded Identity 0 siblings, 0 replies; 19+ messages in thread From: Guarded Identity @ 2008-03-26 5:57 UTC (permalink / raw) Hi, I've been using Sup for a few months now, read up a little more on Ruby, and thought I might be in more of a position to respond to this thread. Excerpts from William Morgan's message of Thu Jan 10 00:34:03 -0600 2008: > Excerpts from Guarded Identity's message of Tue Jan 08 19:39:18 -0800 2008: > > Will migration of maildir mail from new/ to cur/ necessitate an execution > > of sup-sync? > > Hm. I don't think so. As long as the mtime and size of the file are > preserved (which I believe they are during a move) you should be ok. I was thinking about the algorithm for calculating the message's unique "source_info" for Maildir mail (mtime appended to filesize). I know altering the backend might really be something you don't want to support, so maybe I'm voicing a moot concern. It's just that sometimes I get annoyed at people sending me mail with large attachments (especially at work where I just can't control this kind of thing too well). In the past, I'd use mutt to purge mail of their large attachments. However, I guess if I do this en mass, I loose my labeling of these messages when I do a sup-sync because the message's source_info changes, right?, but have you considered using a message's "Message-Id" header as part of the source_info? I'm thinking you might have. For the most part, this should be pretty unique, right? I know the Message-Id is sometimes missing, but we can default to the old source_id if things are really wacky (which should be rarely, right?). The only reason I could think you wouldn't use the Message-Id is because maybe it's less efficient to open up a message than look at it's file stats. Or maybe there's another reason? > > I had done this before, actually, but I noticed that Sup indexing was > > slowed down quite a bit (rss2email and mailing lists generate a lot of new > > mail). > > If that's true, then it must be due to IMAP transmission costs. I'm > certainly willing to believe that pulling a message from IMAP is > significantly slower than a disk read. I was thinking about this too. Is there anything we can do to pull down headers first to fill the index (backgrounding the indexing)? Maybe that might make the delay with IMAP less of a nuisance? However, I won't harp on this too much. Because I wasn't comfortable synchronizing (say with Unison) sup Ferret indexes across hosts, I'm ssh'ing into the machine with the index and using "maildir" sources. But this unfortunately causes me a problem with viewing graphical attachments because X forwarding is really slow. So for now I'm saving attachments and scp'ing them over. I'm not sure there's much anyone can do about any of this. > > I would really like to have time-based auto-expiry (excluding starred or > > special-labeled items) for some of my mail (primarily mailing lists and rss > > feeds). With mboxes, I guess I could do some "deleted"-labeling with a Ruby > > script followed by a call to sup-sync-back. > > That sounds like the right approach. > > > 6. What are my scripting options for mail expiry with Maildirs? The search > > to get the message objects is pretty straight forward. How much further > > work is it to delete the message from the index and to get a filename of > > the message to delete from the maildir? > > Trivial, although the filename/IMAP id require minor API changes to expose > those functions. But with a working sup-sync-back that applies to all source > types, you won't have to do anything other than inject :deleted labels > everywhere and simply call sup-sync-back. Okay, So I've been using a helper script that does some things to help me manage my Maildir mail (all other source types are ignored). My primary idea was to have a script to expire mail, but allowing me to save threads labeled "starred" or "save". Eventually I added in a function to manipulate labels. Unlike sup-tweak-labels, this script gathers it's pool of messages from the index using a Ferret query (Chronic allowed) rather than using source URIs. I got around some of the encapsulation inherent in the API by just brute force breaking encapsulation (which Ruby makes kind of ridiculously easy it seems). I've attached the script, and would like to see any feedback if people have some. In particular, I'm eager to know of any concerns people have with the approach or defects people catch. I'm still pretty new to Ruby, so if I did anything flat out stupid with the language, please let me know. By the way, I was thinking that there seems to be a good deal of healthy participation in this list. Does anybody see any value to hanging out in #sup on Freenode? Sometimes it's just nice to have an IRC forum. -Sukant -------------- next part -------------- A non-text attachment was scrubbed... Name: sup-tweak-maildir Type: application/octet-stream Size: 6652 bytes Desc: not available Url : http://rubyforge.org/pipermail/sup-talk/attachments/20080326/cef1c568/attachment.obj ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2009-03-22 19:22 UTC | newest] Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2008-03-28 0:22 [sup-talk] New User Questions Marc Hartstein 2008-03-28 10:33 ` Marcus Williams 2008-03-28 11:24 ` vasudeva 2008-03-28 17:23 ` Marc Hartstein 2008-03-28 19:51 ` Marcus Williams 2008-03-29 7:57 ` [sup-talk] [PATCH] polling is now done per source Christopher Warrington 2008-03-29 9:59 ` Christopher Warrington 2008-03-29 10:32 ` Christopher Warrington 2008-03-30 3:15 ` Christopher Warrington 2008-04-17 20:58 ` Marc Hartstein 2008-04-17 22:24 ` Christopher Warrington 2008-03-29 16:41 ` [sup-talk] Different Key Bindings (was: New User Questions) Guarded Identity -- strict thread matches above, loose matches on Subject: below -- 2009-02-26 17:33 [sup-talk] New user questions Vadim Gutnik 2009-03-22 17:30 ` William Morgan 2009-03-22 17:56 ` John Bent 2009-03-22 19:22 ` William Morgan 2008-01-09 3:39 [sup-talk] New User Questions Guarded Identity 2008-01-10 6:34 ` William Morgan 2008-03-26 5:57 ` Guarded Identity
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox