sup

A curses threads-with-tags style email client

sup-website.git

git clone https://supmua.dev/git/sup-website/

community/pipermail-archives/sup-talk/2008-12.txt (57161B) - raw

      1 From tocer.deng@gmail.com  Tue Dec  2 09:54:35 2008
      2 From: tocer.deng@gmail.com (tocer)
      3 Date: Tue, 02 Dec 2008 22:54:35 +0800
      4 Subject: [sup-talk] mismatch in mbox file offset 0
      5 Message-ID: <49354C2B.502@gmail.com>
      6 
      7 Hi,
      8 
      9 Thanks you guys great work. the sup is the best mail client I have seen.
     10 
     11 I met a problem. I add a source formatted mbox. and I use getmail program to get 
     12 mail, then the mail is filtered by procmail and is saved in ~/mail/vim. But it 
     13 occure error when sup start:
     14 
     15 Scanning mbox:/home/dm/mail/vim...
     16 problem getting messages from mbox:/home/dm/mail/vim: mismatch in mbox file 
     17 offset 0: "Return-Path: <yaksavage at gmail.com>\n".
     18 
     19 I can't see any mail after entering sup. In fact, there is many mails in vim mbox.
     20 
     21 what's wrong with it?
     22 
     23 
     24 
     25 From wmorgan-sup@masanjin.net  Tue Dec  2 11:17:11 2008
     26 From: wmorgan-sup@masanjin.net (William Morgan)
     27 Date: Tue, 02 Dec 2008 08:17:11 -0800
     28 Subject: [sup-talk] mismatch in mbox file offset 0
     29 In-Reply-To: <49354C2B.502@gmail.com>
     30 References: <49354C2B.502@gmail.com>
     31 Message-ID: <1228234512-sup-3309@entry>
     32 
     33 Reformatted excerpts from tocer.deng's message of 2008-12-02:
     34 > Thanks you guys great work. the sup is the best mail client I have
     35 > seen.
     36 
     37 Thanks!
     38 
     39 > Scanning mbox:/home/dm/mail/vim...
     40 > problem getting messages from mbox:/home/dm/mail/vim: mismatch in mbox file 
     41 > offset 0: "Return-Path: <yaksavage at gmail.com>\n".
     42 
     43 Is the very first line of that mbox "Return-Path: <yaksavage at gmail.com>"?
     44 If so, that's not a valid mbox.
     45 
     46 If not, you need to run "sup-sync --changed mbox:/home/dm/mail/vim" to
     47 update Sup's index. This has to be done whenever the mbox changes in any
     48 way other than adding new messages.
     49 -- 
     50 William <wmorgan-sup at masanjin.net>
     51 
     52 From tocer.deng@gmail.com  Tue Dec  2 22:04:25 2008
     53 From: tocer.deng@gmail.com (tocer)
     54 Date: Tue, 2 Dec 2008 19:04:25 -0800 (PST)
     55 Subject: [sup-talk] mismatch in mbox file offset 0
     56 In-Reply-To: <1228234512-sup-3309@entry>
     57 References: <49354C2B.502@gmail.com> <1228234512-sup-3309@entry>
     58 Message-ID: <20805870.post@talk.nabble.com>
     59 
     60 
     61 > Scanning mbox:/home/dm/mail/vim...
     62 > problem getting messages from mbox:/home/dm/mail/vim: mismatch in mbox
     63 > file 
     64 > offset 0: "Return-Path: <yaksavage at gmail.com>\n".
     65 
     66 Is the very first line of that mbox "Return-Path: <yaksavage at gmail.com>"?
     67 If so, that's not a valid mbox.
     68 
     69 
     70 
     71 No, it's a valid mbox, refer http://en.wikipedia.org/wiki/Return_path
     72 
     73 it works if I relace return_path with "from ". I'll try to define a rule to
     74 do replacement
     75 in procmailrc. But it would be good idea to dealing correctly it in sup :)
     76 
     77 And it seems that sup-sync can't run while sup is running. It means that I
     78 can't open 
     79 sup in whole day because I need sup-sync run when I get new mail if I use
     80 some mbox 
     81 as source
     82 -- 
     83 View this message in context: http://www.nabble.com/mismatch-in-mbox-file-offset-0-tp20794067p20805870.html
     84 Sent from the SUP Talk mailing list archive at Nabble.com.
     85 
     86 
     87 From chrisw@rice.edu  Tue Dec  2 22:39:40 2008
     88 From: chrisw@rice.edu (Christopher Warrington)
     89 Date: Tue, 02 Dec 2008 21:39:40 -0600
     90 Subject: [sup-talk] mismatch in mbox file offset 0
     91 In-Reply-To: <20805870.post@talk.nabble.com>
     92 Message-ID: <C55B5B9C.3675%chrisw@rice.edu>
     93 
     94 "tocer" <tocer.deng at gmail.com> @ 2008-12-2 9:04 PM:
     95 
     96 >>> Scanning mbox:/home/dm/mail/vim... problem getting messages from
     97 >>> mbox:/home/dm/mail/vim: mismatch in mbox file offset 0: "Return-Path:
     98 >>> <yaksavage at gmail.com>\n".
     99 >>>
    100 >> Is the very first line of that mbox "Return-Path: <yaksavage at gmail.com>"? If
    101 >> so, that's not a valid mbox.
    102 >>
    103 > No, it's a valid mbox, refer http://en.wikipedia.org/wiki/Return_path
    104 >
    105 > it works if I relace return_path with "from ". I'll try to define a rule to do
    106 > replacement in procmailrc. But it would be good idea to dealing correctly it
    107 > in sup :)
    108 
    109 Hmmm.... I don't think that that's a valid mbox.
    110 <http://tools.ietf.org/html/rfc4155>
    111 <http://en.wikipedia.org/wiki/Mbox>
    112 
    113 -- 
    114 Christopher Warrington <chrisw at rice.edu>
    115 Jones College
    116 
    117 
    118 
    119 From aardvark@saintaardvarkthecarpeted.com  Wed Dec  3 16:11:21 2008
    120 From: aardvark@saintaardvarkthecarpeted.com (Saint Aardvark the Carpeted)
    121 Date: Wed, 3 Dec 2008 13:11:21 -0800
    122 Subject: [sup-talk] Exception in sup-git when sending postponed email
    123 Message-ID: <20081203211121.GB13504@danconia.atlantis.saintaardvarkthecarpeted.com>
    124 
    125 Hi there -- I'm using the git version of sup, last updated earlier
    126 this week.  I was composing a message that had been postponed; I
    127 exited the editor and sent it off when sup crashed (right term?).  The
    128 contents of .sup/exception-log.txt was:
    129 
    130 
    131 ----
    132 
    133 --- Errno::ENOENT from thread: main
    134 No such file or directory - /home/hugh/.sup/drafts/^A^D
    135 /home/hugh/src/sup-git/lib/sup/draft.rb:38:in `delete'
    136 /home/hugh/src/sup-git/lib/sup/draft.rb:38:in `discard'
    137 /home/hugh/src/sup-git/lib/sup/util.rb:499:in `send'
    138 /home/hugh/src/sup-git/lib/sup/util.rb:499:in `method_missing'
    139 /home/hugh/src/sup-git/lib/sup/modes/resume-mode.rb:36:in `send_message'
    140 /home/hugh/src/sup-git/lib/sup/mode.rb:49:in `send'
    141 /home/hugh/src/sup-git/lib/sup/mode.rb:49:in `handle_input'
    142 /home/hugh/src/sup-git/lib/sup/buffer.rb:240:in `handle_input'
    143 /home/hugh/src/sup-git/bin/sup:190
    144 
    145 ----
    146 
    147 There are two files in ~/.sup/drafts...one called "8" and one called
    148 "10".  "10" appears to be an earlier version of the draft I was
    149 composing, and the timestamp is from about 10 minutes before I sent
    150 off the finished version.  
    151 
    152 The "^A^D" after the "no such file" line appears to be ctrl-a ctrl-d
    153 when I look at the file in vi; when I look using hexcump -C, it's 0x01
    154 0x04 (ie, "...drafts/[0x01][0x04]".  Not sure if that's important.
    155 
    156 Thanks very much for sup, and please let me know if you need any
    157 further info.
    158 
    159 Hugh
    160 
    161 -- 
    162 Saint Aardvark the Carpeted
    163 http://saintaardvarkthecarpeted.com
    164 Because the plural of Anecdote is Myth.
    165 -------------- next part --------------
    166 A non-text attachment was scrubbed...
    167 Name: not available
    168 Type: application/pgp-signature
    169 Size: 187 bytes
    170 Desc: not available
    171 URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081203/a34e1e93/attachment.bin>
    172 
    173 From tocer.deng@gmail.com  Thu Dec  4 21:42:38 2008
    174 From: tocer.deng@gmail.com (tocer)
    175 Date: Fri, 5 Dec 2008 10:42:38 +0800
    176 Subject: [sup-talk]  mismatch in mbox file offset 0
    177 In-Reply-To: <d5ea6e840812041839k11f062b6n5d7f5aa518c82146@mail.gmail.com>
    178 References: <49354C2B.502@gmail.com> <1228234512-sup-3309@entry>
    179 	<20805870.post@talk.nabble.com> <1228294025-sup-7941@ausone.inria.fr>
    180 	<d5ea6e840812041839k11f062b6n5d7f5aa518c82146@mail.gmail.com>
    181 Message-ID: <d5ea6e840812041842mc5838e1m91f78f88a4d9d063@mail.gmail.com>
    182 
    183 2008/12/3 Nicolas Pouillard <nicolas.pouillard at gmail.com>:
    184 > Excerpts from tocer's message of Wed Dec 03 04:04:25 +0100 2008:
    185 >>
    186 >> > Scanning mbox:/home/dm/mail/vim...
    187 >> > problem getting messages from mbox:/home/dm/mail/vim: mismatch in mbox
    188 >> > file
    189 >> > offset 0: "Return-Path: <yaksavage at gmail.com>\n".
    190 >>
    191 >> Is the very first line of that mbox "Return-Path: <yaksavage at gmail.com>"?
    192 >> If so, that's not a valid mbox.
    193 >>
    194 >>
    195 >>
    196 >> No, it's a valid mbox, refer http://en.wikipedia.org/wiki/Return_path
    197 >>
    198 >> it works if I relace return_path with "from ". I'll try to define a rule to
    199 >> do replacement
    200 >> in procmailrc. But it would be good idea to dealing correctly it in sup :)
    201 >
    202 > I think you make the confusion between the email field "From:" and the mbox
    203 > email separator "From ". "Return-Path:" is also an email field, so I think
    204 > your email is not properly wrapped to be in an mbox.
    205 
    206 No, I don't. Now I solve it by adding the following lines into
    207 .procmailrc's header
    208 
    209 :0 fhw
    210 | formail -I "From " -a "From "
    211 
    212 I think it's helpful if someone use procmail as MTA
    213 
    214 >> And it seems that sup-sync can't run while sup is running. It means that I
    215 >> can't open
    216 >> sup in whole day because I need sup-sync run when I get new mail if I use
    217 >> some mbox
    218 >> as source
    219 >
    220 > Yes there is a lock on the index to avoid data-corruption.
    221 >
    222 > --
    223 > Nicolas Pouillard aka Ertai
    224 >
    225 
    226 From daniel@wagner-home.com  Mon Dec  8 00:14:39 2008
    227 From: daniel@wagner-home.com (Daniel Wagner)
    228 Date: Mon, 08 Dec 2008 00:14:39 -0500
    229 Subject: [sup-talk] migrating email to a new computer
    230 In-Reply-To: <1227743600-sup-8882@entry>
    231 References: <1227135030-sup-401@buckwheat> <1227743600-sup-8882@entry>
    232 Message-ID: <1228710381-sup-9800@buckwheat>
    233 
    234 Excerpts from William Morgan's message of Wed Nov 26 19:05:13 -0500 2008:
    235 > I *think* that just moving files from cur/ to cur/ and new/ to new/ will
    236 > work, but moving from new/ to cur/ I think typically changes the
    237 > filename slightly. I don't know if the rename is required or optional
    238 > though. Sup certainly doesn't care, but other tools might.
    239 
    240 As you said, sup didn't care a jot.  Lovely!  Since I'm not using any
    241 other tools for my mail, this is just what I will do.
    242 
    243 > to rebuild your index from scratch. You can use sup-dump to dump out
    244 > state information into a huge text file, and sup-sync --restored
    245 > --restore should be able to make use of that information to keep your
    246 > message state once your new sources have been created.
    247 > 
    248 > But definitely, DEFINITELY test it out first. I haven't tried that stuff
    249 > for a while and it might have experienced bitrot.
    250 
    251 Of *course* I'm going to test first.  I don't trust software any more
    252 since I've seen how it's made. =)
    253 
    254 > That sounds great. Let us know how it goes, and I'll do my best to help
    255 > you though the process if things break. Just be sure to keep a backup of
    256 > everything. Message state is a precious asset.
    257 
    258 I have a project due tomorrow, so naturally I figured I'd spend a few
    259 hours playing with this stuff tonight.  For the most part, I've been
    260 very pleased -- things have gone relatively smoothly compared to my
    261 expectations!
    262 
    263 I sup-dumped, copied all the files in oldmachine/Maildir/**/cur and
    264 oldmachine/Maildir/**/new to newmachine/Maildir/cur, started up sup-sync
    265 --restored --restore, then went and made dinner.  Now for some numbers:
    266 
    267 The Numbers According To sup-sync
    268 entries in the dump file    26462
    269 entries scanned             24779
    270 entries added               24373
    271 warnings                       27
    272 fakes                          75
    273 
    274 The Numbers According To sup
    275                             old machine     new machine
    276 messages in the index       26459           24416
    277 messages with +inbox           50              45
    278 messages with +Starred         67              57
    279 
    280 
    281 The warnings took the form
    282 warning: error (Iconv::InvalidEncoding) decoding message body from XXX: invalid encoding ("utf8//IGNORE", "XXX")
    283 where "XXX" was "unicode-1-1utf7" 17 times, "unknown-8bit" 9 times, and
    284 "X-UNKNOWN" 1 time.  It also faked message-ids 68 times, from headers 3
    285 times, and date headers 4 times.
    286 
    287 So, I'm not sure I understand how to reconcile these numbers.  Things
    288 are mismatched in odd ways; for example, how could there be more entries
    289 in the dump than there are messages in the index?  Or how come "entries
    290 added" + "warnings" != "messages in the index"?
    291 
    292 That's okay, though.  There are people for whom impermanence is a way of
    293 life.  There are a few categories of messages for which I really, really
    294 want the impermanence to start later, though -- specifically +inbox and
    295 +Starred messages.  Since there's only 15 of these lost, is there some
    296 way I can grab/index those manually?
    297 
    298 ...or a way to help identify and fix the underlying problem?
    299 ~d
    300 
    301 P.S. Sorry for all the details.  I realize this is probably much more
    302 interesting to me than to any of you, since it affects me more directly,
    303 but I just couldn't help myself. =P
    304 
    305 From Mr.  Mon Dec  8 00:33:34 2008
    306 From: Mr. (Mr.)
    307 Date: Mon, 08 Dec 2008 00:33:34 -0500
    308 Subject: [sup-talk] migrating email to a new computer
    309 In-Reply-To: <1228710381-sup-9800@buckwheat>
    310 References: <1227135030-sup-401@buckwheat> <1227743600-sup-8882@entry>
    311 	<1228710381-sup-9800@buckwheat>
    312 Message-ID: <1228714133-sup-9592@buckwheat>
    313 
    314 Excerpts from Daniel Wagner's message of Mon Dec 08 00:14:39 -0500 2008:
    315 > The Numbers According To sup-sync
    316 > entries in the dump file    26462
    317 > entries scanned             24779
    318 > entries added               24373
    319 > warnings                       27
    320 > fakes                          75
    321 > 
    322 > The Numbers According To sup
    323 >                             old machine     new machine
    324 > messages in the index       26459           24416
    325 
    326 Ah, I just realized: copying everything to one directory is fine only if
    327 the filenames have unique names.  A quick investigation shows I probably
    328 nuked around 500-600 messages with this assumption, so that's one easily
    329 fixable problem.  But that still leaves ~1500 messages in the aether...
    330 
    331 Love!
    332 ~d
    333 
    334 From wmorgan-sup@masanjin.net  Mon Dec  8 14:44:03 2008
    335 From: wmorgan-sup@masanjin.net (William Morgan)
    336 Date: Mon, 08 Dec 2008 11:44:03 -0800
    337 Subject: [sup-talk] Allow searching of sup wiki?
    338 In-Reply-To: <1227774913-sup-1534@tomsk>
    339 References: <1226340574-sup-8764@ubuntu> <1226349907-sup-5412@entry>
    340 	<1227742325-sup-9058@entry> <1227774913-sup-1534@tomsk>
    341 Message-ID: <1228765419-sup-1323@entry>
    342 
    343 Reformatted excerpts from marcus-sup's message of 2008-11-27:
    344 > github perhaps :) Mind you, you'd need to move sup to github as well
    345 > then....
    346 
    347 It's certainly an option!
    348 -- 
    349 William <wmorgan-sup at masanjin.net>
    350 
    351 From fjbender@googlemail.com  Mon Dec  8 15:59:49 2008
    352 From: fjbender@googlemail.com (Florian Bender)
    353 Date: Mon, 08 Dec 2008 21:59:49 +0100
    354 Subject: [sup-talk] Increase poll frequency
    355 Message-ID: <1228769970-sup-9881@vicodin>
    356 
    357 Hi,
    358 after a quick glance at the documentation, I didn't notice any option to
    359 increase the frequency sup polls a source for new E-Mails. For my GMail Account
    360 I'll like it, if it'd be polled every minute or so.
    361 Any suggestions?
    362 
    363 
    364 -- 
    365 Florian Bender <fjbender at googlemail.com>
    366 GPG Fingerprint: D0BB BDF9 5889 BC30 49F4 681D 84F7 6AC7 7C39 85F8
    367 -------------- next part --------------
    368 A non-text attachment was scrubbed...
    369 Name: signature.asc
    370 Type: application/pgp-signature
    371 Size: 197 bytes
    372 Desc: not available
    373 URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081208/87819a18/attachment.bin>
    374 
    375 From bdwalton@gmail.com  Mon Dec  8 16:29:35 2008
    376 From: bdwalton@gmail.com (Ben Walton)
    377 Date: Mon, 8 Dec 2008 16:29:35 -0500
    378 Subject: [sup-talk] Increase poll frequency
    379 In-Reply-To: <1228769970-sup-9881@vicodin>
    380 References: <1228769970-sup-9881@vicodin>
    381 Message-ID: <f96e0240812081329u595f9583n39f2a3cc5fd9da71@mail.gmail.com>
    382 
    383 On Mon, Dec 8, 2008 at 3:59 PM, Florian Bender <fjbender at googlemail.com> wrote:
    384 > Any suggestions?
    385 
    386 Stop checking your email so much!?! <grin>
    387 
    388 [Just kidding, but I couldn't resist.]
    389 
    390 Cheers.
    391 -Ben
    392 -- 
    393 ---------------------------------------------------------------------------------------------------------------------------
    394 Ben Walton <bdwalton at gmail.com>
    395 
    396 For by doubting we come to inquire, and by inquiring we perceive the truth.
    397 
    398 -Peter Abelard
    399 ---------------------------------------------------------------------------------------------------------------------------
    400 
    401 From fjbender@googlemail.com  Tue Dec  9 01:14:51 2008
    402 From: fjbender@googlemail.com (Florian Bender)
    403 Date: Tue, 09 Dec 2008 07:14:51 +0100
    404 Subject: [sup-talk] Increase poll frequency
    405 In-Reply-To: <8F89701D-3DC1-4BAF-8A42-090D532FBF64@gmail.com>
    406 References: <1228769970-sup-9881@vicodin>
    407 	<8F89701D-3DC1-4BAF-8A42-090D532FBF64@gmail.com>
    408 Message-ID: <1228803122-sup-9451@vicodin>
    409 
    410 Excerpts from fedzor's message of Mo Dez 08 22:59:52 +0100 2008:
    411 > What setup are you using for your gmail account with sup? I've tried  
    412 > it, but it crashes fairly often for me, so it's unusable.
    413 > 
    414 I enabled IMAP in my GMail preferences, then took a look at the configuration data they provide:
    415 http://mail.google.com/support/bin/answer.py?answer=78799
    416 Then added my GMail account as an IMAP source with sup-conf or sup-add if you have configured sup already.
    417 Works like a charm, no crashes so far. Maybe you should report a bug, if the unexpected behaviour continues.
    418 
    419 -- 
    420 Florian Bender <fjbender at googlemail.com>
    421 GPG Fingerprint: D0BB BDF9 5889 BC30 49F4 681D 84F7 6AC7 7C39 85F8
    422 -------------- next part --------------
    423 A non-text attachment was scrubbed...
    424 Name: signature.asc
    425 Type: application/pgp-signature
    426 Size: 197 bytes
    427 Desc: not available
    428 URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081209/3f4089ca/attachment.bin>
    429 
    430 From wmorgan-sup@masanjin.net  Thu Dec 11 15:20:08 2008
    431 From: wmorgan-sup@masanjin.net (William Morgan)
    432 Date: Thu, 11 Dec 2008 12:20:08 -0800
    433 Subject: [sup-talk] Increase poll frequency
    434 In-Reply-To: <1228769970-sup-9881@vicodin>
    435 References: <1228769970-sup-9881@vicodin>
    436 Message-ID: <1229026741-sup-2943@entry>
    437 
    438 Reformatted excerpts from Florian Bender's message of 2008-12-08:
    439 > after a quick glance at the documentation, I didn't notice any option
    440 > to increase the frequency sup polls a source for new E-Mails. For my
    441 > GMail Account I'll like it, if it'd be polled every minute or so.
    442 
    443 There's no way to do this now, but it would be pretty easy to add.
    444 -- 
    445 William <wmorgan-sup at masanjin.net>
    446 
    447 From fjbender@googlemail.com  Thu Dec 11 15:30:39 2008
    448 From: fjbender@googlemail.com (Florian Bender)
    449 Date: Thu, 11 Dec 2008 21:30:39 +0100
    450 Subject: [sup-talk] Increase poll frequency
    451 In-Reply-To: <1229026741-sup-2943@entry>
    452 References: <1228769970-sup-9881@vicodin> <1229026741-sup-2943@entry>
    453 Message-ID: <1229027326-sup-7107@vicodin>
    454 
    455 Excerpts from William Morgan's message of Do Dez 11 21:20:08 +0100 2008:
    456 > There's no way to do this now, but it would be pretty easy to add.
    457 Due to the fact that I'm no Ruby coder, I'd be glad if you could add it to the next release :-)
    458 
    459 -- 
    460 Florian Bender <fjbender at googlemail.com>
    461 GPG Fingerprint: D0BB BDF9 5889 BC30 49F4 681D 84F7 6AC7 7C39 85F8
    462 -------------- next part --------------
    463 A non-text attachment was scrubbed...
    464 Name: signature.asc
    465 Type: application/pgp-signature
    466 Size: 197 bytes
    467 Desc: not available
    468 URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081211/cd78f994/attachment.bin>
    469 
    470 From marc.hartstein@alum.vassar.edu  Thu Dec 11 15:41:55 2008
    471 From: marc.hartstein@alum.vassar.edu (Marc Hartstein)
    472 Date: Thu, 11 Dec 2008 15:41:55 -0500
    473 Subject: [sup-talk] Increase poll frequency
    474 In-Reply-To: <1229026741-sup-2943@entry>
    475 References: <1228769970-sup-9881@vicodin> <1229026741-sup-2943@entry>
    476 Message-ID: <1229027982-sup-7502@cabinet>
    477 
    478 Excerpts from William Morgan's message of Thu Dec 11 15:20:08 -0500 2008:
    479 > Reformatted excerpts from Florian Bender's message of 2008-12-08:
    480 > > after a quick glance at the documentation, I didn't notice any option
    481 > > to increase the frequency sup polls a source for new E-Mails. For my
    482 > > GMail Account I'll like it, if it'd be polled every minute or so.
    483 > 
    484 > There's no way to do this now, but it would be pretty easy to add.
    485 
    486 Christopher Warrington sent a patch to this list in the spring which
    487 allows configuration of poll interval on a per-source basis.  This
    488 should do what's being looked for.
    489 
    490 I run it locally, mostly so sup doesn't waste time polling static
    491 sources (like the mailing list archive from before I joined this list),
    492 and it seems to work nicely.
    493 
    494 Here's the email as I have it saved in my sup-patches/:
    495 
    496 >From chrisw at rice.edu  Sat Mar 29 06:32:05 2008
    497 From: chrisw@rice.edu (Christopher Warrington)
    498 Date: Sat, 29 Mar 2008 05:32:05 -0500
    499 Subject: [sup-talk] [PATCH] polling is now done per source
    500 In-Reply-To: <1206784680-sup-3052 at chris-tablet>
    501 References: <1206784680-sup-3052 at chris-tablet>
    502 Message-ID: <1206786725-5456-1-git-send-email-chrisw at rice.edu>
    503 
    504 Each source has a poll_interval property. This property is used to
    505 determine whether the source should be polled. The user can still for a
    506 poll of all sources.
    507 ---
    508  bin/sup                    |    4 +-
    509  bin/sup-add                |   12 ++-
    510  lib/sup.rb                 |    4 +-
    511  lib/sup/imap.rb            |    6 +-
    512  lib/sup/maildir.rb         |    6 +-
    513  lib/sup/mbox/loader.rb     |    4 +-
    514  lib/sup/modes/poll-mode.rb |    4 +-
    515  lib/sup/poll.rb            |  211 +++++++++++++++++++++++++++++++++-----------
    516  lib/sup/source.rb          |    8 +-
    517  9 files changed, 189 insertions(+), 70 deletions(-)
    518  mode change 100755 => 100644 lib/sup/buffer.rb
    519 
    520 diff --git a/bin/sup b/bin/sup
    521 index 84fd77c..86a2d9f 100644
    522 --- a/bin/sup
    523 +++ b/bin/sup
    524 @@ -197,7 +197,7 @@ begin
    525      end
    526    end unless $opts[:no_initial_poll]
    527    
    528 -  imode.load_threads :num => ibuf.content_height, :when_done => lambda { reporting_thread("poll after loading inbox") { sleep 1; PollManager.poll } unless $opts[:no_threads] || $opts[:no_initial_poll] }
    529 +  imode.load_threads :num => ibuf.content_height, :when_done => lambda { reporting_thread("poll after loading inbox") { sleep 1; PollManager.auto_poll } unless $opts[:no_threads] || $opts[:no_initial_poll] }
    530  
    531    if $opts[:compose]
    532      ComposeMode.spawn_nicely :to_default => $opts[:compose]
    533 @@ -263,7 +263,7 @@ begin
    534      when :compose
    535        ComposeMode.spawn_nicely
    536      when :poll
    537 -      reporting_thread("user-invoked poll") { PollManager.poll }
    538 +      reporting_thread("user-invoked poll") { PollManager.forced_poll }
    539      when :recall_draft
    540        case Index.num_results_for :label => :draft
    541        when 0
    542 diff --git a/bin/sup-add b/bin/sup-add
    543 index 50bbb29..88705e1 100644
    544 --- a/bin/sup-add
    545 +++ b/bin/sup-add
    546 @@ -39,6 +39,7 @@ EOS
    547    opt :unusual, "Do not automatically poll these sources for new messages."
    548    opt :labels, "A comma-separated set of labels to apply to all messages from this source", :type => String
    549    opt :force_new, "Create a new account for this source, even if one already exists."
    550 +  opt :poll_interval, "The interval (in seconds) between new message polls. The default is #{Redwood::DEFAULT_POLL_INTERVAL}.", :type => :int
    551  end
    552  
    553  Trollop::die "require one or more sources" if ARGV.empty?
    554 @@ -84,6 +85,9 @@ index.lock_or_die
    555  begin
    556    index.load_sources
    557  
    558 +  Trollop::die "The poll interval must be a positive integer." if $opts[:poll_interval] <= 0
    559 +  poll_interval = $opts[:poll_interval] || Redwood::DEFAULT_POLL_INTERVAL
    560 +
    561    ARGV.each do |uri|
    562      labels = $opts[:labels] ? $opts[:labels].split(/\s*,\s*/).uniq : []
    563  
    564 @@ -100,14 +104,14 @@ begin
    565          say "For SSH connections, if you will use public key authentication, you may leave the username and password blank."
    566          say ""
    567          username, password = get_login_info uri, index.sources
    568 -        Redwood::MBox::SSHLoader.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, labels
    569 +        Redwood::MBox::SSHLoader.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels
    570        when "imap", "imaps"
    571          username, password = get_login_info uri, index.sources
    572 -        Redwood::IMAP.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, labels
    573 +        Redwood::IMAP.new uri, username, password, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels
    574        when "maildir"
    575 -        Redwood::Maildir.new uri, nil, !$opts[:unusual], $opts[:archive], nil, labels
    576 +        Redwood::Maildir.new uri, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels
    577        when "mbox"
    578 -        Redwood::MBox::Loader.new uri, nil, !$opts[:unusual], $opts[:archive], nil, labels
    579 +        Redwood::MBox::Loader.new uri, nil, !$opts[:unusual], $opts[:archive], nil, poll_interval, labels
    580        when nil
    581          Trollop::die "Sources must be specified with an URI"
    582        else
    583 diff --git a/lib/sup.rb b/lib/sup.rb
    584 index 1946f3c..c27a4bc 100644
    585 --- a/lib/sup.rb
    586 +++ b/lib/sup.rb
    587 @@ -50,6 +50,8 @@ module Redwood
    588    YAML_DOMAIN = "masanjin.net"
    589    YAML_DATE = "2006-10-01"
    590  
    591 +  DEFAULT_POLL_INTERVAL = 300
    592 +
    593  ## record exceptions thrown in threads nicely
    594    def reporting_thread name
    595      if $opts[:no_threads]
    596 @@ -72,7 +74,7 @@ module Redwood
    597    def save_yaml_obj object, fn, safe=false
    598      if safe
    599        safe_fn = "#{File.dirname fn}/safe_#{File.basename fn}"
    600 -      mode = File.stat(fn) if File.exists? fn
    601 +      mode = File.stat(fn).mode if File.exists? fn
    602        File.open(safe_fn, "w", mode) { |f| f.puts object.to_yaml }
    603        FileUtils.mv safe_fn, fn
    604      else
    605 diff --git a/lib/sup/buffer.rb b/lib/sup/buffer.rb
    606 old mode 100755
    607 new mode 100644
    608 diff --git a/lib/sup/imap.rb b/lib/sup/imap.rb
    609 index 1d36976..8b58cba 100644
    610 --- a/lib/sup/imap.rb
    611 +++ b/lib/sup/imap.rb
    612 @@ -51,13 +51,13 @@ class IMAP < Source
    613  
    614    attr_accessor :username, :password
    615    yaml_properties :uri, :username, :password, :cur_offset, :usual,
    616 -                  :archived, :id, :labels
    617 +                  :archived, :id, :poll_interval, :labels
    618  
    619 -  def initialize uri, username, password, last_idate=nil, usual=true, archived=false, id=nil, labels=[]
    620 +  def initialize uri, username, password, last_idate=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[]
    621      raise ArgumentError, "username and password must be specified" unless username && password
    622      raise ArgumentError, "not an imap uri" unless uri =~ %r!imaps?://!
    623  
    624 -    super uri, last_idate, usual, archived, id
    625 +    super uri, last_idate, usual, archived, id, [poll_interval, SCAN_INTERVAL].max
    626  
    627      @parsed_uri = URI(uri)
    628      @username = username
    629 diff --git a/lib/sup/maildir.rb b/lib/sup/maildir.rb
    630 index 584e657..ef71e0b 100644
    631 --- a/lib/sup/maildir.rb
    632 +++ b/lib/sup/maildir.rb
    633 @@ -12,9 +12,9 @@ class Maildir < Source
    634    SCAN_INTERVAL = 30 # seconds
    635  
    636    ## remind me never to use inheritance again.
    637 -  yaml_properties :uri, :cur_offset, :usual, :archived, :id, :labels
    638 -  def initialize uri, last_date=nil, usual=true, archived=false, id=nil, labels=[]
    639 -    super uri, last_date, usual, archived, id
    640 +  yaml_properties :uri, :cur_offset, :usual, :archived, :id, :poll_interval, :labels
    641 +  def initialize uri, last_date=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[]
    642 +    super uri, last_date, usual, archived, id, [poll_interval, SCAN_INTERVAL].max
    643      uri = URI(Source.expand_filesystem_uri(uri))
    644  
    645      raise ArgumentError, "not a maildir URI" unless uri.scheme == "maildir"
    646 diff --git a/lib/sup/mbox/loader.rb b/lib/sup/mbox/loader.rb
    647 index 7fe9129..44317d5 100644
    648 --- a/lib/sup/mbox/loader.rb
    649 +++ b/lib/sup/mbox/loader.rb
    650 @@ -9,7 +9,7 @@ class Loader < Source
    651    attr_accessor :labels
    652  
    653    ## uri_or_fp is horrific. need to refactor.
    654 -  def initialize uri_or_fp, start_offset=nil, usual=true, archived=false, id=nil, labels=[]
    655 +  def initialize uri_or_fp, start_offset=nil, usual=true, archived=false, id=nil, poll_interval=nil, labels=[]
    656      @mutex = Mutex.new
    657      @labels = ((labels || []) - LabelManager::RESERVED_LABELS).uniq.freeze
    658  
    659 @@ -26,7 +26,7 @@ class Loader < Source
    660        @path = uri_or_fp.path
    661      end
    662  
    663 -    super uri_or_fp, start_offset, usual, archived, id
    664 +    super uri_or_fp, start_offset, usual, archived, id, poll_interval
    665    end
    666  
    667    def file_path; @path end
    668 diff --git a/lib/sup/modes/poll-mode.rb b/lib/sup/modes/poll-mode.rb
    669 index 5849f3e..5521cdc 100644
    670 --- a/lib/sup/modes/poll-mode.rb
    671 +++ b/lib/sup/modes/poll-mode.rb
    672 @@ -10,11 +10,11 @@ class PollMode < LogMode
    673      self << s + "\n"
    674    end
    675  
    676 -  def poll
    677 +  def poll sources
    678      puts unless @new
    679      @new = false
    680      puts "Poll started at #{Time.now}"
    681 -    PollManager.do_poll { |s| puts s }
    682 +    PollManager.do_poll_sources(sources) { |s| puts s }
    683    end
    684  end
    685  
    686 diff --git a/lib/sup/poll.rb b/lib/sup/poll.rb
    687 index d32c893..1ff8014 100644
    688 --- a/lib/sup/poll.rb
    689 +++ b/lib/sup/poll.rb
    690 @@ -12,13 +12,22 @@ Variables:
    691  EOS
    692  
    693    HookManager.register "before-poll", <<EOS
    694 +Executes immediately before any poll for new messages commences.
    695 +Variables:
    696 +                 accts: an array of source URIs that will be polled
    697 +EOS
    698 +
    699 +  HookManager.register "before-acct-poll", <<EOS
    700  Executes immediately before a poll for new messages commences.
    701 -No variables.
    702 +Variables:
    703 +                  acct: the URI of the account being polled
    704  EOS
    705  
    706 -  HookManager.register "after-poll", <<EOS
    707 -Executes immediately after a poll for new messages completes.
    708 +  HookManager.register "after-acct-poll", <<EOS
    709 +Executes immediately after a poll for new messages completes. The statistics are
    710 +only for the account specified.
    711  Variables:
    712 +                  acct: the URI of the account being polled
    713                     num: the total number of new messages added in this poll
    714               num_inbox: the number of new messages added in this poll which
    715                          appear in the inbox (i.e. were not auto-archived).
    716 @@ -28,14 +37,25 @@ num_inbox_total_unread: the total number of unread messages in the inbox
    717                          only those messages appearing in the inbox
    718  EOS
    719  
    720 -  DELAY = 300
    721 +  HookManager.register "after-poll", <<EOS
    722 +Executes immediately after any poll for new messages completes. The statistics
    723 +are for all sources polled.
    724 +Variables:
    725 +                 accts: an array of source URIs that were polled
    726 +                   num: the total number of new messages added in this poll
    727 +             num_inbox: the number of new messages added in this poll which
    728 +                        appear in the inbox (i.e. were not auto-archived).
    729 +num_inbox_total_unread: the total number of unread messages in the inbox
    730 +         from_and_subj: an array of (from email address, subject) pairs
    731 +   from_and_subj_inbox: an array of (from email address, subject) pairs for
    732 +                        only those messages appearing in the inbox
    733 +EOS
    734  
    735    def initialize
    736      @mutex = Mutex.new
    737      @thread = nil
    738 -    @last_poll = nil
    739      @polling = false
    740 -    
    741 +
    742      self.class.i_am_the_instance self
    743    end
    744  
    745 @@ -44,47 +64,113 @@ EOS
    746      b
    747    end
    748  
    749 -  def poll
    750 +  def start
    751 +    ## Periodically calls auto_poll, effectively automatically polling
    752 +    ## in the background.
    753 +    ##
    754 +    ## auto_poll is called more frequently than the smallest poll
    755 +    ## interval. auto_poll is not guaranteed to be called in multiples
    756 +    ## of poll intervals. I.e., don't rely on auto_poll to be called
    757 +    ## every x seconds: when auto_poll is called assume an
    758 +    ## indeterminate amount of time has passed.
    759 +    @min_poll_interval = Index.usual_sources.collect{|s| s.poll_interval}.select{|pi| !pi.nil?}.min || Redwood::DEFAULT_POLL_INTERVAL
    760 +
    761 +    Redwood::log "Background poll interval is #{@min_poll_interval} seconds."
    762 +
    763 +    @thread = Redwood::reporting_thread("periodic poll") do
    764 +      while true
    765 +        Redwood::log "Sleeping for #{@min_poll_interval / 2} seconds."
    766 +
    767 +        sleep @min_poll_interval / 2
    768 +        auto_poll
    769 +      end
    770 +    end
    771 +  end
    772 +
    773 +  def stop
    774 +    @thread.kill if @thread
    775 +    @thread = nil
    776 +  end
    777 +
    778 +  def auto_poll
    779 +    ## This method is called by the thread spawned in start. It
    780 +    ## collects the sources that should be polled
    781 +    ## (source.poll_interval has expired or the source has never been
    782 +    ## polled) and polls them.
    783 +    ##
    784 +    ## Returns an array [# of new messages, # of new messages to
    785 +    ## inbox, new message subject/name array pairs, subject/name array
    786 +    ## pairs loaded to inbox]
    787 +
    788 +    sources_to_poll = []
    789 +    @mutex.synchronize do # William, do we need to synchronize here?
    790 +      begin
    791 +        sources_to_poll = Index.usual_sources.select do |source|
    792 +          begin
    793 +            source.last_poll.nil? || (Time.now - source.last_poll) >= source.poll_interval
    794 +          end
    795 +        end
    796 +      end
    797 +    end
    798 +
    799 +    poll_sources sources_to_poll
    800 +  end
    801 +
    802 +  def forced_poll
    803 +    ## This method is called when the user explicitly requests a poll.
    804 +    ##
    805 +    ## Returns an array [# of new messages, # of new messages to
    806 +    ## inbox, new message subject/name array pairs, subject/name array
    807 +    ## pairs loaded to inbox]
    808 +    poll_sources Index.usual_sources
    809 +  end
    810 +
    811 +  def poll_sources sources
    812 +    ## Polls the given sources. Clients of PollManager should call
    813 +    ## this method, not do_poll_sources or poll_source. (Well, only
    814 +    ## PollMode should call do_poll_sources)
    815 +    ##
    816 +    ## Returns an array [# of new messages, # of new messages to
    817 +    ## inbox, new message subject/name array pairs, subject/name array
    818 +    ## pairs loaded to inbox]
    819      return if @polling
    820      @polling = true
    821 -    HookManager.run "before-poll"
    822 +
    823 +    source_uris = sources.map{|s| s.uri}
    824 +
    825 +    HookManager.run "before-acct-poll", :accts => source_uris
    826  
    827      BufferManager.flash "Polling for new messages..."
    828 -    num, numi, from_and_subj, from_and_subj_inbox = buffer.mode.poll
    829 +    num, numi, from_subj, from_subj_inbox = buffer.mode.poll sources
    830      if num > 0
    831        BufferManager.flash "Loaded #{num.pluralize 'new message'}, #{numi} to inbox." 
    832      else
    833        BufferManager.flash "No new messages." 
    834      end
    835  
    836 -    HookManager.run "after-poll", :num => num, :num_inbox => numi, :from_and_subj => from_and_subj, :from_and_subj_inbox => from_and_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] }
    837 +    HookManager.run "after-poll", :accts => source_uris, :num => num, :num_inbox => numi, :from_and_subj => from_subj, :from_and_subj_inbox => from_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] }
    838  
    839      @polling = false
    840 -    [num, numi]
    841 -  end
    842 -
    843 -  def start
    844 -    @thread = Redwood::reporting_thread("periodic poll") do
    845 -      while true
    846 -        sleep DELAY / 2
    847 -        poll if @last_poll.nil? || (Time.now - @last_poll) >= DELAY
    848 -      end
    849 -    end
    850 +    [num, numi, from_subj, from_subj_inbox]
    851    end
    852  
    853 -  def stop
    854 -    @thread.kill if @thread
    855 -    @thread = nil
    856 -  end
    857 +  def do_poll_sources sources, &block
    858 +    ## Polls each source, keeping track of vital statistics (number
    859 +    ## loaded, name/e-mail pairs, &c.) about the poll results.
    860 +    ##
    861 +    ## We need explicit access to the block so that we can pass it to
    862 +    ## poll_source.
    863 +    ##
    864 +    ## Returns an array [total # of new messages, total # of new
    865 +    ## messages to inbox, all new message subject/name array pairs,
    866 +    ## all subject/name array pairs loaded to inbox]
    867  
    868 -  def do_poll
    869      total_num = total_numi = 0
    870 -    from_and_subj = []
    871 -    from_and_subj_inbox = []
    872 +    total_from_and_subj = []
    873 +    total_from_and_subj_inbox = []
    874  
    875      @mutex.synchronize do
    876 -      Index.usual_sources.each do |source|
    877 -#        yield "source #{source} is done? #{source.done?} (cur_offset #{source.cur_offset} >= #{source.end_offset})"
    878 +      sources.each do |source|
    879          begin
    880            yield "Loading from #{source}... " unless source.done? || source.has_errors?
    881          rescue SourceError => e
    882 @@ -93,32 +179,57 @@ EOS
    883            next
    884          end
    885  
    886 -        num = 0
    887 -        numi = 0
    888 -        add_messages_from source do |m, offset, entry|
    889 -          ## always preserve the labels on disk.
    890 -          m.labels = entry[:label].split(/\s+/).map { |x| x.intern } if entry
    891 -          yield "Found message at #{offset} with labels {#{m.labels * ', '}}"
    892 -          unless entry
    893 -            num += 1
    894 -            from_and_subj << [m.from.longname, m.subj]
    895 -            if m.has_label?(:inbox) && ([:spam, :deleted, :killed] & m.labels).empty?
    896 -              from_and_subj_inbox << [m.from.longname, m.subj]
    897 -              numi += 1 
    898 -            end
    899 -          end
    900 -          m
    901 -        end
    902 -        yield "Found #{num} messages, #{numi} to inbox." unless num == 0
    903 +        HookManager.run "before-acct-poll", :acct => source.uri
    904 +
    905 +        BufferManager.flash "Polling #{source.uri} for new messages..."
    906 +        num, numi, from_and_subj, from_and_subj_inbox = poll_source source, &block
    907 +
    908          total_num += num
    909          total_numi += numi
    910 +        total_from_and_subj += from_and_subj
    911 +        total_from_and_subj_inbox += from_and_subj_inbox
    912 +
    913 +        HookManager.run "after-acct-poll", :acct => source.uri, :num => num, :num_inbox => numi, :from_and_subj => from_and_subj, :from_and_subj_inbox => from_and_subj_inbox, :num_inbox_total_unread => lambda { Index.num_results_for :labels => [:inbox, :unread] }
    914        end
    915 +    end
    916 +
    917 +    [total_num, total_numi, total_from_and_subj, total_from_and_subj_inbox]
    918 +  end
    919  
    920 -      yield "Done polling; loaded #{total_num} new messages total"
    921 -      @last_poll = Time.now
    922 -      @polling = false
    923 +  def poll_source source
    924 +    ## Polls the given source for new messages.
    925 +    ##
    926 +    ## @mutex must be held before calling! See do_poll_sources.
    927 +    ##
    928 +    ## Returns an array [# of new messages, # of new messages to
    929 +    ## inbox, new message subject/name array pairs, subject/name array
    930 +    ## pairs loaded to inbox]
    931 +
    932 +    num = 0
    933 +    numi = 0
    934 +    from_and_subj = []
    935 +    from_and_subj_inbox = []
    936 +
    937 +    add_messages_from source do |m, offset, entry|
    938 +      ## always preserve the labels on disk.
    939 +      m.labels = entry[:label].split(/\s+/).map { |x| x.intern } if entry
    940 +      yield "Found message at #{offset} with labels {#{m.labels * ', '}}"
    941 +      unless entry
    942 +        num += 1
    943 +        from_and_subj << [m.from.longname, m.subj]
    944 +        if m.has_label?(:inbox) && ([:spam, :deleted, :killed] & m.labels).empty?
    945 +          from_and_subj_inbox << [m.from.longname, m.subj]
    946 +          numi += 1
    947 +        end
    948 +      end
    949 +      m
    950      end
    951 -    [total_num, total_numi, from_and_subj, from_and_subj_inbox]
    952 +
    953 +    yield "For source #{source.uri}, found #{num} messages, #{numi} to inbox." unless num == 0
    954 +
    955 +    source.last_poll = Time.now
    956 +
    957 +    [num, numi, from_and_subj, from_and_subj_inbox]
    958    end
    959  
    960    ## this is the main mechanism for adding new messages to the
    961 diff --git a/lib/sup/source.rb b/lib/sup/source.rb
    962 index 6510aae..fd8d381 100644
    963 --- a/lib/sup/source.rb
    964 +++ b/lib/sup/source.rb
    965 @@ -62,10 +62,10 @@ class Source
    966    ## dirty? means cur_offset has changed, so the source info needs to
    967    ## be re-saved to sources.yaml.
    968    bool_reader :usual, :archived, :dirty
    969 -  attr_reader :uri, :cur_offset
    970 -  attr_accessor :id
    971 +  attr_reader :uri, :cur_offset, :poll_interval
    972 +  attr_accessor :id, :last_poll
    973  
    974 -  def initialize uri, initial_offset=nil, usual=true, archived=false, id=nil
    975 +  def initialize uri, initial_offset=nil, usual=true, archived=false, id=nil, poll_interval=nil
    976      raise ArgumentError, "id must be an integer: #{id.inspect}" unless id.is_a? Fixnum if id
    977  
    978      @uri = uri
    979 @@ -73,6 +73,8 @@ class Source
    980      @usual = usual
    981      @archived = archived
    982      @id = id
    983 +    @poll_interval = poll_interval || Redwood::DEFAULT_POLL_INTERVAL #seconds
    984 +    @last_poll = nil
    985      @dirty = false
    986    end
    987  
    988 -- 
    989 1.5.4
    990 
    991 -------------- next part --------------
    992 A non-text attachment was scrubbed...
    993 Name: signature.asc
    994 Type: application/pgp-signature
    995 Size: 197 bytes
    996 Desc: not available
    997 URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081211/d2fdb6d5/attachment.bin>
    998 
    999 From wmorgan-sup@masanjin.net  Thu Dec 11 20:31:41 2008
   1000 From: wmorgan-sup@masanjin.net (William Morgan)
   1001 Date: Thu, 11 Dec 2008 17:31:41 -0800
   1002 Subject: [sup-talk] migrating email to a new computer
   1003 In-Reply-To: <1228710381-sup-9800@buckwheat>
   1004 References: <1227135030-sup-401@buckwheat> <1227743600-sup-8882@entry>
   1005 	<1228710381-sup-9800@buckwheat>
   1006 Message-ID: <1229042458-sup-7539@entry>
   1007 
   1008 Reformatted excerpts from Daniel Wagner's message of 2008-12-07:
   1009 > entries in the dump file    26462
   1010 
   1011 What is this number from? Is it a wc -l on the dumpfile, or sup-sync's
   1012 "Restored state on X messages" message? If the latter, what is wc -l?
   1013 (And if the former, what's the other number?)
   1014 
   1015 > entries scanned             24779
   1016 > entries added               24373
   1017 > warnings                       27
   1018 > fakes                          75
   1019 
   1020 Warnings and fakes we can ignore. (They still count as being added.)
   1021 Did sup-sync report a number of updates?
   1022 
   1023 It should be the case that num adds + num updates = num scanned,
   1024 and num adds = index size (when starting from an empty index).
   1025 
   1026 > The Numbers According To sup
   1027 >                             old machine     new machine
   1028 > messages in the index       26459           24416
   1029 > messages with +inbox           50              45
   1030 > messages with +Starred         67              57
   1031 > 
   1032 > So, I'm not sure I understand how to reconcile these numbers.  Things
   1033 > are mismatched in odd ways; for example, how could there be more
   1034 > entries in the dump than there are messages in the index?
   1035 
   1036 Yes, that is weird. 26462 vs 26459 is what concerns me. How did three
   1037 new messages appear at dump time?
   1038 
   1039 > Or how come "entries added" + "warnings" != "messages in the index"?
   1040 
   1041 Messages with warnings or that needed fake headers should still be
   1042 added, so the question is really 24416 vs 24373. That's 43 new messages
   1043 that appeared in the index.
   1044 
   1045 > That's okay, though.  There are people for whom impermanence is a way
   1046 > of life.
   1047 
   1048 Uh oh. 
   1049 
   1050 > There are a few categories of messages for which I really, really
   1051 > want the impermanence to start later, though -- specifically +inbox and
   1052 > +Starred messages.  Since there's only 15 of these lost, is there some
   1053 > way I can grab/index those manually?
   1054 
   1055 The most direct way is to query the index through devel/console.sh. You
   1056 can find examples on the wiki or in the mailing list archives. Something
   1057 like
   1058   Index.ferret.search("label:inbox", :limit => :all).hits.map { |h| Index.ferret[h.doc] }
   1059 
   1060 will give you all the Ferret records with the index label, and you can
   1061 then query them for all the fields listed in index.rb. (They'll appear
   1062 as empty hashes, {}, when irb prints them out, but in reality they're
   1063 just lazily-filled.
   1064 
   1065 That should give you the information you need to compare the two sets of
   1066 documents on both indexes, and figure out what the differences are.
   1067 -- 
   1068 William <wmorgan-sup at masanjin.net>
   1069 
   1070 From wmorgan-sup@masanjin.net  Thu Dec 11 20:37:52 2008
   1071 From: wmorgan-sup@masanjin.net (William Morgan)
   1072 Date: Thu, 11 Dec 2008 17:37:52 -0800
   1073 Subject: [sup-talk] Increase poll frequency
   1074 In-Reply-To: <1229027982-sup-7502@cabinet>
   1075 References: <1228769970-sup-9881@vicodin> <1229026741-sup-2943@entry>
   1076 	<1229027982-sup-7502@cabinet>
   1077 Message-ID: <1229045691-sup-5418@entry>
   1078 
   1079 Reformatted excerpts from marc.hartstein's message of 2008-12-11:
   1080 > Christopher Warrington sent a patch to this list in the spring which
   1081 > allows configuration of poll interval on a per-source basis.  This
   1082 > should do what's being looked for.
   1083 
   1084 >From what I remember there were some minor problems with the patch and
   1085 we were promised a fixed version, and I think that's why it wasn't
   1086 applied to the repo. But it may be sufficient for your purposes right
   1087 now!
   1088 
   1089 (And the miracle of git is that, if you apply it, you can pull --rebase
   1090 and keep the patch on top even when you get new changes.)
   1091 -- 
   1092 William <wmorgan-sup at masanjin.net>
   1093 
   1094 From wmorgan-sup@masanjin.net  Thu Dec 11 20:50:12 2008
   1095 From: wmorgan-sup@masanjin.net (William Morgan)
   1096 Date: Thu, 11 Dec 2008 17:50:12 -0800
   1097 Subject: [sup-talk] Exception in sup-git when sending postponed email
   1098 In-Reply-To: <20081203211121.GB13504@danconia.atlantis.saintaardvarkthecarpeted.com>
   1099 References: <20081203211121.GB13504@danconia.atlantis.saintaardvarkthecarpeted.com>
   1100 Message-ID: <1229046535-sup-9338@entry>
   1101 
   1102 Reformatted excerpts from Saint Aardvark the Carpeted's message of 2008-12-03:
   1103 > --- Errno::ENOENT from thread: main
   1104 > No such file or directory - /home/hugh/.sup/drafts/^A^D
   1105 
   1106 Thanks for the very details report, but I'm having a hard time imagining
   1107 what could've caused this. It almost seems like a memory corruption
   1108 issue. Has it (or anything like it) happened again? 
   1109 -- 
   1110 William <wmorgan-sup at masanjin.net>
   1111 
   1112 From chrisw@rice.edu  Thu Dec 11 20:58:23 2008
   1113 From: chrisw@rice.edu (Christopher Warrington)
   1114 Date: Thu, 11 Dec 2008 19:58:23 -0600
   1115 Subject: [sup-talk] Increase poll frequency
   1116 In-Reply-To: <1229045691-sup-5418@entry>
   1117 Message-ID: <C567215F.39E5%chrisw@rice.edu>
   1118 
   1119 "William Morgan" <wmorgan-sup at masanjin.net> @ 2008-12-11 7:37 PM:
   1120 
   1121 >> From what I remember there were some minor problems with the patch and
   1122 
   1123 Something's up somewhere with "From " and ">From " lines...
   1124 
   1125 > From what I remember there were some minor problems with the patch and
   1126 > we were promised a fixed version, and I think that's why it wasn't
   1127 > applied to the repo. But it may be sufficient for your purposes right
   1128 > now!
   1129 
   1130 Anyone remember what those were? I haven't been using Sup lately, so the
   1131 deficiencies haven't been bugging me. :-)
   1132 
   1133 -- 
   1134 Christopher Warrington <chrisw at rice.edu>
   1135 Jones College
   1136 
   1137 
   1138 
   1139 From aardvark@saintaardvarkthecarpeted.com  Fri Dec 12 09:01:14 2008
   1140 From: aardvark@saintaardvarkthecarpeted.com (Saint Aardvark the Carpeted)
   1141 Date: Fri, 12 Dec 2008 06:01:14 -0800
   1142 Subject: [sup-talk] Exception in sup-git when sending postponed email
   1143 In-Reply-To: <1229046535-sup-9338@entry>
   1144 References: <20081203211121.GB13504@danconia.atlantis.saintaardvarkthecarpeted.com>
   1145 	<1229046535-sup-9338@entry>
   1146 Message-ID: <20081212140114.GI25721@danconia.atlantis.saintaardvarkthecarpeted.com>
   1147 
   1148 William Morgan disturbed my sleep to write:
   1149 > Thanks for the very details report, but I'm having a hard time imagining
   1150 > what could've caused this. It almost seems like a memory corruption
   1151 > issue. Has it (or anything like it) happened again? 
   1152 
   1153 Not yet.  I'll keep an eye out and will try stress-testing the memory
   1154 on this box.
   1155 
   1156 Thanks,
   1157 Hugh
   1158 
   1159 -- 
   1160 Saint Aardvark the Carpeted
   1161 http://saintaardvarkthecarpeted.com
   1162 Because the plural of Anecdote is Myth.
   1163 -------------- next part --------------
   1164 A non-text attachment was scrubbed...
   1165 Name: not available
   1166 Type: application/pgp-signature
   1167 Size: 187 bytes
   1168 Desc: not available
   1169 URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081212/cf731888/attachment.bin>
   1170 
   1171 From marianne.promberger+sup-talk@gmail.com  Wed Dec 17 16:37:11 2008
   1172 From: marianne.promberger+sup-talk@gmail.com (Marianne Promberger)
   1173 Date: Wed, 17 Dec 2008 21:37:11 +0000
   1174 Subject: [sup-talk] a few sup newbie questions
   1175 Message-ID: <1229548441-sup-531@audrey>
   1176 
   1177 Hi,
   1178 
   1179 Thanks for writing such a powerful app! I'm new to sup and am
   1180 wondering whether any of the following are possible:
   1181 
   1182 (1) Automatically apply labels to incoming mail defined on search
   1183 patterns? I know I can apply labels based on source, but I'd really
   1184 like some labels based on subject, etc. I figured out I can do a
   1185 search, then "!!", then tag all and ";" and label. But needless to
   1186 say, that's cumbersome. Pattern-based labeling when mail arrives would
   1187 be great. Is it possible?
   1188 
   1189 (2) Save search history across sup sessions, so that I can use
   1190 up-arrow to access terms I searched for last time I read mail in sup?
   1191 
   1192 (3) Are there search patterns like the "~P" and "~p" in mutt, that is,
   1193 matching everything from or to "me", as defined by the alternates in
   1194 ~/.sup/config.yaml?
   1195 
   1196 (4) Is there any way to send several drafts at once? (I'm assuming
   1197 there is no way to postpone a message other than saving it as a
   1198 draft?). Tagging them and then "enter" or "y" did not work.
   1199 
   1200 Thanks,
   1201 
   1202 m.
   1203 
   1204 
   1205 
   1206 
   1207 From wmorgan-sup@masanjin.net  Wed Dec 17 17:46:15 2008
   1208 From: wmorgan-sup@masanjin.net (William Morgan)
   1209 Date: Wed, 17 Dec 2008 14:46:15 -0800
   1210 Subject: [sup-talk] a few sup newbie questions
   1211 In-Reply-To: <1229548441-sup-531@audrey>
   1212 References: <1229548441-sup-531@audrey>
   1213 Message-ID: <1229552235-sup-1046@entry>
   1214 
   1215 Reformatted excerpts from marianne.promberger+sup-talk's message of 2008-12-17:
   1216 > Thanks for writing such a powerful app! I'm new to sup and am
   1217 > wondering whether any of the following are possible:
   1218 
   1219 Hi and welcome!
   1220 
   1221 > (1) Automatically apply labels to incoming mail defined on search
   1222 > patterns?
   1223 
   1224 Stuff like that is handled by Sup's hook system. The plus side is that
   1225 it's very flexible; the downside is that you have to write Ruby code. In
   1226 this case, check out the before-add-message hook. ("sup -l" will list
   1227 all the hooks and some brief documentation.) 
   1228 
   1229 In my case, I use something like:
   1230 
   1231   to_string = message.recipients.map { |t| t.email }.join(" ")
   1232   case to_string
   1233   when /\bgit at vger.kernel.org\b/
   1234     message.add_label :git
   1235     message.remove_label :inbox
   1236   when /\bsup-talk at rubyforge.org\b/
   1237     message.add_label :sup
   1238   # etc...
   1239 
   1240 > (2) Save search history across sup sessions, so that I can use
   1241 > up-arrow to access terms I searched for last time I read mail in sup?
   1242 
   1243 Not implemented, but a good idea, and probably pretty easy!
   1244 
   1245 > (3) Are there search patterns like the "~P" and "~p" in mutt, that is,
   1246 > matching everything from or to "me", as defined by the alternates in
   1247 > ~/.sup/config.yaml?
   1248 
   1249 You can search for things like "from:me" or "to:me". That should work
   1250 with :alternates, but unfortunately doesn't work if you use :regexen.
   1251 
   1252 > (4) Is there any way to send several drafts at once? (I'm assuming
   1253 > there is no way to postpone a message other than saving it as a
   1254 > draft?). Tagging them and then "enter" or "y" did not work.
   1255 
   1256 Not implemented, but a good idea, and definitely easy to do.
   1257 -- 
   1258 William <wmorgan-sup at masanjin.net>
   1259 
   1260 From marc.hartstein@alum.vassar.edu  Fri Dec 19 20:03:59 2008
   1261 From: marc.hartstein@alum.vassar.edu (Marc Hartstein)
   1262 Date: Fri, 19 Dec 2008 20:03:59 -0500
   1263 Subject: [sup-talk] Increase poll frequency
   1264 In-Reply-To: <C567215F.39E5%chrisw@rice.edu>
   1265 References: <1229045691-sup-5418@entry> <C567215F.39E5%chrisw@rice.edu>
   1266 Message-ID: <1229735030-sup-9615@cabinet>
   1267 
   1268 Excerpts from Christopher Warrington's message of Thu Dec 11 20:58:23 -0500 2008:
   1269 > "William Morgan" <wmorgan-sup at masanjin.net> @ 2008-12-11 7:37 PM:
   1270 > 
   1271 > > From what I remember there were some minor problems with the patch and
   1272 > > we were promised a fixed version, and I think that's why it wasn't
   1273 > > applied to the repo. But it may be sufficient for your purposes right
   1274 > > now!
   1275 > 
   1276 > Anyone remember what those were? I haven't been using Sup lately, so the
   1277 > deficiencies haven't been bugging me. :-)
   1278 
   1279 I do now, as it just bit me again:
   1280 
   1281 If you run a version of sup with the patch, it will crash on startup if
   1282 any usual source lacks the :poll_interval property.
   1283 
   1284 If you have just installed the patch, this will bite you.
   1285 
   1286 If you have run a non-patched version of sup for any reason, it will
   1287 blow away the :poll_interval properties on clean exit (because it
   1288 doesn't recognize them, so it doesn't write them to sources.yaml)
   1289 
   1290 
   1291 Suggest using default value when no value is supplied to solve the first
   1292 case, probably emitting a warning to help slightly with the second case.
   1293 
   1294 I have no idea what to do about the second case more generally, except
   1295 to perpetuate the patch to 'next' and 'master'.
   1296 
   1297 There may be other issues that I'm unaware of.
   1298 -------------- next part --------------
   1299 A non-text attachment was scrubbed...
   1300 Name: signature.asc
   1301 Type: application/pgp-signature
   1302 Size: 197 bytes
   1303 Desc: not available
   1304 URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081219/f3703503/attachment.bin>
   1305 
   1306 From marianne.promberger+sup-talk@gmail.com  Mon Dec 29 12:27:19 2008
   1307 From: marianne.promberger+sup-talk@gmail.com (Marianne)
   1308 Date: Mon, 29 Dec 2008 18:27:19 +0100
   1309 Subject: [sup-talk] a few sup newbie questions
   1310 In-Reply-To: <1229552235-sup-1046@entry>
   1311 References: <1229548441-sup-531@audrey> <1229552235-sup-1046@entry>
   1312 Message-ID: <1230571384-sup-9910@audrey>
   1313 
   1314 Hi,
   1315 
   1316 Thanks for the quick reply; sorry for my slow one.
   1317 
   1318 Excerpts from William Morgan's message of Wed Dec 17 23:46:15 +0100 2008:
   1319 > Reformatted excerpts from marianne.promberger+sup-talk's message of 2008-12-17:
   1320 
   1321 > > (1) Automatically apply labels to incoming mail defined on search
   1322 > > patterns?
   1323 > 
   1324 > Stuff like that is handled by Sup's hook system. 
   1325 
   1326 Thanks! I found more examples on the wiki. Is there any more
   1327 documentation on the hook system? I've found
   1328 /var/lib/gems/1.8/doc/sup-0.6/rdoc/index.html 
   1329 but I have trouble getting anything out of it. Sorry if this is
   1330 obvious to others, but I am not a programmer and haven't used Ruby
   1331 before.
   1332 
   1333 I cobbled two together, but I'm less than sure that they're correct;
   1334 they seem to work for me. If someone could give me the go-ahead, I'd
   1335 post them to the wiki, since I think it would be nice to have some
   1336 more recipes for others to copy.
   1337 
   1338 This one assigns labels according to X-Label header. I'd like to do
   1339 this since I already add some labels with procmail before mail goes to
   1340 the maildir that is a source for sup. (Not sure the regex matching the
   1341 label is exhaustive enough for everyone; it is for me).
   1342 
   1343 if message.raw_header =~ /X-Label: /
   1344   xlabelheader = message.raw_header[/X-Label:.*/]
   1345   xlabelheader.scan(/ [a-z0-9\-_+]+/) { |x| message.add_label x.lstrip }
   1346 end
   1347 
   1348 The second one assigns labels based on an external file containing a
   1349 list of e-mail addresses. The external file is one I maintain from
   1350 mutt; I have a macro set up to quickly add the sender of a message to
   1351 my "private" group.
   1352 
   1353 privatfile = File.open("/home/mpromber/.mutt/privataddr","r")
   1354 if ! privatfile.grep(/#{message.from.email}/).empty?
   1355 message.add_label :privat
   1356 end
   1357 
   1358 Another question: If I add a new rule like this to the
   1359 before-add-message hook, what is the recommended way to get sup to
   1360 rescan messages to apply this rule to all existing mail? 
   1361 
   1362 I am currently using "sup-sync -a --discard -e -x -v", but this takes
   1363 forever (and I can't use regular "sup" in the meantime). (I'm using the
   1364 -e because "sup-sync" on its own marks all mail as unread, even just
   1365 running "sup-sync -c".) Is it correct that there is no faster way?
   1366 
   1367 > > (2) Save search history across sup sessions, so that I can use
   1368 > > up-arrow to access terms I searched for last time I read mail in sup?
   1369 > 
   1370 > Not implemented, but a good idea, and probably pretty easy!
   1371 
   1372 I've been thinking about this some more, and maybe even better would
   1373 be not to save all searches, but to have the possibility to explicitly
   1374 save some searches. This would turn the saved searches into a list of
   1375 quickly accessible virtual mail folders. Maybe a shortcut to do this
   1376 could be added to the buffer-list-mode? To start, it could even just
   1377 be an external text file that has to be edited manually. Sorry I can't
   1378 write patches in ruby :) 
   1379 
   1380 > You can search for things like "from:me" or "to:me". That should work
   1381 > with :alternates, but unfortunately doesn't work if you use :regexen.
   1382 
   1383 Thanks. Another question: Is there a search term like "from:" and
   1384 "to:" but that stands for "anywhere in "from" or "to" or "cc" (like
   1385 "a:" in mairix)?
   1386 
   1387 In general, is there more documentation about the search capacities,
   1388 especially on building search terms, and where could I find it?
   1389 
   1390 One more question: Is there some way to have more fine-grained control
   1391 over what the "From" e-mail address is when replying? As far as I
   1392 understand it, I can have ":accounts:" sections in config.yaml, with
   1393 different "alternates". Sup will reply from the e-mail address that
   1394 someone uses as "to:" when e-mailing me. But for example, I'd like to
   1395 use my "myname+sup-talk at gmail.com" address when replying to this list,
   1396 but obviously, mail to this list has "sup-talk..." as the "to:"
   1397 address. And the point of the "accounts" in config.yaml is just
   1398 picking the right signature, right? I haven't discovered any other
   1399 effect so far.
   1400 
   1401 Thanks,
   1402 
   1403 Marianne
   1404 
   1405 From marc.hartstein@alum.vassar.edu  Mon Dec 29 13:10:48 2008
   1406 From: marc.hartstein@alum.vassar.edu (Marc Hartstein)
   1407 Date: Mon, 29 Dec 2008 13:10:48 -0500
   1408 Subject: [sup-talk] a few sup newbie questions
   1409 In-Reply-To: <1230571384-sup-9910@audrey>
   1410 References: <1229548441-sup-531@audrey> <1229552235-sup-1046@entry>
   1411 	<1230571384-sup-9910@audrey>
   1412 Message-ID: <1230573934-sup-5232@cabinet>
   1413 
   1414 Excerpts from marianne.promberger+sup-talk's message of Mon Dec 29 12:27:19 -0500 2008:
   1415 >
   1416 > One more question: Is there some way to have more fine-grained control
   1417 > over what the "From" e-mail address is when replying? As far as I
   1418 
   1419 Check out the reply-from hook; it's passed the message you're replying
   1420 to so you can do any processing you want on it and return a default
   1421 from address.
   1422 -------------- next part --------------
   1423 A non-text attachment was scrubbed...
   1424 Name: signature.asc
   1425 Type: application/pgp-signature
   1426 Size: 197 bytes
   1427 Desc: not available
   1428 URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20081229/66eb91a1/attachment.bin>
   1429 
   1430 From marianne.promberger+sup-talk@gmail.com  Tue Dec 30 04:53:47 2008
   1431 From: marianne.promberger+sup-talk@gmail.com (Marianne)
   1432 Date: Tue, 30 Dec 2008 10:53:47 +0100
   1433 Subject: [sup-talk] a few sup newbie questions
   1434 In-Reply-To: <1230573934-sup-5232@cabinet>
   1435 References: <1229548441-sup-531@audrey> <1229552235-sup-1046@entry>
   1436 	<1230571384-sup-9910@audrey> <1230573934-sup-5232@cabinet>
   1437 Message-ID: <1230630305-sup-9424@audrey>
   1438 
   1439 Excerpts from Marc Hartstein's message of Mon Dec 29 19:10:48 +0100 2008:
   1440 > Excerpts from marianne.promberger+sup-talk's message of Mon Dec 29 12:27:19 -0500 2008:
   1441 > >
   1442 > > One more question: Is there some way to have more fine-grained control
   1443 > > over what the "From" e-mail address is when replying? As far as I
   1444 > 
   1445 > Check out the reply-from hook; it's passed the message you're replying
   1446 > to so you can do any processing you want on it and return a default
   1447 > from address.
   1448 
   1449 Thanks. Any chance you could give me a pointer on how I can get it to
   1450 "return a person"?
   1451 
   1452 I've tried stuff like ... (in ~/.sup/hook/reply-from.rb)
   1453 
   1454 if message.to =~ /rubyforge/
   1455   hook_reply_from = "My Name <email at domain.com>"
   1456 end
   1457 
   1458 if message.recipient_email =~ /rubyforge/
   1459    return "My name <email at domain.com>"
   1460 end
   1461 
   1462 ... with different variations of patterns I'm testing for and with
   1463 different returned strings. 
   1464 
   1465 Any pointers appreciated! (Including general information where I could
   1466 RTFM ..  I looked at "sup -l" but that's pretty brief.
   1467 
   1468 Marianne
   1469