* [sup-talk] current state of synching upstream?
@ 2010-04-08 9:40 Ryan Barrett
2010-04-11 17:52 ` Ryan Barrett
2010-04-12 0:57 ` Rich Lane
0 siblings, 2 replies; 24+ messages in thread
From: Ryan Barrett @ 2010-04-08 9:40 UTC (permalink / raw)
To: sup-talk; +Cc: sjh
hi all! i've been looking forward to trying sup since i first heard about it a
couple years ago, and i'm getting pretty close. one of my few remaining
concerns is using sup in parallel with other mail clients, including sup
itself.
i know best practice is to just use one sup installation and no other clients,
but i also know that synching back upstream is a frequent topic here. i've seen
the discussions of sup-sync --changed, sup-sync-back, maildir + offlineimap
etc. i'm mostly looking to hear about the current state. is anyone
successfully running sup synched with another sup or other client? if so, how?
it seems like people have generally agreed that the best approach is to make
sup-sync-back support maildir, and then use offlineimap to sync from maildir
to the source. is that still true? cc'ing scott henson, who mentioned he was
working on maildir sync in january.
http://rubyforge.org/pipermail/sup-talk/2010-January/003761.html
more background:
http://rubyforge.org/pipermail/sup-talk/2009-April/002126.html
http://rubyforge.org/pipermail/sup-talk/2009-July/002567.html
-Ryan
--
http://snarfed.org/
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-04-08 9:40 [sup-talk] current state of synching upstream? Ryan Barrett
@ 2010-04-11 17:52 ` Ryan Barrett
2010-04-12 0:57 ` Rich Lane
1 sibling, 0 replies; 24+ messages in thread
From: Ryan Barrett @ 2010-04-11 17:52 UTC (permalink / raw)
Cc: sup-talk, sjh
ping? anyone successfully using sup in parallel with other client(s)?
scott, are you still working on maildir sync? any updates since january?
On Thu, 8 Apr 2010, Ryan Barrett wrote:
> hi all! i've been looking forward to trying sup since i first heard about it
> a couple years ago, and i'm getting pretty close. one of my few remaining
> concerns is using sup in parallel with other mail clients, including sup
> itself.
>
> i know best practice is to just use one sup installation and no other
> clients, but i also know that synching back upstream is a frequent topic
> here. i've seen the discussions of sup-sync --changed, sup-sync-back, maildir
> + offlineimap etc. i'm mostly looking to hear about the current state. is
> anyone successfully running sup synched with another sup or other client? if
> so, how?
>
> it seems like people have generally agreed that the best approach is to make
> sup-sync-back support maildir, and then use offlineimap to sync from maildir
> to the source. is that still true? cc'ing scott henson, who mentioned he was
> working on maildir sync in january.
>
> http://rubyforge.org/pipermail/sup-talk/2010-January/003761.html
>
> more background:
>
> http://rubyforge.org/pipermail/sup-talk/2009-April/002126.html
>
> http://rubyforge.org/pipermail/sup-talk/2009-July/002567.html
>
> -Ryan
>
> --
> http://snarfed.org/
>
-Ryan
--
http://snarfed.org/
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-04-08 9:40 [sup-talk] current state of synching upstream? Ryan Barrett
2010-04-11 17:52 ` Ryan Barrett
@ 2010-04-12 0:57 ` Rich Lane
2010-04-12 5:02 ` Andrew Pimlott
` (2 more replies)
1 sibling, 3 replies; 24+ messages in thread
From: Rich Lane @ 2010-04-12 0:57 UTC (permalink / raw)
To: Ryan Barrett; +Cc: sup-talk, sjh
Excerpts from Ryan Barrett's message of 2010-04-08 05:40:32 -0400:
> hi all! i've been looking forward to trying sup since i first heard about it a
> couple years ago, and i'm getting pretty close. one of my few remaining
> concerns is using sup in parallel with other mail clients, including sup
> itself.
My primary goal for the 0.12 release is making Sup play nice with other
clients. You can see the work in progress on the maildir branch - it's
getting more stable but it's still rough around the edges. Most of the
changes on this branch are to make Sup gracefully handle cases where
another client has moved or deleted mail out from under us.
Sup will automatically detect changes to the Seen flag in the maildir
and update the message's labels. I haven't decided what to do about the
other direction. Historically Sup has never modified mail sources, but
changing flags in a maildir is safe enough that I might implement it.
Running sup-sync-back has the problem that you can't run the UI at the
same time.
Does anyone use the current sup-sync-back that only support mboxes? If
there are no strong objections then the 0.12 sup-sync-back will only
support maildir.
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [sup-talk] current state of synching upstream?
2010-04-12 0:57 ` Rich Lane
@ 2010-04-12 5:02 ` Andrew Pimlott
2010-04-12 12:11 ` Daemian Mack
2010-04-12 9:53 ` Tero Tilus
2010-04-14 13:00 ` William Morgan
2 siblings, 1 reply; 24+ messages in thread
From: Andrew Pimlott @ 2010-04-12 5:02 UTC (permalink / raw)
To: Rich Lane; +Cc: sup-talk, sjh
Excerpts from Rich Lane's message of Sun Apr 11 17:57:42 -0700 2010:
> Does anyone use the current sup-sync-back that only support mboxes? If
> there are no strong objections then the 0.12 sup-sync-back will only
> support maildir.
It wouldn't kill me to lose it, but I use it, and think on principle it
would be better to have an overlap period.
Andrew
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-04-12 5:02 ` Andrew Pimlott
@ 2010-04-12 12:11 ` Daemian Mack
0 siblings, 0 replies; 24+ messages in thread
From: Daemian Mack @ 2010-04-12 12:11 UTC (permalink / raw)
To: Andrew Pimlott; +Cc: sup-talk, sjh
Excerpts from Andrew Pimlott's message of Mon Apr 12 01:02:07 -0400 2010:
> Excerpts from Rich Lane's message of Sun Apr 11 17:57:42 -0700 2010:
> > Does anyone use the current sup-sync-back that only support mboxes? If
> > there are no strong objections then the 0.12 sup-sync-back will only
> > support maildir.
>
> It wouldn't kill me to lose it, but I use it, and think on principle it
> would be better to have an overlap period.
I use sup-sync-back with mbox. I agree with Andrew that overlap would
be a good thing in principle, though personally speaking, I'll
probably take this change as a nudge to convert to maildir. My mbox
files get pretty large (~80-100Mb) between sup-sync-backs, and working with those
files on my aging mail box (1.6GHz Athlon XP) gets pretty clunky in the meantime.
Also looking forward to more interclient operability!
Any interest in kicking off a simple usage survey to find out how
people are using sup? I'd be really curious to see things like...
- the mailstore formats being used
- whether most people use sup as their sole MUA or as one of many
- how multi-MUA users handle working with multiple clients
- the various daily sup workflows in use
Might be helpful from a "what's going on out there" roadmap
perspective, too.
--
http://daemianmack.com/
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-04-12 0:57 ` Rich Lane
2010-04-12 5:02 ` Andrew Pimlott
@ 2010-04-12 9:53 ` Tero Tilus
2010-04-14 13:00 ` William Morgan
2 siblings, 0 replies; 24+ messages in thread
From: Tero Tilus @ 2010-04-12 9:53 UTC (permalink / raw)
To: sup-talk
Rich Lane, 2010-04-12 03:57:
> Sup will automatically detect changes to the Seen flag in the
> maildir and update the message's labels.
What about Trashed, Draft and Flagged? They have obvious counterparts
in Sup labels too.
> I haven't decided what to do about the other direction. Historically
> Sup has never modified mail sources, but changing flags in a maildir
> is safe enough that I might implement it.
For what I know maildir was designed to handle delivery, deletion and
flag changes "safely" (no mail content corruption) even without
locking.
Updating message status upstream is a major point for multi-agent
users. And I have got the impression that there are quire a few of
them already.
--
Tero Tilus ## 050 3635 235 ## http://tero.tilus.net/
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-04-12 0:57 ` Rich Lane
2010-04-12 5:02 ` Andrew Pimlott
2010-04-12 9:53 ` Tero Tilus
@ 2010-04-14 13:00 ` William Morgan
2010-04-14 14:16 ` Ben Walton
2 siblings, 1 reply; 24+ messages in thread
From: William Morgan @ 2010-04-14 13:00 UTC (permalink / raw)
To: sup-talk
Reformatted excerpts from Rich Lane's message of 2010-04-11:
> Does anyone use the current sup-sync-back that only support mboxes?
I do, but I would consider moving to Maildir to support the common good.
(Assuming that the Maildir format won't simply collapse with the result
of converting 29 million lines of mbox goodness.)
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-04-14 13:00 ` William Morgan
@ 2010-04-14 14:16 ` Ben Walton
2010-04-14 15:57 ` William Morgan
0 siblings, 1 reply; 24+ messages in thread
From: Ben Walton @ 2010-04-14 14:16 UTC (permalink / raw)
To: sup-talk
Excerpts from William Morgan's message of Wed Apr 14 09:00:59 -0400 2010:
> (Assuming that the Maildir format won't simply collapse with the result
> of converting 29 million lines of mbox goodness.)
mbox2maildir is (generally) your friend. We've used it here with some
success. The only issue we had is when the first line in a paragraph
of a mail message starts with /From /. As you've seen from dealing
with mboxes in sup, this isn't necessarily an easy problem to fix...in
our case, we're dealing with mboxes created with the likes of ancient
pine and elm, etc, so we might be worse off than normal mutt-made
mboxes for instance.
A conversion solution that is better, if possible, is to use imapsync,
as it leaves the mbox parsing to the imap software. It requires more
overhead to be useful though[1].
Thanks
-Ben
[1] I'll provide more (dovecot) tips if anyone is interested.
--
Ben Walton
Systems Programmer - CHASS
University of Toronto
C:416.407.5610 | W:416.978.4302
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-04-14 14:16 ` Ben Walton
@ 2010-04-14 15:57 ` William Morgan
2010-04-14 16:08 ` Ben Walton
0 siblings, 1 reply; 24+ messages in thread
From: William Morgan @ 2010-04-14 15:57 UTC (permalink / raw)
To: sup-talk
Reformatted excerpts from Ben Walton's message of 2010-04-14:
> mbox2maildir is (generally) your friend.
Oh, I'm sure the conversion is pretty straight-forward (modulo the
"From_" issues you point out). I just wonder if I'm going to hit limits
on the number of files in a directory or something weird like that.
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-04-14 15:57 ` William Morgan
@ 2010-04-14 16:08 ` Ben Walton
2010-12-15 8:19 ` Matthias Vallentin
0 siblings, 1 reply; 24+ messages in thread
From: Ben Walton @ 2010-04-14 16:08 UTC (permalink / raw)
To: sup-talk
Excerpts from William Morgan's message of Wed Apr 14 11:57:28 -0400 2010:
> Oh, I'm sure the conversion is pretty straight-forward (modulo the
> "From_" issues you point out). I just wonder if I'm going to hit
> limits on the number of files in a directory or something weird like
> that.
You might, but I don't know what that limit would be...My inbox is
currently looking like:
--snip--
bwalton @ pinkfloyd : ~/Maildir
$ lc
total 3.1M
drwx------ 2 bwalton user 496K Apr 29 2008 cur
-rw------- 1 bwalton user 1.2K Apr 28 2008 dovecot-uidlist
drwx------ 2 bwalton user 2.6M Apr 14 12:04 new
drwx------ 2 bwalton user 4.0K Apr 14 12:04 tmp
bwalton @ pinkfloyd : ~/Maildir
$ ls -lA new/ | wc -l
36413
--snip--
Big (notice the size of the new/ directory), but not huge. Poll times
increase linearly though, which one drawback to the current approach
of not moving mail into cur/ when it's detected.
Thanks
-Ben
--
Ben Walton
Systems Programmer - CHASS
University of Toronto
C:416.407.5610 | W:416.978.4302
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-04-14 16:08 ` Ben Walton
@ 2010-12-15 8:19 ` Matthias Vallentin
2010-12-15 17:06 ` James Taylor
0 siblings, 1 reply; 24+ messages in thread
From: Matthias Vallentin @ 2010-12-15 8:19 UTC (permalink / raw)
To: Ben Walton; +Cc: sup-talk
On Wed, Apr 14, 2010 at 12:08:14PM -0400, Ben Walton wrote:
> Big (notice the size of the new/ directory), but not huge. Poll times
> increase linearly though, which one drawback to the current approach
> of not moving mail into cur/ when it's detected.
Does the current version of sup still keep mail in new? As a fairly new
sup user, I am trying to understand the limitations of maildir support
and this one seems to be one.
My traditional mindest is still similar to logrotate, i.e., regularly
copying maildirs into an archive. If this is still an issue, what about
an where mail directories with more than X messages are being
automatically copied to an archive? This would bound the linear poll
times at a configurable threshold. I don't know how this would look like
(adding a new source per rotated maildir does not seem a nice solution)
and how it would interplay with Xapian. For example, I worry that
maildir rotation could involve major index rebuilding.
Matthias
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-12-15 8:19 ` Matthias Vallentin
@ 2010-12-15 17:06 ` James Taylor
2010-12-18 5:12 ` Matthias Vallentin
0 siblings, 1 reply; 24+ messages in thread
From: James Taylor @ 2010-12-15 17:06 UTC (permalink / raw)
To: Matthias Vallentin; +Cc: sup-talk, Ben Walton
Excerpts from Matthias Vallentin's message of Wed Dec 15 08:19:55 +0000 2010:
> Does the current version of sup still keep mail in new? As a fairly new
> sup user, I am trying to understand the limitations of maildir support
> and this one seems to be one.
For what it is worth, I've been using maildir-sync for almost six months
every day, no problems. I exercise it pretty hard since I usually have
at least four imap clients (mobile devices and such) accessing the
maildir simultaneously via dovecot imap. Everything works fine.
Currently using Damien's latest sync up with the mainline from here:
http://git.fensalir.fr/?p=dleone/sup.git;a=shortlog;h=refs/heads/maildir-sync-master
Anybody else using this? It seems ready to integrate.
-- jt
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-12-15 17:06 ` James Taylor
@ 2010-12-18 5:12 ` Matthias Vallentin
2010-12-18 5:25 ` James Taylor
0 siblings, 1 reply; 24+ messages in thread
From: Matthias Vallentin @ 2010-12-18 5:12 UTC (permalink / raw)
To: James Taylor; +Cc: sup-talk, Ben Walton
On Wed, Dec 15, 2010 at 05:06:54PM +0000, James Taylor wrote:
> For what it is worth, I've been using maildir-sync for almost six months
> every day, no problems. I exercise it pretty hard since I usually have
> at least four imap clients (mobile devices and such) accessing the
> maildir simultaneously via dovecot imap.
This looks like the scenario I envision. Two questions:
- Do you use the IMAP clients as read-only clients or do you invoke
sup-sync after the devices modify the maildir (or before polling
in sup)?
- Do you use sup-sync-back when changing message state via sup? From
what I understand, this would be desirable to copy messages from
new to cur in order to avoid a linear increase in poll times with
growing number of mails in new.
Matthias
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [sup-talk] current state of synching upstream?
2010-12-18 5:12 ` Matthias Vallentin
@ 2010-12-18 5:25 ` James Taylor
2010-12-18 19:04 ` Matthias Vallentin
0 siblings, 1 reply; 24+ messages in thread
From: James Taylor @ 2010-12-18 5:25 UTC (permalink / raw)
To: Matthias Vallentin; +Cc: sup-talk, Ben Walton
Excerpts from Matthias Vallentin's message of Sat Dec 18 05:12:16 +0000 2010:
> This looks like the scenario I envision. Two questions:
>
> - Do you use the IMAP clients as read-only clients or do you invoke
> sup-sync after the devices modify the maildir (or before polling
> in sup)?
>
> - Do you use sup-sync-back when changing message state via sup? From
> what I understand, this would be desirable to copy messages from
> new to cur in order to avoid a linear increase in poll times with
> growing number of mails in new.
Neither. The imap clients are read write, the only time this becomes a
problem is when the imap client moves a message from new to cur. Sup
will not be able to find the message until the next time it polls (but
this is mostly seamless, if I use the imap client I just poll and
refresh in sup). This does imply that sup must be polling cur as well as
new.
Similarly, sup-sync-back is not needed because messages are synced back
to the maildir immediately.
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-12-18 5:25 ` James Taylor
@ 2010-12-18 19:04 ` Matthias Vallentin
2010-12-18 19:21 ` Ben Walton
0 siblings, 1 reply; 24+ messages in thread
From: Matthias Vallentin @ 2010-12-18 19:04 UTC (permalink / raw)
To: James Taylor; +Cc: sup-talk, Ben Walton
On Sat, Dec 18, 2010 at 05:25:05AM +0000, James Taylor wrote:
> Neither. The imap clients are read write, the only time this becomes a
> problem is when the imap client moves a message from new to cur.
Is that the only operation the imap clients do? What happens if they
delete messages (or moved them to another source) and sup has that
message already in the index.
> Sup will not be able to find the message until the next time it polls
> (but this is mostly seamless, if I use the imap client I just poll and
> refresh in sup). This does imply that sup must be polling cur as well
> as new.
If sup polls both cur and new then there is no more benefit in keeping
new small to avoid long poll times for large maildirs.
This brings me to my main sup question. What is the best strategy to
avoid overly large maildirs? Say I have around 10 maildirs, each of
which represent a separate source. The classic scheme that comes to mind
(and that I use with Mutt and Mairix) would be some horizontal
partitioning scheme according to time, i.e., creating a new tree of
maildirs at a certain time interval (e.g., quarterly could imply a
naming scheme for the top-level directories like mail-2010-1 to
mail-2010-4). Then, only the most recent partition needs to be polled.
The downside appears to be that each rotation adds, in the above
example, 10 new source entries to sources.yaml and requires switching
of polling in the non-current sources.
Does that sound tractable and aligned with sup's mail philosophy?
Matthias
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-12-18 19:04 ` Matthias Vallentin
@ 2010-12-18 19:21 ` Ben Walton
2010-12-18 20:02 ` Tero Tilus
2010-12-21 6:44 ` Matthias Vallentin
0 siblings, 2 replies; 24+ messages in thread
From: Ben Walton @ 2010-12-18 19:21 UTC (permalink / raw)
To: Matthias Vallentin; +Cc: sup-talk
Excerpts from Matthias Vallentin's message of Sat Dec 18 14:04:27 -0500 2010:
> The downside appears to be that each rotation adds, in the above
> example, 10 new source entries to sources.yaml and requires
> switching of polling in the non-current sources.
Here's my current approach that I've been happy with since I
implemented it back in the summer:
1. Procmail files all mail (regardless of originating source) into
Maildirs like .incoming.%Y.%m. This gives me a new maildir each month
that holds all incoming mail for that month. The basics of the
.procmailrc to do this are:
--snip--
MAILDIR=$HOME/Maildir/
DATEDIR=`date +%Y.%m`
:0
$MAILDIR/.incoming.$DATEDIR/
--snip--
2. I have the following hook setup as after-poll:
--snip--
s = "maildir:/path/to/Maildir/.incoming.#{Date.today.strftime("%Y.%m")}"
unless Redwood::SourceManager.source_for(s)
Redwood::Logger.force_message "Adding new source: #{s}"
Redwood::SourceManager.add_source Recoverable.new(Redwood::Maildir.new(s))
end
--snip--
3. I have almost all of my labelling done via the before-add-message
hook. This gets me the per-mailing-list tags that I would have
applied based on source originally. I almost never apply tags
manually any more.
The only downside to this is that my sources.yaml file needs manual
twiddling at restart to add the sources that were added during
runtime. I've yet to be annoyed enough by this to figure out how to
make these dynamic additions sticky across restarts.
HTH.
-Ben
--
Ben Walton
Systems Programmer - CHASS
University of Toronto
C:416.407.5610 | W:416.978.4302
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [sup-talk] current state of synching upstream?
2010-12-18 19:21 ` Ben Walton
@ 2010-12-18 20:02 ` Tero Tilus
2010-12-18 20:12 ` Ben Walton
2010-12-21 6:44 ` Matthias Vallentin
1 sibling, 1 reply; 24+ messages in thread
From: Tero Tilus @ 2010-12-18 20:02 UTC (permalink / raw)
To: sup-talk
Ben Walton, 2010-12-18 21:21:
> The only downside to this is that my sources.yaml file needs manual
> twiddling at restart to add the sources that were added during
> runtime.
How come? For what I know sup should save sources.yml (the call to
SoureManager.save_sources is in Index.save) upon graceful exit.
That's exactly how sup-add modifies sources.yml.
--
Tero Tilus ## 050 3635 235 ## http://tero.tilus.net/
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-12-18 20:02 ` Tero Tilus
@ 2010-12-18 20:12 ` Ben Walton
0 siblings, 0 replies; 24+ messages in thread
From: Ben Walton @ 2010-12-18 20:12 UTC (permalink / raw)
To: sup-talk
Excerpts from Tero Tilus's message of Sat Dec 18 15:02:46 -0500 2010:
> Ben Walton, 2010-12-18 21:21:
> > The only downside to this is that my sources.yaml file needs manual
> > twiddling at restart to add the sources that were added during
> > runtime.
>
> How come? For what I know sup should save sources.yml (the call to
> SoureManager.save_sources is in Index.save) upon graceful exit.
> That's exactly how sup-add modifies sources.yml.
I'm not sure as I haven't had the time to look into it. I only
discovered this after the first time sup restarted after adding this
hook (a few months later). I rarely restart sup and when I do, I
don't usually have time to poke at the problem (work mail and
all...). I might be able to just make the SourceManager.save_sources
call manually.
Thanks
-Ben
--
Ben Walton
Systems Programmer - CHASS
University of Toronto
C:416.407.5610 | W:416.978.4302
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-12-18 19:21 ` Ben Walton
2010-12-18 20:02 ` Tero Tilus
@ 2010-12-21 6:44 ` Matthias Vallentin
2010-12-21 6:48 ` Matthias Vallentin
2010-12-21 11:01 ` Tero Tilus
1 sibling, 2 replies; 24+ messages in thread
From: Matthias Vallentin @ 2010-12-21 6:44 UTC (permalink / raw)
To: Ben Walton; +Cc: sup-talk
On Sat, Dec 18, 2010 at 02:21:40PM -0500, Ben Walton wrote:
> Here's my current approach that I've been happy with since I
> implemented it back in the summer:
Cool, this is exactly how I expected it to look like!
> --snip--
> s = "maildir:/path/to/Maildir/.incoming.#{Date.today.strftime("%Y.%m")}"
>
> unless Redwood::SourceManager.source_for(s)
> Redwood::Logger.force_message "Adding new source: #{s}"
> Redwood::SourceManager.add_source Recoverable.new(Redwood::Maildir.new(s))
> end
> --snip--
Neither master nor Damien's Maildir clone contain the class Recoverable.
Is this code you added somewhere else?
The only (conceptual) issue I see with this code that you miss sources
when having more than a month of idleness. Say the suffix of s is
2012.10 and the next time you check back is in December, then you'll
miss 2012.11. Now whether anyone of us has the freedom to not check
email for that long is a different question :-).
> The only downside to this is that my sources.yaml file needs manual
> twiddling at restart to add the sources that were added during
> runtime.
As Tilo mentioned, this is weird indeed since sup calls Index.save which
in turn should call SourceManager.save_sources. I'll look into this when
I get a chance.
Matthias
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [sup-talk] current state of synching upstream?
2010-12-21 6:44 ` Matthias Vallentin
@ 2010-12-21 6:48 ` Matthias Vallentin
2010-12-21 11:01 ` Tero Tilus
1 sibling, 0 replies; 24+ messages in thread
From: Matthias Vallentin @ 2010-12-21 6:48 UTC (permalink / raw)
To: Ben Walton; +Cc: sup-talk
On Mon, Dec 20, 2010 at 10:44:45PM -0800, Matthias Vallentin wrote:
> As Tilo mentioned [..]
I was referring to Tero, sorry about the confusion.
Matthias
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-12-21 6:44 ` Matthias Vallentin
2010-12-21 6:48 ` Matthias Vallentin
@ 2010-12-21 11:01 ` Tero Tilus
2010-12-21 14:11 ` Ben Walton
1 sibling, 1 reply; 24+ messages in thread
From: Tero Tilus @ 2010-12-21 11:01 UTC (permalink / raw)
To: sup-talk
Matthias Vallentin, 2010-12-21 08:44:
>> The only downside to this is that my sources.yaml file needs manual
>> twiddling at restart to add the sources that were added during
>> runtime.
>
> As Tero mentioned, this is weird indeed since sup calls Index.save
> which in turn should call SourceManager.save_sources. I'll look into
> this when I get a chance.
...
> Neither master nor Damien's Maildir clone contain the class
> Recoverable. Is this code you added somewhere else?
This sources not getting saved might be because of that Recoverable
wrapper class. YAML marshalling might not be able to digest it.
--
Tero Tilus ## 050 3635 235 ## http://tero.tilus.net/
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [sup-talk] current state of synching upstream?
2010-12-21 11:01 ` Tero Tilus
@ 2010-12-21 14:11 ` Ben Walton
2010-12-22 14:42 ` Matthias Vallentin
0 siblings, 1 reply; 24+ messages in thread
From: Ben Walton @ 2010-12-21 14:11 UTC (permalink / raw)
To: sup-talk
Excerpts from Tero Tilus's message of Tue Dec 21 06:01:59 -0500 2010:
> > Neither master nor Damien's Maildir clone contain the class
> > Recoverable. Is this code you added somewhere else?
>
> This sources not getting saved might be because of that Recoverable
> wrapper class. YAML marshalling might not be able to digest it.
When I wrote the hook, I referenced the following from
lib/sup/source.rb:
def load_sources fn=Redwood::SOURCE_FN
source_array = (Redwood::load_yaml_obj(fn) || []).map { |o| Recoverable.new o }
@source_mutex.synchronize do
@sources = Hash[*(source_array).map { |s| [s.id, s] }.flatten]
@sources_dirty = false
end
end
So, after de-marshalling the yaml file, it wraps the object in a
Recoverable. This class is defined in lib/sup/util.rb.
I'm looking at the code again right now as the Recoverable may be the
key as to why this doesn't get saved...
Thanks
-Ben
--
Ben Walton
Systems Programmer - CHASS
University of Toronto
C:416.407.5610 | W:416.978.4302
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [sup-talk] current state of synching upstream?
2010-12-21 14:11 ` Ben Walton
@ 2010-12-22 14:42 ` Matthias Vallentin
2010-12-22 16:27 ` Tero Tilus
0 siblings, 1 reply; 24+ messages in thread
From: Matthias Vallentin @ 2010-12-22 14:42 UTC (permalink / raw)
To: Ben Walton; +Cc: sup-talk
On Tue, Dec 21, 2010 at 09:11:42AM -0500, Ben Walton wrote:
> I'm looking at the code again right now as the Recoverable may be the
> key as to why this doesn't get saved...
I did some printf-debugging at shutdown time and am now really puzzled
why SourceManager.save_source is not invoked. The function Index.save
and SourceManager.save_source look as follows:
def save
debug "saving index and sources..."
FileUtils.mkdir_p @dir unless File.exists? @dir
-> debug "*** Index.save"
SourceManager.save_sources
save_index
end
def save_sources fn=Redwood::SOURCE_FN
-> debug "*** SourceManager.save_sources"
@source_mutex.synchronize do
...
end
end
Adding the with -> marked statement in Index.save printed, however, not
the debug statement in SourceManager.save_sources! Any ideas how this is
possible?
Matthias
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [sup-talk] current state of synching upstream?
2010-12-22 14:42 ` Matthias Vallentin
@ 2010-12-22 16:27 ` Tero Tilus
0 siblings, 0 replies; 24+ messages in thread
From: Tero Tilus @ 2010-12-22 16:27 UTC (permalink / raw)
To: sup-talk
Matthias Vallentin, 2010-12-22 16:42:
> Adding the with -> marked statement in Index.save printed, however, not
> the debug statement in SourceManager.save_sources! Any ideas how this is
> possible?
I think I have a hunch. SourceManager is a singleton, which (see
util.rb) silently drops all the calls if it is in deinstantiated
state. Could you inject some debugging in Singleton#method_missing to
see if SourceManager is already deinstantiated when #save_sources is
called.
--
Tero Tilus ## 050 3635 235 ## http://tero.tilus.net/
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2010-12-22 16:43 UTC | newest]
Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-04-08 9:40 [sup-talk] current state of synching upstream? Ryan Barrett
2010-04-11 17:52 ` Ryan Barrett
2010-04-12 0:57 ` Rich Lane
2010-04-12 5:02 ` Andrew Pimlott
2010-04-12 12:11 ` Daemian Mack
2010-04-12 9:53 ` Tero Tilus
2010-04-14 13:00 ` William Morgan
2010-04-14 14:16 ` Ben Walton
2010-04-14 15:57 ` William Morgan
2010-04-14 16:08 ` Ben Walton
2010-12-15 8:19 ` Matthias Vallentin
2010-12-15 17:06 ` James Taylor
2010-12-18 5:12 ` Matthias Vallentin
2010-12-18 5:25 ` James Taylor
2010-12-18 19:04 ` Matthias Vallentin
2010-12-18 19:21 ` Ben Walton
2010-12-18 20:02 ` Tero Tilus
2010-12-18 20:12 ` Ben Walton
2010-12-21 6:44 ` Matthias Vallentin
2010-12-21 6:48 ` Matthias Vallentin
2010-12-21 11:01 ` Tero Tilus
2010-12-21 14:11 ` Ben Walton
2010-12-22 14:42 ` Matthias Vallentin
2010-12-22 16:27 ` Tero Tilus
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox