* [sup-talk] Handling big mailing lists @ 2009-07-29 6:09 Ilpo Nyyssönen 2009-07-29 12:54 ` Marc Weber 2009-07-29 13:46 ` William Morgan 0 siblings, 2 replies; 5+ messages in thread From: Ilpo Nyyssönen @ 2009-07-29 6:09 UTC (permalink / raw) I'm searching alternatives for my Gnus email + news setup. I don't expect Sup to be able to do everything Gnus can yet, but maybe in future? :) Note that I have never used Gmail. I tried to test Sup, but I wasn't able to get any emails because Sup failed to login to my IMAP server. This happened because the server requires client certificates and it looks like Sup doesn't support those. The bigger question is about handling big amount of mailing list mail. I'm reading many small mailing lists and some big ones. Biggest is linux-kernel. I can split this to two: daily reading routine and expiration. How would my daily reading routine work with Sup? I want to read things in priority order: 1. Personal email 2. Emails related to my programming hobby 3. Emails related to some associations like user groups 4. Mailing lists, also in priority order The order is such that if I need to stop reading, I have read the most important ones already. The order is not static, I want to be able to change it. I have understood that labels are way to get this kind of grouping. Can I get a view where the labels are sorted like this? And can I continue reading the next label in that order after I have finished the one before? Then about the expiration. The linux-kernel mailing list gets so much email that some kind of expiration is a must. Can Sup do such automatic deleting of old emails? Can Sup handle some other process doing such automatic deletion? (I would actually prefer some other process do it.) I'm mostly reading just only few authors from linux-kernel and skipping rest, but I do like to see the rest in the thread view as sometimes some subject is interesting and sometimes some thread gets big and is interesting because of that. How would this work? I would actually recommend reading linux-kernel to test Sup. :) -- Ilpo Nyyss?nen # biny # /* :-) */ ^ permalink raw reply [flat|nested] 5+ messages in thread
* [sup-talk] Handling big mailing lists 2009-07-29 6:09 [sup-talk] Handling big mailing lists Ilpo Nyyssönen @ 2009-07-29 12:54 ` Marc Weber 2009-07-29 13:46 ` William Morgan 1 sibling, 0 replies; 5+ messages in thread From: Marc Weber @ 2009-07-29 12:54 UTC (permalink / raw) You can use hooks to add labels: my "~/.sup/hooks/before-add-message.rb": def matchR(email) not message.recipients.find{ |to| /^#{email}$/i =~ to.email}.nil? end def matchRAddLabel(email, label, inbox = 0) if matchR email then message.add_label label message.remove_label :inbox unless inbox end end def importantFrom(email) message.add_label :Starred if message.from.email == email end importantFrom "info at webkos.de" matchRAddLabel("mod_python at modpython.org","mod-python", 1) matchRAddLabel("mod_python at modpython.org","mod-python", 0) if message.subj =~ /^Project Notification$/ && message.from.email == "info at guru.com" then message.add_label "GURU_PROJECT_NOTIFICATION" message.add_label "delete_after_one_month" end if message.subj =~ /^WEBKOS_/ then message.remove_label :inbox message.add_label "WEBKOS_" end So adding labels onle if a contsraint is met is easy. About deleting mails: There is sup-sync-back. So there must be a way to delete "old" / whatever mails. I haven't used it yet. I'm new to sup myself. Yours Marc Weber ^ permalink raw reply [flat|nested] 5+ messages in thread
* [sup-talk] Handling big mailing lists 2009-07-29 6:09 [sup-talk] Handling big mailing lists Ilpo Nyyssönen 2009-07-29 12:54 ` Marc Weber @ 2009-07-29 13:46 ` William Morgan 2009-07-29 17:24 ` Ilpo Nyyssönen 1 sibling, 1 reply; 5+ messages in thread From: William Morgan @ 2009-07-29 13:46 UTC (permalink / raw) Hi, Excellent set of questions. Handling large volumes of mail is one of my main goals with Sup. Reformatted excerpts from iny+dev's message of 2009-07-28: > I tried to test Sup, but I wasn't able to get any emails because Sup > failed to login to my IMAP server. This happened because the server > requires client certificates and it looks like Sup doesn't support > those. As far as IMAP goes, Sup supports whatever authentication the Ruby IMAP libraries supports, which is probably not much. Any serious use of Sup with IMAP is best done via an intermediate like offlineimap, which syncs an IMAP folder to a local Maildir folder. The Ruby IMAP libraries, and possibly IMAP itself, is a little too slow for how Sup likes to work. > How would my daily reading routine work with Sup? I want to read > things in priority order: > > 1. Personal email > 2. Emails related to my programming hobby > 3. Emails related to some associations like user groups > 4. Mailing lists, also in priority order Sup gives you two tools: search and labels. Sup will present a list of threads that match any search query. So, each of those steps is possible, to the extent that you can compose a search query that reflects it. You can automatically add labels via arbitrary code, so you have a fair amount of flexibility here. > The order is such that if I need to stop reading, I have read the most > important ones already. The order is not static, I want to be able to > change it. I have understood that labels are way to get this kind of > grouping. Can I get a view where the labels are sorted like this? Sup currently only orders chronologically. It would not be difficult to add a second level of user-defined sorting *on top* of the chronological one, but it would be difficult, if not impossible, to order all email in the index by an arbitrary function. (To be pedantic, if you can come up with a single number for each email, which never changes, and which is known at add time, you can order by that, with some work. But it doesn't sound like that's what you want.) > And can I continue reading the next label in that order after I have > finished the one before? Certainly, but not automatically. You can view one label, and then pick another label to view, etc. I suppose if the above second-round of ordering were implemented, you could view both labels at once and make an ordering function that placed one above the other. > Then about the expiration. The linux-kernel mailing list gets so much > email that some kind of expiration is a must. You don't want to just buy a bigger hard drive? > Can Sup do such automatic deleting of old emails? Can Sup handle some > other process doing such automatic deletion? (I would actually prefer > some other process do it.) That's best left to another process. You'll then have to tell Sup which messages were deleted so that it can remove them from the index. Let me know when you're at this point and I can help you---it will require a brief Ruby script. > I would actually recommend reading linux-kernel to test Sup. :) I read ruby-talk, which is probably not too far off. One last comment: large threads are irritatingly slow to create in thread-view-mode with the Ferret index, but there's a new Xapian index which is fast for this. It's still slightly experimental. -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 5+ messages in thread
* [sup-talk] Handling big mailing lists 2009-07-29 13:46 ` William Morgan @ 2009-07-29 17:24 ` Ilpo Nyyssönen 2009-08-03 17:31 ` William Morgan 0 siblings, 1 reply; 5+ messages in thread From: Ilpo Nyyssönen @ 2009-07-29 17:24 UTC (permalink / raw) William Morgan <wmorgan-sup at masanjin.net> writes: > Sup currently only orders chronologically. It would not be difficult to > add a second level of user-defined sorting *on top* of the chronological > one, but it would be difficult, if not impossible, to order all email in > the index by an arbitrary function. I'm not talking about ordering of emails, I'm talking about ordering of the labels. When I start my daily routine, I want to see a list of labels with numbers telling how many unread emails each has. Labels containing nothing new must be hidden. And this list must be sorted to priority order. Then I want to read those labels through in that order. How many labels would I have? At least tens, probably over hundred. >> And can I continue reading the next label in that order after I have >> finished the one before? > > Certainly, but not automatically. You can view one label, and then pick > another label to view, etc. Picking one from alphabetically sorted list or writing it is not usable. I don't remember the order, I just want to get the next one. >> Then about the expiration. The linux-kernel mailing list gets so much >> email that some kind of expiration is a must. > > You don't want to just buy a bigger hard drive? It's not a disk space problem. The problem is that many things would get slow (or I would need to configure exceptions) and I just do not have any need for them. They are all in searchable mailing list archives anyway. And another point is that I actually prefer not to order mailing lists. It is much easier to just get them with nntp from news.gmane.org. How? I have a nntp to imap gateway and I use it to read also from other nntp servers. The thing here is that nntp servers usually do expire messages and so to be able to use that with Sup, it would need to tolerate the expiration. > That's best left to another process. You'll then have to tell Sup which > messages were deleted so that it can remove them from the index. So, I don't do the expiration. I guess it should be possible to find out the ones that went away when checking for new ones. > Let me know when you're at this point and I can help you---it will > require a brief Ruby script. Well, it is looking like in current state I won't even start trying. >> I would actually recommend reading linux-kernel to test Sup. :) > > I read ruby-talk, which is probably not too far off. Hmm, http://dir.gmane.org/gmane.comp.lang.ruby.general http://dir.gmane.org/gmane.linux.kernel I guess so. How long it takes to index few months of ruby-talk? Could Sup use IMAP search features? -- Ilpo Nyyss?nen # biny # /* :-) */ ^ permalink raw reply [flat|nested] 5+ messages in thread
* [sup-talk] Handling big mailing lists 2009-07-29 17:24 ` Ilpo Nyyssönen @ 2009-08-03 17:31 ` William Morgan 0 siblings, 0 replies; 5+ messages in thread From: William Morgan @ 2009-08-03 17:31 UTC (permalink / raw) Reformatted excerpts from iny+dev's message of 2009-07-29: > I guess so. How long it takes to index few months of ruby-talk? Offhand I see index speeds of 50-80 messages/s depending on index size, backend, etc. > Could Sup use IMAP search features? No, that would require a very different architecture and philosophy. -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2009-08-03 17:31 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2009-07-29 6:09 [sup-talk] Handling big mailing lists Ilpo Nyyssönen 2009-07-29 12:54 ` Marc Weber 2009-07-29 13:46 ` William Morgan 2009-07-29 17:24 ` Ilpo Nyyssönen 2009-08-03 17:31 ` William Morgan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox