* [sup-talk] rethinking sup part ii
@ 2008-07-20 21:33 William Morgan
2008-07-20 23:28 ` Guillaume Quintard
` (4 more replies)
0 siblings, 5 replies; 24+ messages in thread
From: William Morgan @ 2008-07-20 21:33 UTC (permalink / raw)
In which the new version of Sup is described!
http://all-thing.net/2008/07/rethinking-sup-part-ii.html
--
William <wmorgan-sup at masanjin.net>
^ permalink raw reply [flat|nested] 24+ messages in thread* [sup-talk] rethinking sup part ii 2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan @ 2008-07-20 23:28 ` Guillaume Quintard 2008-07-21 0:04 ` William Morgan 2008-07-21 12:14 ` Peter Krenn ` (3 subsequent siblings) 4 siblings, 1 reply; 24+ messages in thread From: Guillaume Quintard @ 2008-07-20 23:28 UTC (permalink / raw) Excerpts from William Morgan's message of Sun Jul 20 14:33:24 -0700 2008: > In which the new version of Sup is described! > > http://all-thing.net/2008/07/rethinking-sup-part-ii.html looks cool to me (mostly the server/client idea). I just have one question: I am currently using gmail+mpop, and giving the mbox file to sup. The new sup won't change a lot for me, will it? (maybe a stupid question, but I'd like to be sure I understood the implications). -- Guillaume ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-20 23:28 ` Guillaume Quintard @ 2008-07-21 0:04 ` William Morgan 2008-07-21 5:43 ` Guillaume Quintard 0 siblings, 1 reply; 24+ messages in thread From: William Morgan @ 2008-07-21 0:04 UTC (permalink / raw) Reformatted excerpts from guillaume.quintard's message of 2008-07-20: > I just have one question: I am currently using gmail+mpop, and giving > the mbox file to sup. The new sup won't change a lot for me, will it? > (maybe a stupid question, but I'd like to be sure I understood the > implications). It should be basically the same, except that you can throw away the mbox file when you're done (because STS will handle the storage for you). -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-21 0:04 ` William Morgan @ 2008-07-21 5:43 ` Guillaume Quintard 0 siblings, 0 replies; 24+ messages in thread From: Guillaume Quintard @ 2008-07-21 5:43 UTC (permalink / raw) Excerpts from William Morgan's message of Sun Jul 20 17:04:35 -0700 2008: > It should be basically the same, except that you can throw away the mbox > file when you're done (because STS will handle the storage for you). Ok, that was what I understood. I'd keep the mbox anyway, having a standard backup format is always nice. -- Guillaume ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan 2008-07-20 23:28 ` Guillaume Quintard @ 2008-07-21 12:14 ` Peter Krenn 2008-07-21 15:22 ` William Morgan 2008-07-21 18:43 ` Lionel Ott ` (2 subsequent siblings) 4 siblings, 1 reply; 24+ messages in thread From: Peter Krenn @ 2008-07-21 12:14 UTC (permalink / raw) Sounds great. What kind of interface will this service provide? Or will it be more like a library which one could use to access the sup storage from any application? Something like a specialized database? ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-21 12:14 ` Peter Krenn @ 2008-07-21 15:22 ` William Morgan 2008-07-21 15:26 ` Stephen Patterson 0 siblings, 1 reply; 24+ messages in thread From: William Morgan @ 2008-07-21 15:22 UTC (permalink / raw) Reformatted excerpts from Peter Krenn's message of 2008-07-21: > What kind of interface will this service provide? Or will it be more > like a library which one could use to access the sup storage from any > application? Something like a specialized database? It will be a network service with Thrift defining the interface. -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-21 15:22 ` William Morgan @ 2008-07-21 15:26 ` Stephen Patterson 2008-07-21 16:04 ` John Bent 2008-07-22 2:12 ` William Morgan 0 siblings, 2 replies; 24+ messages in thread From: Stephen Patterson @ 2008-07-21 15:26 UTC (permalink / raw) On 21 Jul 08, William Morgan (wmorgan-sup at masanjin.net) wrote: > Reformatted excerpts from Peter Krenn's message of 2008-07-21: > > What kind of interface will this service provide? Or will it be more > > like a library which one could use to access the sup storage from any > > application? Something like a specialized database? > > It will be a network service with Thrift defining the interface. So we'd then need a collection of custom client applications to search and/or browse the index? This sounds quite a bit like the beagle project - http://beagle-project.org/Main_Page -- Stephen Patterson :: steve at patter.mine.nu :: http://patter.mine.nu/ GPG: B416F0DE :: Jabber: patter at jabber.earth.li "Don't be silly, Minnie. Who'd be walking round these cliffs with a gas oven?" -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Digital signature URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080721/aade96ff/attachment.bin> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-21 15:26 ` Stephen Patterson @ 2008-07-21 16:04 ` John Bent 2008-07-22 2:12 ` William Morgan 1 sibling, 0 replies; 24+ messages in thread From: John Bent @ 2008-07-21 16:04 UTC (permalink / raw) Excerpts from Stephen Patterson's message of Mon Jul 21 09:26:59 -0600 2008: > On 21 Jul 08, William Morgan (wmorgan-sup at masanjin.net) wrote: > > Reformatted excerpts from Peter Krenn's message of 2008-07-21: > > > What kind of interface will this service provide? Or will it be more > > > like a library which one could use to access the sup storage from any > > > application? Something like a specialized database? > > > > It will be a network service with Thrift defining the interface. > > So we'd then need a collection of custom client applications to search > and/or browse the index? > > This sounds quite a bit like the beagle project - > http://beagle-project.org/Main_Page > And google desktop: http://desktop.google.com And spotlight: http://en.wikipedia.org/wiki/Spotlight_(software) And MS Fast Find: http://support.microsoft.com/support/kb/articles/Q166/3/02.ASP There is room for improvement in all of these tools but the margin seems considerably smaller than it did for the sup email client which is vastly superior to anything else. John ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-21 15:26 ` Stephen Patterson 2008-07-21 16:04 ` John Bent @ 2008-07-22 2:12 ` William Morgan 2008-07-23 17:40 ` Nicolas Pouillard 1 sibling, 1 reply; 24+ messages in thread From: William Morgan @ 2008-07-22 2:12 UTC (permalink / raw) Reformatted excerpts from Stephen Patterson's message of 2008-07-21: > So we'd then need a collection of custom client applications to search > and/or browse the index? > > This sounds quite a bit like the beagle project - > http://beagle-project.org/Main_Page Thanks for the pointer. It does sound quite similar. Having played with Beagle a bit, I think they key difference is that I'm envisioning STS as being the primary interface to your precious infos, whereas Beagle seems more of a supplemental index of data that's primarily handled by other apps. This has a couple implications, including who's responsible for storing the files (STS: STS; Beagle: you and your apps), and what the user experience looks like (Beagle: use Thunderbird, then call up Beagle for help; STS: browse everything through your client, starting with a search for label "inbox", and call up other applications for handling foreign mime types). If anything, Beagle is closer to Sup classic, and STS brings us closer to Gmail. -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-22 2:12 ` William Morgan @ 2008-07-23 17:40 ` Nicolas Pouillard 2008-07-23 22:26 ` William Morgan 0 siblings, 1 reply; 24+ messages in thread From: Nicolas Pouillard @ 2008-07-23 17:40 UTC (permalink / raw) Excerpts from William Morgan's message of Tue Jul 22 04:12:09 +0200 2008: > Reformatted excerpts from Stephen Patterson's message of 2008-07-21: > > So we'd then need a collection of custom client applications to search > > and/or browse the index? > > > > This sounds quite a bit like the beagle project - > > http://beagle-project.org/Main_Page > > Thanks for the pointer. It does sound quite similar. Having played with > Beagle a bit, I think they key difference is that I'm envisioning STS as > being the primary interface to your precious infos, whereas Beagle seems > more of a supplemental index of data that's primarily handled by other > apps. > > This has a couple implications, including who's responsible for storing > the files (STS: STS; Beagle: you and your apps), and what the user > experience looks like (Beagle: use Thunderbird, then call up Beagle for > help; STS: browse everything through your client, starting with a > search for label "inbox", and call up other applications for handling > foreign mime types). > > If anything, Beagle is closer to Sup classic, and STS brings us closer > to Gmail. Another design choice, that could (and perhaps *should*) be made; is that stored data don't change in the case of mails, IM..., only their labels and children (threading) do. Then, this makes even more differences, general file system indexers have an harder task since, files change all the time this imply more complex data structures that often provides worse complexity than one could wish. Cheers, -- Nicolas Pouillard aka Ertai -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 194 bytes Desc: not available URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080723/4e09fbde/attachment.bin> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-23 17:40 ` Nicolas Pouillard @ 2008-07-23 22:26 ` William Morgan 2008-07-24 9:16 ` Nicolas Pouillard 0 siblings, 1 reply; 24+ messages in thread From: William Morgan @ 2008-07-23 22:26 UTC (permalink / raw) Reformatted excerpts from nicolas.pouillard's message of 2008-07-23: > Another design choice, that could (and perhaps *should*) be made; is > that stored data don't change in the case of mails, IM..., only their > labels and children (threading) do. I think that's a true fact. Draft messages are sort of an exception, but editing can always be simulated as deleting the old one and adding the new one. -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-23 22:26 ` William Morgan @ 2008-07-24 9:16 ` Nicolas Pouillard 0 siblings, 0 replies; 24+ messages in thread From: Nicolas Pouillard @ 2008-07-24 9:16 UTC (permalink / raw) Excerpts from William Morgan's message of Thu Jul 24 00:26:18 +0200 2008: > Reformatted excerpts from nicolas.pouillard's message of 2008-07-23: > > Another design choice, that could (and perhaps *should*) be made; is > > that stored data don't change in the case of mails, IM..., only their > > labels and children (threading) do. > > I think that's a true fact. Draft messages are sort of an exception, but > editing can always be simulated as deleting the old one and adding the > new one. Perhaps that drafts should be marked as "short-lived", and left in another store. This reminds me how generational Garbage Collectors works, there is a kind of relation... -- Nicolas Pouillard aka Ertai -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 194 bytes Desc: not available URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080724/2cc5f51a/attachment.bin> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan 2008-07-20 23:28 ` Guillaume Quintard 2008-07-21 12:14 ` Peter Krenn @ 2008-07-21 18:43 ` Lionel Ott 2008-07-23 8:45 ` Richard Heycock 2008-07-24 4:55 ` Guarded Identity 4 siblings, 0 replies; 24+ messages in thread From: Lionel Ott @ 2008-07-21 18:43 UTC (permalink / raw) Excerpts from William Morgan's message of Sun Jul 20 23:33:24 +0200 2008: > In which the new version of Sup is described! > > http://all-thing.net/2008/07/rethinking-sup-part-ii.html Sounds nice overall but something I couldn't infer from the post was if you're thinking about doing just the reading and searching part of sup or if some of the mail functionality is going to be kept. Basically will sts be able to send mail or would one have to send it using some other client and sts then would present all that nicely to one, given you feed it with your mails. ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan ` (2 preceding siblings ...) 2008-07-21 18:43 ` Lionel Ott @ 2008-07-23 8:45 ` Richard Heycock 2008-07-23 8:58 ` Nicolas Pouillard 2008-07-23 17:09 ` William Morgan 2008-07-24 4:55 ` Guarded Identity 4 siblings, 2 replies; 24+ messages in thread From: Richard Heycock @ 2008-07-23 8:45 UTC (permalink / raw) Excerpts from William Morgan's message of Mon Jul 21 07:33:24 +1000 2008: > In which the new version of Sup is described! > > http://all-thing.net/2008/07/rethinking-sup-part-ii.html I really like the idea and I have to say that I'm *very* glad that you are keeping the ncurses interface. My one concern is that you are thinking of using Sphinx and as a consequence you are relient on a relational database. Is that really what you want for a mail client? I find this disturbing on two levels: having a full blown database system for email seems like overkill and the idea of an index bring based on a relational model quite appaling. There are a number of other indexs such as Xapian or Hyper Estraier both have ruby bindings and both are quite mature and widely used in other projects. Just my two cents. rgh -- +61 (0) 410 646 369 [e]: rgh at neoss.com.au [im]: rgh at jabber.org You're worried criminals will continue to penetrate into cyberspace, and I'm worried complexity, poor design and mismanagement will be there to meet them - Marcus Ranum ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-23 8:45 ` Richard Heycock @ 2008-07-23 8:58 ` Nicolas Pouillard 2008-07-23 9:27 ` teroz 2008-07-23 17:09 ` William Morgan 1 sibling, 1 reply; 24+ messages in thread From: Nicolas Pouillard @ 2008-07-23 8:58 UTC (permalink / raw) Excerpts from Richard Heycock's message of Wed Jul 23 10:45:06 +0200 2008: > Excerpts from William Morgan's message of Mon Jul 21 07:33:24 +1000 2008: > > In which the new version of Sup is described! > > > > http://all-thing.net/2008/07/rethinking-sup-part-ii.html > > I really like the idea and I have to say that I'm *very* glad that you > are keeping the ncurses interface. > > My one concern is that you are thinking of using Sphinx and as a > consequence you are relient on a relational database. Is that really > what you want for a mail client? I find this disturbing on two levels: > having a full blown database system for email seems like overkill and > the idea of an index bring based on a relational model quite appaling. > > There are a number of other indexs such as Xapian or Hyper Estraier > both have ruby bindings and both are quite mature and widely used in > other projects. I'm also concerned by using relying on relational databases for a mail service. -- Nicolas Pouillard aka Ertai -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 194 bytes Desc: not available URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080723/0a4a45de/attachment.bin> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-23 8:58 ` Nicolas Pouillard @ 2008-07-23 9:27 ` teroz 2008-07-23 21:41 ` Richard Heycock 0 siblings, 1 reply; 24+ messages in thread From: teroz @ 2008-07-23 9:27 UTC (permalink / raw) Well surely if we r eschewing typical mail storage formats like mbox and IMAP then a database is the way to go unless your suggesting we write some form of storage library as well. On Wed, Jul 23, 2008 at 10:58 AM, Nicolas Pouillard < nicolas.pouillard at gmail.com> wrote: > Excerpts from Richard Heycock's message of Wed Jul 23 10:45:06 +0200 2008: > > Excerpts from William Morgan's message of Mon Jul 21 07:33:24 +1000 2008: > > > In which the new version of Sup is described! > > > > > > http://all-thing.net/2008/07/rethinking-sup-part-ii.html > > > > I really like the idea and I have to say that I'm *very* glad that you > > are keeping the ncurses interface. > > > > My one concern is that you are thinking of using Sphinx and as a > > consequence you are relient on a relational database. Is that really > > what you want for a mail client? I find this disturbing on two levels: > > having a full blown database system for email seems like overkill and > > the idea of an index bring based on a relational model quite appaling. > > > > There are a number of other indexs such as Xapian or Hyper Estraier > > both have ruby bindings and both are quite mature and widely used in > > other projects. > > I'm also concerned by using relying on relational databases for a mail > service. > > -- > Nicolas Pouillard aka Ertai > > _______________________________________________ > sup-talk mailing list > sup-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/sup-talk > > -- --Teroz -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080723/0dfa1626/attachment.html> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-23 9:27 ` teroz @ 2008-07-23 21:41 ` Richard Heycock 0 siblings, 0 replies; 24+ messages in thread From: Richard Heycock @ 2008-07-23 21:41 UTC (permalink / raw) Excerpts from terence.namusonge's message of Wed Jul 23 19:27:46 +1000 2008: > Well surely if we r eschewing typical mail storage formats like mbox and > IMAP then a database is the way to go > unless your suggesting we write some form of storage library as well. Two things: I do not have a problem with a database per se, I do however have a problem with a standalone relational database. There are many embedded databases that work a treat and the user never need know about them. That's pretty much what mbox, Maildir, etc are. The other point is that in using Sphinx the STS would not be using the database directly but would require it simply because the index required it. As I mentioned in my previous email there are a number of other indexes which will serve the purpose without the unnecessary complexity of a standalone database system. rgh > On Wed, Jul 23, 2008 at 10:58 AM, Nicolas Pouillard < > nicolas.pouillard at gmail.com> wrote: > > > Excerpts from Richard Heycock's message of Wed Jul 23 10:45:06 +0200 2008: > > > Excerpts from William Morgan's message of Mon Jul 21 07:33:24 +1000 2008: > > > > In which the new version of Sup is described! > > > > > > > > http://all-thing.net/2008/07/rethinking-sup-part-ii.html > > > > > > I really like the idea and I have to say that I'm *very* glad that you > > > are keeping the ncurses interface. > > > > > > My one concern is that you are thinking of using Sphinx and as a > > > consequence you are relient on a relational database. Is that really > > > what you want for a mail client? I find this disturbing on two levels: > > > having a full blown database system for email seems like overkill and > > > the idea of an index bring based on a relational model quite appaling. > > > > > > There are a number of other indexs such as Xapian or Hyper Estraier > > > both have ruby bindings and both are quite mature and widely used in > > > other projects. > > > > I'm also concerned by using relying on relational databases for a mail > > service. > > > > -- > > Nicolas Pouillard aka Ertai > > > > _______________________________________________ > > sup-talk mailing list > > sup-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/sup-talk > > > > > -- +61 (0) 410 646 369 [e]: rgh at neoss.com.au [im]: rgh at jabber.org You're worried criminals will continue to penetrate into cyberspace, and I'm worried complexity, poor design and mismanagement will be there to meet them - Marcus Ranum ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-23 8:45 ` Richard Heycock 2008-07-23 8:58 ` Nicolas Pouillard @ 2008-07-23 17:09 ` William Morgan 2008-07-25 12:11 ` Marcus Williams 1 sibling, 1 reply; 24+ messages in thread From: William Morgan @ 2008-07-23 17:09 UTC (permalink / raw) Reformatted excerpts from Richard Heycock's message of 2008-07-23: > My one concern is that you are thinking of using Sphinx and as a > consequence you are relient on a relational database. Is that really > what you want for a mail client? I find this disturbing on two levels: > having a full blown database system for email seems like overkill and > the idea of an index bring based on a relational model quite appaling. I feel exactly the same way. I'm using Sphinx's XML interface to add things to the index, so there's no RDBMS involved. I hate XML but not as much as I hate RDBMS. The current plan is to actually store email/documents in some kind of persistent key-value store. I have a preliminary very simple implementation that just uses files on disk, but it will be possible to swap that out to something like TokyoCabinet or AWS. > There are a number of other indexs such as Xapian or Hyper Estraier > both have ruby bindings and both are quite mature and widely used in > other projects. Interesting. I will check those out. I only picked Sphinx because it seemed popular and robust, but it is currently making my life more difficult than it really needs to be. -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-23 17:09 ` William Morgan @ 2008-07-25 12:11 ` Marcus Williams 0 siblings, 0 replies; 24+ messages in thread From: Marcus Williams @ 2008-07-25 12:11 UTC (permalink / raw) On 23.7.2008, William Morgan wrote: > > There are a number of other indexs such as Xapian or Hyper Estraier > > both have ruby bindings and both are quite mature and widely used in > > other projects. > > Interesting. I will check those out. I only picked Sphinx because it > seemed popular and robust, but it is currently making my life more difficult > than it really needs to be. You might enjoy datamapper - http://datamapper.org I've been knocking up web apps in ruby/sinatra/datamapper in stupidly few lines with absolutely no contact with the db underlying the system other than through datamapper. Marcus ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan ` (3 preceding siblings ...) 2008-07-23 8:45 ` Richard Heycock @ 2008-07-24 4:55 ` Guarded Identity 2008-07-24 9:21 ` Nicolas Pouillard 2008-07-25 7:18 ` William Morgan 4 siblings, 2 replies; 24+ messages in thread From: Guarded Identity @ 2008-07-24 4:55 UTC (permalink / raw) Excerpts from wmorgan-sup-at-masanjin.net wmorgan-sup-at-masanjin.net |Sup_Talk|'s message of Sun Jul 20 16:33:24 -0500 2008: > In which the new version of Sup is described! > > http://all-thing.net/2008/07/rethinking-sup-part-ii.html Your first blog was intriguing, but this second one is putting Sup in a direction I'm really happy with. I'm happy about: Central Server -------------- I like having a central server, so I can run clients from multiple locations. Modifiable Messages ------------------- There was a post I read in this thread indicating that people didn't want to modify documents. But if we handle this as a "delete/re-add" operation, I'm hoping it will be fine. In all honesty, the modifications I'd like to do are - remove attachments. Sometimes at work, people send out some absolutely useless, gigantic attachments. - delete/expire documents. Documents Beyond E-mail ----------------------- I'm /really/ interested in storing different types of "documents" like RSS. . . maybe newsgroups too? Then I could stop using snownews and slrn. One client. . . actually, for this reason, I'd tried out Emacs Gnus, but I wasn't really happy with the experience/complexity. However, I was hoping to see if I could get in a request while you're in the beginning stages of STS development. True Thread-level Labels ------------------------ I was using rss2email, so the number of messages I had was enormous (although I disabled that due to performance problems, Ferret corruption, and the problem I'm discussion here). I hate wasting disk space needlessly, or cluttering the index for that matter. There's some mailing lists that are mostly fluff, but I wanted to save the occasional thread and expire the rest. But because messages attach to messages really, and not to the thread, negation in search logic breaks. If I search for threads that have "!label:save" I can get threads that have "save" messages if they also have a message not labeled "save". Attaching labels to message always bothered me. I'm hoping that with STS, you can consider designing an architecture that lets you attach labels to a thread grouping. I'm hoping it can be done in a way that's not too terribly performing. While I'm at it, I also have a feature request for STC (Sup The Client) Improved Multiple-Label Support ------------------------------- If I'm only using one label for each thread, then really there's no benefit to having labels over folders. So to really get the benefit of labels, it would be nice to be able to attach more than one of them in a more convenient way. There's a couple of ideas I had on how to do this: Label Hierarchies ----------------- if you use one label, it automatically pulls in another. But I wasn't sure how to manage this with the Sup UI. Label Recommendations --------------------- When using tab-complete for labeling, I see all my labels initially. By now, it's an overwhelming list, and I have to hunt for the label I really wanted. What would be nice is if Sup keeps an index of all label combinations (and perhaps their frequencies). Then when one hits tab the first time only labels that are relevant are presented. If those aren't adequate, then one might hit tab to cycle to a listing including the rest of the labels. Or perhaps without cycling, all labels can be listed by frequency; that way, the hunt for the most relevant label won't be so bad. So Sup probably didn't have this kind of feature because of the stronger advocacy for: - automated label attachment with hooks - not relying on labels as much as one relies on the search engine for keyword queries (the GMail way) But at work, people send me too many messages with too many different types of contexts. It would take quite a bit of AI to auto-categorize these messages. Keyword searches are maybe okay for some of these messages, but it's not really perfect. Sometimes I have to run through synonyms for certain concepts. Better label management would really help me at work. Okay, so those are the requests I have thus far. Hopefully they aren't too far off base from what you are interested in doing with Sup/STS. I'm happy to talk through concepts on this list. And although I'm still working up my Ruby skills, I'm happy to try to contribute code. By the way, are you set up to get these kinds of feature requests with ditz? I didn't see a ditz "bugs" database (folder) in my git clone of Sup. Or are you just using ditz internally for your own personal tracking of Sup development? Seems like there's a number of places to issue feature requests: - this mailing list - ditz, if you can help me figure out how best to use it - a FeatureRequest wiki page, but it would be moot if you didn't make a habit of looking at it Just let us know what you prefer. -Sukant ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-24 4:55 ` Guarded Identity @ 2008-07-24 9:21 ` Nicolas Pouillard 2008-07-25 16:00 ` William Morgan 2008-07-25 7:18 ` William Morgan 1 sibling, 1 reply; 24+ messages in thread From: Nicolas Pouillard @ 2008-07-24 9:21 UTC (permalink / raw) Excerpts from Guarded Identity's message of Thu Jul 24 06:55:38 +0200 2008: > Excerpts from wmorgan-sup-at-masanjin.net wmorgan-sup-at-masanjin.net |Sup_Talk|'s message of Sun Jul 20 16:33:24 -0500 2008: > > In which the new version of Sup is described! > > > > http://all-thing.net/2008/07/rethinking-sup-part-ii.html > > Your first blog was intriguing, but this second one is putting Sup in a > direction I'm really happy with. I'm happy about: > > Central Server > -------------- > I like having a central server, so I can run clients from multiple > locations. > > Modifiable Messages > ------------------- > There was a post I read in this thread indicating that people didn't want > to modify documents. But if we handle this as a "delete/re-add" > operation, I'm hoping it will be fine. > > In all honesty, the modifications I'd like to do are > > - remove attachments. Sometimes at work, people send out some > absolutely useless, gigantic attachments. > > - delete/expire documents. > > Documents Beyond E-mail > ----------------------- > I'm /really/ interested in storing different types of "documents" like RSS. > . . maybe newsgroups too? Then I could stop using snownews and slrn. One > client. . . actually, for this reason, I'd tried out Emacs Gnus, but I > wasn't really happy with the experience/complexity. > > However, I was hoping to see if I could get in a request while you're in the > beginning stages of STS development. > > True Thread-level Labels > ------------------------ Here is another big design decision, note that some labels are truly message based: * unread: of course * spam: to be sure that a spam message a response to a ham message don't throw away the whole thread. * starred: sometime it's there to highlight a message and sometimes to select the whole thread. And some others are truly thread based: * inbox * most of users labels * killed Since there is mandatory requirements for message based labels and thread based labels, one can provide both. Perhaps a syntactic distinction would be sufficient: * a case distinction: INBOX vs unread * a special mark: inbox* vs unread (here the star means the repetition on all messages of the thread). * more ideas to find... > I was using rss2email, so the number of messages I had was enormous > (although I disabled that due to performance problems, Ferret corruption, > and the problem I'm discussion here). I hate wasting disk space > needlessly, or cluttering the index for that matter. There's some mailing > lists that are mostly fluff, but I wanted to save the occasional thread and > expire the rest. But because messages attach to messages really, and not > to the thread, negation in search logic breaks. If I search for threads > that have "!label:save" I can get threads that have "save" messages if they > also have a message not labeled "save". > > Attaching labels to message always bothered me. I'm hoping that with STS, > you can consider designing an architecture that lets you attach labels to a > thread grouping. I'm hoping it can be done in a way that's not too > terribly performing. > > While I'm at it, I also have a feature request for STC (Sup The Client) > > Improved Multiple-Label Support > ------------------------------- > If I'm only using one label for each thread, then really there's no benefit > to having labels over folders. So to really get the benefit of labels, it > would be nice to be able to attach more than one of them in a more > convenient way. > > There's a couple of ideas I had on how to do this: > > Label Hierarchies > ----------------- > if you use one label, it automatically pulls in another. But I wasn't > sure how to manage this with the Sup UI. > > Label Recommendations > --------------------- > When using tab-complete for labeling, I see all my labels initially. > By now, it's an overwhelming list, and I have to hunt for the label I > really wanted. What would be nice is if Sup keeps an index of all > label combinations (and perhaps their frequencies). Then when one hits > tab the first time only labels that are relevant are presented. If > those aren't adequate, then one might hit tab to cycle to a listing > including the rest of the labels. Or perhaps without cycling, all > labels can be listed by frequency; that way, the hunt for the most > relevant label won't be so bad. Hum, that's a very interesting feature. This reminds me delicio.us. In the same line, be able to share popular/public labels for discussion could be very nice. Let's imagine that public tagging is done using public:foo labels, users could share some labels using public ones. Then in answer message an X-Public-Labels header is added and then suggested to users when applying labels. > > So Sup probably didn't have this kind of feature because of the stronger > advocacy for: > > - automated label attachment with hooks > > - not relying on labels as much as one relies on the search engine for > keyword queries (the GMail way) > > But at work, people send me too many messages with too many different types > of contexts. It would take quite a bit of AI to auto-categorize these > messages. Keyword searches are maybe okay for some of these messages, but > it's not really perfect. Sometimes I have to run through synonyms for > certain concepts. Better label management would really help me at work. > > Okay, so those are the requests I have thus far. Hopefully they aren't too > far off base from what you are interested in doing with Sup/STS. I'm happy to > talk through concepts on this list. And although I'm still working up my Ruby > skills, I'm happy to try to contribute code. > > By the way, are you set up to get these kinds of feature requests with ditz? I > didn't see a ditz "bugs" database (folder) in my git clone of Sup. Or are you > just using ditz internally for your own personal tracking of Sup development? It's in the bugs branch. > > Seems like there's a number of places to issue feature requests: > > - this mailing list > > - ditz, if you can help me figure out how best to use it > > - a FeatureRequest wiki page, but it would be moot if you didn't make a > habit of looking at it > > Just let us know what you prefer. I'm not William but I would say that the mailing list is the preferred way. Moreover attaching a git-patch for the ditz issue tracker could help. Cheers, -- Nicolas Pouillard aka Ertai -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 194 bytes Desc: not available URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080724/598bc93c/attachment.bin> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-24 9:21 ` Nicolas Pouillard @ 2008-07-25 16:00 ` William Morgan 2008-07-26 7:32 ` Guarded Identity 0 siblings, 1 reply; 24+ messages in thread From: William Morgan @ 2008-07-25 16:00 UTC (permalink / raw) Reformatted excerpts from nicolas.pouillard's message of 2008-07-24: > Here is another big design decision, note that some labels are truly > message based: > > * unread: of course > * spam: to be sure that a spam message a response to a ham message > don't throw away the whole thread. > * starred: sometime it's there to highlight a message and sometimes > to select the whole thread. What's interesting about the labels you've listed is that they all have special semantics, i.e. the server or the client does something special based on whether they're present. Do people have "user" labels that they use to distinguish individual messages, as opposed to individual threads? If not, we could have labels only on threads by having a bunch of boolean flags on messages: read/unread, spam/ham, starred/regular, draft/not-draft, etc. The search syntax would probably change for those features. > Since there is mandatory requirements for message based labels > and thread based labels, one can provide both. Perhaps a syntactic > distinction would be sufficient: > * a case distinction: INBOX vs unread > * a special mark: inbox* vs unread (here the star means the > repetition on all messages of the thread). Having both would be possible too, but it seems overly complicated to me. I think that's my least favorite option right now. This is a good discussion though. I haven't really thought this all through. -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-25 16:00 ` William Morgan @ 2008-07-26 7:32 ` Guarded Identity 0 siblings, 0 replies; 24+ messages in thread From: Guarded Identity @ 2008-07-26 7:32 UTC (permalink / raw) Excerpts from William Morgan's message of Fri Jul 25 11:00:40 -0500 2008: > > Reformatted excerpts from nicolas.pouillard's message of 2008-07-24: > > > Here is another big design decision, note that some labels are truly > > message based: > > > > * unread: of course > > * spam: to be sure that a spam message a response to a ham message > > don't throw away the whole thread. > > * starred: sometime it's there to highlight a message and sometimes > > to select the whole thread. > > What's interesting about the labels you've listed is that they all have > special semantics, i.e. the server or the client does something special > based on whether they're present. > > Do people have "user" labels that they use to distinguish individual > messages, as opposed to individual threads? I got away with applying the "spam" label at the thread level because I never get any spam that's actually part of a thread; it's normally just a stand-alone piece of mail. I'm not sure what other people use starring for, but I just use it to highlight things I need to reply to, and for that purpose, it works fine at a thread-level because most threads only have one response I'm interested in issuing. But I guess I've seen some of those gigantic threads, so maybe starring at the message-level /might/ be useful, but I'd not sweat it if it wasn't possible. Even for "unread", I get away just managing it at the thread-level, because I'm generally marking an entire thread as read. However, I'd imagine someone might want to revert the unread status of part of a thread. However, that would be a nightmare for a large thread. In that case, I'd probably just use the "@" key-binding to revert labels. So all in all, I never use message-level labels, even though I get understand the instance in which they're needed. The major reason for this is that the presentation of the search is at the thread level, so it makes little sense to think of labels at the message level. > If not, we could have labels only on threads by having a bunch of > boolean flags on messages: read/unread, spam/ham, starred/regular, > draft/not-draft, etc. The search syntax would probably change for those > features. This is /exactly/ what I'm hoping for. I could probably come up with other hypothetical abstractions for indexing, but I'm happy as long as at the end of the day I get negation in reasonably fast searches. I know it sounds like a feature that not everyone might use, but the more messages I put in the index, the more I find I myself trying to make use of negation to prune searches. Also, sometimes I like to manage Sup with scripts and having negation allows me to do some more cool things without having to bother William with feature requests for the odd message/label-managing tasks I have. Hopefully you can continue brain-storming what design you want to do to support this. If you run into a problem, maybe we can jump into the design then. > > Since there is mandatory requirements for message based labels > > and thread based labels, one can provide both. Perhaps a syntactic > > distinction would be sufficient: > > * a case distinction: INBOX vs unread > > * a special mark: inbox* vs unread (here the star means the > > repetition on all messages of the thread). > > Having both would be possible too, but it seems overly complicated to > me. I think that's my least favorite option right now. Yeah, I was trying to think of a way to manage both, and it felt like a kludge. Personally, I like the idea of managing as much as we can at the thread-level. -Sukant ^ permalink raw reply [flat|nested] 24+ messages in thread
* [sup-talk] rethinking sup part ii 2008-07-24 4:55 ` Guarded Identity 2008-07-24 9:21 ` Nicolas Pouillard @ 2008-07-25 7:18 ` William Morgan 1 sibling, 0 replies; 24+ messages in thread From: William Morgan @ 2008-07-25 7:18 UTC (permalink / raw) Reformatted excerpts from Guarded Identity's message of 2008-07-23: > In all honesty, the modifications I'd like to do are > > - remove attachments. Sometimes at work, people send out some > absolutely useless, gigantic attachments. > > - delete/expire documents. Sup 2.0 will definitely handle both of these cases. > I'm /really/ interested in storing different types of "documents" like > RSS. . . maybe newsgroups too? Sure. Anything you can munge into Sup's expected format will be supported. I'm expecting, in addition to a variety of clients (well if other people get excited!), a variety of readers, each of which understands some how to get and import into Sup some kind of information. > There's some mailing lists that are mostly fluff, but I wanted to save > the occasional thread and expire the rest. But because messages > attach to messages really, and not to the thread, negation in search > logic breaks. If I search for threads that have "!label:save" I can > get threads that have "save" messages if they also have a message not > labeled "save". The problem here is not really that there are labels on messages, it's that the current search operation is: for each message that matches the query, return the thread that contains it. That works for positive searches, but clearly doesn't work for negative ones. Making negative search efficient is hard. But I don't think it matters whether the labels are per-message or per-thread... > Attaching labels to message always bothered me. How come? Every other property of a thread is calculated from its component messages---subject, recipients, date---so why not the labels as well? It's a nice symmetry. Anyways, I'm not sure what the right answer is yet. It certainly could be per-thread labels. > Label Hierarchies > ----------------- > if you use one label, it automatically pulls in another. But I wasn't > sure how to manage this with the Sup UI. It would be doable... probably through a hook. > When using tab-complete for labeling, I see all my labels initially. > By now, it's an overwhelming list, and I have to hunt for the label I > really wanted. What would be nice is if Sup keeps an index of all > label combinations (and perhaps their frequencies). Then when one > hits tab the first time only labels that are relevant are presented. > If those aren't adequate, then one might hit tab to cycle to a listing > including the rest of the labels. Or perhaps without cycling, all > labels can be listed by frequency; that way, the hunt for the most > relevant label won't be so bad. The last of those options (list all but by frequency) is definitely doable. The other options are probably a fair amount of work. You can play around with LabelSearchMode and LabelManager if you want to try and mock something up. I don't expect those to change too much, except that LabelManager will probably become a thin shell talking to the server. > By the way, are you set up to get these kinds of feature requests with > ditz? I didn't see a ditz "bugs" database (folder) in my git clone of > Sup. Or are you just using ditz internally for your own personal > tracking of Sup development? As Nicolas pointed out, it's currently on the bugs branch, but I'm planning to move it back into master and starting to add issues there. I also have tentative plans to move Sup's git repo over to gitorious so that people can see pretty pictures when they clone. > Seems like there's a number of places to issue feature requests: > - this mailing list > - ditz, if you can help me figure out how best to use it > - a FeatureRequest wiki page, but it would be moot if you didn't make a > habit of looking at it > > Just let us know what you prefer. The mailing list, ditz patches, or ditz merge requests are all perfectly acceptable. -- William <wmorgan-sup at masanjin.net> ^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2008-07-26 7:32 UTC | newest] Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan 2008-07-20 23:28 ` Guillaume Quintard 2008-07-21 0:04 ` William Morgan 2008-07-21 5:43 ` Guillaume Quintard 2008-07-21 12:14 ` Peter Krenn 2008-07-21 15:22 ` William Morgan 2008-07-21 15:26 ` Stephen Patterson 2008-07-21 16:04 ` John Bent 2008-07-22 2:12 ` William Morgan 2008-07-23 17:40 ` Nicolas Pouillard 2008-07-23 22:26 ` William Morgan 2008-07-24 9:16 ` Nicolas Pouillard 2008-07-21 18:43 ` Lionel Ott 2008-07-23 8:45 ` Richard Heycock 2008-07-23 8:58 ` Nicolas Pouillard 2008-07-23 9:27 ` teroz 2008-07-23 21:41 ` Richard Heycock 2008-07-23 17:09 ` William Morgan 2008-07-25 12:11 ` Marcus Williams 2008-07-24 4:55 ` Guarded Identity 2008-07-24 9:21 ` Nicolas Pouillard 2008-07-25 16:00 ` William Morgan 2008-07-26 7:32 ` Guarded Identity 2008-07-25 7:18 ` William Morgan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox