* [sup-talk] sup-server @ 2009-12-08 18:56 Rich Lane 2009-12-08 22:12 ` Andrei Thorp 2009-12-19 19:16 ` William Morgan 0 siblings, 2 replies; 5+ messages in thread From: Rich Lane @ 2009-12-08 18:56 UTC (permalink / raw) To: sup-talk I've pushed a branch sup-server to my github*. There's a lot to be done before this reaches feature parity with current Sup - for instance, the ncurses client doesn't work yet. *git://github.com/rlane/sup Current status: - Server passes its (incomplete) test suite - sup-cmd executable useful for simple interactions (each protocol request exposed) - ncurses client completely nonfunctional - client/server/common source code split - server implemented with Revactor - Ruby 1.9 only - server stores messages (currently gzip'd mbox-ish) Protocol: - Designed to avoid round-trip latencies - BERT over tcp/unix - thrift/etc could be supported in the future - Requests (full docs in lib/sup/server/requests.rb): - Query - Count - Label - Add - Stream - Cancel - Requests take query arguments instead of messages-ids (thanks to Carl/notmuch for this idea) - These requests should be sufficient to implement a working client. We can add more in the future for optimizations (threading) or new features (contacts). TODO: - Expand test suite - Modify sup-sync to send Add requests to the server - Get the ncurses UI working - Remove dead code - Protocol versioning/negotiation/authentication/extensions - Performance optimizations - Add web/android/iphone/vim/irb/etc UIs - Actorize the index/storage interfaces - Shard index/storage - Distribute indexing/parsing/compression/etc across worker processes - Replication? The test suite is an important part of this effort. With the amount of code churn that's going on it just takes too much work to manually verify that every (affected) feature works before committing. If it's not covered by a test, I don't care if it's broken. For now, I'll plan on adding any new UIs to the main repo. When the protocol stabilizes we can think about splitting them out. I would be very interested if someone could spin up a web UI quickly. I'd like to start using this branch for my own mail and it will take a significant amount of time before I can beat the ncurses UI into shape. For some background on sup-server please read William's various blog posts on the subject. In its current form this project makes some compromises for the sake of efficiency and simplicity. I would be interested in making the protocol more generic while preserving those attributes, so if you have thoughts on this send them to the list. Some questions from the IRC channel: - Do clients need threading logic? Yes. There is no thread abstraction in the protocol, so clients will need to understand email threading. A planned optimization is to expose the index thread-id field to essentially collapse the thread for client-threading purposes. - Will a client connecting to sup-server feel snappier than sup over ssh? Yes (given a well-written client). The protocol is designed to minimize the effects of network latency, but it will take a good amount of work in the client to fully take advantage of this. - Do clients need to know how to parse email? Yes. Right now clients who want to display a message need to query for the raw message text. If we can come up with a simple, comprehensive model of an email message that clients would rather digest than the raw text I'd be willing to add it as an optional extension to the protocol. _______________________________________________ sup-talk mailing list sup-talk@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-talk ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-talk] sup-server 2009-12-08 18:56 [sup-talk] sup-server Rich Lane @ 2009-12-08 22:12 ` Andrei Thorp 2009-12-09 2:33 ` Rich Lane 2009-12-19 19:16 ` William Morgan 1 sibling, 1 reply; 5+ messages in thread From: Andrei Thorp @ 2009-12-08 22:12 UTC (permalink / raw) To: sup-talk, rlane On Tue, Dec 8, 2009 at 1:56 PM, Rich Lane <rlane@club.cc.cmu.edu> wrote: > I've pushed a branch sup-server to my github*. There's a lot to be done > before this reaches feature parity with current Sup - for instance, the > ncurses client doesn't work yet. > > *git://github.com/rlane/sup > > Current status: > - Server passes its (incomplete) test suite > - sup-cmd executable useful for simple interactions (each protocol > request exposed) > - ncurses client completely nonfunctional > - client/server/common source code split > - server implemented with Revactor - Ruby 1.9 only > - server stores messages (currently gzip'd mbox-ish) > > Protocol: > - Designed to avoid round-trip latencies > - BERT over tcp/unix > - thrift/etc could be supported in the future > - Requests (full docs in lib/sup/server/requests.rb): > - Query > - Count > - Label > - Add > - Stream > - Cancel > - Requests take query arguments instead of messages-ids (thanks to > Carl/notmuch for this idea) > - These requests should be sufficient to implement a working client. We > can add more in the future for optimizations (threading) or new > features (contacts). > > TODO: > - Expand test suite > - Modify sup-sync to send Add requests to the server > - Get the ncurses UI working > - Remove dead code > - Protocol versioning/negotiation/authentication/extensions > - Performance optimizations > - Add web/android/iphone/vim/irb/etc UIs > - Actorize the index/storage interfaces > - Shard index/storage > - Distribute indexing/parsing/compression/etc across worker processes > - Replication? > > The test suite is an important part of this effort. With the amount of > code churn that's going on it just takes too much work to manually > verify that every (affected) feature works before committing. If it's > not covered by a test, I don't care if it's broken. > > For now, I'll plan on adding any new UIs to the main repo. When the > protocol stabilizes we can think about splitting them out. > > I would be very interested if someone could spin up a web UI quickly. > I'd like to start using this branch for my own mail and it will take a > significant amount of time before I can beat the ncurses UI into shape. > > For some background on sup-server please read William's various blog > posts on the subject. In its current form this project makes some > compromises for the sake of efficiency and simplicity. I would be > interested in making the protocol more generic while preserving those > attributes, so if you have thoughts on this send them to the list. > > Some questions from the IRC channel: > - Do clients need threading logic? > Yes. There is no thread abstraction in the protocol, so clients will > need to understand email threading. A planned optimization is to > expose the index thread-id field to essentially collapse the thread > for client-threading purposes. > > - Will a client connecting to sup-server feel snappier than sup over > ssh? > Yes (given a well-written client). The protocol is designed to > minimize the effects of network latency, but it will take a good > amount of work in the client to fully take advantage of this. > > - Do clients need to know how to parse email? > Yes. Right now clients who want to display a message need to query for > the raw message text. If we can come up with a simple, comprehensive > model of an email message that clients would rather digest than the > raw text I'd be willing to add it as an optional extension to the > protocol. > _______________________________________________ > sup-talk mailing list > sup-talk@rubyforge.org > http://rubyforge.org/mailman/listinfo/sup-talk > While I'm as usual very impressed that Rich has done so much work, and very happy with the ideas proposed in the STS posts, I can't help but wonder, "Why does this sound like implementing a Sup ui to Wave, but not using Wave?" (Actually mentioned in Rethinking Sup pt. 2) This really seems like it's in direct competition with the Wave system... but Wave has a lot more support. And yes, Wave doesn't have e-mail federation at the moment, but I think they plan to do that in the future. So yes, I like the idea of a Sup server, I like being able to access my (sup) mail from multiple computers, I like the idea of having multiple UIs to the Sup Service... but I think that in the long run, I'd feel just as good about a couple good open source UIs to Google's Wave instead. Honestly, I hope I'm wrong since there seems to be a lot of work done here. Also, I have a Wave account and some invites, so if you want to do some research and get a preview account, I can give some out. Cheers, -Andrei Thorp _______________________________________________ sup-talk mailing list sup-talk@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-talk ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-talk] sup-server 2009-12-08 22:12 ` Andrei Thorp @ 2009-12-09 2:33 ` Rich Lane 2009-12-09 13:45 ` Andrei Thorp 0 siblings, 1 reply; 5+ messages in thread From: Rich Lane @ 2009-12-09 2:33 UTC (permalink / raw) To: Andrei Thorp; +Cc: sup-talk Excerpts from Andrei Thorp's message of Tue Dec 08 17:12:05 -0500 2009: > This really seems like it's in direct competition with the Wave > system... but Wave has a lot more support. And yes, Wave doesn't have > e-mail federation at the moment, but I think they plan to do that in > the future. So yes, I like the idea of a Sup server, I like being able > to access my (sup) mail from multiple computers, I like the idea of > having multiple UIs to the Sup Service... but I think that in the long > run, I'd feel just as good about a couple good open source UIs to > Google's Wave instead. I actually didn't think about this as being competition to Wave. The current implementation is much more narrowly scoped, but the grand generic document store vision does bear a resemblance. I don't know how far down that road the project will go. I don't want to lose mail client functionality for the sake of being as generic as Wave (I haven't seen any information on the subject, but I assume the translation to/from email is lossy). _______________________________________________ sup-talk mailing list sup-talk@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-talk ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-talk] sup-server 2009-12-09 2:33 ` Rich Lane @ 2009-12-09 13:45 ` Andrei Thorp 0 siblings, 0 replies; 5+ messages in thread From: Andrei Thorp @ 2009-12-09 13:45 UTC (permalink / raw) To: Rich Lane, sup-talk On Tue, Dec 8, 2009 at 9:33 PM, Rich Lane <rlane@club.cc.cmu.edu> wrote: > Excerpts from Andrei Thorp's message of Tue Dec 08 17:12:05 -0500 2009: >> This really seems like it's in direct competition with the Wave >> system... but Wave has a lot more support. And yes, Wave doesn't have >> e-mail federation at the moment, but I think they plan to do that in >> the future. So yes, I like the idea of a Sup server, I like being able >> to access my (sup) mail from multiple computers, I like the idea of >> having multiple UIs to the Sup Service... but I think that in the long >> run, I'd feel just as good about a couple good open source UIs to >> Google's Wave instead. > > I actually didn't think about this as being competition to Wave. The > current implementation is much more narrowly scoped, Yeah, if you're sticking strictly to e-mail, this is a much better idea. > but the grand > generic document store vision does bear a resemblance. I don't know how > far down that road the project will go. The slight trouble that I foresee would be when Wave does do e-mail federation. I guess what I'm saying is that the future of this project would be reliant on whether or not Wave takes over, incorporates all of e-mail (as google seems to hope), and phases it out in the long run. I know, I know, there will be blood when this starts to happen (if it does). The reason that sup-server would have trouble if wave became popular would be simply that your e-mail would be fetched by wave servers and sup-server would not really have a place to be inserted, unless it actually became a mail server with some non-standard extensions (presumably for tagging, etc.). This might actually be an interesting direction to take regardless, actually. Upon thinking about how this might work, it seems a lot of data like annotations, labels, etc. could be stored in non-standard e-mail headers, and "sup clients" would just fetch the mail regularly and do stuff with the extra headers they have. The annotation system could, perhaps, list the line of the source + the annotation.The labels would be just a plain list. Then, you could even write back to the mail server your new headers with imap I guess? > I don't want to lose mail client > functionality for the sake of being as generic as Wave (I haven't seen > any information on the subject, but I assume the translation to/from > email is lossy). I guess we don't know whether or not it would be lossy yet. I suppose for the sake of wave internals (sending waves based on e-mails), there will have to be a lot of stuff cut, like most of the e-mail data fields (how would they know what the "to" person is when mapped to wave?). However, for the sake of sending back out to the e-mail system, they would probably store all of the relevant field data. Anyway, a lot of this is speculative, and the e-mail vs. wave battle of the future may not even happen because people are content enough with e-mail. Or it may be far off, or who knows! And I could sure use STS now to handle my mail in a sane way in the meanwhile. Foods for Thoughts, -Andrei Thorp P.S. I'll be excited for the future of sup regardless of what happens. I do honestly like this project a lot. _______________________________________________ sup-talk mailing list sup-talk@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-talk ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-talk] sup-server 2009-12-08 18:56 [sup-talk] sup-server Rich Lane 2009-12-08 22:12 ` Andrei Thorp @ 2009-12-19 19:16 ` William Morgan 1 sibling, 0 replies; 5+ messages in thread From: William Morgan @ 2009-12-19 19:16 UTC (permalink / raw) To: sup-talk Reformatted excerpts from Rich Lane's message of 2009-12-08: > I've pushed a branch sup-server to my github*. Very interesting! The approach is quite different from my (perennially unfinished and unpublished) sup-server code so I'm very curious to see where this goes. Nice work. -- William <wmorgan-sup@masanjin.net> _______________________________________________ sup-talk mailing list sup-talk@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-talk ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2009-12-19 19:27 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2009-12-08 18:56 [sup-talk] sup-server Rich Lane 2009-12-08 22:12 ` Andrei Thorp 2009-12-09 2:33 ` Rich Lane 2009-12-09 13:45 ` Andrei Thorp 2009-12-19 19:16 ` William Morgan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox