* [sup-talk] sup-server
@ 2009-12-08 18:56 Rich Lane
2009-12-08 22:12 ` Andrei Thorp
2009-12-19 19:16 ` William Morgan
0 siblings, 2 replies; 5+ messages in thread
From: Rich Lane @ 2009-12-08 18:56 UTC (permalink / raw)
To: sup-talk
I've pushed a branch sup-server to my github*. There's a lot to be done
before this reaches feature parity with current Sup - for instance, the
ncurses client doesn't work yet.
*git://github.com/rlane/sup
Current status:
- Server passes its (incomplete) test suite
- sup-cmd executable useful for simple interactions (each protocol
request exposed)
- ncurses client completely nonfunctional
- client/server/common source code split
- server implemented with Revactor - Ruby 1.9 only
- server stores messages (currently gzip'd mbox-ish)
Protocol:
- Designed to avoid round-trip latencies
- BERT over tcp/unix
- thrift/etc could be supported in the future
- Requests (full docs in lib/sup/server/requests.rb):
- Query
- Count
- Label
- Add
- Stream
- Cancel
- Requests take query arguments instead of messages-ids (thanks to
Carl/notmuch for this idea)
- These requests should be sufficient to implement a working client. We
can add more in the future for optimizations (threading) or new
features (contacts).
TODO:
- Expand test suite
- Modify sup-sync to send Add requests to the server
- Get the ncurses UI working
- Remove dead code
- Protocol versioning/negotiation/authentication/extensions
- Performance optimizations
- Add web/android/iphone/vim/irb/etc UIs
- Actorize the index/storage interfaces
- Shard index/storage
- Distribute indexing/parsing/compression/etc across worker processes
- Replication?
The test suite is an important part of this effort. With the amount of
code churn that's going on it just takes too much work to manually
verify that every (affected) feature works before committing. If it's
not covered by a test, I don't care if it's broken.
For now, I'll plan on adding any new UIs to the main repo. When the
protocol stabilizes we can think about splitting them out.
I would be very interested if someone could spin up a web UI quickly.
I'd like to start using this branch for my own mail and it will take a
significant amount of time before I can beat the ncurses UI into shape.
For some background on sup-server please read William's various blog
posts on the subject. In its current form this project makes some
compromises for the sake of efficiency and simplicity. I would be
interested in making the protocol more generic while preserving those
attributes, so if you have thoughts on this send them to the list.
Some questions from the IRC channel:
- Do clients need threading logic?
Yes. There is no thread abstraction in the protocol, so clients will
need to understand email threading. A planned optimization is to
expose the index thread-id field to essentially collapse the thread
for client-threading purposes.
- Will a client connecting to sup-server feel snappier than sup over
ssh?
Yes (given a well-written client). The protocol is designed to
minimize the effects of network latency, but it will take a good
amount of work in the client to fully take advantage of this.
- Do clients need to know how to parse email?
Yes. Right now clients who want to display a message need to query for
the raw message text. If we can come up with a simple, comprehensive
model of an email message that clients would rather digest than the
raw text I'd be willing to add it as an optional extension to the
protocol.
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-talk] sup-server
2009-12-08 18:56 [sup-talk] sup-server Rich Lane
@ 2009-12-08 22:12 ` Andrei Thorp
2009-12-09 2:33 ` Rich Lane
2009-12-19 19:16 ` William Morgan
1 sibling, 1 reply; 5+ messages in thread
From: Andrei Thorp @ 2009-12-08 22:12 UTC (permalink / raw)
To: sup-talk, rlane
On Tue, Dec 8, 2009 at 1:56 PM, Rich Lane <rlane@club.cc.cmu.edu> wrote:
> I've pushed a branch sup-server to my github*. There's a lot to be done
> before this reaches feature parity with current Sup - for instance, the
> ncurses client doesn't work yet.
>
> *git://github.com/rlane/sup
>
> Current status:
> - Server passes its (incomplete) test suite
> - sup-cmd executable useful for simple interactions (each protocol
> request exposed)
> - ncurses client completely nonfunctional
> - client/server/common source code split
> - server implemented with Revactor - Ruby 1.9 only
> - server stores messages (currently gzip'd mbox-ish)
>
> Protocol:
> - Designed to avoid round-trip latencies
> - BERT over tcp/unix
> - thrift/etc could be supported in the future
> - Requests (full docs in lib/sup/server/requests.rb):
> - Query
> - Count
> - Label
> - Add
> - Stream
> - Cancel
> - Requests take query arguments instead of messages-ids (thanks to
> Carl/notmuch for this idea)
> - These requests should be sufficient to implement a working client. We
> can add more in the future for optimizations (threading) or new
> features (contacts).
>
> TODO:
> - Expand test suite
> - Modify sup-sync to send Add requests to the server
> - Get the ncurses UI working
> - Remove dead code
> - Protocol versioning/negotiation/authentication/extensions
> - Performance optimizations
> - Add web/android/iphone/vim/irb/etc UIs
> - Actorize the index/storage interfaces
> - Shard index/storage
> - Distribute indexing/parsing/compression/etc across worker processes
> - Replication?
>
> The test suite is an important part of this effort. With the amount of
> code churn that's going on it just takes too much work to manually
> verify that every (affected) feature works before committing. If it's
> not covered by a test, I don't care if it's broken.
>
> For now, I'll plan on adding any new UIs to the main repo. When the
> protocol stabilizes we can think about splitting them out.
>
> I would be very interested if someone could spin up a web UI quickly.
> I'd like to start using this branch for my own mail and it will take a
> significant amount of time before I can beat the ncurses UI into shape.
>
> For some background on sup-server please read William's various blog
> posts on the subject. In its current form this project makes some
> compromises for the sake of efficiency and simplicity. I would be
> interested in making the protocol more generic while preserving those
> attributes, so if you have thoughts on this send them to the list.
>
> Some questions from the IRC channel:
> - Do clients need threading logic?
> Yes. There is no thread abstraction in the protocol, so clients will
> need to understand email threading. A planned optimization is to
> expose the index thread-id field to essentially collapse the thread
> for client-threading purposes.
>
> - Will a client connecting to sup-server feel snappier than sup over
> ssh?
> Yes (given a well-written client). The protocol is designed to
> minimize the effects of network latency, but it will take a good
> amount of work in the client to fully take advantage of this.
>
> - Do clients need to know how to parse email?
> Yes. Right now clients who want to display a message need to query for
> the raw message text. If we can come up with a simple, comprehensive
> model of an email message that clients would rather digest than the
> raw text I'd be willing to add it as an optional extension to the
> protocol.
> _______________________________________________
> sup-talk mailing list
> sup-talk@rubyforge.org
> http://rubyforge.org/mailman/listinfo/sup-talk
>
While I'm as usual very impressed that Rich has done so much work, and
very happy with the ideas proposed in the STS posts, I can't help but
wonder,
"Why does this sound like implementing a Sup ui to Wave, but not using
Wave?" (Actually mentioned in Rethinking Sup pt. 2)
This really seems like it's in direct competition with the Wave
system... but Wave has a lot more support. And yes, Wave doesn't have
e-mail federation at the moment, but I think they plan to do that in
the future. So yes, I like the idea of a Sup server, I like being able
to access my (sup) mail from multiple computers, I like the idea of
having multiple UIs to the Sup Service... but I think that in the long
run, I'd feel just as good about a couple good open source UIs to
Google's Wave instead.
Honestly, I hope I'm wrong since there seems to be a lot of work done
here. Also, I have a Wave account and some invites, so if you want to
do some research and get a preview account, I can give some out.
Cheers,
-Andrei Thorp
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-talk] sup-server
2009-12-08 22:12 ` Andrei Thorp
@ 2009-12-09 2:33 ` Rich Lane
2009-12-09 13:45 ` Andrei Thorp
0 siblings, 1 reply; 5+ messages in thread
From: Rich Lane @ 2009-12-09 2:33 UTC (permalink / raw)
To: Andrei Thorp; +Cc: sup-talk
Excerpts from Andrei Thorp's message of Tue Dec 08 17:12:05 -0500 2009:
> This really seems like it's in direct competition with the Wave
> system... but Wave has a lot more support. And yes, Wave doesn't have
> e-mail federation at the moment, but I think they plan to do that in
> the future. So yes, I like the idea of a Sup server, I like being able
> to access my (sup) mail from multiple computers, I like the idea of
> having multiple UIs to the Sup Service... but I think that in the long
> run, I'd feel just as good about a couple good open source UIs to
> Google's Wave instead.
I actually didn't think about this as being competition to Wave. The
current implementation is much more narrowly scoped, but the grand
generic document store vision does bear a resemblance. I don't know how
far down that road the project will go. I don't want to lose mail client
functionality for the sake of being as generic as Wave (I haven't seen
any information on the subject, but I assume the translation to/from
email is lossy).
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-talk] sup-server
2009-12-09 2:33 ` Rich Lane
@ 2009-12-09 13:45 ` Andrei Thorp
0 siblings, 0 replies; 5+ messages in thread
From: Andrei Thorp @ 2009-12-09 13:45 UTC (permalink / raw)
To: Rich Lane, sup-talk
On Tue, Dec 8, 2009 at 9:33 PM, Rich Lane <rlane@club.cc.cmu.edu> wrote:
> Excerpts from Andrei Thorp's message of Tue Dec 08 17:12:05 -0500 2009:
>> This really seems like it's in direct competition with the Wave
>> system... but Wave has a lot more support. And yes, Wave doesn't have
>> e-mail federation at the moment, but I think they plan to do that in
>> the future. So yes, I like the idea of a Sup server, I like being able
>> to access my (sup) mail from multiple computers, I like the idea of
>> having multiple UIs to the Sup Service... but I think that in the long
>> run, I'd feel just as good about a couple good open source UIs to
>> Google's Wave instead.
>
> I actually didn't think about this as being competition to Wave. The
> current implementation is much more narrowly scoped,
Yeah, if you're sticking strictly to e-mail, this is a much better idea.
> but the grand
> generic document store vision does bear a resemblance. I don't know how
> far down that road the project will go.
The slight trouble that I foresee would be when Wave does do e-mail
federation. I guess what I'm saying is that the future of this project
would be reliant on whether or not Wave takes over, incorporates all
of e-mail (as google seems to hope), and phases it out in the long
run. I know, I know, there will be blood when this starts to happen
(if it does). The reason that sup-server would have trouble if wave
became popular would be simply that your e-mail would be fetched by
wave servers and sup-server would not really have a place to be
inserted, unless it actually became a mail server with some
non-standard extensions (presumably for tagging, etc.). This might
actually be an interesting direction to take regardless, actually.
Upon thinking about how this might work, it seems a lot of data like
annotations, labels, etc. could be stored in non-standard e-mail
headers, and "sup clients" would just fetch the mail regularly and do
stuff with the extra headers they have. The annotation system could,
perhaps, list the line of the source + the annotation.The labels would
be just a plain list. Then, you could even write back to the mail
server your new headers with imap I guess?
> I don't want to lose mail client
> functionality for the sake of being as generic as Wave (I haven't seen
> any information on the subject, but I assume the translation to/from
> email is lossy).
I guess we don't know whether or not it would be lossy yet. I suppose
for the sake of wave internals (sending waves based on e-mails), there
will have to be a lot of stuff cut, like most of the e-mail data
fields (how would they know what the "to" person is when mapped to
wave?). However, for the sake of sending back out to the e-mail
system, they would probably store all of the relevant field data.
Anyway, a lot of this is speculative, and the e-mail vs. wave battle
of the future may not even happen because people are content enough
with e-mail. Or it may be far off, or who knows! And I could sure use
STS now to handle my mail in a sane way in the meanwhile.
Foods for Thoughts,
-Andrei Thorp
P.S. I'll be excited for the future of sup regardless of what happens.
I do honestly like this project a lot.
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-talk] sup-server
2009-12-08 18:56 [sup-talk] sup-server Rich Lane
2009-12-08 22:12 ` Andrei Thorp
@ 2009-12-19 19:16 ` William Morgan
1 sibling, 0 replies; 5+ messages in thread
From: William Morgan @ 2009-12-19 19:16 UTC (permalink / raw)
To: sup-talk
Reformatted excerpts from Rich Lane's message of 2009-12-08:
> I've pushed a branch sup-server to my github*.
Very interesting! The approach is quite different from my (perennially
unfinished and unpublished) sup-server code so I'm very curious to see
where this goes. Nice work.
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2009-12-19 19:27 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-12-08 18:56 [sup-talk] sup-server Rich Lane
2009-12-08 22:12 ` Andrei Thorp
2009-12-09 2:33 ` Rich Lane
2009-12-09 13:45 ` Andrei Thorp
2009-12-19 19:16 ` William Morgan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox