Subject: Re: [sup-talk] sup-server

From: Rich Lane <[email-masked]>
Date: Tue, 08 Dec 2009 21:33:29 -0500

Excerpts from Andrei Thorp's message of Tue Dec 08 17:12:05 -0500 2009:
> This really seems like it's in direct competition with the Wave
> system... but Wave has a lot more support. And yes, Wave doesn't have
> e-mail federation at the moment, but I think they plan to do that in
> the future. So yes, I like the idea of a Sup server, I like being able
> to access my (sup) mail from multiple computers, I like the idea of
> having multiple UIs to the Sup Service... but I think that in the long
> run, I'd feel just as good about a couple good open source UIs to
> Google's Wave instead.

I actually didn't think about this as being competition to Wave. The
current implementation is much more narrowly scoped, but the grand
generic document store vision does bear a resemblance. I don't know how
far down that road the project will go. I don't want to lose mail client
functionality for the sake of being as generic as Wave (I haven't seen
any information on the subject, but I assume the translation to/from
email is lossy).
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Andrei Thorp <[email-masked]>
Date: Wed, 9 Dec 2009 08:45:48 -0500

On Tue, Dec 8, 2009 at 9:33 PM, Rich Lane <[email-masked]> wrote:
> current implementation is much more narrowly scoped,

Yeah, if you're sticking strictly to e-mail, this is a much better idea.

> but the grand
> far down that road the project will go.

The slight trouble that I foresee would be when Wave does do e-mail
federation. I guess what I'm saying is that the future of this project
would be reliant on whether or not Wave takes over, incorporates all
of e-mail (as google seems to hope), and phases it out in the long
run. I know, I know, there will be blood when this starts to happen
(if it does). The reason that sup-server would have trouble if wave
became popular would be simply that your e-mail would be fetched by
wave servers and sup-server would not really have a place to be
inserted, unless it actually became a mail server with some
non-standard extensions (presumably for tagging, etc.). This might
actually be an interesting direction to take regardless, actually.
Upon thinking about how this might work, it seems a lot of data like
annotations, labels, etc. could be stored in non-standard e-mail
headers, and "sup clients" would just fetch the mail regularly and do
stuff with the extra headers they have. The annotation system could,
perhaps, list the line of the source + the annotation.The labels would
be just a plain list. Then, you could even write back to the mail
server your new headers with imap I guess?

> I don't want to lose mail client

I guess we don't know whether or not it would be lossy yet. I suppose
for the sake of wave internals (sending waves based on e-mails), there
will have to be a lot of stuff cut, like most of the e-mail data
fields (how would they know what the "to" person is when mapped to
wave?). However, for the sake of sending back out to the e-mail
system, they would probably store all of the relevant field data.

Anyway, a lot of this is speculative, and the e-mail vs. wave battle
of the future may not even happen because people are content enough
with e-mail. Or it may be far off, or who knows! And I could sure use
STS now to handle my mail in a sane way in the meanwhile.

Foods for Thoughts,

-Andrei Thorp

P.S. I'll be excited for the future of sup regardless of what happens.
I do honestly like this project a lot.
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Andrei Thorp <[email-masked]>
Date: Tue, 8 Dec 2009 17:12:05 -0500

On Tue, Dec 8, 2009 at 1:56 PM, Rich Lane <[email-masked]> wrote:
> I've pushed a branch sup-server to my github*. There's a lot to be done
> before this reaches feature parity with current Sup - for instance, the
> ncurses client doesn't work yet.
>
> *git://github.com/rlane/sup
>
> Current status:
>  - Server passes its (incomplete) test suite
>  - sup-cmd executable useful for simple interactions (each protocol
>   request exposed)
>  - ncurses client completely nonfunctional
>  - client/server/common source code split
>  - server implemented with Revactor - Ruby 1.9 only
>  - server stores messages (currently gzip'd mbox-ish)
>
> Protocol:
>  - Designed to avoid round-trip latencies
>  - BERT over tcp/unix
>  - thrift/etc could be supported in the future
>  - Requests (full docs in lib/sup/server/requests.rb):
>  - Query
>  - Count
>  - Label
>  - Add
>  - Stream
>  - Cancel
>  - Requests take query arguments instead of messages-ids (thanks to
>   Carl/notmuch for this idea)
>  - These requests should be sufficient to implement a working client. We
>   can add more in the future for optimizations (threading) or new
>   features (contacts).
>
> TODO:
>  - Expand test suite
>  - Modify sup-sync to send Add requests to the server
>  - Get the ncurses UI working
>  - Remove dead code
>  - Protocol versioning/negotiation/authentication/extensions
>  - Performance optimizations
>  - Add web/android/iphone/vim/irb/etc UIs
>  - Actorize the index/storage interfaces
>  - Shard index/storage
>  - Distribute indexing/parsing/compression/etc across worker processes
>  - Replication?
>
> The test suite is an important part of this effort. With the amount of
> code churn that's going on it just takes too much work to manually
> verify that every (affected) feature works before committing. If it's
> not covered by a test, I don't care if it's broken.
>
> For now, I'll plan on adding any new UIs to the main repo. When the
> protocol stabilizes we can think about splitting them out.
>
> I would be very interested if someone could spin up a web UI quickly.
> I'd like to start using this branch for my own mail and it will take a
> significant amount of time before I can beat the ncurses UI into shape.
>
> For some background on sup-server please read William's various blog
> posts on the subject. In its current form this project makes some
> compromises for the sake of efficiency and simplicity. I would be
> interested in making the protocol more generic while preserving those
> attributes, so if you have thoughts on this send them to the list.
>
> Some questions from the IRC channel:
>  - Do clients need threading logic?
>   Yes. There is no thread abstraction in the protocol, so clients will
>   need to understand email threading. A planned optimization is to
>   expose the index thread-id field to essentially collapse the thread
>   for client-threading purposes.
>
>  - Will a client connecting to sup-server feel snappier than sup over
>   ssh?
>   Yes (given a well-written client). The protocol is designed to
>   minimize the effects of network latency, but it will take a good
>   amount of work in the client to fully take advantage of this.
>
>  - Do clients need to know how to parse email?
>   Yes. Right now clients who want to display a message need to query for
>   the raw message text. If we can come up with a simple, comprehensive
>   model of an email message that clients would rather digest than the
>   raw text I'd be willing to add it as an optional extension to the
>   protocol.

While I'm as usual very impressed that Rich has done so much work, and
very happy with the ideas proposed in the STS posts, I can't help but
wonder,

"Why does this sound like implementing a Sup ui to Wave, but not using
Wave?" (Actually mentioned in Rethinking Sup pt. 2)

Honestly, I hope I'm wrong since there seems to be a lot of work done
here. Also, I have a Wave account and some invites, so if you want to
do some research and get a preview account, I can give some out.

Cheers,

-Andrei Thorp

---

From: Rich Lane <[email-masked]>
Date: Tue, 08 Dec 2009 13:56:10 -0500

Current status:

Protocol:

TODO:
 - Expand test suite

---

From: William Morgan <[email-masked]>
Date: Sat, 19 Dec 2009 11:16:40 -0800

Reformatted excerpts from Rich Lane's message of 2009-12-08:
> I've pushed a branch sup-server to my github*.

Very interesting!  The approach is quite different from my (perennially
unfinished and unpublished) sup-server code so I'm very curious to see
where this goes. Nice work.
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk