* [sup-devel] sup-server revisited
@ 2011-02-21 21:02 William Morgan
2011-02-22 14:09 ` Nicolas Pouillard
0 siblings, 1 reply; 5+ messages in thread
From: William Morgan @ 2011-02-21 21:02 UTC (permalink / raw)
To: sup-talk, sup-devel
Hello Sup fans,
As I alluded to in an earlier email to sup-devel, I have been working recently
on the ancient goal of splitting Sup functionality out into separate client and
server programs. This will have many advantages; in particular it will remove
email lock-in by making it possible to have non-Sup clients interact with Sup.
I've tried various approaches to this goal at various times, without much
success. But with the release of Whistlepig, I have been making a lot of
progress in a short amount of time.
I wanted to lay out my general plan in this email, so that everyone can follow
along and try things out if they feel inclined.
I'm focusing on the server component first. My specific goals are:
- Encapsulate all storage and indexing logic.
- Formalize the notion of labels and state, and the relationship between them,
messages, and threads. Currently these relationships are fuzzy in Sup, and
things suffer for it.
- Precompute threads, so that search requires only moderate effort, instead of
the large effort it does now. This will make search much, much faster, at the
expense of a little more effort at index time.
- Allow concurrent access from multiple clients.
- Provide something that is transparent and easy to modify without having to go
through an ncurses client.
You can find the work in progress here: https://github.com/wmorgan/heliotrope
Currently it is not actually a server, but accomplishes many of the goals
above. Run bin/email-indexer and then bin/email-searcher to play around with
a simple preview of things to come. Stay tuned for more on this.
Once the server is in a reasonable state, my goals for the curses client reboot
are:
- Move to an event-based model rather than the polling + many threads +
observable model we have now. Dump all events into one big queue,
and block on it.
- Have one, and only one, non-main thread, which it to communicate with the
server. No more mutex insanity.
- Work from the ground up with Ruby 1.9, which means finding the many sources
of encoding errors and punishing them.
- Require the console and ncurses gems so that i18n works out of the box.
- Borrow as much code as possible from the current Sup, because I sure as shit
don't want to have to rewrite it all.
I haven't started on this at all.
So stay tuned, and check out heliotrope if you are so inclined.
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-devel] sup-server revisited
2011-02-21 21:02 [sup-devel] sup-server revisited William Morgan
@ 2011-02-22 14:09 ` Nicolas Pouillard
2011-02-22 18:00 ` [sup-talk] " William Morgan
0 siblings, 1 reply; 5+ messages in thread
From: Nicolas Pouillard @ 2011-02-22 14:09 UTC (permalink / raw)
To: William Morgan, sup-talk, sup-devel
On Mon, 21 Feb 2011 21:02:28 +0000, William Morgan <wmorgan-sup@masanjin.net> wrote:
> Hello Sup fans,
[...]
> I'm focusing on the server component first. My specific goals are:
>
> - Encapsulate all storage and indexing logic.
> - Formalize the notion of labels and state, and the relationship between them,
> messages, and threads. Currently these relationships are fuzzy in Sup, and
> things suffer for it.
> - Precompute threads, so that search requires only moderate effort, instead of
> the large effort it does now. This will make search much, much faster, at the
> expense of a little more effort at index time.
> - Allow concurrent access from multiple clients.
> - Provide something that is transparent and easy to modify without having to go
> through an ncurses client.
On the server side, and more precisely on the interaction between the two,
I would love to see a simple Unix/command/CLI defined for the server. To make
myself clear, a bit like the notmuch CLI. This would allow for a greater
modularity and reusablity between components. For instance I would like to
index my mails with differents backends (disk is cheap), to get a way to
compare different tools on real data, when searching emails, and thus allow
to debug the different tools.
--
Nicolas Pouillard
http://nicolaspouillard.fr
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-talk] [sup-devel] sup-server revisited
2011-02-22 14:09 ` Nicolas Pouillard
@ 2011-02-22 18:00 ` William Morgan
2011-02-23 9:59 ` Nicolas Pouillard
0 siblings, 1 reply; 5+ messages in thread
From: William Morgan @ 2011-02-22 18:00 UTC (permalink / raw)
To: sup-talk
[resend to list]
Reformatted excerpts from Nicolas Pouillard's message of 2011-02-22:
> I would love to see a simple Unix/command/CLI defined for the server. To
> make myself clear, a bit like the notmuch CLI. This would allow for a
> greater modularity and reusablity between components.
I am definitely interested in a set of composable CLI tools that talk to the
server, for basic operations like search, show, add/remove labels, etc.
> For instance I would like to index my mails with differents backends (disk
> is cheap), to get a way to compare different tools on real data, when
> searching emails, and thus allow to debug the different tools.
The server implementation is pretty heavily tied to a particular index and
store mechanism. It's not clear to me how (or whether) the CLI tools will
allow you to swap out different implementations.
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-talk] [sup-devel] sup-server revisited
2011-02-22 18:00 ` [sup-talk] " William Morgan
@ 2011-02-23 9:59 ` Nicolas Pouillard
2011-02-23 18:23 ` William Morgan
0 siblings, 1 reply; 5+ messages in thread
From: Nicolas Pouillard @ 2011-02-23 9:59 UTC (permalink / raw)
To: William Morgan, sup-talk
On Tue, 22 Feb 2011 18:00:06 +0000, William Morgan <wmorgan-sup@masanjin.net> wrote:
> [resend to list]
>
> Reformatted excerpts from Nicolas Pouillard's message of 2011-02-22:
> > I would love to see a simple Unix/command/CLI defined for the server. To
> > make myself clear, a bit like the notmuch CLI. This would allow for a
> > greater modularity and reusablity between components.
>
> I am definitely interested in a set of composable CLI tools that talk to the
> server, for basic operations like search, show, add/remove labels, etc.
Ok, then I think it would make not much sense to make it different than
notmuch. Of course with good arguments changes can go the other direction as
well.
> > For instance I would like to index my mails with differents backends (disk
> > is cheap), to get a way to compare different tools on real data, when
> > searching emails, and thus allow to debug the different tools.
>
> The server implementation is pretty heavily tied to a particular index and
> store mechanism. It's not clear to me how (or whether) the CLI tools will
> allow you to swap out different implementations.
I agree with you and don't to expose such details. What I want is a way to
separate the frontend from the backend completely. This would allow me to
use the frontend with any of:
- heliotrope
- sup (maybe)
- notmuch (not written in ruby)
- my own toy project (not in ruby either)
- any combination of both, where "writes" are dispatched to all of them,
and "reads" are dispatched to one of them (that I can change easily).
I also want to be able to script batch operations on the mail store easily.
Best regards,
--
Nicolas Pouillard
http://nicolaspouillard.fr
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [sup-talk] [sup-devel] sup-server revisited
2011-02-23 9:59 ` Nicolas Pouillard
@ 2011-02-23 18:23 ` William Morgan
0 siblings, 0 replies; 5+ messages in thread
From: William Morgan @ 2011-02-23 18:23 UTC (permalink / raw)
To: sup-talk
Reformatted excerpts from Nicolas Pouillard's message of 2011-02-23:
> I agree with you and don't to expose such details. What I want is a
> way to separate the frontend from the backend completely.
Yes, absolutely.
> I also want to be able to script batch operations on the mail store
> easily.
Agreed. A definite goal. The prevalence of [bug reports about]
sup-tweak-labels usage is a sign that other people desire this too.
--
William <wmorgan-sup@masanjin.net>
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2011-02-23 18:29 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-02-21 21:02 [sup-devel] sup-server revisited William Morgan
2011-02-22 14:09 ` Nicolas Pouillard
2011-02-22 18:00 ` [sup-talk] " William Morgan
2011-02-23 9:59 ` Nicolas Pouillard
2011-02-23 18:23 ` William Morgan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox