Archive of RubyForge sup-talk mailing list
 help / color / mirror / Atom feed
* [sup-talk] rethinking sup part ii
@ 2008-07-20 21:33 William Morgan
  2008-07-20 23:28 ` Guillaume Quintard
                   ` (4 more replies)
  0 siblings, 5 replies; 24+ messages in thread
From: William Morgan @ 2008-07-20 21:33 UTC (permalink / raw)


In which the new version of Sup is described!

http://all-thing.net/2008/07/rethinking-sup-part-ii.html
-- 
William <wmorgan-sup at masanjin.net>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan
@ 2008-07-20 23:28 ` Guillaume Quintard
  2008-07-21  0:04   ` William Morgan
  2008-07-21 12:14 ` Peter Krenn
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 24+ messages in thread
From: Guillaume Quintard @ 2008-07-20 23:28 UTC (permalink / raw)


Excerpts from William Morgan's message of Sun Jul 20 14:33:24 -0700 2008:
> In which the new version of Sup is described!
> 
> http://all-thing.net/2008/07/rethinking-sup-part-ii.html

looks cool to me (mostly the server/client idea). I just have one
question: I am currently using gmail+mpop, and giving the mbox file to
sup. The new sup won't change a lot for me, will it? 
(maybe a stupid question, but I'd like to be sure I understood the
implications).

-- 
Guillaume


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-20 23:28 ` Guillaume Quintard
@ 2008-07-21  0:04   ` William Morgan
  2008-07-21  5:43     ` Guillaume Quintard
  0 siblings, 1 reply; 24+ messages in thread
From: William Morgan @ 2008-07-21  0:04 UTC (permalink / raw)


Reformatted excerpts from guillaume.quintard's message of 2008-07-20:
> I just have one question: I am currently using gmail+mpop, and giving
> the mbox file to sup. The new sup won't change a lot for me, will it?
> (maybe a stupid question, but I'd like to be sure I understood the
> implications).

It should be basically the same, except that you can throw away the mbox
file when you're done (because STS will handle the storage for you).
-- 
William <wmorgan-sup at masanjin.net>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-21  0:04   ` William Morgan
@ 2008-07-21  5:43     ` Guillaume Quintard
  0 siblings, 0 replies; 24+ messages in thread
From: Guillaume Quintard @ 2008-07-21  5:43 UTC (permalink / raw)


Excerpts from William Morgan's message of Sun Jul 20 17:04:35 -0700 2008:
> It should be basically the same, except that you can throw away the mbox
> file when you're done (because STS will handle the storage for you).

Ok, that was what I understood. I'd keep the mbox anyway, having a
standard backup format is always nice.

-- 
Guillaume


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan
  2008-07-20 23:28 ` Guillaume Quintard
@ 2008-07-21 12:14 ` Peter Krenn
  2008-07-21 15:22   ` William Morgan
  2008-07-21 18:43 ` Lionel Ott
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 24+ messages in thread
From: Peter Krenn @ 2008-07-21 12:14 UTC (permalink / raw)


Sounds great.
What kind of interface will this service provide? Or will it be more
like a library which one could use to access the sup storage from any
application? Something like a specialized database?


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-21 12:14 ` Peter Krenn
@ 2008-07-21 15:22   ` William Morgan
  2008-07-21 15:26     ` Stephen Patterson
  0 siblings, 1 reply; 24+ messages in thread
From: William Morgan @ 2008-07-21 15:22 UTC (permalink / raw)


Reformatted excerpts from Peter Krenn's message of 2008-07-21:
> What kind of interface will this service provide? Or will it be more
> like a library which one could use to access the sup storage from any
> application? Something like a specialized database?

It will be a network service with Thrift defining the interface.
-- 
William <wmorgan-sup at masanjin.net>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-21 15:22   ` William Morgan
@ 2008-07-21 15:26     ` Stephen Patterson
  2008-07-21 16:04       ` John Bent
  2008-07-22  2:12       ` William Morgan
  0 siblings, 2 replies; 24+ messages in thread
From: Stephen Patterson @ 2008-07-21 15:26 UTC (permalink / raw)


On 21 Jul 08, William Morgan (wmorgan-sup at masanjin.net) wrote:
> Reformatted excerpts from Peter Krenn's message of 2008-07-21:
> > What kind of interface will this service provide? Or will it be more
> > like a library which one could use to access the sup storage from any
> > application? Something like a specialized database?
> 
> It will be a network service with Thrift defining the interface.

So we'd then need a collection of custom client applications to search
and/or browse the index? 

This sounds quite a bit like the beagle project -
http://beagle-project.org/Main_Page

-- 
Stephen Patterson :: steve at patter.mine.nu :: http://patter.mine.nu/
GPG: B416F0DE :: Jabber: patter at jabber.earth.li 
"Don't be silly, Minnie. Who'd be walking round these cliffs with a gas oven?"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080721/aade96ff/attachment.bin>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-21 15:26     ` Stephen Patterson
@ 2008-07-21 16:04       ` John Bent
  2008-07-22  2:12       ` William Morgan
  1 sibling, 0 replies; 24+ messages in thread
From: John Bent @ 2008-07-21 16:04 UTC (permalink / raw)


Excerpts from Stephen Patterson's message of Mon Jul 21 09:26:59 -0600 2008:
> On 21 Jul 08, William Morgan (wmorgan-sup at masanjin.net) wrote:
> > Reformatted excerpts from Peter Krenn's message of 2008-07-21:
> > > What kind of interface will this service provide? Or will it be more
> > > like a library which one could use to access the sup storage from any
> > > application? Something like a specialized database?
> > 
> > It will be a network service with Thrift defining the interface.
> 
> So we'd then need a collection of custom client applications to search
> and/or browse the index? 
> 
> This sounds quite a bit like the beagle project -
> http://beagle-project.org/Main_Page
> 
And google desktop:
http://desktop.google.com
And spotlight:
http://en.wikipedia.org/wiki/Spotlight_(software)
And MS Fast Find:
http://support.microsoft.com/support/kb/articles/Q166/3/02.ASP

There is room for improvement in all of these tools but the margin seems
considerably smaller than it did for the sup email client which is vastly
superior to anything else.

John


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan
  2008-07-20 23:28 ` Guillaume Quintard
  2008-07-21 12:14 ` Peter Krenn
@ 2008-07-21 18:43 ` Lionel Ott
  2008-07-23  8:45 ` Richard Heycock
  2008-07-24  4:55 ` Guarded Identity
  4 siblings, 0 replies; 24+ messages in thread
From: Lionel Ott @ 2008-07-21 18:43 UTC (permalink / raw)


Excerpts from William Morgan's message of Sun Jul 20 23:33:24 +0200 2008:
> In which the new version of Sup is described!
> 
> http://all-thing.net/2008/07/rethinking-sup-part-ii.html

Sounds nice overall but something I couldn't infer from the post was if
you're thinking about doing just the reading and searching part of sup or
if some of the mail functionality is going to be kept.
Basically will sts be able to send mail or would one have to send it
using some other client and sts then would present all that nicely to
one, given you feed it with your mails.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-21 15:26     ` Stephen Patterson
  2008-07-21 16:04       ` John Bent
@ 2008-07-22  2:12       ` William Morgan
  2008-07-23 17:40         ` Nicolas Pouillard
  1 sibling, 1 reply; 24+ messages in thread
From: William Morgan @ 2008-07-22  2:12 UTC (permalink / raw)


Reformatted excerpts from Stephen Patterson's message of 2008-07-21:
> So we'd then need a collection of custom client applications to search
> and/or browse the index? 
> 
> This sounds quite a bit like the beagle project -
> http://beagle-project.org/Main_Page

Thanks for the pointer. It does sound quite similar. Having played with
Beagle a bit, I think they key difference is that I'm envisioning STS as
being the primary interface to your precious infos, whereas Beagle seems
more of a supplemental index of data that's primarily handled by other
apps.

This has a couple implications, including who's responsible for storing
the files (STS: STS; Beagle: you and your apps), and what the user
experience looks like (Beagle: use Thunderbird, then call up Beagle for
help; STS: browse everything through your client, starting with a
search for label "inbox", and call up other applications for handling
foreign mime types).

If anything, Beagle is closer to Sup classic, and STS brings us closer
to Gmail.
-- 
William <wmorgan-sup at masanjin.net>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan
                   ` (2 preceding siblings ...)
  2008-07-21 18:43 ` Lionel Ott
@ 2008-07-23  8:45 ` Richard Heycock
  2008-07-23  8:58   ` Nicolas Pouillard
  2008-07-23 17:09   ` William Morgan
  2008-07-24  4:55 ` Guarded Identity
  4 siblings, 2 replies; 24+ messages in thread
From: Richard Heycock @ 2008-07-23  8:45 UTC (permalink / raw)


Excerpts from William Morgan's message of Mon Jul 21 07:33:24 +1000 2008:
> In which the new version of Sup is described!
> 
> http://all-thing.net/2008/07/rethinking-sup-part-ii.html

I really like the idea and I have to say that I'm *very* glad that you
are keeping the ncurses interface.

My one concern is that you are thinking of using Sphinx and as a
consequence you are relient on a relational database. Is that really
what you want for a mail client? I find this disturbing on two levels:
having a full blown database system for email seems like overkill and
the idea of an index bring based on a relational model quite appaling.

There are a number of other indexs such as Xapian or Hyper Estraier
both have ruby bindings and both are quite mature and widely used in
other projects.

Just my two cents.

rgh

-- 
+61 (0) 410 646 369
[e]:  rgh at neoss.com.au
[im]: rgh at jabber.org

You're worried criminals will continue to penetrate into cyberspace, and
I'm worried complexity, poor design and mismanagement will be there to meet
them - Marcus Ranum


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-23  8:45 ` Richard Heycock
@ 2008-07-23  8:58   ` Nicolas Pouillard
  2008-07-23  9:27     ` teroz
  2008-07-23 17:09   ` William Morgan
  1 sibling, 1 reply; 24+ messages in thread
From: Nicolas Pouillard @ 2008-07-23  8:58 UTC (permalink / raw)


Excerpts from Richard Heycock's message of Wed Jul 23 10:45:06 +0200 2008:
> Excerpts from William Morgan's message of Mon Jul 21 07:33:24 +1000 2008:
> > In which the new version of Sup is described!
> > 
> > http://all-thing.net/2008/07/rethinking-sup-part-ii.html
> 
> I really like the idea and I have to say that I'm *very* glad that you
> are keeping the ncurses interface.
> 
> My one concern is that you are thinking of using Sphinx and as a
> consequence you are relient on a relational database. Is that really
> what you want for a mail client? I find this disturbing on two levels:
> having a full blown database system for email seems like overkill and
> the idea of an index bring based on a relational model quite appaling.
> 
> There are a number of other indexs such as Xapian or Hyper Estraier
> both have ruby bindings and both are quite mature and widely used in
> other projects.

I'm also concerned by using relying on relational databases for a mail
service.

-- 
Nicolas Pouillard aka Ertai
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080723/0a4a45de/attachment.bin>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-23  8:58   ` Nicolas Pouillard
@ 2008-07-23  9:27     ` teroz
  2008-07-23 21:41       ` Richard Heycock
  0 siblings, 1 reply; 24+ messages in thread
From: teroz @ 2008-07-23  9:27 UTC (permalink / raw)


Well surely if we r eschewing typical mail storage formats like mbox and
IMAP then a database is the way to go
unless your suggesting we write some form of storage library as well.

On Wed, Jul 23, 2008 at 10:58 AM, Nicolas Pouillard <
nicolas.pouillard at gmail.com> wrote:

> Excerpts from Richard Heycock's message of Wed Jul 23 10:45:06 +0200 2008:
> > Excerpts from William Morgan's message of Mon Jul 21 07:33:24 +1000 2008:
> > > In which the new version of Sup is described!
> > >
> > > http://all-thing.net/2008/07/rethinking-sup-part-ii.html
> >
> > I really like the idea and I have to say that I'm *very* glad that you
> > are keeping the ncurses interface.
> >
> > My one concern is that you are thinking of using Sphinx and as a
> > consequence you are relient on a relational database. Is that really
> > what you want for a mail client? I find this disturbing on two levels:
> > having a full blown database system for email seems like overkill and
> > the idea of an index bring based on a relational model quite appaling.
> >
> > There are a number of other indexs such as Xapian or Hyper Estraier
> > both have ruby bindings and both are quite mature and widely used in
> > other projects.
>
> I'm also concerned by using relying on relational databases for a mail
> service.
>
> --
> Nicolas Pouillard aka Ertai
>
> _______________________________________________
> sup-talk mailing list
> sup-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/sup-talk
>
>


-- 
--Teroz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080723/0dfa1626/attachment.html>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-23  8:45 ` Richard Heycock
  2008-07-23  8:58   ` Nicolas Pouillard
@ 2008-07-23 17:09   ` William Morgan
  2008-07-25 12:11     ` Marcus Williams
  1 sibling, 1 reply; 24+ messages in thread
From: William Morgan @ 2008-07-23 17:09 UTC (permalink / raw)


Reformatted excerpts from Richard Heycock's message of 2008-07-23:
> My one concern is that you are thinking of using Sphinx and as a
> consequence you are relient on a relational database. Is that really
> what you want for a mail client? I find this disturbing on two levels:
> having a full blown database system for email seems like overkill and
> the idea of an index bring based on a relational model quite appaling.

I feel exactly the same way. I'm using Sphinx's XML interface to add
things to the index, so there's no RDBMS involved. I hate XML but not as
much as I hate RDBMS.

The current plan is to actually store email/documents in some kind of
persistent key-value store. I have a preliminary very simple
implementation that just uses files on disk, but it will be possible to
swap that out to something like TokyoCabinet or AWS.

> There are a number of other indexs such as Xapian or Hyper Estraier
> both have ruby bindings and both are quite mature and widely used in
> other projects.

Interesting. I will check those out. I only picked Sphinx because it
seemed popular and robust, but it is currently making my life more difficult
than it really needs to be.
-- 
William <wmorgan-sup at masanjin.net>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-22  2:12       ` William Morgan
@ 2008-07-23 17:40         ` Nicolas Pouillard
  2008-07-23 22:26           ` William Morgan
  0 siblings, 1 reply; 24+ messages in thread
From: Nicolas Pouillard @ 2008-07-23 17:40 UTC (permalink / raw)


Excerpts from William Morgan's message of Tue Jul 22 04:12:09 +0200 2008:
> Reformatted excerpts from Stephen Patterson's message of 2008-07-21:
> > So we'd then need a collection of custom client applications to search
> > and/or browse the index? 
> > 
> > This sounds quite a bit like the beagle project -
> > http://beagle-project.org/Main_Page
> 
> Thanks for the pointer. It does sound quite similar. Having played with
> Beagle a bit, I think they key difference is that I'm envisioning STS as
> being the primary interface to your precious infos, whereas Beagle seems
> more of a supplemental index of data that's primarily handled by other
> apps.
> 
> This has a couple implications, including who's responsible for storing
> the files (STS: STS; Beagle: you and your apps), and what the user
> experience looks like (Beagle: use Thunderbird, then call up Beagle for
> help; STS: browse everything through your client, starting with a
> search for label "inbox", and call up other applications for handling
> foreign mime types).
> 
> If anything, Beagle is closer to Sup classic, and STS brings us closer
> to Gmail.

Another design choice, that could (and perhaps *should*) be made; is that
stored data don't change in the case of mails, IM..., only their labels and
children (threading) do.

Then, this makes even more differences, general file system indexers have an
harder task since, files change all the time this imply more complex data
structures that often provides worse complexity than one could wish.

Cheers,

-- 
Nicolas Pouillard aka Ertai
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080723/4e09fbde/attachment.bin>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-23  9:27     ` teroz
@ 2008-07-23 21:41       ` Richard Heycock
  0 siblings, 0 replies; 24+ messages in thread
From: Richard Heycock @ 2008-07-23 21:41 UTC (permalink / raw)


Excerpts from terence.namusonge's message of Wed Jul 23 19:27:46 +1000 2008:
> Well surely if we r eschewing typical mail storage formats like mbox and
> IMAP then a database is the way to go
> unless your suggesting we write some form of storage library as well.

Two things: I do not have a problem with a database per se, I do however
have a problem with a standalone relational database. There are many
embedded databases that work a treat and the user never need know about
them. That's pretty much what mbox, Maildir, etc are.

The other point is that in using Sphinx the STS would not be using
the database directly but would require it simply because the index
required it.

As I mentioned in my previous email there are a number of other indexes
which will serve the purpose without the unnecessary complexity of a
standalone database system.

rgh


> On Wed, Jul 23, 2008 at 10:58 AM, Nicolas Pouillard <
> nicolas.pouillard at gmail.com> wrote:
> 
> > Excerpts from Richard Heycock's message of Wed Jul 23 10:45:06 +0200 2008:
> > > Excerpts from William Morgan's message of Mon Jul 21 07:33:24 +1000 2008:
> > > > In which the new version of Sup is described!
> > > >
> > > > http://all-thing.net/2008/07/rethinking-sup-part-ii.html
> > >
> > > I really like the idea and I have to say that I'm *very* glad that you
> > > are keeping the ncurses interface.
> > >
> > > My one concern is that you are thinking of using Sphinx and as a
> > > consequence you are relient on a relational database. Is that really
> > > what you want for a mail client? I find this disturbing on two levels:
> > > having a full blown database system for email seems like overkill and
> > > the idea of an index bring based on a relational model quite appaling.
> > >
> > > There are a number of other indexs such as Xapian or Hyper Estraier
> > > both have ruby bindings and both are quite mature and widely used in
> > > other projects.
> >
> > I'm also concerned by using relying on relational databases for a mail
> > service.
> >
> > --
> > Nicolas Pouillard aka Ertai
> >
> > _______________________________________________
> > sup-talk mailing list
> > sup-talk at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/sup-talk
> >
> >
> 

-- 
+61 (0) 410 646 369
[e]:  rgh at neoss.com.au
[im]: rgh at jabber.org

You're worried criminals will continue to penetrate into cyberspace, and
I'm worried complexity, poor design and mismanagement will be there to meet
them - Marcus Ranum


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-23 17:40         ` Nicolas Pouillard
@ 2008-07-23 22:26           ` William Morgan
  2008-07-24  9:16             ` Nicolas Pouillard
  0 siblings, 1 reply; 24+ messages in thread
From: William Morgan @ 2008-07-23 22:26 UTC (permalink / raw)


Reformatted excerpts from nicolas.pouillard's message of 2008-07-23:
> Another design choice, that could (and perhaps *should*) be made; is
> that stored data don't change in the case of mails, IM..., only their
> labels and children (threading) do.

I think that's a true fact. Draft messages are sort of an exception, but
editing can always be simulated as deleting the old one and adding the
new one.
-- 
William <wmorgan-sup at masanjin.net>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan
                   ` (3 preceding siblings ...)
  2008-07-23  8:45 ` Richard Heycock
@ 2008-07-24  4:55 ` Guarded Identity
  2008-07-24  9:21   ` Nicolas Pouillard
  2008-07-25  7:18   ` William Morgan
  4 siblings, 2 replies; 24+ messages in thread
From: Guarded Identity @ 2008-07-24  4:55 UTC (permalink / raw)


Excerpts from wmorgan-sup-at-masanjin.net wmorgan-sup-at-masanjin.net |Sup_Talk|'s message of Sun Jul 20 16:33:24 -0500 2008:
> In which the new version of Sup is described!
> 
> http://all-thing.net/2008/07/rethinking-sup-part-ii.html

Your first blog was intriguing, but this second one is putting Sup in a
direction I'm really happy with.  I'm happy about:

    Central Server
    --------------
    I like having a central server, so I can run clients from multiple
    locations.

    Modifiable Messages
    -------------------
    There was a post I read in this thread indicating that people didn't want
    to modify documents.   But if we handle this as a "delete/re-add"
    operation, I'm hoping it will be fine.

    In all honesty, the modifications I'd like to do are

        - remove attachments.  Sometimes at work, people send out some
          absolutely useless, gigantic attachments.

        - delete/expire documents.

    Documents Beyond E-mail
    -----------------------
    I'm /really/ interested in storing different types of "documents" like RSS.
    . . maybe newsgroups too?  Then I could stop using snownews and slrn.  One
    client. . . actually, for this reason, I'd tried out Emacs Gnus, but I
    wasn't really happy with the experience/complexity.

However, I was hoping to see if I could get in a request while you're in the
beginning stages of STS development.

    True Thread-level Labels
    ------------------------
    I was using rss2email, so the number of messages I had was enormous
    (although I disabled that due to performance problems, Ferret corruption,
    and the problem I'm discussion here).  I hate wasting disk space
    needlessly, or cluttering the index for that matter.  There's some mailing
    lists that are mostly fluff, but I wanted to save the occasional thread and
    expire the rest.  But because messages attach to messages really, and not
    to the thread, negation in search logic breaks.  If I search for threads
    that have "!label:save" I can get threads that have "save" messages if they
    also have a message not labeled "save".

    Attaching labels to message always bothered me.  I'm hoping that with STS,
    you can consider designing an architecture that lets you attach labels to a
    thread grouping.  I'm hoping it can be done in a way that's not too
    terribly performing.

While I'm at it, I also have a feature request for STC (Sup The Client)

    Improved Multiple-Label Support
    -------------------------------
    If I'm only using one label for each thread, then really there's no benefit
    to having labels over folders.  So to really get the benefit of labels, it
    would be nice to be able to attach more than one of them in a more
    convenient way.

    There's a couple of ideas I had on how to do this:

        Label Hierarchies
        -----------------
        if you use one label, it automatically pulls in another.  But I wasn't
        sure how to manage this with the Sup UI.

        Label Recommendations
        ---------------------
        When using tab-complete for labeling, I see all my labels initially.
        By now, it's an overwhelming list, and I have to hunt for the label I
        really wanted.  What would be nice is if Sup keeps an index of all
        label combinations (and perhaps their frequencies).  Then when one hits
        tab the first time only labels that are relevant are presented.  If
        those aren't adequate, then one might hit tab to cycle to a listing
        including the rest of the labels.  Or perhaps without cycling, all
        labels can be listed by frequency; that way, the hunt for the most
        relevant label won't be so bad.

    So Sup probably didn't have this kind of feature because of the stronger
    advocacy for:

        - automated label attachment with hooks

        - not relying on labels as much as one relies on the search engine for
          keyword queries (the GMail way)

    But at work, people send me too many messages with too many different types
    of contexts.  It would take quite a bit of AI to auto-categorize these
    messages.  Keyword searches are maybe okay for some of these messages, but
    it's not really perfect.  Sometimes I have to run through synonyms for
    certain concepts.  Better label management would really help me at work.

Okay, so those are the requests I have thus far.  Hopefully they aren't too
far off base from what you are interested in doing with Sup/STS.  I'm happy to
talk through concepts on this list.  And although I'm still working up my Ruby
skills, I'm happy to try to contribute code.

By the way, are you set up to get these kinds of feature requests with ditz?  I
didn't see a ditz "bugs" database (folder) in my git clone of Sup.  Or are you
just using ditz internally for your own personal tracking of Sup development?

Seems like there's a number of places to issue feature requests:

    - this mailing list

    - ditz, if you can help me figure out how best to use it

    - a FeatureRequest wiki page, but it would be moot if you didn't make a
      habit of looking at it

Just let us know what you prefer.

-Sukant


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-23 22:26           ` William Morgan
@ 2008-07-24  9:16             ` Nicolas Pouillard
  0 siblings, 0 replies; 24+ messages in thread
From: Nicolas Pouillard @ 2008-07-24  9:16 UTC (permalink / raw)


Excerpts from William Morgan's message of Thu Jul 24 00:26:18 +0200 2008:
> Reformatted excerpts from nicolas.pouillard's message of 2008-07-23:
> > Another design choice, that could (and perhaps *should*) be made; is
> > that stored data don't change in the case of mails, IM..., only their
> > labels and children (threading) do.
> 
> I think that's a true fact. Draft messages are sort of an exception, but
> editing can always be simulated as deleting the old one and adding the
> new one.

Perhaps  that  drafts  should  be marked as "short-lived", and left in another
store.  This  reminds me how generational Garbage Collectors works, there is a
kind of relation...

-- 
Nicolas Pouillard aka Ertai
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080724/2cc5f51a/attachment.bin>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-24  4:55 ` Guarded Identity
@ 2008-07-24  9:21   ` Nicolas Pouillard
  2008-07-25 16:00     ` William Morgan
  2008-07-25  7:18   ` William Morgan
  1 sibling, 1 reply; 24+ messages in thread
From: Nicolas Pouillard @ 2008-07-24  9:21 UTC (permalink / raw)


Excerpts from Guarded Identity's message of Thu Jul 24 06:55:38 +0200 2008:
> Excerpts from wmorgan-sup-at-masanjin.net wmorgan-sup-at-masanjin.net |Sup_Talk|'s message of Sun Jul 20 16:33:24 -0500 2008:
> > In which the new version of Sup is described!
> > 
> > http://all-thing.net/2008/07/rethinking-sup-part-ii.html
> 
> Your first blog was intriguing, but this second one is putting Sup in a
> direction I'm really happy with.  I'm happy about:
> 
>     Central Server
>     --------------
>     I like having a central server, so I can run clients from multiple
>     locations.
> 
>     Modifiable Messages
>     -------------------
>     There was a post I read in this thread indicating that people didn't want
>     to modify documents.   But if we handle this as a "delete/re-add"
>     operation, I'm hoping it will be fine.
> 
>     In all honesty, the modifications I'd like to do are
> 
>         - remove attachments.  Sometimes at work, people send out some
>           absolutely useless, gigantic attachments.
> 
>         - delete/expire documents.
> 
>     Documents Beyond E-mail
>     -----------------------
>     I'm /really/ interested in storing different types of "documents" like RSS.
>     . . maybe newsgroups too?  Then I could stop using snownews and slrn.  One
>     client. . . actually, for this reason, I'd tried out Emacs Gnus, but I
>     wasn't really happy with the experience/complexity.
> 
> However, I was hoping to see if I could get in a request while you're in the
> beginning stages of STS development.
> 
>     True Thread-level Labels
>     ------------------------

Here is another big design decision, note that some labels are truly message
based:

  * unread: of course
  * spam: to be sure that a spam message a response to a ham message don't
          throw away the whole thread.
  * starred: sometime it's there to highlight a message and sometimes to
             select the whole thread.

And some others are truly thread based:

  * inbox
  * most of users labels
  * killed

Since  there  is  mandatory  requirements  for message based labels and thread
based  labels,  one can provide both. Perhaps a syntactic distinction would be
sufficient:
  * a case distinction: INBOX vs unread
  * a special mark: inbox* vs unread (here the star means the repetition on
    all messages of the thread).
  * more ideas to find...

>     I was using rss2email, so the number of messages I had was enormous
>     (although I disabled that due to performance problems, Ferret corruption,
>     and the problem I'm discussion here).  I hate wasting disk space
>     needlessly, or cluttering the index for that matter.  There's some mailing
>     lists that are mostly fluff, but I wanted to save the occasional thread and
>     expire the rest.  But because messages attach to messages really, and not
>     to the thread, negation in search logic breaks.  If I search for threads
>     that have "!label:save" I can get threads that have "save" messages if they
>     also have a message not labeled "save".
> 
>     Attaching labels to message always bothered me.  I'm hoping that with STS,
>     you can consider designing an architecture that lets you attach labels to a
>     thread grouping.  I'm hoping it can be done in a way that's not too
>     terribly performing.
> 
> While I'm at it, I also have a feature request for STC (Sup The Client)
> 
>     Improved Multiple-Label Support
>     -------------------------------
>     If I'm only using one label for each thread, then really there's no benefit
>     to having labels over folders.  So to really get the benefit of labels, it
>     would be nice to be able to attach more than one of them in a more
>     convenient way.
> 
>     There's a couple of ideas I had on how to do this:
> 
>         Label Hierarchies
>         -----------------
>         if you use one label, it automatically pulls in another.  But I wasn't
>         sure how to manage this with the Sup UI.
> 
>         Label Recommendations
>         ---------------------
>         When using tab-complete for labeling, I see all my labels initially.
>         By now, it's an overwhelming list, and I have to hunt for the label I
>         really wanted.  What would be nice is if Sup keeps an index of all
>         label combinations (and perhaps their frequencies).  Then when one hits
>         tab the first time only labels that are relevant are presented.  If
>         those aren't adequate, then one might hit tab to cycle to a listing
>         including the rest of the labels.  Or perhaps without cycling, all
>         labels can be listed by frequency; that way, the hunt for the most
>         relevant label won't be so bad.

Hum,  that's  a  very  interesting feature. This reminds me delicio.us. In the
same  line,  be  able  to  share popular/public labels for discussion could be
very nice.

Let's  imagine  that  public  tagging  is  done using public:foo labels, users
could  share  some  labels  using  public  ones.  Then  in  answer  message an
X-Public-Labels  header  is  added  and  then suggested to users when applying
labels.

> 
>     So Sup probably didn't have this kind of feature because of the stronger
>     advocacy for:
> 
>         - automated label attachment with hooks
> 
>         - not relying on labels as much as one relies on the search engine for
>           keyword queries (the GMail way)
> 
>     But at work, people send me too many messages with too many different types
>     of contexts.  It would take quite a bit of AI to auto-categorize these
>     messages.  Keyword searches are maybe okay for some of these messages, but
>     it's not really perfect.  Sometimes I have to run through synonyms for
>     certain concepts.  Better label management would really help me at work.
> 
> Okay, so those are the requests I have thus far.  Hopefully they aren't too
> far off base from what you are interested in doing with Sup/STS.  I'm happy to
> talk through concepts on this list.  And although I'm still working up my Ruby
> skills, I'm happy to try to contribute code.
> 
> By the way, are you set up to get these kinds of feature requests with ditz?  I
> didn't see a ditz "bugs" database (folder) in my git clone of Sup.  Or are you
> just using ditz internally for your own personal tracking of Sup development?

It's in the bugs branch.

> 
> Seems like there's a number of places to issue feature requests:
> 
>     - this mailing list
> 
>     - ditz, if you can help me figure out how best to use it
> 
>     - a FeatureRequest wiki page, but it would be moot if you didn't make a
>       habit of looking at it
> 
> Just let us know what you prefer.

I'm not William but I would say that the mailing list is the preferred way.
Moreover attaching a git-patch for the ditz issue tracker could help.

Cheers,

-- 
Nicolas Pouillard aka Ertai
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://rubyforge.org/pipermail/sup-talk/attachments/20080724/598bc93c/attachment.bin>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-24  4:55 ` Guarded Identity
  2008-07-24  9:21   ` Nicolas Pouillard
@ 2008-07-25  7:18   ` William Morgan
  1 sibling, 0 replies; 24+ messages in thread
From: William Morgan @ 2008-07-25  7:18 UTC (permalink / raw)


Reformatted excerpts from Guarded Identity's message of 2008-07-23:
> In all honesty, the modifications I'd like to do are
>
>   - remove attachments.  Sometimes at work, people send out some
>     absolutely useless, gigantic attachments.
> 
>   - delete/expire documents.

Sup 2.0 will definitely handle both of these cases.

> I'm /really/ interested in storing different types of "documents" like
> RSS.  . . maybe newsgroups too?

Sure. Anything you can munge into Sup's expected format will be
supported. I'm expecting, in addition to a variety of clients (well
if other people get excited!), a variety of readers, each of which
understands some how to get and import into Sup some kind of
information.

> There's some mailing lists that are mostly fluff, but I wanted to save
> the occasional thread and expire the rest.  But because messages
> attach to messages really, and not to the thread, negation in search
> logic breaks.  If I search for threads that have "!label:save" I can
> get threads that have "save" messages if they also have a message not
> labeled "save".

The problem here is not really that there are labels on messages, it's
that the current search operation is: for each message that matches the
query, return the thread that contains it. That works for positive
searches, but clearly doesn't work for negative ones.

Making negative search efficient is hard. But I don't think it matters
whether the labels are per-message or per-thread... 

> Attaching labels to message always bothered me.

How come? Every other property of a thread is calculated from its
component messages---subject, recipients, date---so why not the labels
as well? It's a nice symmetry.

Anyways, I'm not sure what the right answer is yet. It certainly could
be per-thread labels.

> Label Hierarchies
> -----------------
> if you use one label, it automatically pulls in another.  But I wasn't
> sure how to manage this with the Sup UI.

It would be doable... probably through a hook.

> When using tab-complete for labeling, I see all my labels initially.
> By now, it's an overwhelming list, and I have to hunt for the label I
> really wanted.  What would be nice is if Sup keeps an index of all
> label combinations (and perhaps their frequencies).  Then when one
> hits tab the first time only labels that are relevant are presented.
> If those aren't adequate, then one might hit tab to cycle to a listing
> including the rest of the labels.  Or perhaps without cycling, all
> labels can be listed by frequency; that way, the hunt for the most
> relevant label won't be so bad.

The last of those options (list all but by frequency) is definitely
doable. The other options are probably a fair amount of work. You can
play around with LabelSearchMode and LabelManager if you want to try and
mock something up. I don't expect those to change too much, except that
LabelManager will probably become a thin shell talking to the server.

> By the way, are you set up to get these kinds of feature requests with
> ditz?  I didn't see a ditz "bugs" database (folder) in my git clone of
> Sup. Or are you just using ditz internally for your own personal
> tracking of Sup development?

As Nicolas pointed out, it's currently on the bugs branch, but I'm
planning to move it back into master and starting to add issues there. I
also have tentative plans to move Sup's git repo over to gitorious so
that people can see pretty pictures when they clone.

> Seems like there's a number of places to issue feature requests:
>   - this mailing list
>   - ditz, if you can help me figure out how best to use it
>   - a FeatureRequest wiki page, but it would be moot if you didn't make a
>     habit of looking at it
> 
> Just let us know what you prefer.

The mailing list, ditz patches, or ditz merge requests are all perfectly
acceptable.
-- 
William <wmorgan-sup at masanjin.net>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-23 17:09   ` William Morgan
@ 2008-07-25 12:11     ` Marcus Williams
  0 siblings, 0 replies; 24+ messages in thread
From: Marcus Williams @ 2008-07-25 12:11 UTC (permalink / raw)


On 23.7.2008, William Morgan wrote:
> > There are a number of other indexs such as Xapian or Hyper Estraier
> > both have ruby bindings and both are quite mature and widely used in
> > other projects.
> 
> Interesting. I will check those out. I only picked Sphinx because it
> seemed popular and robust, but it is currently making my life more difficult
> than it really needs to be.

You might enjoy datamapper  - http://datamapper.org

I've been knocking up web apps in ruby/sinatra/datamapper in stupidly
few lines with absolutely no contact with the db underlying the system
other than through datamapper.

Marcus


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-24  9:21   ` Nicolas Pouillard
@ 2008-07-25 16:00     ` William Morgan
  2008-07-26  7:32       ` Guarded Identity
  0 siblings, 1 reply; 24+ messages in thread
From: William Morgan @ 2008-07-25 16:00 UTC (permalink / raw)


Reformatted excerpts from nicolas.pouillard's message of 2008-07-24:
> Here is another big design decision, note that some labels are truly
> message based:
> 
>   * unread: of course
>   * spam: to be sure that a spam message a response to a ham message
>     don't throw away the whole thread.
>   * starred: sometime it's there to highlight a message and sometimes
>     to select the whole thread.

What's interesting about the labels you've listed is that they all have
special semantics, i.e. the server or the client does something special
based on whether they're present.

Do people have "user" labels that they use to distinguish individual
messages, as opposed to individual threads?

If not, we could have labels only on threads by having a bunch of
boolean flags on messages: read/unread, spam/ham, starred/regular,
draft/not-draft, etc. The search syntax would probably change for those
features.

> Since  there  is  mandatory  requirements  for message based labels
> and thread based  labels,  one can provide both. Perhaps a syntactic
> distinction would be sufficient:
>   * a case distinction: INBOX vs unread
>   * a special mark: inbox* vs unread (here the star means the
>     repetition on all messages of the thread).

Having both would be possible too, but it seems overly complicated to
me. I think that's my least favorite option right now.

This is a good discussion though. I haven't really thought this all
through.
-- 
William <wmorgan-sup at masanjin.net>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* [sup-talk] rethinking sup part ii
  2008-07-25 16:00     ` William Morgan
@ 2008-07-26  7:32       ` Guarded Identity
  0 siblings, 0 replies; 24+ messages in thread
From: Guarded Identity @ 2008-07-26  7:32 UTC (permalink / raw)


Excerpts from William Morgan's message of Fri Jul 25 11:00:40 -0500 2008:
>
> Reformatted excerpts from nicolas.pouillard's message of 2008-07-24:
  >
> > Here is another big design decision, note that some labels are truly
> > message based:
> >
> >   * unread: of course
> >   * spam: to be sure that a spam message a response to a ham message
> >     don't throw away the whole thread.
> >   * starred: sometime it's there to highlight a message and sometimes
> >     to select the whole thread.
>
> What's interesting about the labels you've listed is that they all have
> special semantics, i.e. the server or the client does something special
> based on whether they're present.
>
> Do people have "user" labels that they use to distinguish individual
> messages, as opposed to individual threads?

I got away with applying the "spam" label at the thread level because I never
get any spam that's actually part of a thread; it's normally just a stand-alone
piece of mail.

I'm not sure what other people use starring for, but I just use it to highlight
things I need to reply to, and for that purpose, it works fine at a
thread-level because most threads only have one response I'm interested in
issuing.  But I guess I've seen some of those gigantic threads, so maybe
starring at the message-level /might/ be useful, but I'd not sweat it if it
wasn't possible.

Even for "unread", I get away just managing it at the thread-level, because I'm
generally marking an entire thread as read.  However, I'd imagine someone might
want to revert the unread status of part of a thread.  However, that would be a
nightmare for a large thread.  In that case, I'd probably just use the "@"
key-binding to revert labels.

So all in all, I never use message-level labels, even though I get understand
the instance in which they're needed.  The major reason for this is that the
presentation of the search is at the thread level, so it makes little sense to
think of labels at the message level.

> If not, we could have labels only on threads by having a bunch of
> boolean flags on messages: read/unread, spam/ham, starred/regular,
> draft/not-draft, etc. The search syntax would probably change for those
> features.

This is /exactly/ what I'm hoping for.  I could probably come up with other
hypothetical abstractions for indexing, but I'm happy as long as at the end of
the day I get negation in reasonably fast searches.  I know it sounds like a
feature that not everyone might use, but the more messages I put in the index,
the more I find I myself trying to make use of negation to prune searches.
Also, sometimes I like to manage Sup with scripts and having negation allows me
to do some more cool things without having to bother William with feature
requests for the odd message/label-managing tasks I have.

Hopefully you can continue brain-storming what design you want to do to support
this.  If you run into a problem, maybe we can jump into the design then.

> > Since  there  is  mandatory  requirements  for message based labels
> > and thread based  labels,  one can provide both. Perhaps a syntactic
> > distinction would be sufficient:
> >   * a case distinction: INBOX vs unread
> >   * a special mark: inbox* vs unread (here the star means the
> >     repetition on all messages of the thread).
>
> Having both would be possible too, but it seems overly complicated to
> me. I think that's my least favorite option right now.

Yeah, I was trying to think of a way to manage both, and it felt like a kludge.
Personally, I like the idea of managing as much as we can at the thread-level.

-Sukant


^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2008-07-26  7:32 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-07-20 21:33 [sup-talk] rethinking sup part ii William Morgan
2008-07-20 23:28 ` Guillaume Quintard
2008-07-21  0:04   ` William Morgan
2008-07-21  5:43     ` Guillaume Quintard
2008-07-21 12:14 ` Peter Krenn
2008-07-21 15:22   ` William Morgan
2008-07-21 15:26     ` Stephen Patterson
2008-07-21 16:04       ` John Bent
2008-07-22  2:12       ` William Morgan
2008-07-23 17:40         ` Nicolas Pouillard
2008-07-23 22:26           ` William Morgan
2008-07-24  9:16             ` Nicolas Pouillard
2008-07-21 18:43 ` Lionel Ott
2008-07-23  8:45 ` Richard Heycock
2008-07-23  8:58   ` Nicolas Pouillard
2008-07-23  9:27     ` teroz
2008-07-23 21:41       ` Richard Heycock
2008-07-23 17:09   ` William Morgan
2008-07-25 12:11     ` Marcus Williams
2008-07-24  4:55 ` Guarded Identity
2008-07-24  9:21   ` Nicolas Pouillard
2008-07-25 16:00     ` William Morgan
2008-07-26  7:32       ` Guarded Identity
2008-07-25  7:18   ` William Morgan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox