Subject: Re: [sup-talk] try heliotrope!

From: William Morgan <[email-masked]>
Date: Wed, 29 Jun 2011 04:02:05 +0000

Hi Andrés,

Thanks for the feedback!

Reformatted excerpts from Andrés's message of 2011-06-27:
> As someone already mentioned, the Gmail import is working great, IF you have
> English as your Gmail interface language; otherwise heliotrope just keeps
> looking for the "All Mail" directory;

I think I've fixed this (using the xlist code you discovered). Could you tell
me if it works for you?

> I've found a small bug on the debugging interface: when a thread has
> no subject (some people do that)

Should be fixed.

> The --host parameter does not seem to be implemented (or I am
> misunderstanding what it is supposed to do)

Should be fixed, and defaults to localhost as you suggest.

Keep it coming.
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: William Morgan <[email-masked]>
Date: Tue, 28 Jun 2011 03:54:01 +0000

Reformatted excerpts from sergeig's message of 2011-06-27:
> /home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb.rb:11: [BUG] Segmentation fault
> ruby 1.9.2p180 (2011-02-18 revision 30909) [i686-linux]

Yipes. I've released leveldb-ruby 0.5. Can you try with that please?
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: [email-masked]
Date: Mon, 27 Jun 2011 14:30:32 -0700

Server starts, but mbox add does not work.

On ubuntu 10.04, the gem found for livedb is "leveldb-ruby". Not sure if this is related.

==
$ ruby -Ilib src/git/heliotrope/bin/heliotrope-add -m Mail/fb-mon.mbox -d .heliotrope/fb-mon/
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb.rb:11: [BUG] Segmentation fault
ruby 1.9.2p180 (2011-02-18 revision 30909) [i686-linux]

-- control frame ----------
c:0008 p:---- s:0053 b:0053 l:000052 d:000052 CFUNC  :make
c:0007 p:0017 s:0047 b:0047 l:000046 d:000046 METHOD /home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb.rb:11
c:0006 p:0054 s:0043 b:0043 l:000042 d:000042 METHOD /home/sergei/src/git/heliotrope/lib/heliotrope/index.rb:45
c:0005 p:---- s:0038 b:0038 l:000037 d:000037 FINISH
c:0004 p:---- s:0036 b:0036 l:000035 d:000035 CFUNC  :new
c:0003 p:0798 s:0031 b:0031 l:000ebc d:001680 EVAL   src/git/heliotrope/bin/heliotrope-add:113
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:000ebc d:000ebc TOP   
---------------------------
-- Ruby level backtrace information ----------------------------------------
src/git/heliotrope/bin/heliotrope-add:113:in `<main>'
src/git/heliotrope/bin/heliotrope-add:113:in `new'
/home/sergei/src/git/heliotrope/lib/heliotrope/index.rb:45:in `initialize'
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb.rb:11:in `new'
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb.rb:11:in `make'

-- C level backtrace information -------------------------------------------
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(rb_vm_bugreport+0x72) [0xb7807102]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x56657) [0xb76df657]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(rb_bug+0x3a) [0xb76df74a]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x10a1c4) [0xb77931c4]
[0xb787c410]
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb/leveldb.so(_ZN7leveldb8SkipListIPKcNS_8MemTable13KeyComparatorEE6InsertERKS2_+0x2c) [0xb712feec]
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb/leveldb.so(_ZN7leveldb8MemTable3AddEyNS_9ValueTypeERKNS_5SliceES4_+0x11a) [0xb712f85a]
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb/leveldb.so(+0x29699) [0xb7138699]
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb/leveldb.so(_ZNK7leveldb10WriteBatch7IterateEPNS0_7HandlerE+0x1cd) [0xb71389bd]
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb/leveldb.so(_ZN7leveldb18WriteBatchInternal10InsertIntoEPKNS_10WriteBatchEPNS_8MemTableE+0x40) [0xb7138a60]
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb/leveldb.so(_ZN7leveldb6DBImpl14RecoverLogFileEyPNS_11VersionEditEPy+0x35c) [0xb7126d6c]
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb/leveldb.so(_ZN7leveldb6DBImpl7RecoverEPNS_11VersionEditE+0x261) [0xb71280b1]
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb/leveldb.so(_ZN7leveldb2DB4OpenERKNS_7OptionsERKSsPPS0_+0xbd) [0xb712ad2d]
/home/sergei/.rvm/gems/ruby-1.9.2-p180/gems/leveldb-ruby-0.4/lib/leveldb/leveldb.so(+0x14165) [0xb7123165]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x1681b9) [0xb77f11b9]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x178e1e) [0xb7801e1e]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x16db50) [0xb77f6b50]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x173128) [0xb77fc128]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x174902) [0xb77fd902]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(rb_obj_call_init+0x50) [0xb76e18b0]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(rb_class_new_instance+0x3a) [0xb77302ea]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x168208) [0xb77f1208]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x178e1e) [0xb7801e1e]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x16db50) [0xb77f6b50]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x173128) [0xb77fc128]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(rb_iseq_eval_main+0x1f2) [0xb77fc562]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(+0x58f3a) [0xb76e1f3a]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(ruby_exec_node+0x25) [0xb76e1f75]
/home/sergei/.rvm/rubies/ruby-1.9.2-p180/lib/libruby.so.1.9(ruby_run_node+0x35) [0xb76e3785]
ruby(main+0x68) [0x80487f8]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0xb74b4bd6]
ruby() [0x80486f1]

[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

--
list of gems:

==
$ gem list

*** LOCAL GEMS ***

abstract (1.0.0)
actionmailer (3.0.9)
actionpack (3.0.9)
activemodel (3.0.9)
activerecord (3.0.9)
activeresource (3.0.9)
activesupport (3.0.9)
arel (2.1.3, 2.0.10)
builder (3.0.0, 2.1.2)
bundler (1.0.15)
erubis (2.7.0, 2.6.6)
gettext (2.1.0)
highline (1.6.2)
i18n (0.6.0, 0.5.0)
leveldb-ruby (0.4)
locale (2.0.5)
lockfile (1.4.3)
mail (2.3.0, 2.2.19)
mime-types (1.16)
ncursesw (1.2.4.3)
polyglot (0.3.1)
rack (1.3.0, 1.2.3)
rack-mount (0.8.1, 0.6.14)
rack-test (0.6.0, 0.5.7)
rails (3.0.9)
railties (3.0.9)
rake (0.9.2, 0.8.7 ruby)
rdoc (3.6.1)
rest-client (1.6.3)
rmail (1.0.0)
rubygems-update (1.8.5)
sinatra (1.2.6)
sup (0.12.1)
thor (0.14.6)
tilt (1.3.2)
treetop (1.4.9)
trollop (1.16.2)
tzinfo (0.3.28)
whistlepig (0.7)
xapian-core (1.2.3.1)
xapian-full (1.2.3)

==

Thx,

-Sergei

Excerpts from William Morgan's message of 2011-06-26 22:36:24 -0700:
> Hi guys,
> 
> If any of you are interested in trying out the next generation of Sup, I
> have things in a reasonably stable state. It's still too early for a
> gem, but I have updated the README for both client and server sides. So
> if you'd like to run some highly experimental code and see what the
> future of curses-based email might look like, follow the instructions
> in:
> 
> http://github.com/wmorgan/heliotrope
>   and then
> http://github.com/wmorgan/turnsole
> 
> Lots of stuff is broken, but I have been using it as an end-to-end email
> system for a week or so now, so the basics are there. Tell me how it goes!

---

From: William Morgan <[email-masked]>
Date: Tue, 28 Jun 2011 03:32:03 +0000

Reformatted excerpts from William Morgan's message of 2011-06-27:
> If any of you are interested in trying out the next generation of Sup

Thanks for all the feedback so far. I forgot to say: if you find bugs,
please file issues on github (either
https://github.com/wmorgan/heliotrope/issues or
https://github.com/wmorgan/turnsole/issues) so that I don't lose track
of them.

Thanks!
-- 
William <[email-masked]>
_______________________________________________
Sup-devel mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-devel

---

From: William Morgan <[email-masked]>
Date: Wed, 29 Jun 2011 04:12:46 +0000

Reformatted excerpts from JR Boyens's message of 2011-06-27:
> /Users/jboyens/heliotrope/lib/heliotrope/maildir-walker.rb:50:in `block in get_date_in_file': invalid byte sequence in UTF-8 (ArgumentError)

This should be fixed. But I'm not entirely sure; this encoding stuff is tricky.
Can you try again with the latest master, please?
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Andrés <[email-masked]>
Date: Tue, 28 Jun 2011 01:52:06 +0200

> system for a week or so now, so the basics are there. Tell me how it goes!

Hi there,

it seems to be behaving quite well already!

As someone already mentioned, the Gmail import is working great, IF
you have English as your Gmail interface language; otherwise
heliotrope just keeps looking for the "All Mail" directory; I guess
that's because that directory is hard-coded in lib/gmail-dumper.rb, it
looks like this wouldn't only affect users with other language
configuration on Gmail as sometimes the directories are not
"[Gmail]/All Mail" but rather "[Google Mail]/All Mail" [1]; I've
looked around and found this:
[1] http://groups.google.com/group/Gmail-Help-POP-and-IMAP-en/browse_thread/thread/a154105c54f020fb?pli=1
http://code.google.com/intl/ca/apis/gmail/imap/
which, judging from the xlist bit in lib/imap-dumper.rb, you already
know; maybe support for this kind of not-so-strange set-ups via xlist
is already planned?
I tried editing the lib/gmail-dumper.rb file to
GMAIL_FOLDER = "[Gmail]/Alle Nachrichten"
and it looks like that works for the initial import but I don't knwo
if it'd have implications with the "get labels from IMAP" part.

Haven't tried the mbox import that some people are having issues with
so I can't confirm nor deny those.

I've found a small bug on the debugging interface: when a thread has
no subject (some people do that), there is no way to open that thread
(since there is no link); a fix for that (though maybe not a great one
since I'm no ruby expert) could be: replacing the link_to_threadview
method definition in heliotrope-server (lines 510-512) to:
  def link_to_threadview thread_id, text
    %{<a href="/thread/#{thread_id}">#{if text.empty?; "No subject"
else escape_html text end}</a>}
  end

The --host parameter does not seem to be implemented (or I am
misunderstanding what it is supposed to do), this one is quite
important as by default (ignoring firewall rules and etc) anyone on
the LAN could access all of the data from Heliotrope; wouldn't it be
safer to bind only to localhost by default?

I haven't played around too much with the client, but I could connect
to the server from another computer and looks like its behaviour is
just like sup's.
Even though it's logical, maybe the README should mention that the
lib/* files from heliotrope are needed.

This looks really promising!
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Kevin Mark <[email-masked]>
Date: Mon, 27 Jun 2011 11:02:08 -0400

> system for a week or so now, so the basics are there. Tell me how it goes!

Could you provide a few screenshots of both? (with person details removed) 
just curious about it.
-K
-- 
|  .''`.  == Debian GNU/Linux ==.| http://kevix.myopenid.com......|
| : :' :     The Universal OS....| mysite.verizon.net/kevin.mark/.|
| `. `'   http://www.debian.org/.| http://counter.li.org [#238656]|
|___`-____Unless I ask to be CCd,.assume I am subscribed._________|

NEVER RESPOND TO CRITICAL PRESS.  IT IS A GAME YOU CAN ONLY LOSE, AND IT
MAKES US LOOK BAD.
		-- Bruce Perens
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Andrés <[email-masked]>
Date: Wed, 29 Jun 2011 21:47:41 +0200

> Thanks for the feedback!

Thank YOU for the way the software is advancing ;).

> I think I've fixed this (using the xlist code you discovered). Could you tell
Oh yes, that worked perfectly.

>> no subject (some people do that)
Fixed indeed; your solution is way prettier :).

> Should be fixed, and defaults to localhost as you suggest.

Also seems to be working.

> Keep it coming.

Sure, I'm glad I can help; I already subscribed to the sup-devel
mailing list to be more up-to-date.

Cheers,
--
Andrés

---

From: Matthieu Rakotojaona <[email-masked]>
Date: Wed, 29 Jun 2011 22:29:46 +0200

Heliotrope really is an interesting project ! Thanks a lot!

I have a question : all the mails are stored in the <mailstore>
directory, right ?

I have begun to import some of my gmail mails. I have then removed the
<mailstore> dir, but but when I start importing my mails again, it
doesn't start from the beginning. Is there some info I am supposed to
delete to start again from the beginning ?

Thanks !

-- 
Matthieu RAKOTOJAONA

---

From: William Morgan <[email-masked]>
Date: Mon, 27 Jun 2011 05:36:24 +0000

Hi guys,

---

From: Sergei <[email-masked]>
Date: Mon, 27 Jun 2011 23:32:42 -0700

William,

Am I looking at the right place:

https://github.com/wmorgan/leveldb-ruby

v .4 is shown.

Thx,

-Sergei

---

From: JR Boyens <[email-masked]>
Date: Mon, 27 Jun 2011 08:56:59 -0400

> system for a week or so now, so the basics are there. Tell me how it goes!

Seems to work rather good when I can get the messages into heliotrope...

GMail import seems to work great.

I keep having this issue with the Maildirs I use with Sup:

; loading mail...
; scanning 1 directories...
; found 2623 messages
; reading in dates...
; warning: no date in /Users/jboyens/Mail/Work/INBOX/cur/1302957166_2.57311.marvin.local,U=11400,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,S
; warning: no date in /Users/jboyens/Mail/Work/INBOX/cur/1302957167_0.57311.marvin.local,U=10933,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
/Users/jboyens/heliotrope/lib/heliotrope/maildir-walker.rb:50:in `block in get_date_in_file': invalid byte sequence in UTF-8 (ArgumentError)
	from /Users/jboyens/heliotrope/lib/heliotrope/maildir-walker.rb:48:in `open'
	from /Users/jboyens/heliotrope/lib/heliotrope/maildir-walker.rb:48:in `get_date_in_file'
	from /Users/jboyens/heliotrope/lib/heliotrope/maildir-walker.rb:40:in `block in get_files'
	from /Users/jboyens/heliotrope/lib/heliotrope/maildir-walker.rb:40:in `map'
	from /Users/jboyens/heliotrope/lib/heliotrope/maildir-walker.rb:40:in `get_files'
	from /Users/jboyens/heliotrope/lib/heliotrope/maildir-walker.rb:12:in `load!'
	from ./bin/heliotrope-add:123:in `<main>'

I've tried a few maildirs and it seems endemic. The file that it has an
issue with contains a single, solitary, lower-case h.

Any ideas?
--
JR Boyens
[email-masked]

---

From: William Morgan <[email-masked]>
Date: Tue, 28 Jun 2011 22:10:19 +0000

> https://github.com/wmorgan/leveldb-ruby

Sorry. I am bad at putting files in directories, apparently. Try now please.
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: William Morgan <[email-masked]>
Date: Wed, 29 Jun 2011 21:35:54 +0000

> I have a question : all the mails are stored in the <mailstore>

Yes.

> doesn't start from the beginning.

Ah. This is totally undocumented. Heliotrope-add keeps a file called
~/.heliotropeaddrc-something-something where it stores server state for
IMAP and GMail accounts. You will have to delete this to have it start
over.
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk