* [sup-talk] [bug] NoMemoryError on huge mbox
@ 2008-06-01 17:26 Gabriel Landais
2008-06-03 2:34 ` William Morgan
0 siblings, 1 reply; 4+ messages in thread
From: Gabriel Landais @ 2008-06-01 17:26 UTC (permalink / raw)
Hi,
I'm not able to load my Thunderbird mbox into sup. Added my Inbox
mbox with sup-add/sup-config :
-------------------------------------------------
[glandais at server:~/.mozilla-thunderbird/4v86e51w.default/Mail/pops.codelutin.com]
$ /var/lib/gems/1.8/gems/sup-0.5/bin/sup-config
[Sun Jun 01 19:14:24 +0200 2008] using character set encoding "UTF-8"
[Sun Jun 01 19:14:24 +0200 2008] optional 'chronic' library not found
(run 'gem install chronic' to install)
/var/lib/gems/1.8/gems/highline-1.4.0/lib/highline/system_extensions.rb:125:
warning: Insecure world writable dir /opt/java in PATH, mode 040777
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/crypto.rb:18: warning: Insecure
world writable dir /opt/java in PATH, mode 040777
[Sun Jun 01 19:14:25 +0200 2008] crypto: detected gpg binary in /usr/bin/gpg
Howdy neighbor! This here's sup-config, ready to help you jack in to
the next generation of digital cyberspace: the text-based email
program. Get ready to be the envy of everyone in your internets
with your amazing keyboarding skills! Jump from email to email with
nary a click of the mouse!
Just answer these simple questions and you'll be on your way.
What's your name? (enter for "Gabriel Landais"):
What's your (primary) email address? (enter for "glandais at codelutin.com"):
Ok, your header will look like this:
From: Gabriel Landais <glandais at codelutin.com>
Do you have any alternate email addresses that also receive email?
If so, enter them now, separated by spaces.
Alternate email addresses:
What file contains your signature? (enter for "/home/glandais/.signature"):
What editor would you like to use? (enter for "nano"):
Ok, I've saved you up a nice lil' /home/glandais/.sup/config.yaml.
Now, we'll tell Sup where to find all your email.
Current sources:
* mbox:/home/glandais/.mozilla-thunderbird/4v86e51w.default/Mail/pops.codelutin.com/Inbox
* sup://drafts
* sup://sent
1. Add a new source.
2. Done adding sources!
Your wish? 2
Ok. The final step is to import all your messages into the Sup index.
Depending on how many messages are in the sources, this could take
quite a while.
Run sup-sync to import all messages now? (enter for "n"): y
Ok, trying to run "/var/lib/gems/1.8/gems/sup-0.5/bin/sup-sync --all-sources"...
/var/lib/gems/1.8/gems/sup-0.5/bin/sup-config:225: warning: Insecure
world writable dir /opt/java in PATH, mode 040777
[Sun Jun 01 19:14:37 +0200 2008] using character set encoding "UTF-8"
[Sun Jun 01 19:14:37 +0200 2008] optional 'chronic' library not found
(run 'gem install chronic' to install)
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/crypto.rb:18: warning: Insecure
world writable dir /opt/java in PATH, mode 040777
[Sun Jun 01 19:14:37 +0200 2008] crypto: detected gpg binary in /usr/bin/gpg
[Sun Jun 01 19:14:37 +0200 2008] locking /home/glandais/.sup/lock...
[Sun Jun 01 19:14:37 +0200 2008] loading index...
[Sun Jun 01 19:14:37 +0200 2008] loaded index of 0 messages
Scanning mbox:/home/glandais/.mozilla-thunderbird/4v86e51w.default/Mail/pops.codelutin.com/Inbox...
[Sun Jun 01 19:17:40 +0200 2008] unlocking /home/glandais/.sup/lock...
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/mbox/loader.rb:60:in
`load_header': failed to allocate memory (NoMemoryError)
from /var/lib/gems/1.8/gems/sup-0.5/lib/sup/mbox/loader.rb:56:in
`synchronize'
from /var/lib/gems/1.8/gems/sup-0.5/lib/sup/mbox/loader.rb:56:in
`load_header'
from /var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:534:in `send'
from /var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:534:in `__pass'
from /var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:523:in
`method_missing'
from /var/lib/gems/1.8/gems/sup-0.5/lib/sup/message.rb:62:in
`initialize'
from /var/lib/gems/1.8/gems/sup-0.5/lib/sup/poll.rb:151:in `new'
from /var/lib/gems/1.8/gems/sup-0.5/lib/sup/poll.rb:151:in
`add_messages_from'
... 6 levels...
from /var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:497:in
`method_missing'
from /var/lib/gems/1.8/gems/sup-0.5/bin/sup-sync:136
from /var/lib/gems/1.8/gems/sup-0.5/bin/sup-sync:131:in `each'
from /var/lib/gems/1.8/gems/sup-0.5/bin/sup-sync:131
Rats, that failed. You may have to do it manually.
Try again? (enter for "n"): n
[Sun Jun 01 19:17:47 +0200 2008] loading index...
[Sun Jun 01 19:17:47 +0200 2008] loaded index of 0 messages
Okee doke, you've got yourself an index of 0 messages. Looks
like you're ready to jack in to cyberspace there, cowboy.
Just one last command:
/var/lib/gems/1.8/gems/sup-0.5/bin/sup
Have fun!
-------------------------------------------------
It uses all my RAM then crash. Same when sup is run :
-------------------------------------------------
[glandais at server:~/.mozilla-thunderbird/4v86e51w.default/Mail/pops.codelutin.com]
$ /var/lib/gems/1.8/gems/sup-0.5/bin/sup
[Sun Jun 01 19:18:37 +0200 2008] using character set encoding "UTF-8"
[Sun Jun 01 19:18:38 +0200 2008] optional 'chronic' library not found
(run 'gem install chronic' to install)
[Sun Jun 01 19:18:38 +0200 2008] locking /home/glandais/.sup/lock...
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/crypto.rb:18: warning: Insecure
world writable dir /opt/java in PATH, mode 040777
[Sun Jun 01 19:18:38 +0200 2008] crypto: detected gpg binary in /usr/bin/gpg
[Sun Jun 01 19:18:38 +0200 2008] loading index...
[Sun Jun 01 19:18:38 +0200 2008] loaded index of 0 messages
[Sun Jun 01 19:18:38 +0200 2008] starting curses
[Sun Jun 01 19:21:47 +0200 2008] stopped cursing
[Sun Jun 01 19:21:47 +0200 2008] oh crap, an exception
[Sun Jun 01 19:21:47 +0200 2008] unlocking /home/glandais/.sup/lock...
----------------------------------------------------------------
I'm very sorry. It seems that an error occurred in Sup. Please
accept my sincere apologies. If you don't mind, please send the
contents of sup-exception-log.txt and a brief report of the
circumstances to sup-talk at rubyforge dot orgs so that I might
address this problem. Thank you!
Sincerely,
William
----------------------------------------------------------------
--- NoMemoryError from thread: poll after loading inbox
failed to allocate memory
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/mbox/loader.rb:60:in `load_header'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/mbox/loader.rb:56:in `synchronize'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/mbox/loader.rb:56:in `load_header'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:534:in `send'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:534:in `__pass'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:523:in `method_missing'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/message.rb:62:in `initialize'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/poll.rb:151:in `new'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/poll.rb:151:in `add_messages_from'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/source.rb:98:in `each'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:534:in `send'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:534:in `__pass'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:523:in `method_missing'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/poll.rb:141:in `add_messages_from'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/poll.rb:98:in `do_poll'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/poll.rb:86:in `each'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/poll.rb:86:in `do_poll'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/poll.rb:85:in `synchronize'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/poll.rb:85:in `do_poll'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:497:in `send'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:497:in `method_missing'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/modes/poll-mode.rb:17:in `poll'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/poll.rb:53:in `poll'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:497:in `send'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/util.rb:497:in `method_missing'
/var/lib/gems/1.8/gems/sup-0.5/bin/sup:210
/var/lib/gems/1.8/gems/sup-0.5/lib/sup.rb:60:in `reporting_thread'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup.rb:58:in `initialize'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup.rb:58:in `new'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup.rb:58:in `reporting_thread'
/var/lib/gems/1.8/gems/sup-0.5/bin/sup:210
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/modes/thread-index-mode.rb:536:in `call'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/modes/thread-index-mode.rb:536:in
`__unprotected_load_threads'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/modes/thread-index-mode.rb:478:in `call'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/modes/thread-index-mode.rb:478:in
`load_n_threads_background'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup.rb:60:in `reporting_thread'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup.rb:58:in `initialize'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup.rb:58:in `new'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup.rb:58:in `reporting_thread'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/modes/thread-index-mode.rb:476:in
`load_n_threads_background'
/var/lib/gems/1.8/gems/sup-0.5/lib/sup/modes/thread-index-mode.rb:546:in
`__unprotected_load_threads'
(eval):12:in `load_threads'
/var/lib/gems/1.8/gems/sup-0.5/bin/sup:210
--- SystemExit from thread: main
failed to allocate memory
/var/lib/gems/1.8/gems/sup-0.5/lib/sup.rb:64:in `method_missing'
/var/lib/gems/1.8/gems/sup-0.5/bin/sup:226
-------------------------------------------------
Thanks
Gabriel
^ permalink raw reply [flat|nested] 4+ messages in thread
* [sup-talk] [bug] NoMemoryError on huge mbox
2008-06-01 17:26 [sup-talk] [bug] NoMemoryError on huge mbox Gabriel Landais
@ 2008-06-03 2:34 ` William Morgan
2008-06-03 8:09 ` Gabriel Landais
0 siblings, 1 reply; 4+ messages in thread
From: William Morgan @ 2008-06-03 2:34 UTC (permalink / raw)
Reformatted excerpts from Gabriel Landais's message of 2008-06-01:
> I'm not able to load my Thunderbird mbox into sup. Added my Inbox
> mbox with sup-add/sup-config :
Interesting. This error seems to occur during a string construction.
Others have reported weird problems with Thunderbird mbox files before,
and I now suspect there's something screwy about their mbox format.
Like, they use \r instead of \n for linebreaks. Which wouldn't make any
fucking sense at all, but who knows.
Can you post the output of a
ruby -e 'p File.open("Inbox").read(1000)'
so we can take a look at what these guys are doing? (Or whatever
Thunderbird file you want instead of "Inbox".)
Thanks!
--
William <wmorgan-sup at masanjin.net>
^ permalink raw reply [flat|nested] 4+ messages in thread
* [sup-talk] [bug] NoMemoryError on huge mbox
2008-06-03 2:34 ` William Morgan
@ 2008-06-03 8:09 ` Gabriel Landais
2008-06-04 2:42 ` William Morgan
0 siblings, 1 reply; 4+ messages in thread
From: Gabriel Landais @ 2008-06-03 8:09 UTC (permalink / raw)
On Tue, Jun 3, 2008 at 4:34 AM, William Morgan <wmorgan-sup at masanjin.net> wrote:
> Reformatted excerpts from Gabriel Landais's message of 2008-06-01:
>> I'm not able to load my Thunderbird mbox into sup. Added my Inbox
>> mbox with sup-add/sup-config :
>
> Interesting. This error seems to occur during a string construction.
> Others have reported weird problems with Thunderbird mbox files before,
> and I now suspect there's something screwy about their mbox format.
> Like, they use \r instead of \n for linebreaks. Which wouldn't make any
> fucking sense at all, but who knows.
>
> Can you post the output of a
>
> ruby -e 'p File.open("Inbox").read(1000)'
>
> so we can take a look at what these guys are doing? (Or whatever
> Thunderbird file you want instead of "Inbox".)
>
Here it is (for 10, same thing for 100 000!) :
[glandais at server:~/.mozilla-thunderbird/4v86e51w.default/Mail/pops.codelutin.com]
$ ruby -e 'p File.open("Inbox").read(10)'
"\000\000\000\000\000\000\000\000\000\000"
Perhaps I just shouldn't use that mbox. I never understood how
messages are saved in these directories. My folder looks like that :
[glandais at server:~/.mozilla-thunderbird/4v86e51w.default/Mail] $ durep
[ /home/glandais/.mozilla-thunderbird/4v86e51w.default/Mail/ 520.3M ]
276.2M [%%%%%%%%%%%%%%% ] 1 53.08% Local Folders/
252.9M [%%%%%%%%%%%%%%%%%%%%%%%%%%% ] 5 91.59% Inbox.sbd/
207.8M [%%%%%%%%%%%%%%%%%%%%%%%% ] 5 82.16% Listes.sbd/
202.2M [%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ] 0 97.30% cl.sbd/
5.5M [ ] 0 2.63%
decathlon.sbd/
60.0K [ ] 0 0.03% le.sbd/
36.0K [ ] 14 0.02% [FILES]
28.0K [ ] 1 0.01% bricks.sbd/
12.0K [%%%%%%%%%%%% ] 0
42.86% tapestry.sbd/
8.0K [%%%%%%%% ] 4 28.57% [FILES]
12.0K [ ] 0 0.01% lucene.sbd/
31.3M [%%% ] 16 12.36% [FILES]
7.1M [ ] 0 2.81% Projets.sbd/
4.5M [ ] 0 1.77% formations.sbd/
2.1M [ ] 0 0.84%
simexplorer-si.sbd/
124.0K [ ] 0 0.05% sandra.sbd/
23.2M [%% ] 15 8.41% [FILES]
244.1M [%%%%%%%%%%%%%% ] 0 46.92% pops.codelutin.com/
It was just to give a try to Sup on Thunderbird, I'll try another mbox.
Thanks
Gabriel
^ permalink raw reply [flat|nested] 4+ messages in thread
* [sup-talk] [bug] NoMemoryError on huge mbox
2008-06-03 8:09 ` Gabriel Landais
@ 2008-06-04 2:42 ` William Morgan
0 siblings, 0 replies; 4+ messages in thread
From: William Morgan @ 2008-06-04 2:42 UTC (permalink / raw)
Reformatted excerpts from Gabriel Landais's message of 2008-06-03:
> Here it is (for 10, same thing for 100 000!) :
>
> [glandais at server:~/.mozilla-thunderbird/4v86e51w.default/Mail/pops.codelutin.com
> ]
> $ ruby -e 'p File.open("Inbox").read(10)'
> "\000\000\000\000\000\000\000\000\000\000"
Ok, well, you can't really fault Sup for that one! It's just reading
until it gets a newline. :)
Still, why would Thunderbird produce a file full of zeros?
> Perhaps I just shouldn't use that mbox. I never understood how
> messages are saved in these directories.
Here's what the Thunderbird FAQ claims:
Your mail files are inside your profile (see the Profile Folder), in
the Mail and (if you use IMAP) ImapMail folders. Each mail folder
(Inbox, Sent, etc.) is stored as two files ? one with no extension
(e.g. INBOX), which is the mail file itself (in "mbox" format), and
one with an .msf extension (e.g. INBOX.msf), which is the index (Mail
Summary File) to the mail file. Tell the other program to import mail
from the file with no extension.
http://www.mozilla.org/support/thunderbird/faq
Which implies to me that Inbox was the correct file. But apparently not.
Are there any other files that look like normal mbox files?
--
William <wmorgan-sup at masanjin.net>
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2008-06-04 2:42 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-06-01 17:26 [sup-talk] [bug] NoMemoryError on huge mbox Gabriel Landais
2008-06-03 2:34 ` William Morgan
2008-06-03 8:09 ` Gabriel Landais
2008-06-04 2:42 ` William Morgan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox