Subject: Re: [sup-talk] Problems running sup-convert-ferret-index

From: William Morgan <[email-masked]>
Date: Mon, 18 Jan 2010 12:04:41 -0500

Reformatted excerpts from Anirudh Sanjeev's message of 2010-01-18:
> I'm unable to run sup-convert-ferret-index as well.
> 
> It fails like this:

If you're running from git, you need to use:

  RUBY_INVOCATION="ruby -Ilib" ruby -Ilib bin/sup-convert-ferret-index

Give that a whirl and see if it works.
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: William Morgan <[email-masked]>
Date: Sun, 17 Jan 2010 16:34:49 -0500

Reformatted excerpts from Steve Goldman's message of 2010-01-17:
> It went for about ten minutes and appeared to finish with success.
> But something isn't right.  When I started sup the index was empty,
> and my xapian directory looks like this:

Hm, that sure looks empty. Did the sup-sync look like it was working?
If you run the command manually, does that have the same effect?
Is this with git master or next?
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Anirudh Sanjeev <[email-masked]>
Date: Mon, 18 Jan 2010 11:33:23 +0530

Hi,

It fails like this:

----------BEGIN PASTE------------
[11:20]:~/Desktop/sup-mainline$ xclip -o -selection clipboard
[11:20]:~/Desktop/sup-mainline$ ruby -I lib -w bin/sup-convert-ferret-index 
./lib/sup/util.rb:295: warning: `&' interpreted as argument prefix
./lib/sup/util.rb:9: warning: method redefined; discarding old gen_lock_id
./lib/sup/util.rb:20: warning: method redefined; discarding old dump_lock_id
[Mon Jan 18 11:20:40 +0530 2010] WARNING: can't find character set by using locale, defaulting to utf-8
./lib/sup/message-chunks.rb:36: warning: method redefined; discarding old make_tmpname
./lib/sup/imap.rb:119: warning: ambiguous first argument; put parentheses or even spaces
## Step one: back up all message state to /tmp/sup-state.txt
bin/sup-dump --index ferret > /tmp/sup-state.txt
[Mon Jan 18 11:20:40 +0530 2010] WARNING: can't find character set by using locale, defaulting to utf-8
(completed in 0.8s)

## message state saved to /tmp/sup-state.txt
## Step two: back up sources.yaml file to /home/anirudhs/.sup/sources.yaml-before-xapian-upgrade
cp /home/anirudhs/.sup/sources.yaml /home/anirudhs/.sup/sources.yaml-before-xapian-upgrade
(completed in 0.0s)

## Step three: build the Xapian index
bin/sup-sync --all --all-sources --index xapian --restore /tmp/sup-state.txt 
[Mon Jan 18 11:20:41 +0530 2010] WARNING: can't find character set by using locale, defaulting to utf-8
Loading state dump from /tmp/sup-state.txt...
Read 0 entries from dump file.
/usr/lib/ruby/gems/1.8/gems/lockfile-1.4.3/lib/lockfile.rb:500:in `initialize': No such file or directory - xapian/.deepthought_2282_sup-sync_1263793841_863632_780999264.lck (Errno::ENOENT)
	from /usr/lib/ruby/gems/1.8/gems/lockfile-1.4.3/lib/lockfile.rb:500:in `open'
	from /usr/lib/ruby/gems/1.8/gems/lockfile-1.4.3/lib/lockfile.rb:500:in `create'
	from /usr/lib/ruby/gems/1.8/gems/lockfile-1.4.3/lib/lockfile.rb:432:in `create_tmplock'
	from /usr/lib/ruby/gems/1.8/gems/lockfile-1.4.3/lib/lockfile.rb:213:in `lock'
	from /usr/lib/ruby/gems/1.8/gems/lockfile-1.4.3/lib/lockfile.rb:539:in `attempt'
	from /usr/lib/ruby/gems/1.8/gems/lockfile-1.4.3/lib/lockfile.rb:539:in `catch'
	from /usr/lib/ruby/gems/1.8/gems/lockfile-1.4.3/lib/lockfile.rb:539:in `attempt'
	from /usr/lib/ruby/gems/1.8/gems/lockfile-1.4.3/lib/lockfile.rb:539:in `loop'
	from /usr/lib/ruby/gems/1.8/gems/lockfile-1.4.3/lib/lockfile.rb:539:in `attempt'
	from /usr/lib/ruby/gems/1.8/gems/lockfile-1.4.3/lib/lockfile.rb:210:in `lock'
	from /usr/lib/ruby/gems/1.8/gems/sup-0.9.1/lib/sup/index.rb:38:in `lock'
	from /usr/lib/ruby/gems/1.8/gems/sup-0.9.1/lib/sup/util.rb:520:in `send'
	from /usr/lib/ruby/gems/1.8/gems/sup-0.9.1/lib/sup/util.rb:520:in `method_missing'
	from /usr/lib/ruby/gems/1.8/gems/sup-0.9.1/lib/sup/interactive-lock.rb:26:in `lock_interactively'
	from bin/sup-sync:116
-----------END PASTE--------------

Since Xapian install from gem failed, I installed libxapian-ruby1.8
instead.

[11:29]:~/Desktop/sup-mainline$ apt-cache policy libxapian-ruby1.8 
libxapian-ruby1.8:
  Installed: 1.0.14-1build1
  Candidate: 1.0.14-1build1
  Version table:
 *** 1.0.14-1build1 0
        500 http://archive.ubuntu.com karmic/universe Packages
        100 /var/lib/dpkg/status

Thanks,
Anirudh

---

From: Steve Goldman <[email-masked]>
Date: Sun, 17 Jan 2010 20:44:11 -0500

Excerpts from Steve Goldman's message of Sun Jan 17 19:28:12 -0500 2010:
> Excerpts from William Morgan's message of Sun Jan 17 19:04:46 -0500 2010:
> > > ... until it exited thirty minutes later with seg fault.
> > > 
> > > Anyone else see this?  Any way to debug it?
> > 
> > Is it possible it's running out of memory?
> 
> Possible.  I have 4GB and about 60k messages in my index.  So it's not
> too likely.  I'll run it again and watch the usage.

I ran it in strace.  It hung on a call to munmap, and then crashed
with a lot of a bunch of system call timeout warnings.

So it has something to do with memory.

Would the ruby debugger help narrow it down?  Does the ruby debugger
exist?

Thanks.
-- 

Steve Goldman
[email-masked]

T: 212.219.6014
F: 212.219.6007

Tower Research Capital, LLC
377 Broadway, 11th Fl.
New York, NY 10013
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: William Morgan <[email-masked]>
Date: Tue, 19 Jan 2010 08:37:10 -0500

Reformatted excerpts from Steve Goldman's message of 2010-01-19:
> What are fresh sources?

I just mean a new mbox or maildir, so that Sup can start at offset 0.
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Steve Goldman <[email-masked]>
Date: Sun, 17 Jan 2010 14:50:28 -0500

Here's the message:

[~/sup-src/mainline (master)]$ /usr/bin/ruby -I lib -w bin/sup-convert-ferret-index -v
## Step one: back up all message state to /tmp/sup-state.txt
bin/sup-dump --index ferret > /tmp/sup-state.txt
/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- sup (LoadError)
from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from bin/sup-dump:5

Looks like it's having trouble finding the file called sup.  I tried
running it from the bin directory, but it didn't work there either.

Any ideas?

Thanks.
-- 

Steve Goldman

T: 212.219.6014
F: 212.219.6007

---

From: William Morgan <[email-masked]>
Date: Sun, 17 Jan 2010 15:38:01 -0500

> running it from the bin directory, but it didn't work there either.

Ah, an important note: if you're running from git, you have to set
RUBY_INVOCATION in order to tell sup-con... how to call the ruby
programs.

Try this:
  RUBY_INVOCATION="/usr/bin/ruby -Ilib" /usr/bin/ruby -Ilib bin/sup-convert-ferret-index
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Steve Goldman <[email-masked]>
Date: Mon, 18 Jan 2010 14:43:36 -0500

Excerpts from Rich Lane's message of Sun Jan 17 21:10:17 -0500 2010:
> Excerpts from Steve Goldman's message of 2010-01-17 20:44:11 -0500:
> Try compiling Xapian from source with --enable-assertions. It would also
> be useful to get a core dump (ulimit -c unlimited).

This feels like a wild goose chase.

Can I start a xapian index from scratch?  Any downside to this besides
the time sink?

Thanks.
-- 

Steve Goldman

T: 212.219.6014
F: 212.219.6007

---

From: Steve Goldman <[email-masked]>
Date: Sun, 17 Jan 2010 19:28:12 -0500

Steve Goldman

T: 212.219.6014
F: 212.219.6007

---

From: William Morgan <[email-masked]>
Date: Tue, 26 Jan 2010 08:00:02 -0500

Reformatted excerpts from Steve Goldman's message of 2010-01-25:
> Presumably.  I run it through sup-config, which outputs "Rats, that
> didn't work" when the return code is non-zero.  But sup-sync outputs
> nothing.

And just to confirm, when you run that sup-sync command by itself,
outside of sup-config, it just dies without a backtrace, right?

Since this didn't happen with Ferret, my best guess is that the problem
lies somewhere in the bowels of Xapian or the bindings thereof:

> xapian-bindings-ruby-1.0.17-2.fc11.x86_64

I would try and get the latest xapian-full gem. I have no experience
with the bindings Fedora distributes, but I know that that gem works for
me and for others.

It's possible to install and run gems entirely out of your home
directory (at least in recent versions), so it should be possible to do
this without involving your sysadmin, if that's easier.

Something like:

  gem install gemcutter
  gem install xapian-full

Ought to magically work, if your gems are new enough.

> He says the Fedora release will likely get there soon, but we could
> try to get it in advance if you think this will help.

Let's wait on playing with Ruby versions until we have your Xapian
bindings locked down.
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: William Morgan <[email-masked]>
Date: Sun, 17 Jan 2010 19:04:46 -0500

---

From: Steve Goldman <[email-masked]>
Date: Tue, 26 Jan 2010 22:24:34 -0500

> outside of sup-config, it just dies without a backtrace, right?

Correct.

> Ought to magically work, if your gems are new enough.

Do I need to do something to make ruby use this xapian and not the
Fedora xapian?  How can I tell which it is using?

> Let's wait on playing with Ruby versions until we have your Xapian

I did some sleuthing and found that it hangs on the loop over all the
messages in my first source (imap) after it gets through exactly seven
of them.  Specifically, it hangs at the top of the loop:

  Redwood::PollManager.each_message_from source do |m|

Any ideas?

Thanks.
-- 

Steve Goldman

T: 212.219.6014
F: 212.219.6007

---

From: Steve Goldman <[email-masked]>
Date: Sun, 17 Jan 2010 16:22:03 -0500

>   RUBY_INVOCATION="/usr/bin/ruby -Ilib" /usr/bin/ruby -Ilib bin/sup-convert-ferret-index

Great, that made it run.

[~/.sup/xapian ]$ ll
total 36
-rw-r--r-- 1 sgoldman wheel    0 2010-01-17 16:15 flintlock
-rw-r--r-- 1 sgoldman wheel   12 2010-01-17 16:07 iamflint
-rw-r--r-- 1 sgoldman wheel   13 2010-01-17 16:07 postlist.baseA
-rw-r--r-- 1 sgoldman wheel   14 2010-01-17 16:08 postlist.baseB
-rw-r--r-- 1 sgoldman wheel 8192 2010-01-17 16:08 postlist.DB
-rw-r--r-- 1 sgoldman wheel   13 2010-01-17 16:07 record.baseA
-rw-r--r-- 1 sgoldman wheel   14 2010-01-17 16:08 record.baseB
-rw-r--r-- 1 sgoldman wheel    0 2010-01-17 16:07 record.DB
-rw-r--r-- 1 sgoldman wheel   13 2010-01-17 16:07 termlist.baseA
-rw-r--r-- 1 sgoldman wheel   14 2010-01-17 16:08 termlist.baseB
-rw-r--r-- 1 sgoldman wheel    0 2010-01-17 16:07 termlist.DB

Shouldn't there be more stuff in there?

Thanks.
-- 

Steve Goldman

T: 212.219.6014
F: 212.219.6007

---

From: Anirudh Sanjeev <[email-masked]>
Date: Mon, 18 Jan 2010 22:57:53 +0530

>   RUBY_INVOCATION="ruby -Ilib" ruby -Ilib bin/sup-convert-ferret-index

That worked wonderfully! Thanks!

The search is so fast, oh my!
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: William Morgan <[email-masked]>
Date: Mon, 18 Jan 2010 17:05:10 -0500

> This feels like a wild goose chase.

To be fair, all we know from you at this point is you got a segfault.
We don't even know what Ruby version, what xapian gem, etc.

> Can I start a xapian index from scratch?

If you rename your ~/.sup/ferret directory, any future Sup activity
(e.g. bin/sup or sup-sync) should take place on a fresh new Xapian
index. All of your current messages and message state will be forgotten.

> Any downside to this besides the time sink?

If you ever use sup-sync --all, it will rescan the soucess to pick up
those early messages and presumably incur the same problem. You'll have
to make fresh sources to really avoid that.
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: William Morgan <[email-masked]>
Date: Mon, 25 Jan 2010 16:30:48 -0500

Reformatted excerpts from Steve Goldman's message of 2010-01-25:
> Tried running sup-config from scratch last night.  No dice.  sup-sync
> failed after about ten minutes.
> 
> What info can I supply that would help debug this?

Is it still a segfault?

If so, what does gem list --local say the version is of the "xapian" or
"xapian-full" gem?

> ruby version 1.8.6

How difficult would it be to try with 1.9.1, or at least 1.8.7?
-- 
William <[email-masked]>
_______________________________________________
sup-talk mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-talk

---

From: Rich Lane <[email-masked]>
Date: Sun, 17 Jan 2010 21:10:17 -0500

---

From: Steve Goldman <[email-masked]>
Date: Mon, 25 Jan 2010 11:34:57 -0500

sup on master branch
ruby version 1.8.6
-- 

Steve Goldman

T: 212.219.6014
F: 212.219.6007

---

From: Steve Goldman <[email-masked]>
Date: Mon, 25 Jan 2010 16:56:24 -0500

> If so, what does gem list --local say the version is of the "xapian" or

It's installed with our Fedora release, but my guy tells me this is
the version info:

Thanks.
-- 

Steve Goldman

T: 212.219.6014
F: 212.219.6007

---

From: Steve Goldman <[email-masked]>
Date: Sun, 17 Jan 2010 18:38:45 -0500

> Is this with git master or next?

Ah, yes.  sup-sync exited with error.  I corrected an incorrect
password in my sources.yaml and then it ran...

Thanks.
-- 

Steve Goldman

T: 212.219.6014
F: 212.219.6007

---

From: Steve Goldman <[email-masked]>
Date: Tue, 19 Jan 2010 08:31:23 -0500

> We don't even know what Ruby version, what xapian gem, etc.

Right.  I'd rather start fresh than try to debug this, personally.

What are fresh sources?

Thanks.
-- 

Steve Goldman

T: 212.219.6014
F: 212.219.6007