Subject: Re: [sup-devel] Invalid meta data error in OklahomaMixer

From: Horacio Sanson <[email-masked]>
Date: Tue, 14 Jun 2011 11:37:14 +0900

Not even close... my store.tch is 76MB only. This metadata corruption
in Tokyo Cabinet seems to be a common occurrence and what is scary
about this is that there seems to be no way to recover from it (as far
as google can tell). The only option I see is delete my store/index
and restart sync again.

regards,
Horacio

On Tue, Jun 14, 2011 at 11:14 AM, William Morgan
<[email-masked]> wrote:
> Whoops, I mean this:
>
> http://www.supermind.org/blog/554/2gb-limit-with-tokyo-cabinet-aka-invalid-meta-data
> --
> William <[email-masked]>
> _______________________________________________
> Sup-devel mailing list
> [email-masked]
> http://rubyforge.org/mailman/listinfo/sup-devel
>
_______________________________________________
Sup-devel mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-devel

---

From: William Morgan <[email-masked]>
Date: Tue, 14 Jun 2011 03:30:58 +0000

> as google can tell). The only option I see is delete my store/index

Can you point me to some links about this type of corruption?

I'm far from wedded to Oklahoma Mixer / Tokyo Cabinet. It was just the
easiest thing to get started with. Any kind of mutable persistent
key-value store will do. Other alternatives to consider: another
TokyoCabinet gem besides OH, KyotoCabinet, BDB... I kinda don't want to
move to separate processes like Redis though. If only there were a
libredis (well there is, but it's not what you want.)
-- 
William <[email-masked]>
_______________________________________________
Sup-devel mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-devel

---

From: William Morgan <[email-masked]>
Date: Tue, 14 Jun 2011 02:14:05 +0000

Reformatted excerpts from Horacio Sanson's message of 2011-06-11:
> ruby1.9.1  -Ilib bin/heliotrope-server -d ~/.heliotrope
> /var/lib/gems/1.9.1/gems/oklahoma_mixer-0.4.0/lib/oklahoma_mixer/hash_database.rb:337:in
> `try': invalid meta data (error code 5)
> (OklahomaMixer::Error::CabinetError)

Very disturbing. I found this:

http://www.supermind.org/blog/557/tokyocabinet-hdb-slowdown

Did you hit 2GB on your store.tch file, by any chance?
-- 
William <[email-masked]>
_______________________________________________
Sup-devel mailing list
[email-masked]
http://rubyforge.org/mailman/listinfo/sup-devel

---

From: William Morgan <[email-masked]>
Date: Tue, 14 Jun 2011 02:14:32 +0000

Whoops, I mean this:

---

From: Horacio Sanson <[email-masked]>
Date: Tue, 14 Jun 2011 13:21:56 +0900

I managed to get the store.tch working again by installing
tokyocabinet-bin and running this command on the file:

  tchmgr optimize -nl store.tch   # without -nl nothing works...

at the end of this command I got a write error but the file got back
to a usable state. I am afraid this command simply truncated the file
causing some missing emails but not really sure. Is there a way to
check if the index and the messages are in sync? That is all indexed
messages have a corresponding email message in the store.tch?

Here is a good blog post with tips for using Tokyo Cabinet. It
recommends enabling logging and running optimize from time to time.
Logging would allow us to see what caused the metadata corruption:

  http://www.bigdbahead.com/?p=700

Some posts of people with the same invalid metadata problem:

  http://twitter.com/#!/ono_matope/statuses/17650765308
  http://groups.google.com/group/tokyocabinet-users/browse_thread/thread/f114f078848e20ed

Tokyo Cabinet is pretty good and is actually used for the largest
social network in Japan
(http://alpha.mixi.co.jp/blog/?s=tchdb&paged=2) so I would not change
it. Even big databases (e.g. MySQL, Postgres and Oracle) are not free
of table corruption. The difference is that they provide tools to
detect and recover from such corruption state.

regards,
Horacio

---

From: Horacio Sanson <[email-masked]>
Date: Sun, 12 Jun 2011 00:30:24 +0900

After finishing sync of my gmail account to heliotrope I tried to
start the server but was received with this error message:

ruby1.9.1  -Ilib bin/heliotrope-server -d ~/.heliotrope
/var/lib/gems/1.9.1/gems/oklahoma_mixer-0.4.0/lib/oklahoma_mixer/hash_database.rb:337:in
`try': invalid meta data (error code 5)
(OklahomaMixer::Error::CabinetError)
        from /var/lib/gems/1.9.1/gems/oklahoma_mixer-0.4.0/lib/oklahoma_mixer/hash_database.rb:44:in
`initialize'
        from /var/lib/gems/1.9.1/gems/oklahoma_mixer-0.4.0/lib/oklahoma_mixer.rb:17:in
`new'
        from /var/lib/gems/1.9.1/gems/oklahoma_mixer-0.4.0/lib/oklahoma_mixer.rb:17:in
`open'
        from /media/DATA/Apps/heliotrope/lib/heliotrope/index.rb:44:in
`initialize'
        from bin/heliotrope-server:576:in `new'
        from bin/heliotrope-server:576:in `<main>'

seems something got corrupted and I have no idea how to recover. Any
tips that would fix this without resync my whole account would be
greatly appreciated.

Horacio