* [sup-talk] [PATCH] add Message#load_from_index! shortcut
@ 2009-08-22 18:28 Rich Lane
2009-08-22 18:54 ` [sup-talk] [PATCH] add xapian-specific hack to quickly create a Person Rich Lane
2009-08-24 23:23 ` [sup-talk] [PATCH] add Message#load_from_index! shortcut William Morgan
0 siblings, 2 replies; 6+ messages in thread
From: Rich Lane @ 2009-08-22 18:28 UTC (permalink / raw)
Message#parse_header is slow, taking up to 2/3 of the time spent loading
threads in thread-index-mode. This patch adds a shortcut method that the index
can use to efficiently initialize a message.
---
lib/sup/message.rb | 25 +++++++++++++++++++++++++
lib/sup/xapian_index.rb | 30 +++++++++++-------------------
2 files changed, 36 insertions(+), 19 deletions(-)
diff --git a/lib/sup/message.rb b/lib/sup/message.rb
index 965c10e..56e66de 100644
--- a/lib/sup/message.rb
+++ b/lib/sup/message.rb
@@ -127,6 +127,31 @@ class Message
@list_unsubscribe = header["list-unsubscribe"]
end
+ ## Expected index entry format:
+ ## :message_id, :subject => String
+ ## :date => Time
+ ## :refs, :replytos => Array of String
+ ## :from => Person
+ ## :to, :cc, :bcc => Array of Person
+ def load_from_index! entry
+ @id = entry[:message_id]
+ @from = entry[:from]
+ @date = entry[:date]
+ @subj = entry[:subject]
+ @to = entry[:to]
+ @cc = entry[:cc]
+ @bcc = entry[:bcc]
+ @refs = (@refs + entry[:refs]).uniq
+ @replytos = entry[:replytos]
+
+ @replyto = nil
+ @list_address = nil
+ @recipient_email = nil
+ @source_marked_read = false
+ @list_subscribe = nil
+ @list_unsubscribe = nil
+ end
+
def add_ref ref
@refs << ref
@dirty = true
diff --git a/lib/sup/xapian_index.rb b/lib/sup/xapian_index.rb
index 85f6ef0..c260728 100644
--- a/lib/sup/xapian_index.rb
+++ b/lib/sup/xapian_index.rb
@@ -71,25 +71,17 @@ class XapianIndex < BaseIndex
source = SourceManager[entry[:source_id]]
raise "invalid source #{entry[:source_id]}" unless source
- mk_addrs = lambda { |l| l.map { |e,n| "#{n} <#{e}>" } * ', ' }
- mk_refs = lambda { |l| l.map { |r| "<#{r}>" } * ' ' }
- fake_header = {
- 'message-id' => entry[:message_id],
- 'date' => Time.at(entry[:date]),
- 'subject' => entry[:subject],
- 'from' => mk_addrs[[entry[:from]]],
- 'to' => mk_addrs[entry[:to]],
- 'cc' => mk_addrs[entry[:cc]],
- 'bcc' => mk_addrs[entry[:bcc]],
- 'reply-tos' => mk_refs[entry[:replytos]],
- 'references' => mk_refs[entry[:refs]],
- }
-
- m = Message.new :source => source, :source_info => entry[:source_info],
- :labels => entry[:labels],
- :snippet => entry[:snippet]
- m.parse_header fake_header
- m
+ m = Message.new :source => source, :source_info => entry[:source_info],
+ :labels => entry[:labels], :snippet => entry[:snippet]
+
+ mk_person = lambda { |x| Person.new(*x.reverse!) }
+ entry[:from] = mk_person[entry[:from]]
+ entry[:to].map!(&mk_person)
+ entry[:cc].map!(&mk_person)
+ entry[:bcc].map!(&mk_person)
+
+ m.load_from_index! entry
+ m
end
def add_message m; sync_message m end
--
1.6.4
^ permalink raw reply [flat|nested] 6+ messages in thread
* [sup-talk] [PATCH] add xapian-specific hack to quickly create a Person
2009-08-22 18:28 [sup-talk] [PATCH] add Message#load_from_index! shortcut Rich Lane
@ 2009-08-22 18:54 ` Rich Lane
2009-08-24 23:24 ` William Morgan
2009-08-24 23:23 ` [sup-talk] [PATCH] add Message#load_from_index! shortcut William Morgan
1 sibling, 1 reply; 6+ messages in thread
From: Rich Lane @ 2009-08-22 18:54 UTC (permalink / raw)
Another 10% query performance boost.
---
lib/sup/xapian_index.rb | 10 +++++++++-
1 files changed, 9 insertions(+), 1 deletions(-)
diff --git a/lib/sup/xapian_index.rb b/lib/sup/xapian_index.rb
index c260728..3a23951 100644
--- a/lib/sup/xapian_index.rb
+++ b/lib/sup/xapian_index.rb
@@ -74,7 +74,7 @@ class XapianIndex < BaseIndex
m = Message.new :source => source, :source_info => entry[:source_info],
:labels => entry[:labels], :snippet => entry[:snippet]
- mk_person = lambda { |x| Person.new(*x.reverse!) }
+ mk_person = lambda { |x| QuickPerson.new(*x) }
entry[:from] = mk_person[entry[:from]]
entry[:to].map!(&mk_person)
entry[:cc].map!(&mk_person)
@@ -84,6 +84,14 @@ class XapianIndex < BaseIndex
m
end
+ class QuickPerson < Person
+ def initialize email, name
+ raise ArgumentError, "email can't be nil" unless email
+ @email = email
+ @name = name
+ end
+ end
+
def add_message m; sync_message m end
def update_message m; sync_message m end
def update_message_state m; sync_message m end
--
1.6.4
^ permalink raw reply [flat|nested] 6+ messages in thread
* [sup-talk] [PATCH] add Message#load_from_index! shortcut
2009-08-22 18:28 [sup-talk] [PATCH] add Message#load_from_index! shortcut Rich Lane
2009-08-22 18:54 ` [sup-talk] [PATCH] add xapian-specific hack to quickly create a Person Rich Lane
@ 2009-08-24 23:23 ` William Morgan
1 sibling, 0 replies; 6+ messages in thread
From: William Morgan @ 2009-08-24 23:23 UTC (permalink / raw)
Reformatted excerpts from Rich Lane's message of 2009-08-22:
> Message#parse_header is slow, taking up to 2/3 of the time spent
> loading threads in thread-index-mode. This patch adds a shortcut
> method that the index can use to efficiently initialize a message.
Yes, great catch. I've applied this to xapian-updates branch and
remerged into next.
BTW can you base your patches off master instead of next? It will save
me a couple keystrokes.
--
William <wmorgan-sup at masanjin.net>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [sup-talk] [PATCH] add xapian-specific hack to quickly create a Person
2009-08-22 18:54 ` [sup-talk] [PATCH] add xapian-specific hack to quickly create a Person Rich Lane
@ 2009-08-24 23:24 ` William Morgan
2009-08-26 1:50 ` Rich Lane
0 siblings, 1 reply; 6+ messages in thread
From: William Morgan @ 2009-08-24 23:24 UTC (permalink / raw)
Reformatted excerpts from Rich Lane's message of 2009-08-22:
> - mk_person = lambda { |x| Person.new(*x.reverse!) }
> + mk_person = lambda { |x| QuickPerson.new(*x) }
What about lambda { |x| Person.new x[1], x[0] }. Surely that must be
even faster?
--
William <wmorgan-sup at masanjin.net>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [sup-talk] [PATCH] add xapian-specific hack to quickly create a Person
2009-08-24 23:24 ` William Morgan
@ 2009-08-26 1:50 ` Rich Lane
2009-09-01 20:59 ` William Morgan
0 siblings, 1 reply; 6+ messages in thread
From: Rich Lane @ 2009-08-26 1:50 UTC (permalink / raw)
Excerpts from William Morgan's message of Mon Aug 24 19:24:19 -0400 2009:
> Reformatted excerpts from Rich Lane's message of 2009-08-22:
> > - mk_person = lambda { |x| Person.new(*x.reverse!) }
> > + mk_person = lambda { |x| QuickPerson.new(*x) }
>
> What about lambda { |x| Person.new x[1], x[0] }. Surely that must be
> even faster?
The slow part is the processing in Person#initialize, which QuickPerson
overrides. You might also be able to avoid that by moving the
initialize() code into Person.from_address.
^ permalink raw reply [flat|nested] 6+ messages in thread
* [sup-talk] [PATCH] add xapian-specific hack to quickly create a Person
2009-08-26 1:50 ` Rich Lane
@ 2009-09-01 20:59 ` William Morgan
0 siblings, 0 replies; 6+ messages in thread
From: William Morgan @ 2009-09-01 20:59 UTC (permalink / raw)
Reformatted excerpts from Rich Lane's message of 2009-08-25:
> The slow part is the processing in Person#initialize, which
> QuickPerson overrides. You might also be able to avoid that by moving
> the initialize() code into Person.from_address.
Yeah, I think that's a better approach. There's no need for the
constructor to do all that work. I'll work on this when I get a chance,
or feel free to beat me to it.
--
William <wmorgan-sup at masanjin.net>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2009-09-01 20:59 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-08-22 18:28 [sup-talk] [PATCH] add Message#load_from_index! shortcut Rich Lane
2009-08-22 18:54 ` [sup-talk] [PATCH] add xapian-specific hack to quickly create a Person Rich Lane
2009-08-24 23:24 ` William Morgan
2009-08-26 1:50 ` Rich Lane
2009-09-01 20:59 ` William Morgan
2009-08-24 23:23 ` [sup-talk] [PATCH] add Message#load_from_index! shortcut William Morgan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox