[sup-devel] [PATCH] Saved Search Support

* [sup-devel] [PATCH] Saved Search Support
@ 2010-01-18 23:44 Eric Sherman
  2010-01-19 18:00 ` Rich Lane
  0 siblings, 1 reply; 8+ messages in thread
From: Eric Sherman @ 2010-01-18 23:44 UTC (permalink / raw)
  To: sup-devel

Start an index search with a \ backslash and press enter to get a list
of searches that were previously saved from search-results-mode with %
percent or added from search-list-mode directly.  Saved searches may be
used in other searches by enclosing their names in {} curly braces.
Search names may contain letters, numbers, underscores and dashes.

New Key Bindings
global
  \<CR> open search-list-mode
search-list-mode
  X     Delete selected search
  r     Rename selected search
  e     Edit selected search
  a     Add new search
search-results-mode
  %     Save search

New Hooks
search-list-filter
search-list-format

Search String Expansion
Include saved searches in other searches by enclosing their names in {}
curly braces.  The name and enclosing braces are replaced by the actual
search string.  When expanded, they are enclosed within parens to
preserve logic.

    low_traffic: has:foo OR has:bar
    a_slow_week: {low_traffic} AND after:(7 days ago)

{a_slow_week} expands to "(has:foo OR has:bar) AND after:(7 days ago)"
and may be used in a global search, a refinement or another saved
search.  If a string enclosed in {} curly braces does not match a saved
search name it is ignored.

There is no nesting limit and searches are always expanded completely
before they are turned into proper queries for the index.

Shrinking Search Strings
Search strings are kept as short as possible when displayed or saved.
So a search string of "has:foo OR has:bar OR has:baz" added with the
above searches would be saved as "{low_traffic} OR has:baz".  This may
or may not always be desirable, but it generally makes things easier.

Editing, Renaming, Deleting
Editing a search string that has been included in other searches will
have no effect on those other searches' search strings, but they will
expand with its new contents.

Renaming a search that has been included in other searches will cause
each occurrence in those other searches to be renamed as well.

Deleting a search that has been included in other searches will cause it
to expand into those other searches to prevent breaking them.

Save File Format
Searches are read from ~/.sup/searches.txt on startup and saved at exit.
The format is "name: search_string".  Here's a silly example:

    core: {me} AND NOT {crap} AND NOT {weak}
    crap: is:leadlogger OR is:alert OR is:rzp
    me: to:me OR from:me
    recently: after:(14 days ago)
    top: {core} AND {recently}
    weak: is:feed OR is:list OR is:ham

FLAG_PURE_NOT
I also added FLAG_PURE_NOT to the xapian parse_query requests to allow a
query in the form of "NOT <expression>", which is of questionable
usefulness but at least sup won't bomb when this happens.
---
 bin/sup                              |   11 ++-
 lib/sup.rb                           |    5 +
 lib/sup/modes/search-list-mode.rb    |  164 ++++++++++++++++++++++++++++++++++
 lib/sup/modes/search-results-mode.rb |   14 +++-
 lib/sup/search.rb                    |   87 ++++++++++++++++++
 lib/sup/xapian_index.rb              |    2 +-
 6 files changed, 276 insertions(+), 7 deletions(-)
 create mode 100644 lib/sup/modes/search-list-mode.rb
 create mode 100644 lib/sup/search.rb

diff --git a/bin/sup b/bin/sup
index 19b2a87..b865b6d 100755
--- a/bin/sup
+++ b/bin/sup
@@ -317,9 +317,14 @@ begin
       b, new = bm.spawn_unless_exists("Contact List") { ContactListMode.new }
       b.mode.load_in_background if new
     when :search
-      query = BufferManager.ask :search, "search all messages: "
-      next unless query && query !~ /^\s*$/
-      SearchResultsMode.spawn_from_query query
+      query = BufferManager.ask :search, "Search all messages (enter for saved searches): "
+      unless query.nil?
+        if query.empty?
+          bm.spawn_unless_exists("Saved searches") { SearchListMode.new }
+        else
+          SearchResultsMode.spawn_from_query query
+        end
+      end
     when :search_unread
       SearchResultsMode.spawn_from_query "is:unread"
     when :list_labels
diff --git a/lib/sup.rb b/lib/sup.rb
index b83bbe7..b8a0977 100644
--- a/lib/sup.rb
+++ b/lib/sup.rb
@@ -50,6 +50,7 @@ module Redwood
   LOCK_FN    = File.join(BASE_DIR, "lock")
   SUICIDE_FN = File.join(BASE_DIR, "please-kill-yourself")
   HOOK_DIR   = File.join(BASE_DIR, "hooks")
+  SEARCH_FN  = File.join(BASE_DIR, "searches.txt")
 
   YAML_DOMAIN = "masanjin.net"
   YAML_DATE = "2006-10-01"
@@ -131,12 +132,14 @@ module Redwood
     Redwood::CryptoManager.init
     Redwood::UndoManager.init
     Redwood::SourceManager.init
+    Redwood::SearchManager.init Redwood::SEARCH_FN
   end
 
   def finish
     Redwood::LabelManager.save if Redwood::LabelManager.instantiated?
     Redwood::ContactManager.save if Redwood::ContactManager.instantiated?
     Redwood::BufferManager.deinstantiate! if Redwood::BufferManager.instantiated?
+    Redwood::SearchManager.save if Redwood::SearchManager.instantiated?
   end
 
   ## not really a good place for this, so I'll just dump it here.
@@ -311,6 +314,8 @@ require "sup/modes/file-browser-mode"
 require "sup/modes/completion-mode"
 require "sup/modes/console-mode"
 require "sup/sent"
+require "sup/search"
+require "sup/modes/search-list-mode"
 
 $:.each do |base|
   d = File.join base, "sup/share/modes/"
diff --git a/lib/sup/modes/search-list-mode.rb b/lib/sup/modes/search-list-mode.rb
new file mode 100644
index 0000000..09d081c
--- /dev/null
+++ b/lib/sup/modes/search-list-mode.rb
@@ -0,0 +1,164 @@
+module Redwood
+
+class SearchListMode < LineCursorMode
+  register_keymap do |k|
+    k.add :select_search, "Open search results", :enter
+    k.add :reload, "Discard saved search list and reload", '@'
+    k.add :jump_to_next_new, "Jump to next new thread", :tab
+    k.add :toggle_show_unread_only, "Toggle between showing all saved searches and those with unread mail", 'u'
+    k.add :delete_selected_search, "Delete selected search", "X"
+    k.add :rename_selected_search, "Rename selected search", "r"
+    k.add :edit_selected_search, "Edit selected search", "e"
+    k.add :add_new_search, "Add new search", "a"
+  end
+
+  HookManager.register "search-list-filter", <<EOS
+Filter the search list, typically to sort.
+Variables:
+  counted: an array of counted searches.
+Return value:
+  An array of counted searches with sort_by output structure.
+EOS
+
+  HookManager.register "search-list-format", <<EOS
+Create the sprintf format string for search-list-mode.
+Variables:
+  n_width: the maximum search name width
+  tmax: the maximum total message count
+  umax: the maximum unread message count
+  s_width: the maximum search string width
+Return value:
+  A format string for sprintf
+EOS
+
+  def initialize
+    @searches = []
+    @text = []
+    @unread_only = false
+    super
+    UpdateManager.register self
+    regen_text
+  end
+
+  def cleanup
+    UpdateManager.unregister self
+    super
+  end
+
+  def lines; @text.length end
+  def [] i; @text[i] end
+
+  def jump_to_next_new
+    n = ((curpos + 1) ... lines).find { |i| @searches[i][1] > 0 } || (0 ... curpos).find { |i| @searches[i][1] > 0 }
+    if n
+      ## jump there if necessary
+      jump_to_line n unless n >= topline && n < botline
+      set_cursor_pos n
+    else
+      BufferManager.flash "No saved searches with unread messages."
+    end
+  end
+
+  def focus
+    reload # make sure unread message counts are up-to-date
+  end
+
+  def handle_added_update sender, m
+    reload
+  end
+
+protected
+
+  def toggle_show_unread_only
+    @unread_only = !@unread_only
+    reload
+  end
+
+  def reload
+    regen_text
+    buffer.mark_dirty if buffer
+  end
+
+  def regen_text
+    @text = []
+    searches = SearchManager.all_searches
+
+    counted = searches.map do |name|
+      search_string = SearchManager.search_string_for name
+      query = Index.parse_query(SearchManager.expand(search_string))
+      total = Index.num_results_for :qobj => query[:qobj]
+      unread = Index.num_results_for :qobj => query[:qobj], :label => :unread
+      [name, search_string, total, unread]
+    end
+
+    if HookManager.enabled? "search-list-filter"
+      counts = HookManager.run "search-list-filter", :counted => counted
+    else
+      counts = counted.sort_by { |n, s, t, u| n.downcase }
+    end
+
+    n_width = counts.max_of { |n, s, t, u| n.length }
+    tmax    = counts.max_of { |n, s, t, u| t }
+    umax    = counts.max_of { |n, s, t, u| u }
+    s_width = counts.max_of { |n, s, t, u| s.length }
+
+    if @unread_only
+      counts.delete_if { | n, s, t, u | u == 0 }
+    end
+
+    @searches = []
+    counts.map do |name, search_string, total, unread|
+      fmt = HookManager.run "search-list-format", :n_width => n_width, :tmax => tmax, :umax => umax, :s_width => s_width
+      if !fmt
+        fmt = "%#{n_width + 1}s %5d %s, %5d unread: %s"
+      end
+      @text << [[(unread == 0 ? :labellist_old_color : :labellist_new_color),
+          sprintf(fmt, name, total, total == 1 ? " message" : "messages", unread, search_string)]]
+      @searches << [name, unread]
+    end.compact
+
+    BufferManager.flash "No saved searches with unread messages!" if counts.empty? && @unread_only
+  end
+
+  def select_search
+    name, num_unread = @searches[curpos]
+    return unless name
+    SearchResultsMode.spawn_from_query SearchManager.search_string_for(name)
+  end
+
+  def delete_selected_search
+    name, num_unread = @searches[curpos]
+    return unless name
+    reload if SearchManager.delete name
+  end
+
+  def rename_selected_search
+    old_name, num_unread = @searches[curpos]
+    return unless old_name
+    new_name = BufferManager.ask :save_search, "Rename this saved search: ", old_name
+    return unless new_name && new_name !~ /^\s*$/ && new_name != old_name
+    reload if SearchManager.rename old_name, new_name
+    set_cursor_pos @searches.index([new_name, num_unread])||curpos
+  end
+
+  def edit_selected_search
+    name, num_unread = @searches[curpos]
+    return unless name
+    old_search_string = SearchManager.search_string_for name
+    new_search_string = BufferManager.ask :search, "Edit this saved search: ", (old_search_string + " ")
+    return unless new_search_string && new_search_string !~ /^\s*$/ && new_search_string != old_search_string
+    reload if SearchManager.edit name, new_search_string
+    set_cursor_pos @searches.index([name, num_unread])||curpos
+  end
+
+  def add_new_search
+    search_string = BufferManager.ask :search, "New search: "
+    return unless search_string && search_string !~ /^\s*$/
+    name = BufferManager.ask :save_search, "Name for new search: "
+    return unless name && name !~ /^\s*$/ && !(SearchManager.all_searches.include? name)
+    reload if SearchManager.add name, search_string
+    set_cursor_pos @searches.index(@searches.assoc(name))||curpos
+  end
+end
+
+end
diff --git a/lib/sup/modes/search-results-mode.rb b/lib/sup/modes/search-results-mode.rb
index 121e817..2237295 100644
--- a/lib/sup/modes/search-results-mode.rb
+++ b/lib/sup/modes/search-results-mode.rb
@@ -8,14 +8,21 @@ class SearchResultsMode < ThreadIndexMode
 
   register_keymap do |k|
     k.add :refine_search, "Refine search", '|'
+    k.add :save_search, "Save search", '%'
   end
 
   def refine_search
-    text = BufferManager.ask :search, "refine query: ", (@query[:text] + " ")
+    text = BufferManager.ask :search, "refine query: ", (SearchManager.shrink(@query[:text]) + " ")
     return unless text && text !~ /^\s*$/
     SearchResultsMode.spawn_from_query text
   end
 
+  def save_search
+    name = BufferManager.ask :save_search, "Name this search: "
+    return unless name && name !~ /^\s*$/
+    BufferManager.flash "Saved search." if SearchManager.add name, @query[:text]
+  end
+
   ## a proper is_relevant? method requires some way of asking ferret
   ## if an in-memory object satisfies a query. i'm not sure how to do
   ## that yet. in the worst case i can make an in-memory index, add
@@ -24,9 +31,10 @@ class SearchResultsMode < ThreadIndexMode
 
   def self.spawn_from_query text
     begin
-      query = Index.parse_query(text)
+      query = Index.parse_query(SearchManager.expand(text))
       return unless query
-      short_text = text.length < 20 ? text : text[0 ... 20] + "..."
+      shrunk = SearchManager.shrink text
+      short_text = shrunk.length < 20 ? shrunk : shrunk[0 ... 20] + "..."
       mode = SearchResultsMode.new query
       BufferManager.spawn "search: \"#{short_text}\"", mode
       mode.load_threads :num => mode.buffer.content_height
diff --git a/lib/sup/search.rb b/lib/sup/search.rb
new file mode 100644
index 0000000..e2a1a35
--- /dev/null
+++ b/lib/sup/search.rb
@@ -0,0 +1,87 @@
+module Redwood
+
+class SearchManager
+  include Singleton
+
+  def initialize fn
+    @fn = fn
+    @searches = {}
+    if File.exists? fn
+      IO.foreach(fn) do |l|
+        l =~ /^([^:]*): (.*)$/ or raise "can't parse #{fn} line #{l.inspect}"
+        @searches[$1] = $2
+      end
+    end
+    @modified = false
+    @expanded = {}
+    expand_all
+  end
+
+  def all_searches; return @searches.keys.sort; end
+  def search_string_for name; return @searches[name]; end
+
+  def add name, search_string
+    name.strip!
+    search_string.strip!
+    unless name !~ /^[\w-]+$/ or @searches[name] == search_string
+      @searches[name] = shrink search_string
+      expand_all
+      shrink_all
+      @modified = true
+    end
+  end
+
+  def rename old, new
+    new.strip!
+    return unless new =~ /^[\w-]+$/ and @searches.has_key? old
+    search_string = @searches[old]
+    delete old
+    add new, search_string
+  end
+
+  def edit name, search_string
+    return unless @searches.has_key? name
+    ## we want to delete the old one, but not expand it into searches when doing so
+    @expanded.delete name
+    @searches.delete name
+    add name, search_string
+  end
+
+  def delete name
+    return unless @searches.has_key? name
+    expand_into_searches name
+    @expanded.delete name
+    @searches.delete name
+    @modified = true
+  end
+
+  def expand search_string
+    s = search_string.dup
+    ## stop trying to expand if there are no expansion candidates left, if the none of the remaining candidates represent a search name, or if the string has grown abnormally large due to what would have been infinite recursion
+    until (m = /\{([\w-]+)\}/.match(s)).nil? or m.captures.collect { |n| @expanded.keys.index n }.compact.size == 0 or s.size >= 2048
+      m.captures.each { |n| s.gsub! "{#{n}}", "(#{@expanded[n]})" if @expanded.has_key? n }
+    end
+    return s
+  end
+
+  def shrink search_string
+    s = search_string.dup
+    @expanded.each { |k, v| s.gsub! /\(?#{Regexp.escape v}\)?/, "{#{k}}" unless k == @expanded.index(s) }
+    @searches.each { |k, v| s.gsub! /\(?#{Regexp.escape v}\)?/, "{#{k}}" unless k == @searches.index(s) }
+    return s
+  end
+
+  def save
+    return unless @modified
+    File.open(@fn, "w") { |f| @searches.sort.each { |(n, s)| f.puts "#{n}: #{s}" } }
+    @modified = false
+  end
+
+private
+
+  def expand_into_searches name; @searches.values.each { |v| v.gsub! "{#{name}}", "(#{@searches[name]})" }; end
+  def expand_all; @expanded.replace(@searches).each { |k, v| @expanded[k] = expand v }; end
+  def shrink_all; @searches.each { |k, v| @searches[k] = shrink v }; end
+end
+
+end
diff --git a/lib/sup/xapian_index.rb b/lib/sup/xapian_index.rb
index 0db5010..1bbde5d 100644
--- a/lib/sup/xapian_index.rb
+++ b/lib/sup/xapian_index.rb
@@ -263,7 +263,7 @@ EOS
     qp.add_valuerangeprocessor(Xapian::NumberValueRangeProcessor.new(DATE_VALUENO, 'date:', true))
     NORMAL_PREFIX.each { |k,v| qp.add_prefix k, v }
     BOOLEAN_PREFIX.each { |k,v| qp.add_boolean_prefix k, v }
-    xapian_query = qp.parse_query(subs, Xapian::QueryParser::FLAG_PHRASE|Xapian::QueryParser::FLAG_BOOLEAN|Xapian::QueryParser::FLAG_LOVEHATE|Xapian::QueryParser::FLAG_WILDCARD, PREFIX['body'])
+    xapian_query = qp.parse_query(subs, Xapian::QueryParser::FLAG_PHRASE|Xapian::QueryParser::FLAG_BOOLEAN|Xapian::QueryParser::FLAG_LOVEHATE|Xapian::QueryParser::FLAG_WILDCARD|Xapian::QueryParser::FLAG_PURE_NOT, PREFIX['body'])
 
     debug "parsed xapian query: #{xapian_query.description}"
 
-- 
1.6.6

_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


^ permalink raw reply	[flat|nested] 8+ messages in thread