From mboxrd@z Thu Jan 1 00:00:00 1970 Received: by 10.142.217.10 with SMTP id p10csp87329wfg; Mon, 26 Mar 2012 16:41:09 -0700 (PDT) Received: by 10.60.14.4 with SMTP id l4mr29784549oec.39.1332805269430; Mon, 26 Mar 2012 16:41:09 -0700 (PDT) Return-Path: Received: from rubyforge.org (50-56-192-79.static.cloud-ips.com. [50.56.192.79]) by mx.google.com with ESMTP id k3si10426481obl.101.2012.03.26.16.41.09; Mon, 26 Mar 2012 16:41:09 -0700 (PDT) Received-SPF: pass (google.com: domain of sup-devel-bounces@rubyforge.org designates 50.56.192.79 as permitted sender) client-ip=50.56.192.79; Authentication-Results: mx.google.com; spf=pass (google.com: domain of sup-devel-bounces@rubyforge.org designates 50.56.192.79 as permitted sender) smtp.mail=sup-devel-bounces@rubyforge.org; dkim=neutral (body hash did not verify) header.i=@gmail.com Received: from localhost.localdomain (localhost [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id AD070263040; Mon, 26 Mar 2012 23:41:08 +0000 (UTC) Received: from mail-we0-f178.google.com (mail-we0-f178.google.com [74.125.82.178]) by rubyforge.org (Postfix) with ESMTP id AE39426302B; Mon, 26 Mar 2012 23:40:53 +0000 (UTC) Received: by wera13 with SMTP id a13so5542308wer.23 for ; Mon, 26 Mar 2012 16:40:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=qQ12Yyw7UoKc6NIG5u1njwzbR/RBMPkbDmgdJnCVM2o=; b=P+Y+1LgepW4WV9TYs8QUFL65jgHlpBcO6akuP5ZrlN6hWBnIWCWS/Tu1htaVYHjxRk TBnsJtZkWmzmrpSthviBUoXQ4/ZFvS9O2i271bgXbALKIPU1jDw49QWRrXQkD1QBx+ab ANbwoAvVGtvSPhy+0J18BC/ooxmyFoPspj1DSfYqIsxe3QGFL6/kJTRP8yp1kQA3VmLJ GU/0ODDQ6Mgdn4Rtjn7JSoqZt0kqoA1QDSQIFb2kI7LB7LfXs9XzX3pF7xKNrV9NlD5Y tQw238H1Tdvu69RlVy0npT42+lNRzNiWYImgne8UJHNu9OgfaVPqEzMwaytni5z0tRJE YEwA== Received: by 10.180.8.231 with SMTP id u7mr20981607wia.9.1332803906486; Mon, 26 Mar 2012 16:18:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.96.70 with HTTP; Mon, 26 Mar 2012 16:18:06 -0700 (PDT) From: Matthieu Rakotojaona Date: Tue, 27 Mar 2012 01:18:06 +0200 Message-ID: To: sup-devel , sup-talk Subject: [sup-devel] Filtering rules X-BeenThere: sup-devel@rubyforge.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Sup developer discussion List-Id: Sup developer discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: sup-devel-bounces@rubyforge.org Errors-To: sup-devel-bounces@rubyforge.org Hello everyone, I have been working on a filtering rules system for heliotrope. Basically, I wanted to have something that can apply some rules to my messages, so that I could rearrange my mails' labels/state automatically at any given time, much like what notmuch does (speaking of which : it's sad that common goals in building an efficient mail system are so much separated by divergences in code). I preferred to use an "after-add" strategy, as opposed to a "before-add", for multiple reasons : - In a "after-add", messages are already indexed; a search for a matching criteria is easily done with mechanisms from whistlepig (which was built for the job). In a "before-add", you would have to search for your criterias manually - If you put bad labels/state before adding the message to the store, you have to search for it manually and apply the good ones - In case of a bulk import, a bulk labelling is practical. So I worked on a possible template for rules that is very simple. It's pure yaml, and looks like this : - to:"sup" subject:"Filtering rules" : labels: "+sup" - to:"me@domain.com" body:"Enlarge your" : state: "+spam" - from:"facebook" : state: "-unread" Again, it's very simple, but I believe it is quite readable and can do most of the things one would want. You can have find more (documented) examples at the previous link. So I created a filter.rb script, in the heliotrope folder, on my public repo (https://github.com/rakoo/heliotrope/tree/filter). Here's how you use it : ruby filter.rb --dir /path/to/heliotrope/data/dir --check /path/to/rules/files.yml And it will print you what are the offending threads. I didn't put the changes back to the store yet for testing purposes (it's just a puts of the non-matching threads), but it's just a :dry_run opt away. Plus, state are to be considered differently than labels: they are per-message, not per-threads. Bonus : I guess most of us use Gmail and Gmail's filters. The same script can take your gmail filters and output a valid yaml rules file for you! You just have to export your rules by hand (it's a Lab feature) to a local file, and run : ruby filter.rb --dir /path/to/heliotrope/data/dir --import /path/to/gmail/filters/file.xml I have to work around a few quirks, but you can post-edit the generated file easily to get a usable rules file. There's a way to get your rules automagically from Google servers, but it is reserved to Google Apps users. If you're a simple user, you must use the export function. I'm working on the other way around (heliotrope -> Gmail), but Gmail's import/export rules are XML-the-verbose. Please enjoy this ! Totally unrelated : I happen to have some negative thread_ids. Is it an expected behaviour ? -- Matthieu RAKOTOJAONA _______________________________________________ Sup-devel mailing list Sup-devel@rubyforge.org http://rubyforge.org/mailman/listinfo/sup-devel