community/pipermail-archives/sup-devel/2011-11.txt (19135B) - raw
1 From matthieu.rakotojaona@gmail.com Wed Nov 2 19:55:40 2011
2 From: matthieu.rakotojaona@gmail.com (Matthieu Rakotojaona)
3 Date: Thu, 3 Nov 2011 00:55:40 +0100
4 Subject: [sup-devel] heliotrope-syncback
5 Message-ID: <CAMiZLn1tes53sJAdjzDmv+ntUvNNafPCChtOR9Uq7cnKaeOoQA@mail.gmail.com>
6
7 Hello everyone,
8
9 I am playing around with heliotrope, and I've managed to writo a ruby
10 script to sync changes you made there in your IMAP mailbox. It is
11 primarily intended to be used with GMail, but nothing prevents it from
12 working with others.
13
14 Basically, this is how it works :
15 - when you make a change in heliotrope, it is written in an external file
16 - the script reads the file line by line and applies back any changes
17
18 At the moment, I've only logged messages and state changes. When you
19 send a mail with GMail, it is automatically added in GMail "Sent"
20 mail, so I thought heliotrope could automagically thread it, but I
21 haven't tested yet.
22
23 The changes in thu imap folder happen like this :
24 - search for a thread with the "subject" we are interested in
25 - label/unlabel each message in the thread
26 - change state of each message as written in the log file
27
28 Many problems :
29 - first and foremost, the link between a thread in heliotrope and the
30 same in GMail is done through a basic IMAP search on the entire
31 "subject" string. I don't know about how the threading happens in
32 heliotrope, but I hope both are the same
33 - the ruby Net::IMAP library doesn't want to send non-ascii characters
34 ( or I'm too dumb to see how to). So I decided to do it the hard way :
35 remove any word (blocks of chars separated by spaces) that contains
36 any non-ascii characters. Well, I,ve just removed one character for
37 the moment, because I had an error when trying to match a UTF-8 string
38 with a Unicode regexp.
39 - "states" should be message-centered, as explained in the comments,
40 yet I don't know how to link messages in heliotrope and messages in
41 GMail. And I don't think searching through the whole body of the
42 messages in GMail would be very fast. So, I changed the state of all
43 the messages in the thread. After all, when I have an "Unseen" message
44 in a thread, seeing the whole thread looks interesting; same for
45 "Starred".
46 - I've heard the ruby IMAP lib is buggy; what is it exactly ?
47
48 All in all, the most important part of the process is just the writing
49 of the logfile. (One of) My main problem with switching to heliotrope
50 entirely is the fact that it doesn't sync with anything else. So now,
51 if I have a log of everything I've done, I can at least do the same
52 changes by hand. And the script is to be run apart from everything
53 else, because I don't think it's part of heliotrope job to sync with
54 other mailserver. In fact, this script is more of a "fiddle around to
55 learn ruby" script, so please don't care about the code quality =]
56
57 Cya,
58
59 --
60 Matthieu RAKOTOJAONA
61 -------------- next part --------------
62 A non-text attachment was scrubbed...
63 Name: patch
64 Type: application/octet-stream
65 Size: 8853 bytes
66 Desc: not available
67 URL: <http://rubyforge.org/pipermail/sup-devel/attachments/20111103/92c72780/attachment-0001.obj>
68
69 From wmorgan-sup@masanjin.net Mon Nov 7 01:57:19 2011
70 From: wmorgan-sup@masanjin.net (William Morgan)
71 Date: Mon, 07 Nov 2011 06:57:19 +0000
72 Subject: [sup-devel] heliotrope-syncback
73 In-Reply-To: <CAMiZLn1tes53sJAdjzDmv+ntUvNNafPCChtOR9Uq7cnKaeOoQA@mail.gmail.com>
74 References: <CAMiZLn1tes53sJAdjzDmv+ntUvNNafPCChtOR9Uq7cnKaeOoQA@mail.gmail.com>
75 Message-ID: <1320648596-sup-2551@masanjin.net>
76
77 Hi Matthieu,
78
79 Reformatted excerpts from Matthieu Rakotojaona's message of 2011-11-02:
80 > I am playing around with heliotrope, and I've managed to writo a ruby
81 > script to sync changes you made there in your IMAP mailbox.
82
83 Cool!
84
85 > When you
86 > send a mail with GMail, it is automatically added in GMail "Sent"
87 > mail, so I thought heliotrope could automagically thread it, but I
88 > haven't tested yet.
89
90 That should work.
91
92 > The changes in thu imap folder happen like this :
93 > - search for a thread with the "subject" we are interested in
94
95 This is problematic. Message subjects are definitely not unique. You
96 should match up the Message-Id fields instead, if that's at all possible
97 with IMAP.
98
99 If it's not possible, well, IMAP does have a notion of a per-message
100 ids, but it's fraught with difficulties...
101
102 > I don't know about how the threading happens in heliotrope, but I hope
103 > both are the same
104
105 They will mostly be the same, but it's not guaranteed and you will see
106 some differences.
107
108 > - I've heard the ruby IMAP lib is buggy; what is it exactly ?
109
110 In my experience, I haven't found any large bugs, just hard-to-debug
111 issues in the error-handling, and general slowness (though that is
112 probably attributable to the IMAP protocol itself).
113
114 Keep up the good work!
115 --
116 William <wmorgan-sup at masanjin.net>
117
118 From michael+sup@stapelberg.de Mon Nov 7 13:56:10 2011
119 From: michael+sup@stapelberg.de (Michael Stapelberg)
120 Date: Mon, 07 Nov 2011 18:56:10 +0000
121 Subject: [sup-devel] heliotrope-syncback
122 In-Reply-To: <1320648596-sup-2551@masanjin.net>
123 References: <CAMiZLn1tes53sJAdjzDmv+ntUvNNafPCChtOR9Uq7cnKaeOoQA@mail.gmail.com>
124 <1320648596-sup-2551@masanjin.net>
125 Message-ID: <1320692090-sup-9935@stapelberg.de>
126
127 Hi,
128
129 Excerpts from William Morgan's message of 2011-11-07 06:57:19 +0000:
130 > > - I've heard the ruby IMAP lib is buggy; what is it exactly ?
131 > In my experience, I haven't found any large bugs, just hard-to-debug
132 > issues in the error-handling, and general slowness (though that is
133 > probably attributable to the IMAP protocol itself).
134 When I switched from ruby IMAP to offlineimap, a few "new" messages popped up,
135 which were not in the index before. So my suspicion is that the IMAP backend
136 has been eating messages. Since I was switching away from it, I did not take
137 any time to further investigate, so only take it as an anecdote, not actual
138 evidence.
139
140 Best regards,
141 Michael
142
143 From matthieu.rakotojaona@gmail.com Tue Nov 8 22:22:51 2011
144 From: matthieu.rakotojaona@gmail.com (Matthieu Rakotojaona)
145 Date: Wed, 9 Nov 2011 04:22:51 +0100
146 Subject: [sup-devel] heliotrope-syncback
147 In-Reply-To: <1320692090-sup-9935@stapelberg.de>
148 References: <CAMiZLn1tes53sJAdjzDmv+ntUvNNafPCChtOR9Uq7cnKaeOoQA@mail.gmail.com>
149 <1320648596-sup-2551@masanjin.net> <1320692090-sup-9935@stapelberg.de>
150 Message-ID: <CAMiZLn3dsR_sRfpZV9MLKhJpVhfgsSpd6t3DwHDeg5uydY39Ow@mail.gmail.com>
151
152 Okay, forget about buggy syncback, now comes the IMAP interface ! I've
153 begun to hack ximapd[0], a ruby IMAP server, to work with heliotrope.
154 Unfortunately it is not supported anymore.
155
156 So far, I can connect with it (but with no security), list "mailboxes"
157 (just like GMail, they are just labels) select one and search for
158 something, all using imap4rev1 specs and, of course, heliotrope as a
159 backend. The most interesting parts like FETCHing and STORing are
160 still a few lines of code away. Of course, the goal is to use
161 something like offlineimap, Because It Works, to sync both mail stores
162 in a reliable fashion.
163
164 Considering the implementation, I've considered only UIDs, which are
165 heliotrope's message_ids. Both can and must only go up as messages
166 come and go, so it's cool.
167
168 So if you want to test it, go to https://github.com/rakoo/ximapd-heliotrope.
169 This leads me to the question : what is the license of Heliotrope ?
170 Ximapd seems to be something BSD-like, but Heliotrope has none.
171
172 Cya !
173
174 [0] https://github.com/shugo/ximapd
175 --
176 Matthieu RAKOTOJAONA
177
178 From wmorgan-sup@masanjin.net Wed Nov 9 18:18:08 2011
179 From: wmorgan-sup@masanjin.net (William Morgan)
180 Date: Wed, 09 Nov 2011 23:18:08 +0000
181 Subject: [sup-devel] heliotrope-syncback
182 In-Reply-To: <CAMiZLn3dsR_sRfpZV9MLKhJpVhfgsSpd6t3DwHDeg5uydY39Ow@mail.gmail.com>
183 References: <CAMiZLn1tes53sJAdjzDmv+ntUvNNafPCChtOR9Uq7cnKaeOoQA@mail.gmail.com>
184 <1320648596-sup-2551@masanjin.net>
185 <1320692090-sup-9935@stapelberg.de>
186 <CAMiZLn3dsR_sRfpZV9MLKhJpVhfgsSpd6t3DwHDeg5uydY39Ow@mail.gmail.com>
187 Message-ID: <1320880426-sup-9891@masanjin.net>
188
189 Reformatted excerpts from Matthieu Rakotojaona's message of 2011-11-09:
190 > Okay, forget about buggy syncback, now comes the IMAP interface !
191
192 Now THIS is exciting. Thanks for starting on it!
193
194 > Considering the implementation, I've considered only UIDs, which are
195 > heliotrope's message_ids. Both can and must only go up as messages
196 > come and go, so it's cool.
197
198 That's right. Thread ids can sometimes be non-increasing, but message
199 ids should always be increasing. And the IMAP interface shouldn't have
200 anything to do with thread ids anyways.
201
202 > This leads me to the question : what is the license of Heliotrope ?
203 > Ximapd seems to be something BSD-like, but Heliotrope has none.
204
205 Hm, good question. I will probably GPL it. Let me think for a few days.
206 --
207 William <wmorgan-sup at masanjin.net>
208
209 From eg@gaute.vetsj.com Wed Nov 9 19:12:27 2011
210 From: eg@gaute.vetsj.com (Gaute Hope)
211 Date: Thu, 10 Nov 2011 01:12:27 +0100
212 Subject: [sup-devel] Crash on adding contact
213 Message-ID: <1320883863-sup-9753@qwerzila>
214
215 Hi,
216
217 when adding new contacts (aliases) I get the following error (after
218 upgrade to ruby 1.9.3):
219
220 --- NoMethodError from thread: main
221 undefined method `email' for "":String
222 /home/gaute/dev/ruby/sup.git/lib/sup/contact.rb:34:in `update_alias'
223 /home/gaute/dev/ruby/sup.git/lib/sup/util.rb:572:in `method_missing'
224 /home/gaute/dev/ruby/sup.git/lib/sup/modes/contact-list-mode.rb:17:in `alias_contact'
225 /home/gaute/dev/ruby/sup.git/lib/sup/modes/thread-view-mode.rb:259:in `alias'
226 /home/gaute/dev/ruby/sup.git/lib/sup/mode.rb:59:in `handle_input'
227 /home/gaute/dev/ruby/sup.git/lib/sup/buffer.rb:278:in `handle_input'
228 bin/sup:271:in `<module:Redwood>'
229 bin/sup:80:in `<main>'
230
231 The attached patch seems to fix the problem.
232
233 Regards,
234 Gaute
235 -------------- next part --------------
236 A non-text attachment was scrubbed...
237 Name: 0001-Working-check-for-alias-on-ruby-1.9.3.patch
238 Type: application/octet-stream
239 Size: 793 bytes
240 Desc: not available
241 URL: <http://rubyforge.org/pipermail/sup-devel/attachments/20111110/ced65171/attachment.obj>
242
243 From matthieu.rakotojaona@gmail.com Thu Nov 10 20:25:59 2011
244 From: matthieu.rakotojaona@gmail.com (Matthieu Rakotojaona)
245 Date: Fri, 11 Nov 2011 02:25:59 +0100
246 Subject: [sup-devel] heliotrope-syncback
247 In-Reply-To: <1320880426-sup-9891@masanjin.net>
248 References: <CAMiZLn1tes53sJAdjzDmv+ntUvNNafPCChtOR9Uq7cnKaeOoQA@mail.gmail.com>
249 <1320648596-sup-2551@masanjin.net> <1320692090-sup-9935@stapelberg.de>
250 <CAMiZLn3dsR_sRfpZV9MLKhJpVhfgsSpd6t3DwHDeg5uydY39Ow@mail.gmail.com>
251 <1320880426-sup-9891@masanjin.net>
252 Message-ID: <CAMiZLn1+OFMu-Vht71iEVe6ohkC-kMHV1VE6CumEyvxpbEA2kA@mail.gmail.com>
253
254 On Thu, Nov 10, 2011 at 12:18 AM, William Morgan
255 <wmorgan-sup at masanjin.net> wrote:
256 > Now THIS is exciting. Thanks for starting on it!
257
258 Yes it is. I have now finished (sort of) the FETCH operations. You can
259 grab it here :
260 https://github.com/rakoo/ximapd-heliotrope
261
262 I've struggled a lot with the RFC, not understanding half of it. I
263 don't know if the FETCH results are exact, but they seem to be. Please
264 feel free to test it and tell me what are the problems.
265
266 For those of you not aware : the previous link leads you to a IMAP
267 interface for Heliotrope. I've only implemented the "read-only"
268 commands, so your messages won't have any problems, but you should be
269 able with any IMAP client (again, read-only).
270
271 Cya,
272
273 --
274 Matthieu RAKOTOJAONA
275
276 From matthieu.rakotojaona@gmail.com Wed Nov 16 20:33:07 2011
277 From: matthieu.rakotojaona@gmail.com (Matthieu Rakotojaona)
278 Date: Thu, 17 Nov 2011 02:33:07 +0100
279 Subject: [sup-devel] heliotrope-syncback
280 In-Reply-To: <CAMiZLn1+OFMu-Vht71iEVe6ohkC-kMHV1VE6CumEyvxpbEA2kA@mail.gmail.com>
281 References: <CAMiZLn1tes53sJAdjzDmv+ntUvNNafPCChtOR9Uq7cnKaeOoQA@mail.gmail.com>
282 <1320648596-sup-2551@masanjin.net> <1320692090-sup-9935@stapelberg.de>
283 <CAMiZLn3dsR_sRfpZV9MLKhJpVhfgsSpd6t3DwHDeg5uydY39Ow@mail.gmail.com>
284 <1320880426-sup-9891@masanjin.net>
285 <CAMiZLn1+OFMu-Vht71iEVe6ohkC-kMHV1VE6CumEyvxpbEA2kA@mail.gmail.com>
286 Message-ID: <CAMiZLn3kGuzMUxdMT1M0keSkZgggACWSbE3ykW+o6cdi-AiLkw@mail.gmail.com>
287
288 Hello,
289
290 I'm advancing on the IMAP interface. I can almost use OfflineIMAP, but
291 there are a few things :
292
293 - I needed an other method in the heliotrope-client file :
294 add-message. It is exactly like send_message but doesn't send it.
295 - I needed to have the msgid of a msg if it was already in the store,
296 so a few changes
297
298 Both of these are in the patch attached.
299
300 My problem now is that the addition of messages to the store doesn't
301 seem to work. I use the same things as in heliotrope-add :
302 force_encode("binary") if ...
303 @heliotropeclient.add_message ...
304
305 I send the whole rawbody to heliotrope. But when I want to see the
306 mail, the body is empty (turnsole says 0b, the web interface displays
307 only the header). I even used the RestClient directly, with no luck.
308 So where are my mistakes ?
309
310
311 --
312 Matthieu RAKOTOJAONA
313 -------------- next part --------------
314 A non-text attachment was scrubbed...
315 Name: patch
316 Type: application/octet-stream
317 Size: 2625 bytes
318 Desc: not available
319 URL: <http://rubyforge.org/pipermail/sup-devel/attachments/20111117/04db1efb/attachment.obj>
320
321 From matthieu.rakotojaona@gmail.com Fri Nov 18 14:17:35 2011
322 From: matthieu.rakotojaona@gmail.com (Matthieu Rakotojaona)
323 Date: Fri, 18 Nov 2011 20:17:35 +0100
324 Subject: [sup-devel] rmail gem is faulty
325 Message-ID: <CAMiZLn2-xTSReSjgHwkD2ppaNdm6mwwHmig47P6GpuKajiL6tw@mail.gmail.com>
326
327 Hello everyone,
328
329 I have seen that 'rmail' doesn't parse some of my mails correctly.
330 Specifically, it includes the body in the header fields (I don't know
331 how it manages to do that).
332 Moreover, last version dates from 2008. So I replaced it with the
333 'mail' gem [0], more up-to-date and updated, with some useful methods
334 that we don't have to define specifically in heliotrope.
335
336 The result is the patch enclosed. All I did was in
337 lib/heliotrope/message.rb, so everything else works (mainly).
338
339
340 [0] https://github.com/mikel/mail
341
342 --
343 Matthieu RAKOTOJAONA
344
345 From matthieu.rakotojaona@gmail.com Fri Nov 18 14:19:23 2011
346 From: matthieu.rakotojaona@gmail.com (Matthieu Rakotojaona)
347 Date: Fri, 18 Nov 2011 20:19:23 +0100
348 Subject: [sup-devel] rmail gem is faulty
349 In-Reply-To: <CAMiZLn2-xTSReSjgHwkD2ppaNdm6mwwHmig47P6GpuKajiL6tw@mail.gmail.com>
350 References: <CAMiZLn2-xTSReSjgHwkD2ppaNdm6mwwHmig47P6GpuKajiL6tw@mail.gmail.com>
351 Message-ID: <CAMiZLn1v6PQ75Hfdb9PO+Aw4PZNjakkkVUkgScP6CStC8WYAsw@mail.gmail.com>
352
353 On Fri, Nov 18, 2011 at 8:17 PM, Matthieu Rakotojaona
354 <matthieu.rakotojaona at gmail.com> wrote:
355 > The result is the patch enclosed. All I did was in
356 > lib/heliotrope/message.rb, so everything else works (mainly).
357
358 Hem.
359
360 --
361 Matthieu RAKOTOJAONA
362 -------------- next part --------------
363 A non-text attachment was scrubbed...
364 Name: 0001-modified-message.rb-to-use-mail-gem.patch
365 Type: text/x-patch
366 Size: 8474 bytes
367 Desc: not available
368 URL: <http://rubyforge.org/pipermail/sup-devel/attachments/20111118/f7393609/attachment-0001.bin>
369
370 From michael+sup@stapelberg.de Fri Nov 18 16:14:27 2011
371 From: michael+sup@stapelberg.de (Michael Stapelberg)
372 Date: Fri, 18 Nov 2011 21:14:27 +0000
373 Subject: [sup-devel] Instructions for testing sup with recent ruby/gems
374 Message-ID: <1321650789-sup-7136@stapelberg.de>
375
376 Hi,
377
378 Since I just figured out how to install ruby and sup?s dependencies in a clean
379 environment using RVM (no big deal, but?), I thought I?d share the instructions:
380
381 bash < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)
382 rvm reload
383 rvm install 1.9.3
384 rvm 1.9.3
385 rvm gemset create sup2011-11-18
386 rvm 1.9.3 at sup2011-11-18
387 rvm 1.9.3 at sup2011-11-18 do gem install ncursesw
388 rvm 1.9.3 at sup2011-11-18 do gem install trollop
389 rvm 1.9.3 at sup2011-11-18 do gem install gettext
390 rvm 1.9.3 at sup2011-11-18 do gem install rmail
391 rvm 1.9.3 at sup2011-11-18 do gem install lockfile
392 rvm 1.9.3 at sup2011-11-18 do gem install mime-types
393 rvm 1.9.3 at sup2011-11-18 do gem install xapian-ruby
394 git clone git://gitorious.org/sup/mainline
395 cd mainline
396 ruby -Ilib bin/sup
397
398 Best regards,
399 Michael
400
401 From triumhiz@yandex.ru Tue Nov 22 10:56:28 2011
402 From: triumhiz@yandex.ru (=?utf-8?q?=D0=A1=D0=B5=D1=80=D0=B3=D0=B5=D0=B9_ ?= (Serge))
403 Date: Tue, 22 Nov 2011 19:56:28 +0400
404 Subject: [sup-devel] [BUG] Failure joining threads
405 Message-ID: <1321975663-sup-4280@ss-comp>
406
407 Hello there,
408
409 '#' key sometimes do not join tagged threads in thread-index-mode. And I am not
410 the only one who encountered this problem.
411
412 I attach the mbox file with two messages I cannot join into one thread. Could
413 someone else try to reproduce the bug with this mbox, please?
414
415 Reproducing (in just-installed sup environment):
416
417 1. Type:
418 $ sup-add mbox:/.../failed_join.mbox
419 $ sup
420
421 Two one-message threads are in my inbox at the moment. And it looks like it
422 should be one thread with two messages instead.
423
424 2. Press sequentally tt#
425
426 Threads are now untagged but not joined.
427
428 I have sup-0.12.1 installed from 'xarthisius' overlay on gentoo linux.
429
430 Thanx
431
432 --
433 ?????? (Serge)
434 -------------- next part --------------
435 A non-text attachment was scrubbed...
436 Name: failed_join.mbox
437 Type: application/octet-stream
438 Size: 6921 bytes
439 Desc: not available
440 URL: <http://rubyforge.org/pipermail/sup-devel/attachments/20111122/931b0c03/attachment.obj>
441
442 From triumhiz@yandex.ru Tue Nov 22 13:40:44 2011
443 From: triumhiz@yandex.ru (=?utf-8?q?=D0=A1=D0=B5=D1=80=D0=B3=D0=B5=D0=B9_ ?= (Serge))
444 Date: Tue, 22 Nov 2011 22:40:44 +0400
445 Subject: [sup-devel] Failure joining threads
446 Message-ID: <1321987174-sup-5333@ss-comp>
447
448 Hello there,
449
450 '#' key sometimes do not join tagged threads in thread-index-mode. And I am not
451 the only one who encountered this problem.
452
453 I attach the mbox file with two messages I cannot join into one thread. Could
454 someone else try to reproduce the bug with this mbox, please?
455
456 Reproducing (in just-installed sup environment):
457
458 1. Type:
459 $ sup-add mbox:/.../failed_join.mbox
460 $ sup
461
462 Two one-message threads are in my inbox at the moment. And it looks like it
463 should be one thread with two messages instead.
464
465 2. Press sequentally tt#
466
467 Threads are now untagged but not joined.
468 Nothing appears in status line or log file.
469
470 I have sup-0.12.1 installed from 'xarthisius' overlay on gentoo linux.
471
472 Thanx
473
474 --
475 ?????? (Serge)
476 -------------- next part --------------
477 A non-text attachment was scrubbed...
478 Name: failed_join.mbox
479 Type: application/octet-stream
480 Size: 6921 bytes
481 Desc: not available
482 URL: <http://rubyforge.org/pipermail/sup-devel/attachments/20111122/e2ea5fd6/attachment.obj>
483
484 From michael+sup@stapelberg.de Tue Nov 29 18:20:39 2011
485 From: michael+sup@stapelberg.de (Michael Stapelberg)
486 Date: Tue, 29 Nov 2011 23:20:39 +0000
487 Subject: [sup-devel] [PATCH] Require gpgme 1.0.8 (v2 breaks API)
488 Message-ID: <1322608753-sup-6359@stapelberg.de>
489
490 Hi,
491
492 The latest version of gpgme breaks its API. Pragmatic fix for now is requiring
493 the old version.
494
495 Hamish: Is this patch OK with you?
496
497 Best regards,
498 Michael
499 -------------- next part --------------
500 A non-text attachment was scrubbed...
501 Name: 0001-Require-gpgme-1.0.8-v2-breaks-API.patch
502 Type: application/octet-stream
503 Size: 934 bytes
504 Desc: not available
505 URL: <http://rubyforge.org/pipermail/sup-devel/attachments/20111129/ff5731d0/attachment-0001.obj>
506