* Re: From Gnus to mu4e [not found] <mailman.14.1440105837.31004.help-gnu-emacs@gnu.org> @ 2015-08-24 11:19 ` Joost Kremers 0 siblings, 0 replies; 41+ messages in thread From: Joost Kremers @ 2015-08-24 11:19 UTC (permalink / raw To: help-gnu-emacs Julien Cubizolles wrote: > * Can OfflineImap or whatever imap sync tool used exclude some folders > in the main Maildir folder from being synced to the IMAP server ? I'd > like to be able to archive some old mail without it being sent to > Gmail. Yes, using `folderfilter`, which can be a lambda (offlineimaprc is basically just Python). So I have something like: folderfilter = lambda foldername: not re.search('(^Deleted|^Junk|^Tasks|^Calendar|^Notes|^Contacts|^Drafts)', foldername) which excludes folders matching the regexp in re.search. > * Can mu4e use the Gmail labels for search queries ? There's bin quite some discussion about that on the mu4e mailing list, and AFAIK there is a workable solution for it. But I don't use it myself, so I can't really say. I'd check the archives and ask on the mu4e mailing list if you still have questions. > * If a message is tagged with several labels on Gmail, does it appear in > several folders in the local repository ? On a related note, if one > were to move a message from a folder in a repository synced with Gmail > to another folder not synced (for archiving purposes), I guess the > message would remain at least in the [Gmail]/All Mail folder. Is there > an easy way to remove it altogether from Gmail ? This, too, has been discussed on the mailing list, IIRC. > * Finally, the mailing list for mu4e doesn't seem very active > (especially when you compare it to the one for gnus). Are there any > other channels to get some help ? Perhaps not compared to the gnus mailing list, but I'm subscribed and can say that the list is active enough. Also, the mu/mu4e maintainer reads the list and answers questions. -- Joost Kremers joostkremers@fastmail.fm Selbst in die Unterwelt dringt durch Spalten Licht EN:SiS(9) ^ permalink raw reply [flat|nested] 41+ messages in thread
* From Gnus to mu4e @ 2015-08-20 21:23 Julien Cubizolles 2015-08-21 10:28 ` Julien Cubizolles ` (2 more replies) 0 siblings, 3 replies; 41+ messages in thread From: Julien Cubizolles @ 2015-08-20 21:23 UTC (permalink / raw To: help-gnu-emacs I'm considering moving from Gnus to mu4e, mostly because I spend too much time tinkering with settings I don't really understand/need in Gnus, and because the search feature seems promising. If some of you have already made such a move, or are just happy users of mu4e, I'd appreciate your input on the following issues. * Can OfflineImap or whatever imap sync tool used exclude some folders in the main Maildir folder from being synced to the IMAP server ? I'd like to be able to archive some old mail without it being sent to Gmail. * I'm using gmane to read a lot of mailing lists. I guess there's no way to do that with mu4e and I'm considering sticking with Gnus for that use. Does that seem like a reasonable solution ? * Can mu4e use the Gmail labels for search queries ? * If a message is tagged with several labels on Gmail, does it appear in several folders in the local repository ? On a related note, if one were to move a message from a folder in a repository synced with Gmail to another folder not synced (for archiving purposes), I guess the message would remain at least in the [Gmail]/All Mail folder. Is there an easy way to remove it altogether from Gmail ? * Finally, the mailing list for mu4e doesn't seem very active (especially when you compare it to the one for gnus). Are there any other channels to get some help ? Thanks for your input. Julien. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-20 21:23 Julien Cubizolles @ 2015-08-21 10:28 ` Julien Cubizolles 2015-08-21 10:49 ` Colin Yates 2015-08-21 13:06 ` Suvayu Ali 2015-08-25 5:49 ` Bastien 2 siblings, 1 reply; 41+ messages in thread From: Julien Cubizolles @ 2015-08-21 10:28 UTC (permalink / raw To: help-gnu-emacs Somebody replied by mail to this message to give some advice, but my gnus splitting isn't very fancy these days and I somehow lost it... I just wanted to thank him. Julien. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-21 10:28 ` Julien Cubizolles @ 2015-08-21 10:49 ` Colin Yates 0 siblings, 0 replies; 41+ messages in thread From: Colin Yates @ 2015-08-21 10:49 UTC (permalink / raw To: Julien Cubizolles; +Cc: help-gnu-emacs I think that was me, here is the reply: Not much to add as I am a newbie here, but https://github.com/skeeto/elfeed is a similar tool for rss feeds. I do use mu4e but I use mbsync which seems to have better google-karma than offlineimap. It also allows you to choose which imap folders to pull down. I am actually investigating notmuch as I think it has a getter story around tagging emails; mu4e is folder based. Actually, when I have more time I am going to look into using org-mode and links for organising... I wasn't being self-deprecating with the newbie warning so take the above with a healthy dose of salt :-) ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-20 21:23 Julien Cubizolles 2015-08-21 10:28 ` Julien Cubizolles @ 2015-08-21 13:06 ` Suvayu Ali 2015-08-26 6:53 ` Julien Cubizolles 2015-08-25 5:49 ` Bastien 2 siblings, 1 reply; 41+ messages in thread From: Suvayu Ali @ 2015-08-21 13:06 UTC (permalink / raw To: help-gnu-emacs Hi, I think I can answer some of your queries. On Thu, Aug 20, 2015 at 11:23:36PM +0200, Julien Cubizolles wrote: > > * Can OfflineImap or whatever imap sync tool used exclude some folders > in the main Maildir folder from being synced to the IMAP server ? I'd > like to be able to archive some old mail without it being sent to > Gmail. Yes, it is possible. Just use a folderfilter. Something like this: folderfilter = lambda foldername: foldername in ['INBOX', 'mysent', 'mailinglist', 'org-mode', 'mutt', 'Emacs', 'Tux', 'XFCE', 'bash', 'notmuch', 'travel', '[Google Mail]/Drafts'] So Colin's statement about mbsync being better than OfflineIMAP because of this reason, is inaccurate. Although, I have heard mbsync performs better than OfflineIMAP, I still have to test this though. > * Can mu4e use the Gmail labels for search queries ? Probably not directly, but if it allows for limiting by folder name, it should be possible. You can do this with notmuch (I use notmuch). > * If a message is tagged with several labels on Gmail, does it appear in > several folders in the local repository ? On a related note, if one > were to move a message from a folder in a repository synced with Gmail > to another folder not synced (for archiving purposes), I guess the > message would remain at least in the [Gmail]/All Mail folder. Is there > an easy way to remove it altogether from Gmail ? There are many subtleties when it comes to Gmail's implementation of the IMAP interface. I would recommend reading these two articles: - https://support.google.com/mail/answer/77657?hl=en - https://support.google.com/mail/answer/78755?hl=en That said, in Gmail, you need something like this: When a message is marked as deleted and expunged from the last visible IMAP folder: - [ ] Archive the message (default) - [X] Move the message to the Bin - [ ] Immediately delete the message forever You can also set the last option, choosing the default will leave it in Gmail. On top of this, you should make sure the All Mail folder is _not_ accessible over IMAP. This is important. As long as the message is present in at least one IMAP folder, Gmail will retain it. This is where it becomes important how you choose to delete it. With notmuch, when you delete a message the delete flag will be forwarded to all the respective messages in the maildir, which in turn ensures it is deleted from Gmail the next time you sync. Maybe mu4e does something similar. Hope this helps, -- Suvayu Open source is the future. It sets us free. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-21 13:06 ` Suvayu Ali @ 2015-08-26 6:53 ` Julien Cubizolles 0 siblings, 0 replies; 41+ messages in thread From: Julien Cubizolles @ 2015-08-26 6:53 UTC (permalink / raw To: help-gnu-emacs Suvayu Ali <fatkasuvayu+linux@gmail.com> writes: >> * If a message is tagged with several labels on Gmail, does it appear in >> several folders in the local repository ? On a related note, if one >> were to move a message from a folder in a repository synced with Gmail >> to another folder not synced (for archiving purposes), I guess the >> message would remain at least in the [Gmail]/All Mail folder. Is there >> an easy way to remove it altogether from Gmail ? > > There are many subtleties when it comes to Gmail's implementation of the > IMAP interface. I would recommend reading these two articles: > > - https://support.google.com/mail/answer/77657?hl=en > - https://support.google.com/mail/answer/78755?hl=en > > That said, in Gmail, you need something like this: > > When a message is marked as deleted and expunged from the last visible IMAP folder: > - [ ] Archive the message (default) > - [X] Move the message to the Bin > - [ ] Immediately delete the message forever > > You can also set the last option, choosing the default will leave it in > Gmail. On top of this, you should make sure the All Mail folder is > _not_ accessible over IMAP. This is important. As long as the message > is present in at least one IMAP folder, Gmail will retain it. This is > where it becomes important how you choose to delete it. With notmuch, > when you delete a message the delete flag will be forwarded to all the > respective messages in the maildir, which in turn ensures it is deleted > from Gmail the next time you sync. Maybe mu4e does something similar. Thanks a lot for this info: It will be useful regardless of the MUA I use. Julien. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-20 21:23 Julien Cubizolles 2015-08-21 10:28 ` Julien Cubizolles 2015-08-21 13:06 ` Suvayu Ali @ 2015-08-25 5:49 ` Bastien 2015-08-25 6:28 ` Ian Zimmerman 2015-08-26 7:22 ` Julien Cubizolles 2 siblings, 2 replies; 41+ messages in thread From: Bastien @ 2015-08-25 5:49 UTC (permalink / raw To: Julien Cubizolles; +Cc: help-gnu-emacs Julien Cubizolles <j.cubizolles@free.fr> writes: > I'm considering moving from Gnus to mu4e, mostly because I spend too > much time tinkering with settings I don't really understand/need in > Gnus, and because the search feature seems promising. After a *lot* of scratching and testing, here is my setup: - Gnus for news and mails: because there are too many things I miss in other MUA. A simple example: quote only the selected region when replying. This cannot be implemented easily in mu and notmuch. - notmuch as a search engine: mu is really great (I used it the last three months) but I cannot get used to mu4e. And starting mu4e with a database of 100K emails takes two or three seconds. - mbsync to sync imap folders as local maildirs: I used offlineimap for several years and mbsync is just faster. - postfix with smtp_sender_dependent_authentication to use various smtp servers depending on the email I'm using. I see people use msmtp and msmtp-mta, which I tried for a while, but I'm just too impatient with the .3s you have to wait when sending. Will reconsider in a decade, when Emacs has more asynchronous power :) This is what I'm sticking too now and I'm fine. I was fine for years with a dummy fetchmail+nnml setup... many groups, many posting styles, no complexity. Then I obeyed to the social pressure and moved to all the dovecot, offlineimap, stuff -- and yes, OT discussions on the orgmode mailing lists pushed me in this directions! I must say that I was very frustrated for months that I didn't stick longer to this setup. Especially because I don't use fancy phones and I don't really need to go "mobile" or to sync many devices... but now I'm a grown up and I can haz another device that I'll sync in peace. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-25 5:49 ` Bastien @ 2015-08-25 6:28 ` Ian Zimmerman 2015-08-25 6:56 ` Bastien Guerry 2015-08-26 7:22 ` Julien Cubizolles 1 sibling, 1 reply; 41+ messages in thread From: Ian Zimmerman @ 2015-08-25 6:28 UTC (permalink / raw To: help-gnu-emacs On 2015-08-25 07:49 +0200, Bastien wrote: > After a *lot* of scratching and testing, here is my setup: > > - Gnus for news and mails: because there are too many things I miss in > other MUA. A simple example: quote only the selected region when > replying. This cannot be implemented easily in mu and notmuch. I completely respect your choices - this one and the others. But I still would like to point out that this may also be an example of something else. Choices of this kind are always a tradeoff. Great as Gnus is I don't think it is perfect, or else I'd use it :-) So, how bad are the problems with Gnus, and how bad are the problems with the others? The missing feature you mention doesn't strike _me_ as a particularly serious problem. After all, you have to select the region first, so it's about the same amount of work as deleting the unwanted material in the reply. So if this were the _only_ problem with MUA X, it would not make me drop it and use Gnus. But YMMV! -- Please *no* private copies of mailing list or newsgroup messages. Rule 420: All persons more than eight miles high to leave the court. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-25 6:28 ` Ian Zimmerman @ 2015-08-25 6:56 ` Bastien Guerry 0 siblings, 0 replies; 41+ messages in thread From: Bastien Guerry @ 2015-08-25 6:56 UTC (permalink / raw To: help-gnu-emacs Ian Zimmerman <itz@buug.org> writes: > So if this were the _only_ problem with MUA X, it would not make me drop > it and use Gnus. But YMMV! Yes, sure, I'm not saying Gnus is better, just saying that I'm too used to it. The "switching cost" is too high for me. Or maybe I'm just too old! :) ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-25 5:49 ` Bastien 2015-08-25 6:28 ` Ian Zimmerman @ 2015-08-26 7:22 ` Julien Cubizolles 2015-08-27 12:50 ` Rainer M Krug ` (2 more replies) 1 sibling, 3 replies; 41+ messages in thread From: Julien Cubizolles @ 2015-08-26 7:22 UTC (permalink / raw To: help-gnu-emacs Bastien <bzg@gnu.org> writes: > Julien Cubizolles <j.cubizolles@free.fr> writes: > >> I'm considering moving from Gnus to mu4e, mostly because I spend too >> much time tinkering with settings I don't really understand/need in >> Gnus, and because the search feature seems promising. > > After a *lot* of scratching and testing, here is my setup: > > - Gnus for news and mails: because there are too many things I miss in > other MUA. A simple example: quote only the selected region when > replying. This cannot be implemented easily in mu and notmuch. > > - notmuch as a search engine: mu is really great (I used it the last > three months) but I cannot get used to mu4e. And starting mu4e with > a database of 100K emails takes two or three seconds. > > - mbsync to sync imap folders as local maildirs: I used offlineimap > for several years and mbsync is just faster. That seems very interesting, and I think I'll try to set it up. offlineimap or mbsync seems to be the right way to: + have a fast access to my mail (nnimap from Gnus on a remote server is a bit slow sometimes) + have a complete access from my mobile phone + I can still choose whatever MUA I want on top of that if I want to try something else thant Gnus and still access my old nnml mail. I have a few questions about some details of your setup: + regarding filtering and spam detection, do you rely on the possibilities offered by you mail provider (Gmail or other)? + you said you're running Dovecot. Gnus could also directly access the local maildir without a local imap server the way mu4e does, couldn't it ? What's the benefit of running it ? > - postfix with smtp_sender_dependent_authentication to use various > smtp servers depending on the email I'm using. I see people use > msmtp and msmtp-mta, which I tried for a while, but I'm just too > impatient with the .3s you have to wait when sending. From looking at my fossilized .gnus.el, I'm using smtpmail, which would be equivalent to msmtp. + I'm gonna need to duplicate this setup on two machines (laptop and desktop). I guess running offline imap on the two machines and synchronizing the nnml, nnarchive mail by unison would work. Thanks a lot for all this information. I've a much clearer idea of all the possibilities now. Julien. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-26 7:22 ` Julien Cubizolles @ 2015-08-27 12:50 ` Rainer M Krug 2015-08-27 17:15 ` Bastien 2015-08-27 21:27 ` Julien Cubizolles 2015-08-27 12:58 ` Suvayu Ali 2015-08-27 17:08 ` Bastien 2 siblings, 2 replies; 41+ messages in thread From: Rainer M Krug @ 2015-08-27 12:50 UTC (permalink / raw To: Julien Cubizolles; +Cc: help-gnu-emacs [-- Attachment #1: Type: text/plain, Size: 5700 bytes --] Julien Cubizolles <j.cubizolles@free.fr> writes: > Bastien <bzg@gnu.org> writes: > >> Julien Cubizolles <j.cubizolles@free.fr> writes: >> >>> I'm considering moving from Gnus to mu4e, mostly because I spend too >>> much time tinkering with settings I don't really understand/need in >>> Gnus, and because the search feature seems promising. >> >> After a *lot* of scratching and testing, here is my setup: >> >> - Gnus for news and mails: because there are too many things I miss in >> other MUA. A simple example: quote only the selected region when >> replying. This cannot be implemented easily in mu and notmuch. >> >> - notmuch as a search engine: mu is really great (I used it the last >> three months) but I cannot get used to mu4e. And starting mu4e with >> a database of 100K emails takes two or three seconds. >> >> - mbsync to sync imap folders as local maildirs: I used offlineimap >> for several years and mbsync is just faster. Sorry - I am late to this question. I am using the same setup, only with offlineimap - it works, and speed is not such a problem for me so that I would move to mbsync. > > That seems very interesting, and I think I'll try to set it > up. offlineimap or mbsync seems to be the right way to: > + have a fast access to my mail (nnimap from Gnus on a remote server > is a bit slow sometimes) > > + have a complete access from my mobile phone > > + I can still choose whatever MUA I want on top of that if I want to > try something else thant Gnus and still access my old nnml mail. Exactly - and you can easily make backups from your all your emails in a format which will be supported for quite some time. > > I have a few questions about some details of your setup: > > + regarding filtering and spam detection, do you rely on the > possibilities offered by you mail provider (Gmail or other)? I am using gmail for this - thought for some minutes about implementing it locally, but gmail works reliable. > > + you said you're running Dovecot. Gnus could also directly access the > local maildir without a local imap server the way mu4e does, couldn't > it ? What's the benefit of running it ? There is one program accessing the mail folder: dovecot - no conflicts, race conditions, no slightly different formats, standards, ... And you can also remotely access dovecot if you want to. By the way: the dovecot daemon is not permanently running, as I am using the shell access from gnus: ,---- | (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir"))) `---- The same from offlineimap: ,---- | preauthtunnel = /usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir `---- and offlineimap only reads the emails from the maildir. And I have found the following two functions to open the same message which is open in gnus in notmuch and the other way around: --8<---------------cut here---------------start------------->8--- ** In notmuch jump to gnus folder From [[http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html]]: But seeing just the message that matches is not enough, sometimes we want the whole thread. Here's a snippet of Lisp for your .gnus, based on Tassilo Horn's configuration. Do a Notmuch search, enter one of the results, type C-c C-c, you'll get transported to the folder where that message was, with the context. Note this requires code from org-mode, so you might need to install that. #+begin_src emacs-lisp (defun rmk/notmuch-file-to-group (file) "Calculate the Gnus group name from the given file name. " (let ((group (file-name-directory (directory-file-name (file-name-directory file))))) (setq group (replace-regexp-in-string ".*/Maildir/" "nnimap+Maildir:" group)) (setq group (replace-regexp-in-string "/$" "" group)) (if (string-match ":$" group) (concat group "INBOX") (replace-regexp-in-string ":\\." ":" group)))) (defun rmk/notmuch-goto-message-in-gnus () "Open a summary buffer containing the current notmuch article." (interactive) (let ((group (rmk/notmuch-file-to-group (notmuch-show-get-filename))) (message-id (replace-regexp-in-string "^id:" "" (notmuch-show-get-message-id)))) (setq message-id (replace-regexp-in-string "\"" "" message-id)) (if (and group message-id) (progn (switch-to-buffer "*Group*") (org-gnus-follow-link group message-id)) (message "Couldn't get relevant infos for switching to Gnus.")))) (define-key notmuch-show-mode-map (kbd "C-c C-c") 'rmk/notmuch-goto-message-in-gnus) #+end_src --8<---------------cut here---------------end--------------->8--- Hope this helps, Rainer > >> - postfix with smtp_sender_dependent_authentication to use various >> smtp servers depending on the email I'm using. I see people use >> msmtp and msmtp-mta, which I tried for a while, but I'm just too >> impatient with the .3s you have to wait when sending. > > From looking at my fossilized .gnus.el, I'm using smtpmail, which > would be equivalent to msmtp. > > + I'm gonna need to duplicate this setup on two machines (laptop and > desktop). I guess running offline imap on the two machines and > synchronizing the nnml, nnarchive mail by unison would work. > > Thanks a lot for all this information. I've a much clearer idea of all > the possibilities now. > > Julien. > > > -- Rainer M. Krug email: Rainer<at>krugs<dot>de PGP: 0x0F52F982 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 480 bytes --] ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 12:50 ` Rainer M Krug @ 2015-08-27 17:15 ` Bastien 2015-08-28 8:39 ` Rainer M Krug 2015-08-27 21:27 ` Julien Cubizolles 1 sibling, 1 reply; 41+ messages in thread From: Bastien @ 2015-08-27 17:15 UTC (permalink / raw To: Rainer M Krug; +Cc: help-gnu-emacs, Julien Cubizolles Hi Rainer, Rainer M Krug <Rainer@krugs.de> writes: > There is one program accessing the mail folder: dovecot - no conflicts, race > conditions, no slightly different formats, standards, ... And you can > also remotely access dovecot if you want to. > > By the way: the dovecot daemon is not permanently running, as I am using > the shell access from gnus: > > ,---- > | (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir"))) > `---- > > The same from offlineimap: > > ,---- > | preauthtunnel = /usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir > `---- Interesting -- can you detail this parts of your configuration? I am not using dovecot locally so far, just nnmaildir from Gnus. But I've encountered some annoying race conditions, while sync'ing through mbsync *and* reading/replying from Gnus, so I'm interested in knowing how to avoid them -- ideally letting mbsync to run in the background. Thanks! -- Bastien ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 17:15 ` Bastien @ 2015-08-28 8:39 ` Rainer M Krug 2015-08-28 8:44 ` Bastien 2015-08-28 8:51 ` Bastien 0 siblings, 2 replies; 41+ messages in thread From: Rainer M Krug @ 2015-08-28 8:39 UTC (permalink / raw To: Bastien; +Cc: help-gnu-emacs, Julien Cubizolles [-- Attachment #1: Type: text/plain, Size: 2906 bytes --] Bastien <bzg@gnu.org> writes: > Hi Rainer, > > Rainer M Krug <Rainer@krugs.de> writes: > >> There is one program accessing the mail folder: dovecot - no conflicts, race >> conditions, no slightly different formats, standards, ... And you can >> also remotely access dovecot if you want to. >> >> By the way: the dovecot daemon is not permanently running, as I am using >> the shell access from gnus: >> >> ,---- >> | (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir"))) >> `---- >> >> The same from offlineimap: >> >> ,---- >> | preauthtunnel = /usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir >> `---- > > Interesting -- can you detail this parts of your configuration? This is how I define my secondary select method in GNUS, which is my local, with offlineimap synched, mail: --8<---------------cut here---------------start------------->8--- (add-to-list 'gnus-secondary-select-methods ;; (nnimap-stream network) ;; (nnimap-server-port 143) ;; (nnimap-address "localhost"))) (nnimap-stream shell) (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir"))) --8<---------------cut here---------------end--------------->8--- To ose the daemon, you have to toggle the comments (I hope - haven't done it in a long time...). In offlineimap, I have the following for my local mail location: --8<---------------cut here---------------start------------->8--- [Repository LocalRMKrugGMAIL] type = IMAP preauthtunnel = /usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir holdconnectionopen = yes maxconnections = 3 --8<---------------cut here---------------end--------------->8--- So offlineimap uses the binary (.../imap) to connect to my maildir. > > I am not using dovecot locally so far, just nnmaildir from Gnus. > > But I've encountered some annoying race conditions, while sync'ing > through mbsync *and* reading/replying from Gnus, so I'm interested > in knowing how to avoid them -- ideally letting mbsync to run in > the background. Interesting. I have seen that mbsync can also sync to a *running* imap server, but I do not if you can use something like I use in gnus and offlineimap. Thinking about the race conditions more, I do not know if my approach actually avoids them, as there are two different processes of imap running (one offlineimap, one gnus) accessing the same mail. I can only say this worked perfectly for me so far. If you really want to be sure, run everything through a local docecot imap server. Cheers, Rainer > > Thanks! -- Rainer M. Krug email: Rainer<at>krugs<dot>de PGP: 0x0F52F982 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 480 bytes --] ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 8:39 ` Rainer M Krug @ 2015-08-28 8:44 ` Bastien 2015-08-28 8:51 ` Rainer M Krug 2015-08-28 8:51 ` Bastien 1 sibling, 1 reply; 41+ messages in thread From: Bastien @ 2015-08-28 8:44 UTC (permalink / raw To: Rainer M Krug; +Cc: help-gnu-emacs, Julien Cubizolles Thanks again for the detailed config Rainer. The way to go for me know is to discipline myself not retrieving email too often. An old "papa"-way to avoid race conditions. -- Bastien ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 8:44 ` Bastien @ 2015-08-28 8:51 ` Rainer M Krug 2015-08-28 8:58 ` Rasmus 0 siblings, 1 reply; 41+ messages in thread From: Rainer M Krug @ 2015-08-28 8:51 UTC (permalink / raw To: Bastien; +Cc: help-gnu-emacs, Julien Cubizolles [-- Attachment #1: Type: text/plain, Size: 827 bytes --] Bastien <bzg@gnu.org> writes: > Thanks again for the detailed config Rainer. > > The way to go for me know is to discipline myself not retrieving email > too often. An old "papa"-way to avoid race conditions. True - offlineimap checks in my case every two minutes (offlineimap's quick check), and an more detailed check every 10 minutes - I guess this is often quite often. Cheers, Rainer -- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44 Cell: +33 - (0)6 85 62 59 98 Fax : +33 - (0)9 58 10 27 44 Fax (D): +49 - (0)3 21 21 25 22 44 email: Rainer@krugs.de Skype: RMkrug PGP: 0x0F52F982 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 480 bytes --] ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 8:51 ` Rainer M Krug @ 2015-08-28 8:58 ` Rasmus 2015-08-28 9:15 ` Julien Cubizolles 0 siblings, 1 reply; 41+ messages in thread From: Rasmus @ 2015-08-28 8:58 UTC (permalink / raw To: help-gnu-emacs Rainer M Krug <Rainer@krugs.de> writes: > Bastien <bzg@gnu.org> writes: > >> Thanks again for the detailed config Rainer. >> >> The way to go for me know is to discipline myself not retrieving email >> too often. An old "papa"-way to avoid race conditions. > > True - offlineimap checks in my case every two minutes (offlineimap's > quick check), and an more detailed check every 10 minutes - I guess this > is often quite often. Offlineimap supports IDLE. Rasmus -- Bang bang ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 8:58 ` Rasmus @ 2015-08-28 9:15 ` Julien Cubizolles 2015-08-28 10:03 ` Rainer M Krug 2015-08-28 10:04 ` Rasmus 0 siblings, 2 replies; 41+ messages in thread From: Julien Cubizolles @ 2015-08-28 9:15 UTC (permalink / raw To: help-gnu-emacs Rasmus <rasmus@gmx.us> writes: > Rainer M Krug <Rainer@krugs.de> writes: > >> Bastien <bzg@gnu.org> writes: >> >>> Thanks again for the detailed config Rainer. >>> >>> The way to go for me know is to discipline myself not retrieving email >>> too often. An old "papa"-way to avoid race conditions. >> >> True - offlineimap checks in my case every two minutes (offlineimap's >> quick check), and an more detailed check every 10 minutes - I guess this >> is often quite often. > > Offlineimap supports IDLE. Does this mean you don't need to run offlineimap periodically if you use this mode ? It doesn't help with potential race conditions with Gnus accessing the mailbox, does it ? > > Rasmus ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 9:15 ` Julien Cubizolles @ 2015-08-28 10:03 ` Rainer M Krug 2015-08-28 10:08 ` Rasmus 2015-08-28 10:04 ` Rasmus 1 sibling, 1 reply; 41+ messages in thread From: Rainer M Krug @ 2015-08-28 10:03 UTC (permalink / raw To: Julien Cubizolles; +Cc: help-gnu-emacs [-- Attachment #1: Type: text/plain, Size: 1177 bytes --] Julien Cubizolles <j.cubizolles@free.fr> writes: > Rasmus <rasmus@gmx.us> writes: > >> Rainer M Krug <Rainer@krugs.de> writes: >> >>> Bastien <bzg@gnu.org> writes: >>> >>>> Thanks again for the detailed config Rainer. >>>> >>>> The way to go for me know is to discipline myself not retrieving email >>>> too often. An old "papa"-way to avoid race conditions. >>> >>> True - offlineimap checks in my case every two minutes (offlineimap's >>> quick check), and an more detailed check every 10 minutes - I guess this >>> is often quite often. >> >> Offlineimap supports IDLE. > > Does this mean you don't need to run offlineimap periodically if you use > this mode ? It doesn't help with potential race conditions with Gnus > accessing the mailbox, does it ? Yes - offlineimap is running permanently, but ony checks in specific intervalls - the rest of the time it is idling, but can, if told so, keep the connection open (and this is what I do in my config). No - I don't think it has any impact on potential race conditions. Cheers, Rainer >> >> Rasmus > > > -- Rainer M. Krug email: Rainer<at>krugs<dot>de PGP: 0x0F52F982 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 480 bytes --] ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 10:03 ` Rainer M Krug @ 2015-08-28 10:08 ` Rasmus 2015-08-28 10:41 ` Rainer M Krug 0 siblings, 1 reply; 41+ messages in thread From: Rasmus @ 2015-08-28 10:08 UTC (permalink / raw To: help-gnu-emacs Rainer M Krug <Rainer@krugs.de> writes: > Yes - offlineimap is running permanently, but ony checks in specific > intervalls - the rest of the time it is idling, but can, if told so, > keep the connection open (and this is what I do in my config). This is incorrect. In email technology, IDLE is an IMAP feature described in RFC 2177 that allows a client to indicate to the server that it is ready to accept real-time notifications. (Wikipedia) ^^^^^^^^^^^^^^^^^^^^^^^ What you are talking about is polling; like a chron job. This is inefficient. Rasmus -- m-mm-mmm-mmmm bacon! ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 10:08 ` Rasmus @ 2015-08-28 10:41 ` Rainer M Krug 0 siblings, 0 replies; 41+ messages in thread From: Rainer M Krug @ 2015-08-28 10:41 UTC (permalink / raw To: Rasmus; +Cc: help-gnu-emacs [-- Attachment #1: Type: text/plain, Size: 1181 bytes --] Rasmus <rasmus@gmx.us> writes: > Rainer M Krug <Rainer@krugs.de> writes: > >> Yes - offlineimap is running permanently, but ony checks in specific >> intervalls - the rest of the time it is idling, but can, if told so, >> keep the connection open (and this is what I do in my config). > > This is incorrect. > > In email technology, IDLE is an IMAP feature described in RFC 2177 > that allows a client to indicate to the server that it is ready to > accept real-time notifications. (Wikipedia) > ^^^^^^^^^^^^^^^^^^^^^^^ Sorry - mixed it up with running offlineimap in daemon mode with checking periodically, controlled by the "autorefresh" option. Didn't know about the idlefolders option. > > What you are talking about is polling; like a chron job. This is > inefficient. Why is it inefficient, and inefficient in regards to local resources, time, bandwidth? The connections are kept open, and it is just polling every few minutes. With idlefolders, offlineimap is permanently listening to the server? Rainer > > Rasmus -- Rainer M. Krug email: Rainer<at>krugs<dot>de PGP: 0x0F52F982 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 480 bytes --] ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 9:15 ` Julien Cubizolles 2015-08-28 10:03 ` Rainer M Krug @ 2015-08-28 10:04 ` Rasmus 1 sibling, 0 replies; 41+ messages in thread From: Rasmus @ 2015-08-28 10:04 UTC (permalink / raw To: help-gnu-emacs Julien Cubizolles <j.cubizolles@free.fr> writes: > Does this mean you don't need to run offlineimap periodically if you use > this mode ? Right. It's what TB and k9 uses as well. It works OK. Sometimes it's even able to handle disconnect → suspend → wake up on another network. It's not, however, as good as the IDLE support of "proper" mailclients such as K9 and TB. You need to add something like idlefolders = ['INBOX'] To your offlineimap [remote REPOSITORY]. And maybe maxsyncaccounts = 5 socktimeout = 60 To your [general]. Also, you probably need to add this to [account REPOSITORY] autorefresh = 28 There might be more, I added it to my offlineimap settings a while ago. > It doesn't help with potential race conditions with Gnus > accessing the mailbox, does it ? No idea. I use dovecot for accesing the maildir, like this: (add-to-list 'gnus-secondary-select-methods '(nnimap "mail" (nnimap-stream shell) (nnimap-shell-program "MAIL=maildir:$HOME/mail /usr/lib/dovecot/imap"))) Maybe you can use postsynchook in [Account REPOSITORY] to your advantage. E.g. I get popups via: postsynchook = emacsclient -s gnus --eval "(with-timeout (2) (when (gnus-alive-p) (gnus-group-get-new-news 1)))" Rasmus -- When the facts change, I change my mind. What do you do, sir? ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 8:39 ` Rainer M Krug 2015-08-28 8:44 ` Bastien @ 2015-08-28 8:51 ` Bastien 1 sibling, 0 replies; 41+ messages in thread From: Bastien @ 2015-08-28 8:51 UTC (permalink / raw To: Rainer M Krug; +Cc: help-gnu-emacs, Julien Cubizolles Thanks again for the detailed config Rainer. The way to go for me know is to discipline myself not retrieving email too often. An old "papa"-way to avoid race conditions. -- Bastien ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 12:50 ` Rainer M Krug 2015-08-27 17:15 ` Bastien @ 2015-08-27 21:27 ` Julien Cubizolles 2015-08-27 23:34 ` Suvayu Ali 2015-08-28 8:21 ` Rainer M Krug 1 sibling, 2 replies; 41+ messages in thread From: Julien Cubizolles @ 2015-08-27 21:27 UTC (permalink / raw To: Rainer M Krug; +Cc: help-gnu-emacs Rainer M Krug <Rainer@krugs.de> writes: > There is one program accessing the mail folder: dovecot - no conflicts, race > conditions, no slightly different formats, standards, ... And you can > also remotely access dovecot if you want to. Because otherwise, you would have Gnus and offlineimap competing for access ? > By the way: the dovecot daemon is not permanently running, as I am using > the shell access from gnus: > > ,---- > | (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir"))) > `---- > > The same from offlineimap: > > ,---- > | preauthtunnel = /usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir > `---- > > and offlineimap only reads the emails from the maildir. That looks interesting, I'll try it. > And I have found the following two functions to open the same message > which is open in gnus in notmuch and the other way around: > > ** In notmuch jump to gnus folder I think you forgot the other way around :-) > Hope this helps, I'm sure it will, thanks a lot. Julien. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 21:27 ` Julien Cubizolles @ 2015-08-27 23:34 ` Suvayu Ali 2015-08-27 23:50 ` Ian Zimmerman 2015-08-28 8:21 ` Rainer M Krug 1 sibling, 1 reply; 41+ messages in thread From: Suvayu Ali @ 2015-08-27 23:34 UTC (permalink / raw To: help-gnu-emacs On Thu, Aug 27, 2015 at 11:27:50PM +0200, Julien Cubizolles wrote: > Rainer M Krug <Rainer@krugs.de> writes: > > > There is one program accessing the mail folder: dovecot - no conflicts, race > > conditions, no slightly different formats, standards, ... And you can > > also remotely access dovecot if you want to. > > Because otherwise, you would have Gnus and offlineimap competing for > access ? I do not think this is possible. In fact AFAIK, any kind of race condition in a maildir is not possible by design, as long as the filesystem operations are atomic. -- Suvayu Open source is the future. It sets us free. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 23:34 ` Suvayu Ali @ 2015-08-27 23:50 ` Ian Zimmerman 2015-08-28 0:14 ` Suvayu Ali 0 siblings, 1 reply; 41+ messages in thread From: Ian Zimmerman @ 2015-08-27 23:50 UTC (permalink / raw To: help-gnu-emacs On 2015-08-28 01:34 +0200, Suvayu Ali wrote: > > Because otherwise, you would have Gnus and offlineimap competing for > > access ? > I do not think this is possible. In fact AFAIK, any kind of race > condition in a maildir is not possible by design, as long as the > filesystem operations are atomic. I think this applies to the original delivery, but not necessarily to subsequent manipulation of maildirs. For example, the message flags are stored by appending them to the file name. If two MUAs concurrently scan the maildir for messages, then try to set a flag on one message, a race is possible. I observed wierdness when I tried to access my mail both by IMAP (the with Dovecot as the IMAP server) and directly reading the underlying maildirs with mutt. That was part of the reason why I gave up on IMAP entirely, and instead read over ssh and mutt locally on the server. -- Please *no* private copies of mailing list or newsgroup messages. Rule 420: All persons more than eight miles high to leave the court. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 23:50 ` Ian Zimmerman @ 2015-08-28 0:14 ` Suvayu Ali 2015-08-28 0:24 ` Ian Zimmerman 0 siblings, 1 reply; 41+ messages in thread From: Suvayu Ali @ 2015-08-28 0:14 UTC (permalink / raw To: help-gnu-emacs On Thu, Aug 27, 2015 at 04:50:23PM -0700, Ian Zimmerman wrote: > On 2015-08-28 01:34 +0200, Suvayu Ali wrote: > > > > Because otherwise, you would have Gnus and offlineimap competing for > > > access ? > > > I do not think this is possible. In fact AFAIK, any kind of race > > condition in a maildir is not possible by design, as long as the > > filesystem operations are atomic. > > I think this applies to the original delivery, but not necessarily to > subsequent manipulation of maildirs. > > For example, the message flags are stored by appending them to the file > name. If two MUAs concurrently scan the maildir for messages, then try > to set a flag on one message, a race is possible. > > I observed wierdness when I tried to access my mail both by IMAP (the > with Dovecot as the IMAP server) and directly reading the underlying > maildirs with mutt. That was part of the reason why I gave up on IMAP > entirely, and instead read over ssh and mutt locally on the server. No, it's not. AFAIK on most (all?) *nix filesystems, basic operations are atomic. See for example this (outdated) list: http://rcrowley.org/2010/01/06/things-unix-can-do-atomically.html E.g., this breaks down when you talk about network mounts like NFS (not sure about SSHFS), but it is a valid assumption as long as your storage is local. -- Suvayu Open source is the future. It sets us free. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 0:14 ` Suvayu Ali @ 2015-08-28 0:24 ` Ian Zimmerman 2015-08-28 8:25 ` Rainer M Krug 2015-08-28 9:06 ` Suvayu Ali 0 siblings, 2 replies; 41+ messages in thread From: Ian Zimmerman @ 2015-08-28 0:24 UTC (permalink / raw To: help-gnu-emacs On 2015-08-28 02:14 +0200, Suvayu Ali wrote: > No, it's not. AFAIK on most (all?) *nix filesystems, basic operations > are atomic. See for example this (outdated) list: > > http://rcrowley.org/2010/01/06/things-unix-can-do-atomically.html > > E.g., this breaks down when you talk about network mounts like NFS (not > sure about SSHFS), but it is a valid assumption as long as your storage > is local. Assume there's a message ~/Mail/inbox/cur/1440718280.10956_2.ahiker:2,S MUA 1 wants to set the "tagged" flag, which means renaming 1440718280.10956_2.ahiker:2,S -> 1440718280.10956_2.ahiker:2,FS MUA 2 wants to set the "replied" flag, which means renaming 1440718280.10956_2.ahiker:2,S -> 1440718280.10956_2.ahiker:2,RS Only one of them can succeed, depending on the order they try. And final state also depends on that order. -- Please *no* private copies of mailing list or newsgroup messages. Rule 420: All persons more than eight miles high to leave the court. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 0:24 ` Ian Zimmerman @ 2015-08-28 8:25 ` Rainer M Krug 2015-08-28 9:06 ` Suvayu Ali 1 sibling, 0 replies; 41+ messages in thread From: Rainer M Krug @ 2015-08-28 8:25 UTC (permalink / raw To: Ian Zimmerman; +Cc: help-gnu-emacs [-- Attachment #1: Type: text/plain, Size: 1446 bytes --] Ian Zimmerman <itz@buug.org> writes: > On 2015-08-28 02:14 +0200, Suvayu Ali wrote: > >> No, it's not. AFAIK on most (all?) *nix filesystems, basic operations >> are atomic. See for example this (outdated) list: >> >> http://rcrowley.org/2010/01/06/things-unix-can-do-atomically.html >> >> E.g., this breaks down when you talk about network mounts like NFS (not >> sure about SSHFS), but it is a valid assumption as long as your storage >> is local. > > Assume there's a message ~/Mail/inbox/cur/1440718280.10956_2.ahiker:2,S > > MUA 1 wants to set the "tagged" flag, which means renaming > > 1440718280.10956_2.ahiker:2,S -> 1440718280.10956_2.ahiker:2,FS > > MUA 2 wants to set the "replied" flag, which means renaming > > 1440718280.10956_2.ahiker:2,S -> 1440718280.10956_2.ahiker:2,RS > > Only one of them can succeed, depending on the order they try. And > final state also depends on that order. Interesting discussion - and I am learning that I do not know enough to say that there *can* be race conditions, but I know that they are *bad* when they occur and can lead to corruption or undefined behavior. And I also know that I do not want this, particularly not in email. So using dovecot to RW access the maildir is the safe way to go. Notmuch is also accessing the maildir directly, but only read-only. Cheers, Rainer -- Rainer M. Krug email: Rainer<at>krugs<dot>de PGP: 0x0F52F982 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 480 bytes --] ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 0:24 ` Ian Zimmerman 2015-08-28 8:25 ` Rainer M Krug @ 2015-08-28 9:06 ` Suvayu Ali 2015-08-28 15:31 ` Ian Zimmerman 1 sibling, 1 reply; 41+ messages in thread From: Suvayu Ali @ 2015-08-28 9:06 UTC (permalink / raw To: help-gnu-emacs On Thu, Aug 27, 2015 at 05:24:33PM -0700, Ian Zimmerman wrote: > On 2015-08-28 02:14 +0200, Suvayu Ali wrote: > > > No, it's not. AFAIK on most (all?) *nix filesystems, basic operations > > are atomic. See for example this (outdated) list: > > > > http://rcrowley.org/2010/01/06/things-unix-can-do-atomically.html > > > > E.g., this breaks down when you talk about network mounts like NFS (not > > sure about SSHFS), but it is a valid assumption as long as your storage > > is local. > > Assume there's a message ~/Mail/inbox/cur/1440718280.10956_2.ahiker:2,S > > MUA 1 wants to set the "tagged" flag, which means renaming > > 1440718280.10956_2.ahiker:2,S -> 1440718280.10956_2.ahiker:2,FS > > MUA 2 wants to set the "replied" flag, which means renaming > > 1440718280.10956_2.ahiker:2,S -> 1440718280.10956_2.ahiker:2,RS > > Only one of them can succeed, depending on the order they try. And > final state also depends on that order. I think you are confusing failures on the MUA side with race conditions leading to email corruption. Take your example above, when MUA 1 flags a message, and succeeds, atomicity of filesystem operation ensures setting the reply flag by MUA 2 will fail. It is up to MUA 2 to handle this failure. The maildir is guaranteed to be in a consistent state by the atomicity of filesystem operations. When MUA 2 fails, it is easily handled by reporting to the user and not trying to commit the changes again. E.g. in mutt, this is handled by telling the user something like: file does not exist (I don't recall the exact phrasing), and keeping the folder state as is. The user now has to do two things, mark the folder read-only (so mutt stops trying to write state changes back to the maildir), and reread the present maildir. During all this, your maildir has _not_ been corrupted in any way, and both MUAs are reporting things correctly. It is an entirely different story however if your maildir is over NFS (and possibly SSHFS, actually maybe this is true for any network based filesystem). In that case, say the MUAs are accessing the maildir from different locations. MUA 1 writes the "flagging" action, the NFS client commits the change. However the real files on disk have not been changed, and when MUA 2 tries to write its "replying" action, it succeeds! Now you have a race condition, MUA 1 thinks the message is flagged, MUA 2 thinks it's read. And no one knows what it is in reality other than the NFS daemon. Technically the email is not corrupted, only the meta information about the flag is. This is easily recoverable by quiting both MUAs. And essentially this is the beauty of the maildir format, data integrity. Hopefully I have explained myself clearly enough. Or maybe you were trying to say something else? In that case please feel free to stop me. Cheers, -- Suvayu Open source is the future. It sets us free. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 9:06 ` Suvayu Ali @ 2015-08-28 15:31 ` Ian Zimmerman 2015-08-31 0:16 ` Suvayu Ali 0 siblings, 1 reply; 41+ messages in thread From: Ian Zimmerman @ 2015-08-28 15:31 UTC (permalink / raw To: help-gnu-emacs On 2015-08-28 11:06 +0200, Suvayu Ali wrote: > And essentially this is the beauty of the maildir > format, data integrity. I agree, but that is not the same as absence of races. See below. > Hopefully I have explained myself clearly enough. Or maybe you were > trying to say something else? In that case please feel free to stop me. We have different definitions of "race". To me, corruption is not a necessary ingredient; a race is just a situation where two (or more) threads or processes mutate data in ways that are independent (ie. commutative) in the abstract, and yet the final state of the data depends on the order of the changes. A race can even be completely harmless if the two concrete representations of the final state in fact represent the same abstract state. An extreme example of that would be a log structured system, where the difference is simply in the order of the log records left by the changes. Yes, this is a race by my definition; but it's a feature, not a bug :-) > When MUA 2 fails, it is easily handled by reporting to the user and not > trying to commit the changes again. E.g. in mutt, this is handled by > telling the user something like: file does not exist (I don't recall the > exact phrasing), and keeping the folder state as is. I don't know about you, but if I got this blurb from my MUA I would first stare at the screen for minutes, repeating "wtf wtf", then (if I was having a good day) investigate, and finally mutate the state of my computer to ensure exclusive access :-) IOW, I don't want such weirdness to happen, whether I call it "race" or not. Lastly, it is not even clear what we should consider "corruption", or the absence of it. Is internal consistency enough? Consider the classic motivating example for database transactions, with two threads trying to update an account balance datum, and only one succeeding. Let's say there's no separate transactions table (a stretch, I know). Then the data after the mixup is internally consistent; there just aren't many ways that a single numerical value can be incosistent :-) But it doesn't reflect reality, so maybe it is corrupt? And to close the circle, this example in fact has analogy with our maildir situation: the MUA trying to set the "Replied" flag presumably just sent off a reply. So the failure to set the flag means the maildir state, while internally consistent, doesn't reflect reality. -- Please *no* private copies of mailing list or newsgroup messages. Rule 420: All persons more than eight miles high to leave the court. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-28 15:31 ` Ian Zimmerman @ 2015-08-31 0:16 ` Suvayu Ali 0 siblings, 0 replies; 41+ messages in thread From: Suvayu Ali @ 2015-08-31 0:16 UTC (permalink / raw To: help-gnu-emacs On Fri, Aug 28, 2015 at 08:31:26AM -0700, Ian Zimmerman wrote: > On 2015-08-28 11:06 +0200, Suvayu Ali wrote: > > > When MUA 2 fails, it is easily handled by reporting to the user and not > > trying to commit the changes again. E.g. in mutt, this is handled by > > telling the user something like: file does not exist (I don't recall the > > exact phrasing), and keeping the folder state as is. > > I don't know about you, but if I got this blurb from my MUA I would > first stare at the screen for minutes, repeating "wtf wtf", then (if I > was having a good day) investigate, and finally mutate the state of my > computer to ensure exclusive access :-) IOW, I don't want such weirdness > to happen, whether I call it "race" or not. Okay, I found the error message, it's cryptic indeed. rename: No such file or directory (errno = 2) IIRC, I did understand it quite immediately when I saw it the first time. But maybe it was easy for me because I was quite familiar with the maildir format, having dealt with OfflineIMAP and I had been experimenting with Gmail's IMAP implementation to understand how it works. Having said all that, I do not think there is any easy solution to this issue. This is inherrent in how we are approaching the problem the moment we choose a solution that relies on syncing rather than being live (IMAP access). > And to close the circle, this example in fact has analogy with our > maildir situation: the MUA trying to set the "Replied" flag presumably > just sent off a reply. So the failure to set the flag means the maildir > state, while internally consistent, doesn't reflect reality. This is a very interesting case indeed. However I do not see any possible solutions as long as we choose to sync rather than use IMAP. In this particular case, the inconsistent flags will be corrected on the next sync, since the remote server has the correct flags. Cheers, -- Suvayu Open source is the future. It sets us free. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 21:27 ` Julien Cubizolles 2015-08-27 23:34 ` Suvayu Ali @ 2015-08-28 8:21 ` Rainer M Krug 1 sibling, 0 replies; 41+ messages in thread From: Rainer M Krug @ 2015-08-28 8:21 UTC (permalink / raw To: Julien Cubizolles; +Cc: help-gnu-emacs [-- Attachment #1: Type: text/plain, Size: 2213 bytes --] Julien Cubizolles <j.cubizolles@free.fr> writes: > Rainer M Krug <Rainer@krugs.de> writes: > > >> There is one program accessing the mail folder: dovecot - no conflicts, race >> conditions, no slightly different formats, standards, ... And you can >> also remotely access dovecot if you want to. > > Because otherwise, you would have Gnus and offlineimap competing for > access ? I have not actually experienced this, but I was afraid tat, especially simultaneous write access to the mailbox files cal lead to corruption or errors due to simultaneous access. > >> By the way: the dovecot daemon is not permanently running, as I am using >> the shell access from gnus: >> >> ,---- >> | (nnimap-shell-program "/usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir"))) >> `---- >> >> The same from offlineimap: >> >> ,---- >> | preauthtunnel = /usr/local/opt/dovecot/libexec/dovecot/imap -o mail_location=maildir:$HOME/Maildir >> `---- >> >> and offlineimap only reads the emails from the maildir. > > That looks interesting, I'll try it. > >> And I have found the following two functions to open the same message >> which is open in gnus in notmuch and the other way around: >> >> ** In notmuch jump to gnus folder > > I think you forgot the other way around :-) Sorry - here is it: ,---- | ** In gnus jump to notmuch | Thanks to [[notmuch:id:87fvn4j45h.fsf@thinkpad-t61.fritz.box][Email from Tassilo Horn: Re: How to get message-id from]] | #+begin_src emacs-lisp | (defun rmk/gnus-goto-message-in-notmuch () | (interactive) | (unless (buffer-live-p gnus-summary-buffer) | (user-error "No open Gnus summary buffer!")) | (let ((message-id (substring (with-current-buffer gnus-summary-buffer | (mail-header-id (gnus-summary-article-header))) 1 -1))) | (message message-id) | (notmuch-tree (concatenate 'string "id:" message-id)) ;; Now do the notmuch stuff | )) | #+end_src `---- > >> Hope this helps, > > I'm sure it will, thanks a lot. Great. Cheers, Rainer > > Julien. > > -- Rainer M. Krug email: Rainer<at>krugs<dot>de PGP: 0x0F52F982 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 480 bytes --] ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-26 7:22 ` Julien Cubizolles 2015-08-27 12:50 ` Rainer M Krug @ 2015-08-27 12:58 ` Suvayu Ali 2015-08-27 15:52 ` Glenn Morris 2015-08-27 17:11 ` Bastien 2015-08-27 17:08 ` Bastien 2 siblings, 2 replies; 41+ messages in thread From: Suvayu Ali @ 2015-08-27 12:58 UTC (permalink / raw To: help-gnu-emacs On Wed, Aug 26, 2015 at 09:22:26AM +0200, Julien Cubizolles wrote: > > I have a few questions about some details of your setup: > > + regarding filtering and spam detection, do you rely on the > possibilities offered by you mail provider (Gmail or other)? I don't know about Bastien, but for me, since I use Gmail I heavily rely on both filtering and spam detection. I use Gmails filtering eventhough it is a bit clunky to avoid setting up something on my end; I find it simpler this way. And the spam detection, I find Gmail's spam detection to be very good, specially when you have been training it for a few years by reporting spam and phishing messages, as appropriate. > + you said you're running Dovecot. Gnus could also directly access the > local maildir without a local imap server the way mu4e does, couldn't > it ? What's the benefit of running it ? Probably Bastien has more up to date comments, but last time I looked, I found Gnus's support for maildir rather poor. There were a few issues, it didn't respect the usual maildir flags and stored its own flags withing the maildir. So along with incorrect flag, the mail store took about ~70% more disk space. It was also a bit slow, despite everything being local. My impression is based on the status from about 2-3 years back, so things might have improved a lot by now. Hope this helps, -- Suvayu Open source is the future. It sets us free. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 12:58 ` Suvayu Ali @ 2015-08-27 15:52 ` Glenn Morris 2015-08-27 17:32 ` Suvayu Ali 2015-08-27 17:11 ` Bastien 1 sibling, 1 reply; 41+ messages in thread From: Glenn Morris @ 2015-08-27 15:52 UTC (permalink / raw To: help-gnu-emacs Suvayu Ali wrote: > Probably Bastien has more up to date comments, but last time I looked, I > found Gnus's support for maildir rather poor. There were a few issues, > it didn't respect the usual maildir flags and stored its own flags > withing the maildir. So along with incorrect flag, the mail store took > about ~70% more disk space. It was also a bit slow, despite everything > being local. My impression is based on the status from about 2-3 years > back, so things might have improved a lot by now. Flag handling at least is better since Emacs 24.3 (http://debbugs.gnu.org/8055). ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 15:52 ` Glenn Morris @ 2015-08-27 17:32 ` Suvayu Ali 0 siblings, 0 replies; 41+ messages in thread From: Suvayu Ali @ 2015-08-27 17:32 UTC (permalink / raw To: help-gnu-emacs On Thu, Aug 27, 2015 at 11:52:05AM -0400, Glenn Morris wrote: > Suvayu Ali wrote: > > > Probably Bastien has more up to date comments, but last time I looked, I > > found Gnus's support for maildir rather poor. There were a few issues, > > it didn't respect the usual maildir flags and stored its own flags > > withing the maildir. So along with incorrect flag, the mail store took > > about ~70% more disk space. It was also a bit slow, despite everything > > being local. My impression is based on the status from about 2-3 years > > back, so things might have improved a lot by now. > > Flag handling at least is better since Emacs 24.3 > (http://debbugs.gnu.org/8055). > Thanks Glenn. Maybe one of these days I'll end up experimenting again. :) -- Suvayu Open source is the future. It sets us free. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 12:58 ` Suvayu Ali 2015-08-27 15:52 ` Glenn Morris @ 2015-08-27 17:11 ` Bastien 1 sibling, 0 replies; 41+ messages in thread From: Bastien @ 2015-08-27 17:11 UTC (permalink / raw To: Suvayu Ali; +Cc: help-gnu-emacs Hi Suvayu, Suvayu Ali <fatkasuvayu+linux@gmail.com> writes: >> + you said you're running Dovecot. Gnus could also directly access the >> local maildir without a local imap server the way mu4e does, couldn't >> it ? What's the benefit of running it ? > > Probably Bastien has more up to date comments, but last time I looked, I > found Gnus's support for maildir rather poor. There were a few issues, > it didn't respect the usual maildir flags and stored its own flags > withing the maildir. So along with incorrect flag, the mail store took > about ~70% more disk space. It was also a bit slow, despite everything > being local. My impression is based on the status from about 2-3 years > back, so things might have improved a lot by now. I find it reasonably OK right now, but this is very subjective of course. -- Bastien ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-26 7:22 ` Julien Cubizolles 2015-08-27 12:50 ` Rainer M Krug 2015-08-27 12:58 ` Suvayu Ali @ 2015-08-27 17:08 ` Bastien 2015-08-27 17:30 ` Suvayu Ali 2015-08-27 21:52 ` Julien Cubizolles 2 siblings, 2 replies; 41+ messages in thread From: Bastien @ 2015-08-27 17:08 UTC (permalink / raw To: Julien Cubizolles; +Cc: help-gnu-emacs Julien Cubizolles <j.cubizolles@free.fr> writes: > + regarding filtering and spam detection, do you rely on the > possibilities offered by you mail provider (Gmail or other)? For providers that do spam filtering, obviously yes. For others, I first fetch emails on a server using fetchmail/spamassassin, then expose these emails using dovecot on the server. > > + you said you're running Dovecot. Gnus could also directly access the > local maildir without a local imap server the way mu4e does, couldn't > it ? What's the benefit of running it ? Sorry I wasn't clear. I have 1. an address from my internet provider 2. an address from my own remote server multiple addresses are collected on 2 (using fetchmail/spamassassin) and dovecot is running on this remote server. I use maildir for both 1 and 2 on my local machine, sync'ing all mails with mbsync. >> - postfix with smtp_sender_dependent_authentication to use various >> smtp servers depending on the email I'm using. I see people use >> msmtp and msmtp-mta, which I tried for a while, but I'm just too >> impatient with the .3s you have to wait when sending. > > From looking at my fossilized .gnus.el, I'm using smtpmail, which > would be equivalent to msmtp. > > + I'm gonna need to duplicate this setup on two machines (laptop and > desktop). I guess running offline imap on the two machines and > synchronizing the nnml, nnarchive mail by unison would work. I guess so, yes. > Thanks a lot for all this information. I've a much clearer idea of all > the possibilities now. You're welcome! PS: I secretly love anything related to emails, this is fascinating. -- Bastien ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 17:08 ` Bastien @ 2015-08-27 17:30 ` Suvayu Ali 2015-08-27 21:52 ` Julien Cubizolles 1 sibling, 0 replies; 41+ messages in thread From: Suvayu Ali @ 2015-08-27 17:30 UTC (permalink / raw To: help-gnu-emacs On Thu, Aug 27, 2015 at 07:08:47PM +0200, Bastien wrote: > > PS: I secretly love anything related to emails, this is fascinating. Me too! -- Suvayu Open source is the future. It sets us free. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 17:08 ` Bastien 2015-08-27 17:30 ` Suvayu Ali @ 2015-08-27 21:52 ` Julien Cubizolles 2015-08-27 22:18 ` Bastien 1 sibling, 1 reply; 41+ messages in thread From: Julien Cubizolles @ 2015-08-27 21:52 UTC (permalink / raw To: help-gnu-emacs Bastien <bzg@gnu.org> writes: > Sorry I wasn't clear. > > I have > > 1. an address from my internet provider > 2. an address from my own remote server > > multiple addresses are collected on 2 (using fetchmail/spamassassin) > and dovecot is running on this remote server. > > I use maildir for both 1 and 2 on my local machine, sync'ing all > mails with mbsync. And you access the maildir on your local machine directly with nnimap from Gnus, without dovecot ? > You're welcome! > > PS: I secretly love anything related to emails, this is fascinating. I can see that ? As for me, I'm starting to be a little bit frightened by all I'm gonna have to get running. So far I'm collecting info, and will put it together when I have a few days free. This list has been, as always a source of precious help, thanks a lot. Julien. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 21:52 ` Julien Cubizolles @ 2015-08-27 22:18 ` Bastien 2015-08-27 23:30 ` Julien Cubizolles 0 siblings, 1 reply; 41+ messages in thread From: Bastien @ 2015-08-27 22:18 UTC (permalink / raw To: Julien Cubizolles; +Cc: help-gnu-emacs Julien Cubizolles <j.cubizolles@free.fr> writes: > And you access the maildir on your local machine directly with nnimap > from Gnus, without dovecot ? No, I use the nnmaildir backend from Gnus directly. -- Bastien ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: From Gnus to mu4e 2015-08-27 22:18 ` Bastien @ 2015-08-27 23:30 ` Julien Cubizolles 0 siblings, 0 replies; 41+ messages in thread From: Julien Cubizolles @ 2015-08-27 23:30 UTC (permalink / raw To: Bastien; +Cc: help-gnu-emacs Bastien <bzg@gnu.org> writes: > Julien Cubizolles <j.cubizolles@free.fr> writes: > >> And you access the maildir on your local machine directly with nnimap >> from Gnus, without dovecot ? > > No, I use the nnmaildir backend from Gnus directly. My bad, I meant nnmaildir of course. ^ permalink raw reply [flat|nested] 41+ messages in thread
end of thread, other threads:[~2015-08-31 0:16 UTC | newest] Thread overview: 41+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- [not found] <mailman.14.1440105837.31004.help-gnu-emacs@gnu.org> 2015-08-24 11:19 ` From Gnus to mu4e Joost Kremers 2015-08-20 21:23 Julien Cubizolles 2015-08-21 10:28 ` Julien Cubizolles 2015-08-21 10:49 ` Colin Yates 2015-08-21 13:06 ` Suvayu Ali 2015-08-26 6:53 ` Julien Cubizolles 2015-08-25 5:49 ` Bastien 2015-08-25 6:28 ` Ian Zimmerman 2015-08-25 6:56 ` Bastien Guerry 2015-08-26 7:22 ` Julien Cubizolles 2015-08-27 12:50 ` Rainer M Krug 2015-08-27 17:15 ` Bastien 2015-08-28 8:39 ` Rainer M Krug 2015-08-28 8:44 ` Bastien 2015-08-28 8:51 ` Rainer M Krug 2015-08-28 8:58 ` Rasmus 2015-08-28 9:15 ` Julien Cubizolles 2015-08-28 10:03 ` Rainer M Krug 2015-08-28 10:08 ` Rasmus 2015-08-28 10:41 ` Rainer M Krug 2015-08-28 10:04 ` Rasmus 2015-08-28 8:51 ` Bastien 2015-08-27 21:27 ` Julien Cubizolles 2015-08-27 23:34 ` Suvayu Ali 2015-08-27 23:50 ` Ian Zimmerman 2015-08-28 0:14 ` Suvayu Ali 2015-08-28 0:24 ` Ian Zimmerman 2015-08-28 8:25 ` Rainer M Krug 2015-08-28 9:06 ` Suvayu Ali 2015-08-28 15:31 ` Ian Zimmerman 2015-08-31 0:16 ` Suvayu Ali 2015-08-28 8:21 ` Rainer M Krug 2015-08-27 12:58 ` Suvayu Ali 2015-08-27 15:52 ` Glenn Morris 2015-08-27 17:32 ` Suvayu Ali 2015-08-27 17:11 ` Bastien 2015-08-27 17:08 ` Bastien 2015-08-27 17:30 ` Suvayu Ali 2015-08-27 21:52 ` Julien Cubizolles 2015-08-27 22:18 ` Bastien 2015-08-27 23:30 ` Julien Cubizolles
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.