[PATCH] [debbugs] Add mu4e support.

[PATCH] [debbugs] Add mu4e support.

[Ricardo Wurmus]

[-- Attachment #1.1: Type: text/plain, Size: 648 bytes --]

Hi,

this patch is for the debbugs.el EPLA package.  It adds the mail client
“mu4e” as a debbugs-gnu mail backend.

When “debbugs-gnu-mail-backend” is set to “'mu4e”, visiting a bug will
download the bug’s mbox file, turn it into a separate Maildir tree under
“mu4e-maildir” called “emacs-debbugs.1234” where “1234” is the bug id,
runs “mu index” (synchronously) to update the mail database, and then
shows the emails as usual with mu4e.

I’ve been using this patch for a while with satisfactory results.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net



[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-debbugs-Add-mu4e-support.patch --]
[-- Type: text/x-patch, Size: 4773 bytes --]

From c841c4ad0b77aad0b06a31915d46a7cefafc2058 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Mon, 10 Apr 2017 09:33:51 +0200
Subject: [PATCH] [debbugs] Add mu4e support.

* packages/debbugs/debbugs-gnu.el (debbugs-gnu-mail-backend,
debbugs-gnu-select-report): Handle "mu4e" option.
(debbugs--extract-mbox-to-maildir, debbugs-read-emacs-bug-with-mu4e):
New procedures.
---
 packages/debbugs/debbugs-gnu.el | 65 +++++++++++++++++++++++++++++++++++++++--
 1 file changed, 62 insertions(+), 3 deletions(-)

diff --git a/packages/debbugs/debbugs-gnu.el b/packages/debbugs/debbugs-gnu.el
index ebfa738a4..aea0ddd86 100644
--- a/packages/debbugs/debbugs-gnu.el
+++ b/packages/debbugs/debbugs-gnu.el
@@ -170,9 +170,13 @@
 (autoload 'mail-header-subject "nnheader")
 (autoload 'message-goto-body "message")
 (autoload 'message-make-from "message")
+(autoload 'message-fetch-field "message")
+(autoload 'nndoc-mbox-article-begin "nndoc")
+(autoload 'nnmail-write-region "nnmail")
 (autoload 'rmail-get-new-mail "rmail")
 (autoload 'rmail-show-message "rmail")
 (autoload 'rmail-summary "rmailsum")
+(autoload 'mu4e-headers-search "mu4e-headers")
 (autoload 'vc-dir-hide-up-to-date "vc-dir")
 (autoload 'vc-dir-mark "vc-dir")
 
@@ -293,11 +297,12 @@ suppressed bugs is toggled by `debbugs-gnu-toggle-suppress'."
 
 (defcustom debbugs-gnu-mail-backend 'gnus
   "The email backend to use for reading bug report email exchange.
-If this is `gnus', the default, use Gnus.
-If this is `rmail', use Rmail instead."
+If this is `gnus', the default, use Gnus.  If this is `rmail',
+use Rmail.  If this is `mu4e', use Mu4e instead."
   :group 'debbugs-gnu
   :type '(radio (function-item :tag "Use Gnus" gnus)
-		(function-item :tag "Use Rmail" rmail))
+                (function-item :tag "Use Rmail" rmail)
+                (function-item :tag "Use Mu4e" mu4e))
   :version "25.1")
 
 (defface debbugs-gnu-archived '((t (:inverse-video t)))
@@ -1329,6 +1334,58 @@ MERGED is the list of bugs merged with this one."
     (define-key rmail-mode-map "C" 'debbugs-gnu-send-control-message)
     (rmail-show-message 1)))
 
+(defun debbugs--extract-mbox-to-maildir (bug-str maildir start end)
+  (let* ((msgid (save-restriction
+                  (narrow-to-region start end)
+                  (message-fetch-field "Message-ID")))
+         (file  (concat bug-str "-" (secure-hash 'md5 msgid) ":2,"))
+         (path  (concat maildir "/cur/" file)))
+    (when (not (file-exists-p path))
+      (nnmail-write-region start end path))))
+
+(defun debbugs-read-emacs-bug-with-mu4e (id status merged)
+  "Read email exchange for debbugs bug ID.
+STATUS is the bug's status list.
+MERGED is the list of bugs merged with this one."
+  (let* ((bug-str (number-to-string id))
+         (tmpdir  "/tmp")
+         (mbox    (concat tmpdir "/bug-" bug-str ".mbox"))
+         (subdir  (concat "/emacs-debbugs." bug-str))
+         (maildir (concat mu4e-maildir subdir)))
+    ;; Create temporary maildir structure
+    (dolist (dir '("cur" "new" "tmp"))
+      (make-directory (concat maildir "/" dir) t))
+
+    ;; Download mbox for the given bug
+    (debbugs-get-mbox id 'mboxmaint mbox)
+
+    ;; Split mbox into individual mail files
+    (with-temp-buffer
+      (insert-file-contents mbox)
+      (goto-char (point-min))
+      (setq start (point))
+      (forward-char)
+      (while (nndoc-mbox-article-begin)
+        (setq end (point))
+        (debbugs--extract-mbox-to-maildir bug-str maildir start end)
+        (setq start end)
+        (forward-char))
+      ;; Save last email
+      (debbugs--extract-mbox-to-maildir bug-str maildir start (point-max)))
+
+    ;; Clean up
+    (delete-file mbox)
+
+    ;; We cannot use mu4e-update-index because it runs
+    ;; asynchronously.
+    (call-process-shell-command
+     (concat "mu index -m " mu4e-maildir))
+    (with-current-buffer (window-buffer (selected-window))
+      (let ((msgid (substring (cdr (assq 'msgid status)) 1 -1)))
+        (message msgid)
+        (mu4e-headers-search (concat "maildir:" subdir)
+                             nil nil nil msgid nil)))))
+
 (defun debbugs-read-emacs-bug-with-gnus (id status merged)
   "Read email exchange for debbugs bug ID.
 STATUS is the bug's status list.
@@ -1361,6 +1418,8 @@ MERGED is the list of bugs merged with this one."
       (message "No bug report on the current line"))
      ((eq debbugs-gnu-mail-backend 'rmail)
       (debbugs-read-emacs-bug-with-rmail id status merged))
+     ((eq debbugs-gnu-mail-backend 'mu4e)
+      (debbugs-read-emacs-bug-with-mu4e id status merged))
      ((eq debbugs-gnu-mail-backend 'gnus)
       (debbugs-read-emacs-bug-with-gnus id status merged))
      (t (error "No valid mail backend specified")))))
-- 
2.12.2

Re: [PATCH] [debbugs] Add mu4e support.

[Marcin Borkowski]

On 2017-04-23, at 12:05, Ricardo Wurmus <rekado@elephly.net> wrote:

> Hi,
>
> this patch is for the debbugs.el EPLA package.  It adds the mail client
> “mu4e” as a debbugs-gnu mail backend.
>
> When “debbugs-gnu-mail-backend” is set to “'mu4e”, visiting a bug will
> download the bug’s mbox file, turn it into a separate Maildir tree under
> “mu4e-maildir” called “emacs-debbugs.1234” where “1234” is the bug id,
> runs “mu index” (synchronously) to update the mail database, and then
> shows the emails as usual with mu4e.
>
> I’ve been using this patch for a while with satisfactory results.

That is awesome!  I hope to be able to test it some time soon.

Thanks,

-- 
Marcin Borkowski

Re: [PATCH] [debbugs] Add mu4e support.

[Michael Albinus]

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi,

Hi Ricardo,

> this patch is for the debbugs.el EPLA package.  It adds the mail client
> “mu4e” as a debbugs-gnu mail backend.

Thanks for this!

I hope I'll have time next week to review and test it. If this goes
well, I'm ready to commit your patch.

In the meantime, you might add also some few words to debbugs-ug.texi.

> Ricardo

Best regards, Michael.

Re: [PATCH] [debbugs] Add mu4e support.

[Michael Albinus]

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi,

Hi Ricardo,

> this patch is for the debbugs.el EPLA package.  It adds the mail client
> “mu4e” as a debbugs-gnu mail backend.
>
> When “debbugs-gnu-mail-backend” is set to “'mu4e”, visiting a bug will
> download the bug’s mbox file, turn it into a separate Maildir tree under
> “mu4e-maildir” called “emacs-debbugs.1234” where “1234” is the bug id,
> runs “mu index” (synchronously) to update the mail database, and then
> shows the emails as usual with mu4e.
>
> I’ve been using this patch for a while with satisfactory results.

I've played with this. First of all, I had to install mu 0.9.9.5 from
the source tarball; I didn't find it as package. I've compiled it, there
were errors in compiling mu and warnings in compiling mu4e. Well, I
don't care too much; I don't want to use it myself.

After applying your patch to debbugs-gnu.el, there are further warnings:

--8<---------------cut here---------------start------------->8---
In toplevel form:
packages/debbugs/debbugs-gnu.el:1346:1:Warning: Unused lexical argument
    ‘merged’

In debbugs-read-emacs-bug-with-mu4e:
packages/debbugs/debbugs-gnu.el:1354:27:Warning: reference to free variable
    ‘mu4e-maildir’
packages/debbugs/debbugs-gnu.el:1366:13:Warning: assignment to free variable
    ‘start’
packages/debbugs/debbugs-gnu.el:1369:15:Warning: assignment to free variable
    ‘end’
packages/debbugs/debbugs-gnu.el:1374:57:Warning: reference to free variable
    ‘start’
packages/debbugs/debbugs-gnu.el:1371:21:Warning: reference to free variable
    ‘end’
--8<---------------cut here---------------end--------------->8---

And indeed, after setting debbugs-gnu-mail-backend to 'mu4e, I've got
the error

debbugs-gnu-select-report: Symbol’s value as variable is void: mu4e-maildir

So I've loaded mu4e.el. Now I get the error

debbugs-read-emacs-bug-with-mu4e: Wrong number of arguments: #[(&optional expr prompt edit ignore-history) "Ä\b

The rest of the backtrace cannot be added to this email due to non-ASCII
characters.

Some comments to your code:

> +(defun debbugs-read-emacs-bug-with-mu4e (id status merged)
> +  "Read email exchange for debbugs bug ID.
> +STATUS is the bug's status list.
> +MERGED is the list of bugs merged with this one."
> +  (let* ((bug-str (number-to-string id))
> +         (tmpdir  "/tmp")

Pls use temporary-file-directory.

> +         (mbox    (concat tmpdir "/bug-" bug-str ".mbox"))
> +         (subdir  (concat "/emacs-debbugs." bug-str))
> +         (maildir (concat mu4e-maildir subdir)))
> +    ;; Create temporary maildir structure
> +    (dolist (dir '("cur" "new" "tmp"))
> +      (make-directory (concat maildir "/" dir) t))

Please use expand-file-name where appropriate.

Could you, pls, try to fix these problems? And don't forget the
contribution to debbugs-ug.texi.

> Ricardo

Best regards, Michael.

Re: [PATCH] [debbugs] Add mu4e support.

[Stefan Monnier]

>> +(defun debbugs-read-emacs-bug-with-mu4e (id status merged)
>> +  "Read email exchange for debbugs bug ID.
>> +STATUS is the bug's status list.
>> +MERGED is the list of bugs merged with this one."
>> +  (let* ((bug-str (number-to-string id))
>> +         (tmpdir  "/tmp")

> Pls use temporary-file-directory.

>> +         (mbox    (concat tmpdir "/bug-" bug-str ".mbox"))

Actually, don't use temporary-file-directory but instead use
make-temp-file (which will use temporary-file-directory internally), to
avoid security hazards.


        Stefan

Re: [PATCH] [debbugs] Add mu4e support.

[Ricardo Wurmus]

Hi Michael,

> I've played with this. First of all, I had to install mu 0.9.9.5 from
> the source tarball; I didn't find it as package. I've compiled it, there
> were errors in compiling mu and warnings in compiling mu4e. Well, I
> don't care too much; I don't want to use it myself.

I’m using mu 0.9.18 from Guix.

> After applying your patch to debbugs-gnu.el, there are further warnings:
>
> --8<---------------cut here---------------start------------->8---
> In toplevel form:
> packages/debbugs/debbugs-gnu.el:1346:1:Warning: Unused lexical argument
>     ‘merged’

That’s because “debbugs-read-emacs-bug-with-mu4e” doesn't yet handle
merged bugs.  It will only download messages for the given bug id.

> In debbugs-read-emacs-bug-with-mu4e:
> packages/debbugs/debbugs-gnu.el:1354:27:Warning: reference to free variable
>     ‘mu4e-maildir’

This is defined in mu4e-vars.  How would I mark this as coming from
there without adding a dependency on mu4e?

> packages/debbugs/debbugs-gnu.el:1366:13:Warning: assignment to free variable
>     ‘start’
> packages/debbugs/debbugs-gnu.el:1369:15:Warning: assignment to free variable
>     ‘end’
> packages/debbugs/debbugs-gnu.el:1374:57:Warning: reference to free variable
>     ‘start’
> packages/debbugs/debbugs-gnu.el:1371:21:Warning: reference to free variable
>     ‘end’

I fixed this by defining these variables in the let binding of
“debbugs-read-emacs-bug-with-mu4e”.

> So I've loaded mu4e.el. Now I get the error
>
> debbugs-read-emacs-bug-with-mu4e: Wrong number of arguments: #[(&optional expr prompt edit ignore-history) "Ä\b
>
> The rest of the backtrace cannot be added to this email due to non-ASCII
> characters.

I cannot reproduce this.  What did you do to get this error?

> Some comments to your code:
>
>> +(defun debbugs-read-emacs-bug-with-mu4e (id status merged)
>> +  "Read email exchange for debbugs bug ID.
>> +STATUS is the bug's status list.
>> +MERGED is the list of bugs merged with this one."
>> +  (let* ((bug-str (number-to-string id))
>> +         (tmpdir  "/tmp")
>
> Pls use temporary-file-directory.

I’ve replaced this with “make-temp-file”, as per Stefan’s suggestion.

>> +         (mbox    (concat tmpdir "/bug-" bug-str ".mbox"))
>> +         (subdir  (concat "/emacs-debbugs." bug-str))
>> +         (maildir (concat mu4e-maildir subdir)))
>> +    ;; Create temporary maildir structure
>> +    (dolist (dir '("cur" "new" "tmp"))
>> +      (make-directory (concat maildir "/" dir) t))
>
> Please use expand-file-name where appropriate.

Okay, I’ve made that change.

> Could you, pls, try to fix these problems? And don't forget the
> contribution to debbugs-ug.texi.

I’ve extended the documentation a little.  Before resending the patch,
I’d like to know more about the “wrong number of arguments” bug you got
with “debbugs-read-emacs-bug-with-mu4e”.

Thanks for taking the time to review this!

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: [PATCH] [debbugs] Add mu4e support.

[Michael Albinus]

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi Michael,

Hi Ricardo,

> That’s because “debbugs-read-emacs-bug-with-mu4e” doesn't yet handle
> merged bugs.  It will only download messages for the given bug id.

Mark this argument as unused, like this:

(defun debbugs-read-emacs-bug-with-mu4e (id status _merged)

>> In debbugs-read-emacs-bug-with-mu4e:
>> packages/debbugs/debbugs-gnu.el:1354:27:Warning: reference to free variable
>>     ‘mu4e-maildir’
>
> This is defined in mu4e-vars.  How would I mark this as coming from
> there without adding a dependency on mu4e?

Declare it w/o specifying a value:

(defvar mu4e-maildir)

>> debbugs-read-emacs-bug-with-mu4e: Wrong number of arguments: #[(&optional expr prompt edit ignore-history) "Ä\b
>>
>> The rest of the backtrace cannot be added to this email due to non-ASCII
>> characters.
>
> I cannot reproduce this.  What did you do to get this error?

# ~/src/emacs/src/emacs -Q

(add-to-list 'load-path "~/src/mu/mu4e/")
(add-to-list 'load-path "~/src/elpa/packages/debbugs/")
(require 'debbugs-gnu)
(require 'mu4e-vars)
(setq debug-on-error t)
(setq debbugs-gnu-mail-backend 'mu4e)
(debbugs-gnu-bugs 10000)
<ENTER>

I've removed all *.elc files from the mu4e directory. The backtrace is now

Debugger entered--Lisp error: (wrong-number-of-arguments (lambda (&optional expr prompt edit ignore-history) "Search in the mu database for EXPR, and switch to the output
buffer for the results. This is an interactive function which ask
user for EXPR. PROMPT, if non-nil, is the prompt used by this
function (default is \"Search for:\"). If EDIT is non-nil, instead
of executing the query for EXPR, let the user edit the query before
executing it. If IGNORE-HISTORY is true, do *not* update the query
history stack." (interactive) (let* ((prompt (mu4e-format (or prompt "Search for: "))) (expr (if edit (read-string prompt expr) (or expr (read-string prompt nil (quote mu4e~headers-search-hist)))))) (mu4e-mark-handle-when-leaving) (mu4e~headers-search-execute expr ignore-history))) 6)
  mu4e-headers-search("maildir:/emacs-debbugs.10000" nil nil nil "84k479sz16.fsf@aol.com" nil)
  debbugs-read-emacs-bug-with-mu4e(10000 ((cache_time . 1494335543.4297807) (tags "notabug") (blockedby) (originator . "nyc4bos@aol.com") (location . "archive") (date . 1320812762) (unarchived) (archived . t) (found_versions "24.0.91") (fixed_versions) (subject . "24.0.91; Gnus gnus-get-function error") (affects) (mergedwith) (package "emacs" "gnus") (found (item (key . "24.0.91") (value))) (msgid . "<84k479sz16.fsf@aol.com>") (fixed) (fixed_date) (blocks) (source . "unknown") (severity . "minor") (pending . "done") (found_date) (last_modified . 1328358242) (log_modified . 1328358242) (keywords "notabug") (summary) (done . "Lars Magne Ingebrigtsen <larsi@gnus.org>") (bug_num . 10000) (forwarded) (owner) (id . 10000)) nil)
  debbugs-gnu-select-report()
  funcall-interactively(debbugs-gnu-select-report)
  call-interactively(debbugs-gnu-select-report nil nil)
  command-execute(debbugs-gnu-select-report)

> Ricardo

Best regards, Michael.

[PATCH] [debbugs] Add mu4e support.

[Felix]

This Thread is now more than 5 years old.
Is there any plan for finishing this?
Or is this obsolete because of another way to use mu4e
for debbugs that i didn't find?

Re: [PATCH] [debbugs] Add mu4e support.

[Eli Zaretskii]

> From: Felix <felix.dick@web.de>
> Cc: emacs-devel <emacs-devel@gnu.org>
> Date: Sat, 24 Dec 2022 15:46:34 +0100
> 
> 
> This Thread is now more than 5 years old.
> Is there any plan for finishing this?
> Or is this obsolete because of another way to use mu4e
> for debbugs that i didn't find?

Please state the bug number.

Re: [PATCH] [debbugs] Add mu4e support.

[Felix]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Felix <felix.dick@web.de>
>> Cc: emacs-devel <emacs-devel@gnu.org>
>> Date: Sat, 24 Dec 2022 15:46:34 +0100
>>
>>
>> This Thread is now more than 5 years old.
>> Is there any plan for finishing this?
>> Or is this obsolete because of another way to use mu4e
>> for debbugs that i didn't find?
>
> Please state the bug number.

I'm not shure if it ever was a reported bug.
I was searching for an option to use mu4e for debbugs,
that's how i found the Thread.
Or did i misunderstand you?

Re: [PATCH] [debbugs] Add mu4e support.

[Eli Zaretskii]

> From: Felix <felix.dick@web.de>
> Cc: rekado@elephly.net, emacs-devel@gnu.org
> Date: Sat, 24 Dec 2022 16:28:12 +0100
> 
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Felix <felix.dick@web.de>
> >> Cc: emacs-devel <emacs-devel@gnu.org>
> >> Date: Sat, 24 Dec 2022 15:46:34 +0100
> >>
> >>
> >> This Thread is now more than 5 years old.
> >> Is there any plan for finishing this?
> >> Or is this obsolete because of another way to use mu4e
> >> for debbugs that i didn't find?
> >
> > Please state the bug number.
> 
> I'm not shure if it ever was a reported bug.
> I was searching for an option to use mu4e for debbugs,
> that's how i found the Thread.
> Or did i misunderstand you?

Then please show the URL in the archives for the discussion thread.  I
couldn't find the thread, as you didn't even tell the date when it
happened.

Re: [PATCH] [debbugs] Add mu4e support.

[Felix]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Felix <felix.dick@web.de>
>> Cc: rekado@elephly.net, emacs-devel@gnu.org
>> Date: Sat, 24 Dec 2022 16:28:12 +0100
>>
>>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>> >> From: Felix <felix.dick@web.de>
>> >> Cc: emacs-devel <emacs-devel@gnu.org>
>> >> Date: Sat, 24 Dec 2022 15:46:34 +0100
>> >>
>> >>
>> >> This Thread is now more than 5 years old.
>> >> Is there any plan for finishing this?
>> >> Or is this obsolete because of another way to use mu4e
>> >> for debbugs that i didn't find?
>> >
>> > Please state the bug number.
>>
>> I'm not shure if it ever was a reported bug.
>> I was searching for an option to use mu4e for debbugs,
>> that's how i found the Thread.
>> Or did i misunderstand you?
>
> Then please show the URL in the archives for the discussion thread.  I
> couldn't find the thread, as you didn't even tell the date when it
> happened.

Sorry!

https://lists.gnu.org/archive/html/emacs-devel/2017-04/msg00662.html

Re: [PATCH] [debbugs] Add mu4e support.

[Eli Zaretskii]

> From: Felix <felix.dick@web.de>
> Cc: rekado@elephly.net, emacs-devel@gnu.org
> Date: Sat, 24 Dec 2022 16:45:37 +0100
> 
> > Then please show the URL in the archives for the discussion thread.  I
> > couldn't find the thread, as you didn't even tell the date when it
> > happened.
> 
> Sorry!
> 
> https://lists.gnu.org/archive/html/emacs-devel/2017-04/msg00662.html

Thanks.

AFAIU, there were some unresolved problems discovered by Michael.  If
someone could work on fixing them, I'm sure Michael will gladly accept
the results.

Re: [PATCH] [debbugs] Add mu4e support.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> AFAIU, there were some unresolved problems discovered by Michael.  If
> someone could work on fixing them, I'm sure Michael will gladly accept
> the results.

Indeed.

Best regards, Michael.