bug#49115: Mumi inserts spurious underscore in bug title

bug#49115: Mumi inserts spurious underscore in bug title

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 488 bytes --]

Guix,

See <http://issues.guix.gnu.org/49114>, which Mumi currently 
titles

  ‘guix_ lint’ should catch certificate validation exceptions

There's no underscore in the original subject.  It's also rendered 
correctly in the grey ‘mail view’ div:

  Tobias Geerinckx-Rice wrote seconds ago
  ‘guix lint’ should catch certificate validation exceptions

Perhaps due to my use of Unicode quotes.  There's nothing 
suspicious in the HTML.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#49115: Mumi inserts spurious underscore in bug title

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 300 bytes --]

Tobias Geerinckx-Rice via Bug reports for GNU Guix 写道：
> It's also rendered correctly in the grey ‘mail view’ div:

Otherwise I'd suspect guile-email but now I'm not so sure.

Debbugs doesn't do this: 
<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49114>.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#49115: Mumi inserts spurious underscore in bug title

[Ricardo Wurmus]

> See http://issues.guix.gnu.org/49114, which Mumi currently 
> titles
>
>   ‘guix_ lint’ should catch certificate validation exceptions
>
> There's no underscore in the original subject.

The debbugs “.log” file for issue 49114 contains this line:

  Subject: bug#49114: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?lint=E2=80=99?= should catch certificate validation exceptions

This is MIME Q encoding (similar to “quoted printable” encoding), as
specified in RFC 2047.

The underscore is mentioned in 4.2 (2):

   (2) The 8-bit hexadecimal value 20 (e.g., ISO-8859-1 SPACE) may be
       represented as "_" (underscore, ASCII 95.).  (This character may
       not pass through some internetwork mail gateways, but its use
       will greatly enhance readability of "Q" encoded data with mail
       readers that do not support this encoding.)  Note that the "_"
       always represents hexadecimal 20, even if the SPACE character
       occupies a different code position in the character set in use.

So, the underscore above ought to be replaced with a space.  This seems
to be a bug in guile-email.

-- 
Ricardo

bug#49115: Mumi inserts spurious underscore in bug title

[Arun Isaac]

Hi Ricardo,

Thanks for the bug report!

It's very strange, but I'm not able to reproduce this. I tried the
following. bug_49114.mbox is downloaded from
https://debbugs.gnu.org/cgi/bugreport.cgi?mbox=yes;bug=49114

--8<---------------cut here---------------start------------->8---
(use-modules (email email))

(pk
 (parse-email-headers "Subject: bug#49114: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?lint=E2=80=99?= should catch certificate validation exceptions
"))

(for-each (lambda (bv)
            (pk (assoc-ref (email-headers (parse-email bv))
                           'subject)))
          (call-with-input-file "bug_49114.mbox"
            mbox->emails))
--8<---------------cut here---------------end--------------->8---

Even at https://issues.guix.gnu.org/49114 , only the "bug title" has the
spurious underscore. The subject of the first message does not. Is the
bug title something stored in the xapian index? Could it be that this
was an older bug that has corrupted the xapian index?

If I understand correctly, mumi does not rebuild its xapian index. I
think it should do so from time to time. It would help prevent old bugs
from getting persisted in storage.

Cheers, and wish you a Happy New Year! :-)
Arun

bug#49115: Mumi inserts spurious underscore in bug title

[Ricardo Wurmus]

Hi Arun,

> Thanks for the bug report!
>
> It's very strange, but I'm not able to reproduce this.

Thank you for your analysis!

> Even at https://issues.guix.gnu.org/49114 , only the "bug title" has the
> spurious underscore. The subject of the first message does not. Is the
> bug title something stored in the xapian index? Could it be that this
> was an older bug that has corrupted the xapian index?

As part of #60410 I rebuilt the xapian database (deleted it and then ran
“mumi fetch”), so it’s not that.

[… time passes …]

I think I found the culprit in mumi/debbugs.scm:

    (define qp-pattern "=\\?UTF-8\\?Q\\?([^?]+)\\?=")
    …
    (let ((subject (or (assoc-ref properties "Subject") "")))
      (if (string-contains subject "=?UTF-8?Q?")
          (or (false-if-exception
               (utf8->string
                (quoted-printable-decode
                 (regexp-substitute/global #f qp-pattern
                                           subject 'pre 1 'post))))
              subject)
          subject))

I’m probably not using quoted-printable-decode correctly.  My apologies
for assuming a bug in Guile Email.  I changed this with commit
9d7eb3c5efe6427a1a89fb3e4c7165259edff4dd, using parse-email-headers.

> If I understand correctly, mumi does not rebuild its xapian index. I
> think it should do so from time to time. It would help prevent old bugs
> from getting persisted in storage.

True.  I’ll be sure to rebuild the index on the next upgrade.

> Cheers, and wish you a Happy New Year! :-)

Thank you, and a Happy New Year to you too!

-- 
Ricardo

bug#49115: [PATCH 0/1] Do not check for MIME encoded words

[Arun Isaac]

Hi Ricardo,

I have an additional small improvement. guile-email transparently
handles MIME encoded words. We need not check for them. Patch follows.

Regards,
Arun

Arun Isaac (1):
  debbugs: Do not check for MIME encoded words in subject.

 mumi/debbugs.scm | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

-- 
2.38.1

bug#49115: [PATCH 1/1] debbugs: Do not check for MIME encoded words in subject.

[Arun Isaac]

guile-email transparently handles MIME encoded words. We do not have
to check for them.

* mumi/debbugs.scm (bug-status): Do not check for MIME encoded words
in subject.
---
 mumi/debbugs.scm | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/mumi/debbugs.scm b/mumi/debbugs.scm
index 16bff8e..7e95ddf 100644
--- a/mumi/debbugs.scm
+++ b/mumi/debbugs.scm
@@ -1,5 +1,6 @@
 ;;; mumi -- Mediocre, uh, mail interface
 ;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2023 Arun Isaac <arunisaac@systemreboot.net>
 ;;;
 ;;; This program is free software: you can redistribute it and/or
 ;;; modify it under the terms of the GNU Affero General Public License
@@ -280,10 +281,10 @@ defaults to 30 days."
                      (assoc-ref properties "Submitter")
                      (assoc-ref properties "Owner")
                      (or (assoc-ref properties "Severity") "normal")
-                     (let ((subject (or (assoc-ref properties "Subject") "")))
-                       (if (string-contains subject "=?UTF-8?Q?")
-                           (match (parse-email-headers (string-append "Subject: " subject "\n"))
-                             ((('subject . sub) . rest) sub)
-                             (other subject))
-                           subject))
+                     (assq-ref
+                      (parse-email-headers
+                       (string-append "Subject: "
+                                      (or (assoc-ref properties "Subject") "")
+                                      "\n"))
+                      'subject)
                      (assoc-ref properties "Tags"))))
-- 
2.38.1

bug#49115: [PATCH 1/1] debbugs: Do not check for MIME encoded words in subject.

[Ricardo Wurmus]

Arun Isaac <arunisaac@systemreboot.net> writes:

> guile-email transparently handles MIME encoded words. We do not have
> to check for them.
>
> * mumi/debbugs.scm (bug-status): Do not check for MIME encoded words
> in subject.

Thank you, I applied it.

-- 
Ricardo