bug#62815: Gnus: MIME types of attachments contain regexp quotation

bug#62815: Gnus: MIME types of attachments contain regexp quotation

[Torsten Bronger]

Hallöchen!

Since commit 3faa508eba84a1983732099cbd3cc1eaad404158, I observe
incorrect MIME types in my outgoing mails, e.g.

…
--=-=-=
Content-Type: application/vnd\.oasis\.opendocument\.spreadsheet
Content-Disposition: attachment;
 filename=2022-12-15_Nachbestellung_Studio.ods
Content-Transfer-Encoding: base64
…


Reason is that when attaching a file to a mail in Gnus, the MIME
type is suggested by Gnus, and this suggestion contains the
incorrect backslashes.

Regards,
Torsten.

-- 
Torsten Bronger

bug#62815: Gnus: MIME types of attachments contain regexp quotation

[Eli Zaretskii]

> From: Torsten Bronger <bronger@physik.rwth-aachen.de>
> Date: Thu, 13 Apr 2023 13:46:38 +0200
> 
> Hallöchen!
> 
> Since commit 3faa508eba84a1983732099cbd3cc1eaad404158, I observe
> incorrect MIME types in my outgoing mails, e.g.
> 
> …
> --=-=-=
> Content-Type: application/vnd\.oasis\.opendocument\.spreadsheet
> Content-Disposition: attachment;
>  filename=2022-12-15_Nachbestellung_Studio.ods
> Content-Transfer-Encoding: base64
> …
> 
> 
> Reason is that when attaching a file to a mail in Gnus, the MIME
> type is suggested by Gnus, and this suggestion contains the
> incorrect backslashes.

Lars, could you please look into this?

bug#62815: Gnus: MIME types of attachments contain regexp quotation

[Eli Zaretskii]

> Cc: 62815@debbugs.gnu.org
> Date: Thu, 13 Apr 2023 17:48:22 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > From: Torsten Bronger <bronger@physik.rwth-aachen.de>
> > Date: Thu, 13 Apr 2023 13:46:38 +0200
> > 
> > Hallöchen!
> > 
> > Since commit 3faa508eba84a1983732099cbd3cc1eaad404158, I observe
> > incorrect MIME types in my outgoing mails, e.g.
> > 
> > …
> > --=-=-=
> > Content-Type: application/vnd\.oasis\.opendocument\.spreadsheet
> > Content-Disposition: attachment;
> >  filename=2022-12-15_Nachbestellung_Studio.ods
> > Content-Transfer-Encoding: base64
> > …
> > 
> > 
> > Reason is that when attaching a file to a mail in Gnus, the MIME
> > type is suggested by Gnus, and this suggestion contains the
> > incorrect backslashes.
> 
> Lars, could you please look into this?

Ping!

Andrew, perhaps you could look into this some time soon?  It sounds
like a recent regression, so I'd like to fix this in Emacs 29.

bug#62815: Gnus: MIME types of attachments contain regexp quotation

[Andrew Cohen]

>>>>> "EZ" == Eli Zaretskii <eliz@gnu.org> writes:

    EZ> Andrew, perhaps you could look into this some time soon?  It
    EZ> sounds like a recent regression, so I'd like to fix this in
    EZ> Emacs 29.

(I've added Arash Esbati who filed the original bug that this commit was
intended to fix, and Andreas since I think he understands mime handling
in gnus and can probably comment.)

I have taken a quick look and think I understand what is going on, but
I'm not positive. The mailcap-mime-data can include a regexp for the
subtype; it appears that the only circumstance in which it is used as a
regexp is matching the possible viewer to be used for this subtype.

The patch in commit 3faa508eba84a1983732099cbd3cc1eaad404158 also treats
mailcap-mime-extensions as regexps, which I think is a mistake: these
should be associations between mime types and file extensions.

If all this is right, the simple fix is to revert the part of
3faa508eba84a1983732099cbd3cc1eaad404158 that treated the extension data
as regexps.  I have attached a diff below that does this. If Torsten
and Arash can check if this continues to provide a fix for bug#52038 and
bug#62815 that would be great!

There is one more place I think needs fixing (and maybe others I haven't
spotted yet):  the function mailcap-mime-types returns a list of mime
types, and excludes any type that contains a wildcard:
(unless (string-search "*" type) (push type res))
which suggests that this should be a list of actual types and not
include regexps. Hence we should remove any regexp quoting from the
entries. I have included a line that removes the "\\" in the diff.

diff --git a/lisp/net/mailcap.el b/lisp/net/mailcap.el
index 10c5a7744c1..b8990266069 100644
--- a/lisp/net/mailcap.el
+++ b/lisp/net/mailcap.el
@@ -979,7 +979,7 @@ mailcap-mime-extensions
     (".vox"   . "audio/basic")
     (".vrml"  . "x-world/x-vrml")
     (".wav"   . "audio/x-wav")
-    (".xls"   . "application/vnd\\.ms-excel")
+    (".xls"   . "application/vnd.ms-excel")
     (".wrl"   . "x-world/x-vrml")
     (".xbm"   . "image/xbm")
     (".xpm"   . "image/xpm")
@@ -1051,8 +1051,7 @@ mailcap-parse-mimetype-file
 	(setq save-pos (point))
 	(skip-chars-forward "^ \t\n")
 	(downcase-region save-pos (point))
-	(setq type (mailcap--regexp-quote-type
-                    (buffer-substring save-pos (point))))
+	(setq type (buffer-substring save-pos (point)))
 	(while (not (eolp))
 	  (skip-chars-forward " \t")
 	  (setq save-pos (point))
@@ -1107,7 +1106,7 @@ mailcap-mime-types
         (dolist (info (cdr data))
           (setq type (cdr (assq 'type (cdr info))))
           (unless (string-search "*" type)
-            (push type res))))
+            (push (string-replace "\\" "" type) res))))
       (nreverse res)))))
 
 ;;;

-- 
Andrew Cohen

bug#62815: Gnus: MIME types of attachments contain regexp quotation

[Arash Esbati]

Andrew Cohen <cohen@bu.edu> writes:

> If all this is right, the simple fix is to revert the part of
> 3faa508eba84a1983732099cbd3cc1eaad404158 that treated the extension data
> as regexps.  I have attached a diff below that does this. If Torsten
> and Arash can check if this continues to provide a fix for bug#52038 and
> bug#62815 that would be great!

Thanks for looking at this.  I built the latest repo version
(65735cee71) of Emacs and tried it with and w/o your patch.  It seems to
fix the issues Torsten and I have reported, but ...

> There is one more place I think needs fixing (and maybe others I haven't
> spotted yet):

I see a strange side effect when I use 'e' in article buffer to view a
attachment.  Without your change, Emacs starts Excel like this (this it
at least I see in *Messages* buffer) and it works:

  Displaying start excel.exe "`cygpath -w -a c\:/msys64/tmp/emm.fMzcnA/file_name.xlsx`"

With your change, I get:

  Displaying start excel.exe "`cygpath -w -a "c:/msys64/tmp/emm.QDUAFP/file_name.xlsx"`"

which doesn't work due to extra pair of quotes.  This is the entry in my
.mailcap:

  application/vnd.openxmlformats-officedocument.spreadsheetml.sheet; \
                       start excel.exe "`cygpath -w -a %s`"

This on Win10.

Best, Arash

bug#62815: Gnus: MIME types of attachments contain regexp quotation

[Andrew Cohen]

Arash Esbati <arash@gnu.org> writes:
>Andrew Cohen <cohen@bu.edu> writes:

>> That is the odd part---the changes I made don't touch this at all.  This
>> is in the viewer part, and none of the changes should have any effect on
>> this part. 

>Hi Andy, sorry for the late response.  It took me longer since I went
>through a longer oddity show.  It turned out that your patch works, but
>I had to install Emacs into a directory, i.e., pass a directory to
>configure via --prefix.  Simply starting emacs.exe out of ./src
>directory throws the error I described in my other mail, very odd.  So
>from what I see, your change fixes both problems reported.  Maybe you
>install that change and others can test is as well.

Glad to hear that the patch seems to work. I'm not sure what is going on
with the installation on windows, but since it sounds like this is not
related to the current issue, I'll follow your suggestion and commit the
patch to master. Assuming no one reports a problem I think the patch can
then go to the 29 branch to fix the regression.

Best,
Andy

-- 
Andrew Cohen

bug#62815: Gnus: MIME types of attachments contain regexp quotation

[Andrew Cohen]

My patch has spent almost a week on master with no complaints. Eli, OK
to push this regression fix to emacs-29 and close the bug report?

(I assume to push to emacs-29 I just checkout emacs-29, cherry-pick the
commit, and push?)
-- 
Andrew Cohen

bug#62815: Gnus: MIME types of attachments contain regexp quotation

[Eli Zaretskii]

> From: Andrew Cohen <cohen@bu.edu>
> Cc: Eli Zaretskii <eliz@gnu.org>
> Date: Sun, 07 May 2023 08:21:02 +0800
> 
> My patch has spent almost a week on master with no complaints. Eli, OK
> to push this regression fix to emacs-29 and close the bug report?
> 
> (I assume to push to emacs-29 I just checkout emacs-29, cherry-pick the
> commit, and push?)

Yes to both questions.

When you cherry-pick, be sure to use the -x switch to Git (or its
equivalent, if you don't invoke Git directly from the shell prompt),
so that the commit log message of the backport includes the telltale
"(cherry picked from commit ...)" line.

bug#62815: Gnus: MIME types of attachments contain regexp quotation

[Stefan Kangas]

Version: 29.1

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrew Cohen <cohen@bu.edu>
>> Cc: Eli Zaretskii <eliz@gnu.org>
>> Date: Sun, 07 May 2023 08:21:02 +0800
>>
>> My patch has spent almost a week on master with no complaints. Eli, OK
>> to push this regression fix to emacs-29 and close the bug report?
>>
>> (I assume to push to emacs-29 I just checkout emacs-29, cherry-pick the
>> commit, and push?)
>
> Yes to both questions.
>
> When you cherry-pick, be sure to use the -x switch to Git (or its
> equivalent, if you don't invoke Git directly from the shell prompt),
> so that the commit log message of the backport includes the telltale
> "(cherry picked from commit ...)" line.

It seems like this issue was fixed, but it was left open in the bug
tracker.  I'm therefore closing it now.