bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Drew Adams]

I tried several times to send the bug report that you find in bug
#31350.  I could not do so with Emacs 25 or later.  My mail client is
Outlook 2016.

Each time I tried, I got this Outlook error message:

  The command line argument is not valid.
  Verify the switch you are using.

I have no idea what the command line is that is sent to Outlook.

I finally succeeded in sending the bug report by deleting the two code
lines from it that you see in bug #31350 after this line of text:

  These are the two clauses in question:

Please refer to that bug for the two lines of code.

I deleted those two lines and was able to get the bug report from Emacs
to a new Outlook message, and I inserted the two code lines into that
Outlook message, then sent it.

Something seems wrong in Emacs (or Outlook?) if Emacs cannot get Outlook
to create a message that includes such simple text.

In GNU Emacs 27.0.50 (build 3, x86_64-w64-mingw32)
 of 2018-03-21
Repository revision: e70d0c9e66d7a8609450b2889869d16aeb0363b5
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --without-dbus --host=x86_64-w64-mingw32
 --without-compress-install -C 'CFLAGS=-O2 -static -g3''

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Noam Postavsky]

merge 31351 29074
quit

Drew Adams <drew.adams@oracle.com> writes:

> I tried several times to send the bug report that you find in bug
> #31350.  I could not do so with Emacs 25 or later.  My mail client is
> Outlook 2016.
>
> Each time I tried, I got this Outlook error message:
>
>   The command line argument is not valid.
>   Verify the switch you are using.
>
> I have no idea what the command line is that is sent to Outlook.

I guess the request in
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=29074#11 would be
relevant.

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Drew Adams]

> I guess the request in
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__debbugs.gnu.org_cgi_bugreport.cgi-3Fbug-3D29074-
> 2311&d=DwIBAg&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=kI3P6ljGv6C
> THIKju0jqInF6AOwMCYRDQUmqX22rJ98&m=RBYxpgDmEPhVfvSYC0Wp4U-
> HWUnAn8jq9ldQWaKXFVc&s=OxslwRdnIIJCpInJ7_HkFZmX1t_WQJiSGuWWvIAJs3k&e=
> would be relevant.

Thanks.  I thought it rang a bell, but I couldn't find that.

The new bug is reproducible for me, each time.  Hopefully it will help.

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Eli Zaretskii]

> Date: Wed, 2 May 2018 17:55:14 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> 
> I tried several times to send the bug report that you find in bug
> #31350.  I could not do so with Emacs 25 or later.  My mail client is
> Outlook 2016.
> 
> Each time I tried, I got this Outlook error message:
> 
>   The command line argument is not valid.
>   Verify the switch you are using.
> 
> I have no idea what the command line is that is sent to Outlook.
> 
> I finally succeeded in sending the bug report by deleting the two code
> lines from it that you see in bug #31350 after this line of text:
> 
>   These are the two clauses in question:
> 
> Please refer to that bug for the two lines of code.
> 
> I deleted those two lines and was able to get the bug report from Emacs
> to a new Outlook message, and I inserted the two code lines into that
> Outlook message, then sent it.

I see the problem, but I'm not sure it's the same problem, because it
doesn't fit your description in two details: (a) the problem I see is
not caused by anything in the message body -- in fact, I can reproduce
the problem with a "bug report" whose body consists of just "foo", in
addition to the details collected by the command about my system; and
(b) sending that problematic bug report works just fine in Emacs 25
and all older versions, and is broken only in Emacs 26 and later.

The problem I see is not caused by backticks in the body, it is caused
by quotes ".." in the Subject of the bug.  (I don't see how the body
could have any effect on launching Outlook, since we pass the body
through the system clipboard, and Outlook doesn't see it until you
paste the body into Outlook.  So it cannot possibly prevent Outlook
from starting correctly.)

To fix the problem I see, I installed on the release branch a fix,
which is reproduced below.  Please re-verify that you indeed see a
different problem -- I expect the patch below not to solve the problem
if so.

Thanks.

diff --git a/lisp/net/browse-url.el b/lisp/net/browse-url.el
index 98b0acf..a84a7b1 100644
--- a/lisp/net/browse-url.el
+++ b/lisp/net/browse-url.el
@@ -878,7 +878,21 @@ browse-url-default-windows-browser
 	   (error "Browsing URLs is not supported on this system")))
 	((eq system-type 'cygwin)
 	 (call-process "cygstart" nil nil nil url))
-	(t (w32-shell-execute "open" (url-unhex-string url)))))
+	(t
+         (w32-shell-execute "open"
+                            ;; w32-shell-execute passes file:// URLs
+                            ;; to APIs that expect file names, so we
+                            ;; need to unhex any %nn encoded
+                            ;; characters in the URL.  We don't do
+                            ;; that for other URLs; in particular,
+                            ;; default Windows mail client barfs on
+                            ;; quotes in the MAILTO URLs, so we prefer
+                            ;; to leave the URL with its embedded %nn
+                            ;; encoding intact.
+                            (if (eq t (compare-strings url nil 7
+                                                       "file://" nil nil))
+                                (url-unhex-string url)
+                              url)))))
 
 (defun browse-url-default-macosx-browser (url &optional _new-window)
   "Invoke the macOS system's default Web browser.

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Drew Adams]

> I see the problem, but I'm not sure it's the same problem, because it
> doesn't fit your description in two details: (a) the problem I see is
> not caused by anything in the message body -- in fact, I can reproduce
> the problem with a "bug report" whose body consists of just "foo", in
> addition to the details collected by the command about my system; and
> (b) sending that problematic bug report works just fine in Emacs 25
> and all older versions, and is broken only in Emacs 26 and later.
> 
> The problem I see is not caused by backticks in the body, it is caused
> by quotes ".." in the Subject of the bug.  (I don't see how the body
> could have any effect on launching Outlook, since we pass the body
> through the system clipboard, and Outlook doesn't see it until you
> paste the body into Outlook.  So it cannot possibly prevent Outlook
> from starting correctly.)
> 
> To fix the problem I see, I installed on the release branch a fix,
> which is reproduced below.  Please re-verify that you indeed see a
> different problem -- I expect the patch below not to solve the problem
> if so.
> 
> Thanks.

Great. Thanks for taking a closer look.

It's funny though.  Before I tried simplifying the body, I did
try simplifying the Subject line, by removing the backquote,
the single quote, and the double-quotes.  That didn't fix it
for me.  But perhaps I didn't do what I thought I did.

Anyway, if you've found a fix, that's great.

BTW, it wasn't about having an effect on launching Outlook.
I already had Outlook open.  It was only about not creating
a new Outlook message with the header filled and the body
ready to be pasted as the bug-report text.

(I don't want to try testing this, as that will likely create
a useless bug report.  If you feel this fixes the problem
that's good enough for me, until/unless I run into it again.)

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Eli Zaretskii]

> Date: Thu, 3 May 2018 11:24:39 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 31351@debbugs.gnu.org
> 
> Anyway, if you've found a fix, that's great.

I'd like to close this bug, but it would be good for you to try the
fixed browse-url.el and see if you can still trigger the problem with
that particular bug report.

> BTW, it wasn't about having an effect on launching Outlook.
> I already had Outlook open.  It was only about not creating
> a new Outlook message with the header filled and the body
> ready to be pasted as the bug-report text.

That's an implementation detail.  From Emacs point of view, it
launches Outlook, passing it the addressee and the subject of the bug.
That the running Outlook intercepts the command and opens a new window
in the same executable is out of Emacs control.

> (I don't want to try testing this, as that will likely create
> a useless bug report.

I don't see why you should worry about that: you can always close the
new Outlook window without clicking "Send".  IOW, if you get an
Outlook window with the bug address and the bug title, and can paste
the body with "Ctrl-v", then the bug is fixed, and you don't need to
send anything anywhere (but if you do want to send, you can edit the
addressee).

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Drew Adams]

> > (I don't want to try testing this, as that will likely create
> > a useless bug report.
> 
> I don't see why you should worry about that: you can always close the
> new Outlook window without clicking "Send".  

Of course.  Dunno what I was thinking.

I did that, use this as Subject:

  pcase message: Redundant pcase pattern

And there was no problem.

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Eli Zaretskii]

> Date: Thu, 3 May 2018 12:41:36 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 31351@debbugs.gnu.org
> 
> I did that, use this as Subject:
> 
>   pcase message: Redundant pcase pattern
> 
> And there was no problem.

Great.  Could you try patching browse-url.el, and then making a report
with quotes in the Subject?  That should work after the patch.

Thanks.

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Drew Adams]

> Could you try patching browse-url.el, and then making a report
> with quotes in the Subject?  That should work after the patch.

I did that, but it did not solve the problem - I get the same error.

I tried with both the original message body and with just aaaaaaa
as the body.  So the Subject line is the problem, and it does not
seem to be fixed by your patch.

This is the code I used:

(defun browse-url-default-windows-browser (url &optional _new-window)
  "Invoke the MS-Windows system's default Web browser.
The optional NEW-WINDOW argument is not used."
  (interactive (browse-url-interactive-arg "URL: "))
  (cond ((eq system-type 'ms-dos)
	 (if dos-windows-version
	     (shell-command (concat "start " (shell-quote-argument url)))
	   (error "Browsing URLs is not supported on this system")))
	((eq system-type 'cygwin)
	 (call-process "cygstart" nil nil nil url))
        (t
         (w32-shell-execute
          "open"
          ;; w32-shell-execute passes file:// URLs
          ;; to APIs that expect file names, so we
          ;; need to unhex any %nn encoded
          ;; characters in the URL.  We don't do
          ;; that for other URLs; in particular,
          ;; default Windows mail client barfs on
          ;; quotes in the MAILTO URLs, so we prefer
          ;; to leave the URL with its embedded %nn
          ;; encoding intact.
          (if (eq t (compare-strings url nil 7
                                     "file://" nil nil))
              (url-unhex-string url)
            url)))))

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Eli Zaretskii]

> Date: Thu, 3 May 2018 13:11:47 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 31351@debbugs.gnu.org
> 
> > Could you try patching browse-url.el, and then making a report
> > with quotes in the Subject?  That should work after the patch.
> 
> I did that, but it did not solve the problem - I get the same error.

Sorry for asking the trivial: are you sure you byte-compiled the
modified browse-url.el and loaded it into Emacs before trying?

If indeed the modified code still doesn't solve the problem, please
step with Edebug into browse-url-default-windows-browser when it is
called after you tell report-emacs-bug you want to send via
mailclient, and see what URL is being passed to w32-shell-execute.  It
should be a mailto URL with all the special characters, including the
quotes, encoded as hex %nn codes.  Copy that URL, then paste it into
the Windows Start->Run dialog, and see if that causes the same error
message as you get in your testing.

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Drew Adams]

> > > Could you try patching browse-url.el, and then making a report
> > > with quotes in the Subject?  That should work after the patch.
> >
> > I did that, but it did not solve the problem - I get the same error.
> 
> Sorry for asking the trivial: are you sure you byte-compiled the
> modified browse-url.el and loaded it into Emacs before trying?

No. I just modified that function and evaled the updated defun.

I've done what you suggest now - copied the whole file, updated
it, byte-compiled it, and loaded the *.elc.

And yes, that seems to fix the problem.  Thx.

bug#31351: 27.0; Cannot send bug report with Outlook if text includes backquoted sexps

[Eli Zaretskii]

> Date: Fri, 4 May 2018 08:30:37 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 31351@debbugs.gnu.org
> 
> I've done what you suggest now - copied the whole file, updated
> it, byte-compiled it, and loaded the *.elc.
> 
> And yes, that seems to fix the problem.  Thx.

Great, thanks.  So I'm closing this bug.